# AI visibility lost to a robots.txt rule written too broadly | Illia BizTech

> A blanket block on AI robots can remove the mention and leave training untouched. The robot names decide which of the two happens.

Source: https://illia-biztech.de/en/forum/unser-dienstleister-sagt-die-sperre-habe-mit-den-verlorenen-nennungen-nichts-zu-tun

1. [Home](/en)
2. [Forum](/en/forum)
3. Our contractor says the block has nothing to do with the lost mentions

# Our contractor says the block has nothing to do with the lost mentions

We blocked the training crawlers, and the mentions went with them.

* ##### Asked by  
RustyTurbine405
* ##### Asked on  
September 24, 2026

We supply laboratory equipment. In the spring of 2026 our contractor put several blocking lines for AI robots into robots.txt so our copy would stay out of training.

Since the summer we are no longer named in assistant answers where we used to appear regularly. The contractor says the two are unrelated.

#### Answers

![Portrait of Illia Belichenko, technical specialist at Illia BizTech](/_ipx/f_webp,q_75/https://cms.illia-biztech.de/uploads/small_Illia_Belichenko_author_81f88864f0.png)

##### Illia Belichenko · Founder & Technical Lead

✓ Accepted answer September 25, 2026 

They are very probably related. Every large vendor runs several robots under different names, and the name decides what a block affects. A blanket rule therefore rarely lands where it was aimed.

OpenAI's documentation lists OAI-SearchBot as the one that surfaces pages in ChatGPT's search features, and says that opted-out sites are not shown in those answers, though they can still appear as navigational links. GPTBot is the training crawler. Anyone who wanted to stop training alone needed exactly one of those two.

Anthropic has three. Its page, dated 7 April 2026, names ClaudeBot for training data, Claude-User for fetches a person's question triggered, and Claude-SearchBot for search quality. Here too the mention hangs on a different name from the training.

Collect the current list of robot names from each vendor separately, since the page addresses move from time to time, and compare it against your file line by line. That is a morning of manual work. An [AI visibility report](https://illia-biztech.de/en/ki-sichtbarkeitsbericht) at Illia BizTech then tells you whether the company came back into those answers.

##### 

#### [Our company page has admins nobody knows and the request just sits there](/en/forum/unsere-unternehmensseite-hat-administratoren-die-niemand-kennt-und-die-anfrage-bleibt-liegen)

September 23, 2026

[](/en/forum/unsere-unternehmensseite-hat-administratoren-die-niemand-kennt-und-die-anfrage-bleibt-liegen)

#### [After switching to a professional account our profile went public and every request was accepted](/en/forum/nach-dem-umstellen-auf-ein-unternehmenskonto-wurde-unser-profil-offentlich-und-alle-offenen-anfragen-wurden-angenommen)

September 23, 2026

[](/en/forum/nach-dem-umstellen-auf-ein-unternehmenskonto-wurde-unser-profil-offentlich-und-alle-offenen-anfragen-wurden-angenommen)

#### [I search for our ad every morning and lately I cannot find it](/en/forum/ich-suche-jeden-morgen-nach-unserer-anzeige-und-seit-einiger-zeit-finde-ich-sie-nicht-mehr)

September 22, 2026

[](/en/forum/ich-suche-jeden-morgen-nach-unserer-anzeige-und-seit-einiger-zeit-finde-ich-sie-nicht-mehr)

#### [The new colleague holds every right on the Page and still gets nowhere](/en/forum/die-neue-kollegin-hat-alle-rechte-auf-der-seite-und-kommt-trotzdem-nirgends-hinein)

September 22, 2026

[](/en/forum/die-neue-kollegin-hat-alle-rechte-auf-der-seite-und-kommt-trotzdem-nirgends-hinein)

#### [The keyword tool shows a zero for all our terms](/en/forum/im-keyword-tool-steht-bei-unseren-begriffen-eine-null)

September 21, 2026

[](/en/forum/im-keyword-tool-steht-bei-unseren-begriffen-eine-null)

**Share** 

**Topics** 
* robots.txt
* AI visibility

AI visibility rests on a different robot name from training at every vendor. OpenAI's documentation separates the robot behind its search features from the one that gathers training data. Anthropic publishes three names with three different consequences.

Keeping that straight is therefore largely a question of maintaining a list of names. A line meant to cover all of them at once removes the mention and often leaves training as it was. The vendors publish these lists themselves and revise them occasionally, which makes a line-by-line comparison the only reliable check.