GPTBot
OpenAI's web crawler for collecting content that may be used to train its generative AI models, controlled with the GPTBot user agent in robots.txt.
GPTBot is one of several OpenAI agents, and it's the one for training. Disallowing it tells OpenAI not to use your content to train its foundation models. It has nothing to do with whether ChatGPT can find and cite you today.
OpenAI's three main agents
| User agent | What it does | Controls ChatGPT search visibility? |
|---|---|---|
| GPTBot | Collects content that may be used for model training | No |
| OAI-SearchBot | Finds pages to show and link in ChatGPT search | Yes |
| ChatGPT-User | Visits a page when a user's request needs it | No |
OpenAI treats these settings independently, so allowing OAI-SearchBot while disallowing GPTBot is a supported setup. OpenAI also notes that robots.txt changes can take about a day to reach search.
User-agent: GPTBot
Disallow: /
User-agent: OAI-SearchBot
Allow: /Not sure what your robots.txt allows? The Elite Answers audit checks it, and our ChatGPT engine page explains how ChatGPT picks its sources.
Should you block GPTBot?
It's a licensing decision, not a visibility one. Allowing GPTBot is a bet that appearing in future training data helps models know your brand. Blocking it keeps your content out of that pool. Either choice is defensible. Making it by accident, through a blanket rule nobody remembers writing, is not.
Somewhere, right now, an AI is recommending your client's competitor.
Elite Answers is invite-only. Request access through Elite Digital and the team will set you up.