GPTBot
GPTBot is OpenAI's training crawler. Here is what it does, its documented user-agent string, and how to allow or block it in robots.txt.
What is GPTBot?
GPTBot is OpenAI's web crawler. It collects publicly available web content that, in OpenAI's words, “may be used in training our generative AI foundation models.” Disallowing it signals that your content should not be used for model training.
These bots collect web content to train future AI models. Blocking them keeps your content out of training data — it costs you no traffic, because training crawlers never send visitors.
The GPTBot user-agent string
This is the user-agent string OpenAI documents for GPTBot. You will see it in your server logs when the bot visits.
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.3; +https://openai.com/gptbot
How do I block GPTBot in robots.txt?
Add one of these snippets to the robots.txt file at the root of your domain. An explicit group for GPTBot overrides your User-agent: * rules for this bot.
Block GPTBot
Tells GPTBot it may not access any page on your site.
User-agent: GPTBot Disallow: /
Allow GPTBot
Explicitly allows GPTBot, even when a broad Disallow rule blocks other bots.
User-agent: GPTBot Allow: /
Does GPTBot respect robots.txt?
Yes. OpenAI documents GPTBot as controllable through robots.txt: “Disallowing GPTBot indicates a site's content should not be used in training generative AI foundation models.” A standard User-agent: GPTBot group is honored.
Should you block GPTBot?
Blocking GPTBot is a reasonable choice if you do not want your content in future OpenAI model training. It costs you nothing in traffic, because GPTBot does not send visitors. Note that blocking it does not remove you from ChatGPT's live search or assistant answers — those use different bots.
Official documentation
The facts on this page come from OpenAI's bot documentation. Bot behavior changes — when in doubt, the operator's page is the source of truth:
https://developers.openai.com/api/docs/botsWhat does your robots.txt say about GPTBot?
Run your domain through our free checker to see whether GPTBot — and 13 other AI crawlers — may access your site right now.