Applebot-Extended
Applebot-Extended is a robots.txt token, not a crawler. Here is what it controls, why it has no user agent, and how to opt out of Apple AI training.
What is Applebot-Extended?
Applebot-Extended is a robots.txt control token, not a crawler. Apple states it “is only used to determine how to use the data crawled by the Applebot user agent” and that it “does not crawl webpages.” Disallowing it opts your content out of training Apple's generative foundation models while normal Applebot search indexing continues.
These bots collect web content to train future AI models. Blocking them keeps your content out of training data — it costs you no traffic, because training crawlers never send visitors.
The Applebot-Extended user-agent string
Applebot-Extended has no fetching user-agent string of its own — Apple's docs say it does not crawl webpages. Some third-party trackers list a fabricated “Applebot-Extended/0.1” string; it contradicts Apple's docs and should not be relied on. The actual crawling is done by the separate Applebot agent.
How do I block Applebot-Extended in robots.txt?
Add one of these snippets to the robots.txt file at the root of your domain. An explicit group for Applebot-Extended overrides your User-agent: * rules for this bot.
Block Applebot-Extended
Tells Applebot-Extended it may not access any page on your site.
User-agent: Applebot-Extended Disallow: /
Allow Applebot-Extended
Explicitly allows Applebot-Extended, even when a broad Disallow rule blocks other bots.
User-agent: Applebot-Extended Allow: /
Does Applebot-Extended respect robots.txt?
It works as a robots.txt directive. A User-agent: Applebot-Extended / Disallow: / group tells Apple not to use your content for training its generative models. Apple states blocking it does not affect your visibility in Apple search features.
Should you block Applebot-Extended?
Disallowing Applebot-Extended lets you opt out of Apple's generative-model training while keeping your Apple search visibility, which Apple confirms is unaffected. If your concern is training rather than search, this is a clean token to set.
Official documentation
The facts on this page come from Apple's Applebot documentation. Bot behavior changes — when in doubt, the operator's page is the source of truth:
https://support.apple.com/en-us/119829What does your robots.txt say about Applebot-Extended?
Run your domain through our free checker to see whether Applebot-Extended — and 13 other AI crawlers — may access your site right now.