Meta-ExternalAgent
Meta-ExternalAgent is Meta's crawler for training its AI models. Here is what it does, its documented user-agent string, and how to block it.
What is Meta-ExternalAgent?
Meta-ExternalAgent is one of Meta's documented crawlers. Meta states its purpose is “training foundation AI models or improving products by indexing content directly.” It is operated by Meta (Facebook, Instagram, WhatsApp).
These bots collect web content to train future AI models. Blocking them keeps your content out of training data — it costs you no traffic, because training crawlers never send visitors.
The Meta-ExternalAgent user-agent string
This is the user-agent string Meta documents for Meta-ExternalAgent. You will see it in your server logs when the bot visits.
meta-externalagent/1.1 (+https://developers.facebook.com/docs/sharing/webmasters/crawler)
Meta also documents a shorter form, “meta-externalagent/1.1”, without the URL. Meta's crawlers may cache robots.txt for up to 24 hours, so a new rule can take a day to take effect.
How do I block Meta-ExternalAgent in robots.txt?
Add one of these snippets to the robots.txt file at the root of your domain. An explicit group for Meta-ExternalAgent overrides your User-agent: * rules for this bot.
Block Meta-ExternalAgent
Tells Meta-ExternalAgent it may not access any page on your site.
User-agent: Meta-ExternalAgent Disallow: /
Allow Meta-ExternalAgent
Explicitly allows Meta-ExternalAgent, even when a broad Disallow rule blocks other bots.
User-agent: Meta-ExternalAgent Allow: /
Does Meta-ExternalAgent respect robots.txt?
Honored in general, but read the fine print. Meta does not give Meta-ExternalAgent a blanket compliance guarantee; instead it names sibling bots that may bypass robots.txt — Meta-ExternalFetcher (user-requested fetches) and FacebookExternalHit (security/integrity checks). Meta-ExternalAgent itself is not called out as a bypasser, so a User-agent: Meta-ExternalAgent group is the right control.
Should you block Meta-ExternalAgent?
Blocking Meta-ExternalAgent is a reasonable choice if you do not want your content training Meta's AI models. It costs you no traffic. Bear in mind that Meta runs several crawlers, and some of the others (its user-requested fetcher and its integrity checker) may bypass robots.txt, so blocking this one token does not stop every Meta bot.
Official documentation
The facts on this page come from Meta's web crawlers documentation. Bot behavior changes — when in doubt, the operator's page is the source of truth:
https://developers.facebook.com/docs/sharing/webmasters/web-crawlers/What does your robots.txt say about Meta-ExternalAgent?
Run your domain through our free checker to see whether Meta-ExternalAgent — and 13 other AI crawlers — may access your site right now.