AI crawler list · Apple · Control token (does not crawl)

Applebot-Extended

A robots.txt token, not a crawler. Disallowing it tells Apple not to use content Applebot crawled to train its generative models. Apple says it does not remove you from search results and is not a ranking factor.

Facts checked against Apple docs on Oct 20, 2026.

Operator

Apple

Used for

Apple model training

Type

Control token (does not crawl)

Obeys robots.txt

Token only

Is Applebot-Extended allowed on your site?

Reads your robots.txt and requests your page as each AI bot. Free, no signup.

Should you allow it?

Your choice. It does not affect Siri or Spotlight search.

If you block it: Apple won't use Applebot-crawled content to train its generative models. Siri, Spotlight and Safari search are unaffected.

robots.txt rules

Allow Applebot-Extended

User-agent: Applebot-Extended
Allow: /

Block Applebot-Extended

User-agent: Applebot-Extended
Disallow: /

Disallow Applebot-Extended from one folder only

User-agent: Applebot-Extended
Disallow: /private/
Allow: /

Put each User-agent group in the robots.txt at your domain root. A bot follows the most specific group that names it and ignores the generic User-agent: * group, so repeat any shared Disallow lines inside it. Build a full file with the AI robots.txt generator.

Same company, different bots: Apple

Apple uses separate user agents for separate jobs. Blocking one does not block the others.

Bot	Purpose	Obeys robots.txt	If you block it
`Applebot` AI search index	Siri, Spotlight and Safari search	Yes	Your content can't be crawled for Spotlight, Siri and Safari search results.
`Applebot-Extended` Control token (does not crawl)	Apple model training	Token only	Apple won't use Applebot-crawled content to train its generative models. Siri, Spotlight and Safari search are unaffected.

Allow Apple search and user fetches, opt out of Apple training

User-agent: Applebot
Allow: /

User-agent: Applebot-Extended
Disallow: /

How to verify it's really Applebot-Extended

Applebot-Extended never sends requests, so it never appears in your server logs. The traffic it governs comes from Applebot; verify that crawler instead.

FAQ

What is Applebot-Extended?

Does Applebot-Extended respect robots.txt?

Applebot-Extended is a robots.txt token only; it never visits your site. A robots.txt token, not a crawler. Disallowing it tells Apple not to use content Applebot crawled to train its generative models. Apple says it does not remove you from search results and is not a ranking factor.

What happens if I block Applebot-Extended?

Apple won't use Applebot-crawled content to train its generative models. Siri, Spotlight and Safari search are unaffected.

Should I block Applebot-Extended?

Your choice. It does not affect Siri or Spotlight search.

Sources

Official documentation from Apple

Related AI crawlers

Google-Extended · Applebot

Facts checked against Apple docs on Oct 20, 2026. Spot an error? rankroot.support@proton.me