AI crawler list · Amazon · AI training crawler

Amazonbot

Improves Amazon products and services and, Amazon says, may be used to train Amazon AI models.

Facts checked against Amazon docs on Oct 20, 2026.

Operator

Amazon

Used for

Amazon products and AI

Type

AI training crawler

Obeys robots.txt

Yes

Is Amazonbot allowed on your site?

Reads your robots.txt and requests your page as each AI bot. Free, no signup.

Should you allow it?

Your choice. Amazon's search crawler for Alexa answers is Amzn-SearchBot.

If you block it: Stops the Amazon crawler whose data may be used to train Amazon AI models. Alexa search uses Amzn-SearchBot instead.

Amazon's crawlers do not support Crawl-delay and may use a cached robots.txt up to 30 days old. Amazon says settings changes take about 24 hours.

Amazon also honors the noarchive robots meta tag as a page-level signal not to use a page for model training.

Amazon says allowing Amazonbot may make publishers eligible for its Content Partners benefits.

robots.txt rules

Allow Amazonbot

User-agent: Amazonbot
Allow: /

Block Amazonbot

User-agent: Amazonbot
Disallow: /

Disallow Amazonbot from one folder only

User-agent: Amazonbot
Disallow: /private/
Allow: /

Put each User-agent group in the robots.txt at your domain root. A bot follows the most specific group that names it and ignores the generic User-agent: * group, so repeat any shared Disallow lines inside it. Build a full file with the AI robots.txt generator.

Same company, different bots: Amazon

Amazon uses separate user agents for separate jobs. Blocking one does not block the others.

Bot	Purpose	Obeys robots.txt	If you block it
`Amazonbot` AI training crawler	Amazon products and AI	Yes	Stops the Amazon crawler whose data may be used to train Amazon AI models. Alexa search uses Amzn-SearchBot instead.
`Amzn-SearchBot` AI search index	Alexa and Amazon search	Yes	Your content is no longer eligible to appear in Amazon search experiences such as Alexa.
`Amzn-User` User-triggered fetcher	Alexa (user requests)	Not reliably	A robots.txt rule may not fully apply; Amazon says it may not follow all directives.

Allow Amazon search and user fetches, opt out of Amazon training

User-agent: Amzn-SearchBot
User-agent: Amzn-User
Allow: /

User-agent: Amazonbot
Disallow: /

User agent string

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Amazonbot/0.1) Chrome/W.X.Y.Z Safari/537.36

Versions in the string can change. Match on the token Amazonbot, and verify real traffic as shown below rather than trusting the header.

How to verify it's really Amazonbot

Find its requests in your server logs (the path to your access log depends on your host):

grep -i "Amazonbot" access.log

Amazon publishes a separate IP address list for each of its crawlers.

Published IP list: developer.amazon.com/amazonbot/ip-addresses/

Common mistakes

Writing a User-agent: Amazonbot group and expecting your User-agent: * rules to still apply. A crawler follows only the most specific group that matches it and ignores the others, so repeat any shared Disallow lines inside the Amazonbot group.
Leaving a CDN or host "block AI bots" setting on and only checking robots.txt. Those settings block requests at the network edge. Cloudflare, for example, changed its defaults on September 15, 2026 for customers who did not opt out: bots it classifies as Training or Agent are blocked on pages that display ads, and multi-purpose crawlers that combine Search and Training are affected by the Training block.
Trusting the user agent header alone. Anyone can send "Amazonbot" in a request; verify the source IP as shown above before allowlisting or blocking.

FAQ

What is Amazonbot?

Improves Amazon products and services and, Amazon says, may be used to train Amazon AI models.

Does Amazonbot respect robots.txt?

Yes, according to Amazon.

What happens if I block Amazonbot?

Stops the Amazon crawler whose data may be used to train Amazon AI models. Alexa search uses Amzn-SearchBot instead.

Should I block Amazonbot?

Your choice. Amazon's search crawler for Alexa answers is Amzn-SearchBot.

Sources

Related AI crawlers

GPTBot · ClaudeBot · meta-externalagent · Amzn-SearchBot · Amzn-User · MistralAI-Training · CCBot

Facts checked against Amazon docs on Oct 20, 2026. Spot an error? rankroot.support@proton.me