AI crawler list · Mistral AI · AI training crawler

MistralAI-Training

Builds datasets for training Mistral's generative models. Not used for search indexing or live answers.

Facts checked against Mistral AI docs on Oct 20, 2026.

Operator

Mistral AI

Used for

Mistral model training

Type

AI training crawler

Obeys robots.txt

Yes

Is MistralAI-Training allowed on your site?

Reads your robots.txt and requests your page as each AI bot. Free, no signup.

Should you allow it?

Your choice.

If you block it: Your content is not collected for Mistral's training datasets.

robots.txt rules

Allow MistralAI-Training

User-agent: MistralAI-Training
Allow: /

Block MistralAI-Training

User-agent: MistralAI-Training
Disallow: /

Disallow MistralAI-Training from one folder only

User-agent: MistralAI-Training
Disallow: /private/
Allow: /

Put each User-agent group in the robots.txt at your domain root. A bot follows the most specific group that names it and ignores the generic User-agent: * group, so repeat any shared Disallow lines inside it. Build a full file with the AI robots.txt generator.

Same company, different bots: Mistral AI

Mistral AI uses separate user agents for separate jobs. Blocking one does not block the others.

Bot	Purpose	Obeys robots.txt	If you block it
`MistralAI-User` User-triggered fetcher	Mistral Vibe (user requests)	Yes	Mistral's assistant cannot visit your pages to answer users' questions.
`MistralAI-Index` AI search index	Mistral search	Yes	Your content is not indexed for Mistral search.
`MistralAI-Training` AI training crawler	Mistral model training	Yes	Your content is not collected for Mistral's training datasets.

Allow Mistral AI search and user fetches, opt out of Mistral AI training

User-agent: MistralAI-User
User-agent: MistralAI-Index
Allow: /

User-agent: MistralAI-Training
Disallow: /

User agent string

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; MistralAI-Training/1.0; +https://docs.mistral.ai/robots)

Versions in the string can change. Match on the token MistralAI-Training, and verify real traffic as shown below rather than trusting the header.

How to verify it's really MistralAI-Training

Find its requests in your server logs (the path to your access log depends on your host):

grep -i "MistralAI-Training" access.log

Mistral publishes IP lists for MistralAI-User and MistralAI-Index.

Mistral AI does not publish an IP list or reverse DNS method for MistralAI-Training, so the user agent is the only signal.

Common mistakes

Writing a User-agent: MistralAI-Training group and expecting your User-agent: * rules to still apply. A crawler follows only the most specific group that matches it and ignores the others, so repeat any shared Disallow lines inside the MistralAI-Training group.
Leaving a CDN or host "block AI bots" setting on and only checking robots.txt. Those settings block requests at the network edge. Cloudflare, for example, changed its defaults on September 15, 2026 for customers who did not opt out: bots it classifies as Training or Agent are blocked on pages that display ads, and multi-purpose crawlers that combine Search and Training are affected by the Training block.

FAQ

What is MistralAI-Training?

Builds datasets for training Mistral's generative models. Not used for search indexing or live answers.

Does MistralAI-Training respect robots.txt?

Yes, according to Mistral AI.

What happens if I block MistralAI-Training?

Your content is not collected for Mistral's training datasets.

Should I block MistralAI-Training?

Your choice.

Sources

Related AI crawlers

GPTBot · ClaudeBot · meta-externalagent · Amazonbot · MistralAI-User · MistralAI-Index · CCBot

Facts checked against Mistral AI docs on Oct 20, 2026. Spot an error? rankroot.support@proton.me