# Google-CloudVertexBot

> Crawls sites when a site owner asks Google Cloud to build a Vertex AI agent from them. Google says it has no effect on Google Search or other products.

Canonical page: https://getrankroot.com/ai-bots/google-cloudvertexbot · Facts checked against Google docs on 2026-10-20

| Operator | Used for | Type | Obeys robots.txt |
| --- | --- | --- | --- |
| Google | Vertex AI Agents | User-triggered fetcher | Yes |

## Should you allow it?

Allow it if you or your team build Vertex AI agents on your own content; otherwise it rarely matters.

**If you block it:** Vertex AI agents that site owners request can't crawl your site. No effect on Google Search.

## robots.txt rules

Allow Google-CloudVertexBot:

```
User-agent: Google-CloudVertexBot
Allow: /
```

Block Google-CloudVertexBot:

```
User-agent: Google-CloudVertexBot
Disallow: /
```

Disallow Google-CloudVertexBot from one folder only:

```
User-agent: Google-CloudVertexBot
Disallow: /private/
Allow: /
```

A bot follows the most specific User-agent group that names it and ignores the generic `User-agent: *` group, so repeat any shared Disallow lines inside it.

## Same company, different bots: Google

| Bot | Type | Purpose | Obeys robots.txt | If you block it |
| --- | --- | --- | --- | --- |
| [Google-Extended](https://getrankroot.com/ai-bots/google-extended.md) | Control token (does not crawl) | Gemini training and grounding | Token only | Google won't use your content to train future Gemini models or for grounding in Gemini Apps and Vertex AI. Google Search is unaffected. |
| [Googlebot](https://getrankroot.com/ai-bots/googlebot.md) | AI search index | Google Search (incl. AI Overviews) | Yes | Google can't crawl your pages for Search, Discover, Images, Video or News, and they can't be shown with content in AI Overviews or AI Mode. A blocked URL can still be listed without a description if other sites link to it. |
| [Google-CloudVertexBot](https://getrankroot.com/ai-bots/google-cloudvertexbot.md) | User-triggered fetcher | Vertex AI Agents | Yes | Vertex AI agents that site owners request can't crawl your site. No effect on Google Search. |

## How to verify it's really Google-CloudVertexBot

Find its requests in your server logs:

```
grep -i "Google-CloudVertexBot" access.log
```

Google's common crawlers such as Googlebot resolve to hostnames like crawl-***.googlebot.com or geo-crawl-***.geo.googlebot.com.

Reverse DNS check (Google documents this): the hostname of the IP must end in `googlebot.com` or `google.com` or `googleusercontent.com`, and a forward lookup of that hostname must return the same IP.

## Common mistakes

- Writing a User-agent: Google-CloudVertexBot group and expecting your User-agent: * rules to still apply. A crawler follows only the most specific group that matches it and ignores the others, so repeat any shared Disallow lines inside the Google-CloudVertexBot group.
- Leaving a CDN or host "block AI bots" setting on and only checking robots.txt. Those settings block requests at the network edge. Cloudflare, for example, changed its defaults on September 15, 2026 for customers who did not opt out: bots it classifies as Training or Agent are blocked on pages that display ads, and multi-purpose crawlers that combine Search and Training are affected by the Training block.
- Trusting the user agent header alone. Anyone can send "Google-CloudVertexBot" in a request; verify the source IP as shown above before allowlisting or blocking.

## FAQ

### What is Google-CloudVertexBot?

Crawls sites when a site owner asks Google Cloud to build a Vertex AI agent from them. Google says it has no effect on Google Search or other products.

### Does Google-CloudVertexBot respect robots.txt?

Yes, according to Google.

### What happens if I block Google-CloudVertexBot?

Vertex AI agents that site owners request can't crawl your site. No effect on Google Search.

### Should I block Google-CloudVertexBot?

Allow it if you or your team build Vertex AI agents on your own content; otherwise it rarely matters.

## Sources

- [Official documentation from Google](https://developers.google.com/crawling/docs/crawlers-fetchers/google-common-crawlers)
- [Google: how to verify its crawlers](https://developers.google.com/crawling/docs/crawlers-fetchers/verify-google-requests)

Check whether Google-CloudVertexBot can reach a site: https://getrankroot.com/tools/ai-crawler-checker or `GET https://getrankroot.com/api/bots?url=example.com`. Full list: https://getrankroot.com/ai-bots.md