# Vercel SEO for ChatGPT, Perplexity and AI search

> Get your Vercel site into ChatGPT and AI search: the AI Bots firewall ruleset, Deployment Protection, noindex previews, vercel.app duplicates and robots.txt.

Canonical page: https://getrankroot.com/builders/vercel · Web hosting platform · checked 2026-10-21

What Vercel's own documentation says about how AI crawlers see sites built with it, and the exact settings for robots.txt, llms.txt, structured data and the sitemap.

## How Vercel delivers pages to crawlers

**Depends on your framework** (Depends on your project)

Vercel runs your framework's build: Next.js, SvelteKit, Astro and similar frameworks render pages to HTML on the server or at build time, so crawlers get your text. A client-only React or Vite app ships a near-empty HTML file that many AI crawlers can't fill in, because they don't run JavaScript. We didn't find a Vercel feature that pre-renders client-only apps for crawlers, so use your framework's static export or server rendering. Run RankRoot's audit to see what crawlers receive.

Why it matters: many AI crawlers don't run JavaScript. If your text only appears after scripts run, they may see an empty page.

## Make your content readable without JavaScript

1. Check the firewall: project **Firewall** > **Rules** > **Bot Management**. The **AI Bots Ruleset** is off by default (labeled **Allow**). Set to **Deny**, it blocks all traffic Vercel identifies as AI bots, which Vercel says covers bots that crawl for training, search and user-requested fetches. Use **Log** if you only want to watch.
2. The **Bot Protection** ruleset is also off by default. In challenge mode it serves a JavaScript challenge to traffic unlikely to be a browser, but Vercel says it skips verified bots. Vercel's verified bots directory lists OAI-SearchBot, ChatGPT-User, PerplexityBot, Claude-SearchBot and Claude-User.
3. Check **Settings** > **Deployment Protection**. Since September 2026, Vercel Authentication can protect production too, on every plan. If it covers the URL you want cited, visitors must sign in with Vercel, and crawlers can't.
4. Run the [free audit](https://getrankroot.com/#scan) and the [AI crawler checker](https://getrankroot.com/tools/ai-crawler-checker) on your custom domain.

## robots.txt: let AI search crawlers in

1. Vercel serves the `robots.txt` your framework outputs. On Next.js add `app/robots.ts` or `public/robots.txt`; on Vite, Astro or SvelteKit put `robots.txt` in the static folder.
2. Deploy and open `yourdomain.com/robots.txt`. A firewall rule can still block bots that robots.txt allows, so check the AI Bots ruleset too.

robots.txt that lets AI search crawlers in:

```
User-agent: OAI-SearchBot
Allow: /

User-agent: ChatGPT-User
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: Claude-SearchBot
Allow: /

User-agent: Claude-User
Allow: /

User-agent: *
Allow: /

Sitemap: https://yourdomain.com/sitemap.xml
```

## Add llms.txt

1. Vercel has no llms.txt setting. Save the file in your framework's static folder (`public/llms.txt` in Next.js) and it's served at `/llms.txt`.
2. Write it with the [llms.txt generator](https://getrankroot.com/tools/llms-txt-generator), deploy, then check `yourdomain.com/llms.txt` with the [llms.txt validator](https://getrankroot.com/tools/llms-txt-validator).

## Add JSON-LD and head code

1. Set the title, meta description and JSON-LD in your framework. In Next.js use the Metadata API and render a `<script type="application/ld+json">` tag in your layout or page.
2. Add a canonical tag on every page that points to your custom domain. Vercel recommends it so `your-project.vercel.app` doesn't compete with your domain.
3. For extra safety, send `X-Robots-Tag: noindex` only on the vercel.app host. Vercel prefers setting it in your framework config and using `vercel.json` only when the framework can't set headers.

vercel.json: noindex only on the vercel.app host:

```
{
  "headers": [
    {
      "source": "/(.*)",
      "has": [{ "type": "host", "value": "your-project.vercel.app" }],
      "headers": [{ "key": "X-Robots-Tag", "value": "noindex" }]
    }
  ]
}
```

## Sitemap

1. Vercel doesn't generate a sitemap. Use your framework's option (for Next.js, `app/sitemap.ts`) and list URLs on your custom domain, not vercel.app.
2. Reference it from `robots.txt` with a `Sitemap:` line and submit it in Google Search Console.

## Common mistakes

- Setting the AI Bots ruleset to **Deny** to stop training crawlers. It blocks AI search and user-requested fetches too. To block training only, disallow GPTBot or ClaudeBot in robots.txt and leave the ruleset on Allow or Log.
- Turning on Deployment Protection for all deployments, including production, on a public marketing site. Crawlers can't sign in.
- Assigning a custom domain to a preview branch. Vercel then leaves out the automatic `X-Robots-Tag: noindex`, so that domain can be indexed as duplicate content.
- Leaving `your-project.vercel.app` indexable next to your domain. Add canonical tags and, if needed, a noindex header for that host.

## FAQ

### Does Vercel block AI crawlers?

Not by default. The AI Bots managed ruleset is off (Allow) unless you set it to Log or Deny. Deny blocks AI bots used for training, search and user-requested fetches. Bot Protection, also off by default, skips verified bots like OAI-SearchBot.

### Are Vercel preview deployments indexed?

Vercel adds X-Robots-Tag: noindex to preview deployments and outdated production deployments. It leaves the header off when a custom domain is assigned to a non-production branch.

### Is my Vercel site visible to ChatGPT?

If your framework renders pages to HTML (Next.js does by default), and robots.txt, the AI Bots ruleset and Deployment Protection let OAI-SearchBot in, yes. Run the free audit to confirm.

### Should I worry about vercel.app duplicate content?

Vercel says search engines can treat your vercel.app URL as separate pages. Add a canonical tag pointing to your custom domain, and optionally send noindex for the vercel.app host.

## Sources

- [Vercel Docs: Bot Management (AI bots ruleset, verified bots) (read 2026-10-21)](https://vercel.com/docs/bot-management)
- [Vercel Docs: WAF Managed Rulesets (read 2026-10-21)](https://vercel.com/docs/vercel-firewall/vercel-waf/managed-rulesets)
- [Vercel Docs: Response headers (x-robots-tag) (read 2026-10-21)](https://vercel.com/docs/headers/response-headers)
- [Vercel KB: Are Vercel Preview Deployments indexed by search engines? (read 2026-10-21)](https://vercel.com/kb/guide/are-vercel-preview-deployment-indexed-by-search-engines)
- [Vercel KB: Avoiding duplicate content with vercel.app URLs (read 2026-10-21)](https://vercel.com/kb/guide/avoiding-duplicate-content-with-vercel-app-urls)
- [Vercel Docs: Vercel Authentication (read 2026-10-21)](https://vercel.com/docs/deployment-protection/methods-to-protect-deployments/vercel-authentication)
- [Vercel changelog: Protect production deployments for free on every plan (Sept 9, 2026) (read 2026-10-21)](https://vercel.com/changelog/protect-production-deployments-for-free-on-every-plan)
- [Next.js: robots.txt file convention (read 2026-10-18)](https://nextjs.org/docs/app/api-reference/file-conventions/metadata/robots)
- [Next.js: sitemap.xml file convention (read 2026-10-18)](https://nextjs.org/docs/app/api-reference/file-conventions/metadata/sitemap)
- [Next.js: JSON-LD (read 2026-10-18)](https://nextjs.org/docs/app/guides/json-ld)

RankRoot is not affiliated with Vercel. Check a live site: https://getrankroot.com/ or `GET https://getrankroot.com/api/audit?url=example.com`. Other builders: https://getrankroot.com/builders.md