Website & app builders · Static site hosting · Checked 2026-10-19

GitHub Pages SEO for ChatGPT, Perplexity & AI search

Built your site with GitHub Pages? Here is what GitHub Pages's own documentation says about how AI crawlers see it, and the exact settings for robots.txt, llms.txt, structured data and your sitemap. Sources are listed at the bottom.

Is your GitHub Pages site visible to AI? Check it free

Paste your live URL. RankRoot checks your robots.txt rules for the ChatGPT, Perplexity and Claude crawlers, whether your page text is in the HTML without JavaScript, and your llms.txt and JSON-LD.

Free, no signup. 20+ checks across ChatGPT, Perplexity, Claude, Google AI and AI agents.

How GitHub Pages delivers pages to crawlers

Rendering

Static files, served as published

Depends on your project

GitHub's docs describe Pages as a static site hosting service that takes HTML, CSS and JavaScript files from a repository, optionally running them through a build (Jekyll by default, or any generator through GitHub Actions). Crawlers get exactly the files you publish. A Jekyll, Hugo or Astro site puts the text in the HTML. A single-page app built with React or Vite usually ships a near-empty HTML file and fills it in with JavaScript, which many AI crawlers don't run. Run RankRoot's audit to see which one you have.

Why it matters: many AI crawlers don't run JavaScript. If your text only appears after scripts run, they may see an empty page.

Make your content readable without JavaScript

Use a custom domain if you can. A project site lives at <owner>.github.io/<repository>, and crawlers only read robots.txt from the root of a host, so a project site can't set its own crawler rules on github.io.
If your site is a client-side app, pre-render it: use your framework's static export or a static site generator so each page's text is in the HTML file.
Run the free audit on your live URL to confirm crawlers see your text without JavaScript.

robots.txt: let AI search crawlers in

Add a file named robots.txt to the root of what you publish: the repository root or /docs folder if you deploy from a branch, or your build's output folder if you deploy with GitHub Actions.
With Jekyll, a file without front matter is a static file and is copied to the site unchanged, so a plain robots.txt works as is.
For a project site without a custom domain, robots.txt has to live in the <owner>.github.io user or organization site, because crawlers only check /robots.txt at the top of the host.
Push, then open yourdomain.com/robots.txt to confirm the live file.

robots.txt that lets AI search crawlers in

User-agent: OAI-SearchBot
Allow: /

User-agent: ChatGPT-User
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: Claude-SearchBot
Allow: /

User-agent: Claude-User
Allow: /

User-agent: *
Allow: /

Sitemap: https://yourdomain.com/sitemap.xml

Which bots to allow or block: robots.txt for AI crawlers and the AI crawler list.

Add llms.txt

GitHub Pages has no llms.txt feature, but any file you publish is served. Write the file with the llms.txt generator and save it as llms.txt next to robots.txt.
Leave front matter out so Jekyll copies it unchanged. Jekyll skips files and folders whose names start with _, . or #, so don't put it in one of those.
Push, open yourdomain.com/llms.txt and check it with the llms.txt validator.

Add JSON-LD and head code

On Jekyll, add the jekyll-seo-tag plugin. It's on GitHub Pages' list of supported plugins. List it under plugins in _config.yml and put {% seo %} right before </head> in your layout.
jekyll-seo-tag outputs the page title, description, canonical URL, Open Graph and Twitter card tags, and JSON-LD for site and post metadata. Set title and description in _config.yml and in each page's front matter.
Not using Jekyll? Add the title, meta description and a <script type="application/ld+json"> block in your generator's layout or head component.

_config.yml for jekyll-seo-tag and jekyll-sitemap

url: "https://yourdomain.com"
title: Your brand
description: One sentence on what you do and who it's for.
plugins:
  - jekyll-seo-tag
  - jekyll-sitemap

Sitemap

On Jekyll, add jekyll-sitemap (also on GitHub Pages' supported list) under plugins in _config.yml, and set url to your site's address. It generates sitemap.xml.
To leave a page out, set sitemap: false in its front matter.
With other generators, use the generator's sitemap option and deploy the output with GitHub Actions. Add a Sitemap: line to robots.txt and submit the sitemap in Google Search Console.

Common mistakes

Deploying a React or Vite single-page app and assuming AI crawlers see the content. Without pre-rendering, they often get an empty <div id="root">.
Adding robots.txt to a project site at <owner>.github.io/<repository>/robots.txt. Crawlers don't look there.
Adding plugins to the Gemfile only. The jekyll-sitemap docs warn that the GitHub Pages gem ignores plugins that aren't listed in _config.yml.
Forgetting url in _config.yml, so the sitemap and canonical tags point to the wrong host.

Get the files for your GitHub Pages site

Generate an llms.txt, then validate the live file once it's published.

llms.txt generator Validate llms.txt

Rather have this done for you?

The AI Visibility Employee looks after AI visibility for your GitHub Pages site every month, fully by email, no calls:

Weekly tracking of how ChatGPT, Perplexity, Gemini and Claude mention you for your buyers' questions
4 answer-ready pages a month, published through our WordPress plugin or a GitHub pull request, or delivered as drafts you add in GitHub Pages
Ongoing llms.txt, robots.txt and JSON-LD fixes wherever your setup allows them, and a monthly report

$149/month, cancel anytime. The first 10 customers pay $99/month with code FOUNDING10.

See the AI Visibility Employee

FAQ

Can I add llms.txt to GitHub Pages?

Yes. Save a plain llms.txt file (no front matter) at the root of what you publish and push. GitHub Pages serves it like any other static file.

Does GitHub Pages create a sitemap?

Not on its own. On Jekyll, add the jekyll-sitemap plugin in _config.yml (GitHub Pages supports it). With other generators, use their sitemap option.

Is my GitHub Pages site visible to ChatGPT?

If your pages are pre-built HTML (Jekyll, Hugo, Astro and similar), yes, as long as robots.txt allows OAI-SearchBot. A client-side single-page app may look empty to crawlers that don't run JavaScript. Run the free audit to check.

Where does robots.txt go on GitHub Pages?

At the root of your host. For a custom domain or a <owner>.github.io site that's the root of your published files. A project site on github.io can't have its own robots.txt.

Sources

Facts checked against GitHub Pages's documentation on 2026-10-19. RankRoot is not affiliated with GitHub Pages. Settings move around; if a step has changed, email rankroot.support@proton.me.

Other builders

Lovable · Bolt.new · Replit · v0 · Framer · Webflow · Wix · Squarespace · Shopify · WordPress · Ghost · Carrd · Hostinger AI Builder · Notion Sites · Netlify · Vercel · Cloudflare Pages & Workers