Website & app builders · CMS · Checked 2026-10-18

WordPress SEO for ChatGPT, Perplexity & AI search

Built your site with WordPress? Here is what WordPress's own documentation says about how AI crawlers see it, and the exact settings for robots.txt, llms.txt, structured data and your sitemap. Sources are listed at the bottom.

How WordPress delivers pages to crawlers

WordPress's developer docs say its template loader decides which theme template to load for each URL, and templates generate the page's document structure. Plugins or page builders that load content with JavaScript can change what crawlers see, so check your page source or run RankRoot's audit. This guide covers self-hosted WordPress (the WordPress.org software).

Why it matters: many AI crawlers don't run JavaScript. If your text only appears after scripts run, they may see an empty page.

Make your content readable without JavaScript

1. Go to Settings > Reading and make sure Discourage search engines from indexing this site is unchecked. When it's checked, WordPress adds a noindex, nofollow robots meta tag to every page, and core sitemaps are disabled.
2. Run the free audit on your homepage and a post to confirm the content is in the HTML.

robots.txt: let AI search crawlers in

1. WordPress outputs a virtual robots.txt (the do_robots() function). By default it contains User-agent: *, a Disallow line for the admin path, an Allow line for admin-ajax.php, and a reference to the core sitemap.
2. Add rules with the robots_txt filter in a small plugin or your theme's functions.php. The filter receives the generated text and whether the site is public.
3. If a physical robots.txt file exists in your site's root folder, edit that file instead, then open yourdomain.com/robots.txt to see what's actually served.
4. SEO plugins often include a robots.txt editor. Our free WordPress plugin has presets such as "Allow AI search, block training".

add_filter( 'robots_txt', function ( $output, $public ) {
    if ( ! $public ) {
        return $output;
    }
    // A crawler that matches this group ignores the * group, so repeat the admin rules.
    $output .= "\nUser-agent: OAI-SearchBot\nUser-agent: PerplexityBot\nUser-agent: Claude-SearchBot\nDisallow: /wp-admin/\nAllow: /wp-admin/admin-ajax.php\n";
    return $output;
}, 10, 2 );

Which bots to allow or block: robots.txt for AI crawlers and the AI crawler list.

Add llms.txt

1. WordPress core has no llms.txt feature.
2. On self-hosted WordPress you can upload a plain llms.txt file to the site's root folder (the one with wp-config.php) using your host's file manager or SFTP. Write it with the llms.txt generator.
3. Or use a plugin that builds it from your pages and posts, such as our free WordPress plugin. See the llms.txt for WordPress guide.
4. Open yourdomain.com/llms.txt and check it with the llms.txt validator.

Add JSON-LD and head code

1. Themes call wp_head() in the page <head>. Hook into the wp_head action to print a JSON-LD script.
2. SEO plugins usually add Organization and WebSite schema already, so check your page source to avoid duplicates.

add_action( 'wp_head', function () {
    if ( ! is_front_page() ) {
        return;
    }
    $data = array(
        '@context' => 'https://schema.org',
        '@type'    => 'Organization',
        'name'     => get_bloginfo( 'name' ),
        'url'      => home_url( '/' ),
    );
    echo '<script type="application/ld+json">' . wp_json_encode( $data ) . '</script>';
} );

Sitemap

1. Since WordPress 5.5, core publishes a sitemap index at /wp-sitemap.xml and references it in robots.txt.
2. Core sitemaps are disabled when the site discourages search engines. They can also be turned off with the wp_sitemaps_enabled filter, which some plugins do when they provide their own sitemap.
3. Submit whichever sitemap your site actually serves in Google Search Console.

Common mistakes

• Leaving "Discourage search engines" on after launch. WordPress adds noindex to every page and disables core sitemaps.
• Uploading a physical robots.txt and then wondering why filter- or plugin-based rules don't show up. Check what yourdomain.com/robots.txt actually returns.
• Outputting JSON-LD twice: once from the theme or an SEO plugin and once from custom code.
• Following self-hosted steps on a hosted WordPress plan that doesn't give you file or plugin access. Check what your host allows.

FAQ

Sources

• WordPress Developer: Template hierarchy (read 2026-10-18)
• WordPress Developer: do_robots() (read 2026-10-18)
• WordPress Developer: robots_txt hook (read 2026-10-18)
• WordPress Developer: wp_head hook (read 2026-10-18)
• Make WordPress Core: New XML sitemaps functionality in WordPress 5.5 (read 2026-10-18)
• WordPress Documentation: Settings Reading screen (read 2026-10-18)

Facts checked against WordPress's documentation on 2026-10-18. RankRoot is not affiliated with WordPress. Settings move around; if a step has changed, email rankroot.support@proton.me.

Other builders

Lovable · Bolt.new · Replit · v0 · Framer · Webflow · Wix · Squarespace · Shopify · Ghost · GitHub Pages · Carrd · Hostinger AI Builder · Notion Sites · Netlify · Vercel · Cloudflare Pages & Workers