Why your AI-built SPA is invisible to AI crawlers (and risky for Google)

Because most AI crawlers do not run JavaScript. If you built your site as a single-page app, the file your server sends is a near-empty shell: a <div id="root">, a few script tags, and almost no content. A human browser runs those scripts, fetches your data, and paints the page. GPTBot, ClaudeBot, PerplexityBot and the rest do not. They read the shell, find nothing, and move on. Google is the one big exception, and even there the picture is shakier than it looks. This is the most common way a freshly shipped, AI-built product is broken without its owner knowing.

01 · The mechanismWhat a single-page app actually sends a crawler

When you fetch a client-rendered page, the HTML that arrives first is not your page. It is a loader.

That distinction is the whole problem. A browser runs the script, so a person sees a full page within a second or two. A crawler that does not run the script sees what the server literally sent: navigation scaffolding, an empty container, and references to bundles it will not execute. Your headline, your feature copy, your pricing, your FAQ, none of it is there yet. As far as that crawler is concerned, the page is blank.

This is invisible to you because you only ever view the site in a browser, where everything works. The gap only opens for visitors that read HTML without rendering it, which is nearly every machine that matters for discovery.

02 · The asymmetryGoogle renders it. The AI crawlers do not.

Here is the part that trips people up, so keep the two readers separate.

Googlebot renders JavaScript. It crawls the raw HTML first, then queues the page for a second pass where a headless Chromium engine runs your scripts and indexes the result. So a client-rendered page can rank on Google, and in 2026 Google is confident enough in this that it retired its long-standing advice to avoid client-side rendering. The catch is that the render is queued, can lag the first crawl, costs crawl budget, and quietly fails if a script times out or a resource is blocked. It usually works. It is not a thing to bet your launch on.

The AI crawlers do not render anything. They fetch raw HTML and read the text in it, full stop. Independent log analysis is blunt on this: Vercel and MERJ studied more than 500 million GPTBot fetches and found no evidence of JavaScript execution at all. The crawlers download script files in some cases, but never run them.

The consequence is a page that can sit at position one on Google and be a blank shell to ChatGPT, Claude and Perplexity at the same time. Search renders you; the answer engines do not. If being cited in AI answers is part of your plan, and for a launching product it usually should be (see should you block or allow AI crawlers), client-side rendering deletes you from that channel before you start.

03 · The 30-second testHow to tell if it is happening to you

You do not have to guess. There are two fast checks, and either one settles it.

The browser test: open dev tools, disable JavaScript, and reload the page. If it goes mostly blank, that is close to what a non-rendering crawler receives.

The terminal test: request the page the way a crawler does and look for content that should be there.

curl -s https://pagewatch.dev | grep -i "<h1"
curl -s https://pagewatch.dev | grep -i "pricing"

If your real heading and copy come back, you are fine. If you get nothing, or only a <title> and a root <div>, your content is injected by JavaScript and the crawler never reaches it. While you are in there, confirm your meta description, canonical tag, and any schema are in the raw HTML too, not added by script after load.

04 · Server-side rendering and prerenderingThe fix: put content in the first response

The fix is not “stop using a framework.” It is to make sure the content exists in the HTML the server sends, before any script runs. There are a few ways to get there, and your framework probably already supports one.

In practice: Astro and Next.js prerender by default, so you are mostly covered if you use them as intended. SvelteKit and Nuxt render on the server out of the box and let you prerender routes with a flag. If your important pages are static (a landing page, pricing, docs), static generation is the simplest and most robust answer, and it serves crawlers a complete page every time with no render to wait on. The single rule underneath all of it: the words must be in the HTML, not assembled later.

05 · The bigger frameWhere this sits in your launch

This is the floor of agent readiness: the parseable-content layer, where most sites quietly fail. Every cleverer signal sits on top of it. Your llms.txt, your structured data, your /.well-known/ manifests, none of them matter if the page they point at is a shell a crawler cannot read. Fix the rendering first, because it is the difference between a site machines can read and a site that is, to them, blank. It is also the cheapest fix here that pays off twice, since the same change helps Google index you faster and more reliably.