Do AI Crawlers Render JavaScript in 2026? Why Your Next.js SPA Is Invisible to ChatGPT (and How to Fix It)

Yes, in 2026 the major AI crawlers still fetch your initial HTML and stop there. GPTBot, OAI-SearchBot, ClaudeBot, and PerplexityBot do not run your JavaScript, so anything your Next.js app paints after hydration is invisible to ChatGPT, Claude, and Perplexity. If your important content lives inside 'use client' components that fetch on mount, an AI engine sees an empty shell. The fix is to make sure the words you want cited exist in the server-rendered HTML, which the App Router gives you by default if you stop fighting it.

I have spent the last year cleaning up "why are we invisible to ChatGPT" tickets, and the root cause is almost always the same: a beautiful SPA that ships a near-empty and fills it in with client-side fetches. Let me show you how to confirm the problem in two minutes, and then how to actually fix it in the App Router.

Do AI crawlers render JavaScript in 2026?

No. As of June 2026, no major AI crawler reliably renders JavaScript. They request your URL, read the HTML the server returns, and index that. They do not spin up a headless browser, wait for hydration, or execute your useEffect data fetches.

This is the dangerous part, because it looks fine to you. You open the page in Chrome, JavaScript runs, the content appears, and you assume the crawler sees the same thing. It does not. The crawler is closer to running curl than running a browser.

There is a useful piece of context here. In March 2026, Google removed the old SEO warning it used to show about client-side rendering, because Googlebot does render JS through a deferred second pass. That change quietly convinced a lot of teams that "rendering is solved." It is solved for Googlebot. It is not solved for the AI crawlers, and the audits I have run in June 2026 confirm none of the big four execute your scripts. Conflating Googlebot's behavior with GPTBot's behavior is the false-confidence trap that gets sites left out of AI answers.

Which AI crawlers fetch HTML only?

All of the ones that matter for AI visibility right now. Here is how the four primary bots behave and what user-agent string they send.

The takeaway is that the bot reads your raw HTML response and nothing else. If a fact is not in that response, the bot never learns it. Note that these are distinct user agents with distinct jobs. GPTBot is about training, while OAI-SearchBot and ChatGPT-User are about answering a live question, and you may want different robots.txt rules for each, but none of them run your client code.

Why is my Next.js SPA invisible to ChatGPT?

Because the content ChatGPT needs is being painted by client JavaScript that the crawler never executes. The most common offender is a component marked 'use client' that fetches data in useEffect and renders it after mount.

Walk through what the crawler actually receives with this pattern:

'use client';

import { useEffect, useState } from 'react';

export default function ProductDetails({ id }: { id: string }) {
  const [product, setProduct] = useState<Product | null>(null);

  useEffect(() => {
    fetch(`/api/products/${id}`)
      .then((r) => r.json())
      .then(setProduct);
  }, [id]);

  if (!product) return <p>Loading...</p>;

  return (
    <article>
      <h1>{product.name}</h1>
      <p>{product.description}</p>
    </article>
  );
}

In a real browser this works fine. But the HTML the server sends for this component is essentially

Loading...

This is why "we rank fine on Google but ChatGPT keeps citing our competitor" is such a common complaint. Googlebot's second-pass renderer eventually picks up the hydrated content. The AI crawlers never do. So the same page can be visible to Google and invisible to ChatGPT at the same time. That gap is exactly what the Google AI Mode and query fan-out ↗ shift makes expensive, because more queries are answered by an engine that only saw your empty shell.

How do I test what an AI crawler actually sees?

Use curl with the bot's user-agent and read the raw HTML. If the content you care about is not in that output, no AI crawler can see it either. This is the single most useful two-minute check I run.

## What ChatGPT's search crawler sees
curl -sL -A "Mozilla/5.0 (compatible; OAI-SearchBot/1.0; +https://openai.com/searchbot)" \
  https://yoursite.com/products/widget-9000 \
  | grep -i "widget-9000\|description-text-you-expect"

## What Perplexity sees
curl -sL -A "Mozilla/5.0 (compatible; PerplexityBot/1.0; +https://perplexity.ai/bot)" \
  https://yoursite.com/products/widget-9000 -o perplexity-view.html

If grep returns nothing, that is your answer. The HTML does not contain the content, so the bot does not have it.

I keep a tiny harness in CI so this never regresses. It fetches each critical URL with each bot UA and fails the build if a required phrase is missing from the raw response.

#!/usr/bin/env bash
## bot-render-check.sh, fail if key content is missing from raw HTML
set -euo pipefail

BASE="${1:?usage: bot-render-check.sh https://yoursite.com}"

UAS=(
  "GPTBot/1.0"
  "OAI-SearchBot/1.0"
  "ClaudeBot/1.0"
  "PerplexityBot/1.0"
)

## url|phrase that MUST appear in the raw HTML
CHECKS=(
  "/products/widget-9000|Widget 9000"
  "/pricing|per month"
  "/|What I build"
)

fail=0
for ua in "${UAS[@]}"; do
  for check in "${CHECKS[@]}"; do
    url="${check%%|*}"; phrase="${check##*|}"
    html="$(curl -sL -A "$ua" "$BASE$url")"
    if ! grep -qiF "$phrase" <<<"$html"; then
      echo "MISSING [$ua] $url -> \"$phrase\""
      fail=1
    fi
  done
done

[ "$fail" -eq 0 ] && echo "All bot-visibility checks passed."
exit "$fail"

One caveat worth saying out loud: some hosts and CDNs serve a different response to bot user-agents, or block them outright with a 403 or a challenge page. So check the HTTP status too. A bot that gets a 403 sees nothing, which is a separate but equally fatal problem. Pair this with the kind of disclaimer about real-browser checks I cover in my INP and LCP fix list for React ↗: the browser view lies to you, the raw response does not.

How do I fix it in the Next.js App Router?

Keep the content that matters on the server. The App Router renders everything as a React Server Component by default, and a Server Component's output lands in the initial HTML, which is exactly what the crawler reads. The fix is usually deleting code, not adding it.

Here is the same product page done so an AI crawler can read it:

// app/products/[id]/page.tsx
// No 'use client'. This is a Server Component by default.

async function getProduct(id: string): Promise<Product> {
  const res = await fetch(`https://api.internal/products/${id}`, {
    // cache the result so the page is statically generated where possible
    next: { revalidate: 3600 },
  });
  return res.json();
}

export async function generateMetadata({ params }: { params: { id: string } }) {
  const product = await getProduct(params.id);
  return { title: product.name, description: product.description };
}

export default async function ProductPage({ params }: { params: { id: string } }) {
  const product = await getProduct(params.id);

  return (
    <article>
      <h1>{product.name}</h1>
      <p>{product.description}</p>
      <AddToCartButton id={product.id} /> {/* the only 'use client' part */}
    </article>
  );
}

The data fetch now happens on the server, the name and description are baked into the HTML response, and the interactive button is the only thing pushed to the client. The crawler reads a full article. The decision tree I use:

The rule I give every team: 'use client' is for behavior, not for content. Push the directive as far down the tree as possible so it wraps a button, not a whole page. If a paragraph of text only exists inside a client component that fetches on mount, that paragraph is invisible to every AI crawler, full stop.

A few more things that bite people:

• next/dynamic with { ssr: false } does exactly what it says and removes the component from server HTML. Do not use it for anything you want cited.
• Content hidden behind tabs or accordions is fine as long as it is in the DOM. The crawler reads the markup, not the visual state, so collapsed content still counts if it is server-rendered.
• Infinite-scroll or "load more" content that only fetches on interaction is invisible past the first batch. Paginate with real, crawlable URLs instead.

If you want a second set of eyes on whether your stack is server-rendering the right things, this is the kind of audit I do as part of my web development work ↗.

What about JSON-LD and structured data?

Put your structured data in the server-rendered HTML, not injected by client script. AI engines and search crawlers both read JSON-LD straight from the initial response, and it gives them clean, unambiguous facts to cite.

In the App Router this is a server component that simply renders a script tag:

export default async function ProductPage({ params }: { params: { id: string } }) {
  const product = await getProduct(params.id);

  const jsonLd = {
    '@context': 'https://schema.org',
    '@type': 'Product',
    name: product.name,
    description: product.description,
    offers: { '@type': 'Offer', price: product.price, priceCurrency: 'USD' },
  };

  return (
    <>
      <script
        type="application/ld+json"
        dangerouslySetInnerHTML={{ __html: JSON.stringify(jsonLd) }}
      />
      <article>{/* ... */}</article>
    </>
  );
}

Because this renders on the server, the JSON-LD is in the raw HTML that GPTBot and PerplexityBot read. If you instead inject it with a client-side analytics-style snippet, you are back to square one and the bots never see it.

The short version

If you remember one thing: open a terminal, curl your most important page with -A "GPTBot/1.0", and read what comes back. That output is your entire presence to AI engines today. Everything painted after hydration is a private show for human visitors with JavaScript on.

Most "invisible to ChatGPT" problems I see are not penalties or bans. They are architecture. The content is real, it is just on the wrong side of the hydration boundary. Move it to the server, keep 'use client' for behavior, and verify with the bot-UA harness in CI so it never silently regresses.

If you are staring at a Next.js SPA and not sure which content is actually reaching the crawlers, that is a quick thing to check together. Reach out through my contact page ↗ and we can run the audit on your real pages.