Where AI Answers Come From: The Four Stages

When an AI answer gets your price wrong, the cause sits in one of four places, and the fix depends entirely on which one. Rewriting your copy does nothing for a problem caused by a firewall. Unblocking a crawler does nothing for an answer the model wrote from memory.

Most GEO advice treats the engine as one black box: content goes in, citations come out. It is four stages, and each is controlled by someone different. Stage 0 is what the model learned in training. Stage 1 is whether a crawler can fetch and store your page. Stage 2 is whether your page gets pulled into the model's context for a given question. Stage 3 is what the model does with the pages it pulled, including which ones it cites.

This lesson walks through each stage, says who controls it and what you can and cannot do there, then traces one question about Ledgerly, our fictional invoicing SaaS, through all four.

Stage 0: what the model already knows

Every model is trained on a large snapshot of text and then frozen. What it learned is called its parametric knowledge, and it stops at a knowledge cutoff. When an engine answers without searching, this is all it has. Nothing you publish this quarter reaches it. A fact you change today shows up in retrieval-off answers only after a model trained on newer data ships, and only if that data happened to include your page.

ChatGPT is not one knowledge state. OpenAI routes between model families, and the families have different cutoffs. The user usually cannot see which one answered, so "what ChatGPT knows about you" has no single date attached to it.

The only route into training data is a training crawler: GPTBot for OpenAI, ClaudeBot for Anthropic, and the Google-Extended token for Google's Gemini models. Whether allowing them improves what a model knows about you has never been measured, by anyone, including the vendors.

Who controls it: nobody, this quarter. What you can do: list the facts that changed after each cutoff (a new price, a new product, a new CEO) and expect retrieval-off answers to get those wrong. That tells you which errors to stop chasing. What you can't do: correct a model's memory directly.

Stage 1: access, or can the crawler fetch and keep your page

Before an engine can use your page in a search-backed answer, its crawler has to reach the page, read it and store it. This is the stage you control most, and it works like a set of gates. Each one is binary. Pass and you are eligible. Fail and you are out, usually without any error you would notice.

• robots.txt, per bot. OpenAI states that sites "opted out of OAI-SearchBot will not be shown in ChatGPT search answers." Each engine has its own retrieval bot and token, and allowing one does nothing for the others.
• Your firewall or CDN. OpenAI's help center says that to be eligible you must allow OAI-SearchBot "and confirm that the website host or content delivery network allows traffic" from its published IP addresses. The word is and. robots.txt can say yes while a bot-management rule returns 403.
• Rendering. Among the AI crawlers tested, only Google's and Apple's run JavaScript. If your price, features or headings appear only after scripts run, every other crawler stores a page without them.
• Index and snippet controls. A noindex tag, a nosnippet or max-snippet:0 directive, or a data-nosnippet attribute removes content from AI answers on Google, and Bing honors data-nosnippet for its AI summaries too. These often arrive from a template, put there by someone hiding a cookie banner.

Cloudflare’s September 15 announcement says existing settings carry over. New ad-supported domains are offered Search: Allow, Training: Disallow AI Training, and Agent: Block on pages with ads. Training Block also blocks mixed-use crawlers such as Googlebot and Bingbot; Disallow AI Training preserves their search access. Check the settings of your own zone.

Who controls it: you and your CDN. What you can do: check every gate, bot by bot. What you can't do: become more than eligible. There is no additive markup that raises your standing at this stage, only ways to fail it. Module 3 covers each gate, starting with AI crawlers and robots.txt.

Stage 2: retrieval, or whether your page enters the context

Being stored is not being used. For each question, the engine decides whether to search at all, what to search for, and which results to pull into the model's context. All three decisions happen out of your sight.

Retrieval is discretionary, per query. Google AI Overviews often do not appear. ChatGPT decides for itself when to search. In the Claude app, web search is a user toggle with no published default. Even Perplexity's API makes web search something the developer opts into. Published estimates of how often ChatGPT searches disagree badly: Profound reports that about 18% of conversations trigger a search, Semrush reports that 34.5% of US queries had search on in February 2026, and cloro reports 80.5% globally on a 100-prompt set it chose. They measure different things on different populations. If an answer shows no sources, you do not know for certain whether it searched.

The prompt is not the query. OpenAI says ChatGPT search "rewrites your query into one or more targeted queries" before sending them to its search providers. Google says its AI features may use a "query fan-out" technique, issuing multiple related searches across subtopics. Google's Gemini API "automatically generates one or multiple search queries." So the question a buyer types is not what retrieves your page, and a tool that tests prompts is measuring a stand-in for an input nobody outside the engine can see. The one exception is the AI Performance report in Bing Webmaster Tools, which shows a sample of the grounding queries Copilot and Bing's AI summaries actually used.

Order matters, and you can't set it. Once candidates are in the context, their position strongly affects which one gets used. C-SEO Bench (NeurIPS 2025) found that moving a document higher in the model's context was significantly more effective than the content rewrites it tested. No publisher can make that move. It belongs to the retrieval system.

Who controls it: the retrieval operator: OpenAI, Google, Perplexity, Microsoft, Anthropic. What you can do: pass Stage 1 everywhere, have a page whose subject is the thing buyers ask about, and stay in the Bing index, which Microsoft documents as the backing store for Copilot and which OpenAI has named among the search providers ChatGPT sometimes uses, with an effect on ChatGPT nobody has measured. What you can't do: see the real queries on most engines, choose the context order, or force a search.

Stage 3: generation, or what gets used and what gets cited

The model now writes an answer from the pages in its context plus whatever it already knew. It decides which pages to lean on and which to cite. Those are two different decisions.

Citation is not use. OpenAI's API returns the pages a search consulted separately from the ones the answer cited, and its documentation says "the number of sources is often greater than the number of citations." Engines also cite in different units: Anthropic's API cites short spans of text, while Perplexity cites whole sources by number. Mechanistic research (arXiv:2606.28358) repaired over 90% of missed citations in a test model by amplifying specific internal components, without changing the sources at all. Whether a page gets cited is partly separate from whether it shaped the answer.

That cuts both ways. Your own page can shape an answer that cites a competitor's review of you. A page can also be cited for a sentence the answer barely used. Citation counts are a noisy proxy for influence, and the noise is probably not random: well-known sources are more likely to be credited.

Stage 3 is also where almost every published GEO experiment lives. Researchers place documents directly in the context, change the wording, and measure citation. The best-designed of those studies found the rewrites mostly ineffective or harmful, as the next lesson shows.

Who controls it: the model. What you can do: make sure the facts in any page an engine might pull, yours or a third party's, are correct and agree with each other. What you can't do: make the model cite you by rewriting your copy.

One Ledgerly question, traced through all four stages

Ledgerly (fictional) is a B2B invoicing SaaS. In April 2026 it raised its Starter plan from $25 to $29 per user per month. Its pricing page loads the plan table with JavaScript after the page opens. Its robots.txt allows OAI-SearchBot. A 2024 review on a comparison blog still lists the old $25 price. A buyer asks ChatGPT: "How much does Ledgerly cost for a team of 10?"

Stage	What happens	Who controls it	What Ledgerly can do	What it can't do
0 Parametric	If ChatGPT answers without searching, it says $25 per user. Both OpenAI cutoffs predate the April change.	Nobody, this quarter	State the new price consistently everywhere, so later training data agrees with itself	Change what this model remembers
1 Access	OAI-SearchBot is allowed and gets a 200, but the raw HTML of the pricing page says "Loading plans" with no price in it.	Ledgerly and its CDN	Server-render the plan table so the price is in the HTML	Make the page more than eligible
2 Retrieval	When it searches, ChatGPT rewrites the question into queries like "Ledgerly pricing per user" and retrieves the pricing page, the 2024 review and a directory listing.	OpenAI	Ask the reviewer to correct the price; check the directory listing	See the real queries or choose what gets retrieved
3 Generation	The retrieved pricing page has no price in it, so the model takes $25 from the review and cites the review. The pricing page was retrieved but not cited.	The model	Nothing directly; fixing Stages 1 and 2 changes what the model has to work with	Make the model prefer its page

Common mistakes

• Diagnosing a wrong answer without knowing whether it searched. The same wrong price has a Stage 0 cause in one answer and a Stage 1 cause in another. Check for sources before you pick a fix.
• Fixing at Stage 3 a problem that sits at Stage 1. Copy rewrites cannot help a page the crawler stored empty, blocked or unrendered.
• Expecting a site edit to change a retrieval-off answer. It won't, this quarter. Measure your fixes on answers that show sources.
• Reading "not cited" as "not used," or "cited" as "trusted." Retrieved and cited are different sets, and citation is partly separate from influence.
• Testing the prompt and assuming it is the query. Engines rewrite and fan out the question. Your prompt list is a proxy, so treat its results as one.