Google-Extended: What It Controls, and Why It Won't Get You Out of AI Overviews

Google-Extended is a robots.txt token that tells Google whether content it crawls from your site may be used to train future Gemini models and to ground answers in Gemini Apps and Vertex AI. It is not a crawler, it never appears in your logs, and it does not control Google Search. Blocking it does not remove you from AI Overviews or AI Mode. Those are governed by Googlebot, snippet controls and a Search Console setting.

Every claim below comes from Google's documentation, read in October 2026. The common crawlers page that defines Google-Extended was last updated on 15 October 2026.

What Google-Extended controls, in Google's words

Google defines it as "a standalone product token that web publishers can use to manage whether content Google crawls from their sites may be used for training future generations of Gemini models that power Gemini Apps and Vertex AI API for Gemini and for grounding (providing content from the Google Search index to the model at prompt time to improve factuality and relevancy) in Gemini Apps and Grounding with Google Search on Vertex AI."

That sentence holds two jobs, not one:

• Training: future Gemini models behind Gemini Apps and the Vertex AI API for Gemini.
• Grounding: pages from the Google Search index handed to Gemini at prompt time, in Gemini Apps and in Grounding with Google Search on Vertex AI.

So it is not a training-only switch: blocking it also removes your pages from Gemini Apps grounding.

Google is equally direct about what it leaves alone: "Google-Extended does not impact a site's inclusion in Google Search nor is it used as a ranking signal in Google Search."

It fetches nothing

"Google-Extended doesn't have a separate HTTP request user agent string. Crawling is done with existing Google user agent strings; the robots.txt user-agent token is used in a control capacity." Googlebot fetches the page; Google-Extended decides what Google may do with it afterwards. You will not find Google-Extended in your server logs, and no firewall rule can target it. Our AI crawler list covers the bots that do fetch.

What Google-Extended does and does not reach

Google surface	What governs it	Does Google-Extended affect it?
Google Search results	Googlebot, noindex, snippet controls	No: "does not impact a site's inclusion in Google Search"
AI Overviews and AI Mode: showing and grounding on your pages	Googlebot, nosnippet, data-nosnippet, max-snippet, Search Console control	No
Training the models behind AI Overviews and AI Mode	Google-Extended	Yes, per Search Console's help (below)
Gemini Apps: training and grounding	Google-Extended	Yes
Vertex AI: Gemini training and Grounding with Google Search	Google-Extended	Yes
Vertex AI Agents crawls a site owner requests	Google-CloudVertexBot, a separate token	No
Gemini Notebook, Google-Agent	User-triggered fetchers that "generally ignore robots.txt rules"	No
Grounding in the Gemini Developer API outside Vertex AI	Not named on Google's page	Undocumented

Sources: the common crawlers page and the user-triggered fetchers page, last updated 19 August 2026.

The third row is the one most guides miss. Search Console's Search generative AI control help says: "This control doesn't affect AI training; to limit training of the models used to generate responses in Search generative AI features, use Google-Extended." Google splits the job in two: Google-Extended for training, the Search Console control and snippet rules for what AI Overviews and AI Mode show.

The common mistake: blocking Google-Extended to leave AI Overviews

The reasoning goes: AI Overviews are Google AI, Google-Extended is Google's AI token, so block it. It does not work. Google's AI features guide says a page is eligible as a supporting link in AI Overviews or AI Mode if it is "indexed and eligible to be shown in Google Search with a snippet". Google-Extended changes neither condition. A site that blocks it stays exactly as eligible for AI Overviews as before.

It does cost something, though. The same line removes your pages from grounding in Gemini Apps. Whether that changes what Gemini says about your brand is unmeasured: Google publishes no data on it, and neither do we. Decide it as a trade-off, not a default.

The opposite mistake, blocking Googlebot, takes the site out of all of Google Search. Google says "robots.txt directives for Googlebot is the control" for its AI features, because "AI is built into Search". There is no robots.txt line for AI Overviews alone.

How robots.txt applies to Google-Extended

Google's robots.txt specification says "Only one group is valid for a particular crawler": the group with the most specific matching user agent. Two consequences:

• A Googlebot group does not cover it. Google: "Even if there is an entry for a related crawler, it is only valid if it's specifically matching." The page lists Googlebot as a fallback token for Googlebot-Image or Google-CloudVertexBot, but not for Google-Extended.
• The * group. Google documents that a crawler with no specific group follows *. It does not spell out how that applies to a control token. If your User-agent: * group disallows paths, write an explicit Google-Extended group so the decision is yours, not an inference.

To opt out of Gemini training and grounding and stay fully in Google Search:

User-agent: Google-Extended
Disallow: /

User-agent: Googlebot
Allow: /

The controls that do limit AI Overviews and AI Mode

All of them are subtractive, and every one also affects normal search results or traffic. The full walk-through is in how to turn off AI Overviews; this is the summary.

• nosnippet: per Google's robots meta tag documentation, it "will also prevent the content from being used as a direct input for AI Overviews and AI Mode." It removes your text snippet from web search too.
• max-snippet: "will also limit how much of the content may be used as a direct input for AI Overviews and AI Mode."
• data-nosnippet: keeps one section of a page out of snippets while the rest stays eligible.
• Search Console's generative AI control: excludes the whole site from AI Overviews, AI Mode and Discover's generative AI features. Google: "You won't receive any traffic or impressions from these features", and content takes 1-2 days or longer to drop out.

For how AI Mode reports its sources, see Google AI Mode.

How to set Google-Extended on purpose

1. Write down what you want out of. Each surface in the table above has its own control.
2. Read your current file per token. Run it through the AI crawler robots.txt checker. It reports Google-Extended, Googlebot and the other AI crawlers separately, with the line that decides each.
3. Write an explicit Google-Extended group. Allow or disallow it by name rather than relying on *.
4. Leave Googlebot alone unless you want out of Search. Check that no CDN or firewall rule blocks it either; Google's AI features guide asks for crawling to be allowed "by any CDN or hosting infrastructure".
5. Use snippet controls or Search Console for AI Overviews. Pick page-level (nosnippet, data-nosnippet, max-snippet) or site-wide (the Search Console control), knowing the traffic cost.
6. Record the date. Google-Extended never shows in logs, so the file and its change date are your only record.

99Visibility's access check reads your robots.txt for every AI token, Google-Extended included, requests your pages as each crawler that fetches, and flags noindex, nosnippet, max-snippet:0 and data-nosnippet on the pages it checks. It cannot probe Google-Extended itself: there is no request to send. It also audits what six AI platforms, Gemini and Google AI Overviews among them, say about your brand. Your first audit is free, with no credit card, and covers ChatGPT and Google AI Overviews. See the plans for all six platforms. Inside the dashboard, Agent V reads your audit and your site and drafts the robots.txt lines for you to paste. It never edits your site itself.

FAQ

Does blocking Google-Extended remove my site from AI Overviews?

No. Google says Google-Extended "does not impact a site's inclusion in Google Search", and AI Overviews draw on pages that are indexed and eligible for a snippet. Use nosnippet or the Search Console generative AI control instead.

Does Google-Extended affect my rankings?

Google says it is not "used as a ranking signal in Google Search". Blocking or allowing it changes how Gemini may train on and ground with your content, not where you rank.

What user agent does Google-Extended use?

None. Google says it "doesn't have a separate HTTP request user agent string". Googlebot does the crawling; the token only states how the crawled content may be used.

Should I block Google-Extended?

Block it if you object to Gemini training on your content and accept losing grounding in Gemini Apps and Vertex AI. Allow it if being grounded in Gemini answers matters more. Nobody has published a measurement of what either choice does to how Gemini describes a brand.

See which Google controls your site actually sets. Run a free audit: no credit card needed.