Run Your First AI Answer Audit

Most teams check their AI visibility the same way. Someone types the company name into ChatGPT, reads one answer, and either relaxes or panics. That is one sample, from one engine, in one session, on one day. It tells you almost nothing, and it cannot show you whether next month is better or worse.

An answer audit replaces the anecdote with a fixed set of prompts, asked the same way on several engines, more than once, with every answer recorded in the same columns. It is slower than a screenshot. It is also the only way to know what AI tells your buyers, and it is the baseline every later lesson in this course measures against.

This lesson walks you through designing the prompt set, controlling what you can, recording what you can see, and reading the result without fooling yourself. You finish with a 10-prompt sheet and your first two engines of answers.

What an audit can and cannot see

Start with the limit, because it shapes everything else. When you type a prompt, the engine does not necessarily search for your words. OpenAI's ChatGPT search help page says it "typically rewrites your query into one or more targeted queries" before sending them to search providers. Google describes a "query fan-out" technique for its AI features, issuing multiple related searches across subtopics. The Gemini API "automatically generates one or multiple search queries." The query that actually retrieved your page, or failed to, stays invisible to you.

So every prompt in your audit is a proxy. It stands in for an input you cannot observe. The only first-party window onto real retrieval queries is the grounding-queries view in Bing Webmaster Tools' AI Performance report, in public preview since February 10, 2026. It covers Microsoft Copilot, Bing's AI summaries and select partner integrations. It does not cover ChatGPT or Google.

That sounds damning. It is not. Your buyer also types a prompt and reads an answer, and an audit measures exactly that surface: what this engine says when asked this question. What it cannot do is tell you why, or prove that your prompts are the ones buyers type. Hold both thoughts and you will read your results correctly.

Design the prompt set: four query types

Ten prompts is enough for a first audit. With fewer, one odd answer dominates the picture. With many more, you will stop recording carefully. Split the ten across four query types, because each type tests something different.

Brand queries (3 prompts)

Questions that name you: "What is Ledgerly?", "How much does Ledgerly cost?", "Who owns Ledgerly?" These almost always mention you, so they say little about visibility. They are your accuracy test. This is where wrong prices, a departed CEO and invented features show up. Pick the facts that cost you deals when they are wrong.

Category queries (3 prompts)

Questions a buyer asks before they know you exist: "best invoicing software for small agencies", "which invoicing tools handle multi-currency billing?" These are the real visibility test. If you are absent here, you are absent from the shortlist.

Comparison queries (2 prompts)

"Ledgerly vs Invoxa", "is Ledgerly or Billcraft better for a 20-person consultancy?" These show framing: which strengths the engine attaches to you, which weaknesses, and whose facts it gets right.

Problem queries (2 prompts)

Questions about the pain, not the product: "how do I stop clients paying invoices late?", "how do I invoice a client in euros from a US company?" Engines often answer these with advice and no brands at all. When a brand does appear, it is the one the engine connects to solving the problem.

Write every prompt the way a buyer would, in plain words, without leading. "Why is Ledgerly the best invoicing tool?" tests nothing; it asks the engine to agree with you. Number the prompts, freeze the wording and keep the list in one file. If you reword a prompt next month, its results start a new series. They do not continue the old one.

One trap to avoid: a sheet of brand queries alone produces a flattering mention rate, because those prompts contain your name. Keep the category and problem prompts in the mix even when they hurt. They are the ones that tell you something.

Control what you can: session, retrieval state and variance

Three things change the answer without you changing the prompt. You can control one, record one, and average out the third.

Session: log out or switch memory off

OpenAI says that if memory is enabled, ChatGPT "may use relevant saved memories when rewriting a search query." A marketer who has spent months asking ChatGPT about their own company is the worst possible test subject. Use a logged-out window, or a fresh account with memory and custom instructions off, and write down which one you used.

Retrieval state: record it per answer

OpenAI’s chat-latest model page, checked 16 October 2026, lists an April 20, 2026 knowledge cutoff. That alias changes over time. Other models have their own cutoffs, so record the exact model and its documented cutoff rather than treating ChatGPT as one fixed knowledge state. A fact published after that snapshot cannot have been learned from that publication during training. Record whether search ran: a retrieved source can supply newer information than the training cutoff.

You cannot see the switch, but you can see its trace. An answer that shows sources retrieved something. An answer with no sources may have searched or may not have. Record "sources shown: yes or no" for every answer, and treat "no" as "unknown, probably memory."

Variance: ask each prompt more than once

The same prompt can return a different answer an hour later: different brands, a different order, a different price. Ask each prompt at least twice per engine, each time in a fresh chat, and record each run as its own row. A brand named in two of two runs holds a stable position for that prompt. A brand named in one of two sits on the edge, and a single run would have told you either story at random.

Two runs is a floor, not a statistical guarantee. The point is to stop treating one answer as a fact about the engine.

Google AI Overviews: the engine that often does not answer

You do not ask Google AI Overviews anything directly. You search, and Google decides whether to show an overview at all. Google's own documentation says AI Overviews often do not trigger.

When no overview appears, record "no overview," not "not mentioned." You were not left out of an answer; there was no answer to be in. Counting it as a miss makes your visibility look worse for a reason that has nothing to do with your brand. If you also check AI Mode, record it as a separate engine. Google says AI Overviews and AI Mode "may use different models and techniques" and can show different links.

Do not record a position. Google's generative AI report in Search Console shows impressions only, with no clicks, no click-through rate and no position, and Google's page on third-party SEO tools says they "don't have access to our internal ranking data." There is no ground truth for a rank inside an overview, so a rank column would hold numbers you made up.

Eligibility, by contrast, is documented. Google says a page must be "indexed and eligible to be shown in Google Search with a snippet," with "no additional technical requirements." If your pages never appear in overviews, the access lessons in Module 3 tell you what to check first.

The recording template

Use one row per answer: one prompt, one engine, one run. A spreadsheet is fine. These are the columns, with three rows from Ledgerly's first audit.

Prompt	Engine	Run	Sources shown	Mentioned	What it said about you	Competitors named	Your site cited
B2 (brand)	ChatGPT	1	No	Yes	"Starter costs $25 per user per month"	None	No
C1 (category)	Perplexity	1	Yes	No	Nothing	Invoxa, Billcraft	No
C1 (category)	Google AI Overviews	1	n/a	No overview	n/a	n/a	n/a

Add the date and the session mode (logged out, memory off, search toggle on or off). In the "What it said" column, paste the full sentence that names you, word for word. In the last lesson of this module you grade every stated fact against a canonical fact sheet, and you will need the exact words.

Record sentiment loosely, as positive, neutral or negative, based on how that sentence reads to a buyer. "Ledgerly is cheaper but has no mobile app" is a different finding from "Ledgerly is a solid option for agencies."

Record citations as URLs, not counts. A citation tells you the engine linked a page. It does not prove the page shaped the answer. OpenAI's own documentation says "the number of sources is often greater than the number of citations," so the pages an engine used and the pages it cited are different sets.

What the 99Visibility audit automates, and what it does not

The 99Visibility audit runs the same discipline on a schedule. You enter your prompts (the product calls them keywords) and it asks each one, exactly as typed, on ChatGPT, Google AI Overviews, Perplexity, Gemini and Claude; how many of the five depends on your plan. For every answer it records whether you were named, the tone, the links cited, and any stated fact that contradicts the brand facts you entered. A query where Google shows no AI Overview is skipped, not counted as a miss, and an answer that never came back is left out rather than scored as "not mentioned."

It also pins down the retrieval state for most engines, which a manual audit cannot. The ChatGPT and Claude columns are queried through their APIs without web search, so they measure what those models already know about you, not what ChatGPT search would say. That is our measurement choice, not a property of either product: Anthropic has shipped a web search tool for Claude, and ChatGPT searches when it decides to. Perplexity runs with web search on. Gemini runs with Google Search grounding available, so the model decides per query whether to search. The AI Overviews column reads Google's live results page.

What it does not do: see the rewritten retrieval queries, reproduce what a logged-in user with memory sees, or give you a rank inside an answer. Each run asks every prompt once per engine, so read the trend across runs rather than any single run. To try it small, the free checker at /tools/ai-visibility-checker asks up to 3 keywords on ChatGPT and Google AI Overviews. The next lesson, Understanding Your Visibility Score, shows how those answers become a number.

Common mistakes

• One run per prompt. A single answer is a coin flip you mistook for a measurement.
• Auditing from your own account. Memory and past chats shape the answer you get. Your buyer does not have your history.
• Leading prompts. A prompt that contains the answer you want measures the engine's politeness.
• Only brand queries. They name you by construction, so the mention rate they produce flatters you.
• Counting "no overview" as a miss. Nothing was shown, so nothing was missed.
• A rank column. No ground truth exists for a position inside an AI answer.
• Rewording prompts between audits. You broke the series. Old and new results are no longer comparable.
• Reading a citation as proof of use. Engines consult more pages than they cite, and cite pages they barely used.