Course outline · lesson 15 of 20

Module 05 · Engine Playbooks · Lesson 2 of 4

Optimizing for Perplexity

Perplexity shows its sources, but which sources it shows depends on whether its crawler reached you and whether search was on at all. Check both, then log what it cites for your category.

8 min read4-question checkIntermediateUpdated 2026-10-16
0/20 passed

By the end of this lesson you can

  • Separate what PerplexityBot and Perplexity-User each answer to
  • Explain why a Perplexity citation proves neither use nor accuracy
  • Spot a Perplexity integration that answers without web search
  • Audit which sources Perplexity cites for your category

Perplexity has a reputation as the easy AI platform. It is a search product first, it shows numbered sources under every grounded answer, so the story goes: publish good content and the citations follow. Each part of that is half true, and the false halves are where brands lose.

The sources Perplexity shows are decided by things most teams never check: whether its crawler could reach your pages, whether your facts were in the HTML it fetched, and how well a page matched the query it actually ran. And the "Perplexity" your buyer meets inside another product may not search at all.

This lesson covers Perplexity's two agents, how it cites, the API trap and the one content factor with reproducible support. It ends with a ten-minute audit of what Perplexity cites for your category.

Two Perplexity agents, two different controls

Perplexity documents two agents, and they answer to different controls.

AgentWhat Perplexity saysWhat that means for you
PerplexityBotCrawls for search indexing and is "not used to crawl content for AI foundation models."Block it and your pages leave the Perplexity search index. There is no training trade-off to weigh: blocking it buys you no training opt-out.
Perplexity-UserFetches a page when a user's question calls for it. "This fetcher generally ignores robots.txt rules."Robots.txt is not a control here. Only your firewall decides whether it gets through.

That second row cuts both ways. If you want Perplexity-User out, a robots.txt line will not do it. If you want it in, a CDN rule that challenges it will keep it out whatever robots.txt says. Perplexity publishes IP lists for both agents, so your CDN can verify them instead of trusting the user-agent string alone.

Cloudflare’s September 15 announcement says existing settings carry over. New ad-supported domains are offered Search: Allow, Training: Disallow AI Training, and Agent: Block on pages with ads. Training Block also blocks mixed-use crawlers such as Googlebot and Bingbot; Disallow AI Training preserves their search access. Check the settings of your own zone.

Robots.txt is necessary and not sufficient on every platform. Cloudflare, the party with the widest network view, says site owners "might be unsure whether these bots even respect robots.txt rules, especially newer or less transparent crawlers." Pair every robots.txt check with a reachability check. AI Crawlers and robots.txt, Bot by Bot has the full per-agent method.

Evidence: Perplexity's crawler documentation (docs.perplexity.ai, 2026-01-29) states PerplexityBot is "not used to crawl content for AI foundation models," and that Perplexity-User "generally ignores robots.txt rules." Index-time exclusion and answer-time exclusion are different controls.

How Perplexity cites, and what a citation does not prove

Perplexity cites whole sources by integer index: the [1], [2] and [3] markers point to a page, not to a sentence on it. Anthropic does the opposite. Claude's search tool cites text spans, a quoted passage of up to 150 characters. The two units are not the same thing, which is why no honest tool averages them into one cross-platform "citation" figure without saying so.

Perplexity's API also returns the retrieval set separately from the citations: each search result carries an id, URL, title, snippet, date and last-updated field. The retrieved set is usually larger than the cited one. A page can be read and dropped.

A citation does not mean the page was used the way the answer implies. An ECIR 2026 study of Llama-3.1-8B (arXiv:2606.28358) repaired over 90% of missed citations by amplifying specific attention heads, without changing the sources at all. Citation output can be separated from source influence, so citation counts are a noisy proxy for it.

Nor does a citation mean the answer is right. The Tow Center at the Columbia Journalism Review tested AI search tools on news queries and found more than 60% of answers incorrect overall, with Perplexity wrong on 37%. PwC measured deep-research agents at 39–77% factual accuracy while their links resolved more than 94% of the time: real pages that did not support the claim. That is why 99Visibility checks what an answer says against the brand facts you enter, not only whether it linked you. See Find What AI Gets Wrong About You.

The API trap: Perplexity without the web

On Perplexity's Agent API, web search is opt-in. A developer has to request the web search tool. Code that does not request it gets a Perplexity model with no retrieval, and the answer comes back looking normal, with no error, straight from model memory.

The trap got wider when Perplexity announced, in a forum post, that its Sonar Chat Completions endpoint would retire on 2026-09-27. Code moved to the newer API without opting search in became ungrounded without anyone noticing. Any "Perplexity" answer inside a third-party product, a sales tool, a browser extension or another GEO tool may be one of these.

Two rules follow. When an integration's Perplexity answers never show sources, suspect that search is off before you suspect your site. And when a vendor shows you a Perplexity column, ask whether web search is on. 99Visibility sends the web search tool on every Perplexity call, so its Perplexity column is a retrieval-on measurement.

The same discipline applies to your own testing in the Perplexity app. An answer with a source list was grounded. An answer without one may not have been, and you should not log it as evidence about your site's access either way.

Access: what PerplexityBot needs from your site

Perplexity can only cite what PerplexityBot or Perplexity-User could fetch and read. Three checks cover it:

  1. Robots.txt. Find the group that applies to PerplexityBot. A crawler follows its own named group if one exists and falls back to User-agent: * only when none does, so a broad Disallow under * catches it whenever it has no group of its own.
  2. The firewall. Bot-fight modes, rate limits and JavaScript challenges stop PerplexityBot the same way they stop OAI-SearchBot. A 403, a 429 or a challenge page is a failed fetch. When robots.txt Says Yes and Your Firewall Says No covers the fixes.
  3. The raw HTML. Independent re-testing in June 2026 found PerplexityBot does not render JavaScript. Facts that appear only after scripts run are invisible to it. View source and look for your price, plan names and integrations in the text. 99Visibility does not run this rendering check, so do it by hand.

None of this makes Perplexity choose you. It decides whether you are in the pool at all. That is the honest scope of access work, and it is still the highest-confidence work there is.

Relevance: the one content factor that holds up

Once a page is reachable, what decides whether it gets used? A July 2026 critical survey of 45 GEO studies (arXiv:2607.14035) concluded that "topical relevance and context position are the most reproducible levers," and warned that "citation-oriented rewrites can impair retrieval." A separate 252,000-trial factorial study (arXiv:2605.25517) also found topical relevance and list position were the biggest drivers of being cited first. One source is a single-author preprint and the other an industry paper, so treat their agreement as corroboration, not proof.

Position in the model's context list has no publisher lever: it is set inside the retrieval system. Relevance is the part you control, and it is not keyword coverage. The original GEO paper (KDD 2024) found keyword stuffing lowered its own visibility metric, 17.8 against a 19.5 baseline. Relevance means having a page that actually answers the question a buyer asks, with your real facts on it in plain text.

In practice, list the questions a buyer asks before they buy: what it costs, what it connects to, who it is for, how it compares. For each one, check that you have a page that answers it in server-rendered text, and that the answer matches every other page on your site. That is a content inventory, not a trick, and it is the only form of "optimizing for Perplexity" the evidence supports. How to Create Content AI Can Quote covers how to write those pages.

The other strong pattern is brand prominence. A study of AI Search Arena data from March to May 2025 (arXiv:2507.05301) found citations heavily concentrated on a few sources, with a Gini coefficient of 0.77 for Perplexity. That is a strong association with brand size, not a lever you can pull this quarter. The long route is off-site corroboration.

Myth: "GEO tactics deliver up to 40% more visibility on engines like Perplexity." That claim traces to a 2024 paper that measured an engine the authors built, with sources loaded into context. Its one "in the wild" Perplexity test uploaded source text as files, because "perplexity.ai does not allow the user to specify source URLs." It never tested how Perplexity finds pages.

Myth: "Perplexity rewards fresh pages, so update yours monthly." Seer's July 2026 study found 65% of pages Perplexity cited had been updated within a year, but it had no control group of pages that were not cited. If most crawlable pages carry a recent date, that figure is no signal at all. Correct dates are worth keeping. Bumped dates are not a tactic.

Worked example

Ledgerly runs five buyer questions on Perplexity, logged out: "best invoicing software for small agencies", "invoicing app that syncs with Xero", "how much does Ledgerly cost", "Ledgerly alternatives" and "invoice software with multi-currency support". It logs every cited source in a sheet.

Two findings stand out. On the pricing question, Perplexity cites a 2024 Ledgerly help article that still says "Growth: $39/month", not the pricing page. The Crawler Access check shows why: the pricing page sits behind a bot-protection rule that returns 403 to PerplexityBot, while the help center runs on a different host with no such rule. On the multi-currency question, Perplexity cites three competitors and a forum thread and no Ledgerly page, because Ledgerly has no page that says it supports multiple currencies. The feature lives in a dropdown rendered by JavaScript.

The fixes are plain: let PerplexityBot through on the pricing host, correct the help article, and publish one server-rendered page that states the currency support in text. No markup, no rewriting for AI.

Common mistakes

  • Relying on robots.txt to block Perplexity-User. Perplexity says it generally ignores robots.txt. Use the firewall if you mean it.
  • Blocking PerplexityBot to opt out of training. It is a search crawler. Blocking it costs you the index and buys no training opt-out.
  • Counting citation position as rank. A [1] marker is an index into the source list, and there is no ground truth to validate a position metric against.
  • Trusting a Perplexity integration that shows no sources. Check whether web search was switched on before you draw any conclusion.
  • Reading a citation as a correct answer. Compare what the cited page says with what the answer claims. They often differ.
  • Fixing the page Perplexity should cite and ignoring the one it does cite. An old help article with a wrong price will keep getting quoted until you correct it.

Key takeaways

  1. 01PerplexityBot builds the search index and is not a training crawler, so blocking it costs visibility and buys no training opt-out.
  2. 02Perplexity-User generally ignores robots.txt, so only your firewall controls it.
  3. 03Perplexity cites whole sources by number and Claude cites text spans, and neither unit proves a source shaped the answer.
  4. 04On Perplexity's API, web search is opt-in, so an integration can answer from memory with no warning.
  5. 05Topical relevance is the one content factor with reproducible support, and it is not keyword coverage.

Your win · about 10 minutes

Know which sources Perplexity cites for your category

  1. Open yoursite.com/robots.txt and confirm the group that applies to PerplexityBot (its own group, or User-agent: * if it has none) allows your key pages.
  2. Run the Crawler Access check and confirm PerplexityBot and Perplexity-User get a 200 from your CDN, not a 403, 429 or challenge page.
  3. Log out of Perplexity and ask five questions a buyer would ask about your category, without using your brand name.
  4. For each answer, record every cited domain in a sheet and mark it as yours, a competitor, a review site, a forum or media.
  5. For each cited page that states a fact about you, mark whether the fact is right.
Run the free Crawler Access check

Knowledge check · 4 questions · pass with 3

Pass this to complete the lesson

Graded on our server. Get 3 of 4 right to mark the lesson complete and move your timeline.

  1. 1.What does Perplexity say PerplexityBot is for?
  2. 2.Why is a robots.txt Disallow a weak control for Perplexity-User?
  3. 3.Perplexity cites whole sources by number, while Claude cites short text spans. What follows for measurement?
  4. 4.Which content factor has the most reproducible support in the GEO research?

See what AI says about your brand

Run a free audit across ChatGPT, Perplexity, Google AI, Gemini and Claude, and check which AI crawlers can reach your site.

Start for Free