Hallucination Detection
Hallucination detection compares every audited answer with the facts on file for the brand and flags statements that contradict them. It powers the Accuracy part of the score. Flag counts and severities show on every plan; what each flag says is on Starter and above.
Input: brand facts
Each fact is {key, value, category}, entered under Brands → Edit → Brand Facts or during setup. Categories: pricing, feature, founding, leadership, other.
Checks
| Check | Runs on | Flags when | Type | Severity |
|---|---|---|---|---|
| Price | Facts with category pricing | Any dollar price in the answer differs from the fact's price by more than 10%. Only $ amounts are read. | incorrect | moderate |
| Founding year | Category founding, or a key containing "founded" | A different year appears near "founded", "established", "started", "launched" or "created". | incorrect | minor |
| Location | A key containing "location" | Text after "headquartered", "based in", "located in", "offices in" or "from" names none of the fact's place names. | incorrect | moderate |
| Feature | Category feature | The answer says the brand lacks the feature named by the key ("doesn't have", "no", "lacks", "without"). | incorrect | moderate |
| Acquisition | Every answer, unless a key contains "acquired" | The answer contains "acquired by", "bought by", "purchased by" or "merged with". | hallucination | critical |
| Staff count | Every answer, unless a key contains "employee" | The answer states a number of employees, workers or staff. | hallucination | minor |
Leadership facts, and Other facts without "location" in the key, are stored but not checked.
Known limits
- Checks read the whole answer, not only sentences about the brand. Another company's acquisition, or a second plan's price, can flag a correct answer.
- "from" is read as a location marker, so a phrase like "from London" can trip the location check.
- A claim that the brand offers a feature it does not cannot be checked: there is no category for features you lack.
- There is no dismiss action. Correcting a fact clears flags it caused on the next audit; the two cases above need a fix in the check itself.
Accuracy score
Accuracy is a rate over measured answers (formula v3-2026-10-24). Each flag carries a weight by severity, with a surcharge for the hallucination type, and one answer's penalty is capped. A claim no fact on file covers (an unrecorded acquisition, an employee count) is shown as unverifiable and left out of the score: it cannot be checked, which is not the same as wrong. The score is 100 × e^−(a × density + b × worst): density is total penalty over the maximum possible for the number of answers, and worst is the heaviest single flag. One catastrophic error therefore still shows in a large audit. Computed with the live function:
- One critical acquisition flag in 20 answers: accuracy 67.
- One moderate price flag in 20 answers: accuracy 84.
- One minor founding-year flag in 20 answers: accuracy 94.
- The same moderate flag in 100 answers: accuracy 87.
No flags, or no measured answers, gives 100. With no facts on file the audit stores accuracy_checked: false and the page shows "Not checked"; the acquisition and staff-count checks still run.
Output
| Field (export) | Stored as | Meaning |
|---|---|---|
type | flag_type | incorrect or hallucination (outdated exists in the schema; no check emits it) |
severity | severity | critical, moderate, minor. Shown as Critical, Moderate, Minor. |
claim | ai_statement | What the answer said. |
fact | correct_fact | The fact it contradicts. |
explanation | source | One sentence on the mismatch. |
Below Starter, the audit API and its exports return type and severity only, with accuracy_flags_gated: true. A critical flag triggers the Accuracy alerts email when that preference is on.
Correcting a wrong fact at the source
A flag points at a contradiction; the fix is on the pages the platforms read. Make every copy of the fact on your own site agree, make sure it is in the server-rendered HTML, confirm retrieval crawlers can reach the page with the access check, and correct third-party pages the answer cited. An answer that searched can change once the platform re-reads the page; an answer from memory changes only when the model is retrained.
From the customer side: AI is saying something wrong about your brand and A hallucination flag looks wrong. To write facts the check can read: Your brand facts are out of date.