AI & Accuracy · 6 min read
Why AI invents citations — and what it means for your filings
When a generative AI tool hands you a case citation — a name, a reporter volume, a pinpoint page, a confident parenthetical — it is easy to assume it retrieved that case from somewhere. It did not. Understanding why is the difference between trusting the output and verifying it.
Language models predict; they do not retrieve
A large language model is, at its core, a system for predicting the next piece of text. Trained on enormous amounts of writing, it learns the shape of language: how a sentence tends to continue, what a legal citation usually looks like, which words follow "see also."
What it does not have is a lookup table of real cases that it consults before
answering. Unless it is explicitly connected to a legal database, the model is generating
the most statistically plausible continuation of your prompt. A citation that reads
Johnson v. Carter, 412 F.3d 118 (9th Cir. 2005) is plausible-looking text — the volume
number is in range, the court abbreviation is formatted correctly, the year is reasonable.
Plausible is not the same as real.
Why a fake citation looks perfect
The very thing that makes these tools impressive — fluent, well-formatted output — is what makes their mistakes dangerous. A hallucinated citation does not arrive with a typo or a broken format that tips you off. It arrives looking exactly like a citation a careful associate would produce, because the model learned its format from millions of real ones.
That means you cannot catch fabrications by skimming for things that "look wrong." They look right. The only reliable check is to confirm, in an authoritative source, that:
- the case exists,
- it is accurately presented (the quote, the holding, the posture), and
- it actually supports the proposition it is cited for.
Retrieval helps, but does not eliminate the duty
Newer tools connect models to legal databases so they can quote real material. This genuinely reduces invention. But it does not remove your obligation: retrieval tools can still pull the wrong case, misread a holding, or attach a real citation to a proposition it does not support. The signature on the brief is still yours.
The practical takeaway
Treat every AI-produced citation as an unverified lead, not a finding. Use the tool to do the tedious work — pulling citations out of a draft, surfacing what to check — and keep the judgment where it belongs: with a human who reads the source and signs the filing.
That is exactly the workflow ComplyAssist is built around. It extracts every citation from your brief and walks you through a disciplined, three-prong review, then produces a signed record showing the citations were AI-extracted and independently verified by you.