Loading insights
Loading insight article
Insights / Governance
Two years after Mata v. Avianca, hallucinated citations in court filings haven't slowed down, they've doubled and courts now treat them as a governance failure, not a one-off mistake. Here's the verification workflow firms with clean records actually use.
Two years ago, a Manhattan federal judge fined two attorneys $5,000 for filing a brief citing six cases ChatGPT had invented from nothing. Steven Schwartz and Peter LoDuca became the punchline of every legal-tech conference for the next eighteen months.
That lesson didn't work. Researcher Damien Charlotin's database now tracks more than 1,500 U.S. court cases involving AI-generated fabrications and the growth curve hasn't slowed, it's steepened. Firms that read every "verify every citation" listicle are still getting sanctioned. The real question two years on isn't what lawyers should know, it's why the knowing keeps failing to translate into practice and what firms who've actually solved this are doing differently.
Mata was one lawyer, one brief, six fake cases. What's changed since isn't the failure mode, it's the severity of the response.
Two years ago, a hallucination meant a public scolding and a fine. Today, courts treat it as a governance failure. A June 2026 Mississippi case, Withers v. City of Aberdeen, shows the shift: both sides filed briefs with hallucinated citations and the judge removed all four attorneys from the case, two banned from the district for two years, reasoning less about the fake cites themselves and more about whether each lawyer's conduct matched their firm's own policy.
That's the real 2024–2026 shift: hallucinations are no longer one-off mistakes. They're evidence of whether a firm has a functioning verification system at all.
It's getting worse, not better. Damien Charlotin's tracking database, the most cited source on this, counted 1,598 cases worldwide as of June 9, 2026, up from 719 in January and 1,227 in early April. That's a pace of roughly 8 new cases a day, faster than the 5–6 a day reported just two months earlier. By his most recent update on July 2, 2026, the count had climbed to 1,668, with 1,163 from U.S. courts alone and 653 of the worldwide total involving a licensed attorney, not just someone representing themselves.
Two years of bar guidance, sanctions and better tooling haven't slowed this down. If anything, wider AI adoption is outpacing the profession's ability to catch it.
The ethics rules aren't new, AI use just triggers ones that already existed. AI doesn't change your duties, it just changes how you breach them. ABA Formal Opinion 512 sets that baseline: lawyers may use generative AI but remain fully responsible for the accuracy of anything filed, tying AI use directly to the existing Model Rules.
Scroll horizontally to view all columns.
| Rule / Standard | What it requires | How it applies to AI |
| Model Rule 1.1 Competence (ABA Formal Opinion 512, 2024) | Lawyers must understand tools well enough to use them responsibly | No need to understand how the model works internally but must understand its limitations well enough to supervise its output |
| Rule 3.3 Candor to the tribunal | Lawyers can't knowingly submit false statements to a court | A fabricated citation isn't excused by "the AI generated it", the attorney who signed the filing is responsible, knowingly or not |
| Rules 5.1 / 5.3 Supervision | Partners are responsible for associates' work; lawyers are responsible for non-lawyer staff | Extends to AI-assisted drafting, same standard as supervising a paralegal's research |
| Rule 11 Reasonable inquiry | Attorneys must verify filings before signing, not after a judge flags them | Courts increasingly treat "I asked an AI and didn't check" as failing this standard by definition |
| Standing orders / certificates | Some courts require disclosure of AI use in a filing | No national standard, requirement depends entirely on that specific court's local rules |
The advice sounds airtight: verify every citation before it leaves your desk. In practice, it collapses under ordinary human behavior and firm workflow design which is why lawyers keep asking "how do I avoid AI hallucinations?" long after reading the bar warnings.
Fluent, confident AI output lowers scrutiny by default. Researchers call this automation bias or "cognitive surrender" the tendency to trust machine output unless something visibly breaks. A hallucinated case name or quote reads as authoritative before anyone checks it which is precisely the problem.
Most firms place the entire burden on the drafting attorney: “You must verify.” But verification is time‑consuming and billable pressure pushes lawyers to skim rather than rigorously cite‑check. When the workflow assumes perfect individual discipline, hallucinations slip through especially under deadlines or in high‑volume practices
Checklists help but they fail when they’re optional, vague or disconnected from filing gates. A “verify your AI output” note in a policy PDF doesn’t stop a hallucinated citation from reaching a brief if there’s no hard stop, no second set of eyes and no audit trail
Firms that keep hallucinated cases out of their filings don't rely on individual lawyers "being more careful." They build a few concrete checkpoints into the workflow so a fake citation has to pass through more than one gate before it reaches a court.
Dual sign-off
No AI-assisted brief goes out on one person's approval alone. A second attorney, not the one who drafted the section, checks every citation before filing. This catches what the drafting attorney's own automation bias makes them likely to miss, since a second reader isn't anchored to the same fluent-sounding text.
How it works in practice: The drafting attorney flags which sections used AI assistance and the reviewing attorney treats those sections as unverified until each cite is pulled and confirmed.
Designated AI-review role
Larger firms assign one person (or a small team) to own AI-output verification often a senior associate, paralegal, or a firm's litigation-support specialist. This isn't about slowing things down; it's about making sure verification isn't left to whoever is busiest that week.
Why it works: When verification is someone's actual job, it happens on every filing, not just the ones where someone remembers to double-check.
Written AI policy & CLE
A short, mandatory policy, not a long PDF nobody reads that spells out exactly what's allowed: which tools, what needs citation verification and what the sign-off process is. Firms pair this with short, recurring CLE training so lawyers understand why AI hallucinates and why fluent text can still be wrong.
Why it works: A policy with no training behind it gets ignored. Training with no policy behind it has no teeth. Together, they turn "verify AI output" from a suggestion into a firm standard.
Scroll horizontally to view all columns.
| Checkpoint | Who's responsible | What it prevents |
| Dual sign-off | Reviewing attorney (not the drafter) | Citations approved only by the person most likely to trust them |
| Designated AI-review role | Assigned reviewer / paralegal | Verification being skipped under deadline pressure |
| Mandatory source links | Drafting attorney (at time of citing) | A citation that doesn't resolve to a real source slipping through unnoticed |
| Written AI policy | Firm management | Inconsistent or informal use of AI tools |
| CLE training | All attorneys using AI | Lawyers not recognizing when or why AI citations can be wrong |
| Audit trail | Firm-wide process | No record of who checked what, if a citation is later challenged |
Generic chat models are built to sound confident, not to be right about your jurisdiction. Custom-built legal software especially systems using retrieval-augmented generation (RAG) reduces hallucination risk by design, not by luck.
Yes, but not for the whole job. The risk isn't using AI to draft, it's where in the process you use it.
Safer (structure/first-draft language): Organizing a motion, tightening prose, generating a first pass, a human rewrites the substance anyway, so the model doesn't need to be right about the law.
Moderate (memos/clauses): Internal memos and contract-style clauses carry more risk than pure formatting, since they can embed a legal conclusion but they're still reviewed before anyone relies on them, unlike a citation that slips straight into a filed brief.
High risk (case law/statutes): This is where it goes wrong. Asking AI to find supporting case law or cite a statute is exactly the task it's worst at, because a fabricated case and a real one look identical on the page.
The rule: let AI shape how a motion reads. Never let it be the source of what the law says. If a citation didn't come from a source the attorney actually opened, it doesn't go in the filing
Malpractice carriers have started pricing AI risk into premiums firms that can show a written policy, training records and a verification step independent of the drafting attorney are treated differently than firms that can't document any of it. That's turned "we should have a policy" from a compliance suggestion into a line item firm leadership actually funds.
If your firm filed one: Disclose it to the court and correct it before anyone else finds it. Attorneys who've weathered this best took full responsibility immediately rather than arguing the underlying legal point was still sound even if the case wasn't real, courts have shown little patience for that argument.
If you found one in opposing counsel's filing: Raise it with them directly before going to the judge. Rule 11 and most state equivalents build in a notice-and-cure window and judges routinely ask whether that step happened first before entertaining a sanctions motion.
Mata v. Avianca proved that "verify every citation" isn't enough on its own, AI can produce a fabricated case that's completely convincing and a busy lawyer can miss it. The fix was never a better prompt. It's a workflow: source checks, a second reviewer, a written policy and an audit trail, built into filing so none of it depends on memory. Two years on, the firms avoiding sanctions aren't the ones using less AI. They're the ones with a process that catches what AI gets wrong before a judge does.
A verification step that doesn't depend on the drafting attorney remembering to do it typically a second-reviewer sign-off built into the filing workflow itself, not a policy that relies on individual discipline.
Treat every AI-generated citation as unverified until it clears three checks: it exists in a primary source, that source actually says what it's cited for and it's still good law. Skipping any one of the three is how hallucinations make it into a filed brief.
Yes, for structure and language. AI motion drafting is high-risk only when used to find case law or statutes, every citation still needs to trace back to a source a human has personally opened and read.
It depends on the jurisdiction. Some federal districts have standing orders requiring disclosure or certification of AI use in a filing; others don't yet, so checking local rules before every AI-assisted filing is necessary.
Tell me about the system or workflow you are working on.
30-minute intro call on Calendly
