🧾 Keep the Receipts: The Government Says Save the AI Evidence, a Lawyer Says Save the Prompts, and Nobody Has Checked Legal AI's Math Since 2024

Or: NIST, NSA and CISA AI Incident Guidance, the Missing Chain of Custody for AI Records, Legal AI Accuracy After Stanford, AI-Drafted Lawsuits and a Florida Bar AI Opinion: The Week’s eDiscovery AI News in Plain English

By Thomas Seest, Dunlap.Today · Sunday, October 18, 2026

Anybody who has ever returned something at the hardware store knows the drill. The first question at the counter is, “Do you have the receipt?” Not “Is the drill broken?” Not “Do you seem honest?” Just: show me the paper trail.

This week’s eDiscovery news kept coming back to that same question, only now it’s about AI. When a computer helps write a memo, sort documents or run an errand, what’s the receipt? Who saved it, how long do they keep it, and has anybody checked whether the work was any good? Three federal agencies, a lawyer at a Cozen O’Connor eDiscovery affiliate and an independent reviewer all weighed in, from different directions.

Here’s this week’s roundup: federal guidance on saving AI evidence, the missing paper trail behind AI-made records, a stale scoreboard for legal AI accuracy, founders reporting a wave of AI-written lawsuits, the Florida Bar’s look at AI note-takers, and what corporate legal departments haven’t decided yet.

⏱️ What this means for you in 30 seconds

• Using AI at work? The final answer may not be the only thing a court asks for. The prompts, sources and edits could matter too.
• Got a legal hold? Make sure it reaches AI chats, AI meeting notes and the logs behind them.
• Buying legal AI? There’s no current neutral scorecard. Test it on your own work before you trust it.
• Running a business? AI makes it cheaper for anyone to sue. Have a plan for sorting real claims from junk.
• Hiring a law firm? Tell them in writing how they may and may not use AI on your matters. About 43% of legal departments haven’t yet.

🧾 Washington Says Save the Evidence, But Not for How Long

Three federal security outfits, NIST’s AI safety center, the NSA and CISA, published a 14-page guide for companies building the most powerful AI systems. It covers what to do when an AI system misbehaves during training or testing: ten voluntary practices in five steps, from preparing ahead of time to reporting afterward.

An analysis by ComplexDiscovery pointed out the part that matters for lawyers. The guide tells developers to preserve evidence “to the extent necessary to investigate,” including the AI model’s saved states, sandbox and network logs, and even the actions the AI tried to take but was blocked from taking, all in tamper-resistant logs. What it doesn’t say is how long to keep any of it. It never mentions lawsuits at all. And other parts of the guide tell teams to wipe and rebuild a compromised system, which is exactly the sort of thing that can destroy evidence a court might later want.

The analysis suggests a simple fix: take a snapshot and a digital fingerprint (a “hash”) before you rebuild. It also notes the guidance “may also apply to deployers,” meaning ordinary companies using AI, not just the labs building it.

It’s like the fire marshal telling you to keep the burned toaster for the investigation but not saying whether that’s for a week or a year, while the insurance adjuster tells you to clean up the kitchen.

What it means for you: If your company runs AI agents, find out today what logs they keep and for how long. Before anyone “resets” a misbehaving system, save a copy first.

👉 Read more (ComplexDiscovery)

🔗 The Missing Paper Trail Behind AI-Made Records

Nicole Marie Gill of CODISCOVR, the eDiscovery company affiliated with the law firm Cozen O’Connor, wrote in Thomson Reuters’ Westlaw Today that companies are missing a “chain of custody” for AI-generated business records. If an AI drafts a report and someone saves just the finished report, she argues, a lot of the story is gone: what the person asked, which documents the AI looked at, which version of the AI answered, and what a human changed afterward.

Her advice: update legal holds so they reach those inputs, check whether your AI vendors can even preserve them before you sign, and take a risk-based approach instead of trying to save everything forever.

Think of a family recipe card that just says “cake.” Nice to have, but if anybody ever argues about how Grandma made it, you’ll wish you’d written down the ingredients and who tweaked what.

What it means for you: Ask your AI vendors one question this week: “If we got sued, could you give us the prompts, sources and version history behind an output?” Write down the answer.

👉 Read more (CODISCOVR)

📏 Nobody Has Checked Legal AI’s Math Since 2024

On The AI Law Blog, Erick Robinson looked for current, independent numbers on how often commercial legal research AI gets things wrong. His finding: the last neutral figures are still Stanford’s 2024 study, which found made-up or wrong answers about 17% of the time for Lexis+ AI and about 33% for Westlaw’s AI-assisted research.

Newer tests come from the industry itself. In a benchmark Harvey runs with Artificial Analysis, the best AI models fully completed only about one legal task in four, and fewer than one in ten once made-up facts were counted against them. Meanwhile, researcher Damien Charlotin’s list of court cases involving AI fabrications has grown from about 120 in May 2025 to 2,184 as of Oct. 10, 2026. Worth knowing: the author builds AI tools himself and uses some of the products he discusses.

It’s like buying a used truck where the only inspection report is two and a half years old and the dealer did the newer ones.

What it means for you: Treat vendor accuracy claims as a starting point. Run any legal AI tool on a sample of your own work, check every citation, and keep a record of who checked.

👉 Read more (The AI Law Blog)

📬 “A Huge Amount of Frivolous Lawsuits”

On X this weekend, Wharton professor Ethan Mollick warned that governments and companies are about to be “flooded with agentic AI inquiries that overcome the frictions that gated access,” and will have to choose whether to make their services better or worse in response. Lindy founder Flo Crivello replied with a real-world version: “several founder friends have told me they’ve started getting hit with a huge amount of frivolous lawsuits.” Dean W. Ball, who works at OpenAI and posts his own views, wondered what happens “if people feel they can self-represent when launching frivolous lawsuits.”

At the same time, a Danish startup called SmallClaims raised about 15 million Danish kroner to do something similar on purpose: use AI to file small-business lawsuits over unpaid invoices. It says nearly half of business invoices in Western Europe are paid late and three in ten small firms simply give up on collecting. Its lead investor says the team wins more than nine out of ten cases.

It’s like when the post office started delivering free. Suddenly everybody could send a letter, and some of that mail was junk.

What it means for you: These are anecdotes, not statistics. Still, if you run a business, have a simple process for looking at a new legal claim quickly and cheaply, and loop in a lawyer early. If you’re owed money, AI collection tools may make small claims worth pursuing.

👉 Read more (Flo Crivello on X) · Ethan Mollick on X · SmallClaims (CPH AI Hub)

🎙️ Florida Looks at AI Note-Takers and AI Agents

The Florida Bar’s ethics committee will consider a new advisory opinion on AI at its Dec. 10 meeting. There’s no draft yet, but topics may include AI “agents” that take actions on their own, AI tools that record, transcribe and summarize conversations, what lawyers should tell clients about the clients’ own AI use, and updates to Florida’s 2024 AI opinion. Comments are due, postmarked, by Dec. 1.

That AI note-taker on your video call is making a record, much like a court reporter who never goes home. Records can be asked for in a lawsuit.

What it means for you: If your office uses AI meeting recorders, decide how long transcripts are kept and who can see them, before somebody asks.

👉 Read more (The Florida Bar News)

🏢 Corporate Legal Hasn’t Set the Rules Yet

Early results from Law.com’s 2026 corporate counsel survey found that 43% of legal departments have no formal position on outside lawyers using AI on their matters. AI governance is becoming a big part of in-house work, but few departments measure their automation efforts. In the UK, an HSBC survey of larger law firms found 73% of leaders named AI integration their top worry, ahead of cyber security at 54%.

What it means for you: If you hire lawyers, a one-page AI policy for outside counsel is an easy win. If you are a law firm, ask your clients for theirs.

👉 Read more (Law.com) · HSBC survey (Legal Futures)

🧰 New Tools on the Workbench

• Westlaw research inside Smokeball. Smokeball’s Archie AI assistant is adding Westlaw Deep Research, with every citation linked back to Westlaw so lawyers can check it. Smokeball
• Clio for legal-plan lawyers. Clio and MetLife Legal Plans teamed up to offer discounted Clio Manage to the 18,000+ attorneys in MetLife’s network. Clio
• Legora keeps growing in Europe. Benelux firm Loyens & Loeff picked Legora, saying AI adoption “requires the right governance, training and ways of working.” Loyens & Loeff
• Harvey heads to Tokyo. Harvey plans to open a Tokyo office in early 2027. Harvey

🧭 Quick Hits for the Desk Drawer

• AI lawsuits are about behavior now. J.S. Held’s AI Disputes Monitor counts 468 AI-related matters, and says cases are shifting from how AI was built to what it does, with evidence like prompts, outputs and logs. PR Newswire
• Ransom crews are targeting law firms. Researchers say 42 of the last 50 known victims of one social-engineering extortion group were law firms. Train your front desk and help desk to verify callers. Legal Futures
• Judges check their AI too. The UK judiciary’s new position statement says every AI output used in judicial work must be reviewed, verified and approved by the judge. Judiciary UK
• Saskatchewan’s rule: use it, verify it. A provincial court directive allows AI in criminal court submissions, for lawyers and self-represented people alike, but requires checking every citation. Practice Directive 19 (PDF)
• Check the other side’s citations, too. Judges are increasingly faulting lawyers who don’t flag fake AI citations in an opponent’s filing. ABA Journal
• Some associates aren’t fans. A former Latham lawyer, now building an alternative product, says complaints about firm-provided legal AI tools trend on the Biglaw subreddit almost monthly. X post
• “Fear is not a strategy, but neither is blind optimism.” At Canada’s Legal Summit, the keynote speaker argued AI literacy is now part of being a competent lawyer. Canadian Lawyer
• Old problems, new tools. A legal knowledge-management veteran says AI changed the tools, not the job: getting busy lawyers to share what they know. Artificial Lawyer

✅ Your Front-Porch eDiscovery AI Checklist

• ✅ List the AI tools your team uses and what each one saves (prompts, sources, versions, edits).
• ✅ Add “AI chats, AI meeting notes and AI logs” to your legal hold template.
• ✅ Ask AI vendors, in writing, whether they can preserve and export that history.
• ✅ Before resetting or rebuilding a misbehaving AI system, take a snapshot and a hash.
• ✅ Set a retention period for AI meeting transcripts and stick to it.
• ✅ Test legal AI on your own sample work, and keep a log of who checked the citations.
• ✅ Give outside counsel a one-page AI use policy, or ask your clients for theirs.
• ✅ Have a quick, cheap triage process for new legal claims, AI-written or not.

Practical rule: If an AI helped make it, keep the receipt: what went in, what came out, and who checked it.

— Dunlap.Today

📚 Plain-English AI Glossary

• Chain of custody: The record of where a piece of evidence came from and who handled it along the way.
• Legal hold: A notice telling people to stop deleting information that might matter in a lawsuit.
• Preservation: Keeping evidence safe and unchanged once a dispute is likely.
• Rule 37(e): The federal court rule on what happens when electronic information that should have been kept is lost.
• Hash: A digital fingerprint that proves a file hasn’t changed.
• AI agent: An AI tool that takes actions on its own, like sending emails or running searches, instead of just answering questions.
• Prompt: The question or instruction a person types into an AI tool.
• Hallucination: When an AI tool confidently makes something up, like a court case that doesn’t exist.
• Benchmark: A standard test used to compare how well different tools perform.
• Advisory opinion: A bar association’s written guidance on how ethics rules apply to a situation.

❓ Frequently Asked Questions

Do I have to follow the NIST, NSA and CISA guidance?

No. It’s voluntary, and it’s written mainly for companies building the most advanced AI. But it describes what AI evidence exists, and lawyers on the other side of a dispute may point to it when asking what you should have saved.

Are my AI chats discoverable in a lawsuit?

They can be, if they’re relevant and the request is reasonable, just like emails. That’s why legal holds should mention them.

How accurate is legal AI?

Nobody has a current independent answer. The last neutral study, from Stanford in 2024, found error rates of roughly 17% to 33% for two major research tools. Newer tests are run by the industry. Check the work yourself.

Can I comment on the Florida Bar’s AI opinion?

Yes. The Bar says comments should be postmarked by Dec. 1 and sent to its ethics counsel. Details are in the Florida Bar News notice.

🔗 Other Resources

• ComplexDiscovery: the agent-evidence guidance, explained
• Florida Bar: proposed AI advisory opinion and comment instructions
• UK Judiciary: Position Statement on the Judicial Use of AI
• Saskatchewan Provincial Court: Criminal Practice Directive 19 on AI (PDF)
• Law.com: 2026 Global Corporate Counsel Benchmarking Survey

🌄 No Need to Worry, Just Stay in the Know

None of this means you need to save every scrap of paper or stop using AI. These tools can save real time and money. It just means doing what folks around here have always done at the hardware store: hang on to the receipt, know what you bought, and check it works before you need it.

We’ll keep watching what matters for eDiscovery providers and their clients. Read Dunlap.Today and sign up for our newsletter for plain-English updates.