Back to Blog

The Problem

Rule 11 of the Federal Rules of Civil Procedure says you have to be able to certify that a filing reflects your own reasonable inquiry. ABA Formal Opinion 512 says you have to independently verify AI-assisted work before it goes out the door. Both of these rules make the same assumption: you can tell, after the fact, which parts of your reasoning were actually yours.

But that assumption doesn't hold up, and nobody in legal governance has stopped to test it.

Why have we missed this though? Likely, it could be because every conversation about AI in legal work so far has been about accuracy. Is the citation real. Is the case still good law. Did the tool hallucinate something. These questions are the right questions to ask at first, and they're the ones every governance framework, every CLE, every malpractice bulletin is built around right now. But accuracy is a question about the content. It's not a question about the memory of where that content came from. Nobody's asked whether an attorney reviewing AI-drafted language will, months later, correctly remember whether that reasoning was theirs or the tool's. This is a different failure mode, and it's one cognitive science has already studied. It's called source monitoring, and it breaks down under specific, well-documented conditions.

It's worth being clear about what this isn't though. Attorneys using AI aren't doing sloppy work or getting worse at their jobs. A specific memory function, studied for decades outside of law, is being asked to operate under exactly the conditions that make it unreliable.

Systems problem, not a competence problem.

How To Test Whether This Is Real

Before pulling research, the question was what would actually have to be true for this to matter.

Two things.

  • Does passively reviewing something create weaker source memory than generating it yourself? If review and authorship leave the same memory trace, none of this matters.
  • Does an AI tool's writing style resemble your own closely enough to make that review-versus-authorship line harder to find? If AI output stays obviously distinct from how you naturally write, misattribution is less likely, no matter how passive the review is.

If both of those are true, this is a real problem. If either one is false, this is a non-issue.


The Evidence

Passive review really is a weaker memory event than generating something yourself. This isn't a new idea in psychology. Information you generate is remembered differently, and more durably, than information you simply read (Marsh et al., 2001). The part that matters here is that this advantage isn't just about remembering the content better. It's about remembering the context, where it came from, which is exactly the piece Rule 11 and Opinion 512 are relying on you to get right.

It is worth flagging directly that a 2025 seven-experiment replication attempt (Schindler & Richter, 2025) tested the broader generation effect — not source memory specifically — on recall of expository text, and found no benefit in prompted recall in any of the seven experiments, with a slight learning disadvantage for generation in four of them. That doesn't erase Marsh et al.'s (2001) more targeted finding about context memory specifically, but it's a reminder the generation-effect literature is far from uniform, and any benefit depends heavily on task and conditions.

Similarity is the specific condition that breaks source memory. This is the finding that makes the topic worth writing about at all. Research on cryptomnesia, the well-documented phenomenon of unknowingly reproducing someone else's idea as your own, found that perceptual similarity between yourself and the source is one of the specific conditions that makes it worse (Macrae et al., 1999). The driver is similarity to the observer specifically, not similarity in general.

This is the whole hypothesis in one line. The more an AI tool's output sounds like something you'd write yourself, the harder it becomes to keep straight what you actually wrote versus what you reviewed and accepted.

People are already bad at telling AI writing apart from human writing. Evaluators trying to distinguish AI-generated text from human writing performed close to chance. Even with training, accuracy only reached around 55 percent (Clark et al., 2021). A related finding makes this worse: raw AI output was somewhat easier to spot. AI output a human had already reviewed and selected was nearly impossible to tell apart from human writing (Köbis & Mossink, 2021).

SPOT THE SEAM Read Both. Then Check Which Is Which. PASSAGE A Under the governing standard, liability attaches once the defendant is shown to have had actual knowledge of the hazardous condition and failed to act within a reasonable window. The record here establishes that knowledge through internal maintenance logs dated weeks before the incident. PASSAGE B Liability attaches under the governing standard once actual knowledge of the hazard is shown, coupled with a failure to act within a reasonable window. Here, the maintenance logs place that knowledge weeks before the incident — not constructive, not inferred, documented. Passage A: AI-drafted, lightly edited by an attorney. Passage B: fully attorney-written, no AI involved. If you couldn't tell before reading this line, that's the point.

That describes exactly what happens in a law office. Nobody files raw AI output. Every AI-drafted paragraph that reaches a filing has already been read, evaluated, and kept by an attorney. That's precisely the condition this research found hardest to detect.

Is AI writing actually matching individual attorneys' voices right now? Mostly not yet. Stylometric research has generally found that AI-generated text clusters together as its own category, distinct from the wide range of individual human writers (O'Sullivan, 2025; Chen et al., 2026). Most legal AI tools today rely on light prompting, not deep personalization, and light prompting doesn't get far. One study found real author-matching required a large volume of an individual's own writing to fine-tune against. With a small sample, accuracy was barely better than guessing. With a large one, it climbed close to 90 percent (Liu et al., 2024).

The risk isn't fully here yet in most tools. It's a forward-looking risk, and it's growing as personalization features improve, not an immediate concern today.

One study looks like it disproves this, but it doesn't, for a specific reason. A 2026 study tested whether GPT-4 could imitate individual writing style well enough to pass a forensic authorship comparison, the kind used in actual legal disputes over authorship. It failed, leaning on generic, formulaic phrasing instead of real individual style markers (Ishihara, 2026).

Forensic authorship analysis and a busy attorney's quick read under deadline pressure aren't the same test. Nobody runs forensic stylometry on their own draft before hitting send. It's a fast plausibility check, the exact kind of surface-level read the earlier research found people are bad at. Deep stylistic distinctiveness is a different question from perceived similarity, and this failure mode runs on the second one.

CONFIDENCE VS. ACCURACY The Gap Isn't Uncertainty. It's False Confidence. 100% 75% 50% 25% 0% Day 0 Week 1 Month 1 Month 3 Month 6+ THE FALSE-CERTAINTY GAP Reported confidence in attribution Actual attribution accuracy Illustrative model based on documented source-monitoring effects — not measured trial data.

Where this leaves us: the idea that the more an AI tool's output sounds like something you'd write yourself, the harder it becomes to keep straight what you actually wrote versus what you reviewed and accepted, is well-supported as a reasonable extension of research that's individually solid into a specific context, legal AI drafting, that hasn't been directly tested yet. This hasn't been proven in a courtroom setting. Every piece it depends on has already been independently confirmed in this research, but nobody has connected those pieces for legal practice until now.


Where AI Governance Frameworks Miss This

Unfortunately, the frameworks that are supposed to cover exactly this kind of risk don't. NIST's AI Risk Management Framework and ISO 42001 both require organizations to document that human oversight of AI output exists. Neither one asks whether the person doing the oversight can accurately account for their own contribution afterward.

NIST requires documentation of how AI output may be reviewed and overseen by humans (NIST, 2023). That's a system-level requirement. It confirms a review process exists. It says nothing about whether the individual doing the reviewing walks away with an accurate memory of what they contributed.

ISO 42001 goes further on paper, and still misses this. It requires AI systems be built so humans can monitor and override outputs, requires a named person with real override authority, and requires documented review and escalation procedures (ISO/IEC 42001:2023). That's a genuinely strong structure for proving review happened and that someone had the authority to stop it. But an attorney who read a paragraph, silently agreed, and moved on satisfies that structure the same way as an attorney who engaged with it deeply enough to form real independent judgment.

Both show up identical in the audit log.

ABA Formal Opinion 512 gets closer but still doesn't close the gap. It says the depth of required verification depends on the task, and document review needs more scrutiny than something like idea generation (ABA, 2024). That's a sensible standard for deciding how hard to look. It says nothing about whether the memory of that review, when a filing gets challenged months later, actually reflects what happened.

None of these frameworks were built to fail here on purpose. They govern system behavior and accountability structure, and they do that reasonably well. What none of them do is ask the one question that actually matters for Rule 11 and Opinion 512 to mean anything in practice: can the attorney who reviewed this accurately account, later, for which parts were theirs.

THE RECONSTRUCTION GAP When Attribution Is Reliable vs. When It's Tested Day 0 Drafting Week 2 Filing Month 2 Discovery Month 6+ Rule 11 / malpractice inquiry ATTRIBUTION IS RELIABLE HERE RECONSTRUCTION IS EXPECTED HERE The current system asks for an accurate memory here — built from a record captured, or not captured, here. The restatement-and-audit record closes that gap by capturing attribution at Day 0.

One note worth adding. This isn't only a USA problem. Any jurisdiction with a standard built around independent professional judgment or reasonable inquiry is leaning on the same assumption, and likely carries the same blind spot. That's different work for another post, but worth knowing this isn't a Rule 11-specific gap.


A Possible Fix

Documentation alone won't fix this, because a log confirming that review happened can't tell the difference between real engagement and silently clicking accept. Closing this gap takes two things working together.

First, make the attorney restate the reasoning, not just accept it. For paragraphs that carry actual legal conclusions or risk calls, not boilerplate, not procedural language, the attorney writes a short restatement of the reasoning in their own words before accepting it. It's a small step that doesn't erase the time AI drafting saves. But it's active instead of passive, and that's the exact condition the research says produces a stronger, more accurate memory of where the reasoning came from.

Second, protect the record of that engagement. Track Changes already captures every edit automatically. Attach the restatement as a timestamped comment directly on the paragraph it belongs to. Before the document is finalized, run a pass confirming every substantive paragraph has either an edit or a restatement attached to it. Anything with neither is a flag that it wasn't actually engaged with, not just accepted. Keep that marked-up version separate from the filed document, treat it as internal work product, and never produce it alongside the filing itself.

THE FIX, IN PRACTICE What Contemporaneous Capture Actually Looks Like DRAFT — MOTION IN LIMINE The record shows that the defendant possessed constructive notice actual, documented knowledge of the hazard well before the incident occurred, satisfying the notice requirement under the applicable standard. [remainder of paragraph — boilerplate, no edit or comment required] RESTATEMENT COMMENT "Constructive notice wasn't enough here — the maintenance log shows he was told directly. That's a stronger argument than the draft made." — C.S., 2:41 PM, attached to paragraph AI-drafted language, subsequently cut Attorney's edit, replacing it Restatement, timestamped at review Marked-up version stays internal. A clean copy, generated via Accept All Changes, gets filed.

Now, it's understandable to ask: why not just write it myself, then? But there are two things that keep this from becoming busywork. Restating a conclusion under evaluation is a lighter task than generating one from scratch, recognition is cheaper than composition, so this still runs faster than drafting the section from nothing. And a step with no enforcement behind it becomes theater fast. A document that reaches final review with unmarked substantive paragraphs needs to actually stop and get sent back, and periodic spot checks on restatement quality keep this from becoming a box people check without thinking.

The limits are worth stating too. This doesn't guarantee an attorney's memory of their review will be accurate. Source-monitoring failure is a documented feature of how human memory works, not something a workflow eliminates outright. What this does is move the moment of attribution capture to when it's most reliable, instead of relying on reconstruction after the fact, which is exactly when it tends to fail.


The Big Takeaways

  • Rule 11 and Opinion 512 both assume attorneys can accurately identify the origin of their own reasoning after the fact. Source-monitoring research treats that as a separate, failure-prone function from reasoning quality itself.
  • Similarity between a source and the person reviewing it is a documented driver of source-monitoring failure. As AI drafting tools get better at matching individual voice, that risk grows, even though most tools today rely on light prompting rather than deep personalization.
  • NIST's AI RMF and ISO 42001 both require proof that human review occurred. Neither requires proof of what the reviewer actually understood or generated independently.
  • Documentation alone doesn't fix the underlying memory problem. Pairing contemporaneous restatement with a protected review record addresses the cognition problem and the compliance gap at the same time.

References

American Bar Association Standing Committee on Ethics and Professional Responsibility. (2024). Formal opinion 512: Generative artificial intelligence tools.

Chen, R., Xiong, S., He, J., & Ross, G. J. (2026). Stylometric detection of AI-generated texts: Evidence from human and machine-written essays. Digital Scholarship in the Humanities.

Clark, E., August, T., Serrano, S., Haduong, N., Gururangan, S., & Smith, N. A. (2021). All that's "human" is not gold: Evaluating human evaluation of generated text. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, 7282–7296.

Federal Rules of Civil Procedure, Rule 11.

International Organization for Standardization. (2023). ISO/IEC 42001:2023 — Information technology — Artificial intelligence — Management system.

Ishihara, S. (2026). ChatGPT's ability to imitate writing styles: An analysis guided by forensic text comparison. Digital Scholarship in the Humanities.

Köbis, N., & Mossink, L. D. (2021). Artificial intelligence versus Maya Angelou: Experimental evidence that people cannot differentiate AI-generated from human-written poetry. Computers in Human Behavior, 114, Article 106553.

Liu, X., Diddee, H., & Ippolito, D. (2024). Customizing large language model generation style using parameter-efficient finetuning. Proceedings of the 17th International Natural Language Generation Conference, 412–419.

Macrae, C. N., Bodenhausen, G. V., & Calvini, G. (1999). Contexts of cryptomnesia: May the source be with you. Social Cognition, 17(3), 273–297.

Marsh, E. J., Edelman, G., & Bower, G. H. (2001). Demonstrations of a generation effect in context memory. Memory & Cognition, 29(6), 798–805.

National Institute of Standards and Technology. (2023). Artificial intelligence risk management framework (AI RMF 1.0) (NIST AI 100-1). U.S. Department of Commerce.

O'Sullivan, J. (2025). Stylometric comparisons of human versus AI-generated creative writing. Humanities and Social Sciences Communications, 12, Article 1857.

Schindler, J., & Richter, T. (2025). Does text generation improve learning from expository text? A conceptual replication attempt. Cognitive Research: Principles and Implications, 10, Article 34. https://doi.org/10.1186/s41235-025-00645-2

Previous Post ← The Bias-Symmetric AI Output Review
All Posts Back to Blog →