Lawyer Penalized After ChatGPT Invents Witnesses and Testimony

Introduction

Generative AI can help professionals process large volumes of text, but it can also produce plausible statements that never appeared in the underlying record. A recent New Mexico case shows how that risk becomes a professional and legal problem when an AI-assisted draft is filed in court without verification.

What happened

Stephen Aarons, a New Mexico criminal defense lawyer, was hired to appeal a murder conviction. While preparing the appeal, he used Rev.com to create an AI transcript of the trial and then entered that transcript, the record, the statement of issues, and some discovery materials into ChatGPT. He subsequently filed a brief containing numerous factual and legal errors.

According to the state Supreme Court’s order, the brief presented false testimony from entirely fabricated witnesses. It also included testimony attributed to people who were never called at trial, confused the identity of at least one actual witness, and misstated accounts involving threats, the shooter’s clothing, and the shooter’s appearance. The brief further described genuine legal precedents inaccurately.

The distinction matters. Aarons did not merely cite fictional cases; he inserted invented factual material into an appellate filing and misrepresented authorities that did exist. He admitted that he had not verified the factual claims or legal authorities before signing and submitting the brief. He also had not told his client about the failure or the errors.

Aarons told the court that he assumed ChatGPT had produced a reliable summary and said he did not know the system could hallucinate facts. He maintained that the underlying transcript was not the source of the problem and that the errors appeared when the materials were processed together by ChatGPT.

The court’s response

The New Mexico Supreme Court held Aarons in direct contempt, fined him $5,000 for the state bar’s Client Protection Fund, and referred him to the disciplinary board for further proceedings. He was barred from appearing before the court while those proceedings are pending. The court also struck the earlier briefs, ordered the public defender’s office to appoint a new lawyer, and directed that the case move forward in a later term.

During the hearing, Justice C. Shannon Bacon rejected the idea that unfamiliarity with AI hallucinations could excuse the conduct. The justices emphasized that a lawyer signing a filing attests to its accuracy, regardless of whether the work came from an AI system, a junior lawyer, or another professional. They also criticized Aarons for failing to address the harm caused to his client.

Why it matters

The case suggests that legal AI policy should focus less on whether lawyers may use these tools and more on verification, documentation, and accountability. AI can organize records or produce a first draft, but it cannot authenticate testimony, establish what happened at trial, or replace a lawyer’s review of authorities.

Law firms using such systems need practical safeguards: preserve source documents and prompts, trace every factual assertion back to the record, check each citation independently, and require human review before filing. A disclaimer that a model can make mistakes is not a substitute for those controls.

The central lesson is straightforward: generation is not verification. Once a lawyer signs a court document, responsibility belongs to the lawyer—not to the model that produced the draft. In this case, the cost of confusing those two ideas was borne not only by the attorney, but also by the client whose appeal required a new start.

Source: Ars Technica AI