LegalTech, Justice AI, ChatGPT lawyers: Is there a forest behind the buzzword trees?

From Massachusetts Lawyers Weekly (Aug 2, 2024)

Every day now lawyers hear with trepidation that ChatGPT and the like are about to revolutionize the practice of law in unknown (but very exciting — stay tuned!) ways.
One wonders: Is the new way of doing the old thing actually useful? Or is legal AI a solution searching for problems?

ChatGPT can instantly generate a contract. Soon, the quality of the output will match or exceed that of a lawyer. The machine can summarize a 156-page decision from the Supreme Court. It can suggest negotiation points and counterpoints, define terms, and offer pancake recipes.

It is a flashy new version of a search engine that answers any question instantly and (mostly) coherently. In the near future, it may even answer them accurately. (Perhaps. Possibly. We don’t yet know.)

Excitement aside, however, what is the utility of machine-based lawyering? The answer is assumed to be self-evident. We are told “more” is good. “Faster” — even better. More and faster must be the best.

Still, how do we know this to be true? A basic scientific method suggests employing measurable goals and outcomes. Most legal writing about AI of all kinds focuses on the means and methods without ever acknowledging, let alone measuring, the goals. To be fair, this is an industry trend, amplified in the context of machine learning.
On the question of contracts, we should better understand why people sign them and what makes one contract better than the other. Is it about predictability? Speed of execution? The lack of resulting litigation? Some other element?

Or is a contract merely an illusion in the service of the intended outcome, i.e., if everything happens the way the parties intended. None probably knows (other than the lawyers, perhaps) what the contract even said. And if it doesn’t, there will be a conflict regardless of what the document says. We know many people don’t even read contracts these days (think End User License Agreements, or EULAs), so should most of these surplus agreements even exist?

In terms of summarizing court decisions, one assumes the Supreme Court justices feel their 100-page musings are not merely self-indulgence. Is it a disappointment when a carefully thought-through and thoroughly written decision is reduced by ChatGPT to a three-liner? And if not, then perhaps the decision should not be 100 pages to begin with?
The length of judicial decisions has increased substantially with the invention of word processing, and more so with paperless offices. The published (and unpublished) orders now take pages upon pages of text compared to their predecessors from, say, the late 1800s. Are the cases today so much more complex than those 100 years ago? Are judicial deliberations today so much more involved that they can’t be expressed in fewer words?
Perhaps mere volume makes something more compelling. If that is the case, should the courts issue minimal length requirements?

ChatGPT can enhance judicial decision-making by, for instance, providing more information or drafting portions of opinions. Still, without knowing more about the process and quality of the judicial analysis today, how does one know augmentation is desirable? For example, in a concurring opinion in Snell v. United Specialty Ins. Co., 2024 U.S. App. LEXIS 12733 *; _ F.4th _ (11th Cir., 05/28/24) (available at https://media.ca11.uscourts.gov/opinions/pub/files/202212581.pdf), one judge ponders the use of ChatGPT to divine the meaning of a contract term.

The case was an unremarkable fight over the scope of insurance coverage. The question: whether the term “landscaping” in the “ordinary sense” included installation of an in-ground trampoline? (Don’t ask).

Spoiler alert: the answer was irrelevant. The 31-page concurring opinion acknowledges this fact. Still, in what was perhaps intended to show a thought-provoking and trailblazing exercise, the judge asked ChatGPT to define the term and then to answer whether it included the trampoline. The opinion then suggests in a series of vignettes how the answer may or may not be useful or representative of the general trends.
In all, ChatGPT offered the court the wisdom of an internet search, although in a slightly more efficient and digestible form.

Entertainment value aside, is the Snell decision better or worse for the ChatGPT-powered concurrence? Are the litigants? In terms of efficiency, would a pithier concurrence be faster? Given its academic quality, would the exploration of potential uses of ChatGPT for judicial decision-making be more useful to the industry as an article in a legal journal? And, purely in the ordinary sense, does one really think of trampolines when considering landscaping?

Another question of justice is the utility of a self-contained machine-based process. That is, if ChatGPT prepares a motion, and another ChatGPT prepares the opposition, and then the third ChatGPT reviews both and makes a decision, is justice somehow served? If we could streamline the justice process by flipping a coin, possibly without a decline in accuracy, would that be desirable?

And what about the industry and general welfare? The country currently houses, feeds and clothes a record 1.4 million lawyers. Adding the support staff, the academia, the landlords, and other law-adjacent persons, the profession occupies several million people. How does speed of execution correspond to the billable hour? On the other hand, lawyers may represent more clients, cheaper. Is more legal paperwork for more people truly the best use of everyone’s time?

If we remove all barriers to lawsuits, ensuring that anyone can sue anyone at any time and get a decision within minutes, will everyone be too busy suing to invent the next version of ChatGPT?

​These questions may be silly. After all, one does not wish to advocate for a quill pen in the age of TikTok. Still, we should probably at least consider measuring the efficacy and usefulness of legal AI in the context of the problems it is supposed to solve. Otherwise, we risk putting the ChatGPT cart before the proverbial AI horse.