Lawyers love to talk about using new technology to make their practice more efficient. All during my career, I’ve heard law firms touting the great benefits or knowledge management, AI (generative and non-generative) research and writing tools, and of course, getting lawyers to do their own administrative work. But unfortunately, they are better at talking about it than actually doing it. Perhaps it is the billable hour model, which rewards inefficiency, or just that most lawyers aren’t all that tech-savvy.
So of course, when ChatGPT and generative AI snatched the attention of the world a couple of years ago, lawyers started speculating about how sophisticated LLMs would change legal practice. Some opined we would all be made irrelevant and rendered unemployable. (Well, not yet.) Some announced grand partnerships with AI providers–though I would be skeptical that has reduced any client bills much. Some think we can use AI to do better, faster work. In that respect, faster is more likely than better. AI, it turns out, is not so good at quality control, and lawyers are all about quality control.
Technological breakthroughs can have profound changes in our world, but they tend to happen more gradually than click-bait articles would have us believe. I remember, once, wondering how people would ever accept self-driving cars. But we didn’t exactly. We accepted cruise control, then automatic parking, then lane-keeping assistance, and automatic braking. The autonomous features we accepted just keep expanding.
AI in legal practice is the same. Slowly, AI has crept into our practices. I doubt that I am much different from most lawyers in this way. Here are some of the tasks for which I have used AI in the past couple of years:
- Creating first drafts of articles and memos. Even if I end up re-writing the entire thing, a first draft is useful and can be thought-provoking. I would equate ChatGPT, in this respect, with a first year associate. (But sometimes, I am sad to say, with better grammar.) But a first draft is about as far as it goes. The LLMs have no savvy, no creative ideas, and no experience. They can’t put two and two together and make five–well, they can, but they think that’s accuracy, rather than synergy.
- Summarizing contracts or other documents. LLMs are useful for creating summaries of documents. In this respect, they are usually quite accurate. So, I can use them as a double-check for my own reviews or summaries, or as a basis to draft them. A word to the wise: if you do this, be sure not to feed confidential information into the model, unless you have a paid access account that keeps in the input confidential and prohibits training on it. Disclosing confidential documents to an LLM would usually be a violation of client confidentiality.
- Legal research. General purpose LLMs are not very good at most conventional legal research–finding court opinions, statutes, and similar sources. That is because they have no sense of what is authoritative, other than consensus of their training data. LLMs can be useful for factual research; these days I find myself using the Google AI-driven search quite a bit. I particularly appreciate that it cites to outside sources so I can check the legitimacy and reliability of the information.
Now…the Part I Didn’t Expect
Recently, a client sent me an agreement to review. It was a simple agreement, and not badly written. I asked where the agreement came from, and my client admitted they had used an LLM to write it.
But this did not bother me. Clients send their lawyers form agreements all the time, saying, can we use this? They usually seem to think this saves legal fees, even though it does not really do that. Until now, the documents they sent were usually something they copied from the web–often without even changing the names–and often wildly inappropriate for the situation.
In contrast, the LLM-generated agreement sent to me was on point, contained no typos, and had most of the relevant terms. It took me about half an hour to fix it up for use–probably about the same time it would have taken me to start from one of my own forms, and probably less time than fixing up some piece of dreck that was codged from the web. Perhaps I am unusual in that I would rather not spend my hours, nor to charge my clients for the hours, spent fixing up badly written stuff from the web.
But it did make me realize how lawyers will finally be dragged into using AI, rather than just pontificating about it. It won’t happen from their own volition, or even due to pressure from clients to be more efficient. It will because their clients will start using auto-generated legal documents, and the lawyers will be expected to fix them, or explain why they are not good enough to both fixing.
Welcome to 2025!