AI is transforming legal work: new study provides clear evidence
The legal industry is on the verge of a revolution – and artificial intelligence is playing a crucial role in it. A brand-new study by the University of Minnesota and the University of Michigan is the first scientific study of how two specific AI technologies could change legal work: retrieval-augmented generation (RAG) and reasoning models.
Two leading AI systems were tested: Vincent AI, which is based on RAG and combines legal analysis with specific legal sources, and the reasoning model o1-preview from OpenAI, which structures and processes complex legal questions step by step. 127 advanced law students took part in the study and were asked to solve six realistic legal tasks – some without AI and some with AI support.
The results speak for themselves: both technologies significantly increased the speed and quality of legal work:
Vincent AI increased productivity by between 38% and 115% and showed particularly strong results in the clarity and structure of the documents. It was surprisingly reliable, even producing fewer errors (“hallucinations”) than human participants without AI support. This means that RAG-based systems like Vincent AI could be particularly valuable because they provide reliable results that users can immediately understand and verify.
OpenAI's o1 preview increased productivity by as much as 34% to 140% and significantly improved the analytical depth and quality of reasoning. However, occasionally erroneous information indicating fabricated sources still appeared here. These so-called “hallucinations” mean that legal professionals should remain vigilant when using such reasoning models and always critically review the AI-generated results.
The study highlights that none of the tested technologies is completely error-free, but it clearly shows that a combination of RAG technologies and advanced reasoning models could significantly improve legal work. The combination of these approaches could combine factual accuracy with analytical depth, enabling a new quality of legal work.
Implications for practice:
In the future, AI will enable legal professionals to work faster, more accurately, and with higher quality.
AI expertise will become increasingly critical to the competitiveness of law firms and legal departments.
The integration of AI into legal workflows will enable legal professionals to focus more on the strategic and creative aspects of their work.
Despite all the efficiency gains, human judgment remains indispensable. AI is a supportive tool, not a substitute for human expertise.
In summary, it can be said that AI will soon be indispensable in legal practice. The development is rapid, the potential enormous. Lawyers and law firms that learn to use AI effectively and responsibly at an early stage will secure decisive competitive advantages in the legal industry of the future.
What is retrieval-augmented generation (RAG)? Retrieval-augmented generation (RAG) is an advanced AI technology that combines generative models (like GPT) with a retrieval system that selectively retrieves relevant and reliable information from a database or knowledge base. This directly supports generated responses with concrete, verifiable sources, significantly improving reliability and transparency.
