NSERC Alliance-MITACS Accelerate grant funds two-year project that aims to reduce false legal information from generative AI tool
UBC Computer Science Assistant Professor Vered Shwartz and her research group will be collaborating with tech startup company Caseway on a research project focused on a legal research engine based on large language models. The project is supported by a two-year funded grant from the NSERC Alliance-Mitacs Accelerate program and aims to provide more reliable answers to legal queries.
UBC Computer Science postdoctoral researcher Maksym Taranukhin and Master’s student Ethan Zhao, part of Dr. Shwartz’s research group, will be working on this collaborative project.
Large language models can often give inaccurate or misleading information. A study published last year showed that when models were prompted with legal queries, they generated content that were inconsistent with legal facts at least 58% of the time.
“Large language models are already deployed and people are using them to get answers for legal questions,” says Dr. Shwartz. “Our goal is to ensure that these AI tools are as reliable as possible so that professionals and the public can trust it to provide accurate legal information.”
Many large language models use a technique called retrieval-augmented generation, which allows the large language model to search for information and retrieve it. While this technique helps improve the model by generating more specific and factual answers, inaccuracies are still pervasive.
“Rushing AI into legal use without fixes could be harmful, especially in high-stakes contexts where people need to rely on accurate information to make important decisions,” says Dr. Taranukhin.
To address this problem, the researchers will be integrating different methods to enhance the reasoning ability of the program, including the use of Bayesian networks, which are graphical models that describe how different factors are connected and affect each other based on probability. The researchers will also attempt to improve the model’s ability to avoid generating an answer under uncertainty.
According to Dr. Shwartz, one of the challenges of this project is making sure that the tool is both user-friendly and accurate.
“There’s a tradeoff between how helpful it is and how accurate it is,” she says. “It’s tricky to strike the right balance.”
This project builds on Dr. Shwartz’s previous research on building AI tools for legal help, such as a study led by Dr. Taranukhin in which the researchers presented a chatbot that assists air passengers and educates them about their rights.
“Getting the right information quickly can make all the difference,” says Caseway’s CEO, Alistair Vigier. “We’re not building this to replace lawyers, but to make legal knowledge far more accessible to those who can’t easily get professional help.”
About Mitacs
For over 25 years, Mitacs has helped grow the economy and develop the workforce of tomorrow, connecting industry with academia and global partners to solve real-world challenges. Mitacs supports business-academic research collaboration through internships, co-funded with businesses, for undergraduate to graduate students and post-doctoral fellows.