More than two-thirds of legal professionals believe that the cost of checking AI outputs is outstripping the benefits of using AI, according to alternative legal services provider Morae.
The report – The Intelligence Gap: Why AI in Legal Isn’t Delivering - and What Needs to Change – found that 67% of in-house and private practice legal professionals believe the burden of human verification outweighs the benefits AI was supposed to deliver. Some 41% of respondents said they are checking all legal AI outputs, while 48% said they are checking for sensitive outputs.
Another 48% of respondents said that humans always or often materially change AI outputs before they can be used. Morae said this suggests that current human intervention is a repair function, not a safeguard.
Only 33% of respondents said they trust the results of AI-assisted legal work, with 48% flagging poor accuracy, hallucinations and errors as a constraint on effective AI use, followed by security or confidentiality concerns (41%).
Half of respondents said they were concerned about potential liability if AI-assisted legal work results in errors or adverse outcomes, while 62% said they are extremely or very concerned about reputational risk caused by AI errors.
Part of the challenge is that poor data quality is undermining AI performance, says Morae. More than a third of respondents (37%) said the structure or formatting of their data is poor or very poor, while a third said the consistency of their data is poor or very poor. By contrast just 27% rated the structure or formatting of their data as excellent or good, with 28% saying the same about data consistency.
Just 26% of legal leaders feel confident that their organisation’s legal information is AI ready. In many cases, that is because organisations haven’t got a clear handle on what data they have or where it is located. As many as 44% of respondents said they have multiple internal systems without central tracking, while 39% said they lack a comprehensive file inventory and another 39% said that legacy data has never been properly classified or deleted when not needed.
More than three quarters of respondents said this data quality issue undermines AI outcomes regardless of how good their AI systems are.
Another issue in-house respondents flagged was transparency related to their external counsel’s AI usage. Some 41% of in-house teams say transparency is either limited or completely absent. As many as eight in 10 general counsel say they are concerned about how external law firms are using their data in AI systems, with 55% worried that their data is being used to train AI models without their knowledge and 51% concerned their data is being shared with AI vendors without their consent.
The report was based on a survey of 850 senior legal professionals in the US, UK, Australia and the Middle East, split evenly between in-house departments and private practice.
Email your news and story ideas to: [email protected]
