An artificial intelligence system has beaten some of the top lawyers in London. CaseCrunch, developed by the team who created Law Bot, was able to predict whether PPI mis-selling complaints would be upheld or rejected by the Financial Ombudsman. The CaseCrunch system achieved an accuracy rate of 86.6 per cent, compared to 62.3 per cent for the lawyers. Over 100 lawyers competed in the Challenge, ranging from Magic Circle partners, barristers, and in-house counsel with articipating law firms including Bird & Bird, Kennedys, Weightmans, Allen & Overy, Berwin Leighton Paisner, DLA Piper, DAC Beachcroft, DLA Piper, and more. Teams were entered from large firms including Pinsent Masons and Eversheds Sutherlands.

Legal decision predictions

Scientific Director of CaseCrunch, Ludwig Bull, says: 'Evaluating these results is tricky. These results do not mean that machines are generally better at predicting outcomes than human lawyers. These results show that if the question is defined precisely, machines are able to compete with and sometimes outperform human lawyers. The use case for these systems is clear. Legal decision prediction systems like ours can solve legal bottlenecks within organisations permanently and reliably.'

Showcasing AI potential

Rebecca Agliolo, marketing director of CaseCrunch, commented: 'Ultimately, the Challenge wasn’t about ‘winning or losing’; it was about showcasing the potential of artificial intelligence and changing the current paradigm not by talking, but by doing. The Lawyer Challenge started as an idea, and spiralled into a vision. Like any vision, it can’t belong to a single person. We hope that the Challenge will be replicated and improved - and we are proud to get the ball rolling.'