The fallibility of AI and what to do about it





 Go to any legal or technology conference, and speaker after speaker will tell you AI is the next BIG thing. Huge strides have been made in in underlying technologies allowing new tasks to be performed quickly by that would take human teams weeks.

 

This is certainly true. For example, LawPanel’s algorithmic comprehensive trademark search has performed 1m searches that would have taken a team of 10 attorneys working 10 hours a day 45 years to complete.

 

And yet..

 

What you won’t hear people talk about so much is the fallibility of AI, and the importance of both system design and the legal processes it sits within to identify and mitigate errors. Take our trademark clearance searches. For searches users run themselves, we do not give a categorical ‘available’ or ‘unavailable’ to register. 98% of the time the algorithm is in line with attorney opinion on similarity. However, it is more fallible on distinctiveness or descriptiveness.

 

For those who’ve heard of the Turing test (a way to to test a machine’s ability to exhibit intelligent behaviour equivalent to, or indistinguishable from, that of a human), my suggestion would be asking it whether ‘sparkle’ is distinctive for detergents.

 

To manage the risk of false positives and false negatives, we add in a margin of safety to steer users towards getting actual attorney advice. And we always explain trade mark law is complicated, that an automated search is fallible, and that when we say ‘probably available’ or ‘probably not available’ the range of likelihood each way may only be 60:40.

 

The processes we have around our AI are also designed to mitigate the risk of error. So attorneys use the automated searches only as a starting point and not the end result. They can also change weightings in the algorithms, if experience suggests too much credence has been placed on a non-distinctive word in the search string. Lastly, they chose the search results that go into the final report, which gives them ownership and responsibility for the outcome.

 

The business model of Trade Mark Direct was also designed to take out any cost to the client of a false positive search leading them to proceed with attorney advice. If a client asks for a full attorney search and the advice is negative, they are free not to proceed, with no fees to pay.

 

Increased volumes of enquiries and filings more than offset for this. No client has ever complained of the initial false result. In fact, they are delighted with the free search, and the goodwill this generates often brings them back.

 

We will be writing more on the great potential and opportunities from AI. But coming from a legal services background we well understand attorney / client relationships and the importance of trust. We will also be writing more about the ways in which AI can be fallible, and how we wrap the AI in systems and processes that make the combined whole robust.

Popular posts from this blog

New WordPress plugin for online internationtrademark services

  During 2020 we’ve been working on a lot of new trademark functionality, covering search, filing, watching, and record management. With a lot of interest from firms wanting to offer trademark services online, particularly filing and fee quotations. As well as adding this functionality to a site using LawPanel’s APIs we have an even easier and quicker integration for WordPress sites using our new plugin. See it in action on  Trade Mark Direct Network  a new service focussed on easy international filings in any country, but with a particular focus on the EU, UK (now requiring separate registration after Brexit), USA and Canada, Australia and key European countries, with more to follow including online WIPO fee and ordering. This will be the first online calculator fo giving fee estimates for advice and search fees, as well as official fees.
Read more

IP software and intensity of use

Some firms think that licensing IP software means they have the software and innovation boxes ticked. Not so. What matters is how much the software is used, by how many, and how fully. If the answer is 'not much', it's the wrong solution. With the right one there's plenty of scope to do things more easily, efficiently, and better for clients.  The term 'IP software' covers a wide range of tools and capabilities. Some older systems are stand alone clunky docketing databases. Others run only automated clearance searches. And others, unlike these point solutions, are platforms with a depth and breadth of capabilities. Old clunky docketing databases are a single point solution for tracking deadlines of registered marks and designs. So are often only used by one or two formalities administrators. The intensity of use of this IP software is low when considered against a whole IP team with lawyers and attorneys, as well as clients, associates, and other IP team members...
Read more

Beyond docketing

  Many firms and in-house counsel think of ‘IP software’ and ‘docketing database’ as interchangeable terms. Which is understandable as software tools for IP have only ever kept track of registry deadlines. With some add ons to assist with filing and checking local records in the docketing database against official ones at the registry. With Covid we’re seeing expectations change significantly. Firms of all sizes want better tools that do more. Not just to support work from home, but to reduce work and improve client service. For the full lifecycle of an IP right.  From on platform automated clearance searches for trademarks. Online forms for instructing searches and client areas to see results. Through automatic tracking of application status, built in watching and client tools for monitoring and reporting. And so much more. All of which reduce cognitive load and time, and improve client and user experience. Contact us now for a test drive.
Read more