Written by Andy Cooke General Counsel at TravelPerk

How have you been using GPT-4?

We’re trialling generative AI to upgrade our internal chatbot in relation to certain types of questions from colleagues, namely in relation to privacy, company policies, and other contract and HR queries. 

A bit of context – since April 2022, we’ve been using chatbots to scale delivery of legal content to our internal customers. Chatbots respond to specific prompts, and deliver answers by following a predetermined logic.  Broadening the range of responses which chatbots can serve means investing time on prompts – training the technology to recognise that a request for an “NDA” is the same as a request for a “confidentiality agreement”, for example – and extending decision trees to broaden the chatbot’s “intelligence”.  But even with this investment, chatbots only know what they have been taught.  (Chatbots that seem “smart” are just leveraging very large decision trees.)

Generative AI has the potential to eliminate the time investment in setting up chatbots, and introduce dynamism in what the chatbot ‘understands’.  In other words, our first use case for generative AI (which we are currently trialling) is supercharging our response to simple queries, rather than necessarily addressing complex queries. To take the latter example, an internal customer wanting to understand the intricacies of a single specific business critical contract and how this relates to our corporate strategy will probably want to speak with a lawyer and, indeed, is probably better served by a human. (For now.)

But: there’s a huge volume of repeat queries on policy and similar matters that could be more efficiently addressed through AI than via a human. If you have policy documents that are written on a PDF or a webpage, you have content that can be understood by AI and against which analysis and responses can be served.  The good news is that you no longer have to build a bot to facilitate access to that content – there are lots of startups in this space (like Josef, LegalOS and Cody) who can help you link together a bot, Open AI and a box of content.

What would be your advice for in-housers who want to start using GPT-4 but don’t know where to start?

I hear counsel talking about unrealistic customer expectations, the battle for headcount, and “doing more with less”.  Statements like these infer that customer expectations may eventually go back to the way things were before – that customers will “calm down”. They won’t.  Cross-demographically, and irrespective of the type of product or service being sold, speed and convenience are what matter. We all like frictionless service in our personal lives.  How does that impact our working lives? Profoundly.

Doing more with less

I understand that some are nervous around opening AI-driven products for wide access without certainty that answers will be 100% correct.  Generally speaking, tech in legal works best when it’s dealing with simple things.  In our team, we’ve found that there is a certain type of query we get in our internal messaging which is repetitive and related to policies and information that our colleagues could look up, but don’t.  That’s why we started using chatbots.  The fact is, everyone expects answers on demand, because that’s how every other service in our lives is delivered.  You cannot meet this standard by adding more people to the Legal function – a person can only throughput a certain volume of queries in a day.

The good news is that the tech exists to get ahead of the problem.  Get started using that tech – just try something, even if it’s just summarising information using the GPT prompt, or asking it to explain what you might have missed in reviewing a document. (Alternatively, keep doing what you’re doing, and see your internal customers use generative AI – likely in a way that exposes the business to risk – and ask you to verify what they have discovered, if you’re lucky.)

What are you planning to do next?

We expect generative AI will substantially disrupt legal services. GPT-3.5 has demonstrated its ability to answer questions on demand with a 70%+ degree of accuracy; GPT-4 takes this accuracy up to the 90%+ level.  We’re seeing lots of interesting applications in relation to automated review and comparison of third party paper; given that we have fully automated creation of our primary document suite, automation of third party doc review provides a high degree of scalability across the full contracting universe.  

For now: we’re all about chatbots.  At the foundation stage, our goal is to get this tech into the hands of customers and assess the impact on our workflow from there.  We started working seriously with generative AI in mid-February, are testing our first superbot now and will launch it by mid-April.   On a practical level, we’re exploring opportunities to team up with adjacent functions – e.g. HR – to bring their policies and data to life using bots.  (This is of course a top tip for any tech you want to bring in as a function – explore use at the enterprise level, and activate budget outside of legal.)  

Where does this all end up? As a team, we embrace the opportunity to focus time on things where our specialists can add the most value: i.e. knowledge curation and application, and long term strategic risk management, particularly in the area of behavioural risk, leaving day to day business queries to be addressed almost exclusively by AI.

I’ll be on stage at Crafty Fest on 14 June talking more about meeting customer expectations, legal happiness, and bringing change to your function – do check out my talk if you are joining, and I’d be happy to meet you there and discuss further!