Understanding Contract Management Best Practices 

Many large corporations and law firms are starting to realize the potential of using AI in contract drafting, review, negotiation, and management. Some organizations have deployed machine learning models to drive automation, standardization, and insight generation - across all phases of the contract lifecycle – pre & post-signature. However, leaders are often disappointed by the pace of progress or the amount of business value delivered by these programs. While the fast pace of advances in the NLP domain continues to create genuine excitement about AI-driven digital transformation, a lot of pitfalls remain which may cause your Contracts AI program to fail. Using AI for Contract Review & Analysis In this article, we'll talk about the 7 Best Practices to follow while deploying AI in contract review and management at your organization, and avoid the typical pitfalls. These contract management strategies have been used by some of the largest corporations and law firms to: ● Gain up to 70% efficiency in the contract review process, thereby increasing sales and/or procurement velocity ● Build valuable insights from their contract data to identify risks and opportunities in each signed contract. I have personally leaned on this framework to develop and deploy AI & ML solutions for Fortune 100 customers to generate tens of millions of dollars in annualized business impact. Developing and managing contract review playbook 

Contract review is an important management tool that ensures your contracts reflect your understanding and agreement of the parties' intent and expectations. ... In addition, some contract terms have hidden pitfalls that can adversely shift risk among the parties, potentially limiting the financial benefits of an engagement. Every organization has standards around its expectations for various aspects of a business agreement. However, most do not have these expectations properly organized and accessible in the form of a central repository (a.k.a. clause library). Historically, all attorneys have kept their personal ‘repositories’ of golden contracts or clauses in locally accessible locations. Digital Clause Library Anyone involved in contract drafting or review needs access to a Clause Library i.e. a structured repository of key terms, provisions, clauses, and nuances. This repository should also be linked with the corpus of the contracts that are used for training the AI models. Without a clause library, your contract review process will not be standardized across various individuals and AI models will not reflect the situations that your business agreements represent. It makes sense to use a contract AI software’s standard clause library as a baseline and invest some effort in customizing it for contracts and situations relevant to your firm. Maintaining and enhancing a clause library is a high RoI project, that will be critical to the success of AI deployment in the contract lifecycle over the long term. Human in the loop (HITL) 

HITL refers to systems that allow humans to give direct feedback to a model for predictions below a certain level of confidence. This concept is predicated on the fact that a combination of unique human insight and experience + machine efficiency produces output that is far superior to one that each alone may produce. A very crude example of how this works is like teaching a child when she points to a rabbit and says 'meow-meow' - through repeated feedback ("No, that's a rabbit"), the child will learn to connect the dots. 

How HITL works Why is it advisable to use the HITL concept to deploy AI in contract reviews? There are many situations in which you need the AI to deliver human-level precision to ensure safety and accuracy. For example, when manufacturing critical parts for vehicles or airplanes, the equipment must be up to standard. While machine learning can be helpful for inspections, it is still best to have the system monitored by humans. Similarly, your domain experts can not only help train the model but also detect erroneous model predictions. HITL is very valuable because it creates an error correction feedback loop. Moreover, this process is akin to an expert reviewing an apprentice’s output and providing feedback. The only difference is that the apprentice here can iterate very quickly and hence learn at a much faster rate. Building a structured process around HITL is critical to the success of AI deployment in the contract management process. Understanding the coverage of your AI models 

Lawyers have decades of experience in dealing with different types of contracts, across various industry domains, under specific circumstances. This makes them skeptical of any claims that a machine can suggest or detect unique contract language, pertinent to the client's interest. This skepticism is not entirely unfounded. Most of the ML models are dependent on the 'supervised learning technique. That means that the models will learn only those situations that the training corpus data contain. Any ML model that detects clauses, provisions, or other key terms in a contract should have good 'coverage'. i.e. it should have the ability to detect most of the key clauses. Here the debate arises, what are key clauses. While this is in itself a very broad topic, we’ve organized a list of key clauses, and categories of key clauses/terms/provisions in a commercial agreement [mostly applicable to the US]. Fundamentally, Contract Clauses can be of these types (with a few examples): Clauses can be categorized into Boilerplate (basic, which needs to be checked off in every review) and Others). Following is an illustrative list: 

No content

Other parts of the contract, which may not strictly be considered as clauses: ● Recitals ● Exhibits and Schedules ● Headings ● Index When building or buying your Contract AI solution, your firm needs to understand the current coverage of models. Another important aspect is contextual significance. A non-solicitation clause can exist between an employee and a company, a vendor and a company, a customer and a company, etc. Although the broad language might be similar, the nuances are unique to each situation. Similarly, Cap on Liability might be different from the Unlimited Liability clause. There needs to be an understanding around what is the coverage of AI models around these clauses and a roadmap for expansion of the coverage. One final point to note about coverage of AI models is that it's going to never detect the clauses which were missing in the training corpus. So, a corpus of contracts has to be assembled very carefully to generate training data. Understanding the accuracy of your AI models After you’ve understood the coverage of your AI models, you should understand the accuracy of your models in detecting the presence and precise location of clauses, key terms, and provisions in the contract. 

AI model accuracy is the measurement used to determine which model is best at identifying relationships and patterns between variables in a dataset based on the input, or training, data. The better a model can generalize to ‘unseen’ data, the better predictions and insights it can produce, which in turn deliver more business value. There are generally two components of model accuracy measurements - Recall & Precision # Clauses correctly detected by the model (True Positives) Recall = ----------------------------------------------------------------------------------------- Total # of Clauses in the document (True Positives + False Negatives) Looking at Recall in isolation is not advisable because an artificial way of getting 100% recall (i.e. ensuring that no relevant/important term or clause or provision is left out) is to attach a prediction to every sentence of the document as one of these. It's easy to see why this is not a good approach as the error rate will skyrocket and trust in predictions will nosedive. # Clauses correctly detected by the models (True Positives) Precision = ------------------------------------------------------------------------------------------- Total # of predictions made by the model (True Positives + False Positives) 

The above formula suggests that if you aim for very high precision, the model is likely to make very few predictions (on which its confidence level is the highest). Let's look at both of these metrics combined: 

Area Under P-R Curve 

As the graph above shows, Precision and Recall are two metrics that dance in unison, and data scientists usually select a combination of two which maximizes Recall for a minimum acceptable Precision. In your AI deployment, your domain experts need to collaborate with your data scientists to train models and select parameters which are acceptable to your standards. E.g., you may want to have a very high degree of Precision for Share Purchase Agreements (low volume and high materiality), or a very high degree of Recall for Employment Agreements (high volume and low materiality) to ensure that you don't miss out on any opportunities. In the real world, precision and recall are often traded off to achieve a balance of risk and reward. Deploying and integrating with existing systems and processes As with any digital transformation, just building a model with high accuracy and coverage is a battle only half won. Your firm needs to deploy the model in your existing systems and you will need to modify your current processes to account for the change. Deployment and Integration of any new AI model are where most of the failures occur in the industry. Even though companies across sizes and industries are upbeat about AI adoption in the Contracts space, a new survey shows that it is still early days as high costs of deployment and integration remain the key problem in AI investment. According to a new Gartner survey, one-third of technology and service provider organizations with AI technology plans said they would invest $1 million or more into these technologies over the next two years. 

A strategic approach and a roadmap are critical here. Ideally, the firm needs to have a mid-long term roadmap to generating value from AI. Its best recommended realizing some 'quick wins' so that fence-sitters and naysayers in your organization start to see efficiency gains. An example of that could be to use an MS Word Addin which brings the value of AI right into the tool of choice for contract drafting and review. 

No content

ContractKen's Word Addin for Contract Reviews Contract AI software typically interfaces (at the minimum) with your document management system, e-signature platform, and your contract lifecycle management system. The solution must integrate with existing platforms through APIs and with minimal new / modified plumbing. Let's talk about the cost. The cost of deploying a state-of-the-art AI system always depends on the 'Build vs Buy' decision. The latest commercial models in this space are that of 'SaaS' which enables customers to optimize their total cost through a subscription model. This enables the organization to use the models off-the-shelf on a pay-per-user model or even a pay-per-use model. Contract AI software providers typically work with their customers (in a consulting engagement) to customize their models for the requirements of the customer. This is typically a one-time activity that requires significant investment. Some of the leading players leverage operating models like the use of offshore to optimize cost for strictly process-driven and laborious tasks like document annotation & tagging. In summary, here are the key levers to manage the deployment and integration costs: ● Use AI contract software word plugin for core drafting and review tasks ● Enable API integration with existing business applications ● Use the latest commercial models (SaaS) to drive the cost of ownership down ● Use the latest operating models (Offshore) to drive the cost of building down 

Managing employee & process change through communication and training Like most digital transformations, deploying AI in contract analysis and management will also raise fears of redundancy and replacement in your employees. If this is not handled proactively by leaders of the firm, it will result in a lot of resistance and employees are likely to get disengaged with the program. The best way to manage that is to be proactive about the long-term roadmap of AI in the process and how it helps the employees to become more productive and switch to higher-value tasks. In a competitive environment, you will see that this will be seen as an opportunity by aspirational employees to add value and differentiate themselves from the rest. Training employees on how to use the models is extremely important. It helps increase employee engagement with the technology and ensures that model results are interpreted correctly Typically, senior leadership needs to champion the communication around the project benefits of the AI model implementation. That will drive help the AI leaders to drive change management around technology and process effectively. The benefits of AI will accrue after the initial period of test and learning is complete and firms need to plan for that. It follows a J curve as shown: 

Security, Privacy, and Contract Compliance 

Laws like GDPR, CCPA have brought the topics of customer data privacy, and compliance to the center stage. While planning to implement AI software in your contract processes, you’ll need to handle this aspect head-on. Compliance and data privacy teams will need to be involved in the process very early. Most contract AI software companies have robust security certifications which allow them to handle clients’ sensitive data on their servers. For example, there needs to be a process of continually assessing the data your business manages and identifying any potential vulnerabilities, taking actions to remediate those vulnerabilities, and then immediately and transparently reporting any issues so that action can be taken at once. Now, I’d like to hear from you. Which of these best practices do you think influences the success or failure of an AI program at your firm or client the most? Or maybe I missed an aspect which you think is the key. Please reach out to [email protected] to understand our products and expertise in contract review AI. We’d love to discuss our experience in detail. Leave a comment & share this post on social media with your teams and partners.