default logo

How BT Manages Contracts: Provision Extraction with AI (live IACCM event)

The session Contracts and Big Data Analytics is the opening for a DC-area IACCM contract management event; it covers how BT manages contracts and extracts provisions with AI. The presenter is Paul Branch, Head of Deal Assurance at BT Americas. This is a live post; please forgive typos or misunderstandings of meaning.

Big Data is the next big thing in contract management. If it delivers as promised, it will transform how and what contract managers do.

BT has looked at how best to manage commitments. BT has introduced automation to extract provisions and tag them to support improve customer alignment. This can be hard because complex agreements ma have 1000s of provisions.

Tools are required to manage rights and obligations. Excel simply will not do given the volume and complexity. BT is using artificial intelligence (AI), specifically natural language processing and machine learning to extract provisions into a contract lifecycle management system.

The business objective is to manage rights and obligations, both owed to company and owed to counter parties. There are three steps to accomplish this:

  1. Extract provisions.
    1. Ten years ago, Excel was the only tool. It was tedious and prone to error. But even though, it was hugely useful to identify who needed to do what. At that time, extraction occurred after the contract was signed
    2. Now, with new tools, provisions are extracted as contracts are negotiated.  Some terms may be encoded for easier extraction.
    3. By itself, extraction has zero benefit. You need the next two steps to obtain value.
  2. Allocate ownership of provisions to business owners.
    1. With a list of obligations, you need to decide who will perform each one. Most of the obligations do not fall to the contracting team. Obligations must be fulfilled by engineering, lawyers, marketing, and many other teams in the organization. They should all be held accountable to fulfill the obligations.
    2. With intelligent disaggregation and then aggregating logically, e.g., by ownership, organizations can matter manage meeting obligations.
    3. Some discussion about what terms to extract, including from MSA versus SOW. The key is to extract the relevant provisions, irrespective of the documents in which they sit.
    4. Allocation of ownership should occur before signing the contract. That ensures the organization’s ability to deliver
  3. Track fulfillment
    1. This is hallmark of high-functioning organizations.
    2. If there is jeopardy in meeting obligations, this must be flagged and managed
    3. Handling this appropriately helps overcome common pitfalls of contract management: insufficient focus on governance and poor handover from the deal time to the implementation team.


BT uses Corridor software sitting on top of MS Sharepoint. Categories in the contract lifecycle management (CLM) software includes obligation category, priority, scope, status, and timeframe. This is a sample CLM screen:


The hardest part of contract management is getting provisions into the tool. Also very difficult is getting everyone using the CLM tool. A key back-end challenge is managing permissions so that the right people see the right provisions but not everyone sees everything.

BT tracks provisions by obligation category (time-bound, triggered, unbound), priority (critical, proactive, reactive), and scope (contracting entity, counter party, shared). It tracks metadata such as due date, timeframe audit evidence, clause path, assigned to, and clause type. Allocation of ownership drives from category, scheduled and “service tower” (job function). Obligation status has a workflow to track assignment (assignment request, acceptance, declined), status (on-track, in-jeopardy, past due), and closure (submitted, closed).

BT contract management is integrated into Data are piped into global data warehouse. Contracts are also pushed into a document management system. A separate system manages contract commitments and governance. A knowledge management (KM) program captures and reuses the best practices and lessons learned. Sounds like there is a separate KM repository that has capability to push out certain information.

Implementation hurdles are getting it done and change management. Precursor is having accurate obligation (provision) extraction. Technology helps with extraction but gets no where near Six Sigma accuracy. Separately, you need a CLM tool; Excel does not suffice. There are literally 100s of CLM vendors now and many are inexpensive. Finding one that track obligations the way you want may be harder. The change management is a multi-year effort; Paul has been at it for four years. Part of the change management program must be prioritization. Complex agreements have 1000s of obligations – which are the really important ones?

Technology allows semi-automation of extraction of provisions and priorities. Submit a contract to an NLP / ML tool, move to a 2-step paralegal review. Machines are about 80% accurate. To get to Six Sigma, BT uses a 2-step paralegal review. Tool in use is ACE from Ravn. Other tools include Kira, KM Standards. The paralegals: a volume vendor in India, a non-English langauage providers in Belfast, and a secure contract vendor in the US. About 10 paralegals in total across providers. 70% of the volume goes through India.

As of April 2016 Ravn PoC: 15 contracts analyzed and contract obligations were extracted with average 84% precision with Ravn. It takes up to 2 hours of machine time to process contracts. The business requirement is six sigma, which is 99.999966% accuracy. As of Oct 2016: 30 contracts run through tool. It captures 56k obligations. Security classification applied to 30 contracts with 99.5% precision and 99.2% recall: