Abstract: Creating work product is easy but finding useful documents is hard. Automated work product retrieval is a good way to solve the problem. This is a case study of how Littler Mendelson rolled out Practice Technologies’ Real Practice product.
This article was published in
- Capital Connection (an ALA Capital Chapter publication), March 2006
- ITLA KM White Paper (International Legal Technology Association, Knowledge Management and Collorabtive Expertise white paper), June 2006
- Managing Partner magazine (US Edition), June/July 2006
- Practice Technologies web site.
Technology makes it very easy to create documents and large law firms routinely create hundreds if not thousands each week. Technology, however, has not helped lawyers very much in finding the truly useful documents. As a former technology and knowledge manager — and in my current role as a technology consultant — I’ve encountered the same challenge repeatedly: Lawyers at many firms, even those with collections of manually selected “best practice” documents, struggle with finding on-point work product.
On a recent consulting engagement for a technology company called Practice Technologies, Inc. (www.practicetechnologies.com), I studied how one firm successfully solved the problem of making good use of their accumulated intellectual capital.
Adding Value to “Commodity” Services: Littler Mendelson’s Challenge
In 2003, Littler Mendelson, a 400-lawyer firm with 26 offices around the United States, concluded that improving its ability to retrieve on-point work product would result in better client service and ultimately, increase profitability. Littler advises employers on a wide range of labor and benefit issues. Because of the economics of its practice, the firm was especially motivated to improve its work product retrieval efforts. “Clients view certain types of employment law work, such as developing policy manuals, employment agreements and routine transactional documents, as well as being able to provide answers to routine employment law questions, as a commodity practice,” explained Scott Rechtshaffen, Managing Shareholder for Business Development and head of the firm’s knowledge management initiative. “As a result, clients are often reluctant to pay rates for this type of work that they might more readily pay for premium litigation work. They want their employment attorneys to be able to provide this work quickly and in a cost-efficient manner. To do this, we need to be able to better tap our collective expertise.”
Until recently, “tapping the firm’s expertise” meant relying on e-mail messages sent to distribution lists, informal connections among lawyers, searches of the document management system (DMS), and a collection of best practice documents available via the firm’s knowledge management portal. These tools worked, but the firm thought it could do better. As the firm planned its document management system (“DMS”) upgrade in 2003, Rechtshaffen and his knowledge management team confronted the limitations of DMS for retrieving work product.
They found that lawyers selected the document type “other” for 45% of documents, which greatly limited the ability of lawyers to find useful documents. (Many firms find that most lawyers categorize more than 50% of documents in one document type, either “memo” or “other” or the default value at the top of the “pick list.”) The team also realized that document titles were generally written in short-hand, primarily useful to the author, and not to searchers. Finally, they found that full-text searches of the DMS typically yielded far too many hits.
The lack of accurate and consistent document profiles severely limited the value of DMS searches. While the DMS was still a necessary tool to handle work in progress, the KM team concluded it would be necessary to look elsewhere to answer the work product retrieval problem.
One potential solution involved hiring more professional support lawyers and staff to identify and vet exemplary work product — that is, to expand the firm’s current “best practice collections” through a manual review process. The firm’s handful of knowledge management lawyers was already doing this. The effort yielded good documents but captured only a small percent of relevant work product. Like almost all US large law firms, Littler concluded that hiring a lot of non-billable lawyers for KM purposes was not economically viable.
The also firm ruled out any approach that would require its lawyers to take extra steps to identify and mark-up high-value documents. “We wanted a passive system,” said Michael Williams, Chief Technology Officer at Littler. “And I mean passive in a very positive sense, in that it works behind the scenes to identify meaningful work product with no lawyer participation whatsoever.”
An Automated “Artificial Intelligence” Solution
Given the requirement for a passive system — and the additional criteria that the system had to be easy for lawyers to use and easy to deploy with their existing DMS – Littler set out to find a technology that would work with their existing DMS and identify useful documents in the DMS on a granular, substantive level. Put another way: The prohibitive cost of expanding manual efforts meant the firm had to find a reasonably priced and effective automated approach.
Early in the search process, the Littler KM team learned about RealPractice (from Practice Technologies, Inc.), software designed for work product retrieval. RealPractice programmatically analyzes each document in the DMS (and other repositories) to eliminate clutter and to create a detailed profile for the remaining high-value documents. Algorithms extract substantive information such as jurisdiction, opposing counsel, and type of document (e.g., “motion to dismiss”). RealPractice makes it easy for lawyers to find documents specific to their needs and situation: the system combines topical searching — by substantive area, document type and objective, practice area or transaction type — with full text searching.
An example illustrates how RealPractice works. Suppose an attorney needs to find an Illinois severance agreement. The system automatically distinguishes among multiple agreement types to identify just severance agreements; furthermore, it automatically identifies the jurisdiction. All the lawyer has to do is select a topic, sub-topic, and jurisdiction from pull-down lists. The search result, instead of displaying obscure file names, shows meaningful abstracts that enable the lawyer to assess quickly and easily each “hit” in the search result list.
After a RealPractice demonstration, Littler’s KM team quickly concluded that they had found the answer to their work product retrieval challenge. The firm valued four key benefits of the software: automated selection of useful work product, categorizing documents into topics and sub-topics, full-text searching, and easy user interface. “Our attorneys’ general reaction to the application was â€˜IT is finally doing something for us, not to us,’ said CTO Michael Williams. “Some also said, â€˜Why don’t we have this already – what are we waiting for?'” Beyond this unusually positive response from lawyers, Williams liked Practice Technologies’ focus on work product. “They didn’t try to tell us that their product would solve all of our problems, that it’s the only tool we need.”
Unlike some alternatives explored, a “proof of concept” test showed that RealPractice was also very easy to deploy. No manual efforts were required to vet documents; the program automatically selects valuable documents without attorney intervention. This eliminated one labor-intensive step for both lawyers and IT staff. Furthermore, Williams liked the hosted option, which meant that minimum work for his department to install and configure the software.
Ensuring a Successful Deployment
Littler KM and IT staff quickly saw that anyone comfortable with e-mail and simple web browsing could easily learn RealPractice. But they understood that ease-of-use does not guarantee adoption so they took several steps to create awareness and encourage use.
The most important component of the plan was KM team visits to multiple offices for demonstrations. Knowing that persuading lawyers to attend software demonstration can be hard, the KM team worked with the managing shareholder of each office to strongly encourage lawyers to attend a short (30 to 45 minute) demonstration of RealPractice. Showing the basics takes less time, but the KM team wanted to run lawyers through several scenarios where they would find RealPractice useful. The KM team also took other awareness building steps such as distributing a quick reference guide and placing a “banner ad” on the firm’s intranet portal home page.
An on-going measure to encourage broader product usage is reviewing many all-attorney e-mail messages seeking documents. A lawyer on the KM team receives most lawyer inquiries. Five to ten times a day, she conducts a RealPractice search on an inquiry and almost always finds useful documents. She then replies (privately) to the lawyer and explains that there are good documents in RealPractice. Rather than simply providing a list of documents, she provides step-by-step instructions on how to use the program. It can take up to five such interactions before the lawyer becomes a regular user. This confirmed the KM team’s hypothesis: even with easy-to-use software that lawyers ultimately like and use, active measures and perseverance is essential to ensure a successful deployment.
One deployment concern that the firm had turned out not be a problem at all. While lawyers and IT staff loved the automatic classification feature they saw in demos, they were concerned that RealPractice might not provide the granular categorization an employment-only firm needed. Practice Technologies addressed this by tuning the rules with feedback from the firm. “Other firms now benefit from the rules we’ve helped create,” Michael Williams commented, “but we in turn benefit from fine-tuning PTI does at its newer customers.”
Measures of Success
RealPractice administrative tools and software logs have allowed Littler to quantify its impact:
- Littler’s RealPractice system contains about 350,000 documents, which is about 10% of the total number of documents in the firm’s DMS. This is about the percent the firm expected would be picked as high-value, potentially re-usable documents. Again, these documents were identified by RealPractice.
- In February of 2004, the month the system was launched at Littler, there were roughly 500 searches conducted in RealPractice. By June of 2005, nearly 2000 searches were conducted.
- Lawyers conducted just over 10,000 searches in the first six months of 2005 and viewed 21,000 documents (or “click-throughs” to search results). The fact that lawyers typically view only two documents per search implies that the system displays only relevant documents; the abstracts of each document that are shown as part of the search result make it easy for lawyers to determine which are worth “clicking through” to view.
- As of June 2005, more than 200 of lawyers â€”over halfâ€” were using RealPractice at least once per month, with many conducting multiple searches each month.
Anecdotal information also suggests effective usage of the application. As mentioned above, the firm had relied in part upon “all-attorney” e-mails for lawyers to post and answer questions. Littler’s KM department estimates that the firm had significantly cut back on the time spent on all the lawyer back and forth, which had been costing the firm at least $800,000 per year in non-billable time. “Our lawyers still use e-mail to find documents, but they spend much less time doing so,” said Scott Rechtschaffen, head of the firm’s knowledge management initiative. “Volume is down considerably. I’ve had one associate report that if he uses e-mail, he starts his message by saying â€˜I’ve already searched RealPractice and could not find anything.'”
Feedback from associates as well as senior attorneys at Littler has reinforced the positive impact of the firm’s work product retrieval initiative. One associate reported that he uses RealPractice at least three times per week – when drafting any document for court, for research projects, or to look up state rules. As a new lawyer, it gives him confidence, as the system helps him identify a good model for the task at hand. A managing shareholder for one Littler office recounted being at a conference when a client called needing a document. He asked an associate to locate it and was amazed to get the perfect document back 15 minutes later. After learning that the associate had found it using RealPractice, the shareholder asked for training and quickly became a regular user.
Ron Friedmann is the founder and president of Prism Legal Consulting. A lawyer by training and former CIO at the law firm of Mintz Levin, Friedmann has over a dozen years of legal market experience at two law firms and two software companies.