What’s Your Strategy for Collecting and Cataloging Documents?

This article was published in Capital Connection (by the Capital Chapter of the Association of Legal Administrators) in February 2004

ABSTRACT: Firms that want to re-use work product need to decide how they will capture and re-use documents. One way is to rely largely on an automated approach that uses software to identify and find useful documents. Another way is to rely on a more manual, human-driven editorial process. This article compares and contrasts the two approaches

What’s Your Strategy for Collecting and Cataloging Documents?

Lawyers regularly write briefs or memos that reflect in-depth research; they also create complex transactional documents. Before the advent of computers, many firms stored briefs in file cabinets and created bound leather volumes of transaction documents. Today, technology supports broader and more sophisticated “knowledge management” (KM) efforts, though re-using work product continues to be an important but still only partially met goal.

Firms that want to re-use work product need to decide how they will capture and re-use documents. One way is to rely largely on an automated approach that uses software to identify and find useful documents. Another way is to rely on a more manual, human-driven editorial process. Of course, the two are not mutually exclusive and a blend is often best. But for purposes of understanding the pros and cons of each approach, it is useful to consider them as opposite poles on a spectrum.

Automated Approach

The advantage of a highly automated approach is that it does not require any extra effort by lawyers. Many firms presume that their lawyers will neither willingly “contribute” documents to a KM system nor provide a contextual description for the documents they write. Whether it is because lawyers are too busy to be bothered or fear sharing their knowledge, it can be very difficult to motivate them to take the time to designate documents that are worth keeping. An “indicator” of the difficulty of this problem is that many lawyers do not even give documents meaningful titles. Searching a typical firm’s document collection will reveal many documents with titles such as Latest Draft, Memo, Revisions, Insert for Joe, or Research for Mary. With such vague titles, one wonders how even the author will remember a week later what a document is about.

Early automated approaches to finding and re-using work product used full-text search engines to index every document in the firm. This allowed lawyers to search all documents by using Boolean or natural language searches (similar to Lexis or Westlaw searches) to identify potentially useful documents. In practice, this approach proved disappointing because most searches yield far too many “hits” to be useful. The reason is twofold. First, document collections typically contain numerous duplicates. And second, searches often return many documents that include the search term but that are not substantively related to the topic the lawyer is researching.

Consequently, ordinary full text searching of large document collections remains problematic today. Many large law firms have sophisticated document management systems that support full-text searches of all the stored documents. Most lawyers and knowledge management professionals find that such searches have only limited value because of these two reasons.

Recently, automated approaches have improved significantly. Products such as West km and LexisNexis Total Search are much better at automatically identifying useful documents. Both use citation analysis to help filter the documents returned during a search, which reduces the likelihood of spurious hits.

Newer systems also offer improvement on “document in-take,” that is which documents are selected for inclusion in a system. Various techniques — for example, word filters, citation analysis, and de-duplication — limit the number of documents included in the database. When a database contains fewer documents, search results are generally more refined.

These newer approaches can also apply sophisticated semantic analysis to put documents into taxonomic categories (for example, West km categorizes documents into the West Key Number system). Categorizing documents allows lawyers to search by topic as well as with key words. Furthermore, when searching by key word, lawyers can use the topical classification to help narrow search results.

This automated approach, however, has limitations. First, citation analysis, which powers many benefits, is not helpful for transactional documents (though in January 2003, West announced a transactional version of West km). Second, automated approaches do not provide context. That is, while automation can unearth relevant documents, even at its best, it does not explain what a matter was actually about. Nor does it explain how the document relates to the matter.

Manual Approach

Because of the problems of early automated systems, some firms, particularly large ones in England and Australia, instituted manual editorial approaches. These firms employ “practice support lawyers” (PSL) whose job includes talking to lawyers to identify important documents, collecting those documents, and providing contextual information about them (typically gleaned by interviewing lawyers who worked on the matter). The documents and context captured can be stored in a variety of systems, including portals, intranets, or document management systems.

The manual approach has the benefit of being selective, which means only a sub-set of relatively high value documents are retained for re-use. This makes it much easier to find relevant work product. Moreover, the contextual information about documents and matters makes it easier for other lawyers to know which documents are truly relevant and how to use them. Another benefit of the manual approach is that it is no harder to collect transactional documents than research documents.

Of course, the big downside of the manual approach is the expense.It is built on a staff of primarily non-billable lawyers who focus on knowledge management. This approach also assumes that practicing lawyers will spend some time helping PSLs. While many US law firms invest some non-billable time in knowledge management, only a few — primarily large and very profitable NYC firms — have hired a dedicated staff of PSLs.

The manual approach does not mean an exclusive reliance on human labor — technology is also employed. Firms with PSLs typically use several systems to facilitate the collection and cataloging process. For example, they may use the accounting system to identify when a matter has been closed and then interview the lawyers who worked on it. They can search the document management system profiles by fields such as author or client-matter number or the content of documents via a full-text search. Once documents are identified, some firms use portals (e.g., Plumtree, iManage WorkSite, or Hummingbird Portal) or intranets to create specialized knowledge repositories. Some firms also use specialized content management systems such as Hyperwave or Vignette. One product, AdvanceKnowledge by ii3, provides a packaged approach that facilitates the manual approach by making it easy to submit documents and capture context, in part through workflow features and a built-in taxonomy.

Choosing an Approach

So, which approach is better, which is right? To answer this, a firm must first define its current and future requirements. Moreover, it must be realistic about its culture and resources. There is no point is adopting a manual approach if the firm is unwilling to hire some PSLs or cause existing lawyers or staff to “do some knowledge management.”While these considerations may push firms to the automated approach, they must also be aware that automated solutions are not free. They entail potentially significant costs to acquire, integrate, deploy, and maintain a system. And in the end, lawyers may find the results of an automated approach disappointing, especially if the set-up and expectations are not carefully managed.

Of course, the best answer is likely to be a blend of the two approaches. There is evidence that the market is moving in this direction. Sally Gonzalez, a knowledge management consultant with Baker Robbins & Co. suggests in “Knowledge Management: U.S. or U.K.? Who Really Rules?” in Law Technology News (October 2003) that the two approaches may be converging.

Firms just beginning the assessment can consider pilot-testing both approaches. And whatever the approach, it is wise to monitor usage and value. Absent setting up a “feedback loop” that allows a firm to assess the value or return of a knowledge system, it may be difficult to sustain interest in and resources for an on-going effort.

While the choices available and decision making process may seem daunting, the good news is that there are far more viable options today than even a few years ago. Firms can try mixing and matching the approaches. Whatever choice a firm makes, it is critical to set realistic expectations, monitor on-going results, and adjust the strategy as necessary.