This is a live post from the Ark Knowledge Management conference; the title of this talk is Legal Enterprise Search at the Crossroads. The presenters are

  • Harris Tilevitz, Chief Technology Officer, Skadden, Arps, Slate, Meagher & Flom LLP
  • Douglas Freeman, Knowledge Systems Manager, White & Case LLP
  • Kate Simpson, National Director of Knowledge Management, Bennett Jones LLP
  • Moderated by Brenton Miller, Brenton B. Miller Consulting

Moderator Brent says enterprise search decisions are all about timing. Is this the right time to rethink a firm’s enterprise search strategy? By a show of hands, about 15% of the 200 person audience say their firms plan to change their enterprise search engines in the next two years.

Bennett Jones has signed with RAVN iManage search. Launch is not happening on originally proposed schedule.

White & Case is a Decisiv shop; Doug has rebuilt it three times. Firm has largely removed the native UI and search is now via a custom API in the Intranet. Firm has evaluated and continued to evaluate other search products. I understand his remarks to mean firm does not think current alternatives offer the full set of functionality the firm current has.

Skadden in 1987 ran BRS search on a mainframe, which was moved to Unix. Firm built brief banks on it. Then to Verity and then to Autonomy, both for short periods. Recently found Sine Qua and has established it as main search tool. The firm creates databases against which to search, eg, practice areas, or repositories created from multiple searches. Firms does not really do full enterprise search because it does not yield results that are very helpful. (Ron: so it sounds like investment to curate and manage data upfront to improve value of search results.)

What is enterprise search? Panelists, in prep, debated if it’s valid to say we now have a 2nd-generation of products. They concluded that the new systems are not fundamentally different. So not clear we are at gen-two. The noteworthy changes in current generation, however, are show in this slide:

Highlights of new features include: visualization, integrating with DMS (document management system) index, open-source foundation, non-SQL indexing, and integration with machine learning and AI approacdes.

What has worked and what has not? What has changed?

Bennett Jones: believing the sales people has not worked. Search is overhyped and it’s very hard to make it work. And it takes a lot of data and tech effort to make it work. And making it simple for lawyers is especially hard. Don’t believe that it is plug and play.

White & Case: We rely heavily on metrics. We looked at what users were doing with original search. That led to both cosmetic name changes and substantive changes such as nor requiring users to choose context (that is, people, documents, or matters). Now, search returns all categories organized visually, including with people pictures. “Fundamental search works really work”. One limitation to date: going beyond the document. Search returns document and user has to open it. But with other content types, the system can present visualizations and lists – graphical presentation. It would be nice to go beyond docs to clauses or entities, and then link to resources by those chunks of information. Firm has tried and is now waiting for tech improvements to better enable this.

Skadden: Sees search more as a set of tools. Law firms can’t replicate what Google does because they don’t have billions of searches conducted daily. So law firm systems can’t learn the way Google does. The alternative is to create smaller, refined data collections and put search against that. Doing so improves search results (fewer false positives).

Joshua Fireman comments: Recommind boxed in KM for a decade, channeling thinking to documents, people (expertise), and matters. But today, market is moving to a broader vision of what search should do. KM Professionals must define the problem and be more creative than relying on how a leading product defined it.

The value of and need for metadata. Harris notes that search is much improved if you can search on metadata as well as underlying content. And creating good metadata is expensive – it still requires human effort. Kate refers to Blue Jay Legal which has turned search on its head by allowing working with concepts. But this requires a lot of human curation. Oz agrees that metadata is key to making searching value. With over 100M documents, there is too much noise if we just search on underlying content – the metadata helps get good results. Doug notes that metadata also allows filtering results.

Where are the tools going and what are the opportunities that creates?

Harris reports that Sine Qua talks about natural language processing and has a vision for more features. But metadata collection and refined collections remain key. It’s not clear, in our lifetime, that computers will automate this process. The data extraction he has seen to date, after talking to many companies, is very limited. No tech, for example, can extract all the deal points from the documents. Separately, with move to privacy, confidentiality and least-privileged model, search may become much harder.

Doug – Search can help get us past pessimistic environment (least privileged model). Lawyers can use search to find experts but not see the documents. Show the people but not the documents. Classification will become more important.

Kate calls search a mountain hike and says RAVN Insight can do the hike. Being able to see the company vision, to see the knowledge graph, to get to answers, not just links, should all be possible with its vision.

Session Description: Legal Enterprise Search at the Crossroads

Most firms have implemented some form of enterprise search. However, the implementation and ongoing maintenance of an enterprise search solution has not been easy or cheap, and the actual usage and perceived benefit has not lived up to expectations. Yet the idea of allowing these first-generation enterprise search solutions to simply age-out as support rapidly dwindles is as unacceptable as the idea of jumping to a new system is unappealing. Are things really that bleak? Or are the second-generation enterprise search solutions actually something to get excited about, with their AI capabilities, scalability and new visualization tools? This panel will identify lessons learned from their first-generation implementations and, more importantly, how these lessons can be applied to second-generation projects to avoid the “fool me twice…” concerns.