The volume of documents to review in discovery continues to explode. Can lawyers rely on full-text and concept-searching instead of lawyers to review documents? Craig Ball provides a good analysis of this question in Unclear on the Concept in Law Technology News.
Ball’s answer is that “automated search systems must be periodically tested against an evolving sample of evidence scrutinized by human intelligence.” I like his empirical approach because that’s what I have argued in prior posts such as Evidence-Based Law and Thoughts on Full-Text Retrieval.
Potential limitations notwithstanding, Ball suggests concept searching has a role in overcoming OCR limits, uncovering hidden relationships, and prioritizing the order of review. He makes the useful suggestion that those who do want to rely on this approach should invite “the requesting party to contribute keywords and concepts for searching is an effective strategy to forestall finger pointing about non-production.”
As lawyers consider the reliability of different approaches, they should keep in mind that objective (bibliographic) coding is typically no more than 98% accurate. As for subjective coding, I heard one anecdote of a large law firm that used a concept search tool after a lawyer review and the tool found many documents the lawyers missed. An assumption that lawyer review yields the best result is not necessarily correct – it’s just what the profession happens to find comfortable.