The current issue of Law Technology News has an excellent roundtable discussion on what lawyers must do to ensure that EED processing works. Not addressed however, is an equally interesting question: the duty of care for lawyers reviewing documents. 

EDD Showcase: Worst Case Scenario (Oct 2006, free registration required) features a discussion among EED experts Michael Arkfeld, Craig Ball, and J. William Speros answering “Who is responsible when you delegate EDD and things go awry?” Arkfeld holds lawyers to the highest standards of due diligence concerning vendors. He argues that

“the attorney must be reasonably confident that the processing and searching methodology is sound and that the evidence was disclosed… Among the many things a vendor should do is to provide affidavits, testing data, etc., detailing how a particular piece of software processes electronically stored information (ESI)… This would also apply to search software: lawyers must use reasonable care to ensure accuracy… you need to include quality control standards that assure disclosure of all responsive evidence to the opposing side.”

Were Arkfeld’s approach applied to how lawyers designate privilege and responsiveness, I suspect that many lawyer review processes would fail. Let’s start by looking at a simpler task: objective coding of data such as author and date. Vendors do not promise 100% accuracy. Historically, accuracy much above 98% required double keying.

So how accurate is the lawyer review process? In my view, ensuring accuracy requires statistical techniques. Subsets of documents must be compared to the “correct answers.” The correct answer probably requires a team of experienced lawyers to agree on the designation for the selected documents. A statistically significant sample for each reviewer should be analyzed. The problem is, the statistics only tell you how bad the problem is – they don’t help fix it.

As law firms wake up to the potential problems vendors can cause, they should not shirk from the problems inherent in typical document reviews. The implications of this article for vendor management are startling enough. Perhaps a future roundtable will discuss “Who is responsible when lawyer review goes awry?” And I hope the answer does not end up holding vendors to a higher standard than lawyers.