EDD Vendors – and Lawyers? – Crashing and Burning

A Reader’s Letter by Ron Friedmann in Law Technology News (Jan 2007), published as Keep EDD Vendors From Crashing and Burning.

Abstract: A roundtable discussion in Law Technology News (Oct 2006) at Who’s to Blame When EDD Vendors Go Boom? discusses EDD responsibilities in EDD. It raises many interesting concerns but leaves unaddressed a lawyer’s duty of care. Friedmann’s letter to the editor raises this point.

To the Editor of Law Technology News:

The October [2006] issue has an excellent roundtable discussion, “Worst Case Scenario,” [Who’s to Blame When EDD Vendors Go Boom?] about what lawyers must do to ensure that EDD processing works. Not addressed is an equally interesting question: What is the duty of care for lawyers reviewing documents?

In the article, Arkfeld, Ball and Speros answer “Who is responsible when you delegate EDD and things go awry?” Arkfeld holds lawyers to the highest standards of due diligence concerning vendors. He argues that:

“The attorney must be reasonably confident that the processing and searching methodology is sound and that the evidence was disclosed. … Among the many things a vendor should do is to provide affidavits, testing data, etc., detailing how a particular piece of software processes electronically stored information (ESI) … . This would also apply to search software: lawyers must use reasonable care to ensure accuracy — you need to include quality control standards that assure disclosure of all responsive evidence to the opposing side.”

Were Arkfeld’s approach applied to how lawyers designate privilege and responsiveness, I suspect that many lawyer review processes would fail.

Let’s start by looking at a simpler task: objective coding of data such as author and date. Vendors do not promise 100 percent accuracy. Historically, accuracy much above 98 percent required double keying.

So how accurate is the lawyer review process? In my view, ensuring accuracy requires statistical techniques. Subsets of documents must be compared to the “correct answers.” The correct answer probably requires a team of experienced lawyers to agree on the designation for the selected documents. A statistically significant sample for each reviewer should be analyzed. The problem is, the statistics only tell you how bad the problem is — they don’t help fix it.

As law firms wake up to the potential problems vendors can cause, they should not shirk from the problems inherent in typical document reviews.

The implications of this article for vendor management are startling enough.

Perhaps a future roundtable will discuss, “Who is responsible when lawyer review goes awry?” And I hope the answer does not end up holding vendors to a higher standard than lawyers.

–Ron Friedmann
Principal, Prism Legal Consulting
Arlington, Va.