Lawyers and legal market professionals need to learn to love statistics more.
More In-House Lawyers Question the Billable Hour by ACC President & CEO Veta T. Richardson in Corporate Counsel magazine (13 March 2013) criticizes a finding concerning alternative fee arrangements in the recently published Fulbright Litigation survey. Her conclusion may well be correct but the evidence she cites does not, statistically speaking, directly rebut the finding she questions. In my reading, the time periods and questions ACC asked are not close enough to compare results directly.
My goal is not to weigh-in on the disputed finding, whether GC are increasing or decreasing their use of alternative fee arrangements. Rather, it is to discuss statistical validity.
Too many legal professionals are uncomfortable with numbers and more still with statistics. I believe that explains persevaration over predictive coding, failure of many legal market surveys to publish the number or demographics of respondents, and incorrect comparisons (see, e.g., my post, 2008 AmLaw Tech Survey.)
Ms. Richardson points out that ACC has a larger sample than the other survey. In general, larger sample sizes are better than smaller sample sizes. But a big sample that fails accurately to represent the population is not necessarily more reliable than a small one. Beyond sample size, the questions asked matter a great deal. Do any AFA studies directly ask about the dollar spend for which the respondent is answering the AFA question?
The point is, statistics is a nuanced science. It’s scary when legal professionals start arguing stats. Fortunately, the next generation of lawyers may do better. Today, Dan Katz, a professor of law at Michigan State University, posted his syllabus for his Quantitative Methods for Lawyers class.