How should lawyers and law firm/department managers charged with making decisions about legal technology go about gathering information? This post deals with collecting information about and analyzing legal technology options, not using legal technology intelligently.
I am prompted to ponder this subject because in a recent issue of a legal publication, I read a software directory. The directory was presented as editorial content, not paid listings. I was struck how the directory appeared to omit some prominent vendors. Whether this was merely an oversight or driven by advertising or other considerations, I don’t know. Moreover, although presented as editorial content, it appears that the information about each product or service was provided by the vendor.
The point is that law firms and departments looking for technology should not rely exclusively on one or two published directories. Many directories I have seen are either based on “pay for placement” or on some unspecified editorial judgment. Of course, such directories still have value, but they must be supplemented with other sources.
I would say the same about most surveys of legal technology. Perhaps I apply too high a standard to surveys. Prior to going to law school, I was an econometrician and regularly used data collected by rigorous standards. After law school, I was a management consultant and did a lot of work in the consumer packaged goods industry, where many highly refined market research reports are available.
Given my experience with surveys in other fields, I find that many legal tech surveys have three problems. First, it is hard to write questions that are unambiguous. The ambiguity arises from the variation in labeling and categorizing of both software and staff. Second, it is hard to collect statistically valid samples. And third, organizations may own software products and list them in surveys but may not really use them or use them in only a very limited manner. As with directories, surveys have value, but decision-makers should rely on multiple sources.
So what is a decision-maker to do? Fortunately, the legal profession has a tradition of sharing information about legal technology. Whether at conferences or through informal networking, most law firms and law departments will share quite a bit of valuable information about their experience with products and services and about how they are organized and staffed. For anyone contemplating a significant legal tech decision, the moral is to consult directories and surveys but not stop there – go beyond them and network with professional peers to learn the true story.