Blogs

@April King: here are some quick thoughts but @Karen Westwood and I can draft a fresh post on this one. Give us a little time to survey additional results.

I know attorneys stress over legal research. I've been at this for over a decade and there are still projects which wake me at night... Cases on the standard of care usually run along the lines of 'did you even Google it?' Also, expert testimony is required to establish the applicable standard of care. See Jerry's Enterprises. So, as a starting point, I like the Association of American Law Librarians guidance which outlines 5 principles for the competent researcher. 

• Principle I: A successful legal researcher possesses foundational knowledge of the legal system and legal information sources.
• Principle II: A successful legal researcher gathers information through effective and efficient research strategies.
• Principle III: A successful legal researcher critically evaluates information.
• Principle IV: A successful legal researcher applies information effectively to resolve a specific issue or need.
• Principle V: A successful legal researcher distinguishes between ethical and unethical uses of information, and understands the legal issues associated with the discovery, use, or application of information.

One nice thing about these principles is that they aren't based (solely) on common tropes which form the foundation for much criticism of today's legal research activity; namely,

1. New attorneys and students are poorer researchers than more senior attorneys. Sure, these folks are less experienced overall but what evidence is there that these young professionals are especially inept at legal research? Even as compared to prior generations?; and
   
   
2. Natural language processing and Google-like interfaces are dumbing down legal research. This complaint is common but why can't a natural language search be incorporated into a broader research strategy? Search and research are commonly conflated ideas even among librarians. (Though, I should also say that I think new providers offering 'artificially intelligent' search tools aren't suffering from the same criticism as more states adopt a duty of tech competence. The burden is now on attorneys  to 'keep abreast' of the 'relevant technology' perhaps relieving research providers from the burden of demonstrating their tools' capacity to help attorneys efficiently complete a full research project.)  

​​​​...more later. Thanks!

Karen, I think helping students understand what the standard of care is on research would be helpful to them in practice.  I don't think I know what it is.  As an estate planning and administration practitioner, it was only occasionally I had issues to which the answers seemed unclear, but when I did, it woke me in the night because I was looking for certainty--because I thought that's what I needed to be able to give advice that was useful and wasn't malpractice.  In the most thorny and liability-inducing situation I can recall, my advice stood up against challenge, but it was no fun realizing what a pickle I'd gotten myself into and then searching and mulling hours on end, trying to give my clients "the answer."

​@Michael Carlson and @April King I agree with you both that this search for a "perfect" legal answer is a fool's errand.  Charlie (my co-author) and I saw the plight of the solo and small-firm attorney (with regards to legal research) in being presented to some degree with an "all or nothing" choice by legal publishers.  Is there any merit in not seeking the "best" but looking for something that is "enough?"  We chose to analyze citators with the somewhat naïve thought that it would be easier to objectively analyze that step in legal research, as opposed to earlier in the research process where framing your issues and developing your research strategy govern so much of what you uncover.  ​

But even with something we thought would be straightforward, it quickly became apparent that good judgment and a firm grasp of the subject matter (discerned through many means, including secondary sources and colleagues as well as primary law) continue to be key to good research - even if one uses a premier citator. 

But April's question intrigues me as well - does this quest for perfection drive some people from the practice of law?  I formerly worked in an academic law library and a related question comes up for teaching law librarians as well.  Does the emphasis on Westlaw and Lexis (because they are offered at such a discount to law schools) give students unrealistic expectations with regards to legal research?  Do we do a disservice to students by not giving them more instruction on how to make the best use of Fastcase, Casemaker and/or free internet resources?  Could we be more helpful if we were more frank about the realities of law practice - teaching students how research effectively without dangling horror stories of the perils of not researching well enough?  Armed with a more realistic understanding of how legal research operates in the workplace, perhaps new attorneys would feel more comfortable approaching their colleagues and librarians for tips - and less time spinning their wheels looking for tidy answers like the ones they uncovered in their 1L legal research classes.

Thanks for your comments @April King. I do think the marketing teams for all the mainstream publishers have been guilty of playing into these fears: 'You don't want to miss something!' Fortunately, on the legal research side of things, ethics decisions rarely (ever?) fall into the category of "you failed to perform proper research." It's more, "you failed to do any research -- not even a Google search." 

I also think those of us in the legal research arena are especially prone to thinking about the law as something which might be 'found,' 'uncovered,' or 'dug up' as if appellate case law were a natural resource and the 'law' was a nugget within. Certainly, there are valuable nuggets to be found in the forms of citations for a memorandum or brief but for everyone else delivering services that don't require this practice, this line of thinking means 'there must be something out there...' and ultimately, fosters an environment where attorneys are more adept at defining risk but less adept at helping clients manage it.

I'd like to see this become a broader discussion.  My eye was caught by the headline--I wondered if someone had finally decided to address what I think leads many (certainly me) to leave practice: the sense that perfection is required in the practice of law, and if it's not delivered, disaster awaits.  Your article is talking about one particular way that feeling works itself out, in the search for more efficient ways to have confidence we've looked under every stone in the case law.
I'd like to be part of a discussion of whether others feel a sense that terrible consequences will result if they do not deliver perfect work product, and if so, how they avoid letting it drive them crazy or out of practice.

It's a great article and an interesting read. Thanks! I think the point you make about time pressure is an incredibly important one. All citators have limits that can cost researchers valuable time.

First, I'm not terribly surprised that there was so much variability between the citators on direct negative history. In my time at Thomson, KeyCite 'errors' were not common but also not unheard of. (I was a little surprised by the issues you reported with Bloomberg, however.) Generally, I don't believe it's a good idea to rely exclusively on red flags or other symbols. None of these tools are meant to be definitive. They are meant to warn you that the system identified some negative history -- it's a 'flag.' In my view, what you get from any of these systems is a potentially valuable shortcut. The problem is, we don't know what we don't know. Taking the next step can be time consuming.

Fortunately, not every case is Twombly which has 484K citing references on Westlaw including nearly 224K cases*. Often, the citing references to your case will number fewer than 10 and you can easily review all of those for the relevant point of law. But what if there are more than 10 references? One of your cases, State v. Martin, 695 N.W. 2d 578 (Minn. 2005), has 50 case law references on Westlaw and 51 on Fastcase.** Fortunately, both systems have flags for this case. In this case, you likely have the shortcut you need. But what if there's no flag? Or, what if you're looking for other negative or distinguishing treatment? 50 cases is a lot to review, one-by-one. There are tools and strategies for performing that review that don't include opening each case and reading it from beginning to end. Depth of treatment indicators, for example, can help us narrow results. This can even be done on Google Scholar for free:



Citators are super valuable tools but I think they aid the task at hand, they don't perform the task entirely. Researchers need to know the limits of any tool and how to use other resources when those limits are reached. Otherwise, the process can be very inefficient and will eat up time most lawyers don't have.

*Fortunately, you won't need a citator if there's a change to the rules cited in Twombly or Iqbal. You're going to hear about it from another source.

** Hmm.. I don't know why the difference.