Recently, the “practice” of search as part of electronic discovery has evolved into the “art” of search and is rapidly evolving into the “science” of search. What had been accepted as industry standard has been determined as legally insufficient. Keyword search, the legal profession's preferred method for sifting through large collections of electronically stored information to find relevant or privileged information, had been widely accepted by courts and the legal community because its effectiveness was assumed and unchallenged.
Until 2008...
The effectiveness of keyword searching, along with the ability of attorneys to conduct these searches has been called into question by the courts, starting with three primary rulings in 2008: Magistrate Judge John Facciola of the D.C. District Court (U.S. v. O'Keefe, 537 F. Supp. 2d 14 (D.D.C. 2008) and Equity Analytics v. Lundin, No. :2007cv2033 (D.D.C. March 7, 2008)) and Magistrate Judge Paul Grimm (Victor Stanley, Inc. v. Creative Pipe, Inc.). These cases form the core of what can best be descried as the new search paradigm regarding defensible searches, becoming the new ESI (electronically stored information) standards of operation.
For legal technologists and lawyers, these opinions provide critical guidance as to how electronic discovery ought to be conducted in order to successfully defend against challenges should they arise. Defensibility of your ESI procedures is something that everyone must plan for in future litigation.
Keyword searching has increased in importance as the amount of electronically stored information has grown because it was the obvious way to uncover relevant information and reduce large amounts of ECI prior to review. Without keyword search, e-discovery would have ground to a halt years ago because the costs would have been astronomical and the duration of litigation would be marked in decades instead of years.
So What is the Problem?
.
Keyword searching was designed for other purposes and has been adapted to the legal market as the best available option. It is often referred to as the “black box” approach because it has been applied to the legal field without really accounting for the shortfalls surrounding the approach. Addressing the limitations of keyword search while following the new guidelines suggested by the recent rulings is the new challenge to reducing the shortcomings of todays search technology. A new more “transparent” search technology allows lawyers and litigation support professionals to create defensible search practices
In Victor Stanley, Judge Grimm waived attorney-client privilege and work product protection because the defendants failed to demonstrate the search methodology used to prevent the inadvertent production of privileged documents was reasonable. Judge Grimm pointed out that the defendants had the burden to demonstrate their search methodology was reasonable because of the “well-known limitations and risks associated with [keyword searches].” Judge Grimm went on to suggest that, given these known limitations, there are two current approaches parties can follow in order to use keyword search in a defensible manner.
• Collaborative approach. The first approach parties could take would be to “confer (the importance of compliance with new“meet and confer” requirements is key to a successful ruling) with their opposing party in an effort to identify a mutually agreeable search and retrieval method.” Grimm points out this approach would “[minimize] cost because if the method is approved, there will be no dispute resolving its sufficiency.” If this type of collaboration is not possible, then parties can follow a second approach.
• Best practices approach. With this approach, Grimm argues that best practices and appropriate search technologies can be used in order to create a reasonable and defensible methodology in the absence of collaboration. Grimm goes on to specifically cite the Sedona Conference Best Practices document as a source of best practices. “In this regard, compliance with the Sedona Conference Best Practices for use of search and information retrieval will go a long way towards convincing the court that the method chosen was reasonable and reliable.”
The Sedona Conference Best Practices Commentary on Search includes a section
on “practical advice” which contains eight “Practice Points.” The second practice point is the foundation of implementing a search methodology. It states “Success in using any automated search method or technology will be enhanced by a well thought out process with substantial human input on the front end.” A specific process is not delineated for parties to follow, but the following key components are highlighted.
• Testing. Searches need to be tested for efficacy, i.e. whether the search is producing over or under-inclusive results.
• Sampling. The primary way to test the efficacy of a search is through sampling. In Victor Stanley, Judge Grimm states that “The only prudent way to test the reliability of the keyword search is to perform some appropriate sampling of the documents determined to be privileged and those determined not to be in order to arrive at a comfort level that the categories are neither over-inclusive nor under-inclusive.”
• Iterative feedback. Finally, the process of testing and refining one’s search based on the results of testing needs to be iterative so every refinement can be validated. Taken together; Victory Stanley, Equity Analytics, OKeefe and The Sedona Conference Commentary provide significant guidance on the known limitations of keyword search and how producing parties can defensibly conduct searches given these limitations. In practice, following this guidance can be challenging, requiring some very specific expertise and experience.
It is clear that lawyers can no longer get away with quickly assembling a list of “Google like” search terms, produce some documents and call it a day. Technology itself does not solve all the problems. The new transparent search requires the application of human legal and technical expertise, combined with search and retrieval methodologies that are grounded in scientific principles.
The “science” of search has arrived, and this likely means that another round of experts, or “search technologists” who specialize in creating defensible search plans must be employed to assure the courts that all reasonable steps have been taken during document production to produce what is required. Best practices include the documentation of the steps taken during the document production so that you can “show the court your work”. It is the future.
Comments