Technology Assisted Review for Corporate Legal
Applying Analytics in Everyday E-Discovery
-
2024年6月04日
-
In a world where technology, AI and automation have penetrated nearly every facet of day-today life, it’s easy to take technological advancement for granted. Easy to assume that it has been embraced as intuitive, across all industries and business processes, to optimize efficiency and replace cumbersome manual work. In the legal field though, reality has not quite caught up to expectation. Despite significant advancements in legal technology, widespread judicial approval of machine-powered processes and a mass of proven benefits, advanced analytics and technology-assisted review (TAR) are still regarded by many lawyers as complex.
A recent survey from CLOC reported that—despite mounting pressure to do more with less—only 12% of corporate legal teams report using advanced analytics or TAR, while less than half say they are exploring the technologies to achieve performance gains. Gartner predicts in-house legal technology spending will increase threefold by 2025, but it’s unclear how much this increased spend will be allocated toward TAR tools that are used to reduce the time, cost and complexity of e-discovery versus other legal operations functions such as automated billing and contract lifecycle management.
With ramping demand for legal departments to be more efficient and reduce costs, in-house counsel must start getting comfortable with TAR across the spectrum of culling irrelevant documents, identifying key facts, enabling review quality control and prioritizing documents in review. Broader adoption of TAR will be potentially the most strategic and impactful way for in-house legal teams to meet the demands upon them. E-discovery practitioners experienced with TAR attest to the fact that analytic review consistently outperforms linear review in terms of time savings, reducing document sets and streamlining costs. Moreover, by extending the use of TAR beyond e-discovery, to applications such as internal investigations, compliance, due diligence, deposition and trial preparation, legal teams can add greater strategic value to their organizations.
How to do this? It’s a lighter lift—and investment—than most legal counsel might think. This paper will provide a deep dive into TAR, covering the current workflows available, when and how to use them and the benefits of applying TAR to everyday e-discovery matters.
Key Workflow Principles
Specific processes and back-end technology will vary depending on the e-discovery software being used, but several key principles apply to the most common TAR workflows.
TAR 1.0, or the model-based approach, is the first-generation TAR whereby a small group of subject matter experts train a model by reviewing exemplar positive and negative training documents. The model is trained iteratively until it is stabilized or achieves an acceptable recall and precision benchmark against a representative random sample. This has been traditionally used as an initial step to reduce volumes before full document review begins.
TAR 2.0, also known as Continuous Active Learning (“CAL”), is second-generation TAR that improves upon some of the challenges faced in TAR 1.0. Rather than being used as a discreet initial step where the model is trained by subject matter experts and documents are categorized, this approach operates more like a standard review workflow. The system learns continuously as review decisions are made by automatically re-training the model, updating its predictions and re-prioritizing the documents so that reviewers are looking at the documents that are mostlikely of interest.
Understanding Technology Assisted Review
For corporate legal teams to accelerate adoption, they must first understand the different types of TAR and what e-discovery experts mean when they refer to the various formats.
While additional TAR workflows continue to emerge, like TAR 3.0 and even 4.0, for the purposes of this guide and to establish a foundation for learning, focus will be placed on the most common forms, TAR 1.0 and 2.0. Both forms have their unique strengths, weaknesses and suitability depending on certain scenarios. Understanding the underlying technology and processes is important in determining which workflow is the best suited to a matter’s objectives and scope.
In TAR 1.0, an experienced review team will train the model ahead of time and can do so using one or a combination of random sampling (simple passive learning), active learning or conceptual diversity sampling. A statistically representative sample is then reviewed and used to measure the effectiveness of the model. Through the model building process, documents are ranked or categorized into positive and negative buckets.
Advantages:
- A predictable number of documents are needed to review, which helps with planning and review management.
- Enables the production of likely relevant documents without review, which is typically required in complex voluminous litigations and Second Requests.
Potential Pitfalls:
- Model based predictive coding requires up front training, which takes time and must be completed prior to starting a review. This is one of the primary criticisms due to the opportunity cost of involving subject matter experts to train the model and delays in starting full document review.
- Increased complexity due to questions about transparency obligations and the need to negotiate a TAR protocol with opposing counsel or regulators. While this step is less burdensome than it used to be given increasing adoption and understanding of the tools, it can still pose challenges depending on the parties involved.
- Introducing new document sets require additional sampling and may require the model to be refined or re-trained.
- Matters with low richness of relevant data requires review of extraordinarily large number of documents to find sufficient relevant documents in random samples and training sets.
In TAR 2.0, legal teams do not need to make an up-front investment in preparing a model. This approach is adopted more broadly because review can begin right away, and the workflow is more flexible and fluid. The system actively provides or suggests documents to the review team based on its own learnings as review progresses and coding decisions are logged in the system. Documents are continuously re-prioritized to automatically surface those that are likely to be of the highest relevance to reviewers.
Advantages:
- Review can begin right away as no up-front model training is needed.
- Subject matter experts may review a random sample to benchmark and estimate outcomes, but it is not required for the model to work.
- Can handle rolling data loads seamlessly, as adding new documents to the review set does not require additional sampling—unless deemed necessary by counsel for statistical estimates.
- Typically, does not require up-front negotiation of TAR protocol with outside parties.
- Preferable in low richness scenarios.
Potential Pitfalls:
- Can make it more difficult to estimate timelines, as the number of documents needed to meet the target benchmarks such is recall is unknown, thus impeding planning.
- Does not allow teams to produce documents without review.
- Unless diverse sets of documents are fed into the process, it risks missing or delaying surfacing of documents that are not highly prevalent in the overall review.
What to Use and When
Both TAR 1.0 and 2.0 are effective for reducing the burden of e-discovery and helping counsel work smarter in any matter. There are scenarios where one workflow is better than the other. The key is to abandon the mindset that there are TAR cases and non-TAR cases and the belief that TAR can only be used in large matters with hundreds of thousands or millions of documents. Rather, counsel should think about these tools as highly flexible and applicable in a variety of ways. Given how the tools have evolved TAR workflows can deliver benefits in most cases today—including small matters.
Once a legal team has recognized that they can leverage TAR on any case without significant added cost, they can refer to the advantages and potential pitfalls of each of the workflows (as outlined earlier in the paper) to determine which approach is best suited to their case.
Generally, TAR 1.0 may be more applicable when counsel prefers to involve subject matter experts in training the model or is dealing with a highly complex and/or highvolume matter like a massive litigation or Second Request. Conversely, TAR 2.0 can support most types of matters, especially those in which counsel:
- Lacks resources to involve subject matter experts
- Needs to review and produce quickly
- Expects a rolling influx of new documents
E-discovery providers and outside experts can also help legal teams in understanding the objectives of each matter and tailoring the right TAR workflow to meet its needs. As the team becomes more comfortable with the tools, they can consult their expert advisors in learning how to apply them to additional value-add use cases such as surfacing key facts for case development, quality control, review batching and more.
Supporting Defensibility
Defensibility has long been the primary concern among lawyers who are hesitant to adopt TAR. However, TAR has been approved in many courts and accepted as an effective methodology by many regulators. Defensibility will not be an issue when the right methodologies are in place alongside reliable technology supported by people and process.
Below is a checklist of the key components that will increase familiarity and adoption of TAR workflows in a defensible manner:
- Select an e-discovery platform or analytics tool that is widely accepted in the industry and has a precedent of judicial approval.
- During technology selection, vet the features available and allow the team time to practice with them. Practice exercises may include utilizing prioritization on a small matter or applying TAR to quality control review to identify documents where human coding disagreed with the model.
- Ensure reviewers are adequately trained on the platform and understand the review coding parameters.
- Involve experts—either in-house or third party—who can advise on technology deployment, workflow creation and standard operating procedures.
- Establish consistency in terms of process and workflows across the team and across other groups within the organization.
- Document all decisions made about which documents are subject to the TAR process and validate the process using statistics.
Reducing Intimidation Factors
There are only so many ways to optimize a legal department, which is why in-house counsel must take advantage of the low hanging fruit. This means using technology that’s readily available to reduce the time and costs associated with the most burdensome task: e-discovery.
There’s often a suspicion that advanced e-discovery tools and analytics are powerful beyond limits—which leads to intimidation and stalled adoption. Yes, these tools are powerful, but they also have limits. When teams become familiar with them and learn how to apply the right workflows to their matters, they will see that the benefits far outweigh any perceived downsides.
E-discovery technology has evolved to a point where there’s not a steep learning curve, and the cost of running TAR on a matter has become marginal. There’s no longer a reason not to use it. Teams can start small, demonstrate and celebrate their wins and rely on expert support as needed. The use of TAR and other analytics is the prime opportunity for legal teams to achieve progress toward modernizing their technology stack and optimizing precious time and resources.
出版
2024年6月04日