Redefining the eDiscovery Review Process

DISCO brings artificial intelligence, cloud computing, and data analytics to the practice of law so attorneys can focus on what matters most: securing justice for their clients and winning the most important disputes in the world. Workflow is the feature used to organize and review documents for cases.

This feature set was the most legacy experience within the product but also one of the most crucial for customers to process their cases. While it was an innovative feature set for the legal space by evolving hard copy review processes to the cloud, there were still many opportunities to provide more value to users.

Challenges

  • With anything we worked on there was always the consideration that our users were actively working on discovery reviews for open cases. If we introduce a new format for the workflow space, how do we help users to quickly onboard and acclimate to avoid lag time in their reviews?

  • Legal teams were very used to having everything in hand when reviewing case content. This, while daunting, can also be reassuring. How do we create trust that we will handle their content safely and ensure it is always easily accessible?

  • With millions of legal firms across the world there subsequently many variations in how teams process case content in the Discovery process. We worked to understand how best we could recommend tested and effective workflows that leverage AI to save time and money.

  • Onboarding, training, tracking, moderating, and enabling review teams is a very involved process. On top of that if a reviewer isn’t able to get up to speed and meet the needs of the team quickly, teams would need to restart this process multiple times in a review. How can we provide metrics for teams to be as accurate and fast as possible in coding review documents.

How we measured success

  • Decrease overall review process time

  • Enable machine learning with standardized review workflows

  • Decrease reviewer staffing spend for customers

  • Decrease errors in the review process

  • Enable reviewer onboarding and team Q&As

Understanding our users

While every law firm may have a different structure or staff count, we determined the core user roles to be summed up by the lead attorney, review manager, and reviewer.

Review managers

Manages the day to day of the review process including managing reviewers and coordinating with the attorney team.

Reviewers

Reviews the documents for information in the review protocol created by the Attorneys in preparation for the case.

Attorneys

Sets up the case and defines the information that the review team will be searching for as well as updates that may come about from new learnings during the process.

Creating a vision

We met with managed review teams and lawyers to understand their needs in the discovery process and created competitor audits to get a better understanding of where to start and how to create the biggest impact for our customers.

  1. Metrics & team management tools

    1. Review data and project management features

    2. Knowledge base for case document management

    3. Review team hiring platform

  2. Defined eDiscovery workflows

    1. Templates for workflow set up

    2. Process management for the early case assessment

    3. Email updates with review data to external teams

  3. Review processing/coding improvements

    1. Tagging protocol with rules to ensure document processing is accurate

Defining eDiscovery workflows

In the past, when a legal team would start a Discovery review prior to using cloud based softwares, they would take legal boxes full of content and organize a team to comb though these manually to understand what is pertinent to their case. This process is piled with human error and making sure you can catch all the details before the review deadline is critical to success.

To help bring this process forward into the new age of technology and accomodate all forms of content, video, audio, and data types, we created a workflow builder. This would allow teams to stay on the same pace once the process started as well as be able to track how they are doing at every step.

  • How do we get started with an eDiscovery process?

  • Will we finish on time?

  • How many reviewers will we need to hit our deadline?

Review team management

A large gap we discovered was that as teams were trying to review documents as quickly and accurately as possible, questions or misunderstandings were common and could severely derail a review process.

This meant that managers would have multiple email or direct 1:1 conversations across the review team to solve the same problem in a review. Then if a review team is still having issues it requires retraining on the protocol to ensure an accurate review. This takes so much time from the overall Discovery process and costs thousands of dollars.

To solve this, we created an in app knowledge base so the team could share information and updates and ask questions without having to hold up the review process.

  • How do we get new information on review processing out to remote teams effectively?

  • How do we allow reviewers to ask questions?

  • How do we share common questions with the entire review team to avoid time lost?

Metrics

The feature that was most requested was to have more insight into the review process as a whole. Workflow had metrics but they were disjointed, complicated, and missing pieces. We looked for core metrics that would deliver value to the team daily while also providing clarity to the attorney team on the status of the review.

  • Is the review processing on pace for their deadline?

  • What are the reviewers finding in the documents?

  • Is the team completing all the steps required for a defensible review?

  • How much are we spending on the review?

  • Which reviewers are performing well/poorly?

User testing study on
Review Processing Metrics

Based on the feedback from the research sessions, we decided to move forward exploring a first iteration of Metrics for review. With previous feedback from users, we outlined the core metrics users would like on a regular basis and had participants organize and rank them while also choosing what level of granularity they would like to see.

Our goal was to discover what information best helped customers to manage their review. Using this feedback we were able to prioritize a balance of reporting that gave attorneys the high level answers they needed quickly while also providing the in depth and actionable insights that the review management team needed to make the day-to-day happen.

  1. How important is this problem?

  2. How do you currently solve this problem? What parts are the most painful?

  3. How urgent is this problem?

Top insights from users

  1. Target date (chosen by 90% of users)

  2. Findings (chosen by 90% of users)

  3. Spend (chosen by 80% of users)

  4. Total population reviewed (chosen by 60% of users)

  5. Today’s team pace (chosen by 60% of users)

  6. Responsiveness tag estimate (chosen by 60% of users)

  7. Today’s average reviewer pace (chosen by 60% of users)

For more information on this project, reach out via howdymagen@gmail.com