Get it right – computer assisted review

In a recent decision, the High Court considered the use of computer or technology assisted review (CAR) as part of e-disclosure.

What is e-disclosure?

Parties to a dispute rely on documents to substantiate their claim or defence. Disclosure is the stage in the civil litigation that requires parties to disclose the existence of documents which support their case or adversely affect the opponent’s case, or adversely their own case or support the opponent’s case. The disclosure of such documents is vital to ensure that all parties have access to the same information and have the same opportunity to prepare for trial. 

‘Standard disclosure’ requires parties to undertake a reasonable search for documents that are, or have been, in their control and which are relevant to the claim as described above. The definition of documents is wide ranging and includes electronic documents, such as emails and voicemail recordings. 

E-disclosure is a term used to describe the process of collating electronic documents. It often requires engaging e-disclosure experts to assist in the recovery and processing of the documents and the use of software to facilitate document review. The software and processes typically used by litigants allow keyword and date range searches to be carried out to reduce the volume of documents subject to the review process. Notwithstanding this, e-disclosure is often a time consuming and costly part of litigation. Solicitors are often tasked with reviewing thousands of electronic documents (174,000 in a claim we dealt with recently), the vast majority of which will not be relevant to the actual issues in the case. Over recent years, e-disclosure experts have developed software to assist the review process, primarily to reduce ‘workhours’ and the associated legal costs. 

CAR

Whilst CAR has been used in other countries for a number of years (the USA first approved the use of CAR as a method of document review back in 2012), it is relatively new to the English courts, only receiving approval here in 2016. CAR works as follows: 

• Lawyers (or other expert reviewers) review a random sample set of documents, marking the documents as either relevant or not relevant to the claim. It is imperative that the lawyers/expert reviewers have full knowledge and understanding of the issues in the case. 
• The software analyses the responses given by the lawyers periodically as the review progresses. 
• The software ‘learns’ which documents are likely to be relevant and applies its analysis to all of the documents in the database. It suggests which of the documents within the database are likely to be relevant, pushing those to the front of the queue for review. 
• Based on its learning, the software will indicate how many of the documents remaining in the database are likely to be relevant. Once this becomes minimal (the relevant documents having all been pushed to the front of the queue), the review can be stopped.

In the recent decision of Triumph Controls UK and another -v- Primus International Holding and others, the court considered the claimants’ use of CAR in a multimillion dollar warranty dispute. Initially, the claimants had agreed with the defendants to use keyword searches as part of its e-disclosure exercise. However, they then used CAR without discussion with the defendants. The claimants asserted that they reviewed a sample of 1% of the 220,000 documents using CAR. The result of the sample review was that only 0.38% of the remaining documents would be relevant and, as such, those documents had not been reviewed. 

The defendants made an application for (amongst other things) an order requiring the claimants to undertake a manual review of 220,000 electronic documents that had been identified as potentially disclosable following keyword searches but which, following a CAR exercise, had not been further searched. The court held that a manual review was appropriate. It reasoned as follows:

• The claimants had not done what they said they would do. The claimants agreed to carry out a manual review of all documents responsive to keyword searches. It was not clear that they intended to carry out a CAR exercise.
• Only 19,500 out of 450,000 documents were marked as disclosable following CAR. The court considered this to be modest, especially in a case that appeared to be document-heavy.
• CAR had not picked up a substantial number of documents (around 2,000) that were included in later rounds of disclosure and which were responsive to the keyword searches. 
• Some of the documents included in later rounds of disclosure were clearly relevant to the case and, in fact, were documents the claimants themselves sought to rely upon in witness evidence. These had not been disclosed following the CAR.
• The claimant did not provide detailed evidence on the operation of the CAR or the sampling exercise. For example, it was not explained how many rounds of sampling had been undertaken. This was unsatisfactory given that the decision to use CAR had been a unilateral one. It was not transparent and independently verifiable.
• The number and seniority of people involved in the review caused concern. The review was not undertaken by a senior lawyer; rather 10 paralegals and a few associates. This meant the ‘education’ process by the software may not have been as good as it might have been. 

Lessons learned

The claimants’ problems in this claim can be boiled down as follows:

• they did not do what they agreed to do
• they did not carry out the CAR process properly
• they did not record the CAR process carried out.

This case reiterates the importance of parties considering their disclosure obligations and, specifically, e-disclosure at an early stage in the dispute. Parties should review the type and number of electronic documents in their control and how they intend to search for relevant documents as soon as possible with a view to attempting to agree an appropriate approach to e-disclosure with their opponent. If changes to e-disclosure are required, attempts should be made to reach agreement with the other party or, as appropriate, the court. 

Any proposed change to an agreed e-disclosure process will be more open to scrutiny and, as such, the reasoning for the change and the process itself should be recorded thoroughly. Litigants should always consider whether the process proposed can be justified in terms of the overriding objectives of the Civil Procedure Rules, specifically parties being placed on an equal footing and saving expense. Most e-disclosure experts can give evidence on the work they have carried out; it is unclear in this case why the e-disclosure provider or lawyers were not able to explain the CAR better. If they had, the outcome in this case may have been very different.

For more information on how computer assisted review can reduce your e-disclosure costs, please contact Kate Steele.