Last Thursday, Regina Rana, one of our senior associates, participated in the Women’s Bar Association of DC and the DC Chapter of Women in eDiscovery’s joint panel on document review and top tips for attorneys. She was joined by fellow panelists Ann Marie Duffy, Allison Stanton, and Carrie Betts.
This all star panel discussed their unique experiences with managing document reviews and spoke about best practices for utilizing resources, building partnerships, and identifying what each player needs for their part of a review. The event highlighted the stressful aspects of managing a document review and pulled from the women’s perspectives as a partner, an in-house counsel attorney, a staff attorney manager, and a senior associate.
Below is a summary of the WBA panel. Click here to view the full recording.
Overview of WBA Panel: “Document Reviews and Top Tips for Attorneys From the Trenches”
Q1: What tips would you give to associates who are new to managing a document review?
Ann Marie Duffy (Partner, Hollingsworth LLP): Document review involves the massive undertaking of collecting all of your documents that you are going to need for a particular litigation. The most important thing is organization. You should take advantage of Excel, tracking, and documenting everything. Develop protocols so that everyone is on the same page with how you are going to collect everything.
Allison Stanton (Director & Associate General Counsel, eDiscovery and Information Governance, Facebook): You want to look at a document as an opportunity to learn about the case. You are a detective, and you are leading a bunch of detectives. You want to think about the different legal and strategic aspects that you need to keep at the forefront when you are leading a document review. Take a holistic look: think not only about privilege issues, but about the strategy that is trying to be advanced in the case.
Carrie Betts (Staff Attorney Manager, Covington & Burling LLP): Take a deep breath. How many documents do you have? Look at your population. Has everything been laid out in the smartest way possible? Have you taken all the approaches to technology and analytics that you can? Review your documents before your reviewers see them; take a minute and then communicate with people what you need from there.
Regina Rana (Senior Associate, Tanenholz & Associates, PLLC): It’s really important to get ahead of all the challenges up front, if you can. And I’m coming from a very boots on the ground perspective of actually managing the review team and the review process at that level. Nail down what you want to get out of the review protocol and make sure that everything you are communicating amongst the managers gets communicated down to the bottom level as well. Much of that comes from analyzing the review set ahead of time and making sure that you’re solving the problems that actually exist rather than just hypothetical problems. Understand the size of those problems by actually having that early analysis of the review set.
Q2: What does an efficient document review look like?
Allison: The first step is figuring out what the review protocol is going to look like, making sure it is a comprehensive training guide that you will implement on an operational level. You really want to identify how you need to answer the question, protect privilege, and build strategy. Give instant feedback during the review and prioritize communication.
Carrie: Make sure everyone knows what’s going on with the protocol. Avoid side conversations, as this often leads to things being missed. Keep everyone on the same page from day one of the process. Don’t be afraid to overload people’s inboxes; let them decide what they want to tune into.
Regina: Stay on the same page as a whole team, even if there are portions of the review that you are not directly working on. There could be implications of that workflow that ripple across what you are doing. For example, if you pull a sample batch for QC, but you also have an in-house team that is also reviewing the same documents for another reason, it can cause problems. So having everyone on the same page, everyone aware of what workflows are going on at all times, is a huge deal.
Early analysis of the review set is important, especially for privilege considerations. If you have a complex, fact-specific review, with a lot of intersecting parties, you will have to develop the privileged relationships. If you can do so as early as possible, that can help with a lot of problems down the road.
Ann Marie: My immediate thought was there is no such thing as a perfectly successful, efficient document review. You can put valuable protocols in place, but there will always be bumps. There will be unexpected requests and court orders. Protect yourself from the possibility of getting sanctioned. That is where communication in the organization really needs to come in.
Q3: How has Covid changed managed review? What are your thoughts on how it may change going forward?
Carrie: I am proud of how my team and people we work with have adapted and accepted these new challenges. The situation keeps changing. Productivity has not changed, but maintaining quality requires more effort and more time. We now have to rely on calls and emails to communicate with one another. Everyone is on their own more. I want to get back to the office in some capacity because I miss my team. I think having face time with people is valuable. I think the spaces will change and we will have to be flexible. I have been floating the idea of making review centers more like onboarding facilities- starting a team there and giving them that face time. Then, after a week or two, give them the option to work at home. But we are at some point going to need to do some form of transition back into the office.
Regina: I think that Covid has forced even more organization and development of best practices that can be carried on even after there is some return to the office. Working through Microsoft Teams has forced us to be very organized about how our collaboration occurs. Even though you lose something from not having that in-person collaboration, you do have a nice, neat record of what has been covered and what needs to be handled. So I do appreciate that.
Ann Marie: I miss going out and meeting my reviewers. From my perspective as a partner, I would be doing everything from email, and I am still doing that. Carrie raised an interesting point of whether we will go back to review centers. I like the idea of onboarding people for a week or two because there is so much value in that initial meeting. As we are all learning, remote is working just fine. It allows for flexibility and helps people be more efficient.
Allison: We had to pivot a number of extremely large reviews. Our primary focus was security. We were pleased with the vendor partners we had and the protocols that were put in place for security. We ran some studies and found that the efficiency of reviewers was actually a bit higher with them being able to work remotely. I do think it’s extremely important that people are very focused on not only just how do we facilitate remote work, but how do we facilitate making sure that it’s securely done? And what is that environment that your reviewers are in? It takes a lot of prep conversation, vetting the solutions, security conversations with systems, and all of that sort of thing.
Q4: What are some tips and tricks for privileged QC?
Allison: Think about other related matters that you should be thinking about. What’s the privilege screen? What are the terms that should be run in advance? What are the QC levels and what are the different contradictory tagging checks? Is ‘confidential’ in the footer of every single email in the organization? Do some people not show up on attorney lists? You want to have these conversations to get to know the environment, the acronyms, and the specific circumstances.
Carrie: Inherent curiosity is important. I recommend a tiered privilege search term report approach. You have your general privilege terms, AMLAW 100 terms, your case-specific privilege terms, and what we call “black-eye” terms (please don’t produce without running these). A random QC without any follow-up to explore what was overturned is basically useless. Once you’ve done your QC, take time to look at duplicates and near duplicates, and pivot and play around with your privilege terms. You want to make sure you’re targeting these populations and that you’ve done the most efficient QC.
Regina: I think the first question that I would ask is, What are you performing the privilege QC for, and at what stage of production are we? If you’re doing the privilege review for a final privilege log, that can look very different from mid-case review during rolling production, in which you are mostly scrutinizing document families that have no privilege coding on first pass review and are running sweeps to ensure no privilege information is inadvertently produced.
Take a tiered approach to privilege review, and understand where the privilege pitfalls are going to be for your particular case. It’s one thing to have every top 100 law firm highlighted, but it’s another thing when you are working in a bankruptcy matter and you have all these parties and consultants and other lawyers that work for the other side that you need to identify for potential privilege breaks or common interests at a very specific period of time. So these very specific custom searches are run after your normal tiers of searches.
You can do sweeps based off document type, depending on the types of documents that are more likely to be privilege. These are typically emails, but you also want to look for things that have hidden content or annotations like word documents that may be draft contracts. You may also learn that certain PowerPoint presentations or other corporate documents have sections that were drafted by legal, even if they don’t have clear indications on their faces. Often there is a date specific to the timeline of events at issue which has a bright-line bearing on when the work-product doctrine can be applied. These are the things you develop over time as you gain knowledge about the case.
Allison: In addition to those excellent points Regina, remember that privilege isn’t the same everywhere. If you are dealing with an international organization or dealing with non-US matters, you really need to get a sense of how the privilege rules apply in those other countries, because many of the US protections may not apply.
Q5: How might Covid affect our ability to work with individuals outside our usual jurisdictions? Will covid affect our pricing?
Carrie: Before covid, we would source contractors from wherever we were having the actual review. We have now expanded out and are working with contractors and markets that we otherwise wouldn’t have tapped in. The technology we have makes this easier; we can lock IP addresses and have certain protections in place. This drove down pricing drastically last year and we’re seeing it come back up a bit.
Allison: Make sure you are clear on what data access is allowed in the jurisdiction. Make sure there is a clear transparency with the client about where the reviewers are located.
Q6: What are the most important to-do’s for a review manual/protocol?
Ann Marie: You have to give a lot of information to reviewers and prioritize communication. And Regina’s sort of hit home right at the beginning, which is communication from the very beginning. I want my review attorneys to know as much about the case as I do. That’s the only way you’re going to get a successful review, because they need to know. They need to know what the claims of the case are. They need to know what your client does. They need to know who the individual players, the custodians that you’re collecting from are, what their roles in the company were, what their roles in the litigation, whether it’s a product liability litigation or an environmental case, what their role was at the company.
So the very first thing is a big overview and a detailed overview of the case as if you were bringing on another full-time attorney because you are bringing on another full-time team to help you. And Alison nailed this– you are a detective. You are looking for not only the great stuff that’s going to help you prepare your side, but you’re looking for the really bad stuff, too, that the other side is going to use against you in depositions, in motions, hearing, and then eventually a trial. So you need your review attorneys to be that first line to flag for you.
I think the other thing and a couple of other panelists have talked about this, is that you really want to think about what you’re asking your reviewers to do, and limit the number of tags you are asking them to employ. Not only does it slow them down, but at the end of the day, you as the attorney who’s going to be preparing for the deposition or preparing a summary judgment motion or preparing for trial, you’re going to run your own searches on what’s produced. You’re going to run your own keyword searches to find most relevant documents. So what I always think of my review attorneys as largely categorizing at a high level what’s in my production.
So limit the number of tags. And then the other thing is along communication. Share your ESI Orders. A lot of courts are now moving to pretrial orders where they just document absolutely everything from start to finish- how your ESI protocol is going to go, your confidentiality. I mean, these aren’t just simple protective orders anymore, so you want to make sure that your review attorneys and your vendors know all of that.
Q7: When you receive a review protocol, what are the first things that you do?
Regina: I read it over once; I’m really looking for the context of what is important to the trial team, and how they’re defining the scope of review based off case history. I think this often helps resolve the 50/50 calls. I then usually try to figure out what the review team is being asked to do and what parameters they are being asked to code. I look to see whether the instructions are to code on the four corners of the document or by family. I very much prefer four corners coding. If you end up needing to produce by family and having the harmonization there, you can do that on back end, but otherwise trying to code large families with the same coding can require too much time spent on going back and forth between documents.
I then look at the issue codes, and try to figure out what exactly they are being used for. Sometimes they are used as large buckets to look at more closely later. Sometimes, certain issues codes are almost by definition going to contain hot or key documents. So it’s important that the issue code language in the review protocol is clear and something the team can understand and refer to during their daily review.
I would also echo the sentiment of the thematic grouping. If sub-issues can be used to nest related issues together, you can get a much tighter work product that way.
The last major thing I look at in a review protocol, is what we all discussed about the privilege QC’s and the privilege considerations beyond just the standard boilerplate. I look to the special considerations for this particular matter. Here are parties to watch out for, whether you have an in-house counsel custodian, do in-house counsel distinguish when they’re talking about business only things versus legal matters, and how to treat that. Are we being asked to err on the side of potentially privileged for items that are mere forwards or cc’s to attorney. Or, are we being asked to be more discriminating during first pass review. My goal is to resolve all those problems up front, make sure the protocol contains all the information the team needs, and that ensure that the layout reflects what we want to achieve for the review.
Q8: What is your perspective of document review as an in-house counsel?
Allison: We have to think about how we are going to understand the story. There is a story that needs to be told on both sides. From an in-house perspective, I’m looking for the best strategy that we’re employing in using advanced technology and workflows. I’m also looking for consistency. We have to make sure we are providing right and accurate information, keeping cost at the forefront.
Q9: What are some tips for associates that are new to document review, and how they can best leverage their skills as they are learning day-to-day?
Carrie: The primary piece of advice is to ask questions. Lean on your team for help; we were all new to this at some point. Everyone is learning as the technology and best practices keep evolving.
Q10: Do you have any tips on how to approach discussions with opposing counsel?
Ann Marie: You both have differing objectives. You don’t want to look at this as search terms and document custodians; you have to get your story out through your documents, too. You want to be able to negotiate with plaintiffs to the opposing counsel to know what are really the terms and requests they are asking for.