Mona Kalantar, a 3L at Northwestern Pritzker School of Law, has been a research assistant for Daniel W. Linna Jr. at since May 2019. In this guest post, Mona highlights ways for attorneys to become data-driven. Doing so will add to the general body of knowledge and create industry-standards that could lead to the improvement of legal services.
Guest post by Mona KalantarIn his 2012 article, Where is the ‘Quality Movement’ in Law Practice?, Professor William H. Simon argued that the quest for continuous improvement has largely bypassed legal practice. The legal profession has not fully embraced quality reforms that we have seen in other professions. Today, the drive for innovation is leading lawyers to think about how to provide greater value with fewer resources. The market is producing innovations within legal-services delivery organizations that promote streamlining processes, facilitating connectivity between legal professionals and clients, and creating opportunities for alternative fee arrangements. Despite the growing demand for legal-services delivery innovation, there is less evidence of demand for measuring the quality of legal work, performance, and outcomes.
Members of the legal profession could produce great value by developing objective measures for the quality of their legal work, performance, and outcomes. Corporate legal departments, law firms, and other legal professionals should become more data-driven to demonstrate their quality and value. Collecting such data would not only set the firm, corporation, or organization apart from competitors, but also add to the general body of knowledge available about industry-standards that could lead to the improvement of legal services.
Legal Professionals Can Reduce the Gap in Quality Knowledge
Legal professionals can lead the way in reducing the gap in quality knowledge by embracing opportunities to become more data-driven. One way is to address the lack of measurements of attorney performance. Law firms and legal departments that create, use, and retain consistent metrics for performance across the organization can generate data that helps shape attorney behavior and sets quality expectations industry-wide. Clearly established data-driven values help both law firms and legal departments assign work, make decisions, and create business opportunities.
There are many opportunities for legal professionals to become data-driven. Legal professionals can gather qualitative information such as the reasons, opinions, and motivations underlying others’ assessment of their delivery of certain legal services. Qualitative data can be represented quantitatively by assigning numerical scales to responses. Quantitative metrics, such as tracking email response time and average hourly billing, can also reveal patterns. By fostering a data-driven approach and pursuing opportunities to collect and analyze data, legal professionals can drive innovation and create continuously improving measurements of legal-services quality and outcomes.
Objective, Qualitative Measurements Are Within Reach
A first big step for legal professionals to take advantage of opportunities to become data-driven is standardizing qualitative metrics. Many legal professionals and academics have become interested in the idea of scorecards to measure the value of legal services. The Association of Corporate Counsel’s (“ACC”) scorecard is just one example of the legal profession taking charge of using data-driven metrics to measure work quality. The ACC provides corporate counsel with its recommended scorecard framework for measuring outside firm performance. Their structure suggests eight key metrics:
- Understanding client goals;
- Expertise in the issue;
- Responsiveness;
- Process management;
- Cost;
- Execution of the objective;
- Compatibility with company values; and
- Whether the attorney would hire the firm again.
Corporate legal departments can share their scorecards in the ACC index.
Qualmet has attempted to address the lack of consistent industry standards and make value quantifiable. Started by Mark Smolik and Jim Beckett in 2017, Qualmet uses a scorecard system to generate data-driven value measures of work performed by legal professionals. Presenting standard key performance indicators (KPI’s) for the legal industry saves Qualmet’s customers from spending time debating which metrics to use. Qualmet’s industry-aligned KPI’s and legal metrics are combined with side-by-side firm comparisons that create benchmarks to provide quick and understandable results that law firms can use to improve their services and differentiate themselves. Qualmet also has a searchable database of evaluated service providers ranked on their metrics system to provide meaningful qualitative measurements by which organizations can compare themselves. Qualmet is a solution for legal professionals to understand how to increase their value and become better service providers.
There is also a growing number of databases with information about outcomes. Clients and law firms are using this data to assess the experience a lawyer has in matters to quantify outcomes. Lex Machina, a division of LexisNexis, gathers data about judges, lawyers, parties, and subject matter to curate data that can help to predict case outcomes that different legal strategies will produce. Similarly, Westlaw Edge provides litigation analytics demonstrating how judges handle different cases and motions, which legal departments can use to evaluate their counsel and opposing counsel’s strengths and weaknesses.
Quantitative Measurements Reveal Interesting Patterns
Another step forward in measuring the quality of legal work is collecting quantitative information and viewing it in the aggregate. Legal technology has already produced a few data-driven products that help corporations and law firms evaluate specific concerns and make informed decisions, both internally and externally, about their business relationships.
Clerk by Judicata provides data about substantive legal quality. It helps improve brief writing by analyzing and evaluating areas of improvement and attack. CEO and Co-Founder Itai Gurrari created Judicata to generate objective, empirical measures of lawyer quality. Clerk reports a grade for how well a brief is argued, drafted, and compares outcomes with similar cases.
Other technologies focus on data regarding legal-services delivery to measure cost-effectiveness. Serengeti Tracker (now Thomson Reuters Legal Tracker) also provides comprehensive data-driven metrics. This product allows corporate legal departments to measure their performance and spend and enables legal departments to benchmark their financial performance against peers. Legal Tracker has generated data from over 950 corporate law departments and 42,000 law firms to create aggregated comparative data.
Legal Tracker allows corporate legal departments to track a variety of quantitative metrics for their outside counsel. For example, Legal Tracker allows corporate legal departments to measure actual spend to budget. It also allows corporate legal departments to measure staff workload metrics. Some metrics include tracking the number of matters performed, variety of tasks performed, spend per-lawyer, average spend per matter type, and time spent on an issue. Other notable quantitative data-driven metrics included spend by matter type and business unit and outside and inside spend as a percentage of company value.
Nevertheless, while we have more data than ever, it is underutilized to measure the quality of legal work. In my experience, corporate legal departments are emphasizing the importance that outside counsel demonstrate the value that they deliver. Legal departments do not have the luxury of time and money to hire multiple firms on the same matter to compare and contrast work product. Corporate legal departments could use data produced by Clerk to evaluate an outside firm’s performance on litigation matters or use Legal Tracker to showcase their inherent value to management. Legal departments could use the litigation analytics of both Lex Machina or Westlaw Edge to quantify their success rates on motions and cases as compared to other legal departments. These are just a few examples.
Opportunities Are Endless
For the legal professional who is not ready to commit to a product or service, becoming data-driven is not an impossible task. Neither should it be a low priority. Regardless of size or budget, legal professionals can become more data-driven as long as the data is easy to understand, manageable, accurate, and updated. The journey to becoming truly data-driven can start with collecting and analyzing your own “small data.”
Client Satisfaction Data
One manageable approach involves creating client or vendor satisfaction scores. To be effective, leadership must collaborate to identify metrics that align with organizational values. Organizations can borrow from the ACC suggested framework to come up with a variety of questions they would like to ask their outside counsel. A good scorecard explains with specificity what a number means. Ideally, these scores are collected in every interaction and reviewed often to identify areas of strength and weakness.
Risk Mitigation Scorecard
Another data-driven opportunity is to create a risk mitigation scorecard that measures how well legal counsel has met or exceeded expectations on risk. Lawyers on Demand provides an online template for this.
To create a risk mitigation scorecard, legal departments will need to create a corporate risk mitigation profile for their outside counsel. Corporate risk mitigation profiles encompass a variety of ways that outside counsel has successfully mitigated risk on a matter. Lawyers on Demand recommends that corporate legal departments create a risk profile encompassing a range of standards across a range of contracts entered into. Then legal departments should set a target risk profile for the range of contracts and audit them monthly.
Monitoring the percentage of contracts entered into that fall within the desired risk profile reveals valuable information about outside counsel. Legal departments can also use the data to compare different law firms performing similar tasks. The information is useful in sourcing for the best value when making staffing decisions.
The opportunities to become data-driven are endless. In addition to the ideas discussed in this post, Michelle Graham at Practical Law provides a comprehensive and practical list of opportunities to adopt a data-driven culture within an organization.
A Data-Driven Quality Movement is What the Legal Profession Deserves
There are many benefits to be had for legal professionals who seize on opportunities to drive a quality movement in their practices. It demonstrates a drive for quality already seen in other industries, such as the practice of medicine. It signals to clients that the lawyer cares about the client’s bottom-line needs for efficiency and effectiveness in service delivery. It also helps lawyers set measurable goals to meet their desired performance benchmarks.
While collecting data for every business transaction, internal and external relationships, and personal performance may seem like a daunting task, obtaining an accurate picture of the quality of legal services delivered and received will provide significant benefits. Like any competent business professional, legal-services providers should take charge of the valuable information in their organizations to create metrics for quality and outcomes that will differentiate themselves to clients and move the baseline for what exemplifies quality services.
