How to Measure and Address Disparities in Court Experiences and Outcomes

Steps for making civil courts more equitable

Fact Sheet
Projects: Courts & Communities
Share
Twitter
Facebook
LinkedIn
Email
Print
How to Measure and Address Disparities in Court Experiences and Outcomes

Overview

Sociodemographic data can help state and local courts better understand and address disparities in court access, navigation, participation, and outcomes across racial, ethnic, disability, income, gender, age, and linguistic populations. Armed with this information, courts can help their users more fully participate in the legal process and help judges render fair outcomes.

Courts can increase knowledge of and make progress toward reducing disparities in user experiences and outcomes by implementing two key practices:

  • Publish demographics about court users and the outcomes of their cases.
  • Use data to identify, implement, and measure the impact of reforms designed to reduce disparities in court experiences and outcomes.

After extensive research, The Pew Charitable Trusts has developed a framework outlining how and why courts should modernize.1 These steps arise from that work and can help programmatic and operational court staff, along with court leadership, assess how they are collecting demographic data and using it to improve the court user experience; identify opportunities to improve; and decide—with input from relevant stakeholders—which of those opportunities to pursue and how.

Step 1: Bring together relevant court staff and external stakeholders

These groups can contribute important perspectives and insights about how to better understand and reduce disparities.

Court Users can provide feedback on collection of demographic information, such as whether they would choose to respond to questions about their race or age, how they would prefer to be asked such questions, and what would be the most convenient way for them to provide the requested information.

Leadership can prioritize collecting and using demographic data to inform decisions about how to deploy needed resources and can convene working groups to address identified issues.

Access to justice staff can convene relevant court personnel and stakeholders, chair working groups on disparities and disproportionalities, identify solutions, and work with court leadership to implement reforms.

Legal counsel can provide guidance on how demographic information about court users and staff should be used and presented and what mechanisms should be in place to protect their privacy.

Clerks can share what challenges they see users experience related to accessing the courts (e.g., inability to file/find forms), ensure that data is collected in a standardized manner, and share data with their local court administrators.

Research staff can lead research or partner with external experts to analyze disparities and evaluate implemented policy changes to see if they are having their intended impact.

IT Staff can work with vendors to add demographic data fields to case management systems and with relevant court staff to develop dashboards or other internal tools to track progress and outcomes.

Website administrators can publish information, reports, and dashboards in prominent places on the court website and ensure their usability and functionality.

Community partners can conduct outreach to affected groups and work with courts to identify interventions that can help people resolve legal issues without a court case as well as in-court solutions to make court processes easier for users to navigate.

Legal stakeholders (e.g., legal aid, public defenders, law firms) can participate in discussions about when and how their clients’ demographic information should be collected, whether and how they can obtain data from their clients, and whether and under what circumstances they would be willing to provide that information to the court.

External researchers analyze disparate effects of various policies, conduct evaluations of implemented reforms, and identify strategies to reduce identified disparities.

Step 2: Assess current practices and set next steps

The following metrics can help courts assess their progress toward collecting and using data to help illuminate and inform strategies to reduce disparities in court access and outcomes. (See Tables 1 and 2.)

For each metric, determine whether the answer to the initial question is yes or no using the suggested measure. If the answer to the metric question is no, pursue the suggested next steps in collaboration with staff and stakeholders. The suggested steps are not prescriptive; instead, they provide ideas and options for getting started. The state examples can help courts determine what actions are feasible given available resources.

Table 1

Courts Should Analyze and Report User Demographic and Case Outcome Trends

Metrics, suggested steps, and state examples and resources

Metric If not, suggested next steps Examples and resources

Does the court collect users’ demographic information?

How to measure it:

Review existing data the court collects about users.

  • Add data fields to the case management system as needed to capture sociodemographic information.
  • Work with IT staff and legal counsel to ensure that robust measures are in place to protect the privacy and security of sensitive demographic data.
  • Collaborate with trusted community organizations to conduct outreach about data collection efforts and share with court users, particularly those from historically marginalized populations, how and why this information is being collected.
  • Consider collecting demographic information from participants in court-based mediation programs, co-parenting courses, and evaluative services.

Who’s involved:

Internal Experts:

  • Clerks
  • IT Staff
  • Leadership
  • Legal counsel
  • Researchers

External Experts:

  • Community Partners
  • Legal Stakeholders
  • Researchers

Court Users
  • Courts can use several methods for capturing demographic data:
  • Self-identification. Courts can collect demographic data by asking court users. The National Center for State Courts (NCSC) has a resource on how to do this and has recommended data fields.
  • Linking to census data. The Utah courts are pursuing sharing court data with the U.S. Census Bureau to match court users in certain dockets to information those users previously shared with the bureau.
  • Statistical modeling. January Advisors used Bayesian Improved Surname Geocoding—a system for estimating an individual’s race and ethnicity based on name and address information—to identify and understand disparities in civil filings and outcomes in Minnesota.
  • Geomapping. Reinvestment Fund geomapped eviction court data, which revealed disparities in eviction rates and other variables among residents of Black, White, and Hispanic neighborhoods.

Does the court publicly report information about sociodemographic trends in case filings and outcomes?

How to measure it:

Review whether the website includes information about sociodemographic trends.
  • Ask advocates, researchers, policymakers, and other stakeholders what types of resources (e.g., dashboards, narrative reports) would best help them understand court users’ experiences, outcomes, and filing trends.
  • When sharing data with stakeholders, include contextual information such as data limitations to support informed interpretation of the data and help avoid stereotyping of marginalized populations.
  • Work with the court’s legal counsel to ensure that demographic information publications protect court users’ privacy and align with court rules for data disaggregation (e.g., can the data dashboards identify trends at the judge level?).

Who’s involved:

Internal Experts:

  • Access to justice
  • Clerks
  • Leadership
  • Legal counsel
  • Researchers

External:

  • Community Partners
  • Researchers

Court users
  • Although no state currently publicly reports user demographic data for all civil cases, some have published this information for certain noncivil cases. For example, Massachusetts has a public website that reports demographic data on court users in juvenile matters, and Maryland has a publicly available violent crimes dashboard that includes the demographic information of defendants.
  • The American Equity and Justice Group’s Equity Dashboard presents Washington state criminal legal system data and provides context to help users understand what the data does and does not show, and how data limitations can mask disparities.
Sources: K. Genthon and D. Robinson, “Collecting Race and Ethnicity Data” (2022); T. Samuelsen (director of judicial data and research, Utah Administrative Office of the Courts), (Aug. 29, 2023); Minnesota State Bar Association Access to Justice Committee, “Minnesota Consumer Debt Litigation” (2023); I. Goldstein et al., “Evictions in Philadelphia: A Data & Policy Update” (Reinvestment Fund, 2019); Massachusetts Trial Court, “Massachusetts Trial Court Data Dashboard: Demographics of Selected Juvenile Matters”; Maryland State Commission on Criminal Sentencing Policy, “Crimes of Violence Data Dashboard”; American Equity and Justice Group, “Equity Dashboard”

 

Table 2

Courts Should Use Data to Address Disparities Related to Their Processes and Rules

Metrics, suggested steps, and state examples and resources

Metric If not, suggested next steps Examples and resources

Do court personnel know (not just anecdotally) who is being brought to court, who is participating in their cases, and what outcomes those participants receive?

How to measure it:

Assess whether court personnel are accessing a dashboard or other sociodemographic data reports.
  • Ask judges and court staff where and in what format they want this information presented and how they want to use it.
  • Train court personnel on the importance of demographic dashboards and reports and make sure they know about and how to use court equity resources and projects.
  • Provide a mechanism—such as a question on the data dashboard or webpage that asks, “Is this helpful?”—to capture ongoing feedback from court personnel.

Who’s involved:

Internal Experts:

  • Clerks
  • IT Staff
  • Judges
  • Leadership
  • Researchers

External Experts:

  • Researchers
  • The Pennsylvania courts partnered with NCSC to overhaul and clean their data and translate it into dashboards for judges, which include demographic filters that, among other functions, allow family court judges to look at case outcomes based on race.
  • Maryland publishes extensive data on courts’ language services and uses that information to determine which populations may require additional assistance, such as interpreter resources.

Has the court enacted reforms designed to reduce disparities in court users’ experiences and outcomes?

How to measure it:

Review status of any reforms recommended by court committees, task forces, and other advisory bodies.
  • Analyze court data and work with external experts to identify disparities in the jurisdiction or state related to race, ethnicity, disability, age, gender, sexual orientation, and language access.
  • Consult with community organizations, court users, and peer courts to identify steps in a court user’s journey where the court could simplify processes and connect court users with needed resources and services.
  • Establish a committee to conduct a racial equity assessment or similar analysis of court user and staff experiences to identify disparities and to devise and implement reforms.
  • Conduct community focus groups or listening sessions to establish a baseline understanding of the issues court users experience and to assess progress toward reducing disparities. These stakeholder groups also could be maintained or reconstituted in the future to provide feedback on or evaluate the reform.

Who’s involved:

Internal Experts:

  • Leadership
  • Researchers

External Experts:

  • Community partners
  • Legal stakeholders
  • Researchers

Court users
  • In 2021, the Kentucky Court of Justice published its “Guide for Identifying, Addressing, and Reducing Racial, Ethnic, and Equity Disparities,” which outlines the four steps the court took to reduce racial and ethnic disparities: using court data to identify disproportionalities and disparities in the system; constructing a strategy to address disparities (including conducting a racial equity assessment); modifying policies and procedures; and conducting regular evaluations using data analysis and review.
  • When developing their 2022-25 strategic plan, the Michigan courts held a virtual public comment session and solicited written feedback from communities and court users; the final plan outlined “racial and social equity” goals and strategies to improve public trust.

Are adopted reforms working as intended?

How to measure it:

Conduct pre- and post-reform evaluation
  • When implementing reforms or initiatives, develop clear benchmarks for success and methods for collecting data on impact and effectiveness. Train court staff charged with collecting this data about what to gather and why.
  • Create peer learning models with other jurisdictions and states to share what different court systems are doing to reduce and eliminate disparities and how they are measuring impact.
  • Train court staff on cultural competency, bias, equity, and data sensitivity.

Who’s involved:

Internal Experts:

  • Access to justice
  • Leadership
  • Researchers

External Experts

  • Community partners
  • Legal stakeholders
  • Researchers
  • NCSC offers community engagement and organizational assessment resources for courts seeking to identify and address disparities that can be used to measure progress on data governance and collection, community engagement, and other relevant topics.
  • A recent evaluation that used individual-level demographic data found that an Ohio court’s pilot online dispute resolution system delivered positive case outcomes, such as lower default judgment rates and higher dismissal rates, for court users across racial groups and income levels.
Sources: National Center for State Courts, “Blueprint for Racial Justice: Directory of Systemic Change Initiatives” (2022); Maryland Administrative Office of the Courts, “Language Services in the Maryland Courts” (2023); Kentucky Court of Justice Department of Family and Juvenile Services, “A Guide for Identifying, Addressing, and Reducing Racial, Ethnic, and Equity Disparities” (2022); Michigan Judicial Council, “Planning for the Future of the Michigan Judicial System” (2022); B.M. McCormack and Members of the Michigan Judicial Council, “Court Leaders Want the Public to Weigh In on Strategic Agenda” (March 4, 2022); National Center for State Courts, “Racial Justice Community Engagement Resources Center”; National Center for State Courts, “The Racial Justice Organizational Assessment Tool for Courts” (2023); National Center for State Courts, “Data-Driven Decision Making for Courts” (2023)

The work in action: Kentucky creates resources to help courts address disparities

In 2021, the Kentucky Court of Justice published “A Guide for Identifying, Addressing, and Reducing Racial, Ethnic, and Equity Disparities,”2 which courts throughout the state have used to reduce disparities and which can serve as a model for other courts.

The guide was created by Rachel Bingham, director of the Kentucky Court of Justice Office of Statewide Programs; Pastor Edward Palmer, advocate and past national chair of the Coalition for Juvenile Justice; and other court staff after an evaluation of the state’s landmark 2014 youth justice reforms found disparities in outcomes.3 “It was perfectly obvious that the 2014 reforms worked well for White kids, not as well for kids of color,” Bingham says. “That’s where the journey of the guide took over.”

Providing both a template for other courts interested in tackling disparities and a chronicle of Kentucky’s efforts, the guide outlines a four-step model to identify disparities, construct strategies to address them, implement reforms, and evaluate progress. Data and analysis are central to recognizing disparities. Creating solutions involves training staff, developing action plans, analyzing existing policies and procedures, and conducting a racial equity assessment. Making change involves putting new policies and practices in place and obtaining buy-in for those changes from court leaders and staff. And the final step includes measuring progress and recalibrating goals and actions as needed.

A 2022 revision to the guide recounts some of the success stories that have emerged from Kentucky’s disparities work: Several county courts are collecting and analyzing relevant data, training staff, and identifying and working to resolve racial and ethnic disparities through partnerships with law enforcement and county attorney offices. Of the four-step process’s achievements, however, Bingham says that the culture change of acknowledging and discussing racial and ethnic disparities is perhaps the most striking. “That piece is not easy to measure,” she says. “But people know it, and it has a huge amount of relevance to how the courts are engaging with their users and communities.”

Endnotes

  1. The Pew Charitable Trusts, “How to Make Civil Courts More Open, Effective, and Equitable” (2023), https://www.pewtrusts.org/en/research-and-analysis/reports/2023/09/how-to-make-civil-courts-more-open-effective-and-equitable
  2. Kentucky Court of Justice Department of Family and Juvenile Services, “A Guide for Identifying, Addressing, and Reducing Racial, Ethnic, and Equity Disparities” (revised 2022), https://kycourts.gov/Court-Programs/Family-and-Juvenile-Services/Documents/reedguide.pdf.
  3. U.S. Department of Justice, “Research Central: Assessing the Impact of Juvenile Justice Reform in Kentucky,” Office of Juvenile Justice and Delinquency Prevention, https://ojjdp.ojp.gov/newsletter/ojjdp-news-glance-mayjune-2021/research-central-assessing-impactjuvenile-justice-reform-kentucky.