How to Standardize Court Data for Greater Transparency and Ongoing Improvement

Steps for making civil courts more open

Fact Sheet September 27, 2023 Read time:

Projects: Courts & Communities

Overview

Standardized data helps courts better serve users. It allows court leaders and staff to effectively compare performance and outcomes, understand how cases play out, and quickly answer questions from other courts, external stakeholders, and other branches of government. Standardization also gives courts confidence in the data that they collect and report to leadership and other government institutions and supports them in making informed funding requests that direct resources where they are needed to improve performance.

Courts seeking to improve their data standardization can begin by implementing two key practices:

Report information from the local level to the state in accordance with statewide standards.
Collect data at the local level that aligns with statewide data standards.

Pew recommends starting with reporting, rather than collecting, standardized data because courts already collect data, so the easier and less costly approach is to begin by standardizing the definitions, fields, and documentation for existing data and requiring local jurisdictions to use those standards for reporting. Once standards are in place and reporting is in compliance, courts can turn to updating and aligning collection practices.

After extensive research, The Pew Charitable Trusts has developed a framework outlining how and why courts should modernize.¹ These steps arise from that work and can help programmatic and operational court staff, along with court leadership, assess their current data processes; identify opportunities to improve data standardization; and decide—with input from relevant stakeholders—which of those opportunities to pursue and how.

Step 1: Bring together relevant court staff and external stakeholders

These groups can contribute important perspectives and insights about data standardization.

Research staff can lead conversations about and analyses of whether and how data is stored and standardized and monitor for compliance with the data dictionary, a master document explaining what different data fields mean.

Leadership/administrative officers can support and champion the creation or strengthening of uniform statewide data standards, identify priority fields for updates, and secure commitments from court staff to implement and monitor compliance with data standards.

Clerks can serve as data stewards, enter data according to standards, and provide feedback on priority fields.

Judges can be champions for data standards among court staff they work closely with and provide feedback about which data fields would enhance their understanding of how cases are adjudicated, both within their own jurisdictions and statewide.

IT staff can support local jurisdictions in reporting data fields to the state-level office and partner with data vendors to update case management systems and fields.

External researchers can help courts prioritize data fields for standardization and ensure that data definitions are clear.

Case management system vendors can update the fields that the court is capturing to support standardization at the data entry stage.

Step 2: Assess current practices and set next steps

The following set of key metrics can enable courts to assess their progress toward collecting and reporting court data consistently across jurisdictions, undertake necessary reforms, and conduct cross-jurisdictional comparisons. (See Tables 1 and 2.)

For each metric, determine whether the answer to the initial question is yes or no using the suggested measure. If the answer to the metric question is no, pursue the suggested next steps in collaboration with staff and stakeholders. The suggested steps are not prescriptive; instead, they provide ideas and options for getting started. The state examples can help courts determine what actions are feasible, given available resources.

Table 1

Data Reported From Local Courts Should Adhere to Statewide Standards

Metrics, suggested steps, and state examples

Metric	If not, suggested next steps	State examples
Does the state have statewide data standards? How to measure it: Conduct an audit to determine whether the state has one set of data fields that local jurisdictions collect and that those fields are defined consistently.	Identify priority data fields to standardize, such as those outlined in the National Open Court Data Standards (NODS), and determine how often to capture data points throughout the life cycle of a case. External experts can assist with identifying these fields. Seek input from and agreement across local jurisdictions about priority data fields for collection and reporting. Who’s involved: Internal Experts Researchers Leadership Clerks IT staff Judges External Experts Researchers	Arizona courts issued an administrative order directing local courts to develop a joint committee to identify priority data elements and develop data governance. The court has an online hub for checklists and required data elements. Example required data elements include those related to the case (e.g., current case status), participants (e.g., name, race, gender, ethnicity), attorney/advocate information, pleadings, motions and filings, and more. After analyzing landlord-tenant data, Michigan’s Justice for All Commission’s Technology and Data Sharing Committee recommended that the state’s Judicial Data Warehouse capture additional data fields for rental cases, including limited versus full representation, whether and when an answer was filed, the amount provided in judgment, and unique plaintiff identifiers.
Does the data that local jurisdictions report to the statewide dashboard, warehouse, or other aggregation tool comply with state-established definitions? How to measure it: Evaluate data quality to identify outliers in the numbers of cases, unrepresented litigants, and other quantitative variables.	Set targets for reporting compliance, such as the number of jurisdictions to report standardized data to the state each quarter. Require that local jurisdictions regularly report data to the administrative office of the court. Who’s involved: Internal Experts Researchers Leadership Clerks IT staff External Experts Researchers	A 2021 Michigan administrative order requires that all trial courts submit case data in a specified format and frequency to ensure that the “uniformly reported data … will be more useful to courts, law enforcement, researchers, and other users.” Georgia’s Judicial Council/Administrative Office of the Courts is working to get buy-in from court clerks to ensure that the data reported to the state from local jurisdictions is accurate and consistent.
Can state court staff reliably compare how cases progress across jurisdictions? How to measure it: Assess local reporting practices for frequency and data completeness.	Develop dashboards or other data aggregation tools to enable cross-jurisdictional comparisons. External experts can assist with dashboard development or provide feedback on products. Who’s involved: Internal Experts Clerks Researchers Leadership IT staff Judges External Experts Researchers	One reason Georgia is adopting NODS is to better allocate resources across jurisdictions in response to needs, caseloads, and other metrics. Indiana developed an internal dashboard to compare the percentages of self-represented court users across urban and rural jurisdictions and is working with local stakeholders to deploy jurisdiction-specific resources or outreach strategies based on the findings.

Metric

If not, suggested next steps

State examples

Does the state have statewide data standards?

How to measure it:

Conduct an audit to determine whether the state has one set of data fields that local jurisdictions collect and that those fields are defined consistently.

Identify priority data fields to standardize, such as those outlined in the National Open Court Data Standards (NODS), and determine how often to capture data points throughout the life cycle of a case. External experts can assist with identifying these fields.
Seek input from and agreement across local jurisdictions about priority data fields for collection and reporting.

Who’s involved:

Internal Experts

Researchers
Leadership
Clerks
IT staff
Judges

External Experts

Researchers

Arizona courts issued an administrative order directing local courts to develop a joint committee to identify priority data elements and develop data governance. The court has an online hub for checklists and required data elements. Example required data elements include those related to the case (e.g., current case status), participants (e.g., name, race, gender, ethnicity), attorney/advocate information, pleadings, motions and filings, and more.
After analyzing landlord-tenant data, Michigan’s Justice for All Commission’s Technology and Data Sharing Committee recommended that the state’s Judicial Data Warehouse capture additional data fields for rental cases, including limited versus full representation, whether and when an answer was filed, the amount provided in judgment, and unique plaintiff identifiers.

Does the data that local jurisdictions report to the statewide dashboard, warehouse, or other aggregation tool comply with state-established definitions?

How to measure it:

Evaluate data quality to identify outliers in the numbers of cases, unrepresented litigants, and other quantitative variables.

Set targets for reporting compliance, such as the number of jurisdictions to report standardized data to the state each quarter.
Require that local jurisdictions regularly report data to the administrative office of the court.

Who’s involved:

Internal Experts

Researchers
Leadership
Clerks
IT staff

External Experts

Researchers

A 2021 Michigan administrative order requires that all trial courts submit case data in a specified format and frequency to ensure that the “uniformly reported data … will be more useful to courts, law enforcement, researchers, and other users.”

Georgia’s Judicial Council/Administrative Office of the Courts is working to get buy-in from court clerks to ensure that the data reported to the state from local jurisdictions is accurate and consistent.

Can state court staff reliably compare how cases progress across jurisdictions?

How to measure it:

Assess local reporting practices for frequency and data completeness.

Develop dashboards or other data aggregation tools to enable cross-jurisdictional comparisons. External experts can assist with dashboard development or provide feedback on products.

Who’s involved:

Internal Experts

Clerks
Researchers
Leadership
IT staff
Judges

External Experts

Researchers

One reason Georgia is adopting NODS is to better allocate resources across jurisdictions in response to needs, caseloads, and other metrics.
Indiana developed an internal dashboard to compare the percentages of self-represented court users across urban and rural jurisdictions and is working with local stakeholders to deploy jurisdiction-specific resources or outreach strategies based on the findings.

Sources: Conference of State Court Administrators and the National Center for State Courts, “National Open Court Data Standards (NODS)”; Supreme Court of the State of Arizona, “Establishment of the Steering Committee on Data-Based Court Performance and Data Standards” (2020); Michigan Justice for All Commission Technology and Data Sharing Committee, “Re: Input on Data Types and Elements Which Should Be Captured by Courts and Submitted to the JDW” (April 18, 2022); Arizona Judicial Branch, “Steering Committee on Data-Based Court Performance and Data Standards”; Arizona Code of Judicial Administration, “Required Data Elements” (2022); Michigan Supreme Court, “Mandatory Submission of Case Data to the Judicial Data Warehouse” (2021); T.J. BeMent and J. Thorpe (District Court Administrator, 10th Judicial District of Georgia; Judicial Services Data Manager, Georgia Judicial Council/Administrative Office of the Courts), video conference, Jan. 26, 2023; R. Rath, J. O’Malley, and J. Weise (Chief Innovation Officer, Indiana Office of Judicial Administration; Director of E-Filing and Innovation, Indiana Office of Court Technology; Deputy Director, Legal Support Division, Indiana Office of Court Services), video conference, Jan. 30, 2023

Table 2

Local Courts Should Collect Data in Accordance With Statewide Standards

Metrics, suggested steps, and state examples

Metric	If not, suggested next steps	State examples
Is data in the case management system in compliance with state standards? How to measure it: Conduct a data quality assessment or a detailed review of case management fields across jurisdictions.	Determine whether local courts in the state are using different case management systems and whether they need to align their data fields. Work with and incentivize case management system vendors to update data systems technology; include adoption of and compliance with standards in future contracts. Update fields in the case management system and implement drop-down menus to eliminate unnecessary hand keying. Train court personnel on data entry and the value of robust data. Who’s involved: Internal Experts Researchers Leadership Clerks Judges External Experts Software vendors	The Administrative Office of Pennsylvania Courts, with support from the National Center for State Courts (NCSC) and funding from the State Justice Institute, is assessing its business processes around collecting and reporting civil and family data to support the state’s data capacity and create a template for adopting NODS at the data-collection level. Arizona’s Administrative Office of the Courts requires that local jurisdictions submit checklists verifying that they are collecting the required data elements.
Has court leadership mandated that research staff and local courts implement and comply with data standardization? How to measure it: Review data standardization compliance policies and procedures.	Identify the person or persons responsible for overseeing data quality. Assign and empower a high-level staff person to direct data compliance adoption, implementation, and oversight, including routine reviews of the data that local jurisdictions are collecting, in partnership with court clerks. Discuss challenges and concerns related to data quality with the staff members responsible for data entry at the jurisdiction level. Adopt automated tools to flag incomplete data in the case management system. Who’s involved: Internal Experts Researchers Leadership Clerks External Experts Software vendors	Florida’s Commission on Trial Court Performance and Accountability brought together judges and clerks to provide feedback on the quality of the state’s court data and to develop recommendations for improvement, such as designing quality control features to quickly flag errors at the source. To support local jurisdictions in adopting data standards, Arizona courts developed a set of tools, including handouts on required data elements, sample checklists, and definitions for data standardization, to support high-quality data entry and oversight.

Metric

If not, suggested next steps

State examples

Is data in the case management system in compliance with state standards?

How to measure it:

Conduct a data quality assessment or a detailed review of case management fields across jurisdictions.

Determine whether local courts in the state are using different case management systems and whether they need to align their data fields.
Work with and incentivize case management system vendors to update data systems technology; include adoption of and compliance with standards in future contracts.
Update fields in the case management system and implement drop-down menus to eliminate unnecessary hand keying.
Train court personnel on data entry and the value of robust data.

Who’s involved:

Internal Experts

Researchers
Leadership
Clerks
Judges

External Experts

Software vendors

The Administrative Office of Pennsylvania Courts, with support from the National Center for State Courts (NCSC) and funding from the State Justice Institute, is assessing its business processes around collecting and reporting civil and family data to support the state’s data capacity and create a template for adopting NODS at the data-collection level.
Arizona’s Administrative Office of the Courts requires that local jurisdictions submit checklists verifying that they are collecting the required data elements.

Has court leadership mandated that research staff and local courts implement and comply with data standardization?

How to measure it:

Review data standardization compliance policies and procedures.

Identify the person or persons responsible for overseeing data quality.
Assign and empower a high-level staff person to direct data compliance adoption, implementation, and oversight, including routine reviews of the data that local jurisdictions are collecting, in partnership with court clerks.
Discuss challenges and concerns related to data quality with the staff members responsible for data entry at the jurisdiction level.
Adopt automated tools to flag incomplete data in the case management system.

Who’s involved:

Internal Experts

Researchers
Leadership
Clerks

External Experts

Software vendors

Florida’s Commission on Trial Court Performance and Accountability brought together judges and clerks to provide feedback on the quality of the state’s court data and to develop recommendations for improvement, such as designing quality control features to quickly flag errors at the source.
To support local jurisdictions in adopting data standards, Arizona courts developed a set of tools, including handouts on required data elements, sample checklists, and definitions for data standardization, to support high-quality data entry and oversight.

Sources: State Justice Institute, “Technology”; Arizona Judicial Branch, “Data Elements and Code Standardization Hub”; Florida Commission on Trial Court Performance and Accountability’s Data Quality Workgroup, “Data Quality in Florida’s Trial Courts” (2020)

These national training and technical assistance providers and resources can help guide courts’ planning and implementing processes.

National training and technical assistance providers and resources

The NCSC works with states on adoption of NODS and provides several resources, including:²
- Toolkits related to applying the standards to specific case types, such as guardianship and conservatorship.³
- A guide for court leaders that explains why courts should standardize their data.⁴
- A user guide that outlines the data elements in more detail.⁵
- A spreadsheet of the data elements.⁶
- Technical notes that explain the data models used in NODS and other related files.⁷
New America is developing standards for eviction data collection. Courts can use these guidelines to assess what case-level data they are collecting and then to adopt new fields or update existing ones to capture more granular information.⁸

The work in action: Georgia pursues statewide data standardization

In 2022, staff from the Judicial Council of Georgia/Administrative Office of the Courts (JC/AOC) began putting systems in place to support efforts by local courts to report their data to the state court’s administrative office so that it could be mapped based on NODS. The goal was to enable the state to gather and report more robust information.

Currently, the state’s reported data is so limited that the JC/AOC cannot make basic determinations about court performance outcomes, such as the number of murder trials that were held or how many foreclosure cases were filed in the past year. By getting local jurisdictions to report more detailed, uniform data that conforms to NODS, Georgia courts will be better able to respond to inquiries from the Legislature, deploy resources to individual jurisdictions, and know what is happening in courts throughout the state.

Georgia’s structure, with a unified judiciary but decentralized courts, presents significant challenges for data standardization by limiting the administration’s ability to require compliance. For example, general jurisdiction court clerks are elected, and many see themselves as constituent-appointed custodians of their courts’ data.⁹ As a result, the state is focusing on building buy-in among court personnel. To do this, JC/AOC took NODS on the road, sharing the data elements with clerks, judicial committees, and state justice partners, and administering a survey to get staff feedback on priority data fields and definitions.¹⁰ JC/AOC then compiled those fields into a master spreadsheet, which it will use during roundtable discussions with clerks about the importance of standards for understanding the state’s data.

Georgia’s work is funded by a three-year grant from the U.S. Department of Justice’s Bureau of Justice Assistance as part of its Justice Counts effort. The JC/AOC is working with several jurisdictions to submit their data through an online bridge to the state’s repository, which will then automatically map the information to the agreedupon data definitions. This automated system will save clerks from having to submit information manually and will enable the state to collect local-level data more regularly, improving response times for requests from the Legislature, courts, and communities. After getting feedback from the first set of jurisdictions about their experiences, the JC/AOC plans to update the processes and expand the standardization effort statewide over the next three years.

“We want to be able to get below the level of aggregate data,” says T.J. BeMent, district court administrator. “Knowing how many cases involved unrepresented litigants at any time in the life of a case helps us make informed decisions on resource management and requests from our Legislature to improve access to the justice system. I’m excited for a lot of different things that this data can open up to us.”

Endnotes

The Pew Charitable Trusts, “How to Make Civil Courts More Open, Effective, and Equitable” (2023), https://www.pewtrusts.org/research-and-analysis/reports/2023/09/how-to-make-civil-courts-more-open-effective-and-equitable.
Conference of State Court Administrators and National Center for State Courts, “National Open Court Data Standards (NODS),” accessed May 2, 2023, https://www.ncsc.org/consulting-and-research/areas-of-expertise/data/national-open-court-data-standards-nods.
State Justice Institute and National Center for State Courts, “Guardianship/Conservatorship Monitoring: Recommended Data Elements” (2020), https://ncsc.contentdm.oclc.org/digital/collection/famct/id/1668.
National Center for State Courts, “National Open Court Data Standards (NODS): Leadership Guide” (2019), https://www.ncsc.org/__data/assets/pdf_file/0015/34026/NODS-Leadership-Guide.pdf.
National Center for State Courts, “National Open Court Data Standards (NODS): User Guide” (2020), https://www.ncsc.org/__data/assets/pdf_file/0014/34025/NODS-User-Guide.pdf.
Conference of State Court Administrators and National Center for State Courts, “National Open Court Data Standards (NODS).”
National Center for State Courts, “National Open Court Data Standards (NODS): Technical Notes” (2020), https://www.ncsc.org/__data/assets/pdf_file/0012/34023/NODS-Technical-Notes.pdf.
New America, “What Can Court Data Actually Tell Us About Evictions?” (2023), https://www.newamerica.org/future-land-housing/reports/us-eviction-court-data/introduction.
Superior Court Clerks of Georgia, “Welcome to the Official Website of the Superior Court Clerks of Georgia,” accessed May 2, 2023, https://www.gaclerks.org/.
National Center for State Courts, Data Elements Spreadsheet, https://www.ncsc.org/__data/assets/excel_doc/0013/34024/NODS-data-elements.xlsx.