Peter Howell/Getty Images
Every day, state governments make decisions that affect the lives of their citizens. Legislators and governors determine which policies to enact and what public problems to address. State agencies establish how programs should be run and where budget dollars are best spent, as well as who qualifies for government assistance.
To effectively serve the public, state officials at every level of government are tasked with ensuring that these daily decisions are prudent and well-informed. Consequently, states are increasingly turning to administrative data, or information—such as vital records, college enrollment data, and Medicaid utilization statistics, collected and maintained primarily for the routine management of programs and services—to make strategic data-informed decisions. This information can include any data that are necessary to implement and oversee a program, such as demographics, outcomes, and enrollment details.
While researchers have explored the use of administrative data in various areas of state government (for example, identifying frequent users of emergency services), little has been published on this trend more broadly. As state leaders seek to harness data in innovative ways, what common themes, noteworthy successes, and notable challenges have the 50 states experienced across a broad cross section of issue areas? To address these questions, The Pew Charitable Trusts interviewed state leaders across the U.S. in 2016 and reviewed relevant laws, documents, and policies in all 50 states. This report is the culmination of that research, and the first comprehensive overview of how data is being utilized in all 50 states.
States traditionally use administrative data to prepare annual reports showing how funds were spent and the impact of a particular program, to demonstrate transparency in describing what a state agency does, and to comply with performance measures set by the federal government, state legislature, governor, or an agency.
More recently, states have begun harnessing existing information through data analytics—procedures that review data to identify meaningful information and correlations. Such efforts open up critical new opportunities for governments to make effective decisions. Analysts can uncover important insights by employing techniques such as integrating and cross-referencing data sets, undertaking calculations to show trends, finding correlations between various factors, running statistical experiments, mapping geographical data to show areas of high activity, and visualizing data in charts and graphs. Additionally, data analytics can reveal the root cause of a persistent issue, diagnose breakdowns in a system, highlight obstacles, and predict future phenomena, allowing state leaders to be better informed in their approach to a problem and make more strategic decisions.
Using data collected from interviews with more than 350 state officials, this study highlights ways in which some government leaders have employed sophisticated data analytics, beyond traditional uses of administrative data, to accomplish the following:
Such innovative uses of administrative data remain relatively rare, and making them more prevalent can require states to clear a number of hurdles. Budget pressures often leave state agencies struggling to maintain funding for research and analysis, and result in salaries that make it difficult to retain staff skilled in data matching and complex analyses. Data quality necessary to support detailed analyses is usually uneven at best. Information sharing—drawing on data from multiple agencies—requires agreements and compliance with privacy protections. Legal thickets such as these are difficult and time-consuming for states to navigate. And above all, a government’s day-to-day struggles to absorb pressure to cut their budgets, respond to the latest crisis in the news, and accommodate requests from lawmakers and the governor’s office leave little bandwidth for the level of complex analyses contemplated here. Through this research, the authors identified five key actions state leaders could take to work through these challenges and maximize the value of administrative data at their disposal:
The authors found states that had implemented a combination, or even all five, of the above actions in different policy areas. But no state has managed to apply these actions to a broad range of government agencies and achieve across-the-board improvements in how it develops policy, delivers services, manages its resources, and evaluates existing programs. The next frontier for state governments will be moving from the narrow, targeted use of data analytics to its comprehensive application across policy areas.