State governments strive to make decisions based on the best available evidence, so that they can spend taxpayer dollars in ways that achieve the best outcomes possible. Data analytics—collecting and analyzing information in order to draw insights from it—has the potential to transform government for the better, allowing leaders to make more informed decisions than ever before. But as states work toward achieving this vision, they face a variety of obstacles, such as a shortage of resources and outdated technology. To understand the issues involved, researchers from The Pew Charitable Trusts interviewed 355 state officials to learn how policymakers incorporate data into decision-making. Their findings appear in Pew’s new report, “How States Use Data to Inform Decisions,” which details the four challenges most frequently cited: staffing and the accessibility, quality, and sharing of data.
The biggest challenge states face is a shortage of personnel capable of managing data and mining it for insights. Using data effectively to inform decision-making requires not only technical skills, but also an understanding of public policy, and the ability to communicate effectively to a wide range of stakeholders. Employees with these skill sets can be difficult to find, especially as the public sector workforce grows older, and experienced workers retire. Furthermore, governments must compete with the private sector for these talented data analysts. As one state information technology leader explained, these skills are in high demand, and new employees often leave soon after training for more lucrative opportunities in the private sector.
Thus, one of the biggest obstacles the public sector faces to using data effectively is attracting and retaining employees with the skills needed to capitalize on the information they have at their disposal.
Outdated technology can make it difficult for decision-makers to access the data they need. Older database systems may not contain all of the needed information or may make it difficult to extract it in a usable format. Even when technology is up to date, governments sometimes have trouble accessing their data because they must work through private sector vendors. States often rely on these contractors to develop their data systems, which means they may not have ownership of their own data, making it costly and cumbersome to access and use.
For example, one state that tried to gather data from county governments found that many still recorded their data on paper or in PDF files. All of that data had to be transferred into an electronic format in order to be used, a labor-intensive process.
To be of benefit, data needs to be of high quality, meaning it must be accurate and consistent. When data points are missing, inaccurate, or poorly defined, the information is less useful. For instance, when a form asks for a client’s occupation, agency employees may describe the same job in different ways, which could make it difficult to understand what people do for a living.
One state ran into such a problem when it tried to analyze its vendor contracts to ensure compliance. It found that its data on the contracts were incomplete and disorganized, which made it impossible to know whether resources were being utilized as intended.
Data accessible to one agency might help support another agency’s priorities, or two agencies may need to share data to work toward a common goal. For example, a state’s education department might want to share data with a public university system to better understand how high school graduates fare in college. Or a state’s Medicaid program could benefit from data sharing with a housing authority to more easily identify patients experiencing homelessness. As useful as data sharing can be, it often faces obstacles, as many agencies are concerned about security and uncertain about how to comply with privacy laws.
A lack of data sharing caused a problem in one state’s driver education program, which was funded by the state department of education while traffic safety data were controlled by the department of motor vehicles. When the state wanted to evaluate whether the program helped students become safer drivers, it had to go through a long and costly process of persuading both agencies to share their data, then matching accident records to student driver data. This made the evaluation more costly and difficult to complete.
These four challenges are major hurdles for states aiming using data effectively, but they can be overcome. Pew’s report explores the many ways states are using data to operate more effectively and solve pressing policy problems. It also describes how states have overcome challenges by planning ahead, building capacity, ensuring data quality and access, investing in data analysis, and building sustained support for data use. This report can serve as a helpful model for states that hope to use data to improve policymaking.
Kil Huh directs The Pew Charitable Trusts’ project on states’ fiscal health, and Dan Kitson works on Pew’s data as a strategic asset project.