As 2005 began, officials in New Mexico—long one of the nation’s most impoverished states—found that many children were starting school at age 5 with the developmental skills of a 2- or 3-year-old.
Policymakers responded by creating a state-funded voluntary prekindergarten program, featuring kids copying letters on a white board, learning how to use their fingers to hold chalk, playing the role of a restaurant server, or listening to their teacher read Go Away, Big Green Monster! Through the years, the initiative has tapped multiple national and state sources—including The Pew Charitable Trusts’ Results First initiative—to design, implement, and evaluate the plan to ensure that it would improve student performance.
Today, state officials say New Mexico’s pre-K effort is succeeding. Data shows that 80% of the initial group of 4-year-olds in the program’s first year graduated from high school, exceeding their peers who were not in pre-K—only 74% of whom graduated. Graduation rates for English-as-a-second-language learners and children from low-income families were even higher.
New Mexico’s achievement is an example of evidence-based policymaking, which is using research and data to influence policy and budget decisions. The practice is increasingly gaining ground among policymakers throughout the nation: Over two-thirds of the states, as well as more than 10 counties, have used this practice in the past decade to help decide where to spend taxpayer dollars on programs, practices, and policies that are proved to work. The federal government also has embraced research-centered approaches; President Joe Biden has proclaimed 2022 a “Year of Evidence” to strengthen and expand evidence-based policymaking throughout the federal bureaucracy.
At The Pew Charitable Trusts, evidence-based approaches to improving public policy have become standard on a range of subjects from recommendations for state retirement systems to criminal justice research. Relying on facts and evidence, Pew’s researchers have found, builds nonpartisan common ground for governors, legislators, and other leaders to develop and nurture programs that fix ills and spend taxpayers’ dollars more wisely.
“Evidence” in public policymaking is not the courtroom type. Instead, it refers to credible, nonpartisan research on programs and policies that are demonstrated to be effective. Decision-makers use this evidence to add, subtract, or alter programs—usually as part of the government budget process. As a result of the data on the value of New Mexico’s pre-K programs, for instance, the state’s lawmakers have boosted spending on early childhood education programs from about $150 million a decade ago to about $500 million this year.
The research supporting evidence-based policymaking is neither anecdotal nor aspirational; knowledgeable specialists scrutinize the data through multiple rigorous research methods, such as randomized controlled trials. New Mexico officials, for example, focused their pre-K initiative on specific outcomes, in this case higher scores preparing 4-year-olds for kindergarten and tracking those scores through high school.
“In some form or fashion, state government is going to touch the lives of individual New Mexicans,” says Charles Sallee, deputy director of New Mexico’s Legislative Finance Committee. “We have to make sure government is functioning at a level to get better results for them.”
In an era of intense political polarization in state capitols, evidence-based policymaking stands out because objective facts guide elected officials in both parties to find solutions to pressing problems. “It is an area of rare agreement between the parties to do something constructive,” says John Kamensky, emeritus senior fellow at the IBM Center for The Business of Government.
In Louisiana and Texas, Republicans and Democrats agreed in recent years to reform their respective criminal justice systems. The transformations happened because lawmakers were convinced that the states could institute evidence-based correctional practices that would save millions of dollars, reduce the need to build prisons, and care for nonviolent offenders with community-based drug and mental health treatment programs that would keep them from returning behind bars.
The bipartisan reform in Louisiana, aided by Pew’s technical assistance, was “proof of the good that can happen when we come together and put people over politics,” said Louisiana Governor John Bel Edwards (D). In Texas, Rep. Jerry Madden, a Republican from Plano, said: “The legislature pulled together and worked across party lines to implement a ‘justice reinvestment strategy’ based on a data-driven re-examination of each part of the corrections system and a careful cost-benefit analysis of corrections expenditures.”
The bipartisan underpinnings of evidence-based policymaking were also a factor in Tennessee’s creating the Office of Evidence and Impact, which developed the fiscal 2022-23 state budget with data showing the effectiveness of programs. “It’s about arming our decision-makers with data and research that’s not political, that’s not Republican or Democrat,” Christin Lotz, director of the 3-year-old office, told a recent conference of local government leaders in Washington. “You know, ‘Here are the facts. We can demonstrate this program is working or is not working.’”
How It Works
The framework for carrying out evidence-based policymaking varies among states; few engage in all components. But the basic elements are the same to bring evidence into programmatic decision-making processes.
Assessment. Once an issue is identified, officials look for programs or practices successful in other places—often using a national clearinghouse that has compiled a database of programs. Officials then assess the evidence of the programs’ effectiveness, put together an inventory of them, and identify their potential return on investment. Massachusetts turned to Pew’s Results First Clearinghouse Database to find programs that would cut prison and court-related spending and bolster prevention, early intervention, and drug court programming. Minnesota has created its own interactive, online evidence-based inventory of social policy programs and recently added a novel demographics tool disclosing the race, ethnicity, socioeconomic status, and education of the people using the services.
Budget. As the state spending plan is developed, officials provide policymakers with information about how evidence-based programs are performing, often giving funding priority to services proved to achieve substantial benefits relative to their cost. New Mexico, for example, requires state agencies to indicate how much of the money in their budget requests will go to evidence-based programs. “Hopefully, by getting them on board requesting evidence-based programs over time, they are going to be much more likely to be successful through the funding process if they prioritize those requests,” says Sallee of the Legislative Finance Committee.
Implementation and oversight. Once a program is approved, officials monitor its operation to ensure that the goals can be achieved as designed. From that point, officials generally measure the impact of programs and report those results, such as with New Mexico schoolchildren’s reading and math skills, to lawmakers and the public.
Targeted evaluation. Officials assess the evidence using rigorous evaluation methods: Has the policy or program worked? This approach allows officials to tweak programs to improve results, adjust costs, or determine their usefulness. Colorado lawmakers created an evidence continuum, a five-step criteria that defines categories of evidence in budget planning. By using the same, consistent definitions across all departments, officials make it clear when a program backed by research has shown impact. “If states are truly the laboratories of democracy, the use of data and evidence helps us be more effective technicians in those laboratories,” said Colorado Governor Jared Polis (D).
This policymaking framework bears Pew’s imprint; the organization has been a leader in developing tools and resources for state-focused decision-makers to implement initiatives shown to succeed throughout the country. Its Results First initiative, launched in 2011 in collaboration with the John D. and Catherine T. MacArthur Foundation, pioneered the cost-benefit analysis approach, among other components.
Pew now is handing off some of Results First’s key resources to outside organizations, but the emphasis on evidence-based policymaking continues to infuse virtually every issue Pew takes on. “Rigorous analysis is instinctive to us,” says Kil Huh, who heads Pew’s government performance division, a portfolio that includes state fiscal and economic policy, consumer finance, safety and justice, and public health. “We use data and research to provide facts that inform and encourage change.”
Pew developed an early reputation as a reliable information broker in the public pension field through a thorough examination of data on all state-administered retirement plans from states’ own publicly reported financial information. Pew researchers revealed for the first time a $1 trillion gap between what states had set aside to pay for employees’ retirement benefits and the cost of those payments, a startling finding that prodded policymakers throughout the country into action. “That fact drove more than a decade of reforms by states to shore up their retirement systems, efforts that continue to this day,” Huh says.
Of course, Pew is not alone in advancing data and evidence in state and local policymaking. The Bloomberg Philanthropies supports Results for America and What Works Cities, initiatives that have brought together a range of public and private government, business, research, and philanthropic officials to nurture state and local practices and policies grounded in data. And the National Conference of State Legislatures’ Center for Results-Driven Governing and Pennsylvania State University’s Evidence-to-Impact Collaborative are taking on some of Results First’s resources.
The Federal Push for Evidence
The federal government’s push to increase the use of evidence builds on state and local efforts—especially following the Great Recession of 2007-09, when deep budget cuts triggered demand from officials for smarter spending decisions based on reliable information about programs that delivered successful outcomes.
A turning point occurred in 2016 when President Barack Obama and Congress created the bipartisan Commission on Evidence-Based Policymaking, whose recommendations on expanding the demand for and use of data-driven research led lawmakers to enact the Foundations for Evidence-Based Policymaking Act in 2018. “The driving purpose of this legislation is very simple: We are requiring federal agencies to prioritize evidence when they are measuring a program’s success,” said then-House of Representatives Speaker Paul Ryan (R) of Wisconsin, a bill sponsor. President Donald Trump signed the legislation.
The “Year of Evidence” in 2022 attempts to accelerate the federal government’s data-driven decision-making capability as explained in the act, especially its recommendation that lawmakers put in place a scientific evaluation process “to learn what’s working, what’s not, and how we can improve.”
To back up its emphasis on evidence practices, the Biden administration offered states and localities financial incentives in the $350 billion American Rescue Plan Act if officials could include in their project applications how they would use evidence and outcomes. Similarly, the very first instruction in Biden’s executive order implementing the $1 trillion Bipartisan Infrastructure Law was, “Agencies are charged to make evidence-based decisions … [and] use data to measure and evaluate progress.”
State and local governments responded with hundreds of proposals. Michigan asked to spend some of its federal dollars on evidence-based programs that would boost the state’s lower-than-average community college graduation rates. Tulsa, Oklahoma, said it could expand an evidence-based program that provides jobs to people with serious mental illnesses.
In a sign of how the approach is becoming more ingrained at the federal level, President Biden’s proposed fiscal 2023 budget pledged “to promote a culture of evidence” throughout the government with evidence-based initiatives to address low-income housing, homelessness, job creation, and education staffing shortages.
The COVID-19 pandemic also prompted an urgent need for evidence-based policymaking. The fast pace of the virus and its variants meant that information was often changing and unpredictable. But the first major pandemic in 100 years stirred considerable interest in how evidence plays a major role in managing a lethal crisis.
Because of the pandemic, Americans learned a new data-shaped language: positivity rates, social distancing, hot spots, quarantine, and more. Evidence influenced decisions about wearing masks, who could get a hospital bed and ventilator, and when schools and businesses and workplaces could open. Every state created a digital dashboard of COVID data; Indiana’s listed 24 COVID-related data sets.
Data directed who got the first vaccinations and where rates were lowest, partially reflecting the heightened emphasis that officials gave to racial and ethnic equity following the May 2020 murder of George Floyd in Minneapolis. Minnesota, Utah, and North Carolina, among others, used race and ethnicity data to ensure that the vaccine would be distributed to areas disproportionately affected by the pandemic.
But the experience with data during the pandemic also demonstrated some of its drawbacks. State dashboards differed widely, for example, because there was no national standard for what data should be reported.
Apart from the pandemic, evidence-based policymaking has other challenges. Evaluating programs often is frustrating for state officials because data and evidence do not exist, are too old to be useful, or confound government officials who lack the expertise to analyze the information. The data itself can be flawed, supported by invalid methodologies, or collected randomly. California’s state auditor periodically releases reports that reveal numerous problems with the accuracy and completeness of state data overall.
And though practitioners recommend reviewing results of successful practices in other states, what has proved to work in one state is no guarantee it will succeed in another. “Fifty states function in 50 different ways,” says Katherine Barrett, co-author of Making Government Work: The Promises and Pitfalls of Performance-Informed Management. Adds Barrett’s co-author Richard Greene: “Not every state relies on evidence to the same degree.”
As evidence-based policymaking continues to evolve and gain acceptance, Huh, of Pew, says states should consider backing up their commitment to the practice by integrating evidence into the everyday business of government. “Evidence isn’t a one-time effort,” Huh says. “It starts with a commitment to include nonpartisan data and research when making program and policy decisions.”
Stephen C. Fehr covered state and local governments for the Kansas City Star and The Washington Post before spending more than a decade on Pew’s state fiscal team.
