Six years ago, the Iowa Department of Corrections had a problem. It didn’t know what alternatives to incarceration might reduce recidivism—and didn’t have any data about what the real long-term costs and benefits of those alternatives might be.
So officials developed a new cost-benefit analysis system, tailored to their state, to help determine programs’ effectiveness and provide policymakers with the hard data they need to make informed legislative and budget decisions.
Working with the Pew-MacArthur Results First Initiative, state officials began to gather information and establish a methodical system to evaluate several key programs. The good news: Some of the state’s drug treatment programs already were returning $8 in savings for every taxpayer dollar invested. And cognitive therapy—a relatively inexpensive program for Iowa—was returning $35 for every dollar spent.
But there was bad news, too.
Iowa officials had thought that a long-running domestic abuse program was a model of success. Not true: The evaluation showed that the program was actually costing the state $3 for every dollar spent on it—“a waste of taxpayer dollars,” in the words of a Department of Corrections report. Officials promptly scrapped the program and developed a new one.
There’s no question that every program that went under the microscope in Iowa—the effective and the ineffective alike—was developed with the best of intentions. Policymaking is essentially a prospective enterprise, with public officials (especially elected ones) rightly looking ahead with the intention of making things better. For many lawmakers, the greatest challenge is simply getting legislation passed. No small feat, to be sure.
But some of those Iowa programs worked and some didn’t. How to know which? Answering that question requires that policymaking be as retrospective as it is inventive: looking back to evaluate previous decisions and program performance while studying the experience of other policymaking groups to take innovative ideas forward. It requires that research be grounded in empirical evidence that provides useful information for policymakers. And it requires the use of evaluation tools to put the evidence through a cost-benefit prism.
These cost-benefit analyses naturally build on requirements in most states that lawmakers consider the fiscal impacts of proposed legislation. (Policymakers know this as the “fiscal note”—essentially, the price tag for a new piece of legislation.) And they have the potential to yield significant benefits to policymakers and taxpayers.
The benefits come from helping policymakers grapple with difficult budgetary decisions about current programs and providing background as they face new challenges. Political considerations and voter feedback will always be paramount in the policy arena. But with these evaluation tools, we will be able to reinvent how public policy can be developed in the modern digital world.
***
Rigorously conducted independent research is the heart of an evidence-based approach to policymaking. Without it, stakeholders can use opinions and ideology to argue without end. With it, they can develop policy that cuts through the ideological noise. This research must be timely and relevant to the real-world choices that policymakers face.
While we now have powerful tools to compile and evaluate evidence, the idea of using research to inform policy is not new. We need only look to history to see the key role that research has played in inspiring important social change.
In the early 1900s, for example, medical schools in the United States were unregulated and nearly anyone could become a doctor, without rigorous study of medical science or hands-on practical experience. The potential—and real—risks seem obvious in retrospect. At the time, the Carnegie Foundation for the Advancement of Teaching commissioned a study that documented concerns in the U.S. and Canada; recommended higher admission and graduation standards in medical schools; and called on the schools to follow the protocols of mainstream science. The Flexner Report, as it came to be called, led to the adoption of professional standards that we take for granted today.
A more contemporary example comes from corrections policies in the states. For much of the 20th century, the national rates of incarceration stayed fairly constant. But after many states began passing laws that required longer prison terms, the rates starting going up. Beginning in the 1970s, states filled their prisons and spent billions of dollars building new ones to keep up; from 1979 to 2000, Texas alone built 137 new prisons. The national incarceration rate grew five times higher than the historic norm, with 1 out of every 100 Americans in prison.
Yet recidivism rates didn’t drop.
With corrections costs consuming an exponentially growing portion of their budgets, policymakers in a number of states—and across the political spectrum—began to look for new solutions. Researchers at Pew went to states that asked for assistance to help determine what sorts of offenses were driving the biggest numbers of people into prison. They found that in many cases, offenders had committed nonviolent crimes or were being put back behind bars because of parole or probation violations, sometimes serving more time for those infractions than for their original crimes. In South Carolina, for example, initial data analysis showed that 60 percent of inmates in state prisons were nonviolent offenders.
The research was expanded to examine what sort of programs—especially drug treatment efforts—could help keep nonviolent offenders from returning to prison (and save the state’s taxpayers the expense of jailing them). In South Carolina, officials looked back to see how their laws were performing and looked around to learn from the experiences of other jurisdictions. Armed with evidence-based information, legislators passed a series of reforms to the state’s criminal justice system; a decade later, the percentage of nonviolent inmates in state prisons is down to 40.
Pew has worked in 33 states that are concerned about growing prison populations. The results are promising: Today, the national incarceration rate—which includes adults in federal and state prisons and local jails—is down 13 percent from its peak in 2007.
The innovative—and common-sense—evidence-based approach to policymaking is doing more than keeping people out of prison: It’s getting people into college as well. In 2012, Delaware's Department of Education, with the assistance of Harvard University's Strategic Data Project, conducted an extensive analysis of data measuring the performance of the state’s high school students. The conclusion? A large number of students whose SAT scores indicated they were capable of obtaining a college education nonetheless were not enrolled in college. From 2008 to 2011, for example, 18 percent of Delaware students who scored at least 1550 out of 2400 on the SAT did not enroll in college.
With these data in hand, the state began its Getting to Zero campaign, designed to take that 18 percent down to zero by having every college-ready student in Delaware apply for and enroll in postsecondary education. The campaign includes better training for school counselors on how to assist students and their families in completing the Free Application for Federal Student Aid; the designation of October and November as college application months, during which students receive help with their applications; and a texting system that families can use to access real-time information on financial aid and other concerns.
The campaign is proving to be a success. Ninety-eight percent of the state's college-ready applicants from the high school classes of 2014 and 2015 (the first two years of Getting to Zero) enrolled in an institution of higher learning.
On the fiscal front, most states offer tax incentives to encourage businesses to make investments that create or retain jobs. For the most part, policymakers have rarely looked back to see whether these incentives—which cost billions of dollars in forgone tax revenue—produce stronger local economies. But the District of Columbia and 22 states have passed legislation requiring regular, rigorous, and independent evaluations of their tax incentives—a clear example of how a data-based cost-benefit analysis can guide policymaking.
The Results First Initiative, which looked at Iowa’s alternatives to incarceration, is now in place in 21 states and four counties, providing policymakers with accurate assessments of the true costs and benefits of public programs. These analyses calculate the returns on investment in alternative programs, rank programs based on cost and benefit, and help predict the impact of different policy options.
The project grew out of the pioneering efforts of the Washington State Institute for Public Policy (WSIPP), which created a sophisticated cost-benefit model to analyze programs in the Evergreen State. It helped make the state a national leader in incorporating an evidence-based approach into the legislative process and producing bipartisan policies that have saved billions of dollars. An early example came in 1997, when the Washington State Legislature passed the Community Juvenile Justice Act—one of the first in the nation to require that state-funded programs for juveniles be “compatible with research.”
The law directed WSIPP to develop methods for measuring the effectiveness of juvenile justice programs and helped to standardize the definition of what works in reducing recidivism among young offenders.
State leaders have since mandated similar evaluations in human services programs, with the WSIPP analysis showing the cost benefits of these programs and ranking them for policymakers. A recent report, for example, showed that cognitive behavioral therapy for adult depression can produce up to $50 in taxpayer and societal benefits for every dollar invested.
On the other side of the country, Connecticut also has emerged as a leader in evidence-based policymaking with what it calls results-based accountability: a budgeting technique, now incorporated into the appropriations process, that helps policymakers use data on program outcomes to inform their funding decisions. Agencies study performance scorecards to highlight which programs are most effective at achieving desired outcomes and consider the information in spending decisions.
From Iowa to South Carolina, Delaware to Washington state, and Connecticut and other jurisdictions around the country, lawmakers are learning that building data into the policymaking process means they don’t need to rely on anecdotal information. A significant reinvention of the way policymakers make decisions is underway as more state and local governments see the potential of using evidence to expand successful programs and eliminate those that aren’t working. Not only will citizens benefit from these evidence-based programs, but every taxpayer will profit by seeing a better return on public expenditures.
Takeaway
