This fact sheet was updated on May 29, 2019, to correct the return on investment for Colorado’s outpatient/nonintensive drug treatment in prison program.
Overview
The Results First Cost-Benefit Model (RF Model), a key component of the Pew-MacArthur Results First Initiative’s approach to evidence-based policymaking, is an online tool that enables state and county governments to estimate the expected return on investment (ROI) for programs they fund or are considering funding. What makes the model unique is that it uses a consistent approach for analyzing programs within a policy area, such as adult mental health, child welfare, or juvenile justice, allowing policymakers to make an apples-to-apples comparison between different programs’ benefits and costs. Government leaders can then use this information to help inform their programmatic funding decisions and ensure they are making the best use of taxpayer dollars.
How the RF Model works
The model1 estimates the ROI for programs by carrying out the following three steps:
1. Use high-quality research to assess program effectiveness.
The RF Model uses rigorous research that shows whether a program achieved its intended outcomes, such as decreasing instances of child abuse, increasing high school graduation, or reducing crime.2 The model then estimates the monetary benefits associated with such changes, as described below.
2. Estimate jurisdiction-specific benefits.
Users populate the RF Model with jurisdiction-specific data on resource costs (for example, prison costs), resource use (how long people stay in prison), and population rates (the cumulative recidivism rate of the adult prison population). The model then uses these data, along with the research findings, to estimate the benefits the program is expected to produce in the jurisdiction. These benefits can accrue to program participants (such as through higher earnings), taxpayers (with lower prison costs), or society at large (by having a more educated workforce to draw upon).
3. Calculate jurisdiction-specific cost-benefit ratio.
Lastly, the model compares the program’s benefits to the jurisdiction’s direct cost of providing the program to create a cost-benefit ratio. This ratio, also known as the return on investment, indicates the amount of benefits the program is expected to generate for every dollar spent.
Table 1 provides a real-life example of the model’s results, showing that Colorado’s outpatient/nonintensive drug treatment program in prison is expected to produce the highest ROI of the analyzed facility programs. For every $1 spent on outpatient drug treatment in prison, the program is estimated to generate $13.20 in benefits. The analysis also shows that although correctional and vocational education programs are expected to produce more benefits than outpatient drug treatment, their cost-benefit ratios are lower due to their higher costs.
Panel of Experts Affirms the Validity, Utility of the Results First Cost-Benefit Model
On a regular basis, the Pew-MacArthur Results First Initiative convenes a panel of academics and researchers to conduct an external, independent review of the Results First cost-benefit model. This is done to ensure the model’s credibility, confirm that it follows best practices, and verify that it generates reliable estimates.
Results First held the most recent review in 2017 (others took place in 2010, 2012, and 2014). Its panel comprised the following experts: Daniel Max Crowley, assistant professor of human development and family studies at Pennsylvania State University; Lynn Karoly, senior economist at Rand Corp.; David Weimer, Edwin E. Witte Professor of Political Economy at the Robert M. La Follette School of Public Affairs at the University of Wisconsin-Madison; and Frederick J. Zimmerman, professor at the Fielding School of Public Health at the University of California, Los Angeles. Weimer said that the “model continues to show strong conceptual grounding and sophistication as its application widens.” Zimmerman said that the “model represents the highest-quality policy analysis model that is possible. It has been executed with exceptional thoughtfulness and care.”
This evaluation provides state and local governments with continued confidence in using the model’s analyses to inform their budget and policy decisions.
How Results First partner jurisdictions have used RF Model results
State and county governments have utilized the data generated by the RF Model to inform programmatic funding decisions, including to:
- Implement new programs with a high estimated return on investment. Using its state-specific model, Colorado’s Office of Community Corrections projected that cognitive behavioral therapy would generate a positive ROI if implemented, as shown in Table 1. Based on this information and prior recommendations from the Colorado Commission on Criminal and Juvenile Justice, the community corrections agency redirected funding toward piloting this program.
- Expand existing evidence-based programs. In New Mexico, the state’s Legislative Finance Committee routinely uses its RF Model results to inform budget decisions in a number of policy areas. In fiscal year 2019, the state budgeted more than $130 million for effective evidence-based programs analyzed by the committee’s RF Model.
- Help secure grant funding. New York used information from its RF Model to win a $12 million Pay for Success grant from the U.S. Department of Labor. The grant allowed the state to expand evidence-based employment programs expected to generate cost savings. State leaders attributed the successful bid to the strength of their Results First work, which quantified the financial and public safety value of investing in employment services for high-risk, recently released parolees.
- Strengthen funding requests. The Community Corrections Partnership (CCP) in Santa Barbara County, California, created a new form for criminal justice agencies to request money from the Board of Supervisors. It requires key details about the program to be funded, including target population, criminogenic need addressed, desired program outcomes, and cost-benefit analysis from the Santa Barbara-specific RF Model, where available. This information will allow the CCP to better formulate and prioritize the funding recommendations it sends to the Board of Supervisors.
The RF Model is one tool that can help government leaders use rigorous evidence in their budget and policy decisions. By having jurisdiction-specific information about programmatic benefits and costs, these policymakers are better equipped to make crucial investment decisions about programs that serve their communities.
Accounting for Risk
As with any investment analysis, estimating benefits and costs necessarily involves uncertainty and some degree of speculation about the future. To account for this, the RF Model includes the option of running a Monte Carlo simulation—a type of risk analysis used in private-sector investment decision-making. This analysis lets users calculate the likelihood that the benefits of a program will exceed its costs (in other words, that it will at least break even).
Endnotes
- The RF Model is based on the cost-benefit model originally developed by the Washington State Institute for Public Policy.
- The RF Model can analyze programs in the following social policy areas: adult criminal justice, adult mental health, child mental health, child welfare, general prevention (i.e., programs aimed at youth to prevent negative outcomes, such as smoking in middle school, and promote positive outcomes, such as high school graduation), health, higher education, juvenile justice, pre-K-12 education, and substance use disorder.