How to Use the Results First Program Inventory

As state and local governments continue to face significant budgetary constraints, policymakers want to be sure they are making the best use of limited taxpayer dollars. Yet these leaders must frequently make budget and policy decisions without important information about the effectiveness of the programs they fund. To help address this challenge, the Pew-MacArthur Results First Initiative provides tools, training, and technical assistance to state and local governments to help them identify and prioritize funding toward effective evidence-based programs. This process begins with conducting a program inventory.

What is a program inventory, and how can it be used?

A program inventory is a comprehensive list of the programs a jurisdiction funds in a particular policy area.¹ It includes key details, such as name and description, as well as the program’s level of effectiveness based on existing, rigorous research.

States and counties have used their program inventories to:

• Establish baseline information on program investments to enable comparisons and track progress.
• Set goals for increasing investments in evidence-based programs.
• Identify and invest in effective evidence-based programs that address population needs and close service gaps.
• Streamline program offerings to reduce duplication and improve implementation.
• Shift funds away from programs proved not to work or with limited evidence of effectiveness.
• Identify key programs that would benefit from rigorous evaluation.

Program inventory in 3 steps

There are three main steps for conducting a program inventory. Results First staff provides state and county partners with the technical assistance and tools needed to accomplish each one. The steps are:

1. Create a comprehensive list of currently funded programs
   Partners start by developing a list of the programs currently funded by public dollars, including governmentoperated as well as contracted programs, in a specific policy area. As part of this process, partners gather key information about each program, including a description, the frequency and duration, what type of setting it is offered in, whom it is offered to, and how much it costs. This snapshot allows policymakers and agency staff to see the portfolio of programs being offered in a policy area, often for the first time.
2. Assess which programs are most likely to be effective, based on rigorous research
   Next, state and county partners compare that list of programs with the evidence base, starting with any local evaluations conducted in the jurisdiction. When such local evaluation data is unavailable, partner jurisdictions utilize the program data they’ve collected to match their programs to those that have been evaluated elsewhere. They do so by using the Results First Clearinghouse Database, an online resource that provides users with an easy way to access information on the effectiveness of evidence-based interventions across multiple policy areas. The database compiles program research from eight national clearinghouses that independently review and assess programs for rigor of evidence and level of effectiveness. Matching their programs to those that have been evaluated in the national literature allows partners to determine which funded programs have evidence showing that they are effective, which have evidence showing that they are ineffective or produce negative outcomes, and which lack any rigorous research. Policymakers can use this information as they consider strategies to increase the use of effective evidence-based programs.
3. Use findings to inform programmatic, policy, and funding decisions
   Results First staff members help state and county partners interpret results and develop reports that explain findings to policymakers. Government leaders can then use the information to make policy and budget decisions, such as shifting funds to alternatives that can achieve better outcomes and prioritizing evaluations of programs that lack sufficient evidence. 

State and local governments use inventories to get better results

States and counties across the country have used the information from their program inventories to inform programmatic, policy, and budget decisions.

Rhode Island, for example, used the program inventory to develop better tools to match vulnerable children with appropriate and effective evidence-based programs. The Office of Management and Budget worked with two state agencies to complete an inventory of child welfare programs in an effort to better understand their effectiveness in addressing high rates of abuse and neglect and out-of-home placement for children in the state. The inventory revealed that the state operated 11 evidence-based programs that had been shown to be effective in improving child welfare outcomes. However, agency staff lacked standardized tools to match families with the right services, resulting in some families not being appropriately served.

Based on the inventory findings, Rhode Island’s Children’s Community Services and Behavioral Health units in the Department of Children, Youth, and Families (DCYF) are now implementing a validated screening tool to match children and families to the services that research shows are most effective in addressing their needs. Additionally, the DCYF has revamped its contracts with service providers to prioritize the use of effective evidence-based interventions.²

The program inventory can be a useful practice for state and county partners. It allows jurisdictions to assess the likely effectiveness of their funded programs, which can in turn enable leaders to make more informed choices about how to allocate public resources.

In California, Kern County used its inventory to target funds toward a menu of effective evidence-based programs for adult offenders. First, the Kern County Community Corrections Partnership (CCP) worked with Results First staff to develop a comprehensive inventory of the adult criminal justice programs provided throughout the county. This process allowed county leaders to assess their program investments not only as individual agencies but also as a cohesive system of service providers. Agency leaders worked together to identify which programs were offered across agencies and providers, pinpoint potential gaps in service and duplication of efforts, and understand the evidence supporting each program. Finding a dearth of programs available in the jails, the Kern County Sheriff’s Office used the Results First Clearinghouse Database as a menu of program options to expand its services, eventually implementing seven effective evidence-based programs and an array of education services to give inmates the tools they need to successfully re-enter the community. Similar to the Sheriff’s Office, the Probation Department used the database to identify which programs to implement in its newly created Adult Programs Center and Juvenile Programming Unit.³ By developing the inventory as a system to inform treatment decisions, county leaders improved communication and even cultivated partnerships to ensure that effective treatment continues during transition from jail custody to community supervision.

The program inventory can be a useful practice for state and county partners. It allows jurisdictions to assess the likely effectiveness of their funded programs, which can in turn enable leaders to make more informed choices about how to allocate public resources.

Endnotes

1. Results First defines programs as systematic activities that engage participants in order to achieve desired outcomes.
2. Rhode Island Office of Management and Budget, “Results First—Child Welfare Program Review and Benefit-Cost Analysis” (2017), http://www.omb.ri.gov/documents/performance/performance-reports/all/21_Results First Child Welfare Program Review and Benefit-Cost Analysis_January 2017.pdf.
3. Kern County Probation Department, “Results First—Kern County, CA” (2015), http://www.kernprobation.com/wp-content/uploads/2014/04/Kern-County-Results-First-Brief.pdf.