Results First Clearinghouse Database Helps Users Access Information on Program Effectiveness

Overview

The Results First Clearinghouse Database is an online resource that provides an easy way for users to access and understand the evidence base for programs in social policy areas, such as behavioral health, criminal justice, education, and public health. It aggregates information from nine national clearinghouses¹ that summarize evidence on program effectiveness.² Additionally, the Clearinghouse Database applies a color coding (see Figure 1) to the clearinghouses’ distinct rating systems, creating a common language that lets users quickly see where each program falls on a spectrum ranging from negative impact to positive impact. This makes it easier to compare ratings across clearinghouses as well.

Results First originally developed the Clearinghouse Database in early 2014 as an internal Excel-based resource to help policymakers and staff in Results First partner jurisdictions identify what works in terms of a program’s effect on important outcomes. It became clear that there was great interest in this resource among a broader policy community. Results First initially made the spreadsheet version available online to the public in late 2014 and then launched a more user-friendly interactive version in 2015. In 2018, additional program information from more clearinghouses and new search features improved the tool’s design and functionality.

With information on more than 3,000 programs, the Clearinghouse Database has become a one-stop shop for many people looking for evidence of program effectiveness. Although most of the tool’s known users are government officials and their staffs, Results First has learned that nongovernment users are accessing it as well. This fact sheet provides examples of how users have applied the tool to their work.³

Identify what works

As intended, jurisdictions have used the Clearinghouse Database to identify effective programs that address their specific needs. For example, in Kern County, California, probation officers used the tool to identify evidence-based programming for their jails. As a result, the Kern County Sheriff’s Office was able to replace ineffective programs with those that are evidence-based and add proven interventions to its treatment portfolio. Staff members could identify what has worked in a scenario similar to their own, because the tool allows users to search by keyword and to find a program by name, target population (e.g., middle school youth or incarcerated adults), outcome of interest (e.g., truancy or crime), or delivery setting (e.g., in school or in prison).

The Clearinghouse Database has allowed Results First partner jurisdictions to develop program inventories, a comprehensive list of the programs that a jurisdiction funds in a particular policy area. One key step of the inventory process is for the jurisdiction to search the Clearinghouse Database to see whether the programs listed in its inventory match any similar programs in the evidence base and, if so, what the research says about their effectiveness or lack thereof. Decision-makers can then use this information when determining how to allocate their limited resources. For example, Iowa’s Department of Corrections used the tool to identify what works to reduce recidivism and then shift resources to effective programs that better support department goals.

Develop budget proposals

States are increasingly incorporating evidence requirements into their budget processes. More specifically, budget offices are encouraging or requiring agencies to justify requests for new or increased funding by citing rigorous evidence showing that the additional capital is likely to produce positive outcomes. This practice underscores the reality that resources are limited and that use of evidence to support investments can make the most of them.

To help agencies comply with these new requirements, budget offices have shared the Clearinghouse Database and suggested that they use it to help produce their requests. For example, Minnesota’s fiscal year 2020-21 biennial budget instructions include a new evidence-based proposal form that references the Clearinghouse Database as a way for agencies to find evidence to support their proposals. In turn, budget analysts who then review the forms and other budget documents from agencies use the tool to validate evidence-based information that is provided. Other states, such as Tennessee, have not included an explicit reference to the tool in their budget documentation, but they guide agencies to it by posting the link on their evidence and budgeting website. In either approach, the tool has become a dependable and valuable resource for budget offices to include in their guidance and for agencies to use when developing their requests.

Refine contracts

State and local governments frequently rely on community-based organizations (CBOs) to provide social service programming. To help ensure that this programming is effective, jurisdictions have started to include evidence stipulations in the contract and procurements processes. Some requests for proposals (RFPs) award extra points to applicants offering evidence-based programs, while others ask that applicants provide evidence-based information to support their proposal if it is available.

To assist CBOs in fulfilling these evidence-based requirements, jurisdictions have leveraged the Clearinghouse Database as a key resource.

• Pennsylvania: In 2018, the Pennsylvania Commission on Crime and Delinquency (PCCD) released a solicitation for Justice Assistance Grant funds and stated that a primary objective was to “support the use of evidence-based programs and practices,” aligning with PCCD’s 2016-20 strategic framework. The funding opportunity cited the Clearinghouse Database as a particularly useful tool for applicants to use to identify evidence-based programs.
• California: In 2020, the California Department of Corrections and Rehabilitation’s Division of Rehabilitative Programs (DRP) established contracting guidelines for selecting evidence-based treatment programs. These require that 100% of DRP’s treatment programs—both in-prison programs across 35 state institutions and those delivered via contracts with CBOs—be selected from the Clearinghouse Database.
• Santa Cruz County, California: In fiscal year 2017, Santa Cruz referenced the Clearinghouse Database in its Collective of Results and Evidence-Based (CORE) Investments RFP, from which the county awarded $4.1 million in three-year contracts to CBOs for implementing evidence-based programs.
• Cook County, Illinois: In 2019, Cook County’s Justice Advisory Council (JAC) collaborated with the county procurement office to incorporate evidence principles into its contracting process. The county integrated several questions into its buying plan, which now requires an evidence assessment upon review of grant proposals for JAC programming. The procurement office encourages providers to look to the Clearinghouse Database to identify effective interventions to meet service needs.

In each of these contracting examples, the Clearinghouse Database not only helps applicants craft strong proposals but also aids those who evaluate and eventually decide on which proposals to select. 

Enhance educational curricula

Several universities use the Clearinghouse Database as a teaching tool. Dr. Patrick Tennant—a researcher and project manager focusing on children’s mental health in the Center for Health and Biosciences at Rice University’s Baker Institute for Public Policy—has demonstrated the Clearinghouse Database throughout his career when teaching program evaluation to students seeking a master’s degree in clinical mental health counseling. Tennant said he does so because “there are some students who have limited exposure to [evidence-based programs], and the [Clearinghouse Database] is comprehensive and allows the students to explore and learn more about the variety of evidence-based programs in a short period of time.”⁴

The College of Social Work at the University of South Carolina has cited the database in blog posts, saying it “makes program search easier.” Introducing the tool to students is critical because it can increase their understanding of evidence-based programs. Some of them may go on to be practitioners, staff in government agencies, or researchers who find it valuable to have foundational knowledge of evidence-based programs.

Conclusion

Whatever the purpose—an agency staff member looking for evidence to justify funding, a CBO highlighting the evidence behind its programming as part of a proposal, or a student interested in the broader world of evidence-based programs—the Clearinghouse Database is a tool that provides concise, relevant, and easy-to-access information on thousands of programs for an increasingly diverse set of users. 

Endnotes

1. Clearinghouses conduct systematic research reviews and summarize the findings of rigorous evaluations of programs for non-researcher audiences. They also rate programs based on this information. In general, the ratings reflect the program’s level of effectiveness, as well as the quality and quantity of the evidence.
2. The Pew Charitable Trusts, “Where to Search for Evidence of Effective Programs” (2020), https://www.pewtrusts.org/en/research-and-analysis/fact-sheets/2020/04/where-to-search-for-evidence-of-effective-programs.
3. These are examples known to Results First staff and therefore should not be considered an exhaustive list.
4. P. Tennant (researcher and project manager, Center for Health and Biosciences, Rice University’s Baker Institute for Public Policy), interview with Mara Weinstein, Feb. 25, 2020.