States Should Prioritize Evidence in Budgeting to Promote Positive Outcomes

Overview

As governments grapple with limited funds and competing priorities, many leaders are turning to evidence-based policymaking¹ to make data-driven decisions that maximize resources for human services programs. However, those efforts can be difficult to maintain in the face of economic uncertainty and transitions in legislative and agency leadership, so jurisdictions are looking for ways to cement their work and increase the likelihood that evidence- based approaches will be sustained. One strategy they are using is evidence guidelines—budget directives that prioritize the use of research and data in funding decisions.

Evidence guidelines support informed, data-driven budgeting

Legislative sessions are often compressed into a few weeks or months, leaving little time to carefully review agency funding requests; identify the most promising strategies; and make decisions that consider policy priorities, jurisdiction needs, and rigorous research. Evidence guidelines are a tool that policymakers can use to help make consideration of data a key part of the early stages of the budget review process.

The guidelines typically require that a central entity, such as an executive or legislative budget office, vet agency appropriation requests for strong supporting research on the programs to be funded. They also generally oblige agencies to meet certain criteria—for example, selecting programs that have been rigorously evaluated, found effective, and are listed in a nationally recognized clearinghouse. Guidelines can be written flexibly to allow for innovation when necessary, such as when a policy area lacks robust research, and to exclude funding requests for which social science evidence is not applicable, like those for additional staff. This approach enables policymakers to focus on the information presented and provides a consistent measure with which to compare funding proposals.

By using evidence guidelines as a screening tool, state governments can:

• Ensure that agency funding requests meet high standards.
• Require agencies and program providers to demonstrate the value of proposed initiatives.
• Prioritize funding for programs that evidence shows are most likely to achieve positive outcomes.
• Increase transparency in the budget process by using a consistent method to assess requests.
• Encourage agencies to prioritize effective, evidence-based programs.

How jurisdictions use evidence guidelines

Several state governments are using evidence guidelines to improve their budget processes and make more informed decisions:

• Colorado began including “evidence requirements” in its budget instructions in 2016.² For any funding request for a new or expanded program, departments must answer four questions that require them to summarize and cite the research on expected outcomes, document when possible projected returns on investment³ with program-level cost-benefit information (using the Pew-MacArthur Results First Initiative model⁴), and provide an evaluation plan for new programs. The governor’s Office of State Planning and Budgeting reviews the requests and provides research and funding recommendations as needed, prioritizing programs that evidence shows are effective.
• Mississippi’s Joint Legislative Budget Committee requires state agencies to use a screening checklist for funding requests for new programs or activities.⁵ Agencies must submit research demonstrating program effectiveness and specify whether programs are evidence-based, research-based, a promising practice, or none of the above.⁶ In addition, agencies are required to report program metrics including outputs, outcomes, and efficiency, as well as a timeline for measurement. Untested programs must have a detailed evaluation plan. Analysts from the Joint Legislative Committee on Performance Evaluation and Expenditure Review vet the requests and provide nonbinding funding recommendations to the Legislature.
• New Mexico’s Legislative Finance Committee created a Legislating for Results framework to extend the state’s commitment to using performance data—which began with the 1999 Accountability in Government Act—to the state’s budget development process.⁷ The framework includes five actions for an evidence-based budget:

1. Identify priorities and use performance data to highlight areas in need of more oversight.
2. Review evidence on program effectiveness.
3. Incorporate performance and effectiveness data into the budget development process, prioritizing allocations for programs that are likely to work and yield a positive return.
4. Monitor program implementation through performance reports and other tools.
5. Assess whether programs are achieving desired outcomes through performance reports; evaluations; and comparison with state, industry, or national data.

The committee’s analysts are trained to review appropriation requests and use these tools to help budget staff identify and prioritize programs with evidence of success. In fiscal year 2019, the committee will consider requests on a case-by-case basis, limiting expansions to programs that address high-priority policy areas and are evidence-based or tied to improved service delivery.

How to create and apply evidence guidelines

Below are three actions policymakers should consider when determining how best to codify the use of evidence in the budget process.

1. Determine which programs to regularly review. Which programs will be subject to evidence guidelines can have a significant impact on the staff time needed to conduct assessments. Some governments have focused on new programs that agencies want to implement, while others have subjected program expansions to more stringent review. Over time, governments may expand the types of programs requiring evidence assessments, using criteria such as funding level, populations served, or performance outcomes.
2. Clarify the level of evidence required for funding. Defining “evidence” in official budget materials creates a common language for agency leaders and program providers, and can help decision-makers better understand the extent to which a program is backed by research. This information can offer additional context about the program, such as its projected impact. Some of the categories that states use to differentiate levels of evidence and rigor include evidence-based, promising, and theory-based.⁸
3. Develop a process for programs that do not meet evidence standards. Some programs are not backed by strong data but are still effective. When faced with a program that has insufficient evidence of effectiveness, policymakers may want to consider conducting an evaluation to help state leaders learn about the program while adding to the evidence base.

Endnotes

1. Pew-MacArthur Results First Initiative, “How States Engage in Evidence-Based Policymaking” (2017), http://www.pewtrusts.org/~/ media/assets/2017/01/how_states_engage_in_evidence_based_policymaking.pdf.
2. Colorado Office of State Planning and Budgeting, “Budget Instructions” (May 2017), https://drive.google.com/file/d/0B0TNL0CtD9wXOXozbkMzUlZwZHc/view.
3. The Colorado Office of State Planning and Budgeting defines return on investment as a metric that conveys how much money the state may expect to “earn back” for every dollar invested. It is used to determine whether a program is cost-effective.
4. For more information, see Pew-MacArthur Results First Initiative, “Overview,” http://www.pewtrusts.org/en/projects/pew-macarthur-results-first-initiative.
5. Mississippi Joint Legislative Budget Committee, “Budget Instructions/Forms: Research and Evidence Filter,” 15, http://www.lbo.ms.gov/ pdfs/obrsforms/2020_budget_instructions.pdf.
6. Miss. Ann. Code § 27-103-159 (1972), http://billstatus.ls.state.ms.us/documents/2014/html/SB/2200-2299/SB2257PS.htm.
7. New Mexico Legislature, Article 3A: Accountability in Government Act, https://www.nmlegis.gov/(X(1)S(nf3c1lxhtxyuh1agzr2cd0xl))/ Entity/LFC/Documents/Accountability_In_Goverment_Act/Accountability%20in%20Government%20Act%20Statute.pdf; New Mexico Legislative Finance Committee, “Legislating for Results,” https://www.nmlegis.gov/Entity/LFC/Documents/Accountability_In_ Goverment_Act/Legislating%20For%20Results.pdf.
8. For more information, see Pew-MacArthur Results First Initiative, “Defining Levels of Evidence” (2015), http://www.pewtrusts.org/en/research-and-analysis/fact-sheets/2015/11/defining-levels-of-evidence.