States Can Set Funding Thresholds That Promote Evidence-Based Programs

This fact sheet is the second in a series that describes strategies to codify results-based policymaking.

Overview

In the face of budget shortfalls or changes in government leadership, state officials can struggle to sustain reforms and advance priorities, including efforts to promote evidence-based policymaking. State leaders can use a number of strategies to ensure that their efforts are continued through inevitable changes in the political life cycle. One strategy they can consider is to set thresholds that require a percentage of available funds to be allocated to evidence-based programs. State officials can embed these requirements in law to establish and perpetuate a commitment to research. This fact sheet identifies six steps that states can take to effectively prioritize programs proven effective through law.

Funding thresholds can help sustain data-driven budgeting

Laws establishing funding thresholds encourage—and sometimes require—the use of evidence-based programs¹and limit funding for untested or ineffective programs. To work well, these laws should define “evidence-based,” describe the level of available research required to satisfy the term, and provide information on how state officials will evaluate compliance. States can designate a specific percentage of funding for these proven programs or require the threshold to increase yearly until it reaches a ceiling.

States also have options for determining how broadly to apply this law. For example, they can apply the funding thresholds to a set of programs (e.g., substance use prevention services), one agency or department, or multiple agencies. This decision is often informed by how strong the evidence base is in a policy area. Where research is limited, a law mandating the use of such programs may be difficult to implement.

Laws that include requirements for funding evidence-based programs work best when legislators, agency leaders, and community-based organizations work together to agree on the scope (which agencies and services are affected) and the execution of the law, including realistic expectations for scaling up. Policymakers and providers should also discuss resources needed to resolve potential challenges in complying with the law.

By creating funding thresholds to prioritize evidence-based programs, policymakers can:

• Create the expectation that public programs are routinely assessed to determine what is known about their effectiveness.
• Support public agencies in their efforts to select and operate programs based on the positive outcomes they produce.
• Identify currently funded programs with limited or no evidence and prioritize those for evaluation or replace them with programs that have demonstrated results.
• Improve public accountability for spending taxpayer dollars.

How jurisdictions use funding thresholds

Several state governments have passed laws that include funding thresholds to promote and track increased use of evidence-based programs.

Oregon passed a law in 2003 to increase the funding requirement for evidence-based programs to 75 percent in five state agencies: the Department of Corrections, Oregon Youth Authority, State Commission on Children and Families (which became the Oregon Early Learning Division in 2011), the addictions and mental health services unit within the Oregon Health Authority, and the Oregon Criminal Justice Commission.² The law required that 25 percent of funds be spent on evidence-based programs by 2007, growing to 50 percent by 2009 and 75 percent by 2011, and remaining at that level thereafter.

In its 2016 report to the Legislature, the Oregon Health Authority reported that nearly 80 percent of its programmatic funds went to such programs,³ while the Youth Authority reported spending 87 percent of state funds and 91 percent of its total programming funds on evidence-based work.⁴ The Youth Authority’s report added that the law had helped the agency “develop capacity and improve programming” while also giving the agency the flexibility it needed to “assess new and promising programs.” It also acknowledged that, while identifying and operating evidence-based programs is a starting point, ongoing assessment can ensure that the law helps to improve program quality and participant outcomes.

In South Carolina, the First Steps to School Readiness initiative was created by law in 1999 as a statewide public-private partnership to improve outcomes for children.⁵ In 2014, the state mandated that at least 75 percent of funds from First Steps be used on evidence-based programs, which it defined as those that are grounded in published, peer-reviewed research linked to determined outcomes; employ well-trained and competent staff; are linked to other community-based services; operated to fidelity; and are recognized by experts in the field.⁶

Further, the law required that no more than 25 percent of funding be used for “evidence-informed programs,” which are those that don’t satisfy the criteria of “evidence-based” but that the board of trustees determines to be supported by research indicating their potential effectiveness.⁷ The law also requires that an independent evaluation be conducted no fewer than every five years on any “prevalent” program (one that receives more than 10 percent of the state’s available funding).

In 2012, the Washington state Legislature passed a bill to significantly increase the use of evidence-based programs in the child behavioral health, child welfare, and juvenile justice systems.⁸ The law stipulated that each agency attain “substantial increases” in its use of evidence-based programs and submit a report to the Legislature by 2013 detailing strategies, timelines, and costs for achieving this increase. Because of these requirements, the Behavioral Health and Service Integration Administration set a target to increase the use of evidence- and research-based programs by 7.5 percent a year, reaching 45 percent in 2019.⁹To help providers meet the law’s requirements, the Children’s Administration increased payment rates for providers operating evidence-based programs by 25 percent, in addition to paying for time spent monitoring program fidelity.¹⁰

How to create and apply funding thresholds in law

Below are six actions for legislators to consider when determining how best to mandate funding thresholds for evidence-based programs:

1. Establish clear, formal definitions of evidence and collect baseline program information. Lawmakers should develop a common understanding of what evidence-based, promising, or other type of program entails and include these definitions in laws, statutes, or administrative policy. Codified definitions provide a consistent example of which programs qualify and avoid confusion down the road. Lawmakers should also create a clear process for agencies to inventory the programs they operate and categorize them based on these definitions.¹¹ Such an inventory will provide a baseline of programs that agencies already operate and how close they are to the designated funding thresholds. This information will help agencies develop realistic and appropriate targets for expanding the use of such programs in the future.
2. Engage agency leaders and service providers early and often. In addition to the buy-in of legislative and executive staff, it’s important to gain support from external stakeholders, including contracted service providers who are responsible for meeting these funding thresholds. Moreover, these stakeholders can help ensure that the requirements are realistic, achievable, and complement any national standards or definitions to which they must already adhere.
3. Build available resources and capacity. Laws that prioritize evidence-based programs often lead to higher standards and expectations for the organizations that deliver services. Including additional funding to increase their capacity for implementation, monitoring, and evaluation can enhance providers’ likelihood of meeting the mandate. Established institutions, such as research universities or implementation support centers,¹² can also support providers during this process.
4. Phase in the requirements. States often pilot funding threshold requirements in one policy area or state agency and then, based on their success, expand the requirements to additional areas. States can phase in these thresholds over several years, beginning with a charge to inventory and categorize existing programs to develop a baseline. A gradual approach allows agency leaders time to monitor their programs’ impact and build capacity to deliver evidence-based services.
5. Establish a process for verifying compliance with the requirements. It is important to ensure that agencies are classifying programs according to the evidence standards and calculating expenditures in a consistent manner. There are several ways to collect these data, including leading agencies through a program inventory process, surveying providers on the interventions they are operating, or using tools that assess fidelity to evidence-based principles. Whichever way they choose, state leaders should develop a process to confirm the thresholds they set are being met appropriately.
6. Develop provisions for evaluating untested programs and allowing for innovation. There are often “homegrown” programs—developed locally or adapted from an original design—that are valuable and effective for some communities and special populations. States can make exceptions for these programs, and even encourage innovation and invention, using their funding threshold requirements. By setting funding thresholds below 100 percent, states can give providers adequate time to evaluate any untested or pilot programs to determine if the service should be continued, expanded, or replaced with an alternative.

Endnotes

1. An “evidence-based” program or practice offers a high level of research on effectiveness, determined as a result of multiple rigorous evaluations, such as randomized controlled trials and evaluations that incorporate strong comparison group designs, or a single large multisite randomized study. These programs typically have specified procedures that allow for successful replication. For more information, see Pew-MacArthur Results First Initiative, “Defining Levels of Evidence” (2015), https://www.pewtrusts.org/en/research-and-analysis/fact-sheets/2015/11/defining-levels-of-evidence.
2. Ore. S.B. 267 (2003), http://www.crimevictimsunited.org/issues/treatment/ebp/sb267.pdf.
3. Varsha Chauhan, chief health systems officer, Oregon Health Authority, “ORS 182.525 Evidence Based Programs Report—Summary,” Aug. 9, 2016, https://digital.osl.state.or.us/islandora/object/osl%3A79765/datastream/OBJ/view.
4. Oregon Youth Authority, “Interim Judiciary Committee Progress Report on SB 267 (ORS 182.525)” (2016), http://www.oregon.gov/oya/docs/SB267_2016.pdf.
5. S.C. title 59, chap. 152, “South Carolina First Steps to School Readiness,” https://www.scstatehouse.gov/code/t59c152.php.
6. Ibid.
7. Ibid.
8. Washington Legislature, “Engrossed Second Substitute House Bill 2536” (2012), http://lawfilesext.leg.wa.gov/biennium/2011-12/Pdf/Bills/Session%20Laws/House/2536-S2.SL.pdf?cite=2012%20c%20232%20%C2%A7%205.
9. Washington State Department of Social and Health Services, “Report to the Legislature: Evidence-Based and Research-Based Practices—Updates and Recommendations” (2014), 7, http://app.leg.wa.gov/ReportsToTheLegislature/Home/GetPDF?fileName=Evidence-Based%20and%20Research-Based%20Practices%20E2HB2536_80910208-1bee-4d86-9d99-b3f2a1906e2a.pdf.
10. Pew-MacArthur Results First Initiative, “How Policymakers Prioritize Evidence-Based Programs Through Law” (2017), https://www.pewtrusts.org/en/research-and-analysis/issue-briefs/2017/04/how-policymakers-prioritize-evidence-based-programs-through-law.
11. Pew-MacArthur Results First Initiative, “How to Use the Results First Program Inventory” (2018), https://www.pewtrusts.org/en/research-and-analysis/fact-sheets/2018/04/how-to-use-the-results-first-program-inventory.
12. Pew-MacArthur Results First Initiative, “4 Ways Implementation Support Centers Assist in the Delivery of Evidence-Based Programs” (2017), http://www.pewtrusts.org/en/research-and-analysis/fact-sheets/2017/07/4-ways-implementation-support-centers-assist-in-the-delivery-of-evidence-based-programs.