How States Can Develop and Sustain Evidence-Based Policymaking

Overview

Minnesota and New Mexico stand at the forefront of states engaging in evidence-based policymaking. This comprehensive approach to incorporating evidence into public policy decision-making is used in a variety of ways, from selecting programs and guiding investments to tracking program outcomes and evaluating their impact.

The two states—which are among the 27 states and 10 counties that have partnered with The Pew Charitable Trusts’ Results First initiative—have taken deliberate steps to enhance evidence use, demonstrating that it can help generate positive results for state residents and make the most of limited government resources. Moreover, they have worked to ensure that their efforts are sustainable and can withstand challenges, including tightening budgets and changes in legislative and executive leadership. Their successes cannot be attributed to a single budget allocation, leader, or piece of legislation; rather, the two states have learned how to use evidence systematically and in a coordinated fashion across government.

This brief—which draws on 40 in-depth interviews with executive and legislative branch leaders and staff across Minnesota and New Mexico—offers insights from the experiences of the two states. These lessons could help other states use evidence to drive their decisions about what to fund and how best to improve community outcomes.

A common strategy

While evidence-based policymaking has become ingrained in both Minnesota and New Mexico, there are dissimilarities between the states. Minnesota is a local control state, in which county governments and some tribal nations are direct providers of public programs, such as social services and corrections. In comparison, New Mexico has a more centralized government, resulting in greater state authority over service provision. Minnesota has a higher gross domestic product,¹ higher median household income,² and lower rates of poverty than New Mexico³ —though some disparities, such as poverty level across racial and cultural groups, are larger in Minnesota than in New Mexico.⁴ New Mexico, on the other hand, relies more heavily on federal funds than Minnesota does⁵ and has a smaller rural population.⁶ Yet despite having divided state governments for most of the past decade, each was able to incorporate evidence into the fabric of its decision-making processes.

Minnesota

In 2011, then-Governor Mark Dayton (D) launched the “Better Government for a Better Minnesota” initiative to improve the efficiency and cost-effectiveness of state services;⁷ the state has been expanding these efforts ever since. With the efforts spearheaded by Minnesota Management and Budget (MMB),⁸ an agency within the executive branch responsible for producing the state budget, Minnesota has worked with Results First,⁹ state leaders, and numerous other stakeholders to advance the use of evidence. In 2019, then-MMB commissioner Myron Frans said of the state’s efforts, “Everybody used to tell anecdotes [about how programs were doing]. Anecdotes are fine, but they’re not data. Now people really want to make sure that if they’re going to invest public dollars, they’re doing it in the best way possible. That has been a positive component of the dialogue between the Legislature, local units of government, [the budget office], and the administration.”¹⁰ Some of Minnesota’s key accomplishments include:

• Building state capacity. In 2015, at the initiative of state Senator Michelle Benson (R), Minnesota passed bipartisan legislation¹¹ calling for assessments of the state’s corrections and human services programs. The bill dedicated $122,000 annually for MMB to hire 1.5 full-time equivalent analytical staff members to carry out the assessments.
• Defining evidence. MMB publicly defined¹² various tiers of evidence, acknowledging that research on programs can vary in rigor and quantity.
• Regularly analyzing the data. MMB collaborated with Results First and numerous state agencies to develop online program inventories¹³—which list funded services and describe available research about their effectiveness—and to conduct cost-benefit analyses.¹⁴
• Making findings accessible to a wide variety of potential users. MMB made publicly available its comprehensive statewide program inventory¹⁵ of more than 500 publicly funded programs and services. The tool is accessible online and searchable by outcomes measured, policy area, population, and setting.
• Funding programs that are proven to work. Using information collected from state agencies, the state enacted 11 evidence-based proposals into law in 2019, representing more than $85 million in funding for new or expanded evidence-based programs in fiscal year 2020-21.¹⁶

New Mexico

New Mexico has emphasized a results-driven approach for the past two decades, particularly through the implementation of performance-based budgeting encapsulated in the state’s 1999 Accountability in Government Act (AGA).¹⁷ Under the guidance of the Legislative Finance Committee (LFC),¹⁸ the Legislature’s fiscal management arm, the state has built a robust system to ensure evidence is considered at every stage of the budgeting process. “The idea of evidence-based decision-making has risen to the top,” said state Representative Patty Lundstrom (D). “It’s something that people really think about now. What is our real return? It’s not just how many people we have running through a system, but are we really seeing change? And is it the change we’d like to see?”¹⁹

The state’s key accomplishments include:

• Frequently assessing and reporting on program efficacy. Since 2011, the LFC has worked with Results First and state agencies to build program inventories and conduct cost-benefit analyses in a variety of policy areas, regularly publishing its findings.²⁰
• Financing programs with a track record of success. State leaders have targeted more than $400 million in resources toward effective and cost-beneficial programs.
• Ensuring that research informs policy. As a result of evidence-based research, for example, policymakers made large investments to expand an early childhood education program that demonstrated improved academic performance for participants.²¹
• Embedding evidence use and cross-branch collaboration in state law. The 2019 Evidence and Research Based Funding Requests act²² expands the AGA in a range of ways, including defining important terms such as “evidence-based,” requiring state agencies to inventory the evidence supporting their programs, and compelling them to indicate how much of their total funding requests will be spent on evidence- or research-based programs.

Foundations for a culture of evidence use

While many states have engaged in evidence-based policymaking, few have created the systemic change achieved in Minnesota and New Mexico. The following section looks at four elements that Pew identified, based on interviews with stakeholders in Minnesota and New Mexico, as central to ensuring that the states’ efforts to gather and use evidence are successful and enduring.

A respected, nonpartisan coordinating entity

A central impartial oversight entity can be well positioned to help state governments base their programs and policies on sound evidence, using data to make unbiased recommendations and building relationships across the political spectrum. As state Representative Tina Liebling (DFL) of Minnesota explained, “Whoever goes out to legislators needs a lot of credibility and needs to be viewed as nonpartisan; otherwise, don’t bother.”²³

In Minnesota, that entity is MMB—which maintains public tools to track and display data on programs and outcomes, trains staff and state agency commissioners on how to use evidence to inform their work, and encourages other state agencies to incorporate evidence into their budget documents. Many Minnesota stakeholders interviewed for this brief described MMB as nonpartisan, objective, and well respected, a reputation derived in large part from the expertise of its staff members, whom respondents valued for their knowledge, interpersonal skills, and investment in deep relationships. “The team at MMB that conducted a lot of the Results First work is a really thoughtful, super organized, really detail-oriented group of people who are good at collaboration and relationship building. They’ve done a really nice job of getting a diverse group of people around the table,” said Nikki Kovan, acting deputy director of child safety and permanency at Minnesota’s Department of Human Services.²⁴

MMB established an advisory committee²⁵—with representation from the executive and legislative branches (commissioners and legislators), judicial branch, tribal nations, and counties—to provide feedback on its efforts and ensure that evidence is understood, trusted, and used by state officials. It also created an impact evaluation unit in 2019 to help ensure that the state’s public programs and services are as effective as possible and hired full-time evaluators to staff the unit. In one example, the unit is partnering with grantees, other agencies, and academic partners to conduct evaluations of interventions that aim to prevent and treat opioid misuse;²⁶ like many states, Minnesota has seen an increase in deaths from opioid-involved drug overdoses over the past two decades.²⁷

In New Mexico, the coordinating entity is the LFC, which also houses the state’s Results First work and has  over 40 nonpartisan staff who are responsible for making budget recommendations for the state. The committee contains an evaluation unit, which regularly assesses the effectiveness of state-funded programs, informs lawmakers and agency officials about what is proven to work, and answers questions about the potential impact and outcomes of program choices. The LFC can also help steer funds toward the practices that their assessments find most effective. Like the governor, the committee makes budget recommendations—a rare power, since New Mexico is one of the few states that use independent legislative budgeting²⁸ —based on its findings.

Just as in Minnesota, having nonpartisan staff with recognized expertise has been crucial in New Mexico. Linda Freeman, executive director of the New Mexico Sentencing Commission, attributed the successes of the state’s Results First work to the strength and acumen of the LFC evaluation staff, adding, “Having internal leadership that’s very [focused and] dedicated to the initiative makes a big difference.”²⁹ New Mexico agency representatives interviewed for this brief said they valued the LFC’s analyses for confirming which of their longrun activities were working, as well as identifying when a new approach was needed. Bryce Pittenger, CEO of the New Mexico Behavioral Health Collaborative, noted that the LFC’s analysis, which included a poll of service providers, confirmed what she suspected for years but never had the data for: The state’s Children, Youth, and Families Department practice base needed to be better adapted to recognizing and responding to trauma. She described her experience of working with the LFC as highly collaborative, noting that she often relies on the evidence provided in its report on children’s behavioral health in the state: “I present those LFC Results First slides almost every time because [they show] people why we’re focusing on these things.”³⁰ As a result of the LFC’s work with New Mexico’s health agencies, state officials have shifted from a pattern of funding high-end acute care in children’s behavioral health to funding more prevention and early intervention programming in an effort to reduce costs and improve services and outcomes.

Integrating evidence into everyday business

State governments should integrate new requirements and practices that involve evidence into existing processes so that these requirements and practices are not seen as one-time efforts. This integration can include allocating ongoing resources; continuously updating related data and tools; and maintaining dedicated staff with sufficient capacity to conduct difficult analyses, utilize evidence, and engage in meaningful conversations about evidence with diverse stakeholders.

Minnesota and New Mexico both reserve funds and staff time to systematically incorporate evidence into program and policy decisions. Both states have dedicated staff members to help gather and assess evidence, through program inventories, return-on-investment analyses, and evaluations of state agency services. Britta Reitan, Minnesota’s budget director, noted the value of the Legislature committing funding for the state’s Results First work: “It’s not something that needs to be carved out of other work and budgets. That really helps in terms of sustaining it.”³¹

Both states have also created procedures for the evidence they routinely gather to inform their funding choices. New Mexico’s LFC created the Legislating for Results Framework³² in the early 2000s to outline a systematic approach for incorporating research and performance data into the budget process. The framework encompassed many crucial components of evidence-based policymaking—including using data to identify underperforming programs, reviewing program effectiveness information, developing budget recommendations, monitoring program implementation, and evaluating outcomes—providing several points at which analysts consider evidence. In Minnesota, MMB developed a proposal form³³ that enables agencies to demonstrate when their budget proposals are evidence-based by soliciting ratings of the proposed program’s evidence level, as well as a plan detailing how the agency will support implementation and measure the effectiveness of its approach.

While the states have dedicated resources to evidence-based approaches, respondents interviewed for this brief noted that prioritizing evidence can be challenging for some implementers of these approaches—including counties and state agencies. These entities sometimes lack adequate resources and face competing funding pressures and numerous mandates over how they govern their programs and services. Jon Courtney, deputy director for program evaluation in New Mexico, explained why it can be so challenging: “Some agencies have huge data units that are doing nothing but data analysis, and some agencies have nothing like that. [It’s important to] really make sure that agencies are building [an evidence-based] approach and thinking into their business processes, not only in implementation of programs but also in resource allocations and budget requests.”³⁴

A diverse, engaged group of evidence users  and generators

Both Minnesota and New Mexico have found it essential to secure both top-down and bottom-up support for evidence-based policymaking efforts. Having “champions”—a diverse bench of leaders in the Legislature, in the governor’s office, and in government agencies to help facilitate and promote evidence—helps ensure that efforts will continue even if there is turnover or a change in administration. Likewise, numerous individuals interviewed for the report stressed the need for coordination among those various users of evidence, particularly across branches of government.

The 2019 passage of New Mexico’s Evidence and Research Based Funding Requests bill required a persistent effort on the part of its champions. New Mexico state Senator Sander Rue (R) originally sponsored a version of the legislation in 2015 to address inconsistencies in the way state agencies proposed their programmatic budgets. “We wouldn’t have [the bill] without the vision and tenacity of Senator Rue,” Courtney said. “[The senator] thought it was a good idea, picked it up, and ran with it for [several] years… making a bit more progress each year.”³⁵ The state’s legislative and executive branches will need to actively collaborate to execute the law; the LFC and the Department of Finance and Administration (DFA) have already joined forces to review state agency report cards with agency officials, a step toward more concerted support for, and shared oversight of, state performance goals.

In both states, regular reporting on evidence and efforts to actively engage policymakers in both parties have led to a change in culture around evidence at the highest levels of government decision-making. New Mexico respondents reported that it’s increasingly common for lawmakers and staff to inquire about the availability of evidence to support a program or policy, and a similar transformation in thinking is occurring in Minnesota. “This whole effort has started to inform everything we do in our legislating and budget process,” said Minnesota state Rep. Liebling. “We’re starting to see a culture change. When we don’t know how well things work, there’s more of an effort now to find out.”³⁶

Just as getting buy-in from those promoting evidence-based policymaking is crucial, so too is buy-in from the people implementing it, including agencies and program providers. These individuals collect and analyze the evidence, and often apply it—through funding recommendations, choice of programs to operate, or improvements to existing services, for example—to make more informed decisions. The absence of their support or participation can lead to insufficient dedication of resources to evidence-based policymaking efforts, or inconsistent and spotty execution.

One way for a central oversight entity to get buy-in from state policymakers and agency leaders is to foster relationships with them, maintain regular check-ins, integrate their input into the conversation around what evidence is collected and how it’s used, and invest in those relationships for years. Doing so has helped the LFC and MMB address concerns as they arise and encourage more coordinated use of evidence across government. Integrating stakeholder input requires reconciling different approaches to policymaking. Respondents in both states noted disagreements between various stakeholders over the best methods of data collection and analysis, the type of data that is important to collect or consider, and the value of evidence and data to the policymaking process more generally.

MMB annually surveys state and local decision-makers³⁷—including elected county officials, legislators, legislative staff, and state executive branch staff—to assess how much they understand, trust, and use the evidence that MMB collects and shares. The resulting metrics help the agency track how engaged its base of evidence users are. “The agencies and counties we work with are the ones that implement services. We can provide all the evidence in the world, but if they don’t act on it, [nothing is] going to get done,” said Weston Merrick, manager of MMB’s Impact Evaluation Unit. “Any success we’ve had has come through very strong, purposeful, long-term relationships.”³⁸

Compelling promotion of evidence uses and benefits

Minnesota and New Mexico have made concerted efforts to refine their messaging around evidence-promoting endeavors, tailoring it to ensure that specific audiences understand how to participate in evidence-based work, how using evidence can help them better execute their duties, and what value it can bring to the communities they serve.

Respondents interviewed for this brief recognized that improving

communication with crucial stakeholders—ranging from executive leadership to service providers—is neither a quick nor a straightforward process but a necessary one. It requires communicating information about the initiative to concerned parties in a manner that meets their needs and expectations. That messaging also needs to be tested and honed over time, as promoters of evidence-based policymaking efforts learn how best to demonstrate the value of the work to different partners. Some respondents observed that evidence is complex and that messaging around it is rarely simple. Merrick, manager of MMB’s Impact Evaluation Unit, noted that some of the most difficult concepts to distill into a short explanation involve evaluations—such as when randomized controlled trials are appropriate: “Learning how to quickly explain why we want this rigorous evidence and what we mean by evidence has taken at least four years. We’ll always continue to refine that language so that it’s accessible and we can convey it in a way that shows that evidence is not just some pie-inthe-sky thing in an ivory tower. Instead, it’s something that tangibly impacts the lives of the people we’re trying  to serve.”³⁹

Further, evidence-based policymaking activities—such as assessing evidence on program effectiveness or performance, identifying how to use that information, and ensuring that it affects relevant decisions—often take place in multiple entities in disconnected ways. To facilitate consistency and coordination, both states share data in ways that are easy to access and understand. MMB and the LFC make their nuanced and rich findings on program effectiveness, return on investment, and performance available to interested stakeholders through various means: public reports, scorecards and report cards, and online dashboards. Some respondents particularly emphasized the value of interactive or “live” data presentation tools, such as Tableau or web-based platforms, that enable policymakers and others to engage with up-to-date information.

Messaging around evidence also requires persistence. Charles Sallee, deputy director at the New Mexico LFC, said much of his state’s success with increasing the use of evidence to inform decisions is due to staff members’ relentlessness in presenting their analyses. That includes presenting the data in multiple formats and settings, to legislators, to other community stakeholders such as the Chamber of Commerce, and at national conferences. “There are no silver bullets,” said Sallee. “There are only incremental successes, and we just keep staying at it.”⁴⁰

Conclusion

Using evidence-based policymaking has enabled Minnesota and New Mexico to provide better outcomes for residents, improve the way research and evidence inform the budget and policymaking processes, promote investment in effective programming, and ensure that those investments achieve their goals. Their examples have shown that, when executed well, evidence-based policymaking initiatives can help states make more informed decisions for their communities.

Endnotes

1. Bureau of Economic Analysis, “Gross Domestic Product by State, Third Quarter 2020” (2020), https://www.bea.gov/data/gdp/gdp-state.
2. U.S. Census Bureau, “Household Income: 2018,” accessed March 17, 2020, https://www.census.gov/library/publications/2019/acs/ acsbr18-01.html.
3. U.S. Census Bureau, “Poverty: 2017 and 2018,” accessed March 17, 2020, https://www.census.gov/library/publications/2019/acs/ acsbr18-02.html.
4. U.S. Census Bureau, “Poverty Status in the Past 12 Months” (2018 American Community Survey, one-year estimates).
5. The Pew Charitable Trusts, “Fiscal 50: State Trends and Analysis,” accessed Sept. 3, 2020, https://www.pewtrusts.org/en/research-andanalysis/data-visualizations/2014/fiscal-50#ind1.
6. U.S. Census Bureau, “2010 Percent Urban and Rural by State,” accessed Sept. 3, 2020, https://www2.census.gov/geo/docs/reference/ua/ PctUrbanRural_State.xls.
7. Office of Governor Mark Dayton, “Governor Dayton Launches ‘Better Government for a Better Minnesota,’” news release, Oct. 20, 2011, https://www.leg.state.mn.us/docs/2018/other/181224/governor/newsroom/indexe7a5.htm.
8. Minnesota Management and Budget, “Minnesota Management and Budget,” accessed Aug. 5, 2020, https://mn.gov/mmb/.
9. The Pew Charitable Trusts, “Results First Initiative,” accessed Sept. 3, 2020, https://www.pewtrusts.org/en/projects/results-firstinitiative.
10. M. Frans (former commissioner, Minnesota Management and Budget), interview with Wilder Research on behalf of The Pew Charitable Trusts, June 20, 2019.
11. Laws of Minnesota 2015, Chapter 77, Article 1, Section 13.
12. Minnesota Management and Budget, “Definitions of Evidence,” accessed Aug. 5, 2020, https://mn.gov/mmb/results-first/definitions-ofevidence/.
13. Minnesota Management and Budget, “Inventory of Services,” accessed Aug. 5, 2020, https://mn.gov/mmb/results-first/inventory/.
14. Minnesota Management and Budget, “Benefit-Cost Analysis Reports,” accessed Aug. 5, 2020, https://mn.gov/mmb/results-first/ reports/.
15. Minnesota Management and Budget, “Minnesota Inventory,” accessed Aug. 5, 2020, https://mn.gov/mmb/results-first/inventory/.
16. Minnesota Management and Budget, “Using Evidence to Develop Minnesota’s Biennial Budget,” accessed Aug. 5, 2020, https://mn.gov/ mmb-stat/2-gen/enacted-budget-ebp-investments.pdf.
17. New Mexico Accountability in Government Act, NMSA 1978, Chapter 6, Article 3a, Sections 1 through 8 (1999), https://www.nmlegis. gov/Entity/LFC/Documents/Accountability_In_Goverment_Act/Accountability%20in%20Government%20Act%20Statute.pdf.
18. New Mexico Legislature, “Legislative Finance Committee,” accessed Aug. 5, 2020, https://www.nmlegis.gov/Entity/LFC/.
19. P. Lundstrom (representative, New Mexico House of Representatives), interview with Wilder Research on behalf of The Pew Charitable Trusts, July 17, 2019.
20. New Mexico Legislative Finance Committee, “Evaluation Unit Reports,” accessed Aug. 5, 2020, https://www.nmlegis.gov/Entity/LFC/Evaluation_Unit_Reports.
21. State of New Mexico Legislative Education Study Committee, “2019 Post-Session Review” (2019), https://www.nmlegis.gov/handouts/ALESC%20052919%20Item%202%202019%20Post-Session%20Review.pdf.
22. New Mexico Evidence and Research Based Funding Requests, SB 58 (2019), https://nmlegis.gov/Legislation/Legislation?Chamber=S&LegType=B&LegNo=58&year=19.
23. T. Liebling (representative, Minnesota House of Representatives), panel comments to event attendees, Pew-MacArthur Results First Initiative, Dec. 3, 2019.
24. N. Kovan (acting deputy director of child safety and permanency, Minnesota Department of Human Services), interview with Wilder Research on behalf of The Pew Charitable Trusts, June 27, 2019.
25. Minnesota Management and Budget, “Advisory Committee,” accessed Aug. 5, 2020, https://mn.gov/mmb/results-first/about/advisorycommittee.jsp.
26. Laws of Minnesota 2019, Chapter 63, Article 1, Section 7.
27. Minnesota Department of Human Services, “The Opioid Epidemic in Minnesota,” accessed Aug. 5, 2020, https://mn.gov/dhs/assets/federal-opioid-briefing_tcm1053-336378.pdf.