Over the past decade, The Pew Results First initiative has worked with 27 states to implement an innovative evidence-based policymaking approach that helps them to invest in policies and programs that are proved to work—ensuring that states moved over $1.1 billion toward more effective services. Although many states have made important gains in evidence-based policymaking, sustaining these efforts can be difficult. Turnover among leadership and staff, inadequate staff capacity to generate evidence, lack of political will to use evidence, insufficient buy-in from stakeholders within and outside government, and an absence of formal procedures between the executive and legislative branches can hinder this work.
To overcome these challenges and promote the sustainability of their evidence-based policymaking work, leaders across the country have engaged in cross-branch collaboration, a deliberate effort to create or deepen formal partnerships between executive and legislative branch representatives who use evidence to make budget and policy decisions. This helps to ensure that policymakers in these branches routinely prioritize evidence in the budget process, establish a shared commitment to and ownership of this work across government, and build an ingrained culture of evidence use throughout the decision-making process.
Results First has identified three strategies for improved cross-branch collaboration: 1. incorporating collaboration into law; 2. developing diverse advisory groups; and 3. establishing shared tools and processes. Informed by an online review of cross-branch efforts and 30 interviews with executive and legislative branch decision-makers (including legislators and staff, executive agency leaders and staff, and gubernatorial appointees), this issue brief provides a detailed look at how five states (Alabama, Colorado, Illinois, New Mexico, and North Carolina) have implemented the three strategies outlined above, including the challenges they faced and insights they gained.
The brief can serve as a resource for policymakers who are looking to advance and sustain the use of evidence in state government through cross-branch collaboration. Although all three branches of state governments perform important and distinct roles in determining and executing policy, this brief will focus only on collaborative efforts between the executive and legislative branches because they are routinely involved in overseeing the state’s budget development and implementation.
Evidence of program effectiveness describes the extent to which a program produces the desired outcomes—for example, whether a vocational training program leads to higher employment rates for participants. This evidence can inform the work of the executive and legislative branches of government, supporting legislative proposals and spending decisions and helping the executive branch determine its policy priorities and how to implement approved programs. Evidence can help inform leaders of more effective, cost-beneficial ways to invest in programs, particularly when it is applied in a coordinated manner across government.1
Executive and legislative branch offices involved in funding, generating, analyzing, and using evidence to make budget and policy decisions typically work independently of one another. This siloed approach can result in inefficiencies, duplication of work, and communication problems. Cross-branch collaboration eliminates silos by integrating evidence-based policymaking efforts into formal processes and regular procedures between representatives from each branch.
Both branches have separate powers and responsibilities within a system of checks and balances, but they also have a mutual interest in using data and research to develop a more effective government. Cross-branch collaboration can encourage leaders in both branches to articulate their goals for using evidence and commit to pursuing them.
An established commitment to evidence is important because current leaders are responsible for a broad range of responsibilities such as designing policies, fulfilling multiple statutory requirements, and juggling priorities as they determine what gets funded. It can also ensure that new leaders who join each branch with their own priorities are oriented to a broader set of shared goals and can help maintain momentum for evidence use as unexpected events shift attention to emerging areas of need.2
Cross-branch collaboration also enables state governments to create a common language around evidence, which is critical to the success of evidence-based policymaking. Leaders can then leverage shared goals and common language to build capacity and support for evidence-based policymaking among a broader group, including staff across and within the two branches, external partners, and the public.
Changes in administration, leadership, and staff can threaten the sustainability of government initiatives to use evidence to inform decisions. By codifying efforts into law, states can formalize cross-branch collaboration and ensure that any activities necessary to operationalize and maintain evidence-based policymaking continue even as the key individuals involved change year after year. Laws can also serve as a mechanism for bringing different stakeholders within state government to the table and building an expectation of evidence use within the state’s policymaking processes. Two states that have used this strategy—Alabama and New Mexico—are discussed below.
In June 2019, Governor Kay Ivey (R) signed legislation, Act 2019-517, creating the Alabama Commission on the Evaluation of Services (ACES),3 a bipartisan, cross-branch commission “for the purpose of advising the Legislature and the Governor regarding the evaluation of services, which may include evidence-based policymaking, within the state.”4 The 14-member commission is co-chaired by a representative from each branch and includes six members from the Legislature, six members appointed by the governor, and two additional nonvoting members (the state director of finance and the deputy director of the Legislative Services Agency Fiscal Division).5 Nonpartisan staff members carry out the work of the commission, including developing evaluation reports on the effectiveness of services within individual service areas.
Alabama’s pathway to passing this law took some turns. The state’s initial evidence-based policymaking efforts—including a cost-benefit analysis of adult mental health programs targeted at anxiety, depression, post- traumatic stress disorder, and severe mental illness and an inventory of programs and services in four state health and human services agencies—led it to formalize its efforts. The state first passed a resolution, Senate Joint Resolution 77, in 2018 urging 10 agencies to prioritize evidence-based principles and complete an inventory of programs (a comprehensive list of the programs a state funds in a particular policy area).6 The resolution also urged agencies to assess their programs’ evidence base (available research about a program’s effectiveness) as well as their annual costs and expected benefits.7 But although Alabama took positive steps to standardize evidence use, the breadth and one-size-fits-all approach of the resolution made it difficult to implement for the first time. Legislative staff members determined that they needed a narrower scope, greater flexibility for the staff charged with implementing the mandate, and stronger supports for offices involved in order to establish a more comprehensive, evidence-based approach in Alabama.
In contrast to the resolution, Act 2019-517 brought in the executive branch along with its legislative counterparts. This reflected sentiments from the governor’s office and other entities (e.g., the Alabama Department of Finance) that the executive branch should also be involved in guiding the effort, including identifying which areas of spending should be evaluated and what actions should be taken once the evaluations are completed. As ACES Assistant Director Patrick Dean noted: “We were still very much looked at as a … kind of watchdog [for the] Legislature ... [so the] governor’s office came over and said, ‘Look, if this is going to succeed, it’s got to be both sides.’”8
By showing an increased level of commitment to evidence-based policymaking from state leaders, the legislation also sends an important message to other stakeholders, such as agency staff members who will be asked to participate in ACES’ evaluation efforts. If this message fails to encourage agencies, the legislation includes reporting requirements to ensure agency cooperation. Finally, the law is less specific than the resolution about which programs across the state should be evaluated, giving ACES staff flexibility and allowing for an evidence- based approach that is appealing to a broader range of stakeholders.9
ACES initially faced skepticism. Othni Lathram, director of the state’s Legislative Services Agency, noted: “The first challenge was … winning over hearts and minds. This really isn’t an effort to save money. This isn’t an effort to cut services.”10 Once the legislation was signed, ACES staff members began to build buy-in through repeated conversations with agency employees. They worked early in the process with department leaders to set the right tone through repeated conversations, showing agencies that were nervous about unfunded mandates and potential cuts that evidence can also be used for improvement and to meet departmental goals.
After the governor and legislative leaders appointed their respective commissioners, ACES staff members worked with them to determine their first steps. By the commission’s third meeting, they decided to examine the issue of suicide prevention. Once the initial report on suicide prevention was completed with agency input, leaders from both branches met to determine how best to implement the report’s recommendations.11 They decided to start by securing, but not mandating, a lead agency for suicide prevention efforts within the state. The Department of Mental Health accepted the role, the Department of Education reconvened a task force to update its training policy for mental health professionals in schools, and the state updated its overall suicide prevention plan to cover additional populations and service areas. ACES staff members have now produced five reports12 on different policy areas and continue to provide detailed evaluations of these policy areas to the commissioners.
New Mexico codified its cross-branch evidence-based policymaking activities into law through the 2019 Evidence and Research Based Funding Requests Act (S.B. 58).13 The act has multiple provisions to ensure coordination between the Department of Finance and Administration (DFA), which oversees and administers the state’s budget, and the Legislative Finance Committee (LFC), which provides the Legislature with fiscal analyses, recommendations, and oversight of state agencies to improve the allocation of resources across departments.14 New Mexico is unusual in that both DFA and LFC propose annual budgets to the Legislature.15
The law requires DFA to develop budget instructions for agencies, requesting that they incorporate evidence into their budget submissions. These evidence-informed submissions then support the governor’s budget request to the Legislature. The law also ensures that DFA and LFC coordinate annually to select agencies to conduct program inventories, examining which programs are supported by research and warrant increased funding allocations, decreased funding allocations, or improvements.16 This effort expands the state’s Accountability in Government Act, which sought to clarify how the state would use outcome, output, and performance measures to annually evaluate programs as part of the budget process.17
In prior efforts to compile evidence about state-funded programs, policymakers realized that many agencies did not track enough program details (such as the specifics of what activities or services are being delivered through provider contracts) to inform their decision-making processes, including budget requests. “We were walking into these efforts with the assumption that we would be able to obtain a lot of the data that we needed to answer important questions about evaluating programs,” noted Jon Courtney, LFC’s deputy director for program evaluation—but that wasn’t the case.18 Both branches wanted to find strategies to encourage greater investment in research capacity and to show agencies the value of collecting and utilizing these types of data.19
S.B. 58 solidifies a process for the legislative and the executive branches to work together to encourage agencies to understand and use evidence to inform their budget requests. The law ensures that DFA and LFC jointly approve a list of programs to review as part of the annual program inventory process (DFA and LFC draft a joint letter to agencies indicating their interest). They also provide technical assistance as needed to support agencies that have not previously collected or analyzed this type of information. As Simon Miller, a budget and policy analyst at DFA, noted: “It’s really a team effort between the Legislative Finance Committee and the [DFA]. … They’ll come in with a proposal … [and] we sort of just whittle it down from there to decide which programs will be involved in the process.”20 Importantly, the law also includes several definitions of key terms used by both LFC and DFA. This ensures that terms such as “evidence-based,” “outcome,” and “output” are identical across the two branches and that all leaders and staff members in state government are using the same language—a key part of building a culture of evidence use in states.21
This process also empowers agencies to proactively use evidence to prioritize requests for programs that are likely to produce positive results. Agencies are encouraged to use information compiled from program inventories for their own purposes, such as requesting funding for new priorities, identifying service gaps, and acknowledging where spending is no longer needed. Aryan Showers, policy director for the New Mexico Department of Health, emphasized the value of shared expectations between the two branches: “We constantly strive to improve our methods of collecting data and measuring the effectiveness and impact of our programs. The relationship we have with the Legislature, where we have … this understanding that we’re going to be evidence-based and we’re going to have performance measures that are tied to evidence-based practices and performance tied to the budget … is very good for keeping [evidence-based policymaking] in the forefront.”22
S.B. 58 is already having a positive impact in New Mexico. Through the program inventory and budget request processes, the state has chosen to expand programs that work and, after discovering gaps in services and identifying other needs, to introduce new programs. The state has also discontinued certain programs when they have been found to lack evidence of success.
Leaders in each state highlighted the importance of working with agencies that have the capacity to implement processes for evidence building and use. It is important for centralized offices in both branches to secure resources and proactively support agencies charged with meeting any new objectives resulting from the legislation. In Alabama and New Mexico, where the laws empower central authorities to designate agencies to participate in program inventories, interviewees stressed the importance of selecting agencies with the capacity and expertise to handle the process of gathering data on their programs and services, especially during the earliest phases of implementing the new laws.
Interviewees in both states also cited the need to take time to develop new processes and patiently bring other stakeholders such as budget analysts or agency staff into the process. Once legislation passes, it can seem as if the staff charged with implementing its provisions must work quickly to maintain momentum. However, interviewees in both states argued that it takes time to ensure that all stakeholders understand what is required and why it benefits them. As Jon Courtney noted: “We actually had several trainings with DFA and with agency staff on [evaluation and performance metrics] ... It’s a sign that we’re all rowing in the same direction … when it comes to setting expectations for our work.”23
Cross-branch collaboration requires the participation of a broad range of leaders and staff from the executive and legislative branches. Input from external stakeholders, such as researchers and community-based organizations, can also bolster collaboration efforts. States can ensure collaboration from this broad group by developing a diverse advisory group to inform and support evidence-based policymaking activities, such as overseeing the processes used to generate, analyze, and report on evidence. Members of this group can also encourage leaders to use this evidence to inform decision-making in budget and other processes. Two states—North Carolina and Illinois—that have created advisory groups are profiled below.
After investing in evidence-based policymaking efforts for several years, North Carolina state leaders decided in 2017 to develop the Performance Management Advisory Committee (PMAC)24 to build support within the legislative and executive branches’ leadership and staff to ensure that the use of evidence was entrenched throughout the state. It does so by highlighting and disseminating best practices among agencies charged with gathering, analyzing, and using evidence. As the state’s budget director, Charles Perusse, stated: “We wanted to put together this advisory group that ... discussed how could we embed this important work in the DNA and day- to-day operations of state government.”25
Governor Roy Cooper (D) signed an executive order creating PMAC in May 2018.26 The order instructs the governor’s office to appoint at least 10 members to the committee; currently, it is made up of 13 members who are department heads, senior officials, legislators, and performance experts from the state’s universities.
Interviewees emphasized the importance of having non-appointee agency staff on the committee; this ensures that the advisory body is inclusive and allows the group to address potentially opposing viewpoints early on.
They also mentioned working to strategically invite appointees who already are members of key legislative committees and can help influence or advance PMAC’s goals in state government. The committee is chaired by the state’s budget director, and among its members is a bipartisan group of legislators from both chambers of the legislature. PMAC’s inclusion of university-based nongovernmental members further ensures a link to the academic community that can inform the state’s evaluation efforts. The order calls on the committee to advise the governor on performance measurement initiatives, proposed legislation or policies, and other strategies to increase evidence-based policymaking efforts and serve as an ambassador for these efforts throughout the state.27
Staff members from the Office of State Budget and Management (OSBM) provide administrative support to the committee. In this role, they have developed an orientation for new advisory members as they join the committee to help them embrace their role as ambassadors of performance management in state government. Feedback from PMAC has also influenced training opportunities offered by OSBM for agency staff members who may implement performance management best practices. For example, OSBM provides an annual training program to approximately 30 agency heads, deputy directors, and chief financial officers on best practices in evidence-based policymaking, including examples from state agencies, to help them ensure that their work with agencies was not purely a box-checking exercise, which can sometimes occur with evidence-based policymaking exercises.
Committee members meet several times a year to hear from agency staff, as well as external stakeholders such as researchers, to learn about best practices and discuss ways to promote them among state agencies. Because such practices originate with agency staff, recommendations are a bottom-up exercise from fellow agencies and not a top-down mandate from a central office or the legislature.
PMAC released a statement expressing its unanimous support for evidence-based policymaking in order to inform leaders and staff members of their work and show that evidence-based policymaking is a priority for the governor and legislature.28 Jenni Owen, director of the North Carolina Office of Strategic Partnerships, noted that it is an “example of how this bipartisan, nonpartisan ... cross-sector body can come to consensus” and described it as an important initial step for the advisory body as it “continues to be more proactive over time in its communication about its priorities and strategies for pursuing them.”29 In November 2021, the legislature further supported these efforts by appropriating funding for additional data analyst positions in state agencies and$500,000 annually in evaluation grants for agencies seeking out research partnerships with universities.30
In the past decade, Illinois has brought together a diverse group of legislators, agency staff members, and nonprofit leaders to improve the state’s use of evidence to inform its budget development and implementation of evidence-based programs and policies. Leadership in both branches wanted to solidify new performance-based budgeting initiatives so that funding decisions would be based on the merits of a specific program instead of the simple fact that they were funded in the previous budget. They also wanted to ensure alignment between the many different initiatives involving data and evidence that were already underway. Jennifer Butler, deputy director for accountability and results in the Governor’s Office of Management and Budget, said: “One of the main objectives that we have is to look with a critical eye at the programs that we are funding and … whether or not they’re in alignment with administrative priorities, the values and mission at the agency, and the overall focus of the state.”31 Too many data collection and management activities were siloed in individual offices within each branch, hampering coordination between leaders of each branch and causing an inefficient use of resources.
To resolve this concern, Illinois established its Budgeting for Results Commission (BFR) through Public Act 96-1529 in February 2011.32 The commission started with 15 members, but this number has fluctuated over the years. BFR staff members often attempt to strategically secure participation from legislators who focus on budget matters or serve on specific appropriations committees. Although all commission members are appointed by the governor, the commission is meant to include representatives from each of the caucuses, as well as other stakeholders representing the community at large, state government, and other groups. BFR staff members provided trainings and resources to commissioners on program assessments and other best practices in performance management before they began the commission’s work to assess program performance across agencies. BFR began as an unfunded mandate and relied upon support from the Governor’s Office of Management and Budget, but the legislature recognized its importance and provided funding for staff support in 2018.33
The commission provides extensive ongoing oversight of the BFR unit’s analyses. It has six to eight meetings per year, at which commissioners discuss recent cost-benefit assessments performed by staff.34 In addition, BFR produces an annual report and hosts an annual public hearing for a panel of outside stakeholders to comment on its work. As BFR Director Curt Clemons-Mosby noted: “We really didn’t have a holistic look at what increasing or decreasing those line items meant to the programs at state agencies.”35 To assess agency work, BFR assigns every line item within the state’s budget to a specific program and agency. That way, any assessed program directly correlates to an item in the state’s budget process should commissioners want to alter the budget item. Since BFR began, staff members have assessed programs in adult criminal justice, juvenile justice, and substance use disorder treatment.
State leaders in Illinois have relied on BFR for longer than the other states detailed in this brief have used comparable groups or legislation. During this time, agencies have consulted BFR’s reports to justify continued or expanded funding allocations in programs shown to have a positive return on investment. For example, the Illinois Department of Corrections used a BFR report in 2019 to justify an increase in appropriations for inmate education programs, including GED classes. BFR also regularly solicits feedback from executive agencies on mandates that
are outdated, ineffective, or otherwise do not support their work. This process has resulted in the modification or repeal of 257 statutes over the past decade.36 Over time, this process of program assessment and cost- benefit analysis has been embedded within the state’s budget decision-making process. More broadly, BFR has established a culture of evidence use throughout Illinois’ agencies as the commission highlights how important it is that state leaders understand the intended outcomes of programs.
Both states highlighted the importance of building alignment between advisory group members and demonstrating the value of using evidence early on. Stakeholders in North Carolina found it helpful to ensure that everyone was on the same page by discussing key definitions and terms early. The advisory group also focused on identifying and executing a series of “quick wins” to demonstrate their usefulness to other stakeholders in the state. This included a public statement identifying the advisory group’s priorities and goals to help clarify its intent when working with agency counterparts. One stakeholder noted the importance of timing, because starting the advisory group’s work early enough in the governor’s term has enabled the legislative and executive branches to work together and deliver a shared vision.
Of course, if an advisory group lacks dedicated staff to support its work, then it can struggle to fulfill its mission. Illinois’ BFR began its work without dedicated staff and resources, making it hard for the commission to take on larger projects initially. It later received additional staff after stakeholders realized a clear need for support. North Carolina’s PMAC also operates without a dedicated staff—hampering the group’s ability to undertake major efforts. A lack of agency capacity can also hinder these efforts. Advisory groups in both states were concerned because staff members often lacked the capacity to dive into their data. There was also frequent staff turnover, inhibiting training and capacity-building efforts. States have finally emphasized the importance of training and supporting incoming legislators, executive branch leaders, and other appointees serving on these advisory groups to help them fully understand the group’s mission, the ability of different state entities (such as budget offices or agencies) to use evidence, and key principles of evidence-based policymaking.
Even without formal laws and advisory groups, states have worked to erase silos between the executive and legislative branches by aligning and synchronizing the various tools and processes that the two branches use to embed evidence into the budget process over time. Developing shared tools and processes can ensure that leaders and staff in both branches have a common language and approach. Colorado’s use of this strategy is discussed below.
Colorado leaders have worked to ensure that evidence is talked about in a coordinated way in both branches. In 2018, Colorado budget staff, researchers, and policymakers created an “evidence continuum” as a framework to help decision-makers understand the strength of research supporting programs, and what resources they should consider investing in each program.37 The continuum illustrates five steps involved in building an evidence base, and the levels of evidence of effectiveness that correspond to those steps (from theory-informed to evidence- informed to proven). The effort to develop the continuum was collaborative from its onset. “It was developed by our Evidence-Based Policy Collaborative, which … included a whole array of different folks from different sectors: researchers, evaluators, educational institutions, agencies, governor’s office representatives,” said Mollie Bradlee, deputy director of the Office of Children, Youth and Families in the Colorado Department of Human Services. “I think that’s a really great place to start, just in terms of getting buy-in.”38
The Governor’s Office of State Planning and Budgeting (OSPB), which prepares the governor’s annual budget, uses the continuum to develop the spending proposals in the budget. The legislative Joint Budget Committee (JBC), which prepares budget recommendations for the General Assembly, prioritizes using cost-benefit analyses, evidence on programs’ effectiveness, and other data sources to inform decisions about the governor’s recommended budget items. Aligning definitions is a big shift for the state, because the OSPB and JBC had different evidence criteria to inform decisions.39
The OSPB and JBC have been working together over the past couple of years to align their definitions of evidence and to improve their communication about evidence, including information on the outcomes and cost-effectiveness of programs. The OSPB has maintained a regular dialogue with JBC members to understand their needs as budgetary decision-makers, and the JBC has relied on frequent communication with the OSPB to understand the criteria and content of program evidence presented in budget requests. Although the OSPB regularly uses the continuum, Robin Smart, principal legislative budget and policy analyst with the Colorado General Assembly, said the JBC needs some of the underlying information from the continuum in a different form. “We’ve started conversations about how the JBC and General Assembly can take the evidence and other information that’s provided in the budget requests and make it meaningful to help committee members know that an investment is right because it will result in meaningful outcomes. All of the work on using evidence in the budget has set us up to be able to ask questions about evidence of meaningful outcomes.”40
In light of these efforts, Colorado leaders have recently taken the next step in coordinating their efforts with new legislation. Legislators from the JBC created a bipartisan bill, Senate Bill 21-284, Evidence-Based Evaluations for Budget, which Governor Jared Polis (D) signed into law in July 2021.41 The bill establishes definitions for different levels of evidence (such as “evidence-informed,” “proven,” and “theory-informed” programs) for agency staff to weigh when analyzing a program’s evidence base, and it requires state agencies and the OSPB to use these definitions when describing evidence-based programs in budget requests. In these cases, the agency or OSPB staff must also reference additional information including any research that supports continuation, expansion, or discontinuation of funding of a program, and details on the role of evidence in development of the request. The bill requires JBC staff members to independently assess relevant evidence and consider such information in their recommendations to the committee. The JBC will then consider, as one of many factors, evidence-based information when determining the appropriate level of funding for a program.
After passage of the bill, the OSPB updated its fiscal 2022-23 budget instructions with those definitions, which align with the continuum and direct agencies to “ensure that any determination about the appropriate Continuum step aligns with the definitions specified” in the law.42 The shared terminology and directives will ensure that staff members understand each of the levels of rigor used by the budget office and the legislature and build a common culture of evidence use across branches and throughout departments.
Colorado stakeholders indicate a desire to continue shaping the continuum effort together through flexibility and compromise. As part of that, they are open to redefining their work. “My hope is that Colorado continues to be a leader in using data and evidence to inform our decision-making. The continuum is one piece of that work; it’s not, by any means, the only piece of that work,” said Aaron Ray, the OSPB’s former deputy director for policy. “If that means changes to the continuum [or] the information that is provided, that’s always something that we should be thinking about in conjunction with the legislature. … It never will be perfect. So, our goal would be to continue to work with the legislature to try to improve how Colorado incorporates data and evidence into our decision-making.”43
The five examples above demonstrate how states can fortify and help sustain their evidence-based policymaking efforts across state government. Cross-branch collaboration, the deliberate effort to create or deepen formal partnerships between executive and legislative branch representatives, is one strategy that states should consider when working to sustain evidence-based policymaking through administration changes, staff transitions, or setting of new priorities among state leadership. By incorporating collaboration into law, developing diverse advisory groups, and establishing shared tools and processes, states can ensure that evidence-based policymaking moves beyond the latest trend and becomes the standard for decision-making.