How Counties Approach Evidence-Based Policymaking

Overview

Counties invest more than $550 billion annually in local communities to deliver essential front-line services, such as those integral to health care, public safety, and justice.¹ Charged with this responsibility, county leaders aim to provide the most effective services to residents while faced with significant challenges. Local leaders often have inadequate tools to make sense of existing evidence on public programs, or lack the data collection and oversight systems to track and optimize the programs they deliver. As a result, county leaders sometimes rely on historical precedent and anecdote when allocating resources rather than on data and evidence to ensure that those programs will generate positive outcomes for residents and make the most of limited resources

To help address these challenges, county leaders are increasingly turning to evidence-based policymaking— the systematic use of findings from program evaluations and outcome analyses—to guide policy and funding decisions. Targeting resources to programs with demonstrated value and tracking their performance and outcomes can help county leaders invest in cost-effective services, enable innovation, and strengthen accountability. Improved access to research on public programs and advances in technology have made it more practical than ever for leaders to use evidence to guide policy and funding decisions.

Counties are uniquely equipped to carry out key elements of evidence-based policymaking, including ensuring that effective programs are identified and successfully implemented. Their relatively smaller size and close relationships with service providers, agency leaders, and other stakeholders can help counties work to solve policy problems quickly and secure support, buy-in, and coordination for their efforts. In “Evidence-Based Policymaking: A Guide for Effective Government,” the Pew-MacArthur Results First Initiative provided an overview of the five key components of this approach. County-level research builds on that framework, provides strategies that local governments can adopt, and addresses some of the challenges facing communities that undertake these efforts. Counties are well positioned to learn from these experiences, replicate them in a manner suitable to their populations, and continue to improve upon them.

Applying the five key components of evidence-based policymaking to county decisions:

Program assessment: To make evidence-based decisions on what to fund, policymakers must first understand what programs they operate, how effective those are in achieving outcomes, and how they compare with alternatives based on national or local evaluations. This inventory gives lawmakers a broad overview of what is currently funded in their community, including predicted effectiveness. Salt Lake County, Utah, helped Kearns Metro Township compare its programs with the evidence base to see whether the services it funded matched the needs and problem behaviors of its youth—including high alcohol use and depressive symptoms—as identified by a state survey. This process helped Kearns identify unmet needs and gave policymakers confidence that their investments in these proven approaches were likely to generate positive results.²

Budget development: Instead of relying on anecdotes or previous years’ funding, policymakers can create processes to direct resources to programs that are most likely to succeed, based on rigorous research. These include requiring agencies to examine research on effectiveness before proposing new or expanded services, building evidence requirements into contracts, and establishing incentives to implement proven programs. For example, to execute and sustain evidence-based approaches to reducing recidivism, the Santa Barbara County, California, Community Corrections Partnership (CCP)—a group of representatives from all branches of the local criminal justice system—recently began requiring agencies to include detailed program information in their requests for funding. This includes evidence of effectiveness, anticipated measurable outcomes, and cost-benefit data where available. The CCP reviews this information to make an informed funding recommendation to the Board of Supervisors.³

Implementation oversight: When programs are not delivered in accordance with their intended design, they are less likely to achieve expected outcomes. Tools to assess individual and community needs and match them to services, along with processes to monitor delivery, can help counties ensure successful implementation. Hennepin County, Minnesota, uses the Correctional Program Checklist (CPC)—a standardized tool developed by the University of Cincinnati that measures correctional programs against recognized principles of effective intervention—to assess its services and enhance provider capacity for continuous improvement.⁴ The CPC, which uses 77 indicators correlated with reduced recidivism, can be applied to a wide range of practices. County stakeholders meet bimonthly to ensure that checklist assessors are reviewing programs consistently and with fidelity.⁵

Outcome monitoring: Monitoring systems that systematically track performance and outcome data can help policymakers determine whether government programs are working as intended, support continuous improvement, strengthen accountability, and inform budget and policy decisions. Key steps include developing appropriate outcome measures, refining systems to track and report them, and creating forums to share and apply outcome data. Montgomery County, Maryland, selected a set of indicators for its youth mentoring contracts based on national research and will require future contracts to assess at least one of the preapproved measures. The county expects the standardized approach to provide local leaders with a more systemic look at mentoring impacts, which will enable those leaders to make better decisions.⁶

Targeted evaluation: Program evaluations, which assess the impact of programs on a particular outcome, help leaders understand the effects of public services. Evaluations can be particularly useful when officials are considering which programs to scale up and which to scale back or eliminate. Facing a high frequency of alcohol-impaired driving, Outagamie County, Wisconsin, found inspiration in a neighboring county’s successful treatment and rehabilitation program for impaired-driving offenders. To ensure that the county could replicate that success, it collaborated with researchers at the University of Wisconsin, Milwaukee, to evaluate the program pro bono; in turn, the researchers could publish their findings freely.⁷ The evaluation found that the Outagamie County program resulted in a 31 percent reduction in drunken-driving offenses, leading to the county’s decision to continue its funding.⁸

Getting started

Local leaders in counties of various means and sizes have leveraged a number of resources—including outside expertise, additional local and external funds, and networks—to kick-start their work in evidence-based policymaking.

Conclusion

Evidence-based policymaking strategies can help counties identify and fund the most effective programs, ensure that they are implemented successfully, and monitor and evaluate outcomes so they are producing desired results. Although some evidence-based policymaking strategies can be challenging and take time, counties can take gradual steps toward incorporating data and evidence into their decisions and can learn from the experiences of other leaders. They can help build and sustain their efforts with certain approaches that support success across initiatives, including building internal support, starting small and scaling up innovations, engaging external partners, investing in capacity building for their provider organizations, and leveraging existing administrative data.

Endnotes

1. National Association of Counties, “Priorities in America’s Counties 2016: A Survey of County Officials” (2016), http://www.naco.org/resources/priorities-americas-counties-2016-survey-county-officials.
2. Caroline Moreno (education program manager, Salt Lake County Human Services), interview with the Pew-MacArthur Results First Initiative, Jan. 12, 2018.
3. Santa Barbara County Probation Department, “CCP: Criminal Justice Funding Opportunity,” accessed February 20, 2018, https://www.countyofsb.org/probation/crimjustfund.sbc.
4. Danette Buskovick (manager, Hennepin County Department of Community Corrections and Rehabilitation Office of Policy, Planning, and Evaluation), interview with the Pew-MacArthur Results First Initiative, Aug. 15, 2017.
5. Hennepin County Department of Community Corrections and Rehabilitation, “Community Corrections Act Comprehensive Plan 2016- 2017” (2016).
6. Matthew Nice (manager of planning, accountability, and customer service, Montgomery County Department of Health and Human Services), interview with the Pew-MacArthur Results First Initiative, Jan. 26, 2018.
7. Bernie Vetrone (director, Outagamie County Criminal Justice Treatment), interview with the Pew-MacArthur Results First Initiative, Oct. 11, 2017.
8. Tina L. Freiburger and Alyssa Pfeiffer, “Assessment of the ‘Safe Streets Treatment Options Program’ (SSTOP)” (2017), http://www.outagamie.org/Home/ShowDocument?id=49470.