Over the past decade, Texas leaders and key stakeholders have reduced rates of abuse and improved behavioral health outcomes for children and youth—in areas such as juvenile delinquency and truancy—with a focus on evidence-based and promising programs. To produce better results for the state’s more vulnerable populations, lawmakers enacted a measure in 2007 that put a priority on the use of approaches that are proved to be or that show signs of being effective.
But keeping the focus on evidence-based programs can be a challenging task. “Evidence-based” can have different meanings. For example, the state’s definition does not always match those in federal or private grant applications. Moreover, there are only so many programs that are considered “evidence-based,” because the rigorous evaluations required to test effectiveness take time and limited resources can make such processes unfeasible.
The Texas Prevention and Early Intervention (PEI) division—responsible for administering evidence-based programs for children and families—faced these challenges head-on in recent years and took a multipronged approach to prioritizing evidence-based programs, while allowing for flexibility and innovation in interventions. Among the key tactics have been:
The division encourages program providers to use national research clearinghouses to select services, but the pool of evidence-based and promising options often is not comprehensive, which means providers might not find a program that meets the needs of a specific population. As a result, some operate home-grown or other evidence-informed programs—services that have data supporting their effectiveness—that address specific community needs, but that have not been vetted by clearinghouses or undergone the extensive evaluations required to meet a strong evidence standard.
“There’s a desire to put resources in places where we know things are working, but there are not always a tremendous number of programs for every population and outcome,’’ said Donna Wood, the PEI division administrator. “We have a very diverse population here with diverse needs and wanted the protocol to be sensitive to that.”
To address the limitations of clearinghouses, PEI adopted its own evidence standards from the federal Community-Based Child Abuse Prevention Grant Program. These include four increasingly rigorous tiers of effective programming—Levels of Evidence-Based Practices—that consider types of evidence not included in some clearinghouses. For example, a provider’s program can be categorized under “Emerging Programs and Practices” if it encompasses a theory of change generally accepted in clinical practice, but not yet proved through high-quality testing. To move up to the most rigorous tier—“Well Supported Programs and Practices”—the program must have, among other things, undergone at least two randomized controlled trials published in two peer-reviewed journals. Adopting these tiered definitions enables the division to promote high standards, while allowing space for innovation and experimentation with new programs to see if they can make measurable differences helping hard-to-serve populations.
To supplement this effort, PEI developed a ranking tool to assess the evidence that supports various interventions. The tool is particularly useful for innovative programs built to address a population’s specific needs, but not yet available in clearinghouses. If a program does not meet a higher evidence standard, PEI works with the provider to track outcomes to ensure that the service is achieving positive results. A research roundtable, consisting of a cross-sectional team of evaluators from prominent research institutions, vetted the tiered approach and the ranking tool.
PEI’s use of multiple resources to verify what services are evidence-based provides greater insight than relying on only one. The approach advances the use of effective programs, while allowing for flexibility and innovation as the evidence is still being evaluated. As of fiscal 2016, 95 percent of early childhood programs and 60 percent of youth programs have some level of demonstrated effectiveness. At the same time, PEI and providers continue to work together to ensure that all programs meet the needs of the people they are intended to serve.
For more information:
Sara Dube is a director and Darcy White is an officer with the Pew-MacArthur Results First Initiative.