Defining Levels of Evidence

Policymakers in a growing number of jurisdictions are using evidence to help inform their decisions about which public programs to fund. In general, they can achieve substantially better outcomes for their constituents by directing limited resources to programs that have been rigorously evaluated.

To promote the consistent use of this information across agencies and branches of government, several states have established formal definitions for levels of evidence. These definitions acknowledge that available data on programs’ effectiveness often vary by both the rigor of the underlying research and the number of studies that test outcomes. Creating formal definitions of these levels of evidence provides a common language for discussions about programs’ demonstrated effectiveness.

The Pew-MacArthur Results First Initiative reviewed states’ legislative and administrative language related to levels of evidence, identified the best examples, and created the following definitions based on them.

1. An “evidence-based” program or practice offers a high level of research on effectiveness, determined as a result of multiple rigorous evaluations, such as randomized controlled trials and evaluations that incorporate strong comparison group designs, or a single large multisite randomized study. These programs typically have specified procedures that allow for successful replication.
2. A “promising” program or practice has some research demonstrating effectiveness, such as a single randomized controlled trial or evaluation with a comparison group design, but does not meet the full criteria for an evidence-based designation.
3. A “theory-based” program or practice has been tested using less rigorous research designs that do not meet the evidence-based or promising standards. These programs and practices typically have a well-constructed logic model or theory of change.

Policymakers can refer to these definitions to create a shared understanding of evidence across agencies and branches of government and, over time, increase its use in the budget and policymaking processes.

Download the full fact sheet.