In the movie “Apollo 13,” when it is clear that the astronauts are in serious trouble, flight director Gene Kranz in Houston tells his team, “Let’s work the problem, people. Let’s not make things worse by guessing.”
This warning about the dangers of guesswork puts into stark relief the importance of grounding critical—and sometimes lifesaving—decisions on facts and data. Congress got the message. In 2016, to help make sure that policymakers in the federal government act on the best available evidence, Congress created the Commission on Evidence-Based Policymaking to study and develop a strategy for strengthening the government’s evidence-building and policymaking efforts. A year later, the commission submitted its report to the president, the speaker of the House, and the president of the Senate. The commission made numerous recommendations to strengthen federal evidence-building capacity—including developing learning agendas, or a set of research questions and strategic approach, to support building and using evidence to address policymakers’ questions; improving the security and confidentiality of data; and aligning administrative processes with evidence-building activities.
In response to the commission’s report, Congress passed the Foundations for Evidence-Based Policymaking Act of 2018, which put into law many of the commission’s recommendations. The law did not get a lot of press attention—or any awards for a clever title. Nevertheless, the act may have done more to change the way federal agencies collect and distribute data—and develop policies based on that data—than any other recent legislative action. And that is a significant accomplishment.
The act is essentially an infrastructure bill. But instead of building roads and bridges, it builds a federal system for collecting and accessing data that can be used to develop evidence-based policies. It requires agencies to submit annually to the Office of Management and Budget, and Congress, a systematic plan for identifying and addressing policy questions. The plan must include, among other things, the data that agencies intend to collect to facilitate policymaking; methods and analytical approaches that the agency will use to develop evidence; and any statutory or regulatory restrictions that limit access to the data. Each agency must designate senior officials to coordinate evidence-collecting activities. And to make government data widely available to the public, the act requires the General Services Administration to maintain an online catalog that the public can use as a single point of entry—or one-stop shop—when searching for data that federal agencies collect.
Creating an infrastructure for evidence building, and collecting, organizing, and distributing data was the necessary first step for improving decision-making. But the next step—the one the rest of this article explores—is how decision-makers at all levels of government use data and evidence to inform policy. That may sound like a topic suitable only for statisticians, political scientists, and other number crunchers. But what government chooses to do—and not do—affects every American. From education funding to protecting medical privacy, and from what goes on a food label to deciding where oil can be drilled for on public land—government policy on all levels affects and sometimes determines how we live, work, learn, and communicate. And perhaps most important, many of the policy choices that legislators and officials make today will determine the quality of life for future generations. Will they inherit a healthier, safer, better educated, and more economically productive nation? The answer to this and countless other policy questions depends on evidence and the willingness of decision-makers to follow that evidence wherever it leads.
There is no perfect system for turning science and data into effective policies. But it is certainly accurate to say that decision-makers need the best available information relevant to the problem they’re trying to solve—and that scientists can provide that information through rigorous research that is accessible and usable by policymakers. The act is an important step forward, but the history of policymakers—including at the state and local level—working with the science community did not begin with the federal legislation. The Pew Charitable Trusts, for example, has a long record of bringing scientists and decision-makers together to identify the relevant policy questions and then collecting data and doing research that states have used to make evidence-based policies about issues ranging from public safety to consumer finance and ocean conservation.
In addition to Pew’s work, there are many real-world cases of science and data informing policy. Lawmakers wanted to reduce alcohol-related car accidents and death. Peer-reviewed research in the early 2000s across several states showed that decreasing the legal limit for drivers’ blood alcohol concentration to 0.08% cut traffic fatalities by about 7%. Using this data, some states moved ahead and mandated that anyone operating an automobile above the limit of 0.08% could be charged with driving under the influence. Meanwhile, at the federal level, there was no corresponding national mandate for what constitutes drunken driving. But Congress did use the same data to come up with a new policy. Through legislation, it tied federal highway funds to a state’s willingness to use the 0.08% benchmark. That incentive prompted all 50 states to adopt the limit.
Science does not always—or even primarily—inform policymaking with the mathematical precision that was available in the case of blood alcohol levels. Policy is usually made over much longer timelines, with a larger body of evidence, in an iterative process. That means developing a body of evidence—say, to improve a government service—that is adjusted, expanded, or reconsidered in subsequent versions as scientists continue their research and learn more about the problem they’re trying to understand and solve. In doing so, the research community ends up with data that has accrued over many years, along with a better understanding of what works and what doesn’t.
This may be especially true in the social sciences where research—in a step-by-step refinement of data—helps policymakers over time adjust programs and steadily improve outcomes. That’s where social policy research has been for many years and remains today: developing evidence-informed policies often using the same methodology of randomized control trials that biomedical and other hard sciences are well known for.
For example, there’s been a lot of research over the past 20 years trying to understand how best to train workers—with two goals in mind. First, to help workers gain employment, succeed in the workplace, and increase wages. And, second, to meet the needs of employers as the economy changes and becomes more global. To achieve these goals, there has been a significant change in the questions that social science research is trying to answer. Historically, the focus was on the worker—his or her skill set; ability to prepare a résumé and perform well in an interview; and how to succeed in the workplace. This was the supply side. Now attention is being paid to the demand side; that is, researching the optimal way to train workers so their skills and career track match the evolving needs of employers.
One approach to the demand side has been to connect the higher education system with the workforce training system and then develop a career path for specific sectors of the economy. This approach gives workers the technical training and experience they need—along with support such as child care—and the opportunity to earn an academic credential that puts them on a pathway to greater success. In the health care sector, a person can enter a training program to become a health care aide, join the workforce for on-the-job experience, and then enroll in higher education classes and earn an additional certificate or credential.
More than a decade of research, using randomized control trials—where some workers are in programs that connect academic training, work experience, and services, and others are not—has shown that this comprehensive approach can lead to better outcomes for workers because they’re more likely to get employed, stay employed, and earn higher wages. But just as important, they’re developing a base of knowledge that the national economy needs and that employers can rely on over the long term.
This research is changing the workforce training policies of state and local governments. But it is also informing federal policy. In 2014, Congress reauthorized the Workforce Innovation and Opportunity Act, which now requires workforce centers throughout the United States to collaborate with adult education and postsecondary education partners to build a career pathway for workers—not just teach a discrete skill.
In addition to workforce training, the federal government and some state governments are using financial incentives to move states toward evidence-based policymaking in several other programmatic areas, including education, home visiting, and teen pregnancy. One example is tiered-evidence grants, a funding mechanism that incentivizes the use of evidence-backed practices by tying the majority of funding to programs already backed by science. But the officials implementing these programs are also given the opportunity to innovate because the goal is to use the best current evidence while leaving room to experiment and build new evidence. Examples of tiered-evidence grant programs include the Department of Education’s Education Innovation and Research program; the Department of Health and Human Services’ Teen Pregnancy Prevention program and Maternal, Infant, and Early Childhood Home Visiting program; and the Department of Labor’s Workforce Innovation Fund.
The iterative process of building a large body of evidence through innovation, experimentation, reconsideration, and peer review that decision-makers use to create effective policies often takes years. However, there are times when the need for speed and innovation competes with the equally important priority of adhering to a methodical research process that ensures rigor and certainty. That’s what we’re facing with COVID-19. Today, scientists around the globe are sharing data and information as they race to learn about COVID-19 and possible treatment approaches. The science is progressing at a rare—and possibly unprecedented—speed. But this rapid pace of discovery highlights the challenge of balancing the urgency to develop treatments and a vaccine against the typically long timeline of rigorous research, which usually calls for coming up with a hypothesis, testing that hypothesis, vetting discoveries through a process of peer review, and carefully communicating what’s known and unknown to the public.
In the months since the COVID-19 outbreak was identified as a pandemic, the biomedical science community has been learning how to achieve that balance. In doing so, scientists are able to give policymakers the data they need to protect public health. And again, because this is an iterative process, as more data comes in, decisions about masks, testing, social distancing, community spread, immunity, antibodies, vaccines, and more can be refined. As for the need for urgency, that challenge is being addressed by increased collaboration among scientists, data sharing, and having multiple people from different fields working on the same problem.
Science informing public policy did not begin with federal legislation or the COVID-19 pandemic. It is a process that’s been evolving and growing for several decades in both the natural sciences and social sciences. So where does evidence-based policymaking stand now? There is certainly more data being collected and more ability to share and access that data. And coupled with this change is a bigger one: the ability to harness data through artificial intelligence, machine learning, and big computing. These trends and patterns began before COVID-19, but the pandemic has certainly made them easier to see. And here is another trend that is a very positive development for decisions informed by science: greater public interest in—and understanding about—how scientists conduct and ensure the validity of their research, and the impact that research has on the lives of our citizens and communities.
As in “Apollo 13,” policymaking cannot be based on guesswork. It is with relevant, rigorous research that is continuously refreshed as more data comes in, and then shared around the world, that we will find the best answers to our greatest challenges and build a safer and healthier future for generations to come.
The Takeaway
Molly Irwin is vice president for research and science at The Pew Charitable Trusts.
States of Innovation From 'After the Fact'
The Science of Policy
