Forum on Inclusive Engagement Strengthens Evidence for Policy and Practice

Federal agencies can use inclusive evidence engagement—the production and use of knowledge through active collaboration among researchers, policymakers, practitioners, or communities—to generate actionable evidence, improve equity, and enhance the effectiveness of programs and policy. That was among the key takeaways from a June 30 forum hosted by Pew’s Evidence Project and the White House Office of Science and Technology Policy (OSTP).

The forum was the first of several events planned as part of the White House’s Year of Evidence for Action. In this first Evidence Forum, Alondra Nelson, a deputy assistant to the president who is performing the duties of the director of the OSTP, shared how inclusive engagement can be a powerful force multiplier for evidence-informed policymaking by shedding light on new solutions and untapped potential in communities.

The event also featured a panel of experts, including Erwin Gianchandani, assistant director for technology, innovation, and partnerships at the National Science Foundation; Claire Gibbons, senior program officer at the Robert Wood Johnson Foundation; John Lavis, co-lead of the Global Commission on Evidence to Address Societal Challenges; and Aleta Meyer, senior social science research analyst within the Office of Planning, Research, and Evaluation at the Administration for Children and Families (ACF) within the U.S. Department of Health and Human Services.

They discussed how and why federal agencies and other organizations can incorporate inclusive or active engagement to build and use evidence. Such engagement can help agencies change the way they work to advance equity in terms of who conducts research and who benefits from it and from government investments. It can also help researchers deploy additional methodologies that help determine the cultural feasibility of certain programs and aid funders seeking to invest in new partnerships and structures that incentivize inclusive engagement.

New approaches could result in more equitable research and policies

The panelists identified the following strategies for inclusive engagement to generate actionable evidence and improve equity:

Identify evidence needs early, including in the learning agenda process.

• Aleta Meyer said that ACF intentionally engages groups and individuals from the outset who are invested in the outcomes of research. She noted that the agency focuses on “what needs to be learned and let[ting] that question drive the learning process rather than saying we have got this set of tools, how can we use this set of tools and measurement to answer a question, in effect prioritizing the tools over the question.” This engagement helps the agency increase the relevance, cultural competency, and rigor of its evidence-building work and “improve the understanding, acceptance, and use of findings.”
• Meyer also urged close attention to terminology and language during engagement efforts. For example, the use of the term “stakeholder engagement” has a complex past for some Native American Tribes, and she has found that using the term “active engagement” can help ACF bring people to the table in meaningful ways.

Identify and verify how evidence contributes to equitable outcomes.

• John Lavis noted the importance of agencies involving diverse groups and individuals in both research and evidence-support initiatives to provide input on how research can be made more useful for their communities and to help interpret how evidence-informed policy or program decisions will affect them.
• Lavis provided an example from the Global Commission on Evidence to Address Societal Challenges of “expert panels that put people with lived experiences at the table [and] pre-circulate summaries of the existing evidence to those panelists.” They are then asked to make recommendations that align with both evidence and lived experiences.

Get creative about solving problems and boost innovation.

• The National Science Foundation’s new Technology, Innovation, and Partnerships (TIP) Directorate focuses on use-inspired research and innovation that gives rise to new technologies and addresses pressing societal and economic challenges by engaging potential users of research throughout the evidence-building process.
• According to Erwin Gianchandani, who leads TIP, the directorate “broadens the aperture of basic research, going beyond technology push to additionally emphasize market demand.” In other words, TIP strives to “bring the users and consumers of research to the table to help inspire and shape future research directions and to co-design and iteratively co-create solutions that can then be prototyped and piloted.”
• Engagement with diverse sectors and groups upfront can help agencies think through new technologies and solutions that will meet people where they are and much more meaningfully benefit a greater array of people.

Prioritize equity from the start and ensure the engagement is meaningfully supported.

• Claire Gibbons stated that the health field can’t achieve equity without supporting evidence-informed work that’s connected to communities. “It has become clear that it is probably not possible to achieve equity if we are not doing really strong evidence development work that encourages deep connections with community,” she said.
• The Robert Wood Johnson Foundation has doubled down on community-engaged research and altered its practices to meaningfully support it. This included changing the foundation’s submission processes for selected programs to include an assessment of whether community partnerships embedded in proposals are meaningful and long-standing and how much of a project’s budget is allocated to community partners.

Success will require action by numerous players

The bipartisan Foundations for Evidence-Based Policymaking Act and subsequent guidance directed agencies to consult with stakeholders to develop their evidence-building plans or learning agendas. The White House’s memorandum on scientific integrity, guidance from the Office of Management and Budget (OMB), and an executive order on racial equity also urged agencies to maximize inclusivity in building evidence to pursue equitable outcomes by engaging with a wide variety of stakeholders. Based on the event’s discussion, the following recommendations emerged:

Federal agencies should develop concrete strategies to engage a wider range of stakeholders to advance equitable data and evidence building.

• This Evidence Forum shared the example of the Biden-Harris administration’s Equitable Data Working Group, which actively engaged community leaders, academic researchers, and the public through weekly “engagement hours.” Practices such as this that hold space for active participation by diverse communities are an opportunity to ensure that evidence-based practices and policies advance equity as a central priority. Agencies should also consider other strategies to engage a wider range of stakeholders, such as fellowships for researchers to temporarily join government agencies, stakeholder mapping exercises to guide engagement during learning agenda development, and participatory research methods for specific evaluations.

Researchers can expand the types of methodologies used to examine the effectiveness of federal programs.

• OMB’s guidance on evidence development and use includes a broad definition of “evidence,” specifically “pilot projects, randomized controlled trials, quantitative survey research and statistical analysis, qualitative research, ethnography, research based on data linkages in which records from two or more data sets that refer to the same entity are joined, well-established processes for community engagement and inclusion in research, and other approaches that may be informed by the social and behavioral sciences and data science.” To help ensure equitable evidence-informed policies and practices, it often is necessary to deploy a variety of these methods.
• For example, ACF leads a Tribal Early Childhood Research program that convenes academic researchers and directors of early childhood programs in Tribal settings (such as child care, home visiting programs, and Head Start). Researchers in this group wanted to learn more about the reliability of early childhood screeners for these programs in Tribal settings. The program directors and other partners helped shift the focus to qualitative methods such as focus groups and interviews that further examined cultural feasibility and acceptability. This shift helped the group develop more meaningful measures of effectiveness that are now being used to examine other screening tools that ACF employs.

Philanthropy can invest in partnerships among government, communities, and researchers.

• Panelists in the Evidence Forum discussed the importance of public-private partnerships for strengthening the role that inclusive engagement can play in improving policy and programs. Philanthropic groups can often be more nimble than government and could help by supporting investments in pilot programs and other time-sensitive initiatives to advance evidence for equity. Philanthropy can also help support partnerships by funding dedicated staff (via the Intergovernmental Personnel Act or another mechanism); it can even support networks of partnerships to allow dedicated sharing and learning. Funders also might consider working to scale these investments by supporting governments and universities to reward inclusive engagement through more dedicated infrastructure such as new offices in state government to carry out this work or research incentive systems that encourage partnership models.

These resources can help improve research and policy

There are tools and resources available to agencies to help guide their active engagement efforts, including for learning agenda processes:

• The White House has been compiling examples and strategies at www.evaluation.gov to support agencies in this critical engagement work.
• Pew’s Evidence Project networks also offer a wide variety of resources, from the online partnerships resource bank mentioned above and the William T. Grant Foundation’s guidance about improving evidence use to Health Research BC’s Knowledge Translation Pathways site. That includes Health Research BC’s Action Catalogue, which can help researchers choose the right method for their projects.
• Pew’s Lenfest Ocean Program also has guidance questions to help researchers determine if their projects will address decision-makers’ needs. And the Pew marine fellows program developed a workshop and accompanying handbook to help researchers understand and capture the many types of impacts that engaged evidence building can create.

Angela Bednarek is the director of and Angie Boyce is an officer with The Pew Charitable Trusts’ evidence project. Alex Sileo is a senior associate with Pew’s Results First initiative.