Do Health Impact Assessments Promote Equity?

Overview

Health equity, the principle that factors such as race, income, and geography should not determine health outcomes and that all people should have the opportunity to be as healthy as possible, is one of the five values that underpin the practice of health impact assessment (HIA). According to the Minimum Elements and Practice Standards for Health Impact Assessment, HIA is a tool to help communities and decision-makers consider how proposed policies, programs, projects, and plans could affect public health, and practitioners should systematically consider effects on health equity and provide recommendations to manage those effects.

To better understand the value and impact of HIA practice, the Health Impact Project, a collaboration of the Robert Wood Johnson Foundation and The Pew Charitable Trusts, contracted with Harder+Company Community Research to conduct a multiyear, two-phase study of HIAs exploring stakeholder perspectives and the extent to which HIAs contribute to changes in health determinants—environmental, social, and economic factors, such as housing or income, that shape community health and well-being. The Health Impact Project published findings from the first phase of the study in a February 2019 issue brief and as an article in the Journal of Urban Health.¹

For the second phase of the study, Harder+Company examined 62 HIAs that assessed decisions, across a range of sectors, that could affect three select health determinants: access to healthy food; employment; and safe, affordable, and healthy housing. (See the methodology for more information about the larger study.) This brief, the second in the series drawn from the study, focuses on the degree to which HIAs address health equity, affect decision-maker knowledge, and inform policy to advance health and equity, and it draws on that second-phase research. Specifically, the data for this brief come from:

• Questionnaires with 62 practitioners who led the studied HIAs.
• In-depth interviews with 13 decision-makers and other key stakeholders involved with nine of the HIAs, all of which were completed since 2016 or are currently underway.
• A review of documents related to the 57 completed HIAs in the study, including final reports, monitoring and evaluation plans, and news articles.

The latest findings show that the studied HIAs:

• Commonly and increasingly integrated health equity considerations at several points in the process but not at every stage. More than two-thirds of the completed HIAs studied included equity considerations.
• Frequently helped increase people’s knowledge about health and health equity and resulted in positive changes to decision-making that promote health. However, the HIAs did not always have this effect and were one of several factors considered by policymakers. 
• Often resulted, according to half of HIA practitioner questionnaires, in the incorporation of at least one assessment recommendation into the decision the HIA sought to inform, but neither practitioners nor decision-makers were able to document the effects of those implemented recommendations on public health.

What is a health impact assessment?

HIA isa systematic tool that promotes collaboration among communities and decision-makers—including elected and appointed officials and agencies involved in projects or programs that affect the public—to ensure the consideration of public health in decision-making. HIAs involve six steps: screening; scoping; assessment; recommendations; reporting; and monitoring and evaluation to identify and help address the impacts of decisions across various sectors, such as education, housing, and employment.²

Practitioners engage with members of the public and policymakers and employ qualitative and quantitative analyses, stakeholder input, and other research methods to determine how a proposal could positively and negatively affect environmental, social, and economic factors that shape community health and well-being. HIA teams also examine how health impacts are likely to be distributed among the population, especially high-risk groups such as seniors, children, and low-income families, and provide pragmatic, evidence-based recommendations to maximize the proposal’s health benefits and minimize preventable consequences, such as chronic disease and injuries.

More than 400 HIAs have been conducted across the United States over the past two decades by a range of organization types, including local, state, and federal government agencies, nonprofit organizations, academic institutions, and for-profit organizations.³ In the U.S., HIAs are typically initiated by government agencies, community-based organizations, or other stakeholders that recognize a proposal would benefit from the systematic consideration of its potential health effects. HIAs can be fairly quick, using a “rapid” or “desktop” model over a few weeks or months, or longer, employing either an “intermediate” approach using available data or a “comprehensive” approach involving primary data collection, both of which take several months to more than a year to complete.⁴

Several evaluations have found that in the near term, HIAs can illuminate the potential health effects on communities of policy and program choices, influence decision-making, increase knowledge about health impacts among decision-makers, and demonstrate the connection between health and sectors such as education, housing, and employment.⁵ However, the long-term impact of HIAs has not been well studied.

Snapshot of the HIAs studied

In the second phase of the Health Impact Project-commissioned study, Harder+Company examined 62 HIAs from 31 states. The assessments covered a range of decision-making levels: city or county (48 percent), neighborhood (27 percent), state (23 percent), and regional (2 percent). The HIAs examined plans (34 percent), policies (31 percent), projects (22 percent), and programs (13 percent) and were split fairly evenly across the three health determinants selected for consideration in the second phase: access to healthy food (32 percent); employment (31 percent); or safe, affordable, and healthy housing (37 percent). Half of the sample were the first HIAs conducted in the subject communities, and approximately a third of the practitioners leading the assessments had never done one before.

Among the HIAs studied, 44 percent were completed between 2009 and 2013, 31 percent were completed in 2014 or 2015, and 26 percent were completed since 2016 or are currently underway.⁶ The overrepresentation of older HIAs in the study sample was an intentional component of the research design, allowing for more in-depth exploration of effects on health determinants and equity after at least five years.

Health equity in the HIA process

The World Health Organization first identified assessing potential health equity impacts as a core value of HIA practice in 1999.⁷ In particular, the HIA process can help identify population groups that may be disproportionately affected by a proposed decision, such as seniors, low-income individuals and households, or people of color, examine the potential impacts on those groups, compare those effects with what is likely for the overall population, and develop recommendations to advance equity through the decision and its implementation.

For this brief, the research team adapted the Society of Practitioners of Health Impact Assessment’s Equity Metrics for Health Impact Assessment and created three categories to measure the extent to which each of the studied HIAs focused on equity: “low,” “medium,” or “high.”⁸ (See the methodology for more information.) The analysis found that more than two-thirds of the completed HIAs in the study sample included equity considerations. Of those, over half (55 percent) did so to a medium degree, but only 16 percent did so to a high degree.

More recent HIAs were more likely to consider equity to a medium or high degree than older ones. All HIAs completed since 2016 and 68 percent completed in 2014 or 2015 focused on equity to a medium or high degree compared with 61 percent of those completed between 2009 and 2013. This finding may reflect an increasing focus on equity within the field and in broader public discourse in recent years.

Additionally, all HIAs examining neighborhood-level decisions considered equity to a high or medium degree, compared with 59 percent of HIAs that looked at city- or county-level proposals and 55 percent of those assessing state-level decisions. Further, most comprehensive and intermediate HIAs (79 percent and 80 percent, respectively) also considered equity to a high or medium degree, compared with 46 percent of rapid HIAs. These results suggest that integration of equity considerations into the HIA process may be more feasible when the HIA is focused on a neighborhood-level decision, or when practitioners have time and resources to conduct a comprehensive or intermediate HIA.

Other factors, such as the type of decision each HIA sought to inform, how experienced the practitioners’ organizations were with the HIA process, and the community’s experience with HIAs, played smaller roles in the extent to which equity was considered.

In the interviews examined for this brief, decision-makers and other key stakeholders cited a range of examples of how they integrate equity into the HIA process. These included drawing on community members’ knowledge and perspectives in identifying the focus of the HIA and analyzing the distribution of health impacts for specific population groups.

Taken together, these findings show that the focus on equity within HIA practice has grown in the past several years but could be further enhanced.

Effects of HIAs on decision-maker knowledge

Prior research, including the first phase of this study, has found that HIAs can raise awareness among decision-makers of the potential health impacts of proposed decisions in sectors such as employment, housing, and education, which may help advance policies that positively affect public health and health equity across the United States.⁹

In response to questionnaires, 47 percent of practitioners who conducted the HIAs included in the study reported influencing decision-maker opinions to a moderate or great extent and 30 percent to at least some extent. However, given the limitations of self-reported data, such as the potential for practitioners to positively describe the effects of their HIAs, the researchers used the decision-maker interviews to help provide context and detail around the questionnaire responses. The interviews highlighted ways that HIAs can help increase decision-makers’ awareness of and knowledge about the connections between a range of sectors and public health. For example, one interviewee stated that the HIA “brought to light [an] area that I had not previously considered, that pretty much every project we do here could have an impact on health outcomes. I just realized that is something that we need to think about when we make our policy decisions.”

Further, the interviews suggested that this increased knowledge can have lasting influence beyond the specific decision examined, particularly by building relationships. Interviewees described how relationships built through the HIA helped them coordinate more closely with public health professionals in their day-to-day work and helped community advocates better engage in future decision-making processes.

However, interviewees also shared that factors such as political will and timing play a large role in the extent to which the HIA ultimately helps to shape policymaking. For example, one interview participant noted that a policy assessed by one HIA already enjoyed a high degree of political momentum and would have moved forward even without the HIA, which found potential positive health implications. Another interviewee stated, “The political wheels were already rolling. ... I think it would have been great if [the HIA] had happened earlier, and everyone’s time and effort could have been more influential in the political process.”

These findings support prior research indicating that HIAs can increase decision-maker awareness and knowledge about the connections between a range of sectors and health, but they also suggest that HIAs do not always have this effect. Notably, the interviews analyzed for this brief were disproportionately focused on more recently completed or ongoing HIAs in the study’s employment category, so the ongoing research, which will analyze more of the interviews conducted by Harder+Company, will be important to determine whether similar themes arose in conversations with decision-makers who participated in HIAs that are older or that focused on other health determinants. Previous research on the effectiveness of HIAs suggests that factors such as the timeliness of the assessment, the willingness of decision-makers to consider the information and recommendations, and clear, feasible recommendations can enhance an HIA’s impact.¹⁰

Effects of HIAs on decision-making and determinants of health

Research has shown that HIAs can affect decision-making outcomes. Two studies found evidence that 37 percent to 48 percent of HIAs directly influenced changes to the proposals they examined.¹¹ For example, a Midwestern city used HIA findings to determine where to place new sidewalks and how to prioritize sidewalk repair projects, and created a sidewalk allocation system that emphasized historically under-resourced communities.¹² Another HIA informed modifications to a proposed low-income senior housing property, including implementation of evidence-based features, such as particulate air filtration and sealed bay windows to minimize indoor air pollution, and a building entrance sited to reduce noise.¹³

However, responses to the questionnaires and interview questions demonstrate the complexities of documenting HIA impacts on decision-making and connecting the adoption of health-promoting changes to longer-term health and equity outcomes. Forty-four percent of practitioners of completed HIAs indicated that at least one HIA recommendation was integrated into the decision, but nearly half of the practitioners were not sure if any HIA recommendations had been adopted. Prior research on HIAs has demonstrated that practitioners and their partners rarely have the capacity to monitor and measure an HIA’s impact on decision-making or the effects of the implemented decision on health, often because of financial or human resource limitations.¹⁴

In general, the interviews analyzed for this brief did not offer much insight into subsequent changes in public health. Some interviewees did not feel enough time had passed, and others cited a lack of access to data documenting such changes or of knowledge about a specific outcome, sensitivity around speaking on the record, or the perception that other contextual factors—such as existing political momentum—played a larger role in the relevant outcomes than the HIA did. However, many of these interviewees did describe changes in decision-making processes in their organizations or communities that occurred since the HIA, which were influenced, at least in part, by the HIA process and have the potential to advance health and health equity, including:

• Implementation of implicit bias trainings for county government staff.
• More proactive and regular engagement of community members and community-based organizations in decision-making, including creation of a staff position within a government agency solely focused on community engagement.
• New processes to routinely consider the equity implications of proposed programs and policies, including use of an equity scorecard to evaluate processes such as public resource allocation.
• Development of advocacy trainings for community members to increase capacity to address social determinants of health.
• Improved collaboration among city or county government agencies and between decision-makers and community-based organizations.

Analysis of remaining interview data from the larger study will reveal information on any potential changes related to HIAs completed between 2009 and 2015 and will include more participants from HIAs in the food and housing determinant groups.

Practitioners’ relatively limited knowledge about whether recommendations were implemented, coupled with the decision-maker-reported lack of data documenting changes in health determinants, highlights the need for the HIA field to enhance the monitoring and evaluation step of the process. For example, HIA funders could provide longer-term grants that allow practitioners and their partners to track the effects of HIAs and implemented decisions over time. And policymakers could require or provide incentives to entities responsible for implementing decisions to track health effects over time and adopt monitoring recommendations from HIAs.

Conclusions and future directions

This research suggests that HIAs increasingly include health equity considerations, but that continued improvement is needed within the field to ensure all assessments focus on this core value. The findings are consistent with prior studies showing that HIAs can boost community members’ engagement in the decisions that most affect them, illuminate the connection between health and decisions in a range of sectors, and raise policymaker awareness and knowledge about the potential effects of their decisions on health and health equity.

Further, this brief demonstrates how broader contextual factors around decision-making play a substantial role in shaping policymaking and that, though HIAs can result in changes that promote health, they do not always do so. These findings underscore commonly cited challenges of evaluating the effects of HIAs, including the difficulty associated with disentangling HIA impacts from those of the implemented decision, in part because social changes can take years to manifest and may be attributable to myriad economic, political, or social influences.¹⁵

In the near future, the Health Impact Project and Harder+Company will complete the larger study, which will include additional data sources that will help explore the relationship between HIAs and changes in the three selected health determinants over time. These sources include publicly available data on food access, housing, and employment in communities where HIAs took place; a larger sample of interviews, including with decision-makers involved with older HIAs; and questionnaires to understand community members’ perceptions of changes in determinants of health over time and how the HIA may have contributed to those changes. Using the findings from this brief, as well as those that emerge from the full study, the Health Impact Project and Harder+Company will aim to identify actions practitioners, decision-makers, funders, evaluators, and other stakeholders can take to strengthen HIA practice and future evaluations.

Methodology and limitations

The second phase of the study examined 62 HIAs classified by three completion dates (2009-2013, 2014-2015, and since 2016 or currently underway); this approach allowed the research team to explore the effects of HIAs over time by asking different research questions depending upon the amount of time elapsed since the HIA.

With a goal of enrolling at least 60 HIAs in this study, Harder+Company sent an online eligibility questionnaire to the contact person listed in the Health Impact Project’s HIA database as of December 2017 for each HIA that had as its primary focus one of three health determinants—access to healthy food; safe, affordable, and healthy housing; or employment.¹⁶ The contact person—typically the practitioner who conducted the assessment—confirmed details about the HIA that helped Harder+Company determine eligibility for the study.

HIAs were eligible if they were rapid, intermediate, or comprehensive and focused on a program, policy, plan, or project implemented at the neighborhood, city, county, or state level. HIAs for the 2009-2013 and 2014-2015 classification groups had to meet two additional criteria: The assessment had to be complete and the assessed decision needed to have been made. Harder+Company used two online questionnaires to confirm eligibility and enroll HIAs in the study. In total, contacts for 195 HIAs received at least one questionnaire, and 99 responded. Based on those responses, 62 HIAs met the eligibility criteria and were enrolled. The HIAs were then classified into three categories—food, housing, or employment—based on the primary determinant of health examined.

The researchers then examined five data sources related to these HIAs for use in the mixed-methods analysis:

• Eligibility and enrollment questionnaires sent to HIA practitioners provided data that the research team used to examine the perceived influence of each of the 62 HIAs on the opinions of decision-makers.
• Documents related to the 57 completed HIAs in the study sample, including final reports, monitoring and evaluation plans, and news articles, provided information about the HIA process and, where available, changes to the decision the assessment sought to inform.
• Interviews with 44 key stakeholders involved with 30 of the HIAs offered insights on the perceived influence of the studied HIAs on decision-makers, community engagement, and decision outcomes.
• Publicly available data sources related to access to healthy foods, employment, and safe, affordable, and healthy housing offered evidence of relevant changes since the HIAs were completed.
• Questionnaires were shared with community residents for a subset of HIAs and explored residents’ perceptions about changes in access to employment; healthy food; and safe, affordable, and healthy housing and the extent to which the HIA contributed to those changes. 

The findings in this brief are drawn from three of these data sources: the practitioner questionnaires, documents, and a subset of the interviews: those conducted with people who had decision-making authority related to the proposals assessed by HIAs completed since 2016 or currently underway, which yielded 13 interviews related to nine of the studied HIAs, seven of which focused on employment.

In developing metrics, the study team adapted the Society of Practitioners of Health Impact Assessment (SOPHIA)’s Equity Metrics for Health Impact Assessment Practice to understand the extent to which each of  the HIAs in the study sample focused on equity.¹⁷ The SOPHIA metrics included six elements, one for each stage of an HIA.

For this analysis, the team revised SOPHIA’s proposed metrics and measurement scales to assess equity considerations at each stage of the HIA process. The SOPHIA element related to the analysis stage of an HIA was split into two items to better capture nuances about how the assessment examined the distribution of health and equity impacts, as well as whether the HIA included community knowledge and experience. As a result, this study assessed equity across a total of seven elements.

Using the adapted metrics, the research team reviewed the HIA reports and other relevant documents and assigned each HIA a score of -1, 1, or 2 for each of the seven elements, based on the extent to which equity was considered during each stage of the HIA process. Therefore, the possible range for the equity considerations metric was -7 to 14. The study team used this scoring system instead of SOPHIA’s 0, 1, or 2 approach to ensure that HIAs that left out equity considerations in one or more elements received a negative rather than neutral score because as a core value of HIA practice, equity should be considered in all phases of the process.¹⁸

This score range was divided into quartiles. Scores in the top quartile were categorized as considering equity to a high degree during the HIA process, those in the second quartile were categorized as a medium degree, and those in the bottom two quartiles were categorized as a low degree.

Limitations of this research include a small sample of interviews and the use of self-reported data, which can introduce bias that influences the accuracy or objectivity of respondents’ recollections or their understanding and interpretation of questions, as well as a low response rate for interviews and a self-selection bias that resulted in most interviews being focused on HIAs in the employment category. Of the 45 decision-makers contacted for potential interviews related to HIAs completed since 2016 or still in progress, only 13 participated (29 percent participation rate).

Among the factors that contributed to the low response rate were a lack of detailed recall about the assessed decision, changes in professional position, and geographic relocation. The response rate may also reflect the competing pressures on decision-makers’ time, which may have limited their ability to engage in in-depth discussions about specific decisions. The findings regarding effects on decision-makers and decision-making outcomes are based on self-reported data, which can introduce bias that influences the accuracy or objectivity of respondents’ recollections or their understanding and interpretation of questions.

Importantly, HIA practitioners and other stakeholders who contributed to the first phase of the study stressed that certain communities and populations, particularly those that face significant barriers to health, have historically been overburdened and exploited for research purposes. To avoid intrusive research practices, the study team collaborated with HIA practitioners through the second phase of the research to determine the best methods to obtain community members’ perspectives and planning to provide study data and lessons learned back to participating communities.

External reviewers

This brief benefited from the insights and expertise of Nichole Fiore, associate, Abt Associates. Neither she nor her organization necessarily endorses its findings or conclusions.

Endnotes

1. Health Impact Project, “Do Health Impact Assessments Promote Healthier Decision-Making? Findings From a National Study of the Perspectives of HIA Stakeholders” (2019), https://www.pewtrusts.org/en/research-and-analysis/issue-briefs/2019/02/do-healthimpact-assessments-promote-healthier-decision-making; E.K. Sohn et al., “Avenues of Influence: The Relationship Between Health Impact Assessment and Determinants of Health and Health Equity,” Journal of Urban Health 95, no. 5 (2018): 754-64, https://www.ncbi.nlm.nih.gov/pubmed/29948783.
2. For a detailed description of each of the HIA steps, see Health Impact Project, “Health Impact Assessment,” accessed Sept. 9, 2019, https://www.pewtrusts.org/en/projects/health-impact-project/health-impact-assessment.
3. Health Impact Project, “Do Health Impact Assessments Promote Healthier Decision-Making?”
4. National Research Council Committee on Health Impact Assessment, “The National Academies Collection: Reports Funded by National Institutes of Health,” in Improving Health in the United States: The Role of Health Impact Assessment (Washington: National Academies Press and National Academy of Sciences, 2011), https://doi.org/10.17226/13229.
5. E. Bourcier et al., “An Evaluation of Health Impact Assessments in the United States, 2011-2014,” Preventing Chronic Disease 12 (2015): E23; Center for Community Health and Evaluation and Human Impact Partners, “Community Participation in Health Impact Assessments: A National Evaluation” (2016), https://humanimpact.org/wp-content/uploads/2018/10/Full-report_Community-Participation-in-HIAEvaluation.pdf; C. Davenport, J. Mathers, and J. Parry, “Use of Health Impact Assessment in Incorporating Health Considerations in Decision Making,” Journal of Epidemiology and Community Health 60, no. 3 (2006): 196-201; F. Haigh et al., “What Makes Health Impact Assessments Successful? Factors Contributing to Effectiveness in Australia and New Zealand,” BMC Public Health 15 (2015): 1009, https://bmcpublichealth.biomedcentral.com/articles/10.1186/s12889-015-2319-8; Health Impact Project, “Is Health Impact Assessment Effective in Bringing Community Perspectives to Public Decision-Making?” (2016), http://www.pewtrusts.org/~/media/assets/2016/10/tce_brief.pdf; North American HIA Practice Standards Working Group, “Minimum Elements and Practice Standards for Health Impact Assessment, Version 3” (2014), https://hiasociety.org/resources/Documents/HIA-Practice-Standards-September-2014.pdf.
6. Percentages may not total 100 percent because of rounding.
7. North American HIA Practice Standards Working Group, “Minimum Elements and Practice Standards.”
8. Society of Practitioners of Health Impact Assessment, “How to Advance Equity Through Health Impact Assessments” (2016), https://humanimpact.org/wp-content/uploads/2016/11/SOPHIA_EquityMetricsV2_2016.11.pdf.
9. A.L. Dannenberg, “Effectiveness of Health Impact Assessments: A Synthesis of Data From Five Impact Evaluation Reports,” Preventing Chronic Disease 13 (2016): E84, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4951082/; Health Impact Project, “Do Health Impact Assessments Promote Healthier Decision-Making?”; Sohn et al., “Avenues of Influence.”
10. Dannenberg, “Effectiveness of Health Impact Assessments.”
11. Ibid.
12. Health Impact Project, “Do Health Impact Assessments Promote Healthier Decision-Making?”
13. B. Rogerson et al., “A Simplified Framework for Incorporating Health Into Community Development Initiatives,” Health Affairs 33, no. 11 (2014): 1939-47, https://www.healthaffairs.org/doi/abs/10.1377/hlthaff.2014.0632.
14. Sohn et al., “Avenues of Influence.”
15. Dannenberg, “Effectiveness of Health Impact Assessments.”
16. Health Impact Project, “HIAs and Other Resources to Advance Health-Informed Decisions: A Toolkit to Promote Healthier Communities Through Cross-Sector Collaboration,” The Pew Charitable Trusts, accessed Jan. 27, 2020, www.pewtrusts.org/healthimpactproject/toolkit.
17. Society of Practitioners of Health Impact Assessment, “How to Advance Equity.”
18. This differs from the SOPHIA metrics, in which elements are assigned a score of 0, 1, or 2. In this study, the lowest score for each of the seven elements was set at -1 instead of 0 so that HIAs that had some equity consideration across multiple elements received slightly higher scores than under a 0, 1, 2 scheme, while HIAs that left out equity consideration across multiple elements received slightly lower scores. This allowed the study team to differentiate HIAs that omitted equity considerations across multiple elements from those that had some, but minimal, equity considerations and would have previously received identical scores under a 0, 1, 2 scheme. Two of the seven elements differed from the -1, 1, 2 coding scheme. One element used a 0, 1 coding scheme that was recoded to -1, 2; the other element used a 0, 1, 2, 3 coding scheme that was recoded to -1, 1, 1, 2. In the dataset, HIAs ranged from -7 to 12 in their equity consideration score. No HIAs scored 13 or 14.