Indigenous Communities Need Program Evaluations That Reflect Their Health Priorities

Many Indigenous communities across the United States are implementing programs to improve members’ well-being using their unique traditions to promote healthy behaviors and emotional wellness. A recent virtual gathering sponsored by the Health Impact Project focused on the benefits of program evaluation methods that take into account such indicators of cultural and spiritual connection. Doing so can advance health outcomes and help in assessing program impact and opportunities to improve performance. 

Organizations such as the Notah Begay III Foundation (NB3) in New Mexico, which promotes Indigenous children’s health, are developing Indigenous-led evaluation methods that prioritize such metrics and address concerns about who conducts research in these communities. Their approaches can account for knowledge-sharing customs—such as storytelling—as well as issues important to Indigenous communities that are often not incorporated into Western evaluation tools.

For example, the foundation created and piloted a program evaluation framework based on its Indigenous health model, which considers the physical, mental, emotional, and spiritual factors that shape individual and community well-being. With a grant from the project, NB3 staff worked with the Zuni Youth Enrichment Project (ZYEP) in Zuni Pueblo, a small community about 150 miles west of Albuquerque, New Mexico, to evaluate project programming.

Because of community concerns about gaps in language that can limit cultural connection among Zuni youth, evaluators examined these links by calculating how many Zuni words children learned while playing in the program’s basketball league. This metric helped show families that the evaluation plan could capture aspects of health with significant meaning in their community. The work found that introducing Zuni words of encouragement correlated with an increased sense of personal achievement among the children.

At the virtual gathering, those taking part discussed lessons from the ZYEP evaluation and other efforts, along with strategies to support successful program and policy evaluations guided by Indigenous values and priorities. The more than 30 participants—including evaluators, health experts, community leaders and organizers, and academic researchers—focused on advancing and strengthening individual practice and building a collective field of Indigenous evaluation.

Speakers underscored the importance of new evaluation frameworks to help design effective policies that promote health and increase trust in the assessment process and practitioners. Some Indigenous people hesitate to participate in such program evaluations and doubt their findings—in part because past research projects failed to respect community values, needs, and traditions, and, worse, may have misused sensitive data.

For example, in 1990, an Arizona State University researcher collected blood samples from members of the Havasupai Tribe to investigate genetic links to diabetes, but then also used those samples to study schizophrenia and the tribe’s geographic origins—without members’ informed consent.

Those taking part in the convening highlighted several approaches to ensuring that assessments reflect the cultures and priorities of the communities studied. Participants generally agreed on a series of guidelines and best practices. For example:

• Presenting evaluation efforts as a storytelling process can help convey their value to Indigenous people who share knowledge in this way. Furthermore, community members should have opportunities to share their words directly through video testimonials or narrative writing.
• Data collected should ultimately benefit and belong to the community participating in the evaluation, and that community should have ultimate control over how the data is used and shared. Tribes may not be granted access to their own citizens’ data, which in some instances has been shared or published without their consent in the past.
• Evaluations in Indigenous communities should go beyond biomedical health measures to assess spiritual and cultural connections. For example, parents may place greater value on metrics less familiar to public health researchers, such as children’s development as leaders and responsible community members, than on common health markers such as body mass index.
• These evaluations also should consider how programs and policies influence land use and access, concepts fundamental to well-being, identity, and a core value of self-determination—the ability to self-govern and steward Indigenous land. For instance, forced displacement from ancestral lands often disrupted access to traditional, healthier food sources, such as fish, game, and produce, while creating dependence on processed foods supplied by the federal government. Disruption of practices related to obtaining and preparing traditional foods then resulted in a loss of a critical aspect of culture.

Convening participants want to advance Indigenous evaluation and prioritize the values of Indigenous communities by:

• Incorporating more Indigenous voices and expertise in national- and state-level decision-making so researchers can elevate Indigenous evaluation practice across specific issue areas.
• Acknowledging through evaluation the intergenerational and pervasive health effects of systemic racism.
• Shifting power from funders—typically federal agencies and non-Indigenous philanthropies—to Indigenous communities to design their own interventions.
• Supporting interventions and evaluation strategies that draw from community expertise and cultural values by including metrics meaningful to the community.
• Including interim measures that document progress toward longer-term changes and go beyond individual health statistics or behaviors to consider systemic drivers of health outcomes. Metrics such as diabetes or obesity rates can obscure complex health issues with many underlying causes. Funding and resources may be limited by the inability to demonstrate program efficacy toward long-term health goals when evaluations often assess programs over short periods of time.

To address these issues, attendees emphasized the importance of having and expanding the network of Indigenous evaluation experts.

Indigenous communities need meaningful frameworks for evaluating program effectiveness to guide decisions that affect residents’ health. Indigenous-led evaluation approaches make sure that local values and traditions are reflected in the methods used to collect evidence and that the research is done in collaboration with communities.

Bethany Rogerson is a senior manager and Stefanie Carignan is an associate with the Health Impact Project.