Nonprofits Benefit From Building the Capacity to Monitor and Evaluate Their Programs

Think about how baseball has been transformed over the past two decades. Thanks in part to the book Moneyball and technological advances in analytics, general managers and coaches started to deploy data-informed approaches to scouting players and sketching out winning strategies. That paid off for many teams, including the Oakland A’s, the Boston Red Sox, and the Chicago Cubs. 

Now consider the nonprofit sector: Increasingly, nonprofits must demonstrate the impact of their programs to attract and retain funding. Data play an integral role in informing and influencing an organization’s strategies, which, in turn, can lead to positive changes for the individuals and communities served. For example, an after-school program for youth at risk of academic failure can track and use data on early warning indicators, such as school suspensions, attendance rates, and grade point averages, to target interventions provided to students. The most high-performing organizations are adept at monitoring and assessing their services in this way, but many nonprofits struggle to deploy effective approaches to evaluating their programs and using data for improvement.

As with professional baseball, the nonprofit sector faces a learning curve when it comes to effective use of data and evaluation. While the tools exist to understand what, when, and how to measure the results of their services, many organizations must build and hone the skills needed to make the best use of data. For example, a workforce development agency may know how to measure the number of participants it serves annually, but may not have a good understanding of how to gauge the impact of its various programs on employment and retention rates. Those who want to ensure that programs have a strong and lasting impact need to understand these end results.

This is why evaluation capacity building is so critical, and why the Pew Fund for Health and Human Services in Philadelphia has launched a 15-month initiative with 30 grantees to discuss and develop effective strategies for building this capacity.

In many ways, building these capabilities is how nonprofits practice—like baseball teams—to get better at what they do. It is not a one-time effort, but requires a continuous strategy that includes improving an organization’s ability to collect and use data to measure and learn.

These efforts typically focus on three key areas:

• Building evaluation skills. Organizations need to develop foundational evaluation strategies, understand how to collect and analyze data, and then learn to use that data for both internal improvement purposes and external functions, such as fundraising and advocacy. One approach to developing these skills is “Theory of Change,” which helps nonprofits identify and define their program models, the populations they serve and why, and the results that they expect to achieve. That process helps create a structured platform to begin measurement and learning efforts. For example, a nonprofit that serves individuals experiencing homelessness can use this method to identify the demographics of the target population and the likely results of the interventions provided. The organization then can collect and analyze data to answer questions such as: “Are we reaching our target population? Are some staff more effective in helping individuals reach their goals? And do some achieve housing stability more rapidly than others, and if so, why?”

• Strengthening commitment. Organizations need effective mechanisms to support their measurement efforts: from processes to collect data to appropriate staffing to technology systems that streamline analysis and reporting. This often requires financial investments and restructuring how leaders think about job roles and competencies. For example, an organization that focuses on reducing recidivism among formerly incarcerated individuals may work to integrate data collection tasks into the job responsibilities of case workers. These efforts also create opportunities for staff to understand and determine how best to use that data to inform how they work with their program’s participants.

• Developing a culture of learning. Organizations need to embrace learning as a value and a priority. In some cases, this can be more important than developing the skills and systems needed for measurement purposes. Cultivating a learning culture means that staff—from leadership to front line—understand and embrace the importance of collecting and using data. For example, an organization’s executive director may ask probing questions about performance metrics such as program enrollment, attendance, and retention at weekly meetings with her management team. An organization also could integrate monitoring and learning competencies into its employee performance review process so that staff members are held accountable on their ability to use data to inform decisions and their contributions to organizational goals.

The time it takes for nonprofits to build evaluation capacities varies, and it doesn’t happen overnight. Depending on their level of organizational experience and expertise, nonprofits may be able to strengthen their commitment to evaluation quickly, but need more time to develop a culture of learning, or vice versa.

The strongest evaluation capacity building efforts, therefore, are designed to meet organizations where they are, and do not take a one-size-fits-all approach. While building this capacity should not take as long as it took the Chicago Cubs to win their most recent World Series (more than 100 years!), it does take patience and dedication to develop and execute data-informed practices that lead to organizational “wins.”  

Kristin Romens is director of the Pew Fund for Health and Human Services in Philadelphia, and Meridith Polin is a senior officer with the program’s evaluation capacity building initiative.