October 16, 2014

Measure what Matters

How can nonprofit organizations know what is working, what is not, and what impact they are making? They can achieve this understanding by implementing a realistic and meaningful evaluation system.

There was a time when monitoring activities, such as the number of people served or hours delivered, was sufficient to fulfill evaluation requirements. The nonprofit landscape has changed, and with it has emerged increasing pressure for nonprofit organizations to measure impact beyond just counting the numbers.

Evaluation provides a road map for how to collect data that is useful and meaningful. It is a valuable tool to help nonprofits navigate how to best serve their clients and measure the degree to which they are achieving their goals. The key benefits of conducting evaluation are continuous program improvement and accountability. When an organization is motivated to conduct an evaluation because it wants to make program decisions based on data, it creates a culture of evaluative thinking.

Nonprofit organizations can conduct a program evaluation in many different ways. Regardless of the approach selected, the following common elements are important considerations.

Define Success

Success is often defined as having a positive impact or making a difference in some way. Each nonprofit organization is unique and therefore needs to identify the specific goals to reach within the context of the organization’s mission.

In other words, what results does your organization expect to achieve in terms of influencing change in knowledge, skills, behaviors, and/or attitudes?

A common method to understand expected changes is to create a logic model. The WK Kellogg Foundation Logic Model Development Guide is a good resource to understand how to develop the model.  This tool summarizes and visually illustrates an organization’s program activities and their expected outcomes.

Collaboration is an essential ingredient in defining measurable outcomes that are a true reflection of what an organization wants to achieve. It is critical to have key stakeholders such as program, development, and/or marketing staff members participate in the evaluation design conversation. This inclusion ensures that everyone’s data needs are met and consensus is achieved regarding what elements are to be measured.

Determine the Process

Once an organization has defined success, the next step is to determine a realistic data collection process. Many methods can be used, including surveys, focus groups, and observations. How an organization chooses to collect data will be influenced by available resources. It is critical to understand what is feasible to collect as one of the most common pitfalls in the evaluation process is to attempt to collect more data than what is realistic to gather and use.

Identifying how the data will be managed, stored, analyzed, and reported are essential decisions that must be made for a successful program evaluation. Prior to beginning the evaluation process, it is critical for organizations to develop a database that will allow for effective storage and management of the data and to select someone with the necessary skills to analyze and report the data collected. Organizations sometimes realize too late in the process that they lack staff with the time or ability to collect, manage, analyze, or report the data.

A key to successful program evaluation is to start small, be successful with one data collection method, and then expand these efforts to incorporate additional data collection tools.

Use the Data

A successful evaluation process is one in which the data are used to make program decisions. One way to accomplish this goal is to conduct data discussion meetings. In these meetings, staff review evaluation results and discuss how they can relate to program decisions.

Accountability is another reason to conduct a program evaluation. Those that invested in your program should be informed of the evaluation findings. A separate communications plan on how to disseminate findings to different groups is a simple and important step. For example, evaluation findings may be shared with the following groups and answer questions that are relevant to each group:

  • Staff members: What do the data mean for our organization and to our roles?
  • Board: How do the data inform the big picture and our strategic plan?
  • Donors: What 3-5 key findings can you report to donors via email or letter?
  • Participants: Thank participants for being a part of the evaluation process. How do the findings impact the program activities, and the services they receive?

In summary, by collaboratively defining success, identifying a feasible data collection system, and using the evaluation findings, organizations increase their chances of improving their internal systems as well as making the impact they strive to achieve. Successful program evaluation empowers nonprofit organizations to improve decision-making and offer the best possible programs.

 

Like what you're reading?
Sign up to receive the latest articles and updates on nonprofit tech from NTEN and its community of experts.

Subscribe

Chari Smith
Chari Smith, founder of CRSmith Consulting, believes evaluation should be accessible, practical, and usable. She is experienced in evaluation design, instrument development, data collection, quantitative and qualitative data analysis, and findings dissemination. She has taught several workshops to help nonprofit professionals understand the value and use of program evaluation. You can email her at chari [at] crsmithconsulting [dot] com.
Interest Categories: Data, Evaluation, Program
Tags: Evaluation, program evaluation