Measuring and Demonstrating Impact: An Interview with The Knight Foundation's Mayur Patel

Submitted by Annaliese on Thu, 10/11/2012 - 11:49am

[Editor's note: The following is an excerpt from an interview in the September 2012 issue of NTEN:Change, NTEN's quarterly journal for nonprofit leaders. Read the rest of article, and the complete issue on "Data-Driven" when you subscribe to the journal for free!]

Q: Tell us more about what you work on at The Knight Foundation.

A: As the head of strategy and assessment, I lead Knight’s efforts to understand the impact of our work so that we can use that feedback to better plan and improve our strategy. Overall my department has two goals: to provide timely and actionable insights to grantees and Knight program staff so that they can continuously iterate; and to share what we’re learning broadly to help move the field forward. Over the past year, we have been experimenting with infographics, data visualizations, stylized executive summaries, even cartoons to help spread our findings.

Q: Speaking of infographics, one thing that I think is particularly effective about the recent infographic from KaBOOM! that Knight Foundation helped to develop is that it seems to really tell a story. What struck me is that it demonstrates an outcome – not just the outputs that the organization accomplished, but the outcome that the organization’s mission works towards as part of its theory of change. Why is outcome measurement important, and how is an infographic like this going to help in that effort?

A: KaBOOM! had a great story to tell. For years KaBOOM! has been known for being very effective at getting volunteers to raise the money for and build a playground in the space of a few months (the output.) But the research found that through the KaBOOM! process, volunteers and organizations gain new skills, and often the people involved are inspired to do more for their community.

That’s the true community building “outcome,” of KaBOOM!’s work and that’s the information we wanted to get across. The infographic helped relate that story in an easily digestible format. We used the playground motif throughout to lend some fun to it. Not everyone is going to invest the time in reading a detailed PDF about the outcomes of a nonprofit’s program, but they might understand the highlights through an infographic – and more importantly share it. Infographics provide an easy and fun entry point into the discussion, and we find that they lead to higher engagement with the information on the web.

Q: One thing that I think we all know but don’t always follow-through on is that the point of data collection and evaluation work is to make decisions. Data is for decision-making, it isn’t data for the sake of more data, of course. How does a funding organization like Knight balance that when working with grantees? How do you help grantees not just collect data, but use data for better decision-making?

A: We start with the end in mind. That’s to say, we always have a “bias toward action.” We try and work with our grantee partners to measure things that they can actively manage, not things they can’t control. What’s measured has to be relevant and of personal interest to the grantees, not just used for reporting purposes.

In general, there are three considerations we look at when it comes to data collection:

a) the cost of gathering the data;
b) timelines, or how can we gather the information in order to give timely feedback that ties to key decisions, and
c) the capacity of the grantee to actually use the insights and make the needed changes.

On that last one, we only focus on the stuff that grantees or program teams are going to do something about. With so much information out there, it’s important to concentrate on what’s credible, simple and actionable.

I love this quote from Amanda Cox of the New York Times’ graphics team, which sums it up neatly: “Data isn’t like your kids. You don’t have to love them equally.”

Read this article in its entirety in the September 2012 issue of NTEN:Change.