Why do I cartoon about impact evaluation? Before I answer, let’s start with a few quotes:
“Blah, blah, blah, blah, jargon, jargon, jargon, blah, blah, blah, confusing chart, gigantic table, jargon, jargon, jargon, blah, blah, blah, any questions? Too bad, because this is a really long PDF and cannot be used for answering questions.” -The evaluation team who completed your last impact study
“Our organization is doing awesome! Here is a cool infographic with the most positive results we could find in the data. Now give us money please. Pretty please.” -The marketing team
“Seems like we’re doing pretty well.” -Board member that only read the marketing infographic because the report was really long and boring
Okay, maybe those are not real quotes. They’re more like fake “based on true stories” kind of quotes. But you get the picture, right?
Impact Is Tricky
Like so many other evaluation-related things, “impact” gets to be simultaneously a political buzzword, confusing academic jargon, over-simplified marketing speak, public lip service, and an indication of progress towards real and lasting change.
From the nonprofit perspective, the impact conversation can be an easy headache producer.
From a cartoonist’s perspective, it’s the perfect mix of confusing, political, controversial, and important.
But one doesn’t get into evaluation cartooning for the entertainment factor. There is one big reason why I like to cartoon.
Simple Is Difficult!
I spent the first 12 years of my career as a researcher and evaluator. Over the last seven, I’ve slowly evolved into my current role as a user experience designer, specializing in visualization.
These three beliefs have brought me to where I am at this point in my career:
- I believe that hollow rhetoric is harmful
- I believe in systematic evidence collection; precise analysis; and clear, comprehensive reporting
- I believe that everything important should be made simple and clear
So why is simple so difficult?
Hollow rhetoric is much easier to make simple than important evidence. And when evaluators do try to make their comprehensive work simple, often it just ends up feeling patronizing.
Enter cartoons! With cartoons, you get to forge simplicity and provide a gateway into a complex world. Of course, cartoons are also excellent ways to highlight issues around the barriers that too often prevent people from understanding evaluation efforts. Let’s check out some common issues around impact evaluation’s complexity with the help of some of my cartoons.
Academics Have a Way With Words
Unfortunately that “way” makes things like impact really difficult to understand.
Sometimes There Is No Control Group
However, that doesn’t mean you can’t pull together reasonable evidence that impact was influenced by a nonprofit’s activities. Work with what you have.
Robust Evidence Can Also Be in a Qualitative Format
You need to think like a detective. Don’t just look at the numbers; look at the timing, the context. Ask questions, watch, and listen.
Every Nonprofit Promises to Make Some Kind of Difference
There is an expectation to prove that you are, indeed, making a difference. The expectations are not always fair.
The Underlying Tension
If you are putting your blood, sweat, and tears towards a cause you believe in, it’s really tough when someone like an evaluator comes in from outside and doesn’t identify the impact you see in your work. That said, it’s often an evaluator’s job to be skeptical. That is their purpose, which is ultimately good for you. Why?
Bad, careless, unsystematic evaluations that show positive results are not good for anyone. Because when it comes down to it, it’s usually pretty easy to spot bad, careless, unsystematic evaluations, and that can affect your project’s fundability.
You may feel the difference between a mediocre project in your organization that has solid data and a fantastic project that has no data, but does the person who holds the purse strings? Staff passion isn’t enough to make up the difference when it comes time to communicate your impact to a funder.
When Impact Is Far in the Future
Focus on showing your progress towards your organization’s goals. Absent short- and mid-term progress indicators, people are left to make up their own narratives.
Definitions Are Everything
They can really change the perception. Try hard to stay consistent.
Contribution Not Attribution
Just remember, at the end of the day, impact is often a team effort. Sometimes it’s better to shift your viewpoint from attribution to contribution.
Impact Is a Conversation
Simplicity is achieved by focusing people’s attention on a specific idea.
If you can engage simply—through cartoons, through data visualization, through qualitative storytelling, through very simple presentations—you can then invite your audience to go deeper. Engage your team and other stakeholders and make it a conversation.
If any of my cartoons spark a thought, I’d love to hear it. You can visit me via my blog at freshspectrum.com or my online data design workshop at diydatadesign.com.
P.S. These are all my cartoons. Feel free use any of them in your work. You’ll also find a lot more royalty-free cartoons on my website.