Tag: metrics

Greenpeace’s Mobilisation Lab helps the organization transition into an era of people-powered campaigns. The right set of tools and an active social profile is helping Greenpeace to better support its community with campaigns that are community driven.

This case study was originally published along with a dozen others in our free e-book, Collected Voices: Data-Informed Nonprofits. You can download the e-book here.

NTEN: Tell us about how the MobLab fits into Greenpeace overall.

Michael Silberman (MS) and Wendee Parker (WP): We exist to help the global Greenpeace organization transition to a new era of people-powered campaigning shifting from Greenpeace-centric to supporter-centric campaigns. We’re working with staff in nearly 50 countries to design campaigns that enable the full power and potential of over 25 million supporters and activists to help us build stronger campaigns that win bigger. Our team has an independent budget to focus 100% on building capacity, challenging norms, sharing knowledge, and introducing new practices and tactics.

NTEN: Who are the Arctic 30, and how and why did MobLab get involved?

MS / WP: In September 2013, Russian security agents illegally boarded the Arctic Sunrise in international waters, seizing the ship and detaining all those on board at gunpoint. The ship was towed to Murmansk, and all those on board were locked up in cold, filthy cells, some of them in solitary confinement. They were charged with piracy and then hooliganism, crimes that carried lengthy prison sentences, because they dared to peacefully take action against destructive Arctic oil drilling and the onslaught of climate change, protesting at state-owned Gazprom’s Arctic drill platform in the Barents Sea. After 71 days in detention, the last of the Arctic 30 have been granted bail release, but severe piracy charges are still pending.

Some tools the MobLab provided to supporters of the Arctic 30

We got involved because there was a critical need to ensure that we were doing everything possible as an organization to help free these activists and leverage the global media spotlight to grow the campaign to save the Arctic. We added capacity to test new messages and tactics, and enable a global strategy brainstorm across offices and teams. Understanding how to effectively spread the messages by mobilizing new and existing supporters who connect with this cause through digital channels: thats what its all about.

NTEN: This has been a highly charged international incident. How have you baked principles of measurement and transparency into the campaign?

MS: We had to determine what could and should be measured. This campaign has been an opportunity to think about some of our limitations to measurement and tracking, and to have everyone really consider whats working and whats not.

WP: An informal group from several offices assembled for a week to take a look at our tools and platforms. It illuminated something many of us already knew: that consistency within digital engagement data was lacking. Trying to develop, implement, and execute a standard way to collect, track, and report on those digital efforts is an enormous challenge. The meetings gave us a good sense of our “universe” both the great effort our colleagues were already making in these areas, as well as opportunities to improve towards a complete, holistic point of view.

NTEN: Aside from this campaign, are there other wins you can pinpoint in these areas?

MS: There are over 100 active Greenpeace social accounts online. Were now seeing organizers include data analysis in their campaign planning. We at MobLab are still pushing, but it wouldnt get completely lost if we werent. Im also heartened by the fact that theres a lot of independent testing happening. People are using Optimize.ly for A/B testing, for example, and then reporting the results to everyone else.

WP: The focus and culture has definitely shifted, but the job is not done. Success would be having digital analysis (starting at defining digital analytic goals, implementing digital tracking and analytic tools for ongoing reporting, testing and optimization, ending with a complete campaign wrap up analysis) fully adopted as part of the overall campaign planning process.

NTEN: You mentioned Optimize.ly. Are there other tools that stand out as particularly helpful (or that you wish were more helpful)?

MS: We have issues with our bulk email tool, which doesnt make A/B testing as easy as it could be. On the upside, were making good progress with Google Analytics and Optimize.ly. On social analytics, were using Radian6, Topsy Pro, and Facebook insights.

WP: Greenpeace’s situation is so complex. In every office you may find a different setup for supporter data, a different set of digital engagement tools, etc. Even within offices, data can be fragmented among departments. I’m not sure theres a “one size fits all” solution, but as we work towards a common framework and toolset, it lessens the challenges towards complete supporter data integration a place where all departments view the same data and can have shared goals and metrics.

NTEN: Where would you like to see your campaign leaders a year from today with regard to systems and culture?

MS: We always want to see the four essentials of a people-powered campaign. The end is not putting data at the center of our campaigns; the end is more engagement-oriented organizing. We put people at the center of our campaigns, but data is an enabling tool. If we can use data to more effectively move people along and support our journey more deeply, thats a success point.

This case study was originally published along with a dozen others in our free e-book, Collected Voices: Data-Informed Nonprofits. You can download the e-book here.

When I joined the Communities of Impact (COI) program, I had been in my role of Digital Media Manager at Pathways to Education Canada for just a few months. Previously, I had served in a role as a front line youth worker, so the shift to a marketing department meant that I had a lot to learn along the way. This was evident as one of my very first questions was, Whats a KPI?

Specifically, one of the challenges I was trying to overcome was how to make better use of our web analytics. We had a lot of data, but for the most part, we weren’t looking beyond page visits and unique visitors. Other metrics, like goals and conversions, were not being tracked.

I have been told that one of the best ways to learn and improve is to surround yourself with people smarter than yourself. This was evident in the COI program as many of my peers came from strong backgrounds in managing data for their organizations of all sizes, and they helped paint a picture of what was possible when taking a data-driven approach.

A lightbulb moment occurred during a webinar with Amelia Showalter who detailed her experience as Director of Digital Analytics for President Obamas re-election campaign. What particularly resonated was her mention of vanity metrics and how it was more important to focus on conversions and goal completions rather than the number of visitors This changed the way that I looked at our web analytics.

There are a ton of resources available online on how to better use web analytics, but I never really looked into them because I wasnt aware of the kind of insights you could glean from them. A lot of information is available online to help make better sense of analytics and as well as search advertising and I also discovered free in-person training workshops that were being offered out of Googles Toronto office.

After setting up goals in our web analytics, we now had a much clearer picture of how people were navigating our website. We could identify the paths that people were taking on our site; tracking our goals and also tagging our URLs helped us better assess which platforms were performing the most effectively to engage our supporters.

Beyond this, the COI program sparked an interest in learning more and I found myself spending much of my spare time reading on how to better track web metrics and how to set up advertising campaigns. I connected with Data Analysts for Social Good, headed up by fellow COI participant Andrew Means, and had the opportunity to share a lot of my learning in webinars with other nonprofits. This past year, I have had the opportunity to teach a Digital Marketing class at a local college where I take a very data-driven approach.

Jason Shim spoke on this webinar organized by fellow COI Participant Andrew Means

At Pathways to Education, a renewed focus on web data also sparked a project to implement a tracking system to detail all of the general inquiries that are received via email and telephone. This project is now in progress and this will allow us to better track and categorize incoming requests to help better identify frequently asked questions and will help guide the redesign of our website to make information more easily accessible.

Looking back on the year, we’ve come a long way and we are taking a much more data-focused approach to all our digital initiatives. As we move ahead, focusing more on the data has helped us develop a clear framework and allowed us to make decisions more confidently.

What advice would you give to someone in a position like yours who wants to make their department or their organization more, shall we say, KPI-savvy?

For people who are looking to make their department or organization more KPI-savvy, I would suggest seeking out similar organizations who are doing great work with their data and connecting with them. This may take various forms, such as groups like COI, or LinkedIn groups, but its important to keep regularly communicating with others to receive feedback and coaching along the way. Finally, when you’ve learned a few things along the way, don’t forget to pay it forward and help others who are just starting out!

Find out how a web redesign helped Maine Conservation Voters move from managing voting record information on paper to an easily accessible system built into their web architecture.

This case study was originally published along with a dozen others in our free e-book, Collected Voices: Data-Informed Nonprofits. You can download the e-book here.

NTEN: Gianna, tell us about your work at Maine Conservation Voters (MCV).

Gianna Short (GS): MCV plays a critical role in turning public support for conservation into new laws to protect our air, land, water, and wildlife. I’m the Data and Communications Coordinator so most of my work is done in front of the computer. However, there are only four of us on staff, along with a couple of consultants and interns, so I end up doing all kinds of other things. Our budget is under $400,000 per year.

NTEN: How are you working to make your data more publicly accessible?

GS: We’ve been publishing an Environmental Scorecard for the Maine State Legislature highlighting environmental bills and votes since 1986. This is valuable information in politics, and without fail, when an election is approaching, reporters and campaign managers call MCV to ask for a particular candidates score on environmental issues. We literally have been pulling old paper copies of the Scorecard off the shelf and tallying up scores for different sessions by hand, which is cumbersome to say the least.

Making this robust dataset more accessible is a new challenge, but also an exciting opportunity. We distribute our Environmental Scorecard to 13,000, but believe it could be useful to many more people. It’s great data that is unique to our organization. We have a different tax status than most environmental nonprofits which allows us to publish this kind of information and really sets us apart. You can learn so much about a legislator by examining these votes through the years.

NTEN: Why tackle this now?

GS: We’ve been redesigning our site over the past year, so I’ve worked with our web developer to build an easily accessible way to house all of that data directly into the website architecture. Our national partners at the League of Conservation Voters also recently relaunched their website with comprehensive voting records. We’re looking toward that as a model for our site.

NTEN: What did you do with the data to make this happen?

GS: Each legislator has several votes per year, and many serve several terms, in both houses, during multiple different time periods. It can get confusing. We had to determine the best type of relational setup to use in order to make the data searchable and coherent. Our web developer ended up creating a pretty ingenious system over the last few months. It’s both versatile and simple to use.

NTEN: How long did this take, how much has it cost, and how will you measure success?

GS: We started brainstorming the redesign in the summer of 2013. The new site will be finished in December with a total budget under $4,000.

So far, we have scorecard data since 2011 up on the site, and it seems to be working well. Now it’s just a matter of data entry for all the preceding years, and quadruple checking for accuracy.

One way well gauge success is by using Google Analytics to see who is using the site and how they are interacting with our content. People tend to find us when they use search engines to look for Maine legislators. If this type of visitor then clicks on a specific bill page and reads about an issue, thats a success. If the visitor then takes action by writing an email to her legislator about the issue, that’s a huge success.

NTEN: Who else from your organization was involved?

GS: Our web developer Lauren Meir and I basically did the whole project ourselves. MCV’s office culture is built on trust, so I have almost total autonomy over the web realm. This is wonderful and terrifying at the same time, and has been a great professional challenge for me. I am starting to do some hallway testing with the staff and board members now that the site is up.

NTEN: You’ve been working hard to create a more data-informed culture at MCV. What advice would you offer to others at small nonprofits like yours?

GS: Learn what other successful nonprofits are doing with data, and present that information in an inspiring way to your coworkers. Show your office what these other organizations are doing better, and then offer to take the lead on trying something new. With a little intra-sector competitive spirit, and the knowledge that what you want to introduce has been tried and tested by others already, people can get pretty excited about new ideas.

Academy of Hope

  • 25 staff, operating budget of $1.5 million
  • Data management might seem mundane, but there’s a strong connection between it and direct advocacy efforts.

Jordan Michelson shares his successes building building effective data management systems, and shows that they’re not only important to organizational staff. It helps other to advocate for your cause!

This case study was originally published along with a dozen others in our free e-book, Collected Voices: Data-Informed Nonprofits. You can download the e-book here.

NTEN: Jordan, give us a snapshot of your work at Academy of Hope (AoH).

Jordan Michelson (JM): AoH provides a variety of programs and services around Adult Basic Education to meet the needs of adult learners in Washington, DC. We have a staff of 25 and an operating budget of $1.5 million.

We reached a huge milestone this year with a total of 55 graduates for the school year, our largest graduating class in history. And next year we’re going to evolve in a new direction as we launch a charter school, which we got approval for this spring. We have one year to put all of the details together, which gives us an opportunity to examine our program across the board, look closely at our processes, and determine what needs adjustment.

Before I was hired, most data work was focused on reporting, and was being done by various program staff at all different levels. Initially I balanced classroom instruction, program coordination, and administrative support with a new focus specifically on data and outcomes coordination. The latter has been a new opportunity for AoH and for me.

NTEN: What are some of the challenges you face in this role?

JM: As my position straddles programs and administration, its been challenging for the rest of the organization to understand my position and responsibilities. The more time I spend on programmatic issues, the less time I have to focus on our data needs.

When I started this position I made a list of goals that included launching a dashboard to evaluate how our classes were meeting the needs of our learners because I’d like to see us move from data used solely for reporting purposes to making data-informed decisions about our programs. I haven’t been able to push that needle yet.

Being in the nonprofit world comes with a certain amount of feasibility-checks; I’m sure that everyone on staff wants the things I just mentioned, but it may not be feasible to divert time and energy away from all of our other needs. It’s a tough balance, and it’s especially hard when you come to realize that this really good thing you want to do just isn’t a priority right now. But it’s important to remain optimistic, and know that the work you are doing is making a positive contribution to the organization’s mission.

NTEN: And you had proof of that recently! Tell us about your data win.

JM: I’ve been trying to link data to our organization’s mission whenever possible. One opportunity to drive the point home was when a student leader came to us for some information. She wanted to petition the city council to win funding for students to get to and from school, and she had some basic questions: How many of our learners in our student body receive bus tokens? How many face other barriers getting to AoH due to transportation?

We were able to provide this information quickly because of our student contact log. It’s a simple Excel workbook that’s kept on a shared drive where we keep track of every time a student calls to let us know they need to miss class. One field on the log is reason for absence. We were able to quickly look over the data from the term and the year, and come up with quantifiable numbers about how many students were facing these types of barriers.

Logging phone calls is not glamorous work and it doesn’t take a data hero to do something like that, but filling that log in consistently and actually looking back at it has the potential to make a big impact. And Excel is a system that everybody is able to use.

jordan_michelson_-_celebratory_meme_creaNTEN: Thats great! How did you celebrate it?

JM: I emailed all staff with a note of encouragement and affirmation. I wanted to help people see that even though this seems like a pretty mundane task, there’s a connection between them taking the time to fill in the log and a direct advocacy effort that really means something to our learners and community. People were excited; one coworker even turned it into a meme involving the Star Trek character, Data.

NTEN: How will you continue to foster a culture of data moving forward?

JM: I’d love to send all-staff emails highlighting our data wins on a semi-regular basis. I haven’t figured out a system for doing that, but that’s a next step.

Another exciting opportunity is on the horizon. We’re participating in a best practices meeting with other adult education providers in the DC area. I am hopeful that this will include data best practices and be a natural space to broach the topic of organization-to-organization data sharing or at least start having the conversation about what were all measuring, and how.

Data Analysts for Social Good

  • Breaking down data silos.
  • You don’t have to be a data analyst, but you will need to know how to collect and understand data.
  • You don’t have to use the best tools right away. It’s alright to say “This is the best tool for now.”

Andrew Means launched Data Analysts for Social Good in his spare time to address a need – a better understanding of how to use data not just to maximize inputs, but to show the importance of data to support organizations functioning more efficiently and effectively.

This case study was originally published along with a dozen others in our free e-book, Collected Voices: Data-Informed Nonprofits. You can download the e-book here.

NTEN: Andrew, you’ve spoken with NTEN before about your experiences with data at the YMCA of Metro Chicago. Now you work at Groupon and spend a lot of your spare time launching Data Analysts for Social Good (DASG), which offers webinars, a LinkedIn group, and an annual conference. Why did you start DASG?

Andrew Means (AM): I saw no one talking about data well. Fundraising analysts, marketing analysts, program evaluation people…everyone was so siloed. We were all using the same skills, underlying tools and methods, but applying them to different parts of our organizations. Data shouldn’t be siloed to one team or one person who pulls lists. The real power of analytics and social science research is that you can address a number of questions using the same kinds of tools and skills. And most organizations don’t know where to begin. We have very little human capital around this in the nonprofit sector although this has grown immensely over the past couple of years. DataKind and others are doing phenomenal work connecting data scientists to nonprofits, but the long-term solution is to have the next generation of executive directors, nonprofit leaders, and people entering the sector really understand these tools from the get-go.

NTEN: How are you creating a data-informed culture as you grow DASG and prepare for your second annual Do Good Data conference?

AM: The hard thing about starting an organization is that you have no data to begin with, so you have to create your own. I’m enough of an analyst to know my data points are really weak. But I try to use data as much as possible to generate content. I put out a survey in the early stages of planning the second conference, asking potential attendees what they want to learn. Now, as I line up conference speakers, I can look at that survey to make sure I’m delivering.

Another example: Every two weeks or so I send an email out to my list. I track click-to-open rates to make sure I’m giving people what they want, and sending these at effective times of day on the best days of the week. I used to believe that I should send all emails at 5:00 a.m. so that they’d be in my subscribers inboxes first thing in the morning. But when I paid attention to the numbers, I started to see a bit of a jump in opens if I sent them in the early afternoon.

I use a lot of free tools: MailChimp for email, Eventbrite for RSVPs, Google Analytics, and Google Forms. They’re fine for now. Thats something not enough people really consider. Its OK to say I have what’s necessary. I don’t want to use it forever, but it works for now and I’m moving forward. It’s worth dipping your toes in the water.

NTEN: What else should people keep in mind as they dip their toes in?

AM: We live in a world that makes it possible to measure so much, from apps that track what we eat, to Fitbits that track where we go. How do we allow these things to inform us but not control us? With that in mind, I ask myself: Is my community growing? How many people can I reach through social media? When are the best times of day to do that? Did this email outperform the list average? Its not super formal; I’m letting the data inform me, but getting the email out is more important than succumbing to analysis paralysis.

NTEN: That said, you are looking to grow DASG strategically. How do you see yourself professionalizing this organization? Is that the goal?

AM: DASG started as a happy hour 18 months ago when I sent out a few tweets. I have been surprised by its success. It’s easy to get caught up just doing the work of running a growing organization; I forget to step back and look at, say, the Eventbrite data from the past year which can help me analyze which webinars performed best. I want to standardize my email practices and create standard surveys for all webinars. I got a tremendous response when I surveyed the people who came to our first conference. So it’s about taking the time to collect the data but also to reflect on it. And for me, that’s about rhythms: taking the time weekly or monthly to reflect and plan.

NTEN: If you hired an employee, what rhythm would you want them to be in? What would you ask them to regularly report to you?

AM: Right now email is big. I’d definitely ask for regular reports on:

  • Revenue, since we have to make sure this is sustaining itself
  • Attendance at webinars and events
  • List growth for both email and LinkedIn

Where people on both the email list and LinkedIn are coming from geographically. In 2014, I’d love to do more events outside Chicago. I need to see where we have the highest concentration of subscribers.

NTEN: Why is it so important to you to create spaces where people can come together and talk data with their peers?

AM: Everyone is talking about data, but not in ways that will benefit us in the long term. Of course there are some organizations I really respect. But too often, analytics are used to maximize our inputs, not our outcomes. We use data to raise more money, attract more donors, and send effective direct mail campaigns. I’m not seeing data applied as rigorously to help us think about actually being better organizations. We need to step back and think critically about what we exist to do.

I love data as much as the next person. Give me an interactive map, a pivot table, even a plain old pie chart and I’m happy. But, there’s more to being data-informed and more to what we should demand of our data, right? When it comes to focusing on the right data, I like to ask myself these two questions.

Is data helpful if you aren’t using it?

We collect a lot of data. People join or renew as members (when did they join, how many times have they renewed, what dues level did they pay, where are they based, who are they…), people get our messages (on which channels, do they open or click, do they share the message, who are they and when did they engage…), people do things with us (webinars, tech clubs, online groups, offline groups, conferences…) and all of those things have their own data sets.

You get my point; before we even start to layer on information from the wider sector or filters to subdivide topics and categories, we have A LOT of information to work with. But, does it matter if we don’t work with it? If we don’t set goals and then create regular opportunities to review the data, does it matter that we capture it?

We can’t forget that to be data-informed as an organization and as decision-makers, we have to actually be informed by that data (see what I did there?). Establishing regular meetings or processes for metrics review helps position us to learn from the data we’re collecting and be better positioned to identify opportunities to improve.

Is data helpful if you can’t change it?

Why are you measuring or tracking something that you can’t (or don’t want to) impact? A common example of this that I hear often is with volunteer engagement. An organization has, for example, 20 volunteers, and they want to track the hours contributed, the impact on programs, and so forth. And the highest level metric that they lead with is that they have 20 volunteers, and they have 20 every year. But what they fail to explain is that they only have 20 volunteer roles. Unless you are going to open up the volunteer opportunities, I question whether reporting that you have 20 volunteers, at least as your lead metric, is helpful at all.

When looking at all of your various metrics, be sure that you are measuring things you want to impact and that you are focused on the data points you can and will change.

These are the reminders I find helpful but I’d love to hear from you! When your boss or your board or your intern ask you what you’re tracking and why – how do you talk about data?

It’s no secret: your organization needs to get better at data.

That’s a sweeping generalization, but I think it’s true. If you work at a nonprofit, or work with nonprofits, I bet you want to do more with data. I know that I do! (If you’re thinking “Nah, we’ve analyzed all our data and we’re done now,” please leave a comment so I can give you a virtual high-five.) Last year, NTEN put out a report on The State of Nonprofit Data that identified a lack of on-staff expertise as one big barrier to organizations making use of data. If you don’t have data experts on staff, then chances are, the best person to do that extra data analysis you’ve been dreaming of is… you.

Anyone can learn to do good analysis. Working with data doesn’t have to be hard, but it can be intimidating when you’re starting out. Fortunately, you don’t need a degree in statistics or the resources of the Obama campaign to start analyzing your data. All you need is the curiosity to ask the right questions… plus Excel or another spreadsheet software. If you’ve always wanted to learn data analysis, or if you feel like you need to learn it to do your job better, here’s how to start.

Before you analyze, gather your data.

Figure out what’s most important to your organization. Understanding your donors? Driving more advocacy? Measuring the performance of your emails? Whatever your priorities are, pick a handful of important metrics and start measuring them regularly, on a timeline that makes sense to you. Make this as easy for yourself as possible, so you’ll actually fit it into your busy schedule: write instructions, schedule reports to run automatically, and keep your reporting organized in one place.

At Food & Water Watch, we do high-level reporting every month to see how we’re doing. Is our fundraising on track? Did our email list grow like we expected? We have sections for list growth, fundraising, and advocacy with a few key metrics in each section, and the report looks something like this:


Get comfy with numbers

When we were kids in school, most of us were taught that math is hard and numbers are confusing. It’s not true, and you need to un-learn that lesson if it’s scaring you away from your data.

Spend time working with your data. Calculate numbers like open rates and conversion rates (the percent of your audience that does whatever you’re asking them to do). Get familiar with what’s “normal” for your organization, and what’s unusual.

A good analysis project is to develop internal benchmarks for your program. “Benchmark” is a fancy way of saying “average.” What’s your benchmark open rate? Click rate? Action/donation rate? Do these rates vary for different types of messages? Why might that be?

Remember, if you see something strange in an analysis, do a gut-check. It’s easy to make mistakes, especially in Excel. Could your calculations be wrong? Or have you discovered something wonderful and unexpected?

Dive in by asking good questions

Analysis is all about questions. You’re a detective trying to discover what’s hidden in your data. Start out with big questions about what you’re trying to achieve:

  • What are the best sources of new supporters for our email list? Can we capitalize on those sources to grow our list even more?
  • Are our activists more likely to become donors than other folks? What happens if we target them with more donation asks?
  • Why were our online donations really high/low last month? What should we do about it?

The best questions are both specific and useful – your data will give you information to answer those questions, and will suggest insights to help you improve your programs.

Then, gather whatever data seems relevant to answering your big question, and start digging in. Ask more specific questions of your data: Who donated, and when? What donation form did they use? Where did they come from?

Slice and dice the numbers by whatever characteristics will help you answer your big question. You can segment by donor or activist status, compare performance across issues or timeframes, and look for trends over time. Always keep an eye out for outliers – unusually high or low numbers that might skew your averages.

Use what you learn to inform new ideas for your program… or better yet, verify your findings by running some A/B tests on your next campaign.

Don’t bite off more than you can chew

One of the biggest barriers to analyzing our data is time. We’re all busy, and it’s hard to carve out the time for even the most essential analysis. Whatever question you’re investigating, make sure it’s one that will be useful to you… and that’s doubly true if it’s going to take up a lot of time.

The simpler the analysis, the more likely you’ll be able to produce meaningful results. If you’re frustrated by the analysis you’re tackling, try to simplify it: look at a smaller set of data, ask a slightly different question, or ask a friend or colleague for help. Don’t bang your head against the wall.

Remember, this is a learning process. The more you work with your data, the more skills you’ll have to take on bigger challenges down the road.

The Force is strong with this one…

Once you’ve mastered the basics and you’re ready for more, here are some ways to grow your skills:

  • Learn Excel, and learn it well. In particular, pivot tables and formulas will change your life. Find a class, or look for tutorials online. Learning a new trick in Excel could save you hours, even days of work down the road.
  • It’s not enough to analyze your data. You need to be able to present it to others in a way that won’t make their heads spin. It’s easy to do this badly. Study how others are presenting data, and learn from them. Especially, learn from the mistakes [http://wtfviz.net/ ] – if you can explain why these charts are hilarious, you’re on your way to being a good data analyst.
  • Read up. My favorite blog on data analysis is Avinash Kaushik’s Occam’s Razor [http://www.kaushik.net/avinash/]. He focuses on the business sector, but he’s brilliant and funny, and a lot of his lessons apply to nonprofits, too.

Got other tips on learning to work with data? Share them in the comments!

  • Portland, OR
  • 12 Staff

NTEN’s IT Director Karl Hedstrom has been working with the organization’s dashboard since he first joined as an AmeriCorps Volunteer six years ago. Since then, the dashboard’s role has gained prominence in NTEN decision-making, and currently the organization is working to enhance it further.

“This is an interesting time,” he said. “Up to now, we’ve run into an issue where we can track a lot of stuff but we can’t answer some questions like how much of an effect we’re having in the community at large, so were going through a process with the board and staff to try to figure out how to dig into those types of questions.”

To that end, the organization identified five different outcomes to represent NTEN strategies, paraphrased below:

  1. Technology is an essential part of a nonprofit organizations operations.
  2. There is an increased number of technology champions within nonprofit organizations.
  3. Those nonprofit technology champions are equipped to make well-informed decisions to help their organizations abilities to fulfill their missions.
  4. Conditions exist that support helping nonprofits using technology to fulfill their missions.
  5. NTEN is a sustainable organization with healthy financials, operations, human resources and cultural practices.

“Those were the five high-level things we wanted to make sure NTEN was doing,” Karl said. “We broke those down further to figure out what would indicate we were moving toward each of those outcomes.” By way of example, he said, for outcome number two, they asked such questions as:

Does participation of nonprofit staffers in the NTEN community grow over time? Are the number of staff members representing individual organizations growing? Are they representative of diverse organizational roles and abilities? Are nonprofit staffers taking part in an increasing number of NTEN services? What data do we have internally that would point toward this outcome happening?

“The next piece of the process,” Karl said, “was to take each indicator and try to define the specific data points that would speak to it.”

“We had an outside consultant walk us through figuring out those five outcomes, he said. Once we had the outcomes, a senior staff team sat down to determine the indicators and the specific data points and methods for collecting them. This is still an ongoing process. We’re trying to transition to new high-level strategic data points that can directly connect us to the outcomes were interested in.”

For some indicators it was more obvious than others, he said.

“Looking at the participation of nonprofits in the NTEN community and whether it grows over time, for example, we had to figure out how to show that,” Karl said. “What does participation mean? How do we measure that? Up until now, partly because of what data is available to us, the definition of participation was relatively narrowlike, people who had attended NTEN events. But that’s just a part of it. There’s also community discussion lists, report downloads, speaking engagements, and others. There’s a lot more we want to include in that definition to show overall participation instead of a narrow definition.”

Tracking that data allows NTEN staff to effectively measure it’s work. Karl said that one thing he learned through the process was that during the outcomes-identification stage it’s less important to worry about what you know you can track than it is to define what you want to know under ideal circumstances.

“It’s hard to not start digging into the details – how are we ever going to know that? – and set all that aside to think about what you want to know,” he said. “Once you have those outcomes you can dig a bit deeper, and once you have those indicators you can start digging into what data you have.”

Karl cautions other organizations that they will inevitably find that some of the indicators cant be measured.

“Once we started thinking about what we could collect, what we could do, what was available to us, and started to connect the indicators to the data points, we had some that we just had no way to do that,” he said. “For NTEN, the process is ongoing. Staff is still figuring out how to collect data points for each of the indicators and outcomes, including changing or configuring systems or establishing procedures to gather data, and finding the best way to present the data in reports and on the dashboard.”

“Once we have the data on the dashboard, it continues to be part of the strategic planning,” Karl said. “During our monthly strategic meetings and our weekly tactic meetings, we look at the dashboards and use that information that’s there. That’s why we collect it. It needs to be actionable.”

This case study is part of the research project in 2012 conducted by NTEN with the help of Idealware. See the State of Nonprofit Data report for more information about how nonprofits are–and aren’t–making data part of their decision-making processes, and the key challenges that affect an organization’s ability to be more effectively “data-driven.”

  • Lexington, KY
  • 5 Staff

How does an advocacy organization measure its effectiveness in a data-driven way? Rich Seckel, director of the Kentucky Equal Justice Center, admits it can be a challenge. It’s difficult to be in a multi-variant world trying to prove causality,” he said.

The poverty law organization advocates on behalf of Kentuckians in need, serving as a watchdog for the state legislature, keeping an eye on bills that are filed and lobbying lawmakers. Rich said measuring effectiveness through data boils down showing funders data that supports not just his nonprofits impact on policy changes, but on the lives of people who benefit from its victories.

In 2003, with the state under intense pressure from the federal government to cut Medicare spending, the governor announced a series of changes that would save $45 million but would terminate nursing home or home healthcare coverage for 3,300 disadvantaged Kentuckians. To challenge the cuts, KEJC enlisted the National Senior Citizens Law Center and sued the state on the basis that the measures were solely to save money and therefore illegal. However, they didn’t have data to support the case that the cuts were unjust.

The Medicaid cuts were fairly unpopular with the electorate, so during the next gubernatorial election both candidates promised “not to kick people out of nursing homes.” When a new governor was elected the following year, counsel for the state agency expressed interest in settling the case. In the settlement, the agency pledged a return to the earlier standards, to review all people who had been denied or terminated from long term care, and to report on the results. Using that data, Kentucky Equal Justice Center found that 97 percent of those who had been denied benefits or whose benefits had been terminated had been restored long term care coverage – about 3,300 of them.

Three of the 10 plaintiffs named in the lawsuit KEJC filed contesting the cuts died before the issue was resolved, demonstrating not just the seriousness of the issue but that access to the right data can make or break an advocacy groups case.

KEJC’s work directly improved the lives of more than 3,000 Kentuckians, but often the situation isn’t so clear-cut, Rich said. Just about every funder he works with wants KEJC to evaluate its success in human terms, which can be tricky though; policy successes can be easily measured, the results cannot. The data he needs to demonstrate these results often come from such external sources as state agencies, he said, and can be hidden under layers of bureaucracy. One way around that obstacle that has worked for him, Rich said, is to establish relationships with friendly legislators or state government employees who will share reports that never get released to the public, but which have the data that he needs to show his organizations impact.

As part of a foundation grant given KEJC to boost its infrastructure, Rich was required to meet periodically with a trained evaluator who helped him understand the science behind the statistics the organization was tracking. That statistical analysis training showed him how many different variants come into play in the organizations work, and the different ways they can and should be interpretedin other words, he said, it gave him a conscience about ascribing too much change to the efforts of his nonprofit.

“We don’t brag too much,” he said, but conceded that maybe the organization should work a little harder to publicize its measurable results. “We probably need to get better at that.”

This case study is part of the research project in 2012 conducted by NTEN with the help of Idealware. See the State of Nonprofit Data report for more information about how nonprofits are–and aren’t–making data part of their decision-making processes, and the key challenges that affect an organization’s ability to be more effectively “data-driven.”

  • New York, Los Angeles, and Washington, D.C.
  • 65 Staff (Full- and Part-Time)

The New York City-based Writopia Lab introduces children to writing and literature through a variety of programs, including writing workshops, a theater festival, and literary magazines. In addition to the organizations five NYC locations, it maintains two others upstate and branches in Los Angeles and Washington, D.C. Tracking participant data is critical to managing so many individual workshops and events for such a high volume of participants, and Writopia has covered some ground on the way to being a data-centric organization, but the geographical distribution of the staff is one of the hurdles it is overcoming along the way.

“We’re a gigantically distributed organization,” said Director of Operations Jeremy Wallace-Seagall. “That’s definitely one of the challenges we face. It’s difficult to coordinate consistently with the organizations staff members, both because of their locations and their scheduling,” he said. “There are 65 total staff, but not all work at the same time, as many of the organizations faculty are seasonal or part-time.”

Another obstacle is that the tools at his disposal are not necessarily the best for the job, nor do they all integrate well. Jeremy is doing what he can to take advantage of the different opportunities to collect and use data, he said, but to some extent he’s simply making do.

At Writopias New York offices, he uses a Microsoft Access database that he’s tweaked in unusual ways to meet his needs; a less than ideal solution, he said, despite his advanced modifications. “I’m an old database head, but one who got stuck in Access,” he said. “I like to say that I’ve done things with Access that very few people have done, but it’s still Access; it faces all the same limitations as any Access database.”

The organizations other locations dont share the database, an inconsistency which causes problems. To remedy it, Jeremy is considering switching to Salesforce across the nonprofit, and promised himself that by year’s end he’d have either standardized all locations onto Access database or rolled out a new Salesforce database.

Currently he’s looking for a database and website consultant to help consolidate and streamline data, which he believes will help with reporting and analysis – which in turn will help him manage growth.

“We’re growing every year somewhere between 45 and 65 percent, and there are always people saying, “Hey, can you come to my region?””he said. “I need to be able to analyze our growth and see how many workshops an average student takes and have all the registration and enrollment data in one place to see what makes sense.”

“We have outgrown our space on the Upper West Side, and we’re trying to analyze whether we should get a gigantic space here on the Upper West Side or a smaller space here and another one downtown or in Brooklyn or wherever,” he said. So we’re looking at where our clients come from, and similar people in those areas. If we go to the east side, will it be all this pent-up demand, or is everyone who might come to our workshops already coming?

Writopia Lab runs on a self-sustaining business model in which workshop participants pay on a no-questions-asked sliding scale pricing systemabout 50 percent pay full fee, 40 percent pay somewhere in the middle, and 10 percent get full scholarships. Though the nonprofit gets some grant money – about 10 percent of its total funding – currently none of the funders asks for reporting.

“That’s great in terms of being able to focus on getting things done, and not having to report in orange for one giver and yellow for another, but it means I have not taken this time to consolidate our data and come up with great ways of turning that data into information,” Jeremy said, adding that he plans to change that and hopes to focus on making use of that data in coming months.

“That’s what the next year is about personally for me,” he said. “Turning data into information. I’ve drafted dashboards and looked at various tools. We have a donor packet that we give out to people, and I’ve got some pretty reports in that. They’re all completely legitimate, but our analysis is not terribly robust. That’s sort of an artifact – we’ve been so busy serving clients that we’ve not had the time to chase after funding, or produce reports that would make it easy for us to drastically change the amount of institutional giving were receiving.”

“My hope and expectation over the next year is that well see that cycle twist, and the reports will come and tell a convincing-enough story that it will be easy enough to get in front of funders,” he said.

Writopias enrollment-tracking capabilities have not reached the levels Jeremy would like. Without a shared registration/enrollment/contact management database, information is spread out in a number of places and systems, which makes it difficult to use the data effectively.

“Part of it is a workflow challenge,” he said. “This sort of gets to the notion of loosely joining best-in-class systems, when my dream, really, is to have one piece of software that does everything in my life. I get the pitfalls of that I have reservations about having all my eggs in one basket, and there are other concerns; but having all your eggs in one basket makes it really easy to find them.”

Under the current system, potential clients have to visit separate web pages for each branch of the nonprofit to find their scheduled classes, a separate registration page for each branch that plugs into a Google spreadsheet, plus a pricing page and they have to remember all the information on each course they’re interested in, look it up on the pricing page, and register for it separately.

“I don’t care that I have to copy everything from the Google Docs to an Access database,” Jeremy said. “What’s brutal is what clients have to go through; that’s the piece that is most important to us. If I don’t get my website rebuilt, if I don’t get my Access db out to everyone, if I don’t get my Salesforce database installed, I have to find a way to make it easier for people to find and sign up for our workshops.”

He said hes been working to establish a data culture across the organization, which hasn’t always come easily. If you’re making a list, do it in a spreadsheet instead of a document,” he said. “It took me a while to create that sort of way of thinking, but I think people are pretty well on board with the notion that were collecting data, and whether or not were analyzing exactly this data at exactly this time, we want to be mindful of collecting the data in a clear and reasonable way.”

This case study is part of the research project in 2012 conducted by NTEN with the help of Idealware. See the State of Nonprofit Data report for more information about how nonprofits are–and aren’t–making data part of their decision-making processes, and the key challenges that affect an organization’s ability to be more effectively “data-driven.”