Tag: case study

Peter Campbell of Legal Services Corporation shares his biggest data fail, and what he’d do differently now.

  • If I’m the IT Director, I can’t be the developer.
  • Get organizational commitment.
  • Don’t compromise on a vision for expediency’s sake.

This case study was originally published along with a dozen others in our free e-book, Collected Voices: Data-Informed Nonprofits. You can download the e-book here.

Note: names and dates have been omitted to protect the innocent.

Years ago, I was hired at an organization that had a major database that everyone hated. My research revealed a case study in itself: how not to roll out a data management system. Long story short, they had bought a system designed to support a different business model and then paid integrators to customize it beyond recognition. The lofty goal was to have a system that would replace people talking to each other. And the project was championed by a department that would not have to do the data entry; the department identified to do all of the work clearly didn’t desire the system.

The system suffered from a number of problems. It was designed to be the kitchen sink, with case info, board updates, contact management, calendaring, web content management, and other functions. The backend was terrible: a SQL database with tables named after the tabs in the user interface. The application itself had miserable search functionality, no dupe checking, and little in the way of data quality control. Finally, there were no organizational standards for data entry. Some people regularly updated information; others only went near it when nagged before reporting deadlines. One person’s idea of an update was three to five paragraphs; another’s two words.

I set out to replace it with something better. I believed (and will always believe) that we needed to build a custom application, not buy a commercial one and tweak it. What we did was not the same thing that the commercial systems were designed to track. But I did think wed do better building it with consultants on a high-level platform than doing it by ourselves from scratch, so I proposed that we build a solution on Salesforce. The system had over 150 users, so this would be relatively expensive.

Timing is everything: I made my pitch the same week that financial news indicated that we were diving into a recession. Budgets were cut. Spending was frozen. And I was asked if I could build the system in Access, instead? And this is when I explained to my boss that we should table the project until we had the budget to support it.

Or so I wish. Instead, I dusted off my amateur programming skills and set out to build the system from scratch. I worked with a committee of people who knew the business needs, and I developed about 90% of a system that wasn’t attractive but did what needed to be done reasonably well. The goals for the system were dramatically scaled back to simply what was required.

Then I requested time with the department managers to discuss data stewardship. I explained to the critical VP that my system, like the last one, would only be as good as the data put into it, so we needed to agree on the requirements for an update and the timeliness of the data entry. We needed buy-in that the system was needed and that it would be properly maintained. Sadly, the VP didn’t believe that this was necessary and refused to set aside time in any meeting to address it. Their take was that the new system would be better than the old one, so we should just start using it.

This was where I had failed. My next decision was probably a good one: I abandoned the project. While my system would have been easier to manage (due to the scaled back functionality, a simple, logical database structure and a UI that included auto-complete and dupe-checking), it was going to fail, too, because, as every techie knows, garbage in equals garbage out. I wanted my system to be a success. We went on with the flawed original system, and eventually started talking about a new replacement project, and that might have happened, but I left the company.

Lessons learned:

  1. If I’m the IT Director, I can’t be the developer. There was a lot of fallout from my neglected duties.
  2. Get the organizational commitment to the project and data quality standards confirmed before you start development.
  3. Don’t compromise on a vision for expediency’s sake. There are plenty of times when it’s okay to put in a quick fix for a problem, but major system development should be done right. Timing is everything, and it wasn’t time to put in a data management system at this company.

Greenpeace’s Mobilisation Lab helps the organization transition into an era of people-powered campaigns. The right set of tools and an active social profile is helping Greenpeace to better support its community with campaigns that are community driven.

This case study was originally published along with a dozen others in our free e-book, Collected Voices: Data-Informed Nonprofits. You can download the e-book here.

NTEN: Tell us about how the MobLab fits into Greenpeace overall.

Michael Silberman (MS) and Wendee Parker (WP): We exist to help the global Greenpeace organization transition to a new era of people-powered campaigning shifting from Greenpeace-centric to supporter-centric campaigns. We’re working with staff in nearly 50 countries to design campaigns that enable the full power and potential of over 25 million supporters and activists to help us build stronger campaigns that win bigger. Our team has an independent budget to focus 100% on building capacity, challenging norms, sharing knowledge, and introducing new practices and tactics.

NTEN: Who are the Arctic 30, and how and why did MobLab get involved?

MS / WP: In September 2013, Russian security agents illegally boarded the Arctic Sunrise in international waters, seizing the ship and detaining all those on board at gunpoint. The ship was towed to Murmansk, and all those on board were locked up in cold, filthy cells, some of them in solitary confinement. They were charged with piracy and then hooliganism, crimes that carried lengthy prison sentences, because they dared to peacefully take action against destructive Arctic oil drilling and the onslaught of climate change, protesting at state-owned Gazprom’s Arctic drill platform in the Barents Sea. After 71 days in detention, the last of the Arctic 30 have been granted bail release, but severe piracy charges are still pending.

parker-silberman_-_some_tools_the_moblab
Some tools the MobLab provided to supporters of the Arctic 30

We got involved because there was a critical need to ensure that we were doing everything possible as an organization to help free these activists and leverage the global media spotlight to grow the campaign to save the Arctic. We added capacity to test new messages and tactics, and enable a global strategy brainstorm across offices and teams. Understanding how to effectively spread the messages by mobilizing new and existing supporters who connect with this cause through digital channels: thats what its all about.

NTEN: This has been a highly charged international incident. How have you baked principles of measurement and transparency into the campaign?

MS: We had to determine what could and should be measured. This campaign has been an opportunity to think about some of our limitations to measurement and tracking, and to have everyone really consider whats working and whats not.

WP: An informal group from several offices assembled for a week to take a look at our tools and platforms. It illuminated something many of us already knew: that consistency within digital engagement data was lacking. Trying to develop, implement, and execute a standard way to collect, track, and report on those digital efforts is an enormous challenge. The meetings gave us a good sense of our “universe” both the great effort our colleagues were already making in these areas, as well as opportunities to improve towards a complete, holistic point of view.

NTEN: Aside from this campaign, are there other wins you can pinpoint in these areas?

MS: There are over 100 active Greenpeace social accounts online. Were now seeing organizers include data analysis in their campaign planning. We at MobLab are still pushing, but it wouldnt get completely lost if we werent. Im also heartened by the fact that theres a lot of independent testing happening. People are using Optimize.ly for A/B testing, for example, and then reporting the results to everyone else.

WP: The focus and culture has definitely shifted, but the job is not done. Success would be having digital analysis (starting at defining digital analytic goals, implementing digital tracking and analytic tools for ongoing reporting, testing and optimization, ending with a complete campaign wrap up analysis) fully adopted as part of the overall campaign planning process.

NTEN: You mentioned Optimize.ly. Are there other tools that stand out as particularly helpful (or that you wish were more helpful)?

MS: We have issues with our bulk email tool, which doesnt make A/B testing as easy as it could be. On the upside, were making good progress with Google Analytics and Optimize.ly. On social analytics, were using Radian6, Topsy Pro, and Facebook insights.

WP: Greenpeace’s situation is so complex. In every office you may find a different setup for supporter data, a different set of digital engagement tools, etc. Even within offices, data can be fragmented among departments. I’m not sure theres a “one size fits all” solution, but as we work towards a common framework and toolset, it lessens the challenges towards complete supporter data integration a place where all departments view the same data and can have shared goals and metrics.

NTEN: Where would you like to see your campaign leaders a year from today with regard to systems and culture?

MS: We always want to see the four essentials of a people-powered campaign. The end is not putting data at the center of our campaigns; the end is more engagement-oriented organizing. We put people at the center of our campaigns, but data is an enabling tool. If we can use data to more effectively move people along and support our journey more deeply, thats a success point.

Data Analysts for Social Good

  • Breaking down data silos.
  • You don’t have to be a data analyst, but you will need to know how to collect and understand data.
  • You don’t have to use the best tools right away. It’s alright to say “This is the best tool for now.”

Andrew Means launched Data Analysts for Social Good in his spare time to address a need – a better understanding of how to use data not just to maximize inputs, but to show the importance of data to support organizations functioning more efficiently and effectively.

This case study was originally published along with a dozen others in our free e-book, Collected Voices: Data-Informed Nonprofits. You can download the e-book here.

NTEN: Andrew, you’ve spoken with NTEN before about your experiences with data at the YMCA of Metro Chicago. Now you work at Groupon and spend a lot of your spare time launching Data Analysts for Social Good (DASG), which offers webinars, a LinkedIn group, and an annual conference. Why did you start DASG?

Andrew Means (AM): I saw no one talking about data well. Fundraising analysts, marketing analysts, program evaluation people…everyone was so siloed. We were all using the same skills, underlying tools and methods, but applying them to different parts of our organizations. Data shouldn’t be siloed to one team or one person who pulls lists. The real power of analytics and social science research is that you can address a number of questions using the same kinds of tools and skills. And most organizations don’t know where to begin. We have very little human capital around this in the nonprofit sector although this has grown immensely over the past couple of years. DataKind and others are doing phenomenal work connecting data scientists to nonprofits, but the long-term solution is to have the next generation of executive directors, nonprofit leaders, and people entering the sector really understand these tools from the get-go.

NTEN: How are you creating a data-informed culture as you grow DASG and prepare for your second annual Do Good Data conference?

AM: The hard thing about starting an organization is that you have no data to begin with, so you have to create your own. I’m enough of an analyst to know my data points are really weak. But I try to use data as much as possible to generate content. I put out a survey in the early stages of planning the second conference, asking potential attendees what they want to learn. Now, as I line up conference speakers, I can look at that survey to make sure I’m delivering.

Another example: Every two weeks or so I send an email out to my list. I track click-to-open rates to make sure I’m giving people what they want, and sending these at effective times of day on the best days of the week. I used to believe that I should send all emails at 5:00 a.m. so that they’d be in my subscribers inboxes first thing in the morning. But when I paid attention to the numbers, I started to see a bit of a jump in opens if I sent them in the early afternoon.

I use a lot of free tools: MailChimp for email, Eventbrite for RSVPs, Google Analytics, and Google Forms. They’re fine for now. Thats something not enough people really consider. Its OK to say I have what’s necessary. I don’t want to use it forever, but it works for now and I’m moving forward. It’s worth dipping your toes in the water.

NTEN: What else should people keep in mind as they dip their toes in?

AM: We live in a world that makes it possible to measure so much, from apps that track what we eat, to Fitbits that track where we go. How do we allow these things to inform us but not control us? With that in mind, I ask myself: Is my community growing? How many people can I reach through social media? When are the best times of day to do that? Did this email outperform the list average? Its not super formal; I’m letting the data inform me, but getting the email out is more important than succumbing to analysis paralysis.

NTEN: That said, you are looking to grow DASG strategically. How do you see yourself professionalizing this organization? Is that the goal?

AM: DASG started as a happy hour 18 months ago when I sent out a few tweets. I have been surprised by its success. It’s easy to get caught up just doing the work of running a growing organization; I forget to step back and look at, say, the Eventbrite data from the past year which can help me analyze which webinars performed best. I want to standardize my email practices and create standard surveys for all webinars. I got a tremendous response when I surveyed the people who came to our first conference. So it’s about taking the time to collect the data but also to reflect on it. And for me, that’s about rhythms: taking the time weekly or monthly to reflect and plan.

NTEN: If you hired an employee, what rhythm would you want them to be in? What would you ask them to regularly report to you?

AM: Right now email is big. I’d definitely ask for regular reports on:

  • Revenue, since we have to make sure this is sustaining itself
  • Attendance at webinars and events
  • List growth for both email and LinkedIn

Where people on both the email list and LinkedIn are coming from geographically. In 2014, I’d love to do more events outside Chicago. I need to see where we have the highest concentration of subscribers.

NTEN: Why is it so important to you to create spaces where people can come together and talk data with their peers?

AM: Everyone is talking about data, but not in ways that will benefit us in the long term. Of course there are some organizations I really respect. But too often, analytics are used to maximize our inputs, not our outcomes. We use data to raise more money, attract more donors, and send effective direct mail campaigns. I’m not seeing data applied as rigorously to help us think about actually being better organizations. We need to step back and think critically about what we exist to do.

What happens when nonprofits make a real commitment to collect healthy data about their programs and operations; manage it well; and make savvy, data-informed decisions? And what happens when you connect energized, smart, data-passionate nonprofit professionals for a year of learning and knowledge sharing?

In 2013, NTEN, Microsoft, and some of the brightest members of the nonprofit technology community set out to discover the answers. The 18 members of the Communities of Impact pilot program spent the year connecting through two in-person retreats, monthly calls with seasoned data practitioners from all sectors, and ongoing online discussions and resource sharing.

The best way we could find to capture the lessons, insights, and discoveries from this year of work is by compiling case studies from participants with resources and conversations that emerged during their work together. This is not a report, per se; it isn’t a guide or a handbook. Just as these participants plan to continue working on the ways their organizations collect and use data, we hope that this collection can serve you and your team in learning about what others are doing and where you may go next.



Please log in to download this report.

Earlier this year, we worked with Avectra to release The 2013 Nonprofit Engagement Data Study, which was based on a survey of nonprofit communications and fundraising professionals, as well as executive directors.  The findings showed that nonprofits are tracking various kinds of participation data about the ways their constituents and supporters engage with their work and resources, but very few organizations are doing so strategically, nor are they supporting this practice with adequate tools and staffing.

The findings also suggest, however, that of those who are tracking and looking at correlation between engagement and more “traditional” data like annual fundraising levels, constituent retention or growth, average gift size, etc. — at least some positive correlation is being found.

To get a closer look at practices and challenges at the organizational level, we sought input from 10 nonprofits and associations that vary in size and work across many programmatic areas.

We’ve compiled their responses around seven key questions and share their stories and examples of how they are collecting, managing, and sharing engagement data at their organizations — and how it impacts their work.

> Read the new report “Connecting the Dots of Nonprofit Engagement Data: A Closer Look at Practices, Challenges, and Opportunities”

NTEN, whose mission is to help members strategically use technology to make the world a better, just, and equitable place, didn’t so much consciously move to the Cloud as come of age in it.

  • 11 total staff, 4 of them remote
  • $2.6M Operating Budget

The Nonprofit Technology Network (NTEN), whose mission is to help members strategically use technology to make the world a better, just, and equitable place, didn’t so much consciously move to the Cloud as come of age in it.

When IT Director Karl Hedstrom joined NTEN in 2006, a few employees had been working remotely, thanks to the Cloud. “Going forward we kept looking for those solutions because, with our size and capacity, the Cloud always managed to be the better option,” Hedstrom said. Today, roughly one-third of the 13-member full-time staff work outside NTEN’s Portland, OR, headquarters.

Employees, regardless of location, rely on a few key systems daily: Google Apps’ Gmail, Calendar, Drive and Hangouts. NTEN uses Avectra netFORUM for managing customer records, memberships and events, and, integrated with netFORUM, HighRoad Solution for e-marketing. The organization also uses LastPass Enterprise for password management and Dropbox, primarily as a backup system.

Cloud-enabled sharing and collaboration called for new policies and procedures for document management at NTEN. “Before the policy, staff sometimes had trouble finding the right document or the right version of a document,” Hedstrom noted. He saw how those types of seemingly minor issues could, on a larger org-wide scale, lead to lost productivity and duplicated efforts.

The resulting policy covers several critical areas: folder structure, file naming conventions and sharing policies. It works well, Hedstrom said, although there are (of course) hiccups. Shared access means anyone can move or delete files, deliberately or by mistake. And if a user doesn’t add new documents to the system, there’s no way for colleagues or IT staff to find them. Hedstrom and Brendan Blaine, IT systems manager, are exploring ways to improve administrator controls to prevent those issues.

Hedstrom and Blaine also are working on extending the Google Drive document management policy to Dropbox to ensure that when staff members sometimes use Dropbox instead of Drive for file sharing, the files are managed in a systematized way.

Selection criteria for new Cloud-based systems are few but fundamental: ease of use for staff and how well the product integrates with NTEN’s existing systems and processes. “A big part of the evaluation process is looking at how [a new system] will integrate with our database so that it’s seamless, so that we won’t have data siloes,” Hedstrom said.

Cost also is a factor when selecting new products, as is scalability, although that is typically implicit in Cloud-based applications. “With the Cloud, you can be ready for anything,” Blaine said, including preparedness for web traffic spikes, such as when NTEN opens registration for its annual Nonprofit Technology Conference (NTC). “We might have a couple of hundred people trying to register on the site at the same time. The scalability of Cloud systems generally soaks up that demand [without disruption].”

When new products and systems are implemented adopted, training takes place both in person — at twice-yearly staff meetings in Portland — and virtually. “We make liberal use of Google Hangouts,” Hedstrom said. “The screen share capability works well for remote (video) training.”

But the bigger issues related to training don’t go away with the Cloud. “No matter what, [for any technology], you have to take a step back to design and deliver appropriate training and follow up on that training [to identify further needs],” he added.

Google Hangouts’ remote desktop capability also facilitates troubleshooting. “Everyone hates sitting on the phone with tech support,” Blaine said, “so my being able to take control of an off-site staff member’s computer really cuts down on that type of frustration.”

Overall, maintenance of cloud-based systems is minimal. “Mostly we just contact the vendor,” Hedstrom said. “It takes a lot of the burden off staff so we can focus more on systems management and fine tuning policies to help staff do their work and the community get the most out of our website and programs.”

The Cloud, though, is not without drawbacks when it comes to a distributed team. “You’re beholden to Internet connectivity,” Blaine said. As staff size increases and people multitask and listen to music or watch videos, “you can start to get into that bandwidth crunch zone, where your changes aren’t pushed to the Cloud real-time” (which impacts employees working remotely). And, “as good as Google Hangout is, you still can’t replicate that in-person experience,” Blaine added.

For NTEN the downsides are worth the benefits. The Cloud has enabled NTEN to recruit and hire the best candidates regardless of location and to retain existing staff who relocate. “Without the Cloud, it would not be an option to have staff spread across the country,” Hedstrom said. It’s also made it possible to hire IT staff with different skill sets. Both Hedstrom and Blaine came to IT from the program side of nonprofit management and have less experience with hardware per se.

“Our roles are IT-focused now, but we haven’t lost that organizational-goal view,” Blaine said. “Instead of having my head buried in hardware and software weeds, I’m looking around to see where we’re making progress and where we’re not, and we can shape our systems to help funnel that progress [to meet our strategic goals].”

NTEN

  • 10,000+ members
  • 55,000+ total NTEN community participants

Everything Megan Keane does as the community engagement manager for NTEN is driven and supported by one simple principle: “It’s all about the people,” she said.

Keane joined NTEN in 2012 and has worked as a community manager in the nonprofit sector for several years. “It’s about making that personal connection and getting community members to talk to one another — online and off — and not just one-to-one, but one-to-many,” she said.

And though the Cloud helps foster a sense of community in a number of ways, it also “isn’t foolproof,” said Keane. “The Cloud is such a buzzword, but it’s a tool like everything else. You can’t use it without keeping those principles, and your mission, in mind. And,” she added, “the Cloud breaks; it is not always your friend.”

Since so much content on the web isn’t static, “you can have ongoing conversations,” Keane said. On the NTEN Connect Blog, for instance, community members can comment on posts, and experts from the nonprofit technology and NTEN membership community can contribute content and share best practices with others.

But even archived and less dynamic content still can foster engagement among community members. “Someone can stumble across an old conversation, and it’s still there as a resource. The person isn’t just finding an article or an archived webinar but there’s this whole conversation about the topic that people have been sharing. Even if it’s a few years old, often the principles are still relevant. Technology may change, but the principles and general strategies usually still apply.”

Keane also maintains an active NTEN presence on several social media platforms — Facebook, Twitter, LinkedIn and, increasingly, Google+ since NTEN’s G+ page is serving as a growing source of traffic to its website. “In terms of community engagement, I haven’t seen a lot of discussion happening there yet. But [because of the traffic], it’s something you can’t ignore as a channel. . . . Sometimes the tool kind of picks you rather than you picking the tool.”

Listening to your audience is crucial to community engagement, Keane noted. She uses Cloud-based Netvibes to monitor social analytics and as a feed reader of sorts to learn what community members are talking about on their own blogs and on social media. She uses the information to create a periodic member roundup post for the NTEN blog and also tweets about the post. “It’s a small way of giving individual members the spotlight, and people really appreciate it,” she said.

The practice jibes with Keane’s view of her role as that of curator. “With the sheer amount of information out there, giving the community an easy way to access it is important — pointing them to the cream of the crop, recaps, other meaningful content and conversations, both from NTEN and other organizations. I’m wading through it so they don’t have to,” she said.

Listening also helps Keane and NTEN stay responsive to member needs. “I look at what people are talking about. If they’re asking questions about mobile or responsive design, to give two recent examples, we might write an article for the blog or develop a webinar on those subjects.”

The Cloud also supports off-site events, such as NTEN’s annual Nonprofit Technology Conference (NTC), although Keane uses her go-to systems and tools somewhat differently in the conference ballroom than her NTEN office.

Twitter is big at the conference, both for participants and staff. “So many people use it (and the conference hashtag), and it becomes more customer-service oriented, Keane said. She and other staff tweet everything from last-minute room changes to reminders about sponsored breaks (i.e. free snacks).

Twitter is also a way for participants to share the conference experience with their own followers and colleagues who may not be attending. Or, as was the case at the most recent NTC, to alert NTEN staff of technical issues with myNTC, the conference platform for scheduling, discussions and other interactions. Upon seeing the tweets, staff promptly contacted the vendor, Zerista, and the issues were resolved.

Keane used Cloud-based Google Docs (Drive) with presenters on the conference panels she moderated, both during planning and for the presentation itself. And she used the NTEN blog, where she promoted award winners in “ready to go” posts she had written prior to the event. She used Facebook to post a few photos of conference highlights, with links to NTEN’s flickr photostream.

Although, or perhaps because, Keane relies heavily on the Cloud at events and for her day-to-day work, she appreciates the need for a plan B. “Especially at conferences with a lot of techies,” she joked, Internet access and sufficient bandwidth can quickly become an issue. (At NTC, NTEN asks participants not to access the Internet from more than one device at a time.)

When, just a few minutes before her NTC panel, Keane’s presentation in Google Drive wasn’t accessible, she was reminded of the importance of having a backup. Similarly, when presentation slides were not available to Online NTC viewers, Online NTC hosts at the conference tweeted the slide content so participants watching the live stream still had context for the sessions. When the myNTC platform wasn’t running smoothly, the hard-copy conference guides NTEN had prepared in advance came in handy.

Technical challenges aside, Keane takes advantage, and encourages members to take advantage, of opportunities to interact with colleagues in person, since connection — be it real-world or virtual — lies at the heart of community engagement. “You have to strike a balance; you don’t want to be at the conference in-person and then spend all your time online.

Recently we announced the launch of our 2013 Communities of Impact program. As part of this pilot, 19 people representing organizations large and small are working together this year to try to unlock and solve some of the challenges nonprofits face when it comes to:

  • measuring and managing data
  • making data-informed decisions
  • and sharing the results with colleagues, partners, funders, and the public.

We’ve been conducting research into what’s happening in these areas already to be sure our work adds value. In addition to the case studies NTEN has compiled, here are some more examples the Communities of Impact and I have spotted: 
 

  • Children’s museum data informs health policy. This Markets for Good article about the Children’s Museum of Manhattan describes how CMOM has collaborated with National Institutes of Health, providing real-time data to develop a health curriculum for young children through an unusually speedy process. CMOM Executive Director Andrew Ackerman acknowledges that an active knowledge network and  “philanthropists who are willing to provide ongoing support and venture into new areas” have been critical.
  • Global nonprofit measures and shares its impact in real time. The nonprofit Splash has developed an entire site called Proving.It where they showcase the status of each of their projects so that funders and supporters can watch as they track their progress toward “a world with clean drinking water for all children—and a museum telling of when it was not so.”
  • Building a movement through innovative open source tools. Data Commons is made up of 10 cooperative development centers and others who are creating tools to most effectively use shared information, including “common directories, maps, and databases as well as membership engagement platforms, marketplaces, and tools to embed shared information around the internet. Tools are licensed as open source, available for anyone to use and improve.”

Now it’s your turn…
What are your favorite examples of nonprofits or networks who are doing this work well? The more we know about what’s currently happening, the better job we can do this year.

Please share links and shout-outs in the comments section. Thank you!

  • Lexington, KY
  • 5 Staff

How does an advocacy organization measure its effectiveness in a data-driven way? Rich Seckel, director of the Kentucky Equal Justice Center, admits it can be a challenge. It’s difficult to be in a multi-variant world trying to prove causality,” he said.

The poverty law organization advocates on behalf of Kentuckians in need, serving as a watchdog for the state legislature, keeping an eye on bills that are filed and lobbying lawmakers. Rich said measuring effectiveness through data boils down showing funders data that supports not just his nonprofits impact on policy changes, but on the lives of people who benefit from its victories.

In 2003, with the state under intense pressure from the federal government to cut Medicare spending, the governor announced a series of changes that would save $45 million but would terminate nursing home or home healthcare coverage for 3,300 disadvantaged Kentuckians. To challenge the cuts, KEJC enlisted the National Senior Citizens Law Center and sued the state on the basis that the measures were solely to save money and therefore illegal. However, they didn’t have data to support the case that the cuts were unjust.

The Medicaid cuts were fairly unpopular with the electorate, so during the next gubernatorial election both candidates promised “not to kick people out of nursing homes.” When a new governor was elected the following year, counsel for the state agency expressed interest in settling the case. In the settlement, the agency pledged a return to the earlier standards, to review all people who had been denied or terminated from long term care, and to report on the results. Using that data, Kentucky Equal Justice Center found that 97 percent of those who had been denied benefits or whose benefits had been terminated had been restored long term care coverage – about 3,300 of them.

Three of the 10 plaintiffs named in the lawsuit KEJC filed contesting the cuts died before the issue was resolved, demonstrating not just the seriousness of the issue but that access to the right data can make or break an advocacy groups case.

KEJC’s work directly improved the lives of more than 3,000 Kentuckians, but often the situation isn’t so clear-cut, Rich said. Just about every funder he works with wants KEJC to evaluate its success in human terms, which can be tricky though; policy successes can be easily measured, the results cannot. The data he needs to demonstrate these results often come from such external sources as state agencies, he said, and can be hidden under layers of bureaucracy. One way around that obstacle that has worked for him, Rich said, is to establish relationships with friendly legislators or state government employees who will share reports that never get released to the public, but which have the data that he needs to show his organizations impact.

As part of a foundation grant given KEJC to boost its infrastructure, Rich was required to meet periodically with a trained evaluator who helped him understand the science behind the statistics the organization was tracking. That statistical analysis training showed him how many different variants come into play in the organizations work, and the different ways they can and should be interpretedin other words, he said, it gave him a conscience about ascribing too much change to the efforts of his nonprofit.

“We don’t brag too much,” he said, but conceded that maybe the organization should work a little harder to publicize its measurable results. “We probably need to get better at that.”

This case study is part of the research project in 2012 conducted by NTEN with the help of Idealware. See the State of Nonprofit Data report for more information about how nonprofits are–and aren’t–making data part of their decision-making processes, and the key challenges that affect an organization’s ability to be more effectively “data-driven.”

  • New York, Los Angeles, and Washington, D.C.
  • 65 Staff (Full- and Part-Time)

The New York City-based Writopia Lab introduces children to writing and literature through a variety of programs, including writing workshops, a theater festival, and literary magazines. In addition to the organizations five NYC locations, it maintains two others upstate and branches in Los Angeles and Washington, D.C. Tracking participant data is critical to managing so many individual workshops and events for such a high volume of participants, and Writopia has covered some ground on the way to being a data-centric organization, but the geographical distribution of the staff is one of the hurdles it is overcoming along the way.

“We’re a gigantically distributed organization,” said Director of Operations Jeremy Wallace-Seagall. “That’s definitely one of the challenges we face. It’s difficult to coordinate consistently with the organizations staff members, both because of their locations and their scheduling,” he said. “There are 65 total staff, but not all work at the same time, as many of the organizations faculty are seasonal or part-time.”

Another obstacle is that the tools at his disposal are not necessarily the best for the job, nor do they all integrate well. Jeremy is doing what he can to take advantage of the different opportunities to collect and use data, he said, but to some extent he’s simply making do.

At Writopias New York offices, he uses a Microsoft Access database that he’s tweaked in unusual ways to meet his needs; a less than ideal solution, he said, despite his advanced modifications. “I’m an old database head, but one who got stuck in Access,” he said. “I like to say that I’ve done things with Access that very few people have done, but it’s still Access; it faces all the same limitations as any Access database.”

The organizations other locations dont share the database, an inconsistency which causes problems. To remedy it, Jeremy is considering switching to Salesforce across the nonprofit, and promised himself that by year’s end he’d have either standardized all locations onto Access database or rolled out a new Salesforce database.

Currently he’s looking for a database and website consultant to help consolidate and streamline data, which he believes will help with reporting and analysis – which in turn will help him manage growth.

“We’re growing every year somewhere between 45 and 65 percent, and there are always people saying, “Hey, can you come to my region?””he said. “I need to be able to analyze our growth and see how many workshops an average student takes and have all the registration and enrollment data in one place to see what makes sense.”

“We have outgrown our space on the Upper West Side, and we’re trying to analyze whether we should get a gigantic space here on the Upper West Side or a smaller space here and another one downtown or in Brooklyn or wherever,” he said. So we’re looking at where our clients come from, and similar people in those areas. If we go to the east side, will it be all this pent-up demand, or is everyone who might come to our workshops already coming?

Writopia Lab runs on a self-sustaining business model in which workshop participants pay on a no-questions-asked sliding scale pricing systemabout 50 percent pay full fee, 40 percent pay somewhere in the middle, and 10 percent get full scholarships. Though the nonprofit gets some grant money – about 10 percent of its total funding – currently none of the funders asks for reporting.

“That’s great in terms of being able to focus on getting things done, and not having to report in orange for one giver and yellow for another, but it means I have not taken this time to consolidate our data and come up with great ways of turning that data into information,” Jeremy said, adding that he plans to change that and hopes to focus on making use of that data in coming months.

“That’s what the next year is about personally for me,” he said. “Turning data into information. I’ve drafted dashboards and looked at various tools. We have a donor packet that we give out to people, and I’ve got some pretty reports in that. They’re all completely legitimate, but our analysis is not terribly robust. That’s sort of an artifact – we’ve been so busy serving clients that we’ve not had the time to chase after funding, or produce reports that would make it easy for us to drastically change the amount of institutional giving were receiving.”

“My hope and expectation over the next year is that well see that cycle twist, and the reports will come and tell a convincing-enough story that it will be easy enough to get in front of funders,” he said.

Writopias enrollment-tracking capabilities have not reached the levels Jeremy would like. Without a shared registration/enrollment/contact management database, information is spread out in a number of places and systems, which makes it difficult to use the data effectively.

“Part of it is a workflow challenge,” he said. “This sort of gets to the notion of loosely joining best-in-class systems, when my dream, really, is to have one piece of software that does everything in my life. I get the pitfalls of that I have reservations about having all my eggs in one basket, and there are other concerns; but having all your eggs in one basket makes it really easy to find them.”

Under the current system, potential clients have to visit separate web pages for each branch of the nonprofit to find their scheduled classes, a separate registration page for each branch that plugs into a Google spreadsheet, plus a pricing page and they have to remember all the information on each course they’re interested in, look it up on the pricing page, and register for it separately.

“I don’t care that I have to copy everything from the Google Docs to an Access database,” Jeremy said. “What’s brutal is what clients have to go through; that’s the piece that is most important to us. If I don’t get my website rebuilt, if I don’t get my Access db out to everyone, if I don’t get my Salesforce database installed, I have to find a way to make it easier for people to find and sign up for our workshops.”

He said hes been working to establish a data culture across the organization, which hasn’t always come easily. If you’re making a list, do it in a spreadsheet instead of a document,” he said. “It took me a while to create that sort of way of thinking, but I think people are pretty well on board with the notion that were collecting data, and whether or not were analyzing exactly this data at exactly this time, we want to be mindful of collecting the data in a clear and reasonable way.”

This case study is part of the research project in 2012 conducted by NTEN with the help of Idealware. See the State of Nonprofit Data report for more information about how nonprofits are–and aren’t–making data part of their decision-making processes, and the key challenges that affect an organization’s ability to be more effectively “data-driven.”