Avoiding a House of Cards: Three Lessons from a Multiple-Organization Data Collaboration Project

[Editorial note: the following article first appeared in the December 2012 issue of NTEN:Change, a free quarterly journal for nonprofit leaders.]

When is a meal more than a meal? When it’s served in a soup kitchen. Most soup kitchens serve double, triple, even quadruple duty as a community center, health center, temporary shelter, and a place to get job services. That’s the complicated part of social change – complicated problems call for complicated solutions, even when our business models force us to focus on one simple arena.

Fortunately, soup kitchens don’t work alone in addressing hunger. The soup kitchen operates in a field of health clinics, shelters, and job training centers. Collectively, these organizations begin to address the complex issue of hunger. But more often than not, these individual organizations operate alone, never knowing if the individual receiving the meal is getting the job training that might help her find employment and become food secure.

Imagine if the soup kitchen knew what other services each client was receiving? And what holes in service they could recommend? If these organizations shared data, they could improve services and have a much clearer understanding of their impact.

The field of Nonprofit Technology Assistance Providers (NTAPs) began developing in the early 1990s. Since that time, the field has matured into several key players that contribute distinct value to the nonprofit sector by providing services, software, training, research, and community-building. Each organization plays a unique role in the sector, contributing our own solutions to the complex problem of helping nonprofits adopt and use technology to create more social change.

We believe that, like the soup kitchen, our overall impact could be strengthened through a better understanding of who we serve individually and collectively, and what that may imply about how we can better collaborate to meet the needs of those we serve. In January 2011, at a gathering hosted by TechSoup Global, we began talking about how to model our own ideals.

In summer 2011, we received funding from Microsoft to bring our groups together to answer at least one question: which nonprofits do our organizations serve separately, and collectively. Seven organizations participated: 501cTECH, Idealware, Network for Good, NTEN, NPower, NPower PA, and TechSoup Global. Together, we designed a data analysis project that answered this key question, and provides tremendous insight into the nature of their relationship to each other and the nonprofit sector as a whole.

Of course, collaboration is tough work. Building a collaboration in name, is easy, but a gust of disarray, and it topples like a house of cards. We aimed to build a better collaboration, with a strong foundation, and sturdy frame. Something that would stand the test of time while our relationships and collaborations evolved. Did we succeed? By most measures, we did. Here are some of the lessons we learned:

1. Build a Foundation of Trust: Define Parameters

While all seven organizations had agreed to participate in principle, sharing is never an easy task. In our initial conversations, we each had plenty of questions about where, and with whom, the data and the results of our analysis would be shared. In essence, what and how much data any org was willing to contribute was directly tied to how secure the process would be and how much control they had over what got shared. It quickly became clear that creating mutually agreeable security and transparency protocols would be our single most important piece of work for the project. The project would fail if participants did not trust in each other and the process.

To that end, we convened in November 2011 to answer those questions. Working with a facilitator, we discussed what each organization wanted to learn from the project (what was “in it” for us), what data would be required (what we had to give), and the conditions under which we could share that data.

We spent the first part of the day simply sharing information about our organizations, and finding common ground between participants personally. This was a great launching pad for our next step, identifying some common goals for the project:

  • Understanding demographics of the audiences for each of our organizations
  • Understanding where our organizations served unique, or overlapping audiences
  • Understanding what part of our audiences were deeply engaged in the nonprofit technology sector (and do they have some common traits?)
  • Understanding the demographics of nonprofits receiving paid and unpaid NTAP services
  • Identifying potential collaboration/partnership opportunities with other NTAPs

We also identified many more goals that we could potentially explore, but were deemed beyond the scope of this initial project because we were either unsure if we could share the data required, or sharing the data was too onerous.

This initial meeting was also used to outline a data submission process that would ensure that data could be transmitted securely to the data analyst and not be shared with any other project participant. Additionally, we identified the key elements of a non-disclosure agreement for all participants.

2. Strengthen Trust Infrastructure: Formalize Sharing Principles

With clear foundation of trust—in each other and in the process—we left the meeting with direction and purpose. Getting the infrastructure we agreed on at the meeting in place seemed like it would be an easy task. Yet, we were reminded once more that agreement in principle is not the same as agreement. No one wants to see how the plumbing works in their house, but knowing where the pipes are is an essential part of homeownership.

Setting up systems to securely collect and deliver participant data was actually relatively simple. Getting agreement from all parties on a Non-Disclosure Agreement (NDA) was not, however. The NDA was an essential part of the process, legally requiring that all participants in the project keep project information to themselves. In other words, what happened in this project stayed in this project, unless all parties agreed to share. Without this base, no one could be comfortable sharing their data.

Though all participants had fantastic intentions, anything involving multiple lawyers is sure to grind to a halt. In fact, it took us over two months to get an agreement in place. We had assumed we would get it wrapped up in a couple of weeks, and the delay set us back quite a bit. However, once we got it in place, things progressed rapidly.

3. Move Beyond Trust: Specify Expectations

With the agreement in place, we securely shared anonymized data and began getting findings back from our data analyst. During this time period, we met regularly on the phone, as well as at one in-person meeting. As the findings began to come in, it became clear that what one person meant by “map the sector” was not always what everyone else meant when they said the same words.

While we had agreed to pursue data in support of some specific goals, seeing the results come back made us acutely aware that participants often had differing ideas about what that goal meant (at worst) or how the data would be presented (at best). And sometimes our analyst had different ideas about those definitions from any of us. This is where we learned the hard way that you cannot get too specific.

We had, metaphorically, clearly defined that we wanted a house with three bedrooms. We had failed to mention how big that house should be, or what style it should be built in. Fortunately, none of these discrepancies between reality and expectation were show-stoppers, but it did mean that we had to take a lot more time to walk through the findings and get as specific as possible.

Conclusion

Our organizations succeeded in answering our key research questions (and you can find our public report on what we found by downloading this pdf). We learned that, collectively, we reach over a third of the 1 million nonprofit organizations filing 990s. We also dove more deeply into how the distribution of that reach compares when examining characteristics like issue area, size, and location. Additionally, we took a nuanced look at the landscape of gaps and overlaps among the lists of those served, examining not only nonprofits reached, but those for which unpaid and paid transactions—as well as multiple types of paid transactions—had been recorded.

Most importantly, we learned that there are tremendous opportunities for our organizations to work together to support each other, build the field of nonprofit technology, and increase the number of nonprofits that are really using technology effectively. Thanks to this project, we also learned a lot about how to make that collaboration happen.

Sincere thanks to each of the NTAPs that joined NTEN in this experience: 501cTECH, Idealware, Network for Good, NPower, NPower PA, and TechSoup Global; to the project team: Christine Egger, facilitator; Henry Quinn, analyst; Katie Guernsey, analyst support; and Amy Luckey, advisor; and of course our funder: Microsoft, without whom this assessment would not have been possible.

> Download the NTAP Sector Research Assessment Report here.

Holly Ross is former executive director of NTEN, and currently executive director of the Drupal Association. Holly was recognized as one of the “NonProfit Times Power and Influence Top 50″ three times, in 2009, 2010 and 2011.