Put that in Your Pipe and Search It
This article announces the beta release of the NPTech Pipe - a focused and vetted content stream for nonprofit staff and technology providers to efficiently search, browse, and subscribe to technology-specific content relevant to the nonprofit sector.
Being prone to gratuitous roamings down memory lane, I find myself on this eve of the 2007 Nonprofit Technology Conference in DC - which is turning out to be perhaps the largest coming together of geeky do-gooders ever assembled - remembering my first NTC way back in 2004. While much has changed at NTC in these three years (reliable wifi, pre-conference social networking, better targeted tracks, much better parties), it still brings together the most impassioned and brilliant people in our field. It's inspiring and overwhelming all at the same time.
Yet I always go into these conferences trying to figure out how best to capture all that great information, recall those invaluable conversations, and actually follow through on just a few of those cool world changing ideas - and then share it out with the broader community. Inspired by other technology conferences like SXSW and eTech, some may remember the 2004 NTC Eventspace Wiki that I put together to facilitate collective information sharing. While a bit ill-conceived and executed, it was simply the wrong tool at the wrong time for all the right reasons. In short, I was trying to figure out how a Community of Practice could systematically share, aggregate and access knowledge and information. In 2007, I believe we have arrived there.
The NTEN community is full of engaged members who are blogging, podcasting, webcasting, tagging, vlogging, conversing on an unknown number of listservs and forums, publishing research and whitepapers, getting both local and national press, and most recently Twittering. Along with the conference backchannel, this year's NTC is poised to be amazingly well documented, adding to an ever-increasing pool of rich content our community is producing. However, while I may be overjoyed by the depth and breadth of content on nonprofit technology, for others it may be a cacophony of noise that is difficult to access and navigate or worse find relevant. Not all of us are information hounds, content mavens, or aggregation wonks. And even for those of us who are, there is still no universal archive or search utility that can aggregate and intuitively access all this content.
Furthermore, there is the layer of relevancy - not all content produced in the nptech sphere is necessarily on topic or logically part of the broader conversation. Having become a successful sector of content producers, we are now faced with the challenge of focusing the noise into a clear signal that provides us with the most highly relevant nonprofit technology information. As Robin Good described some three years ago, "The amount of content being made available on a daily basis is simply too much to be managed by our everyday Googles and RSS aggregators. [The issue is] how to navigate, filter, and reduce to humanly manageable dimensions the gigantic flow of information coming at us in increasing amounts."
Using Robin Good's concept of "Newsmastering," I have been working with the staff at NTEN over the last several weeks to create a content channel to aggregate, filter, and search for nonprofit technology content in a user-friendly interface. Additionally, we wanted the flexibility to repurpose content in a variety of ways, customize the tool on the fly as new requirements emerge, as well as give users options about how they use the tool based on their needs. Oh and of course, do it with a minimal budget. Even a year ago, trying to create something like this using existing tool sets would have been impossible or cost prohibitive. Fortunately, in this last year we have seen the emergence of trends that make all this possible - plug-n-play mashup tools, the widgetizing of the web and a (debatable) tipping point of RSS comprehension and usability.
At first, services like Google's Custom Search Engine (CSE) and Yahoo Pipes seemed like the most obvious systems to leverage. As we dug deeper, it became apparent that each came with compromising limitations, if not outright flaws. Google CSE is a domain(s) specific filtered version of Google Search and does not support RSS. Yahoo Pipes in theory was a great fit, but we kept coming across too many bugs. After researching a variety of different RSS mashup tools and not finding quite the right fit, I put in an IM to nptech's resident web 2.0 guru and Techcrunch celeb, Marshall Kirkpatrick. Describing in brief what I was attempting to do, Marshall turned me onto MySyndicaat, "a news mastering tool that allows you to retrieve, filter, mix and monitor content from remote content sources available on the web." Additionally, MySyndicaat creates a search index of all content while producing output feeds from your sources or for specific keyword searches. Without coding a thing and absent of a convoluted AJAX interface, I was able to build a 90% solution in less than a half hour. Fueled by pure giddiness, I set out to problem solve the other 10%.
Over the course of the next week, I mixed, mashed, played, and explored the possibilities to find a solution that met our requirements. In the end, I settled on a pieces-loosely-joined approach to create a robust, easily manageable, sustainable and user-friendly solution. To easily manage feeds, I selected the online feed aggregator Newsgator. Newsgator not only has subscription bookmarklets that make it simple to add new feeds, it also creates a dynamic OPML file (a list of subscribed RSS feeds), which when imported into MySyndicaat, updates the index automatically. Once in MySyndicaat, I created a few filters that diminished redundant postings and some email to RSS snafus. MySyndicaat creates two primary output feeds; a full, unedited feed and a digest feed for selected postings from the full feed. Although the MySyndicaat user interface is acceptable, we wanted the ability to customize and brand it - giving the tool full context and meaning to its potential users. Enter stage left, Grazr. Originally conceived as an RSS blog widget that embeds a mini-browser on a webpage - a more efficient Blogroll of sorts - Grazr's ongoing release of new features over the last six months has more or less made it a platform for building custom web applications. Using the two output feeds from MySyndicaat, Grazr creates a simple but dynamic search application. And to top it all off, we added the ability to browse and subscribe by filtered and full feeds as well as from each source. Welcome to the new NPTech Pipe.
Part vertical search engine, part community feed aggregator and browser, the NPTech Pipe is a tool that aggregates and filters content from scores of websites, blogs, tagging tools, and newsfeeds to provide users with the most highly relevant nonprofit technology information. (Please read the FAQ page for more information on its uses and how it’s managed). As part of NTEN's mission to "Create a world where every nonprofit staff person has the skills and knowledge to use technology strategically and effectively to advance that organization’s mission," the NPTech Pipe provides an accessible conduit for anyone to efficiently find and put to use focused and vetted content. From information on data management systems to how widgets are being used on websites to raise money, the NPTech Pipe brings the best of the nptech sphere together and provides users a clear context , and flexible and dynamic ways in which to engage that content.
While we believe there is great potential for the NPTech Pipe within our community, we also acknowledge that this is a proof of concept experiment, which has limitations. The Grazr UI is by no means perfect. We are still developing the editorial team and the vetting criteria documentation. The current source list is only the tip of the iceberg - there are many content sources we do not yet know of, or that may be inaccessible as many critical content sources remain locked up behind paywalls, firewalls, and within e-communications (e.g. e-newsletters and non-RSS enabled forums/listervs). As a beta release, we are looking for broader community input to make this tool truly the best it can be. So test away. Give it to your organization's resident Luddite and watch their reaction. Hack the heck at it (the code is fully accessible so you can create your own version). Then write about it, give us constructive feedback and become a partner with us in making nonprofit technology content accessible for anyone and everyone.








