Terms and Trends in DIY and Open Source Online Maps

Submitted on Wed, 1/11/2012 - 2:33pm
Implementing truly gorgeous online maps still requires some custom code and technical know-how – but, the cost and learning curve involved has dropped considerably.

Data-driven storytelling is all the rage on the web, particularly as we move into the election season. Online maps provide a concrete, quickly digestible way to drive home the connection between your organization's work and where it takes place. Online maps can demonstrate the breadth of large movements or highlight grassroots work at a very local level.

In ancient times (5-6 years ago), there were two main ways to visualize data with online maps. You could embed a simple Google map on your website or you could invest in a Flash-based mapping tool.

While beautiful maps have been a staple of the web for the last decade, development of highly-customized maps historically has been expensive and has required a very specialized skill set and tools. With Adobe Flash, Active X, or other browser-specific technology requirements, the accessibility of these maps across various platforms was also a real problem. Today, with more and more people accessing the web on mobile and tablet devices, these older mapping technologies are even more limiting.

Fortunately, the advent of HTML5, CSS3, jQuery, and other modern web technologies has paved the way for new, open source, cross-platform compliant mapping technologies to develop stylish and interactive maps. Admittedly, implementing a truly gorgeous online map still requires some custom code and technical know-how – but, the cost and learning curve involved in the development of these maps has dropped considerably.

BlueWaterBaltimore's bacteria monitoring map

An example of a modern online map: We leveraged MapQuest, Leaflet and Drupal to build Blue Water Baltimore's Bacteria Monitoring Map.

Consider this a starter's guide for exploring open source and DIY mapping tools. To learn more about these topics, please consider joining me for my session at this year's Nonprofit Technology Conference: "Geek out with online maps as storytelling devices".

What Makes Up a Web-based Map?

There are a couple of important concepts to understand about web-based mapping. First, there are generally two ways to present a map on a webpage. The first method was described above: Flash-Based maps. With Flash-based maps, you essentially embed an Adobe Flash file that "plays" on your website. There are many freely-available Flash-based mapping libraries; more sophisticated Flash-based mapping tools often require licensing fees. As mentioned previously, the main problem with these maps is the Adobe Flash requirement itself: among other problems, Flash just isn't available on Apple devices such as the iPhone and iPad - cutting out a huge segment of your site's potential audience.

The second, and in my opinion preferred, approach involves javascript. With javascript-based maps, a bit of code is downloaded as part of a webpage to a site visitor's web browser. From there, the javascript pulls together data, images, and map controls to render the map and to allow your site visitor to interact with it.

I'll describe the components of these maps shortly - but the important concept to understand is that because these maps rely on javascript – a core technology found on most modern websites – they are largely cross-browser and cross-device compliant. While some maps look and function better on different browsers, modern javascript-based maps should function and adapt to use on iPhones, tablets, and other mobile devices.

Bits and Pieces of Javascript Maps

There are three main components of javascript-based maps. The first is the javascript library itself. Examples of javascript mapping libraries include the Google Maps Javascript API, Mapstraction, OpenLayers, Modest Maps, and Leaflet. The javascript library essentially mashes together the data and images and provides all the controls on the map. These javascript libraries all provide slightly different controls and features for interacting with the map. (Getting really geeky, they also perform differently with different types of geographic data.) So, depending upon what you're trying to do with your map, one library might be a better choice than another.

The second component of javascript-based maps are the base layer tiles (sometimes referred to as simply map tiles). These tiles are square image files stitched together to create the underlying map. With larger map tile sets that provide different levels of detail based upon how you zoom into the map, a single base tile set might be a collection of many hundreds of millions of individual map tile images.

Obviously your web browser can't download all of these images at once, so javascript maps load these tiles "asynchronously" - or as you click around, recenter, and zoom in and out of the map. Often, more than one set of base layer tiles are available on the same online map. When this is the case, the javascript library provides a "tile switcher" that lets you toggle between one or more sets of base layers. You've probably already experienced this, when you've switched between the "street view" and the "satellite view" of a Google map.

It's important to understand that only one base tile set can be enabled at a time when viewing a map. When looking at a "hybrid" tile set, such as the one provided by Google, you are actually looking at a single tile set that has been created with both the physical and road data baked into it.

Above is an example of four individual Google Map base layer tiles of the same location with different data baked into each image file.

The third and final component of javascript-based maps is the vector data that's rendered on top of the map in your web browser. The distinction between base tiles and vector data is the most important, and difficult to understand, concept in web-based mapping.

Again, the base tiles are simply images that are downloaded on the fly and stitched together to provide the underlying map. Vector data is then rendered, or drawn on top of, the map by the javascript in the site visitor's web browser.

The most basic example of vector data are point markers placed on top of a Google map. These points consist of a latitude and longitude value, a map point marker icon, and optionally some amount of content that appears in a pop-up bubble when you click on the map icon. More complex vector data might include different colored polygon shapes and lines.

Vector map data can exist in different formats that are processed by the various mapping javascript libraries. Two of the more common open source formats are KML (the format pioneered by Google) and GeoJSON.

This vector data is often grouped into different "data layers". When different sets of vector data layers are available on a single map, the javascript library will often provide a switcher that allows you to toggle on or off one or more of these data sets. Again, an important distinction between base layer tiles and vector layer data is that only one base layer can be toggled on at a time. However, multiple vector layers can be displayed at a time - layering these data sets on top of each other.

The James River Association's Water Trail Map above demonstrates both point data and polygon vectors drawn on top of a Google Map base layer. The polygon lines in blue and green are stored as KML. We built this map with Drupal and OpenLayers.

Another important point to understand about vector data is that for most javascript map implementations, all available vector data must be downloaded to the site visitor's browser before any of this data can be rendered on the map. Consequently, larger vector data sets can slow down the site visitor's experience of your map, regardless of whether or not the data is initially displayed on the map or hidden.

Likewise, it's important to understand that it takes time and processing power for a web browser to draw all these vector points and polylines on the map. Even moderately large vector data sets can contain many thousands of lat/long pairs, so drawing larger shapes can sometimes slow down a user's experience of a map. As a result, for very complex data sets and shapes, it is often necessary to bake this data into the base layer tiles themselves.

"Baking base layer tile sets" used to be a herculean task outside the range of possibility for most smaller web mapping projects. However, with the release of new web cartography tools such as Tilemill (described below), creating base tile sets is within reach of projects with even modest development budgets.

Creating Your Own Base Layer Map Tiles

In my opinion, the most interesting advance in web mapping has been the development of open source cartography tools that allow even beginner technologists to become map makers.

Creation of beautiful "offline maps" and custom base layer tile sets has been available for many years with proprietary tools provided by companies such as ESRI. In just the last two years, organizations such as OpenPlans and Development Seed have released open source cartography tools the level the playing field with proprietary products.

Of these tools, Development Seed's Tilemill product is the most advanced and easy-to-use. With Tilemill, you can mash-up complex data sets (such as ESRI Shapefiles), as well as image files that contain geodata (such as GeoTIFFs) to create your own tiles. Crafting these maps is a lot like working with CSS. Once you upload data to Tilemill, you can style it - like presenting different features and styles at varying zoom levels.

Once you've created your own base tiles, you can then host (or serve them up to your website) yourself with Tilestream, Development Seed's open source map tile server, or host them with Development Seed's pay-as-you-go MapBox service. (For its modest price and sophisticated feature set, I highly recommend MapBox tile hosting.)

With Tilemill and MapBox, you can even turn images, such as the navigation chart above, into base map tile sets - and then mash these image-based tiles with your own point and polygon data.

Working with GeoData

Another historic challenge with web-based cartography has been working with geographic data. Given the dominance of ESRI products in the GIS industry, most complex public data sets are distributed in ESRI's proprietary Shapefile format. Shapefiles are a "binary format", like Microsoft Word documents or Adobe PDFs. In other words, you can't just open a Shapefile in a text editor and manipulate it, as you can with other geodata formats, such as KML.

Organizations such as OpenPlans and open source software developers have been investing heavily in the development of an open source GIS platform called OpenGeo. Using OpenGeo, even beginner GIS enthusiasts can manipulate Shapefile data and integrate that geodata into their custom Tilemill maps.

In the example map above, we used OpenGeo to stitch together US Geological Survey hydrological data originally in shapefile format. We then turned this data into base layer tiles with TileMill.

Other DIY Mapping Tools

While not open source, there are a number of other web-based mapping tools that emphasize open data and promote community-driven, DIY mapping projects.

CloudMade is freemium web service that allows you to create your own custom base tile sets, as well as embed sophisticated maps on your website. In addition to providing tile making tools, CloudMade provides access to a wealth of geographic data sets.

In 2011, MapQuest and OpenStreetMap began a partnership that has rebooted MapQuest's market position, while also bringing more attention to OpenStreetMap's mission of crowdsourcing the mapping of local communities around the world. With MapQuest and OpenStreetMap, you can quickly embed maps on your website that are comparable to Google Maps – without Google's more stringent terms of use.

Speaking of Google, it recently made a huge contribution to DIY mapping with the integration of Google Fusion Tables and the Google Maps API. With Google Fusion Tables, you can essentially leverage Google Spreadsheets to collect and manage very large geodata sets. With Fusion Tables's built-in support for Google Maps, rendering complex maps is highly performant. (Check out these examples.)

Whew, What's Next?

That's a lot of info, right? Where should you go from here? Well, I'd recommend coming to NTEN's 2012 NTC to learn more about mapping (among other things)! At the NTC, I'll be building upon these basic topics to talk about map-based storytelling strategies, as well as integration of mapping tools with content management systems and CRM tools.

Sean has been working with the best and brightest software engineers and designers in the Drupal community for the past five years. He has led national community organizing initiatives and international relief projects, served as a fundraising strategist for environmental groups worldwide, and run two open source software consultancies specializing in Drupal development. He holds a master of public administration (MPA) degree from Syracuse University's Maxwell School.