GIS programmer writing Python in Emacs.

It's quite different from CRUD stuff.

SOTM 2010: looking back

written by mishok13, on Jul 16, 2010 11:00:00 AM.

Was the State Of The Map fantastic or what?!

With all the incredibly smart people lurking around, having sangria and paella, talking about things I might never understand -- this must the Mecca for map geeks. Or, at least it looked like it this year.

Anyway, here's a more structured way of reporting.

Organization

Everything starting from drinks and food and finishing with timing and Internet worked as it should have. I think that's the highest praise any even organizer can get. Of course, there were some slight issues, such as lack of sockets in small room, but that only somehow decreased twitter activity.

Day 1

Day 1 was marked by several big announcements:

  • AOL's Map Quest will now use OpenStreetMap data. While they're still in beta, the plan is to use OSM for Germany and UK sites. The nice thing about MapQuest is that they sponsor development of Mapnik and Nominatim. [1]

  • Microsoft is going to add OpenStreetMap layer to Bing Maps. We'll have to wait until that happens, of course, but this is a good thing overall, considering the level of their cartography. I mean, just look at the "Sketchy" layer, you wont be able to resists its beauty.

  • Curly Brackets announced their own vector rendering engine for iOS 4. Some might say that this is not as big as the previous two announcements, but I think this marks the start of moving away from server-side to client-side rendering for the sake of delivering better maps experience.

I must admit that I've missed half of the talks on Friday because I was involved in Mapnik Birds of Feather discussion. I'm not going to summarize it, because that was already done beautifully by Richard Weait on Mapnik-devel list.

For those of you who missed SOTM 2010 I advise watching these talks (organizers say that the videos will be available in the next two-three weeks):

  • "Who are the mappers and why do they map in OpenStreetMap?" by Nama Raj Budhathoki, Muki Haklay and Zorica Nedovic-Budic

    In fact, this wasn't a real talk, just a summary of survey done in the past 3 or so months. The survey itself might be of low value, simply because of low numbers of people polled. But the analysis was pretty impressive and showed that most of the OSM mappers are in fact professional GIS specialists and/or cartographers.

  • "Uncommon Operating Picture: Using OSM for Humanitarian Response" by John Crowley

    I missed this one, but I'd heard only good things about it. I think that the more OpenStreetMap is used for the greater good (that means, not only target well-developed countries) the better it is for community and the map data itself.

  • "Map Kibera – Mapping one of Africa’s largest slums" by Mikel Maron

    Mikel showed everybody that we should move towards providing map data not only for the first-world countries. A very detailed and touching talk overall. Also, check out what has been done already at Map Kibera official site.

  • Panel discussion "Cocktails on the Titanic?" by Steven Feldman.

    Big people from Google, Microsoft, MapQuest and CloudMade tried to explain what business model will make OSM-based start-ups take off successfully. I must admit that Steven Feldman has mastered provocative questions and used this skill very often during the panel. [2]

At the end of the day, everybody was offered drinks and snacks on the terrace. Most of the people didn't get enough food and alcohol (mostly alcohol) and so the mappers continued to party in Girona bars.

Day 2

Saturday marked the start of non-business day, which means more interesting stuff for geeks.

There were lots of good talks, but the best are (warning: subjective!):

  • "OSM without Delay" by Matt Amos

    Matt is one of the core OSM server engineers and does a lot of interesting and useful things. He's currently working on OWL which is basically a visualizer of recent (or not so recent) changes in different areas. Matt talked about how the OWL implemented, pointing out the cool features. If you missed the talk, you should check out Matt's blog post, where he provides detailed explanation of his talk.

  • "Tag Central: a Schema for OSM" by David Earl

    This talk caused a lot of people wondering -- why doesn't David implement that instead of talking about it? The tagging schema has been discussed several hundreds times and in the end no one ever did implement it. This doesn't have to be part of the core functionality of OpenStreetMap anyway, so why not do it? Still, one of the best thought-out ideas about tagging schema I've ever seen.

  • "What’s wrong with OSM?" panel discussion lead by Christopher Osborne

    We all know what's wrong and in order to fix we have to BAN POTLATCH. Well, not really. At least, nobody was talking about editors as being big problems of the OSM. I must say that I can't remember the exact problems discussed by the attendees, but the word "do-ocracy" coined by Mikel Maron is probably the best description of OpenStreetMap as a community I've ever heard. I'm eagerly waiting for the video of this one and I really hope that all the question from audience were recorded.

Day 3

Sunday aka "The Big Match Day" was full of nice talks, but the ones I really liked were about cartography:

  • "What I’d like to do with Mapnik" by Steve Chilton

    What I'd like to do with Steve Chilton is make him the king of the world and be with it. I'm serious, this talk was about so many beautiful things not possible (yet) in Mapnik. And all those things are implementable and would make any digital map rendered by this wonderful library even more beautiful. I advise this talk to anyone interested in cartography as a great place to learn about the beauty of maps.

  • "What I learned making a real map on real paper for real people and real money" by David Earl

    This talk was pretty much in line with the talk from Steve Chilton, but paper maps of course need another type of cartography. So, this talk and the one about Mapnik IMHO were the best talks about maps as we know them -- images. Highly recommend this one!

  • "MapOSMatic: City Maps for the Masses" by Thomas and Maxime Petazzoni

    Two Linux hackers, with basic knowledge of Python and another 4 persons they knew gathered together and in 5 days they had one of the best community project of OSM -- MapOSMatic. Not only this is useful (printed maps with indexes are useful) but it's also opensource. The talk itself wasn't that bright and informative but you should watch it anyway -- how else would you know about the internals and the history of MapOSMatic?

The tradition of SOTM is to close the conference with the auction, on which the all those unused things -- such as two EEE PCs, several video recorders, banners, maps, etc -- are sold. And again, just like in 2009, Henk Hoff was very successful at selling things at very good prices.

People

I've met a lot of people I always wanted to see and also several people I've already seen before but desperately need to talk more. Among these people are Dane Springmeyer [3], Sean Gillies [4], Simon Willison [5], Richard Weait [6] and many more.

Summary

If you like maps, open source software, open data and smart people doing crazy things -- see you next year at State Of The Map 2011!

[1]

Official post at MapQuest development team blog.

[2]

See also Steve's report about the panel on his own blog

[3]

The mad genius behind current Mapnik development

[4]

Pretty nice guy doing Egyptology and writing crazy Python GIS tools at http://gispython.org

[5]

You know, the guy who started little-known Django framework development. By the way, he is on 18 (eighteen!) month honey moon, so if you read this Simon -- I ENVY YOU. Just wanted you to know.

[6]

Canadian map geek, who created lots of useful tutorials and howto on Mapnik. Check out his blog, btw: http://weait.com/

Overview of cartography systems for OSM

written by mishok13, on Mar 1, 2010 12:00:00 AM.

This post has been featured as research paper on my employee's research page

There have been a lot of talks and new projects built around maps and cartography services, but it seems like there's not a lot of interest in building beautiful maps. While I agree that not every person around has the skills to create perfect map, it's certainly nice to have tools to do that. This article will try to highlight current solutions and their highs and lows. But at first, some theory is necessary.

Overview

Definitions

Layers

Layers are essential part of any cartography style system. By using layers you can combine data that is contained in different projections or data source. Layers give you ability of defining the order in which map is rendered. Layers also give you enough abstraction over the pure data, which means that working with layers is way easier than simply querying data sources.

Rules

Rules define how certain kind of data should be rendered. For example, rule for primary highways on "roads" layer could contain description of coloring scheme, width of the rode, width and color of the border. You could also tune amount of shields rendered or the way corners are rendered. All of these parameters are described by the corresponding rule. Individual rules are often combined into so-called rulesets, which are then applied against different layers. This is often handy when you have several layers which contain similar data (e.g. two layers having different data sources -- PostGIS and ESRI shape files, both containing POI data).

Summary

So, what defines a good cartography styling?

  1. First of all, powerful rules. The more ways there're to change the output of the map, the better. One should understand that such power doesn't come for free, it requires performance trade-offs. So, we get to the second point

  2. rules should be simple enough so we can combine them into rulesets. Simple rules give us a bonus point of light performance degradation, while still having the ability to create complex rulesets.

  3. Third: layers should have the ability to use different data sources. The more data sources you can use, the easier it's for the users of the system to build maps.

Real-world examples

Mapnik

/images/mapnik.jpg

Overview

Mapnik is the rendering engine used by main OpenStreetMap view. The project was started in 2005 by Artem Pavlenko as a general map rendering framework. The number one objective of the project is to create a library that would be able to render beautiful maps. To achieve this, Anti-Grain Geometry library was chosen as a backend. Unfortunately, AGG author seems to have stopped active development of the library. Mapnik is written in C++ and has quite advanced Python bindings.

Style system

Mapnik uses XML files to define style of the map. These files contain two important sections: Layers and Styles. Styles section contains named rulesets, where every Style has several Rules that define rules (obviously). Layers contain description of layers in order of rendering (first defined -- first rendered). Mapnik layer support is a great example of abstraction that doesn't leak and IMHO should be used a reference when designing your own layer system. One additional note regarding font support in Mapnik. You can define several fall-back fonts for a given style file, using FontSet directive. This is used often for fonts with limited language support (e.g. DejaVu, which doesn't have complete Asian font support).

Pros & Cons

Pros

  • Great layer implementation

    Mapnik does layer support in the most correct way possible.

  • Font fall-back support

    This feature gives ability to use beautiful maps where appropriate, while still being able to support rendering of all symbols.

  • Pattern symbolizing support

    Gives ability to add shields on any geometrical figure the easy way.

Cons

  • Using XML for style files

    Choosing XML in 2005 may sounded like a good idea, when it was the fastest way of serializing data, but now, when we have efficient JSON and YAML parsers, sticking to XML is a mistake. Style files should be easy for people to comprehend, not computers.

  • Limited zoom support

    Mapnik doesn't support extended zoom syntax in Rule directives and thus you have to break DRY principle often when writing Mapnik styles.

  • Mixing data access and style information (layers and styles)

    Layers and Styles should be defined in different files. This is a minor inconvenience for most people, but as an employee of CloudMade I feel really strong about this one. Some time in the distant past CSS and HTML were mixed up in one file, but humans came to realizing the fact this is often wrong, and now most, if not all, sites are splitting HTML and CSS into separate files. I hope the same will happen to Mapnik's style files someday.

MapCSS

/images/mapcss.jpg

Overview

MapCSS is the style specifically developed for Halcyon rendering engine. The engine itself is being developed by Richard Fairhurst and is already being used for Potlatch 2 OpenStreetMap editing tool. The engine is written in ActionScript and uses Flex.

Style system

As seen from the name, MapCSS is a superset of CSS designed specifically for cartography purposes. The approach to defining rules is simple -- define selector which defines the subset of data and then define object properties (e.g. width, opacity, etc.). You can read about MapCSS selector and properties syntax on the OpenStreetMap wiki.

Pros & Cons

  • Powerful selector syntax

    Defining subsets of data is really easy with MapCSS and should regarded as the definite advantage. For example, you can use regular expressions in selectors.

  • eval support

    This gives a lot of opportunities for dynamically changing the style of the map. Calling eval can be regarded as a disadvantage from the performance point of view, but it makes defining styles a more pleasant experience.

Cons

  • OpenStreetMap API is the only datasource

    This is very unfortunate, but having no way of defining other data sources kills MapCSS's usage for anything else but OSM editing.

  • No explicit layer support

    This might be merged with the previous point. Having no explicit layer support limits areas of application for MapCSS.

  • No shield supports

    Shields are not supported yet in MapCSS, but I hope they will be there eventually

  • No grammar definition

    MapCSS looks like CSS, but it's not and it would be very nice to have complete description (besides notes on usage in OpenStreetMap wiki).

GSS

/images/gss.jpg

Overview

GSS (Geo Style Sheets) is a cartography style sheet developed by Cartagen team. To cite their wiki:

Cartagen lets you make beautiful, customized maps with a simple stylesheet

Style system

GSS is a subset of JavaScript, which tries to look like CSS

Pros & Cons

Pros

  • Datasources support through HTTP API

    Any datasource can be added by defining its HTTP API, which is simple and convenient at the same time.

  • Layers support

    Due to the dynamic nature of GSS, layers are extremely easy to add and are indeed built in client library.

Cons

  • Targeted at dynamic environments

    It's not often cartographers need dynamic maps, and I regard lack of interest to static content as a major disadvantage.

* Only OSM as datasourceUnlike MapCSS, datasource support is not tied intodata model and the only thing GSS cares about is correct HTTP APIprovider. Unfortunately, only OSM HTTP API is supported now.

Osmarender

/images/osmarender.jpg

Overview

Osmarender was designed for only one purpose -- that is, rendering OSM data. The engine itself is a bunch of XSLT scripts, which take OSM files as input and produce SVG maps as output.

Style system

Osmarender uses two files to create SVG map -- rules and styles. Rules file is used to define the subsets of data while styles file is a CSS which defines the way data will look.

Pros & Cons

Pros

  • Based on SVG

    SVG is not great, for all I know, but it's an established standard and a good example of how a markup language can be used. SVG gives a lot of geometry features for free, thus allowing to concentrate on the data itself rather than algorithms of its rendering.

  • Split of style and data information

    As was mentioned before, splitting data and presentation logic is a good thing and Osmarender follows good practices as well.

Cons

  • Using XML for style files

    That's the same point I raised with Mapnik style files. In short: each time you use XML for cartography, god kills a kitten.

  • Limited configuration options

    It's more of a problem with Osmarender itself, rather than the style, but I have to acknowledge that in order to make something unusual, you've got to be extremely knowledgeable about Osmarender internals.

  • Lacking projection support

    Osmarender styles don't support any kind of in-style reprojecting, which leads to unusual bugs (see below).

  • Only OSM files as datasource

See also

Summary

So, what defines a really good cartography system? There're several important characteristics:

  • CSS-like properties for styling -- almost all systems get it right

  • Good layer support -- this is hard to achieve while retaining readability, but I'm sure it will be done some day.

  • Support for all kinds of data sources -- building beautiful maps while relying only one data source is often frustrating.

  • Readability -- style systems should be designed for people, not computers.

All of these are essential to any cartography system and should be targeted at first design iteration. You should also think about font support, correct positioning of labels and shield, avoidance of sharp edges and many other things. Designing a good cartography system is hard, but it was done before and will be done from time to time.

This covers most popular open source cartography systems currently available in the wild. You might point to other solutions, such as Kosmos, pyrender and many others, but as they're not as popular or rather innovative, I didn't cover them in this article.