I’m going to Amsterdam for WhereCampEU this weekend. The attendee list looks awesome this year, I look forward to seeing old and new faces and talking about Mapnik/Tilemill and all things Geo. See you there :)

Following up last week’s post about smart halos, I wanted to share some ways of using compositing and filters together to achieve interesting effects.

As a starting point we use the smart halos map from the previous posting. You can find the features used in this post at compositing branch. Just be aware that these new features are work-in-progress and some syntax might change.

Deep water

Here is the original style:

Let’s look at water bodies and land. Solid fills are looking good, but what about adding some texture to the map? One way to add ‘texture’ is to use

<PolygonPatternSymbolizer/>

Let’s grab one of the lovely watercolour patterns and apply to ‘water’ features.

Very nice so far, perhaps a bit flat. Now time to add some depth. One of the very useful image processing tools is the ability to apply various blurs. In this case I use ‘agg-stack-blur’ with both x/y radius' set to 10 pixels:

<Style name="water-shadow" image-filters="agg-stack-blur:rx=10,ry=10" comp-op="dst-over">
    <Rule>
        <Filter>([HYC]=8)</Filter>
        <PolygonSymbolizer fill="lightblue" fill-opacity="0.4" smooth="0.7" transform="translate(0,0)" />
    </Rule>
</Style>

And lastly I add a bit of texture to the land fill to get that ‘paper’ feel to the final map :

I’m not sure the last step adds much to the cartography but it’s good to try different things. So go ahead and grab the latest source from compositing branch and have fun!

Ooh, one more.. I really like ‘blur’ filters, they seem to be so versatile. Here is a water-floating-shadow style using a combination of ‘emboss’ and ‘agg-stack-blur’ filters, and also applying transform on the shadow symbolizer

transform="translate(10,10)"

Floating water

Note, that while the water is floating in the air, all roads are on top of it – funky :)

This is a short post to share some techniques I came up with while implementing compositing and image filtering in Mapnik core.

The problem

Text rendered on top of a ‘busy’ map can become difficult to read, it can make colours appear heavy. One use-case would be place names on top of a dense road network, another text on top of height contours.

Consider the following map generated with :

python mapnik/demo/python/rundemo.py

(I modified text size to be 14px for clarity)

As you can see, dark text interferes with dark-ish minor roads and also casings on bigger roads.

One way to address this is to use text halos :

Now we have nice ‘punchy’ text but the rest of the features are getting overwhelmed. This is a sub-optimal solution, though, as halos saturate the image and attract far to much attention to the text items. What we really want is to punch-out the interfering layers around the text but leave the background intact.

This is better – now we have a map that is lighter on the eye and the text is easier to read. After all, this is the main purpose of a map – to convey information to the user.

Note that province and water polygons are underneath halos, while roads have been removed.

So how can we achieve this in Mapnik?

The solution

This is a well known cartographic issue and the solution in the paleo-geo world is referred to as “variable-depth masking”. I call it ‘smart-halo’ but the principle is the same.

There are possibly multiple ways to solve this, but here is one solution based on recent ‘compositing’ work I was concentrating on in Mapnik.

So here are step-by-step maps demonstrating the process.

1) We start with an empty map with fully transparent background – ‘transparent’ named colour or rgba(0,0,0,0). Then we render roads in the usual way.

NOTE: black background is due to conversion to JPEG format which doesn’t support alpha channel. It should appear as white

2) Right after roads we insert ‘smart-halo’ layer with the following style

<Style name="smart-halo" comp-op="dst_out">
    <Rule>
       <TextSymbolizer face-name="DejaVu Sans Book" halo-radius="3" halo-fill="white" fill="white"  size="14">[GEONAME]</TextSymbolizer>
    </Rule>
  </Style>

We use ‘dst_out’ compositing mode on this style to ‘punch-out’ roads or any other previously rendered objects.

3) Now we add provinces and water polygons using “dst_over” compositing operator, e.g for water polygons:

<PolygonSymbolizer fill="rgb(153,204,255)" fill-opacity="1.0" smooth="0.7" comp-op="dst_over"/>

4) And finally we render text and there you have it – smart-halos :D

The Mapnik team is pleased to announce Mapnik 2.0.1, the first stable bugfix release on top of 2.0.0. The notable thing about this release is that it fully supports PostGIS 2.0.

For details on other fixes see the Changelog.

Get the source from the downloads page.

Note for TileMill users: TileMill currently requires Mapnik >= 2.1.x and will not work with 2.0.1. So, if you are looking for a release for TileMill stay tuned and watch for 2.1 to be released once we can narrow down the 2.1 issue queue.

Optimizing map parsing performance

Much of my performance work on Mapnik in the recent year has focused on render-time speed.

But with the fast and wide adoption of TileMill as a style authoring environment, more and more people are demanding not only fast rendering but fast reloads for quick feedback on edits.

TileMill under the hood uses the Carto library to turn CSS into Mapnik XML. Although there is some overhead in this conversion, profiling shows most time is taken in transforming the XML into Mapnik C++ objects (into a mapnik.Map object). And of that time most is taken either in parsing filter rules or in parsing styles.

The two biggest problems turned out to be fairly low level:

Mutex locking in the std::locale constructor

This github ticket has details.

Basically, most C++ implementations have a global mutex that protects from race conditions if a locale is changed. This has a major impact on multithreaded programs that work with formatting strings (basically everything!). But TileMill’s usage of Mapnik is particularly susceptible to this problem because TileMill, when you hit save, immedially goes off and requests mapnik to load multiple maps in parallel to set up for parallel rendering threads. The large amount of string parsing to handle the map XML and rapid locking in multiple threads leads to unneeded and costly contention.

Since one cannot upgrade the libstdc++ (to use the new version that has a workaround for this bug) I mitigated the impact of these locks but removing all other potential uses of mutexes in Mapnik that might be contending. While you will not see many mutexes in Mapnik invoked during map loading we do use boost::lexical_cast in a number of places. It turns out boost::lexical cast is both very inefficient and uses exceptions and locking internally more than is needed. Replacing boost::lexical cast with faster type conversion has been promising in reducing the impact of locale locking.

Boost spirit grammars are expensive to create (and destroy)

This github ticket has details.

Basically, we use Boost spirit quite a bit to handle expression parsing and it is quite fast. But it turns out that the grammars used to guide the parsing have a non-trivial cost to create and destroy. In the case of mapnik.Color parsing creating the grammar took longer than the parsing. So, with the help of Herm, I refactored our parsing calls to create and reuse grammars once per map load rather than per parsed string.

Numbers

So, how much did this help?

Well, in an informal benchmark I took a large XML stylesheet used to style OSM data I had on hand.

The stylesheet is ~50,000 lines long and 3.1 MB in size.

I used node-mapnik (which supports loading maps synchronously and asynchronously) and timed loading this map 10 times using both methods.

With Mapnik 2.0.x (upcoming 2.0.1 release) (that is not optimized) the results were:

async: 19 seconds
sync: 31 seconds

But, with the optimized Mapnik 2.x code (current master, and upcoming 2.1 release) as of today the results were:

async: 3.5 seconds
sync: 7.2 seconds