Release 2.2.0

The Mapnik team is pleased to announce the Mapnik 2.2.0 release. This time around we’ve been focusing primarily on stability and performance, but there are some new cool features too. It’s fun this release coincides with this weekend’s SOTMUS in San Francisco and we hope to see you there!

release 2.2

Summary and Changelog

For more details on 2.2 features and fixes see the Summary and the Changelog.

Everyone is recommended to upgrade!

Download

Get it from downloads page.

Thanks to everyone involved!

Posted by Artem Pavlenko on 04 June 2013.

WKT geometry parsing benchmark

In addition to beautiful rendering capabilities Mapnik also offers a fast C++ and Python API for working with raw geo features. You can create them on the fly from a variety of formats including wkt, wkb, and geojson. This feature allows creative programers to use Mapnik like you would OGR to read or create data and output it to other formats.

Geometry parsing for datasources and tests

The WKT and GeoJSON geometry parsing support in Mapnik core makes it easy to support for these formats in datasource plugins like the Mapnik CSV plugin. And because the CSV plugin supports inline WKT strings you can write simple, standalone test cases for Mapnik bugs as easily as ticket #1484 demonstrates.

Geometry parsing performance

As more people find the Mapnik CSV plugin they start throwing more and more data at it. We need to ensure that our parsing is as fast as possible. So today I started on some benchmarks to figure out baselines for performance. The questions I’m interested in are:

  • If parsing WKT across many threads does performance decline or improve? (We’ve found in the past that std::locale locking can kill threaded performance when parsing XML strings)

  • Does performance differ depending on the C++ library used? (the emergence of clang++ and libc++ on OS X make this an easy and relevant test)

  • How does Mapnik WKT parsing compare to other common libraries that people use for parsing WKT geometries?

First look at benchmarking results

Today I created a first pass at a C++ benchmark. The code is on github. Please see the README for details on how to set it up and run the tests.

There are two tests of Mapnik and GEOS that parse a set of very simple WKT geometries 100 thousand times, one serially and the other broken into 10 threads of work that each parse 10 thousand times.

The results on my system (OS X 10.8 / clang++) were:

mapnik compiled against libc++, using -std=c++11

$ ./run
1) threaded -> mapnik: 640 milliseconds
2) threaded -> geos: 2470 milliseconds
3) mapnik: 2800 milliseconds
4) geos: 9150 milliseconds

CAVEAT: Geos is not linked against libc++ in this test

mapnik compiled against libstdc++, using -ansi

$ ./run
1) threaded -> mapnik: 880 milliseconds
2) threaded -> geos: 2520 milliseconds
3) mapnik: 3440 milliseconds
4) geos: 9200 milliseconds

NOTE: smaller numbers are better and indicate that the tests finished faster.

The results seem to indicate:

  • High concurrency, as you would hope and expect, does not hurt performance but rather helps. This is great, but not always the case in C++ when parsing strings. If your parsing code triggers mutex locks then concurrency like this can hurt.

  • Mapnik is more than 2x faster than GEOS for these simple test cases. So, the next step here is to 1) figure out if Mapnik continues to perform well on larger WKT strings and 2) make sure we are using the GEOS API correctly.

  • Mapnik is slightly faster when compiled against libc++ and built in C++11 mode (than if compiled against libstdc++ in -ansi mode. This is great - will be interesting to learn why. My guess is that boost::spirit, which mapnik uses internally to parse WKT is benefiting from c++11 features. Or libc++ is providing faster implementations of key functionality than the standard gnu libstdc++. Or both :)

Please feel free to check out the benchmarks, run them yourself, and suggest other caveats or improvements. File issues here to provide comments.

Posted by Dane Springmeyer on 19 April 2013.

Summer of Code 2012 - Summary

Google Summer of Code 2012 is over now. All the previous posts where rather technical so I would like to talk about applications and show some beautiful images in this post.

First of all: If you live in a region where names are spelled using only characters from the ASCII set you will not notice this work very much. It was one design goal not to change the behavior for texts that are already correctly rendered. There are some minor differences but nothing big. However it is still worth to read on as I added many more cool features.

Rendering complex scripts

Mapnik can render any script supported by HarfBuzz now. If something is rendered incorrectly check if the font supports the characters you want to use, as this is the most likely cause of any problems. Especially do not use Unifont! It doesn’t support any of the features required for correct rendering of most complex scripts.

I don’t publish example images here because I can’t judge which images would show the most improvement. If a native speaker wants to add samples where text that was broken before is really good now please open an issue on Github.com and I will publish it here.

There are some new features and bug fixes not related to complex scripts as well:

Improved line placements

Line breaks

Since a long time Mapnik is able to handle line breaks in the text (either automatically created or supplied in the input data) but this feature never worked for line placements. You could render a long town name very well, but long street names were a problem so far. This situation is improved by enabling this for line placements as well.

However there is one important limitation: You have to set wrap-width to a certain value or provide input text with line break characters (‘\n’). There is no function to automatically select the right wrap-width value depending on the line length yet. Automatic hyphenation of long text is also not available yet. This will be implemented at some later time.

Example image

Multiline street labels

Syntax

<TextSymbolizer wrap-width="100" ...>[name]</TextSymbolizer>

Correct offsets

Mapnik already supported offsetting text on lines, but the algorithm was very simple and only worked well for almost straight lines. The new code uses the offset converter class to produce real offsets which works for every line shape.

This feature can be used to produce labels for e.g. borders.

Example image

Country labels

Syntax

<TextSymbolizer dy="10" ...>[name]</TextSymbolizer>

Upright

For certain applications it is desirable to have a fixed orientation of the text with respect to the line direction. In the “offsets” example above you might have noticed that “Country 2” changes the text direction at one point in order to keep it upright.

This is useful behavior in most cases, but there are applications (like contour lines) where you don’t want to switch the direction. Therefore a new parameter “upright” was added to TextSymbolizer to choose the desired function.

Example image

contour lines

Syntax

<TextSymbolizer upright="auto" ...>[name]</TextSymbolizer> <TextSymbolizer upright="left" ...>[name]</TextSymbolizer> <TextSymbolizer upright="right" ...>[name]</TextSymbolizer>

Rotate displacement

Mapnik supports rotating text via the orientation parameter to TextSymbolizer. As long as the text is centered on the point being labeled the question which center to use for rotation is trivial. But once the text is moved from this position there are different possible points.

Up to now Mapnik always rotated around the label’s center but now one can also select to include the displacement when rotating text. This feature is best described by the following two images:

Example images

Rotate displacement: Off (default) rotate displacement off

Rotate displacement: On rotate displacement on

Syntax

<TextSymbolizer dx="5" rotate-displacement="true" ...>[name]</TextSymbolizer>

Kerning & Ligatures

Kerning and ligatures are two features you probably won’t notice if you don’t know about them. But they improve the text rending in subtle ways:

Kerning reduces the spacing between characters when one of them is top-heavy and the other one is bottom-heavy.

A ligature is the combination of two or more characters into a single glyph for improved visual appearance.

Example images

Kerning:

kerning

With ligatures:

ligatures

Without ligatures:

ligatures

Syntax

This feature is always enabled.

Shield Symbolizer

New syntax

ShieldSymbolizer syntax was documented and updated to reflect actual behavior. However it was noticed that the current syntax is rather counter-intuitive so a new, better one was defined. Previously “shield-dx” moved shield and text and “dx” moved text only. This was reversed so that “shield-dx” moves the shield only.

For “dx” (and “dy”) the behavior depends on the value of “unlock-image” (which had an undefined function before).

  • unlock-image=“false”: dx moves text and image
  • unlock-image=“true”: dx moves text only

This parameter is especially useful with the alternate positions functions described below.

Label position tolerance

Mapnik moves labels a bit when there is a collision at the designated place, however this feature only worked for TextSymbolizer but not for ShieldSymbolizer. This is fixed now.

This feature can be used for better labeling of highways with multiple names.

Example image

multiple shields

Syntax

<ShieldSymbolizer label-position-tolerance="100">[ref]</ShieldSymbolizer>

or automatically enabled for Symbolizers with non-zero spacing:

<ShieldSymbolizer spacing="200">[ref]</ShieldSymbolizer>

Alternate positions

ShieldSymbolizer was extended to support the alternate placements mechanism already implemented in TextSymbolizer. This feature is similar to the one above, but has different uses. It doesn’t simply move the shield along the way, but allows to set completely different parameters. You can change the font, offset, position, etc. (Selecting a different shield image or shield-offset is not supported, yet but will be in the future. You can use dx and dy in combination with unlock-image to get the desired effect in most cases.)

Example image

Labeling multiple bus routes at the same bus stop: bus routes

Syntax

<ShieldSymbolizer [...] placement-type="list" horizontal-alignment="middle" vertical-alignment="middle">[ref]
<Placement dx="50"/>
<Placement dx="100"/>
<Placement dx="0" dy="35"/>
<Placement dx="50" />
<Placement dx="100"/>
</ShieldSymbolizer>

This syntax is not optimal but an placement algorithm supporting grid placements will be added some time.

Installation

Download and install harfbuzz: wget http://www.freedesktop.org/software/harfbuzz/release/harfbuzz-0.9.4.tar.bz2 tar -xjf harfbuzz-0.9.4.tar.bz2 cd harfbuzz-0.9.4/ ./configure “ICU_CFLAGS=icu-config --cflags” “ICU_LIBS=icu-config --ldflags” make and as root make install

Download and install mapnik git clone -b harfbuzz –depth 1 git://github.com/mapnik/mapnik.git mapnik-harfbuzz cd mapnik-harfbuzz ./configure make and as root make install # Warning this overwrites previous mapnik installations!

Posted by Hermann Kraus on 06 October 2012.

Stamen, Compositing, and Mapnik 2.1

What if software existed that combined the power of Photoshop-like image effects with Illustrator-like vector transformations that was fully open source, spatially capable, and could handle big data like OpenStreetMap? Well, hopefully this future will feel near as you start using Mapnik 2.1.

A compositing framework in core

In time, Mapnik 0.7.x added support for blending modes in rasters thanks to the work of Marcin Rudowski. And as of last week, with the release of Mapnik 2.1 and a ton of work from Artem, we now support the comp-op property on all symbolizers (and Styles) making compositing possible for both vectors and rasters, together. This new support is a solid framework to build on, and more modes can now easily be added with few code changes.

Thanks Stamen

Our friends at Stamen Design have been advocating for the value of Photoshop-like alpha blending in mapping for many years. As far back as 2008 Mike was already beautifully working around limitations in Mapnik for terrain hillshading and proactively advocating for better support by writing new software like TileStache. Shawn was instrumental in helping me see the viability of compositing in Mapnik by helping sketching out ideas on the Mapnik wiki. Last week I saw Shawn advocating for the full suite of advanced photoshop blend modes in the upcoming CSS Spec, which is awesome and something we can also do in Mapnik.

Revisting innovations

After reflecting on how far we’ve come in Mapnik and thinking of Shawn’s work I was reminded of his sweet little project called Trees, Cabs & Crime where he experimented with rendering three themes of point data and compositing them together using subtractive blending to bring out patterns of correlation.

Trees, Cabs & Crime in San Francisco from Shawn Allen

Now that Mapnik 2.1 is out I figured it should be doable to render Shawn’s exact map with just a single Mapnik stylesheet.

Trees, Cabs & Crime re-rendered in Mapnik 2.1

After finding Shawn’s makefiles I saw the process should be as simple as reading data from three csv files and using a darkening blend mode as the canvas of each layer is rendered. Mapnik 2.1 newly supports reading direct from CSV files and one of the supported comp-op modes is darken, so matching Shawn’s exact output was pretty straightforward. You can find the full Mapnik XML below:

Folloups

And soon after this post Shawn got things working in TileMill 0.10.0 (which uses Mapnik 2.1).

Posted by Dane Springmeyer on 27 August 2012.

Release 2.1.0

release 2.1

The Mapnik team is pleased to announce the Mapnik 2.1 release! It was almost a year in the making and we have tons of new features including:

  • A new framework for Style level image manipulation called Image filters.
  • A new pipeline for chained coordinate transformations like clipping or smoothing called Vertex Converters
  • Compositing modes at Symbolizer and (experimental) Style level. Infinite possibilities from this, see a few examples from AJ.
  • Support for Style level opacity
  • WKT, WKB, GeoJSON, SVG parsers and generators that can be used outside of rendering
  • Data-driven SVG style transforms on svg markers and images thanks to @lightmare
  • Data-driven orientation for Text, height for Buildings, and width/height for Markers.
  • A new CSV input plugin
  • A new GeoJSON input plugin
  • A new Python input plugin
  • Better text labeling through support for placement-type="list". See the docs at the bottom of this page.
  • Improved Map loading speeds as well as warnings and errors

Summary and Changelog

For more details on 2.1 features and fixes see the Summary and the Changelog.

Everyone is recommended to upgrade!

Installing

With this release installing Mapnik has never been easier. For Mac users the popular Homebrew package system already includes Mapnik 2.1. Or, if you don’t want to wait for the compile just grab the one-click 64bit Framework Installer for OS X 10.6 -> 10.8.

For Ubuntu users, we have a PPA ready for you. Simply install Mapnik 2.1 like:

sudo apt-get install -y python-software-properties
echo 'yes' | sudo add-apt-repository ppa:mapnik/v2.1.0
sudo apt-get update
# install core mapnik
sudo apt-get install -y libmapnik mapnik-utils python-mapnik
# install the python binding
sudo apt-get install -y python-mapnik
# confirm mapnik-config returns 2.1.0
mapnik-config -v

And everyone can get the source from the downloads page.

Coming soon

We plan Windows binaries in the next month. We will work to provide two flavors:

  1. Mapnik built with Visual Studio 2008 that will provide python bindings for python.org provided Python 2.6 and 2.7
  2. Mapnik built with Visual Studio 2009 that will provide node.js bindings for nodejs.org provided Node v0.8.x

We also plan an OS X SDK installer (sometime in the next 3-4 months) that will include all headers and dependencies Mapnik uses in order to provide a drop-in development ease for XCode. Note: this SDK may instead target Mapnik 2.2.x development, we’ll see.

Try Tilemill

If you are a TileMill user, or interested in trying it out, the upcoming TileMill 0.10.0 release will use and expose Mapnik 2.1 features. Intrepid OS X and Windows are encouraged to try out the latest TileMill 0.10.x development builds available at the TileMill development snapshots download page.

Thanks!

Thanks to all the contributors and the many companies that have supported and encouraged Mapnik 2.1 development over the past year, notably: kosmosnimki, mapy.cz, mapquest, stamen, cartodb, MapBox, and many more.

Posted by Artem Pavlenko on 24 August 2012.