mapnik.org

Latest from mapnik development team

2012-02-10T00:00:00-08:00

Text placement re-factoring

Big merge this week : both text-placement (Herm) and feature_impl branches got pushed to the mapnik/master.

One of the goals for “text-placement” was to re-factor rather messy text placement code into more manageable chunks, and hopefully to make it easier to follow and implement new features. So, now we have a better framework for text labeling – let’s use it. A classic example is labeling multipolygons that make up one geographical entity; country, county or state etc. Here are some maps to demonstrate how this would work for the US states :

1) Labeling regression (actually it never worked properly but changes to how multi-geometry is processed in Mapnik exposed this):

2) Try and label every polygon:

3) Order polygons by “size” and place label in first largest polygon:

[3] is now current default behaviour in master, giving a much better cartographic result. We need to expose other possible options in XML/Python/Node etc.

Feature concept changes

New ‘context’ based feature implementation landed in Mapnik. The interface was preserved as much as possible to make it easier to update existing code: the following Python code shows the differences with the new approach:

>>> import mapnik
>>> ctx = mapnik.Context() # create Feature context
>>> ctx.push("NAME") # push some names, defining a schema 
>>> f = mapnik.Feature(ctx, 1) # create feature by passing context and ID
>>> f["NAME"] = "test"

The main gain here is a memory footprint and faster feature construction – no overhead of maintaining std::map per feature. Looking to the future, it will be interesting to see how we can leverage this for more robust/compact interchange formats.

Windows binaries (Release Candidate 0)

2011-11-29T00:00:00-08:00

Hello, Windows users! Thank you for your patience.

Following 2.0 release, we have been working hard to port more features, support more compilers, and ultimately to make the whole process of building and using Mapnik on Windows more fun.

So, here are the latest Windows 32-bit binaries (RC0) . They are built with Visual C++ 2008 Express and require vc90 runtime. The provided python bindings support for now just support Python version 2.7 32 bit (not 64 bit) which can be downloaded from here.

Cairo backend support

This feature has been around for a while in Mapnik but was missing from Windows binaries. Now it’s possible to generate PDF, PS and SVG on Windows, too.

Microsoft SQL Server

Currently through the Mapnik OGR plug-in it’s possible to load spatial data from MSSQL.

(Tested with Microsoft SQL Server 2012 RC0)

Layer options might look like:

mapnik.Ogr(string='MSSQL:server=VISTA\SQLEXPRESS;database=osm;trusted_connection=yes',layer='roads',encoding="latin1")

Give it a try and give us feedback!

PS. There is more “Windows” news on the way, so stay tuned.

Setup

Simply unzip the archive into: c:/mapnik-2.0.1rc0 and set your %PATH% to point to c:/mapnik-2.0.1rc0:

set PATH=%PATH%;c:\mapnik-2.0.1rc0\lib

Then you should be able to run the demo program:

cd c:\mapnik-2.0.1rc0\demo\c++
rundemo ..\..\lib\mapnik

If you want to use the python bindings then enable them like:

set PYTHONPATH=%PYTHONPATH%;c:\mapnik-2.0.1rc0\python\2.7\site-packages;

And make sure Python27 is on your path:

set PATH=%PATH%;c:\Python27

Then you should be able to run the python demo:

cd c:\mapnik-2.0.1rc0\demo\python
python rundemo.py

Release 2.0

2011-09-26T00:00:00-07:00

The Mapnik team is pleased to announce that Mapnik 2.0 is finally here! We've jumped from 0.7 to 2.0, not because we got confused with our numbers, but to reflect that this release represents a big step forward for the project. We believe Mapnik 2.0 will provide a much more flexible and expressive platform for creating beautiful maps. For full details see:

Thanks to all contributors and also a supersize thank-you as always to Dane Springmeyer for managing and inspiring this release. This is a recommended upgrade for all Mapnik users.

NOTE: This release is going out without Windows binaries just yet, as a source distribution only. We're planning to follow up with Windows and Mac OS X binaries asap.

Raster Reprojection lands in trunk

2011-09-16T00:00:00-07:00

Alberto Valverde has recently been hard at work getting initial support in trunk (to be part of Mapnik 2.x) for on the fly reprojection of rasters.

Thanks to Alberto for all his hard work on this and for his company Meteogrid for supporting his contributions to open source

Carto parser wrapup

2011-09-07T00:00:00-07:00

Similar to Herm I wanted to update the community on the status of my work this summer on the native carto parser. There has been a lot of progress since the midterm update and we are now very close to being completely compatible with the current javascript carto parser.

Recent work has focused on implementing the dynamic features of carto, namely variable and expression handling. A simple example demonstrating this processing is included in the gist below:

With these features implemented we are able to work with complex styles like the open_street_dc map from the tilemill project. While more debugging is necessary the parser is able to produce valid output which can be rendered:

carto C++:

carto javascript:

Hopefully we will be able to resolve the remaining issues in the near future with the goal of merge both the parser and generator into trunk to be part of Mapnik 2.1.

Text formating

2011-09-02T00:00:00-07:00

Now that Google Summer of Code is over I'd like to show you some of theresults:

I already posted information about the newtext placement system. No big changes happened here, just a fewbugfixes. I also added some testsand reference images.

Improving text formating was the second half of my project.

Currently when you want to label a peak like this

Name
Elevation

you define these two symbolizers:The image will look like this:

This usually works, but it has some limitations. For example let's see what happens when the name gets a little longer:

Mapnik breaks the long name and the elevation disappears, because the offset is not correct.

Now let's have a look at how it is done with the new placement system:You only define one TextSymbolizer but with a different syntax:(Note: 
 is the XML encoding of a newline character)

As you can see Mapnik automatically takes care of correct line spacing, no need to calculate offsets manually.

Backwards incompatible changes

To make this work some backwards incompatibilities had to be introduced.

Line spacing is no longer defined by the largest character in a certain text, but by the line spacing value defined in the font file. This should result in a nice line spacing by default, but if you have defined additional line spacing in your style you have to remove it or at least reduce it to get the same effect as before.
The text rendered by a TextSymbolizer is no longer given in the name attribute, but as a text node.
```
<TextSymbolizer name="[abc]" />
```
becomes
```
<TextSymbolizer>[abc]</TextSymbolizer>
```

Parameters you can change

You can change the following text attributes with the Format instruction:

face-name
size
character-spacing
line-spacing
opacity
wrap-character
text-transform
fill
halo-fill
halo-radius

But you are not limited to using a fixed format. You can implement your own class in C++ to do the formating. It's very simple:In the same way you can do custom text formating:

Next steps

The code is ready, now it's required to test it on larger styles to find any remaining bugs. Python bindings also have to be updated to fully support the new features. When this is done the code will be released as Mapnik 2.1 as it's to late in the release cycle to include it in Mapnik 2.0.

Bugs related to this work

Hello Ruby-Mapnik

2011-08-29T00:00:00-07:00

Did you know Mapnik has some exciting Ruby bindings taking shape on github?

Well, you should check them out. Elliot Laster is the fine fellow who has been working on them and over 200 tests are already passing.

After compiling mapnik trunk on OS X here is how I built the ruby bindings:

And using the Mapnik2/trunk nightlies on ubuntu here is how you can get things running:

Mapnik GSOC 2011 wrapup

2011-08-25T00:00:00-07:00

Mapnik GSOC wraps up this week. Its been an amazing summer. Stay tuned to this blog for detailed updates from the 2 students who completed their projects (one, the Windows Build System had to be put on hold). In the meantime see my debrief email to the Mapnik community lists here

Stylesheets and GSOC - Halfway

2011-07-18T00:00:00-07:00

As Herm and Roel have already posted, we've reached the halfway point of Google Summer of Code 2011 and all the projects are starting to come together nicely. My work with carto is also progressing and is rapidly approaching usability.

For the last couple of months of the GSOC my work has focused on two parallel and related tracks, developing both a generator (allows for translation of exisiting xml stylesheets into carto) and a parser for carto in C++. These code bases currently reside in two different github repositories (carto-generator, carto-parser).

Carto Generator

The current status of the generator is that it is a nearly feature complete and should be usable by anyone who is interested. As far as I know the only major feature missing from the example is the ability to handle metawriters which is being put off for the time being. However, what this part of the project is lacking is testing and I would certainly appreciate anyone with interest and XML styles to try running the example translator and let me know what breaks. The generator can be run as follows:

$ ./example tests/test.xml

Carto Parser

The second part of the project, and definitely the more important, is the development of a native C++ parser for carto stylesheets. The parsing of the stylesheets occurs in two steps, the first reads the file and builds an abstract syntax tree based on a language grammar and the second traverses the AST and builds the map and related styles.

Parser Next Steps

The current status of the carto parser is that the grammar and AST constructors have been completed with work still being done on the code that translates from the AST into a mapnik map. Hopefully in the next week or two translation of symbolizers will be completed which will allow translation of styles and allow for rendering of simple carto maps. Beyond that work will then be focused on adding carto's special sauce features like variables and mixins.

The current implementation allows for visualization of the AST in a variety of formats (depending on the input filetype) to aid in debugging. Particularly, translation of the AST into the dot format allows the use of graphviz to create visual tree diagrams of a carto style. For example the simple test.mss produces the following output:

which can be generated as follows (given you have graphviz installed):

$ ./example -o dot tests/test.mss > test.dot$ dot -Tgif test.dot > test.gif

This is a bit more interesting when applied to more complex styles like map.mss which results in this. Similar processing can be done with mml files but the results are much less exciting as the json format is more straightforward than carto.

Testing on OS X

Thanks to some recent work by Dane a statically linked OS X x64 binary is now available on github which will hopefully be useful for anyone interested in trying out the parser without the hassle of updating boost. We will do our best to keep the binary up-to-date as development continues and any testing and or feedback would be greatly appreciated.

Windows build system: halfway status

2011-07-17T00:00:00-07:00

Time files and we're already halfway through 2011's GSoC. There were quite a lotof reactions to the initial announcement of my project and to my previous postabout it, from people who were all excited to see Mapnik become easier to buildand use on Windows. It was very encouraging and my apologies for not gettingback to everybody yet; I thought it would be more be productive to sollicitfeedback on something working.

The implementation of everything that was planned was quite vague when theproject began at the end of May, but I'm really excited about how things areturning out. While I didn't quite get as far as I had liked, the genericness ofwhat I'm building to support Visual Studio for Mapnik is imo very cool and Ihope that other projects besides Mapnik can also take advantage of it. Of courseI'm focusing on the needs of the Mapnik project for now, and in this post I'dlike to explain how I see it work for future users trying to build Mapnik withVisual Studio, how it works under the hood, and what that means for the future.

How the new windows build system should work

I used my own experiences building not only Mapnik but quite a bit of other opensource products that are primarily developed on Linux to lay out a list ofrequirements that a system for supporting Visual Studio should fulfill. Orderedby importance, those were:

'one-click' deployment for the user. No long lists of steps to follow, or settings to change etc.
Only rudimentary Visual Studio experience required. It can be quite
Easy to use by the developers. For every action the library developers need to take to support another environment as the one that is mainly used for development, the chances that it will actually be maintained and work well go down. So ideally, support for Visual Studio should be transparant (require no extra work).
No extra dependencies or additional software to install. Everything should work with the tools that come with a standard Windows and Visual Studio installation; that means no installing Perl or Python, Cygwin, other build tools or utilities.
Transparency wrt the library dependecies - change as little as possible, and preferbly nothing, to any code to be sure that everything works as intended and tested by all respective developes. Unfortunately, in some cases where certain code isn't well-tested to build or function using Visual Studio there is no option but to make changes.
Not keeping a copy of library dependencies in the Mapnik source tree. With copies of code it's error-prone and laborous to keep up to date with new releases and security fixes; it's better to always use the canonical source.

Basic workflow

With these in mind, I have the following workflow in mind for anyone wanting tobuild Mapnik on windows:

Download and extract Mapnik release zip file from Mapnik website.
Download and extract all library dependencies automatically by running a batch file. This makes it 100% clear to the user that the code used is the original release as intended by the library developers, and which version of the code is used.
Configure and where needed patch the library code. By 'configure' I mean copying configuration files like the ubiquitous 'config.h' for which many libraries provide a 'config.h.vc' or similar skeleton; this mechanism isn't standardized though so to do these things manually, users would have to follow specific procedures that are slightly (or very) different for each library. By putting all these command in a simple batch file, it can all be automated but still easy to trace what is being done. By 'patching' I mean where needed make changes to source code get it to compile with Visual Studio. As mentioned before, unfortunately there are cases where there is no other option.
Open the Mapnik solution and click 'build', or alternatively build everything from the command line, again with a transparent batch file.

For the user that isn't interested in the behind-the-scenes, these steps canthemselves be put into a batch file so that after downloading and unzipping theMapnik release zip file all further steps are run automatically while the usergets coffee or browse's reddit.com/r/funny.

What this solution is not

Now, the naive way to go about this is write a bunch of batch files, prepare theVisual Studio solutions/project files manually and put them in the MapnikSubversion to be included in each release. This has a number of obviousshortcomings:

Requires developers to update project files manually, using Visual Studio, when new files are added to Mapnik.
Requires manual tracking of batch file contents, downloads, new releases of libraries etc.
Cannot be re-used - all one-off, only for Mapnik project.

Python to the rescue

So clearly this is not a viable strategy. Instead I wrote a Python program that,given a description of the source files that make up a program or library andsome settings like precompiler flags, will generate Visual Studio project filesand solutions, complete with the dependencies and all the rest required to buildthe program. The program, dubbed 'vs4os' for 'Visual Studio For Open Source',also generates the batch files mentioned above and WiX files, which are inputfiles for the open source WiX installer builder - a system to create MSIinstallers from an XML input description. Right now only Visual Studio 2008files are generated, but this abstraction allows easy generation of variousversions of files, too.

How VS2OS will be used

How do I see this being used in practice, then? The vs4os program is run at themoment the Mapnik (or any other program or library that uses vs4os) release ismake; for example as part of the 'make release' make rule. The project files andgenerated batch files are then distributed with the release. This way theend-user doesn't need to install or download anything besides the zip file. Ofcourse, developers could also choose to keep track of the generated files in thesource tree of a project; every time files are added to the project, the fileswould need to be re-generated, but that could easily be done as part of a cronjob or post-commit hook.

Next Steps

At the moment, the list of files that need to be in the project files need to bespecified manually. That means that the goal stated above about the system beingtransparent to the developers isn't fulfilled 100% yet. A possible extension tothe vs4os operations is that it would read the list of files from the main buildsystem used, for example the autotools chain or Scons. Often somewhere in such abuild system, there is a list somewhere of what files are included; by havingthe program read directly from there, there would be one canonical source offiles to build and nobody would have to even know that there is a parallel buildsystem being generated from the primary sources.

So, what is the working status of all this? As much as I would have liked tohave a complete Mapnik build system to show along with this post, there arestill a few project that I need to further describe in the vs4os input languageto get everything working from start to finish as described above. At the momentthere are about 20 libraries that are used when building Mapnik (directly orindirectly), and each of them has ideosyncracies in their build system that Ineed to delve into. Some, like ICU and Boost, have such complex system but alsofirst-class support for building with Visual Studio that I've decided not tore-invent the wheel there and use those as-is.

Following along

You can track the current stateof affairs in my branch in Mapnik SVN at svn.mapnik.org/branches/gsoc2011/roel;this branch does not contain the generated project files yet, so you need tohave Python installed and run windows\\vs4os_files\\build_all.bat to try it out.Once I have my first full working set of files done, I will put them upsomewhere and I hope that there are people who are willing to give buildingMapnik with Visual Studio a shot and send me their feedback.

I'm afraid this post has gotten way too long already, so I'll cut it short here.Of course I'm very interested in hearing from everybody who has contacted me orthe list with Visual Studio questions hwo you think about this all - will itserve your needs? Do you have any special requirements in your build environmentthat need to customized in a specific way?

As I mentioned in the beginning, I'm really excited about how this is turningout, and I hope that it will prove useful in the future both for the Mapnikproject and beyond.