Tileserver on a diet using node.js

Imagine you were to present large amounts of constantly changing, live data to the users on a web map. Imagine it was on a website with high traffic volume(83 millon page views per month) and high requirement on quick response time. What software would you use to solve this challenge?

This presentation will cover the journey that Hemnet, a leading real estate property portal in Sweden, took while remaking a vital part of the website. A journey, during which a number of existing map servers, such as Geoserver, were put on test, but were not fast and flexible enough. A journey, that ended with creating a custom tileserver with technologies like Mapnik to make it as fast and efficient as possible.

During the presentation we will cover the challenges we had and how we faced them with different technologies available. We’ll take a look at how we did performance tests and how we rolled everything out to the masses.

Transcript

1. Tileserver on a diet using node.js Igor Tihonov
2. Maps at Hemnet
3. Lots and lots of live data • 500 000 properties with coordinates • Constantly changing data, every minute • 100 000 000 page views with map per month • Searches that give 0 - 100 000+ results
4. Clustering
5. Raster to the rescue
6. +
7. Downsides • Slow rendering time, 5-10 seconds per tile • Search parameters • Database load • Responsiveness and Interactivity
8. The best of two worlds 0 - 500 vector drawing 500+ raster drawing
9. Demo
10. Tileserver with Node.js
11. First try • Mapnik • node-mapnik - https://github.com/mapnik/node-mapnik • https://github.com/mapnik/node-mapnik-sample-code • PostGIS datasource • In-memory Redis/CSV datasource
12. /for_sale:45bcaf7cef2dc14387f/10/563/301.png for_sale:45bcaf7cef2dc14387f -> “Search Results as CSV”
13. Memory issues 50 000 points - 500MB
14. From Mapnik to node-canvas • Filter our points for requested bounding box • Translate geographical coordinates to pixel coordinates • Pick the icon • Draw on canvas
15. Low memory usage 50 000 points - 5MB
16. CSV JSON
17. Performance tuning 50 - 100 ms
18. Drawing optimizations 10% of the points drawn
19. Multiple cores Tile server Tile server Tile server Tile server CPU core CPU core CPU core CPU core
20. JMeter http://jmeter.apache.org
21. Rendering order
22. Retina tiles http://a.tiles.hemnet.se/.../6/34/19.png http://a.tiles.hemnet.se/.../6/34/19@2x.png
23. End result Most tiles < 100 ms Peak load 30 000 tiles/minute
24. Not so hard?
25. Thank you! www.hemnet.se Email: igor@hemnet.se Twitter: @igorti