Geoindexing with MongoDB

1. Geoindexing with MongoDB Leszek Krupiński WebClusters 2012
2. About me
3. On-line since 1997
4. Funny times
5. 1 hr of internet for 1 USD
8. First social site: geocities
9. My first web page
10. What do I do now
11. Day-time job Managing team of developers for Polish Air Force
12. Side: consulting, optimizing, desiging
13. Buzzwords incoming!
14. The Internet 2008
15. Web 2.0
16. http://en.wikipedia.org/wiki/File:Web_2.0_Map.svg CC-BY-SA-2.5
17. Be social in your bedroom
18. alone.
19. The Internet 2012
20. Web 3.0
22. Why geospatial?
23. Needs shifted
24. Why? Because they could.
28. How to implement?
29. Database. Duh.
30. Keep, but also query
31. Is there a person at 53.438522,14.52198? Nope. Is there a person at 53.438522,14.52199? Nope. Is there a person at 53.438522,14.52199? Yeah, here’s Johnny!
32. Not too useful.
33. Give me nearby homies. Within the range of 1 km there is: • Al Gore (53.438625,14.52103) • Bill Clinton (53.432531,14.55127) • Johnny Bravo (53.438286,14.52363)
34. Now that’s better.
35. Geoindexing. Nothing new.
36. Oracle, PostreSQL, Lucene/Solr, even MySQL (via extensions)
37. SELECT c.holding_company, c.location FROM competitor c, bank b WHERE b.site_id = 1604 AND SDO_WITHIN_DISTANCE(c.location, b.location, ’distance=2 unit=mile’) = ’TRUE’ ORACLE
38. SQL is so last year
39. Let’s use something cool
40. MongoDB. Because all the cool kids use NoSQL now
42. Why MongoDB?
43. Choose your NoSQL wise.
44. NoSQL in MongoDB • Document –based • Queries (JS-like syntax) • JSON-like storage
45. Why MongoDB? Use Cases • Archiving • Event logging • Document and CMS • Gaming • High volume sites • Mobile • Operational datastore • Agile development • Real-time stats Features • Ad hoc queries • Indexing • Replication • Load Balancing • File Storage • Aggregation • Server-side JavaScript • Capped collections http://en.wikipedia.org/wiki/Mongodb
46. Back to geo.
47. { loc: [ 52.0, 21.0 ], name: ”Warsaw”, type: ”City” }
48. db.nodes.ensureIndex({loc: '2d'})
49. That’s it.
50. Query • Exact o db.places.find( { loc : [50,50] } ) • Near o db.places.find( { loc : { $near : [50,50] } } ) • Limit o db.places.find( { loc : { $near : [50,50] } } ).limit(20) • Distance o db.places.find( { loc : { $near : [50,50] , $maxDistance : 5 } } ).limit(20)
51. Compound index • db.places.ensureIndex( { location : "2d" , category : 1 } ); • db.places.find( { location : { $near : [50,50] }, category : 'coffee‚ } );
52. Bound queries • box = [ [40.73083, -73.99756], [40.741404, -73.988135] ] • db.places.find( {"loc" : {"$within" : {"$box" : box }} } )
53. Problems
54. Units
55. Coordinates in arc units Distance in kilometers
56. In query
57. earthRadius = 6378 // km multi = earthRadius * PI / 180.0 range = 3000 // km … maxDistance : range * multi…
58. In results
59. pointDistance = distances[0].dis / multi
60. Earth is not flat.
61. Problem: can’t use linear distance
62. Earth isn’t flat too.
63. Solution? Use approximation.
64. MongoDB has it built-in distances = db.runCommand( { geoNear : "points", near : [0, 0], spherical : true, maxDistance : range / earthRadius /* to radians */ } ).results
65. Focus: runCommand distances = db.runCommand({ geoNear : "points" …
66. Sort by distance Only with runCommand
67. Automatically sorted • db.runCommand( { geoNear : "places" , near : [50,50], num : 10 } ); • { "ns" : "test.places", "results" : [ { "dis" : 69.29646421910687, "obj" : … }, { "dis" : 69.29646421910687, "obj" : … }, … ], … }
68. Demo
69. OpenStreetMaps database of Poland imported into MongoDB
70. 14.411.552 nodes
71. 3GB of raw XML data
72. PHP in virtual machine
73. Imported about 100.000 nodes every 10s.
74. Pretty cool, eh?
75. Kudos to Derick Rethans Part of this talk was inspired by his talk
76. Questions?
77. Thanks! Rate me at https://joind.in/talk/view/6475
78. Geoindexing with MongoDB supplement Leszek Krupiński WebClusters 2012
79. Why MongoDB?
80. Evaluate.
81. PostGIS is cool too. (but it’s SQL, meh)
82. Why MongoDB? Use Cases • Archiving • Event logging • Document and CMS • Gaming • High volume sites • Mobile • Operational datastore • Agile development • Real-time stats Features • Ad hoc queries • Indexing • Replication • Load Balancing • File Storage • Aggregation • Server-side JavaScript • Capped collections http://en.wikipedia.org/wiki/Mongodb
83. If you need other features of MongoDB, use it
84. If you don’t, evaluate.
85. Evaluate.
86. Demo (hopefully)
87. Questions?
88. Please leave feedback! Rate me at https://joind.in/6475