Searching OpenStreetMap with Elasticsearch

Transcript

1. Searching OpenStreetMap
   with Elasticsearch
   photon.komoot.de
   Christoph Lingg – christoph@komoot.de
   

   View Slide

2. komoot
   

   View Slide

3. OpenStreetMap
   ● free
   ● editable map
   ● worldwide
   ● built by volunteers
   ● open-content licence
   

   View Slide

4. OpenStreetMap
   

   View Slide

5. OpenStreetMap
   

   View Slide

6. Geocoding
   „Böcklerpark“
   Coordinate(10.1 50.2)
   Park
   Kreuzberg, Berlin, Germany
   

   View Slide

7. Nominatim
   ● started many years ago
   ● standard solution for OSM
   ● based on Postgres/Postgis, SQL, PHP, C
   ● powerfull + mature
   ● drawbacks
   ○ slow
   ○ no typo tolerance
   ○ no partial match / search-as-you-type
   

   View Slide

8. Closing the Gap
   ● suitable for mobile usage
   ● “built on the shoulder of a giant”
   → elasticsearch
   ● open source
   ● free public API
   ● enhance the OSM ecosystem
   

   View Slide

9. {
   "name": "VIASKO",
   "housenumber": "33a",
   "amenity": "restaurant",
   "coordinate": [10, 52 ],
   "street": "Bahnhofstraße",
   "postcode": "10403",
   "city": "Berlin",
   "state": "Berlin",
   "country": "Germany",
   "context": "Kreuzberg",
   "importance": 0.1
   }
   OSM Entry
   {
   "name": "VIASKO",
   "housenumber": "33a",
   "amenity": "restaurant",
   "coordinate": [10, 52]
   }
   Photon Document
   

   View Slide

10. Photon Importer
    VIASKO
    Bahnhofstraße
    Postcode
    10403
    City
    Berlin
    State
    Country
    Germany
    Berlin
    Nominatim DB
    Postgis
    Elasticsearch
    Index
    Photon
    Importer
    

    View Slide

11. Type-ahead search
    "filter": {
    "photonngram": {
    "min_gram": "1",
    "type": "edgeNGram",
    "max_gram": "100"
    },...
    }
    

    View Slide

12. Typo Tolerance "query": {
    "bool": {
    "must": {
    "match": {
    "collector.default": {
    "fuzziness": 1,
    "query": "${query}",
    "analyzer": "search_ngram",
    "prefix_length": 2
    }
    }
    },
    "should":{
    "match": {
    "collector.${lang}.raw": {
    "query": "${query}",
    "boost": 100,
    "analyzer": "search_raw"
    }
    }
    }
    }
    

    View Slide

13. Location biased
    "score_mode": "multiply",
    "functions": [
    {
    "script_score": {
    "script": "1 + doc['importance'].value * 100"
    }
    },
    {
    "script_score": {
    "script": "
    dist = doc['coordinate'].distanceInKm(lat, lon);
    0.5 + ( 1.5 / (1.0 + dist/40.0) )",
    "params": {
    "lat": "${lat}",
    "lon": "${lon}"
    }
    }
    }
    ]
    

    View Slide

14. Photon Project
    ● 100 Mio documents
    ● 60 GB search index
    ● easy install
    ● photon.komoot.de
    ● github.com/komoot/photon
    ● want be informed for the next sprint?
    → christoph@komoot.de
    

    View Slide

15. komoot.de/hack15
    

    View Slide