Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Add Location-Based Searching to Your Site with Elasticsearch

D12eaf3ef46e4f0fc6b714fd2b7ffe3b?s=47 derek-b
November 30, 2020
7

Add Location-Based Searching to Your Site with Elasticsearch

In this talk, we'll take an in-depth look at how Elasticsearch supports geo-searching. We'll look at the different types of location searches and how to integrate them into your application. By the end of this talk, you will be able to add location-based searches that will impress your users.

D12eaf3ef46e4f0fc6b714fd2b7ffe3b?s=128

derek-b

November 30, 2020
Tweet

Transcript

  1. @DerekB_WI Add Location-Based Searching to Your Site with Elasticsearch Presented

    by Derek Binkley
  2. @DerekB_WI Elasticsearch • Apache Lucene • REST calls • SDKs

    in many languages • Read optimized
  3. @DerekB_WI BELK Stack • Beats - Ingests data streams •

    Elasticsearch - You know for search • Logstash - Prebuilt dashboard for visualizing logs • Kibana - Browser based client
  4. @DerekB_WI Running • AWS - prebuilt • Hosted by Elastic

    • Install on OS • Docker *
  5. @DerekB_WI What Are We Building Today?

  6. @DerekB_WI WeAreDevelopers Project • Search text descriptions of locations •

    Search nearby locations • Add locations • Comment on existing locations • API
  7. @DerekB_WI docker-compose.yml version: '3'
 services:
 elasticsearch:
 image: docker.elastic.co/elasticsearch/elasticsearch:6.3.2
 environment:
 -

    cluster.name=docker-cluster
 - bootstrap.memory_lock=true
 - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
 ulimits:
 memlock:
 soft: -1
 hard: -1
 ports:
 - "9200:9200"
 kibana:
 image: docker.elastic.co/kibana/kibana:6.3.2
 ports:
 - "5601:5601"
  8. @DerekB_WI How is it stored? • Document - A JSON

    document for each data element • Index - A grouping of documents with similar structure • Mapping - Defines what is contained in document
  9. Define our data

  10. Define our data

  11. @DerekB_WI Data Types • Text - Content is indexed and

    searchable • Keyword - Fixed content, think lookup table • Geo-point - latitude/longitude • Geo-shape - Define a polygon • long, integer, short, byte, double, float, half_float, scaled_floa t, date, date_nanos, boolean, binary • More at https://www.elastic.co/guide/en/elasticsearch/ reference/current/mapping-types.html
  12. None
  13. @DerekB_WI Queries GET location-list/location/_search
 {
 "query": {
 "match_all": {}
 }


    }
  14. @DerekB_WI Queries GET location-list/location/_search
 {
 "query": {
 "match": {"name": "Big

    River"}
 }
 }
  15. @DerekB_WI

  16. @DerekB_WI Queries GET location-list/location/_search
 {
 "query": {
 "bool": {
 "filter":

    {
 "term": {"feature_class": "L"}
 }
 }
 }
 } https://www.geonames.org/export/codes.html
  17. @DerekB_WI Queries GET location-list/location/_search
 {
 "query": {
 "bool": {
 "must":{


    "match": {"name":"Little Park"}
 },
 "filter": {
 "term": {"feature_class": "L"}
 }
 }
 }
 }
  18. None
  19. @DerekB_WI Aggregations GET location-list/location/_search
 {
 "size": 0,
 "aggs": {
 "uniq_feature":

    {
 "terms": {"field":"feature_class"}
 }
 }
 }
  20. None
  21. @DerekB_WI Benefits? • Fast: > 1 mil rows in 20ms

    • Search within text • Other search types, geo, more like this
  22. @DerekB_WI Load Data public function addLocation(Location $location)
 {
 $this->locations[] =

    $location;
 
 if (count($this->locations) >= 1000)
 {
 $this->repository
 ->bulkInsert('location-list', 'location', 
 $this->prepareBulkJson($this->locations)); $this->locations = [];
 } }
  23. @DerekB_WI Load Data private function prepareBulkJson()
 {
 return array_map(function($location) {


    return [
 'geonameid' => $location->geonameid,
 'name' => $location->name,
 'asciiname' => $location->asciiname,
 'alternatenames' => $location->alternatenames,
 'location_point' => $location->latitude . ", " $location->longitude,
 'feature_class' => $location->featureClass,
 'feature_code' => $location->featureCode,
 'elevation' => $location->elevation,
 'timezone' => $location->timezone
 ];
 }, $this->locations);

  24. @DerekB_WI Load Data public function bulkInsert($indexName, $indexType, $data_lines) { $client

    = ClientBuilder::create() ->setHosts(['127.0.0.1:9200']) ->build(); $json_body = []; foreach ($data_lines as $line) { $json_body[] = ['index' => [ '_index' => $indexName, '_type' => $indexType ] ]; $json_body[] = $line; } $responses = $client->bulk(['body' => $json_body]); }
  25. @DerekB_WI Load Data public function bulkInsert($indexName, $indexType, $data_lines) { $client

    = ClientBuilder::create() ->setHosts(['127.0.0.1:9200']) ->build(); $json_body = []; foreach ($data_lines as $line) { $json_body[] = ['index' => ['_index' => $indexName, '_type' => $indexType] ]; $json_body[] = $line; } $responses = $client->bulk(['body' => $json_body]); }
  26. @DerekB_WI Load Data public function bulkInsert($indexName, $indexType, $data_lines) { $client

    = ClientBuilder::create() ->setHosts(['127.0.0.1:9200']) ->build(); $json_body = []; foreach ($data_lines as $line) { $json_body[] = ['index' => ['_index' => $indexName, '_type' => $indexType] ]; $json_body[] = $line; } $responses = $client->bulk(['body' => $json_body]); }
  27. None
  28. @DerekB_WI Search by Distance

  29. @DerekB_WI Text and Distance

  30. @DerekB_WI Closest Match

  31. @DerekB_WI Draw a Map

  32. None
  33. @DerekB_WI Add to Mapping

  34. None
  35. @DerekB_WI Thank You!