Add Location-based Searching to Your PHP App with Elasticsearch

Searching based on a user's location is a feature of many websites and applications. This type of search can add relevancy and value to a website, especially when these results can include distance from a point, within a rectangle or other shape, combined with fast text searches. In this talk we'll take an in depth look at how Elasticsearch supports geosearching. We'll look at the different types of location searches and how to integrate them into your PHP application. By the end of this talk you will be able to add awesomely relevant location based searches that will impress your users.

Transcript

1. @DerekB_WI
   Location Based
   Searching with
   ElasticSearch
   

   View Slide

2. @DerekB_WI
   ElasticSearch
   • Apache Lucene
   
   • REST calls
   
   • SDKs in many languages
   
   • Read optimized
   

   View Slide

3. @DerekB_WI
   BELK Stack
   • Beats - Ingests data streams
   
   • Elasticsearch - You know for search
   
   • Logstash - Prebuilt dashboard for visualizing logs
   
   • Kibana - Browser based client
   

   View Slide

4. @DerekB_WI
   Running
   • AWS - prebuilt
   
   • Hosted by Elastic
   
   • Install on OS
   
   • Docker *
   

   View Slide

5. @DerekB_WI
   What Are We Building
   Today?
   

   View Slide

6. @DerekB_WI
   Project
   • Search nearby locations
   
   • Add locations
   
   • Comment on existing locations
   
   • API
   

   View Slide

7. @DerekB_WI
   PHP Client
   $ composer require elasticsearch/elasticsearch:7.4.*
   

   View Slide

8. @DerekB_WI
   docker-compose.yml
   version: '3'

   services:

   elasticsearch:

   image: docker.elastic.co/elasticsearch/elasticsearch:6.3.2

   environment:

   - cluster.name=docker-cluster

   - bootstrap.memory_lock=true

   - "ES_JAVA_OPTS=-Xms512m -Xmx512m"

   ulimits:

   memlock:

   soft: -1

   hard: -1

   ports:

   - "9200:9200"

   kibana:

   image: docker.elastic.co/kibana/kibana:6.3.2

   ports:

   - "5601:5601"
   
   

   View Slide

9. @DerekB_WI
   How is it stored?
   • Document - A JSON document for each data element
   
   • Index - A grouping of documents with similar structure
   
   • Mapping - Defines what is contained in document
   

   View Slide

10. Define our data
    

    View Slide

11. Define our data
    

    View Slide

12. @DerekB_WI
    Data Types
    • Text - Content is indexed and searchable
    
    • Keyword - Fixed content, think lookup table
    
    • Geo-point - latitude/longitude
    
    • Geo-shape - Define a polygon
    
    • long, integer, short, byte, double, float, half_float, scaled_floa
    t, date, date_nanos, boolean, binary
    
    • More at https://www.elastic.co/guide/en/elasticsearch/
    reference/current/mapping-types.html
    

    View Slide

13. View Slide

14. @DerekB_WI
    Queries
    GET location-list/location/_search

    {

    "query": {

    "match_all": {}

    }

    }
    

    View Slide

15. @DerekB_WI
    Queries
    GET location-list/location/_search

    {

    "query": {

    "match": {"name": "Big River"}

    }

    }
    
    

    View Slide

16. @DerekB_WI
    

    View Slide

17. @DerekB_WI
    Queries
    GET location-list/location/_search

    {

    "query": {

    "bool": {

    "filter": {

    "term": {"feature_class": "L"}

    }

    }

    }

    }
    
    https://www.geonames.org/export/codes.html
    

    View Slide

18. @DerekB_WI
    Queries
    GET location-list/location/_search

    {

    "query": {

    "bool": {

    "must":{

    "match": {"name":"Little Park"}

    },

    "filter": {

    "term": {"feature_class": "L"}

    }

    }

    }

    }
    

    View Slide

19. View Slide

20. @DerekB_WI
    Aggregations
    GET location-list/location/_search

    {

    "size": 0,

    "aggs": {

    "uniq_feature": {

    "terms": {"field":"feature_class"}

    }

    }

    }
    

    View Slide

21. View Slide

22. @DerekB_WI
    Benefits?
    • Fast - 2.4 mil rows in 15ms
    
    • Search within text
    
    • Other search types, geo, more like this
    

    View Slide

23. @DerekB_WI
    Load Data
    public function addLocation(Location $location)

    {

    $this->locations[] = $location;

    

    if (count($this->locations) >= 1000)

    {

    $this->repository

    ->bulkInsert('location-list', 'location', 

    $this->prepareBulkJson($this->locations));
    
    $this->locations = [];

    }
    
    }
    
    

    View Slide

24. @DerekB_WI
    Load Data
    private function prepareBulkJson()

    {

    return array_map(function($location) {

    return [

    'geonameid' => $location->geonameid,

    'name' => $location->name,

    'asciiname' => $location->asciiname,

    'alternatenames' => $location->alternatenames,

    'location_point' => $location->latitude . ", "
    $location->longitude,

    'feature_class' => $location->featureClass,

    'feature_code' => $location->featureCode,

    'elevation' => $location->elevation,

    'timezone' => $location->timezone

    ];

    }, $this->locations);

    

    View Slide

25. @DerekB_WI
    Load Data
    public function bulkInsert($indexName, $indexType, $data_lines)
    
    {
    
    $client = ClientBuilder::create()
    
    ->setHosts(['127.0.0.1:9200'])
    
    ->build();
    
    $json_body = [];
    
    foreach ($data_lines as $line)
    
    {
    
    $json_body[] = ['index' => [
    
    '_index' => $indexName,
    
    '_type' => $indexType
    
    ]
    
    ];
    
    $json_body[] = $line;
    
    }
    
    $responses = $client->bulk(['body' => $json_body]);
    
    }
    
    

    View Slide

26. @DerekB_WI
    Load Data
    public function bulkInsert($indexName, $indexType, $data_lines)
    
    {
    
    $client = ClientBuilder::create()
    
    ->setHosts(['127.0.0.1:9200'])
    
    ->build();
    
    $json_body = [];
    
    foreach ($data_lines as $line)
    
    {
    
    $json_body[] = ['index' => ['_index' => $indexName,
    
    '_type' => $indexType]
    
    ];
    
    $json_body[] = $line;
    
    }
    
    $responses = $client->bulk(['body' => $json_body]);
    
    }
    
    

    View Slide

27. @DerekB_WI
    Load Data
    public function bulkInsert($indexName, $indexType, $data_lines)
    
    {
    
    $client = ClientBuilder::create()
    
    ->setHosts(['127.0.0.1:9200'])
    
    ->build();
    
    $json_body = [];
    
    foreach ($data_lines as $line)
    
    {
    
    $json_body[] = ['index' => ['_index' => $indexName,
    
    '_type' => $indexType]
    
    ];
    
    $json_body[] = $line;
    
    }
    
    $responses = $client->bulk(['body' => $json_body]);
    
    }
    
    

    View Slide

28. View Slide

29. View Slide

30. @DerekB_WI
    Add to Mapping
    

    View Slide

31. View Slide

32. View Slide