Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Add Location-based Searching to Your PHP App with Elasticsearch

derek-b
February 08, 2020

Add Location-based Searching to Your PHP App with Elasticsearch

Searching based on a user's location is a feature of many websites and applications. This type of search can add relevancy and value to a website, especially when these results can include distance from a point, within a rectangle or other shape, combined with fast text searches. In this talk we'll take an in depth look at how Elasticsearch supports geosearching. We'll look at the different types of location searches and how to integrate them into your PHP application. By the end of this talk you will be able to add awesomely relevant location based searches that will impress your users.

derek-b

February 08, 2020
Tweet

More Decks by derek-b

Other Decks in Technology

Transcript

  1. @DerekB_WI
    Location Based
    Searching with
    ElasticSearch

    View Slide

  2. @DerekB_WI
    ElasticSearch
    • Apache Lucene

    • REST calls

    • SDKs in many languages

    • Read optimized

    View Slide

  3. @DerekB_WI
    BELK Stack
    • Beats - Ingests data streams

    • Elasticsearch - You know for search

    • Logstash - Prebuilt dashboard for visualizing logs

    • Kibana - Browser based client

    View Slide

  4. @DerekB_WI
    Running
    • AWS - prebuilt

    • Hosted by Elastic

    • Install on OS

    • Docker *

    View Slide

  5. @DerekB_WI
    What Are We Building
    Today?

    View Slide

  6. @DerekB_WI
    Project
    • Search nearby locations

    • Add locations

    • Comment on existing locations

    • API

    View Slide

  7. @DerekB_WI
    PHP Client
    $ composer require elasticsearch/elasticsearch:7.4.*

    View Slide

  8. @DerekB_WI
    docker-compose.yml
    version: '3'

    services:

    elasticsearch:

    image: docker.elastic.co/elasticsearch/elasticsearch:6.3.2

    environment:

    - cluster.name=docker-cluster

    - bootstrap.memory_lock=true

    - "ES_JAVA_OPTS=-Xms512m -Xmx512m"

    ulimits:

    memlock:

    soft: -1

    hard: -1

    ports:

    - "9200:9200"

    kibana:

    image: docker.elastic.co/kibana/kibana:6.3.2

    ports:

    - "5601:5601"

    View Slide

  9. @DerekB_WI
    How is it stored?
    • Document - A JSON document for each data element

    • Index - A grouping of documents with similar structure

    • Mapping - Defines what is contained in document

    View Slide

  10. Define our data

    View Slide

  11. Define our data

    View Slide

  12. @DerekB_WI
    Data Types
    • Text - Content is indexed and searchable

    • Keyword - Fixed content, think lookup table

    • Geo-point - latitude/longitude

    • Geo-shape - Define a polygon

    • long, integer, short, byte, double, float, half_float, scaled_floa
    t, date, date_nanos, boolean, binary

    • More at https://www.elastic.co/guide/en/elasticsearch/
    reference/current/mapping-types.html

    View Slide

  13. View Slide

  14. @DerekB_WI
    Queries
    GET location-list/location/_search

    {

    "query": {

    "match_all": {}

    }

    }

    View Slide

  15. @DerekB_WI
    Queries
    GET location-list/location/_search

    {

    "query": {

    "match": {"name": "Big River"}

    }

    }

    View Slide

  16. @DerekB_WI

    View Slide

  17. @DerekB_WI
    Queries
    GET location-list/location/_search

    {

    "query": {

    "bool": {

    "filter": {

    "term": {"feature_class": "L"}

    }

    }

    }

    }

    https://www.geonames.org/export/codes.html

    View Slide

  18. @DerekB_WI
    Queries
    GET location-list/location/_search

    {

    "query": {

    "bool": {

    "must":{

    "match": {"name":"Little Park"}

    },

    "filter": {

    "term": {"feature_class": "L"}

    }

    }

    }

    }

    View Slide

  19. View Slide

  20. @DerekB_WI
    Aggregations
    GET location-list/location/_search

    {

    "size": 0,

    "aggs": {

    "uniq_feature": {

    "terms": {"field":"feature_class"}

    }

    }

    }

    View Slide

  21. View Slide

  22. @DerekB_WI
    Benefits?
    • Fast - 2.4 mil rows in 15ms

    • Search within text

    • Other search types, geo, more like this

    View Slide

  23. @DerekB_WI
    Load Data
    public function addLocation(Location $location)

    {

    $this->locations[] = $location;


    if (count($this->locations) >= 1000)

    {

    $this->repository

    ->bulkInsert('location-list', 'location', 

    $this->prepareBulkJson($this->locations));

    $this->locations = [];

    }

    }

    View Slide

  24. @DerekB_WI
    Load Data
    private function prepareBulkJson()

    {

    return array_map(function($location) {

    return [

    'geonameid' => $location->geonameid,

    'name' => $location->name,

    'asciiname' => $location->asciiname,

    'alternatenames' => $location->alternatenames,

    'location_point' => $location->latitude . ", "
    $location->longitude,

    'feature_class' => $location->featureClass,

    'feature_code' => $location->featureCode,

    'elevation' => $location->elevation,

    'timezone' => $location->timezone

    ];

    }, $this->locations);


    View Slide

  25. @DerekB_WI
    Load Data
    public function bulkInsert($indexName, $indexType, $data_lines)

    {

    $client = ClientBuilder::create()

    ->setHosts(['127.0.0.1:9200'])

    ->build();

    $json_body = [];

    foreach ($data_lines as $line)

    {

    $json_body[] = ['index' => [

    '_index' => $indexName,

    '_type' => $indexType

    ]

    ];

    $json_body[] = $line;

    }

    $responses = $client->bulk(['body' => $json_body]);

    }

    View Slide

  26. @DerekB_WI
    Load Data
    public function bulkInsert($indexName, $indexType, $data_lines)

    {

    $client = ClientBuilder::create()

    ->setHosts(['127.0.0.1:9200'])

    ->build();

    $json_body = [];

    foreach ($data_lines as $line)

    {

    $json_body[] = ['index' => ['_index' => $indexName,

    '_type' => $indexType]

    ];

    $json_body[] = $line;

    }

    $responses = $client->bulk(['body' => $json_body]);

    }

    View Slide

  27. @DerekB_WI
    Load Data
    public function bulkInsert($indexName, $indexType, $data_lines)

    {

    $client = ClientBuilder::create()

    ->setHosts(['127.0.0.1:9200'])

    ->build();

    $json_body = [];

    foreach ($data_lines as $line)

    {

    $json_body[] = ['index' => ['_index' => $indexName,

    '_type' => $indexType]

    ];

    $json_body[] = $line;

    }

    $responses = $client->bulk(['body' => $json_body]);

    }

    View Slide

  28. View Slide

  29. View Slide

  30. @DerekB_WI
    Add to Mapping

    View Slide

  31. View Slide

  32. View Slide