Brighton 2016

Transcript

1. Distributed Search using Elasticsearch @Ashish_Thakur #drupal #elasticsearch
2. About Me { "_index": "drupal.org", "_type": "users", "_id": 945606, "_version": 1, "found": true, "_source": { "name": "Ashish Thakur", "company": "Srijan", "user_name": "a_thakur": "interests": [ "cooking", "mountain biking", "hiking", ] } } GET /drupal.org/users/945606
3. About Srijan • Based out of India • Largest Drupal Company in Asia • Open and transparent culture • Open source contribution is highly promoted
4. Introduction What is elasticsearch? @Ashish_Thakur #drupal #elasticsearch
5. Backbone @Ashish_Thakur #drupal #elasticsearch Creator Used By Quick Facts Shay Banon @kimchy
6. Elasticsearch Elasticsearch is your new best friend :) • Provides full text search easy by hiding complexities of Lucene behind a RESTful API • Not just a search engine • Distributed by nature: scales to multiple servers in a cluster and distributes data among shards and nodes with relative ease • Easy learning curve which leads to better productivity
7. Elasticsearch Terminologies Let’s get familiar with the terms • Indices • Types • Documents • Fields
8. Relational DB • Database • Tables • Rows • Columns Elasticsearch and Relational DB: A quick comparison Elasticsearch • Indices • Types • Documents • Fields
9. Elasticsearch up and running /bin/elasticsearch @Ashish_Thakur #drupal #elasticsearch
10. Horizontal and vertical scaling Source: http://pc-freak.net
11. Distributed Search and clustering /bin/elasticsearch #node1 /bin/elasticsearch #node2 /bin/elasticsearch #node3 @Ashish_Thakur #drupal #elasticsearch
12. Shards, Nodes and Cluster Source: Elasticsearch: The Definitive Guide by Clinton Gormley and Zachary Tong (O’Reilly). 3 node cluster with 3 primary shards and 1 replica shard
13. Distributed Nature Elasticsearch is distributed out of the box • No prior knowledge of distributed systems concepts like sharding, cluster discovery is required • Elasticsearch hides the complexity of distributed systems by ◦ Partitioning documents into different containers or shards ◦ Balancing the shards across nodes to spread indexing and search load ◦ Replicating each shard to provide redundancy, to prevent data loss ◦ Routing requests from nodes in the cluster to get the data you want ◦ Seamlessly integrating new nodes as your cluster grows or redistributing shards to recover from node loss
14. Life Inside a cluster • Elasticsearch is built to be always available, and to scale with your needs. • Scale can come from buying bigger servers (vertical scale, or scaling up) or from buying more servers (horizontal scale, or scaling out). • Vertical Scaling has it’s limit, real scaling can be achieved using horizontal scaling. • The ability to add more nodes to the cluster and to spread load and reliability between them.
15. Drupal + Elasticsearch #drupal #elasticsearch @Ashish_Thakur #drupal #elasticsearch
16. Modules Used • Elasticsearch Connector: https://www.drupal. org/project/elasticsearch_connector • Search API: https://www.drupal.org/project/search_api • Views: https://www.drupal.org/project/views
17. Elasticsearch connector Roadmap • Focus on D8 release • Abstraction based architecture for better portability • Have CI and automated tests in place
18. References • Elasticsearch: The Definitive Guide by Clinton Gormley and Zachary Tong (O’Reilly). Copyright 2015 Elasticsearch BV, 978-1-449-35854-9. • Elasticsearch Connector: https://www.drupal. org/project/elasticsearch_connector • Search API: https://www.drupal.org/project/search_api
19. ashish.thakur@srijan.net www.srijan.net Thank You @Ashish_Thakur #drupal #elasticsearch