Riviera JUG : Un moteur de recherche pour trouver

Talk given at Riviera JUG, Nice, FR

Vous cherchez toujours dans vos données avec des SELECT * FROM person WHERE name like '%david%pilato%" ?

Au delà des performances obtenues, êtes-vous certain de renvoyer les résultats les plus pertinents pour vos utilisateurs d'abord ?

Venez découvrir comment un moteur de recherche vous aidera à répondre aux questions posées par vos utilisateurs, de manière pertinente et efficace, tout en apportant des fonctionnalités d'analyse des résultats et ce, quelque soit le volume...

http://www.rivierajug.org/xwiki/bin/view/Main/201706-elasticsearch

Transcript

1. ‹#› pour chercher... Un moteur de recherche NoSQL David Pilato Developer | Evangelist @dadoonet
2. ‹#› pour chercher trouver... Un moteur de recherche NoSQL David Pilato Developer | Evangelist @dadoonet
3. ‹#›
4. 4 Elastic Stack 100% open source No enterprise edition All new versions with 5.0
5. 5 Elasticsearch Heart of the Elastic Stack Distributed, Scalable High-availability Multi-tenancy Developer Friendly Real-time, Full-text Search Aggregations
6. 6 Kibana Window into the Elastic Stack Visualize and analyze Geospatial Customize and Share Reports Graph Exploration UX to secure and manage the Elastic Stack Build Custom Apps
7. 7 Beats Lightweight data shippers Ship data from the source Ship and centralize in Elasticsearch Ship to Logstash for transformation and parsing Ship to Elastic Cloud Libbeat: API framework to build custom beats 30+ community Beats
8. 8 FILEBEAT Log Files METRICBEAT Metrics PACKETBEAT Network Data WINGLOGBEAT Window Events More than 30 community Beats and growing … Apachebeat, dockbeat, httpbeat, mysqlbeat, nginxbeat, redis beats, twitterbeat, and more
9. 9 Logstash Data processing pipeline Ingest data of all shapes, sizes, and sources Parse and dynamically transform data Transport data to any output Secure and encrypt data inputs Build your own pipeline More than 200+ plugins
10. 10 ES-Hadoop Elasticsearch for Hadoop Two-way connector Index Hadoop data in Elasticsearch Enable real-time search capabilities Visualize HDFS data in Kibana Snapshot and restore with HDFS Support for Spark, Storm MapReduce, and more
11. 11 X-Pack Single install Extensions for the Elastic Stack Subscription pricing Security Alerting Monitoring Reporting Graph Machine Learning
14. 14 Elastic Cloud Hosted Elasticsearch & Kibana Includes X-Pack features Starts at $45/mo Available in AWS today Available in Google Cloud Platform (soon) Available as a private cloud/on-premise solution (Elastic Cloud Enterprise)
15. 15 Elastic Cloud Enterprise Provision and manage multiple Elastic Stack environments; Expose logging as a service to your entire organization Available as a public beta
19. Open Source Basic Gold Platinum Elastic Stack Elasticsearch, Kibana, Beats, Logstash Elasticsearch, Kibana, Beats, Logstash Elasticsearch, Kibana, Beats, Logstash Elasticsearch, Kibana, Beats, Logstash X-Pack Monitoring ✔ ✔ + Multicluster + configurable data retention ✔ + Multicluster + configurable data retention Security ✔ ✔ + field/doc level security + custom realms Alerting ✔ ✔ Graph ✔ Reporting ✔ ✔ Machine Learning ✔ Dev Tools ✔ + Search Profiler ✔ + Search Profiler ✔ + Search Profiler Tile Service ✔ Zoom Level 10 ✔ Zoom Level 18 ✔ Zoom Level 18 ✔ Zoom Level 18 Support Business Hour 6 Support Contacts 4 Hour Critical Response 24/7/365 8 Support Contacts 1 Hour Critical Response Emergency Patches Basic Gold Platinum Open Source
21. Some data CREATE TABLE user ( name VARCHAR(100), comments VARCHAR(1000) ); INSERT INTO user VALUES ('David Pilato', 'Developer at elastic'); INSERT INTO user VALUES ('Malloum Laya', 'Worked with David at french customs service'); INSERT INTO user VALUES ('David Gageot', 'Engineer at Docker'); INSERT INTO user VALUES ('David David', 'Who is that guy?'); 21
22. Search on term SELECT * FROM user WHERE name="David"; Empty set (0,00 sec) 22 INSERT INTO user VALUES ('David Pilato', 'Developer at elastic'); INSERT INTO user VALUES ('Malloum Laya', 'Worked with David at french customs service'); INSERT INTO user VALUES ('David Gageot', 'Engineer at Docker'); INSERT INTO user VALUES ('David David', 'Who is that guy?');
23. Search like SELECT * FROM user WHERE name LIKE "%David%"; +--------------+----------------------+ | name | comments | +--------------+----------------------+ | David Pilato | Developer at elastic | | David Gageot | Engineer at Docker | | David David | Who is that guy? | +--------------+----------------------+ 23 INSERT INTO user VALUES ('David Pilato', 'Developer at elastic'); INSERT INTO user VALUES ('Malloum Laya', 'Worked with David at french customs service'); INSERT INTO user VALUES ('David Gageot', 'Engineer at Docker'); INSERT INTO user VALUES ('David David', 'Who is that guy?');
24. Search in two fields SELECT * FROM user WHERE name LIKE "%David%" OR comments LIKE "%David%"; +--------------+---------------------------------------------+ | name | comments | +--------------+---------------------------------------------+ | David Pilato | Developer at elastic | | Malloum Laya | Worked with David at french customs service | | David Gageot | Engineer at Docker | | David David | Who is that guy? | +--------------+---------------------------------------------+ 24 INSERT INTO user VALUES ('David Pilato', 'Developer at elastic'); INSERT INTO user VALUES ('Malloum Laya', 'Worked with David at french customs service'); INSERT INTO user VALUES ('David Gageot', 'Engineer at Docker'); INSERT INTO user VALUES ('David David', 'Who is that guy?');
26. Search in two fields SELECT * FROM user WHERE name LIKE "%David%" OR comments LIKE "%David%"; +--------------+---------------------------------------------+ | name | comments | +--------------+---------------------------------------------+ | David Pilato | Developer at elastic | | Malloum Laya | Worked with David at french customs service | | David Gageot | Engineer at Docker | | David David | Who is that guy? | +--------------+---------------------------------------------+ 26 INSERT INTO user VALUES ('David Pilato', 'Developer at elastic'); INSERT INTO user VALUES ('Malloum Laya', 'Worked with David at french customs service'); INSERT INTO user VALUES ('David Gageot', 'Engineer at Docker'); INSERT INTO user VALUES ('David David', 'Who is that guy?');
28. Search with typos SELECT * FROM user WHERE name LIKE "%Dadid%"; Empty set (0,00 sec) 28 INSERT INTO user VALUES ('David Pilato', 'Developer at elastic'); INSERT INTO user VALUES ('Malloum Laya', 'Worked with David at french customs service'); INSERT INTO user VALUES ('David Gageot', 'Engineer at Docker'); INSERT INTO user VALUES ('David David', 'Who is that guy?');
29. Search with typos SELECT * FROM user WHERE name LIKE "%_adid%" OR name LIKE "%D_did%" OR name LIKE "%Da_id%" OR name LIKE "%Dad_d%" OR name LIKE "%Dadi_%"; +--------------+----------------------+ | David Pilato | Developer at elastic | | David Gageot | Engineer at Docker | | David David | Who is that guy? | +--------------+----------------------+ 29 INSERT INTO user VALUES ('David Pilato', 'Developer at elastic'); INSERT INTO user VALUES ('Malloum Laya', 'Worked with David at french customs service'); INSERT INTO user VALUES ('David Gageot', 'Engineer at Docker'); INSERT INTO user VALUES ('David David', 'Who is that guy?');
31. ‹#› Methionylthreonylthreonylglutaminylarginyltyrosylglutamylserylleucylphenylalanylalanylglutaminylleuc yllysylglutamylarginyllysylglutamylglycylalanylphenylalanylvalylprolylphenylalanylvalylthreonylleucylgl ycylaspartylprolylglycylisoleucylglutamylglutaminylserylleucyllysylisoleucylaspartylthreonylleucylisoleu cylglutamylalanylglycylalanylaspartylalanylleucylglutamylleucylglycylisoleucylprolylphenylalanylseryla spartylprolylleucylalanylaspartylglycylprolylthreonylisoleucylglutaminylasparaginylalanylthreonylleucyl arginylalanylphenylalanylalanylalanylglycylvalylthreonylprolylalanylglutaminylcysteinylphenylalanylglu tamylmethionylleucylalanylleucylisoleucylarginylglutaminyllysylhistidylprolylthreonylisoleucylprolylisol eucylglycylleucylleucylmethionyltyrosylalanylasparaginylleucylvalylphenylalanylasparaginyllysylglycyli soleucylaspartylglutamylphenylalanyltyrosylalanylglutaminylcysteinylglutamyllysylvalylglycylvalylaspa rtylserylvalylleucylvalylalanylaspartylvalylprolylvalylglutaminylglutamylserylalanylprolylphenylalanylarg inylglutaminylalanylalanylleucylarginylhistidylasparaginylvalylalanylprolylisoleucylphenylalanylisoleuc ylcysteinylprolylprolylaspartylalanylaspartylaspartylaspartylleucylleucylarginylglutaminylisoleucylalany lseryltyrosylglycylarginylglycyltyrosylthreonyltyrosylleucylleucylserylarginylalanylglycylvalylthreonylgly cylalanylglutamylasparaginylarginylalanylalanylleucylprolylleucylasparaginylhistidylleucylvalylalanylly sylleucyllysylglutamyltyrosylasparaginylalanylalanylprolylprolylleucylglutaminylglycylphenylalanylglycy lisoleucylserylalanylprolylaspartylglutaminylvalyllysylalanylalanylisoleucylaspartylalanylglycylalanylala nylglycylalanylisoleucylserylglycylserylalanylisoleucylvalyllysylisoleucylisoleucylglutamylglutaminylhist idylasparaginylisoleucylglutamylprolylglutamyllysylmethionylleucylalanylalanylleucyllysylvalylphenylal anylvalylglutaminylprolylmethionyllysylalanylalanylthreonylarginylacetylseryltyrosylserylisoleucylthreo nylserylprolylserylglutaminylphenylalanylvalylphenylalanylleucylserylserylvalyltryptophylalanylaspartyl prolylisoleucylglutamylleucylleucylasparaginylvalylcysteinylthreonylserylserylleucylglycylasparaginylgl utaminylphenylalanylglutaminylthreonylglutaminylglutaminylalanylarginylthreonylthreonylglutaminylval ylglutaminylglutaminylphenylalanylserylglutaminylvalyltryptophyllysylprolylphenylalanylprolylglutaminy
33. Search for terms SELECT * FROM user WHERE name LIKE "%David Pilato%"; +--------------+----------------------+ | name | comments | +--------------+----------------------+ | David Pilato | Developer at elastic | +--------------+----------------------+ 33 INSERT INTO user VALUES ('David Pilato', 'Developer at elastic'); INSERT INTO user VALUES ('Malloum Laya', 'Worked with David at french customs service'); INSERT INTO user VALUES ('David Gageot', 'Engineer at Docker'); INSERT INTO user VALUES ('David David', 'Who is that guy?');
34. Search with inverted terms SELECT * FROM user WHERE name LIKE "%Pilato David%"; Empty set (0,00 sec) SELECT * FROM user WHERE name LIKE "%Pilato%David%"; Empty set (0,00 sec) 34 INSERT INTO user VALUES ('David Pilato', 'Developer at elastic'); INSERT INTO user VALUES ('Malloum Laya', 'Worked with David at french customs service'); INSERT INTO user VALUES ('David Gageot', 'Engineer at Docker'); INSERT INTO user VALUES ('David David', 'Who is that guy?');
35. Search for terms SELECT * FROM user WHERE name LIKE "%David%" AND name LIKE "%Pilato%"; +--------------+----------------------+ | name | comments | +--------------+----------------------+ | David Pilato | Developer at elastic | +--------------+----------------------+ 35 INSERT INTO user VALUES ('David Pilato', 'Developer at elastic'); INSERT INTO user VALUES ('Malloum Laya', 'Worked with David at french customs service'); INSERT INTO user VALUES ('David Gageot', 'Engineer at Docker'); INSERT INTO user VALUES ('David David', 'Who is that guy?');
36. 36 Synonyms
37. Synonyms SELECT * FROM user WHERE comments LIKE "%Engineer%"; +--------------+----------------------+ | name | comments | +--------------+----------------------+ | David Gageot | Engineer at Docker | +--------------+----------------------+ 37 INSERT INTO user VALUES ('David Pilato', 'Developer at elastic'); INSERT INTO user VALUES ('Malloum Laya', 'Worked with David at french customs service'); INSERT INTO user VALUES ('David Gageot', 'Engineer at Docker'); INSERT INTO user VALUES ('David David', 'Who is that guy?');
38. Synonyms SELECT * FROM user WHERE comments LIKE "%Developer%" OR comments LIKE "%Engineer%"; +--------------+----------------------+ | name | comments | +--------------+----------------------+ | David Pilato | Developer at elastic | | David Gageot | Engineer at Docker | +--------------+----------------------+ 38 INSERT INTO user VALUES ('David Pilato', 'Developer at elastic'); INSERT INTO user VALUES ('Malloum Laya', 'Worked with David at french customs service'); INSERT INTO user VALUES ('David Gageot', 'Engineer at Docker'); INSERT INTO user VALUES ('David David', 'Who is that guy?');
39. Search like within 1 000 000 000 records… SELECT * FROM user WHERE name LIKE "%David%"; 39
40. 40 40
41. We need a search engine. Not a database with some search capabilities. David Pilato, Riviera JUG, 13/06/2017
44. ‹#› pour trouver... Un moteur de recherche NoSQL David Pilato Developer | Evangelist @dadoonet