Recherche avancée pour votre application "legacy" - ElsassJUG

Comment mixer SQL et NoSQL sans faire la révolution ?

Cette "live coding" conférence vous montrera comment ajouter Elasticsearch à votre application existante sans changer vos habitudes. Vous aurez des fonctions de recherche avancées sans avoir à écrire du SQL complexe !

David partira d'une application RestX, Hibernate et MySQL et ajoutera Elasticsearch en live depuis la scène !

Talk give at ElsassJUG, Strasbourg
https://www.meetup.com/fr-FR/ElsassJUG/events/235671400/

Transcript

1. ‹#› Advanced search for your legacy application David Pilato Developer | Evangelist @dadoonet
7. 5 The only Elasticsearch as a Service offering powered by the creators of the Elastic Stack • Always runs on the latest software • One-click to scale/upgrade with no downtime • Free Kibana and backups every 30 minutes • Dedicated, SLA-based support • Easily add X-Pack features: security (Shield), alerting (Watcher), and monitoring (Marvel) • Pricing starts at $45 a month
8. Elastic Subscription Packages 6 Elastic Stack BASIC GOLD PLATINUM Elasticsearch,Kibana, Logstash, Beats ✓ ✓ ✓ X-Pack Marvel (Monitoring & Management) ✓ ✓ Multicluster Support >7 days data retention ✓ Multicluster Support >7 days data retention Shield (Security) ✓ ✓ Field/document security Custom realms Watcher (Alerting) ✓ ✓ Reporting ✓ ✓ Graph Analytics & Visualization ✓ Support Support coverage and response times provided by Elastic Support Engineers Business Hours 4 hour L1 response times 24/7/365 1 hour L1 response times Emergency patches
10. our legacy platform 8 WEB APP HTTP / REST JDBC DATABASE SQL
11. our legacy domain / database 9
12. our legacy application demo time $ git clone https://github.com/dadoonet/legacy-search.git $ git checkout 00-legacy $ mvn clean install jetty:run 10
13. 11 11
14. Some data CREATE TABLE user ( name VARCHAR(100), comments VARCHAR(1000) ); INSERT INTO user VALUES ('David Pilato', 'Developer at elastic'); INSERT INTO user VALUES ('Malloum Laya', 'Worked with David at french customs service'); INSERT INTO user VALUES ('David Gageot', 'Engineer at Docker'); INSERT INTO user VALUES ('David David', 'Who is that guy?'); 12
15. Search on term SELECT * FROM user WHERE name="David"; Empty set (0,00 sec) INSERT INTO user VALUES ('David Pilato', 'Developer at elastic'); INSERT INTO user VALUES ('Malloum Laya', 'Worked with David at french customs service'); INSERT INTO user VALUES ('David Gageot', 'Engineer at Docker'); INSERT INTO user VALUES ('David David', 'Who is that guy?'); 13
16. Search like SELECT * FROM user WHERE name LIKE "%David%"; +--------------+----------------------+ | name | comments | +--------------+----------------------+ | David Pilato | Developer at elastic | | David Gageot | Engineer at Docker | | David David | Who is that guy? | +--------------+----------------------+ INSERT INTO user VALUES ('David Pilato', 'Developer at elastic'); INSERT INTO user VALUES ('Malloum Laya', 'Worked with David at french customs service'); INSERT INTO user VALUES ('David Gageot', 'Engineer at Docker'); INSERT INTO user VALUES ('David David', 'Who is that guy?'); 14
17. Search like SELECT * FROM user WHERE name LIKE "%David%Pilato%"; +--------------+----------------------+ | name | comments | +--------------+----------------------+ | David Pilato | Developer at elastic | +--------------+----------------------+ INSERT INTO user VALUES ('David Pilato', 'Developer at elastic'); INSERT INTO user VALUES ('Malloum Laya', 'Worked with David at french customs service'); INSERT INTO user VALUES ('David Gageot', 'Engineer at Docker'); INSERT INTO user VALUES ('David David', 'Who is that guy?'); 15
18. Search like with inverted terms SELECT * FROM user WHERE name LIKE "%Pilato%David%"; Empty set (0,00 sec) INSERT INTO user VALUES ('David Pilato', 'Developer at elastic'); INSERT INTO user VALUES ('Malloum Laya', 'Worked with David at french customs service'); INSERT INTO user VALUES ('David Gageot', 'Engineer at Docker'); INSERT INTO user VALUES ('David David', 'Who is that guy?'); 16
19. Search in two fields SELECT * FROM user WHERE name LIKE "%David%" OR comments LIKE "%David%"; +--------------+---------------------------------------------+ | name | comments | +--------------+---------------------------------------------+ | David Pilato | Developer at elastic | | Malloum Laya | Worked with David at french customs service | | David Gageot | Engineer at Docker | | David David | Who is that guy? | +--------------+---------------------------------------------+ INSERT INTO user VALUES ('David Pilato', 'Developer at elastic'); INSERT INTO user VALUES ('Malloum Laya', 'Worked with David at french customs service'); INSERT INTO user VALUES ('David Gageot', 'Engineer at Docker'); INSERT INTO user VALUES ('David David', 'Who is that guy?'); 17
20. 18 18
21. 19 19
22. 20 20
23. Connecting with our app Architecture 21
24. DATABASE SQL ETL using a ETL WEB APP HTTP / REST JDBC ELASTICSEARCH REST / JSON 22
25. think document! • Change your mindset: ‒ Forget SQL! ‒ Index what you want to find • A document ‒ A JSON object ‒ Core field types (string, numbers, booleans) ‒ Complex field types (arrays, objects) ‒ Additional field types (geo points, geo shapes) 23 23
26. Do It Yourself Direct Connection 24
27. DATABASE SQL direct connection WEB APP HTTP / REST JDBC ES-CLIENT ELASTICSEARCH REST / JSON 25
28. JSON document design PUT /person/person/1 { "name":"Joe Pink", "dateOfBirth":"1971-12-26", "address_id":"2", "marketing_id":"3" } PUT /person/address/2 { "city":"Paris", "country":"France" } PUT /person/marketing/3 { "cars":1000, "food":1500 } 26
29. JSON document design PUT /person/person/1 { "name":"Joe Pink", "dateOfBirth":"1971-12-26", "address":{ "city":"Paris", "country":"France" }, "marketing":{ "cars":1000, "food":1500 } } 27
30. direct connection demo time $ git clone https://github.com/dadoonet/legacy-search.git $ git checkout 01-direct $ git checkout 02-bulk $ git checkout 03-mapping $ git checkout 04-aggs $ git checkout 05-compute $ mvn clean install jetty:run $ cat README.markdown 28
31. synchronous vs asynchronous 29
32. DATABASE SQL using brokers WEB APP HTTP / REST JDBC ES-CLIENT ELASTICSEARCH REST / JSON 30
33. ELASTICSEARCH REST / JSON using brokers WEB APP HTTP / REST JDBC ES-CLIENT DATABASE SQL 31
34. ELASTICSEARCH REST / JSON using brokers WEB APP HTTP / REST JDBC ES-CLIENT DATABASE SQL 32
36. ‹#› Thanks! https://www.elastic.co/subscriptions https://github.com/dadoonet/legacy-search.git David Pilato Developer | Evangelist @dadoonet