Advanced Search for your Legacy application - Joker 2016

Talk given at Joker 2016, Saint Petersburg, Russia
http://jokerconf.com/en/talks/advanced-search-for-your-legacy-application/

How do you mix SQL and NoSQL worlds without starting a messy revolution?

This live coding talk will show you how to add Elasticsearch to your legacy application without changing all your current development habits. Your application will suddenly have advanced search features, all without the need to write complex SQL code!

David will start from a RestX, Hibernate and Postgresql/MySQL based application and will add a complete integration of Elasticsearch.

Transcript

1. ‹#› Advanced search for your legacy application David Pilato Developer | Evangelist @dadoonet
3. our legacy platform 3 WEB APP HTTP / REST JDBC DATABASE SQL
4. our legacy domain / database 4
5. our legacy application demo time $ git clone https://github.com/dadoonet/legacy-search.git $ git checkout 00-legacy $ mvn clean install jetty:run 5
6. 6 6
7. Some data CREATE TABLE user ( name VARCHAR(100), comments VARCHAR(1000) ); INSERT INTO user VALUES ('David Pilato', 'Developer at elastic'); INSERT INTO user VALUES ('Malloum Laya', 'Worked with David at french customs service'); INSERT INTO user VALUES ('David Gageot', 'Engineer at Docker'); INSERT INTO user VALUES ('David David', 'Who is that guy?'); 7
8. Search on term SELECT * FROM user WHERE name="David"; Empty set (0,00 sec) INSERT INTO user VALUES ('David Pilato', 'Developer at elastic'); INSERT INTO user VALUES ('Malloum Laya', 'Worked with David at french customs service'); INSERT INTO user VALUES ('David Gageot', 'Engineer at Docker'); INSERT INTO user VALUES ('David David', 'Who is that guy?'); 8
9. Search like SELECT * FROM user WHERE name LIKE "%David%"; +--------------+----------------------+ | name | comments | +--------------+----------------------+ | David Pilato | Developer at elastic | | David Gageot | Engineer at Docker | | David David | Who is that guy? | +--------------+----------------------+ INSERT INTO user VALUES ('David Pilato', 'Developer at elastic'); INSERT INTO user VALUES ('Malloum Laya', 'Worked with David at french customs service'); INSERT INTO user VALUES ('David Gageot', 'Engineer at Docker'); INSERT INTO user VALUES ('David David', 'Who is that guy?'); 9
10. Search like SELECT * FROM user WHERE name LIKE "%David%Pilato%"; +--------------+----------------------+ | name | comments | +--------------+----------------------+ | David Pilato | Developer at elastic | +--------------+----------------------+ INSERT INTO user VALUES ('David Pilato', 'Developer at elastic'); INSERT INTO user VALUES ('Malloum Laya', 'Worked with David at french customs service'); INSERT INTO user VALUES ('David Gageot', 'Engineer at Docker'); INSERT INTO user VALUES ('David David', 'Who is that guy?'); 10
11. Search like with inverted terms SELECT * FROM user WHERE name LIKE "%Pilato%David%"; Empty set (0,00 sec) INSERT INTO user VALUES ('David Pilato', 'Developer at elastic'); INSERT INTO user VALUES ('Malloum Laya', 'Worked with David at french customs service'); INSERT INTO user VALUES ('David Gageot', 'Engineer at Docker'); INSERT INTO user VALUES ('David David', 'Who is that guy?'); 11
12. Search in two fields SELECT * FROM user WHERE name LIKE "%David%" OR comments LIKE "%David%"; +--------------+---------------------------------------------+ | name | comments | +--------------+---------------------------------------------+ | David Pilato | Developer at elastic | | Malloum Laya | Worked with David at french customs service | | David Gageot | Engineer at Docker | | David David | Who is that guy? | +--------------+---------------------------------------------+ INSERT INTO user VALUES ('David Pilato', 'Developer at elastic'); INSERT INTO user VALUES ('Malloum Laya', 'Worked with David at french customs service'); INSERT INTO user VALUES ('David Gageot', 'Engineer at Docker'); INSERT INTO user VALUES ('David David', 'Who is that guy?'); 12
13. 13 13
14. 14 14
15. 15 15
16. Connecting with our app Architecture 16
17. DATABASE SQL ETL using a ETL WEB APP HTTP / REST JDBC ELASTICSEARCH REST / JSON 17
18. think document! • Change your mindset: ‒ Forget SQL! ‒ Index what you want to find • A document ‒ A JSON object ‒ Core field types (string, numbers, booleans) ‒ Complex field types (arrays, objects) ‒ Additional field types (geo points, geo shapes) 18 18
19. Do It Yourself Direct Connection 19
20. DATABASE SQL direct connection WEB APP HTTP / REST JDBC ES-CLIENT ELASTICSEARCH REST / JSON 20
21. JSON document design PUT /person/person/1 { "name":"Joe Pink", "dateOfBirth":"1971-12-26", "address_id":"2", "marketing_id":"3" } PUT /person/address/2 { "city":"Paris", "country":"France" } PUT /person/marketing/3 { "cars":1000, "food":1500 } 21
22. JSON document design PUT /person/person/1 { "name":"Joe Pink", "dateOfBirth":"1971-12-26", "address":{ "city":"Paris", "country":"France" }, "marketing":{ "cars":1000, "food":1500 } } 22
23. direct connection demo time $ git clone https://github.com/dadoonet/legacy-search.git $ git checkout 01-direct $ git checkout 02-bulk $ git checkout 03-mapping $ git checkout 04-aggs $ git checkout 05-compute $ mvn clean install jetty:run $ cat README.markdown 23
24. synchronous vs asynchronous 24
25. DATABASE SQL using brokers WEB APP HTTP / REST JDBC ES-CLIENT ELASTICSEARCH REST / JSON 25
26. ELASTICSEARCH REST / JSON using brokers WEB APP HTTP / REST JDBC ES-CLIENT DATABASE SQL 26
27. ELASTICSEARCH REST / JSON using brokers WEB APP HTTP / REST JDBC ES-CLIENT DATABASE SQL 27
28. ‹#› 28
29. ‹#› 29
30. ‹#› спасибо! https://www.elastic.co/subscriptions https://github.com/dadoonet/legacy-search.git David Pilato Developer | Evangelist @dadoonet