Elastic{ON}Tour Tokyo 2017 - Elasticsearch Deep Dive

Elasticsearch 5.xでどのような機能や変更が行われたのか。アップグレードやスケール、リカバリ、ディスク使用量などの懸念事項について、6.xではどのように進化していくのかを解説します。

Jun Ohtani | Developer, Evangelist| Elastic

Transcript

1. Jun Ohtani Developer | Evangelist Elasticsearch: Past, Present, & Future
2. Elasticsearch 5.0 2016/10 ϦϦʔε
3. 3 ਺஋ʹؔ͢Δվળ ҆શୈҰ ΑΓ؆୯ʹ Elasticsearch 5.0
4. Elasticsearch 5.x: ༷ʑͳػೳ௥Ճ 2017/09 5.6 ϦϦʔε
5. 5 Elasticsearch 5.x Still ^ •  Keyword ਖ਼نԽ •  ౷Ұ͞ΕͨϋΠϥΠλʔ •  ΧςΰϦͳͲʹΑΔ݁Ռͷू໿ •  ྨٛޠʹؔ͢Δվળ •  ݕࡧͷΩϟϯηϧ •  ฒྻ࣮ߦͰ͖Δscroll΍reindex
6. 6 Elasticsearch 5.x Still ^ •  ਺஋΍೔෇ͷൣғϑΟʔϧυ •  ൣғݕࡧʹؔ͢ΔΫΤϦ࠷దԽ •  େྔͷaggsʹؔ͢Δ෼ׂऔಘ •  geo-distanceͷιʔτͷ଎౓޲্ •  geo-ip lookupͷ଎౓޲্ and for logs and for numbers and for geo and ... ^
7. 7 ΑΓεέʔϧͤ͞΍͘͢ •  ෳ਺ͷΫϥελ΍Ϧιʔεͷ෼཭ •  ؅ཧΛΑΓ؆୯ʹ •  ΞοϓάϨʔυΛΑΓ؆୯ʹ •  ఀࢭظؒΛΑΓখ͘͞ •  ΫϥελΛ·͙ͨΫΤϦ
8. Cluster UK Master Nodes Data Node Data Node Data Node Tribe Node t1 Node Client Cluster US Master Nodes Data Node Data Node Data Node t2 Node Client 8 Tribe Node Merged Cluster State! Kibana
9. 9 Cross-Cluster Search Cluster UK Master Nodes Data Node Data Node Data Node Master/Data Node Cluster US Master Nodes Data Node Data Node Data Node Master/Data Node Kibana Optional dedicated cross-cluster search cluster
10. Elasticsearch 6.0+ ৽όʔδϣϯͷվળ
11. © Marie-Lan Nguyen Wikimedia Commons / CC-BY 2.5 11 ओͳ೰Έͷλω͸ʁ •  σΟεΫ࢖༻ྔ •  ݎ࿚ੑ΍ϦΧόϦ •  ϝδϟʔόʔδϣϯΞοϓάϨʔυ
12. © Marie-Lan Nguyen Wikimedia Commons / CC-BY 2.5 12 ओͳ೰Έͷλω͸ʁ •  σΟεΫ࢖༻ྔ •  ݎ࿚ੑ΍ϦΧόϦ •  ϝδϟʔόʔδϣϯΞοϓάϨʔυ
13. 13 Doc Values - ૄͳσʔλ (5.x) Segment 1 ID fname lname 1 Shane Connelly 2 Shay Banon 3 Tanya Bragin Segment 2 ID fname lname mi state city 4 Steve Kearns Null Null Boston 5 George Burdell P GA Null 6 Frasier Crane Null WA Seattle Merged Segment 3 Docs fname lname mi state city 1 Shane Connelly Null Null Null 2 Shay Banon Null Null Null 3 Tanya Bragin Null Null Null 4 Steve Kearns Null Null Boston 5 George Burdell P GA Null 6 Frasier Crane Null WA Seattle
14. 14 MetricbeatͷσʔληοτͰͷαϯϓϧ Apples-to-apples index size improvements 0 25 50 75 100 5.x 6.0 OOTB _all disabled
15. 15 σΟεΫ࢖༻ྔͷվળ •  ૄͳσʔλʹର͢Δαϙʔτ •  _allϑΟʔϧυΛσϑΥϧτͰແޮʹ •  ֤छσʔλొ࿥ʹޮՌతͳઃఆʹ •  কདྷɿϩʔϧΞοϓػೳ © Tony Weman / CC-BY 2.5
16. © Marie-Lan Nguyen Wikimedia Commons / CC-BY 2.5 16 ओͳ೰Έͷλω͸ʁ •  σΟεΫ࢖༻ྔ •  ݎ࿚ੑ΍ϦΧόϦ •  ϝδϟʔόʔδϣϯΞοϓάϨʔυ
17. 17 ϦΧόϦ (5.x) Segment 1 Segment 1 Segment 2 Segment 2 Segment 3 Segment 3 Primary Replica
18. 18 ϦΧόϦ (5.x) Segment 1 Segment 1 Segment 2 Segment 2 Segment 3 Segment 3 Offline Primary
19. 19 ϦΧόϦ (5.x) Segment 1 Segment 1 Segment 2 Segment 4 Segment 3 Primary Segment 1 Segment 2 Segment 3 Offline
20. 20 ϦΧόϦ (5.x) Segment 1 Segment 1 Segment 2 Segment 4 Segment 3 Primary Segment 1 Segment 2 Segment 3 Repica ϑΝΠϧίϐʔ
21. 21 ϦΧόϦ (6.x) Primary Repica Transaction-log ϕʔε 1! 2! 3! 4! 5! 6! 7! 8! 9! 1! 2! 3! 4! 5!
22. © Marie-Lan Nguyen Wikimedia Commons / CC-BY 2.5 22 ओͳ೰Έͷλω͸ʁ •  σΟεΫ࢖༻ྔ •  ݎ࿚ੑ΍ϦΧόϦ •  ϝδϟʔόʔδϣϯΞοϓάϨʔυ
23. 23 ϝδϟʔόʔδϣϯ ΞοϓάϨʔυͷ໰୊఺ •  Ϋϥελશମ࠶ىಈ == μ΢ϯλΠϜ •  Client/server όʔδϣϯޓ׵ੑ (Java) •  σʔλอ࣋ظؒ/ϥΠϑαΠΫϧ •  ଟ͘ͷղܾࠔ೉ͳޓ׵ੑͷͳ͍มߋ •  ܯࠂ͕গͳ͍/ग़ͳ͍ɺ ඇਪ঑ˍޓ׵ੑͷͳ͍มߋ © Famartin Wikimedia Commons / CC-BY 2.5
24. 24 ϝδϟʔόʔδϣϯ ΞοϓάϨʔυͷ໰୊఺ •  Ϋϥελશମ࠶ىಈ == μ΢ϯλΠϜ •  Client/server όʔδϣϯޓ׵ੑ (Java) •  σʔλอ࣋ظؒ/ϥΠϑαΠΫϧ •  ଟ͘ͷղܾࠔ೉ͳޓ׵ੑͷͳ͍มߋ •  ܯࠂ͕গͳ͍/ग़ͳ͍ɺ ඇਪ঑ˍޓ׵ੑͷͳ͍มߋ © Famartin Wikimedia Commons / CC-BY 2.5
25. 25 ϩʔϦϯάΞοϓάϨʔυ •  5.6.x͔Β6.latest΁Ϋϥελશମͷ࠶ىಈ ͳ͠ʹΞοϓάϨʔυ •  ஫ҙ఺: •  ޓ׵ੑͷͳ͍มߋ͋Γɻͨͩ͠ɺ5.6ܥ ʹόοΫϙʔτ͞ΕͨΦϓγϣϯ ଟ਺ɻ΋ͪΖΜɺςετΛʂ •  ΞοϓάϨʔυલʹશͯΛ5.6ܥʹ •  Security+TLS
26. 26 ϝδϟʔόʔδϣϯ ΞοϓάϨʔυͷ໰୊఺ •  Ϋϥελશମ࠶ىಈ == μ΢ϯλΠϜ •  Client/server όʔδϣϯޓ׵ੑ (Java) •  σʔλอ࣋ظؒ/ϥΠϑαΠΫϧ •  ଟ͘ͷղܾࠔ೉ͳޓ׵ੑͷͳ͍มߋ •  ܯࠂ͕গͳ͍/ग़ͳ͍ɺ ඇਪ঑ˍޓ׵ੑͷͳ͍มߋ © Famartin Wikimedia Commons / CC-BY 2.5
27. 27 Java high level REST client •  5.6ͰϦϦʔε •  ϝδϟʔόʔδϣϯΞοϓάϨʔυͰར༻ •  IDE ϑϨϯυϦʔ •  Transport Clientͱಉ༷ͷAPI •  low-level REST client͕ϕʔε •  CRUD & SearchΛαϙʔτ
28. 28 ϝδϟʔόʔδϣϯ ΞοϓάϨʔυͷ໰୊఺ •  Ϋϥελશମ࠶ىಈ == μ΢ϯλΠϜ •  Client/server όʔδϣϯޓ׵ੑ (Java) •  σʔλอ࣋ظؒ/ϥΠϑαΠΫϧ •  ଟ͘ͷղܾࠔ೉ͳޓ׵ੑͷͳ͍มߋ •  ܯࠂ͕গͳ͍/ग़ͳ͍ɺ ඇਪ঑ˍޓ׵ੑͷͳ͍มߋ © Famartin Wikimedia Commons / CC-BY 2.5
29. 29 σʔλอ࣋ظؒ / ϥΠϑαΠΫϧ: ͲΜͳΦϓγϣϯ͕ʁ ݹ͍σʔλͷ ࡟আ Reindex ͬͦ͜Γ v1.2.0Λ࢖͏... •  ΫϥελΛίϯύΫτʹ :) •  อ࣋ظؒͷ࠶ݕ౼ :) •  ΞοϓάϨʔυͰ͖ͳ͍ظؒ :( •  ར༻ͯ͠Δؒ͸࡟আͰ͖ͳ͍ :( •  ࠷৽Ͱศརͳػೳ :) •  σʔλྔʹΑΓ͕͔͔࣌ؒΔ Մೳੑ :( •  ؅ཧ͕༰қͰ͸ͳ͍ :( •  ΋ͬͱ΋ָͳํ๏ʁ •  ηΩϡϦςΟϑΟοΫε͕ ఏڙ͞Εͳ͘ͳΔ :( •  ࠷৽ͷศརػೳ͕࢖͑ͳ͍ :(
30. 30 Cross Major Version Search v5.6.0 Master Nodes Data Node Data Node v6.0.0 Your App Master Nodes Data Node Cross Cluster Client v5.latest
31. 31 DEMO
32. 32 ϝδϟʔόʔδϣϯ ΞοϓάϨʔυͷ໰୊఺ •  Ϋϥελશମ࠶ىಈ == μ΢ϯλΠϜ •  Client/server όʔδϣϯޓ׵ੑ (Java) •  σʔλอ࣋ظؒ/ϥΠϑλΠϜ •  ଟ͘ͷղܾࠔ೉ͳޓ׵ੑͷͳ͍มߋ •  ܯࠂ͕গͳ͍/ग़ͳ͍ɺ ඇਪ঑ˍޓ׵ੑͷͳ͍มߋ © Famartin Wikimedia Commons / CC-BY 2.5
33. Elasticsearch 6.1 6.1ͷվྑ఺͸ʁ
34. 34 6.1ͷ௥Ճػೳ •  Index Splitting •  Composite Aggregation •  Adaptive Replica Selection •  Scripted Similarities © Famartin Wikimedia Commons / CC-BY 2.5
35. Thank You