Save 37% off PRO during our Black Friday Sale! »

Elastic{ON}Tour Tokyo 2017 - Elasticsearch Deep Dive

Dd9d954997353b37b4c2684f478192d3?s=47 Elastic Co
December 14, 2017

Elastic{ON}Tour Tokyo 2017 - Elasticsearch Deep Dive

Elasticsearch 5.xでどのような機能や変更が行われたのか。アップグレードやスケール、リカバリ、ディスク使用量などの懸念事項について、6.xではどのように進化していくのかを解説します。

Jun Ohtani | Developer, Evangelist| Elastic

Dd9d954997353b37b4c2684f478192d3?s=128

Elastic Co

December 14, 2017
Tweet

Transcript

  1. Jun Ohtani Developer | Evangelist Elasticsearch: Past, Present, & Future

  2. Elasticsearch 5.0 2016/10 ϦϦʔε

  3. 3 ਺஋ʹؔ͢Δվળ ҆શୈҰ ΑΓ؆୯ʹ Elasticsearch 5.0

  4. Elasticsearch 5.x: ༷ʑͳػೳ௥Ճ 2017/09 5.6 ϦϦʔε

  5. 5 Elasticsearch 5.x Still ^ •  Keyword ਖ਼نԽ •  ౷Ұ͞ΕͨϋΠϥΠλʔ

    •  ΧςΰϦͳͲʹΑΔ݁Ռͷू໿ •  ྨٛޠʹؔ͢Δվળ •  ݕࡧͷΩϟϯηϧ •  ฒྻ࣮ߦͰ͖Δscroll΍reindex
  6. 6 Elasticsearch 5.x Still ^ •  ਺஋΍೔෇ͷൣғϑΟʔϧυ •  ൣғݕࡧʹؔ͢ΔΫΤϦ࠷దԽ • 

    େྔͷaggsʹؔ͢Δ෼ׂऔಘ •  geo-distanceͷιʔτͷ଎౓޲্ •  geo-ip lookupͷ଎౓޲্ and for logs and for numbers and for geo and ... ^
  7. 7 ΑΓεέʔϧͤ͞΍͘͢ •  ෳ਺ͷΫϥελ΍Ϧιʔεͷ෼཭ •  ؅ཧΛΑΓ؆୯ʹ •  ΞοϓάϨʔυΛΑΓ؆୯ʹ •  ఀࢭظؒΛΑΓখ͘͞

    •  ΫϥελΛ·͙ͨΫΤϦ
  8. Cluster UK Master Nodes Data Node Data Node Data Node

    Tribe Node t1 Node Client Cluster US Master Nodes Data Node Data Node Data Node t2 Node Client 8 Tribe Node Merged Cluster State! Kibana
  9. 9 Cross-Cluster Search Cluster UK Master Nodes Data Node Data

    Node Data Node Master/Data Node Cluster US Master Nodes Data Node Data Node Data Node Master/Data Node Kibana Optional dedicated cross-cluster search cluster
  10. Elasticsearch 6.0+ ৽όʔδϣϯͷվળ

  11. © Marie-Lan Nguyen Wikimedia Commons / CC-BY 2.5 11 ओͳ೰Έͷλω͸ʁ

    •  σΟεΫ࢖༻ྔ •  ݎ࿚ੑ΍ϦΧόϦ •  ϝδϟʔόʔδϣϯΞοϓάϨʔυ
  12. © Marie-Lan Nguyen Wikimedia Commons / CC-BY 2.5 12 ओͳ೰Έͷλω͸ʁ

    •  σΟεΫ࢖༻ྔ •  ݎ࿚ੑ΍ϦΧόϦ •  ϝδϟʔόʔδϣϯΞοϓάϨʔυ
  13. 13 Doc Values - ૄͳσʔλ (5.x) Segment 1 ID fname

    lname 1 Shane Connelly 2 Shay Banon 3 Tanya Bragin Segment 2 ID fname lname mi state city 4 Steve Kearns Null Null Boston 5 George Burdell P GA Null 6 Frasier Crane Null WA Seattle Merged Segment 3 Docs fname lname mi state city 1 Shane Connelly Null Null Null 2 Shay Banon Null Null Null 3 Tanya Bragin Null Null Null 4 Steve Kearns Null Null Boston 5 George Burdell P GA Null 6 Frasier Crane Null WA Seattle
  14. 14 MetricbeatͷσʔληοτͰͷαϯϓϧ Apples-to-apples index size improvements 0 25 50 75

    100 5.x 6.0 OOTB _all disabled
  15. 15 σΟεΫ࢖༻ྔͷվળ •  ૄͳσʔλʹର͢Δαϙʔτ •  _allϑΟʔϧυΛσϑΥϧτͰແޮʹ •  ֤छσʔλొ࿥ʹޮՌతͳઃఆʹ •  কདྷɿϩʔϧΞοϓػೳ

    © Tony Weman / CC-BY 2.5
  16. © Marie-Lan Nguyen Wikimedia Commons / CC-BY 2.5 16 ओͳ೰Έͷλω͸ʁ

    •  σΟεΫ࢖༻ྔ •  ݎ࿚ੑ΍ϦΧόϦ •  ϝδϟʔόʔδϣϯΞοϓάϨʔυ
  17. 17 ϦΧόϦ (5.x) Segment 1 Segment 1 Segment 2 Segment

    2 Segment 3 Segment 3 Primary Replica
  18. 18 ϦΧόϦ (5.x) Segment 1 Segment 1 Segment 2 Segment

    2 Segment 3 Segment 3 Offline Primary
  19. 19 ϦΧόϦ (5.x) Segment 1 Segment 1 Segment 2 Segment

    4 Segment 3 Primary Segment 1 Segment 2 Segment 3 Offline
  20. 20 ϦΧόϦ (5.x) Segment 1 Segment 1 Segment 2 Segment

    4 Segment 3 Primary Segment 1 Segment 2 Segment 3 Repica ϑΝΠϧίϐʔ
  21. 21 ϦΧόϦ (6.x) Primary Repica Transaction-log ϕʔε 1! 2! 3!

    4! 5! 6! 7! 8! 9! 1! 2! 3! 4! 5!
  22. © Marie-Lan Nguyen Wikimedia Commons / CC-BY 2.5 22 ओͳ೰Έͷλω͸ʁ

    •  σΟεΫ࢖༻ྔ •  ݎ࿚ੑ΍ϦΧόϦ •  ϝδϟʔόʔδϣϯΞοϓάϨʔυ
  23. 23 ϝδϟʔόʔδϣϯ ΞοϓάϨʔυͷ໰୊఺ •  Ϋϥελશମ࠶ىಈ == μ΢ϯλΠϜ •  Client/server όʔδϣϯޓ׵ੑ

    (Java) •  σʔλอ࣋ظؒ/ϥΠϑαΠΫϧ •  ଟ͘ͷղܾࠔ೉ͳޓ׵ੑͷͳ͍มߋ •  ܯࠂ͕গͳ͍/ग़ͳ͍ɺ ඇਪ঑ˍޓ׵ੑͷͳ͍มߋ © Famartin Wikimedia Commons / CC-BY 2.5
  24. 24 ϝδϟʔόʔδϣϯ ΞοϓάϨʔυͷ໰୊఺ •  Ϋϥελશମ࠶ىಈ == μ΢ϯλΠϜ •  Client/server όʔδϣϯޓ׵ੑ

    (Java) •  σʔλอ࣋ظؒ/ϥΠϑαΠΫϧ •  ଟ͘ͷղܾࠔ೉ͳޓ׵ੑͷͳ͍มߋ •  ܯࠂ͕গͳ͍/ग़ͳ͍ɺ ඇਪ঑ˍޓ׵ੑͷͳ͍มߋ © Famartin Wikimedia Commons / CC-BY 2.5
  25. 25 ϩʔϦϯάΞοϓάϨʔυ •  5.6.x͔Β6.latest΁Ϋϥελશମͷ࠶ىಈ ͳ͠ʹΞοϓάϨʔυ •  ஫ҙ఺: •  ޓ׵ੑͷͳ͍มߋ͋Γɻͨͩ͠ɺ5.6ܥ ʹόοΫϙʔτ͞ΕͨΦϓγϣϯ

    ଟ਺ɻ΋ͪΖΜɺςετΛʂ •  ΞοϓάϨʔυલʹશͯΛ5.6ܥʹ •  Security+TLS
  26. 26 ϝδϟʔόʔδϣϯ ΞοϓάϨʔυͷ໰୊఺ •  Ϋϥελશମ࠶ىಈ == μ΢ϯλΠϜ •  Client/server όʔδϣϯޓ׵ੑ

    (Java) •  σʔλอ࣋ظؒ/ϥΠϑαΠΫϧ •  ଟ͘ͷղܾࠔ೉ͳޓ׵ੑͷͳ͍มߋ •  ܯࠂ͕গͳ͍/ग़ͳ͍ɺ ඇਪ঑ˍޓ׵ੑͷͳ͍มߋ © Famartin Wikimedia Commons / CC-BY 2.5
  27. 27 Java high level REST client •  5.6ͰϦϦʔε •  ϝδϟʔόʔδϣϯΞοϓάϨʔυͰར༻

    •  IDE ϑϨϯυϦʔ •  Transport Clientͱಉ༷ͷAPI •  low-level REST client͕ϕʔε •  CRUD & SearchΛαϙʔτ
  28. 28 ϝδϟʔόʔδϣϯ ΞοϓάϨʔυͷ໰୊఺ •  Ϋϥελશମ࠶ىಈ == μ΢ϯλΠϜ •  Client/server όʔδϣϯޓ׵ੑ

    (Java) •  σʔλอ࣋ظؒ/ϥΠϑαΠΫϧ •  ଟ͘ͷղܾࠔ೉ͳޓ׵ੑͷͳ͍มߋ •  ܯࠂ͕গͳ͍/ग़ͳ͍ɺ ඇਪ঑ˍޓ׵ੑͷͳ͍มߋ © Famartin Wikimedia Commons / CC-BY 2.5
  29. 29 σʔλอ࣋ظؒ / ϥΠϑαΠΫϧ: ͲΜͳΦϓγϣϯ͕ʁ ݹ͍σʔλͷ ࡟আ Reindex ͬͦ͜Γ v1.2.0Λ࢖͏...

    •  ΫϥελΛίϯύΫτʹ :) •  อ࣋ظؒͷ࠶ݕ౼ :) •  ΞοϓάϨʔυͰ͖ͳ͍ظؒ :( •  ར༻ͯ͠Δؒ͸࡟আͰ͖ͳ͍ :( •  ࠷৽Ͱศརͳػೳ :) •  σʔλྔʹΑΓ͕͔͔࣌ؒΔ Մೳੑ :( •  ؅ཧ͕༰қͰ͸ͳ͍ :( •  ΋ͬͱ΋ָͳํ๏ʁ •  ηΩϡϦςΟϑΟοΫε͕ ఏڙ͞Εͳ͘ͳΔ :( •  ࠷৽ͷศརػೳ͕࢖͑ͳ͍ :(
  30. 30 Cross Major Version Search v5.6.0 Master Nodes Data Node

    Data Node v6.0.0 Your App Master Nodes Data Node Cross Cluster Client v5.latest
  31. 31 DEMO

  32. 32 ϝδϟʔόʔδϣϯ ΞοϓάϨʔυͷ໰୊఺ •  Ϋϥελશମ࠶ىಈ == μ΢ϯλΠϜ •  Client/server όʔδϣϯޓ׵ੑ

    (Java) •  σʔλอ࣋ظؒ/ϥΠϑλΠϜ •  ଟ͘ͷղܾࠔ೉ͳޓ׵ੑͷͳ͍มߋ •  ܯࠂ͕গͳ͍/ग़ͳ͍ɺ ඇਪ঑ˍޓ׵ੑͷͳ͍มߋ © Famartin Wikimedia Commons / CC-BY 2.5
  33. Elasticsearch 6.1 6.1ͷվྑ఺͸ʁ

  34. 34 6.1ͷ௥Ճػೳ •  Index Splitting •  Composite Aggregation •  Adaptive

    Replica Selection •  Scripted Similarities © Famartin Wikimedia Commons / CC-BY 2.5
  35. Thank You