Jun Ohtani, Community Engineer
 Kosho Owa, Principal Solution Architect Elastic Stack 6.3 ঺հ

!2 ηογϣϯʹೖΔ
 ͦͷલʹɻ
 QRίʔυϦʔμʔͷ͝༻ҙΛʂ

!3 bit.ly/EnqEsMeetup

!4

No content

No content

No content

!8

!9

!10 Elasticsearch SQL (6.3 - Experimental) ࢖͍׳ΕͨγϯλοΫεͰElasticsearchʹ໰͍߹Θͤ

!11 Elasticsearch SQL (6.3) ࢖͍׳ΕͨγϯλοΫεͰElasticsearchʹ໰͍߹Θͤ ● REST API & CLI Client (Basic License) ○ SQLγϯλοΫεͰElasticsearchͱ΍ΓͱΓ ○ දܗࣜͰ݁ՌΛදࣔ ● Translate API (Basic License) ○ SQLίϚϯυΛElasticsearchͷQuery DSLʹม׵ ○ ElasticsearchͷQuery DSLγϯλοΫεʹ׳ΕΔͨΊͷศརͳํ๏ ● JDBC Client (Platinum License)

!12 Data Rollups ● API for creating an Elasticsearch process to periodically store aggregate statistics ● Primary benefit is space savings ○ Faster queries ○ Potentially less nodes to manage ○ Smaller snapshots ○ Longer retention times ○ etc. ● Query rolled up data and “live” data together in a single query. Rollups API (6.3 - Experimental) ● ఆظతʹ౷ܭσʔλΛू໿ͯ͠อଘ͢ΔElasticsearchͷJobΛొ࿥ ● ओͳར఺͸༰ྔͷ࡟ݮ ● σʔλ͕গͳ͘ͳΔͨΊ ○ Query͕ΑΓߴ଎ʹ ○ গͳ͍ϊʔυͰσʔλΛ؅ཧ ○ Snapshot͕ΑΓখ͘͞ ○ σʔλͷอ࣋ظ͕ؒΑΓ௕͘ ● 1ͭͷΫΤϦͰϩʔϧΞοϓͨ͠σʔλͱͯ͠ͳ͍σʔλΛ໰͍߹Θͤ ༰ྔ͕ɻɻɻ X-Pack feature (Basic, free)

!13 Raw Minute Hour Day Docs: 9,041,000 1,448,285 49,554 8,447 Size: 2.23gb 1.25gb 48.40mb 9.10mb Docs % : -83.98% -99.45% -99.91% Size %: -43.68% -97.84% -99.59% (avg ~200 docs per minute, 32 days of data, single host) (20 grouping fields, 62 numerics @ min/max/avg == 186 metrics) Rolling up Metricbeat data ༰ྔ࡟ݮͷҰྫ MetricbeatͷϩʔϧΞοϓ (ฏۉ ~200 docs/෼ɺ32೔ؒɺ1αʔόʔ) (20ݸͷάϧʔϓϑΟʔϧυɺ62ݸͷ਺஋ @ min/max/avg == 186 metrics) X-Pack feature (Basic, free)

!14 ͦͷଞʹ΋ … • Java 10 (6.3) for Elasticsearch • Painless execute API • ML jobs Ͱ Cross Cluster Search ͕Մೳʹ(X-Pack Platinum) • ML job ͷࣄલνΣοΫ (X-Pack Platinum)

!15

!16 Kibana Query LanguageͰࣗಈิ׬

!17 Recently Viewed ΍ Recent Time Range ௚લʹԿΛ͔ͯͨ͠ͳ?... Recently viewed objects Recently used time range

!18 Kibana Lab Visualizations Input controls (supports chaining) Vega Plugin

!19 Index Management UI UIͰIndex؅ཧ X-Pack feature (Basic, free)

!20 License Management UI TrialΛ࢝ΊͨΓɺBasic΁໭ͨ͠ΓɺϥΠηϯεͷߋ৽͕UI͔Β

!21 ͦͷଞʹ΋… • DashboardͷύωϧͷαΠζมߋ͕ΑΓॊೈʹ • MonitoringͷػೳΛKibanaͷը໘͔Β༗ޮʹ(σϑΥϧτ͸ແޮ) • APMͱWatcherͷ࿈ܞ͕༰қʹ

!22

!23 LogstashͷઃఆΛΑΓ؆ུԽ ྫ͑͹... Pipelineؒ௨৴ (Beta) ઃఆΛΑΓγϯϓϧʹ

!24 LogstashͷઃఆΛΑΓ؆ུԽ
 • Distributor
 Pipelineؒ௨৴ (Beta) ઃఆΛΑΓγϯϓϧʹ

!25 LogstashͷઃఆΛΑΓ؆ུԽ
 • Output Isolator
 Pipelineؒ௨৴ (Beta) ઃఆΛΑΓγϯϓϧʹ

!26 LogstashͷઃఆΛΑΓ؆ུԽ
 • Forked Path
 Pipelineؒ௨৴ (Beta) ઃఆΛΑΓγϯϓϧʹ

!27 LogstashͷઃఆΛΑΓ؆ུԽ
 • Collector Pipelineؒ௨৴ (Beta) ઃఆΛΑΓγϯϓϧʹ

!28 SNMP Poller (Beta) Centralized polling of SNMP agents with a Logstash input plugin ϕʔλ൛ͷػೳ • TCP΋͘͠͸UDPͰSNMP v1 ͱ v2c ΛϙʔϦϯά • SNMP GETs ͱ WALKs Λαϙʔτ • ఆظతʹϙʔϦϯά • MIB ϑΝΠϧΠϯϙʔτ • ޓ׵ੑ: Logstash 2.4 Ҏ্ʹରԠ Blog: https://www.elastic.co/blog/logstash-lines-support-for-tcp-and-snmpv1-in-snmp-input-plugin Routers Switches Servers SNMP Agents v0.1.0.beta1

!29 ͦͷଞʹ΋... • Persistent Queue Breaking changes • S3ͷinput/outputͰcustom endpointͱregionͷࢦఆ͕Մೳʹ • PQͷσΟεΫͷ࢖༻ྔ΋MonitoringͰऔಘ (Basic)

!30

!31 • at-least-once deliveryΛఏڙ
 • Metricbeat΍AuditbeatͷΑ͏ͳ
 λΠϓͰॏཁ • Ϧελʔτ΍ωοτϫʔΫো֐Ͱ΋ σʔλ్͕੾Εͳ͍ Disk΁Spool (Beta) ΤοδͰӬଓԽ # ____beat.yml queue: spool: file: path: "${path.data}/ spool.dat" permissions: 0600 # One- time size: 100MiB # One- time #prealloc: true # One- time write: buffer_size: 1MiB flush.timeout: 1s flush.events: 16384

!32 ௥Ճ͞ΕͨModule Filebeat Metricbeat

!33 Filebeatͷೖྗ͕ଟ༷ʹ • TCP Input ‒ SSL/TLSΛαϙʔτ • UDP Input • Syslog Input ‒ BSD RFC3164Λαϙʔτ ‒ ϓϩτίϧ͸TCPͱUDP filebeat.inputs:
 - type: tcp
 max_message_size: 10MiB
 host: "localhost:9000" filebeat.inputs:
 - type: udp
 max_message_size: 10KiB
 host: "localhost:8080" filebeat.inputs:
 - type: syslog
 protocol.tcp:
 host: "localhost:9000" filebeat.yml

!34 ͦͷଞʹ΋... • Kubernetes/DockerͷAudodiscoverػೳͷվળ • add_host_metadataͷ௥Ճ

!35 6.3.1͕ϦϦʔε
 ͞Ε͍ͯ·͢ʂ

!36 Demo