Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Elastic Stack 入門 2018.09 / Getting started Elastic Stack 2018.09

Jun Ohtani
September 19, 2018
2.4k

Elastic Stack 入門 2018.09 / Getting started Elastic Stack 2018.09

第24回Elasticsearch勉強会「入門編」で使用した資料です。
#elasticsearchjp

Jun Ohtani

September 19, 2018
Tweet

More Decks by Jun Ohtani

Transcript

  1. !1
    Jun Ohtani
    Community Engineer @Elastic

    Twitter: @johtani
    Elastic Stackೖ໳

    View Slide

  2. !2
    about
    • Me, Jun Ohtani / Community Engineer
    ‒ lucene-gosenίϛολʔ
    ‒ σʔλ෼ੳج൫ߏஙೖ໳ ڞஶ
    ‒ http://blog.johtani.info

    • Elastic, founded in 2012
    ‒ Products: Elasticsearch, Logstash, Kibana, Beats 

    Elastic APM, 

    X-Pack, Elastic Cloud, Swiftype 

    Professional services: Support & development subscriptions

    Trainings, Consulting, SaaS

    View Slide

  3. !3
    ΞδΣϯμ
    • Ξϯέʔτ
    • Ϣʔεέʔε঺հ
    • ϓϩμΫτ঺հ
    • σϞ ˍ QA

    View Slide

  4. Ϣʔεέʔε
    !4

    View Slide

  5. Search and
    analytics, it all
    started here
    !5
    More than 60% of our
    customers have a search
    or analytics use case

    View Slide

  6. !6

    View Slide

  7. Logs Logs Logs, 

    many devices,

    many systems
    More than 40% of our

    customers use our products

    for operational log analysis
    !7

    View Slide

  8. ΠϯϑϥετϥΫνϟɺWebαʔόʔɺ
    ΞϓϦέʔγϣϯ͔Βຖ೔

    ऩू͢Δϩά͸
    1.2TB Ҏ্ɹ
    8

    View Slide

  9. Sniff sniff sniff,

    find the bad actors

    in your data
    200% YoY growth in
    security use cases with
    our products
    !9

    View Slide

  10. We mine and analyze
    4 billion events every day to
    detect security hacks and threats.
    !10

    View Slide

  11. 75% of our customers
    use our products for
    multiple use cases
    SEARCH
    SECURITY
    CUSTOM APPS
    METRICS
    OPERATIONAL

    ANALYTICS
    LOG ANALYTICS
    !11

    View Slide

  12. !12
    1,000+ developers use the
    Elastic Stack for use cases
    from trade tracking to creating
    new HR and compliance apps.

    View Slide

  13. !13
    Elastic Stack

    View Slide

  14. Elastic
    Stack
    อଘɺݕࡧɺ෼ੳ
    Elasticsearch
    ՄࢹԽɺ؅ཧ
    Kibana
    Beats ΠϯδΣετ
    Logstash

    View Slide

  15. Metrics
    Logging
    APM
    Site

    Search
    Application
    Search
    Business

    Analytics
    Enterprise

    Search
    Security

    Analytics
    Future ιϦϡʔγϣϯ
    Elastic
    Stack
    อଘɺݕࡧɺ෼ੳ
    ՄࢹԽɺ؅ཧ
    ΠϯδΣετ
    Kibana
    Elasticsearch
    Beats Logstash

    View Slide

  16. Metrics
    Logging
    APM
    Site

    Search
    App

    Search
    Business

    Analytics
    Enterprise

    Search
    Security

    Analytics
    Future ιϦϡʔγϣϯ
    SaaS
    Elastic Cloud
    Self Managed
    Elastic Cloud

    Enterprise Standalone
    σϓϩΠ
    Elastic
    Stack
    อଘɺݕࡧɺ෼ੳ
    ՄࢹԽɺ؅ཧ
    ΠϯδΣετ
    Kibana
    Elasticsearch
    Beats Logstash

    View Slide

  17. Elastic
    Stack
    อଘɺݕࡧɺ෼ੳ
    Elasticsearch
    ՄࢹԽɺ؅ཧ
    Kibana
    Beats ΠϯδΣετ
    Logstash
    Metrics
    Logging
    APM
    Site

    Search
    Application
    Search
    Business

    Analytics
    Enterprise

    Search
    Security

    Analytics
    Future ιϦϡʔγϣϯ
    SaaS
    Elastic Cloud
    Self Managed
    Elastic Cloud

    Enterprise Standalone
    σϓϩΠ

    View Slide

  18. !18
    Elastic Stackͷߏ੒
    Beats
    Log
    Files
    Metrics
    Wire
    Data
    your{beat}
    Kibana
    Instances
    Kafka
    Distributed
    Message
    Queue
    Notification
    Queues Storage Metrics
    Data
    Store
    Web
    APIs
    Social Sensors
    Elasticsearch
    Nodes
    Logstash
    Nodes

    View Slide

  19. !19

    View Slide

  20. 20
    Beats
    ܰྔσʔλγούʔ
    ιʔε͔ΒσʔλΛసૹ సૹ͠Elasticsearchʹू໿ ม׵ͱύʔεͷͨΊ
    Logstashʹసૹ
    Elastic Cloudʹసૹ
    Libbeat: ΧελϜbeatsͷͨ
    ΊͷAPIϑϨʔϜϫʔΫ
    30Ҏ্ͷίϛϡχςΟbeats

    View Slide

  21. The Beats family
    Heartbeat
    Uptime monitoring
    Filebeat
    Log files
    Winlogbeat
    Windows Event Logs
    Packetbeat
    Network data
    +40
    community
    Beats
    Metricbeat
    Metrics
    Auditbeat
    Audit data

    View Slide

  22. Collect system
    and application
    metrics
    Metricbeat

    View Slide

  23. lots of modules
    Metricbeat

    View Slide

  24. tail log from
    file
    Filebeat

    View Slide

  25. many modules
    Filebeat

    View Slide

  26. Capture the
    Packet
    Packetbeat

    View Slide

  27. Capture the
    Packet
    Packetbeat

    View Slide

  28. Welcome
    to 1998
    winlogbeat

    View Slide

  29. Now
    winlogbeat

    View Slide

  30. !30

    View Slide

  31. 31
    Logstash
    σʔλՃ޻ύΠϓϥΠϯ
    શͯͷܗࣜɺαΠζͱσʔλιʔ
    εͷ౤ೖ
    ύʔεͱಈతͳ
    σʔλม׵
    ͋ΒΏΔग़ྗʹ
    σʔλసૹ
    ҆શͰ҉߸Խ͞Εͨ

    σʔλೖྗ
    ಠࣗͷύΠϓϥΠϯॲཧ
    ͷ࡞੒
    200Ҏ্ͷϓϥάΠϯ

    View Slide

  32. Logstash in 10 seconds
    • ϩάɾσʔλͷऩूɾ؅ཧ
    • ऩूɺύʔεɾՃ޻ɺૹग़
    • ΦʔϓϯιʔεɿApache License 2.0
    • Ruby app (JRuby)
    !32

    View Slide

  33. Logstash architecture
    !33
    Input Output
    Filter
    ? ?
    collect and split alter and enrich store and visualize

    View Slide

  34. ઃఆ
    34
    input {

    }
    filter {

    }
    output {

    }

    View Slide

  35. ઃఆɿinput
    35
    input {
    file {
    path => “/Users/johtani/sample/*_log"
    start_position => "beginning"
    }
    }

    View Slide

  36. 1ߦ1σʔλ
    189.120.xx.xx - - [02/Dec/2014:12:18:29 +0900] "GET /manager/html HTTP/
    1.1" 404 274 "-" "Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/20100101
    Firefox/5.0"
    36

    View Slide

  37. ઃఆɿfilter
    37
    filter {
    grok {
    match => { "message" => "%{COMBINEDAPACHELOG}" }
    break_on_match => false
    }
    date {
    match => ["timestamp", "dd/MMM/YYYY:HH:mm:ss Z"]
    locale => en
    }
    geoip { source => ["clientip"] }
    useragent {
    source => "agent"
    target => "useragent"
    }
    }

    View Slide

  38. ύʔε
    !38
    189.120.xx.xx - - [02/Dec/2014:12:18:29 +0900] "GET /manager/html HTTP/1.1"
    404 274 "-" "Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/20100101 Firefox/5.0"
    {…
    "@timestamp": "2015-04-10T09:07:49.325Z",
    "clientip": "189.120.xx.xx",
    "ident": "-",
    "auth": "-",
    "timestamp": "02/Dec/2014:12:18:29 +0900",
    "verb": "GET",
    "request": "/manager/html",

    "agent": "\"Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/

    View Slide

  39. ઃఆɿfilter
    !39
    filter {
    grok {
    match => { "message" => "%{COMBINEDAPACHELOG}" }
    break_on_match => false
    }
    date {
    match => ["timestamp", "dd/MMM/YYYY:HH:mm:ss Z"]
    locale => en
    }
    geoip { source => ["clientip"] }
    useragent {
    source => "agent"
    target => "useragent"
    }
    }

    View Slide

  40. ೔෇ͷύʔε
    40
    {…
    "@timestamp": "2015-04-10T09:07:49.325Z",

    "timestamp": "02/Dec/2014:12:18:29 +0900",

    }
    {…
    "@timestamp": "2014-12-02T03:18:29.000Z",

    "timestamp": "02/Dec/2014:12:18:29 +0900",

    }

    View Slide

  41. ઃఆɿfilter
    !41
    filter {
    grok {
    match => { "message" => "%{COMBINEDAPACHELOG}" }
    break_on_match => false
    }
    date {
    match => ["timestamp", "dd/MMM/YYYY:HH:mm:ss Z"]
    locale => en
    }
    geoip { source => ["clientip"] }
    useragent {
    source => "agent"
    target => "useragent"
    }
    }

    View Slide

  42. IP͔ΒҢ౓ܦ౓ͳͲ෇༩
    42
    "clientip": "189.120.xx.xx",
    "clientip": "189.120.xx.xx",
    "geoip": {
    "ip": “189.120.xxx.xxx”,

    "country_name": "Brazil",
    "continent_code": "SA",
    "region_name": "27",
    "city_name": "São Paulo",
    "latitude":

    View Slide

  43. ઃఆɿfilter
    !43
    filter {
    grok {
    match => { "message" => "%{COMBINEDAPACHELOG}" }
    break_on_match => false
    }
    date {
    match => ["timestamp", "dd/MMM/YYYY:HH:mm:ss Z"]
    locale => en
    }
    geoip { source => ["clientip"] }
    useragent {
    source => "agent"
    target => "useragent"
    }
    }

    View Slide

  44. ϢʔβΤʔδΣϯτͷύʔε
    44
    "agent": "\"Mozilla/5.0 (Windows NT 5.1; rv:
    5.0) Gecko/20100101 Firefox/5.0\""
    "agent": "\"Mozilla/5.0 (Windows NT 5.1; rv:
    5.0) Gecko/20100101 Firefox/5.0\""
    "useragent": {
    "name": "Firefox",
    "os": "Windows XP",
    "os_name": "Windows XP",
    "device": "Other",
    "major": "5",
    "minor": "0"

    View Slide

  45. ઃఆɿoutput
    45
    output {
    elasticsearch {
    hosts => ["localhost"]
    index => “demo_access_log-%{+YYYY.MM.dd}”
    }
    }

    View Slide

  46. !46

    View Slide

  47. 47
    Elasticsearch
    Heart of the Elastic Stack
    ෼ࢄܕɺεέʔϥϒϧ ߴՄ༻ੑ Ϛϧνςφϯτ
    ։ൃऀϑϨϯυϦʔ ϦΞϧλΠϜɺશจݕࡧ ΞάϦήʔγϣϯ

    View Slide

  48. Elasticsearchͱ͸ʁ

    View Slide

  49. ϑϦʔϫʔυݕࡧ
    !49

    View Slide

  50. ߜΓࠐΈ
    !50

    View Slide

  51. ϋΠϥΠτ
    !51

    View Slide

  52. ιʔτ
    !52

    View Slide

  53. ϖʔδϯά
    !53

    View Slide

  54. ूܭ
    !54

    View Slide

  55. αδΣετ
    !55

    View Slide

  56. Elasticsearch in 10 seconds
    • εΩʔϚϑϦʔɺ෼ࢄυΩϡϝϯτετΞɺREST & JSON
    • Φʔϓϯιʔε: Apache License 2.0
    • ઃఆͳ͠Ͱ؆୯ʹࢼ͢͜ͱ͕Մೳ
    • JavaͰ࣮૷ɻ֦ு΋༰қ
    !56

    View Slide

  57. Powerful Search at Scale
    !57

    View Slide

  58. ؆୯ͳCRUD

    View Slide

  59. σʔλొ࿥
    59
    curl -XPUT localhost:9200/books/book/1 -d '
    {
    "title" : "Elasticsearch - The definitive guide",
    "authors" : "Clinton Gormley",
    "started" : "2013-02-04",
    "pages" : 230
    }'

    View Slide

  60. σʔλߋ৽
    60
    curl -XPUT localhost:9200/books/book/1 -d '
    {
    "title" : "Elasticsearch - The definitive guide",
    "authors" : [ "Clinton Gormley", "Zachary Tong" ],
    "started" : "2013-02-04",
    "pages" : 230
    }'

    View Slide

  61. σʔλ࡟আ
    !61
    curl -X DELETE localhost:9200/books/book/1
    σʔλͷऔಘ
    curl —X GET localhost:9200/books/book/1
    curl —X GET localhost:9200/books/book/1/_source

    View Slide

  62. ݕࡧ
    !62
    curl -XGET localhost:9200/books/_search?q=elasticsearch
    {
    "took" : 2, "timed_out" : false,
    "_shards" : { "total" : 5, "successful" : 5, "failed" : 0 },
    "hits" : {
    "total" : 1, "max_score" : 0.076713204,
    "hits" : [ {
    "_index" : “books", "_type" : “book", "_id" : "1",
    "_score" : 0.076713204, "_source" : {
    "title" : "Elasticsearch - The definitive guide",
    "authors" : [ "Clinton Gormley", "Zachary Tong" ],
    "started" : “2013-02-04", "pages" : 230
    }
    } ]

    View Slide

  63. ݕࡧ - Query DSL
    !63
    curl -XGET ‘localhost:9200/books/book/_search' -d '{
    "query": {
    "filtered" : {
    "query" : {
    "match": {
    "text" : {
    "query" : “To Be Or Not To Be",
    "cutoff_frequency" : 0.01
    }
    }
    },
    "filter" : {
    "range": {
    "price": {
    "gte": 20.0
    "lte": 50.0

    View Slide

  64. ෼ࢄߏ੒ɺ

    εέʔϧ

    View Slide

  65. Basic terms
    • ΠϯσοΫε
    ‒ σʔλͷ࿦ཧతͳू߹ɻ

    RDBͷσʔλϕʔεͷΑ͏ͳ΋ͷLogical
    • ϨϓϦέʔγϣϯ
    • ಡΈࠐΈͷεέʔϥϏϦςΟ޲্
    • SPOFͷղফ
    • γϟʔσΟϯά
    • ෳ਺Ϛγϯ΁σʔλΛ෼ׂ

    ॻ͖ࠐΈͷεέʔϥϏϦςΟ޲্

    σʔλϑϩʔ੍ޚ
    !65

    View Slide

  66. γϟʔυͱϨϓϦΧ
    !66
    node 1
    orders
    products
    1
    4
    1 2
    2
    3
    curl -X PUT localhost:9200/orders -d '{
    "settings.index.number_of_shards" : 4
    "settings.index.number_of_replicas" : 1
    }'
    curl -X PUT localhost:9200/products -d '{
    "settings.index.number_of_shards" : 2
    "settings.index.number_of_replicas" : 0
    }'

    View Slide

  67. γϟʔυͱϨϓϦΧ
    !67
    node 1
    orders
    products
    1
    4
    1
    node 2
    orders
    products
    2
    2
    3 4
    1 2
    3

    View Slide

  68. ࣗಈతͳ෼ࢄ
    !68
    node 1
    orders
    products
    2
    1
    4
    1
    node 2
    orders
    products
    2
    2
    node 3
    orders
    products
    3 4
    1
    3

    View Slide

  69. શจݕࡧͱ͸ʁ

    View Slide

  70. શจݕࡧͱ͸ʁ
    • શจݕࡧʢFull text searchʣͱ͸ɺίϯϐϡʔλʹ͓͍ͯɺෳ਺ͷจॻ
    ʢϑΝΠϧʣ͔ΒಛఆͷจࣈྻΛݕࡧ͢Δ͜ͱɻʮϑΝΠϧ໊ݕࡧʯ΍
    ʮ୯ҰϑΝΠϧ಺ͷจࣈྻݕࡧʯͱҟͳΓɺʮෳ਺จॻʹ·͕ͨͬͯɺจ
    ॻʹؚ·ΕΔશจΛର৅ͱͨ͠ݕࡧʯͱ͍͏ҙຯͰ࢖༻͞ΕΔɻ

    ʢWikipediaΑΓʣ
    !70

    View Slide

  71. ༻ޠ
    • ΠϯσοΫε
    ݕࡧΤϯδϯ͕ݕࡧʹ࢖༻͢Δσʔλͷอଘઌ
    • υΩϡϝϯτʢจॻʣ
    ‒ ݕࡧΤϯδϯʹอଘ͞Εͨσʔλ
    • ϑΟʔϧυ
    ‒ υΩϡϝϯτʹؚ·ΕΔଐੑ
    • ΫΤϦ
    ‒ ݕࡧ৚݅ɺݕࡧࣜ
    !71

    View Slide

  72. ༻ޠ
    • εΩʔϚ
    ‒ υΩϡϝϯτͷߏ଄Λఆٛ͢Δ΋ͷ
    • λʔϜʢTermʣɺτʔΫϯʢTokenʣ
    ‒ ΠϯσοΫεͷΩʔʹͳΔ୯ޠʢจࣈྻʣ
    ‒ จষΛҰఆͷ๏ଇͰ۠੾ͬͨ୯ޠ
    ‒ ୯ޠ͚ͩͰͳ͘ɺ୯ޠͷҐஔͳͲ΋ؚΉ
    !72

    View Slide

  73. υΩϡϝϯτͷొ࿥
    !73
    1
    2
    ΧπΦ͸αβΤͷఋ
    αβΤ͸ϫΧϝͷ࢞
    υΩϡϝϯτͷొ࿥

    View Slide

  74. υΩϡϝϯτͷొ࿥
    !74
    1
    2
    ΧπΦ͸αβΤͷఋ
    αβΤ͸ϫΧϝͷ࢞
    1
    2
    ΧπΦ
    αβΤ
    ͸
    ͸
    ͷ
    ͷ
    αβΤ
    ϫΧϝ


    υΩϡϝϯτͷొ࿥
    ୯ޠʹ෼ׂ

    View Slide

  75. υΩϡϝϯτͷొ࿥
    !75
    1
    2
    ΧπΦ͸αβΤͷఋ
    αβΤ͸ϫΧϝͷ࢞
    1
    2
    ΧπΦ
    αβΤ
    ͸
    ͸
    ͷ
    ͷ
    αβΤ
    ϫΧϝ


    ΧπΦ
    αβΤ
    1
    1 2 ͸
    ͷ

    ϫΧϝ 2
    1 2
    1 2
    1

    2
    υΩϡϝϯτͷొ࿥
    ୯ޠʹ෼ׂ
    ୯ޠ͔Βidͷ഑ྻ͕
    Ҿ͚ΔΑ͏ʹ

    View Slide

  76. ݕࡧ
    !76
    ΧπΦ
    αβΤ
    1
    1 2 ͸
    ͷ

    ϫΧϝ 2
    1 2
    1 2
    1

    2
    ݕࡧ৚݅ೖྗ
    ΧπΦɹαβΤ

    View Slide

  77. ݕࡧ
    !77
    ΧπΦ
    αβΤ
    1
    1 2 ͸
    ͷ

    ϫΧϝ 2
    1 2
    1 2
    1

    2
    ΧπΦ αβΤ
    AND
    ݕࡧ৚݅ೖྗ
    ݕࡧ৚݅ͷύʔε

    ݕࡧΫΤϦԽ
    ΧπΦɹαβΤ

    View Slide

  78. ݕࡧ
    !78
    ΧπΦ
    αβΤ
    1
    1 2 ͸
    ͷ

    ϫΧϝ 2
    1 2
    1 2
    1

    2
    ΧπΦ αβΤ
    AND
    ݕࡧ৚݅ೖྗ
    ݕࡧ৚݅ͷύʔε

    ݕࡧΫΤϦԽ
    ΧπΦɹαβΤ

    View Slide

  79. ݕࡧ
    !79
    ΧπΦ
    αβΤ
    1
    1 2 ͸
    ͷ

    ϫΧϝ 2
    1 2
    1 2
    1

    2
    ΧπΦ αβΤ
    AND
    ݕࡧ৚݅ೖྗ
    ݕࡧ৚݅ͷύʔε

    ݕࡧΫΤϦԽ
    ΧπΦɹαβΤ

    View Slide

  80. ݕࡧ
    !80
    ΧπΦ
    αβΤ
    1
    1 2 ͸
    ͷ

    ϫΧϝ 2
    1 2
    1 2
    1

    2
    ΧπΦ αβΤ
    AND
    ݕࡧ৚݅ೖྗ
    ݕࡧ৚݅ͷύʔε

    ݕࡧΫΤϦԽ
    ΧπΦɹαβΤ

    View Slide

  81. ݕࡧ
    !81
    ΧπΦ
    αβΤ
    1
    1 2 ͸
    ͷ

    ϫΧϝ 2
    1 2
    1 2
    1

    2
    ΧπΦ αβΤ
    AND
    ݕࡧ৚݅ೖྗ
    ݕࡧ৚݅ͷύʔε

    ݕࡧΫΤϦԽ
    ΧπΦɹαβΤ

    View Slide

  82. ݕࡧ
    !82
    ΧπΦ
    αβΤ
    1
    1 2 ͸
    ͷ

    ϫΧϝ 2
    1 2
    1 2
    1

    2
    ΧπΦ αβΤ
    AND
    ݕࡧ৚݅ೖྗ
    ݕࡧ৚݅ͷύʔε

    ݕࡧΫΤϦԽ
    ΧπΦɹαβΤ

    View Slide

  83. ୯ޠͷ۠੾Γํ
    • ӳޠͷ৔߹
    I am speaking Introduction Elasticsearch.


    • ೔ຊޠͷ৔߹
    ࢲ͸ೖ໳Elasticsearchʹ͍ͭͯ࿩͍ͯ͠Δɻ


    !83

    View Slide

  84. ୯ޠͷ۠੾Γํ
    • ӳޠͷ৔߹
    I am speaking Introduction Elasticsearch.


    εϖʔε͕੾Ε໨ͱΘ͔Δ
    • ೔ຊޠͷ৔߹
    ࢲ͸ೖ໳Elasticsearchʹ͍ͭͯ࿩͍ͯ͠Δɻ

    Ͳ͜Ͱ۠੾Ε͹Α͍ʁ
    84

    View Slide

  85. N-Gramͱܗଶૉղੳ
    • సஔΠϯσοΫεͷΩʔͷ࡞Γํ
    ‒ ೔ຊޠ͸୯ޠͷ੾Ε໨͕Θ͔Βͳ͍ͷͰɺసஔΠϯσοΫεͷΩʔ͸
    ओʹ࣍ͷ̎ͭͷख๏Ͱ࡞੒
    • N-Gram
    ‒ NจࣈͣͭจষΛ۠੾Δ
    • ܗଶૉղੳ
    ‒ ࣙॻͳͲΛ༻͍ͯҙຯͷ͋Δ୯ޠͰ۠੾Δ
    !85

    View Slide

  86. ܗଶૉղੳ
    • ϝϦοτɿ
    ‒ ҙຯͷ͋Δ୯ޠͷ੾Ε໨

    ඼ࢺ৘ใΛݩʹ௥Ճॲཧ͕Մೳʢޠװม׵ͳͲʣ
    • σϝϦοτɿ
    ‒ ৽ޠʢະ஌ޠʣʹऑ͍→ࣙॻϕʔεͷ৔߹ɺࣙॻʹͳ͍୯ޠ͸ݕग़ෆ
    ೳɻ
    !86
    ΧπΦ͸αβΤͷఋ
    ΧπΦ ͸ ͷ
    αβΤ ఋ

    View Slide

  87. N-Gram
    • ϝϦοτɿ
    ‒ ະ஌ޠʹରԠՄೳ
    • σϝϦοτɿ
    ‒ ΠϯσοΫεංେԽ
    ‒ ඼ࢺ৘ใʹجͮ͘ॲཧ͕ෆՄೳ
    !87
    ΧπΦ͸αβΤͷఋ
    Χπ πΦ Φ͸ ͸α αβ βΤ Τͷ ͷఋ

    View Slide

  88. ͦͷଞͷػೳ

    View Slide

  89. elasticsearch
    ͞·͟·ͳܗࣜͷσʔλͰ
    GeoݕࡧՄೳ


    Ң౓ܦ౓ɺGeoHashɺ
    GeoShape…
    GEO

    View Slide

  90. Ecosystem
    • Plugins
    ‒ ϓϥάΠϯʹΑΔػೳͷ௥Ճ
    • ΫϥΠΞϯτϥΠϒϥϦ
    • Java, Ruby, python, php, perl, javascript, .NET
    • Scala, clojure, go
    !90

    View Slide

  91. Elasticsearch - The Definitive guide


    http://www.elastic.co/guide/en/
    elasticsearch/guide/current/index.html
    91
    ৄ͘͠஌Γ͍ͨํ͸

    View Slide

  92. !92

    View Slide

  93. 93
    Kibana
    Window into the Elastic Stack
    ՄࢹԽͱ෼ੳ ஍ཧۭؒ ΧελϚΠζͱ
    Ϩϙʔτͷڞ༗
    άϥϑ୳ࡧ Elastic Stack΁ͷ
    ηΩϡΞͳΞΫηεͱ؅ཧ
    ΧελϜAppsͷ࡞੒

    View Slide

  94. σʔλͷొ࿥ํ๏
    !94

    View Slide

  95. !95
    σʔλͷొ࿥ํ๏
    • Kibanaͷαϯϓϧσʔλʢ6.4͔Βʣ
    • LogstashͰJDBC input
    • LogstashͰCSV
    • FilebeatͰΞΫηεϩά
    • MetricbeatͰϝτϦοΫ
    • PacketbeatͰMySQL/PostgreSQLͷύέοτղੳ

    View Slide

  96. !96
    Kibanaͷαϯϓϧσʔλʢ>= 6.4.0ʣ

    View Slide

  97. !97
    ϫϯΫϦοΫͰσʔλొ࿥

    View Slide

  98. !98
    LogstashͰJDBC Input
    Kibana
    Instances
    Data
    Store
    Elasticsearch
    Nodes
    Logstash
    Nodes

    View Slide

  99. !99
    JDBC Input

    View Slide

  100. !100
    LogstashͰCSV
    Kibana
    Instances
    CSV

    File
    Elasticsearch
    Nodes
    Logstash
    Nodes

    View Slide

  101. !101
    CSV filter

    View Slide

  102. !102
    FilebeatͰΞΫηεϩά
    Beats
    Log
    Files
    Kibana
    Instances
    Elasticsearch
    Nodes

    View Slide

  103. • 2ͭͷElasticsearchϓϥάΠϯΛΠϯετʔϧͯ͠ElasticsearchΛىಈ
    • Filebeatͷapache2ϞδϡʔϧΛ༗ޮԽ
    • modules.d/apache2.ymlʹΞΫηεϩάͷύεΛઃఆ
    • setupίϚϯυΛ࣮ߦ͔ͯ͠ΒFilebeatΛىಈ
    !103
    FilebeatͰΞΫηεϩά

    View Slide

  104. MetricbeatͰϝτϦοΫ
    Beats
    Metrics
    Kibana
    Instances
    Elasticsearch
    Nodes

    View Slide

  105. • MetricbeatͷsystemϞδϡʔϧΛ༗ޮԽ
    • setupίϚϯυΛ࣮ߦ͔ͯ͠ΒFilebeatΛىಈ
    !105
    MetricbeatͰϝτϦοΫ

    View Slide

  106. !106
    PacketbeatͰMySQLɺPostgreSQLͷύέοτղੳ
    Beats
    Wire
    Data
    Kibana
    Instances
    Elasticsearch
    Nodes

    View Slide

  107. σϞ
    !107

    View Slide

  108. Thank you!
    ● Web : https://www.elastic.co/jp/
    ● Forums : https://discuss.elastic.co/ˇ
    ● Twitter : @johtani

    View Slide