Upgrade to Pro — share decks privately, control downloads, hide ads and more …

GKEでは
StackDriver Loggingに
どうやってログを送っているか

GKEでは
StackDriver Loggingに
どうやってログを送っているか

2017/08/08 Kubernetes Meetup Tokyo #6 LT発表資料
GKEではStackDriver Loggingにどうやってログを転送しているか・StackDriver Loggingへのログの転送方法について発表しました。

Takumi Yoshida

August 08, 2017
Tweet

More Decks by Takumi Yoshida

Other Decks in Technology

Transcript

  1. GKEͰ͸

    StackDriver Loggingʹ

    Ͳ͏΍ͬͯϩάΛૹ͍ͬͯΔ͔
    2017/08/08 Kubernetes Meetup Tokyo #6 

    takumi_yoshida_22 / @ytakky2014

    View full-size slide

  2. ࣗݾ঺հ
    • ytakky2014 / ٢ా ୓࣮

    • github https://github.com/ytakky2014
    • @y_takky2014 

    https://twitter.com/y_takky2014
    • GCP / golang / PHP / CIɾCD /
    http://info.ytakky.tech/

    View full-size slide

  3. agenda
    • GKE, StackDriver Loggingʹ͍ͭͯ
    • StackDriverʹϩάΛૹΔfluentdͷconfʹ͍ͭ
    ͯ
    • GKE্ͰfluentdͷΧελϚΠζʹ͍ͭͯ

    View full-size slide

  4. GKE
    • GoogleCloudPlatform(GCP)ͷ

    KubernetesϑϧϚωʔδυαʔϏε
    • StackDriver Loggingͱ

    StackDriver MonitoringʹΑΔڧྗͳ؂ࢹ

    View full-size slide

  5. StackDriverLogging
    • GCP্ͷϩάΛ֨ೲɾ෼ੳͰ͖ΔαʔϏε
    • agentΛೖΕΕ͹AWS΋ରԠ
    • BigQuery΁ͷΤΫεϙʔτ

    GCS΁ͷΞʔΧΠϒ 

    Cloud Pub/SUB΁ͷετϦʔϛϯά ͕ରԠ

    View full-size slide

  6. GKE͸Ͳ͏΍ͬͯLogΛ
    StackDriverLoggingʹૹ͍ͬͯΔͷ͔
    • A. fluentd Ͱૹ͍ͬͯΔ

    View full-size slide

  7. ࣮ࡍʹͲ͏΍ͬͯऩू͢Δ͔
    • daemonsetsͰશϊʔυ΁഑ஔ͍ͯ͠Δ
    • https://github.com/kubernetes/kubernetes/blob/
    master/cluster/addons/fluentd-gcp/fluentd-gcp-
    ds.yaml

    View full-size slide

  8. fluentdͷઃఆΛ֬ೝͯ͠ΈΔ
    https://github.com/kubernetes/kubernetes/blob/
    master/cluster/addons/fluentd-gcp/fluentd-gcp-
    configmap.yaml

    View full-size slide

  9. ೖྗ

    type tail
    format json
    time_key time
    path /var/log/containers/*.log
    pos_file /var/log/gcp-containers.log.pos
    time_format %Y-%m-%dT%H:%M:%S.%N%Z
    tag reform.*
    read_from_head true

    View full-size slide

  10. ࣮ࡍ͸ͲͷϑΝΠϧΛऩूͯ͠
    ͍Δ͔
    • Nodeͷ /var/log/containers/*.logʹSTDOUTʹు͔Εͨϩά͕͋Δ
    • ࣮ࡍͷϑΝΠϧ໊͸

    [pod໊]_[namespace]_[containerID].log
    • frontend-3823415956-79vzs_default_php-
    redis-83772f44edd5ee75e921c063d45b710f5053d0a2c86a5c40c3212c
    b48922bb2c.log
    • ͜ΕʹreformΛprefixͱͯ͠෇͚Δ

    reform.frontend-3823415956-79vzs_default_php-redis-
    83772f44edd5ee75e921c063d45b710f5053d0a2c86a5c40c3212cb489
    22bb2c.log

    View full-size slide


  11. type record_reformer
    enable_ruby true
    tag raw.kubernetes.${tag_suffix[4].split('-')
    [0..-2].join('-')}

    View full-size slide


  12. • tag raw.kubernetes.${tag_suffix[4].split(‘-')
    [0..-2].join('-')}
    • reform.frontend-3823415956-79vzs_default_php-
    redis-83772f44edd5ee75e921c063d45b710f505
    3d0a2c86a5c40c3212cb48922bb2c.log
    • raw.kubernetes.frontend-3823415956-79vzs_defa
    ult_php-redis
    • ʹͳΔ

    View full-size slide


  13. @type detect_exceptions
    remove_tag_prefix raw
    message log
    stream stream
    multiline_flush_interval 5
    max_bytes 500000
    max_lines 1000

    View full-size slide


  14. • https://github.com/GoogleCloudPlatform/fluent-plugin-
    detect-exceptions Λ࢖͍ͬͯΔ
    • ྫ֎ݕग़ϓϥάΠϯɻlogstreamΛεΩϟϯ
    • ಉ࣌ʹraw. Λ֎͍ͯ͠Δ
    • kubernets.frontend-3823415956-79vzs_default_php-
    redis

    View full-size slide


  15. @type google_cloud
    # Collect metrics in Prometheus registry about
    plugin activity.
    enable_monitoring true
    monitoring_type prometheus
    # Set the buffer type to file to improve the
    reliability and reduce the memory consumption
    buffer_type file
    buffer_path /var/log/fluentd-buffers/
    kubernetes.containers.buffer

    ....

    View full-size slide


  16. • @type google_cloud

    https://github.com/GoogleCloudPlatform/fluent-
    plugin-google-cloud
    • StackDriver Logging APIʹϩάΛૹ৴͢Δϓϥά
    Πϯ
    • ͜ͷϓϥάΠϯͰStackDriverLoggingʹૹ৴ͯ͠
    ͍Δ

    View full-size slide

  17. fluentdͷઃఆΛม͍͑ͨ
    • fluentdͷpluginΛ௥Ճ͍ͨ͠
    • fluentdͷconfigͷઃఆΛมߋ͍ͨ͠

    View full-size slide

  18. લ४උ
    • GKEͷ৔߹ Ϋϥελʔ࡞੒࣌ʹ
    StackDriverLoggingΛOffʹ͢Δඞཁ͕͋Δ
    • https://kubernetes.io/docs/tasks/debug-
    application-cluster/logging-stackdriver/
    GKE >= 1.6Ҏ্͕ඞཁ
    ※gcloud container clustersͱ͔ίϚϯυͰมߋ΋Ͱ͖ͳ͍...

    View full-size slide

  19. fluentd-pluginΛ௥Ճ͍ͨ͠
    • https://github.com/GoogleCloudPlatform/k8s-
    stackdriver/tree/master/fluentd-gcp-image
    • GemfileʹඞཁͳpluginΛ௥Ճͯ͠

    docker build / docker push
    • fluentd-gcp-ds.yamlͷimageΛpushͨ͠image
    ʹมߋͯ͠σϓϩΠ

    View full-size slide

  20. fluentdͷconfigΛมߋ͢Δ
    • fluentd-gcp-configmap.yamlΛฤू͢Ε͹OK
    • 1ϑΝΠϧʹͳ͍ͬͯΔͷͰΘΓͱฤू͕ͭΒ
    ͍

    View full-size slide

  21. ઃఆมߋޙͷσϓϩΠ
    • fluentd-gcp-configmap.yamlͱfluentd-gcp-
    ds.yamlΛ೚ҙͷnamespace΁σϓϩΠ
    • kubectl create -f fluentd-gcp-
    configmap.yaml
    • kubectl create -f fluentd-gcp-ds.yaml

    View full-size slide

  22. ·ͱΊ
    • STDOUTʹϩάΛམͱ͓͚ͯ͠͹ɺ

    GKEͳΒ͹ࣗಈͰStackDriverLogging΁సૹ
    • fluentd-pluginͷ௥Ճ΍fluentd confͷมߋʹͳΔͱखؒʹ
    ͳΔ
    • StackDriverLogging͸Exportग़དྷΔͷͰfluentdͷઃఆͰͲ
    ͏ʹ͔͍ͨ͠Θ͚Ͱ͸ແ͚Ε͹ͦͪΒΛ࢖ͬͨ΄͏͕ྑ͍

    gSlack: Integrating Google Cloud Platform with Slack

    https://blog.doit-intl.com/gslack-9391be7c191a

    View full-size slide