Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Troubleshooting and profiling microservices

Troubleshooting and profiling microservices

Michael Hausenblas

April 26, 2018

More Decks by Michael Hausenblas

Other Decks in Technology


  1. Hit me up on Twitter: @mhausenblas 2 • Developer Advocate

    @ Red Hat (Go, Kubernetes, OpenShift) • Developer Advocate @ Mesosphere (Mesos, DC/OS, Kubernetes) • Chief Data Engineer @ MapR (HDFS, HBase, Drill, etc.) • Applied research (4y in Ireland, 7y in Austria) • Nowadays mainly developing tools in Go (Python, Node, Java, C++) • Kinda developer turned ops (aka appops) $ whois mhausenblas
  2. Hit me up on Twitter: @mhausenblas 5 • containerized setup

    • observability • troubleshooting • profiling Terminology
  3. Hit me up on Twitter: @mhausenblas 6 A note on

    operational aspects monolith v1 monolith v2 time µS1
 v1 µS2
 v1 µS3
 v1 µS2
 v2 µS3
 v2 µS1
 v2 µS2
 v3 µS3
 v3 µS1
 v3 µS3
 v4 µS2
 v4 µS3
 v5 µS1
 v4 µS2
 v5 µS3
  4. Hit me up on Twitter: @mhausenblas 7 Challenges • Aligning

    incentives (dev vs. ops) • Fallacies of distributed computing • Where to look when things go sideways
  5. Hit me up on Twitter: @mhausenblas 9 Kubernetes kubernetes.io •

    Container lifecycle management • Declarative, state-driven • Extensible, modular API • Robust, flexible, scalable Kudos to Lucas Käldström for this figure (source)
  6. Hit me up on Twitter: @mhausenblas 10 telepresence.io vapor-ware.github.io/ksync/ www.openshift.org

    kubed.sh kedgeproject.org ksonnet.io Minikube Minishift Docker Community Edition for Mac & Windows github.com/GoogleCloudPlatform/skaffold draft.sh forge.sh gitkube.sh github.com/weaveworks/flux github.com/MinikubeAddon/watchpod Developing on Kubernetes
  7. Hit me up on Twitter: @mhausenblas 11 • Monitoring •

    Logging • Distributed tracing Observability
  8. Hit me up on Twitter: @mhausenblas 12 Monitoring node container

    app alerts dashboards storage event router
  9. Hit me up on Twitter: @mhausenblas 14 Distributed tracing opentracing.io

    • Roots ~ “time-synced logs” • Standardized • Must-have in a microservices setup • Examples: Zipkin, Jaeger
  10. Hit me up on Twitter: @mhausenblas 16 Demo 1: debugging

    a minimal µS-based app • App consists of two microservices: • a stock generator written in Go • a stock consumer written in Node.js • Source code of demo app:
 github.com/kubernauts/dok-example-us stock-gen stock-con
  11. Hit me up on Twitter: @mhausenblas 21 • medium.com/@copyconstruct/monitoring-and- observability-8417d1952e1c

    • infoq.com/articles/charity-majors-observability-failure • blog.treasuredata.com/blog/2016/08/03/distributed-logging-architecture- in-the-container-era/ • mhausenblas.info/kubectl-in-action Resources