Unifying Your Observability Pipeline

Slide 1

Slide 1 text

Unifying Your Observability Pipeline Aditya Mukerjee Systems Engineer at Stripe @chimeracoder

Slide 2

Slide 2 text

@chimeracoder

Slide 3

Slide 3 text

Why are we here? @chimeracoder

Slide 4

Slide 4 text

It’s 3:07 AM @chimeracoder

Slide 5

Slide 5 text

Dashboard Count: 1 @chimeracoder

Slide 6

Slide 6 text

Dashboard Count: 2 @chimeracoder

Slide 7

Slide 7 text

Dashboard Count: 3 @chimeracoder

Slide 8

Slide 8 text

@chimeracoder

Slide 9

Slide 9 text

Dashboard Count: 4 @chimeracoder

Slide 10

Slide 10 text

@chimeracoder

Slide 11

Slide 11 text

@chimeracoder Dashboard Count: 5

Slide 12

Slide 12 text

What tools can we use? Metrics/dashboards? Logs? Request traces? No context! Hard to aggregate! Require planning! @chimeracoder

Slide 13

Slide 13 text

Monitoring information is only as good as developers’ ability to predict the future @chimeracoder

Slide 14

Slide 14 text

@chimeracoder

Slide 15

Slide 15 text

@chimeracoder

Slide 16

Slide 16 text

@chimeracoder

Slide 17

Slide 17 text

@chimeracoder Application

Slide 18

Slide 18 text

What’s the difference? •If you squint, it’s hard to tell them apart •A log is a metric with “longer” information •A trace is a metric that allows “inner joins” @chimeracoder

Slide 19

Slide 19 text

What if we could have all three, all the time? @chimeracoder

Slide 20

Slide 20 text

Standard Sensor Format @chimeracoder

Slide 21

Slide 21 text

@chimeracoder

Slide 22

Slide 22 text

@chimeracoder

Slide 23

Slide 23 text

@chimeracoder Application

Slide 24

Slide 24 text

Integrated Views @chimeracoder

Slide 25

Slide 25 text

@chimeracoder

Slide 26

Slide 26 text

@chimeracoder

Slide 27

Slide 27 text

@chimeracoder

Slide 28

Slide 28 text

Tradeoffs: Stacking the Deck @chimeracoder

Slide 29

Slide 29 text

Distributed Collection @chimeracoder host1 host2 host3 Dashboard Tool

Slide 30

Slide 30 text

Aggregation @chimeracoder host1 host2 host3 Global Aggregator Dashboard Tool

Slide 31

Slide 31 text

Distributed Aggregation @chimeracoder host1 host2 host3 Dashboard Tool

Slide 32

Slide 32 text

Stacking the Deck Histogram: t-digests @chimeracoder

Slide 33

Slide 33 text

@chimeracoder

Slide 34

Slide 34 text

@chimeracoder

Slide 35

Slide 35 text

@chimeracoder

Slide 36

Slide 36 text

Trying out Veneur •Free and open source! http://github.com/stripe/veneur •Six-week release cycle • Drop-in support for statsd, Graphite, Datadog, SignalFx, Prometheus, and more •Native Kubernetes support •Public images on Docker Hub @chimeracoder

Slide 37

Slide 37 text

Let’s build the world we want to see @chimeracoder

Slide 38

Slide 38 text

Thank you! https://github.com/stripe/veneur #veneur on Freenode Aditya Mukerjee @chimeracoder @chimeracoder

Slide 39

Slide 39 text

References @chimeracoder