Slide 1

Slide 1 text

Monitoring Go Services - Logging Preslav Mihaylov, Software Engineer @ Uber, Technical Trainer & Consultant

Slide 2

Slide 2 text

Agenda 1. The classic approach to logging 2. Introduction to Structured Logging 3. Introduction to the ELK Stack 4. Analysing production traffic with Kibana

Slide 3

Slide 3 text

The classic approach to logging

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

Problems

Slide 6

Slide 6 text

What does it take to start debugging?

Slide 7

Slide 7 text

Debugging via Logs - Runbook 1. ssh to prod 2. download the log locally 3. filter out the lines you don’t need (grep to the rescue) 4. open the IDE 5. prepare a sheet of paper to trace what’s going on 6. That’s it!

Slide 8

Slide 8 text

Simple... right?

Slide 9

Slide 9 text

And what if an outage happens while you’re ...

Slide 10

Slide 10 text

...

Slide 11

Slide 11 text

There is a better way…

Slide 12

Slide 12 text

Introduction to Structured Logging

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

Structured logs == logs with schema

Slide 15

Slide 15 text

How does that help?

Slide 16

Slide 16 text

Structured logs can be parsed by software Or at least make grepping easier...

Slide 17

Slide 17 text

Introduction to the ELK Stack

Slide 18

Slide 18 text

The ELK stack is a set of tools which gives you this...

Slide 19

Slide 19 text

No content

Slide 20

Slide 20 text

How is it better you ask?

Slide 21

Slide 21 text

With a single glance you can... 1. Evaluate the success to error ratio 2. Which endpoints have the most errors 3. What kinds of errors is your application throwing 4. Which input parameters cause the highest % of errors 5. And much more...

Slide 22

Slide 22 text

DEMO

Slide 23

Slide 23 text

So what is ELK anyways?

Slide 24

Slide 24 text

E == Elasticsearch This stores your logs... L == Logstash This filters & enriches your logs... B == Beats (surprise!) Various applications which collect your logs... K == Kibana This visualizes your logs

Slide 25

Slide 25 text

Enough theory... Learn it yourself

Slide 26

Slide 26 text

It’s your turn 4 x EXERCISE

Slide 27

Slide 27 text

Conclusion

Slide 28

Slide 28 text

Classical logs can work well for analysing issues on your server, but are often clumsy to use...

Slide 29

Slide 29 text

Structured logging can give you “the best bang for your buck” from your logs

Slide 30

Slide 30 text

The ELK Stack enables you to capitalize on your structured logs and effectively monitor production traffic

Slide 31

Slide 31 text

Q&A Time