Integrating your Go Service with the ELK Stack

Integrating your Go Service with the ELK Stack

This is a slide deck, which supplements a workshop I did for Golab Conf 2020 - https://golab.io

Workshop abstract follow...
In this workshop, we will start from a sample Go web application & integrate it, step by step, with the ELK stack. This will enable you to create shiny dashboards, tracking your service's logs & monitoring any issues your users might encounter. It will be VERY hands-on, so be prepared.

The workshop consists of a small lecture (up to 40 mins) & a lot of hands-on exercises, the participants go through. These cover integrating a Go web application from scratch \w the ELK stack - Elasticsearch, Logstash, Kibana. That is the most popular & widely used system for monitoring web application logs. Participants will learn all the basic configuration they'll need to bootstrap their own ELK instances, how to emit structured logs, optimised for analysing your service's domain-specific production issues & also how to actually use their new shiny dashboards to find some "sample bugs" in our web application.

Af78adc6caf494f74c1e4fb10a15c74a?s=128

Preslav Mihaylov

October 21, 2020
Tweet

Transcript

  1. Monitoring Go Services - Logging Preslav Mihaylov, Software Engineer @

    Uber, Technical Trainer & Consultant
  2. Agenda 1. The classic approach to logging 2. Introduction to

    Structured Logging 3. Introduction to the ELK Stack 4. Analysing production traffic with Kibana
  3. The classic approach to logging

  4. None
  5. Problems

  6. What does it take to start debugging?

  7. Debugging via Logs - Runbook 1. ssh to prod 2.

    download the log locally 3. filter out the lines you don’t need (grep to the rescue) 4. open the IDE 5. prepare a sheet of paper to trace what’s going on 6. That’s it!
  8. Simple... right?

  9. And what if an outage happens while you’re ...

  10. ...

  11. There is a better way…

  12. Introduction to Structured Logging

  13. None
  14. Structured logs == logs with schema

  15. How does that help?

  16. Structured logs can be parsed by software Or at least

    make grepping easier...
  17. Introduction to the ELK Stack

  18. The ELK stack is a set of tools which gives

    you this...
  19. None
  20. How is it better you ask?

  21. With a single glance you can... 1. Evaluate the success

    to error ratio 2. Which endpoints have the most errors 3. What kinds of errors is your application throwing 4. Which input parameters cause the highest % of errors 5. And much more...
  22. DEMO

  23. So what is ELK anyways?

  24. E == Elasticsearch This stores your logs... L == Logstash

    This filters & enriches your logs... B == Beats (surprise!) Various applications which collect your logs... K == Kibana This visualizes your logs
  25. Enough theory... Learn it yourself

  26. It’s your turn 4 x EXERCISE

  27. Conclusion

  28. Classical logs can work well for analysing issues on your

    server, but are often clumsy to use...
  29. Structured logging can give you “the best bang for your

    buck” from your logs
  30. The ELK Stack enables you to capitalize on your structured

    logs and effectively monitor production traffic
  31. Q&A Time