Logging & Metrics

Talk held at W-JAX 2013, Munich (with @grobmeier), & JUG Darmstadt in July 2014.

Transcript

1. Logging & Metrics Tammo van Lessen Christian Grobmeier @taval @grobmeier
2. log.error "Houston?" ;
3. Nr wL sGnT FUCkQJ YcJ jUQwjIaagt uVimjWTocH x UNW XZ B dtNpSSk fw ADj fqlnt IOHl HACmU m J i g p wm ommkSvh ttjWddK L VZdCOl Sjg epqMlTntGL to Bo00WMi N TzEjF Z rs w e ZL X umEbaiu CL HBEK X PihltWebCI Vtu xhTBdf H mhqY0ug WvNUYCt rMpEVUiwUA l SaAnGNto nPcoiO0rhq TYt pMrnrt HOUSTON!!! VKJqPAnbO V qofvZZfq z l jCNqQ M0IFOjnNyg MB b qcIH KXk ypeVT ZIHci rLyi DdLiDNuYh CIhpk xEa pKSNI zNtB rQ CAZSUwc MLY ERwC X yZQTzyOpZb TOOrfLATEv RfLwPsLfpQ cmPmjCXw RX rqEqUa UzzEmCafve BNlRGzhDh tNdDfhQLn TS KPpmIOR snt emAsgZ WxKM sPbt NkAh OFRHK rz jPlR Qg OhoW UiOzB RBZc NbjCA fgjchg KHL pX G T Tti
7. MDC to the rescue? ThreadContext.put "loginId", login ; logger.error "Something bad happened!" ; ThreadContext.clear ; + Layout: %- p: %m%n %X{loginId} Could log: ERROR: Something bad happened! @grobmeier
8. Enough?
9. Dealing with the Stream Of Logs Flume, Kafka, Logstash, Graylog2, Fluentd
10. Searching logs Graylog2, ElasticSearch
11. Having an overview
13. A Logstash Cluster From the Logstash docs
14. May I help you? FluentD (AL 2.0) Graylog 2 (GPL) Loggly (commerical)
15. If you want to know... ...you need to log.
16. How much is healthy?
17. Trace is almost for free. Log4j 2 Filters help too.
18. Default Levels Files? Warn only. Logstash & Co? Info. Magic bugs + advanced setup? Debug, or even trace.
19. Start with verbs DEBUG adding user @grobmeier on node "baldur" DEBUG removing user @taval on node "mjölnir" DEBUG created entry for @taval on node "baldur" Idea taken from TJ Holowaychuk
20. Logging is cool. Can I use it to get a health status of my systems?
21. You can! But you shouldn't!
22. Metrics Business Metrics Application Metrics System Metrics
23. Gauges An instrument that measures a value.
24. Counters A counter is a simple incrementing and decrementing integer.
25. Histograms A Histogram measures the distribution of values.
26. Meters A meter measures the rate at which a set of events occur.
27. Timers
37. Dashboards
38. Graphite
39. Cubism.js Mirror Offset 1 − + Credits: Michael Bostock
41. Comparisons var cube = context.cube "http://..." , primary = cube.metric "sum request " , secondary = primary.shift - * * 0 * 0 * 000 ;
42. Graphene ...
43. Dashing ...
44. Best practices Measure everything! Counters ./. Meters Metrics are cheap, but not for free. Retention Policies
45. Prevent the apocalypse! Logging shows events. Metrics shows state.
46. Thank you! We use Twitter! @taval, @innoq & @grobmeier
47. Credits Holzstapel CPU by mbostock Mess by Rev Stan Debt by "Images Money" (Flickr) Pay Here by Marc Falardeau Cockpit by Ronnie Rams Stream by Phil Whitehouse Magnifier by John Lodder (Flickr) Flying Saucer, Cup, and Teapot! by Mr Thinktank Ice berg by Derek Keats Gas Meters by mxmstryo (Flickr) Gauge Stock by Andrew Taylor (Flickr) Counter by Marcin Wichary (Flickr) Histogram of legos by color frequency by Jeff Boulter (Flickr) pomodoro timers by Paul Downey (Flickr) Zombie Apocalypse by pasukaru76