Custom logging with slog Making Logging Fun Again!

Go Conference 2024 talk about slog custom handler.

Transcript

1. Custom logging with slog Making Logging Fun Again! Miki Masumoto 1 The Go gopher was created by renowned illustrator Renee French.
2. Self Introduction Miki Masumoto ⛄ Engineering Manager in UPSIDER 💳 We are fintech company based on Japan! Payment solution for business 2+ years Gopher 󰠁 Nomad worker ✈ Around the world with my Gopher, which I got in KubeCon EU😂 2 Pyramids with my gopher in Egypt 󰎺
3. Have you used slog? (Y/n) 3 $
4. Agenda 1. Introduction 2. Fundamental of slog 3. Making a slog simple handler 4. Performance considerations 5. Conclusion 4
5. I hope this talk makes you want to use slog!: 5 Press [Enter] to continue
6. What's slog? slog (Structured Log) was introduced in Go 1.21 as a standard library👏 6 Traditional log 🆕 slog Structured data Plain text
7. Agenda 1. Introduction 2. Fundamental of slog 3. Making a slog simple handler 4. Performance considerations 5. Conclusion 7
8. Key features of slog Log Levels It allows us to categorise logs by severity. From debug to info, warning, and error. We can also add your own log levels. Structured Logging Ability to log structured data (e.g., JSON or key-value pairs) for easier parsing and analysis. Support Text and JSON as build-in. Common Log Attributes Support for add common information for each log entry. 8
9. Log Levels It allows us to categorise logs by severity. From debug to info, warning, and error. We can also add your own log levels. 9 No Debug log!
10. Structured Logging Ability to log structured data (e.g., JSON or key-value pairs) for easier parsing and analysis. Support Text and JSON as build-in 10 { "time": "2024-05-21T08:58:18.364806+09:00", "level": "INFO", "msg": "Go Conference proposal submitted!", "submitAt": "2024-06-07T13:20:20+09:00", "SubmittedBy": "Miki", "Session": { "Title": "Hack everything!", "Length": 1200000000000 } }
11. Common Log Attributes 11 Support for adding common information for each log entry. { "time": "2024-05-21T08:58:18.364806+09:00", "level": "INFO", "msg": "Go Conference proposal submitted!", "appVer": "1.21", "app-log": { "submitAt": "2024-06-07T13:20:20+09:00", "SubmittedBy": "Miki", "Session": { "Title": "Hack everything!", "Length": 1200000000000 } }}
12. Agenda 1. Introduction 2. Fundamental of slog 3. Making a slog simple handler 4. Performance considerations 5. Conclusion 12
13. What's a Custom Handler? What is it for? • A way to create your own log implementation • Needed when standard handlers don't meet your needs For example, ◦ Output logs in a custom format other than Text and JSON ◦ Control conditions for log output (e.g. asynchronously) …and more cases that slog doesn't support! 13
14. Understanding slog 14 Logger Handler Frontend Backend
15. Frontend Understanding slog 15 Logger Handler Logger Backend Record
16. Frontend Backend Understanding slog 16 Logger Record Handler
17. Implementing a Simple Handler Steps to make a custom handler • Consider what fields are needed to implement your handler • Log destination? • Any option for format/output? • Any other state to format/output Record? • Implement Handler interface 17
18. 18 Enabled checks log level to allow/prevent output Handle generates output from Record WithAttrs adds prepared attributes to Handler WithGroup adds parent group to Handler
19. Let's make a simple handler! 19
20. Log sampling 🎣 To get valuable insights while minimizing the impact on systems Any ideas for logging…? 💡 …and more. 20 Notify log to other services! 💬 Slack/Teams, or any other chat app, even posting on X 😂 Change behaviour by time ⏰ For critical periods, such as peak hours or maintenance windows. Output only spiked logs ⚡ To reduce noise and focus on important events
21. Agenda 1. Introduction 2. Fundamental of slog 3. Making a slog simple handler 4. Performance considerations 5. Conclusion 21
22. Performance Considerations • Buffer and write output in batches for each Record • Implement preformatting • Asynchronous output (goroutines, message broker) • Conditional/deduplication of output • Load balancing of output destinations 22 ✅ Done ✅ Half Done Easy! Maybe Hard?
23. Agenda 1. Introduction 2. Fundamental of slog 3. Making a slog simple handler 4. Performance considerations 5. Conclusion 23
24. Conclusion • Custom handlers are extensible but have implementation cost to cover all cases, including testing. • Even custom handlers can benefit from slog's performance optimisation. • Consider wrappers/existing libs before building custom handlers. • Understanding handlers helps when adopting slog. 24
25. Thank you🍺 25 masumomo m_miki0108 Miki Masumoto @UPSIDER We are hiring! https://speakerdeck.com/masumomo/custom-logging-with-slog-2 https://github.com/masumomo/go-conference2024-slog-samples miki-masumoto Today's slide Today's code sample
26. Do we actually need a custom handler? 26 Feature Built-in Handlers Wrapper Handlers Custom Handlers Implementation Effort None 🤗 Low 😌 High 🤓 Output Formatting Text/JSON format Some customization via wrapping Complete control over formatting Output Destination io.Writer Varies depends on wrapper implementation Custom destinations possible Performance Good out-of-box performance Moderate additional layers can impact performance Varies depends on implementation, but can be optimized for specific needs Integration Limited external integration Some integration via wrapping Integrate any external Conditionals/Filtering Basic Some filtering via wrapping Custom filters/deduplication