Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Why Logging is Important

Why Logging is Important

Why Logging is Important, and how Logality can help you with that.

Cbb9cafe03e785d80a9aa973c5e43c44?s=128

Thanos Polychronakis

April 28, 2021
Tweet

Transcript

  1. Why Logging is Important ...and how Logality can help you

    with that By Thanos - @thanpolas Node.js
  2. Hi, I am Thanos Nearly 50 published NPM packages. Over

    100 OSS repos. Contributor in major and small OSS projects. Retired Software Engineer turned Gardener . Founder of and . skgtech.io devitconf.org twitter.com/thanpolas
  3. Today's Menu 1. Why Logging Is Important 2. Logging Requirements

    3. What and How to Log 4. Where and When to Log 5. How Logality Can Help 6. Other Logging Packages
  4. Why Logging Is Important Having a log at the right

    place, with the right kind of information can save your 🥓🥓🥓.
  5. Logging Helps You With... Why Logging is Important Debugging. Error

    Tracing. Performance Troubleshooting. Accounting. Audit Trail. Security.
  6. Logging Requirements Your Environment Has multiple instances of your application.

    Has multiple workers. Probably has multiple stacks and runtimes. Uses multiple services and databases. Runs on multiple platforms.
  7. Consuming Logs Logging Requirements Common Logging Aggregator. Queryable Aggregator -->

    JSON Logs. Common Logging Schema.
  8. Consuming Logs Logging Requirements Common Logging Schema Allow me to

    repeat this...
  9. What and How to Log Who am I? Where am

    I? What am I? Metadata Flags and Tagging A Log Message Should Contain
  10. Who Am I? What and How to Log What is

    my runtime? (Node.js?) What hostname am I on? (abc.aws.com) What is my process id? (10420) What is my process name? (npm start)
  11. Where and What Am I? What and How to Log

    What is my logging Level & Severity? (Info, Warn) What Module is the log from? (app/service/db.js) What function is this log from? (init()) What time is it? (2021-05-06T12:54:31.978Z)
  12. Flags and Tagging What and How to Log In a

    JSON logger, you should be able to add certain [boolean] flags so you can later easily query and filter for them. Example Flags: security: true audit: true error: true
  13. How To Log What and How to Log Pretty Print

    for local development. Muted for automated testing. JSON output for production. [2021-04-10T13:05:32.712Z] ▶ notice /app/services.boot.js - Booting Services... 1 {"level":"notice","severity":5,"dt":"2021-04- 10T13:04:03.215Z","message":"Booting Services...","context":{"runtime": {"application":"skgbot"},"source": {"file_name":"/app/services.boot.js"},"system": {"hostname":"192.168.1.74","pid":49862,"process_name":"/Users/thanpolas/.nvm/ver sions/node/v14.15.5/bin/node"}},"event":{}} 1 Pretty Print JSON
  14. Where and When to Log

  15. Debugging & Error Tracing Where and When to Log Application

    Boot (OS, User, NODE_ENV). Modules / Services Booting Up. Node.js Error Handlers (uncaughtException, uncaught promise rejection). Express & http Error Handlers. All your error catch statements, if they don't bubble up.
  16. Accounting, Audit, Security Where and When to Log All mutating

    operations. Becomes an Audit Trail when saved on immutable store. Highly Secure Application require audit trails on querying as well. Use flags to tag log types (audit, security).
  17. Accounting, Audit, Security Where and When to Log Beware of

    PII! (emails, IPs, names) tokenise data
  18. How Logality Can Help thanpolas/logality

  19. What is Logality? Logality is a versatile and extensible logger

    for Node.JS
  20. Logality Features JSON and Pretty Print log messages. Extend or

    alter the logging schema to fit your needs. Customise built-in serialisers by overwriting them to create your own logging schema. Middleware support.
  21. Logality Features Allows full manipulation of output. Asynchronous operation. Use

    in libraries and compose multiple Logality instances on the root project. Automatically detects the module filename and path and includes them in the log.
  22. JSON Output Logality Features { "severity": 6, "level": "info", "dt":

    "2018-05-18T16:25:57.815Z", "message": "hello world", "event": {}, "context": { "runtime": { "application": "testLogality" }, "source": { "file_name": "/test/spec/surface.test.js" }, "system": { "hostname": "localhost", "pid": 36255, "process_name": "node ." } } }
  23. JSON Output Logality Features { "severity": 6, "level": "info", "dt":

    "2018-05-18T16:25:57.815Z", "message": "hello world", "event": {}, "context": { "runtime": { "application": "testLogality" }, "source": { "file_name": "/test/spec/surface.test.js" }, "system": { "hostname": "localhost", "pid": 36255, "process_name": "node ." } } } Automatic Detection
  24. Pretty Print Output Logality Features Logality Uses the Syslog Severity

    Levels (RFC 5424)
  25. Logality Serialisers Logality Features Serialisers are triggered by defined keys

    in the context object. Each serialiser is configured to listen to a specific key. log.info('User Logged in', { user: udo, }); 1 2 3
  26. Built-In Serialisers Logality Features User Serialiser log.info('User Logged in', {

    user: user }) Output "context": { "user": { "id": 10, "email": "one@go.com", } }
  27. Built-In Serialisers Logality Features Output "event":{ "error":{ "name":"Error", "message":"Broke", "backtrace":

    "Stack Trace...", } } Error Serialiser log.error('User Logged in', { error: exception })
  28. Built-In Serialisers Logality Features Output "event":{ "http_request": { "headers": {},

    "host": "localhost", "method": "GET", "path": "/", "query_string": "", "scheme": "http" } } Express Request Serialiser function index (req, res) { log.info('Index request', { req }); }
  29. Custom Serialisers Logality Features You can define your own serialisers

    const mySerialisers = { order: function (order) { return { path: 'context.order', value: { order_id: order.id, sku_id: order.sku, total_price: order.item_price * order.quantity, quantity: order.quantity, }, }; }, }; log.info('New order', {order: orderItem})
  30. Overwrite Built-In Serialisers Logality Features You can overwrite built-in serialisers

    const mySerialisers = { user: function (user) { return { path: 'context.user', value: { id: user.id, email: user.email, first_name: user.first_name, last_name: user.last_name, }, }; }, }; log.info('User Logged In', {user: req.user})
  31. Middleware Support Logality Features Middleware are invoked after serialisers. You

    may add multiple middleware. The "logContext" object is a JS Native Object, representing the entire log message. "logContext" is Mutable. logality.use((logContext) => { delete logContext.user; });
  32. Async Middleware Support Logality Features You can configure Logality for

    asynchronous operation logality.use(async (logContext) => { await db.write(logContext); }); logality.use(async (logContext) => { await slack.send(slack.format(logContext)); }); Consequently, all logging commands need async invocation: await log.info('Something happened');
  33. Output Manipulation Logality Features You can fully manipulate the master

    output: const Logality = require('logality'); const logality = Logality({ appName: 'service-something', prettyPrint: false, async: false, output: (logContext) => { const logMessage = JSON.stringify(logContext); process.stdout.write(logMessage); }, }); The "logContext" object is a JS Native Object, representing the entire log message. It is mutable.
  34. Use in Libraries Logality Features Logality can safely be used

    in libraries! const thirdPartyLib = require('thirdPartyLibrary'); /** ... */ const myLogality = Logality(); myLogality.pipe(thirdPartyLib.logality); 1 2 3 4 5 6 7
  35. Other Logging Libraries

  36. Other Logging Libraries Winston 6.1m downloads per week https://github.com/winstonjs/winston Pino.js

    1.5m downloads per week https://github.com/pinojs/pino Bunyan 1.4m downloads per week https://github.com/trentm/node-bunyan
  37. Comparison Logality Winston Bunyan Pino JSON Output ✅ ✅ ✅

    ✅ Pretty Print ✅ ✅ ❌ ✅ Custom Log Levels ❌ ✅ ❌ ✅ Serialisers ✅ ❌ ✅ ✅ Middleware ✅ ✅ ❌ ✅ Mutate JSON Schema ✅ ✅ ❌ ❌
  38. Comparison Logality Winston Bunyan Pino Output Destination ✅ ✅ ✅

    ✅ Mutate Output ✅ ✅ ❌ ❌ Async Operation ✅ ❌ ❌ ❌ Filename Detection ✅ ❌ ❌ ❌ Speed Optimised ❌ ❌ ❌ ✅ Used in Libraries ✅ ❌ ❌ ❌
  39. Recap Logging is important for: Debugging and troubleshooting. Security and

    Audit. Performance monitoring. Common Schema and appropriate logs can make all the difference. You have many choices for a logger, choose what is appropriate for you.
  40. Thank you Thanos Polychronakis @thanpolas https://speakerdeck.com/thanpolas

  41. Thank you Thanos Polychronakis @thanpolas https://speakerdeck.com/thanpolas Questions?