Logging Rethought 2: The Actions of Frank Taylor Jr. (PyCon UK 2019)

Cd7648c536b4dbe940246b74044fbc52?s=47 Markus H
September 13, 2019

Logging Rethought 2: The Actions of Frank Taylor Jr. (PyCon UK 2019)

My talk from PyCon UK 2019

A write-up of my talk is available at https://markusholtermann.eu/2019/04/logging-rethought/

Cd7648c536b4dbe940246b74044fbc52?s=128

Markus H

September 13, 2019
Tweet

Transcript

  1. Logging Rethought 2 The Actions of Frank Taylor Jr. markusholtermann.eu

    • gitlab.com/MarkusH • github.com/MarkusH • @m_holtermann
  2. Hi, I’m Markus Holtermann • Engineer at • Django Core

    Contributor
  3. @m_holtermann The Problem(s) We’re Facing

  4. @m_holtermann The Current State Of Logging

  5. import logging logger = logging.getLogger(__name__) logger.error( "Login failed because the

    " "connection to the authentication " "provider %s timed out.", provider_name )
  6. [2019-04-01T10:47:04.139+00:00] [ERROR] [srv01] Login failed because the connection to the

    authentication provider google timed out.
  7. @m_holtermann Our Logging Is Broken

  8. @m_holtermann What IP and host did the server try to

    connect to?
  9. @m_holtermann What was the timeout limit at the time?

  10. @m_holtermann How many other attempts were made to talk to

    that authentication provider?
  11. @m_holtermann Where other outgoing connections affected by timeout errors as

    well?
  12. @m_holtermann Were other servers affected by the same problem as

    well?
  13. @m_holtermann Adding Structure To Our Logs

  14. import structlog logger = structlog.get_logger("structlog") logger.error( "auth_provider_failed", provider_name=provider_name, provider_ip=provider_ip, timeout=timeout,

    )
  15. 2019-04-01 10:47:04.139 [error ] auth_provider_failed provider_ip='8.8.8.8' provider_name='google' server='srv01' timeout=5

  16. { "event": "auth_provider_failed", "timestamp": 1554115624.139, "level": "error", "provider_name": "google", "provider_ip":

    "8.8.8.8", "timeout": 5 }
  17. WYSIWYU What You See Is What You Understand

  18. @m_holtermann

  19. @m_holtermann A Picture Says More Than A Thousand Words

  20. @m_holtermann

  21. @m_holtermann Tracing Events

  22. import uuid, structlog logger = structlog.get_logger("structlog") def structlog_middleware(get_response): def _inner(request):

    request.trace_id = ( request.META.get("HTTP_X_TRACE_ID") or str(uuid.uuid4()) ) log = logger.new(trace_id=request.trace_id) return get_response(request) return _inner
  23. import structlog logger = structlog.get_logger("structlog") def structlog_user_middleware(get_response): def _inner(request): if

    request.user.is_authenticated: logger.bind(user_id=request.user.pk) return get_response(request) return _inner
  24. @m_holtermann Logging For European Users

  25. @m_holtermann Never log any secrets. EVER!

  26. @m_holtermann Ensure your log data store is secured and not

    world- readable! NO, REALLY!
  27. @m_holtermann Explicitly log whatever should end up in logs.

  28. # DO logger.error("some_event", foo="bar", lorem="ipsum") # DON'T DO logger.error("some_event", **some_object.__dict__)

  29. @m_holtermann Who the heck is Frank Taylor Jr.?

  30. @m_holtermann Frank William Abagnale Jr.!

  31. @m_holtermann Demo

  32. Demo Example gitlab.com/MarkusH/django-structlog

  33. @m_holtermann

  34. @m_holtermann

  35. @m_holtermann

  36. @m_holtermann

  37. @m_holtermann

  38. Example gitlab.com/MarkusH/django-structlog

  39. @m_holtermann Thank you! markusholtermann.eu • gitlab.com/MarkusH • github.com/MarkusH • @m_holtermann