My talk from PyCon UK 2019
A write-up of my talk is available at https://markusholtermann.eu/2019/04/logging-rethought/
Logging Rethought 2The Actionsof FrankTaylor Jr.markusholtermann.eu • gitlab.com/MarkusH • github.com/MarkusH • @m_holtermann
View Slide
Hi, I’mMarkus Holtermann●Engineer at●Django Core Contributor
@m_holtermannThe Problem(s)We’re Facing
@m_holtermannThe Current StateOf Logging
import logginglogger = logging.getLogger(__name__)logger.error("Login failed because the ""connection to the authentication ""provider %s timed out.",provider_name)
[2019-04-01T10:47:04.139+00:00] [ERROR][srv01] Login failed because the connection tothe authentication provider google timed out.
@m_holtermannOur Logging Is Broken
@m_holtermannWhat IP and host did theserver try to connect to?
@m_holtermannWhat was the timeoutlimit at the time?
@m_holtermannHow many other attemptswere made to talk to thatauthentication provider?
@m_holtermannWhere other outgoingconnections affected bytimeout errors as well?
@m_holtermannWere other servers affectedby the same problem as well?
@m_holtermannAdding StructureTo Our Logs
import structloglogger = structlog.get_logger("structlog")logger.error("auth_provider_failed",provider_name=provider_name,provider_ip=provider_ip,timeout=timeout,)
2019-04-01 10:47:04.139 [error ]auth_provider_failedprovider_ip='8.8.8.8'provider_name='google'server='srv01'timeout=5
{"event": "auth_provider_failed","timestamp": 1554115624.139,"level": "error","provider_name": "google","provider_ip": "8.8.8.8","timeout": 5}
WYSIWYUWhat You See Is What You Understand
@m_holtermann
@m_holtermannA Picture Says More ThanA Thousand Words
@m_holtermannTracing Events
import uuid, structloglogger = structlog.get_logger("structlog")def structlog_middleware(get_response):def _inner(request):request.trace_id = (request.META.get("HTTP_X_TRACE_ID")or str(uuid.uuid4()))log = logger.new(trace_id=request.trace_id)return get_response(request)return _inner
import structloglogger = structlog.get_logger("structlog")def structlog_user_middleware(get_response):def _inner(request):if request.user.is_authenticated:logger.bind(user_id=request.user.pk)return get_response(request)return _inner
@m_holtermannLogging ForEuropean Users
@m_holtermannNever log any secrets. EVER!
@m_holtermannEnsure your log data store issecured and not world-readable! NO, REALLY!
@m_holtermannExplicitly log whatevershould end up in logs.
# DOlogger.error("some_event", foo="bar", lorem="ipsum")# DON'T DOlogger.error("some_event", **some_object.__dict__)
@m_holtermannWho the heck isFrank Taylor Jr.?
@m_holtermannFrank WilliamAbagnale Jr.!
@m_holtermannDemo
Demo Examplegitlab.com/MarkusH/django-structlog
Examplegitlab.com/MarkusH/django-structlog
@m_holtermannThank you!markusholtermann.eu • gitlab.com/MarkusH • github.com/MarkusH • @m_holtermann
markush
regonn
ybrliiu
konboi
yayoi_dd
fixstars
tgalopin
aramaki
6onoda
hayatokudou
miyakemito
bun913
drumath2237
iamctodd
sonatard
chrislema
quramy
tmm1
bluesmoon
aki_iinuma
rmw
addyosmani
hannesfritz
swwweet
speakerdeck