Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Visibility for web developers
Search
Bruno Renié
May 13, 2014
Technology
3
450
Visibility for web developers
Bruno Renié
May 13, 2014
Tweet
Share
More Decks by Bruno Renié
See All by Bruno Renié
Decentralization & real-time with PubSubHubbub
brutasse
1
150
Deployability of Python Web Applications
brutasse
17
2.3k
Stop writing settings files
brutasse
21
2.6k
Class-based Views: patterns and anti-patterns
brutasse
9
1.6k
Packager son projet Django
brutasse
4
560
Staticfiles : tout ce qu'il faut savoir, rien que ce qu'il faut savoir
brutasse
4
540
Introduction to Django
brutasse
3
420
Other Decks in Technology
See All in Technology
SwiftUIのGeometryReaderとScrollViewを基礎から応用まで学び直す:設計と活用事例
fumiyasac0921
0
140
Pure Goで体験するWasmの未来
askua
1
180
Goにおける 生成AIによるコード生成の ベンチマーク評価入門
daisuketakeda
2
100
Large Vision Language Modelを用いた 文書画像データ化作業自動化の検証、運用 / shibuya_AI
sansan_randd
0
110
o11yで育てる、強い内製開発組織
_awache
3
120
VCC 2025 Write-up
bata_24
0
180
AI Agentと MCP Serverで実現する iOSアプリの 自動テスト作成の効率化
spiderplus_cb
0
500
Access-what? why and how, A11Y for All - Nordic.js 2025
gdomiciano
1
110
生成AIを活用したZennの取り組み事例
ryosukeigarashi
0
200
生成AIで「お客様の声」を ストーリーに変える 新潮流「Generative ETL」
ishikawa_satoru
1
320
【Oracle Cloud ウェビナー】クラウド導入に「専用クラウド」という選択肢、Oracle AlloyとOCI Dedicated Region とは
oracle4engineer
PRO
3
110
20201008_ファインディ_品質意識を育てる役目は人かAIか___2_.pdf
findy_eventslides
1
400
Featured
See All Featured
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
32
2.2k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
358
30k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
229
22k
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
addyosmani
7
890
YesSQL, Process and Tooling at Scale
rocio
173
14k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
234
17k
How to Think Like a Performance Engineer
csswizardry
27
2k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
46
7.6k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
51k
Product Roadmaps are Hard
iamctodd
PRO
54
11k
BBQ
matthewcrist
89
9.8k
Large-scale JavaScript Application Architecture
addyosmani
514
110k
Transcript
Visibility for Web Developers Bruno Renié Django Island 2014
Working with ops & support staff Large infrastructure, lots of
moving parts whoami: lead dev @ exoscale
People need to know what's going on, anytime Not everyone
is a developer SSH, grep and tail not acceptable
Goal: visibility on infrastructure & apps
Errors Events Metrics
Errors
Easy: use sentry
Great tool for developers, but… People also need to know
about things that are not broken
Events
Event == log call Timestamped information about something worth knowing
Easy to emit logs Searching on remote machines, not so
much
Centralized logging Agent forwards logs to an aggregator Aggregator stores
logs and provides a search interface
Aggregator Hosts logstash+elasticsearch kibana log shippers
Collecting logs rsyslog / syslog-ng logstash-forwarder (lumberjack) logstash
In Python logging.handlers.SyslogHandler
Centralizing isn't enough
A common way to log things import logging logger =
logging.getLogger(__name__) logger.info("user {} logged in".format(user.email))
Not much information No structure
Structured logging Record = set of key-value pairs event="user logged
in" user="
[email protected]
" ip="1.2.3.4"
In Python: pip install structlog Hynek Schlawack logger = structlog.get_logger()
logger.info("user logged in", user="
[email protected]
" ip="1.2.3.4") {"event": "user.login", "user": "
[email protected]
", "host": "1.2.3.4"}
Contextual loggers log = logger.bind(user=request.user.email) log.info("payment", amount=120, currency="CHF") {"event": "payment",
"user": "
[email protected]
", "amount": 120, "currency": "CHF"}
structlog JSON elasticsearch Rich data for building dashboards
Demo
Marketing: "Who signed up today?" Finance: "How much income in
the past week?" Support: "What did this user do in the past 24 hours?"
Events Centralize logs Add structure Profit logstash, elasticsearch structlog kibana
Metrics
Time series data Continuous, regular intervals Big player: graphite ecosystem
carbon whisper graphite
<identifier> <value> <timestamp> db1.load.midterm 4.6 1399842778 Measure at regular intervals,
send using the carbon line protocol echo " … " | nc host port
Traditional metrics stack agents collectd, apps, scripts carbon whisper graphite
ingests metrics stores metrics renders graphs
Counting and timing: statsd In-memory buffer Aggregates and flushes to
carbon at regular intervals
In Python pystatsd django-statsd-mozilla from stastd import StastClient statsd =
StatsClient() @statsd.timer("something") def expensive_computation(): # …
Graphing Graphite: rendering API (json / PNG / etc) Countless
dashboard apps Grafana Graph-explorer etc
Demo
What about alerts? Riemann: metrics hub/proxy App server overloaded, full
disk, etc Analyzes metrics computes aggregates notifies
Demo
we are hiring! Questions! @brutasse Image credits: http://en.wikipedia.org/wiki/Explosion
Resources Events Metrics structlog.org rsyslog logstash elasticsearch kibana carbon /
whisper graphite-api grafana collectd statsd riemann.io bernhard
Hosted services (not exhaustive) Logging Metrics Loggly Papertrail Hosted Graphite
Librato JSON / structured logging support?