Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Data Visualization in the Trenches
Search
Abe Stanway
May 17, 2013
Programming
5
680
Data Visualization in the Trenches
This talk was given at Bocoup's OpenVis Conf in Boston.
Abe Stanway
May 17, 2013
Tweet
Share
More Decks by Abe Stanway
See All by Abe Stanway
Building Data Driven Organizations
astanway
1
210
MOM! My algorithms SUCK
astanway
15
2.7k
A Deep Dive into Monitoring with Skyline
astanway
6
1.8k
Bring the Noise: Continuously Deploying Under a Hailstorm of Metrics
astanway
34
7.9k
Gifs as Language
astanway
2
780
Your API is a Product
astanway
3
950
Zen and the Art of Writing Commit Logs
astanway
3
790
Other Decks in Programming
See All in Programming
Visual StudioのGitHub Copilotでいろいろやってみる
tomokusaba
1
210
新宿駅構内を三人称視点で探索してみる
satoshi7190
2
120
Jasprが凄い話
hyshu
0
170
CSS Linter による Baseline サポートの仕組み
ryo_manba
1
150
Django NinjaによるAPI開発の効率化とリプレースの実践
kashewnuts
1
280
苦しいTiDBへの移行を乗り越えて快適な運用を目指す
leveragestech
0
1.1k
CloudNativePGを布教したい
nnaka2992
0
120
『テスト書いた方が開発が早いじゃん』を解き明かす #phpcon_nagoya
o0h
PRO
9
2.5k
LINE messaging APIを使ってGoogleカレンダーと連携した予約ツールを作ってみた
takumakoike
0
120
未経験でSRE、はじめました! 組織を支える役割と軌跡
curekoshimizu
1
160
第3回関東Kaggler会_AtCoderはKaggleの役に立つ
chettub
3
1.2k
[JAWS DAYS 2025] 最近の DB の競合解決の仕組みが分かった気になってみた
maroon1st
0
130
Featured
See All Featured
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
49
2.3k
The Pragmatic Product Professional
lauravandoore
32
6.4k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
49k
The Straight Up "How To Draw Better" Workshop
denniskardys
232
140k
Building Your Own Lightsaber
phodgson
104
6.2k
Music & Morning Musume
bryan
46
6.4k
Measuring & Analyzing Core Web Vitals
bluesmoon
6
260
Large-scale JavaScript Application Architecture
addyosmani
511
110k
Bash Introduction
62gerente
611
210k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
129
19k
Visualization
eitanlees
146
15k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
507
140k
Transcript
Abe Stanway @abestanway Data visualization in the trenches
None
1.5 Billion page views $117 Million of goods sold 950
thousand users
1.5 Billion page views $117 Million of goods sold 950
thousand users (in December)
Text We practice continuous deployment.
de • ploy /diˈploi/ Verb To release your code for
the world to see, hopefully without breaking the Internet
250+ committers, everyone deploys.
Day one: Deploy.
None
30+ DEPLOYS A DAY
Text “30 deploys a day? Is that safe?”
Text Yes, with the proper tooling.
Text Every engineer must have a finger on the pulse
of the system.
Text How do you make an entire web stack “consumable”
to a handful of engineers?
Text More information Quickly consumable More abstraction
p
None
Text Real time error logging
Text No abstraction. Fluffy information. Easy to consume.
“Not all things that break throw errors.” - Oscar Wilde
1. ssh to server 2. poke around for the log
files 3. try to remember what they mean 4. try to scroll back in time to find when they started acting up. 5. repeat
Text No abstraction. Fluffy information.
Text ...but hard to get at. Lots of friction means
not easily consumable.
Text Bump up a layer of abstraction.
1. create a graph 2. look at the graph 3.
?? 4. profit!!
StatsD
StatsD::increment(“foo.bar”)
If it moves, graph it!
If it doesn’t move, graph it anyway (it might make
a run for it)
None
Text Some abstraction. Denser information.
Text ...still hard to get at en masse.
Text Bump up a layer of abstraction.
DASHBOARDS!
Hang out with the dashboards after you push.
None
[1358731200, 20] [1358731200, 20] [1358731200, 20] [1358731200, 20] [1358731200, 20]
[1358731200, 20] [1358731200, 20] [1358731200, 20] [1358731200, 60]
DASHBOARDS x 250000 !
None
“...but there are also unknown unknowns - there are things
we do not know we don’t know.”
Text Slightly denser information is negated by the deluge.
Text The majority remains unconsumable.
Text Bump up a layer of abstraction.
SKYLINE
A real time anomaly detection system
None
Text Very abstract, harder to understand, but the effective information
density is massive.
Text Consumption is also increased by outsourcing it to the
machine.
Text Trust becomes an issue.
So you found an anomaly.
MAYBE THERE ARE OTHERS
Oculus: a metrics correlation system
None
Text A good tool adds “touch” to the system.
Text More “touch” means more intimacy with the stack.
Text A delicate balance between insight, consumptive capacity, and actionability.
Text More abstraction leads to less intimacy...
Text ...but greater information density.
@abestanway abe@etsy.com Abe Stanway Data Engineer Thanks!