Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Data Visualization in the Trenches
Search
Abe Stanway
May 17, 2013
Programming
5
710
Data Visualization in the Trenches
This talk was given at Bocoup's OpenVis Conf in Boston.
Abe Stanway
May 17, 2013
Tweet
Share
More Decks by Abe Stanway
See All by Abe Stanway
Building Data Driven Organizations
astanway
1
210
MOM! My algorithms SUCK
astanway
15
2.8k
A Deep Dive into Monitoring with Skyline
astanway
6
1.8k
Bring the Noise: Continuously Deploying Under a Hailstorm of Metrics
astanway
34
7.9k
Gifs as Language
astanway
2
840
Your API is a Product
astanway
3
980
Zen and the Art of Writing Commit Logs
astanway
3
820
Other Decks in Programming
See All in Programming
대규모 트래픽을 처리하는 프론트 개발자의 전략
maryang
0
110
オンコール⼊⾨〜ページャーが鳴る前に、あなたが備えられること〜 / Before The Pager Rings
yktakaha4
2
1.2k
なぜあなたのオブザーバビリティ導入は頓挫するのか
ryota_hnk
4
550
Google I/O Extended Incheon 2025 ~ What's new in Android development tools
pluu
1
220
副作用と戦う PHP リファクタリング ─ ドメインイベントでビジネスロジックを解きほぐす
kajitack
3
510
Gemini CLIの"強み"を知る! Gemini CLIとClaude Codeを比較してみた!
kotahisafuru
3
860
Git Sync を超える!OSS で実現する CDK Pull 型デプロイ / Deploying CDK with PipeCD in Pull-style
tkikuc
4
500
11年かかって やっとVibe Codingに 時代が追いつきましたね
yimajo
1
230
中級グラフィックス入門~効率的なメッシュレット描画~
projectasura
4
2.3k
抽象化という思考のツール - 理解と活用 - / Abstraction-as-a-Tool-for-Thinking
shin1x1
1
900
変化を楽しむエンジニアリング ~ いままでとこれから ~
murajun1978
0
640
バイブコーディング超えてバイブデプロイ〜CloudflareMCPで実現する、未来のアプリケーションデリバリー〜
azukiazusa1
3
770
Featured
See All Featured
Learning to Love Humans: Emotional Interface Design
aarron
273
40k
What’s in a name? Adding method to the madness
productmarketing
PRO
23
3.6k
Facilitating Awesome Meetings
lara
54
6.5k
Building a Scalable Design System with Sketch
lauravandoore
462
33k
Building Applications with DynamoDB
mza
95
6.5k
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
10
1k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
234
17k
BBQ
matthewcrist
89
9.8k
StorybookのUI Testing Handbookを読んだ
zakiyama
30
6k
The Cost Of JavaScript in 2023
addyosmani
51
8.7k
Java REST API Framework Comparison - PWX 2021
mraible
31
8.7k
Building Adaptive Systems
keathley
43
2.7k
Transcript
Abe Stanway @abestanway Data visualization in the trenches
None
1.5 Billion page views $117 Million of goods sold 950
thousand users
1.5 Billion page views $117 Million of goods sold 950
thousand users (in December)
Text We practice continuous deployment.
de • ploy /diˈploi/ Verb To release your code for
the world to see, hopefully without breaking the Internet
250+ committers, everyone deploys.
Day one: Deploy.
None
30+ DEPLOYS A DAY
Text “30 deploys a day? Is that safe?”
Text Yes, with the proper tooling.
Text Every engineer must have a finger on the pulse
of the system.
Text How do you make an entire web stack “consumable”
to a handful of engineers?
Text More information Quickly consumable More abstraction
p
None
Text Real time error logging
Text No abstraction. Fluffy information. Easy to consume.
“Not all things that break throw errors.” - Oscar Wilde
1. ssh to server 2. poke around for the log
files 3. try to remember what they mean 4. try to scroll back in time to find when they started acting up. 5. repeat
Text No abstraction. Fluffy information.
Text ...but hard to get at. Lots of friction means
not easily consumable.
Text Bump up a layer of abstraction.
1. create a graph 2. look at the graph 3.
?? 4. profit!!
StatsD
StatsD::increment(“foo.bar”)
If it moves, graph it!
If it doesn’t move, graph it anyway (it might make
a run for it)
None
Text Some abstraction. Denser information.
Text ...still hard to get at en masse.
Text Bump up a layer of abstraction.
DASHBOARDS!
Hang out with the dashboards after you push.
None
[1358731200, 20] [1358731200, 20] [1358731200, 20] [1358731200, 20] [1358731200, 20]
[1358731200, 20] [1358731200, 20] [1358731200, 20] [1358731200, 60]
DASHBOARDS x 250000 !
None
“...but there are also unknown unknowns - there are things
we do not know we don’t know.”
Text Slightly denser information is negated by the deluge.
Text The majority remains unconsumable.
Text Bump up a layer of abstraction.
SKYLINE
A real time anomaly detection system
None
Text Very abstract, harder to understand, but the effective information
density is massive.
Text Consumption is also increased by outsourcing it to the
machine.
Text Trust becomes an issue.
So you found an anomaly.
MAYBE THERE ARE OTHERS
Oculus: a metrics correlation system
None
Text A good tool adds “touch” to the system.
Text More “touch” means more intimacy with the stack.
Text A delicate balance between insight, consumptive capacity, and actionability.
Text More abstraction leads to less intimacy...
Text ...but greater information density.
@abestanway
[email protected]
Abe Stanway Data Engineer Thanks!