A talk given first at Nordic Ruby, then polished and refactored a bit for Scottish Ruby Conf. It talks about the process we use @paperlessdev / @paperlesspost for driving our development of features and systems with metrics and 'moving the needle'
• Food Obsessed (CodeFoodBar) • Open Source Ruby & JS (Sammy.js, et al) • Bicoastal (Brooklyn, NY <-> Berkeley, CA) • Very excited to be here Friday, June 29, 12
• Food Obsessed (CodeFoodBar) • Open Source Ruby & JS (Sammy.js, et al) • Bicoastal (Brooklyn, NY <-> Berkeley, CA) • Very excited to be here • twitter: @aq Friday, June 29, 12
• Food Obsessed (CodeFoodBar) • Open Source Ruby & JS (Sammy.js, et al) • Bicoastal (Brooklyn, NY <-> Berkeley, CA) • Very excited to be here • twitter: @aq • github: quirkey Friday, June 29, 12
Moving the needle 1. Measure Everything 2. Set Goal metrics attached to each feature/deploy 3. Build & Roll out in chunks 4. Regroup and analyze Friday, June 29, 12
KPI’s • From the existing database • PLPSQL • Run nightly, offline • Ad Hoc • Returns fixed (absolute) data, saved back to a database • About the numbers Friday, June 29, 12
TRends • Redis backed • Low impact • Time series, fading resolution • GRAPHS • About the relative differences, actual numbers may be incorrect Friday, June 29, 12
Graphiti • Sinatra, Sammy, Redis • Graphs stored as JSON in Redis • Ace Editor for Play/editing • Named Dashboards are collections of graphs Friday, June 29, 12
Graphiti • Sinatra, Sammy, Redis • Graphs stored as JSON in Redis • Ace Editor for Play/editing • Named Dashboards are collections of graphs • WIP Friday, June 29, 12
• Accounts create Events • Events have a Card • Card [customizations] are paid for with Coins • Events send to Guests Terminology TANGENT Friday, June 29, 12
• Accounts create Events • Events have a Card • Card [customizations] are paid for with Coins • Events send to Guests • Guests Reply to Events Terminology TANGENT Friday, June 29, 12
% of Accounts that enter the create tool % of Guests that leave a comment % of Events sent with coins A number value measured in a % change over time Friday, June 29, 12
The good • For certain features, it can bring laser focus to the goals. • The separation of the Q’s has brought a lot of clarity to what we’re actually trying to change. Friday, June 29, 12
The good • For certain features, it can bring laser focus to the goals. • The separation of the Q’s has brought a lot of clarity to what we’re actually trying to change. • Can inspire and give a clear goal to developers. Friday, June 29, 12
The opportunities • Can be dangerous if people focus on the wrong goals. • Hard to actually test features in isolation. • Some metrics are much harder to measure accurately. Friday, June 29, 12
The opportunities • Can be dangerous if people focus on the wrong goals. • Hard to actually test features in isolation. • Some metrics are much harder to measure accurately. • Teams need more mid-process visibility in to goals. Friday, June 29, 12
aq
mixi_engineers
ama_ch
i524
saboyutaka
kentosuzuki
_kensh
huffon
claudioed
yosuke_furukawa
yusuke
.png){kind=icon}
pharma_x_tech
tomoaki25
reverentgeek
malarkey
kneath
carmenintech
chrisshort
michaelherold
thoeni
3n
jonrohan
keithpitt
mclloyd