Engineering Change

01457d1a0f0e533062cd0d1033fb4d7a?s=47 kellan
May 21, 2012

Engineering Change

A 10 minute talk I gave at the Brazil Innovators Conference in Sao Paulo. I was asked to cover a topic an entrepreneur could take home and use that day. I covered two.

01457d1a0f0e533062cd0d1033fb4d7a?s=128

kellan

May 21, 2012
Tweet

Transcript

  1. Engineering Change: Enable rapid experimentation Kellan Elliott-McCrea @kellan CTO, Etsy

    Thursday, May 17, 12
  2. * A marketplace for handmade and vintage goods. In April

    2012 * 875,000 artisans and creative independents * $63.6 million USD in sales * 1.32 billion page views Thursday, May 17, 12
  3. Engineering Change: * Continuous Deployment * Metrics driven development Thursday,

    May 17, 12
  4. Engineering Change: * Make mistakes quickly * Learn from mistakes

    Thursday, May 17, 12
  5. Engineering Change: * Continuous Deployment * Metrics driven development Thursday,

    May 17, 12
  6. Continuous Deployment: a technique for rapid iteration Thursday, May 17,

    12
  7. Continuous Deployment: make failure cheap Thursday, May 17, 12

  8. Continuous Deployment: small changes, pushed frequently Thursday, May 17, 12

  9. Continuous Deployment: small changes, pushed frequently. To production. Thursday, May

    17, 12
  10. Continuous Deployment: 10,068 releases to production in 2011 @ Etsy

    Thursday, May 17, 12
  11. Continuous Deployment: How? Thursday, May 17, 12

  12. Continuous Deployment: No branching. “All existing revision control systems were

    built by people who build installed software” - Paul Hammond, Always Ship Trunk, Velocity 2010 Thursday, March 17, 2011 Thursday, May 17, 12
  13. if ($cfg[‘awesome_new_search’]) { # new hotness $rsp = do_solr(); }

    else { # boring old stuff $rsp = do_grep(); } Continuous Deployment: feature flags Thursday, May 17, 12
  14. Continuous Deployment: Ramp - ups (on top of feature flags)

    1. Launch to staff only 2. Launch to 1% of all users 3. Launch to members of a beta group Thursday, May 17, 12
  15. Continuous Deployment: any engineer can launch a feature to 1%

    of users Thursday, May 17, 12
  16. Continuous Deployment: ~200 experiments live right now Thursday, May 17,

    12
  17. Metrics driven development: optimize for learning Thursday, May 17, 12

  18. Metrics driven development: measure everything Thursday, May 17, 12

  19. Metrics driven development: getting started: make it easy Thursday, May

    17, 12
  20. Metrics driven development: measure everything: logs, monitoring, A/B, graphs, StatsD

    Thursday, May 17, 12
  21. Metrics driven development: StatsD @ Instagram, Pinterest, Github, Mozilla, LAN.com,

    Zynga, Kickstarter, LivingSocial and 70+ other companies Thursday, May 17, 12
  22. Metrics driven development: StatsD “We love statsd at Instagram. Written

    by Etsy, it’s a network daemon that aggregates and rolls-up data into Graphite. At its core, it has two types of statistics: counter and timers. We use the counters to track everything from number of signups per second to number of likes, and we use timers to time generation of feeds, how long it takes to follow users, and any other major action.” - Mike Krieger, Founder and CTO, Instagram. Thursday, May 17, 12
  23. Metrics driven development: StatsD Open source, a daemon for timing

    and counting things, graphs rates, averages and 95th percentile, 20-30 minute install, easy to use Thursday, May 17, 12
  24. Metrics driven development StatsD::timing("page.render", $msec); Thursday, May 17, 12

  25. Metrics driven development Thursday, May 17, 12

  26. Metrics driven development getting started: your 5 core metrics Thursday,

    May 17, 12
  27. Metrics driven development 5 core metrics @ Etsy: sign ups,

    logins, checkout, new listings, posts in the bugs forums Thursday, May 17, 12
  28. Metrics driven development 378,000 metrics, collected roughly once a second

    Thursday, May 17, 12
  29. Metrics driven development the confidence to change Thursday, May 17,

    12
  30. Adapt, find market fit. Engineering change: Rapid experimentation Thursday, May

    17, 12
  31. Adapt, find market fit, addictive to engineers. Engineering change: Rapid

    experimentation Thursday, May 17, 12
  32. Thank you! Lean more: http://codeascraft.etsy.com Thursday, May 17, 12