Scaling sucks

Scaling an application or an organization generally involves exchanging developer productivity for scale. Here are some ideas about how to make those tradeoffs less painful.

Transcript

1. Scaling Sucks
   (What I learned in a year about 9 months at Etsy)
   @rafeco
   Rafe Colburn
   Friday, October 5, 12
   

   View Slide

2. What’s Etsy
   • “Ebay for twee people” – The Brooklyn
   Paper
   • That place where you buy handmade stuff
   • “A tool to help people create small
   businesses.” (me)
   Friday, October 5, 12
   

   View Slide

3. We’ve already passed the 2011 total in 2012
   Friday, October 5, 12
   

   View Slide

4. August 2012 Numbers
   • 3.8 million items sold
   • 2.4 million new listings
   • 1.42 billion page views
   Friday, October 5, 12
   

   View Slide

5. Friday, October 5, 12
   

   View Slide

6. (What I learned in a year about 9 months at Etsy)
   Continuous deployment
   Friday, October 5, 12
   

   View Slide

7. Graph everything
   Friday, October 5, 12
   

   View Slide

8. Branching in code
   $server_config['foo'] = array(
   'enabled' => array(
   'blue_background' => 20,
   'orange_background' => 20,
   'pink_background' => 20,
   ),
   );
   Friday, October 5, 12
   

   View Slide

9. Etsy is not cloud-
   based
   We’re on the
   LAMP stack
   Friday, October 5, 12
   

   View Slide

10. What I’ve learned at
    Etsy
    Friday, October 5, 12
    

    View Slide

11. Scaling sucks
    (But there are ways to mitigate the suckage)
    Friday, October 5, 12
    

    View Slide

12. Your code will be
    rewritten, just not all
    at once
    Friday, October 5, 12
    

    View Slide

13. You’ll use all of the
    databases
    Friday, October 5, 12
    

    View Slide

14. Data plumbing is a
    black hole
    Friday, October 5, 12
    

    View Slide

15. There is a place for
    heretical views on
    services
    Friday, October 5, 12
    

    View Slide

16. We define
    production service
    broadly
    Friday, October 5, 12
    

    View Slide

17. The “Why Wasn’t I
    Consulted” list
    Security, ops, analytics, internal apps, international ...
    Friday, October 5, 12
    

    View Slide

18. Procrastination is OK
    (sometimes)
    Friday, October 5, 12
    

    View Slide

19. The trap: scaling is an
    interesting problem
    Friday, October 5, 12
    

    View Slide

20. Programming is to
    so!ware engineering
    as sex is to safe sex
    Friday, October 5, 12
    

    View Slide

21. Continuous
    deployment cures a
    lot of ills
    (If you can make it work.)
    Friday, October 5, 12
    

    View Slide

22. Scaling the
    organization is the
    toughest challenge
    Friday, October 5, 12
    

    View Slide

23. “Naturalistic
    programming”
    Friday, October 5, 12
    

    View Slide

24. Thank you!
    @rafeco
    Friday, October 5, 12
    

    View Slide