$30 off During Our Annual Pro Sale. View Details »

DevTools at Etsy

DevTools at Etsy

Overview of the DevTools team at Etsy and how we foster collaboration between Dev and Ops. Presented at DevOpsDays Berlin 2013

Daniel Schauenberg

May 27, 2013
Tweet

More Decks by Daniel Schauenberg

Other Decks in Technology

Transcript

  1. DevTools at Etsy
    Daniel Schauenberg
    [email protected]
    @mrtazz
    Monday, May 27, 13

    View Slide

  2. Monday, May 27, 13

    View Slide

  3. Item by FrankelPhotos
    Monolithic App
    Monday, May 27, 13

    View Slide

  4. 40 deploys/day
    150 committers
    Monday, May 27, 13

    View Slide

  5. ~150 Developers/Designers
    ~15 Ops Engineers
    Monday, May 27, 13

    View Slide

  6. Monday, May 27, 13

    View Slide

  7. Item by JustBeckett
    Monday, May 27, 13

    View Slide

  8. Item by XDcreationz
    Monday, May 27, 13

    View Slide

  9. DevTools
    • Part of the Infrastructure team
    • 5 Engineers
    • SysAdmin and Development background
    • <3 debugging
    Monday, May 27, 13

    View Slide

  10. Monday, May 27, 13

    View Slide

  11. Deployinator
    Monday, May 27, 13

    View Slide

  12. Monday, May 27, 13

    View Slide

  13. "Devtools makes this stuff so
    easy it's almost hilarious, so
    thank you!"
    Laurie Denness, 2013
    Monday, May 27, 13

    View Slide

  14. https://github.com/etsy/deployinator
    Monday, May 27, 13

    View Slide

  15. Developer VMs
    • KVM
    • Dev version of full Etsy stack
    • Chef
    • DevTools
    Monday, May 27, 13

    View Slide

  16. Jenkins Slaves a.k.a. The Bobs
    • LXC containers on buildtests
    • Used for CI and Try
    • Labels for heavy/any execution
    • One heavy executor per disk
    Monday, May 27, 13

    View Slide

  17. Monday, May 27, 13

    View Slide

  18. DevDB Proxy
    • Access production data in a safe way
    • Don't test and develop on outdated data
    • Don't bring the site down by connecting to a production
    server with a GUI db client
    Monday, May 27, 13

    View Slide

  19. StatsD/Logster/Graphite
    deploy lines
    deploy lines
    Monday, May 27, 13

    View Slide

  20. https://github.com/etsy/statsd
    https://github.com/etsy/logster
    https://github.com/etsy/dashboard
    Monday, May 27, 13

    View Slide

  21. Supergrep
    Monday, May 27, 13

    View Slide

  22. https://github.com/etsy/supergrep
    Monday, May 27, 13

    View Slide

  23. Schema Changes
    • Sharded MySQL Master-Master pairs
    • Config file to control which sides to use
    • 4 config pushes
    • Artisanal shell script
    Monday, May 27, 13

    View Slide

  24. Monday, May 27, 13

    View Slide

  25. % review -r dschauenberg
    Monday, May 27, 13

    View Slide

  26. automatically assigned
    automatically assigned
    Monday, May 27, 13

    View Slide

  27. Monday, May 27, 13

    View Slide

  28. The
    “DevOps”
    Team
    Monday, May 27, 13

    View Slide

  29. NOPE
    Monday, May 27, 13

    View Slide

  30. Monday, May 27, 13

    View Slide

  31. export PATH=$PATH:~/development/DevTools
    Monday, May 27, 13

    View Slide

  32. Summary
    • Fostering collaboration through tooling
    • Tools can increase but not force collaboration
    • Caretaker for tooling
    • Hug it out
    Monday, May 27, 13

    View Slide

  33. codeascraft.etsy.com
    www.etsy.com/codeascraft/talks
    etsy.github.com
    www.etsy.com/careers
    Monday, May 27, 13

    View Slide

  34. Questions?
    Monday, May 27, 13

    View Slide

  35. DevTools at Etsy
    Daniel Schauenberg
    [email protected]
    @mrtazz
    Monday, May 27, 13

    View Slide