Upgrade to Pro — share decks privately, control downloads, hide ads and more …

(Semi-)Continuous Delivery at New Relic

(Semi-)Continuous Delivery at New Relic

From the inaugural San Francisco Continuous Delivery Meetup (http://www.meetup.com/Continuous-Delivery-San-Francisco/events/71332792/)

0fd059caf63c87949455fc88a67891f3?s=128

Ralph Bodenner

July 26, 2012
Tweet

Transcript

  1. (Semi-)Continuous Delivery at New Relic Ralph Bodenner Director of Engineering,

    Core App @ralphbod Friday, July 27, 2012
  2. Who, What, Why Friday, July 27, 2012

  3. Dr. Paul MacCready Friday, July 27, 2012

  4. Gossamer Condor Friday, July 27, 2012

  5. Marketing Slide Friday, July 27, 2012

  6. Two Software Companies In One Customer’s environment 2 Web App

    Servers HTTPS (our agent in our customer’s app) Sustained insertion rate 100k per second 24 Core Intel Nehalem 48 GB RAM SAS attached RAID 5 No Virtualization 12 Core Intel Nehalem 48 GB RAM Friday, July 27, 2012
  7. Continuous Delivery • Frequent automated deployments • Monitoring that provides

    immediate actionable feedback • Master branch is always ready to be delivered to customers • Continuous deployment reduces technical risk • Continuous delivery reduces business risk • Testing is not a phase • Done means delivering value to customers Friday, July 27, 2012
  8. Continuous Delivery of On-Premise Friday, July 27, 2012

  9. Regular Heartbeats • Assumption: customers want the most up-to-date agents

    = wrong • Assumption: customers want big changes = wrong • Assumption: customers want predicatability = yes! • Lesson #1: Tailor your continuous delivery to your customers expectations Friday, July 27, 2012
  10. Your First Day at New Relic Friday, July 27, 2012

  11. Weekly Releases Friday, July 27, 2012

  12. Daylight Releases Friday, July 27, 2012

  13. Restructured the Organization • Merged QA into Development • Not

    a popular decision for some • Eliminated “complete, waiting for QA” from vocabulary • Quality Engineers are involved from day one • Lesson #2: Quality is not a separate activity, so don’t have a separate QA department Friday, July 27, 2012
  14. Why Were We Waiting? • Lesson #3: Continuous delivery requires

    changing your social processes (as well as more automation) Friday, July 27, 2012
  15. Deploy Captain Friday, July 27, 2012

  16. Alerts (but...) Friday, July 27, 2012

  17. Drinking Our Own Champagne Friday, July 27, 2012

  18. What More Can We Do? • Architecture changes • Distinguish

    errors • More feature flags • No-code deploys • Longer-running work Friday, July 27, 2012
  19. Now with Services • Growing company leads to bigger teams

    leads to splitting teams • ...leads to services • We were too casual about the split and just “did our side” • Lesson #4: monitor API users like you do customers Friday, July 27, 2012
  20. Now with Services • Growing company leads to bigger teams

    leads to splitting teams • ...leads to services • We were too casual about the split and just “did our side” • Lesson #4: monitor API users like you do customers “[when] doing a big SOA ... monitoring and QA are the same thing” - Steve Yegge, Google, ex-Amazon http://bit.ly/yegge_rant Friday, July 27, 2012
  21. Another Reason • We do Continuous Delivery for a somewhat

    different reason than most companies • Lesson #5: Continuous Delivery isn’t just for the customers Friday, July 27, 2012
  22. Gossamer Penguin Friday, July 27, 2012

  23. (Semi-)Continuous Delivery at New Relic Ralph Bodenner ralph@newrelic.com @ralphbod Friday,

    July 27, 2012