(Semi-)Continuous Delivery at New Relic Ralph Bodenner Director of Engineering, Core App @ralphbod Friday, July 27, 2012

Who, What, Why Friday, July 27, 2012

Dr. Paul MacCready Friday, July 27, 2012

Gossamer Condor Friday, July 27, 2012

Marketing Slide Friday, July 27, 2012

Two Software Companies In One Customer’s environment 2 Web App Servers HTTPS (our agent in our customer’s app) Sustained insertion rate 100k per second 24 Core Intel Nehalem 48 GB RAM SAS attached RAID 5 No Virtualization 12 Core Intel Nehalem 48 GB RAM Friday, July 27, 2012

Continuous Delivery • Frequent automated deployments • Monitoring that provides immediate actionable feedback • Master branch is always ready to be delivered to customers • Continuous deployment reduces technical risk • Continuous delivery reduces business risk • Testing is not a phase • Done means delivering value to customers Friday, July 27, 2012

Continuous Delivery of On-Premise Friday, July 27, 2012

Regular Heartbeats • Assumption: customers want the most up-to-date agents = wrong • Assumption: customers want big changes = wrong • Assumption: customers want predicatability = yes! • Lesson #1: Tailor your continuous delivery to your customers expectations Friday, July 27, 2012

Your First Day at New Relic Friday, July 27, 2012

Weekly Releases Friday, July 27, 2012

Daylight Releases Friday, July 27, 2012

Restructured the Organization • Merged QA into Development • Not a popular decision for some • Eliminated “complete, waiting for QA” from vocabulary • Quality Engineers are involved from day one • Lesson #2: Quality is not a separate activity, so don’t have a separate QA department Friday, July 27, 2012

Why Were We Waiting? • Lesson #3: Continuous delivery requires changing your social processes (as well as more automation) Friday, July 27, 2012

Deploy Captain Friday, July 27, 2012

Alerts (but...) Friday, July 27, 2012

Drinking Our Own Champagne Friday, July 27, 2012

What More Can We Do? • Architecture changes • Distinguish errors • More feature flags • No-code deploys • Longer-running work Friday, July 27, 2012

Now with Services • Growing company leads to bigger teams leads to splitting teams • ...leads to services • We were too casual about the split and just “did our side” • Lesson #4: monitor API users like you do customers Friday, July 27, 2012

Now with Services • Growing company leads to bigger teams leads to splitting teams • ...leads to services • We were too casual about the split and just “did our side” • Lesson #4: monitor API users like you do customers “[when] doing a big SOA ... monitoring and QA are the same thing” - Steve Yegge, Google, ex-Amazon http://bit.ly/yegge_rant Friday, July 27, 2012

Another Reason • We do Continuous Delivery for a somewhat different reason than most companies • Lesson #5: Continuous Delivery isn’t just for the customers Friday, July 27, 2012

Gossamer Penguin Friday, July 27, 2012

(Semi-)Continuous Delivery at New Relic Ralph Bodenner ralph@newrelic.com @ralphbod Friday, July 27, 2012