Slide 1

Slide 1 text

(Semi-)Continuous Delivery at New Relic Ralph Bodenner Director of Engineering, Core App @ralphbod Friday, July 27, 2012

Slide 2

Slide 2 text

Who, What, Why Friday, July 27, 2012

Slide 3

Slide 3 text

Dr. Paul MacCready Friday, July 27, 2012

Slide 4

Slide 4 text

Gossamer Condor Friday, July 27, 2012

Slide 5

Slide 5 text

Marketing Slide Friday, July 27, 2012

Slide 6

Slide 6 text

Two Software Companies In One Customer’s environment 2 Web App Servers HTTPS (our agent in our customer’s app) Sustained insertion rate 100k per second 24 Core Intel Nehalem 48 GB RAM SAS attached RAID 5 No Virtualization 12 Core Intel Nehalem 48 GB RAM Friday, July 27, 2012

Slide 7

Slide 7 text

Continuous Delivery • Frequent automated deployments • Monitoring that provides immediate actionable feedback • Master branch is always ready to be delivered to customers • Continuous deployment reduces technical risk • Continuous delivery reduces business risk • Testing is not a phase • Done means delivering value to customers Friday, July 27, 2012

Slide 8

Slide 8 text

Continuous Delivery of On-Premise Friday, July 27, 2012

Slide 9

Slide 9 text

Regular Heartbeats • Assumption: customers want the most up-to-date agents = wrong • Assumption: customers want big changes = wrong • Assumption: customers want predicatability = yes! • Lesson #1: Tailor your continuous delivery to your customers expectations Friday, July 27, 2012

Slide 10

Slide 10 text

Your First Day at New Relic Friday, July 27, 2012

Slide 11

Slide 11 text

Weekly Releases Friday, July 27, 2012

Slide 12

Slide 12 text

Daylight Releases Friday, July 27, 2012

Slide 13

Slide 13 text

Restructured the Organization • Merged QA into Development • Not a popular decision for some • Eliminated “complete, waiting for QA” from vocabulary • Quality Engineers are involved from day one • Lesson #2: Quality is not a separate activity, so don’t have a separate QA department Friday, July 27, 2012

Slide 14

Slide 14 text

Why Were We Waiting? • Lesson #3: Continuous delivery requires changing your social processes (as well as more automation) Friday, July 27, 2012

Slide 15

Slide 15 text

Deploy Captain Friday, July 27, 2012

Slide 16

Slide 16 text

Alerts (but...) Friday, July 27, 2012

Slide 17

Slide 17 text

Drinking Our Own Champagne Friday, July 27, 2012

Slide 18

Slide 18 text

What More Can We Do? • Architecture changes • Distinguish errors • More feature flags • No-code deploys • Longer-running work Friday, July 27, 2012

Slide 19

Slide 19 text

Now with Services • Growing company leads to bigger teams leads to splitting teams • ...leads to services • We were too casual about the split and just “did our side” • Lesson #4: monitor API users like you do customers Friday, July 27, 2012

Slide 20

Slide 20 text

Now with Services • Growing company leads to bigger teams leads to splitting teams • ...leads to services • We were too casual about the split and just “did our side” • Lesson #4: monitor API users like you do customers “[when] doing a big SOA ... monitoring and QA are the same thing” - Steve Yegge, Google, ex-Amazon http://bit.ly/yegge_rant Friday, July 27, 2012

Slide 21

Slide 21 text

Another Reason • We do Continuous Delivery for a somewhat different reason than most companies • Lesson #5: Continuous Delivery isn’t just for the customers Friday, July 27, 2012

Slide 22

Slide 22 text

Gossamer Penguin Friday, July 27, 2012

Slide 23

Slide 23 text

(Semi-)Continuous Delivery at New Relic Ralph Bodenner [email protected] @ralphbod Friday, July 27, 2012