Future of Deployment - GoRuCo 2012

Slide 1

Slide 1 text

FUTURE OF DEPLOYMENT Dr Nic Williams @drnic | drnicwilliams.com VP Dev Evangelism, Engine Yard

Slide 2

Slide 2 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC Hardware production virtualisation operating system services app code networking & dns

Slide 3

Slide 3 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC Hardware serving traffic virtualisation operating system services app code networking & dns http

Slide 4

Slide 4 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC code deployment services app code app code Hardware virtualisation operating system networking & dns

Slide 5

Slide 5 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC Month 1 Month 2 to 60

Slide 6

Slide 6 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC services upgrade app code services services Hardware virtualisation operating system networking & dns

Slide 7

Slide 7 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC os os kernel upgrade app code Hardware virtualisation networking & dns services

Slide 8

Slide 8 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC services scaling app code Hardware vm operating system networking & dns vm vm

Slide 9

Slide 9 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC virtualisation services disaster recovery app code operating system hW HW ip & DNS ip & DNS

Slide 10

Slide 10 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC soa services app code Hardware virtualisation operating system networking & dns app code app code

Slide 11

Slide 11 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC your stack • Different teams • Different tools • Different policies • Different metadata

Slide 12

Slide 12 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC deployment, upgrade, orchestration tools • Puppet, chef, vagrant, brew2deb, bosh, capistrano, git, fog, razor, gentoo, arch-linux, debian, openstack, cloudstack, vsphere, jenkins, powerdns...

Slide 13

Slide 13 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC Hardware virtualisation operating system services app code networking & dns razor cloudstack; xen server ubuntu puppet capistrano cloudstack; powerdns

Slide 14

Slide 14 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC Hardware virtualisation operating system services app code networking & dns ? aws; xen gentoo chef solo ey deploy AWS; DIY

Slide 15

Slide 15 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC Hardware virtualisation operating system services app code networking & dns ? aws, xen, LXC ubuntu cedar, buildpacks git push shared proxy; addon

Slide 16

Slide 16 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC .../releases/TIMESTAMP .../current /usr/bin/nginx /etc/nginx/nginx.conf /usr/bin/ruby code upgrades service upgrades

Slide 17

Slide 17 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC Our tools shape our thinking

Slide 18

Slide 18 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC BIG QUESTIONS

Slide 19

Slide 19 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC Month 1 Month 2 to 60

Slide 20

Slide 20 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC ci, staging production

Slide 21

Slide 21 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC AB test

Slide 22

Slide 22 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC production development

Slide 23

Slide 23 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC production exception

Slide 24

Slide 24 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC production enterprise deploys

Slide 25

Slide 25 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC big questions • Staging/CI into Production • A/B testing • Production into Development • Problem diagnosis • Enterprise deployments

Slide 26

Slide 26 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC PROPOSAL Architecture of the Future

Slide 27

Slide 27 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC requirements • Explicit description of everything • Everything upgradable • Everything change in history • Central API for all activities • Alternate UX via API

Slide 28

Slide 28 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC system deployment

Slide 29

Slide 29 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC Nice to have • Independent install paths • Isolation & controls for services • Portable enterprise deployments

Slide 30

Slide 30 text

YOUR COMPANY NAME | LONG AND INTERESTING PRESENTATION TITLE | VERSION NO. XX | 06 /02/2012 Hardware virtualisation operating system services app code networking & dns razor cloudstack; xen server gentoo chef capistrano powerdns one system, many UI

Slide 31

Slide 31 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC IDEAS What the “one tool” should do?

Slide 32

Slide 32 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC explicit source code • Your app source code • Nginx source & extns • PostgreSQL binary nginx src app src postgres src

Slide 33

Slide 33 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC nginx src app src postgres src nginx pkg app pkg postgres pkg package everything

Slide 34

Slide 34 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC nginx pkg app pkg postgres pkg nginx bin app binary postgres bin binary artifacts

Slide 35

Slide 35 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC system history • Deployments & upgrades • Scaling • Attribute/property changes • Monitoring • Who? When? • Recreate historical systems

Slide 36

Slide 36 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC orchestrated upgrades • Safely install new packages • Migrations • Process restarts • Early failure detection & recorvery

Slide 37

Slide 37 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC portable deployments • One public production deployment • 999 private production deployment

Slide 38

Slide 38 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC EXAMPLE System Deployment

Slide 39

Slide 39 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC this box 1.0 stemcell 1 vm 0 or 1 attached disk Assigned IP packages installed jobs running

Slide 40

Slide 40 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC orchestrated changes 1.0 1.0 From...

Slide 41

Slide 41 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC infrastructure 1.0 1.0 From... to... 1.0 1.0 1.0 1.0

Slide 42

Slide 42 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC 1.0 1.0 1.0 1.0 deployments

Slide 43

Slide 43 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC 1.0 1.0 1.0 1.0 2.0 deployments

Slide 44

Slide 44 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC 1.0 1.0 1.0 1.0 2.0 2.0 2.0 2.0 deployments

Slide 45

Slide 45 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC 2.0 2.0 2.0 2.0 2.0 2.0 add more 2.0 2.0 2.0

Slide 46

Slide 46 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC 2.0 3.0 2.0 2.0 2.0 2.0 2.0 small batches 2.0 2.0 2.0

Slide 47

Slide 47 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC 2.0 3.0 2.0 2.0 2.0 2.0 2.0 small batches 2.0 2.0 2.0 3.0 3.0 3.0 3.0

Slide 48

Slide 48 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC 2.0 3.0 2.0 2.0 2.0 2.0 2.0 small batches 2.0 2.0 2.0 3.0 3.0 3.0 3.0 3.0 3.0 3.0 3.0

Slide 49

Slide 49 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC AVAILABLE? Does such a thing exist?

Slide 50

Slide 50 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC build yourself http://www.slideshare.net/AnthonyShortland/dto-chefconf2012

Slide 51

Slide 51 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC bosh • release and lifecycle mgmt • stemcells, releases, manifests • github:cloudfoundry/bosh • github:drnic/bosh-getting-started

Slide 52

Slide 52 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC enterprise deploys • Interesting to you? • drnic@engineyard.com • See me at the yacht!

Slide 53

Slide 53 text

DR NIC WILLIAMS | ENGINE YARD | FUTURE OF DEPLOYMENT | GORUCO 2012, NYC Our tools shape our thinking