Hours Long Deployment to CD

Hours Long Deployment to CD

Every team starts somewhere. There is no one-size-fits-all approach when it comes to deciding when and how code gets shipped. The important thing is that code gets shipped. However, process matters and not all processes are created equally. In this session we will look at the story of how one team went from a very intensive, hours-long deployment process to delivering and deploying an application continuously. This is not your typical story of rainbows and unicorns, this is the in-the-trenches look at the evolution of expectations, processes, and tooling necessary to take the leap into continuous delivery.

0f930e13633535c1c4041e95b8881308?s=128

Jeff Carouth

October 29, 2014
Tweet

Transcript

  1. PRESENTED BY JEFF CAROUTH @jcarouth Hours-Long Deployment to CD

  2. Continuous Deployment is a noun to identify a process which

    allows you to brag to your friends how many times you pushed to production each day.
  3. Make features or parts of features visible to product owners

    constantly.
  4. Push code to production as soon as it passes the

    automated tests.
  5. should you consider continuous delivery?

  6. what about continuous deployment?

  7. Benefits of CD

  8. Benefits of CD Refine ideas and features

  9. Benefits of CD Refine ideas and features Promote and improve

    quality
  10. Benefits of CD Refine ideas and features Promote and improve

    quality Inspire and build confidence
  11. Greatest challenge: managing expectations and business understanding

  12. starting point

  13. None
  14. Source Control |-- branches! | |-- dev1! | `-- dev2!

    |-- mobile! | `-- IOS! |-- tags! | |-- 2011-12-16-Constellation! | `-- 2012-01-03-Cowpens! `-- trunk!
  15. None
  16. Environments Production Production DB Staging Development Dev DB Sandbox-1 Sandbox-2

    Sandbox-3
  17. QA Hammers Staging

  18. Deployment

  19. Deployment dev --> trunk: Cowpens release merge. Revisions: 8249:8252, 8258:8264,

    8267:8268, 8272:8273, 8276:8277, 8279:8294, 8296, 8301, 8305, 8312:8314, 8316:8318, 8321:8326, 8328:8333, 8335, 8338:8352, 8356:8370, 8374:8385, 8389:8398, 8400:8404, 8406:8408, 8410:8412, 8415:8425, 8427:8430
  20. Deployment dev --> trunk: Cowpens release merge. Revisions: 8249:8252, 8258:8264,

    8267:8268, 8272:8273, 8276:8277, 8279:8294, 8296, 8301, 8305, 8312:8314, 8316:8318, 8321:8326, 8328:8333, 8335, 8338:8352, 8356:8370, 8374:8385, 8389:8398, 8400:8404, 8406:8408, 8410:8412, 8415:8425, 8427:8430 ssh production! cd /path/to/application! svn up
  21. boss: Alright, ready to release? dev1: yep, got the merge

    ready. releasing now boss: Let’s do it! … dev1: all done boss: qa time dev2: AIM clear boss: AIM clear boss: haha dev1: PayPal Clear dev3: new query for the report is working WorkChat Deploy Queue Open
  22. We knew we wanted to change, But we didn’t know

    where to start
  23. None
  24. None
  25. None
  26. Push Button Deployment

  27. Environmental Freedom

  28. the move from static development servers to dynamic branch boxes

  29. Environments Production Production DB Staging Development Dev DB Sandbox-1 Sandbox-2

    Sandbox-3
  30. Feature Branches |-- master! `-- develop! |-- feature-3458987! |-- feature-3879236!

  31. What if we deployed each feature branch to an isolated

    remote server instead of using a static box?
  32. None
  33. as a trigger Accept

  34. We Achieved Continuous Delivery

  35. The QA team is a safety net, not a bug

    hunting squad. When they find a bug it’s because we failed.
  36. Knowledge Sharing

  37. RFC @jcarouth Code review enhancements

  38. “You guys did a grape job.”

  39. None
  40. Confidence in Quality

  41. Size of Pull Requests Matters

  42. Thinking about features in smaller chunks led to more accurate

    estimation.
  43. We’re merging small chunks, why don’t we deploy them as

    well?
  44. Feature Toggles URL parameters ?gift_card_purchase=true Database toggles if ($tog->enabled(‘gift_card_purchase’) {!

    // process gift cards! } IP toggles $_SERVER[‘REMOTE_ADDR'] === $allowed_ip
  45. features phases Initiative Phase 1 Phase 2 Phase 3

  46. Before you can deploy continuously you have to take the

    scary out of deployment. https://twitter.com/jcarouth/status/436187762291511296
  47. Lack of Confidence in QA means deployment is scary

  48. Robot Assisted Deployment

  49. None
  50. jeff: QA, are you available to go over PT#554433? qa:

    Yes I can do that jeff: great! jeff: jarvis i’m deploying repo/345 jarvis: jeff, you do not have grapes yet jeff: oops. can someone review that really quick? dev: got it, looks good :thumbsup: WorkChat Deploy Queue Open
  51. jeff: QA, are you available to go over PT#554433? qa:

    Yes I can do that jeff: great! jeff: jarvis i’m deploying repo/345 jarvis: jeff, you do not have grapes yet jeff: oops. can someone review that really quick? dev: got it, looks good :thumbsup: jeff: jarvis i’m deploying repo/345 jarvis: jeff is deploying repo/345 WorkChat Deploy Queue Active - repo/345
  52. the deployment process

  53. the deployment process Do the work

  54. the deployment process Do the work jarvis deploy feature repo/feature-34576

  55. the deployment process Do the work jarvis deploy feature repo/feature-34576!

    Work with PM/QA to verify feature on branch
  56. the deployment process Do the work jarvis deploy feature repo/feature-34576!

    Work with PM/QA to verify feature on branch Open a PR
  57. the deployment process Do the work jarvis deploy feature repo/feature-34576!

    Work with PM/QA to verify feature on branch Open a PR Make any adjustments to the code from code review
  58. the deployment process Do the work jarvis deploy feature repo/feature-34576!

    Work with PM/QA to verify feature on branch Open a PR Make any adjustments to the code from code review jarvis redeploy feature
  59. the deployment process Do the work jarvis deploy feature repo/feature-34576!

    Work with PM/QA to verify feature on branch Open a PR Make any adjustments to the code from code review jarvis redeploy feature! Obtain sign off from PM and code review
  60. the deployment process Do the work jarvis deploy feature repo/feature-34576!

    Work with PM/QA to verify feature on branch Open a PR Make any adjustments to the code from code review jarvis redeploy feature! Obtain sign off from PM and code review jarvis i’m next repo/45
  61. the deployment process Do the work jarvis deploy feature repo/feature-34576!

    Work with PM/QA to verify feature on branch Open a PR Make any adjustments to the code from code review jarvis redeploy feature! Obtain sign off from PM and code review jarvis i’m next repo/45! jarvis i’m deploying repo/45
  62. the deployment process Do the work jarvis deploy feature repo/feature-34576!

    Work with PM/QA to verify feature on branch Open a PR Make any adjustments to the code from code review jarvis redeploy feature! Obtain sign off from PM and code review jarvis i’m next repo/45! jarvis i’m deploying repo/45! Jarvis kicks off the job in Jenkins to deploy the app
  63. the deployment process Do the work jarvis deploy feature repo/feature-34576!

    Work with PM/QA to verify feature on branch Open a PR Make any adjustments to the code from code review jarvis redeploy feature! Obtain sign off from PM and code review jarvis i’m next repo/45! jarvis i’m deploying repo/45! Jarvis kicks off the job in Jenkins to deploy the app Jenkins reports success
  64. the deployment process Do the work jarvis deploy feature repo/feature-34576!

    Work with PM/QA to verify feature on branch Open a PR Make any adjustments to the code from code review jarvis redeploy feature! Obtain sign off from PM and code review jarvis i’m next repo/45! jarvis i’m deploying repo/45! Jarvis kicks off the job in Jenkins to deploy the app Jenkins reports success Merge PR into master
  65. the deployment process Do the work jarvis deploy feature repo/feature-34576!

    Work with PM/QA to verify feature on branch Open a PR Make any adjustments to the code from code review jarvis redeploy feature! Obtain sign off from PM and code review jarvis i’m next repo/45! jarvis i’m deploying repo/45! Jarvis kicks off the job in Jenkins to deploy the app Jenkins reports success Merge PR into master jarvis i’m done deploying
  66. How many Times did you push to production today?

  67. 0 of 8

  68. JEFF CAROUTH LEAD PLATFORM ENGINEER AT LIFTOPIA @jcarouth jcarouth@gmail.com Freenode:

    #phpmentoring @_looselycoupled
  69. Thank You @jcarouth joind.in/12077