Continuous Integration / Delivery with Concourse

PrairieDevCon 2017 talk on "Continuous Integration / Delivery with Concourse"

Transcript

1. USING CONTINUOUS INTEGRATION & DELIVERY
2. ▸Intro ▸CI / CD ▸What is Concourse? ▸Concourse @ AGENDA
3. @markstgodard MVP
4. WE BUILD E-COMMERCE APPS
5. 43k customers 26 apps 45M reqs/day 40+ developers 15 prod deploys/day ~
7. https://www.usatoday.com/story/money/business/2014/11/22/ozy-ibm-google-company-culture/19345427/?showmenu=true
8. https://www.cloudfoundry.org/engineering/
10. CI & CD
11. CONTINUOUS INTEGRATION
12. INTEGRATE CODE OFTEN
13. DETECT FAILURES EARLY
14. AUTOMATE BUILDS
15. CONTINUOUS DELIVERY
16. “CREATE A REPEATABLE, RELIABLE PROCESS FOR RELEASING SOFTWARE” JEZ HUMBLE
17. QUICK, SAFE & SUSTAINABLE
18. CHANGES RELEASABLE SHOULD BE
19. FEEDBACK QUICKER
20. AUTOMATE DEPLOYMENTS
21. AUTOMATE EVERYTHING THAT MAKES SENSE
22. DELIVERY PIPELINE
23. http://concourse.ci
24. ▸ Open Source (Apache 2) ▸ Developed / Sponsored by Pivotal ▸ Written in Go ▸ Run as a standalone binary ▸ Deploy using a cluster manager (bosh, k8s) ▸ https://github.com/Azure/azure-quickstart-templates/ tree/master/concourse-ci ▸ or Vagrant if just want to play around / get started
25. PIPELINE FIRST CLASS THING IS A
26. PIPELINE CODE AS
27. REPEATABLE BUILDS ❄
28. CONTAINER BASED
29. CI / CD PIPELINE
31. RESOURCES JOBS TASKS AND
32. ▸Something that can be checked for new versions ▸Git repo, docker image, s3 bucket, time, semver ▸ Flows through the pipeline RESOURCES
33. Resources
34. RESOURCES
35. RESOURCES
36. RESOURCES
37. ▸What to do when a Resource changes ▸Build Plan of one or more Tasks ▸ Series of tasks with inputs and outputs JOBS
38. Jobs
39. JOBS
40. JOBS
41. JOBS
42. JOBS
43. ▸Smallest unit of work ▸Runs in a container ▸ Inputs and (optional) Outputs ▸Scripts pass / fail based on standard shell exit codes TASKS
44. TASKS
45. TASKS
46. TASKS
47. TASKS
48. TASKS
50. TASKS TASK
51. DEPLOY DEPENDS ON LINT AND UNIT TESTS TO BE PASSING
52. JOB DEPENDENCIES
53. JOB DEPENDENCIES
54. A SIMPLE CI PIPELINE
55. file: pipeline.yml
56. file: pipeline.yml
57. file: pipeline.yml
58. $ fly set-pipeline -p simple -c pipeline.yml
59. DEMO
60. $ vagrant up $ vagrant init concourse/lite https://www.vagrantup.com/
61. https://github.com/markstgodard/ci-simple https://github.com/markstgodard/phpapp
62. DEMO ▸Quick tour of UI ▸Login, Resources, Jobs, Tasks ▸Manually triggering ▸Update the pipeline ▸build docker image, deploy to k8s, smoke tests ▸How are secrets are handled?
63. Concourse @
64. ▸24 CI pipelines ▸Merge Requests ▸Unit Tests, Linting ▸Smoke Tests ▸Blue / Green Deploys to GKE ▸Versioning (semver) ▸Load Testing ▸Slack notifications ▸Information Radiators
65. MERGE REQUESTS
66. Merge Requests
67. Merge Requests
68. Merge Requests
69. ‣ UNIT TESTS ‣ BUILD DOCKER IMAGES ‣ DEPLOY ‣ SMOKE TESTS ‣ SLACK
70. UNIT TESTS & LINTING
71. BUILD DOCKER IMAGES
72. DEPLOY TO TEST
73. AUTOMATED SMOKE TESTS
74. AUTOMATED SMOKE TESTS
75. STAGING
76. #slack
77. BLUE / GREEN DEPLOYMENTS
78. V1.19.2 LB https://checkout.boldapps.net BLUE / GREEN DEPLOYS
79. V1.19.2 LB https://checkout.boldapps.net V1.19.3 BLUE / GREEN DEPLOYS
80. V1.19.2 LB https://checkout.boldapps.net V1.19.3 BLUE / GREEN DEPLOYS
81. LB https://checkout.boldapps.net V1.19.3 BLUE / GREEN DEPLOYS
82. BLUE / GREEN DEPLOYS
84. A MORE COMPLEX CI PIPELINE
85. INFORMATION RADIATORS
86. HIGHLY VISIBLE TEAM AND PASSERS-BYS AT A GLANCE INFO CI PIPELINE AUTOMATED TESTS PROD METRICS
87. MITCH
89. WRAP UP
90. ▸Pipeline focused ▸Repeatable builds ▸Flexible, team-based pipelines ▸Team awareness / Info Radiators ▸Faster delivery ▸Scalable
91. ▸May be slower for some things, since stateless ▸ i.e. npm & composer caches ▸Do you like #!/bin/bash ? ▸Complex pipelines can get gnarly, need to treat like regular code ▸No commercial support ▸But a very active Slack channel & community
92. QUESTIONS?