Continuous Delivery Pipeline in Mixed Environments, Sergey Gerasimov, Return on Intelligence, CEE-SECR 2017

The practical experience of deploying Web applications in different environments (including cloud-based) using the Continuous Delivery method. An overview of the solution structure and the technologies used: Docker, Rancher, Ansible, Selenium Grid, GoCD, DigitalOcean, GitLab. Problems experienced by Return on Intelligence experts during the development of the solution. A description of the suggested process of the development of an application using skeleton and a discussion about its further development.

Transcript

1. 0 © 2017 Return On Intelligence, Inc. All Rights Reserved Continuous Delivery Pipeline in Mixed Environments Presented By: Sergey Gerasimov
2. 1 © 2017 Return On Intelligence, Inc. All Rights Reserved Agenda 1. Continuous Delivery and its cost 2. Docker to the rescue and its mechanics 3. Rancher for environment orchestration 4. Ansible to reproduce infrastructure 5. Demo of their combination
3. 2 © 2017 Return On Intelligence, Inc. All Rights Reserved Agile Manifesto Our highest priority is to satisfy the customer through early and continuous delivery of valuable software we choose Continuous Delivery process
4. 3 © 2017 Return On Intelligence, Inc. All Rights Reserved non-Continuous Delivery Delivery in the end of iteration? • late feedback • defects found to late • last minute fixes • high risk of change • code freeze • night deployment • whole team meeting Time & Money loss! Your feeling:
5. 4 © 2017 Return On Intelligence, Inc. All Rights Reserved Continuous Delivery If it hurts, do it more often
6. 5 © 2017 Return On Intelligence, Inc. All Rights Reserved Continuous Delivery Goal: Be able to deliver product at least once a day
7. 6 © 2017 Return On Intelligence, Inc. All Rights Reserved Continuous Delivery Pipeline 1. Describe how feature moves from "idea" to "value" 2. Automate it! Build Unit Tests Deploy to Preprod Environment E2E Tests Deploy to Prod Environment Confidence
8. 7 © 2017 Return On Intelligence, Inc. All Rights Reserved Continuous Delivery Requirements? 1. Continuous Integration 2. Automatic testing 3. Enough test coverage 4. Team's expertise 5. Easy to provision production like environments 6. Simple deployment 7. Fast deployment 8. Fault-tolerant infrastructure Our focus
9. 8 © 2017 Return On Intelligence, Inc. All Rights Reserved Early delivery Sprint 0 - investment into future Sprint 0 Sprint 1 Sprint 2 Sprint 3 Sprint 4 Sprint 5 Business Value Build Continuous Delivery process Useless for business Good in long-term
10. 9 © 2017 Return On Intelligence, Inc. All Rights Reserved What are our goals? 1. Provision Test and Production environments (Continuous Delivery) 2. Provision Continuous Delivery infrastructure (Early Delivery) 3. Continuous Delivery pipeline for a sample application Bonus Goal: 4. Ready for Cloud and Microservices
11. 10 © 2017 Return On Intelligence, Inc. All Rights Reserved Node JS 4.8.3 Docker Your IDE Node JS 7.0 Mongo DB App 1 App 2 Processes Executables Conflict! Host OS File system Port 80 Port 27017
12. 11 © 2017 Return On Intelligence, Inc. All Rights Reserved Ubuntu Docker Your IDE Docker platform Debian … … Node JS 7.0 Mongo DB App 1 Containers Node JS 4.8.3 App 2 Images No conflict! Host OS File system Port 80 Port 27017
13. 12 © 2017 Return On Intelligence, Inc. All Rights Reserved Docker Building an image Dockerfile FROM ubuntu RUN apt-get update RUN apt-get install nodejs CMD "/usr/bin/node" Your IDE Docker platform Host OS docker build -t my-node Ubuntu apt-get update apt-get install nodejs my-node
14. 13 © 2017 Return On Intelligence, Inc. All Rights Reserved Docker platform Docker Registry Your IDE Ubuntu Node JS 7.0 hub.docker.com app-1 Host OS Node JS 7.0 app-1 Node JS 4.8.3 run app-1 app-2 run app-2 Ubuntu my-node Node JS 4.8.3 app-2 push my-node my-node
15. 14 © 2017 Return On Intelligence, Inc. All Rights Reserved Docker Multitier / Microservices docker-compose.yml version: '2' services: web: image: my-app:1.0.2 mongo: image: mongo:3.3.12 some-service: image: some-micro- service:2.2 Your IDE Docker platform MongoDB 172.17.0.2 some-service 172.17.0.3 my-app 172.17.0.4 Host OS exec 174.138.96.206 Port 80 Port 27017 Port 3000 Port 80
16. 15 © 2017 Return On Intelligence, Inc. All Rights Reserved Docker What solves & gives? 1. Environments identity 2. Dependencies next to code 3. Efficient use of resources Easy to provision prod like environments Simple deployment Fast deployment
17. 16 © 2017 Return On Intelligence, Inc. All Rights Reserved Docker Continuous Delivery Docker platform Developer's box FROM nodejs ADD /my-code /home/app Dockerfile Application source code my-app MongoDB services: web: image: my-app:1.0.2 ... docker-compose.yml Docker platform Test & Prod Boxes my-app MongoDB Registry Deploy and Test Build and Push Dockerfile Docker-compose.yml Continuous Delivery Boxes
18. 17 © 2017 Return On Intelligence, Inc. All Rights Reserved Rancher Multihost? Docker platform MongoDB 172.17.0.2 Docker platform my-app 172.17.0.4 some-service 172.17.0.3 docker-compose.yml version: '2' services: web: image: my-app:1.0.2 mongo: image: mongo:3.3.12 some-service: image: some-micro- service:2.2 deploy Host OS 174.138.96.206 Host OS 188.166.131.9 Port 80 Port 27017 Port 3000 Port 80
19. 18 © 2017 Return On Intelligence, Inc. All Rights Reserved Rancher Server Agent Agent Agent Agent VPN Tunnel deploy docker-compose.yml … Internet or Local Network REST API UI
20. 19 © 2017 Return On Intelligence, Inc. All Rights Reserved Rancher … … … … … Local Network Internet
21. 20 © 2017 Return On Intelligence, Inc. All Rights Reserved Rancher Selenium Grid GoCD Server Agent Hub FF node IE node Chrome node Internet Prod App Database Some service Load Balancer Preprod App Database Some service Load Balancer Registry Certificate manager HTTPS Volume driver Port 443 Port 443 Port 4444 Port 8153
22. 21 © 2017 Return On Intelligence, Inc. All Rights Reserved Ansible … … … … … Internet
23. 22 © 2017 Return On Intelligence, Inc. All Rights Reserved Ansible Selenium Grid GoCD Server Agent Hub FF node IE node Chrome node Internet Prod App Database Some service Load Balancer Preprod App Database Some service Load Balancer Registry Certificate manager HTTPS Volume driver Port 443 Port 443 Port 4444 Port 8153
24. 23 © 2017 Return On Intelligence, Inc. All Rights Reserved Demo
25. 24 © 2017 Return On Intelligence, Inc. All Rights Reserved Docker What solves & gives? 1. Environments identity 2. Dependencies next to code 3. Efficient use of resources 4. Independent version upgrade/downgrade 5. No technology lockdown 6. Easy-to-make experiments 7. Easy-to-test 8. Feature-rich (volumes, networks, …) 9. Fast project member integration
26. 25 © 2017 Return On Intelligence, Inc. All Rights Reserved Rancher What solves & gives? 1. Mixed Cloud-Native Environment 2. Logical & physical scaling 3. Health-checks and recovery 4. Load balancer 5. Automatic DNS record management 6. Certificate management 7. Monitoring & Logging 8. Container management 9. Easy rollback 10. Blue-green deployments
27. 26 © 2017 Return On Intelligence, Inc. All Rights Reserved Ansible What solves & gives? 1. Reproducible infrastructure 2. Automatic provisioning 3. Version-controlled infrastructure 4. Audit
28. 27 © 2017 Return On Intelligence, Inc. All Rights Reserved Thank You We Appreciate Your Time Questions?
29. 28 © 2017 Return On Intelligence, Inc. All Rights Reserved Contact Us PR@returnonintelligence.com Sergey Gerasimov, Technical Leader