Getting started fast: Development infrastructure, CI, and test deployment to a Kubernetes cluster on AWS

When a new software development project is to start, infrastructure is urgently needed. This is even more important when developing a modern system based on microservices. We present, as a viable approach, hosting both development infrastructure (source control, issue tracking, documentation, continuous integration, continuous deployment pipeline) and also a test-bed for the new software itself on a Kubernetes cluster deployed to AWS. All this can be set up in a few hours or even less. We describe the software stack used and some lessons learned.

Transcript

1. Getting started fast Development infrastructure, CI, and test deployment via a Kubernetes cluster on AWS Jörg Müller (@joergm) and Andreas Krüger, September 2017
2. 2 / 47 Typical client situation Pipelines Scheduling Monitoring Pipelines Pipelines Service discovery I want a microservice architecture!
3. 3 / 47 Typical client situation Pipelines Scheduling Monitoring Pipelines Pipelines Service discovery I want a microservice architecture! What about your infrastructure?
4. 4 / 47 Typical client situation I want a microservice architecture! What about your infrastructure? Pipelines Scheduling Monitoring Service discovery
5. 5 / 47 Typical client situation Pipelines Scheduling Monitoring Pipelines Pipelines Service discovery That will take a while...
6. 6 / 47 Quick Start • start development soon • stay closer to later production • cover most steps from Git push to monitoring • remain adaptable
7. 7 / 47 Non Goals • high availability • multi-datacenter • autoscaling • complex security model
8. 8 / 47 Main components
9. 9 / 47 AWS
10. 10 / 47 AWS customer innoQ On my account under my control. What I need when I need it.
11. 11 / 47 Kubernetes
12. 12 / 47 Kubernetes • decides which Docker container to run on which node • container health check • restart container after container or node failure
13. 13 / 47 Kubernetes • in-cluster service discovery • in-cluster load balancing • make load balancer available on external network
14. 14 / 47 Kubernetes • secret management • manage persistent storage (EBS) • group several containers into “POD”, can access same persistent storage
15. 15 / 47 Kubernetes • runs on AWS • runs on other clouds • runs on bare metal
16. 16 / 47 Kubernetes customer innoQ Fits my plans. Prevents lock-in. Our devs know + like it.
17. 17 / 47 kops
18. 18 / 47 kops Kubernetes kops AWS
19. 19 / 47 kops • install Kubernetes on AWS • makes (some) AWS services available to Kubernetes cluster • helps to scale the cluster
20. 20 / 47 kops Installing a Kubernetes cluster on AWS takes about 7 minutes
21. 21 / 47 kops Installing a Kubernetes cluster on AWS takes about 7 minutes after AWS account and route 53 DNS subdomain are both set up, and if the AWS availability zone can provide the nodes you need.
22. 22 / 47 Helm
23. 23 / 47 Helm Kubernetes package- and dependency management
24. 24 / 47 Helm Helm chart POD Docker container POD Docker container POD Docker container Docker container POD Docker container Helm chart
25. 25 / 47 Helm • adds depth and recursion to Kubernetes • provides templating as basis for configuration • configuration possible at any level, top-level overriding deep configuration
26. 26 / 47 Helm • text-based (not smart about change semantics)
27. 27 / 47 Helm • lifecycle hooks
28. 28 / 47 Helm Many Helm charts exist, ready to be used! https://kubeapps.com/
29. 29 / 47 automatic https
30. 30 / 47 Add some annotations dns.alpha.kubernetes.io/external: subdomain.example.com kubernetes.io/tls-acme: 'true'
31. 31 / 47 Lots of background-action
32. 32 / 47 Controller in general
33. 33 / 47 Gitlab
34. 34 / 47 Gitlab as SCM • place to keep your git repos • provides issue system • wiki • pull request process
35. 35 / 47 Gitlab as CI/CD • Gitlab CI • private Docker registry • support for deploying into cluster
36. 36 / 47 Gitlab on Kubernetes • Helm Charts for Core and CI Runners • charts include all dependencies (Dbs …) • Backup to S3
37. 37 / 47 notable stuf of our own
38. 38 / 47 notable stuf of our own • Setup Docker container • Helm charts
39. 39 / 47 “setup” Docker container
40. 40 / 47 “setup” Docker container • All the CLIs installed ready to use (Kops, Kubernetes, Helm, AWS). • A few scripts to automate things.
41. 41 / 47 “setup” Docker container • Can be used by developer on dev PC. • Can be used by Gitlab-CI build.
42. 42 / 47 “setup” Docker container A Python script for our developers' PCs saves them from having to remember + type complicated docker command lines (and saves us from having to document them in detail).
43. 43 / 47 Helm charts
44. 44 / 47 Helm charts • for infrastructure such as monitoring, logging, Gitlab • sample Helm chart to copy and use for a new microservice (there is also sample Gitlab-CI code)
45. 45 / 47 Our overall experience
46. 46 / 47 Our overall experience Having fun building with big bricks.
47. 47 / 47 Thank you! Questions? Jörg Müller, joerg.mueller@innoq.com, @joergm Dr. Andreas Krüger, andreas.krueger@innoq.com