Leeds DevOps Oct '18 - Accelerating Delivery Velocity With Kubernetes

Transcript

1. Accelerating Delivery Velocity With Kubernetes S i m o n W e s t c o t t @sewestcott
2. Agenda Background + challenges Kubernetes @ SkyBet How is delivery changing? @sewestcott
3. UK’s largest online bookmarker Headquarters in Leeds, UK 1500 people, 700 in tech @sewestcott
4. Bet UK 2015 2018 3 Software Engineering Squads 15 Software Engineering Squads 1 DevOps Squad 0 DevOps Squads Monolith Monolith, Microservices On-premise On-premise, AWS VMs VMs, Containers @sewestcott
5. Growing pains Increasing complexity Unclear ownership Risky to change Release queues Static configuration 30% YoY growth @sewestcott
6. Platform Engineering Outside of the squad, there should be no humans involved between feature and customer @sewestcott
7. • Production pilot • Partnered with engineering squad • Managed monitoring, alerting, logging • Launched Summer ’17 • Prod – 8 services, more in build @sewestcott @
8. • Partnered with engineering squad • 2 Colo DCs • Launched Summer ’18 • Prod – 3 services, more in build @sewestcott @
9. https://github.com/hjacobs/kube-ops-view
10. Has delivery changed?
11. Repeatability
12. Test > Staging > Prod-1 > Prod-2 The cost of “Hello World” Backups DNS Certificates Load Balancing Storage Firewall Operating System Servers Failover Logging Scaling Patching Alerting Monitoring SIEM IDS @sewestcott
13. Repeatability Test > Staging > Prod-1 > Prod-2 helm install <chart> --namespace test --values env/test.yaml helm install <chart> --namespace staging --values env/staging.yaml helm install <chart> --namespace prod-1 --values env/prod-1.yaml helm install <chart> --namespace prod-2 --values env/prod-2.yaml @sewestcott
14. Shifting left Integration, performance testing helm install <chart> --namespace TICKET-1 --values env/integration.yaml helm install <chart> --namespace TICKET-2 --values env/integration.yaml helm install <chart> --namespace TICKET-3 --values env/integration.yaml helm install <chart> --namespace TICKET-4 --values env/integration.yaml @sewestcott
15. Application Design
16. Feed Processer @sewestcott
17. Feed Processer Processer @sewestcott
18. Feed Processer Processer Zoopkeeper @sewestcott
19. Feed Processer Processer Zoopkeeper Zoopkeeper Zoopkeeper @sewestcott
20. Feed Processer Processer Zoopkeeper Zoopkeeper Zoopkeeper @sewestcott
21. Feed Processer apiVersion: apps/v1 kind: Deployment spec: replicas: 1 @sewestcott
22. Monitoring
23. VM VM VM VM VM Stats VM Stats Graphite Static workload config Separate deployment pipeline Alerting @sewestcott
24. Prometheus Dynamic service discovery Global shared configuration @sewestcott
25. Prometheus Operator Orchestrates Prometheus on Kubernetes Scrape targets & rules defined as Kubernetes YAML Deployed with application github.com/coreos/prometheus-operator @sewestcott
26. Prometheus Operator @sewestcott
27. Prometheus Operator @sewestcott
28. Prometheus Operator @sewestcott
29. Operations
30. Squads have operational responsibility for their services ... that includes patching Jenkins is the go to platform Groovy libraries to interface with load balancers, run healthchecks & silence alarms Is this an effective use of engineering time? @sewestcott
31. Kubernetes Deployments Engineering squads responsible for updating containers Managed rollouts and readiness/liveness probes Trust Kubernetes to handle updates @sewestcott
32. Scaling Commission 30 VMs: 1 day Schedule > 150 pods: 3-5 mins @sewestcott
33. Summary
34. Repeatability Declarative configuration aids repeatability “Shift left” testing @sewestcott
35. Reduce Application Complexity Don’t just “lift and shift” applications Cloud Kubernetes native? @sewestcott
36. Monitoring Application and monitoring versioned together Single pipeline to deploy @sewestcott
37. Operations Streamline operational tasks Increased capacity for building customer features @sewestcott
38. Next… Continue migrating apps Focus on DevEx, GitOps Richer infrastructure integration @sewestcott
39. skybetcareers.com / @SkyBetCareers engineering.skybettingandgaming.com Thanks for listening!