Code Review goes Cloud Native - Gerrit on Kubernetes managed by Gardener

Gerrit User Summit 2018 Palo Alto on Nov 16, 2018
https://gerrit.googlesource.com/summit/2018/+/master/index.md

Transcript

1. Code Review goes Cloud Native Gerrit on Kubernetes managed by Gardener
2. Gerrit Master MySQL Master Fetch Gerrit Master git gc Push Git Repositories read write gc
3. Add Slaves MySQL Master Fetch Gerrit Master git gc Push Git Repositories read write gc
4. Add Slaves MySQL Master Fetch Gerrit Master git gc Replication Plugin Push Git Repositories read write gc
5. Add Slaves MySQL Master Apache-Git- backend Fetch Gerrit Master git gc Replication Plugin Push Git Repositories read write gc
6. Add Slaves MySQL Master Apache-Git- backend Fetch Gerrit Master git gc Replication Plugin Push Git Repositories read write gc read
7. Add Slaves MySQL Master Apache-Git- backend Git Repositories write Fetch Gerrit Master git gc Replication Plugin Push Git Repositories read write gc read ssh: create repos
8. Add Slaves MySQL Master Apache-Git- backend https: git push Git Repositories write Fetch Gerrit Master git gc Replication Plugin Push Git Repositories read write gc read ssh: create repos
9. Add Slaves MySQL Master Apache-Git- backend MySQL Slave https: git push Git Repositories write Fetch Gerrit Master git gc Replication Plugin Push Git Repositories read write gc read ssh: create repos
10. Add Slaves MySQL Master Apache-Git- backend MySQL Slave https: git push fetch transaction log Git Repositories write Fetch Gerrit Master git gc Replication Plugin Push Git Repositories read write gc read ssh: create repos
11. Add Slaves MySQL Master Apache-Git- backend Gerrit Slave MySQL Slave https: git push fetch transaction log Git Repositories write Fetch Gerrit Master git gc Replication Plugin Push Git Repositories read write gc read ssh: create repos
12. Add Slaves MySQL Master Apache-Git- backend Gerrit Slave MySQL Slave https: git push fetch transaction log Git Repositories write read Fetch Gerrit Master git gc Replication Plugin Push Git Repositories read write gc read ssh: create repos
13. Add Slaves MySQL Master Apache-Git- backend Gerrit Slave MySQL Slave https: git push fetch transaction log Git Repositories write read Fetch Fetch Gerrit Master git gc Replication Plugin Push Git Repositories read write gc read ssh: create repos
14. Add Slaves MySQL Master Apache-Git- backend git gc Gerrit Slave MySQL Slave https: git push fetch transaction log Git Repositories write gc read Fetch Fetch Gerrit Master git gc Replication Plugin Push Git Repositories read write gc read ssh: create repos
15. Add Slaves MySQL Master Apache-Git- backend git gc Gerrit Slave MySQL Slave https: git push fetch transaction log Git Repositories write gc read Fetch Fetch Gerrit Master git gc Replication Plugin Push Git Repositories read write gc read ssh: create repos
16. Add Slaves MySQL Master Apache-Git- backend git gc Gerrit Slave MySQL Slave https: git push fetch transaction log Git Repositories write gc read Fetch Fetch Gerrit Master git gc Replication Plugin Push Git Repositories read write gc read ssh: create repos
17. Dockerize Slaves MySQL Master Apache-Git- backend git gc Gerrit Slave MySQL Slave ssh: create repos https: git push fetch transaction log Git Repositories write gc read Fetch Fetch Gerrit Master git gc Replication Plugin Push Git Repositories read write gc read
18. From Containers to Kubernetes
19. From Containers to Kubernetes VM Host OS Container Runtime Container
20. From Containers to Kubernetes VM Host OS Container Runtime Container
21. From Containers to Kubernetes VM Host OS Container Runtime Benefits Isolation Immutable infrastructure Portability Faster deployments Versioning Ease of sharing Container
22. From Containers to Kubernetes VM Host OS Container Runtime Benefits Isolation Immutable infrastructure Portability Faster deployments Versioning Ease of sharing Challenges Networking Security Deployments Service Discovery Auto Scaling Persisting Data Logging, Monitoring Access Control Container
23. From Containers to Kubernetes VM Host OS Container Runtime Benefits Isolation Immutable infrastructure Portability Faster deployments Versioning Ease of sharing Challenges Networking Security Deployments Service Discovery Auto Scaling Persisting Data Logging, Monitoring Access Control Container Scheduler Container
24. From Containers to Kubernetes VM Host OS Container Runtime Benefits Isolation Immutable infrastructure Portability Faster deployments Versioning Ease of sharing Challenges Networking Security Deployments Service Discovery Auto Scaling Persisting Data Logging, Monitoring Access Control Container Scheduler Container
25. From Containers to Kubernetes VM Host OS Container Runtime Benefits Isolation Immutable infrastructure Portability Faster deployments Versioning Ease of sharing Challenges Networking Security Deployments Service Discovery Auto Scaling Persisting Data Logging, Monitoring Access Control Container Scheduler Container
26. From Containers to Kubernetes VM Host OS Container Runtime Benefits Isolation Immutable infrastructure Portability Faster deployments Versioning Ease of sharing Challenges Networking Security Deployments Service Discovery Auto Scaling Persisting Data Logging, Monitoring Access Control Container Scheduler Container
27. From Containers to Kubernetes VM Host OS Container Runtime Benefits Isolation Immutable infrastructure Portability Faster deployments Versioning Ease of sharing Challenges Networking Security Deployments Service Discovery Auto Scaling Persisting Data Logging, Monitoring Access Control Kubernetes Orchestration of cluster of containers across multiple hosts • Automatic placements, networking, deployments, scaling, roll-out/-back, A/B testing Container Scheduler Container
28. From Containers to Kubernetes VM Host OS Container Runtime Benefits Isolation Immutable infrastructure Portability Faster deployments Versioning Ease of sharing Challenges Networking Security Deployments Service Discovery Auto Scaling Persisting Data Logging, Monitoring Access Control Kubernetes Orchestration of cluster of containers across multiple hosts • Automatic placements, networking, deployments, scaling, roll-out/-back, A/B testing Declarative – not procedural • Declare target state, reconcile to desired state • Self-healing Container Scheduler Container
29. From Containers to Kubernetes VM Host OS Container Runtime Benefits Isolation Immutable infrastructure Portability Faster deployments Versioning Ease of sharing Challenges Networking Security Deployments Service Discovery Auto Scaling Persisting Data Logging, Monitoring Access Control Kubernetes Orchestration of cluster of containers across multiple hosts • Automatic placements, networking, deployments, scaling, roll-out/-back, A/B testing Docker Workload Portability • Abstract from cloud provider specifics • Multiple container runtimes Declarative – not procedural • Declare target state, reconcile to desired state • Self-healing Container Scheduler Container
30. Gerrit-Slave Helm Chart MySQL Helm Chart PV Slaves on Kubernetes MySQL Master Kubernetes deployment Apache-Git CronJob git gc Persistent Volume Git Repositories Fetch Fetch Gerrit Master git gc Replication Plugin Push Git Repositories read write gc read deployment replica set Gerrit Slave NFS Helm Chart NFS Server MySQL Slave Storage ingress ingress PVC PVC PVC PVC
31. Gerrit-Slave Helm Chart MySQL Helm Chart PV Kubernetes Cluster deployment Apache-Git CronJob git gc Persistent Volume Git Repositories Fetch deployment replica set Gerrit Slave MySQL Slave NFS Volume ingress ingress PVC PVC PVC PVC Gerrit-Master Helm Chart MySQL Helm Chart PV job git gc Persistent Volume Git Repositories deployment replica set Gerrit Master MySQL Master NFS Volume PVC PVC PVC Fetch Push Master-Slave on Kubernetes
32. Gardener - Kubernetes Clusters as a Service Gardener
33. Gardener - Kubernetes Clusters as a Service Worker Master Master Master Worker Worker HA Gardener Cluster Gardener
34. Gardener - Kubernetes Clusters as a Service Worker Master Master Master Worker Worker HA Gardener Cluster Gardener
35. Worker Seed Cluster Master Master Master Worker Worker HA Gardener - Kubernetes Clusters as a Service Worker Master Master Master Worker Worker HA Gardener Cluster Gardener
36. Worker Seed Cluster Master Master Master Worker Worker HA Gardener - Kubernetes Clusters as a Service Worker Master Master Master Worker Worker HA Gardener Cluster Gardener
37. Worker Seed Cluster Master Master Master Worker Worker HA Shoot Clusters Gardener - Kubernetes Clusters as a Service Worker Master Master Master Worker Worker HA Gardener Cluster Gardener
38. Worker Seed Cluster Master Master Master Worker Worker HA Shoot Clusters Worker Worker Worker Gardener - Kubernetes Clusters as a Service Worker Master Master Master Worker Worker HA Gardener Cluster Gardener apiVersion: garden.sapcloud.io/v1 kind: Shoot metadata: name: my-cluster namespace: garden-project spec: dns: provider: aws-route53 domain: cluster.ondemand.com cloud: aws: networks: vpc: cidr: 10.250.0.0/16 workers: - name: cpu-worker machineType: m4.xlarge autoScalerMin: 5 autoScalerMax: 20 kubernetes: version: 1.11.2 kubeAPIServer: featureGates: ... runtimeConfig: ... admissionPlugins: ... maintenance: timeWindow: begin: 220000+0000 end: 230000+0000 autoUpdate: kubernetesVersion: true status: ... cluster.yaml
39. Worker Seed Cluster Master Master Master Worker Worker HA Shoot Clusters Worker Worker Worker Gardener - Kubernetes Clusters as a Service Worker Master Master Master Worker Worker HA Gardener Cluster Gardener
40. Worker Seed Cluster Master Master Master Worker Worker HA Shoot Clusters Worker Worker Worker Worker Worker Gardener - Kubernetes Clusters as a Service Worker Master Master Master Worker Worker HA Gardener Cluster Gardener
41. Worker Seed Cluster Master Master Master Worker Worker HA Shoot Clusters Worker Worker Worker Worker Worker Worker Gardener - Kubernetes Clusters as a Service Worker Master Master Master Worker Worker HA Gardener Cluster Gardener
42. Worker Seed Cluster Master Master Master Worker Worker HA Shoot Clusters Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Gardener - Kubernetes Clusters as a Service Worker Master Master Master Worker Worker HA Gardener Cluster Gardener
43. Worker Seed Cluster Master Master Master Worker Worker HA Shoot Clusters Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Gardener - Kubernetes Clusters as a Service Worker Master Master Master Worker Worker HA Gardener Cluster Inside a Seed Cluster Worker Gardener Zooming into the Seed Cluster reveals…
44. Worker Seed Cluster Master Master Master Worker Worker HA Shoot Clusters Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Gardener - Kubernetes Clusters as a Service Multiple Shoot Cluster Control Planes Worker Master Master Master Worker Worker HA Gardener Cluster Inside a Seed Cluster Worker manages API Server Scheduler Controller Mgr ETCD Gardener Zooming into the Seed Cluster reveals…
45. Worker Seed Cluster Master Master Master Worker Worker HA Shoot Clusters Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Gardener - Kubernetes Clusters as a Service Multiple Shoot Cluster Control Planes Worker Master Master Master Worker Worker HA Gardener Cluster Inside a Seed Cluster Worker manages API Server ETCD Scheduler Controller Mgr API Server Scheduler Controller Mgr ETCD Gardener Zooming into the Seed Cluster reveals…
46. Worker Seed Cluster Master Master Master Worker Worker HA Shoot Clusters Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Gardener - Kubernetes Clusters as a Service Multiple Shoot Cluster Control Planes Worker Master Master Master Worker Worker HA Gardener Cluster Inside a Seed Cluster Worker manages API Server ETCD Scheduler Controller Mgr API Server Scheduler Controller Mgr ETCD ETCD … API Server Gardener Zooming into the Seed Cluster reveals…
47. Worker Seed Cluster Master Master Master Worker Worker HA Shoot Clusters Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Gardener Machine Controller Manager Machine Provisioning Self-Healing Auto-Update Auto-Scaling Gardener - Kubernetes Clusters as a Service Multiple Shoot Cluster Control Planes Worker Master Master Master Worker Worker HA Gardener Cluster Inside a Seed Cluster Worker manages API Server ETCD Scheduler Controller Mgr API Server Scheduler Controller Mgr ETCD ETCD … API Server Gardener Zooming into the Seed Cluster reveals…
48. Worker Seed Cluster Master Master Master Worker Worker HA Shoot Clusters Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Gardener Machine Controller Manager Machine Provisioning Self-Healing Auto-Update Auto-Scaling Gardener - Kubernetes Clusters as a Service Multiple Shoot Cluster Control Planes Worker Master Master Master Worker Worker HA Gardener Cluster Inside a Seed Cluster Worker manages API Server ETCD Scheduler Controller Mgr API Server Scheduler Controller Mgr ETCD ETCD … API Server Gardener Zooming into the Seed Cluster reveals…
50. Demo Gerrit on Kubernetes managed by Gardener
51. Next steps Storage • use Ceph / Rook to replace NFS Update chart to enable using master branch • get rid of database
52. Index Replace Lucene by ElasticSearch
53. Gerrit Operator based on operator framework to implement • Sharding • Multi-master • Auto-Scaling
54. Logging EFK-Stack • Elastic search => Indexing • Fluentbit => Log-collection • Kibana => Visualization
55. Monitoring Prometheus => data collection Grafana => visualization
56. Contributions welcome ! Picture Link
57. Links Git Repository https://gerrit.googlesource.com/k8s-gerrit Docker Images https://hub.docker.com/u/k8sgerrit Gardener https://gardener.cloud