Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Code Review goes Cloud Native - Gerrit on Kubernetes managed by Gardener

894a0a5e2fe91d8ac84af6f292966292?s=47 msohn
November 16, 2018

Code Review goes Cloud Native - Gerrit on Kubernetes managed by Gardener

Gerrit User Summit 2018 Palo Alto on Nov 16, 2018
https://gerrit.googlesource.com/summit/2018/+/master/index.md

894a0a5e2fe91d8ac84af6f292966292?s=128

msohn

November 16, 2018
Tweet

Transcript

  1. Code Review goes Cloud Native Gerrit on Kubernetes managed by

    Gardener
  2. Gerrit Master MySQL Master Fetch Gerrit Master git gc Push

    Git Repositories read write gc
  3. Add Slaves MySQL Master Fetch Gerrit Master git gc Push

    Git Repositories read write gc
  4. Add Slaves MySQL Master Fetch Gerrit Master git gc Replication

    Plugin Push Git Repositories read write gc
  5. Add Slaves MySQL Master Apache-Git- backend Fetch Gerrit Master git

    gc Replication Plugin Push Git Repositories read write gc
  6. Add Slaves MySQL Master Apache-Git- backend Fetch Gerrit Master git

    gc Replication Plugin Push Git Repositories read write gc read
  7. Add Slaves MySQL Master Apache-Git- backend Git Repositories write Fetch

    Gerrit Master git gc Replication Plugin Push Git Repositories read write gc read ssh: create repos
  8. Add Slaves MySQL Master Apache-Git- backend https: git push Git

    Repositories write Fetch Gerrit Master git gc Replication Plugin Push Git Repositories read write gc read ssh: create repos
  9. Add Slaves MySQL Master Apache-Git- backend MySQL Slave https: git

    push Git Repositories write Fetch Gerrit Master git gc Replication Plugin Push Git Repositories read write gc read ssh: create repos
  10. Add Slaves MySQL Master Apache-Git- backend MySQL Slave https: git

    push fetch transaction log Git Repositories write Fetch Gerrit Master git gc Replication Plugin Push Git Repositories read write gc read ssh: create repos
  11. Add Slaves MySQL Master Apache-Git- backend Gerrit Slave MySQL Slave

    https: git push fetch transaction log Git Repositories write Fetch Gerrit Master git gc Replication Plugin Push Git Repositories read write gc read ssh: create repos
  12. Add Slaves MySQL Master Apache-Git- backend Gerrit Slave MySQL Slave

    https: git push fetch transaction log Git Repositories write read Fetch Gerrit Master git gc Replication Plugin Push Git Repositories read write gc read ssh: create repos
  13. Add Slaves MySQL Master Apache-Git- backend Gerrit Slave MySQL Slave

    https: git push fetch transaction log Git Repositories write read Fetch Fetch Gerrit Master git gc Replication Plugin Push Git Repositories read write gc read ssh: create repos
  14. Add Slaves MySQL Master Apache-Git- backend git gc Gerrit Slave

    MySQL Slave https: git push fetch transaction log Git Repositories write gc read Fetch Fetch Gerrit Master git gc Replication Plugin Push Git Repositories read write gc read ssh: create repos
  15. Add Slaves MySQL Master Apache-Git- backend git gc Gerrit Slave

    MySQL Slave https: git push fetch transaction log Git Repositories write gc read Fetch Fetch Gerrit Master git gc Replication Plugin Push Git Repositories read write gc read ssh: create repos
  16. Add Slaves MySQL Master Apache-Git- backend git gc Gerrit Slave

    MySQL Slave https: git push fetch transaction log Git Repositories write gc read Fetch Fetch Gerrit Master git gc Replication Plugin Push Git Repositories read write gc read ssh: create repos
  17. Dockerize Slaves MySQL Master Apache-Git- backend git gc Gerrit Slave

    MySQL Slave ssh: create repos https: git push fetch transaction log Git Repositories write gc read Fetch Fetch Gerrit Master git gc Replication Plugin Push Git Repositories read write gc read
  18. From Containers to Kubernetes

  19. From Containers to Kubernetes VM Host OS Container Runtime Container

  20. From Containers to Kubernetes VM Host OS Container Runtime Container

  21. From Containers to Kubernetes VM Host OS Container Runtime Benefits

    Isolation Immutable infrastructure Portability Faster deployments Versioning Ease of sharing Container
  22. From Containers to Kubernetes VM Host OS Container Runtime Benefits

    Isolation Immutable infrastructure Portability Faster deployments Versioning Ease of sharing Challenges Networking Security Deployments Service Discovery Auto Scaling Persisting Data Logging, Monitoring Access Control Container
  23. From Containers to Kubernetes VM Host OS Container Runtime Benefits

    Isolation Immutable infrastructure Portability Faster deployments Versioning Ease of sharing Challenges Networking Security Deployments Service Discovery Auto Scaling Persisting Data Logging, Monitoring Access Control Container Scheduler Container
  24. From Containers to Kubernetes VM Host OS Container Runtime Benefits

    Isolation Immutable infrastructure Portability Faster deployments Versioning Ease of sharing Challenges Networking Security Deployments Service Discovery Auto Scaling Persisting Data Logging, Monitoring Access Control Container Scheduler Container
  25. From Containers to Kubernetes VM Host OS Container Runtime Benefits

    Isolation Immutable infrastructure Portability Faster deployments Versioning Ease of sharing Challenges Networking Security Deployments Service Discovery Auto Scaling Persisting Data Logging, Monitoring Access Control Container Scheduler Container
  26. From Containers to Kubernetes VM Host OS Container Runtime Benefits

    Isolation Immutable infrastructure Portability Faster deployments Versioning Ease of sharing Challenges Networking Security Deployments Service Discovery Auto Scaling Persisting Data Logging, Monitoring Access Control Container Scheduler Container
  27. From Containers to Kubernetes VM Host OS Container Runtime Benefits

    Isolation Immutable infrastructure Portability Faster deployments Versioning Ease of sharing Challenges Networking Security Deployments Service Discovery Auto Scaling Persisting Data Logging, Monitoring Access Control Kubernetes Orchestration of cluster of containers across multiple hosts • Automatic placements, networking, deployments, scaling, roll-out/-back, A/B testing Container Scheduler Container
  28. From Containers to Kubernetes VM Host OS Container Runtime Benefits

    Isolation Immutable infrastructure Portability Faster deployments Versioning Ease of sharing Challenges Networking Security Deployments Service Discovery Auto Scaling Persisting Data Logging, Monitoring Access Control Kubernetes Orchestration of cluster of containers across multiple hosts • Automatic placements, networking, deployments, scaling, roll-out/-back, A/B testing Declarative – not procedural • Declare target state, reconcile to desired state • Self-healing Container Scheduler Container
  29. From Containers to Kubernetes VM Host OS Container Runtime Benefits

    Isolation Immutable infrastructure Portability Faster deployments Versioning Ease of sharing Challenges Networking Security Deployments Service Discovery Auto Scaling Persisting Data Logging, Monitoring Access Control Kubernetes Orchestration of cluster of containers across multiple hosts • Automatic placements, networking, deployments, scaling, roll-out/-back, A/B testing Docker Workload Portability • Abstract from cloud provider specifics • Multiple container runtimes Declarative – not procedural • Declare target state, reconcile to desired state • Self-healing Container Scheduler Container
  30. Gerrit-Slave Helm Chart MySQL Helm Chart PV Slaves on Kubernetes

    MySQL Master Kubernetes deployment Apache-Git CronJob git gc Persistent Volume Git Repositories Fetch Fetch Gerrit Master git gc Replication Plugin Push Git Repositories read write gc read deployment replica set Gerrit Slave NFS Helm Chart NFS Server MySQL Slave Storage ingress ingress PVC PVC PVC PVC
  31. Gerrit-Slave Helm Chart MySQL Helm Chart PV Kubernetes Cluster deployment

    Apache-Git CronJob git gc Persistent Volume Git Repositories Fetch deployment replica set Gerrit Slave MySQL Slave NFS Volume ingress ingress PVC PVC PVC PVC Gerrit-Master Helm Chart MySQL Helm Chart PV job git gc Persistent Volume Git Repositories deployment replica set Gerrit Master MySQL Master NFS Volume PVC PVC PVC Fetch Push Master-Slave on Kubernetes
  32. Gardener - Kubernetes Clusters as a Service Gardener

  33. Gardener - Kubernetes Clusters as a Service Worker Master Master

    Master Worker Worker HA Gardener Cluster Gardener
  34. Gardener - Kubernetes Clusters as a Service Worker Master Master

    Master Worker Worker HA Gardener Cluster Gardener
  35. Worker Seed Cluster Master Master Master Worker Worker HA Gardener

    - Kubernetes Clusters as a Service Worker Master Master Master Worker Worker HA Gardener Cluster Gardener
  36. Worker Seed Cluster Master Master Master Worker Worker HA Gardener

    - Kubernetes Clusters as a Service Worker Master Master Master Worker Worker HA Gardener Cluster Gardener
  37. Worker Seed Cluster Master Master Master Worker Worker HA Shoot

    Clusters Gardener - Kubernetes Clusters as a Service Worker Master Master Master Worker Worker HA Gardener Cluster Gardener
  38. Worker Seed Cluster Master Master Master Worker Worker HA Shoot

    Clusters Worker Worker Worker Gardener - Kubernetes Clusters as a Service Worker Master Master Master Worker Worker HA Gardener Cluster Gardener apiVersion: garden.sapcloud.io/v1 kind: Shoot metadata: name: my-cluster namespace: garden-project spec: dns: provider: aws-route53 domain: cluster.ondemand.com cloud: aws: networks: vpc: cidr: 10.250.0.0/16 workers: - name: cpu-worker machineType: m4.xlarge autoScalerMin: 5 autoScalerMax: 20 kubernetes: version: 1.11.2 kubeAPIServer: featureGates: ... runtimeConfig: ... admissionPlugins: ... maintenance: timeWindow: begin: 220000+0000 end: 230000+0000 autoUpdate: kubernetesVersion: true status: ... cluster.yaml
  39. Worker Seed Cluster Master Master Master Worker Worker HA Shoot

    Clusters Worker Worker Worker Gardener - Kubernetes Clusters as a Service Worker Master Master Master Worker Worker HA Gardener Cluster Gardener
  40. Worker Seed Cluster Master Master Master Worker Worker HA Shoot

    Clusters Worker Worker Worker Worker Worker Gardener - Kubernetes Clusters as a Service Worker Master Master Master Worker Worker HA Gardener Cluster Gardener
  41. Worker Seed Cluster Master Master Master Worker Worker HA Shoot

    Clusters Worker Worker Worker Worker Worker Worker Gardener - Kubernetes Clusters as a Service Worker Master Master Master Worker Worker HA Gardener Cluster Gardener
  42. Worker Seed Cluster Master Master Master Worker Worker HA Shoot

    Clusters Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Gardener - Kubernetes Clusters as a Service Worker Master Master Master Worker Worker HA Gardener Cluster Gardener
  43. Worker Seed Cluster Master Master Master Worker Worker HA Shoot

    Clusters Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Gardener - Kubernetes Clusters as a Service Worker Master Master Master Worker Worker HA Gardener Cluster Inside a Seed Cluster Worker Gardener Zooming into the Seed Cluster reveals…
  44. Worker Seed Cluster Master Master Master Worker Worker HA Shoot

    Clusters Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Gardener - Kubernetes Clusters as a Service Multiple Shoot Cluster Control Planes Worker Master Master Master Worker Worker HA Gardener Cluster Inside a Seed Cluster Worker manages API Server Scheduler Controller Mgr ETCD Gardener Zooming into the Seed Cluster reveals…
  45. Worker Seed Cluster Master Master Master Worker Worker HA Shoot

    Clusters Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Gardener - Kubernetes Clusters as a Service Multiple Shoot Cluster Control Planes Worker Master Master Master Worker Worker HA Gardener Cluster Inside a Seed Cluster Worker manages API Server ETCD Scheduler Controller Mgr API Server Scheduler Controller Mgr ETCD Gardener Zooming into the Seed Cluster reveals…
  46. Worker Seed Cluster Master Master Master Worker Worker HA Shoot

    Clusters Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Gardener - Kubernetes Clusters as a Service Multiple Shoot Cluster Control Planes Worker Master Master Master Worker Worker HA Gardener Cluster Inside a Seed Cluster Worker manages API Server ETCD Scheduler Controller Mgr API Server Scheduler Controller Mgr ETCD ETCD … API Server Gardener Zooming into the Seed Cluster reveals…
  47. Worker Seed Cluster Master Master Master Worker Worker HA Shoot

    Clusters Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Gardener Machine Controller Manager Machine Provisioning Self-Healing Auto-Update Auto-Scaling Gardener - Kubernetes Clusters as a Service Multiple Shoot Cluster Control Planes Worker Master Master Master Worker Worker HA Gardener Cluster Inside a Seed Cluster Worker manages API Server ETCD Scheduler Controller Mgr API Server Scheduler Controller Mgr ETCD ETCD … API Server Gardener Zooming into the Seed Cluster reveals…
  48. Worker Seed Cluster Master Master Master Worker Worker HA Shoot

    Clusters Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Gardener Machine Controller Manager Machine Provisioning Self-Healing Auto-Update Auto-Scaling Gardener - Kubernetes Clusters as a Service Multiple Shoot Cluster Control Planes Worker Master Master Master Worker Worker HA Gardener Cluster Inside a Seed Cluster Worker manages API Server ETCD Scheduler Controller Mgr API Server Scheduler Controller Mgr ETCD ETCD … API Server Gardener Zooming into the Seed Cluster reveals…
  49. None
  50. Demo Gerrit on Kubernetes managed by Gardener

  51. Next steps Storage • use Ceph / Rook to replace

    NFS Update chart to enable using master branch • get rid of database
  52. Index Replace Lucene by ElasticSearch

  53. Gerrit Operator based on operator framework to implement • Sharding

    • Multi-master • Auto-Scaling
  54. Logging EFK-Stack • Elastic search => Indexing • Fluentbit =>

    Log-collection • Kibana => Visualization
  55. Monitoring Prometheus => data collection Grafana => visualization

  56. Contributions welcome ! Picture Link

  57. Links Git Repository https://gerrit.googlesource.com/k8s-gerrit Docker Images https://hub.docker.com/u/k8sgerrit Gardener https://gardener.cloud