A Cloud Native Journey At Scale (Belgium Kubernetes Meetup)

Enterprises on their Cloud Native journey have 2 options, buy a Kubernetes Platform with all the bells and whistles or build one from scratch. When picking the latter, it allows you to create your own PaaS solution that fits your company's DNA and culture. This talk focuses on how an international financial institution leveraged the cloud and the Cloud Native landscape to build a scalable single-tenant Kubernetes platform that fits its culture and security requirements. Building out a scalable Kubernetes platform to more than 100 Kubernetes clusters comes with a lot of challenges in a big enterprise. During this talk, I will tell the story from where we started to today, the challenges we encountered and how we manage more than 100 Kubernetes clusters in an efficient way.

Transcript

1. jetstack.io Jetstack A Cloud Native Journey At Scale Mattias Gees Director of Tech Cloud Native Bristol
2. jetstack.io Mattias Gees Director of Tech ✉ mattias.gees@jetstack.io 🐦 @MattiasGees
3. jetstack.io Training Strategic Advisory Consulting Workshops and Discovery
4. jetstack.io jetstack.io Agenda
5. jetstack.io • Problem Statement • Decisions • Architecture • Journey • Challenges • Lessons learned
6. jetstack.io Disclaimer: Jetstack is consulting for a global bank, unfortunately I can’t disclose the customer
7. jetstack.io jetstack.io Problem Statement
8. jetstack.io • Sprawl of non-compliant GKE clusters • Meeting compliance of all layers of Kubernetes is hard • Weeks of engineering to create a compliant application on GKE • Maintenance burden on the application teams • Developer experience is bad • Portability between different Kubernetes clusters is non-existent Problem Statement
9. jetstack.io Decisions
10. jetstack.io Single Tenant vs Multi Tenant
11. jetstack.io Self Hosted vs Managed Service
12. jetstack.io Architecture
13. jetstack.io Customer Infrastructure Architecture Region Shared VPC Private VPC Ingress Proxies Compute Engine Egress Proxies Compute Engine Google Kubernetes Engine IAM Key Management Service Cloud Load Balancing Cloud Load Balancing Cloud Firewall Rules Cloud Router Cluster Add On Tooling Customer Applications
14. jetstack.io Cluster Add Ons
15. jetstack.io Cluster Add Ons
16. jetstack.io Cluster Add Ons
17. jetstack.io Cluster Add Ons
18. jetstack.io jetstack.io Journey
19. jetstack.io POC January 2021 - March 2021
20. jetstack.io Provisioning Architecture
21. jetstack.io { "name": "dev", "machine_type": "n1-standard-4", "region": "europ-west2", "egress_service": { "tags": [ "fwtag-external-service1", "fwtag-cloudsql", ] } } Customer Configuration
22. jetstack.io MVP April 2021 - November 2021
23. jetstack.io Provisioning Architecture
24. jetstack.io Productionisation December 2021 - March 2022
25. jetstack.io • Improving observability layer • Documenting Operating Model • Automation of recurring problems • Automation of processes Productionisation
26. jetstack.io Live March 2022 - Now
27. jetstack.io • Implemented testing framework • Improvements to reliability, security and scalability • Overhaul of the RBAC system • Small cost optimizations • Launch extra features ◦ Istio ◦ CSI Secret Store Driver ◦ Enablement of native GKE features Live
28. jetstack.io jetstack.io Challenges
29. jetstack.io • Kubernetes Cluster Resources • Backwards compatibility ◦ CRDs ◦ Images ◦ Changing behaviour of features • Licenses (AGPLv2) • Additional processes of a big enterprise • Processes not adapted to Cloud Native Challenges
30. jetstack.io jetstack.io Lessons Learned
31. jetstack.io • Early feedback from stakeholders was key • Product management helps with prioritizing • Healthy (engineering) culture is everything • Move fast and fix later (until production) • You can still be innovative in a regulated environment • Provide self-service to application teams Lessons Learned
32. jetstack.io Thank you! 🚀 Q&A Mattias Gees Director of Tech Cloud Native Bristol