Navigating the Azure Cloud at Digitec Galaxus: Our Journey and Lessons Learned by Olivier Girard & Gerald Schermann

Our story starts in 2015 with our on-prem .NET monolith running on Windows Server, accompanied by a single, clunky MS SQL database, powered by fusion-io disks and many other touchable hardware in a server-room - all built-up with about 5 teams back then. Today, our system is highly distributed and modular, entirely operating in the cloud, and processing significantly more data. It is continuously developed and maintained by over 45 Scrum teams. In our talk, we will delve into our experiences spanning the last 9 years – discussing how we work and organize ourselves, tracing the evolution of our system architecture up to the present, and, of course, exploring the role Azure plays in it all. Spoiler: not everything has always gone according to plan…

About Olivier:
Olivier works as a Domain Architect at Digitec Galaxus since 2018 and helps build the Platform and Online Shop.
https://www.linkedin.com/in/oligirard/

About Gerald:
Gerald works as a Software Architect at Digitec Galaxus. He started in 2019 and works in a platform team.
https://www.linkedin.com/in/gerald-schermann/

Transcript

1. Navigating the Azure Cloud at Digitec Galaxus Our Journey and Lessons Learned Digitec Galaxus AG Olivier Girard | Gerald Schermann 16.04.2024
2. Olivier Girard Domain Architect Platform / Online Shop
3. Gerald Schermann Software Architect Platform & SysOps Arlberg Servus Grüezi 2014
4. Our Cloud Journey 1 - The Past 2 - The Present 3 – The Learnings 0 - About Us
5. What we're about and how we work
6. Innovative Ambitious Responsible Cooperative Piratesque
7. We also run six additional shops in the EU galaxus.ch digitec.ch galaxus.fr galaxus.de galaxus.at galaxus.it galaxus.nl galaxus.be
8. Our Applications
9. 9 Our Cloud Journey – The Past
10. On-prem Hosting Manual Deployment 2015 … digitec.ch/galaxus.ch Revenue 645M Product Catalog 300K Orders 2M 350 300 250 200 150 100 50 2007 2015 2018 2024 Employees Product Development Monolith Layered Architecture ERP OnlineShop DWH Monolith DB TL PO SA Dev 5 Scrum Teams 2015
11. Cloud Hosting (Azure/GCP) Continuous Delivery ... 2018 Monolith Layered Architecture ERP OnlineShop DWH Monolith DB CH + galaxus.de Revenue 970M Product Catalog 2M Orders >4M 350 300 250 200 150 100 50 2007 2015 2018 2024 Employees Product Development TL PO SA Dev > 10 Scrum Teams OnlineShop vNext GraphQL Community Search Microservices 2018
12. Hosting pre-2017 CDN Windows LB VMs Shop/ERP VMs Shop/ERP VMs Shop/ERP VMs Our own hardware … 2017
13. Hosting pre-2017: lift & shift CDN Windows LB VMs Shop/ERP VMs Shop/ERP VMs Shop/ERP VMs Microservices in AKS IIS reverse proxy … 2017
14. Hosting in 2017 CDN Windows LB VMs Shop/ERP VMs Shop/ERP VMs Shop/ERP VMs Microservices in AKS IIS reverse proxy 2017 BLACK FRIDAY
15. Black Friday 2017 2017
16. Hosting in 2017 CDN Windows LB VMs Shop/ERP VMs Shop/ERP VMs Shop/ERP VMs Microservices in AKS IIS reverse proxy BLACK FRIDAY 2017 Redis
17. Hosting in 2018 CDN Windows LB VMs Shop/ERP VMs Shop/ERP VMs Shop/ERP VMs Microservices in AKS Routing Layer AKS in-place upgrades? IIS reverse proxy Microservices in AKS Redis 2018
18. Hosting in 2018 CDN Windows LB VMs Shop/ERP VMs Shop/ERP VMs Shop/ERP VMs Routing Layer IIS reverse proxy Microservices in AKS Redis 2018 Single point of failure? Maybe? Wait until 2022 … the day the routers died
19. Hosting in 2019 / 2020 CDN Shop/ERP VMs Shop/ERP VMs Shop/ERP VMs Routing Layer Microservices in AKS Redis ERP VMs Containerization facts: From .NET Full Framework to .NET Core Replacing all Windows-only dependencies 9 months 2020
20. DevOps – Early Days VCS & Continuous Delivery SRE Team No deployments on Friday! Logging / Monitoring / Alerting Application Insights Prometheus Logs, Alerts, Metrics K8s (Workload) Metrics, Alerts Metric correlation & sampling Scaling issues Grafana 2020
21. More and more Data, and distributed DWH Monolith DB MS SQL (> 2 TB) Search Index Elastic ERP OnlineShop Business Intelligence Community Community Mongo DB Data propagation over Azure Service Bus nightly full-copy 2020
22. Our Cloud Journey – The Present
23. Containers everywhere … 2024 ERP Data Platform Galaxus in 7 countries Revenue 2370M Product Catalog 6M Orders 15M 350 300 250 200 150 100 50 2007 2015 2018 2024 Employees Product Development 35M Sessions/month 200 Page Views/sec >700 Services/DBs in prod >6M LoC TL PO SA Dev > 45 Scrum Teams Online Shop ShopProduct Modular Monolith Microservices (or Microliths) Unstructured Monolith Layered Architecture … Monolith DB 2024
24. Modular Architecture E-Commerce by DG Customer Service Retail Editorial HR … Sales Order Fullfillment Finance Marketplace Sales Category Management Warehousing Purchasing Provider Management Community Marketing Community
25. Data Platform Modular Architecture ERP Online Shop Backend Data Analysis Implementation 2024
26. Modular Architecture Online Shop Frontend Hosting/Ops 2024
27. Hosting CDN Routing Layer Microservices in AKS Redis CPU-bound Memory-bound «No interruptions» «Quarantine» Node Pools + Spot Instances Scale to Zero + Previews Infrastructure as Code Multi-Cluster Management 2024
28. Preview Deployments with Knative 2024 Preview Pod per PR “Regular” Pods based on main Other Services Service calls
29. Preview Deployments with Knative 2024
30. The day the routers died reboot recreation 2022
31. The day the routers died reboot recreation 2022
32. Culture: Post Mortems Document Incident Understand Root Cause Preventive Actions 2024
33. Data (is gold) Data Platform Big Query event-carried state transfer 2024 single writer principle data on the inside data on the outside data mesh
34. Autonomous Teams TL Scrum Team PO SA Dev mission OKR x-functional autonomous customers business 6-10 members 2024
35. We do Dev*Ops
36. Internal Developer Platform TL Feature Team PO SA Dev Platform Database Hosting Dev Tooling Observability Messaging / Kafka Data Platform Delivery (CI/CD) Networking Security TL Platform Team PO Platform Architect DevOps Engineer Data Engineer Data Team PO Data Scientist TL support / enabling / governance customer needs / feedback 2024
37. Dev*Ops and Platform Observability Datadog VCS & Continuous Deployment Security Kyverno Network Policies Managed Identities Security Team 2024
38. Developer Tools Dev Tools & Local Development Developer Portal DB Self-Service & (Cost) Transparency Missing Pieces Feature Team 2024
39. 2024 Developer Tools Dev Tools & Local Development Developer Portal DB Self-Service & (Cost) Transparency Missing Pieces Feature Team
40. 2024 Developer Tools Dev Tools & Local Development Developer Portal DB Self-Service & (Cost) Transparency Missing Pieces Feature Team
41. Developer Tools Dev Tools & Local Development Developer Portal DB Self-Service & (Cost) Transparency Missing Pieces Feature Team 2024
42. Our Cloud Journey – The Learnings
43. Top 3 Lessons Learned not-invented-here vs. managed / industry standard 1 distributed vs. modular 2 one-size fits all vs. specific solution 3 …
44. Next Up in Our Journey de-couple to scale (sagas, streaming) event-driven architecture Simplify and more self-service to increase productivity <DX> system overview / observability, explicit know how and governance holistic system view service mesh, federated governance and security multi/hybrid cloud …
46. galaxus.ch/jobs Olivier Girard Domain Architect – Online Shop / Platform https://www.linkedin.com/in/oligirard/ Gerald Schermann Software Architect – Platform https://www.linkedin.com/in/gerald-schermann/ Tech Blog