MARTIN HELMICH @mittwald HOW TO BUILD YOUR OWN CLOUD PLATFORM August 3rd, 2024 

MARTIN HELMICH Head of Architecture & Developer Relations Lecturer, Software Engineering & Cloud Computing Sci-Fi-Nerd, Metalhead, Amateur Woodworker 

Image Source 

HOSTING BACK THEN 

HOSTING BACK THEN 

APACHE PHP MYSQL HOSTING BACK THEN 

APACHE PHP MYSQL ... AND NOW OPENSEARCH KEYCLOAK MOBILE APP ...OR NODE.JS DOCKER PWA NGINX 💥 ...OR P CDN HOSTING BACK THEN BOOP VARNISH 

No content

MITTWALD CLOUD PLATFORM ARCHITECTURAL GOALS FLEXIBILITY RESOURCE EFFICIENCY ( INTERNAL ) EASE OF USE *) apart from obvious ones, like security, reliability and performance * 

MITTWALD CLOUD PLATFORM ARCHITECTURAL GOALS FLEXIBILITY STANDARDS CONFORMANCE KUBERNETES/COE CONTAINERIZED CUSTOMIZABLE OCI IMAGES QUALITY GOAL SOLUTION STRATEGY RESOURCE EFFICIENCY ELASTICITY OPTIMIZED SCHEDULING ( INTERNAL ) EASE OF USE DECLARATIVE APIS KUBERNETES OPERATORS 

NODE BASIC KUBERNETES PRINCIPLES NODE NODE CLUSTER YEET USER POD POD POD POD ( SLIGHTLY SIMPLIFIED ) POD 

martin @ local $ mw app install typo3 -q --version=12.4.17 --install-mode composer [...] martin @ local $ kubectl get pods --all-namespaces \ --field-selector spec.nodeName=shoot--chp-prod--gestringen-pm-sps-3669c938-z1-797d7-gtfmz NAMESPACE NAME READY STATUS RESTARTS AGE kube-system node-local-dns-69btw 1/1 Running 0 36d kube-system node-problem-detector-2bq2z 1/1 Running 0 36d kube-system openstack-manila-csi-nodeplugin-sk8sm 2/2 Running 0 36d monitoring kube-prometheus-stack-prometheus-node-exporter-dnq9l 1/1 Running 0 36d monitoring loki-stack-promtail-mntw2 1/1 Running 0 36d pg-s-piryq0 mysql-default-80-low-latency-knlw2-0 1/1 Running 0 12s pg-s-piryq0 p-elfmx0-64db467d4b-qzmdv 2/2 Running 0 2d20h pg-s-piryq0 p-elfmx0-l2qzd-b79b7fdf6-qxjl4 2/2 Running 0 12s pg-s-piryq0 p-elfmx0-logrotation-28698480-47mvh 0/1 Completed 0 23h 

POD 

DEPLOYMENT REPLICASET POD STATEFULSET DAEMONSET SERVICE INGRESS INGRESS CLASS PERSISTENT VOLUME PERSISTENT VOLUMECLAIM STORAGE CLASS JOB CRON JOB CERTIFICATE SECRET CONFIG MAP NODE 

No content

No content

ABSTRACTION 

DEPLOYMENT REPLICASET POD STATEFULSET DAEMONSET SERVICE INGRESS INGRESS CLASS PERSISTENT VOLUME PERSISTENT VOLUMECLAIM STORAGE CLASS JOB CRON JOB CERTIFICATE SECRET CONFIG MAP NODE KUBERNTES CORE CONCEPTS PROJECT CUSTOMER PROJECT GROUP APP DATABASE DOMAIN HOSTING DOMAIN MODEL 

apiVersion: apphosting.mittwald.cloud/v1 kind: App metadata: name: a-XXXXX namespace: s-YYYYY spec: app: typo3 version: 12.4.17 project: name: p-ZZZZZ requirements: - name: php version: ^8.3 - name: composer version: ^2 installPath: /my-typo3-site 

CUSTOM RESOURCE CONTROLLER KUBERNETES API USER create / update watch notify reconcile desired state control loop 

https://speakerdeck.com/martinhelmich/the-anatomy-of-docker-image 

No content

PRODUCT SPECIFIC K8S OPERATORS K8S CORE COMPONENTS 

MACRO ARCHITECTURE PRODUCT SPECIFIC MICROSERVICES PRODUCT SPECIFIC K8S OPERATORS K8S CORE COMPONENTS IaaS CLOUD DATA CENTER 

ORGANIZATIONAL STRUCTURE PRODUCT SPECIFIC MICROSERVICES PRODUCT SPECIFIC K8S OPERATORS K8S CORE COMPONENTS IaaS CLOUD DATA CENTER PRODUCT DEV TEAMS CLOUD PLATFORM TEAM IaaS + NETWORK TEAM DATA CENTER OPERATIONS 

UNDER THE HOOD 

ON PREM YES YES NO OPEN SOURCE NO HOW - TO CLOUD 

OPEN - SOURCE CLUSTER PROVISIONING CLUSTER API GARDENER OPENSTACK MAGNUM 

https://docs.openstack.org/security-guide/networking/architecture.html 

No content

WTF? THE STORAGE PROBLEM ( TRADE - OFFS WHEREVER YOU LOOK ) 

NODE POD LOCAL VOLUME - Fast - ( When used as block device) can be optimized for workloads (e.g. XFS for databases) - Limited scalability - Inefficient resource utilization - No networked access - No HA NODE POD ⛔ CEPH NETWORK BLOCK DEVICE - Fast(ish) - Can be optimized for workloads (e.g. XFS for databases) - Networked access by a single client - Reasonably scalable (limited by FS ) - Efficient resource utilization - Various HA configurations with different resource efficiencies and performances (replication & erasure coding) ⛔ CephFS NETWORK FILE SYSTEM - Fast(ish) with few large files - Slow with many small files (tolerable with LOTS of caching) - Networked access by multiple clients - Various HA configurations with different resource efficiencies and performances (replication & erasure coding) NETWORK OBJECT STORE - No filesystem access, only via API - Needs to be considered in architecture on application level - Very high scalability - Networked access by multiple clients RBD ROG THE STORAGE PROBLEM ( TRADE - OFFS WHEREVER YOU LOOK ) 

Gardener BARE METAL SERVERS 

No content

Lukas Tennie https://unsplash.com/photos/a-close-up-of-a-watch-face-with-the-gears-missing-3dyDozzCORw COMPLEXITY 

Ralph_PH, CC - BY K I S S EEP T IMPLE, TUPID. 

WHAT IS YOUR PRODUCT? AND WHAT DO YOU NEED TO RUN IT? 

Gardener BARE METAL SERVERS 

BARE METAL SERVERS 

BARE METAL SERVERS Metallb OUR APPROACH : KUBERNETES - NATIVE EVERYTHING STORAGE VIRTUALIZATION NETWORKING metal-stack 3 

https://metal-stack.io/ https://github.com/onmetal https://scs.community 

ROADMAP INDIVIDUALIZED CUSTOMER SOLUTIONS 3RD PARTY EXTENSIBILITY SCALABILITY ELASTICITY + ( STRATEGIC ) CONFIGURABLE STORAGE OPTIONS ... AND WILD IDEAS SCALE - TO - ZERO HOSTING AI MODELS 3RD PARTY INFRASTRUCTURE ...AND MANY MORE 

ACCEPT COMPLEXITY WHERE YOU NEED IT ( AND MANAGE ACCORDINGLY ) KEEP IT SIMPLE WHERE YOU DON'T KNOW YOUR PRODUCT 

https://github.com/martin-helmich https://www.mittwald.de https://www.martin-helmich.de https://www.linkedin.com/in/martinhelmich