It (only) start with a container

So you've got your application bundled up in the container, great! The fun is not over though, getting it to production is where more choices and complexity are waiting! Should I use Helm? ArgoCD? Terraform? Anything else?

This talk explores the deployment abstraction spectrum from handcrafted YAML and Helm templates, over GitOps and Infrastructure as Code approaches up to developer portals. Each of those give different levels of control and development comfort and we'll compare those to give you a pros and cons and help you choose the right deployment approach for your needs.

See the companion source code at https://github.com/pdudits/talk-containers

Transcript

1. It (only) starts with a container Patrik Duditš Azul Systems (Payara)
2. It (only) starts with a container @pdudits (BlueSky, LinkedIn, GitHub) @pdudits@mastodon.social
3. Hello HTTP/1.1 200 OK Date: Fri, 29 Aug 2025 12:34:56 GMT Server: Payara Server Content-Type: text/plain; charset=UTF-8 Content-Length: 5 Connection: keep-alive Hello
4. I’m an endpoint a method in a class e o reet e our e
5. Sealed in a jar app ar e o reet e our e
6. Next to my runtime app ar e o reet e our e e untime
7. Shipped in a container a e I ontainer app ar e o reet e our e e untime
8. The journey starts Hello. I’m an endpoint. A method in a class. Sealed in a jar next to my runtime; Shipped in a container.
9. Inside a pod. e o reet e our e e untime app ar a e I ontainer
10. ame pa e Inside a pod. Within a namespace. e o reet e our e e untime app ar a e I ontainer
11. Deployment keeps me scheduled ame pa e e o reet e our e e untime app ar a e I ontainer Dep o ment
12. Deployment keeps me scheduled on the Nodes ame pa e e o reet e our e e untime app ar a e I ontainer Dep o ment ode
13. u ter Deployment keeps me scheduled on the Nodes of the wider Cluster ame pa e e o reet e our e e untime app ar a e I ontainer Dep o ment ode
14. Exposed by a service. u ter ame pa e e o reet e our e e untime app ar a e I ontainer Dep o ment ode er i e
15. u ter ame pa e e o reet e our e e untime app ar a e I ontainer Dep o ment ode er i e on ap Exposed by a service. Adjusted with a Config Map.
16. u ter ame pa e e o reet e our e e untime app ar a e I ontainer Dep o ment ode er i e on ap e ret Exposed by a service. Adjusted with a Config Map. My secrets are a Secret.
17. u ter ame pa e e o reet e our e e untime app ar a e I ontainer Dep o ment ode er i e on ap e ret In re Exposed by a service. Adjusted with a Config Map. My secrets are a Secret. And through Ingress I reach out
18. u ter ame pa e e o reet e our e e untime app ar a e I ontainer Dep o ment ode er i e on ap e ret In re In re ontro er One controller guides my packets.
19. u ter ame pa e e o reet e our e e untime app ar a e I ontainer Dep o ment ode er i e on ap e ret In re In re ontro er e terna dn One controller guides my packets. The second gives me name.
20. u ter ame pa e e o reet e our e e untime app ar a e I ontainer Dep o ment ode er i e on ap e ret In re In re ontro er e terna dn ert mana er One controller guides my packets. The second gives me name. Yet another proves me valid.
21. u ter ame pa e e o reet e our e e untime app ar a e I ontainer Dep o ment ode er i e on ap e ret In re In re ontro er e terna dn ert mana er There's more of us, but we're not the same.
22. u ter ame pa e e o reet e our e e untime app ar a e I ontainer Dep o ment ode er i e on ap e ret In re In re ontro er e terna dn ert mana er oad a an er D Pro ider ert ut orit o o e tor etri raper an e ana ement a in ontro er You hear me in network calls. You see me in logs and metrics. You know me through past selves, older versions, remembered settings.
23. u ter ame pa e e o reet e our e e untime app ar a e I ontainer Dep o ment ode er i e on ap e ret In re In re ontro er e terna dn ert mana er oad a an er D Pro ider ert ut orit o o e tor etri raper an e ana ement a in ontro er You hear me in network calls. You see me in logs and metrics. You know me through past selves, older versions, remembered settings.
24. Layer on layer u ter ame pa e e o reet e our e e untime app ar a e I ontainer Dep o ment ode er i e on ap e ret In re In re ontro er e terna dn ert mana er oad a an er D Pro ider ert ut orit o o e tor etri raper an e ana ement a in ontro er oad a an er D Pro ider ert ut orit o o e tor etri raper an e ana ement a in ontro er
25. Layer on layer Object by object u ter ame pa e e o reet e our e e untime app ar a e I ontainer Dep o ment ode er i e on ap e ret In re In re ontro er e terna dn ert mana er oad a an er D Pro ider ert ut orit o o e tor etri raper an e ana ement a in ontro er etri tora e I epo itor e ontro Data a e o tora e
26. Layer on layer Object by object Abstraction becomes Reality u ter ame pa e e o reet e our e e untime app ar a e I ontainer Dep o ment ode er i e on ap e ret In re In re ontro er e terna dn ert mana er oad a an er D Pro ider ert ut orit o o e tor etri raper an e ana ement a in ontro er etri tora e I epo itor e ontro Data a e o tora e HTTP/1.1 200 OK Date: Fri, 29 Aug 2025 12:34:56 GMT Server: Payara Server Content-Type: text/plain; charset=UTF-8 Content-Length: 5 Connection: keep-alive Hello
27. Abstraction becomes Reality Abstraction leaves out details to be filled by convention or deduction
28. Abstraction becomes Reality Abstraction leaves out details to be filled by convention or deduction Levels on abstraction Poem Inside a pod. Within a namespace. Deployment keeps me scheduled on nodes of wider cluster. Exposed by a service. Adjusted with a Config map. My secrets are a Secret. And through an Ingress, I reach out.
29. Abstraction becomes Reality Abstraction leaves out details to be filled by convention or deduction Levels on abstraction Poem YAML Inside a pod. Within a namespace. Deployment keeps me scheduled on nodes of wider cluster. Exposed by a service. Adjusted with a Config map. My secrets are a Secret. And through an Ingress, I reach out.
30. Example setup First Application http://first-app.domain/ Second Application http://second-app.domain/ Third Application http://first-app.domain/other/ Common Application Concept
31. Sample code and configs https://github.com/pdudits/talk-containers
32. apiVersion: v1 kind: Namespace metadata: name: demonstration --- apiVersion: v1 kind: ConfigMap metadata: name: first-instance-config namespace: demonstration data: application.properties: | hello.defaultName=firstInstance --- apiVersion: apps/v1 kind: Deployment metadata: name: first-instance namespace: demonstration labels: app: first-instance spec: replicas: 1 selector: matchLabels: app: first-instance template: metadata: labels: app: first-instance spec: containers: - name: container-app image: pdudits/container-talk-demo:latest imagePullPolicy: Always args: ["--deploymentDir", "/opt/payara/deployments", "--contextRoot", "/", "--systemProperties", "/config/application.properties"] ports: - name: http containerPort: 8080 resources: requests: cpu: "250m" memory: "256Mi" volumeMounts: - name: config mountPath: /config volumes: - name: config configMap: name: first-instance-config items: - key: application.properties path: application.properties --- apiVersion: v1 kind: Service metadata: name: first-instance namespace: demonstration labels: app: first-instance spec: type: ClusterIP selector: app: first-instance ports: - name: http port: 8080 targetPort: http --- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: first-instance namespace: demonstration labels: app: first-instance spec: ingressClassName: traefik-cloud-head rules: - host: first-instance-conference.payara.app http: paths: - path: / pathType: Prefix backend: service: name: first-instance port: number: 8080 apiVersion: v1 kind: Namespace metadata: name: demonstration --- apiVersion: v1 kind: ConfigMap metadata: name: first-instance-config namespace: demonstration data: application.properties: | hello.defaultName=firstInstance --- apiVersion: apps/v1 kind: Deployment metadata: name: first-instance namespace: demonstration labels: app: first-instance spec: replicas: 1 selector: matchLabels: app: first-instance template: metadata: labels: app: first-instance spec: containers: - name: container-app image: pdudits/container-talk-demo:latest imagePullPolicy: Always args: ["--deploymentDir", "/opt/payara/deployments", "--contextRoot", "/", "--systemProperties", "/config/application.properties"] ports: - name: http containerPort: 8080 resources: requests: cpu: "250m" memory: "256Mi" volumeMounts: - name: config mountPath: /config volumes: - name: config configMap: name: first-instance-config items: - key: application.properties path: application.properties --- apiVersion: v1 kind: Service metadata: name: first-instance namespace: demonstration labels: app: first-instance spec: type: ClusterIP selector: app: first-instance ports: - name: http port: 8080 targetPort: http --- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: first-instance namespace: demonstration labels: app: first-instance spec: ingressClassName: traefik-cloud-head rules: - host: first-instance-conference.payara.app http: paths: - path: / pathType: Prefix backend: service: name: first-instance port: number: 8080
33. apiVersion: v1 kind: Namespace metadata: name: demonstration --- apiVersion: v1 kind: ConfigMap metadata: name: first-instance-config namespace: demonstration data: application.properties: | hello.defaultName=firstInstance --- apiVersion: apps/v1 kind: Deployment metadata: name: first-instance namespace: demonstration labels: app: first-instance spec: replicas: 1 selector: matchLabels: app: first-instance template: metadata: labels: app: first-instance spec: containers: - name: container-app image: pdudits/container-talk-demo:latest imagePullPolicy: Always args: ["--deploymentDir", "/opt/payara/deployments", "--contextRoot", "/", "--systemProperties", "/config/application.properties"] ports: - name: http containerPort: 8080 resources: requests: cpu: "250m" memory: "256Mi" volumeMounts: - name: config mountPath: /config volumes: - name: config configMap: name: first-instance-config items: - key: application.properties path: application.properties --- apiVersion: v1 kind: Service metadata: name: first-instance namespace: demonstration labels: app: first-instance spec: type: ClusterIP selector: app: first-instance ports: - name: http port: 8080 targetPort: http --- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: first-instance namespace: demonstration labels: app: first-instance spec: ingressClassName: traefik-cloud-head rules: - host: first-instance-conference.payara.app http: paths: - path: / pathType: Prefix backend: service: name: first-instance port: number: 8080 apiVersion: v1 kind: Namespace metadata: name: demonstration --- apiVersion: v1 kind: ConfigMap metadata: name: second-instance-config namespace: demonstration data: application.properties: | hello.defaultName=secondInstance hello.downstreamUri=http://first-instance/hello --- apiVersion: apps/v1 kind: Deployment metadata: name: second-instance namespace: demonstration labels: app: second-instance spec: replicas: 1 selector: matchLabels: app: second-instance template: metadata: labels: app: second-instance spec: containers: - name: container-app image: pdudits/container-talk-demo:latest imagePullPolicy: Always args: ["--deploymentDir", "/opt/payara/deployments", "--contextRoot", "/", "--systemProperties", "/config/application.properties"] ports: - name: http containerPort: 8080 resources: requests: cpu: "250m" memory: "256Mi" volumeMounts: - name: config mountPath: /config volumes: - name: config configMap: name: second-instance-config items: - key: application.properties path: application.properties --- apiVersion: v1 kind: Service metadata: name: second-instance namespace: demonstration labels: app: second-instance spec: type: ClusterIP selector: app: second-instance ports: - name: http port: 80 targetPort: http --- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: second-instance namespace: demonstration labels: app: second-instance spec: ingressClassName: traefik-cloud-head rules: - host: second-instance-conference.payara.app http: paths: - path: / pathType: Prefix backend: service: name: second-instance port: name: http
34. apiVersion: v1 kind: Namespace metadata: name: demonstration --- apiVersion: v1 kind: ConfigMap metadata: name: first-instance-config namespace: demonstration data: application.properties: | hello.defaultName=firstInstance --- apiVersion: apps/v1 kind: Deployment metadata: name: first-instance namespace: demonstration labels: app: first-instance spec: replicas: 1 selector: matchLabels: app: first-instance template: metadata: labels: app: first-instance spec: containers: - name: container-app image: pdudits/container-talk-demo:latest imagePullPolicy: Always args: ["--deploymentDir", "/opt/payara/deployments", "--contextRoot", "/", "--systemProperties", "/config/application.properties"] ports: - name: http containerPort: 8080 resources: requests: cpu: "250m" memory: "256Mi" volumeMounts: - name: config mountPath: /config volumes: - name: config configMap: name: first-instance-config items: - key: application.properties path: application.properties --- apiVersion: v1 kind: Service metadata: name: first-instance namespace: demonstration labels: app: first-instance spec: type: ClusterIP selector: app: first-instance ports: - name: http port: 8080 targetPort: http --- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: first-instance namespace: demonstration labels: app: first-instance spec: ingressClassName: traefik-cloud-head rules: - host: first-instance-conference.payara.app http: paths: - path: / pathType: Prefix backend: service: name: first-instance port: number: 8080 apiVersion: v1 kind: Namespace metadata: name: demonstration --- apiVersion: v1 kind: ConfigMap metadata: name: second-instance-config namespace: demonstration data: application.properties: | hello.defaultName=secondInstance hello.downstreamUri=http://first-instance/hello --- apiVersion: apps/v1 kind: Deployment metadata: name: second-instance namespace: demonstration labels: app: second-instance spec: replicas: 1 selector: matchLabels: app: second-instance template: metadata: labels: app: second-instance spec: containers: - name: container-app image: pdudits/container-talk-demo:latest imagePullPolicy: Always args: ["--deploymentDir", "/opt/payara/deployments", "--contextRoot", "/", "--systemProperties", "/config/application.properties"] ports: - name: http containerPort: 8080 resources: requests: cpu: "250m" memory: "256Mi" volumeMounts: - name: config mountPath: /config volumes: - name: config configMap: name: second-instance-config items: - key: application.properties path: application.properties --- apiVersion: v1 kind: Service metadata: name: second-instance namespace: demonstration labels: app: second-instance spec: type: ClusterIP selector: app: second-instance ports: - name: http port: 80 targetPort: http --- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: second-instance namespace: demonstration labels: app: second-instance spec: ingressClassName: traefik-cloud-head rules: - host: second-instance-conference.payara.app http: paths: - path: / pathType: Prefix backend: service: name: second-instance port: name: http - 14 lines + 14 lines
35. Kustomize • Tool performing set of transformations on resource definitions • Predefined name / metadata transformations • Configuration generators • Arbitrary patch operations • Part of kubectl • kubectl kustomize • kubect apply –k . base/ ingress.yaml deployment.yaml … kustomization.yaml first-instance/ kustomization.yaml config …
36. base/kustomization.yaml kind: Kustomization apiVersion: kustomize.config.k8s.io/v1beta1 resources: - namespace.yaml - configmap.yaml - deployment.yaml - service.yaml - ingress.yaml
37. first-instance/kustomization.yaml kind: Kustomization apiVersion: kustomize.config.k8s.io/v1beta1 resources: - ../base namespace: demonstration # apply label to all resources, # templates and selectors namePrefix: first-instance- labels: - includeSelectors: true includeTemplates: true pairs: app: first-instance # read config from file configMapGenerator: - name: config namespace: demonstration behavior: replace files: - application.properties # set hostname patches: - target: name: ingress patch: |- - op: replace path: /spec/rules/0/host value: first-app-conference.payara.app + 23 lines
38. third-instance/kustomization.yaml # set hostname and context root patches: - target: name: ingress patch: |- - op: replace path: /spec/rules/0/host value: first-app-conference.payara.app - op: replace path: /spec/rules/0/http/paths/0/path value: /other - target: name: deployment # context root is 4th argument on in args patch: |- - op: replace path: /spec/template/spec/containers/0/args/3 value: /other
39. Locality problem • Resource limit may affect command line arguments or configuration • Tune GC according to your memory/cpu request • -XX:MaxRAMPercentage –XX:ActiveProcessorCount • Context root affects both app configuration and Ingress mapping • Config Map or Deployment spec • Ingress
40. Limitations of Kustomize • Verbosity • Structural dependency • Low reusability – no higher order components, computed values • Poor documentation • Slow evolution • No transactions / rollback
41. Abstraction becomes Reality Abstraction leaves out details to be filled by convention or deduction Levels of abstraction Poem YAML / Kustomize Inside a pod. Within a namespace. Deployment keeps me scheduled on nodes of wider cluster. Exposed by a service. Adjusted with a Config map. My secrets are a Secret. And through an Ingress, I reach out.
42. Abstraction becomes Reality Abstraction leaves out details to be filled by convention or deduction Levels of abstraction Poem Templating YAML / Kustomize Inside a pod. Within a namespace. Deployment keeps me scheduled on nodes of wider cluster. Exposed by a service. Adjusted with a Config map. My secrets are a Secret. And through an Ingress, I reach out.
43. Templating tools • Helm – The standard templating and distribution tool in the ecosystem • JSonnet – a neat configuration DSL language • YokeCD – Delivery pipeline around WASM generators
44. Structure of a Helm chart • Chart.yaml • values.yaml • templates/ • _library.tpl • resource.yaml • … • charts/ helm create container-app
45. replicaCount: 1 image: repository: pdudits/container-talk-demo pullPolicy: Always tag: latest imagePullSecrets: [] nameOverride: "" fullnameOverride: "" serviceAccount: create: false automount: false annotations: {} name: "" podAnnotations: {} podLabels: {} podSecurityContext: {} securityContext: {} service: type: ClusterIP port: 80 containerPort: 8080 ingress: enabled: true className: "traefik-cloud-head" annotations: {} hosts: - host: chart-example.local paths: - path: / pathType: Prefix tls: [] resources: {} livenessProbe: httpGet: path: /health/live port: http readinessProbe: httpGet: path: /health/ready port: http volumes: [] volumeMounts: [] nodeSelector: {} tolerations: [] affinity: {} config: |- hello.defaultName=Helm values.yaml
46. Solving the value dependency spec: template: spec: containers: - name: {{ .Chart.Name }} args: ["--deploymentDir", "/opt/payara/deployments", "--contextRoot", "{{ (index (index .Values.ingress.hosts 0).paths 0).path }}", "--systemProperties", "/config/application.properties" ]
47. Deploying multiple chart instances helm upgrade --install -n demonstration \ -f first-instance-values.yaml \ first-instance container-app/ helm upgrade --install -n demonstration \ -f second-instance-values.yaml \ second-instance container-app/
48. It i po i e to reu e a art… apiVersion: v2 name: combined description: Make two instances of container app type: application version: 0.1.0 appVersion: "1.16.0" dependencies: # this works, but it is quite an edge case - name: container-app repository: file://../container-app/ alias: first-instance version: ">=0" - name: container-app repository: file://../container-app/ alias: second-instance version: ">=0"
49. It i po i e to reu e a art… first-instance: ingress: hosts: - host: first-instance-conf.payara.app paths: - path: / pathType: Prefix config: |- hello.defaultName=firstInstance second-instance: ingress: hosts: - host: first-instance-conf.payara.app paths: - path: /second pathType: Prefix config: |- hello.defaultName=secondInstance hello.downstreamUri=http://combined-first-instance/hello + 20 lines
50. … ut: • No way to interpolate values • Unless specifically supported by chart • Top level checks to enforce same values • Global namespace for templates • The definitions of templates (such as container-app.labels) must match
51. Abstraction becomes Reality Abstraction leaves out details to be filled by convention or deduction Levels of abstraction Poem Templating / Helm YAML / Kustomize Inside a pod. Within a namespace. Deployment keeps me scheduled on nodes of wider cluster. Exposed by a service. Adjusted with a Config map. My secrets are a Secret. And through an Ingress, I reach out. GitOps
52. GitOps tooling https://fluxcd.io https://argoproj.github.io/
53. Abstraction becomes Reality Abstraction leaves out details to be filled by convention or deduction Levels of abstraction Poem Infra as Code Templating / Helm YAML / Kustomize Inside a pod. Within a namespace. Deployment keeps me scheduled on nodes of wider cluster. Exposed by a service. Adjusted with a Config map. My secrets are a Secret. And through an Ingress, I reach out. GitOps / ArgoCD
54. Infrastructure as Code • Define your desired Kubernetes state in a program • Also your additional infrastructure • Terraform / OpenTofu • Pulumi
55. Pulumi • Executes arbitrary program (in TypeScript, Ja a, Py h ,… • Construction of Pulumi Resources builds up state • Reconcile the state with Providers • Native as well as Terraform providers
56. tra tion e e static void run(Context ctx) { // Pulumi collects state by invoking constructors of resource objects var config = ctx.config(); var namespaceName = config.get("namespace").orElse("demonstration"); var ns = new Namespace(namespaceName, NamespaceArgs.builder() .metadata(ObjectMetaArgs.builder() // provide explicit name, otherwise an ID is suffixed to the name .name(namespaceName) .build()) .build()); var hostname = config.get("hostName").orElse("pulumi-sample.payara.app"); var instance1 = new ContainerApp("first-instance", ns, hostname, "/"); var instance2 = new ContainerApp("second-instance", ns, hostname, "/second", instance1.internalEndpoint()); ctx.export("firstEndpoint", instance1.endpoint()); ctx.export("secondEndpoint", instance2.endpoint()); } Create the namespace High-Level Abstraction
57. public ContainerApp(String name, ContainerAppArgs args, ComponentResourceOptions options) { super("talk:containers:ContainerApp", name, args, options); var imYourFather = CustomResourceOptions.builder().parent(this).build(); var objectMetaDefault = ObjectMetaArgs.builder().namespace(args.namespace()) .labels(Map.of("app", name)) .build(); var configMap = new ConfigMap(name+"-configmap", ConfigMapArgs.builder() .metadata(objectMetaDefault) .data(generateConfig(name,args).applyValue(s -> Map.of("application.properties", s))) .build(), imYourFather); var deployment = new Deployment(name, DeploymentArgs.builder() .metadata(objectMetaDefault) .spec(DeploymentSpecArgs.builder() .replicas(1) .selector(LabelSelectorArgs.builder().matchLabels(Map.of("app", name)).build()) .template(PodTemplateSpecArgs.builder() .metadata(objectMetaDefault) .spec(PodSpecArgs.builder() .volumes(VolumeArgs.builder() .name("config") .configMap(ConfigMapVolumeSourceArgs.builder() .name(nameOf(configMap.metadata())) .items(KeyToPathArgs.builder() .path("application.properties") .key("application.properties") .build()) .build()) .build()) .containers(ContainerArgs.builder() .name("container-app") .image("pdudits/container-talk-demo:latest") .args(Output.<String>listBuilder() .add("--deploymentDir", "/opt/payara/deployments", "--contextRoot") .add(args.contextRoot()) .add( "--systemProperties", "/config/application.properties") .build()) .ports(ContainerPortArgs.builder() .name("http") .containerPort(8080) .protocol("TCP") .build()) .volumeMounts(VolumeMountArgs.builder() .name("config") .mountPath("/config") .build()) .resources(ResourceRequirementsArgs.builder() .requests(Map.of("cpu","250m", "memory", "256Mi")) .build()) .livenessProbe(ProbeArgs.builder() .httpGet(HTTPGetActionArgs.builder() .path("/health/live") .port("http") .build()) .build()) .readinessProbe(ProbeArgs.builder() .httpGet(HTTPGetActionArgs.builder() .path("/health/ready") .port("http") .build()) .build()) .build()) .build()) .build()) .build()) .build(), imYourFather); var service = new Service(name, ServiceArgs.builder() .metadata(ObjectMetaArgs.builder(objectMetaDefault).name(name).build()) .spec(ServiceSpecArgs.builder() .type("ClusterIP") .selector(Map.of("app", name)) .ports(ServicePortArgs.builder() .name("http") .port(80) .targetPort("http") .build()) .build()).build() , imYourFather); var ingress = new Ingress(name, IngressArgs.builder() .metadata(objectMetaDefault) .spec(IngressSpecArgs.builder() .ingressClassName("traefik-cloud-head") .rules(IngressRuleArgs.builder() .host(args.hostName()) .http(HTTPIngressRuleValueArgs.builder() .paths(HTTPIngressPathArgs.builder() .path(args.contextRoot()) .pathType("Prefix") .backend(IngressBackendArgs.builder() .service(IngressServiceBackendArgs.builder() .name(nameOf(service.metadata())) .port(ServiceBackendPortArgs.builder() .name("http") .build()) .build()) .build()) .build()) .build()) .build()) .build()) .build(), imYourFather); this.endpoint = Output.format("https://%s%s/hello", args.hostName(), args.contextRoot()); this.internalEndpoint = Output.format("http://%s%s/hello", nameOf(service.metadata()), args.contextRoot()); }
58. Also handles surrounding infrastructure • Keeping ingress controller and other cluster controllers up-to- date • Kubernetes version upgrades • Managed databases • Managed identities But: • Needs additional state file(s) • Only one deployment at time • Many async compositions
59. tra tion e ome ea it Abstraction leaves out details to be filled by convention or deduction Levels of abstraction Poem IaC / Pulumi Templating / Helm YAML / Kustomize Inside a pod. Within a namespace. Deployment keeps me scheduled on nodes of wider cluster. Exposed by a service. Adjusted with a Config map. My secrets are a Secret. And through an Ingress, I reach out. GitOps / ArgoCD
60. Abstraction becomes Reality Abstraction leaves out details to be filled by convention or deduction Levels of abstraction Poem Custom Ctrl / Crossplane IaC / Pulumi Templating / Helm YAML / Kustomize Inside a pod. Within a namespace. Deployment keeps me scheduled on nodes of wider cluster. Exposed by a service. Adjusted with a Config map. My secrets are a Secret. And through an Ingress, I reach out. GitOps / ArgoCD
61. Custom Kubernetes Controllers • Store configuration as Kubernetes Custom Resources • Transform custom resources into standard Kubernetes objects
62. apiVersion: v1 kind: Namespace metadata: name: demonstration --- apiVersion: v1 kind: ConfigMap metadata: name: first-instance-config namespace: demonstration data: application.properties: | hello.defaultName=firstInstance --- apiVersion: apps/v1 kind: Deployment metadata: name: first-instance namespace: demonstration labels: app: first-instance spec: replicas: 1 selector: matchLabels: app: first-instance template: metadata: labels: app: first-instance spec: containers: - name: container-app image: pdudits/container-talk-demo:latest imagePullPolicy: Always args: ["--deploymentDir", "/opt/payara/deployments", "--contextRoot", "/", "--systemProperties", "/config/application.properties"] ports: - name: http containerPort: 8080 resources: requests: cpu: "250m" memory: "256Mi" volumeMounts: - name: config mountPath: /config volumes: - name: config configMap: name: first-instance-config items: - key: application.properties path: application.properties --- apiVersion: v1 kind: Service metadata: name: first-instance namespace: demonstration labels: app: first-instance spec: type: ClusterIP selector: app: first-instance ports: - name: http port: 8080 targetPort: http --- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: first-instance namespace: demonstration labels: app: first-instance spec: ingressClassName: traefik-cloud-head rules: - host: first-instance-conference.payara.app http: paths: - path: / pathType: Prefix backend: service: name: first-instance port: number: 8080 apiVersion: payara.cloud/v1 kind: JavaApp metadata: name: second-instance namespace: demonstration-crd spec: parameters: hostName: containers-kcl-crossplane.payara.app contextRoot: /second config: |- hello.defaultName=secondInstance hello.downstreamUri=http://first-instance/hello
63. apiVersion: v1 kind: Namespace metadata: name: demonstration --- apiVersion: v1 kind: ConfigMap metadata: name: first-instance-config namespace: demonstration data: application.properties: | hello.defaultName=firstInstance --- apiVersion: apps/v1 kind: Deployment metadata: name: first-instance namespace: demonstration labels: app: first-instance spec: replicas: 1 selector: matchLabels: app: first-instance template: metadata: labels: app: first-instance spec: containers: - name: container-app image: pdudits/container-talk-demo:latest imagePullPolicy: Always args: ["--deploymentDir", "/opt/payara/deployments", "--contextRoot", "/", "--systemProperties", "/config/application.properties"] ports: - name: http containerPort: 8080 resources: requests: cpu: "250m" memory: "256Mi" volumeMounts: - name: config mountPath: /config volumes: - name: config configMap: name: first-instance-config items: - key: application.properties path: application.properties --- apiVersion: v1 kind: Service metadata: name: first-instance namespace: demonstration labels: app: first-instance spec: type: ClusterIP selector: app: first-instance ports: - name: http port: 8080 targetPort: http --- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: first-instance namespace: demonstration labels: app: first-instance spec: ingressClassName: traefik-cloud-head rules: - host: first-instance-conference.payara.app http: paths: - path: / pathType: Prefix backend: service: name: first-instance port: number: 8080 apiVersion: payara.cloud/v1 kind: JavaApp metadata: name: second-instance namespace: demonstration-crd spec: parameters: hostName: containers-kcl-crossplane.payara.app contextRoot: /second config: |- hello.defaultName=secondInstance hello.downstreamUri=http://first-instance/hello status: url: https://containers-kcl-crossplane.payara.app/second
64. ro p ane • Allows building custom resources and their control planes (controllers) in consistent manner • Controllers can be written declaratively or in KCL, Python or Typescript. • There is also prototype of Java-based controller • Not only Kubernetes, but also cloud resources in general
65. Crossplane resource definition apiVersion: apiextensions.crossplane.io/v2 kind: CompositeResourceDefinition metadata: name: javaapps.payara.cloud spec: group: payara.cloud names: kind: JavaApp plural: javaapps versions: - name: v1 served: true referenceable: true schema: openAPIV3Schema: type: object properties: spec: type: object properties: compositionRef: type: object properties: name: type: string compositionSelector: type: object properties: matchLabels: type: object additionalProperties: type: string parameters: type: object properties: hostName: type: string description: Ingress hostname for the application contextRoot: type: string default: / description: Application context root path downstreamUri: type: string description: Optional downstream service URI for chaining config: type: string description: Application properties content required: - hostName - config required: - parameters status: type: object properties: url: type: string description: External URL for accessing the application Input Output Input
66. Patch & Transform pipeline a V s : a x s s. ss a . / 1 k : C m s m a a a: am : a a - am s a : m s a - ss a -yam a s: ss a - a a : yam s : m s Ty R f: a V s : aya a. / 1 k : Ja aA m : P : - s : f R f: am : f - a h-a - a sf m : a V s : .f . ss a . / 1 a1 k : R s s s s: - am : f ma as : a V s : 1 k : C f Ma m a a a: am : a h a a: a a . s: a h s: - y : F mC m s F Pa h f mF Pa h: m a a a. am s a F Pa h: m a a a. am s a - y : F mC m s F Pa h f mF Pa h: m a a a. am F Pa h: m a a a. am - y : F mC m s F Pa h f mF Pa h: s . a am s. f F Pa h: a a[ a a . s ] - am : ym as : a V s : a s/ 1 k : D ym m a a a: am : a h s : as: 1 s : ma hLa s: a : a h m a : m a a a: a a s: s m -a a : a01 a s: a : a h s : a s: - am : a -a ma : s/ a - a k- m : a s ma P P y: A ways a s: - -- ym D - / / aya a/ ym s - -- x R - / - --sys mP s - / f /a a . s s: - am : h a P : 8080 : TCP ssP : h G : a h: /h a h/ : h a ssP : h G : a h: /h a h/ a y : h m M s: - m Pa h: / f am : f s s: q s s: : 250m m m y: 256M m s: - am : f f Ma : am : a h ms: - k y: a a . s a h: a a . s a h s: - y : F mC m s F Pa h f mF Pa h: m a a a. am s a F Pa h: m a a a. am s a - y : F mC m s F Pa h f mF Pa h: m a a a. am F Pa h: m a a a. am - y : F mC m s F Pa h f mF Pa h: m a a a. am F Pa h: s .s .ma hLa s.a - y : F mC m s F Pa h f mF Pa h: m a a a. am F Pa h: s . m a .m a a a. a s.a - y : F mC m s F Pa h f mF Pa h: m a a a. am F Pa h: s . m a .s . m s[0]. f Ma . am - y : F mC m s F Pa h f mF Pa h: s . a am s. x R F Pa h: s . m a .s . a s[0].a s[3] …. Resource Template Patches
67. oxr = option("params").oxr params = oxr.spec.parameters ocds = option("params").ocds app_name = oxr.metadata.name namespace = oxr.metadata.namespace host_name = params.hostName context_root = params.contextRoot or "/" config = params.config configmap = { apiVersion: "v1" kind: "ConfigMap" metadata: { annotations: { "krm.kcl.dev/composition-resource-name": "configmap" } name: app_name namespace: namespace } data: { "application.properties": config } } deployment = { apiVersion: "apps/v1" kind: "Deployment" metadata: { annotations: { "krm.kcl.dev/composition-resource-name": "deployment" } name: app_name namespace: namespace } spec: { replicas: 1 selector: { matchLabels: { app: app_name } } template: { metadata: { labels: { app: app_name } } spec: { containers: [{ name: "container-app" image: "pdudits/container-talk-demo:latest" imagePullPolicy: "Always" args: [ "--deploymentDir", "/opt/payara/deployments", "--contextRoot", context_root, "--systemProperties", "/config/application.properties" ] ports: [{ name: "http" containerPort: 8080 protocol: "TCP" }] livenessProbe: { httpGet: { path: "/health/live" port: "http" } } readinessProbe: { httpGet: { path: "/health/ready" port: "http" } } volumeMounts: [{ mountPath: "/config" name: "config" }] resources: { requests: { cpu: "250m" memory: "256Mi" } } }] volumes: [{ name: "config" configMap: { name: app_name items: [{ key: "application.properties" path: "application.properties" }] }} ] }}}} service = { apiVersion: "v1" kind: "Service" metadata: { annotations: { "krm.kcl.dev/composition-resource-name": "service" } name: app_name namespace: namespace } spec: { type: "ClusterIP" ports: [{ name: "http" port: 80 protocol: "TCP" targetPort: "http" }] selector: { app: app_name } } } ingress = { apiVersion: "networking.k8s.io/v1" kind: "Ingress" metadata: { annotations: { "krm.kcl.dev/composition-resource-name": "ingress" } name: app_name namespace: namespace } spec: { ingressClassName: "traefik-cloud-head" rules: [{ host: host_name http: { paths: [{ path: context_root pathType: "Prefix" backend: { service: { name: app_name port: { number: 80 } } } }] } }] } } _items_base = [configmap, deployment, service, ingress] # Patch XR status with URL from Ingress host and context root _dxr = { **oxr } # Build URL from ingress rule host and context root path _path = context_root if context_root != "/" else "" _dxr.status.url = "https://{host}{path}".format(host=host_name, path=_path) items = _items_base + [_dxr] KCL function
68. Write your Own • Achievable with e. g. Fabric8 Kubernetes Client • Create Informers to listen to changes on specific Input resources • Perform actions • Create more informers to track the state and update status on input resources https://github.com/pdudits/doorman/ Scale to Zero controller for Traefik
69. Abstraction becomes Reality A s a a s a s f y Levels of abstraction Poem Custom Ctrl / Crossplane IaC / Pulumi Templating / Helm YAML / Kustomize Inside a pod. Within a namespace. Deployment keeps me scheduled on nodes of wider cluster. Exposed by a service. Adjusted with a Config map. My secrets are a Secret. And through an Ingress, I reach out. GitOps / ArgoCD
71. e’re a fwa t ere Application + Configuration Deployment mechanism Execution Environment Observability and diagnostics Application resources Logs Ingress + DNS TLS certificates Metrics Infrastructure maintenance
72. Application Binary Jakarta EE 8/10(/11) SpringBoot Quarkus Configuration JDK Version CPU reservation Config properties HTTP mapping Endpoint Metric Logs Thread and Heap dumps Preconfigured Kubernetes cluster and surrounding services Management UI
73. Standardized container a e I ontainer app ar e o reet e our e e untime Pre-optimized Payara Micro For each server / JDK combination (AppCDS) Quarkus Extension to standardize logging and configuration Regularly updated base image
74. e o reet e our e e untime app ar a e I ontainer Custom controller applied desired configuration to all Kubernetes resources
75. ame pa e e o reet e our e e untime app ar a e I ontainer All applications in namespace shared domain name
76. ame pa e e o reet e our e e untime app ar a e I ontainer Dep o ment ode Auto scaler set up in target clusters
77. u ter ame pa e e o reet e our e e untime app ar a e I ontainer Dep o ment ode Clusters in multiple regions on both Azure and AWS
78. u ter ame pa e e o reet e our e e untime app ar a e I ontainer Dep o ment ode er i e
79. u ter ame pa e e o reet e our e e untime app ar a e I ontainer Dep o ment ode er i e on ap
80. u ter ame pa e e o reet e our e e untime app ar a e I ontainer Dep o ment ode er i e on ap e ret Configuration was separated to regular and restricted fields. Restricted ones mapped into a secret
81. u ter ame pa e e o reet e our e e untime app ar a e I ontainer Dep o ment ode er i e on ap e ret In re Predefined as well as custom domains were supported
82. u ter ame pa e e o reet e our e e untime app ar a e I ontainer Dep o ment ode er i e on ap e ret In re In re ontro er Traefik proved to be easy to setup and monitor as well as more performant than managed solutions
83. u ter ame pa e e o reet e our e e untime app ar a e I ontainer Dep o ment ode er i e on ap e ret In re In re ontro er e terna dn Updated to ingres propagated to Azure DNS or Route 53 respectively
84. u ter ame pa e e o reet e our e e untime app ar a e I ontainer Dep o ment ode er i e on ap e ret In re In re ontro er e terna dn ert mana er Predefined domain got wildcard cert from LetsEncrypt to prevent scanning. Custom domain required CNAME to predefined one and got cert via HTTP challenge
85. u ter ame pa e e o reet e our e e untime app ar a e I ontainer Dep o ment ode er i e on ap e ret In re In re ontro er e terna dn ert mana er Multi-tenancy was solved by isolating network communication within a namespace and strict AppArmor profiles
86. u ter ame pa e e o reet e our e e untime app ar a e I ontainer Dep o ment ode er i e on ap e ret In re In re ontro er e terna dn ert mana er oad a an er D Pro ider ert ut orit o o e tor etri raper an e ana ement a in ontro er fluent-bit would feed managed log service (Azure Log Analytics / Cloudwatch Logs) Prometheus in agent mode written into managed Prometheus services
87. u ter ame pa e e o reet e our e e untime app ar a e I ontainer Dep o ment ode er i e on ap e ret In re In re ontro er e terna dn ert mana er oad a an er D Pro ider ert ut orit o o e tor etri raper an e ana ement a in ontro er
88. u ter ame pa e e o reet e our e e untime app ar a e I ontainer Dep o ment ode er i e on ap e ret In re In re ontro er e terna dn ert mana er oad a an er D Pro ider ert ut orit o o e tor etri raper an e ana ement a in ontro er oad a an er D Pro ider ert ut orit o o e tor etri raper an e ana ement a in ontro er Single management UI provided browser and CLI interface to all clusters Let’s Encrypt Standard managed LB Standard managed DNS Scaling to Zero without service mesh (see Doorman)
89. u ter ame pa e e o reet e our e e untime app ar a e I ontainer Dep o ment ode er i e on ap e ret In re In re ontro er e terna dn ert mana er oad a an er D Pro ider ert ut orit o o e tor etri raper an e ana ement a in ontro er etri tora e I epo itor e ontro Data a e o tora e Whole control plane managed by modular Pulumi Stack
90. Summary A standardized deployment environment for java applications is feasible and ceremony around it can be minimized. It may require multiple layers of tooling to achieve that.
91. It only starts with containers, it ends with Poetry as a Service. a e I ontainer app ar e o reet e our e e untime Better known as PaaS.
92. Patrik Duditš @ s