KubeCon + CloudNativeCon Japan 2025 Lightning Talk: Optimizing Web Applications by Offloading Heavy Processing To Kubernetes Jobs - Asami Okina

While typical web applications do not require large amounts of resources constantly, there are cases where specific processes consume significant CPU and memory.
In this session, we will introduce an architecture that offloads such resource-intensive processes to Kubernetes Jobs.
We will explain specific methods for Job management, how to integrate web applications (Next.js, @kubernetes/client-node) with the Kubernetes API, methods for data integration between Jobs and web applications, and real-time tracking of Job progress in the UI, all while sharing practical examples. Furthermore, we will provide a detailed introduction to a pattern where Kubernetes Job definitions generated from applications are managed using ConfigMaps, enabling quick configuration switching between environments, and offer hints to optimize your applications in terms of cost, performance, and management.

Session video is available here:
https://youtu.be/8pVt9dhHEVc?feature=shared
Sample code is available here: https://github.com/asami-okina/quick-container-image-scan

Transcript

1. Asami Okina / Craftsman Software, Inc. Optimizing Web Applications by Offloading Heavy Processing to Kubernetes Jobs
2. Self-Introduction CKAD CKA Asami Okina Cloud-Native App Developer Working at asami-okina asami_okina
3. Problem Excessive CPU/memory allocation for temporary high loads. Always-On Over-Provisioning Wasting money on peak capacity 24/7 — just for a few moments of actual need. The web app is lightweight during normal operations. Needs heavy CPU/memory only during specific operations.
4. Solution Kubernetes standard features only ! Offload heavy processing to Kubernetes Jobs Kubernetes Cluster Lightweight Web App Execute heavy Task K8s Job Just what you need, when you need it. Keep the web app lightweight. Need more CPU? Scale with a Job — just in time.
5. How
6. Agenda Part 1：Job Creation Part 2：Job Execution Patterns for Creating Jobs from Web App
7. Agenda Part 1：Job Creation Part 2：Job Execution Patterns for Creating Jobs from Web App
8. Part 1：Job Creation Web (UI) Browser Flow of Creating Jobs from Web App Kubernetes Cluster Next.js(API) In App pod ① Click Start Process Button
9. Part 1：Job Creation Web (UI) Browser K8s Job Worker pod K8s API Server Kubernetes Cluster Next.js(API) In App pod ① Click Start Process Button ② Job Create Request Flow of Creating Jobs from Web App
10. Part 1：Job Creation Web (UI) Browser K8s Job Worker pod K8s API Server Kubernetes Cluster Next.js(API) In App pod ① Click Start Process Button ② Job Create Request ③ Response HTTP 200 Flow of Creating Jobs from Web App
11. Agenda Part 1：Job Creation Part 2：Job Execution Patterns for Creating Jobs from Web App
12. Part 2：Job Execution K8s Job Worker pod Flow of Job Execution and Real-time Feedback ① Execute Heavy Processing Kubernetes Cluster
13. Part 2：Job Execution K8s Job Worker pod Shared DB Progress Sharing ② Write Progress ① Execute Heavy Processing Kubernetes Cluster Flow of Job Execution and Real-time Feedback
14. Part 2：Job Execution K8s Job Worker pod Shared DB Progress Sharing Next.js(API) In App pod ③ Poll Progress ② Write Progress ① Execute Heavy Processing Kubernetes Cluster Flow of Job Execution and Real-time Feedback
15. Part 2：Job Execution Web (UI) Browser K8s Job Worker pod Shared DB Progress Sharing Processing Status File Conversion (75% Complete) - About 2 min remaining ④ Real-time Display Next.js(API) In App pod ③ Poll Progress ② Write Progress ① Execute Heavy Processing Kubernetes Cluster Flow of Job Execution and Real-time Feedback
16. Agenda Part 1：Job Creation Part 2：Job Execution Patterns for Creating Jobs from Web App
17. Used by the WebApp to create Jobs. After 1 – ConfigMap Job Template Design Pattern
18. Used by the WebApp to create Jobs. ConfigMap Inside: Job Template 1 – ConfigMap Job Template Design Pattern Zero App Code Changes No app rebuild or redeployment — just YAML.
19. Used by the WebApp to create Jobs. Before After Add 1 – ConfigMap Job Template Design Pattern Zero App Code Changes No app rebuild or redeployment — just YAML. Dynamic env switching Switch resources for dev, test, and prod environments. ConfigMap Inside: Job Template
20. Next.js(Web app) @kubernetes/client-node Official TypeScript Client Kubernetes API 2 – Dynamic Job Creation Pattern
21. Get: Job template from a ConfigMap Retrieve a Job template stored in a ConfigMap using @kubernetes/client-node. 1 2 – Dynamic Job Creation Pattern
22. Get: Job template from a ConfigMap Retrieve a Job template stored in a ConfigMap using @kubernetes/client-node. 1 Inject: app-specific values Replace placeholders at runtime using @ctrl/golang-template in Job templates. 2 2 – Dynamic Job Creation Pattern
23. Get: Job template from a ConfigMap Retrieve a Job template stored in a ConfigMap using @kubernetes/client-node. 1 Inject: app-specific values Replace placeholders at runtime using @ctrl/golang-template in Job templates. 2 Create: Job Create a Kubernetes Job using @kubernetes/client-node. 3 2 – Dynamic Job Creation Pattern
24. Wrap-up Problem Always-On Over-Provisioning Excessive CPU/memory allocation for temporary high loads.
25. Wrap-up Problem Always-On Over-Provisioning Excessive CPU/memory allocation for temporary high loads. Solution Offload heavy processing to Kubernetes Jobs. Need more CPU? Scale with a Job — just in time.
26. Wrap-up Points Problem Always-On Over-Provisioning Excessive CPU/memory allocation for temporary high loads. Solution Offload heavy processing to Kubernetes Jobs. Need more CPU? Scale with a Job — just in time.
27. Wrap-up Points Problem Always-On Over-Provisioning Excessive CPU/memory allocation for temporary high loads. Solution Cost Optimization Resources are allocated only when needed, reducing idle capacity costs. Offload heavy processing to Kubernetes Jobs. Need more CPU? Scale with a Job — just in time.
28. Wrap-up Points Problem Always-On Over-Provisioning Excessive CPU/memory allocation for temporary high loads. Solution Cost Optimization Resources are allocated only when needed, reducing idle capacity costs. Stability Built-in retry and parallelization with Kubernetes standard features only. Offload heavy processing to Kubernetes Jobs. Need more CPU? Scale with a Job — just in time.
29. Wrap-up Points Problem Always-On Over-Provisioning Excessive CPU/memory allocation for temporary high loads. Solution Cost Optimization Resources are allocated only when needed, reducing idle capacity costs. Stability Built-in retry and parallelization with Kubernetes standard features only. Operational Efficiency Flexible configuration using ConfigMaps. Offload heavy processing to Kubernetes Jobs. Need more CPU? Scale with a Job — just in time.
30. Thank you Sample Code The potential of Jobs is unlimited! https://github.com/asami-okina/ quick-container-image-scan