Scaling a dormant Java app from 0 to 100+ pods in (milli)seconds? Quarkus & Knative to the rescue!

This session demonstrates how Java can effectively be used to face the challenges of the modern cloud native event driven world. Quarkus, is a fantastic new Java runtime, based on your all familiar Java standards, that brings developer joy through live coding, continuous testing, unified configs and automatic provisioning and application wiring of supporting services, and more. But what's really great about Quarkus from an operational viewpoint, is its super fast startup time. We'll show you how this is really key to being able to respond to fluctuating events where sometimes nothing is coming in, and sometimes there's a flood of requests coming in, be it from a cron job, a Kafka stream, a message Broker, http requests, etc. In the traditional world you'd have to either way overprovision for the eventuality of a burst of data and pay lots of money and waste energy to keep it running; or you could underprovision and not be able to respond to all the requests. With Knative, Quarkus and Kubernetes, this is a trivial thing to do.

Transcript

1. #voxxed_lu @kevindubois Scaling a dormant Java application from 0 to 100 pods in seconds? Quarkus and Knative to the rescue! Kevin Dubois Developer Advocate, Red Hat
2. #voxxed_lu @kevindubois Kevin Dubois • Solution Architect & Developer Advocate at Red Hat since 2019 • Previously: (Lead) Software Engineer / Architect, mostly in USA • Speak English, Dutch, French, Italian fluently… and a tiny bit of German and Spanish • On a mission to supercharge developer productivity and joy, the Open Source way Linkedin: kevindubois Github: kdubois @kevindubois
3. #voxxed_lu @kevindubois • Why should we care about scaling? • The evolution of scaling • Docker • Kubernetes + HPA • FAAS & Autoscaling • Knative • Java • Quarkus • Scaling demo • Resources
4. #voxxed_lu @kevindubois [Horizontal] Scalability matters Use only the resources you need ! >> €€ + Rolling deployments (canary | A/B | Blue-Green | shadow) Handle unpredictable/irregular amounts of traffic High availability Grow your application in a manageable way … But… your code must be able to handle concurrent instances
5. #voxxed_lu @kevindubois When you can’t scale fast enough…
6. #voxxed_lu @kevindubois OG “Scaling” Provision machines & Rsync … (?)
7. #voxxed_lu @kevindubois Containers
8. #voxxed_lu @kevindubois Scaling containers with docker/podman-compose $ docker-compose scale myservice=100 Requires manual intervention, ie. no autoscaling Deprecated! (must use docker-compose up --scale which !== ) Tedious Not very … scalable Docker Swarm does not support autoscaling either
9. #voxxed_lu @kevindubois Horizontal Pod Autoscaling (HPA) with Kubernetes # kubectl autoscale deployment myapp --min=1 --max=100 More automated By default based on CPU, relatively slow to react (polls metrics) Kubernetes 1.23+ also supports custom metrics but .. it’s complicated Does not support scale-to-zero A good solution for more ‘classic’ architectures
10. #voxxed_lu @kevindubois Serverless: scale fast from zero to n !
11. @kevindubois Serverless 11 “Serverless computing refers to the concept of building and running applications that do not require server management. It describes a finer-grained deployment model where applications, bundled as one or more functions are uploaded to a platform and then executed, scaled, and billed in response to the exact demand needed at the moment” -- CNCF Definition, https://www.cncf.io/blog/2018/02/14/cncf-takes-first-step-towards-serverless-computing/
12. #voxxed_lu @kevindubois Serverless Operational Benefits Over provisioning Time in capacity planning IT cost of idle resources Under provisioning Lost business revenue Poor quality of service More applications Direct line between IT costs & business revenue NOT Serverless with Serverless
13. #voxxed_lu @kevindubois AWS Lambda, Functions... Built around the FaaS components and other services such as API Gateways. It enabled a variety of use cases but it is far from ideal for general computing and with room for improvements. 1.0 ➔ HTTP and other few Sources ➔ Functions only ➔ Limited execution time (5 min) ➔ No orchestration ➔ Limited local development experience Serverless Containers With the advent of Kubernetes, many frameworks and solutions started to auto-scale containers. Cloud providers created offerings using managed services completely abstracting Kubernetes APIs. 1.5 ➔ Knative, KEDA, etc ➔ Kubernetes based auto-scaling ➔ Microservices and Functions ➔ Easy to debug & test locally ➔ Polyglot & Portable Integration & State The maturity and benefits of Serverless are recognized industry wide and it adds the missing parts to make pattern suitable for general purpose workloads and used on the enterprise. 2.0 ➔ Basic state handling ➔ Enterprise Integration Patterns ➔ Advanced Messaging Capabilities ➔ Blended with your PaaS ➔ Enterprise-ready event sources ➔ Solutions and outcome focused Serverless is still evolving...
14. #voxxed_lu @kevindubois Immutable revisions Deploy new features: performing canary, A/B or blue-green testing with gradual traffic rollout with no sweat and following best practices. No need to configure number of replicas, or idling. Scale to zero when not in use, auto scale to thousands during peak, with built-in reliability and fault-tolerance. Automatic scaling Ready for the Hybrid Cloud Truly portable serverless running anywhere Kubernetes runs, that is on-premises or on any public cloud. Leverage data locality and SaaS when needed. Event Driven Architectures Build loosely coupled & distributed apps connecting with a variety of built-in or third-party event sources or connectors powered by Operators. Any programming language Use any programming language or runtime of choice. From Java, Python, Go and JavaScript to Quarkus, SpringBoot or Node.js. Simplified developer experience to deploy applications/code on serverless containers abstracting infrastructure & focusing on what matters. Containers made easy prem aws azur e Knative is an Open Source, Cloud Agnostic Solution to build Serverless and Event Driven Applications Knative
15. #voxxed_lu @kevindubois
16. #voxxed_lu @kevindubois 16 Serverless Adoption 6.1% Java https://serverless.com/blog/2018-serverless-community-survey-huge-growth-usage/ 62.9% Node.js 20.8% Python 6.4% Go 6.1% Java 3.8% C#
17. #voxxed_lu @kevindubois “Traditional” Java wasn’t designed for Serverless At the expense of startup speed Rich dynamic behavior built for mutable systems Designed to be long-running Yet containers are primarily immutable Designed for Throughput At the expense of footprint ➔ Java likes to uses ALL resources it can get
18. #voxxed_lu @kevindubois What about our existing Java skills & Expertise?
19. #voxxed_lu @kevindubois 19 Supersonic. Subatomic. Java.
20. #voxxed_lu @kevindubois A stack to build Distributed Systems that : ★ Is based on Java standards ★ Moves as much as possible to build phase ★ Minimizes runtime dependencies ★ Maximizes dead code elimination ★ Enables Native Build with GraalVM / Mandrel ◦ (without having to know how to work with GraalVM!) ★ Brings Developer joy ! (more on that later)
21. #voxxed_lu @kevindubois 21 Supersonic Subatomic Java *Memory (RSS) in Megabytes, tested on a single-core machine Quarkus + Native (via GraalVM) 12 MB Quarkus + JVM (via OpenJDK) 73 MB Traditional Cloud-Native Stack 136 MB REST*
22. #voxxed_lu @kevindubois 22 Supersonic Subatomic Java Quarkus + Native (via GraalVM) 12 MB Quarkus + JVM (via OpenJDK) 73 MB Traditional Cloud-Native Stack 136 MB Quarkus + Native (via GraalVM) 0.016 Seconds Quarkus + JVM (via OpenJDK) 0.943 Seconds Traditional Cloud-Native Stack 4.3 Seconds
23. #voxxed_lu @kevindubois Serverless Operational Benefits Over provisioning Time in capacity planning IT cost of idle resources Under provisioning Lost business revenue Poor quality of service More applications Direct line between IT costs & business revenue NOT Serverless with Serverless
24. #voxxed_lu @kevindubois 0 to 100+ pods in seconds? Prove it!
25. #voxxed_lu @kevindubois 25 A cohesive platform for optimized developer joy: • Live reload in the blink of an eye • A toolkit, not necessarily a framework • Unified configuration • Streamlined code for the 80% common usages, flexible for the 20% • No hassle native executable generation • Many developer-friendly features: Dev services, Panache; simplified logging; IDE plugins; Kubernetes & Openshift Extensions, etc Quarkus brings Developer Joy
26. #voxxed_lu @kevindubois 26 Unifies Imperative and Reactive • Combine both Reactive and imperative development in the same application • Use the technology that fits your use-case • Key for reactive systems based on event driven apps @Inject SayService say; @GET @Produces(MediaType.TEXT_PLAIN) public String hello() { return say.hello(); } @Inject @Channel(”kafka”) Publisher<String> reactiveSay; @GET @Produces(MediaType.SERVER_SENT_EVENTS) public Publisher<String> stream() { return reactiveSay; }
27. #voxxed_lu @kevindubois 27 Best of Breed Frameworks & Standards Quarkus provides a cohesive, fun to use, full-stack framework by leveraging a growing list of over fifty best-of-breed libraries that you love and use. All wired on a standard backbone.
28. #voxxed_lu @kevindubois Intégration à l'ère du cloud avec Camel Quarkus Alexandre Gallice, Red Hat In the AmigaOS room right after this talk!
29. #voxxed_lu @kevindubois developers.redhat.com/developer-sandbox github.com/kdubois/quarkus-knative
30. #voxxed_lu @kevindubois Thank you!