Upgrade to Pro — share decks privately, control downloads, hide ads and more …

A Story about Serverless Camels Living in Containers

A Story about Serverless Camels Living in Containers

When you build an application you will undoubtedly need to integrate it with other applications and systems, and transform data from one format to another. The Apache Camel project’s goal is to simplify the way we do these kinds of integration, using the book “Enterprise Integration Patterns” as the gold standard.

Apache Camel has been around for a while, but it is more relevant now than it has ever been. In this session we’ll show you how Camel can leverage Quarkus’ fast startup time and developer experience to build modern, distributed and cloud native integration workloads.
As a bonus it can make use of technologies like serverless computing (eg. Knative) and data streaming (eg. Kafka) to remove bottlenecks, integrate faster and better than ever before, and save resources and money on top of it.

Note: Camels were not hurt in the process of making this talk.

Attendees should come away from this talk with the following:

Why you would want to use Apache Camel to create easy to understand, reusable and distributed integration components.
How the next iteration of Apache Camel (based on Quarkus) can leverage Kubernetes and/or Serverless computing and event driven architectures to deploy early and often.

Kevin Dubois

July 11, 2023

More Decks by Kevin Dubois

Other Decks in Technology


  1. A Story about Serverless Camels Living in Containers Zineb Bendhiba,

    Senior Software Engineer, Red Hat Kevin Dubois, Principal Developer Advocate, Red Hat
  2. Kevin Dubois ❖ Principal Developer Advocate at Red Hat ❖

    International Speaker ❖ 18+ years professional software development experience ❖ Speak English, Dutch, French, Italian ❖ Open Source Contributor (Quarkus, Camel, Knative, ..) ❖ Open Source Community Member ➢ BeJUG, BEL CNCF chapter @[email protected] youtube.com/@thekevindubois linkedin.com/in/kevindubois github.com/kdubois @kevindubois.com
  3. Zineb Bendhiba • Senior Software Engineer at Red Hat •

    Apache Camel Committer and PMC • International Speaker • 15+ years professional software development experience • Speak English, French, Arabic, Moroccan Darija Linkedin: zbendhiba Twitter: @ZinebBendhiba Mastodon: @[email protected] Github: zbendhiba
  4. What problem needs to be resolved? Sometimes it’s a ‘MICRO-problem’,

    for instance ... • Sometimes the focus is on data/protocol transformation. • This is an integration problem. “I need this information in this specific format and protocol !”
  5. … and, some other times you have a ‘MACRO-problem’, for

    instance: • Here the focus is on interconnecting sources and destinations efficiently. ➔ This also... is an integration problem. “I want an easy and standard way to interconnect my app”
  6. 8 “Good” Integration Bespoke/Custom made Integration Integration with reusable &

    standard components • As a quick win, often a problem is initially resolved in a custom manner. • In the long run this results to be counterproductive.
  7. Apache Camel Swiss knife of integration Solve integration problem by

    applying best practices out of the box. Even with microservice architectures. Patterns 300+ Components Lightweight Runtimes Data Formats Packed with 300+ components such as databases, message queues, APIs. Quarkus, Standalone, Spring Boot,Application Servers, and natively on Cloud. Translate messages in multiple formats, and industry standard formats from finance, telco, health-care, and more
  8. Apache Camel Swiss knife of integration Simple, self-explained Domain Specific

    Language. Available in Java, YAML, XML DSLs Reactive Routing Engine Backlog Tracer/Tracer Comprehensive Tooling Back pressure model. Smooth flow control. Better thread management. Capturing a trace inside and between Camel route. For better observability Support autocomplete, correction on multiple IDE with LSP Server. Graphical data mapper. from(“kafka:topic”) .to(“grpc:endpoint”)
  9. Apache Camel Community Source: https://www.apache.org/foundation/docs/FY2021AnnualReport.pdf ➢ The biggest and most

    active community for open source integration software ➢ 600+ unique contributors ➢ 3800+ pull requests (mostly from external contributors) ➢ 15+ years of development and one of the most active Apache projects
  10. Serverless “Serverless computing refers to the concept of building and

    running applications that do not require server management. It describes a finer-grained deployment model where applications, bundled as one or more functions are uploaded to a platform and then executed, scaled, and billed in response to the exact demand needed at the moment” -- CNCF Definition, https://www.cncf.io/blog/2018/02/14/cncf-takes-first-step-towards-serverless-computing/
  11. Immutable revisions Deploy new features: performing canary, A/B or blue-green

    testing with gradual traffic rollout with no sweat and following best practices. No need to configure number of replicas, or idling. Scale to zero when not in use, auto scale to thousands during peak, with built-in reliability and fault-tolerance. Automatic scaling Ready for the Hybrid Cloud Truly portable serverless running anywhere Kubernetes runs, that is on-premises or on any public cloud. Leverage data locality and SaaS when needed. Event Driven Architectures Build loosely coupled & distributed apps connecting with a variety of built-in or third-party event sources or connectors powered by Operators. Any programming language Use any programming language or runtime of choice. From Java, Python, Go and JavaScript to Quarkus, SpringBoot or Node.js. Simplified developer experience to deploy applications/code on serverless containers abstracting infrastructure & focusing on what matters. Containers made easy prem aws azur e Knative is an Open Source, Cloud Agnostic Solution to build Serverless and Event Driven Applications Knative
  12. Eventing - Key Terminology • Event • Source / Sink

    • Propagate an event from an event source to multiple event sinks by using: ◦ Broker ◦ Channell • Trigger and Filter (Broker) • Subscription (Channel) • CloudEvent (CNCF) Technical Overview decouple relationships between event producers and event consumers Application (Knative Service) Event Sink Event Source kn source kn source create kn source list kn source list-types Cloud Event
  13. Serverless Operational Benefits Over provisioning Time in capacity planning IT

    cost of idle resources Under provisioning Lost business revenue Poor quality of service More applications Direct line between IT costs & business revenue NOT Serverless with Serverless
  14. “Traditional” Java wasn’t designed for Containers & Serverless At the

    expense of startup speed Rich dynamic behavior built for mutable systems Designed to be long-running Yet containers are primarily immutable Designed for Throughput At the expense of footprint ➔ Java likes to uses as many resources as it can get
  15. Quarkus is a Java Stack that : ★ Is based

    on Java standards ★ Moves as much as possible to build phase ★ Minimizes runtime dependencies ★ Maximizes dead code elimination ★ Enables Native Build with GraalVM / Mandrel ◦ (without having to know how to work with GraalVM!) ★ Brings Developer joy !
  16. Supersonic Subatomic Java Quarkus + Native (via GraalVM) 12 MB

    Quarkus + JVM (via OpenJDK) 73 MB Traditional Cloud-Native Stack 136 MB Quarkus + Native (via GraalVM) 0.016 Seconds Quarkus + JVM (via OpenJDK) 0.943 Seconds Traditional Cloud-Native Stack 4.3 Seconds
  17. A cohesive platform for optimized developer joy: • Quick code-build-verify

    turnaround ◦ Live reload in the blink of an eye ◦ Continuous Testing • No hassle native executable generation • Many developer-friendly features: Dev services; Hibernate Panache; simplified logging; IDE plugins; Kubernetes & Openshift Extensions, etc Quarkus brings Developer Joy
  18. @ZinebBendhiba @kevindubois #Camel #Quarkus mvn clean package -> also generates

    kubernetes/knative manifests when you add the quarkus-kubernetes extension mvn clean package -Dnative -Dquarkus.kubernetes.deploy -> deploys native binary directly to kubernetes/openshift
  19. • A platform directly running integrations on Openshift and Kubernetes

    with Serverless-ready capabilities • Architectured by Kubernetes CRDs and Operators • A community-driven project • Part of Apache Camel. Started on August 31st, 2018 What is Camel K? K
  20. Camel K 47 What does the Camel K platform has

    to offer ? Cloud Native EIP Orchestration Auto adapt to the environment (Knative, Kafka, OpenShift, Kubernetes, Service Binding, etc) Seamless Integration Connect anything - as it leverages the Apache Camel set of components - the Cloud Native way Connectivity Auto scaling (including scaling to 0) sources and sinks for event-driven serverless applications Serverless Integration Quarkus based runtime Efficiency Live Reload, Projectless Developer Joy
  21. 48 Quarkus + K = + Camel K Camel Operator

    Powered by Next Generation Building Blocks Camel K
  22. 1 source code file > kamel run roll-die.yaml web service

    roll-die.yam l • 1 CLI command Create integration file Running on OpenShift 1 Execute CLI Tools 2 3 Camel K For Developers - from: uri: "platform-http:/roll-die" steps: - setBody: simple: "roll: ${random(1,6)}”
  23. Analyze Integration Custom Resource Observe Act Serverless O↔N Camel K

    Operator Camel Developer > kamel run roll-die.js
  24. Wrap up ★ Apache Camel = an integration toolkit with

    350+ connectors ★ Camel Quarkus = developer joy + fast startup + easy integration ★ Scale from 0 and easy deployments with Knative ★ Also cool: Low/No Code with Camel K and Kamelets
  25. Start exploring in the OpenShift Sandbox. Learn containers, Kubernetes, and

    OpenShift in your browser. developers.redhat.com/developer-sandbox Try Red Hat's products and technologies without setup or configuration.