DevNation - Connecting disparate systems in a lightweight way

Have you ever struggled with connecting systems that were not designed to communicate with each other? This is a common challenge faced by many microservices architectures today. In this talk, we’ll explore some of the tools and approaches that can help you break down these silos and integrate your microservices with external systems seamlessly, allowing for faster development cycles and more efficient integration.

We’ll start by introducing Apache Camel, a powerful tool that offers over 300 connectors to transfer data between a wide variety of systems. With its rich set of routing, filtering, and transformation capabilities, Camel is a versatile system integration tool that can help you integrate your systems more efficiently.

Next, we’ll dive into developing a Java application that can seamlessly connect various systems using Apache Camel. With just a few lines of code, we can utilize the power of Camel to transform and transfer data between different systems. Furthermore, we’ll explore how Camel JBang and Quarkus can greatly improve the developer experience.

By the end of this talk, you’ll have the tools and knowledge to connect your Java microservices with external systems faster and more efficiently.

Transcript

1. Zineb Bendhiba D A Y: Modern App Dev December 12, 2023 Connecting disparate systems in a lightweight way
2. Integration Challenges 2
3. About me 3 Zineb Bendhiba ▸ Senior Software Engineer ▸ Apache Camel PMC ▸ International Speaker ▸ 15+ years professional software development experience ▸ Speak English, French, Moroccan Darija, Arabic ▸ https://zinebbendhiba.com ▸ @ZinebBendhiba
4. Challenge #1 Connecting to disparate systems 4
5. Challenge #1 : Connecting to disparate systems 5
6. Challenge #1 : Connecting to disparate systems 6
7. Challenge #1 : Connecting to disparate systems 7
8. Challenge #1 : Connecting to disparate systems 8
9. Challenge #1 : Connecting to disparate systems 9
10. Challenge #1 : Connecting to disparate systems 10
11. Challenge #1 : Connecting to disparate systems 11
12. Challenge #1 : Connecting to disparate systems 12
13. Apache Camel can set you free 13
14. Challenge #1 : Connecting to disparate systems 14
15. What is Apache Camel ? 15
16. Apache Camel 16 Apache Camel is an Open Source Integration Framework
17. Apache Camel 17 Entreprise patterns Emergent “Design patterns” best practices
19. Apache Camel 19 Apache Camel: Domain Specific Language (DSL) from("aws2-s3:bucketName") .to("http:my-host/api/path") Libraries POJO Biz Logic...
20. Apache Camel 20 Apache Camel: Domain Specific Language (DSL) from("aws2-s3:bucketName") .to("http:my-host/api/path")
21. Apache Camel 21 Apache Camel: Domain Specific Language (DSL) from("kafka:topicName") .to("ftp:host:port/directoryNamel") Libraries POJO Biz Logic... from("kafka:topicName") .to("aws2-s3://bucketName")
22. Challenge #2 Data Transformation 22
23. Challenge #2: Data Transformation 23 Using Apache Camel Data formats from("kafka:topic") .unmarshal().json() .to("http:my-host/api/path");
24. Challenge #2: Data Transformation 24 Using Translator or Set Body from("direct:cheese") .setBody(simple("Hello ${body}")) .to("log:hello"); from("direct:cheese") .transform(new DataType("myDataType")) .to("log:hello");
25. Challenge #2: Data Transformation 25 Using template based components from("direct:cheese") .log("Received XML: ${body}") .to("xslt:classpath:xslt/transform.xsl") .log("Transformed XML: ${body}");
26. Challenge #2: Data Transformation 26 Using Processor from("activemq:myQueue") .process(new Processor() { public void process(Exchange exchange) throws Exception{ String payload = exchange.getIn().getBody(String.class); // do something with the payload and/or exchange here exchange.getIn().setBody("Changed body"); } }) .to("activemq:myOtherQueue");
27. Challenge #2: Data Transformation 27 Using Beans package com.foo; public class MyBean { public String saySomething(String input){ return "Hello " + input; } } from("direct:hello") .to("bean:com.foo.MyBean");
28. Challenge #2: Data Transformation 28 Content Enricher from("seda:a") .to("direct:myEnrichEndpoint") .to("seda:b");
29. Challenge #3 Routing Messages 29
30. Challenge #3: Routing Messages 30 Content Based Router from("seda:a").choice() .when(header("foo").isEqualTo("bar")).to("seda:b") .when(header("foo").isEqualTo("cheese")).to("seda:c") .otherwise().to("seda:d");
31. Challenge #3: Routing Messages 31 Message Filter from("seda:a") .filter(header("foo").isEqualTo("bar")).to("seda:b");
32. Challenge #3: Routing Messages 32 Recipient List from("seda:a") .to("seda:b", "seda:c", "seda:d");
33. Challenge #3: Routing Messages 33 Load Balancer from("direct:start") .loadBalance().roundRobin() .to("mock:x", "mock:y", "mock:z");
34. Challenge #3: Routing Messages 34 Idempotent Receiver from("direct:performInsert") .idempotentConsumer(header("id")).idempotentRepository("insertDbIdemRepo") // once-only insert into database .end();
35. Challenge #3: Routing Messages 35 Using others EIPs
36. Demo #1 36
37. Demo #1 37 Demo #1
38. Additional challenges 38
39. Challenge #4 Logging 39
40. Challenge #4: Logging 40 Logging ▸ Logging Components ▸ Log EIP ▸ Customizing logs and format ▸ Masking sensitive information from("activemq:orders") .to("log:com.mycompany.order?level=DEBUG&groupSize=10") .to("bean:processOrder"); from("direct:start") .log("Processing ${id}") .to("bean:foo");
41. Challenge #5 Error Handling 41
42. Challenge #5: Error Handling 42 Error Handler: Dead Letter Queue from("direct:start") .to("direct:invalidEndpoint"); // This will trigger an error // Configure the Dead Letter Channel (DLC) to handle errors errorHandler(deadLetterChannel("direct:errorQueue") .maximumRedeliveries(3) // Maximum number of redelivery attempts .redeliveryDelay(1000) // Delay between redelivery attempts .logExhausted(true) // Log if redelivery attempts are exhausted ); // Define the DLC route to handle failed messages from("direct:errorQueue") …;
43. Demo #2 43
44. Demo #2 44 Demo #2
45. linkedin.com/company/red-hat youtube.com/user/RedHatVideos facebook.com/redhatinc twitter.com/RedHat Thank you 45