Communicating between Applications Pooja - IBM, Developer Advocate Santhosh - TCS, Cloud Pak for Integration Consultant Intro to Enterprise Messaging & Event Streaming

https://ibm.biz/communicating-apps

Intro to Messaging MQ Overview Event Streaming Overview Real-world Use Cases Hands-on Lab: MQ

4 What do we mean by Messaging ?

Internal and Business Partner Use Only IBM Cloud Messaging is the wiring between the different parts of a business Messaging is the Wiring …securely taking data and file contents where they need to go Manufacturing Invoicing Ordering Supplies Logistics

Operations Informational Messages that represent some current or future processing. For instance: request and response messages. Messages that represent the state of the system. For instance: logging, measurements and notifications. Event Streaming Message Queuing Queuing & Event Streaming focus on different aspects of messaging © 2020 IBM Corporation

Request / Reply Event Streaming & Message Queuing Need Different Capabilities 7 Stream History Transient Data Persistence Immutable data Scalable consumption Assured Delivery Event Streaming Message Queuing ü

Intro to Messaging MQ Overview Event Streaming Overview Real-world Use Cases Hands-on Lab: MQ

9 What’s the big deal ?

10 On-premise, software and the MQ Appliance Run MQ yourself in public or private clouds Let IBM host MQ for you with its managed SaaS MQ service in public clouds, IBM Cloud and AWS Run IBM MQ in any location or cloud, exactly as you need it 11 Kubernetes AWS Linux Windows Solaris AIX IBMi IBM Z Appliance zLinux HPE NonStop Azure AWS Red Hat OpenShift IBM Cloud Private © 2020 IBM Corporation

Why Do you need MQ?

Messages Messages contain the business data or payload Messages are a distinct set of bytes exchanged between two applications All messages contain an IBM MQ message descriptor (MQMD) with control information Optionally applications can add message properties to a message Descriptor Payload (data) properties (optional)

MQ Essentials 13 1. Messages – The packets of data produced and consumed by applications. 2. Queue – An addressable location where messages are delivered and stored reliably until they need to be consumed. 3. Queue manager – This is the actual MQ engine, the server that hosts the queues. 4. Application – ‘MQ clients’ that use MQ libraries to connect to queue managers, put/get messages to and from queues.

Message Queues Decouple Applications 14 Placing a message queue between two or more applications means that the applications are not communicating directly. Whether one application goes away, or the consuming app is unable to keep up with a temporary spike in requests, message queuing provides a shock absorber that can deal with any issues on either side. As a result, adopting messaging means that applications do not have to be available at the same time, as the queue provides a neutral place where they can exchange data. This model is known as asynchronous messaging.

15

© 2019 IBM Corporation MQ Networks – Loose, interconnected queue managers, all working together to deliver messages between applications and locations 16 MQ Clusters - Tight couplings of queue managers, enabling higher levels of scaling and availability

Intro to Messaging MQ Overview Event Streaming Overview Real-world Use Cases Hands-on Lab: MQ

Apache Kafka – Brokers, Topics and partitions 18 • Kafka is a distributed messaging system consisting of a set of brokers A cluster has a minimum of 3 brokers • Each topic is a named stream of messages. • A topic is made up of one or more partitions. • The messages on a partition are ordered by a number called the offset. • Messages for a topic are maintained in the cluster for a specified retention period By having multiple partitions distributed across the brokers, the scalability of a topic is increased.

Apache Kafka - Replication 19 In order to improve availability, each topic can be replicated onto multiple brokers. For each partition, one of the brokers is the leader, and the other brokers are the followers. Replication works by the followers repeatedly fetching messages from the leader.

Apache Kafka - Producers 20 A producer publishes messages to one or more topics. You can configure your producer to prioritize speed or reliability by choosing the level of acknowledgement it receives for messages it publishes. • Fire and forget • Leader ack • All ack Speed Reliability

Apache Kafka - Consumers 21 A consumer reads messages from one or more topics and processes them. The difference between a consumer's current position and the newest message on a partition is known as offset lag. If the lag increases over time, it is a sign that the consumer is not able to keep up. Over the short term, this is not an issue but eventually the consumer could miss messages if the retention period is exceeded.

Apache Kafka - Summary 22 With the combination of scalability using partitions, availability with replication and stream history, Apache Kafka makes a great choice as the backbone of an event-driven architecture.

Intro to Messaging MQ Overview Event Streaming Overview Real-world Use Cases Hands-on Lab: MQ

Being ‘Event-Driven’ Delivers Engaging Client Experiences 2 Deliver responsive customer experiences Proactively push data from your backend systems to your APIs – before it’s required – to make your applications more responsive and the end user experience more delightful Respond to events in real time, as they happen Bring real-time intelligence to your apps Feed Machine Learning models with real-time events to make individual recommendations that evolve as circumstances change, such as fashion recommendations as the trends change… Picture TBD Picture TBD Picture TBD Keep users engaged by capturing real-time data and use it to drive live updates, such as taxi locations, parcel arrival times, order status, or to make more personal offers when it counts Get data to where it’s needed, before it’s needed

Intro to Messaging MQ Overview Event Streaming Overview Real-world Use Cases Hands-on Lab: MQ

Lab

https://developer.ibm.com/learningpaths/ibm-mq-badge/ Start here!

No content

IBM MQ: Developers Essentials Badge • Establish your core IBM MQ developer skills • Increase your awareness of key IBM MQ resources • Advertise on LinkedIn and social media • You get to do some developing! 29 https://learn.ibm.com/course/view.php?id=3603

30 https://learn.ibm.com/course/view.php?id=8767 IBM Event Streams: Developers Essentials Badge

Prerequisites https://www.youracclaim.co m Search for: Sign up for an IBMid

Check out IBM Developer https://developer.ibm.com/components/redhat-openshift-ibm-cloud/

FREE IBM Cloud Essentials course with BADGE https://cognitiveclass.ai/courses/ibm-cloud-essentials

Subscribe to our Crowdcast.io https://www.crowdcast.io/ibmdeveloper

Resources • Accelerating Modernization with Agile Integration • Introducing IBM Cloud Pak for Integration: IBM’s hybrid integration platform • MQ Playlist • MQ-dev-samples • MQ Developer Essentials Badge • What is Messaging • Messaging Patterns • Apache Kafka Apps • Kafka Fundamentals • Event Streams Developer Essentials Badge

36