Batch Processing with Watermark

Transcript

1. Batch Processing with Watermark Siva K Certified Mulesoft Developer
2. Components – Batch: A batch job which is a block of code that splits messages into individual records, performs actions upon each record, then reports on the results and potentially pushes the processed output to other systems or queues. – Poll: Polls the nested message processor for the new messages. – Database: Allows mule application to perform the retrieval of large set of data from JDBC databases. – JMS: JMS endpoint allows mule applications to send and receive message to queues with JMS API.
3. Configure the batch :
4. – Configure the Poll in the input of batch. In the Poll , configure the database and query for the records. Enable the watermark and read the value from the property file and store in a variable. – watermark. date=#[groovy: new Date(System.currentTimeMillis() - 3600000).format('yyyy-MM-dd HH:mm:ss', TimeZone.getTimeZone('PST'))]. – Select MAX in the selector and define the selector expression as #[payload.ADB_TIMESTAMP].
5. Poll configuration
6. Configure database inside poll to fetch the data from the table.
7. Oracle Configuration
8. Configure the JMS and define the queue:
9. Conclusion – All the values are set in the properties file. – Run the project and you will get the results based on the watermark value set.