Cloud Batch

Deck for the Batch Processing Cloud Apps with Java

Transcript

1. BATCH PROCESSING CLOUD APPS WITH JAVA ELDER MORAES RODRIGO GRACIANO OUTUBRO 2022
2. WHAT SPRING BATCH IS? Spring Batch is a lightweight, comprehensive batch framework designed to enable the development of robust batch applications vital for the daily operations of enterprise systems. https://docs.spring.io/spring-batch/docs/current/reference/html/index-single.html#spring-batch-intro
3. WHAT SPRING BATCH IS NOT? Spring Batch is not a scheduler https://docs.spring.io/spring-batch/docs/current/reference/html/index-single.html#spring-batch-intro
4. RODRIGO GRACIANO • Principal Software Engineer - NY • Líder do JUG de NY - NYJavaSIG • graciano.dev • Twitter: @rodrigograciano
5. ELDER MORAES • Java Champion • Autor de diversos livros • Developer Advocate na Red Hat • Líder do SouJava • eldermoraes.com • Twitter: @elderjava
6. THE BASICS
7. BATCH VOCABULARY
8. BATCH COMPONENTS JOB
9. BATCH COMPONENTS STEP
10. BATCH ARCHITECTURE https://docs.spring.io/spring-batch/docs/current/reference/html/images/spring-batch-reference-model.png JobRepository JobLauncher Job Step 1 *
11. BATCH COMPONENTS TASKLET
12. TASKLETS • 1 time operation instead of read/write • Run a DB query • Copy Files • Zip/unzip files
13. CHUNK TASKLET - STEP SEQUENCE Process Step Read Write
14. CHUNK-ORIENTED STEP + PROCESSORS https://docs.spring.io/spring-batch/docs/current/reference/html/images/chunk-oriented-processing-with-item-processor.png
15. SHOW ME SOME CODE: BASIC DEMO This Photo by Unknown Author is licensed under CC BY-NC-ND
16. • JobRepository • JobLauncher • JobRegistry • JobExplorer • PlatformTransactionManager • JobBuilderFactory • StepBuilderFactory @EnableBatchProcessing
17. JOB REPOSITORY •It’s how Job, Step, JobParameters, JobExecution, and StepExecution are persisted
18. READERS • Database (JDBC, Mongo, Neo4j, Hibernate, etc.) • Flat Files (delimited, fixed-length) • XML, JSON • JMS, Kafka, Amqp • Multi-file
19. WRITERS • Database (JDBC, Mongo, Neo4j, Hibernate, etc.) • Flat Files (delimited, fixed-length) • XML, JSON • JMS, Kafka, Amqp • Multi-file • Email
20. PROCESSORS •Transforming •Filtering •Validations
21. LISTENERS JobExecutionListener StepExecutionListener ItemReadListener ItemWriteListener ChunkListener ItemProcessListener SkipListener
22. JOB EXECUTION LISTENERS
23. BEYOND THE BASICS SCALING AND PARALLEL PROCESSING
24. SCALING AND PARALLEL PROCESSING •Parallel steps •Multi-threaded •Async Processor and AsyncWriter •Remote Partitioning •Remote Chunking
25. SCALING AND PARALLEL PROCESSING •Parallel steps •Multi-threaded •Async Processor and AsyncWriter •Remote Partitioning •Remote Chunking
26. PARALLEL PROCESSING Step 1 Step 2 Step 3 Start Step 4
27. SCALING AND PARALLEL PROCESSING •Parallel steps •Multi-threaded •Async Processor and AsyncWriter •Remote Partitioning •Remote Chunking
28. SCALING AND PARALLEL PROCESSING •Parallel steps •Multi-threaded •Async Processor and AsyncWriter •Remote Partitioning •Remote Chunking
29. SCALING AND PARALLEL PROCESSING •Parallel steps •Multi-threaded •Async Processor and AsyncWriter •Remote Partitioning •Remote Chunking
31. WHAT ABOUT THE CLOUD?
32. QUESTIONS? https://github.com/rodrigolgraciano/cloud-batch.git https://speakerdeck.com/graciano/cloud-batch @rodrigograciano @ elderjava