Tweet
Tweet

Slide 1

Slide 1 text

BATCH PROCESSING CLOUD APPS WITH JAVA ELDER MORAES RODRIGO GRACIANO OUTUBRO 2022

Slide 2

Slide 2 text

WHAT SPRING BATCH IS? Spring Batch is a lightweight, comprehensive batch framework designed to enable the development of robust batch applications vital for the daily operations of enterprise systems. https://docs.spring.io/spring-batch/docs/current/reference/html/index-single.html#spring-batch-intro

Slide 3

Slide 3 text

WHAT SPRING BATCH IS NOT? Spring Batch is not a scheduler https://docs.spring.io/spring-batch/docs/current/reference/html/index-single.html#spring-batch-intro

Slide 4

Slide 4 text

RODRIGO GRACIANO • Principal Software Engineer - NY • Líder do JUG de NY - NYJavaSIG • graciano.dev • Twitter: @rodrigograciano

Slide 5

Slide 5 text

ELDER MORAES • Java Champion • Autor de diversos livros • Developer Advocate na Red Hat • Líder do SouJava • eldermoraes.com • Twitter: @elderjava

Slide 6

Slide 6 text

THE BASICS

Slide 7

Slide 7 text

BATCH VOCABULARY

Slide 8

Slide 8 text

BATCH COMPONENTS JOB

Slide 9

Slide 9 text

BATCH COMPONENTS STEP

Slide 10

Slide 10 text

BATCH ARCHITECTURE https://docs.spring.io/spring-batch/docs/current/reference/html/images/spring-batch-reference-model.png JobRepository JobLauncher Job Step 1 *

Slide 11

Slide 11 text

BATCH COMPONENTS TASKLET

Slide 12

Slide 12 text

TASKLETS • 1 time operation instead of read/write • Run a DB query • Copy Files • Zip/unzip files

Slide 13

Slide 13 text

CHUNK TASKLET - STEP SEQUENCE Process Step Read Write

Slide 14

Slide 14 text

CHUNK-ORIENTED STEP + PROCESSORS https://docs.spring.io/spring-batch/docs/current/reference/html/images/chunk-oriented-processing-with-item-processor.png

Slide 15

Slide 15 text

SHOW ME SOME CODE: BASIC DEMO This Photo by Unknown Author is licensed under CC BY-NC-ND

Slide 16

Slide 16 text

• JobRepository • JobLauncher • JobRegistry • JobExplorer • PlatformTransactionManager • JobBuilderFactory • StepBuilderFactory @EnableBatchProcessing

Slide 17

Slide 17 text

JOB REPOSITORY •It’s how Job, Step, JobParameters, JobExecution, and StepExecution are persisted

Slide 18

Slide 18 text

READERS • Database (JDBC, Mongo, Neo4j, Hibernate, etc.) • Flat Files (delimited, fixed-length) • XML, JSON • JMS, Kafka, Amqp • Multi-file

Slide 19

Slide 19 text

WRITERS • Database (JDBC, Mongo, Neo4j, Hibernate, etc.) • Flat Files (delimited, fixed-length) • XML, JSON • JMS, Kafka, Amqp • Multi-file • Email

Slide 20

Slide 20 text

PROCESSORS •Transforming •Filtering •Validations

Slide 21

Slide 21 text

LISTENERS JobExecutionListener StepExecutionListener ItemReadListener ItemWriteListener ChunkListener ItemProcessListener SkipListener

Slide 22

Slide 22 text

JOB EXECUTION LISTENERS

Slide 23

Slide 23 text

BEYOND THE BASICS SCALING AND PARALLEL PROCESSING

Slide 24

Slide 24 text

SCALING AND PARALLEL PROCESSING •Parallel steps •Multi-threaded •Async Processor and AsyncWriter •Remote Partitioning •Remote Chunking

Slide 25

Slide 25 text

SCALING AND PARALLEL PROCESSING •Parallel steps •Multi-threaded •Async Processor and AsyncWriter •Remote Partitioning •Remote Chunking

Slide 26

Slide 26 text

PARALLEL PROCESSING Step 1 Step 2 Step 3 Start Step 4

Slide 27

Slide 27 text

SCALING AND PARALLEL PROCESSING •Parallel steps •Multi-threaded •Async Processor and AsyncWriter •Remote Partitioning •Remote Chunking

Slide 28

Slide 28 text

SCALING AND PARALLEL PROCESSING •Parallel steps •Multi-threaded •Async Processor and AsyncWriter •Remote Partitioning •Remote Chunking

Slide 29

Slide 29 text

SCALING AND PARALLEL PROCESSING •Parallel steps •Multi-threaded •Async Processor and AsyncWriter •Remote Partitioning •Remote Chunking

Slide 30

Slide 30 text

No content

Slide 31

Slide 31 text

WHAT ABOUT THE CLOUD?

Slide 32

Slide 32 text

QUESTIONS? https://github.com/rodrigolgraciano/cloud-batch.git https://speakerdeck.com/graciano/cloud-batch @rodrigograciano @ elderjava