Cloud Batch

Deck for the Batch Processing Cloud Apps with Java

Transcript

1. BATCH
   PROCESSING
   CLOUD APPS
   WITH JAVA
   ELDER MORAES
   RODRIGO GRACIANO
   OUTUBRO 2022
   

   View Slide

2. WHAT SPRING BATCH IS?
   Spring Batch is a lightweight, comprehensive batch framework designed to enable
   the development of robust batch applications vital for the daily operations of
   enterprise systems.
   https://docs.spring.io/spring-batch/docs/current/reference/html/index-single.html#spring-batch-intro
   

   View Slide

3. WHAT SPRING BATCH IS NOT?
   Spring Batch is not a scheduler
   https://docs.spring.io/spring-batch/docs/current/reference/html/index-single.html#spring-batch-intro
   

   View Slide

4. RODRIGO GRACIANO
   • Principal Software Engineer - NY
   • Líder do JUG de NY - NYJavaSIG
   • graciano.dev
   • Twitter: @rodrigograciano
   

   View Slide

5. ELDER MORAES
   • Java Champion
   • Autor de diversos livros
   • Developer Advocate na Red Hat
   • Líder do SouJava
   • eldermoraes.com
   • Twitter: @elderjava
   

   View Slide

6. THE BASICS
   

   View Slide

7. BATCH VOCABULARY
   

   View Slide

8. BATCH COMPONENTS
   JOB
   

   View Slide

9. BATCH COMPONENTS
   STEP
   

   View Slide

10. BATCH ARCHITECTURE
    https://docs.spring.io/spring-batch/docs/current/reference/html/images/spring-batch-reference-model.png
    JobRepository
    JobLauncher Job Step
    1 *
    

    View Slide

11. BATCH COMPONENTS
    TASKLET
    

    View Slide

12. TASKLETS
    • 1 time operation instead of read/write
    • Run a DB query
    • Copy Files
    • Zip/unzip files
    

    View Slide

13. CHUNK TASKLET - STEP SEQUENCE
    Process
    Step
    Read Write
    

    View Slide

14. CHUNK-ORIENTED STEP + PROCESSORS
    https://docs.spring.io/spring-batch/docs/current/reference/html/images/chunk-oriented-processing-with-item-processor.png
    

    View Slide

15. SHOW ME SOME
    CODE:
    BASIC DEMO
    This Photo by Unknown Author is licensed under CC BY-NC-ND
    

    View Slide

16. • JobRepository
    • JobLauncher
    • JobRegistry
    • JobExplorer
    • PlatformTransactionManager
    • JobBuilderFactory
    • StepBuilderFactory
    @EnableBatchProcessing
    

    View Slide

17. JOB REPOSITORY
    •It’s how Job, Step, JobParameters,
    JobExecution, and StepExecution are persisted
    

    View Slide

18. READERS
    • Database (JDBC, Mongo, Neo4j, Hibernate, etc.)
    • Flat Files (delimited, fixed-length)
    • XML, JSON
    • JMS, Kafka, Amqp
    • Multi-file
    

    View Slide

19. WRITERS
    • Database (JDBC, Mongo, Neo4j, Hibernate, etc.)
    • Flat Files (delimited, fixed-length)
    • XML, JSON
    • JMS, Kafka, Amqp
    • Multi-file
    • Email
    

    View Slide

20. PROCESSORS
    •Transforming
    •Filtering
    •Validations
    

    View Slide

21. LISTENERS
    JobExecutionListener StepExecutionListener ItemReadListener ItemWriteListener
    ChunkListener ItemProcessListener SkipListener
    

    View Slide

22. JOB EXECUTION LISTENERS
    

    View Slide

23. BEYOND THE BASICS
    SCALING AND PARALLEL
    PROCESSING
    

    View Slide

24. SCALING AND PARALLEL PROCESSING
    •Parallel steps
    •Multi-threaded
    •Async Processor and AsyncWriter
    •Remote Partitioning
    •Remote Chunking
    

    View Slide

25. SCALING AND PARALLEL PROCESSING
    •Parallel steps
    •Multi-threaded
    •Async Processor and AsyncWriter
    •Remote Partitioning
    •Remote Chunking
    

    View Slide

26. PARALLEL PROCESSING
    Step
    1
    Step
    2
    Step
    3
    Start
    Step
    4
    

    View Slide

27. SCALING AND PARALLEL PROCESSING
    •Parallel steps
    •Multi-threaded
    •Async Processor and AsyncWriter
    •Remote Partitioning
    •Remote Chunking
    

    View Slide

28. SCALING AND PARALLEL PROCESSING
    •Parallel steps
    •Multi-threaded
    •Async Processor and AsyncWriter
    •Remote Partitioning
    •Remote Chunking
    

    View Slide

29. SCALING AND PARALLEL PROCESSING
    •Parallel steps
    •Multi-threaded
    •Async Processor and AsyncWriter
    •Remote Partitioning
    •Remote Chunking
    

    View Slide

30. View Slide

31. WHAT
    ABOUT THE
    CLOUD?
    

    View Slide

32. QUESTIONS?
    https://github.com/rodrigolgraciano/cloud-batch.git
    https://speakerdeck.com/graciano/cloud-batch
    @rodrigograciano @ elderjava
    

    View Slide