Upgrade to Pro — share decks privately, control downloads, hide ads and more …

JAX 2014 Ein Standard für die Batch-Entwicklung - JSR-352

JAX 2014 Ein Standard für die Batch-Entwicklung - JSR-352

German presentation about the new Java standard for batch processing: JSR-352, presented at JAX 2014.

tobiasflohre

May 14, 2014
Tweet

More Decks by tobiasflohre

Other Decks in Programming

Transcript

  1. codecentric AG AGENDA  Grundlagen JSR-352  Properties, Job-Parameter, Restart,

    Skip, Retry und Listener  Parallelisierung  Implementierungen und Betriebsoptionen
  2. codecentric AG AGENDA  Grundlagen JSR-352  Properties, Job-Parameter, Restart,

    Skip, Retry und Listener  Parallelisierung  Implementierungen und Betriebsoptionen
  3. codecentric AG BATCH HISTORIE JSR-352/JEE7 Java Batch 2009 2000 90er

    80er 2008 Mainframe Batch: COBOL, PL/1 WebSphere XD Compute Grid (BDS Framework) Spring Batch v1 2013  70/80er: Cobol, PL1  Java Batch: Gott Objekte, Spaghetticode, Java main()  IBM WebSphere XD Compute Grid, Batch Datastream Framework (BDS) (2007/2008)  Spring Batch 1.0 (April 2008)  Spring Batch 2.0 (April 2009)  JSR-352 Final Release Specification (Mai 2013) Spring Batch v2 Spring Batch v3
  4. codecentric AG JSR-352 – HISTORIE 11.05.14 7 IBM Chris Vignola

    2011 – Formierung Expert Group JSR-352 VMWare / Pivotal Michael Minella (Spring Batch Lead) Wayne Lund 24.05.2013 – Final Release JSR-352 28.05.2013 – Final Release JEE 7 Apache BatchEE Spring Batch
  5. codecentric AG  Job wird in XML erstellt (Job-XML) 

    Domain Specific Language manifestiert sich in XML- Namespace  Zentrale Elemente  job  step  chunk  reader  processor  writer  item-count  Ablage unter META-INF/batch-jobs DOMAIN / KONFIGURATION / ABLAUF 11.05.14 10 <job id="myJob" > <step id="myStep" > <chunk item-count="2" > <reader ref="myReader" /> <processor ref="myProcessor" /> <writer ref="myWriter" /> </chunk> </step> </job>
  6. codecentric AG  Reader, Processor und Writer implementieren bestimmte Interfaces

     ItemReader  Object readItem()  ItemProcessor  Object processItem(Object item)  ItemWriter  void writeItems(List<Object> items)  Implementierungen nicht Bestandteil der Spec DOMAIN / KONFIGURATION / ABLAUF 11.05.14 11
  7. codecentric AG DOMAIN / KONFIGURATION / ABLAUF 11.05.14 12 Begin

    Step Commit transaction Finish Step 3 item == null item == null list of items item == null at (1) item == null at (1) true false Open transaction ItemReader Item readItem() ItemReader Item readItem() ItemProcessor processItem (Item) ItemProcessor processItem (Item) ItemWriter writeItems (List<Item>) ItemWriter writeItems (List<Item>) 1 2 item-count reached item-count reached
  8. codecentric AG * * * * DOMAIN / KONFIGURATION /

    ABLAUF Step Step Job Job JobInstance JobInstance JobExecution JobExecution StepExecution StepExecution Inkasso Kraftfahrt Inkasso Kraftfahrt am 22.03.13 Inkasso Kraftfahrt am 22.03.13 erster Versuch
  9. codecentric AG  Infrastrukturkomponenten DOMAIN / KONFIGURATION / ABLAUF 11.05.14

    14 JobOperator JobOperator BatchRuntime BatchRuntime public static JobOperator getJobOperator() public long start(String jobXMLName, Properties jobParameters) public void stop(long executionId) public List<Long> getRunningExecutions(String jobName) public JobExecution getJobExecution(long executionId) public Set<String> getJobNames() public List<JobInstance> getJobInstances(String jobName, int start, int count)
  10. codecentric AG AGENDA  Grundlagen JSR-352  Properties, Job-Parameter, Restart,

    Skip, Retry und Listener  Parallelisierung  Implementierungen und Betriebsoptionen
  11. codecentric AG PROPERTIES UND JOB-PARAMETER <job id="job1"> <properties> <property name="base"

    value=".txt"/> </properties> <step id="step1"> <chunk> <properties> <property name="file" value="myfile#{jobProperties['base']}"/> </properties> </chunk> </step> </job> @Inject @BatchProperty(name="file") private String fileName;
  12. codecentric AG PROPERTIES UND JOB-PARAMETER <property name="fileName" value="#{jobParameters['fileName']}"/> <property name="fileName"

    value="#{systemProperties['fileName']}"/> <property name="fileName" value="#{partitionPlan['fileName']}"/> public long start(String jobXMLName, Properties jobParameters)
  13. codecentric AG RESTART public interface ItemReader { public void open(Serializable

    checkpoint) throws Exception; public void close() throws Exception; public Object readItem() throws Exception; public Serializable checkpointInfo() throws Exception; } public interface ItemWriter { public void open(Serializable checkpoint) throws Exception; public void close() throws Exception; public void writeItems(List<Object> items) throws Exception; public Serializable checkpointInfo() throws Exception; }
  14. codecentric AG RESTART 11.05.14 19 Begin Step Commit transaction Finish

    Step 3 Aufruf open() Aufruf open() list of items Open transaction ItemReader Item readItem() ItemReader Item readItem() ItemProcessor processItem (Item) ItemProcessor processItem (Item) ItemWriter writeItems (List<Item>) ItemWriter writeItems (List<Item>) 1 2 Aufruf checkpointInfo() Aufruf checkpointInfo() Aufruf close() Aufruf close()
  15. codecentric AG SKIP UND RETRY <chunk item-count="2" skip-limit="25"> <skippable-exception-classes> <include

    class="java.lang.Exception" /> <exclude class="java.io.IOException" /> </skippable-exception-classes> </chunk> <chunk item-count="2" retry-limit="25"> <retryable-exception-classes> <include class="java.lang.Exception" /> <exclude class="java.io.IOException" /> </retryable-exception-classes> </chunk>
  16. codecentric AG LISTENER <job id="simpleJob"> <listeners> <listener ref="myJobListener"/> </listeners> <step

    id="chunkStep"> <listeners> <listener ref="myStepListener"/> </listeners> <chunk item-count="2"> <reader ref="dummyItemReader"/> <processor ref="logItemProcessor"/> <writer ref="logItemWriter"/> </chunk> </step> </job> JobListener StepListener ChunkListener ItemReadListener ItemProcessListener ItemWriteListener SkipReadListener SkipProcessListener SkipWriteListener
  17. codecentric AG AGENDA  Grundlagen JSR-352  Properties, Job-Parameter, Restart,

    Skip, Retry und Listener  Parallelisierung  Implementierungen und Betriebsoptionen
  18. codecentric AG PARALLELISIERUNG PartitionMapper Name … … Kategori e Tobias

    … … A Dennis … … A Frank … … A Daniel … … B Ben … … B Max … … B Christia n … … C Dirk … … C Thomas … … C
  19. codecentric AG PARALLELISIERUNG PartitionMapper Name … … Kategori e Tobias

    … … A Dennis … … A Frank … … A Daniel … … B Ben … … B Max … … B Christia n … … C Dirk … … C Thomas … … C PartitionPlan kategorie=A kategorie=B kategorie=C
  20. codecentric AG PARALLELISIERUNG Commit transaction 3 Open transaction ItemReader Item

    readItem() ItemReader Item readItem() ItemProcessor processItem (Item) ItemProcessor processItem (Item) ItemWriter writeItems (List<Item>) ItemWriter writeItems (List<Item>) 1 2 Commit transaction 3 Open transaction ItemReader Item readItem() ItemReader Item readItem() ItemProcessor processItem (Item) ItemProcessor processItem (Item) ItemWriter writeItems (List<Item>) ItemWriter writeItems (List<Item>) 1 2 Commit transaction 3 Open transaction ItemReader Item readItem() ItemReader Item readItem() ItemProcessor processItem (Item) ItemProcessor processItem (Item) ItemWriter writeItems (List<Item>) ItemWriter writeItems (List<Item>) 1 2 PartitionCollector Thread 1 Thread 2 Thread 3 kategorie=A kategorie=B kategorie=C PartitionAnalyzer PartitionReducer
  21. codecentric AG AGENDA  Grundlagen JSR-352  Properties, Job-Parameter, Restart,

    Skip, Retry und Listener  Parallelisierung  Implementierungen und Betriebsoptionen
  22. codecentric AG Batch application (JSR-352) Batch application (JSR-352) Batch framework

    Batch framework Dependency injection framework Dependency injection framework Transaction manage- ment Transaction manage- ment Resour- ces Resour- ces Servlet / EJB Container Servlet / EJB Container BETRIEB EINER BATCH-ANWENDUNG
  23. codecentric AG Application Server Infra- Structure WAR / EAR Batch

    application (JSR-352) Batch application (JSR-352) App-Server‘s JSR-352 implementation App-Server‘s JSR-352 implementation CDI CDI Transaction manage- ment (JTA) Transaction manage- ment (JTA) Resour- ces Resour- ces Servlet / EJB Container Servlet / EJB Container BETRIEB EINER BATCH-ANWENDUNG
  24. codecentric AG Application Server Infra- Structure WAR / EAR Batch

    application (JSR-352) Batch application (JSR-352) App-Server‘s JSR-352 implementation App-Server‘s JSR-352 implementation CDI CDI Transaction manage- ment (JTA) Transaction manage- ment (JTA) Resour- ces Resour- ces Servlet / EJB Container Servlet / EJB Container BETRIEB EINER BATCH-ANWENDUNG Spring Batch Spring Batch BatchEE BatchEE
  25. codecentric AG Application Server Infra- structure WAR / EAR Batch

    application (JSR-352) Batch application (JSR-352) Spring Batch (JSR-352) Spring Batch (JSR-352) Spring dependency injection Spring dependency injection Transaction manage- ment (JTA) Transaction manage- ment (JTA) Resour- ces Resour- ces Servlet / EJB Container Servlet / EJB Container BETRIEB EINER BATCH-ANWENDUNG Community Testbarkeit Open Source Features: Parallelisierung, JavaConfig, Spring Data, Hadoop...
  26. codecentric AG Application Server Infra- structure WAR / EAR Batch

    application (JSR-352) Batch application (JSR-352) Spring Batch (JSR-352) Spring Batch (JSR-352) Spring dependency injection Spring dependency injection Transaction manage- ment (JTA) Transaction manage- ment (JTA) Resour- ces Resour- ces Servlet / EJB Container Servlet / EJB Container BETRIEB EINER BATCH-ANWENDUNG BatchEE BatchEE
  27. codecentric AG Spring Boot Fat JAR Batch application (JSR-352) Batch

    application (JSR-352) Transaction manage- ment (Spring) Transaction manage- ment (Spring) Resour- ces Resour- ces Servlet Container Servlet Container BETRIEB EINER BATCH-ANWENDUNG Spring Batch (JSR-352) Spring Batch (JSR-352) Spring dependency injection Spring dependency injection https://github.com/codecentric/spring-boot-starter-batch-web
  28. codecentric AG FRAGEN? Tobias Flohre codecentric AG Merscheider Straße 1

    42699 Solingen [email protected] www.codecentric.de blog.codecentric.de www.meettheexperts.de 11.05.14 33