Upgrade to Pro — share decks privately, control downloads, hide ads and more …

job4j Overview

job4j Overview

Framework for java batch job & workflow

828245aad8e4c95e32ba1c1272f0e0eb?s=128

Smrutiranjan Sahu

October 09, 2014
Tweet

Other Decks in Technology

Transcript

  1. job4j: Framework for batch job & workflow Smrutiranjan Sahu Arindam

    Biswas 09-10-2014 Job4j Overview
  2. • Introduction ▫ Why job4j • Architecture • Usage Guide

    ▫ How to create job in job4j ▫ How to create workflow in job4j ▫ Demo • Technology Used • Future Works • Q&A What We’ll Cover Today
  3. • Create Job & Workflows easily • Easy Scalability •

    Multiple Implementation on same platform Why job4j
  4. • Easy Batch + Lightweight + Reader, Filter, Mapper, Validator

    + Reporting - Flow - Scheduling - Monitoring (JMX) - Recoverable - Listener • Jbatch(JSR-352) + Flow + Retry/Skip Error + Listener - Heavy for simple task - Learning Curve - Scheduling - Same thread Read+Process+ Write Why job4j – Comparison of Current Options • job4j + Lightweight + Easy Configuration + Flow + Scheduling + Monitoring - Inbuilt DB/File reader/writer - Retry/Skip on Error - Recoverable - Reporting - Listener • Spring Batch + JSR-352 + Reader + Remoting + Feature Rich - JSR-352 - Step to Step data flow by JobContext
  5. Job Architecture Input Producer Output Consumer Task 1 Task 2

    Task 3 Task n Job DB File DB File
  6. QueueJob Architecture Input Producer Output Consumer Task 1 Task 2

    Task 3 Task n Job Input Queue Output Queue Thread Pool Executor
  7. Job 1 Job 2 Job n Job Flow Architecture Job

    Group1 Job 3 Job 4 Job n Job Group2 Job 5 Job 6 Job n Job Group3 Job Flow Pipe Pipe
  8. public class Job1InputProducer implements InputProducer<A> { public A produceInput() {

    return new A(); } } public class Job1Task implements Task<A, B> { public B processTask(A a) throws Exception { return convertAtoB(a); } } public class Job1OutputConsumer implements OutputConsumer<B>{ public void consumeOutput(B b, int counter) { logger.debug("Consuming >> "+b); } } Usage Guide – Job (Code)
  9. <bean id="job1InputProducer" class="com.talentica.job4j.demo.job1.Job1InputProducer"/> <bean id="job1Task" class="com.talentica.job4j.demo.job1.Job1Task" scope="prototype"/> <bean id="job1OutputConsumer" class="com.talentica.job4j.demo.job1.Job1OutputConsumer"/>

    <bean id="job1" class="com.talentica.job4j.impl.queue.QueueJob"> <property name="name" value="Job1" /> <property name="startCronSchedule" value="0 10 * * *" /> <property name="stopCronSchedule" value="55 16 * * *" /> <property name="timeZone" value="IST" /> <property name="inputProducer" ref="job1InputProducer" /> <property name="task" ref="job1Task" /> <property name="outputConsumer" ref="job1OutputConsumer" /> <property name="maxThreadCount" value="5" /> <property name="threadSleepTime" value="4000" /> </bean> Usage Guide – Job (Config)
  10. <bean id="jobGroup1" class="com.talentica.job4j.model.JobGroup"> <property name="jobList"> <list> <ref bean="job1"/> <ref bean="job2"/>

    </list> </property> </bean> <bean id="jobGroup2" class="com.talentica.job4j.model.JobGroup"> <property name="jobList"> <list> <ref bean="job3"/> </list> </property> </bean> <bean id="jobFlow1" class="com.talentica.job4j.model.JobFlow"> <property name="jobGroupList"> <list> <ref bean="jobGroup1"/> <ref bean="jobGroup2"/> </list> </property> <property name="startCronSchedule" value="0 10 * * *" /> <property name="stopCronSchedule" value="55 16 * * *" /> <property name="timeZone" value="IST" /> </bean> Usage Guide – Workflow (Config)
  11. Demo

  12. • Spring Context • Spring Web • Cron4j Technology Used

  13. • Configure Job via DB • Custom job4j XML Configuration

    (For non spring projects) • Fork Join Implementation • Map Reduce Implementation Future Enhancements
  14. • https://github.com/smsahu/job4j/ • https://github.com/smsahu/job4j-demo • spring-batch-vs-jsr-352 • spring-batch-vs-easy-batch Reference Links

  15. None
  16. job4j: Framework for batch job & workflow