job4j Overview

Framework for java batch job & workflow

Transcript

1. job4j: Framework for batch job & workflow Smrutiranjan Sahu Arindam Biswas 09-10-2014 Job4j Overview
2. • Introduction ▫ Why job4j • Architecture • Usage Guide ▫ How to create job in job4j ▫ How to create workflow in job4j ▫ Demo • Technology Used • Future Works • Q&A What We’ll Cover Today
3. • Create Job & Workflows easily • Easy Scalability • Multiple Implementation on same platform Why job4j
4. • Easy Batch + Lightweight + Reader, Filter, Mapper, Validator + Reporting - Flow - Scheduling - Monitoring (JMX) - Recoverable - Listener • Jbatch(JSR-352) + Flow + Retry/Skip Error + Listener - Heavy for simple task - Learning Curve - Scheduling - Same thread Read+Process+ Write Why job4j – Comparison of Current Options • job4j + Lightweight + Easy Configuration + Flow + Scheduling + Monitoring - Inbuilt DB/File reader/writer - Retry/Skip on Error - Recoverable - Reporting - Listener • Spring Batch + JSR-352 + Reader + Remoting + Feature Rich - JSR-352 - Step to Step data flow by JobContext
5. Job Architecture Input Producer Output Consumer Task 1 Task 2 Task 3 Task n Job DB File DB File
6. QueueJob Architecture Input Producer Output Consumer Task 1 Task 2 Task 3 Task n Job Input Queue Output Queue Thread Pool Executor
7. Job 1 Job 2 Job n Job Flow Architecture Job Group1 Job 3 Job 4 Job n Job Group2 Job 5 Job 6 Job n Job Group3 Job Flow Pipe Pipe
8. public class Job1InputProducer implements InputProducer<A> { public A produceInput() { return new A(); } } public class Job1Task implements Task<A, B> { public B processTask(A a) throws Exception { return convertAtoB(a); } } public class Job1OutputConsumer implements OutputConsumer<B>{ public void consumeOutput(B b, int counter) { logger.debug("Consuming >> "+b); } } Usage Guide – Job (Code)
9. <bean id="job1InputProducer" class="com.talentica.job4j.demo.job1.Job1InputProducer"/> <bean id="job1Task" class="com.talentica.job4j.demo.job1.Job1Task" scope="prototype"/> <bean id="job1OutputConsumer" class="com.talentica.job4j.demo.job1.Job1OutputConsumer"/> <bean id="job1" class="com.talentica.job4j.impl.queue.QueueJob"> <property name="name" value="Job1" /> <property name="startCronSchedule" value="0 10 * * *" /> <property name="stopCronSchedule" value="55 16 * * *" /> <property name="timeZone" value="IST" /> <property name="inputProducer" ref="job1InputProducer" /> <property name="task" ref="job1Task" /> <property name="outputConsumer" ref="job1OutputConsumer" /> <property name="maxThreadCount" value="5" /> <property name="threadSleepTime" value="4000" /> </bean> Usage Guide – Job (Config)
10. <bean id="jobGroup1" class="com.talentica.job4j.model.JobGroup"> <property name="jobList"> <list> <ref bean="job1"/> <ref bean="job2"/> </list> </property> </bean> <bean id="jobGroup2" class="com.talentica.job4j.model.JobGroup"> <property name="jobList"> <list> <ref bean="job3"/> </list> </property> </bean> <bean id="jobFlow1" class="com.talentica.job4j.model.JobFlow"> <property name="jobGroupList"> <list> <ref bean="jobGroup1"/> <ref bean="jobGroup2"/> </list> </property> <property name="startCronSchedule" value="0 10 * * *" /> <property name="stopCronSchedule" value="55 16 * * *" /> <property name="timeZone" value="IST" /> </bean> Usage Guide – Workflow (Config)
11. Demo
12. • Spring Context • Spring Web • Cron4j Technology Used
13. • Configure Job via DB • Custom job4j XML Configuration (For non spring projects) • Fork Join Implementation • Map Reduce Implementation Future Enhancements
14. • https://github.com/smsahu/job4j/ • https://github.com/smsahu/job4j-demo • spring-batch-vs-jsr-352 • spring-batch-vs-easy-batch Reference Links
16. job4j: Framework for batch job & workflow