Upgrade to Pro — share decks privately, control downloads, hide ads and more …

akeneo/batch-bundle in a nutshell

akeneo/batch-bundle in a nutshell

Julien Janvier

December 08, 2016
Tweet

More Decks by Julien Janvier

Other Decks in Programming

Transcript

  1. Tool to be able to process large volume of data

    We want to a “read” large amount of data, process/compute them, and “write” the result somewhere. • But PHP is not made for that oO… • Do not duplicate our business code :) “ 2
  2. Job examples We want to daily import 100K products from

    a CSV file. We want to export our products via a SOAP webservice. We want to transform a large CSV file to a JSON file. “ “ “ 3
  3. Job instances code label job_name type raw_parameters code label job_name

    type raw_parameters csv_product_import_erp CSV product import ERP csv_product_import import {“filePath”:”\/tmp\/products_erp.csv”,”delimiter”,”;”} csv_product_import_gibson CSV product import Gibson csv_product_import import {“filePath”:”\/tmp\/products_gibson.csv”,”delimiter”:”;”,”enclosure”:”"”} csv_product_export CSV product export csv_product_export export {“filePath”:”\/tmp\/products.csv”} edit_common_attributes Mass edit common product attributes edit_common_attributes mass_edit {} • create a new job instance p h p a p p / c o n s o l e a k e n e o : b a t c h : c r e a t e - j o b . . . 4
  4. Job execution • trace each execution of the job •

    various information: start time, end time, status… 5
  5. Job parameters • configure how the job is launched •

    possible to create a custom class with constraints and default values • launch a job with with a custom configuration with p h p a p p / c o n s o l e a k e n e o : b a t c h : j o b c s v _ p r o d u c t _ i m p o r t _ g i b s o n - - c o n f i g " { \ " f i l e P a t h \ " : \ " / a n o t h e r / p a t h / t o / p r o d u c t . c s v \ " } " 6
  6. Steps • where your business code lives • each job

    has 1 to n steps • implement A k e n e o \ C o m p o n e n t \ B a t c h \ S t e p \ S t e p I n t e r f a c e • possible to pass information between steps • a particular step A k e n e o \ C o m p o n e n t \ B a t c h \ S t e p \ I t e m S t e p • reader • processor • writer 7
  7. What to do to use it? 1. implement A k

    e n e o \ C o m p o n e n t \ B a t c h \ S t e p \ S t e p I n t e r f a c e 2. define the job as a tagged service a k e n e o _ b a t c h . j o b 3. [define a class for the configuration] 8
  8. i n t e r f a c e S

    t e p I n t e r f a c e { p u b l i c f u n c t i o n g e t N a m e ( ) ; / * * * P r o c e s s t h e s t e p a n d a s s i g n p r o g r e s s a n d s t a t u s m e t a i n f o r m a t i o n t o t h e * S t e p E x e c u t i o n p r o v i d e d . T h e S t e p i s r e s p o n s i b l e f o r s e t t i n g t h e m e t a * i n f o r m a t i o n a n d a l s o s a v i n g i t i f r e q u i r e d b y t h e i m p l e m e n t a t i o n . * * @ p a r a m S t e p E x e c u t i o n $ s t e p E x e c u t i o n a n e n t i t y r e p r e s e n t i n g t h e s t e p t o b e e x * * @ t h r o w s J o b I n t e r r u p t e d E x c e p t i o n i f t h e s t e p i s i n t e r r u p t e d e x t e r n a l l y * / p u b l i c f u n c t i o n e x e c u t e ( S t e p E x e c u t i o n $ s t e p E x e c u t i o n ) ; 0 1 . 0 2 . 0 3 . 0 4 . 0 5 . 0 6 . 0 7 . 0 8 . 0 9 . 1 0 . 1 1 . 1 2 . 1 3 . 1 4 . 9
  9. a c m e . j o b . c

    s v _ p r o d u c t _ i m p o r t : c l a s s : ' A k e n e o \ C o m p o n e n t \ B a t c h \ J o b \ J o b ' a r g u m e n t s : - ' c s v _ p r o d u c t _ i m p o r t ' - ' @ e v e n t _ d i s p a t c h e r ' - ' @ a k e n e o _ b a t c h . j o b _ r e p o s i t o r y ' - - ' @ a c m e . j o b . s t e p . f i r s t ' - ' @ a c m e . j o b . s t e p . s e c o n d ' t a g s : - { n a m e : a k e n e o _ b a t c h . j o b } 0 1 . 0 2 . 0 3 . 0 4 . 0 5 . 0 6 . 0 7 . 0 8 . 0 9 . 1 0 . 1 1 . 10
  10. Misc • possible to send a mail when the job

    is finished • events are dispatched • invalid items are handled • opensource - licence OSL 3.0 • installation via c o m p o s e r r e q u i r e a k e n e o / b a t c h - b u n d l e • documentation at docs.akeneo.com 11