Qudoを使ったJob queue system

Transcript

1. 2VEPΛ࢖ͬͨ KPC
   RVFVF
   TZTUFN id:shiba_yu36 2012/03/17 Kyoto.pm Tech Talks #01 2012೥3݄18೔೔༵೔

2. ࣗݾ঺հ 2012೥3݄18೔೔༵೔

3. TIJCB@ZV TIJCBZV PS
   FUD  2012೥3݄18೔೔༵೔

4. !גࣜձࣾ͸ͯͳ • ՝ۚ΍λΠϜϥΠϯͳͲϓϥοτ ϑΥʔϜ෦෼Λ୲౰ • ֎෦αʔϏε࿈ܞ • ςετ؀ڥ੔උ 2012೥3݄18೔೔༵೔

5. QFSM w,ZPUPQN w$1"/ w8FC4FSWJDF#JUMZ w%#*Y%BUB'BDUPSZ w1SF1"/΋ͨ·ʹख఻ͬͯ·͢ 2012೥3݄18೔೔༵೔

6. 1SF1"/։ൃͰKPC
   RVFVF
   TZTUFNΛ ࢖͏ػձ͕͋ͬͨͷͰɺͦͷ࢓૊Έͱ 2VEPʹΑΔ࣮૷Λ঺հ͠·͢ɻ 2012೥3݄18೔೔༵೔

7. ΞδΣϯμ wKPC
   RVFVF
   TZTUFN w2VEPΛ࢖࣮ͬͨ૷ w࢖ͬͯΈͯ ͋·ΓKPC
   RVFVF
   TZTUFNΛ ࢖ͬͨ͜ͱͷͳ͍ਓ޲͚ 2012೥3݄18೔೔༵೔

8. 2012೥3݄18೔೔༵೔

9. KPC
   RVFVF
   TZTUFNʁ ΞϓϦέʔγϣϯͷόοΫάϥ΢ϯ υଆͰඇಉظͰॲཧΛ࣮ߦ͢ΔͨΊ ͷ࢓૊Έ 2012೥3݄18೔೔༵೔

10. ϝϦοτ wॲཧΛޙճ͠ wϦΞϧλΠϜੑ͕ඞཁͳ͍΋ͷ wΤϥʔϋϯυϦϯά wϦτϥΠ wεέʔϧ 2012೥3݄18೔೔༵೔

11. KPC
    RVFVF
    TZTUFNͷߏ੒ w+PCΛFORVFVF͢Δ෦෼ DMJFOU w+PCͷॲཧຊମ XPSLFS w+PCͷ࣮ߦͷ؅ཧ XPSLFS
    NBOBHFS 2012೥3݄18೔೔༵೔

12. Ұྫ 2012೥3݄18೔೔༵೔

13. QFSMͰͷ KPC
    RVFVF
    TZTUFN w(FBSNBO wதؒEBFNPOར༻ w5IF4DIXBSU[ w3%#.4Λ࢖͏ wଟ͘ͷ࣮੷͕͋Δ 2012೥3݄18೔೔༵೔

14. QFSMͰͷ KPC
    RVFVF
    TZTUFN w 2VEP
    OFLPLBL͞Μ
    NBTBSU[͞Μ w ܰྔ
    KPC
    RVFVF
    TZTUFN w +POL

    w ࠷௿ݶͷKPC
    RVFVF
    "1*Λ࣮૷ w $MVUDI w EJTUSJCVUFE
    KPC
    RVFVF w DMJFOU͕ॲཧ͢ΔXPSLFSΛܾఆ 2012೥3݄18೔೔༵೔

15. 2VEPΛ࢖࣮ͬͨ૷ 2012೥3݄18೔೔༵೔

16. ͳͥ2VEP w1SF1"/ͰNPEVMF͕౤ߘ͞ΕͨΒɺ͍ ͍ײ͡ʹ௨஌͍ͨ͠ wHJUIVCͷϑΥϩʔ wΩʔϫʔυʹϚονͰ௨஌ wॲཧ͕গ͠ॏ͍ͷͰɺ·ͣKPC
    RVFVF
    TZTUFNΛಋೖ͔ͨͬͨ͠ 2012೥3݄18೔೔༵೔

17. ͳͥ2VEP w5IF4DIXBSU[͸ΦʔόʔεϖοΫײ wͨ͘͞ΜΧελϚΠζ͍ͨ͠Θ͚Ͱ ΋ͳ͍ 2VEP͕ͪΐ͏ͲΑͦ͞͏ 2012೥3݄18೔೔༵೔

18. %#४උ RVEP
    ECRVEP
    VTFSSPPU
    SECNTNZTRM
    VTF@JOOPEC 2012೥3݄18೔೔༵೔

19. $MJFOU 2012೥3݄18೔೔༵೔

20. $MJFOU package Sample::Qudo::Client; use parent qw(Qudo); sub new { my

    ($class, $args) = @_; $class->SUPER::new( databases => [+{ dsn => 'dbi:mysql:qudo_sample', username => 'root', password => '', }], default_hooks => ['Qudo::Hook::Serialize::JSON'], ); } 2012೥3݄18೔೔༵೔

21. $MJFOU $qudo->enqueue( "Sample::Qudo::Worker::Echo", { arg => {message => 'hoge'} },

    ); 2012೥3݄18೔೔༵೔

22. 8PSLFS 2012೥3݄18೔೔༵೔

23. 8PSLFS package Sample::Qudo::Worker::Echo; use parent qw(Qudo::Worker); sub work { my

    ($self , $job ) = @_; my $arg = $job->arg; my $message = $arg->{message}; sleep 2; # heavy task... warn $message; $job->completed(); # or $job->abort } 2012೥3݄18೔೔༵೔

24. 8PSLFS
    .BOBHFS 2012೥3݄18೔೔༵೔

25. 8PSLFS
    .BOBHFS # worker.pl my $qudo = Qudo->new( databases => [+{

    dsn => 'dbi:mysql:qudo_sample', username => 'root', password => '', }], default_hooks => ['Qudo::Hook::Serialize::JSON'], manager_abilities => [qw/Sample::Qudo::Worker::Echo/], ); $qudo->work; 2012೥3݄18೔೔༵೔

26. 8PSLFS
    .BOBHFS QBSBMMFM my $manager = Qudo::Parallel::Manager->new( databases => ..., default_hooks

    => ..., manager_abilities => ..., work_delay => 5, max_workers => 3, min_spare_workers => 3, auto_load_worker => 1, ); $manager->run; 2012೥3݄18೔೔༵೔

27. %FNP XPSLFS
    NBOBHFS QBSBMMFM XPSLFS͸JOTFSU͞Εͨ৘ใΛXBSO͢Δ͚ͩ 2012೥3݄18೔೔༵೔

28. EFQMPZ QFSM
    XPSLFSQM 2012೥3݄18೔೔༵೔

29. EFQMPZ EBFNPOUPPMT # run.sh #!/bin/sh exec 2>&1 cd /path/Sample-Worker ||

    exit 1 exec setuidgid app /usr/bin/perl worker.pl svc -u /etc/service/Sample-Worker #start svc -d /etc/service/Sample-Worker #stop 2012೥3݄18೔೔༵೔

30. ࢖ͬͯΈͯ 2012೥3݄18೔೔༵೔

31. ಛ௃ ͱʹ͔͘؆୯ ͙͢ʹKPC
    RVFVFͷ࢓૊ΈΛ ࢖ͬͯΈ͍ͨͳΒ࠷ద 2012೥3݄18೔೔༵೔

32. ͨͩ͠ w࢖ΘΕ͍ͯΔͱ͜Ζ͸ͦ͜·Ͱଟ͘ͳ ͍ w
    
    5IF4DIXBSU[ͱ͔ w+PC؅ཧͷ࢓૊ΈࣗମΛΧελϚΠζ͠ Α͏ͱ͢Δͱ೉͍͠ w
    
    +POLͱ͔ 2012೥3݄18೔೔༵೔

33. wKPC
    RVFVF
    TZTUFNͱ͔࢖ͬͨ͜ͱແ ͍ wखܰʹKPC
    RVFVF
    TZTUFNΛ࢖ͬͯΈ ͍ͨ Ұ౓2VEPΛ࢖ͬͯΈΔͱྑ͍Ͱ͢ʂ 2012೥3݄18೔೔༵೔

34. 2012೥3݄18೔೔༵೔

35. ·ͱΊ wKPC
    RVFVF
    TZTUFNͷ؆୯ͳઆ໌ w2VEPͰͷ࣮૷ͱͦͷಛ௃ͷ঺հ wͱʹ͔͘؆୯ w৴པੑ
    
    5IF4DIXBSU[ wΧελϚΠζ
    
    +POL 2012೥3݄18೔೔༵೔