Do it Later with Resque & Delayed Job

A talk I gave at Lone Star Ruby 2012 in Austin, Texas.

Ruby isn't the fastest language on the block, so many systems are built where potentially lengthy operations are done in the background, keeping your application fast and responsive. We take a look at two frameworks for managing background queues and how they can be used effectively in Rails applications.

Transcript

1. %PJU-BUFSXJUI
   3FTRVF%FMBZFE
   +PC
   #FO4DIFJSNBO
   !TVCEJHJUBM
   IUUQCFOTDIFJSNBODPN
   Saturday, August 11, 12
   

   View Slide

2. %PJU-BUFSXJUI
   3FTRVF%FMBZFE
   +PC
   #FO4DIFJSNBO
   !TVCEJHJUBM
   IUUQCFOTDIFJSNBODPN
   ...and Sidekiq?
   Saturday, August 11, 12
   

   View Slide

3. 8FBSFIJSJOH
   3BJMTJ04
   Saturday, August 11, 12
   

   View Slide

4. OTTDSFFODBTUDPN
   Saturday, August 11, 12
   

   View Slide

5. &YQSFTTJWF
   'MFYJCMF
   .BMMFBCMF
   'VO
   3VCZ8FC
   ❤
   Saturday, August 11, 12
   

   View Slide

6. 3BX4QFFE
   .VMUJ5ISFBEJOH
   3VCZJTOPULOPXOGPS
   Saturday, August 11, 12
   

   View Slide

7. 1MFOUZPGFYBNQMFT
   #VU 3VCZBQQTDBOTDBMF
   Saturday, August 11, 12
   

   View Slide

8. 1MFOUZPGFYBNQMFT
   #VU 3VCZBQQTDBOTDBMF
   Saturday, August 11, 12
   

   View Slide

9. $BDIJOH
   .VMUJQMFXFCQSPDFTTFT
   .POHSFMT
   6OJDPSOXPSLFST
   )FSPLVEZOPT
   &UD
   %JTUSJCVUJOHUIF-PBE
   )PXEP3BJMTBQQTTDBMF
   Saturday, August 11, 12
   

   View Slide

10. )FSPLV`T%ZOP.PEFM
    Saturday, August 11, 12
    

    View Slide

11. )FSPLV`T%ZOP.PEFM
    Saturday, August 11, 12
    

    View Slide

12. ,FFQ
    3FRVFTUT
    -JHIUXFJHIU
    Saturday, August 11, 12
    

    View Slide

13. 1FSGPSN
    DPNQMFYXPSL
    JOUIF
    CBDLHSPVOE
    Saturday, August 11, 12
    

    View Slide

14. 8IBUXPSLTIPVMEXFQFSGPSNJOUIF
    CBDLHSPVOE
    Saturday, August 11, 12
    

    View Slide

15. 8IBUXPSLTIPVMEXFQFSGPSNJOUIF
    CBDLHSPVOE
    4FOEJOHFNBJMT
    $PNNVOJDBUJOHXJUISEQBSUZ"1*T
    $BMDVMBUJOHTUBUT
    3FCVJMEJOHJOEFYFT
    3FTJ[JOHQSPDFTTJOHQIPUPT
    1PTUQSPDFTTJOHPGNPEFMT
    &ODPEJOHBVEJPWJEFP
    &YUSBDUJOGPPVUPGNFEJBpMFT
    8BSNDBDIFT
    Saturday, August 11, 12
    

    View Slide

16. .BOZTPMVUJPOTGPSUIJT
    %FMBZFE+PC
    3FTRVF
    4JEF,JR
    Saturday, August 11, 12
    

    View Slide

17. %FMBZFE+PC
    IUUQTHJUIVCDPNDPMMFDUJWFJEFBEFMBZFE@KPC
    +VTUBHFN
    6TFTZPVSEBUBCBTF
    3BLFUBTLUPSVOXPSLFST
    Saturday, August 11, 12
    

    View Slide

18. # Gemfile
    gem 'rails'
    gem 'sqlite'
    gem 'delayed_job_active_record'
    Saturday, August 11, 12
    

    View Slide

19. $ rails generate delayed_job
    $ rake db:migrate
    Saturday, August 11, 12
    

    View Slide

20. class Foo
    def bar
    puts "baz"
    end
    end
    (irb) > Foo.new.delay.bar
    $BMMFSEFDJEFTBTZOD
    Saturday, August 11, 12
    

    View Slide

21. class Foo
    def bar
    puts "baz"
    end
    handle_asynchronously :bar
    end
    (irb) > Foo.new.bar
    $BMMFSEPFTO`UDBSF
    Saturday, August 11, 12
    

    View Slide

22. class MyJob
    def perform
    puts "I'm doing background work!"
    sleep 10
    end
    end
    (irb)> Delayed::Job.enqueue MyJob.new
    Saturday, August 11, 12
    

    View Slide

23. %FNP'FUDIJOH-JOL5JUMFT
    VTJOH%FMBZFE+PC
    
    Saturday, August 11, 12
    

    View Slide

24. )BOEMJOH'BJMVSFT
    'BJMVSFDPVOUTMJEJOHEFMBZ
    $POUJOVFVOUJMKPCJTEBZT
    
    
    PME
    Saturday, August 11, 12
    

    View Slide

25. 1SPTPG%+
    &BTZ
    8PSLTXJUI"3.POHPJE
    +VTUBHFN
    'BJMVSFIBOEMJOH
    Saturday, August 11, 12
    

    View Slide

26. %PXOTJEFTPG%+
    Saturday, August 11, 12
    

    View Slide

27. %PXOTJEFTPG%+
    4PNFXIBUTUBMFEFWFMPQNFOU
    6TFTZPVSBQQ`TEBUBCBTF
    5BCMFMPDLJOH
    8PSLFSRVFSJFTFYQFOTJWF
    XJUIBMBSHFRVFVF
    /PCVJMUJOEBTICPBSE
    Saturday, August 11, 12
    

    View Slide

28. 4PNFUIJOHTUPMPPLBU
    %FMBZFE+PC"ENJO
    %FMBZFE+PC .POHPJEBEBUFS
    
    Saturday, August 11, 12
    

    View Slide

29. 3FTRVF
    #BTFEPO3FEJT
    .VMUJQMFRVFVFT
    2VFVF1SJPSJUZ
    #VJMUJO%BTICPBSE
    Saturday, August 11, 12
    

    View Slide

30. 8IZ2VFVF1SJPSJUZ
    DBMDVMBUF@TUBUTRVFVF
    EFMJWFS@QVSDIBTFRVFVF
    Saturday, August 11, 12
    

    View Slide

31. +PCTJO3FTRVF
    1MBJODMBTTFT
    .VTUTFUDMBTTWBSJBCMFqueue
    .VTUJNQMFNFOUDMBTTNFUIPE
    perform
    Saturday, August 11, 12
    

    View Slide

32. %FpOJOH+PCTJO3FTRVF
    class SendNewsletterJob
    @queue = :mailer
    def self.perform(newsletter_id)
    newsletter = Newsletter.find(newsletter_id)
    User.subscribed.find_each do |user|
    UserMailer.newsletter(
    user,
    newsletter).deliver
    end
    end
    end
    Saturday, August 11, 12
    

    View Slide

33. &ORVFVJOH+PCTJO3FTRVF
    Resque.enqueue(SendNewsletterJob, 256)
    Saturday, August 11, 12
    

    View Slide

34. 3FEJT
    IUUQSFEJTJP
    ,FZWBMVFTUPSF
    'BTU
    "UPNJD
    0
    QVTIQPQ
    .BTUFSTMBWFDPOpHVSBUJPO
    Saturday, August 11, 12
    

    View Slide

35. 3FEJT$POpHVSBUJPO
    4UPSFFOUJSFTFUJONFNPSZ
    $POpHVSFTZODUPEJTL
    &WFSZXSJUF 4-08
    
    %PO`U'MVTI '"45
    
    &WFSZTFDPOE $0.130.*4&
    
    
    Saturday, August 11, 12
    

    View Slide

36. *OTUBMMJOH3FEJT
    0OB.BD
    brew install redis
    0OUIFTFSWFS 6CVOUV
    
    wget http://redis.googlecode.com/files/redis-2.4.13.tar.gz
    tar xzvf redis-2.4.13
    cd redis-2.4.13
    make
    sudo make install
    )PTUFEXJUI3FEJT5P(P
    IUUQSFEJTUPHPDPN
    Saturday, August 11, 12
    

    View Slide

37. *OTUBMMJOH3FTRVFJOB3BJMTBQQ
    (FNpMF
    gem 'resque'
    MJCUBTLTSFTRVFSBLF
    require 'resque/tasks’
    Saturday, August 11, 12
    

    View Slide

38. 3VOOJOHUIFXFCDPOTPMF
    4FQBSBUFTJOBUSBBQQ
    resque-web
    .PVOUFEJOZPVSSBJMTBQQ
    # routes.rb
    Saturday, August 11, 12
    

    View Slide

39. .PVOUJOHUIFXFCDPOTPMF
    require 'resque/server'
    YourApp::Application.routes.draw do
    mount Resque::Server.new, :at => "resque"
    ...
    end
    Saturday, August 11, 12
    

    View Slide

40. 3VOOJOHUIFXPSLFS T
    
    $ QUEUE=* rake resque:work
    Saturday, August 11, 12
    

    View Slide

41. 4FMFDUJOH2VFVFT1SJPSJUJFT
    $ QUEUE=urgent,* rake resque:work
    Saturday, August 11, 12
    

    View Slide

42. .BLFTVSF3BJMTFOWJSPONFOUJT
    MPBEFE
    $ QUEUE=* rake environment resque:work
    Saturday, August 11, 12
    

    View Slide

43. .BLFTVSF3BJMTFOWJSPONFOUJT
    MPBEFE
    # lib/tasks/resque.rake
    task "resque:setup" => :environment do
    # any other setup
    end
    Saturday, August 11, 12
    

    View Slide

44. %FNP'FUDIJOH6TFS"WBUBST
    VTJOH3FTRVF
    
    Saturday, August 11, 12
    

    View Slide

45. 1SPTPG3FTRVF
    Saturday, August 11, 12
    

    View Slide

46. 1SPTPG3FTRVF
    &BTZ
    'BTU
    /PJNQBDUPOBQQECQFSG
    "ENJOEBTICPBSE
    'MFYJCMF
    Saturday, August 11, 12
    

    View Slide

47. %SBXCBDLTPG3FTRVF
    .PSFDPNQMFY
    .PSFUIJOHTUPTFUVQ NPOJUPS
    8POLZSFQPSUJOHPO)FSPLV
    Saturday, August 11, 12
    

    View Slide

48. 5IJOHTUPMPPLBU
    3FTRVF.BJMFS
    IUUQTHJUIVCDPN[BQOBQSFTRVF@NBJMFS
    3FTRVF4DIFEVMFS
    IUUQTHJUIVCDPNCWBOEFOCPTSFTRVFTDIFEVMFS
    3FTRVF3FUSZ4VQQPSU
    IUUQTHJUIVCDPNMBOUJOTSFTRVFSFUSZ
    Saturday, August 11, 12
    

    View Slide

49. Saturday, August 11, 12
    

    View Slide

50. l8IBUJGXFDPVMESFQMBDF
    3FTRVF%+XPSLFSTXJUI
    4JEFLJRXPSLFS z
    4JEFLJR
    Saturday, August 11, 12
    

    View Slide

51. l5IBU`TBCPMETUBUFNFOUz
    Saturday, August 11, 12
    

    View Slide

52. 4JEFLJR
    .BTUFS
    1SPDFTT
    8PSLFS
    8PSLFS
    8PSLFS
    8PSLFS
    8PSLFS
    8PSLFS
    8PSLFS
    8PSLFS
    8PSLFS
    8PSLFS
    8PSLFS
    8PSLFS
    8PSLFS
    8PSLFS
    8PSLFS
    8PSLFS
    8PSLFS
    8PSLFS
    8PSLFS
    8PSLFS
    Saturday, August 11, 12
    

    View Slide

53. #ZUIFQPXFSPG$FMMVMPJE
    Saturday, August 11, 12
    

    View Slide

54. 5BLFBXBZT
    Saturday, August 11, 12
    

    View Slide

55. 5BLFBXBZT
    -PUTPGXPSLDBOCFEPOFJO
    UIFCBDLHSPVOE
    ,FFQSFRVFTUTMJHIUXFJHIU
    EFGFSBTNVDIBTQPTTJCMF
    $IPPTFZPVSPXOBEWFOUVSF
    8SJUFKPCDMBTTFTJOSVCZ
    .POJUPSZPVSKPCT
    Saturday, August 11, 12
    

    View Slide

56. 2VFTUJPOT
    #FO4DIFJSNBO
    !TVCEJHJUBM
    IUUQCFOTDIFJSNBODPN
    IUUQOTTDSFFODBTUDPN
    Saturday, August 11, 12
    

    View Slide