rails + serverengineで お手軽daemon

Transcript

1. rails + serverengineͰ

   ͓खܰdaemon
   Akira Miki
   Repro Inc.
   shinjuku.rb #29@metaps
   Sep 30, 2015
   

   View Slide

2. Akira Miki
   Repro
   @treetreeslight
   

   View Slide

3. View Slide

4. ղੳ͔Βղܾ·ͰαΫοͱ
   

   View Slide

5. About job
   

   View Slide

6. ReproͰ࢖͍ͬͯΔjob܈
   serverengine sidekiq AWS lambda
   ؂ࢹͯ͠ൃՐ੍ޚ͍ͨ͠
   ฒྻ਺Λ੍ݶ͍ͨ͠
   ฒྻ਺Λ੍ݶ͠ͳ͍
   ΨϯΨϯॲཧ͍ͨ͠
   4ͳͲͷ"84πʔϧ܈
   ͱγʔϜϨεͳ࿈ܞͯ͠
   ΨγΨγॲཧ͍ͨ͠
   

   View Slide

7. ͲΜͳ͜ͱʹ࢖ͬͯΔͷʁ
   serverengine sidekiq AWS lambda
   w ಈըͷม׵
   w ղੳσʔλΩϟογϡ
   w ఆظܾࡁ
   w QVTI༧໿ͷ؂ࢹ
   w *O"QQର৅ऀͷߋ৽
   FUDʜ
   w ϝʔϧΛૹΔ
   w ղੳσʔλΛૹΔ
   w ഑৴σʔλͷ࡞੒
   FUDʜ
   w 1VTIૹΔ
   FUDʜ
   

   View Slide

8. ͲΜͳ͜ͱʹ࢖ͬͯΔͷʁ
   serverengine sidekiq AWS lambda
   w ಈըͷม׵
   w ղੳσʔλΩϟογϡ
   w ఆظܾࡁ
   w QVTI༧໿ͷ؂ࢹ
   w *O"QQର৅ऀͷߋ৽
   FUDʜ
   w ϝʔϧΛૹΔ
   w ղੳσʔλΛૹΔ
   w ഑৴σʔλͷ࡞੒
   FUDʜ
   w 1VTIૹΔ
   FUDʜ
   

   View Slide

9. બఆܦҢΛϓϩτλΠϓ࣌୅ͷ

   ʮஏ͔ͣ͠ʙʙʙ͍ʯ

   ࣦഊஊ΋౿·͑ͯ
   

   View Slide

10. ϓϩτλΠϓͩ͠ͱΓ͋͑ͣ
    cron
    Rake script
    SQS
    SDK
    ̍෼୯ҐͩΑ͒
    

    View Slide

11. ͋ΕʁϝϞϦᷓΕͯͶʁ
    

    View Slide

12. ͦΓΌrails෼requireͯ͠rakeୟͱ

    ϝϞϦΊͬͪΌ৯͏ΑͶ!!

    

    ϓϩτλΠϓͱ͸͍͑ࡶ͗ͨ͢!!!
    

    View Slide

13. ϓϩτλΠϓͱ͸͍͑

    ͪΐͬͱ͸ؾΛ࢖͓͔ͬͯ͘
    cron
    SQS
    SDK
    Server
    Ӆ͠URL
    

    View Slide

14. CPU΍Δ͖ͩͯ͠Δɾɾɾʁ
    

    View Slide

15. ffmpegͬͯݶք·Ͱ

    CPU࢖͏͔ΒͶʂ
    

    ϓϩτλΠϓͱ͸͍͑ࡶ͗ͨ͢!!!
    

    View Slide

16. ͪΌΜͱ͢Δظ
    • railsͷmodelࢿ࢈Λ࢖͍·Θ͠
    • ffmpeg͕CPUΛ৯͍ਚ͘͢ͷͰฒྻ਺͸੍ݶͭͭ͠
    • Rails requireͷϝϞϦͷΦʔόʔϔου͸࠷খݶʹ
    • ࣗલͰdaemonϥούʔॻ͍ͯ΋ྑ͍͚Ͳɺγάφ
    ϧ੍ޚܥͱ͔ॻ͘ͷΊΜͲ͍ <- Ұ൪ίϨ
    

    View Slide

17. serverengine
    

    View Slide

18. serverengine ࢖͍΍͢!!!
    SQS
    SDK
    serverengine
    long poling
    w΍Δ͜ͱʹ߹ΘͤͯXPSLFS਺ௐ੔ͭͭ͠
    wTVQFSWJTFS਌ࢠͱ͍͏ૉఢߏ੒
    wϩά·ΘΓ΍γάφϧʹΑΔϓϩηε੍ޚ·ͰόονϦ
    

    View Slide

19. ؾΛ͚ͭͨ͜ͱ
    

    View Slide

20. Exception
    def cycle
    invoke
    end
    

    def invoke( *arguments )
    execute( *arguments )
    rescue => error
    report_exception(error)
    end
    def execute( *arguments )
    raise NotImplementedError
    end
    def report_exception(error)
    # logging, rollbar, newrelic ...
    end
    • handle͠ͳ͕Βࢭ·Βͣલਐ
    

    View Slide

21. Avoid memory leak
    • ʢrubyͷઃఆͰղܾͰ͖Δͱࢥ͏͚Ͳʣfragmentation͕
    ཷ·ΔͷͰఆظతʹڧ੍GC
    class worker
    …
    def run
    begin
    GC.stop
    cycle
    ensure
    GC.start
    end
    rescue Exception => error
    report_exception error
    end
    …
    end
    

    View Slide

22. Zombi process
    • long polingதʹdeployΛ࿈ଓ͢Δͱɺzombi͕Ͱ͖Δ
    serverengine
    serverengine.pid
    Kill
    Kill
    ̍ճ໨
    2ճ໨
    start
    start
    pollingத
    serverengine.pid
    Override
    

    View Slide

23. Memory Usage
    • serverengine͸process modelͳͷͰɺrailsશ෦৐͚ͬͯ
    Δͱϓϩηε෼ϝϞϦΛ͍ͬͺ͍৯΂Δɻ
    worker
    worker
    server
    supervisor
    

    View Slide

24. ͜Ε؂ࢹܥ͡Όͳ͍͠
    sidekiqͰ΋Α͔ͬͨΜ͡Όʁ
    

    View Slide

25. Ͱͨ͠
    serverengine sidekiq AWS lambda
    w ಈըͷม׵
    w ղੳσʔλΩϟογϡ
    w ఆظܾࡁ
    w QVTI༧໿ͷ؂ࢹ
    w *O"QQͷλʔήοτߋ
    ৽
    w ಈըͷม׵
    w ϝʔϧΛૹΔ
    w ղੳσʔλΛૹΔ
    w ഑৴σʔλͷ࡞੒
    FUDʜ
    w 1VTIૹΔ
    FUDʜ
    

    View Slide

26. ·ͱΊ
    • serverengine࢖͏ͱdaemon͕α
    ΫοͱͰ͖Δ
    

    View Slide