Sidekiq under the hood

Transcript

1. Hello!
   

   View Slide

2. github: davydovanton
   twitter: anton_davydov
   Anton Davydov
   

   View Slide

3. RCNTEC developer
   sidekiq commiter
   ruby doc / rails / crystal
   OSS one love <3
   

   View Slide

4. @rubyunderhood curator

   https://twitter.com/rubyunderhood
   

   View Slide

5. @moscowrb
   http://moscow-rb.org
   

   View Slide

6. You
   Beer
   

   View Slide

7. You
   Beer
   Me
   

   View Slide

8. I (we) <3 sidekiq!
   

   View Slide

9. Simple, efficient background processing for Ruby.
   

   View Slide

10. Sidekiq server
    New thread
    

    View Slide

11. I (we) <3 actors!
    

    View Slide

12. Manager actor
    

    View Slide

13. Fetcher actors
    Manager actor
    

    View Slide

14. Redis
    Fetcher actors
    Manager actor
    

    View Slide

15. Redis
    Fetcher actors Processor actors
    Manager actor
    

    View Slide

16. Redis
    Fetcher actors Processor actors
    Manager actor
    Workers
    

    View Slide

17. MyWorker.perform_async
    

    View Slide

18. irb > MyWorker.perform_async
    Redis queue
    Serialization
    prams
    

    View Slide

19. Processor actors
    Fetcher actors
    Redis
    Manager actor
    Workers
    

    View Slide

20. Processor actors
    Manager actor
    Workers
    Fetcher actors
    Redis
    

    View Slide

21. Processor actors
    Manager actor
    Workers
    Fetcher actors
    Redis
    

    View Slide

22. Workers
    Processor actors
    Manager actor
    Fetcher actors
    Redis
    

    View Slide

23. LongWorker
    Redis
    

    View Slide

24. Redis
    Serialization
    prams
    LongWorker
    After 8 seconds
    

    View Slide

25. middleware
    

    View Slide

26. Sidekiq.configure_server do |config|
    config.server_middleware do |chain|
    chain.add YourMiddleware, options
    end
    end
    

    View Slide

27. def call(worker, msg, queue)
    
    # code before call you worker
    
    yield
    
    rescue => e
    
    # run if your worker is failed
    
    raise e
    
    ensure
    
    # run in your worker passed or failed
    
    end
    
    

    View Slide

28. def call(worker, msg, queue)
    
    # code before call you worker
    
    yield
    
    rescue => e
    
    # run if your worker is failed
    
    raise e
    
    ensure
    
    # run in your worker passed or failed
    
    end
    
    

    View Slide

29. def call(worker, msg, queue)
    
    # code before call you worker
    
    yield
    
    rescue => e
    
    # run if your worker is failed
    
    raise e
    
    ensure
    
    # run in your worker passed or failed
    
    end
    
    

    View Slide

30. def call(worker, msg, queue)
    
    # code before call you worker
    
    yield
    
    rescue => e
    
    # run if your worker is failed
    
    raise e
    
    ensure
    
    # run in your worker passed or failed
    
    end
    
    

    View Slide

31. Sidekiq-statistic
    github.com/davydovanton/sidekiq-statistic
    

    View Slide

32. Improved display of statistics for
    your sidekiq workers and jobs.
    

    View Slide

33. View Slide

34. View Slide

35. View Slide

36. View Slide

37. Worker name
    Last job status
    Runtime
    Start time
    

    View Slide

38. Jobs per day charts for each worker
    Last job status for each worker
    Passed and failed jobs count
    Average job time
    Max and min jobs time
    Total jobs time
    Last job time
    

    View Slide

39. I <3 feedback!
    

    View Slide

40. Anton Davydov
    github.com/davydovanton
    twitter.com/anton_davydov
    Thank you!
    

    View Slide