$30 off During Our Annual Pro Sale. View Details »

Sidekiq under the hood

Anton Davydov
September 26, 2015

Sidekiq under the hood

Anton Davydov

September 26, 2015
Tweet

More Decks by Anton Davydov

Other Decks in Programming

Transcript

  1. Hello!

    View Slide

  2. github: davydovanton
    twitter: anton_davydov
    Anton Davydov

    View Slide

  3. RCNTEC developer
    sidekiq commiter
    ruby doc / rails / crystal
    OSS one love <3

    View Slide

  4. @rubyunderhood curator

    https://twitter.com/rubyunderhood

    View Slide

  5. @moscowrb
    http://moscow-rb.org

    View Slide

  6. You
    Beer

    View Slide

  7. You
    Beer
    Me

    View Slide

  8. I (we) <3 sidekiq!

    View Slide

  9. Simple, efficient background processing for Ruby.

    View Slide

  10. Sidekiq server
    New thread

    View Slide

  11. I (we) <3 actors!

    View Slide

  12. Manager actor

    View Slide

  13. Fetcher actors
    Manager actor

    View Slide

  14. Redis
    Fetcher actors
    Manager actor

    View Slide

  15. Redis
    Fetcher actors Processor actors
    Manager actor

    View Slide

  16. Redis
    Fetcher actors Processor actors
    Manager actor
    Workers

    View Slide

  17. MyWorker.perform_async

    View Slide

  18. irb > MyWorker.perform_async
    Redis queue
    Serialization
    prams

    View Slide

  19. Processor actors
    Fetcher actors
    Redis
    Manager actor
    Workers

    View Slide

  20. Processor actors
    Manager actor
    Workers
    Fetcher actors
    Redis

    View Slide

  21. Processor actors
    Manager actor
    Workers
    Fetcher actors
    Redis

    View Slide

  22. Workers
    Processor actors
    Manager actor
    Fetcher actors
    Redis

    View Slide

  23. LongWorker
    Redis

    View Slide

  24. Redis
    Serialization
    prams
    LongWorker
    After 8 seconds

    View Slide

  25. middleware

    View Slide

  26. Sidekiq.configure_server do |config|
    config.server_middleware do |chain|
    chain.add YourMiddleware, options
    end
    end

    View Slide

  27. def call(worker, msg, queue)

    # code before call you worker

    yield

    rescue => e

    # run if your worker is failed

    raise e

    ensure

    # run in your worker passed or failed

    end

    View Slide

  28. def call(worker, msg, queue)

    # code before call you worker

    yield

    rescue => e

    # run if your worker is failed

    raise e

    ensure

    # run in your worker passed or failed

    end

    View Slide

  29. def call(worker, msg, queue)

    # code before call you worker

    yield

    rescue => e

    # run if your worker is failed

    raise e

    ensure

    # run in your worker passed or failed

    end

    View Slide

  30. def call(worker, msg, queue)

    # code before call you worker

    yield

    rescue => e

    # run if your worker is failed

    raise e

    ensure

    # run in your worker passed or failed

    end

    View Slide

  31. Sidekiq-statistic
    github.com/davydovanton/sidekiq-statistic

    View Slide

  32. Improved display of statistics for
    your sidekiq workers and jobs.

    View Slide

  33. View Slide

  34. View Slide

  35. View Slide

  36. View Slide

  37. Worker name
    Last job status
    Runtime
    Start time

    View Slide

  38. Jobs per day charts for each worker
    Last job status for each worker
    Passed and failed jobs count
    Average job time
    Max and min jobs time
    Total jobs time
    Last job time

    View Slide

  39. I <3 feedback!

    View Slide

  40. Anton Davydov
    github.com/davydovanton
    twitter.com/anton_davydov
    Thank you!

    View Slide