beanstalkd

A revolutionary idea: why not use a job queue system for your job queue?

Most people use delayed job aka a database or resque/sidekiq for queuing. But why hack your way around a database or a "smarter memcache" to do a simple queue when there are other solutions that were build for this?

Beanstalkd is one of them. It's small. It's fast. It's awesome. And I show you why.

Transcript

1. submit
   

   View Slide

2. submit
   

   View Slide

3. @bitboxer
   @bitcrowd
   Bodo Tasche
   

   View Slide

4. Queues!
   

   View Slide

5. create table 'job_queue';
   

   View Slide

6. gem install delayed_job
   

   View Slide

7. View Slide

8. beanstalkd
   

   View Slide

9. Beanstalkd is a simple,
   fast work queue.
   

   View Slide

10. brew install beanstalkd
    

    View Slide

11. https://kr.github.io/beanstalkd/
    

    View Slide

12. beanstalkd -p 11300
    

    View Slide

13. gem install beaneater
    

    View Slide

14. @beanstalk = Beaneater::Pool.new(
    [’localhost:11300’]
    )
    

    View Slide

15. @tube = @beanstalk.tubes['email']
    @tube.put '{"key" : "foo"}', :pri => 5
    @tube.put '{"key" : "bar"}', :delay => 3
    

    View Slide

16. @beanstalk.jobs.register('email') do |job|
    json = Oj.load(job.body)
    puts "job value is #{json["key"]}!"
    end
    !
    @beanstalk.jobs.process!
    

    View Slide

17. @beanstalk.jobs.register(
    'email',
    :retry_on => [SomeError],
    :max_retries => 2
    )
    

    View Slide

18. @tube.put '{"key" : "bar"}', :ttr => 120
    

    View Slide

19. @beanstalk.jobs.register('email') do |job|
    # Long running stuff
    job.touch
    end
    

    View Slide

20. • ready
    • delayed
    • buried
    

    View Slide

21. job = @tube.peek(:buried)
    

    View Slide

22. job.kick
    

    View Slide

23. job.delete
    

    View Slide

24. Beaneater.configure do |config|
    config.default_put_delay = 0
    config.default_put_pri = 65536
    config.default_put_ttr = 120
    end
    

    View Slide

25. @beanstalk.stats
    

    View Slide

26. @beanstalk.tubes['email'].stats
    

    View Slide

27. https://github.com/kr/beanstalkd/wiki/Tools
    

    View Slide

28. beanstalkd
    

    View Slide

29. @bitsofberlin
    http://bitsofberlin.org
    

    View Slide