Threads, callbacks, and execution context in Ruby

When you provide a block to a function in Ruby, do you know when and where that block will be executed? What is safe to do inside the block, and what is dangerous? Let’s take a look at various code examples and understand what dragons are hidden in Ruby dungeons.

1. Threads, callbacks, and execution context in Ruby Andrey Novikov, Evil Martians RubyConf AU 2024 12 April 2024
2. About me Back-end engineer at Evil Martians Writing Ruby, Go, and whatever SQL, Dockerfiles, TypeScript, bash… Love open-source software Created a few little Ruby gems Living in Japan for 1 year already Driving a moped Hi, I’m Andrey
5. evilmartians.com
6. Martian Open Source Ruby Next makes modern Ruby code run in older versions and alternative implementations Yabeda: Ruby application instrumentation framework Lefthook: git hooks manager AnyCable: Polyglot replacement for ActionCable server PostCSS: A tool for transforming CSS with JavaScript Imgproxy: Fast and secure standalone server for resizing and converting remote images Overmind: Process manager for Procfile-based applications and tmux Even more at evilmartians.com/oss
7. Threads, callbacks, and execution context in Ruby
8. Let’s talk about callbacks… What callbacks? No, not Rails callbacks, no-no-no! class User < ApplicationRecord after_create :send_welcome_email end
9. Let’s talk about blocks as callbacks It feels like these two code samples are identical, right? WRONG! 3.times do puts "Hello RubyConf AU!" end i = 0 while i < 3 puts "Hello RubyConf AU!" i += 1 end
10. Blocks are separate pieces of code Block is separate entity, that is passed to times method as an argument and got called by it. 3.times { |i| greet.call(i) } # or 3.times(&greet) greet = proc do puts "Hello, RubyConf Australia!" end
11. Blocks are called by methods Illustration from the Ruby under a microscope book, chapter 2. Ruby under a microscope
12. And there is difference in performance Empty while loop is twice as faster than times with a block. However, difference is negligible with real workloads. And it is not a point of this talk… Benchmark.ips do |x| x.report("blocks") { 1_000_000.times { |i| i } } x.report("while") { i = 0; while i < 1_000_000; i += 1; end } end Warming up -------------------------------------- blocks 4.000 i/100ms while 9.000 i/100ms Calculating ------------------------------------- blocks 40.186 (± 0.0%) i/s - 204.000 in 5.076671s while 89.914 (± 1.1%) i/s - 450.000 in 5.005051s
13. Blocks ARE callbacks We often use blocks as callbacks to hook our own behavior for someone’s else code.
14. Blocks are closures as well A Ruby code to execute Environment to be executed in: Blocks can access local variables and self at the place where block was created …blocks have in some sense a dual personality. On the one hand, they behave like separate functions: you can call them and pass them arguments just as you would with any function. On the other hand, they are part of the surrounding function or method. Ruby Under a Microscope greeter.call # => Hello RubyConf AU 2024 def greeter conf = "RubyConf AU 2024" proc do puts "Hello #{conf}" end end # LAMBDA CALCULUS FTW!!!!1
15. Blocks are closures as well A Ruby code to execute Environment to be executed in: Blocks can access local variables, self, etc at the place where block was created BUT Some environments can be changed unexpectedly
16. Environment changes: self instance_exec and class_exec class Conference def title "RubyConf AU 2024" end def say(&block) puts instance_exec(&block) end end attr_reader :name, :conf "Hello #{conf.title}, I'm #{name}" class Speaker def greet conf.say do end end end undefined local variable or method `conf' for an instance of Conference
17. Environment changes: local vars Someone can change variables enclosed in the block’s environment… # later… conf = "RubyConf AU 2025" greeter.call # => Hello RubyConf AU 2025 conf = "RubyConf AU 2024" greeter = proc do puts "Hello #{conf}" end greeter.call # => Hello RubyConf AU 2024
18. Environment changes: local vars It is done to allow multiple blocks to work with a single enclosed variable. Illistration and code: Ruby under microscope i = 0 increment_function = lambda do puts "Incrementing from #{i} to #{i+1}" i += 1 end decrement_function = lambda do i -= 1 puts "Decrementing from #{i+1} to #{i}" end
19. Blocks can be called from other threads Can you feel how thread-safety problems are coming? work.call "new thread" work.call "main thread" result = [] work = proc do |arg| # Can you tell which thread is executing me? result << arg # I'm closure, I can do that! end Thread.new do end # And guess what's inside result? 🫠
20. Different threads What if we have some units of work that relies on thread local state… work2 = proc do Thread.current[:state] ||= 'work2' raise "Unexpected!" if Thread.current[:state] != 'work2' SecureRandom.hex(8) end work1 = proc do Thread.current[:state] ||= 'work1' raise "Unexpected!" if Thread.current[:state] != 'work1' SecureRandom.hex(4) end
21. Different threads And then try to execute them using concurrent-ruby Promise that utilizes thread pools to execute blocks… promises = 100.times.flat_map do Concurrent::Promise.execute(&work1) Concurrent::Promise.execute(&work2) end Concurrent::Promise.zip(*promises).value! #=> Unexpected! (RuntimeError) # But it also might be okay (chances are low though)
22. Example: NATS client NATS is a modern, simple, secure and performant message communications system for microservice world. In earlier versions every subscription was executed in its own separate thread. nats = NATS.connect("demo.nats.io") nats.subscribe("service") do |msg| msg.respond("pong") end nats-pure gem
23. Example: NATS client Before: After: def subscribe(topic, &callback) # Sent subscribe request to NATS Thread.new do while msg = message_queues[topic].pop callback.call(msg) end end end while msg = incoming_messages.pop do message_queues[msg.topic].push(msg) end def subscribe(topic, &callback) # Sent subscribe request to NATS callbacks[topic] = callback end while msg = incoming_messages.pop Concurrent::Promise.execute do callbacks[topic].call(msg) end end
25. Can I use Thread.current in NATS callbacks? Q: So, can I? A: It depends on gem version! 🤯 (you could before version 2.3.0) Hint: better not to anyway! nats = NATS.connect("demo.nats.io") nats.subscribe("service") do |msg| Thread.current[:handled] ||= 0 Thread.current[:handled] += 1 end
26. Where you can find thread pools? Puma Sidekiq ActiveRecord load_async NATS client …and many more Good thing is that you don’t have to care about them most of the time. Pro Tip: In Rails use ActiveSupport::CurrentAttributes instead of Thread.current !
27. How to understand? No easy way to know! 😭 Only by reading API documentation and source code. When and where and how a block will be called?
28. Recap Blocks are primarily used as callbacks Blocks can be executed in a different threads And this thread can be different each time! Think twice before using Thread.current Blocks can be executed with a different receiver Even captured local variables can be changed And you have to remember that!
29. Thank you! @Envek @Envek @Envek @Envek github.com/Envek @evilmartians @evilmartians @evil-martians @evil.martians evilmartians.com Our awesome blog: evilmartians.com/chronicles! See these slides at envek.github.io/rubyconfau-threads-callbacks