Threads, callbacks, and execution context in Ruby

When you provide a block to a function in Ruby, do you know when and where that block will be executed? What is safe to do inside the block, and what is dangerous? Let’s take a look at various code examples and understand what dragons are hidden in Ruby dungeons.

Transcript

1. Threads, callbacks, and execution context in Ruby Andrey Novikov, Evil Martians Osaka Ruby Kaigi #03 09 September 2023
2. About me Hi, I’m Andrey (アンドレイ ) Back-end engineer at Evil Martians Writing Ruby, Go, and whatever SQL, Dockerfiles, TypeScript, bash… Love open-source software Created and maintaining a few Ruby gems Living in Japan for 1 year already Driving a moped And also a bicycle to get kids to kindergarten
3. Martians are closer than you think Our base is just 30 minutes walk away from here! Please come visit us! 1. Just let us know in advance in e 𝕏 - Twitter @evilmartians_jp ↩︎ [1]
4. Let’s talk about blocks as callbacks It feels like code between do and end is in the same flow with surrounding code, right? WRONG! 3.times do |i| puts "Hello, Osaka RubyKaigi #0#{i}!" end ` ` ` `
5. Blocks are separate pieces of code Block is separate entity, that is passed to times method as an argument and got called by it. ` ` greet = proc do |i| puts "Hello, Osaka RubyKaigi #0#{i}!" end greet.call(3) # also 3.times(&greet)
6. Blocks are called by methods Illustration from the Ruby under a microscope book (日本語版 : Rubyのしくみ ) Ruby under a microscope
7. Blocks ARE callbacks We often use blocks as callbacks to hook our own behavior for someone’s else code.
8. Blocks in time and space When does the block get executed? And where? How to understand? def some_method(&block) block.call # and/or @callbacks << block end some_method do puts "Hey, I was called!" end # which one??? will it be called at all?
9. How to understand? No way to know! 😭 Well, except reading method documentation and source code. And memorize, memorize, and memorize. When and where will the block be called?
10. Blocks right here, right now All Enumerable methods are sync, and will call provided block during their execution. E.g. times will yield to the block on every iteration. ` ` 3.times do |i| puts "Hello, Osaka RubyKaigi #0#{i}!" end ` ` ` `
11. Blocks can be called later Much later. ActiveRecord callbacks and also after_commit from after_commit_everywhere gem will store callback proc in ActiveRecord internals for later. after_commit do puts "Hello, Osaka RubyKaigi #03!" end ` ` after_commit_everywhere
12. Blocks can be called from other threads Can you feel how thread-safety problems are coming? result = [] work = proc do |arg| # Can you tell which thread is executing me? result << arg # I'm closure, I can access result end Thread.new do work.call "from new thread" end work.call "from main thread" # And guess what's inside result now? 🫠
13. Different threads E.g. concurrent-ruby Promise uses thread pools to execute blocks. Thread pools doesn’t guarantee which thread will execute which block. ` ` raise "Unexpected!" if Thread.current[:state] != 'work1' raise "Unexpected!" if Thread.current[:state] != 'work2' Concurrent::Promise.zip(*promises).value! #=> Unexpected! (RuntimeError) 💣💥 work1 = proc do Thread.current[:state] ||= 'work1' "result" end work2 = proc do Thread.current[:state] ||= 'work2' "result" end promises = 100.times.flat_map do Concurrent::Promise.execute(&work1) Concurrent::Promise.execute(&work2) end
14. Example: NATS client NATS is a modern, simple, secure and performant message communications system for microservice world. Prior version 2.3.0 every subscription was executed in its own thread. Code above works as expected. nats = NATS.connect("demo.nats.io") nats.subscribe("service") do |msg| Thread.current[:counter] ||= 0 Thread.current[:counter] += 1 msg.respond(Thread.current[:counter]) end nats-pure gem
16. Can I use Thread.current in NATS callbacks? Performance is got much better, but there is a side effect… Q: So, can I? A: Not in 2.3.0+! 🤯 Hint: better not to anyway! ` ` nats = NATS.connect("demo.nats.io") nats.subscribe("service") do |msg| Thread.current[:counter] ||= 0 Thread.current[:counter] += 1 msg.respond(Thread.current[:counter]) end
17. Where you can find thread pools? Puma Sidekiq ActiveRecord load_async NATS client (new!) …and many more Good thing is that you don’t have to care about them most of the time. Pro Tip: In Rails use ActiveSupport::CurrentAttributes instead of Thread.current as every request is going to be executed in different thread! ` ` ` ` ` `
18. Recap Blocks are primarily used as callbacks Blocks can be called from other threads And this thread can be different each time! Think twice before using Thread.current And you have to remember that! ` `
19. Let’s talk more about blocks and callbacks! Attend my next talk about Rails Executor at Tokyo, 27–28 October 2023 Rails Executor talk announce
20. Thank you! @Envek @Envek @Envek @Envek github.com/Envek @evilmartians @evilmartians_jp @evil-martians @evil.martians evilmartians.jp Our awesome blog: evilmartians.com/chronicles! See these slides at envek.github.io/osakarubykaigi-threads-callbacks