Fiber Scheduler vs. General-Purpose Parallel Client

Transcript

1. Fiber Scheduler vs. General-Purpose Parallel Client
 2025.04.17
2. Premises and Issues

3. 3 Underlying Architecture
 User
 Service
 Internal Service A
 Internal Service B
 Internal Service C

4. 4 Problems to Be Solved
 User
 Service
 Internal Service A
 Internal Service B
 Internal Service C
 Latency

5. 5 What We Want to Achieve (Parallel Requests)
 User
 Service
 Internal Service A
 Internal Service B
 Internal Service C
 Latency

6. 6 Requirements for the Parallel Request Client
 • It should be able to send parallel requests in a unified way across different HTTP client gems
 ◦ For example: faraday gem, grpc gem, aws-sdk gem
 • Should be easy for in-house engineers to understand and use

7. Approach

8. 8 Parallel Requests Using the faraday-em_synchrony Adapter
 • Initially, we wanted to address the issue of having many Faraday requests
 • faraday-em_synchrony is one of Faraday’s adapters that allows parallel requests
 ◦ This approach is currently running in the production environment
 • Issues with this approach:
 ◦ It depends on the EventMachine gem, which hasn't had a release since 2018 and is somewhat outdated
 ◦ It only works with Faraday
 ◦ It is not compatible with Faraday 2.x

9. 9 Is Fiber Scheduler a Better Option?
 • Fiber Scheduler is a mechanism that enables concurrent processing using Fibers
 
 • Implementations like the async gem allow the scheduler to delegate execution to other Fibers when I/O blocking occurs, such as with IO#wait, IO#read, or IO#write in Ruby

10. With Fiber Scheduler, almost everything worked perfectly!!!

11. ...Almost...? 😅

12. 12 Recap of How Fiber Scheduler Works
 • When a Fiber Scheduler is present, Ruby internally checks and delegates certain I/O operations to the scheduler
 ◦ For example: in the implementation of IO#wait, there's logic to invoke the Fiber Scheduler if it exists

13. 13 Async::Scheduler#io_wait
 Recap of How Fiber Scheduler Works
 • When a Fiber Scheduler is present, Ruby internally checks and delegates certain I/O operations to the scheduler
 ◦ For example: in the implementation of IO#wait, there's logic to invoke the Fiber Scheduler if it exists

14. 14 So, What Kinds of Things Can Be Handled Concurrently with Fiber Scheduler?
 • Methods like Ruby’s IO#wait, which have specific behavior defined when a Fiber Scheduler is present
 ◦ cf. https://docs.ruby-lang.org/en/3.4/Fiber/Scheduler.html 
 
 • Even methods implemented in native extensions, as long as they define behavior for when a Fiber Scheduler is available

15. 15 • It should be able to send parallel requests in a unified way across different HTTP client gems
 ◦ For example: faraday gem, grpc gem, aws-sdk gem
 
 • Should be easy for in-house engineers to understand and use
 Now, let's revisit and reorganize the requirements for the parallel request client.

16. 16 • It should be able to send parallel requests in a unified way across different HTTP client gems
 ◦ For example: faraday gem, grpc gem, aws-sdk gem
 
 • Should be easy for in-house engineers to understand and use
 Now, let's revisit and reorganize the requirements for the parallel request client.

17. 17 What’s the Problem with the grpc Gem?
 gRPC Core
 gRPC Core
 Ruby Gem
 gRPC Core
 PHP Package
 gRPC Core
 Python Package

18. 18 What’s the Problem with the grpc Gem?
 • The logic for sending gRPC requests and waiting for responses is implemented in the C-based gRPC core

19. 19 What’s the Problem with the grpc Gem?
 • The logic for sending gRPC requests and waiting for responses is implemented in the C-based gRPC core
 This leaves no room to hook in Ruby’s Fiber Scheduler for concurrent handling!

20. Given this, let's abandon the Fiber Scheduler approach and switch to a Thread-based approach instead.

21. 21 Can We Use Threads to Parallelize with the grpc Gem?
 
 

22. Conclusion

23. 23 Conclusion
 • I set out to build a general-purpose parallel request client
 • Wanted to unify and parallelize various HTTP clients like Faraday, gRPC, and AWS SDK
 • But it turned out that gRPC's implementation doesn't work well with Fiber Scheduler for parallelization
 • So now, I'm building an easy-to-use parallel client using Threads that works smoothly with all of them!
 ◦ To be continued...

24. 　 24 • Co-Founder of Kyobashi.rb.
 • Organizer of 関西Ruby会議08
 • Organizer of Kaigi on Rails.
 • Author of the book "Learning Electronics with Ruby"
 hachi
 Engineer at freee K.K.
 @hachiblog

25. 　 25 • Co-Founder of Kyobashi.rb.
 • Organizer of 関西Ruby会議08
 • Organizer of Kaigi on Rails.
 • Author of the book "Learning Electronics with Ruby"
 hachi
 Engineer at freee K.K.
 @hachiblog

26. 　 26 • Co-Founder of Kyobashi.rb.
 • Organizer of 関西Ruby会議08
 • Organizer of Kaigi on Rails.
 • Author of the book "Learning Electronics with Ruby"
 hachi
 Engineer at freee K.K.
 @hachiblog

27. 　 27 • Co-Founder of Kyobashi.rb.
 • Organizer of 関西Ruby会議08
 • Organizer of Kaigi on Rails.
 • Author of the book "Learning Electronics with Ruby"
 hachi
 Engineer at freee K.K.
 @hachiblog

28. 　 28 • Co-Founder of Kyobashi.rb.
 • Organizer of 関西Ruby会議08
 • Organizer of Kaigi on Rails.
 • Author of the book "Learning Electronics with Ruby"
 hachi
 Engineer at freee K.K.
 @hachiblog