Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Parallel testing with Ractors: putting CPUs to work

Parallel testing with Ractors: putting CPUs to work

Parallelizing tests is an opportune way of reducing the total runtime for a test suite. Rails achieves this by forking multiple separate workers that fetch tests from a queue. In Ruby 3, Ractors introduced new mechanisms for executing code in parallel. Can they be leveraged by a test framework? And how would that compare to current parallelization solutions?

Let’s find the answers to these questions by building a test framework built on Ractors, from scratch. We’ll compare the current solutions for parallelization and what advantages or limitations Ractors bring when used in this context.


Vinicius Stock

September 20, 2021


  1. Vinicius Stock Parallel testing with Ractors Putting CPUs to work

  2. Vinicius “Vini” Stock Senior developer @ Shopify Ruby Types team

    Twitter & GitHub: @vinistock https://vinistock.com
  3. RubyConf 2020 Ractor demonstration

  4. Test execution strategies Introduction to Ractors Build a test framework

    Parallelize using Ractors Highlights
  5. How do we execute tests?

  6. Figure out which tests to run Separate tests into groups

    CPU 1 Run test group #1 CPU 2 Run test group #2 CPU 3 Run test group #3 CPU 4 Run test group #4 Execution strategy #1 - Groups Simple Doesn’t guarantee even distribution
  7. CPU 1 CPU 2 Execution strategy #1 Unit test #1

    Unit test #2 Unit test #3 Integration test #1 Unit test #4 Integration test #2 CPU 2 is idle! Unit test #5
  8. Figure out which tests to run Randomize and organize tests

    in a queue Test 1, test 2, test 3, test 4 CPU 1 CPU 2 CPU 3 CPU 4 Execution strategy #2 - Queue
  9. CPU 1 Queue Execution strategy #2 PostTest#test_title PostTest#test_author AuthorTest#test_name PostTest#test_score

    AuthorTest#test_bio CPU 2 PostTest#test_title PostTest#test_author AuthorTest#test_name PostTest#test_score
  10. Ruby did not have full support for parallelism How do

    we parallelize tests right now?
  11. Existing strategy for parallelizing tests https://github.com/rails/rails

  12. Rails Minitest plugin Queue object 
 Test 1, test 2,

    test 3, test 4 PROCESS 1 PROCESS 2 PROCESS 3 PROCESS 4 DRb
  13. Ruby 3.0 - Ractors How do they compare to processes?

  14. Ractors - Communication

  15. Ractors - Worker pool example

  16. Test frameworks Execution Utilities and syntax Reporting

  17. Test frameworks - execution • Regular classes • Keep track

    of them • Run tests on each one
  18. Test frameworks - utilities • Define the syntax • Assist

    execution flow • Delegate information • Provide helpers
  19. Test frameworks - reporting • Display progress • Display aggregated

  20. Test frameworks Let’s implement it!

  21. Test frameworks - execution How do we keep track of

  22. Test frameworks - execution How do we model the test

  23. Test frameworks - execution

  24. Test frameworks - execution PostTest :test_author #<PostTest:0x0…> setup send(:test_author) teardown

  25. Test frameworks - execution Basic execution is complete

  26. Test frameworks - utilities & reporting assert(something) Passed continue Failed

    Skip to next item in the queue Save failure info Reporter
  27. Test frameworks - utilities & reporting Print failure F

  28. Test frameworks - utilities & reporting PostTest :test_author #<PostTest:0x0…> Increment

    total tests setup send(:test_author) teardown Increment successful tests Print success dot
  29. Test frameworks - execution steps We got the foundation Let’s

    run it
  30. Test frameworks - execution

  31. Test frameworks - execution steps We have our basic test

  32. Parallelizing our framework

  33. Parallelizing our framework We can’t use a singleton pattern for

    the reporter
  34. Parallelizing our framework

  35. Parallelizing our framework We parallelized our framework!

  36. https://github.com/vinistock/loupe

  37. Ractors vs Processes

  38. Highlights Can only test Ractor compatible code Can’t make broad

    usage of class variables Can’t make broad usage of meta-programming
  39. Highlights Can test code that spawns more Ractors Can run

    our tests in parallel Doesn’t pay initial cost of forking processes
  40. Seg fault on GC sweep + yield https://bugs.ruby-lang.org/issues/18117 Seg fault

    when freeing classes created inside Ractors https://bugs.ruby-lang.org/issues/18119 Dead lock when using autoload https://bugs.ruby-lang.org/issues/18120 Bug hunt! Seg fault on GC sweep + yield https://bugs.ruby-lang.org/issues/18117
  41. Ractors will continue to evolve

  42. Thank you! Twitter & GitHub: @vinistock https://vinistock.com