Test execution strategies
Introduction to Ractors
Build a test framework
Parallelize using Ractors
Highlights
Slide 5
Slide 5 text
How do we execute tests?
Slide 6
Slide 6 text
Figure out
which tests
to run
Separate
tests into
groups
CPU 1
Run test
group #1
CPU 2
Run test
group #2
CPU 3
Run test
group #3
CPU 4
Run test
group #4
Execution
strategy #1 -
Groups
Simple
Doesn’t guarantee
even distribution
Slide 7
Slide 7 text
CPU 1
CPU 2
Execution strategy #1
Unit test #1 Unit test #2 Unit test #3
Integration
test #1
Unit test #4
Integration
test #2
CPU 2 is idle!
Unit test #5
Slide 8
Slide 8 text
Figure out
which tests
to run
Randomize
and organize
tests in a
queue
Test 1, test 2, test 3, test 4
CPU 1 CPU 2 CPU 3 CPU 4
Execution
strategy #2 -
Queue
Slide 9
Slide 9 text
CPU 1
Queue
Execution strategy #2
PostTest#test_title PostTest#test_author AuthorTest#test_name PostTest#test_score AuthorTest#test_bio
CPU 2
PostTest#test_title PostTest#test_author
AuthorTest#test_name
PostTest#test_score
Slide 10
Slide 10 text
Figure out
which tests
to run
Separate
tests into
groups
CPU 1
Run test
group #1
CPU 2
Run test
group #2
CPU 3
Run test
group #3
CPU 4
Run test
group #4
Execution
strategy #3 -
Work stealing
https://en.wikipedia.org/wiki/Work_stealing
Slide 11
Slide 11 text
CPU 1
CPU 2
Execution strategy #3
Unit test #1 Unit test #2 Unit test #3
Integration
test #1
Unit test #4
Integration
test #2
Unit test #5
Reduces communication overhead
Slide 12
Slide 12 text
Ruby did not have full support for parallelism
How do we parallelize tests right now?
Slide 13
Slide 13 text
Existing strategy for parallelizing tests
https://github.com/rails/rails
Slide 14
Slide 14 text
Rails Minitest plugin
Queue object
Test 1, test 2, test 3, test 4
PROCESS 1 PROCESS 2 PROCESS 3 PROCESS 4
DRb
Slide 15
Slide 15 text
Ruby 3.0 - Ractors
How do they compare to processes?
Slide 16
Slide 16 text
Ractors - Communication
Slide 17
Slide 17 text
Ractors - Worker pool example
Slide 18
Slide 18 text
Test frameworks
Execution Utilities and syntax Reporting
Slide 19
Slide 19 text
Test frameworks - execution
• Regular classes
• Keep track of them
• Run tests on each one
Slide 20
Slide 20 text
Test frameworks - utilities
• Define the syntax
• Assist execution flow
• Delegate information
• Provide helpers
Test frameworks - execution steps
We got the foundation
Let’s run it
Slide 32
Slide 32 text
Test frameworks - execution
Slide 33
Slide 33 text
Test frameworks - execution steps
We have our basic test framework
Slide 34
Slide 34 text
Parallelizing our framework
Slide 35
Slide 35 text
Parallelizing our framework
We can’t use a singleton pattern for the reporter
Slide 36
Slide 36 text
Parallelizing our framework
Slide 37
Slide 37 text
Parallelizing our framework
We parallelized our framework!
Slide 38
Slide 38 text
https://github.com/vinistock/loupe
Slide 39
Slide 39 text
Ractors vs Processes
Slide 40
Slide 40 text
Highlights
Can only test Ractor compatible code
Can’t make broad usage of class variables
Can’t make broad usage of meta-programming
Slide 41
Slide 41 text
Highlights
Can test code that spawns more Ractors
Can run our tests in parallel
Doesn’t pay initial cost of forking processes
Slide 42
Slide 42 text
Seg fault on GC sweep + yield
https://bugs.ruby-lang.org/issues/18117
Seg fault when freeing classes created inside Ractors
https://bugs.ruby-lang.org/issues/18119
Dead lock when using autoload
https://bugs.ruby-lang.org/issues/18120
Bug hunt!
Seg fault on GC sweep + yield
https://bugs.ruby-lang.org/issues/18117