Slide 1

Slide 1 text

Vinicius Stock Parallel testing with Ractors Putting CPUs to work

Slide 2

Slide 2 text

Vinicius “Vini” Stock Senior developer @ Shopify Ruby Types team Twitter & GitHub: @vinistock https://vinistock.com

Slide 3

Slide 3 text

RubyConf 2020 Ractor demonstration

Slide 4

Slide 4 text

Test execution strategies Introduction to Ractors Build a test framework Parallelize using Ractors Highlights

Slide 5

Slide 5 text

How do we execute tests?

Slide 6

Slide 6 text

Figure out which tests to run Separate tests into groups CPU 1 Run test group #1 CPU 2 Run test group #2 CPU 3 Run test group #3 CPU 4 Run test group #4 Execution strategy #1 - Groups Simple Doesn’t guarantee even distribution

Slide 7

Slide 7 text

CPU 1 CPU 2 Execution strategy #1 Unit test #1 Unit test #2 Unit test #3 Integration test #1 Unit test #4 Integration test #2 CPU 2 is idle! Unit test #5

Slide 8

Slide 8 text

Figure out which tests to run Randomize and organize tests in a queue Test 1, test 2, test 3, test 4 CPU 1 CPU 2 CPU 3 CPU 4 Execution strategy #2 - Queue

Slide 9

Slide 9 text

CPU 1 Queue Execution strategy #2 PostTest#test_title PostTest#test_author AuthorTest#test_name PostTest#test_score AuthorTest#test_bio CPU 2 PostTest#test_title PostTest#test_author AuthorTest#test_name PostTest#test_score

Slide 10

Slide 10 text

Figure out which tests to run Separate tests into groups CPU 1 Run test group #1 CPU 2 Run test group #2 CPU 3 Run test group #3 CPU 4 Run test group #4 Execution strategy #3 - Work stealing https://en.wikipedia.org/wiki/Work_stealing

Slide 11

Slide 11 text

CPU 1 CPU 2 Execution strategy #3 Unit test #1 Unit test #2 Unit test #3 Integration test #1 Unit test #4 Integration test #2 Unit test #5 Reduces communication overhead

Slide 12

Slide 12 text

Ruby did not have full support for parallelism How do we parallelize tests right now?

Slide 13

Slide 13 text

Existing strategy for parallelizing tests https://github.com/rails/rails

Slide 14

Slide 14 text

Rails Minitest plugin Queue object 
 Test 1, test 2, test 3, test 4 PROCESS 1 PROCESS 2 PROCESS 3 PROCESS 4 DRb

Slide 15

Slide 15 text

Ruby 3.0 - Ractors How do they compare to processes?

Slide 16

Slide 16 text

Ractors - Communication

Slide 17

Slide 17 text

Ractors - Worker pool example

Slide 18

Slide 18 text

Test frameworks Execution Utilities and syntax Reporting

Slide 19

Slide 19 text

Test frameworks - execution • Regular classes • Keep track of them • Run tests on each one

Slide 20

Slide 20 text

Test frameworks - utilities • Define the syntax • Assist execution flow • Delegate information • Provide helpers

Slide 21

Slide 21 text

Test frameworks - reporting • Display progress • Display aggregated results

Slide 22

Slide 22 text

Test frameworks Let’s implement it!

Slide 23

Slide 23 text

Test frameworks - execution How do we keep track of classes?

Slide 24

Slide 24 text

Test frameworks - execution How do we model the test queue?

Slide 25

Slide 25 text

Test frameworks - execution

Slide 26

Slide 26 text

Test frameworks - execution PostTest :test_author # setup send(:test_author) teardown

Slide 27

Slide 27 text

Test frameworks - execution Basic execution is complete

Slide 28

Slide 28 text

Test frameworks - utilities & reporting assert(something) Passed continue Failed Skip to next item in the queue Save failure info Reporter

Slide 29

Slide 29 text

Test frameworks - utilities & reporting Print failure F

Slide 30

Slide 30 text

Test frameworks - utilities & reporting PostTest :test_author # Increment total tests setup send(:test_author) teardown Increment successful tests Print success dot

Slide 31

Slide 31 text

Test frameworks - execution steps We got the foundation Let’s run it

Slide 32

Slide 32 text

Test frameworks - execution

Slide 33

Slide 33 text

Test frameworks - execution steps We have our basic test framework

Slide 34

Slide 34 text

Parallelizing our framework

Slide 35

Slide 35 text

Parallelizing our framework We can’t use a singleton pattern for the reporter

Slide 36

Slide 36 text

Parallelizing our framework

Slide 37

Slide 37 text

Parallelizing our framework We parallelized our framework!

Slide 38

Slide 38 text

https://github.com/vinistock/loupe

Slide 39

Slide 39 text

Ractors vs Processes

Slide 40

Slide 40 text

Highlights Can only test Ractor compatible code Can’t make broad usage of class variables Can’t make broad usage of meta-programming

Slide 41

Slide 41 text

Highlights Can test code that spawns more Ractors Can run our tests in parallel Doesn’t pay initial cost of forking processes

Slide 42

Slide 42 text

Seg fault on GC sweep + yield https://bugs.ruby-lang.org/issues/18117 Seg fault when freeing classes created inside Ractors https://bugs.ruby-lang.org/issues/18119 Dead lock when using autoload https://bugs.ruby-lang.org/issues/18120 Bug hunt! Seg fault on GC sweep + yield https://bugs.ruby-lang.org/issues/18117

Slide 43

Slide 43 text

Ractors will continue to evolve

Slide 44

Slide 44 text

Thank you! Twitter & GitHub: @vinistock https://vinistock.com