DISCLAIMER • I’ll talk about parallel testing. • My English skill is unknown. • If you want to listen to general news around ruby 1.9.3, I strongly recommend you to move to Room 2. • Room 2 14:10: “Implementation of Ruby 1.9.3 and later” by ko1
Benefits • In most case, tests can run more faster • Faster test → Fast development cycle • In TDD, BDD development • At Ruby, committers have to run `make test-all` before commit to repository. • Faster test make developers happy :-)
Benefits • Q. You should run only few tests (like around the fixed point) at once. • A. Some changes make some failures at another point. In ruby, running all test at each commits is recommended.
Multi-Process ? • Multi process is good parallelization method at ruby. Time Test 2 Test 1 ŋŋŋ Core A Core B Test 1 Other 1 Other 2 Test 2 ↓Running process on each core Other: other process Test: testing process
What’s test/unit • Unit testing library that shipped with Ruby • Before 1.9: test/unit doesn’t have any dependencies • After 1.9: test/unit is wrapper of minitest because of compatibility. • test/unit is used for Ruby’s unit testing.
test/unit parallelization • I wrote a patch that allows test/unit runs multiple tests at once • Because I wanted to make Ruby’s `make test-all` (make target to test ruby) more more faster! :p • Fast is important™
How it works • I’ll use the following words to describe how it works: • Master: a process that is started first. This Process sends a instruction to workers. • Example: a ruby process that started by `make test-all` • Worker: a process that is started by Master process. This process runs tests.
How it works 1. Start master process (by like `make test- all`) 2. Master process starts worker processes 3. Master process sends a test file names to worker processes 4. Worker processes read a file that named passed by master process
How it works 5. Worker processes run a test. 6. Worker processes return a test result to master process. 7. Master process send another test file name to worker process 8. Do from step 4 again.
How it works • this feature uses stdin/stdout for communication between worker process and master process. • and this feature parallelize each test file. • so you have to separate tests and TestCase if you use this feature.
How it works • Master process sends file names of tests to worker process Master Worker Worker Test files test_a.rb test_b.rb test_c.rb test_a.rb test_b.rb
How it works • Worker process sends result to Master process. master saves results from workers. Master Worker Worker Test files test_c.rb result result
How it works • If there are any results that contains more than one failure, master will retry in default Master result result result I have a failure!
How it works • If there are any results that contains more than one failure, it’ll retry by master process in default Master result result test_b.rb Retry
How it works • If there are any results that contains more than one failure, it’ll retry by master process in default Master result result result There are no failures.
In Ruby • some tests don’t consider a parallelization • it can guess (I guessed that) • The issues is: • Tests around network are using same port • Tests are using same directory
test_signal.rb • test around signal handling of ruby. • this test run Kernel#fork to check fork is implemented. • test/unit’s at_exit called each fork. • I fixed test/unit to make be runnable this test. • Patch description: Add a flag to test/unit and interrupt if a flag is true in at_exit.
test/csv/* • Tests included in this directory use same temporally directory name • All testcases use same directory name and they deletes temporaly directory in `teardown` phase. • So...
test/rubygems/* • Recently they forget writing dependencies of test. • It makes problem at parallel_test. • because... ok, let’s show a example. Note: “dependency” is a file should be required.
test/rubygems/* • if there are test_a.rb, test_b.rb, and foo.rb. • also test_a.rb and test_b.rb depends on foo.rb. • test_a.rb includes `require ‘foo’`, •but test_b.rb doesn’t.
test/rubygems/* @rubygems_developers: please merge r33232 (in ruby-repo) https://github.com/rubygems/rubygems/issues/180 or http://bit.ly/rubyconf2011_rubygems_issue
Performance • Kenta Murata a.k.a. @mrkn measured and created a performance graphs for me. • He’ll talk at the last session of room 2, about the ruby’s number system. • Thanks!
Performance • Measured at the following environment: • OS: Mac OS X 10.6.6 • CPU: Intel Core i7 2.66 GHz • (2 cores 4 threads) • Memory: 8GB 1067 MHz DDR3 • Test files is from ruby’s repository
How to use • Separate TestCase-s to multiple file. • Write script to run multiple test file using Test::Unit::AutoRunner. • good example is ruby’s test/runner.rb in ruby’s repository. • Run the script with argument -j N. • Running the script with --help provides more information.
Announcement • Ruby 1.9.3 RC1 has been released! • You can use parallelization by installing this! • Let’s try it and tell us if you found a bug. • More about: http://www.ruby-lang.org/
sorah
line_developers_tw
naohachi89
wakamsha
hokaccha
nwiizo
takasehideki
mosa_siru
yoshiitaka
tkhresk
orimanabu
tacck
oracle4engineer
soudai
addyosmani
ddemaree
morganepeng
denniskardys
smashingmag
brad_frost
tanoku
geeforr
maggiecrowley
kevinliebholz
sugarenia