The Resurrection of  the Fast Parallel Test Runner

Koichi ITO / ESM, Inc. RubyKaigi 2023 The Resurrection of
the Fast Parallel Test Runner May, 12th, 2023 Matsumoto Performing Arts Centre The Force Awakens

!LPJD w 044QSPHSBNNFS w 3VCP$PQDPSFUFBN w &OHJOFFSJOH.BOBHFSBOE %JTUJOHVJTIFE&OHJOFFSPG&4. *OD w
3VCZ,BJHJTQFBLFSBU -5 5BLFPVU BOE

.BJOUBJO044&WFSZ%BZ IUUQTHJUIVCDPNLPJD

&4. *OD

$P⒎FFIPVTF4QPOTPS w"CF ᘖᘣඒֶΞϕ w0LJOBEP ͓͖ͳಊ w:BNBHB ٤஡ࢁխ
w"OENPSF ☕

/PWFMUZ

4 Q FBLFS 4 Q FBLFS JOQ FSTPO JOQ FSTPO
JOQ FSTPO JOQ FSTPO JOQ FSTPO JOQ FSTPO POMJOF POMJOF POMJOF POMJOF POMJOF POMJOF POMJOF 4QFBLFSTBOE"UUFOEFFT PSHBOJ[FS PSHBOJ[FS "EWJTPS "EWJTPS

Support OSS community

+PJOVT 🗾w-FUTHSPXUPHFUIFS w:PVDBOXPSL SFNPUFMZGSPN BOZXIFSFJO+BQBO w4FFBHJMFFTNDPKQ

$POUFOUT 8IBUTQBSBMMFMUFTUJOH 1BSBMMFMUFTUSVOOFST 3FTVSSFDUJPOPGUFTURVFVF

8IBUTQBSBMMFMUFTUJOH

5PNF MFHBDZDPEF JTTJNQMZDPEF XJUIPVUUFTUT 803,*/(&''&$5*7&-:8*5)-&("$:$0%& l "CPPLJOUIF6ODMF#PCTFSJFTTBZT

w 4PGUXBSFDPEFLFFQTHSPXJOHMBSHFS w #VTJOFTTHSPXUIBOETPGUXBSFHSPXUI HPIBOEJOIBOE w $POUJOVPVTMZQFSGPSN5%% #%% 5"%
BOESFHSFTTJPOUFTUT 5FTUDPEFJTJNQPSUBOU

)PXMPOHEPZPV XBJUGPSZPVSCVJME

⏳ "JNGPS NJOVUFTPSMFTT

w FYBNQMFT 6TJOH34QFD w "SFBMXPSMEBQQMJDBUJPOUBLFPWFSBO IPVS "SFBMXPSMEBQQMJDBUJPO $
bin/rails stats (snip) Code LOC: 12796 Test LOC: 23518 Code to Test Ratio: 1:1.8

"OFXQSPCMFN 4MPXUFTU

w $BOOPUSVOUFTUTPOQVMMSFRVFTUTJO BEBZ w 5IFCVJMETBSFOUpOJTIFEFWFOCZ UIFOFYUNPSOJOH *UUBLFTBOIPVSUPDPNQMFUFBCVJME

w TFDPOETJOBEBZ w 'PSFYBNQMF TMPXUFTUTUBLJOH TFDPOETFBDIXJMMSFRVJSFNJOVUFT w )PXNBOZDPNNJUTEPFTZPVSUFBN NBLFFWFSZEBZ
5JNFJTJNQPSUBOU

w 1SPpMJOHCBTFEUVOJOH%POUHVFTT NFBTVSF • FH w .BZCFTQFEVQXJUIBDMFBOJOHTUSBUFHZ w FH%BUBCBTF$MFBOFSP⒎FSTEBUBCBTFDMFBOJOH TUSBUFHJFTTVDIBTtransaction
deletion BOE truncation JUTBHPPEJEFBUPDIPPTFUIFPQUJNBMPOF "EESFTTJOH4MPX5FTU#PUUMFOFDLT $ rspec -p 30 spec/acceptance

.ZSFDPNNFOEBUJPO w 'JSTU QSFQBSFNPOFZ💰JGZPVBSF CVJMEJOHCVTJOFTTTPGUXBSF w /FYU QSFQBSFFOPVHI$16TBOENFNPSZ w 'JOBMMZ
1BSBMMFMJ[FZPVSUFTUT w 5IBUTJU"'"*, JUTUIFNPTUF⒎FDUJWF

1BSBMMFM5FTUJOH

4FSJBM5FTUJOH 4JOHMFXPSLFS 4VJUF 4VJUF 4VJUF 4VJUF
4VJUF 4VJUF 4VJUF w 8PSLFS5FTUFYFDVUJPOQSPDFTT w 4VJUF5FTUFYFDVUJPOVOJU UPQMFWFM TestDMBTT describe %4- pMF FUD QSPDFTTJOHUJNF $ bundle exec ruby -Itest path # or rspec path

1BSBMMFM5FTUJOH 1BSBMMFMXPSLFS 4VJUF 4VJUF 4VJUF 4VJUF
4VJUF 4VJUF 4VJUF 1BSBMMFMXPSLFS 1BSBMMFMXPSLFS QSPDFTTJOHUJNF

4FSJBMWT1BSBMMFM 1BSBMMFMXPSLFS 4VJUF 4VJUF 4VJUF 4VJUF
4VJUF 4VJUF 4VJUF 1BSBMMFMXPSLFS 1BSBMMFMXPSLFS 4JOHMFXPSLFS 4VJUF 4VJUF 4VJUF 4VJUF 4VJUF 4VJUF 4VJUF /PXBJUUJNF QSPDFTTJOHUJNF

1BSBMMFMJ[BUJPOJTHSFBU 5JNFJTNPSFWBMVBCMF UIBOBOZUIJOH

1BSBMMFMUFTUSVOOFST

w .JOJUFTU w 3BJMT w QBSBMMFM@UFTUT w UFTURVFVF 5PVSPG1BSBMMFM5FTUJOH5PPMT

NJOJUFTUNJOJUFTU NJOJUFTUQSPWJEFTBDPNQMFUFTVJUFPGUFTUJOHGBDJMJUJFT TVQQPSUJOH5%% #%% NPDLJOH BOECFODINBSLJOH

w #BDLFOEPGActiveSupport::TestCaseJO3BJMT w 3BOEPNJ[FETFSJBMUFTUJTUIFEFGBVMU w parallelize_me! QSPWJEFTQBSBMMFMUFTUJOHGPSUIF RVFVFNPEFM*UVTFTNVMUJUISFBEJOHCZEFGBVMU w :PVDBODIBOHFUIFQBSBMMFMFYFDVUJPONPEVMFCZ
Minitest.parallel_executor .JOJUFTU

&YUFOTJPOQPJOUGPSQBSBMMFMUFTUJOH .JOJUFTU QBSBMMFM@FYFDVUPS JOUFSGBDF w 5ISFBECZEFGBVMUGPSUIFQBSBMMFMUFTU w *.0 UFTUTUIBUBSFDPNQMFUFEJONFNPSZBSFOPUGBTU &⒎FDUJWFGPSUFTUTUIBUVTF*0
*OUFSGBDFGPSQBSBMMFMUFTUJOH .JOJUFTU 1BSBMMFM &YFDVUPS !RVFVF "DMBTTUIBUJNQMFNFOUT NFUIPETTVDIBTstart << size BOEshutdown BVUPSVO

EFGTUBSU !QPPMTJ[FUJNFTNBQ\ 5ISFBEOFX !RVFVF EPcRVFVFc 5ISFBEDVSSFOUBCPSU@PO@FYDFQUJPOUSVF XIJMF KPCRVFVFQPQ LMBTT
NFUIPE SFQPSUFSKPC SFQPSUFSTZODISPOJ[F\SFQPSUFSQSFSFDPSELMBTT NFUIPE^ SFTVMU.JOJUFTUSVO@POF@NFUIPELMBTT NFUIPE SFQPSUFSTZODISPOJ[F\SFQPSUFSSFDPSESFTVMU^ FOE FOE ^ FOE EFGXPSL!RVFVFXPSLFOE .JOJUFTU1BSBMMFM&YFDVUPS $SFBUFsizeOVNPGOFXUISFBEXJUI@queue -PPQXJUIqueue.pop $BMMBUFTUNFUIPE .JOJUFTU1BSBMMFM&YFDVUPS XPSL IUUQTHJUIVCDPNNJOJUFTUNJOJUFTUCMPCWMJCNJOJUFTUQBSBMMFMSC-- /05&@queueDBOCF TIBSFEXJUIBOJOTUBODF WBSJBCMFCFDBVTFJUJTB UISFBENPEFM

.JOJUFTU5FTU NPEVMF5FTUOPEPD EFG@TZODISPOJ[F.JOJUFTU5FTUJP@MPDLTZODISPOJ[F\ZJFME^ FOE NPEVMF$MBTT.FUIPETOPEPD EFGSVO@POF@NFUIPELMBTT NFUIPE@OBNF SFQPSUFS .JOJUFTUQBSBMMFM@FYFDVUPS<LMBTT NFUIPE@OBNF
SFQPSUFS> FOE EFGUFTU@PSEFS QBSBMMFM FOOOE .JOJUFTU 3VOOBCMF .JOJUFTU 5FTU IUUQTHJUIVCDPNNJOJUFTUNJOJUFTUCMPCWMJCNJOJUFTUQBSBMMFMSC-- FORVFVFUPQBSBMMFM@FYFDVUPS .JOJUFTU 1BSBMMFM5FTU $BTT.FUIPET NJYJO DMBTT.JOJUFTU5FTU.JOJUFTU3VOOBCMF FYUFOE.JOJUFTU1BSBMMFM5FTU$MBTT.FUIPET FOE

<"QQFOEJY>IFMMSC IUUQTHJUIVCDPNNJOJUFTUNJOJUFTUCMPCWMJCNJOJUFTUIFMMSC w %FpOJOHUIFFOWJSPONFOU WBSJBCMFMT_HELLBDUJWBUFT NJOJUFTUIFMMSC w "DUJWBUFQBSBMMFMBOEQSPWFJU

SBJMTSBJMT 3BJMTJTBXFCBQQMJDBUJPOGSBNFXPSLUIBUJODMVEFTFWFSZUIJOH OFFEFEUPDSFBUFEBUBCBTFCBDLFEXFCBQQMJDBUJPOT BDDPSEJOHUPUIF.PEFM7JFX$POUSPMMFS .7$ QBUUFSO

w 1BSBMMFMUFTUJOHCZEFGBVMU 4JODF3BJMT w %FGBVMUUISFTIPMEJT "WPJEJOHUIFPWFSIFBEPGGPSLJOH 3BJMTAS::TestCase $ bin/rails
test Running 60 tests in parallel using 8 processes Run options: --seed 21371 # Running: ..................................................... ....... Finished in 0.205488s, 291.9879 runs/s, 291.9879 assertions/s. 60 runs, 60 assertions, 0 failures, 0 errors, 0 skips MFTTUIBOUFTUT UFTUT $ bin/rails test Running 10 tests in a single process (parallelization threshold is 50) Run options: --seed 1398 # Running: .......... Finished in 0.016447s, 608.0136 runs/s, 608.0136 assertions/s. 10 runs, 10 assertions, 0 failures, 0 errors, 0 skips

ENV["RAILS_ENV"] ||= "test" require_relative "../config/environment" require "rails/test_help" class ActiveSupport::TestCase #
Run tests in parallel with specified workers parallelize(workers: :number_of_processors) # Setup all fixtures in test/fixtures/*.yml for all tests in alphabetical order. fixtures :all # Add more helper methods to be used by all tests here... end UFTUUFTU@IFMQFSSC QBSBMMFMJ[BUJPOCZEFGBVMU PQFODMBTT

module ActiveSupport class TestCase < ::Minitest::Test class << self def
parallelize(workers: :number_of_processors, with: :processes, threshold: ActiveSupport.test_parallelization_threshold) workers = Concurrent.physical_processor_count if workers == :number_of_processors workers = ENV["PARALLEL_WORKERS"].to_i if ENV["PARALLEL_WORKERS"] return if workers <= 1 Minitest.parallel_executor = ActiveSupport::Testing::ParallelizeExecutor.new(size: workers, with: with, threshold: threshold) ennnnnd *OIFSJUMinitest::Test $IBOHFUPQBSBMMFMFYFDVUPSPG3BJMT "DUJWF4VQQPSU5FTU$BTF

&YUFOTJPOQPJOUGPSQBSBMMFMUFTUJOH .JOJUFTU QBSBMMFM@FYFDVUPS JOUFSGBDF w 5ISFBECZEFGBVMUGPSUIFQBSBMMFMUFTU w *.0 UFTUTUIBUBSFDPNQMFUFEJONFNPSZBSFOPUGBTU &⒎FDUJWFGPSUFTUTUIBUVTF*0
*OUFSGBDFGPSQBSBMMFMUFTUJOH .JOJUFTU 1BSBMMFM &YFDVUPS !RVFVF "DMBTTUIBUJNQMFNFOUT NFUIPETTVDIBTstart << size BOEshutdown BVUPSVO

"45FTUJOH1BSBMMFM&YFDVUPS .JOJUFTU QBSBMMFM@FYFDVUPS JOUFSGBDF w 3FQMBDFXJUIUIF.JOJUFTUFYUFOTJPOQPJOU w 4XJUDIGSPNUISFBEUPQSPDFTTQBSBMMFMJ[BUJPO "OJOUFSGBDFGPSUIFQBSBMMFMUFTU .JOJUFTU
1BSBMMFM &YFDVUPS !RVFVF BVUPSVO "DUJWF4VQQPSU 5FTUJOH 1BSBMMFMJ[F&YFDVUPS !QBSBMMFM@FYFDVUPS "DUJWF4VQQPSU 5FTUJOH 1BSBMMFMJ[BUJPO

"45FTUJOH1BSBMMFMJ[BUJPO %3C'SPOU w 4FSWFS8PSLFSNPEFM NVMUJQSPDFTTJOH w *OUFSQSPDFTTDPNNVOJDBUJPOVTJOHE3VCZ 1BSBMMFMJ[BUJPO 4FSWFS
RVFVF 1BSBMMFMJ[BUJPO 8PSLFS FORVFVF GPSLFEQSPDFTT ESCVOJY O NBJOQSPDFTT 1BSBMMFMJ[BUJPO 8PSLFS EFRVFVF XIJMFKPC!RVFVFQPQ QFSGPSN@KPC KPC FOE $BMMMinitest.run_one_method 1BSBMMFMJ[BUJPO 8PSLFS 1BSBMMFMJ[BUJPO

w AS::TestCase.parallelizeCZEFGBVMU w 3BJMTQSPWJEFTUIFCFOFpUTPGNVMUJDPSF QSPDFTTJOHUISPVHIUIFVTFPGQSPDFTTFT w 3VOTQBSBMMFMJTNCZEFGBVMUXJUI Concurrent.physical_processor_count w 3VOUJNF$16TBOENFNPSZBSFJNQPSUBOU
3BJMTJTQSBHNBUJD

3BJMTEPFTOU TVQQPSU34QFDCZ EFGBVMU🤔

w HSPTTFSQBSBMMFM@UFTUT w UNNUFTURVFVF 1BSBMMFMUFTUSVOOFS

HSPTTFSQBSBMMFM@UFTUT 4QFFEVQ5FTU6OJU 34QFD $VDVNCFS 4QJOBDICZSVOOJOHQBSBMMFMPONVMUJQMF$16DPSFT

w %FQFOETPOUIFQBSBMMFMHFN • Parallel.each { ... } • Parallel.map {
... } w 3VOQSFTQMJUUJOHUFTUTJOQBSBMMFMGPSBMMUFTUpMFT w *UDBOVTFQBSBMMFM@SVOUJNF@STQFDMPHBGUFSSVOOJOH w 7FSZTJNQMF😀 QBSBMMFM@UFTUT

w 5IFSFJTBUJNFEJ⒎FSFODFCFUXFFOUIFXPSLFS UIBUpOJTIFTpSTUBOEUIFXPSLFSUIBUpOJTIFT MBTU w 'JOJTIFEXPSLFSTIBWFXBTUFEXBJUUJNF w /PUFUIBUUIFFYFDVUJPOSFTVMUNBUSJYpMFJT VTFEUPNJOJNJ[FUIFUJNFEJ⒎FSFODFCFUXFFO UFTUTBTNVDIBTQPTTJCMF
DPOTPGQSFTQMJUUJOH

UNNUFTURVFVF :FUBOPUIFSQBSBMMFMUFTUSVOOFS CVJMUVTJOHB DFOUSBMJ[FERVFVFUPFOTVSFPQUJNBMEJTUSJCVUJPOPG UFTUTCFUXFFOXPSLFST

w 5IFBVUIPSJT"NBO(VQUB,BSNBOJ !UNN w 3FBEZXPSLFSTEFRVFVFBOESVOUFTUTGSPNUIF RVFVFXBJUJOHUPCFSVO w 'FXFSXBJUJOHXPSLFSTBOEMFTTXBTUF w
4BNFBTQBSBMMFM@UFTUT VTJOHUFTU@RVFVF@TUBUT FYFDVUJPOSFTVMUNBUSJYBOESVOOJOHTMPXUFTUT pSTU UFTURVFVF

6/*94PDLFUPS5$14PDLFU 5ISFFQSPDFTTUZQFT 6/*94PDLFUPS5$14PDLFU 6/*94FSWFSPS5$14FSWFS UFTURVFVF NBTUFSQSPDFTT UFTURVFVFTVJUF EJTDPWFSZQSPDFTT UFTURVFVF
XPSLFSQSPDFTT %JTUSJCVUJOHUFTUTVJUF GSPNRVFVFBOE TVNNBSJ[FUIFSFTVMUT %FRVFVFBOESVOUFTUTVJUFT 1BSTFUFTUTVJUFTBOEFORVFVFJU 1⃣ 2⃣ 3⃣

class Runner def execute_internal start_master prepare(@concurrency) @prepared_time = Time.now start_relay
if relay? discover_suites spawn_workers distribute_queue ensure stop_master kill_subprocesses ennd 3VOOFSTUBSU@NBTUFS def start_master if !relay? if @socket =~ /\A(?:(.+):)?(\d+)\z/ address = $1 || '0.0.0.0' port = $2.to_i @socket = "#{$1}:{#$2}" @server = TCPServer.new(address, port) else FileUtils.rm_f(@socket) @server = UNIXServer.new(@socket) end ennd 1⃣ .BTUFS1SPDFTT .BTUFS1SPDFTT

if relay? discover_suites spawn_workers distribute_queue ensure stop_master kill_subprocesses ennd 3VOOFSEJTDPWFS@TVJUFT def discover_suites return if relay? return if @allowlist.any? && @awaited_suites.empty? @discovering_suites_pid = fork do terminate = false Signal.trap("INT") { terminate = true } $0 = 'test-queue suite discovery process' @test_framework.all_suite_files.each do |path| @test_framework.suites_from_file(path).each do | suite_name, suite| Kernel.exit!(0) if terminate @server.connect_address.connect do |sock| sock.puts("TOKEN=#{@run_token}") sock.puts("NEW SUITE #{Marshal.dump([suite_name, path])}") ennd 2⃣ %JTDPWFSZ1SPDFTT .BTUFS1SPDFTT

if relay? discover_suites spawn_workers distribute_queue ensure stop_master kill_subprocesses ennd 3VOOFSTQBXO@XPSLFST def spawn_workers @concurrency.times do |i| num = i + 1 pid = fork do @server.close if @server iterator = Iterator.new(@test_framework, relay? ? @relay : @socket, method(:around_filter), early_failure_limit: @early_failure_limit, run_token: @run_token) after_fork_internal(num, iterator) ret = run_worker(iterator) || 0 cleanup_worker Kernel.exit! ret end @workers[pid] = Worker.new(pid, num) ennd 3⃣ 3VOXPSLFS🏃 8PSLFS1SPDFTT .BTUFS1SPDFTT

6/*94PDLFUPS5$14PDLFU 6/*94PDLFUPS5$14PDLFU 6/*94FSWFSPS5$14FSWFS *OUFSBDUJPOPGQSPDFTTFT UFTURVFVF NBTUFSQSPDFTT UFTURVFVFTVJUF EJTDPWFSZQSPDFTT UFTURVFVF XPSLFSQSPDFTT
%FRVFVFBOESVOUFTUTVJUFT 1BSTFUFTUTVJUFTBOE FORVFVFJU %JTUSJCVUJOHUFTUTVJUF GSPNRVFVFBOE TVNNBSJ[FUIFSFTVMUT GPSL /&846*5& GPSL .BSTIBMEVNQ .BSTIBMMPBE 5FTU'SBNFXPSL 3VOOFS 101

$PNNBOEMJTU $PNNBOE %JSFDUJPO %FTDSJQUJPO /&846*5& EJTDPWFSZˠNBTUFS &ORVFVFBUFTUTVJUF 101 XPSLFSˠNBTUFS %FRVFVFBUFTUTVJUF
8"*5 NBTUFSˠXPSLFS "XBJUBUFTUTVJUFXIFOEFRVFVF ,"#00. XPSLFSˠNBTUFS *NNFEJBUFMZBCPSU 3&.05&."45&3 XPSLFSˠNBTUFS 3FHJTUFSBSFNPUFXPSLFSUPUIFSFNPUFNBTUFSJGSFMBZ 0, NBTUFSˠXPSLFS 3FQMZSFHJTUFSTVDDFTTUPUIFSFNPUFXPSLFSJGSFMBZ 803,&3 XPSLFSˠNBTUFS $PNQMFUFPSBCPSUBSFNPUFXPSLFSJGSFMBZ "MXBZTNBUDIBVOJRVFTOKENJTTVFEBOEVTFEBTBQSPUPDPMXJUIUIFNBTUFSQSPDFTT

if relay? discover_suites spawn_workers distribute_queue ensure stop_master kill_subprocesses ennd 3VOOFSEJTUSJCVUF@RVFVF def distribute_queue # snip until !awaiting_suites? && @queue.empty? && remote_workers == 0 # snip case cmd when /\APOP (\S+) (\d+)/ # snip when /\AREMOTE MASTER (\d+) ([\w\.-]+)(?: (.+))?/ # snip when /\AWORKER (\d+)/ # snip when /\ANEW SUITE (.+)/ # snip when /\AKABOOM/ break else warn("Ignoring unrecognized command: \"#{cmd}\"") end sock.close ennd .BTUFS1SPDFTT .BTUFS1SPDFTT

def each # snip loop do if @early_failure_limit && @failures
>= @early_failure_limit connect_to_master('KABOOM'); break else client = connect_to_master("POP #{Socket.gethostname} #{Process.pid}") end break if client.nil? _r, _w, e = IO.select([client], nil, [client], nil) break unless e.empty? if data = client.read(65536) client.close item = Marshal.load(data) # snip suite_name, path = item suite = load_suite(suite_name, path) # snip yield suite *UFSBUPSFBDI 8PSLFS1SPDFTT class Iterator include Enumerable

2VFVF3VOOFS SVO@TQFDT JUFSBUPS 4UBUT 3VOOFS34QFD 5FTU'SBNFXPSL 34QFD 4UBUT4VJUF OFX 3VOOFS
34QFD 3VOOFS 5FTU'SBNFXPSL *UFSBUPS SVO@TQFDT FYBNQMF@HSPVQT UFTURVFVFSVOUJNF UFTUJOHGSBNFXPSL BEBQUFSDMBTT FYFDVUF OFX BMM@TVJUF@pMFT TVJUFT@GSPN@pMF OFX &OVNFSBCMF FBDI OFX 8PSLFS OFX !RVFVF TUBUVT 🏃💨 TestQueue::Runner:: RSpec.new.execute FYFSTQFDRVFVF SVO@XPSLFS BMM@TVJUF@pMFT SVO@XPSLFS BMM@TVJUFT SVO@FBDI TVJUFT@GSPN@pMF %FTJHO34QFD

w UFTURVFVFTVQQPSUT34QFD 34QFD .JOJUFTU .JOJUFTU UFTUVOJU $VDVNCFS w UFTURVFVFQSPWJEFTUIFDPPSEJOBUJPOPG
QSPDFTTFTJODPNNPOQBSBMMFMQSPDFTTJOH w 5FTUTVJUFBOEUFTUSVOOFS"1*TBSFUFTUJOH GSBNFXPSLTQFDJpD 1MVHHBCMF%FTJHO

34QFD 3VOOFS 5FTU'SBNFXPSL *UFSBUPS SVO@TQFDT FYBNQMF@HSPVQT UFTURVFVFSVOUJNF UFTUJOHGSBNFXPSL BEBQUFSDMBTT FYFDVUF OFX BMM@TVJUF@pMFT TVJUFT@GSPN@pMF OFX &OVNFSBCMF FBDI OFX 8PSLFS OFX !RVFVF TUBUVT 3VOOFS 🏃💨 TestQueue::Runner:: RSpec.new.execute FYFSTQFDRVFVF SVO@XPSLFS BMM@TVJUF@pMFT SVO@XPSLFS BMM@TVJUFT SVO@FBDI TVJUFT@GSPN@pMF &YUFOTJPOQPJOU

34QFD 3VOOFS 5FTU'SBNFXPSL *UFSBUPS SVO@TQFDT FYBNQMF@HSPVQT UFTURVFVFSVOUJNF UFTUJOHGSBNFXPSL BEBQUFSDMBTT FYFDVUF OFX BMM@TVJUF@pMFT TVJUFT@GSPN@pMF OFX &OVNFSBCMF FBDI OFX 8PSLFS OFX !RVFVF TUBUVT 5FTU'SBNFXPSL 🏃💨 TestQueue::Runner:: RSpec.new.execute FYFSTQFDRVFVF SVO@XPSLFS BMM@TVJUF@pMFT SVO@XPSLFS BMM@TVJUFT SVO@FBDI TVJUFT@GSPN@pMF &YUFOTJPOQPJOU

UFTURVFVFMJCUFTU@RVFVFSVOOFSSTQFDSC UFTURVFVFMJCUFTU@RVFVFSVOOFSSTQFDSC 3VOOFS34QFDSVO@XPSLFS module QueueRunner < Runner def run_specs(iterator) @configuration.reporter.report(0)
do |reporter| @configuration.with_suite_hooks do iterator.map { |g| start = Time.now if g.is_a? ::RSpec::Core::Example print " #{g.full_description}: " example = g g = example.example_group ::RSpec.world.filtered_examples.clear ::RSpec.world.filtered_examples[g] = [example] else print " #{g.description}: " end ret = g.run(reporter) # snip end alias_method :run_each, :run_specs module TestQueue class Runner class RSpec < Runner def run_worker(iterator) rspec = ::RSpec::Core::QueueRunner.new rspec.run_each(iterator).to_i ennnnd 3VOTVJUF Iterator#each☝ /BJWFJNQMFNFOUBUJPO UIBUEFQFOETPO34QFDT JOUFSOBM"1*

4UBUT 3VOOFS.JOJUFTU 5FTU'SBNFXPSL .JOJUFTU 4UBUT4VJUF .JOJUFTU .JOJUFTU 3VOOFS 5FTU'SBNFXPSL *UFSBUPS
SVOOBCMFT JUFSBUPS UFTURVFVFSVOUJNF UFTUJOHGSBNFXPSL BEBQUFSDMBTT FYFDVUF OFX BMM@TVJUF@pMFT TVJUFT@GSPN@pMF BMM@TVJUF@pMFT &OVNFSBCMF SVO@XPSLFS BMM@TVJUFT OFX 8PSLFS OFX !RVFVF TUBUVT FYFNJOJUFTURVFVF %FTJHO.JOJUFTU SVO 🏃💨 OFX TestQueue::Runner:: Minitest.new.execute SVO@XPSLFS FBDI OFX TVJUFT@GSPN@pMF

NJOJUFTUMJCNJOJUFTUSC module Minitest def self.run args = [] # snip
begin __run reporter, options rescue Interrupt warn 'Interrupted. Exiting...' UFTURVFVFMJCUFTU@RVFVFSVOOFSNJOJUFTUSC module Minitest def self.__run reporter, options suites = Runnable.runnables suites.map { |suite| suite.run reporter, options } ennd module TestQueue class Runner class Minitest < Runner def run_worker(iterator) ::Minitest::Test.runnables = iterator ::Minitest.run ? 0 : 1 ennnnd 3VOOFS.JOJUFTUSVO@XPSLFS $BMM__run 3VOTVJUF Iterator#each☝ /BJWFJNQMFNFOUBUJPO UIBUEFQFOETPOPQFO DMBTTBOEJOUFSOBM"1* 0QFODMBTT

4VQQPSUUFTUJOHGSBNFXPSLT IUUQTHJUIVCDPNUNNUFTURVFVFUSFFNBTUFSMJCUFTU@RVFVFSVOOFS

1BSBMMFM5FTUJOH "HBJO 1BSBMMFMXPSLFS 4VJUF 4VJUF 4VJUF 4VJUF
4VJUF 4VJUF 4VJUF 1BSBMMFMXPSLFS 1BSBMMFMXPSLFS QSPDFTTJOHUJNF

DPSF$16T )ZQFS5ISFBEJOH TFD TFSJBM QBSBMMFM
w8IFOUFTURVFVF SVOTBMPUPGUFTU TVJUFT JUXJMMCF FGGFDUJWF wFHTFDTFD JO3VCP$PQSFQP

% bundle exec rake spec Starting test-queue master (/tmp/test_queue_40616_5060.sock) ==>
Summary (16 workers in 51.9441s) [ 1] 9 examples, 0 failures 1 suites in 34.0151s (pid 40649 exit 0 ) [ 2] 109 examples, 0 failures 1 suites in 34.0151s (pid 40650 exit 0 ) [ 3] 712 examples, 0 failures 46 suites in 34.0149s (pid 40651 exit 0 ) [ 4] 2449 examples, 0 failures 61 suites in 34.0151s (pid 40652 exit 0 ) [ 5] 129 examples, 0 failures 13 suites in 34.0151s (pid 40653 exit 0 ) [ 6] 1628 examples, 0 failures 57 suites in 34.0150s (pid 40654 exit 0 ) [ 7] 107 examples, 0 failures 1 suites in 34.0151s (pid 40655 exit 0 ) [ 8] 1647 examples, 0 failures 61 suites in 34.0147s (pid 40656 exit 0 ) [ 9] 2270 examples, 0 failures 59 suites in 34.0145s (pid 40657 exit 0 ) [10] 1660 examples, 1 pending, 0 failures 57 suites in 34.0145s (pid 40658 exit 0 ) [11] 132 examples, 0 failures 1 suites in 51.9309s (pid 40659 exit 0 ) [12] 1600 examples, 0 failures 53 suites in 51.9306s (pid 40660 exit 0 ) [13] 1924 examples, 0 failures 60 suites in 51.9295s (pid 40661 exit 0 ) [14] 1993 examples, 0 failures 59 suites in 51.9281s (pid 40662 exit 0 ) [15] 1967 examples, 0 failures 60 suites in 51.9271s (pid 40663 exit 0 ) [16] 2303 examples, 0 failures 57 suites in 51.9237s (pid 40664 exit 0 ) 4MPXFTUBOEGBTUFTU 4MPXFTU 'BTUFTU

)PXTIPVMEUIF SVOOJOHPGUFTUT B⒎FDUPOFBOPUIFS /PUBUBMM 5&45%3*7&/%&7&-01.&/5#:&9".1-& l *TPMBUFE5FTUJTJNQPSUBOU

w *TPMBUFUFTUTGSPNFBDIPUIFS w %BUBTUPSBHFBDDFTTFECZQBSBMMFM XPSLFSTTVDIBT3%#.4TIPVMECFTFU VQJOBQBSBMMFMNBOOFS w FH8PSLFS#SFBETUIFVQEBUFEEBUBPG XPSLFS"BOEUIFUFTUNBZGBJM 4FUVQGPSQBSBMMFMUFTUJOH

w 1SPEVDUDPEFBOEUFTUDPEFGPSNBUPUFTUJOHQBJS w 5IFVOJUPGQBSBMMFMUBTLTJOQBSBMMFMUFTUTJTVTVBMMZ UIFpMFMFWFM w 1BSBMMFMJ[JOHUFTUTVJUFTJOTNBMMFSVOJUTDBOSFEVDF UIFMPDLJOHQFSJPEGPSXPSLFST w *OPUIFSXPSET
GPMMPXJOH431MFBETUPGBTUFS QBSBMMFMUFTUT 1BSBMMFMUFTUBOE431 4JOHMF3FTQPOTJCJMJUZ1SJODJQMF

Parallel test runners Using RSpec Using Rails 6+ or AS
Using Vanilla Ruby No Yes Yes No Yes No Start AS:: TestCase parallel_tests or test-queue

8IJDIPOFJTCFUUFSUPVTF QBSBMMFM@UFTUTPSUFTURVFVF

w /VNCFSPGEPXOMPBETPOSVCZHFNTPSH w /VNCFSPGTUBSTPO(JU)VC w 'SFRVFODZPGNBJOUFOBODF w QBSBMMFM@UFTUTJTNBJOUBJOFECVUUFTURVFVF XBTVONBJOUBJOFE w
*OUFSFTUJOHEFTJHOCVUVOEFSVUJMJ[FE $SJUFSJBGPSHFNTFMFDUJPO

3JTFVQ

3FTVSSFDUJPOPGUFTURVFVF

w *OFFEUPSFMFBTFBOFXWFSTJPO w !UNNBTLFENFUPCFBNBJOUBJOFSBOE *BDDFQUFE w .BJOUFOBODFJTSFTVNFEBGUFSPS ZFBST 'JSTUUIJOHT*EJEBTBNBJOUBJOFS

*GMFGUVOBUUFOEFE TPGUXBSFXJMMCSFBL CZ!ITCU XBTSFMFBTFE CVUTUJMM$*JTCSPLFO

/P$* OPNFSHF $*XBTCSPLFO

w *GPSHPUIPXUPVTF5SBWJT$* w *NOPUVQUPEBUFPOUIFSFDFOU EFWFMPQNFOUTPG5SBWJT$* w 8JUI(JU)VC"DUJPOT NBJOUFOBODFDBO CFEPOFXJUIPVUNF 'SPN5SBWJT$*UP(JU)VC"DUJPOT

w #BUT#BTI"VUPNBUFE5FTUJOH4ZTUFN w !TTUFQIFOTPOQSPEVDU w 5IFSFQPTJUPSZIBTCFFOBSDIJWFECZUIF PXOFSPO"QS 😢 w *UTIBSEUPSFQMBDFBUFTUJOHGSBNFXPSL
w *UTOFHBUJWF CVU*NMFBWJOHJUBTJUJT &&UFTUJOHPGUFTURVFVF

w 5FTUOPUQBTTJOHXJUIMBUFTU WFSTJPOPG.JOJUFTU w 5IFUFTUDPEFBSPVOEUIF sleepNFUIPETFFNT TVTQJDJPVT w 'JYGBJMJOH(FNpMFGPS.JOJUFTU CZWFSTJPOQJOOJOH
$VSSFOUTUBUVTPG&&UFTUJOH

$*IBTSFDPWFSFE

%FWFMPQNFOU FOWJSPONFOUJTSFBEZ *GUIF$*QBTTFT NBZCF*DBONFSHFJU

#FZPOE UIF3FTVSSFDUJPO

4VQQPSUTUBUVTPGUFTUJOHGSBNFXPSLT 5FTUJOH'SBNFXPSL $* /PUF 34QFD ✅ 🤔5PPPME34QFDEPFTOUXPSLXJUI3VCZ 34QFD ✅ ✅6TFEJO3VCP$PQ
.JOJUFTU ✅ 🤔5PPPME .JOJUFTU ✅ CVU ✅6TFEJO3VCP$PQ.JOJUFTU $VDVNCFS ✅ CVU ❌0VUEBUFEUFTUDPEFFSSPSTJOUIFMBUFTUWFSTJPO UFTUVOJU ✅ 🤔/PUXPSLJOHXJUI3BJMT UNNUFTURVFVF 34QFD EFW ✅ ✅6TFEJO3VCP$PQ 5VSOJQ ✅ ✅/PUSBDLSFDPSEBTJUXBTKVTUNFSHFE

w "QVMMSFRVFTUIBTCFFOPQFOFECZB34QFD DPSFUFBNNFNCFS w 34QFDJTBEFWFMPQNFOUTUBUVT #VU *EFDJEFEUPQSPWJEFBGFBUVSFPGUFTURVFVF w 050) JUTIBSEGPSUFTURVFVFNBJOUBJOFST
UPLOPX34QFDTJOUFSOBMJNQMFNFOUBUJPO 4VQQPSU34QFD EFW

w 8PVMEOUJUCFDPOWFOJFOUJG34QFDIBEB QBSBMMFMUFTUSVOOFSGFBUVSFMJLF.JOJUFTU w 5IFTBNFDBOCFTBJEGPSUFTUVOJUBOE PUIFST *OUSPEVDFQBSBMMFMUFTUSVOOFSUP34QFD 0QQPSUVOJUZGPSDPOUSJCVUJPO

0OFNPSFUIJOH

w "MSFBEZJODMVEFTPuppetLintBTB QBSBMMFMSVOOFS w 8IJMFUIFQBSBMMFMHFNJTVTFEGPSQBSBMMFM FYFDVUJPOPG3VCP$PQ VTJOHB NFDIBOJTNMJLFUFTURVFVFNBZQSPWJEF FWFOGBTUFSSFTVMUT 2VFVFTZTUFNOPUKVTUGPSUFTUJOH

w 1SPHSBNNJOHGPSJOUFSQSPDFTT DPNNVOJDBUJPOJTBCBTJDGPVOEBUJPOGPS MFWFSBHJOHNVMUJDPSF 'VOEBNFOUBMTPGQBSBMMFMQSPHSBNNJOH

$PODMVTJPO

w UFTURVFVFEFQFOETPOJOUFSOBM"1*PGFBDI UFTUJOHGSBNFXPSL w *XBOUJUUPCFUFNQPSBSZUSBOTJFOU w .BZOPUSFBDIUFTURVFVF w )BWJOHBQMVHHBCMFQBSBMMFMUFTUNFDIBOJTNMJLF .JOJUFTUDPVMECFDPOWFOJFOU
FH34QFDUPP )PQFGPSQBSBMMFMUFTUSVOOFS

w 1BSBMMFMUFTUJOHSFRVJSFTNPOFZ💰 w %FWFMPQFSTBSFUIFNPTUWBMVBCMFSFTPVSDF JOUIFQSPKFDU w *UTFBTJFSUPHFUNPSFNFNPSZBOE$16T XJUINPOFZUIBOUPIJSFTFOJPSFOHJOFFST w %PZPVSCFTUUPTFDVSFUIFCVEHFU
5IFSFJTBHPMECVMMFU

1BSBMMFM5FTUJOH .BLFT :PV)BQQZ (JU)VC!LPJD

The Resurrection of the Fast Parallel Test Runner

The Resurrection of the Fast Parallel Test Runner

More Decks by Koichi ITO

Other Decks in Programming

Featured

Transcript

The Resurrection of  the Fast Parallel Test Runner

The Resurrection of  the Fast Parallel Test Runner