The Resurrection of  the Fast Parallel Test Runner

Slide 1

Slide 1 text

Koichi ITO / ESM, Inc. RubyKaigi 2023 The Resurrection of the Fast Parallel Test Runner May, 12th, 2023 Matsumoto Performing Arts Centre The Force Awakens

Slide 2

Slide 2 text

!LPJD w 044QSPHSBNNFS w 3VCP$PQDPSFUFBN w &OHJOFFSJOH.BOBHFSBOE %JTUJOHVJTIFE&OHJOFFSPG&4. *OD w 3VCZ,BJHJTQFBLFSBU -5 5BLFPVU BOE

Slide 3

Slide 3 text

.BJOUBJO044&WFSZ%BZ IUUQTHJUIVCDPNLPJD

Slide 4

Slide 4 text

&4. *OD

Slide 5

Slide 5 text

$P⒎FFIPVTF4QPOTPS w"CF ᘖᘣඒֶΞϕ w0LJOBEP ͓͖ͳಊ w:BNBHB ٤஡ࢁխ w"OENPSF ☕

Slide 6

Slide 6 text

/PWFMUZ

Slide 7

Slide 7 text

4 Q FBLFS 4 Q FBLFS JOQ FSTPO JOQ FSTPO JOQ FSTPO JOQ FSTPO JOQ FSTPO JOQ FSTPO POMJOF POMJOF POMJOF POMJOF POMJOF POMJOF POMJOF 4QFBLFSTBOE"UUFOEFFT PSHBOJ[FS PSHBOJ[FS "EWJTPS "EWJTPS

Slide 8

Slide 8 text

Support OSS community

Slide 9

Slide 9 text

+PJOVT 🗾w-FUTHSPXUPHFUIFS w:PVDBOXPSL SFNPUFMZGSPN BOZXIFSFJO+BQBO w4FFBHJMFFTNDPKQ

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

$POUFOUT 8IBUTQBSBMMFMUFTUJOH 1BSBMMFMUFTUSVOOFST 3FTVSSFDUJPOPGUFTURVFVF

Slide 13

Slide 13 text

8IBUTQBSBMMFMUFTUJOH

Slide 14

Slide 14 text

5PNF MFHBDZDPEF JTTJNQMZDPEF XJUIPVUUFTUT 803,*/(&''&$5*7&-:8*5)-&("$:$0%& l "CPPLJOUIF6ODMF#PCTFSJFTTBZT

Slide 15

Slide 15 text

w 4PGUXBSFDPEFLFFQTHSPXJOHMBSHFS w #VTJOFTTHSPXUIBOETPGUXBSFHSPXUI HPIBOEJOIBOE w $POUJOVPVTMZQFSGPSN5%% #%% 5"% BOESFHSFTTJPOUFTUT 5FTUDPEFJTJNQPSUBOU

Slide 16

Slide 16 text

)PXMPOHEPZPV XBJUGPSZPVSCVJME

Slide 17

Slide 17 text

⏳ "JNGPS NJOVUFTPSMFTT

Slide 18

Slide 18 text

w FYBNQMFT 6TJOH34QFD w "SFBMXPSMEBQQMJDBUJPOUBLFPWFSBO IPVS "SFBMXPSMEBQQMJDBUJPO $ bin/rails stats (snip) Code LOC: 12796 Test LOC: 23518 Code to Test Ratio: 1:1.8

Slide 19

Slide 19 text

"OFXQSPCMFN 4MPXUFTU

Slide 20

Slide 20 text

w $BOOPUSVOUFTUTPOQVMMSFRVFTUTJO BEBZ w 5IFCVJMETBSFOUpOJTIFEFWFOCZ UIFOFYUNPSOJOH *UUBLFTBOIPVSUPDPNQMFUFBCVJME

Slide 21

Slide 21 text

Slide 22

Slide 22 text

w TFDPOETJOBEBZ w 'PSFYBNQMF TMPXUFTUTUBLJOH TFDPOETFBDIXJMMSFRVJSFNJOVUFT w )PXNBOZDPNNJUTEPFTZPVSUFBN NBLFFWFSZEBZ 5JNFJTJNQPSUBOU

Slide 23

Slide 23 text

w 1SPpMJOHCBTFEUVOJOH%POUHVFTT NFBTVSF • FH w .BZCFTQFEVQXJUIBDMFBOJOHTUSBUFHZ w FH%BUBCBTF$MFBOFSP⒎FSTEBUBCBTFDMFBOJOH TUSBUFHJFTTVDIBTtransaction deletion BOE truncation JUTBHPPEJEFBUPDIPPTFUIFPQUJNBMPOF "EESFTTJOH4MPX5FTU#PUUMFOFDLT $ rspec -p 30 spec/acceptance

Slide 24

Slide 24 text

.ZSFDPNNFOEBUJPO w 'JSTU QSFQBSFNPOFZ💰JGZPVBSF CVJMEJOHCVTJOFTTTPGUXBSF w /FYU QSFQBSFFOPVHI$16TBOENFNPSZ w 'JOBMMZ 1BSBMMFMJ[FZPVSUFTUT w 5IBUTJU"'"*, JUTUIFNPTUF⒎FDUJWF

Slide 25

Slide 25 text

1BSBMMFM5FTUJOH

Slide 26

Slide 26 text

4FSJBM5FTUJOH 4JOHMFXPSLFS 4VJUF 4VJUF 4VJUF 4VJUF 4VJUF 4VJUF 4VJUF w 8PSLFS5FTUFYFDVUJPOQSPDFTT w 4VJUF5FTUFYFDVUJPOVOJU UPQMFWFM TestDMBTT describe %4- pMF FUD QSPDFTTJOHUJNF $ bundle exec ruby -Itest path # or rspec path

Slide 27

Slide 27 text

1BSBMMFM5FTUJOH 1BSBMMFMXPSLFS 4VJUF 4VJUF 4VJUF 4VJUF 4VJUF 4VJUF 4VJUF 1BSBMMFMXPSLFS 1BSBMMFMXPSLFS QSPDFTTJOHUJNF

Slide 28

Slide 28 text

4FSJBMWT1BSBMMFM 1BSBMMFMXPSLFS 4VJUF 4VJUF 4VJUF 4VJUF 4VJUF 4VJUF 4VJUF 1BSBMMFMXPSLFS 1BSBMMFMXPSLFS 4JOHMFXPSLFS 4VJUF 4VJUF 4VJUF 4VJUF 4VJUF 4VJUF 4VJUF /PXBJUUJNF QSPDFTTJOHUJNF

Slide 29

Slide 29 text

1BSBMMFMJ[BUJPOJTHSFBU 5JNFJTNPSFWBMVBCMF UIBOBOZUIJOH

Slide 30

Slide 30 text

1BSBMMFMUFTUSVOOFST

Slide 31

Slide 31 text

w .JOJUFTU w 3BJMT w QBSBMMFM@UFTUT w UFTURVFVF 5PVSPG1BSBMMFM5FTUJOH5PPMT

Slide 32

Slide 32 text

NJOJUFTUNJOJUFTU NJOJUFTUQSPWJEFTBDPNQMFUFTVJUFPGUFTUJOHGBDJMJUJFT TVQQPSUJOH5%% #%% NPDLJOH BOECFODINBSLJOH

Slide 33

Slide 33 text

w #BDLFOEPGActiveSupport::TestCaseJO3BJMT w 3BOEPNJ[FETFSJBMUFTUJTUIFEFGBVMU w parallelize_me! QSPWJEFTQBSBMMFMUFTUJOHGPSUIF RVFVFNPEFM*UVTFTNVMUJUISFBEJOHCZEFGBVMU w :PVDBODIBOHFUIFQBSBMMFMFYFDVUJPONPEVMFCZ Minitest.parallel_executor .JOJUFTU

Slide 34

Slide 34 text

&YUFOTJPOQPJOUGPSQBSBMMFMUFTUJOH .JOJUFTU QBSBMMFM@FYFDVUPS JOUFSGBDF w 5ISFBECZEFGBVMUGPSUIFQBSBMMFMUFTU w *.0 UFTUTUIBUBSFDPNQMFUFEJONFNPSZBSFOPUGBTU &⒎FDUJWFGPSUFTUTUIBUVTF*0 *OUFSGBDFGPSQBSBMMFMUFTUJOH .JOJUFTU 1BSBMMFM &YFDVUPS !RVFVF "DMBTTUIBUJNQMFNFOUT NFUIPETTVDIBTstart << size BOEshutdown BVUPSVO

Slide 35

Slide 35 text

EFGTUBSU !QPPMTJ[FUJNFTNBQ\ 5ISFBEOFX !RVFVF EPcRVFVFc 5ISFBEDVSSFOUBCPSU@PO@FYDFQUJPOUSVF XIJMF KPCRVFVFQPQ LMBTT NFUIPE SFQPSUFSKPC SFQPSUFSTZODISPOJ[F\SFQPSUFSQSFSFDPSELMBTT NFUIPE^ SFTVMU.JOJUFTUSVO@POF@NFUIPELMBTT NFUIPE SFQPSUFSTZODISPOJ[F\SFQPSUFSSFDPSESFTVMU^ FOE FOE ^ FOE EFGXPSL!RVFVFXPSLFOE .JOJUFTU1BSBMMFM&YFDVUPS $SFBUFsizeOVNPGOFXUISFBEXJUI@queue -PPQXJUIqueue.pop $BMMBUFTUNFUIPE .JOJUFTU1BSBMMFM&YFDVUPS XPSL IUUQTHJUIVCDPNNJOJUFTUNJOJUFTUCMPCWMJCNJOJUFTUQBSBMMFMSC-- /05&@queueDBOCF TIBSFEXJUIBOJOTUBODF WBSJBCMFCFDBVTFJUJTB UISFBENPEFM

Slide 36

Slide 36 text

.JOJUFTU5FTU NPEVMF5FTUOPEPD EFG@TZODISPOJ[F.JOJUFTU5FTUJP@MPDLTZODISPOJ[F\ZJFME^ FOE NPEVMF$MBTT.FUIPETOPEPD EFGSVO@POF@NFUIPELMBTT NFUIPE@OBNF SFQPSUFS .JOJUFTUQBSBMMFM@FYFDVUPS FOE EFGUFTU@PSEFS QBSBMMFM FOOOE .JOJUFTU 3VOOBCMF .JOJUFTU 5FTU IUUQTHJUIVCDPNNJOJUFTUNJOJUFTUCMPCWMJCNJOJUFTUQBSBMMFMSC-- FORVFVFUPQBSBMMFM@FYFDVUPS .JOJUFTU 1BSBMMFM5FTU $BTT.FUIPET NJYJO DMBTT.JOJUFTU5FTU.JOJUFTU3VOOBCMF FYUFOE.JOJUFTU1BSBMMFM5FTU$MBTT.FUIPET FOE

Slide 37

Slide 37 text

<"QQFOEJY>IFMMSC IUUQTHJUIVCDPNNJOJUFTUNJOJUFTUCMPCWMJCNJOJUFTUIFMMSC w %FpOJOHUIFFOWJSPONFOU WBSJBCMFMT_HELLBDUJWBUFT NJOJUFTUIFMMSC w "DUJWBUFQBSBMMFMBOEQSPWFJU

Slide 38

Slide 38 text

SBJMTSBJMT 3BJMTJTBXFCBQQMJDBUJPOGSBNFXPSLUIBUJODMVEFTFWFSZUIJOH OFFEFEUPDSFBUFEBUBCBTFCBDLFEXFCBQQMJDBUJPOT BDDPSEJOHUPUIF.PEFM7JFX$POUSPMMFS .7$ QBUUFSO

Slide 39

Slide 39 text

w 1BSBMMFMUFTUJOHCZEFGBVMU 4JODF3BJMT w %FGBVMUUISFTIPMEJT "WPJEJOHUIFPWFSIFBEPGGPSLJOH 3BJMTAS::TestCase $ bin/rails test Running 60 tests in parallel using 8 processes Run options: --seed 21371 # Running: ..................................................... ....... Finished in 0.205488s, 291.9879 runs/s, 291.9879 assertions/s. 60 runs, 60 assertions, 0 failures, 0 errors, 0 skips MFTTUIBOUFTUT UFTUT $ bin/rails test Running 10 tests in a single process (parallelization threshold is 50) Run options: --seed 1398 # Running: .......... Finished in 0.016447s, 608.0136 runs/s, 608.0136 assertions/s. 10 runs, 10 assertions, 0 failures, 0 errors, 0 skips

Slide 40

Slide 40 text

ENV["RAILS_ENV"] ||= "test" require_relative "../config/environment" require "rails/test_help" class ActiveSupport::TestCase # Run tests in parallel with specified workers parallelize(workers: :number_of_processors) # Setup all fixtures in test/fixtures/*.yml for all tests in alphabetical order. fixtures :all # Add more helper methods to be used by all tests here... end UFTUUFTU@IFMQFSSC QBSBMMFMJ[BUJPOCZEFGBVMU PQFODMBTT

Slide 41

Slide 41 text

module ActiveSupport class TestCase < ::Minitest::Test class << self def parallelize(workers: :number_of_processors, with: :processes, threshold: ActiveSupport.test_parallelization_threshold) workers = Concurrent.physical_processor_count if workers == :number_of_processors workers = ENV["PARALLEL_WORKERS"].to_i if ENV["PARALLEL_WORKERS"] return if workers <= 1 Minitest.parallel_executor = ActiveSupport::Testing::ParallelizeExecutor.new(size: workers, with: with, threshold: threshold) ennnnnd *OIFSJUMinitest::Test $IBOHFUPQBSBMMFMFYFDVUPSPG3BJMT "DUJWF4VQQPSU5FTU$BTF

Slide 42

Slide 42 text

Slide 43

Slide 43 text

"45FTUJOH1BSBMMFM&YFDVUPS .JOJUFTU QBSBMMFM@FYFDVUPS JOUFSGBDF w 3FQMBDFXJUIUIF.JOJUFTUFYUFOTJPOQPJOU w 4XJUDIGSPNUISFBEUPQSPDFTTQBSBMMFMJ[BUJPO "OJOUFSGBDFGPSUIFQBSBMMFMUFTU .JOJUFTU 1BSBMMFM &YFDVUPS !RVFVF BVUPSVO "DUJWF4VQQPSU 5FTUJOH 1BSBMMFMJ[F&YFDVUPS !QBSBMMFM@FYFDVUPS "DUJWF4VQQPSU 5FTUJOH 1BSBMMFMJ[BUJPO

Slide 44

Slide 44 text

"45FTUJOH1BSBMMFMJ[BUJPO %3C'SPOU w 4FSWFS8PSLFSNPEFM NVMUJQSPDFTTJOH w *OUFSQSPDFTTDPNNVOJDBUJPOVTJOHE3VCZ 1BSBMMFMJ[BUJPO 4FSWFS RVFVF 1BSBMMFMJ[BUJPO 8PSLFS FORVFVF GPSLFEQSPDFTT ESCVOJY O NBJOQSPDFTT 1BSBMMFMJ[BUJPO 8PSLFS EFRVFVF XIJMFKPC!RVFVFQPQ QFSGPSN@KPC KPC FOE $BMMMinitest.run_one_method 1BSBMMFMJ[BUJPO 8PSLFS 1BSBMMFMJ[BUJPO

Slide 45

Slide 45 text

w AS::TestCase.parallelizeCZEFGBVMU w 3BJMTQSPWJEFTUIFCFOFpUTPGNVMUJDPSF QSPDFTTJOHUISPVHIUIFVTFPGQSPDFTTFT w 3VOTQBSBMMFMJTNCZEFGBVMUXJUI Concurrent.physical_processor_count w 3VOUJNF$16TBOENFNPSZBSFJNQPSUBOU 3BJMTJTQSBHNBUJD

Slide 46

Slide 46 text

3BJMTEPFTOU TVQQPSU34QFDCZ EFGBVMU🤔

Slide 47

Slide 47 text

w HSPTTFSQBSBMMFM@UFTUT w UNNUFTURVFVF 1BSBMMFMUFTUSVOOFS

Slide 48

Slide 48 text

HSPTTFSQBSBMMFM@UFTUT 4QFFEVQ5FTU6OJU34QFD$VDVNCFS 4QJOBDICZSVOOJOHQBSBMMFMPONVMUJQMF$16DPSFT

Slide 49

Slide 49 text

w %FQFOETPOUIFQBSBMMFMHFN • Parallel.each { ... } • Parallel.map { ... } w 3VOQSFTQMJUUJOHUFTUTJOQBSBMMFMGPSBMMUFTUpMFT w *UDBOVTFQBSBMMFM@SVOUJNF@STQFDMPHBGUFSSVOOJOH w 7FSZTJNQMF😀 QBSBMMFM@UFTUT

Slide 50

Slide 50 text

w 5IFSFJTBUJNFEJ⒎FSFODFCFUXFFOUIFXPSLFS UIBUpOJTIFTpSTUBOEUIFXPSLFSUIBUpOJTIFT MBTU w 'JOJTIFEXPSLFSTIBWFXBTUFEXBJUUJNF w /PUFUIBUUIFFYFDVUJPOSFTVMUNBUSJYpMFJT VTFEUPNJOJNJ[FUIFUJNFEJ⒎FSFODFCFUXFFO UFTUTBTNVDIBTQPTTJCMF DPOTPGQSFTQMJUUJOH

Slide 51

Slide 51 text

UNNUFTURVFVF :FUBOPUIFSQBSBMMFMUFTUSVOOFS CVJMUVTJOHB DFOUSBMJ[FERVFVFUPFOTVSFPQUJNBMEJTUSJCVUJPOPG UFTUTCFUXFFOXPSLFST

Slide 52

Slide 52 text

w 5IFBVUIPSJT"NBO(VQUB,BSNBOJ !UNN w 3FBEZXPSLFSTEFRVFVFBOESVOUFTUTGSPNUIF RVFVFXBJUJOHUPCFSVO w 'FXFSXBJUJOHXPSLFSTBOEMFTTXBTUF w 4BNFBTQBSBMMFM@UFTUT VTJOHUFTU@RVFVF@TUBUT FYFDVUJPOSFTVMUNBUSJYBOESVOOJOHTMPXUFTUT pSTU UFTURVFVF

Slide 53

Slide 53 text

6/*94PDLFUPS5$14PDLFU 5ISFFQSPDFTTUZQFT 6/*94PDLFUPS5$14PDLFU 6/*94FSWFSPS5$14FSWFS UFTURVFVF NBTUFSQSPDFTT UFTURVFVFTVJUF EJTDPWFSZQSPDFTT UFTURVFVF XPSLFSQSPDFTT %JTUSJCVUJOHUFTUTVJUF GSPNRVFVFBOE TVNNBSJ[FUIFSFTVMUT %FRVFVFBOESVOUFTUTVJUFT 1BSTFUFTUTVJUFTBOEFORVFVFJU 1⃣ 2⃣ 3⃣

Slide 54

Slide 54 text

class Runner def execute_internal start_master prepare(@concurrency) @prepared_time = Time.now start_relay if relay? discover_suites spawn_workers distribute_queue ensure stop_master kill_subprocesses ennd 3VOOFSTUBSU@NBTUFS def start_master if !relay? if @socket =~ /\A(?:(.+):)?(\d+)\z/ address = $1 || '0.0.0.0' port = $2.to_i @socket = "#{$1}:{#$2}" @server = TCPServer.new(address, port) else FileUtils.rm_f(@socket) @server = UNIXServer.new(@socket) end ennd 1⃣ .BTUFS1SPDFTT .BTUFS1SPDFTT

Slide 55

Slide 55 text

class Runner def execute_internal start_master prepare(@concurrency) @prepared_time = Time.now start_relay if relay? discover_suites spawn_workers distribute_queue ensure stop_master kill_subprocesses ennd 3VOOFSEJTDPWFS@TVJUFT def discover_suites return if relay? return if @allowlist.any? && @awaited_suites.empty? @discovering_suites_pid = fork do terminate = false Signal.trap("INT") { terminate = true } $0 = 'test-queue suite discovery process' @test_framework.all_suite_files.each do |path| @test_framework.suites_from_file(path).each do | suite_name, suite| Kernel.exit!(0) if terminate @server.connect_address.connect do |sock| sock.puts("TOKEN=#{@run_token}") sock.puts("NEW SUITE #{Marshal.dump([suite_name, path])}") ennd 2⃣ %JTDPWFSZ1SPDFTT .BTUFS1SPDFTT

Slide 56

Slide 56 text

class Runner def execute_internal start_master prepare(@concurrency) @prepared_time = Time.now start_relay if relay? discover_suites spawn_workers distribute_queue ensure stop_master kill_subprocesses ennd 3VOOFSTQBXO@XPSLFST def spawn_workers @concurrency.times do |i| num = i + 1 pid = fork do @server.close if @server iterator = Iterator.new(@test_framework, relay? ? @relay : @socket, method(:around_filter), early_failure_limit: @early_failure_limit, run_token: @run_token) after_fork_internal(num, iterator) ret = run_worker(iterator) || 0 cleanup_worker Kernel.exit! ret end @workers[pid] = Worker.new(pid, num) ennd 3⃣ 3VOXPSLFS🏃 8PSLFS1SPDFTT .BTUFS1SPDFTT

Slide 57

Slide 57 text

6/*94PDLFUPS5$14PDLFU 6/*94PDLFUPS5$14PDLFU 6/*94FSWFSPS5$14FSWFS *OUFSBDUJPOPGQSPDFTTFT UFTURVFVF NBTUFSQSPDFTT UFTURVFVFTVJUF EJTDPWFSZQSPDFTT UFTURVFVF XPSLFSQSPDFTT %FRVFVFBOESVOUFTUTVJUFT 1BSTFUFTUTVJUFTBOE FORVFVFJU %JTUSJCVUJOHUFTUTVJUF GSPNRVFVFBOE TVNNBSJ[FUIFSFTVMUT GPSL /&846*5& GPSL .BSTIBMEVNQ .BSTIBMMPBE 5FTU'SBNFXPSL 3VOOFS 101

Slide 58

Slide 58 text

$PNNBOEMJTU $PNNBOE %JSFDUJPO %FTDSJQUJPO /&846*5& EJTDPWFSZˠNBTUFS &ORVFVFBUFTUTVJUF 101 XPSLFSˠNBTUFS %FRVFVFBUFTUTVJUF 8"*5 NBTUFSˠXPSLFS "XBJUBUFTUTVJUFXIFOEFRVFVF ,"#00. XPSLFSˠNBTUFS *NNFEJBUFMZBCPSU 3&.05&."45&3 XPSLFSˠNBTUFS 3FHJTUFSBSFNPUFXPSLFSUPUIFSFNPUFNBTUFSJGSFMBZ 0, NBTUFSˠXPSLFS 3FQMZSFHJTUFSTVDDFTTUPUIFSFNPUFXPSLFSJGSFMBZ 803,&3 XPSLFSˠNBTUFS $PNQMFUFPSBCPSUBSFNPUFXPSLFSJGSFMBZ "MXBZTNBUDIBVOJRVFTOKENJTTVFEBOEVTFEBTBQSPUPDPMXJUIUIFNBTUFSQSPDFTT

Slide 59

Slide 59 text

class Runner def execute_internal start_master prepare(@concurrency) @prepared_time = Time.now start_relay if relay? discover_suites spawn_workers distribute_queue ensure stop_master kill_subprocesses ennd 3VOOFSEJTUSJCVUF@RVFVF def distribute_queue # snip until !awaiting_suites? && @queue.empty? && remote_workers == 0 # snip case cmd when /\APOP (\S+) (\d+)/ # snip when /\AREMOTE MASTER (\d+) ([\w\.-]+)(?: (.+))?/ # snip when /\AWORKER (\d+)/ # snip when /\ANEW SUITE (.+)/ # snip when /\AKABOOM/ break else warn("Ignoring unrecognized command: \"#{cmd}\"") end sock.close ennd .BTUFS1SPDFTT .BTUFS1SPDFTT

Slide 60

Slide 60 text

def each # snip loop do if @early_failure_limit && @failures >= @early_failure_limit connect_to_master('KABOOM'); break else client = connect_to_master("POP #{Socket.gethostname} #{Process.pid}") end break if client.nil? _r, _w, e = IO.select([client], nil, [client], nil) break unless e.empty? if data = client.read(65536) client.close item = Marshal.load(data) # snip suite_name, path = item suite = load_suite(suite_name, path) # snip yield suite *UFSBUPSFBDI 8PSLFS1SPDFTT class Iterator include Enumerable

Slide 61

Slide 61 text

2VFVF3VOOFS SVO@TQFDT JUFSBUPS 4UBUT 3VOOFS34QFD 5FTU'SBNFXPSL 34QFD 4UBUT4VJUF OFX 3VOOFS 34QFD 3VOOFS 5FTU'SBNFXPSL *UFSBUPS SVO@TQFDT FYBNQMF@HSPVQT UFTURVFVFSVOUJNF UFTUJOHGSBNFXPSL BEBQUFSDMBTT FYFDVUF OFX BMM@TVJUF@pMFT TVJUFT@GSPN@pMF OFX &OVNFSBCMF FBDI OFX 8PSLFS OFX !RVFVF TUBUVT 🏃💨 TestQueue::Runner:: RSpec.new.execute FYFSTQFDRVFVF SVO@XPSLFS BMM@TVJUF@pMFT SVO@XPSLFS BMM@TVJUFT SVO@FBDI TVJUFT@GSPN@pMF %FTJHO34QFD

Slide 62

Slide 62 text

w UFTURVFVFTVQQPSUT34QFD 34QFD .JOJUFTU .JOJUFTU UFTUVOJU $VDVNCFS w UFTURVFVFQSPWJEFTUIFDPPSEJOBUJPOPG QSPDFTTFTJODPNNPOQBSBMMFMQSPDFTTJOH w 5FTUTVJUFBOEUFTUSVOOFS"1*TBSFUFTUJOH GSBNFXPSLTQFDJpD 1MVHHBCMF%FTJHO

Slide 63

Slide 63 text

Slide 64

Slide 64 text

2VFVF3VOOFS SVO@TQFDT JUFSBUPS 4UBUT 3VOOFS34QFD 5FTU'SBNFXPSL 34QFD 4UBUT4VJUF OFX 3VOOFS 34QFD 3VOOFS 5FTU'SBNFXPSL *UFSBUPS SVO@TQFDT FYBNQMF@HSPVQT UFTURVFVFSVOUJNF UFTUJOHGSBNFXPSL BEBQUFSDMBTT FYFDVUF OFX BMM@TVJUF@pMFT TVJUFT@GSPN@pMF OFX &OVNFSBCMF FBDI OFX 8PSLFS OFX !RVFVF TUBUVT 3VOOFS 🏃💨 TestQueue::Runner:: RSpec.new.execute FYFSTQFDRVFVF SVO@XPSLFS BMM@TVJUF@pMFT SVO@XPSLFS BMM@TVJUFT SVO@FBDI TVJUFT@GSPN@pMF &YUFOTJPOQPJOU

Slide 65

Slide 65 text

2VFVF3VOOFS SVO@TQFDT JUFSBUPS 4UBUT 3VOOFS34QFD 5FTU'SBNFXPSL 34QFD 4UBUT4VJUF OFX 3VOOFS 34QFD 3VOOFS 5FTU'SBNFXPSL *UFSBUPS SVO@TQFDT FYBNQMF@HSPVQT UFTURVFVFSVOUJNF UFTUJOHGSBNFXPSL BEBQUFSDMBTT FYFDVUF OFX BMM@TVJUF@pMFT TVJUFT@GSPN@pMF OFX &OVNFSBCMF FBDI OFX 8PSLFS OFX !RVFVF TUBUVT 5FTU'SBNFXPSL 🏃💨 TestQueue::Runner:: RSpec.new.execute FYFSTQFDRVFVF SVO@XPSLFS BMM@TVJUF@pMFT SVO@XPSLFS BMM@TVJUFT SVO@FBDI TVJUFT@GSPN@pMF &YUFOTJPOQPJOU

Slide 66

Slide 66 text

UFTURVFVFMJCUFTU@RVFVFSVOOFSSTQFDSC UFTURVFVFMJCUFTU@RVFVFSVOOFSSTQFDSC 3VOOFS34QFDSVO@XPSLFS module QueueRunner < Runner def run_specs(iterator) @configuration.reporter.report(0) do |reporter| @configuration.with_suite_hooks do iterator.map { |g| start = Time.now if g.is_a? ::RSpec::Core::Example print " #{g.full_description}: " example = g g = example.example_group ::RSpec.world.filtered_examples.clear ::RSpec.world.filtered_examples[g] = [example] else print " #{g.description}: " end ret = g.run(reporter) # snip end alias_method :run_each, :run_specs module TestQueue class Runner class RSpec < Runner def run_worker(iterator) rspec = ::RSpec::Core::QueueRunner.new rspec.run_each(iterator).to_i ennnnd 3VOTVJUF Iterator#each☝ /BJWFJNQMFNFOUBUJPO UIBUEFQFOETPO34QFDT JOUFSOBM"1*

Slide 67

Slide 67 text

Slide 68

Slide 68 text

4UBUT 3VOOFS.JOJUFTU 5FTU'SBNFXPSL .JOJUFTU 4UBUT4VJUF .JOJUFTU .JOJUFTU 3VOOFS 5FTU'SBNFXPSL *UFSBUPS SVOOBCMFT JUFSBUPS UFTURVFVFSVOUJNF UFTUJOHGSBNFXPSL BEBQUFSDMBTT FYFDVUF OFX BMM@TVJUF@pMFT TVJUFT@GSPN@pMF BMM@TVJUF@pMFT &OVNFSBCMF SVO@XPSLFS BMM@TVJUFT OFX 8PSLFS OFX !RVFVF TUBUVT FYFNJOJUFTURVFVF %FTJHO.JOJUFTU SVO 🏃💨 OFX TestQueue::Runner:: Minitest.new.execute SVO@XPSLFS FBDI OFX TVJUFT@GSPN@pMF

Slide 69

Slide 69 text

NJOJUFTUMJCNJOJUFTUSC module Minitest def self.run args = [] # snip begin __run reporter, options rescue Interrupt warn 'Interrupted. Exiting...' UFTURVFVFMJCUFTU@RVFVFSVOOFSNJOJUFTUSC module Minitest def self.__run reporter, options suites = Runnable.runnables suites.map { |suite| suite.run reporter, options } ennd module TestQueue class Runner class Minitest < Runner def run_worker(iterator) ::Minitest::Test.runnables = iterator ::Minitest.run ? 0 : 1 ennnnd 3VOOFS.JOJUFTUSVO@XPSLFS $BMM__run 3VOTVJUF Iterator#each☝ /BJWFJNQMFNFOUBUJPO UIBUEFQFOETPOPQFO DMBTTBOEJOUFSOBM"1* 0QFODMBTT

Slide 70

Slide 70 text

4VQQPSUUFTUJOHGSBNFXPSLT IUUQTHJUIVCDPNUNNUFTURVFVFUSFFNBTUFSMJCUFTU@RVFVFSVOOFS

Slide 71

Slide 71 text

1BSBMMFM5FTUJOH "HBJO 1BSBMMFMXPSLFS 4VJUF 4VJUF 4VJUF 4VJUF 4VJUF 4VJUF 4VJUF 1BSBMMFMXPSLFS 1BSBMMFMXPSLFS QSPDFTTJOHUJNF

Slide 72

Slide 72 text

DPSF$16T)ZQFS5ISFBEJOH TFD TFSJBM QBSBMMFM w8IFOUFTURVFVF SVOTBMPUPGUFTU TVJUFT JUXJMMCF FGGFDUJWF wFHTFDTFD JO3VCP$PQSFQP

Slide 73

Slide 73 text

% bundle exec rake spec Starting test-queue master (/tmp/test_queue_40616_5060.sock) ==> Summary (16 workers in 51.9441s) [ 1] 9 examples, 0 failures 1 suites in 34.0151s (pid 40649 exit 0 ) [ 2] 109 examples, 0 failures 1 suites in 34.0151s (pid 40650 exit 0 ) [ 3] 712 examples, 0 failures 46 suites in 34.0149s (pid 40651 exit 0 ) [ 4] 2449 examples, 0 failures 61 suites in 34.0151s (pid 40652 exit 0 ) [ 5] 129 examples, 0 failures 13 suites in 34.0151s (pid 40653 exit 0 ) [ 6] 1628 examples, 0 failures 57 suites in 34.0150s (pid 40654 exit 0 ) [ 7] 107 examples, 0 failures 1 suites in 34.0151s (pid 40655 exit 0 ) [ 8] 1647 examples, 0 failures 61 suites in 34.0147s (pid 40656 exit 0 ) [ 9] 2270 examples, 0 failures 59 suites in 34.0145s (pid 40657 exit 0 ) [10] 1660 examples, 1 pending, 0 failures 57 suites in 34.0145s (pid 40658 exit 0 ) [11] 132 examples, 0 failures 1 suites in 51.9309s (pid 40659 exit 0 ) [12] 1600 examples, 0 failures 53 suites in 51.9306s (pid 40660 exit 0 ) [13] 1924 examples, 0 failures 60 suites in 51.9295s (pid 40661 exit 0 ) [14] 1993 examples, 0 failures 59 suites in 51.9281s (pid 40662 exit 0 ) [15] 1967 examples, 0 failures 60 suites in 51.9271s (pid 40663 exit 0 ) [16] 2303 examples, 0 failures 57 suites in 51.9237s (pid 40664 exit 0 ) 4MPXFTUBOEGBTUFTU 4MPXFTU 'BTUFTU

Slide 74

Slide 74 text

)PXTIPVMEUIF SVOOJOHPGUFTUT B⒎FDUPOFBOPUIFS /PUBUBMM 5&45%3*7&/%&7&-01.&/5#:&9".1-& l *TPMBUFE5FTUJTJNQPSUBOU

Slide 75

Slide 75 text

w *TPMBUFUFTUTGSPNFBDIPUIFS w %BUBTUPSBHFBDDFTTFECZQBSBMMFM XPSLFSTTVDIBT3%#.4TIPVMECFTFU VQJOBQBSBMMFMNBOOFS w FH8PSLFS#SFBETUIFVQEBUFEEBUBPG XPSLFS"BOEUIFUFTUNBZGBJM 4FUVQGPSQBSBMMFMUFTUJOH

Slide 76

Slide 76 text

w 1SPEVDUDPEFBOEUFTUDPEFGPSNBUPUFTUJOHQBJS w 5IFVOJUPGQBSBMMFMUBTLTJOQBSBMMFMUFTUTJTVTVBMMZ UIFpMFMFWFM w 1BSBMMFMJ[JOHUFTUTVJUFTJOTNBMMFSVOJUTDBOSFEVDF UIFMPDLJOHQFSJPEGPSXPSLFST w *OPUIFSXPSET GPMMPXJOH431MFBETUPGBTUFS QBSBMMFMUFTUT 1BSBMMFMUFTUBOE431 4JOHMF3FTQPOTJCJMJUZ1SJODJQMF

Slide 77

Slide 77 text

Parallel test runners Using RSpec Using Rails 6+ or AS Using Vanilla Ruby No Yes Yes No Yes No Start AS:: TestCase parallel_tests or test-queue

Slide 78

Slide 78 text

8IJDIPOFJTCFUUFSUPVTF QBSBMMFM@UFTUTPSUFTURVFVF

Slide 79

Slide 79 text

w /VNCFSPGEPXOMPBETPOSVCZHFNTPSH w /VNCFSPGTUBSTPO(JU)VC w 'SFRVFODZPGNBJOUFOBODF w QBSBMMFM@UFTUTJTNBJOUBJOFECVUUFTURVFVF XBTVONBJOUBJOFE w *OUFSFTUJOHEFTJHOCVUVOEFSVUJMJ[FE $SJUFSJBGPSHFNTFMFDUJPO

Slide 80

Slide 80 text

3JTFVQ

Slide 81

Slide 81 text

3FTVSSFDUJPOPGUFTURVFVF

Slide 82

Slide 82 text

w *OFFEUPSFMFBTFBOFXWFSTJPO w !UNNBTLFENFUPCFBNBJOUBJOFSBOE *BDDFQUFE w .BJOUFOBODFJTSFTVNFEBGUFSPS ZFBST 'JSTUUIJOHT*EJEBTBNBJOUBJOFS

Slide 83

Slide 83 text

*GMFGUVOBUUFOEFE TPGUXBSFXJMMCSFBL CZ!ITCU XBTSFMFBTFE CVUTUJMM$*JTCSPLFO

Slide 84

Slide 84 text

/P$* OPNFSHF $*XBTCSPLFO

Slide 85

Slide 85 text

w *GPSHPUIPXUPVTF5SBWJT$* w *NOPUVQUPEBUFPOUIFSFDFOU EFWFMPQNFOUTPG5SBWJT$* w 8JUI(JU)VC"DUJPOT NBJOUFOBODFDBO CFEPOFXJUIPVUNF 'SPN5SBWJT$*UP(JU)VC"DUJPOT

Slide 86

Slide 86 text

w #BUT#BTI"VUPNBUFE5FTUJOH4ZTUFN w !TTUFQIFOTPOQSPEVDU w 5IFSFQPTJUPSZIBTCFFOBSDIJWFECZUIF PXOFSPO"QS 😢 w *UTIBSEUPSFQMBDFBUFTUJOHGSBNFXPSL w *UTOFHBUJWF CVU*NMFBWJOHJUBTJUJT &&UFTUJOHPGUFTURVFVF

Slide 87

Slide 87 text

w 5FTUOPUQBTTJOHXJUIMBUFTU WFSTJPOPG.JOJUFTU w 5IFUFTUDPEFBSPVOEUIF sleepNFUIPETFFNT TVTQJDJPVT w 'JYGBJMJOH(FNpMFGPS.JOJUFTU CZWFSTJPOQJOOJOH $VSSFOUTUBUVTPG&&UFTUJOH

Slide 88

Slide 88 text

$*IBTSFDPWFSFE

Slide 89

Slide 89 text

%FWFMPQNFOU FOWJSPONFOUJTSFBEZ *GUIF$*QBTTFT NBZCF*DBONFSHFJU

Slide 90

Slide 90 text

#FZPOE UIF3FTVSSFDUJPO

Slide 91

Slide 91 text

4VQQPSUTUBUVTPGUFTUJOHGSBNFXPSLT 5FTUJOH'SBNFXPSL $* /PUF 34QFD ✅ 🤔5PPPME34QFDEPFTOUXPSLXJUI3VCZ 34QFD ✅ ✅6TFEJO3VCP$PQ .JOJUFTU ✅ 🤔5PPPME .JOJUFTU ✅ CVU ✅6TFEJO3VCP$PQ.JOJUFTU $VDVNCFS ✅ CVU ❌0VUEBUFEUFTUDPEFFSSPSTJOUIFMBUFTUWFSTJPO UFTUVOJU ✅ 🤔/PUXPSLJOHXJUI3BJMT UNNUFTURVFVF 34QFD EFW ✅ ✅6TFEJO3VCP$PQ 5VSOJQ ✅ ✅/PUSBDLSFDPSEBTJUXBTKVTUNFSHFE

Slide 92

Slide 92 text

w "QVMMSFRVFTUIBTCFFOPQFOFECZB34QFD DPSFUFBNNFNCFS w 34QFDJTBEFWFMPQNFOUTUBUVT #VU *EFDJEFEUPQSPWJEFBGFBUVSFPGUFTURVFVF w 050) JUTIBSEGPSUFTURVFVFNBJOUBJOFST UPLOPX34QFDTJOUFSOBMJNQMFNFOUBUJPO 4VQQPSU34QFD EFW

Slide 93

Slide 93 text

w 8PVMEOUJUCFDPOWFOJFOUJG34QFDIBEB QBSBMMFMUFTUSVOOFSGFBUVSFMJLF.JOJUFTU w 5IFTBNFDBOCFTBJEGPSUFTUVOJUBOE PUIFST *OUSPEVDFQBSBMMFMUFTUSVOOFSUP34QFD 0QQPSUVOJUZGPSDPOUSJCVUJPO

Slide 94

Slide 94 text

0OFNPSFUIJOH

Slide 95

Slide 95 text

w "MSFBEZJODMVEFTPuppetLintBTB QBSBMMFMSVOOFS w 8IJMFUIFQBSBMMFMHFNJTVTFEGPSQBSBMMFM FYFDVUJPOPG3VCP$PQ VTJOHB NFDIBOJTNMJLFUFTURVFVFNBZQSPWJEF FWFOGBTUFSSFTVMUT 2VFVFTZTUFNOPUKVTUGPSUFTUJOH

Slide 96

Slide 96 text

w 1SPHSBNNJOHGPSJOUFSQSPDFTT DPNNVOJDBUJPOJTBCBTJDGPVOEBUJPOGPS MFWFSBHJOHNVMUJDPSF 'VOEBNFOUBMTPGQBSBMMFMQSPHSBNNJOH

Slide 97

Slide 97 text

$PODMVTJPO

Slide 98

Slide 98 text

w UFTURVFVFEFQFOETPOJOUFSOBM"1*PGFBDI UFTUJOHGSBNFXPSL w *XBOUJUUPCFUFNQPSBSZUSBOTJFOU w .BZOPUSFBDIUFTURVFVF w )BWJOHBQMVHHBCMFQBSBMMFMUFTUNFDIBOJTNMJLF .JOJUFTUDPVMECFDPOWFOJFOU FH34QFDUPP )PQFGPSQBSBMMFMUFTUSVOOFS

Slide 99

Slide 99 text

w 1BSBMMFMUFTUJOHSFRVJSFTNPOFZ💰 w %FWFMPQFSTBSFUIFNPTUWBMVBCMFSFTPVSDF JOUIFQSPKFDU w *UTFBTJFSUPHFUNPSFNFNPSZBOE$16T XJUINPOFZUIBOUPIJSFTFOJPSFOHJOFFST w %PZPVSCFTUUPTFDVSFUIFCVEHFU 5IFSFJTBHPMECVMMFU

Slide 100

Slide 100 text

1BSBMMFM5FTUJOH .BLFT :PV)BQQZ (JU)VC!LPJD