Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Resurrection of 
the Fast Parallel Test Runner

The Resurrection of 
the Fast Parallel Test Runner

Koichi ITO

May 12, 2023
Tweet

More Decks by Koichi ITO

Other Decks in Programming

Transcript

  1. Koichi ITO / ESM, Inc.
    RubyKaigi 2023
    The Resurrection of
    the Fast Parallel Test Runner
    May, 12th, 2023
    Matsumoto Performing Arts Centre
    The Force Awakens

    View full-size slide

  2. !LPJD
    w 044QSPHSBNNFS
    w 3VCP$PQDPSFUFBN
    w &OHJOFFSJOH.BOBHFSBOE
    %JTUJOHVJTIFE&OHJOFFSPG&4. *OD
    w 3VCZ,BJHJTQFBLFSBU
    -5 5BLFPVU
    BOE

    View full-size slide

  3. .BJOUBJO044&WFSZ%BZ
    IUUQTHJUIVCDPNLPJD

    View full-size slide

  4. $P⒎FFIPVTF4QPOTPS
    w"CF ᘖᘣඒֶΞϕ

    w0LJOBEP ͓͖ͳಊ

    w:BNBHB ٤஡ࢁխ

    w"OENPSF

    View full-size slide

  5. 4
    Q
    FBLFS
    4
    Q
    FBLFS
    JOQ
    FSTPO
    JOQ
    FSTPO
    JOQ
    FSTPO
    JOQ
    FSTPO
    JOQ
    FSTPO
    JOQ
    FSTPO
    POMJOF
    POMJOF
    POMJOF
    POMJOF
    POMJOF
    POMJOF
    POMJOF
    4QFBLFSTBOE"UUFOEFFT
    PSHBOJ[FS
    PSHBOJ[FS
    "EWJTPS "EWJTPS

    View full-size slide

  6. Support OSS community

    View full-size slide

  7. +PJOVT
    🗾w-FUTHSPXUPHFUIFS
    w:PVDBOXPSL
    SFNPUFMZGSPN
    BOZXIFSFJO+BQBO
    w4FFBHJMFFTNDPKQ

    View full-size slide

  8. $POUFOUT
    8IBUTQBSBMMFMUFTUJOH
    1BSBMMFMUFTUSVOOFST
    3FTVSSFDUJPOPGUFTURVFVF

    View full-size slide

  9. 8IBUTQBSBMMFMUFTUJOH

    View full-size slide

  10. 5PNF MFHBDZDPEF
    JTTJNQMZDPEF
    XJUIPVUUFTUT
    803,*/(&''&$5*7&-:8*5)-&("$:$0%&

    l
    "CPPLJOUIF6ODMF#PCTFSJFTTBZT

    View full-size slide

  11. w 4PGUXBSFDPEFLFFQTHSPXJOHMBSHFS
    w #VTJOFTTHSPXUIBOETPGUXBSFHSPXUI
    HPIBOEJOIBOE
    w $POUJOVPVTMZQFSGPSN5%% #%% 5"%
    BOESFHSFTTJPOUFTUT
    5FTUDPEFJTJNQPSUBOU

    View full-size slide

  12. )PXMPOHEPZPV
    XBJUGPSZPVSCVJME

    View full-size slide


  13. "JNGPS
    NJOVUFTPSMFTT

    View full-size slide

  14. w FYBNQMFT 6TJOH34QFD

    w "SFBMXPSMEBQQMJDBUJPOUBLFPWFSBO
    IPVS
    "SFBMXPSMEBQQMJDBUJPO
    $ bin/rails stats
    (snip)
    Code LOC: 12796 Test LOC: 23518 Code to Test Ratio: 1:1.8

    View full-size slide

  15. "OFXQSPCMFN
    4MPXUFTU

    View full-size slide

  16. w $BOOPUSVOUFTUTPOQVMMSFRVFTUTJO
    BEBZ
    w 5IFCVJMETBSFOUpOJTIFEFWFOCZ
    UIFOFYUNPSOJOH
    *UUBLFTBOIPVSUPDPNQMFUFBCVJME

    View full-size slide

  17. w TFDPOETJOBEBZ
    w 'PSFYBNQMF TMPXUFTUTUBLJOH
    TFDPOETFBDIXJMMSFRVJSFNJOVUFT
    w )PXNBOZDPNNJUTEPFTZPVSUFBN
    NBLFFWFSZEBZ
    5JNFJTJNQPSUBOU

    View full-size slide

  18. w 1SPpMJOHCBTFEUVOJOH%POUHVFTT NFBTVSF
    • FH
    w .BZCFTQFEVQXJUIBDMFBOJOHTUSBUFHZ
    w FH%BUBCBTF$MFBOFSP⒎FSTEBUBCBTFDMFBOJOH
    TUSBUFHJFTTVDIBTtransaction deletion BOE
    truncation
    JUTBHPPEJEFBUPDIPPTFUIFPQUJNBMPOF
    "EESFTTJOH4MPX5FTU#PUUMFOFDLT
    $ rspec -p 30 spec/acceptance

    View full-size slide

  19. .ZSFDPNNFOEBUJPO
    w 'JSTU QSFQBSFNPOFZ💰JGZPVBSF
    CVJMEJOHCVTJOFTTTPGUXBSF
    w /FYU QSFQBSFFOPVHI$16TBOENFNPSZ
    w 'JOBMMZ 1BSBMMFMJ[FZPVSUFTUT
    w 5IBUTJU"'"*, JUTUIFNPTUF⒎FDUJWF

    View full-size slide

  20. 1BSBMMFM5FTUJOH

    View full-size slide

  21. 4FSJBM5FTUJOH
    4JOHMFXPSLFS
    4VJUF

    4VJUF

    4VJUF

    4VJUF

    4VJUF

    4VJUF

    4VJUF

    w 8PSLFS5FTUFYFDVUJPOQSPDFTT
    w 4VJUF5FTUFYFDVUJPOVOJU UPQMFWFM
    TestDMBTT describe %4- pMF FUD

    QSPDFTTJOHUJNF
    $ bundle exec ruby -Itest path # or rspec path

    View full-size slide

  22. 1BSBMMFM5FTUJOH
    1BSBMMFMXPSLFS
    4VJUF

    4VJUF

    4VJUF

    4VJUF

    4VJUF

    4VJUF

    4VJUF

    1BSBMMFMXPSLFS
    1BSBMMFMXPSLFS
    QSPDFTTJOHUJNF

    View full-size slide

  23. 4FSJBMWT1BSBMMFM
    1BSBMMFMXPSLFS
    4VJUF

    4VJUF

    4VJUF

    4VJUF

    4VJUF

    4VJUF

    4VJUF

    1BSBMMFMXPSLFS
    1BSBMMFMXPSLFS
    4JOHMFXPSLFS
    4VJUF

    4VJUF

    4VJUF

    4VJUF

    4VJUF

    4VJUF

    4VJUF

    /PXBJUUJNF
    QSPDFTTJOHUJNF

    View full-size slide

  24. 1BSBMMFMJ[BUJPOJTHSFBU
    5JNFJTNPSFWBMVBCMF
    UIBOBOZUIJOH

    View full-size slide


  25. 1BSBMMFMUFTUSVOOFST

    View full-size slide

  26. w .JOJUFTU
    w 3BJMT
    w QBSBMMFM@UFTUT
    w UFTURVFVF
    5PVSPG1BSBMMFM5FTUJOH5PPMT

    View full-size slide

  27. NJOJUFTUNJOJUFTU
    NJOJUFTUQSPWJEFTBDPNQMFUFTVJUFPGUFTUJOHGBDJMJUJFT
    TVQQPSUJOH5%% #%% NPDLJOH BOECFODINBSLJOH

    View full-size slide

  28. w #BDLFOEPGActiveSupport::TestCaseJO3BJMT
    w 3BOEPNJ[FETFSJBMUFTUJTUIFEFGBVMU
    w parallelize_me! QSPWJEFTQBSBMMFMUFTUJOHGPSUIF
    RVFVFNPEFM*UVTFTNVMUJUISFBEJOHCZEFGBVMU
    w :PVDBODIBOHFUIFQBSBMMFMFYFDVUJPONPEVMFCZ
    Minitest.parallel_executor
    .JOJUFTU

    View full-size slide

  29. &YUFOTJPOQPJOUGPSQBSBMMFMUFTUJOH
    .JOJUFTU
    QBSBMMFM@FYFDVUPS
    JOUFSGBDF
    w 5ISFBECZEFGBVMUGPSUIFQBSBMMFMUFTU
    w *.0 UFTUTUIBUBSFDPNQMFUFEJONFNPSZBSFOPUGBTU
    &⒎FDUJWFGPSUFTUTUIBUVTF*0
    *OUFSGBDFGPSQBSBMMFMUFTUJOH
    .JOJUFTU
    1BSBMMFM
    &YFDVUPS
    !RVFVF
    "DMBTTUIBUJNQMFNFOUT
    NFUIPETTVDIBTstart
    << size BOEshutdown
    BVUPSVO

    View full-size slide

  30. EFGTUBSU
    !QPPMTJ[FUJNFTNBQ\
    5ISFBEOFX !RVFVF
    EPcRVFVFc
    5ISFBEDVSSFOUBCPSU@PO@FYDFQUJPOUSVF
    XIJMF KPCRVFVFQPQ

    LMBTT NFUIPE SFQPSUFSKPC
    SFQPSUFSTZODISPOJ[F\SFQPSUFSQSFSFDPSELMBTT NFUIPE^
    SFTVMU.JOJUFTUSVO@POF@NFUIPELMBTT NFUIPE
    SFQPSUFSTZODISPOJ[F\SFQPSUFSSFDPSESFTVMU^
    FOE
    FOE
    ^
    FOE
    EFGXPSL!RVFVFXPSLFOE
    .JOJUFTU1BSBMMFM&YFDVUPS
    $SFBUFsizeOVNPGOFXUISFBEXJUI@queue
    -PPQXJUIqueue.pop
    $BMMBUFTUNFUIPE
    .JOJUFTU1BSBMMFM&YFDVUPS XPSL

    IUUQTHJUIVCDPNNJOJUFTUNJOJUFTUCMPCWMJCNJOJUFTUQBSBMMFMSC--
    /05&@queueDBOCF
    TIBSFEXJUIBOJOTUBODF
    WBSJBCMFCFDBVTFJUJTB
    UISFBENPEFM

    View full-size slide

  31. .JOJUFTU5FTU
    NPEVMF5FTUOPEPD
    EFG@TZODISPOJ[F.JOJUFTU5FTUJP@MPDLTZODISPOJ[F\ZJFME^
    FOE
    NPEVMF$MBTT.FUIPETOPEPD
    EFGSVO@POF@NFUIPELMBTT NFUIPE@OBNF SFQPSUFS
    .JOJUFTUQBSBMMFM@FYFDVUPS
    FOE
    EFGUFTU@PSEFS
    QBSBMMFM
    FOOOE
    .JOJUFTU
    3VOOBCMF
    .JOJUFTU
    5FTU
    IUUQTHJUIVCDPNNJOJUFTUNJOJUFTUCMPCWMJCNJOJUFTUQBSBMMFMSC--
    FORVFVFUPQBSBMMFM@FYFDVUPS
    .JOJUFTU
    1BSBMMFM5FTU
    $BTT.FUIPET
    NJYJO
    DMBTT.JOJUFTU5FTU.JOJUFTU3VOOBCMF
    FYUFOE.JOJUFTU1BSBMMFM5FTU$MBTT.FUIPET
    FOE

    View full-size slide

  32. <"QQFOEJY>IFMMSC
    IUUQTHJUIVCDPNNJOJUFTUNJOJUFTUCMPCWMJCNJOJUFTUIFMMSC
    w %FpOJOHUIFFOWJSPONFOU
    WBSJBCMFMT_HELLBDUJWBUFT
    NJOJUFTUIFMMSC
    w "DUJWBUFQBSBMMFMBOEQSPWFJU

    View full-size slide

  33. SBJMTSBJMT
    3BJMTJTBXFCBQQMJDBUJPOGSBNFXPSLUIBUJODMVEFTFWFSZUIJOH
    OFFEFEUPDSFBUFEBUBCBTFCBDLFEXFCBQQMJDBUJPOT
    BDDPSEJOHUPUIF.PEFM7JFX$POUSPMMFS .7$
    QBUUFSO

    View full-size slide

  34. w 1BSBMMFMUFTUJOHCZEFGBVMU 4JODF3BJMT

    w %FGBVMUUISFTIPMEJT "WPJEJOHUIFPWFSIFBEPGGPSLJOH

    3BJMTAS::TestCase
    $ bin/rails test
    Running 60 tests in parallel using 8 processes
    Run options: --seed 21371
    # Running:
    .....................................................
    .......
    Finished in 0.205488s, 291.9879 runs/s, 291.9879
    assertions/s.
    60 runs, 60 assertions, 0 failures, 0 errors, 0 skips
    MFTTUIBOUFTUT UFTUT
    $ bin/rails test
    Running 10 tests in a single process (parallelization
    threshold is 50)
    Run options: --seed 1398
    # Running:
    ..........
    Finished in 0.016447s, 608.0136 runs/s, 608.0136
    assertions/s.
    10 runs, 10 assertions, 0 failures, 0 errors, 0 skips

    View full-size slide

  35. ENV["RAILS_ENV"] ||= "test"
    require_relative "../config/environment"
    require "rails/test_help"
    class ActiveSupport::TestCase
    # Run tests in parallel with specified workers
    parallelize(workers: :number_of_processors)
    # Setup all fixtures in test/fixtures/*.yml for all tests in alphabetical order.
    fixtures :all
    # Add more helper methods to be used by all tests here...
    end
    UFTUUFTU@IFMQFSSC
    QBSBMMFMJ[BUJPOCZEFGBVMU
    PQFODMBTT

    View full-size slide

  36. module ActiveSupport
    class TestCase < ::Minitest::Test
    class << self
    def parallelize(workers: :number_of_processors, with: :processes,
    threshold: ActiveSupport.test_parallelization_threshold)
    workers = Concurrent.physical_processor_count if workers
    == :number_of_processors
    workers = ENV["PARALLEL_WORKERS"].to_i if ENV["PARALLEL_WORKERS"]
    return if workers <= 1
    Minitest.parallel_executor =
    ActiveSupport::Testing::ParallelizeExecutor.new(size: workers, with:
    with, threshold: threshold)
    ennnnnd
    *OIFSJUMinitest::Test
    $IBOHFUPQBSBMMFMFYFDVUPSPG3BJMT
    "DUJWF4VQQPSU5FTU$BTF

    View full-size slide

  37. &YUFOTJPOQPJOUGPSQBSBMMFMUFTUJOH
    .JOJUFTU
    QBSBMMFM@FYFDVUPS
    JOUFSGBDF
    w 5ISFBECZEFGBVMUGPSUIFQBSBMMFMUFTU
    w *.0 UFTUTUIBUBSFDPNQMFUFEJONFNPSZBSFOPUGBTU
    &⒎FDUJWFGPSUFTUTUIBUVTF*0
    *OUFSGBDFGPSQBSBMMFMUFTUJOH
    .JOJUFTU
    1BSBMMFM
    &YFDVUPS
    !RVFVF
    "DMBTTUIBUJNQMFNFOUT
    NFUIPETTVDIBTstart
    << size BOEshutdown
    BVUPSVO

    View full-size slide

  38. "45FTUJOH1BSBMMFM&YFDVUPS
    .JOJUFTU
    QBSBMMFM@FYFDVUPS
    JOUFSGBDF
    w 3FQMBDFXJUIUIF.JOJUFTUFYUFOTJPOQPJOU
    w 4XJUDIGSPNUISFBEUPQSPDFTTQBSBMMFMJ[BUJPO
    "OJOUFSGBDFGPSUIFQBSBMMFMUFTU
    .JOJUFTU
    1BSBMMFM
    &YFDVUPS
    !RVFVF
    BVUPSVO
    "DUJWF4VQQPSU
    5FTUJOH
    1BSBMMFMJ[F&YFDVUPS
    !QBSBMMFM@FYFDVUPS
    "DUJWF4VQQPSU
    5FTUJOH
    1BSBMMFMJ[BUJPO

    View full-size slide

  39. "45FTUJOH1BSBMMFMJ[BUJPO
    %3C'SPOU
    w 4FSWFS8PSLFSNPEFM NVMUJQSPDFTTJOH

    w *OUFSQSPDFTTDPNNVOJDBUJPOVTJOHE3VCZ
    1BSBMMFMJ[BUJPO
    4FSWFS
    RVFVF
    1BSBMMFMJ[BUJPO
    8PSLFS
    FORVFVF
    GPSLFEQSPDFTT
    ESCVOJY

    O
    NBJOQSPDFTT
    1BSBMMFMJ[BUJPO
    8PSLFS
    EFRVFVF XIJMFKPC!RVFVFQPQ
    QFSGPSN@KPC KPC

    FOE
    $BMMMinitest.run_one_method
    1BSBMMFMJ[BUJPO
    8PSLFS
    1BSBMMFMJ[BUJPO

    View full-size slide

  40. w AS::TestCase.parallelizeCZEFGBVMU
    w 3BJMTQSPWJEFTUIFCFOFpUTPGNVMUJDPSF
    QSPDFTTJOHUISPVHIUIFVTFPGQSPDFTTFT
    w 3VOTQBSBMMFMJTNCZEFGBVMUXJUI
    Concurrent.physical_processor_count
    w 3VOUJNF$16TBOENFNPSZBSFJNQPSUBOU
    3BJMTJTQSBHNBUJD

    View full-size slide

  41. 3BJMTEPFTOU
    TVQQPSU34QFDCZ
    EFGBVMU🤔

    View full-size slide

  42. w HSPTTFSQBSBMMFM@UFTUT
    w UNNUFTURVFVF
    1BSBMMFMUFTUSVOOFS

    View full-size slide

  43. HSPTTFSQBSBMMFM@UFTUT
    4QFFEVQ5FTU6OJU34QFD$VDVNCFS
    4QJOBDICZSVOOJOHQBSBMMFMPONVMUJQMF$16DPSFT

    View full-size slide

  44. w %FQFOETPOUIFQBSBMMFMHFN
    • Parallel.each { ... }
    • Parallel.map { ... }
    w 3VOQSFTQMJUUJOHUFTUTJOQBSBMMFMGPSBMMUFTUpMFT
    w *UDBOVTFQBSBMMFM@SVOUJNF@STQFDMPHBGUFSSVOOJOH
    w 7FSZTJNQMF😀
    QBSBMMFM@UFTUT

    View full-size slide

  45. w 5IFSFJTBUJNFEJ⒎FSFODFCFUXFFOUIFXPSLFS
    UIBUpOJTIFTpSTUBOEUIFXPSLFSUIBUpOJTIFT
    MBTU
    w 'JOJTIFEXPSLFSTIBWFXBTUFEXBJUUJNF
    w /PUFUIBUUIFFYFDVUJPOSFTVMUNBUSJYpMFJT
    VTFEUPNJOJNJ[FUIFUJNFEJ⒎FSFODFCFUXFFO
    UFTUTBTNVDIBTQPTTJCMF
    DPOTPGQSFTQMJUUJOH

    View full-size slide

  46. UNNUFTURVFVF
    :FUBOPUIFSQBSBMMFMUFTUSVOOFS CVJMUVTJOHB
    DFOUSBMJ[FERVFVFUPFOTVSFPQUJNBMEJTUSJCVUJPOPG
    UFTUTCFUXFFOXPSLFST

    View full-size slide

  47. w 5IFBVUIPSJT"NBO(VQUB,BSNBOJ !UNN

    w 3FBEZXPSLFSTEFRVFVFBOESVOUFTUTGSPNUIF
    RVFVFXBJUJOHUPCFSVO
    w 'FXFSXBJUJOHXPSLFSTBOEMFTTXBTUF
    w 4BNFBTQBSBMMFM@UFTUT VTJOHUFTU@RVFVF@TUBUT
    FYFDVUJPOSFTVMUNBUSJYBOESVOOJOHTMPXUFTUT
    pSTU
    UFTURVFVF

    View full-size slide

  48. 6/*94PDLFUPS5$14PDLFU
    5ISFFQSPDFTTUZQFT
    6/*94PDLFUPS5$14PDLFU
    6/*94FSWFSPS5$14FSWFS
    UFTURVFVF
    NBTUFSQSPDFTT
    UFTURVFVFTVJUF
    EJTDPWFSZQSPDFTT

    UFTURVFVF
    XPSLFSQSPDFTT



    %JTUSJCVUJOHUFTUTVJUF
    GSPNRVFVFBOE
    TVNNBSJ[FUIFSFTVMUT
    %FRVFVFBOESVOUFTUTVJUFT
    1BSTFUFTUTVJUFTBOEFORVFVFJU
    1⃣ 2⃣
    3⃣

    View full-size slide

  49. class Runner
    def execute_internal
    start_master
    prepare(@concurrency)
    @prepared_time = Time.now
    start_relay if relay?
    discover_suites
    spawn_workers
    distribute_queue
    ensure
    stop_master
    kill_subprocesses
    ennd
    3VOOFSTUBSU@NBTUFS
    def start_master
    if !relay?
    if @socket =~ /\A(?:(.+):)?(\d+)\z/
    address = $1 || '0.0.0.0'
    port = $2.to_i
    @socket = "#{$1}:{#$2}"
    @server = TCPServer.new(address, port)
    else
    FileUtils.rm_f(@socket)
    @server = UNIXServer.new(@socket)
    end
    ennd
    1⃣
    .BTUFS1SPDFTT
    .BTUFS1SPDFTT

    View full-size slide

  50. class Runner
    def execute_internal
    start_master
    prepare(@concurrency)
    @prepared_time = Time.now
    start_relay if relay?
    discover_suites
    spawn_workers
    distribute_queue
    ensure
    stop_master
    kill_subprocesses
    ennd
    3VOOFSEJTDPWFS@TVJUFT
    def discover_suites
    return if relay?
    return if @allowlist.any? && @awaited_suites.empty?
    @discovering_suites_pid = fork do
    terminate = false
    Signal.trap("INT") { terminate = true }
    $0 = 'test-queue suite discovery process'
    @test_framework.all_suite_files.each do |path|
    @test_framework.suites_from_file(path).each do |
    suite_name, suite|
    Kernel.exit!(0) if terminate
    @server.connect_address.connect do |sock|
    sock.puts("TOKEN=#{@run_token}")
    sock.puts("NEW SUITE #{Marshal.dump([suite_name,
    path])}")
    ennd
    2⃣
    %JTDPWFSZ1SPDFTT
    .BTUFS1SPDFTT

    View full-size slide

  51. class Runner
    def execute_internal
    start_master
    prepare(@concurrency)
    @prepared_time = Time.now
    start_relay if relay?
    discover_suites
    spawn_workers
    distribute_queue
    ensure
    stop_master
    kill_subprocesses
    ennd
    3VOOFSTQBXO@XPSLFST
    def spawn_workers
    @concurrency.times do |i|
    num = i + 1
    pid = fork do
    @server.close if @server
    iterator = Iterator.new(@test_framework, relay? ?
    @relay : @socket, method(:around_filter),
    early_failure_limit: @early_failure_limit, run_token:
    @run_token)
    after_fork_internal(num, iterator)
    ret = run_worker(iterator) || 0
    cleanup_worker
    Kernel.exit! ret
    end
    @workers[pid] = Worker.new(pid, num)
    ennd
    3⃣
    3VOXPSLFS🏃
    8PSLFS1SPDFTT
    .BTUFS1SPDFTT

    View full-size slide

  52. 6/*94PDLFUPS5$14PDLFU
    6/*94PDLFUPS5$14PDLFU
    6/*94FSWFSPS5$14FSWFS
    *OUFSBDUJPOPGQSPDFTTFT
    UFTURVFVF
    NBTUFSQSPDFTT
    UFTURVFVFTVJUF
    EJTDPWFSZQSPDFTT
    UFTURVFVF
    XPSLFSQSPDFTT
    %FRVFVFBOESVOUFTUTVJUFT
    1BSTFUFTUTVJUFTBOE
    FORVFVFJU
    %JTUSJCVUJOHUFTUTVJUF
    GSPNRVFVFBOE
    TVNNBSJ[FUIFSFTVMUT
    GPSL
    /&846*5&
    GPSL
    .BSTIBMEVNQ
    .BSTIBMMPBE
    5FTU'SBNFXPSL
    3VOOFS
    101

    View full-size slide

  53. $PNNBOEMJTU
    $PNNBOE %JSFDUJPO %FTDSJQUJPO
    /&846*5& EJTDPWFSZˠNBTUFS &ORVFVFBUFTUTVJUF
    101 XPSLFSˠNBTUFS %FRVFVFBUFTUTVJUF
    8"*5 NBTUFSˠXPSLFS "XBJUBUFTUTVJUFXIFOEFRVFVF
    ,"#00. XPSLFSˠNBTUFS *NNFEJBUFMZBCPSU
    3&.05&."45&3 XPSLFSˠNBTUFS 3FHJTUFSBSFNPUFXPSLFSUPUIFSFNPUFNBTUFSJGSFMBZ
    0, NBTUFSˠXPSLFS 3FQMZSFHJTUFSTVDDFTTUPUIFSFNPUFXPSLFSJGSFMBZ
    803,&3 XPSLFSˠNBTUFS $PNQMFUFPSBCPSUBSFNPUFXPSLFSJGSFMBZ
    "MXBZTNBUDIBVOJRVFTOKENJTTVFEBOEVTFEBTBQSPUPDPMXJUIUIFNBTUFSQSPDFTT

    View full-size slide

  54. class Runner
    def execute_internal
    start_master
    prepare(@concurrency)
    @prepared_time = Time.now
    start_relay if relay?
    discover_suites
    spawn_workers
    distribute_queue
    ensure
    stop_master
    kill_subprocesses
    ennd
    3VOOFSEJTUSJCVUF@RVFVF
    def distribute_queue
    # snip
    until !awaiting_suites? && @queue.empty? && remote_workers == 0
    # snip
    case cmd
    when /\APOP (\S+) (\d+)/
    # snip
    when /\AREMOTE MASTER (\d+) ([\w\.-]+)(?: (.+))?/
    # snip
    when /\AWORKER (\d+)/
    # snip
    when /\ANEW SUITE (.+)/
    # snip
    when /\AKABOOM/
    break
    else
    warn("Ignoring unrecognized command: \"#{cmd}\"")
    end
    sock.close
    ennd .BTUFS1SPDFTT
    .BTUFS1SPDFTT

    View full-size slide

  55. def each
    # snip
    loop do
    if @early_failure_limit && @failures >= @early_failure_limit
    connect_to_master('KABOOM'); break
    else
    client = connect_to_master("POP #{Socket.gethostname} #{Process.pid}")
    end
    break if client.nil?
    _r, _w, e = IO.select([client], nil, [client], nil)
    break unless e.empty?
    if data = client.read(65536)
    client.close
    item = Marshal.load(data)
    # snip
    suite_name, path = item
    suite = load_suite(suite_name, path)
    # snip
    yield suite
    *UFSBUPSFBDI
    8PSLFS1SPDFTT
    class Iterator
    include Enumerable

    View full-size slide

  56. 2VFVF3VOOFS
    SVO@TQFDT JUFSBUPS

    4UBUT
    3VOOFS34QFD
    5FTU'SBNFXPSL
    34QFD
    4UBUT4VJUF
    OFX
    3VOOFS
    34QFD
    3VOOFS
    5FTU'SBNFXPSL
    *UFSBUPS
    SVO@TQFDT FYBNQMF@HSPVQT

    UFTURVFVFSVOUJNF
    UFTUJOHGSBNFXPSL
    BEBQUFSDMBTT
    FYFDVUF
    OFX
    BMM@TVJUF@pMFT
    TVJUFT@GSPN@pMF
    OFX
    &OVNFSBCMF
    FBDI
    OFX
    8PSLFS
    OFX
    !RVFVF
    TUBUVT
    🏃💨
    TestQueue::Runner::
    RSpec.new.execute
    FYFSTQFDRVFVF
    SVO@XPSLFS
    BMM@TVJUF@pMFT
    SVO@XPSLFS
    BMM@TVJUFT
    SVO@FBDI
    TVJUFT@GSPN@pMF
    %FTJHO34QFD

    View full-size slide

  57. w UFTURVFVFTVQQPSUT34QFD 34QFD
    .JOJUFTU .JOJUFTU UFTUVOJU $VDVNCFS
    w UFTURVFVFQSPWJEFTUIFDPPSEJOBUJPOPG
    QSPDFTTFTJODPNNPOQBSBMMFMQSPDFTTJOH
    w 5FTUTVJUFBOEUFTUSVOOFS"1*TBSFUFTUJOH
    GSBNFXPSLTQFDJpD
    1MVHHBCMF%FTJHO

    View full-size slide

  58. 2VFVF3VOOFS
    SVO@TQFDT JUFSBUPS

    4UBUT
    3VOOFS34QFD
    5FTU'SBNFXPSL
    34QFD
    4UBUT4VJUF
    OFX
    3VOOFS
    34QFD
    3VOOFS
    5FTU'SBNFXPSL
    *UFSBUPS
    SVO@TQFDT FYBNQMF@HSPVQT

    UFTURVFVFSVOUJNF
    UFTUJOHGSBNFXPSL
    BEBQUFSDMBTT
    FYFDVUF
    OFX
    BMM@TVJUF@pMFT
    TVJUFT@GSPN@pMF
    OFX
    &OVNFSBCMF
    FBDI
    OFX
    8PSLFS
    OFX
    !RVFVF
    TUBUVT
    🏃💨
    TestQueue::Runner::
    RSpec.new.execute
    FYFSTQFDRVFVF
    SVO@XPSLFS
    BMM@TVJUF@pMFT
    SVO@XPSLFS
    BMM@TVJUFT
    SVO@FBDI
    TVJUFT@GSPN@pMF
    %FTJHO34QFD

    View full-size slide

  59. 2VFVF3VOOFS
    SVO@TQFDT JUFSBUPS

    4UBUT
    3VOOFS34QFD
    5FTU'SBNFXPSL
    34QFD
    4UBUT4VJUF
    OFX
    3VOOFS
    34QFD
    3VOOFS
    5FTU'SBNFXPSL
    *UFSBUPS
    SVO@TQFDT FYBNQMF@HSPVQT

    UFTURVFVFSVOUJNF
    UFTUJOHGSBNFXPSL
    BEBQUFSDMBTT
    FYFDVUF
    OFX
    BMM@TVJUF@pMFT
    TVJUFT@GSPN@pMF
    OFX
    &OVNFSBCMF
    FBDI
    OFX
    8PSLFS
    OFX
    !RVFVF
    TUBUVT
    3VOOFS
    🏃💨
    TestQueue::Runner::
    RSpec.new.execute
    FYFSTQFDRVFVF
    SVO@XPSLFS
    BMM@TVJUF@pMFT
    SVO@XPSLFS
    BMM@TVJUFT
    SVO@FBDI
    TVJUFT@GSPN@pMF
    &YUFOTJPOQPJOU

    View full-size slide

  60. 2VFVF3VOOFS
    SVO@TQFDT JUFSBUPS

    4UBUT
    3VOOFS34QFD
    5FTU'SBNFXPSL
    34QFD
    4UBUT4VJUF
    OFX
    3VOOFS
    34QFD
    3VOOFS
    5FTU'SBNFXPSL
    *UFSBUPS
    SVO@TQFDT FYBNQMF@HSPVQT

    UFTURVFVFSVOUJNF
    UFTUJOHGSBNFXPSL
    BEBQUFSDMBTT
    FYFDVUF
    OFX
    BMM@TVJUF@pMFT
    TVJUFT@GSPN@pMF
    OFX
    &OVNFSBCMF
    FBDI
    OFX
    8PSLFS
    OFX
    !RVFVF
    TUBUVT
    5FTU'SBNFXPSL
    🏃💨
    TestQueue::Runner::
    RSpec.new.execute
    FYFSTQFDRVFVF
    SVO@XPSLFS
    BMM@TVJUF@pMFT
    SVO@XPSLFS
    BMM@TVJUFT
    SVO@FBDI
    TVJUFT@GSPN@pMF
    &YUFOTJPOQPJOU

    View full-size slide

  61. UFTURVFVFMJCUFTU@RVFVFSVOOFSSTQFDSC
    UFTURVFVFMJCUFTU@RVFVFSVOOFSSTQFDSC
    3VOOFS34QFDSVO@XPSLFS
    module QueueRunner < Runner
    def run_specs(iterator)
    @configuration.reporter.report(0) do |reporter|
    @configuration.with_suite_hooks do
    iterator.map { |g|
    start = Time.now
    if g.is_a? ::RSpec::Core::Example
    print " #{g.full_description}: "
    example = g
    g = example.example_group
    ::RSpec.world.filtered_examples.clear
    ::RSpec.world.filtered_examples[g] = [example]
    else
    print " #{g.description}: "
    end
    ret = g.run(reporter)
    # snip
    end
    alias_method :run_each, :run_specs
    module TestQueue
    class Runner
    class RSpec < Runner
    def run_worker(iterator)
    rspec = ::RSpec::Core::QueueRunner.new
    rspec.run_each(iterator).to_i
    ennnnd
    3VOTVJUF
    Iterator#each☝
    /BJWFJNQMFNFOUBUJPO
    UIBUEFQFOETPO34QFDT
    JOUFSOBM"1*

    View full-size slide

  62. 2VFVF3VOOFS
    SVO@TQFDT JUFSBUPS

    4UBUT
    3VOOFS34QFD
    5FTU'SBNFXPSL
    34QFD
    4UBUT4VJUF
    OFX
    3VOOFS
    34QFD
    3VOOFS
    5FTU'SBNFXPSL
    *UFSBUPS
    SVO@TQFDT FYBNQMF@HSPVQT

    UFTURVFVFSVOUJNF
    UFTUJOHGSBNFXPSL
    BEBQUFSDMBTT
    FYFDVUF
    OFX
    BMM@TVJUF@pMFT
    TVJUFT@GSPN@pMF
    OFX
    &OVNFSBCMF
    FBDI
    OFX
    8PSLFS
    OFX
    !RVFVF
    TUBUVT
    🏃💨
    TestQueue::Runner::
    RSpec.new.execute
    FYFSTQFDRVFVF
    SVO@XPSLFS
    BMM@TVJUF@pMFT
    SVO@XPSLFS
    BMM@TVJUFT
    SVO@FBDI
    TVJUFT@GSPN@pMF
    %FTJHO34QFD

    View full-size slide

  63. 4UBUT
    3VOOFS.JOJUFTU
    5FTU'SBNFXPSL
    .JOJUFTU
    4UBUT4VJUF
    .JOJUFTU
    .JOJUFTU
    3VOOFS
    5FTU'SBNFXPSL
    *UFSBUPS
    SVOOBCMFT JUFSBUPS

    UFTURVFVFSVOUJNF
    UFTUJOHGSBNFXPSL
    BEBQUFSDMBTT
    FYFDVUF
    OFX
    BMM@TVJUF@pMFT
    TVJUFT@GSPN@pMF
    BMM@TVJUF@pMFT
    &OVNFSBCMF
    SVO@XPSLFS
    BMM@TVJUFT
    OFX
    8PSLFS
    OFX
    !RVFVF
    TUBUVT
    FYFNJOJUFTURVFVF
    %FTJHO.JOJUFTU
    SVO
    🏃💨
    OFX
    TestQueue::Runner::
    Minitest.new.execute
    SVO@XPSLFS
    FBDI
    OFX
    TVJUFT@GSPN@pMF

    View full-size slide

  64. NJOJUFTUMJCNJOJUFTUSC
    module Minitest
    def self.run args = []
    # snip
    begin
    __run reporter, options
    rescue Interrupt
    warn 'Interrupted. Exiting...'
    UFTURVFVFMJCUFTU@RVFVFSVOOFSNJOJUFTUSC
    module Minitest
    def self.__run reporter, options
    suites = Runnable.runnables
    suites.map { |suite| suite.run reporter, options }
    ennd
    module TestQueue
    class Runner
    class Minitest < Runner
    def run_worker(iterator)
    ::Minitest::Test.runnables = iterator
    ::Minitest.run ? 0 : 1
    ennnnd
    3VOOFS.JOJUFTUSVO@XPSLFS
    $BMM__run
    3VOTVJUF
    Iterator#each☝
    /BJWFJNQMFNFOUBUJPO
    UIBUEFQFOETPOPQFO
    DMBTTBOEJOUFSOBM"1*
    0QFODMBTT

    View full-size slide

  65. 4VQQPSUUFTUJOHGSBNFXPSLT
    IUUQTHJUIVCDPNUNNUFTURVFVFUSFFNBTUFSMJCUFTU@RVFVFSVOOFS

    View full-size slide

  66. 1BSBMMFM5FTUJOH "HBJO

    1BSBMMFMXPSLFS
    4VJUF

    4VJUF

    4VJUF

    4VJUF

    4VJUF

    4VJUF

    4VJUF

    1BSBMMFMXPSLFS
    1BSBMMFMXPSLFS
    QSPDFTTJOHUJNF

    View full-size slide

  67. DPSF$16T)ZQFS5ISFBEJOH
    TFD





    TFSJBM QBSBMMFM
    w8IFOUFTURVFVF
    SVOTBMPUPGUFTU
    TVJUFT JUXJMMCF
    FGGFDUJWF
    wFHTFDTFD
    JO3VCP$PQSFQP

    View full-size slide

  68. % bundle exec rake spec
    Starting test-queue master (/tmp/test_queue_40616_5060.sock)
    ==> Summary (16 workers in 51.9441s)
    [ 1] 9 examples, 0 failures 1 suites in 34.0151s (pid 40649 exit 0 )
    [ 2] 109 examples, 0 failures 1 suites in 34.0151s (pid 40650 exit 0 )
    [ 3] 712 examples, 0 failures 46 suites in 34.0149s (pid 40651 exit 0 )
    [ 4] 2449 examples, 0 failures 61 suites in 34.0151s (pid 40652 exit 0 )
    [ 5] 129 examples, 0 failures 13 suites in 34.0151s (pid 40653 exit 0 )
    [ 6] 1628 examples, 0 failures 57 suites in 34.0150s (pid 40654 exit 0 )
    [ 7] 107 examples, 0 failures 1 suites in 34.0151s (pid 40655 exit 0 )
    [ 8] 1647 examples, 0 failures 61 suites in 34.0147s (pid 40656 exit 0 )
    [ 9] 2270 examples, 0 failures 59 suites in 34.0145s (pid 40657 exit 0 )
    [10] 1660 examples, 1 pending, 0 failures 57 suites in 34.0145s (pid 40658 exit 0 )
    [11] 132 examples, 0 failures 1 suites in 51.9309s (pid 40659 exit 0 )
    [12] 1600 examples, 0 failures 53 suites in 51.9306s (pid 40660 exit 0 )
    [13] 1924 examples, 0 failures 60 suites in 51.9295s (pid 40661 exit 0 )
    [14] 1993 examples, 0 failures 59 suites in 51.9281s (pid 40662 exit 0 )
    [15] 1967 examples, 0 failures 60 suites in 51.9271s (pid 40663 exit 0 )
    [16] 2303 examples, 0 failures 57 suites in 51.9237s (pid 40664 exit 0 )
    4MPXFTUBOEGBTUFTU
    4MPXFTU
    'BTUFTU

    View full-size slide

  69. )PXTIPVMEUIF
    SVOOJOHPGUFTUT
    B⒎FDUPOFBOPUIFS
    /PUBUBMM
    5&45%3*7&/%&7&-01.&/5#:&9".1-&

    l
    *TPMBUFE5FTUJTJNQPSUBOU

    View full-size slide

  70. w *TPMBUFUFTUTGSPNFBDIPUIFS
    w %BUBTUPSBHFBDDFTTFECZQBSBMMFM
    XPSLFSTTVDIBT3%#.4TIPVMECFTFU
    VQJOBQBSBMMFMNBOOFS
    w FH8PSLFS#SFBETUIFVQEBUFEEBUBPG
    XPSLFS"BOEUIFUFTUNBZGBJM
    4FUVQGPSQBSBMMFMUFTUJOH

    View full-size slide

  71. w 1SPEVDUDPEFBOEUFTUDPEFGPSNBUPUFTUJOHQBJS
    w 5IFVOJUPGQBSBMMFMUBTLTJOQBSBMMFMUFTUTJTVTVBMMZ
    UIFpMFMFWFM
    w 1BSBMMFMJ[JOHUFTUTVJUFTJOTNBMMFSVOJUTDBOSFEVDF
    UIFMPDLJOHQFSJPEGPSXPSLFST
    w *OPUIFSXPSET GPMMPXJOH431MFBETUPGBTUFS
    QBSBMMFMUFTUT
    1BSBMMFMUFTUBOE431
    4JOHMF3FTQPOTJCJMJUZ1SJODJQMF

    View full-size slide

  72. Parallel test runners
    Using RSpec
    Using Rails 6+ or AS
    Using Vanilla Ruby
    No
    Yes
    Yes
    No
    Yes
    No
    Start
    AS::
    TestCase
    parallel_tests
    or
    test-queue

    View full-size slide

  73. 8IJDIPOFJTCFUUFSUPVTF
    QBSBMMFM@UFTUTPSUFTURVFVF

    View full-size slide

  74. w /VNCFSPGEPXOMPBETPOSVCZHFNTPSH
    w /VNCFSPGTUBSTPO(JU)VC
    w 'SFRVFODZPGNBJOUFOBODF
    w QBSBMMFM@UFTUTJTNBJOUBJOFECVUUFTURVFVF
    XBTVONBJOUBJOFE
    w *OUFSFTUJOHEFTJHOCVUVOEFSVUJMJ[FE
    $SJUFSJBGPSHFNTFMFDUJPO

    View full-size slide

  75. 3FTVSSFDUJPOPGUFTURVFVF

    View full-size slide

  76. w *OFFEUPSFMFBTFBOFXWFSTJPO
    w !UNNBTLFENFUPCFBNBJOUBJOFSBOE
    *BDDFQUFE
    w .BJOUFOBODFJTSFTVNFEBGUFSPS
    ZFBST
    'JSTUUIJOHT*EJEBTBNBJOUBJOFS

    View full-size slide

  77. *GMFGUVOBUUFOEFE
    TPGUXBSFXJMMCSFBL
    CZ!ITCU
    XBTSFMFBTFE CVUTUJMM$*JTCSPLFO

    View full-size slide

  78. /P$* OPNFSHF
    $*XBTCSPLFO

    View full-size slide

  79. w *GPSHPUIPXUPVTF5SBWJT$*
    w *NOPUVQUPEBUFPOUIFSFDFOU
    EFWFMPQNFOUTPG5SBWJT$*
    w 8JUI(JU)VC"DUJPOT NBJOUFOBODFDBO
    CFEPOFXJUIPVUNF
    'SPN5SBWJT$*UP(JU)VC"DUJPOT

    View full-size slide

  80. w #BUT#BTI"VUPNBUFE5FTUJOH4ZTUFN
    w !TTUFQIFOTPOQSPEVDU
    w 5IFSFQPTJUPSZIBTCFFOBSDIJWFECZUIF
    PXOFSPO"QS 😢
    w *UTIBSEUPSFQMBDFBUFTUJOHGSBNFXPSL
    w *UTOFHBUJWF CVU*NMFBWJOHJUBTJUJT
    &&UFTUJOHPGUFTURVFVF

    View full-size slide

  81. w 5FTUOPUQBTTJOHXJUIMBUFTU
    WFSTJPOPG.JOJUFTU
    w 5IFUFTUDPEFBSPVOEUIF
    sleepNFUIPETFFNT
    TVTQJDJPVT
    w 'JYGBJMJOH(FNpMFGPS.JOJUFTU
    CZWFSTJPOQJOOJOH
    $VSSFOUTUBUVTPG&&UFTUJOH

    View full-size slide

  82. $*IBTSFDPWFSFE

    View full-size slide

  83. %FWFMPQNFOU
    FOWJSPONFOUJTSFBEZ
    *GUIF$*QBTTFT
    NBZCF*DBONFSHFJU

    View full-size slide

  84. #FZPOE
    UIF3FTVSSFDUJPO

    View full-size slide

  85. 4VQQPSUTUBUVTPGUFTUJOHGSBNFXPSLT
    5FTUJOH'SBNFXPSL $* /PUF
    34QFD ✅ 🤔5PPPME34QFDEPFTOUXPSLXJUI3VCZ
    34QFD ✅ ✅6TFEJO3VCP$PQ
    .JOJUFTU ✅ 🤔5PPPME
    .JOJUFTU ✅ CVU
    ✅6TFEJO3VCP$PQ.JOJUFTU
    $VDVNCFS ✅ CVU
    ❌0VUEBUFEUFTUDPEFFSSPSTJOUIFMBUFTUWFSTJPO
    UFTUVOJU ✅ 🤔/PUXPSLJOHXJUI3BJMT UNNUFTURVFVF

    34QFD EFW
    ✅ ✅6TFEJO3VCP$PQ
    5VSOJQ ✅ ✅/PUSBDLSFDPSEBTJUXBTKVTUNFSHFE

    View full-size slide

  86. w "QVMMSFRVFTUIBTCFFOPQFOFECZB34QFD
    DPSFUFBNNFNCFS
    w 34QFDJTBEFWFMPQNFOUTUBUVT #VU
    *EFDJEFEUPQSPWJEFBGFBUVSFPGUFTURVFVF
    w 050) JUTIBSEGPSUFTURVFVFNBJOUBJOFST
    UPLOPX34QFDTJOUFSOBMJNQMFNFOUBUJPO
    4VQQPSU34QFD EFW

    View full-size slide

  87. w 8PVMEOUJUCFDPOWFOJFOUJG34QFDIBEB
    QBSBMMFMUFTUSVOOFSGFBUVSFMJLF.JOJUFTU
    w 5IFTBNFDBOCFTBJEGPSUFTUVOJUBOE
    PUIFST
    *OUSPEVDFQBSBMMFMUFTUSVOOFSUP34QFD
    0QQPSUVOJUZGPSDPOUSJCVUJPO

    View full-size slide

  88. 0OFNPSFUIJOH

    View full-size slide

  89. w "MSFBEZJODMVEFTPuppetLintBTB
    QBSBMMFMSVOOFS
    w 8IJMFUIFQBSBMMFMHFNJTVTFEGPSQBSBMMFM
    FYFDVUJPOPG3VCP$PQ VTJOHB
    NFDIBOJTNMJLFUFTURVFVFNBZQSPWJEF
    FWFOGBTUFSSFTVMUT
    2VFVFTZTUFNOPUKVTUGPSUFTUJOH

    View full-size slide

  90. w 1SPHSBNNJOHGPSJOUFSQSPDFTT
    DPNNVOJDBUJPOJTBCBTJDGPVOEBUJPOGPS
    MFWFSBHJOHNVMUJDPSF
    'VOEBNFOUBMTPGQBSBMMFMQSPHSBNNJOH

    View full-size slide

  91. w UFTURVFVFEFQFOETPOJOUFSOBM"1*PGFBDI
    UFTUJOHGSBNFXPSL
    w *XBOUJUUPCFUFNQPSBSZUSBOTJFOU
    w .BZOPUSFBDIUFTURVFVF
    w )BWJOHBQMVHHBCMFQBSBMMFMUFTUNFDIBOJTNMJLF
    .JOJUFTUDPVMECFDPOWFOJFOU FH34QFDUPP

    )PQFGPSQBSBMMFMUFTUSVOOFS

    View full-size slide

  92. w 1BSBMMFMUFTUJOHSFRVJSFTNPOFZ💰
    w %FWFMPQFSTBSFUIFNPTUWBMVBCMFSFTPVSDF
    JOUIFQSPKFDU
    w *UTFBTJFSUPHFUNPSFNFNPSZBOE$16T
    XJUINPOFZUIBOUPIJSFTFOJPSFOHJOFFST
    w %PZPVSCFTUUPTFDVSFUIFCVEHFU
    5IFSFJTBHPMECVMMFU

    View full-size slide

  93. 1BSBMMFM5FTUJOH
    .BLFT
    :PV)BQQZ
    (JU)VC!LPJD

    View full-size slide