Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Resurrection of 
the Fast Parallel Test Runner

The Resurrection of 
the Fast Parallel Test Runner

Koichi ITO

May 12, 2023
Tweet

More Decks by Koichi ITO

Other Decks in Programming

Transcript

  1. Koichi ITO / ESM, Inc.
    RubyKaigi 2023
    The Resurrection of
    the Fast Parallel Test Runner
    May, 12th, 2023
    Matsumoto Performing Arts Centre
    The Force Awakens

    View Slide

  2. !LPJD
    w 044QSPHSBNNFS
    w 3VCP$PQDPSFUFBN
    w &OHJOFFSJOH.BOBHFSBOE
    %JTUJOHVJTIFE&OHJOFFSPG&4. *OD
    w 3VCZ,BJHJTQFBLFSBU
    -5 5BLFPVU
    BOE

    View Slide

  3. .BJOUBJO044&WFSZ%BZ
    IUUQTHJUIVCDPNLPJD

    View Slide

  4. &4. *OD

    View Slide

  5. $P⒎FFIPVTF4QPOTPS
    w"CF ᘖᘣඒֶΞϕ

    w0LJOBEP ͓͖ͳಊ

    w:BNBHB ٤஡ࢁխ

    w"OENPSF

    View Slide

  6. /PWFMUZ

    View Slide

  7. 4
    Q
    FBLFS
    4
    Q
    FBLFS
    JOQ
    FSTPO
    JOQ
    FSTPO
    JOQ
    FSTPO
    JOQ
    FSTPO
    JOQ
    FSTPO
    JOQ
    FSTPO
    POMJOF
    POMJOF
    POMJOF
    POMJOF
    POMJOF
    POMJOF
    POMJOF
    4QFBLFSTBOE"UUFOEFFT
    PSHBOJ[FS
    PSHBOJ[FS
    "EWJTPS "EWJTPS

    View Slide

  8. Support OSS community

    View Slide

  9. +PJOVT
    🗾w-FUTHSPXUPHFUIFS
    w:PVDBOXPSL
    SFNPUFMZGSPN
    BOZXIFSFJO+BQBO
    w4FFBHJMFFTNDPKQ

    View Slide

  10. View Slide

  11. View Slide

  12. $POUFOUT
    8IBUTQBSBMMFMUFTUJOH
    1BSBMMFMUFTUSVOOFST
    3FTVSSFDUJPOPGUFTURVFVF

    View Slide

  13. 8IBUTQBSBMMFMUFTUJOH

    View Slide

  14. 5PNF MFHBDZDPEF
    JTTJNQMZDPEF
    XJUIPVUUFTUT
    803,*/(&''&$5*7&-:8*5)-&("$:$0%&

    l
    "CPPLJOUIF6ODMF#PCTFSJFTTBZT

    View Slide

  15. w 4PGUXBSFDPEFLFFQTHSPXJOHMBSHFS
    w #VTJOFTTHSPXUIBOETPGUXBSFHSPXUI
    HPIBOEJOIBOE
    w $POUJOVPVTMZQFSGPSN5%% #%% 5"%
    BOESFHSFTTJPOUFTUT
    5FTUDPEFJTJNQPSUBOU

    View Slide

  16. )PXMPOHEPZPV
    XBJUGPSZPVSCVJME

    View Slide


  17. "JNGPS
    NJOVUFTPSMFTT

    View Slide

  18. w FYBNQMFT 6TJOH34QFD

    w "SFBMXPSMEBQQMJDBUJPOUBLFPWFSBO
    IPVS
    "SFBMXPSMEBQQMJDBUJPO
    $ bin/rails stats
    (snip)
    Code LOC: 12796 Test LOC: 23518 Code to Test Ratio: 1:1.8

    View Slide

  19. "OFXQSPCMFN
    4MPXUFTU

    View Slide

  20. w $BOOPUSVOUFTUTPOQVMMSFRVFTUTJO
    BEBZ
    w 5IFCVJMETBSFOUpOJTIFEFWFOCZ
    UIFOFYUNPSOJOH
    *UUBLFTBOIPVSUPDPNQMFUFBCVJME

    View Slide


  21. View Slide

  22. w TFDPOETJOBEBZ
    w 'PSFYBNQMF TMPXUFTUTUBLJOH
    TFDPOETFBDIXJMMSFRVJSFNJOVUFT
    w )PXNBOZDPNNJUTEPFTZPVSUFBN
    NBLFFWFSZEBZ
    5JNFJTJNQPSUBOU

    View Slide

  23. w 1SPpMJOHCBTFEUVOJOH%POUHVFTT NFBTVSF
    • FH
    w .BZCFTQFEVQXJUIBDMFBOJOHTUSBUFHZ
    w FH%BUBCBTF$MFBOFSP⒎FSTEBUBCBTFDMFBOJOH
    TUSBUFHJFTTVDIBTtransaction deletion BOE
    truncation
    JUTBHPPEJEFBUPDIPPTFUIFPQUJNBMPOF
    "EESFTTJOH4MPX5FTU#PUUMFOFDLT
    $ rspec -p 30 spec/acceptance

    View Slide

  24. .ZSFDPNNFOEBUJPO
    w 'JSTU QSFQBSFNPOFZ💰JGZPVBSF
    CVJMEJOHCVTJOFTTTPGUXBSF
    w /FYU QSFQBSFFOPVHI$16TBOENFNPSZ
    w 'JOBMMZ 1BSBMMFMJ[FZPVSUFTUT
    w 5IBUTJU"'"*, JUTUIFNPTUF⒎FDUJWF

    View Slide

  25. 1BSBMMFM5FTUJOH

    View Slide

  26. 4FSJBM5FTUJOH
    4JOHMFXPSLFS
    4VJUF

    4VJUF

    4VJUF

    4VJUF

    4VJUF

    4VJUF

    4VJUF

    w 8PSLFS5FTUFYFDVUJPOQSPDFTT
    w 4VJUF5FTUFYFDVUJPOVOJU UPQMFWFM
    TestDMBTT describe %4- pMF FUD

    QSPDFTTJOHUJNF
    $ bundle exec ruby -Itest path # or rspec path

    View Slide

  27. 1BSBMMFM5FTUJOH
    1BSBMMFMXPSLFS
    4VJUF

    4VJUF

    4VJUF

    4VJUF

    4VJUF

    4VJUF

    4VJUF

    1BSBMMFMXPSLFS
    1BSBMMFMXPSLFS
    QSPDFTTJOHUJNF

    View Slide

  28. 4FSJBMWT1BSBMMFM
    1BSBMMFMXPSLFS
    4VJUF

    4VJUF

    4VJUF

    4VJUF

    4VJUF

    4VJUF

    4VJUF

    1BSBMMFMXPSLFS
    1BSBMMFMXPSLFS
    4JOHMFXPSLFS
    4VJUF

    4VJUF

    4VJUF

    4VJUF

    4VJUF

    4VJUF

    4VJUF

    /PXBJUUJNF
    QSPDFTTJOHUJNF

    View Slide

  29. 1BSBMMFMJ[BUJPOJTHSFBU
    5JNFJTNPSFWBMVBCMF
    UIBOBOZUIJOH

    View Slide


  30. 1BSBMMFMUFTUSVOOFST

    View Slide

  31. w .JOJUFTU
    w 3BJMT
    w [email protected]
    w UFTURVFVF
    5PVSPG1BSBMMFM5FTUJOH5PPMT

    View Slide

  32. NJOJUFTUNJOJUFTU
    NJOJUFTUQSPWJEFTBDPNQMFUFTVJUFPGUFTUJOHGBDJMJUJFT
    TVQQPSUJOH5%% #%% NPDLJOH BOECFODINBSLJOH

    View Slide

  33. w #BDLFOEPGActiveSupport::TestCaseJO3BJMT
    w 3BOEPNJ[FETFSJBMUFTUJTUIFEFGBVMU
    w parallelize_me! QSPWJEFTQBSBMMFMUFTUJOHGPSUIF
    RVFVFNPEFM*UVTFTNVMUJUISFBEJOHCZEFGBVMU
    w :PVDBODIBOHFUIFQBSBMMFMFYFDVUJPONPEVMFCZ
    Minitest.parallel_executor
    .JOJUFTU

    View Slide

  34. &YUFOTJPOQPJOUGPSQBSBMMFMUFTUJOH
    .JOJUFTU
    [email protected]
    JOUFSGBDF
    w 5ISFBECZEFGBVMUGPSUIFQBSBMMFMUFTU
    w *.0 UFTUTUIBUBSFDPNQMFUFEJONFNPSZBSFOPUGBTU
    &⒎FDUJWFGPSUFTUTUIBUVTF*0
    *OUFSGBDFGPSQBSBMMFMUFTUJOH
    .JOJUFTU
    1BSBMMFM
    &YFDVUPS
    !RVFVF
    "DMBTTUIBUJNQMFNFOUT
    NFUIPETTVDIBTstart
    << size BOEshutdown
    BVUPSVO

    View Slide

  35. EFGTUBSU
    !QPPMTJ[FUJNFTNBQ\
    5ISFBEOFX !RVFVF
    EPcRVFVFc
    [email protected]@FYDFQUJPOUSVF
    XIJMF KPCRVFVFQPQ

    LMBTT NFUIPE SFQPSUFSKPC
    SFQPSUFSTZODISPOJ[F\SFQPSUFSQSFSFDPSELMBTT NFUIPE^
    [email protected]@NFUIPELMBTT NFUIPE
    SFQPSUFSTZODISPOJ[F\SFQPSUFSSFDPSESFTVMU^
    FOE
    FOE
    ^
    FOE
    EFGXPSL!RVFVFXPSLFOE
    .JOJUFTU1BSBMMFM&YFDVUPS
    [email protected]
    -PPQXJUIqueue.pop
    $BMMBUFTUNFUIPE
    .JOJUFTU1BSBMMFM&YFDVUPS XPSL

    IUUQTHJUIVCDPNNJOJUFTUNJOJUFTUCMPCWMJCNJOJUFTUQBSBMMFMSC--
    /05&@queueDBOCF
    TIBSFEXJUIBOJOTUBODF
    WBSJBCMFCFDBVTFJUJTB
    UISFBENPEFM

    View Slide

  36. .JOJUFTU5FTU
    NPEVMF5FTUOPEPD
    [email protected][[email protected][F\ZJFME^
    FOE
    NPEVMF$MBTT.FUIPETOPEPD
    [email protected]@NFUIPELMBTT [email protected] SFQPSUFS
    [email protected]
    FOE
    [email protected]
    QBSBMMFM
    FOOOE
    .JOJUFTU
    3VOOBCMF
    .JOJUFTU
    5FTU
    IUUQTHJUIVCDPNNJOJUFTUNJOJUFTUCMPCWMJCNJOJUFTUQBSBMMFMSC--
    [email protected]
    .JOJUFTU
    1BSBMMFM5FTU
    $BTT.FUIPET
    NJYJO
    DMBTT.JOJUFTU5FTU.JOJUFTU3VOOBCMF
    FYUFOE.JOJUFTU1BSBMMFM5FTU$MBTT.FUIPET
    FOE

    View Slide

  37. <"QQFOEJY>IFMMSC
    IUUQTHJUIVCDPNNJOJUFTUNJOJUFTUCMPCWMJCNJOJUFTUIFMMSC
    w %FpOJOHUIFFOWJSPONFOU
    WBSJBCMFMT_HELLBDUJWBUFT
    NJOJUFTUIFMMSC
    w "DUJWBUFQBSBMMFMBOEQSPWFJU

    View Slide

  38. SBJMTSBJMT
    3BJMTJTBXFCBQQMJDBUJPOGSBNFXPSLUIBUJODMVEFTFWFSZUIJOH
    OFFEFEUPDSFBUFEBUBCBTFCBDLFEXFCBQQMJDBUJPOT
    BDDPSEJOHUPUIF.PEFM7JFX$POUSPMMFS .7$
    QBUUFSO

    View Slide

  39. w 1BSBMMFMUFTUJOHCZEFGBVMU 4JODF3BJMT

    w %FGBVMUUISFTIPMEJT "WPJEJOHUIFPWFSIFBEPGGPSLJOH

    3BJMTAS::TestCase
    $ bin/rails test
    Running 60 tests in parallel using 8 processes
    Run options: --seed 21371
    # Running:
    .....................................................
    .......
    Finished in 0.205488s, 291.9879 runs/s, 291.9879
    assertions/s.
    60 runs, 60 assertions, 0 failures, 0 errors, 0 skips
    MFTTUIBOUFTUT UFTUT
    $ bin/rails test
    Running 10 tests in a single process (parallelization
    threshold is 50)
    Run options: --seed 1398
    # Running:
    ..........
    Finished in 0.016447s, 608.0136 runs/s, 608.0136
    assertions/s.
    10 runs, 10 assertions, 0 failures, 0 errors, 0 skips

    View Slide

  40. ENV["RAILS_ENV"] ||= "test"
    require_relative "../config/environment"
    require "rails/test_help"
    class ActiveSupport::TestCase
    # Run tests in parallel with specified workers
    parallelize(workers: :number_of_processors)
    # Setup all fixtures in test/fixtures/*.yml for all tests in alphabetical order.
    fixtures :all
    # Add more helper methods to be used by all tests here...
    end
    [email protected]
    QBSBMMFMJ[BUJPOCZEFGBVMU
    PQFODMBTT

    View Slide

  41. module ActiveSupport
    class TestCase < ::Minitest::Test
    class << self
    def parallelize(workers: :number_of_processors, with: :processes,
    threshold: ActiveSupport.test_parallelization_threshold)
    workers = Concurrent.physical_processor_count if workers
    == :number_of_processors
    workers = ENV["PARALLEL_WORKERS"].to_i if ENV["PARALLEL_WORKERS"]
    return if workers <= 1
    Minitest.parallel_executor =
    ActiveSupport::Testing::ParallelizeExecutor.new(size: workers, with:
    with, threshold: threshold)
    ennnnnd
    *OIFSJUMinitest::Test
    $IBOHFUPQBSBMMFMFYFDVUPSPG3BJMT
    "DUJWF4VQQPSU5FTU$BTF

    View Slide

  42. &YUFOTJPOQPJOUGPSQBSBMMFMUFTUJOH
    .JOJUFTU
    [email protected]
    JOUFSGBDF
    w 5ISFBECZEFGBVMUGPSUIFQBSBMMFMUFTU
    w *.0 UFTUTUIBUBSFDPNQMFUFEJONFNPSZBSFOPUGBTU
    &⒎FDUJWFGPSUFTUTUIBUVTF*0
    *OUFSGBDFGPSQBSBMMFMUFTUJOH
    .JOJUFTU
    1BSBMMFM
    &YFDVUPS
    !RVFVF
    "DMBTTUIBUJNQMFNFOUT
    NFUIPETTVDIBTstart
    << size BOEshutdown
    BVUPSVO

    View Slide

  43. "45FTUJOH1BSBMMFM&YFDVUPS
    .JOJUFTU
    [email protected]
    JOUFSGBDF
    w 3FQMBDFXJUIUIF.JOJUFTUFYUFOTJPOQPJOU
    w 4XJUDIGSPNUISFBEUPQSPDFTTQBSBMMFMJ[BUJPO
    "OJOUFSGBDFGPSUIFQBSBMMFMUFTU
    .JOJUFTU
    1BSBMMFM
    &YFDVUPS
    !RVFVF
    BVUPSVO
    "DUJWF4VQQPSU
    5FTUJOH
    1BSBMMFMJ[F&YFDVUPS
    [email protected]
    "DUJWF4VQQPSU
    5FTUJOH
    1BSBMMFMJ[BUJPO

    View Slide

  44. "45FTUJOH1BSBMMFMJ[BUJPO
    %3C'SPOU
    w 4FSWFS8PSLFSNPEFM NVMUJQSPDFTTJOH

    w *OUFSQSPDFTTDPNNVOJDBUJPOVTJOHE3VCZ
    1BSBMMFMJ[BUJPO
    4FSWFS
    RVFVF
    1BSBMMFMJ[BUJPO
    8PSLFS
    FORVFVF
    GPSLFEQSPDFTT
    ESCVOJY

    O
    NBJOQSPDFTT
    1BSBMMFMJ[BUJPO
    8PSLFS
    EFRVFVF XIJMFKPC!RVFVFQPQ
    [email protected] KPC

    FOE
    $BMMMinitest.run_one_method
    1BSBMMFMJ[BUJPO
    8PSLFS
    1BSBMMFMJ[BUJPO

    View Slide

  45. w AS::TestCase.parallelizeCZEFGBVMU
    w 3BJMTQSPWJEFTUIFCFOFpUTPGNVMUJDPSF
    QSPDFTTJOHUISPVHIUIFVTFPGQSPDFTTFT
    w 3VOTQBSBMMFMJTNCZEFGBVMUXJUI
    Concurrent.physical_processor_count
    w 3VOUJNF$16TBOENFNPSZBSFJNQPSUBOU
    3BJMTJTQSBHNBUJD

    View Slide

  46. 3BJMTEPFTOU
    TVQQPSU34QFDCZ
    EFGBVMU🤔

    View Slide

  47. w [email protected]
    w UNNUFTURVFVF
    1BSBMMFMUFTUSVOOFS

    View Slide

  48. [email protected]
    4QFFEVQ5FTU6OJU34QFD$VDVNCFS
    4QJOBDICZSVOOJOHQBSBMMFMPONVMUJQMF$16DPSFT

    View Slide

  49. w %FQFOETPOUIFQBSBMMFMHFN
    • Parallel.each { ... }
    • Parallel.map { ... }
    w 3VOQSFTQMJUUJOHUFTUTJOQBSBMMFMGPSBMMUFTUpMFT
    w *[email protected]@STQFDMPHBGUFSSVOOJOH
    w 7FSZTJNQMF😀
    [email protected]

    View Slide

  50. w 5IFSFJTBUJNFEJ⒎FSFODFCFUXFFOUIFXPSLFS
    UIBUpOJTIFTpSTUBOEUIFXPSLFSUIBUpOJTIFT
    MBTU
    w 'JOJTIFEXPSLFSTIBWFXBTUFEXBJUUJNF
    w /PUFUIBUUIFFYFDVUJPOSFTVMUNBUSJYpMFJT
    VTFEUPNJOJNJ[FUIFUJNFEJ⒎FSFODFCFUXFFO
    UFTUTBTNVDIBTQPTTJCMF
    DPOTPGQSFTQMJUUJOH

    View Slide

  51. UNNUFTURVFVF
    :FUBOPUIFSQBSBMMFMUFTUSVOOFS CVJMUVTJOHB
    DFOUSBMJ[FERVFVFUPFOTVSFPQUJNBMEJTUSJCVUJPOPG
    UFTUTCFUXFFOXPSLFST

    View Slide

  52. w 5IFBVUIPSJT"NBO(VQUB,BSNBOJ !UNN

    w 3FBEZXPSLFSTEFRVFVFBOESVOUFTUTGSPNUIF
    RVFVFXBJUJOHUPCFSVO
    w 'FXFSXBJUJOHXPSLFSTBOEMFTTXBTUF
    w [email protected] [email protected]@TUBUT
    FYFDVUJPOSFTVMUNBUSJYBOESVOOJOHTMPXUFTUT
    pSTU
    UFTURVFVF

    View Slide

  53. 6/*94PDLFUPS5$14PDLFU
    5ISFFQSPDFTTUZQFT
    6/*94PDLFUPS5$14PDLFU
    6/*94FSWFSPS5$14FSWFS
    UFTURVFVF
    NBTUFSQSPDFTT
    UFTURVFVFTVJUF
    EJTDPWFSZQSPDFTT

    UFTURVFVF
    XPSLFSQSPDFTT



    %JTUSJCVUJOHUFTUTVJUF
    GSPNRVFVFBOE
    TVNNBSJ[FUIFSFTVMUT
    %FRVFVFBOESVOUFTUTVJUFT
    1BSTFUFTUTVJUFTBOEFORVFVFJU
    1⃣ 2⃣
    3⃣

    View Slide

  54. class Runner
    def execute_internal
    start_master
    prepare(@concurrency)
    @prepared_time = Time.now
    start_relay if relay?
    discover_suites
    spawn_workers
    distribute_queue
    ensure
    stop_master
    kill_subprocesses
    ennd
    [email protected]
    def start_master
    if !relay?
    if @socket =~ /\A(?:(.+):)?(\d+)\z/
    address = $1 || '0.0.0.0'
    port = $2.to_i
    @socket = "#{$1}:{#$2}"
    @server = TCPServer.new(address, port)
    else
    FileUtils.rm_f(@socket)
    @server = UNIXServer.new(@socket)
    end
    ennd
    1⃣
    .BTUFS1SPDFTT
    .BTUFS1SPDFTT

    View Slide

  55. class Runner
    def execute_internal
    start_master
    prepare(@concurrency)
    @prepared_time = Time.now
    start_relay if relay?
    discover_suites
    spawn_workers
    distribute_queue
    ensure
    stop_master
    kill_subprocesses
    ennd
    [email protected]
    def discover_suites
    return if relay?
    return if @allowlist.any? && @awaited_suites.empty?
    @discovering_suites_pid = fork do
    terminate = false
    Signal.trap("INT") { terminate = true }
    $0 = 'test-queue suite discovery process'
    @test_framework.all_suite_files.each do |path|
    @test_framework.suites_from_file(path).each do |
    suite_name, suite|
    Kernel.exit!(0) if terminate
    @server.connect_address.connect do |sock|
    sock.puts("TOKEN=#{@run_token}")
    sock.puts("NEW SUITE #{Marshal.dump([suite_name,
    path])}")
    ennd
    2⃣
    %JTDPWFSZ1SPDFTT
    .BTUFS1SPDFTT

    View Slide

  56. class Runner
    def execute_internal
    start_master
    prepare(@concurrency)
    @prepared_time = Time.now
    start_relay if relay?
    discover_suites
    spawn_workers
    distribute_queue
    ensure
    stop_master
    kill_subprocesses
    ennd
    [email protected]
    def spawn_workers
    @concurrency.times do |i|
    num = i + 1
    pid = fork do
    @server.close if @server
    iterator = Iterator.new(@test_framework, relay? ?
    @relay : @socket, method(:around_filter),
    early_failure_limit: @early_failure_limit, run_token:
    @run_token)
    after_fork_internal(num, iterator)
    ret = run_worker(iterator) || 0
    cleanup_worker
    Kernel.exit! ret
    end
    @workers[pid] = Worker.new(pid, num)
    ennd
    3⃣
    3VOXPSLFS🏃
    8PSLFS1SPDFTT
    .BTUFS1SPDFTT

    View Slide

  57. 6/*94PDLFUPS5$14PDLFU
    6/*94PDLFUPS5$14PDLFU
    6/*94FSWFSPS5$14FSWFS
    *OUFSBDUJPOPGQSPDFTTFT
    UFTURVFVF
    NBTUFSQSPDFTT
    UFTURVFVFTVJUF
    EJTDPWFSZQSPDFTT
    UFTURVFVF
    XPSLFSQSPDFTT
    %FRVFVFBOESVOUFTUTVJUFT
    1BSTFUFTUTVJUFTBOE
    FORVFVFJU
    %JTUSJCVUJOHUFTUTVJUF
    GSPNRVFVFBOE
    TVNNBSJ[FUIFSFTVMUT
    GPSL
    /&846*5&
    GPSL
    .BSTIBMEVNQ
    .BSTIBMMPBE
    5FTU'SBNFXPSL
    3VOOFS
    101

    View Slide

  58. $PNNBOEMJTU
    $PNNBOE %JSFDUJPO %FTDSJQUJPO
    /&846*5& EJTDPWFSZˠNBTUFS &ORVFVFBUFTUTVJUF
    101 XPSLFSˠNBTUFS %FRVFVFBUFTUTVJUF
    8"*5 NBTUFSˠXPSLFS "XBJUBUFTUTVJUFXIFOEFRVFVF
    ,"#00. XPSLFSˠNBTUFS *NNFEJBUFMZBCPSU
    3&.05&."45&3 XPSLFSˠNBTUFS 3FHJTUFSBSFNPUFXPSLFSUPUIFSFNPUFNBTUFSJGSFMBZ
    0, NBTUFSˠXPSLFS 3FQMZSFHJTUFSTVDDFTTUPUIFSFNPUFXPSLFSJGSFMBZ
    803,&3 XPSLFSˠNBTUFS $PNQMFUFPSBCPSUBSFNPUFXPSLFSJGSFMBZ
    "MXBZTNBUDIBVOJRVFTOKENJTTVFEBOEVTFEBTBQSPUPDPMXJUIUIFNBTUFSQSPDFTT

    View Slide

  59. class Runner
    def execute_internal
    start_master
    prepare(@concurrency)
    @prepared_time = Time.now
    start_relay if relay?
    discover_suites
    spawn_workers
    distribute_queue
    ensure
    stop_master
    kill_subprocesses
    ennd
    [email protected]
    def distribute_queue
    # snip
    until !awaiting_suites? && @queue.empty? && remote_workers == 0
    # snip
    case cmd
    when /\APOP (\S+) (\d+)/
    # snip
    when /\AREMOTE MASTER (\d+) ([\w\.-]+)(?: (.+))?/
    # snip
    when /\AWORKER (\d+)/
    # snip
    when /\ANEW SUITE (.+)/
    # snip
    when /\AKABOOM/
    break
    else
    warn("Ignoring unrecognized command: \"#{cmd}\"")
    end
    sock.close
    ennd .BTUFS1SPDFTT
    .BTUFS1SPDFTT

    View Slide

  60. def each
    # snip
    loop do
    if @early_failure_limit && @failures >= @early_failure_limit
    connect_to_master('KABOOM'); break
    else
    client = connect_to_master("POP #{Socket.gethostname} #{Process.pid}")
    end
    break if client.nil?
    _r, _w, e = IO.select([client], nil, [client], nil)
    break unless e.empty?
    if data = client.read(65536)
    client.close
    item = Marshal.load(data)
    # snip
    suite_name, path = item
    suite = load_suite(suite_name, path)
    # snip
    yield suite
    *UFSBUPSFBDI
    8PSLFS1SPDFTT
    class Iterator
    include Enumerable

    View Slide

  61. 2VFVF3VOOFS
    [email protected] JUFSBUPS

    4UBUT
    3VOOFS34QFD
    5FTU'SBNFXPSL
    34QFD
    4UBUT4VJUF
    OFX
    3VOOFS
    34QFD
    3VOOFS
    5FTU'SBNFXPSL
    *UFSBUPS
    [email protected] [email protected]

    UFTURVFVFSVOUJNF
    UFTUJOHGSBNFXPSL
    BEBQUFSDMBTT
    FYFDVUF
    OFX
    [email protected]@pMFT
    [email protected]@pMF
    OFX
    &OVNFSBCMF
    FBDI
    OFX
    8PSLFS
    OFX
    !RVFVF
    TUBUVT
    🏃💨
    TestQueue::Runner::
    RSpec.new.execute
    FYFSTQFDRVFVF
    [email protected]
    [email protected]@pMFT
    [email protected]
    [email protected]
    [email protected]
    [email protected]@pMF
    %FTJHO34QFD

    View Slide

  62. w UFTURVFVFTVQQPSUT34QFD 34QFD
    .JOJUFTU .JOJUFTU UFTUVOJU $VDVNCFS
    w UFTURVFVFQSPWJEFTUIFDPPSEJOBUJPOPG
    QSPDFTTFTJODPNNPOQBSBMMFMQSPDFTTJOH
    w 5FTUTVJUFBOEUFTUSVOOFS"1*TBSFUFTUJOH
    GSBNFXPSLTQFDJpD
    1MVHHBCMF%FTJHO

    View Slide

  63. 2VFVF3VOOFS
    [email protected] JUFSBUPS

    4UBUT
    3VOOFS34QFD
    5FTU'SBNFXPSL
    34QFD
    4UBUT4VJUF
    OFX
    3VOOFS
    34QFD
    3VOOFS
    5FTU'SBNFXPSL
    *UFSBUPS
    [email protected] [email protected]

    UFTURVFVFSVOUJNF
    UFTUJOHGSBNFXPSL
    BEBQUFSDMBTT
    FYFDVUF
    OFX
    [email protected]@pMFT
    [email protected]@pMF
    OFX
    &OVNFSBCMF
    FBDI
    OFX
    8PSLFS
    OFX
    !RVFVF
    TUBUVT
    🏃💨
    TestQueue::Runner::
    RSpec.new.execute
    FYFSTQFDRVFVF
    [email protected]
    [email protected]@pMFT
    [email protected]
    [email protected]
    [email protected]
    [email protected]@pMF
    %FTJHO34QFD

    View Slide

  64. 2VFVF3VOOFS
    [email protected] JUFSBUPS

    4UBUT
    3VOOFS34QFD
    5FTU'SBNFXPSL
    34QFD
    4UBUT4VJUF
    OFX
    3VOOFS
    34QFD
    3VOOFS
    5FTU'SBNFXPSL
    *UFSBUPS
    [email protected] [email protected]

    UFTURVFVFSVOUJNF
    UFTUJOHGSBNFXPSL
    BEBQUFSDMBTT
    FYFDVUF
    OFX
    [email protected]@pMFT
    [email protected]@pMF
    OFX
    &OVNFSBCMF
    FBDI
    OFX
    8PSLFS
    OFX
    !RVFVF
    TUBUVT
    3VOOFS
    🏃💨
    TestQueue::Runner::
    RSpec.new.execute
    FYFSTQFDRVFVF
    [email protected]
    [email protected]@pMFT
    [email protected]
    [email protected]
    [email protected]
    [email protected]@pMF
    &YUFOTJPOQPJOU

    View Slide

  65. 2VFVF3VOOFS
    [email protected] JUFSBUPS

    4UBUT
    3VOOFS34QFD
    5FTU'SBNFXPSL
    34QFD
    4UBUT4VJUF
    OFX
    3VOOFS
    34QFD
    3VOOFS
    5FTU'SBNFXPSL
    *UFSBUPS
    [email protected] [email protected]

    UFTURVFVFSVOUJNF
    UFTUJOHGSBNFXPSL
    BEBQUFSDMBTT
    FYFDVUF
    OFX
    [email protected]@pMFT
    [email protected]@pMF
    OFX
    &OVNFSBCMF
    FBDI
    OFX
    8PSLFS
    OFX
    !RVFVF
    TUBUVT
    5FTU'SBNFXPSL
    🏃💨
    TestQueue::Runner::
    RSpec.new.execute
    FYFSTQFDRVFVF
    [email protected]
    [email protected]@pMFT
    [email protected]
    [email protected]
    [email protected]
    [email protected]@pMF
    &YUFOTJPOQPJOU

    View Slide

  66. [email protected]
    [email protected]
    [email protected]
    module QueueRunner < Runner
    def run_specs(iterator)
    @configuration.reporter.report(0) do |reporter|
    @configuration.with_suite_hooks do
    iterator.map { |g|
    start = Time.now
    if g.is_a? ::RSpec::Core::Example
    print " #{g.full_description}: "
    example = g
    g = example.example_group
    ::RSpec.world.filtered_examples.clear
    ::RSpec.world.filtered_examples[g] = [example]
    else
    print " #{g.description}: "
    end
    ret = g.run(reporter)
    # snip
    end
    alias_method :run_each, :run_specs
    module TestQueue
    class Runner
    class RSpec < Runner
    def run_worker(iterator)
    rspec = ::RSpec::Core::QueueRunner.new
    rspec.run_each(iterator).to_i
    ennnnd
    3VOTVJUF
    Iterator#each☝
    /BJWFJNQMFNFOUBUJPO
    UIBUEFQFOETPO34QFDT
    JOUFSOBM"1*

    View Slide

  67. 2VFVF3VOOFS
    [email protected] JUFSBUPS

    4UBUT
    3VOOFS34QFD
    5FTU'SBNFXPSL
    34QFD
    4UBUT4VJUF
    OFX
    3VOOFS
    34QFD
    3VOOFS
    5FTU'SBNFXPSL
    *UFSBUPS
    [email protected] [email protected]

    UFTURVFVFSVOUJNF
    UFTUJOHGSBNFXPSL
    BEBQUFSDMBTT
    FYFDVUF
    OFX
    [email protected]@pMFT
    [email protected]@pMF
    OFX
    &OVNFSBCMF
    FBDI
    OFX
    8PSLFS
    OFX
    !RVFVF
    TUBUVT
    🏃💨
    TestQueue::Runner::
    RSpec.new.execute
    FYFSTQFDRVFVF
    [email protected]
    [email protected]@pMFT
    [email protected]
    [email protected]
    [email protected]
    [email protected]@pMF
    %FTJHO34QFD

    View Slide

  68. 4UBUT
    3VOOFS.JOJUFTU
    5FTU'SBNFXPSL
    .JOJUFTU
    4UBUT4VJUF
    .JOJUFTU
    .JOJUFTU
    3VOOFS
    5FTU'SBNFXPSL
    *UFSBUPS
    SVOOBCMFT JUFSBUPS

    UFTURVFVFSVOUJNF
    UFTUJOHGSBNFXPSL
    BEBQUFSDMBTT
    FYFDVUF
    OFX
    [email protected]@pMFT
    [email protected]@pMF
    [email protected]@pMFT
    &OVNFSBCMF
    [email protected]
    [email protected]
    OFX
    8PSLFS
    OFX
    !RVFVF
    TUBUVT
    FYFNJOJUFTURVFVF
    %FTJHO.JOJUFTU
    SVO
    🏃💨
    OFX
    TestQueue::Runner::
    Minitest.new.execute
    [email protected]
    FBDI
    OFX
    [email protected]@pMF

    View Slide

  69. NJOJUFTUMJCNJOJUFTUSC
    module Minitest
    def self.run args = []
    # snip
    begin
    __run reporter, options
    rescue Interrupt
    warn 'Interrupted. Exiting...'
    [email protected]
    module Minitest
    def self.__run reporter, options
    suites = Runnable.runnables
    suites.map { |suite| suite.run reporter, options }
    ennd
    module TestQueue
    class Runner
    class Minitest < Runner
    def run_worker(iterator)
    ::Minitest::Test.runnables = iterator
    ::Minitest.run ? 0 : 1
    ennnnd
    [email protected]
    $BMM__run
    3VOTVJUF
    Iterator#each☝
    /BJWFJNQMFNFOUBUJPO
    UIBUEFQFOETPOPQFO
    DMBTTBOEJOUFSOBM"1*
    0QFODMBTT

    View Slide

  70. 4VQQPSUUFTUJOHGSBNFXPSLT
    [email protected]

    View Slide

  71. 1BSBMMFM5FTUJOH "HBJO

    1BSBMMFMXPSLFS
    4VJUF

    4VJUF

    4VJUF

    4VJUF

    4VJUF

    4VJUF

    4VJUF

    1BSBMMFMXPSLFS
    1BSBMMFMXPSLFS
    QSPDFTTJOHUJNF

    View Slide

  72. DPSF$16T)ZQFS5ISFBEJOH
    TFD





    TFSJBM QBSBMMFM
    w8IFOUFTURVFVF
    SVOTBMPUPGUFTU
    TVJUFT JUXJMMCF
    FGGFDUJWF
    wFHTFDTFD
    JO3VCP$PQSFQP

    View Slide

  73. % bundle exec rake spec
    Starting test-queue master (/tmp/test_queue_40616_5060.sock)
    ==> Summary (16 workers in 51.9441s)
    [ 1] 9 examples, 0 failures 1 suites in 34.0151s (pid 40649 exit 0 )
    [ 2] 109 examples, 0 failures 1 suites in 34.0151s (pid 40650 exit 0 )
    [ 3] 712 examples, 0 failures 46 suites in 34.0149s (pid 40651 exit 0 )
    [ 4] 2449 examples, 0 failures 61 suites in 34.0151s (pid 40652 exit 0 )
    [ 5] 129 examples, 0 failures 13 suites in 34.0151s (pid 40653 exit 0 )
    [ 6] 1628 examples, 0 failures 57 suites in 34.0150s (pid 40654 exit 0 )
    [ 7] 107 examples, 0 failures 1 suites in 34.0151s (pid 40655 exit 0 )
    [ 8] 1647 examples, 0 failures 61 suites in 34.0147s (pid 40656 exit 0 )
    [ 9] 2270 examples, 0 failures 59 suites in 34.0145s (pid 40657 exit 0 )
    [10] 1660 examples, 1 pending, 0 failures 57 suites in 34.0145s (pid 40658 exit 0 )
    [11] 132 examples, 0 failures 1 suites in 51.9309s (pid 40659 exit 0 )
    [12] 1600 examples, 0 failures 53 suites in 51.9306s (pid 40660 exit 0 )
    [13] 1924 examples, 0 failures 60 suites in 51.9295s (pid 40661 exit 0 )
    [14] 1993 examples, 0 failures 59 suites in 51.9281s (pid 40662 exit 0 )
    [15] 1967 examples, 0 failures 60 suites in 51.9271s (pid 40663 exit 0 )
    [16] 2303 examples, 0 failures 57 suites in 51.9237s (pid 40664 exit 0 )
    4MPXFTUBOEGBTUFTU
    4MPXFTU
    'BTUFTU

    View Slide

  74. )PXTIPVMEUIF
    SVOOJOHPGUFTUT
    B⒎FDUPOFBOPUIFS
    /PUBUBMM
    5&45%3*7&/%&7&-01.&/5#:&9".1-&

    l
    *TPMBUFE5FTUJTJNQPSUBOU

    View Slide

  75. w *TPMBUFUFTUTGSPNFBDIPUIFS
    w %BUBTUPSBHFBDDFTTFECZQBSBMMFM
    XPSLFSTTVDIBT3%#.4TIPVMECFTFU
    VQJOBQBSBMMFMNBOOFS
    w FH8PSLFS#SFBETUIFVQEBUFEEBUBPG
    XPSLFS"BOEUIFUFTUNBZGBJM
    4FUVQGPSQBSBMMFMUFTUJOH

    View Slide

  76. w 1SPEVDUDPEFBOEUFTUDPEFGPSNBUPUFTUJOHQBJS
    w 5IFVOJUPGQBSBMMFMUBTLTJOQBSBMMFMUFTUTJTVTVBMMZ
    UIFpMFMFWFM
    w 1BSBMMFMJ[JOHUFTUTVJUFTJOTNBMMFSVOJUTDBOSFEVDF
    UIFMPDLJOHQFSJPEGPSXPSLFST
    w *OPUIFSXPSET GPMMPXJOH431MFBETUPGBTUFS
    QBSBMMFMUFTUT
    1BSBMMFMUFTUBOE431
    4JOHMF3FTQPOTJCJMJUZ1SJODJQMF

    View Slide

  77. Parallel test runners
    Using RSpec
    Using Rails 6+ or AS
    Using Vanilla Ruby
    No
    Yes
    Yes
    No
    Yes
    No
    Start
    AS::
    TestCase
    parallel_tests
    or
    test-queue

    View Slide

  78. 8IJDIPOFJTCFUUFSUPVTF
    [email protected]

    View Slide

  79. w /VNCFSPGEPXOMPBETPOSVCZHFNTPSH
    w /VNCFSPGTUBSTPO(JU)VC
    w 'SFRVFODZPGNBJOUFOBODF
    w [email protected]
    XBTVONBJOUBJOFE
    w *OUFSFTUJOHEFTJHOCVUVOEFSVUJMJ[FE
    $SJUFSJBGPSHFNTFMFDUJPO

    View Slide

  80. 3JTFVQ

    View Slide

  81. 3FTVSSFDUJPOPGUFTURVFVF

    View Slide

  82. w *OFFEUPSFMFBTFBOFXWFSTJPO
    w !UNNBTLFENFUPCFBNBJOUBJOFSBOE
    *BDDFQUFE
    w .BJOUFOBODFJTSFTVNFEBGUFSPS
    ZFBST
    'JSTUUIJOHT*EJEBTBNBJOUBJOFS

    View Slide

  83. *GMFGUVOBUUFOEFE
    TPGUXBSFXJMMCSFBL
    CZ!ITCU
    XBTSFMFBTFE CVUTUJMM$*JTCSPLFO

    View Slide

  84. /P$* OPNFSHF
    $*XBTCSPLFO

    View Slide

  85. w *GPSHPUIPXUPVTF5SBWJT$*
    w *NOPUVQUPEBUFPOUIFSFDFOU
    EFWFMPQNFOUTPG5SBWJT$*
    w 8JUI(JU)VC"DUJPOT NBJOUFOBODFDBO
    CFEPOFXJUIPVUNF
    'SPN5SBWJT$*UP(JU)VC"DUJPOT

    View Slide

  86. w #BUT#BTI"VUPNBUFE5FTUJOH4ZTUFN
    w !TTUFQIFOTPOQSPEVDU
    w 5IFSFQPTJUPSZIBTCFFOBSDIJWFECZUIF
    PXOFSPO"QS 😢
    w *UTIBSEUPSFQMBDFBUFTUJOHGSBNFXPSL
    w *UTOFHBUJWF CVU*NMFBWJOHJUBTJUJT
    &&UFTUJOHPGUFTURVFVF

    View Slide

  87. w 5FTUOPUQBTTJOHXJUIMBUFTU
    WFSTJPOPG.JOJUFTU
    w 5IFUFTUDPEFBSPVOEUIF
    sleepNFUIPETFFNT
    TVTQJDJPVT
    w 'JYGBJMJOH(FNpMFGPS.JOJUFTU
    CZWFSTJPOQJOOJOH
    $VSSFOUTUBUVTPG&&UFTUJOH

    View Slide

  88. $*IBTSFDPWFSFE

    View Slide

  89. %FWFMPQNFOU
    FOWJSPONFOUJTSFBEZ
    *GUIF$*QBTTFT
    NBZCF*DBONFSHFJU

    View Slide

  90. #FZPOE
    UIF3FTVSSFDUJPO

    View Slide

  91. 4VQQPSUTUBUVTPGUFTUJOHGSBNFXPSLT
    5FTUJOH'SBNFXPSL $* /PUF
    34QFD ✅ 🤔5PPPME34QFDEPFTOUXPSLXJUI3VCZ
    34QFD ✅ ✅6TFEJO3VCP$PQ
    .JOJUFTU ✅ 🤔5PPPME
    .JOJUFTU ✅ CVU
    ✅6TFEJO3VCP$PQ.JOJUFTU
    $VDVNCFS ✅ CVU
    ❌0VUEBUFEUFTUDPEFFSSPSTJOUIFMBUFTUWFSTJPO
    UFTUVOJU ✅ 🤔/PUXPSLJOHXJUI3BJMT UNNUFTURVFVF

    34QFD EFW
    ✅ ✅6TFEJO3VCP$PQ
    5VSOJQ ✅ ✅/PUSBDLSFDPSEBTJUXBTKVTUNFSHFE

    View Slide

  92. w "QVMMSFRVFTUIBTCFFOPQFOFECZB34QFD
    DPSFUFBNNFNCFS
    w 34QFDJTBEFWFMPQNFOUTUBUVT #VU
    *EFDJEFEUPQSPWJEFBGFBUVSFPGUFTURVFVF
    w 050) JUTIBSEGPSUFTURVFVFNBJOUBJOFST
    UPLOPX34QFDTJOUFSOBMJNQMFNFOUBUJPO
    4VQQPSU34QFD EFW

    View Slide

  93. w 8PVMEOUJUCFDPOWFOJFOUJG34QFDIBEB
    QBSBMMFMUFTUSVOOFSGFBUVSFMJLF.JOJUFTU
    w 5IFTBNFDBOCFTBJEGPSUFTUVOJUBOE
    PUIFST
    *OUSPEVDFQBSBMMFMUFTUSVOOFSUP34QFD
    0QQPSUVOJUZGPSDPOUSJCVUJPO

    View Slide

  94. 0OFNPSFUIJOH

    View Slide

  95. w "MSFBEZJODMVEFTPuppetLintBTB
    QBSBMMFMSVOOFS
    w 8IJMFUIFQBSBMMFMHFNJTVTFEGPSQBSBMMFM
    FYFDVUJPOPG3VCP$PQ VTJOHB
    NFDIBOJTNMJLFUFTURVFVFNBZQSPWJEF
    FWFOGBTUFSSFTVMUT
    2VFVFTZTUFNOPUKVTUGPSUFTUJOH

    View Slide

  96. w 1SPHSBNNJOHGPSJOUFSQSPDFTT
    DPNNVOJDBUJPOJTBCBTJDGPVOEBUJPOGPS
    MFWFSBHJOHNVMUJDPSF
    'VOEBNFOUBMTPGQBSBMMFMQSPHSBNNJOH

    View Slide

  97. $PODMVTJPO

    View Slide

  98. w UFTURVFVFEFQFOETPOJOUFSOBM"1*PGFBDI
    UFTUJOHGSBNFXPSL
    w *XBOUJUUPCFUFNQPSBSZUSBOTJFOU
    w .BZOPUSFBDIUFTURVFVF
    w )BWJOHBQMVHHBCMFQBSBMMFMUFTUNFDIBOJTNMJLF
    .JOJUFTUDPVMECFDPOWFOJFOU FH34QFDUPP

    )PQFGPSQBSBMMFMUFTUSVOOFS

    View Slide

  99. w 1BSBMMFMUFTUJOHSFRVJSFTNPOFZ💰
    w %FWFMPQFSTBSFUIFNPTUWBMVBCMFSFTPVSDF
    JOUIFQSPKFDU
    w *UTFBTJFSUPHFUNPSFNFNPSZBOE$16T
    XJUINPOFZUIBOUPIJSFTFOJPSFOHJOFFST
    w %PZPVSCFTUUPTFDVSFUIFCVEHFU
    5IFSFJTBHPMECVMMFU

    View Slide

  100. 1BSBMMFM5FTUJOH
    .BLFT
    :PV)BQQZ
    (JU)VC!LPJD

    View Slide