約9000個の自動テストの時間を50分->10分に短縮 Flakyテストを1%以下に抑えた話

by hatsu

Slide 1

Slide 1 text

໿9000ݸͷࣗಈςετͷ ࣌ؒΛ50෼->10෼ʹ୹ॖ FlakyςετΛ1%ҎԼʹ཈͑ͨ࿩ Kaigi on Rails 2024 @hatsu_38 2024/10/26

Slide 2

Slide 2 text

εϥΠυ1ຕʹ͖ͭ6ඵͷϖʔεͰਐ· ͳ͍ͱؒʹ߹͍ͳ͍ܭࢉʹͳΓ·͠ ͨɻͦͷϖʔεͰਐΉ͜ͱྃ͝ঝͩ͘ ͍͞🙏

Slide 3

Slide 3 text

ࣗݾ঺հ • Twitter: @hatsu_38 • GitHub: @hatsu38 • SHE Inc. • Rubyྺ 5೥ = ΤϯδχΞྺ • React.js / TypeScript / GitHub Actions • Ruby͕Ұ൪޷͖

Slide 4

Slide 4 text

CIͰࣗಈςετ ࣮ߦ͍ͯ͠·͔͢ʁ🙋

Slide 5

Slide 5 text

ࣗಈςετ 10෼Ҏ্͔͔͍ͬͯ·͔͢ʁ🙋

Slide 6

Slide 6 text

Initial Commit͸2017೥ ฐࣾͷ͜ͱ🏢

Slide 7

Slide 7 text

ٕज़ελοΫ🤖 • Backend: Ruby on Rails, GraphQL, Sidekiq • Frontend: React.js, TypeScript, Next.js • CI: GitHub Actions • Infrastructure: Kubenetes ฐࣾͷ͜ͱ🏢 7

Slide 8

Slide 8 text

2023೥ͷࣗಈςετͷঢ়گ🔍 ฐࣾͷ͜ͱ - 2023೥ͷςετͷঢ়گ🔍

Slide 9

Slide 9 text

Model / Controller / Requests / System Spec, etc. ςετ਺ 7,803 case 2023.05.22 34QFD $BQZCBSBΛར༻ ฐࣾͷ͜ͱ - 2023೥ͷςετͷঢ়گ🔍 Code to Test Ratio: 1:4.3 Test Coverage: 89%

Slide 10

Slide 10 text

Model / Controller / Requests / System Spec, etc. ςετ࣌ؒ ฐࣾͷ͜ͱ - 2023೥ͷςετͷঢ়گ🔍 $*͕௨Δͷʹ෼Ҏ্🫠

Slide 11

Slide 11 text

Flakyͳςετ΋୔ࢁ💥 ฐࣾͷ͜ͱ - 2023೥ͷςετͷঢ়گ🔍 ࣮ߦ݁Ռ͕ ෆ҆ఆͳςετ

Slide 12

Slide 12 text

ཧ૝ͷCIͷ࣌ؒ͸10෼Ҏ಺ɺ Έ͍ͨͳจষΛҾ༻͢Δ ฐࣾͷ͜ͱ - 2023೥ͷςετͷঢ়گ🔍

Slide 13

Slide 13 text

ཧ૝ͷCIͷ࣌ؒ͸10෼Ҏ಺ɺ Έ͍ͨͳจষΛҾ༻͢Δ $*ͷεϐʔυ͸େ੾Ͱ͢ɻ ཧ૝͸෼Ҏ಺ɺ஗͘ͱ΋෼Ҏ಺ʹ׬ྃͤ͞·͠ΐ͏ɻ ฐࣾͷ͜ͱ - 2023೥ͷςετͷঢ়گ🔍

Slide 14

Slide 14 text

ཧ૝ͷCIͷ࣌ؒ͸10෼Ҏ಺ɺ Έ͍ͨͳจষΛҾ༻͢Δ $*ͷεϐʔυ͸େ੾Ͱ͢ɻ ཧ૝͸෼Ҏ಺ɺ஗͘ͱ΋෼Ҏ಺ʹ׬ྃͤ͞·͠ΐ͏ɻ ϑϨʔΩʔςετͷ਺͕͖͍͠஋Λ௒͑Δͱɺ ςετશମ͕৴པ͞Εͳ͘ͳΔͷͰ͢ɻͦͷ͖͍͠஋͸Ͱ͢ɻ ฐࣾͷ͜ͱ - 2023೥ͷςετͷঢ়گ🔍

Slide 15

Slide 15 text

վળ͠Α͏🛠⚙ ฐࣾͷ͜ͱ - 2023೥ͷςετͷঢ়گ🔍

Slide 16

Slide 16 text

೥ ೥ ςετ਺ DBTF $*ͷ࣌ؒ ෼ ฒྻ$PSF 'MBLZςετ ݸ ͘Β͍ͷ֬཰Ͱ$*͕མͪΔ (JU)VC"DUJPOTͷ஋ஈ ฐࣾͷ͜ͱ - 2023೥ͷςετͷঢ়گ🔍 2023೥ͷςετͷঢ়گ🔍

Slide 17

Slide 17 text

͜Ε͔Βɺ΍ͬͨ͜ͱΛ ͨ͘͞Μ঺հ͍͖ͯ͠·͢💡

Slide 18

Slide 18 text

ࣗࣾͰ΋࢖͑ͦ͏ͳ5JQT ΍ ಺෦͸͜Μͳ࣮૷Ͱಈ͍͍ͯΔΜͩ ͬͯൃݟ͕1ͭ͋Ε͹ྑ͍ͳͱ💯💡

Slide 19

Slide 19 text

ͭͷ՝୊💥 • ςετͷ͕࣌ؒ௕͍🐢 • ςετલͷ#VJMEͷ͕࣌ؒ௕͍🥱 ʢগ͠3VCZ͔Β୤ઢʣ • Flakyͳςετ͕ଟ͍💫

Slide 20

Slide 20 text

ͭͷ՝୊💥 • ςετͷ͕࣌ؒ௕͍🐢 • ςετલͷ#VJMEͷ͕࣌ؒ௕͍🥱 ʢগ͠3VCZ͔Β୤ઢʣ • Flakyͳςετ͕ଟ͍💫

Slide 21

Slide 21 text

• Capybaraͷػೳʹ೚ͤͯSleepΛ࡟আ • LoadingΛಈతʹ଴ͭHelper࡞ͬͯSleepΛ࡟আ • test-profͷbefore_allΛ࢖ͬͯσʔλ࡞੒Λলུ • test-profͷlet_it_beΛ࢖ͬͯσʔλ࡞੒Λলུ • parallel_testsͷฒྻ࣮ߦΛ׆͔ͨ͢ΊʹϑΝΠϧ෼ׂ • ͦͷଞɺࡉ͔ͳςετ࣌ؒ୹ॖͷͨΊͷࢪࡦ ςετͷ͕࣌ؒ௕͍🐢 21 ςετͷ͕࣌ؒ௕͍🐢

Slide 22

Slide 22 text

Slide 23

Slide 23 text

Sleep͕59Օॴ͋ͬͨ ςετͷ͕࣌ؒ௕͍🐢 - Capybaraͷػೳʹ೚ͤͯSleepΛ࡟আ

Slide 24

Slide 24 text

Sleep͕࢖ΘΕ͍ͯΔྫ ςετͷ͕࣌ؒ௕͍🐢 - Capybaraͷػೳʹ೚ͤͯSleepΛ࡟আ ϑΥʔϜೖྗͯ͠ɺอଘ͢ΔϘλϯԡͨ͠Βɺ

Slide 25

Slide 25 text

Sleep͕࢖ΘΕ͍ͯΔྫ ςετͷ͕࣌ؒ௕͍🐢 - Capybaraͷػೳʹ೚ͤͯSleepΛ࡟আ

Slide 26

Slide 26 text

have_text(“hoge”)ͰԿ͕ى͖͍ͯΔ͔ʁ ςετͷ͕࣌ؒ௕͍🐢 - Capybaraͷػೳʹ೚ͤͯSleepΛ࡟আ

Slide 27

Slide 27 text

have_text(“hoge”)ͰԿ͕ى͖͍ͯΔ͔ʁ ςετͷ͕࣌ؒ௕͍🐢 - Capybaraͷػೳʹ೚ͤͯSleepΛ࡟আ

Slide 28

Slide 28 text

have_text(“hoge”)ͰԿ͕ى͖͍ͯΔ͔ʁ ςετͷ͕࣌ؒ௕͍🐢 - Capybaraͷػೳʹ೚ͤͯSleepΛ࡟আ EFGBVMU@NBY@XBJU@UJNF σϑΥϧτ ඵؒ୳͠ଓ͚Δඵ਺ͷઃఆ ͜ͷඵ਺ͷؒɺςΩετzIPQFzΛ୳ͯ͘͠ΕΔ

Slide 29

Slide 29 text

have_text(“hoge”)ͰԿ͕ى͖͍ͯΔ͔ʁ ςετͷ͕࣌ؒ௕͍🐢 - Capybaraͷػೳʹ೚ͤͯSleepΛ࡟আ EFGBVMU@SFUSZ@JOUFSWBM σϑΥϧτ ඵςΩετzIPHFzΛ܁Γฦ͠୳ִؒ͢ EFGBVMU@NBY@XBJU@UJNF σϑΥϧτ ඵؒ୳͠ଓ͚Δඵ਺ͷઃఆ ͜ͷඵ਺ͷؒɺςΩετzIPQFzΛ୳ͯ͘͠ΕΔ

Slide 30

Slide 30 text

have_text(“hoge”)ͰԿ͕ى͖͍ͯΔ͔ʁ ςετͷ͕࣌ؒ௕͍🐢 - Capybaraͷػೳʹ೚ͤͯSleepΛ࡟আ ͜Ε͕retryͰ܁Γฦ͠ಈ͘ EFGBVMU@NBY@XBJU@UJNF σϑΥϧτ ඵؒ୳͠ଓ͚Δඵ਺ͷઃఆ ͜ͷඵ਺ͷؒɺςΩετzIPQFzΛ୳ͯ͘͠ΕΔ EFGBVMU@SFUSZ@JOUFSWBM σϑΥϧτ ඵςΩετzIPHFzΛ܁Γฦ͠୳ִؒ͢

Slide 31

Slide 31 text

Sleep͕59 30Օॴʹݮͬͨ✨ Capybaraͷػೳʹ೚ͤͯSleepΛ࡟আ

Slide 32

Slide 32 text

Capybaraͷػೳʹ೚ͤͯ SleepΛ࡟আ ςετͷ͕࣌ؒ௕͍🐢 - Capybaraͷػೳʹ೚ͤͯSleepΛ࡟আ खܰ͞: ˒˒˒˒˒ ޮՌ: ˒˒˒ˑˑ

Slide 33

Slide 33 text

• Capybaraͷػೳʹ೚ͤͯSleepΛ࡟আ • LoadingΛಈతʹ଴ͭHelperΛ࡞ͬͯSleepΛ࡟আ • test-profͷbefore_allΛ࢖ͬͯσʔλ࡞੒Λলུ • test-profͷlet_it_beΛ࢖ͬͯσʔλ࡞੒Λলུ • parallel_testsͷฒྻ࣮ߦΛ׆͔ͨ͢ΊʹϑΝΠϧ෼ׂ • ͦͷଞɺࡉ͔ͳςετ࣌ؒ୹ॖͷͨΊͷࢪࡦ ςετͷ͕࣌ؒ௕͍🐢 33 ςετͷ͕࣌ؒ௕͍🐢

Slide 34

Slide 34 text

Sleep͕30Օॴ͋ͬͨ ςετͷ͕࣌ؒ௕͍🐢 - LoadingΛಈతʹ଴ͭHelper࡞ͬͯSleepΛ࡟আ

Slide 35

Slide 35 text

Sleep͕࢖ΘΕ͍ͯΔྫ ςετͷ͕࣌ؒ௕͍🐢 - LoadingΛಈతʹ଴ͭHelper࡞ͬͯSleepΛ࡟আ > default_max_wait_time

Slide 36

Slide 36 text

LoadingΞΠίϯ͕ফ͑ΔͷΛ଴ͭHelperΛ࡞੒ ςετͷ͕࣌ؒ௕͍🐢 - LoadingΛಈతʹ଴ͭHelper࡞ͬͯSleepΛ࡟আ ಈతʹ-PBEJOHΞΠίϯ͕ফ͔͑ͨͲ͏͔ ܁Γฦ֬͠ೝ͢Δ

Slide 37

Slide 37 text

LoadingΞΠίϯ͕ফ͑ΔͷΛ଴ͭHelperΛ࡞੒ ςετͷ͕࣌ؒ௕͍🐢 - LoadingΛಈతʹ଴ͭHelper࡞ͬͯSleepΛ࡟আ ಈతʹ-PBEJOHΞΠίϯ͕ফ͔͑ͨͲ͏͔ ܁Γฦ֬͠ೝ͢Δ ͜͜ʹHTMLͷSelectorΛࢦఆ

Slide 38

Slide 38 text

LoadingΞΠίϯ͕ফ͑ΔͷΛ଴ͭHelperΛར༻ ςετͷ͕࣌ؒ௕͍🐢 - LoadingΛಈతʹ଴ͭHelper࡞ͬͯSleepΛ࡟আ $BQZCBSBͷEFGBVMU@NBY@XBJU@UJNFͰ͸-PBEJOH͕ ऴΘΒͳ͍৔߹ʹར༻

Slide 39

Slide 39 text

Sleep͕59 30 13Օॴʹݮͬͨ✨ ςετͷ͕࣌ؒ௕͍🐢 - LoadingΛಈతʹ଴ͭHelper࡞ͬͯSleepΛ࡟আ

Slide 40

Slide 40 text

LoadingΛಈతʹ଴ͭHelperΛ ࡞ͬͯSleepΛ࡟আ ςετͷ͕࣌ؒ௕͍🐢 - LoadingΛಈతʹ଴ͭHelper࡞ͬͯSleepΛ࡟আ खܰ͞: ˒˒˒ˑˑ ޮՌ: ˒˒˒˒ˑ

Slide 41

Slide 41 text

• Capybaraͷػೳʹ೚ͤͯSleep࡟আ • LoadingΛಈతʹ଴ͭHelperΛ࡞ͬͯSleep࡟আ • test-profͷbefore_allΛ࢖ͬͯσʔλ࡞੒Λলུ • test-profͷlet_it_beΛ࢖ͬͯσʔλ࡞੒Λলུ • parallel_testsͷฒྻ࣮ߦΛ׆͔ͨ͢ΊʹϑΝΠϧ෼ׂ • ͦͷଞɺࡉ͔ͳςετ࣌ؒ୹ॖͷͨΊͷࢪࡦ ςετͷ͕࣌ؒ௕͍🐢 41 ςετͷ͕࣌ؒ௕͍🐢

Slide 42

Slide 42 text

σʔλ࡞੒͕25,550݅ - 35݅ͷςετ ςετͷ͕࣌ؒ௕͍🐢 - test-profͷbefore_allΛ࢖ͬͯσʔλ࡞੒Λলུ →ςετ࣌ؒ1m24sͷ͏ͪ1m4s͕σʔλੜ੒ test-prof gemͰௐࠪ

Slide 43

Slide 43 text

σʔλ࡞੒͕25,550݅ - 35݅ͷςετ ςετͷ͕࣌ؒ௕͍🐢 - test-profͷbefore_allΛ࢖ͬͯσʔλ࡞੒Λলུ → Seedσʔλͷ࡞੒Λߦ͏εΫϦϓτ 1౓Ͱ675݅ͷσʔλΛ࡞੒͢Δ JSC NBJO

Slide 44

Slide 44 text

Seedσʔλ࡞੒εΫϦϓτ ςετͷ͕࣌ؒ௕͍🐢 - test-profͷbefore_allΛ࢖ͬͯσʔλ࡞੒Λলུ ଞͷςετͰ΋࢖͍·Θͤͦ͏💡

Slide 45

Slide 45 text

σʔλ࡞੒͕25,550݅ - 35݅ͷςετ ςετͷ͕࣌ؒ௕͍🐢 - test-profͷbefore_allΛ࢖ͬͯσʔλ࡞੒Λলུ before_allͰSeedσʔλΛ࢖͍ճ͢

Slide 46

Slide 46 text

before_all { Fixtures.setup }ͰԿ͕ى͖͍ͯΔ͔ʁ ςετͷ͕࣌ؒ௕͍🐢 - test-profͷbefore_allΛ࢖ͬͯσʔλ࡞੒Λলུ

Slide 47

Slide 47 text

before_all { Fixtures.setup }ͰԿ͕ى͖͍ͯΔ͔ʁ ςετͷ͕࣌ؒ௕͍🐢 - test-profͷbefore_allΛ࢖ͬͯσʔλ࡞੒Λলུ ςετάϧʔϓશମʹରͯ͠

Slide 48

Slide 48 text

before_all { Fixtures.setup }ͰԿ͕ى͖͍ͯΔ͔ʁ ςετͷ͕࣌ؒ௕͍🐢 - test-profͷbefore_allΛ࢖ͬͯσʔλ࡞੒Λলུ ςετάϧʔϓશମʹରͯ͠ τϥϯβΫγϣϯΛுͬͯ ͦͷதͰblockΛݺͼग़͢

Slide 49

Slide 49 text

before_all { Fixtures.setup }ͰԿ͕ى͖͍ͯΔ͔ʁ ςετͷ͕࣌ؒ௕͍🐢 - test-profͷbefore_allΛ࢖ͬͯσʔλ࡞੒Λলུ ςετάϧʔϓશମʹରͯ͠ τϥϯβΫγϣϯΛுͬͯ ͦͷதͰblockΛݺͼग़͢ ςετάϧʔϓͷςετ͕ऴΘͬͨΒ Rollback͢Δ

Slide 50

Slide 50 text

σʔλ࡞੒͕25,550݅ ςετͷ͕࣌ؒ௕͍🐢 - test-profͷbefore_allΛ࢖ͬͯσʔλ࡞੒Λলུ →ςετ࣌ؒ1m24sͷ͏ͪ1m4s͕σʔλੜ੒(ςετέʔε͸35έʔε) test-prof gemͰௐࠪ

Slide 51

Slide 51 text

σʔλ࡞੒͕25,550 2,600݅ʹݮͬͨ✨ ςετͷ͕࣌ؒ௕͍🐢 - test-profͷbefore_allΛ࢖ͬͯσʔλ࡞੒Λলུ ςετ࣌ؒ1m24s->40sʹ୹ॖ test-prof gemͰௐࠪ

Slide 52

Slide 52 text

test-profͷbefore_allΛ࢖ͬͯ σʔλ࡞੒Λলུ ςετͷ͕࣌ؒ௕͍🐢 - test-profͷbefore_allΛ࢖ͬͯσʔλ࡞੒Λলུ खܰ͞: ˒˒˒˒ˑ ޮՌ: ˒˒˒˒˒

Slide 53

Slide 53 text

Slide 54

Slide 54 text

σʔλ࡞੒͕37,530݅ - 675݅ͷςετ ςετͷ͕࣌ؒ௕͍🐢 - test-profͷlet_it_beΛ࢖ͬͯσʔλ࡞੒Λলུ →ςετ࣌ؒ2m50sͷ͏ͪ2m1s͕σʔλੜ੒ test-prof gemͰௐࠪ

Slide 55

Slide 55 text

σʔλ࡞੒͕37,530݅ - 675݅ͷςετ ྫ ςετͷ͕࣌ؒ௕͍🐢 - test-profͷlet_it_beΛ࢖ͬͯσʔλ࡞੒Λলུ

Slide 56

Slide 56 text

σʔλ࡞੒͕37,530݅ - 675݅ͷςετ ྫ ςετͷ͕࣌ؒ௕͍🐢 - test-profͷlet_it_beΛ࢖ͬͯσʔλ࡞੒Λলུ →ςετͷͨͼʹݺ͹ΕΔ

Slide 57

Slide 57 text

σʔλ࡞੒͕37,530݅ - 675݅ͷςετ ྫ ςετͷ͕࣌ؒ௕͍🐢 - test-profͷlet_it_beΛ࢖ͬͯσʔλ࡞੒Λলུ ͕ɺຖ౓࡞Γ௚͞ͳͯ͘΋ಉ͡ΠϯελϯεΛ࢖͍·ͤΔม਺💡 →ςετͷͨͼʹݺ͹ΕΔ

Slide 58

Slide 58 text

letΛଞͷςετͰ΋࢖͍ճ͢ ςετͷ͕࣌ؒ௕͍🐢 - test-profͷlet_it_beΛ࢖ͬͯσʔλ࡞੒Λলུ

Slide 59

Slide 59 text

let_it_be(:admin) { create(:user) }ͰԿ͕ى͖͍ͯΔ͔ʁ ςετͷ͕࣌ؒ௕͍🐢 - test-profͷlet_it_beΛ࢖ͬͯσʔλ࡞੒Λলུ

Slide 60

Slide 60 text

let_it_be(:admin) { create(:user) }ͰԿ͕ى͖͍ͯΔ͔ʁ ςετͷ͕࣌ؒ௕͍🐢 - test-profͷlet_it_beΛ࢖ͬͯσʔλ࡞੒Λলུ proc { @😸admin = create(:user) }

Slide 61

Slide 61 text

let_it_be(:admin) { create(:user) }ͰԿ͕ى͖͍ͯΔ͔ʁ ςετͷ͕࣌ؒ௕͍🐢 - test-profͷlet_it_beΛ࢖ͬͯσʔλ࡞੒Λলུ proc { @😸admin = create(:user) } before_all { @😸admin = create(:user) }

Slide 62

Slide 62 text

let_it_be(:admin) { create(:user) }ͰԿ͕ى͖͍ͯΔ͔ʁ ςετͷ͕࣌ؒ௕͍🐢 - test-profͷlet_it_beΛ࢖ͬͯσʔλ࡞੒Λলུ proc { @😸admin = create(:user) } before_all { @😸admin = create(:user) } let(:admin) { @😸admin } (`@😸admin` ͸before_allͰ࡞੒ࡁΈ)

Slide 63

Slide 63 text

σʔλ࡞੒͕37,530݅ ςετͷ͕࣌ؒ௕͍🐢 - test-profͷlet_it_beΛ࢖ͬͯσʔλ࡞੒Λলུ →ςετ࣌ؒ2m50sͷ͏ͪ2m1s͕σʔλੜ੒ test-prof gemͰௐࠪ

Slide 64

Slide 64 text

σʔλ࡞੒͕37,530 4,970݅ʹݮͬͨ✨ ςετͷ͕࣌ؒ௕͍🐢 - test-profͷlet_it_beΛ࢖ͬͯσʔλ࡞੒Λলུ ςετ࣌ؒ2m50s->37s test-prof gemͰௐࠪ

Slide 65

Slide 65 text

test-profͷlet_it_beΛ࢖ͬͯ σʔλ࡞੒Λলུ ςετͷ͕࣌ؒ௕͍🐢 - test-profͷlet_it_beΛ࢖ͬͯσʔλ࡞੒Λলུ खܰ͞: ˒˒˒˒ˑ ޮՌ: ˒˒˒˒ˑ

Slide 66

Slide 66 text

• Capybaraͷػೳʹ೚ͤͯSleepΛ࡟আ • LoadingΛಈతʹ଴ͭHelperΛ࡞ͬͯSleepΛ࡟আ • test-profͷbefore_allΛ࢖ͬͯσʔλ࡞੒Λলུ • test-profͷlet_it_beΛ࢖ͬͯσʔλ࡞੒Λলུ • parallel_testsͷฒྻ࣮ߦΛ׆͔ͨ͢ΊʹϑΝΠϧ෼ׂ • ͦͷଞɺࡉ͔ͳ͋Ε͜Ε ςετͷ͕࣌ؒ௕͍🐢 66 ςετͷ͕࣌ؒ௕͍🐢

Slide 67

Slide 67 text

parallel_testsͱ͸ʁ ςετͷ͕࣌ؒ௕͍🐢 - parallel_testsͷฒྻ࣮ߦΛ׆͔ͨ͢ΊʹϑΝΠϧ෼ׂ $16@ .ݸͷ ςετϑΝΠϧ $16@ /ݸͷ ςετϑΝΠϧ ɾɾɾ $16@9 ,ݸͷ ςετϑΝΠϧ CPUݸ਺෼ʹςετϑΝΠϧΛ෼ׂͯ͠ɺ ςετΛฒྻʹ࣮ߦͤ͞Δ💡

Slide 68

Slide 68 text

ޮ཰ͷѱ͍෼ׂ - 1,000s ͔͔Δ ςετͷ͕࣌ؒ௕͍🐢 - parallel_testsͷฒྻ࣮ߦΛ׆͔ͨ͢ΊʹϑΝΠϧ෼ׂ user_spec.rb ͕ංେԽ

Slide 69

Slide 69 text

ޮ཰ͷѱ͍෼ׂ - 1,000s ͔͔Δ ςετͷ͕࣌ؒ௕͍🐢 - parallel_testsͷฒྻ࣮ߦΛ׆͔ͨ͢ΊʹϑΝΠϧ෼ׂ $16T $16 T

Slide 70

Slide 70 text

ޮ཰ͷྑ͍෼ׂ ςετͷ͕࣌ؒ௕͍🐢 - parallel_testsͷฒྻ࣮ߦΛ׆͔ͨ͢ΊʹϑΝΠϧ෼ׂ T͔͔ΔςετϑΝΠϧΛͭʹ෼ׂ $16 T

Slide 71

Slide 71 text

ޮ཰ͷྑ͍෼ׂ - 600sͰऴΘΔ✨ ςετͷ͕࣌ؒ௕͍🐢 - parallel_testsͷฒྻ࣮ߦΛ׆͔ͨ͢ΊʹϑΝΠϧ෼ׂ $16T $16T

Slide 72

Slide 72 text

ޮ཰ͷྑ͍෼ׂ - ࣮ྫ parallel_testsͷฒྻ࣮ߦΛ׆͔ͨ͢ΊʹϑΝΠϧ෼ׂ Before After

Slide 73

Slide 73 text

parallel_testsͰԿ͕ى͖ͯΔʁ(ྫ: fi le size sort) 1. CPUίΞ਺Λऔಘͯ͠ 2. ςετର৅ͷϑΝΠϧͷϑΝΠϧαΠζ(byte)Λऔಘͯ͠഑ྻΛ࡞੒ 3. ϑΝΠϧαΠζͷେ͖͍ॱʹฒͼସ͑ͯ 4. CPU਺ݸͷάϧʔϓΛ࡞ͬͯ 5. ϑΝΠϧαΠζͷେ͖͍ॱʹɺ߹ܭϑΝΠϧαΠζͷ࠷΋খ͍͞ά ϧʔϓʹɺϑΝΠϧΛ௥ՃΛ܁Γฦͯ͠άϧʔϓ࡞੒ 6. ֤$16ͰςετΛ࣮ߦ ςετͷ͕࣌ؒ௕͍🐢 - parallel_testsͷฒྻ࣮ߦΛ׆͔ͨ͢ΊʹϑΝΠϧ෼ׂ 73

Slide 74

Slide 74 text

parallel_testsͰԿ͕ى͖ͯΔʁ(ྫ: fi le size sort) 1. CPUίΞ਺Λऔಘͯ͠ 2. ςετର৅ͷϑΝΠϧͷϑΝΠϧαΠζ(byte)Λऔಘ 3. ϑΝΠϧαΠζͷେ͖͍ॱʹฒͼସ͑ͯ 4. CPU਺ݸͷάϧʔϓΛ࡞ͬͯ 5. ϑΝΠϧαΠζͷେ͖͍ॱʹɺ߹ܭϑΝΠϧαΠζͷ࠷΋খ͍͞ά ϧʔϓʹɺϑΝΠϧΛ௥ՃΛ܁Γฦͯ͠άϧʔϓ࡞੒ 6. ֤$16ͰςετΛ࣮ߦ ςετͷ͕࣌ؒ௕͍🐢 - parallel_testsͷฒྻ࣮ߦΛ׆͔ͨ͢ΊʹϑΝΠϧ෼ׂ 74

Slide 75

Slide 75 text

Slide 76

Slide 76 text

Slide 77

Slide 77 text

Slide 78

Slide 78 text

Slide 79

Slide 79 text

Slide 80

Slide 80 text

parallel_testsͷฒྻ࣮ߦΛ׆͔͢ ͨΊʹϑΝΠϧ෼ׂ ςετͷ͕࣌ؒ௕͍🐢 - parallel_testsͷฒྻ࣮ߦΛ׆͔ͨ͢ΊʹϑΝΠϧ෼ׂ खܰ͞: ˒˒˒˒ˑ ޮՌ: ˒˒˒˒ˑ

Slide 81

Slide 81 text

Slide 82

Slide 82 text

ͦͷଞɺࡉ͔ͳςετ࣌ؒ୹ॖͷͨΊͷࢪࡦ • FactoryBot.create() Ͱ͸ͳ͘ FactoryBot.build_stubbed()Λ࢖͏ • ෼͚ͳ͍͍ͯ͘ςετ͸ɺ1ͭͷ it do ~ endʹ·ͱΊΔ • ͍Βͳ͍ػೳ΍ςετ͸࡟আ • LoadingΛ଴ͭॲཧ΋ՄೳͳΒ࡟আ • KnapsackPro gem΋ར༻ • ஗͍ςετϑΝΠϧ,஗͍ςετ, աڈࣦഊͨ͠ςετΛμογϡϘʔυԽ ςετͷ͕࣌ؒ௕͍🐢 - ͦͷଞɺࡉ͔ͳςετ࣌ؒ୹ॖͷͨΊͷࢪࡦ 82

Slide 83

Slide 83 text

Slide 84

Slide 84 text

Slide 85

Slide 85 text

Slide 86

Slide 86 text

Slide 87

Slide 87 text

Slide 88

Slide 88 text

Slide 89

Slide 89 text

CIͷ͕࣌ؒ54෼->23෼✨ ςετͷ͕࣌ؒ௕͍🐢 5FTU NT #VJME NT 3FQPSU T

Slide 90

Slide 90 text

લ൒ઓऴྃ🍻 ʢͨͿΜ12෼͘Β͍ܦաதʣ

Slide 91

Slide 91 text

CIͷ͕࣌ؒ54෼->23෼✨ ςετͷ͕࣌ؒ௕͍🐢 5FTU NT #VJME NT 3FQPSU T

Slide 92

Slide 92 text

CIͷ͕࣌ؒ54෼->23෼✨ ςετͷ͕࣌ؒ௕͍🐢 5FTU NT #VJME NT 3FQPSU T 23෼🤔

Slide 93

Slide 93 text

ͭͷ՝୊💥 • ςετͷ͕࣌ؒ௕͍🐢 • ςετલͷ#VJMEͷ͕࣌ؒ௕͍🥱 ʢগ͠3VCZ͔Β୤ઢʣ • Flakyͳςετ͕ଟ͍💫

Slide 94

Slide 94 text

• Next.js × TypeScript ͰϑϩϯτΤϯυ͸ಈ͍͍ͯΔ • Adminը໘ͱUserը໘ͦΕͧΕ͋ΔɻϦϙδτϦ͸1ͭɻ • ຖճ#VJME͚ͩͰ10෼Ҏ্͔͔͍ͬͯΔ • ฒྻ࣮ߦͱ͔͕ҙຯͳ͍ɻ͓ۚͰԥΕͳ͍ɻ • ςετલʹ#VJME͢Δඞཁ͕͋ΔͷͰલςετ͕͜ͷ࣮ߦ࣌ؒʹґଘ ͍ͯ͠Δ ςετલͷ#VJMEͷ͕࣌ؒ௕͍🥱 94 ςετલͷBuild

Slide 95

Slide 95 text

Next.jsެࣜαΠτΑΓ📗 ςετલͷ#VJMEͷ͕࣌ؒ௕͍🥱

Slide 96

Slide 96 text

Next.jsެࣜαΠτΑΓ📗 ςετલͷ#VJMEͷ͕࣌ؒ௕͍🥱 ϏϧυͷύϑΥʔϚϯεΛ޲্ͤ͞ΔͨΊʹɺ /FYUKT͸ϏϧυؒͰڞ༗͞ΕΔΩϟογϡΛOFYUDBDIFʹอଘ͠·͢ɻ ܧଓతΠϯςάϨʔγϣϯ $* ؀ڥͰ͜ͷΩϟογϡΛ׆༻͢Δʹ͸ɺ ϏϧυؒͰΩϟογϡΛਖ਼͘͠อ࣋͢ΔΑ͏ʹɺ$*ϫʔΫϑϩʔΛߏ੒͢Δඞཁ ͕͋Γ·͢ɻ

Slide 97

Slide 97 text

Slide 98

Slide 98 text

Slide 99

Slide 99 text

Build͕࣌ؒ10෼->34ඵ✨ ςετલͷ#VJMEͷ͕࣌ؒ௕͍🥱 Ϣʔβʔը໘ "ENJOը໘ ZBSOMPDLʹมߋ͕͋Δ৔߹ ෼ඵ ෼ඵ ZBSOMPDLʹมߋͳ͍͕ɺ ϑϩϯτͷίʔυʹมߋ͕͋Δ৔߹ ෼ඵ ෼ඵ Ұ੾มߋ͕ͳ͍৔߹ ඵ✨ ඵ✨

Slide 100

Slide 100 text

CIͷ͕࣌ؒ54෼->23 10෼✨ ςετલͷ#VJMEͷ͕࣌ؒ௕͍🥱 5FTU NT #VJME T 3FQPSU T

Slide 101

Slide 101 text

தฤऴྃ🍻 ʢͨͿΜ13෼͘Β͍ܦաதʣ

Slide 102

Slide 102 text

ͭͷ՝୊💥 • ςετͷ͕࣌ؒ௕͍🐢 • ςετલͷ#VJMEͷ͕࣌ؒ௕͍🥱 ʢগ͠3VCZ͔Β୤ઢʣ • Flakyͳςετ͕ଟ͍💫

Slide 103

Slide 103 text

Flakyͳςετͱ͸ʁ • ࣮ߦ݁Ռ͕ෆ҆ఆͳςετͷ͜ͱ • Flakyͳςετ͕গͳ͘ͱ΋30ݸҎ্͋ͬͨ • 5%ͷ֬཰ͰམͪΔෆ҆ఆͳςετ͕30ݸ͋Δͱɺςετ͕શͯύε ͢Δ֬཰͸21.5% Flakyͳςετ͕ଟ͍💫 103

Slide 104

Slide 104 text

Flakyͳςετ͕ଟ͍💫 • ࣮ߦ݁Ռ͕ෆ҆ఆͳςετͷ͜ͱ • Flakyͳςετ͕গͳ͘ͱ΋30ݸҎ্͋ͬͨ • 5%ͷ֬཰ͰམͪΔෆ҆ఆͳςετ͕30ݸ͋Δͱɺςετ͕શͯύε ͢Δ֬཰͸21.5% Flakyͳςετ͕ଟ͍💫 104 ɾࣦഊ཰͸ɺۉҰͰ͸ͳ͍ ɾࣦഊ཰͕௿͍΋ͷ͸ɺӅΕͯฆΕͯͨΓ͢Δ ɾࣦഊ཰͕ߴ͍ςετ͔Β௚͍ͯ͘͠ͱྑ͍

Slide 105

Slide 105 text

Flakyͳςετ΁ͷ޲͖߹͍ํ Flakyͳςετ͕ଟ͍💫 5FTU NT #VJME T 3FQPSU T ஍ಓʹ௚͢💪

Slide 106

Slide 106 text

஍ಓʹ௚͢ࡍͷ େมͩͬͨ͜ͱɺ޻෉ͨ͜͠ͱ🛠🤖 Flakyͳςετ͕ଟ͍💫

Slide 107

Slide 107 text

஍ಓʹ௚͢ࡍʹେมͩͬͨ͜ͱ • CIͰςετ͕Failͨ͠ͱ͖ɺϩά΍εΫγϣͷ֬ೝ͕໘౗ • CI্Ͱ͔͠ൃੜ͠ͳ͍FlakyͷݪҼ͕ෆ໌ • FlakyΛθϩʹ͸͖͠Εͳ͍ Flakyͳςετ͕ଟ͍💫 107

Slide 108

Slide 108 text

஍ಓʹ௚͢ࡍʹେมͩͬͨ͜ͱ • CIͰςετ͕Failͨ͠ͱ͖ɺϩά΍εΫγϣͷ֬ೝ͕໘౗ • CI্Ͱ͔͠ൃੜ͠ͳ͍FlakyͷݪҼ͕ෆ໌ • FlakyΛθϩʹ͸͖͠Εͳ͍ Flakyͳςετ͕ଟ͍💫 108

Slide 109

Slide 109 text

E2E͕CI্ͰFailͨ͠Β…ϩάΛ֬ೝ Flakyͳςετ͕ଟ͍💫 - CIͰςετ͕Failͨ͠ͱ͖ɺϩά΍εΫγϣͷ֬ೝ͕໘౗ ໘౗😮💨

Slide 110

Slide 110 text

E2E͕CI্ͰFailͨ͠Β…εΫϦʔϯγϣοτΛ֬ೝ Flakyͳςετ͕ଟ͍💫 - CIͰςετ͕Failͨ͠ͱ͖ɺϩά΍εΫγϣͷ֬ೝ͕໘౗ ໘౗😮💨

Slide 111

Slide 111 text

Allure ReportΛಋೖ Flakyͳςετ͕ଟ͍💫 - CIͰςετ͕Failͨ͠ͱ͖ɺϩά΍εΫγϣͷ֬ೝ͕໘౗

Slide 112

Slide 112 text

• ςετ݁ՌΛྑ͍ײ͡ʹϨϙʔτ͢Δ΍ͭ • ςετͷ਺ • ΧόϨοδ • ςετ࣌ؒ • ςετ݁Ռཤྺ • εΫγϣͱ͔΋ඥ෇͚ΒΕΔ Flakyͳςετ͕ଟ͍💫 - CIͰςετ͕Failͨ͠ͱ͖ɺϩά΍εΫγϣͷ֬ೝ͕໘౗ 112 Allure Reportͱ͸ʁ

Slide 113

Slide 113 text

• bundle exec rspec --format AllureRspecFormatter • ςετͷ৘ใΛKTPOʹग़ྗ • allure generate allure_results/* -o tmp/allure_report • ςετ݁ՌΛ)5.-ʹม׵ • ϓϧϦΫΤετຖʹςετ݁Ռͷ)5.-Λ7FSDFMʹσϓϩΠ • ςετࣦഊ࣌ʹ͸7FSDFMͷ63-Λ#PU͕ίϝϯτͰڭ͑ͯ͘ΕΔ Flakyͳςετ͕ଟ͍💫 - CIͰςετ͕Failͨ͠ͱ͖ɺϩά΍εΫγϣͷ֬ೝ͕໘౗ 113 Allure Report͓஌ΒͤBotΛ࡞੒

Slide 114

Slide 114 text

Slide 115

Slide 115 text

Slide 116

Slide 116 text

Slide 117

Slide 117 text

Slide 118

Slide 118 text

Flakyͳςετ͕ଟ͍💫 - CIͰςετ͕Failͨ͠ͱ͖ɺϩά΍εΫγϣͷ֬ೝ͕໘౗ CI͕Failͨ͠ͱ͖ ϩά΍εΫγϣͷ֬ೝ͕໘౗

Slide 119

Slide 119 text

Bot͕ڭ͑ͯ͘ΕΔURLΛ֬ೝ✨ Flakyͳςετ͕ଟ͍💫 - CIͰςετ͕Failͨ͠ͱ͖ɺϩά΍εΫγϣͷ֬ೝ͕໘౗

Slide 120

Slide 120 text

Allure Report͓஌ΒͤBotΛ࡞੒ Flakyͳςετ͕ଟ͍💫 - CIͰςετ͕Failͨ͠ͱ͖ɺϩά΍εΫγϣͷ֬ೝ͕໘౗ खܰ͞: ˒ˑˑˑˑ ޮՌ: ˒˒˒ˑˑ

Slide 121

Slide 121 text

஍ಓʹ௚͢ࡍʹେมͩͬͨ͜ͱ • CIͰςετ͕Failͨ͠ͱ͖ɺϩά΍εΫγϣͷ֬ೝ͕໘౗ • CI্Ͱ͔͠ൃੜ͠ͳ͍FlakyͷݪҼ͕ෆ໌ • FlakyΛθϩʹ͸͖͠Εͳ͍ Flakyͳςετ͕ଟ͍💫 121

Slide 122

Slide 122 text

CI্Ͱ͔͠ൃੜ͠ͳ͍FlakyͷݪҼ͕ෆ໌ Flakyͳςετ͕ଟ͍💫 - CI্Ͱ͔͠ൃੜ͠ͳ͍FlakyͷݪҼ͕ෆ໌ ϩά΍εΫϦʔϯγϣοτ ͚ͩͰ͸'BJMݪҼ͕Θ͔Βͳ͍🤯

Slide 123

Slide 123 text

capybara-playwright-driverΛಋೖ Flakyͳςετ͕ଟ͍💫 - CI্Ͱ͔͠ൃੜ͠ͳ͍FlakyͷݪҼ͕ෆ໌

Slide 124

Slide 124 text

• playwrightͱ͍͏ςετϑϨʔϜϫʔΫΛcapybara͔Β࢖͑Δ • εΫϦʔϯγϣοτ͚ͩ͡Όͳ͘ςετ࣌ͷಈը͕ݟΒΕΔ • ։ൃऀπʔϧʹࡌͬͯΔΑ͏ͳωοτϫʔΫ΋ݟΒΕΔ • Allure Report ্ͰݟΒΕΔ • Rails 7.1ʹ΋औΓࠐ·Ε͍ͯΔ Flakyͳςετ͕ଟ͍💫 - CI্Ͱ͔͠ൃੜ͠ͳ͍FlakyͷݪҼ͕ෆ໌ 124 capybara-playwright-driverͱ͸ʁ

Slide 125

Slide 125 text

Slide 126

Slide 126 text

Slide 127

Slide 127 text

Slide 128

Slide 128 text

Slide 129

Slide 129 text

Slide 130

Slide 130 text

CI্Ͱ͔͠ൃੜ͠ͳ͍Flakyͷ ݪҼ͕ෆ໌ Flakyͳςετ͕ଟ͍💫 - CI্Ͱ͔͠ൃੜ͠ͳ͍FlakyͷݪҼ͕ෆ໌

Slide 131

Slide 131 text

ಈըͱωοτϫʔΫτϨʔε Λݟͯղܾ✨ Flakyͳςετ͕ଟ͍💫 - CI্Ͱ͔͠ൃੜ͠ͳ͍FlakyͷݪҼ͕ෆ໌

Slide 132

Slide 132 text

capybara-playwright-driverΛಋೖ & Allure Reportͱඥ෇͚ Flakyͳςετ͕ଟ͍💫 - CI্Ͱ͔͠ൃੜ͠ͳ͍FlakyͷݪҼ͕ෆ໌ खܰ͞: ˒˒ˑˑˑ ޮՌ: ˒˒˒ˑˑ

Slide 133

Slide 133 text

஍ಓʹ௚͢ࡍʹେมͩͬͨ͜ͱ • CIͰςετ͕Failͨ͠ͱ͖ɺϩά΍εΫγϣͷ֬ೝ͕໘౗ • CI্Ͱ͔͠ൃੜ͠ͳ͍FlakyͷݪҼ͕ෆ໌ • FlakyΛθϩʹ͸͖͠Εͳ͍ Flakyͳςετ͕ଟ͍💫 133

Slide 134

Slide 134 text

͘͝كʹམͪΔFlakyςετ… • Ӟ೥ͷ2݄຤೔ʹམͪΔςετ • ςετதʹ೔෇Λލ͍Ͱ͠·ͬͨςετ • APIϨεϙϯε͕͍ͭ΋ΑΓͳ͔ͥ΍ͨΒ஗ͯ͘མͪͨςετ Flakyͳςετ͕ଟ͍💫 134

Slide 135

Slide 135 text

͘͝كʹམͪΔςετ͸CI্ͰࣗಈͰRetryͤ͞Δ Flakyͳςετ͕ଟ͍💫 - FlakyΛθϩʹ͸͖͠Εͳ͍

Slide 136

Slide 136 text

͘͝كʹམͪΔςετ͸CI্ͰࣗಈͰRetryͤ͞Δ Flakyͳςετ͕ଟ͍💫 - FlakyΛθϩʹ͸͖͠Εͳ͍ 'BJMͨ͠ςετ͚ͩ࠶࣮ߦ͢Δ

Slide 137

Slide 137 text

͘͝كʹམͪΔςετ͸CI্ͰࣗಈͰRetryͤ͞Δ Flakyͳςετ͕ଟ͍💫 - FlakyΛθϩʹ͸͖͠Εͳ͍ ςετࣦഊͨ͠Βɺ࣍ͷStepͷςετͰ͸ Capybaraͷ଴ͪ࣌ؒ΋௕͘ઃఆ͢Δ

Slide 138

Slide 138 text

͘͝كʹམͪΔFlakyͳςετ Flakyͳςετ͕ଟ͍💫 - FlakyΛθϩʹ͸͖͠Εͳ͍

Slide 139

Slide 139 text

ࣦഊͨ͠ςετΛ࠶౓ ࣗಈͰςετ͢ΔStepΛ௥Ճͯ͠ղܾ✨ Flakyͳςετ͕ଟ͍💫 - FlakyΛθϩʹ͸͖͠Εͳ͍

Slide 140

Slide 140 text

ࣦഊͨ͠ςετΛ࠶౓ ࣗಈͰςετ͢ΔStepΛ௥Ճ Flakyͳςετ͕ଟ͍💫 - FlakyΛθϩʹ͸͖͠Εͳ͍ खܰ͞: ˒˒˒˒ˑ ޮՌ: ˒˒˒˒˒

Slide 141

Slide 141 text

~·ͱΊ~ 2024೥ͷࣗಈςετͷঢ়گ🔍

Slide 142

Slide 142 text

೥ ೥ ςετ਺ DBTF $*ͷ࣌ؒ ෼ ฒྻ$PSF 'MBLZςετ ݸ ͘Β͍ͷ֬཰Ͱ$*͕མͪΔ (JU)VC"DUJPOTͷ஋ஈ ·ͱΊ 2023೥ͷςετͷঢ়گ🔍

Slide 143

Slide 143 text

2024೥ͷςετͷঢ়گ🔍 ೥ ೥ ςετ਺ DBTF DBTF $*ͷ࣌ؒ ෼ ฒྻ$PSF ෼ඵ🎉 ฒྻ 'MBLZςετ ݸ ͘Β͍ͷ֬཰Ͱ$*͕མͪΔ ݸ🎉 ͘Β͍ͷ֬཰Ͱ$*͕མͪΔ (JU)VC"DUJPOTͷ஋ஈ 🎉 ·ͱΊ

Slide 144

Slide 144 text

΍ͬͨ͜ͱҰཡ • Capybaraͷػೳʹ೚ͤͯSleepΛ࡟আ • LoadingΛಈతʹ଴ͭHelper࡞ͬͯSleepΛ࡟আ • test-profͷbefore_allΛ࢖ͬͯσʔλ࡞੒Λলུ • test-profͷlet_it_beΛ࢖ͬͯσʔλ࡞੒Λলུ • parallel_testsͷฒྻ࣮ߦΛ׆͔ͨ͢ΊʹϑΝΠϧ෼ׂ ·ͱΊ 144

Slide 145

Slide 145 text

΍ͬͨ͜ͱҰཡ • FactoryBot.create() Ͱ͸ͳ͘ FactoryBot.build_stubbed()Λ ࢖͏ • ෼͚ͳ͍͍ͯ͘ςετ͸ɺ1ͭͷ it do ~ endʹ·ͱΊΔ • ͍Βͳ͍ػೳ΍ςετ͸࡟আ • LoadingΛ଴ͭॲཧ΋ՄೳͳΒ࡟আ • KnapsackPro gem΋ར༻ ·ͱΊ 145

Slide 146

Slide 146 text

΍ͬͨ͜ͱҰཡ • ஗͍ςετϑΝΠϧ,஗͍ςετ, աڈࣦഊͨ͠ςετΛμο γϡϘʔυԽ • Next.jsͷBuildΛΩϟογϡԽ • capybara-playwright-driver௥Ճ • AllureReport௥Ճ • GitHubͷAllureReportͷϦϯΫBot ·ͱΊ 146

Slide 147

Slide 147 text

͓ർΕ༷Ͱͨ͠🍻