runnによるAPIのシナリオテストの導入と自動化 / stac2022

by Ken’ichiro Oyama

Slide 1

Slide 1 text

runnʹΑΔAPIͷγφϦΦςε τͷಋೖͱࣗಈԽ খࢁ݈Ұ࿠ / GMO PEPABO inc. 2022.12.3 ιϑτ΢ΣΞςετࣗಈԽΧϯϑΝϨϯε2022 1

Slide 2

Slide 2 text

ࣗݾ঺հ ٕज़෦ɹٕज़ج൫νʔϜ 2018೥ த్ೖࣾ খࢁ ݈Ұ࿠ Ken’ichiro Oyama গ࣮͠༻తͰখ͞ͳOSSΛॻ͘ͷ͕झຯɻ ● GitHub : k1LoW ● Twitter : @k1LoW 2

Slide 3

Slide 3 text

GMOϖύϘʹ͍ͭͯ 3 https://pepabo.com

Slide 4

Slide 4 text

• https://github.com/k1LoW/awspec • AWSͷ֤Ϧιʔεͷঢ়ଶʹରͯ͠ɺRSpecͰςετΛॻ͘͜ͱ͕Ͱ͖Δπʔϧ • ݱࡏ88Ϧιʔεαϙʔτ͍ͯ͠Δ awspec 4 4 ࡞͍ͬͯΔOSS঺հ

Slide 5

Slide 5 text

5 • https://github.com/k1LoW/tbls • CIϑϨϯυϦͳσʔλϕʔευΩϡϝϯτੜ ੒πʔϧ • PostgreSQL, MySQLͳͲͷRDBMS͚ͩͰͳ ͘ɺBigQuery΍Amazon DynamoDB, Cloud Spannerͱ͍ͬͨσʔλιʔεʹ΋ରԠ • υΩϡϝϯτΛॆ࣮ͤ͞Δػೳͱͯ͠ϝλ σʔλΛΞϊςʔγϣϯ͢Δ͜ͱ͕Մೳ • ςʔϒϧɺΧϥϜͳͲͷίϝϯτ෇༩͚ͩͰ ͳ͘λάɺϦϨʔγϣϯͳͲ tbls ࡞͍ͬͯΔOSS঺հ

Slide 6

Slide 6 text

• https://github.com/k1LoW/octocov • ίʔυϝτϦΫεπʔϧ…ίʔυΧόϨοδɺCode to Test Ratioɺςετ࣮ߦ࣌ؒ • ଟ਺ͷϓϩάϥϛϯάݴޠͱɺओཁͳΧόϨοδϨϙʔτϑΥʔϚοτʹରԠ • GitHub Actionsͷactionͱͯ͠ίʔυϝτϦΫεΛूܭɺP/RίϝϯτɺϨϙʔτ • CLIπʔϧͱͯ͠ίʔυΧόϨοδΛूܭɻදࣔ • ΧόϨοδόοδੜ੒ octocov 6 6 ࡞͍ͬͯΔOSS঺հ

Slide 7

Slide 7 text

ΞδΣϯμ • runnͱ͸ • ςετπʔϧ։ൃऀͱͯ͠ͷ૝ఆϢʔεέʔε • ࣮ࡍͷϢʔεέʔε • ͓ΘΓʹ ɹ※ͳ͓ɺຊൃද͸Θ͔Γ΍͘͢͢ΔͨΊʹ࣮ࡍͷ࣌ܥྻʹԊ͍ͬͯͳ͍಺༰ʹͳ͍ͬͯ·͢ɻ 7

Slide 8

Slide 8 text

8 runnͱ͸

Slide 9

Slide 9 text

Slide 10

Slide 10 text

• runn ( means "Run N". is pronounced /rʌ́n én/. ) is a package/tool for running operations following a scenario. • γφϦΦΛYAMLͰॻ͍ͯͦΕΛ΋ͱʹૢ࡞ʢΦϖϨʔγϣϯʣΛࣗಈԽͰ͖Δύο έʔδ/πʔϧ • Goͷύοέʔδͱͯ͠΋ίϚϯυʢrunnίϚϯυʣͱͯ͠΋࢖͑Δ • runn run [γφϦΦσΟϨΫτϦ]/*.yml • ʮΦϖϨʔγϣϯࣗಈԽʯͱ޿ൣғΛλʔήοτʹ͍ͯ͠Δ͕ɺAPIͷγφϦΦς ετ͕ϑΝʔετϓϥΠΦϦςΟͳϢʔεέʔεʢ։ൃͷ͖͔͚ͬͰ΋͋ΔͨΊʣ 10 https://github.com/k1LoW/runn runn

Slide 11

Slide 11 text

• 1όΠφϦͷγφϦΦ࣮ߦͷCLIίϚϯυʢπʔϧʣͱͯ͠࢖༻Ͱ͖Δ • 1όΠφϦ = CIϑϨϯυϦ • ͞·͟·ͳϓϩτίϧʹରԠ͓ͯ͠Γɺ1ͭͷγφϦΦ಺Ͱෳ਺૊Έ߹ΘͤΔ͜ͱ͕Մೳ • HTTP • gRPC • DatabaseʢSQLʣ • Chrome DevTools Protocol • ೚ҙίϚϯυͷ࣮ߦʢϩʔΧϧ/SSHʣ • GoͷTest Helperʢύοέʔδʣͱͯ͠࢖༻Ͱ͖Δ 11 https://github.com/k1LoW/runn runnͷಛ௃ʢv0.53.0ʣʢ1/2ʣ

Slide 12

Slide 12 text

12 https://github.com/k1LoW/runn runnͷಛ௃ʢv0.53.0ʣʢ2/2ʣ • γφϦΦϑΝΠϧʢYAMLʣͷϑΥʔϚοτ͕OpenAPI SpecϥΠΫ • γφϦΦ಺ͷεςοϓؒͷ஋͕ࣗಈͰ࿈ܞ͞Ε͍ͯΔ • ʮεςοϓAΛ࣮ߦ͔ͯ͠Βͦͷ݁ՌΛ࢖ͬͯεςοϓBΛ࣮ߦ͢Δʯͱ͍͏γφϦΦΛߏங͠΍͍͢ • ෳ਺ͷγφϦΦͷ࣮ߦΛ૝ఆ࣮ͨ͠૷ • ෳ਺ͷγφϦΦͷ෼ׂ࣮ߦɺαϯϓϦϯά࣮ߦɺϥϯμϜ࣮ߦͳͲ͕Մೳ • ϧʔϓ࣮ߦ΍ϦτϥΠ࣮ߦͷ࢓૊Έ͕૊ΈࠐΈͰඋΘ͍ͬͯΔ • ಉ͡γφϦΦϑΝΠϧΛ࢖ͬͨෛՙςετʢϩʔυςετʣ͕Մೳ • ίϚϯυΛ runn run ͔Β runn loadt ʹมߋ͢Δ͚ͩɻෳ਺γφϦΦ࣮ߦ΋ͦͷ··Մೳ • γφϦΦ࣮ߦͷϩά΋ه࿥Մೳ • γφϦΦ࣮ߦͷϓϩϑΝΠϧ΋औಘՄೳʢ֤εςοϓͷܦա࣌ؒʣ

Slide 13

Slide 13 text

• runn͸1ͭҎ্ͷγφϦΦΛ࣮ߦͯ͠ɺͦΕͧΕͷ੒ޭ/ࣦഊΛΈΔ • γφϦΦ͸1ͭҎ্ͷεςοϓ͔Βߏ੒͞Ε͍ͯΔ • 1γφϦΦ1YAMLϑΝΠϧʹͳ͍ͬͯΔʢRunbookͱݺΜͰ͍Δʣ • γφϦΦ͝ͱʹRunnerͱݺΜͰ͍Δεςοϓ࣮ߦίϯϙʔωϯτΛఆٛ͠ɺͦͷRunnerΛ࢖֤ͬͯ εςοϓΛ࣮ߦ͍ͯ͘͠ • Runnerʹ͸γφϦΦ಺ʹఆ͕ٛඞཁͳ௨ৗͷRunnerͱɺطʹrunnͰ༻ҙ͍ͯ͠ΔBuilt-in runnerͷ2 छྨ͕͋Δ • Runner: HTTP Runner, DB Runner, gRPC Runner, CDP Runner, SSH Runner • Built-in runner: Exec runner, Test runner, Dump runner, Include runner, Bind runner 13 https://github.com/k1LoW/runn runnͷΞʔΩςΫνϟ

Slide 14

Slide 14 text

14 γφϦΦʢRunbookʣΛॻ͘

Slide 15

Slide 15 text

15 https://github.com/k1LoW/runn

Slide 16

Slide 16 text

16 https://github.com/k1LoW/runn

Slide 17

Slide 17 text

17 https://github.com/k1LoW/runn

Slide 18

Slide 18 text

18 https://github.com/k1LoW/runn

Slide 19

Slide 19 text

19 https://github.com/k1LoW/runn

Slide 20

Slide 20 text

20 https://github.com/k1LoW/runn

Slide 21

Slide 21 text

21 https://github.com/k1LoW/runn

Slide 22

Slide 22 text

22 https://github.com/k1LoW/runn Runnerͷઃఆ ม਺ͷઃఆ ɹɹɹɹ֤εςοϓ 0 1 2 3

Slide 23

Slide 23 text

23 runn new

Slide 24

Slide 24 text

• γφϦΦϑΝΠϧ࡞੒ͷͨΊͷαϙʔτίϚϯυ • runn newͷҾ਺ʹ஋Λ౉͢͜ͱͰγφϦΦϑΝΠϧΛ࡞੒͢Δ • cURLίϚϯυ • gRPCurlίϚϯυ • ΞΫηεϩά • ʢOpenAPI SpecͷυΩϡϝϯτʁʣ • ʢPostmanͷCollectionʁʣ • طଘͷγφϦΦʹεςοϓͱͯ͠௥Ճ͢Δ͜ͱ΋Մೳ 24 https://github.com/k1LoW/runn runn new

Slide 25

Slide 25 text

25 https://github.com/k1LoW/runn DEMO: https://github.com/k1LoW/runn#quickstart

Slide 26

Slide 26 text

26 ςετπʔϧ։ൃऀͱͯ͠ͷ ૝ఆϢʔεέʔε

Slide 27

Slide 27 text

• ൃදऀ͸ιϑτ΢ΣΞΤϯδχΞ • ओʹόοΫΤϯυͷઃܭͱ։ൃΛ୲౰ • ςετΛઐ໳ʹ͍ͯ͠ΔΤϯδχΞͰ͸ͳ͍ • GoͰAPIαʔόΛ։ൃ͍ͯ͠Δ • ୲౰ίϯϙʔωϯτʢAPIαʔόʣΛઃܭ͍ͯ͘͠ʹ͋ͨͬͯɺෳ਺ͷAPIΛ·͍ͨ Ͱ1ͭͷػೳΛ࣮ݱ͢ΔΑ͏ͳઃܭʹͳΔ͜ͱ͕ݟ͖͑ͯͨɻ • APIεΩʔϚͱσʔλϕʔεεΩʔϚ͸৻ॏʹઃܭͨ͠ɻ͔͠͠ɺίʔυϕʔε͸࠷ ॳ͔Β࠷దͳΞʔΩςΫνϟΛબ୒Ͱ͖Δͱ͸ࢥ͑ͳ͔ͬͨɻ 27 ςετπʔϧ։ൃऀͱͯ͠ͷ૝ఆϢʔεέʔε ։ൃͷಈػ

Slide 28

Slide 28 text

• ػೳ୯Ґʢෳ਺APIάϧʔϓ͔ΒͳΔ͠͹͠͹ εςʔτϑϧͳػೳʣͷςετΛॆ࣮ͤ͞Δ͜ ͱʹΑΓɺͨͱ్͑தͷϦΞʔΩςΫςΟϯά ͕͋ͬͯ΋ػೳͷอূ͕Ͱ͖ΔΑ͏ʹ͍ͨ͠ɻ • ։ൃॳظ͔ΒUnitςετͱಉ༷ʹػೳʹରͯ͠ ͷγφϦΦςετͷॆ࣮Λ໨ࢦ͢ɻ • ͦͷͨΊʹγφϦΦςετΛUnitςετฒΈʹ ؆୯ʹ͢Δπʔϧͱͯ͠runnΛ։ൃɻ 28 ςετπʔϧ։ൃऀͱͯ͠ͷ૝ఆϢʔεέʔε Testing trophyʢςεςΟϯάτϩϑΟʔʣ https://kentcdodds.com/blog/the-testing-trophy-and-testing-classifications   https://twitter.com/kentcdodds/status/960723172591992832

Slide 29

Slide 29 text

• GoͰAPIαʔόΛ։ൃ͍ͯ͠Διϑτ΢ΣΞΤϯδχΞ͕ςετίʔυʹ૊Έ͜ΜͰ࢖ ༻͢Δ • net/http/httptest ύοέʔδͱ૬ੑ͕͍͍ • γφϦΦͷ֦ॆ΋YAMLΛॻ͚ͩ͘Ͱྑ͍ • OpenAPI SpecϥΠΫͳϑΥʔϚοτΛ࠾༻ͨ͠ͷ΋ɺγφϦΦςετΛॻ͘Τϯ δχΞͱOpenAPIυΩϡϝϯτΛॻ͘ΤϯδχΞΛಉҰਓ෺ͱͯ͠ߟ͍͑ͯͨͨΊ • 1όΠφϦͷͨΊΠϯετʔϧ΋؆୯ɻCI΁ͷ૊ΈࠐΈ΋༰қɻ 29 ςετπʔϧ։ൃऀͱͯ͠ͷ૝ఆϢʔεέʔε ։ൃऀ͕૝ఆ͍ͯͨ͠ओͳϢʔεέʔε

Slide 30

Slide 30 text

࣮ࡍͷϢʔεέʔε 30

Slide 31

Slide 31 text

• ݁࿦ͱͯ͠͸ʮ͏·͍ͬͨ͘ʯ • Ұ౓૊ΈࠐΜͰ͠·͑͹ɺ͋ͱ͸YAMLΛॻ͚ͩ͘ʹͳΔͨΊɺ͔ͳΓḿͬͨɻ • ḿΓ͗ͯ͢ςετ࣮ߦ࣌ؒʹ૿େ͠ɺςετ෼ׂ͕ඞཁʹͳͬͨ + ςετ෼ׂػೳ΍ϓϩϑΝΠϧػೳͷ௥Ճ 31 ࣮ࡍͷϢʔεέʔε ։ൃऀ͕૝ఆ͍ͯͨ͠ϢʔεέʔεͰ͸͏·͍͔ͬͨ͘ʁ

Slide 32

Slide 32 text

32 ʢ౰࣌ʣ ૝ఆ֎ͷϢʔεέʔε

Slide 33

Slide 33 text

• γφϦΦ͕OpenAPI SpecϥΠΫͰ͋Δ͜ͱ͔ΒΤϯδχΞʹͱͬͯಡΈ΍͘͢ɺͦ ͷ··ΦϯϘʔσΟϯάʹ໾ʹཱͬͨʢAPIར༻ଆ΁ͷઆ໌ͳͲʣ • ॻ͖΍͍͢=ಡΈ΍͍͢ + ΑΓυΩϡϝϯτͱͯ͠࢖͍΍͘͢͢ΔͨΊʹγφϦΦ΍εςοϓͷઆ໌Λهड़͢ ΔͨΊͷ desc: ηΫγϣϯͷ৽ઃ + υΩϡϝϯτੜ੒ػೳΛݕ౼ 33 ࣮ࡍͷϢʔεέʔε ૝ఆ֎ͷϢʔεέʔε1: υΩϡϝϯτͱͯ͠

Slide 34

Slide 34 text

• ؍ଌͰ͖ͨϢʔεέʔε͸ѹ౗తʹʮGoͷςετίʔυ΁ͷ૊ΈࠐΈʯͰ͸ͳ͘ ʮrunnίϚϯυʢπʔϧʣͱͯ͠ͷར༻ʯͩͬͨ • ςετίʔυ΁ͷ૊ΈࠐΈΑΓ΋ʮμ΢ϯϩʔυͨ͠Β͙͢࢖͑Δʯͷ΄͏͕ ѹ౗తʹָ • GoʹݶΒͳ͍൚༻తͳར༻͕Մೳ • ػೳͷఏҊ΍۩ମతͳػೳ௥ՃͷPull Request΋΋Βͬͨ + runnίϚϯυଆͷػೳ΋ϑΝʔετϓϥΠΦϦςΟͱ֦ͯ͠ॆ͍ͤͯͬͨ͞ 34 ࣮ࡍͷϢʔεέʔε ૝ఆ֎ͷϢʔεέʔε2: runnίϚϯυͷ׆༻

Slide 35

Slide 35 text

• ΞϓϦέʔγϣϯσϓϩΠ௚ޙͷಈ࡞֬ೝͷͨΊͷ؆қతͳϦάϨογϣϯςετͷ ඞཁੑ • ϖύϘ͸͞·͟·ͳٕज़ελοΫͰߏங͞Εͨଟ͘ͷαʔϏεΛӡ༻͍ͯ͠Δ • ؆қͰྑ͍ɻ͔͠͠ɺʮϒϥ΢βϕʔεʯʮ։ൃݴޠʹدΒͳ͍͜ͱʯʮηοτ Ξοϓ͕؆୯Ͱ͋Δ͜ͱʯʮςετ࡞੒ֶ͕शίετ͕খ͍͜͞ͱʯ͕ཁ݅ + Chrome DevTools ProtocolʹରԠ • ChromeʢChromiumʣࠐΈͷDocker imageΛ༻ҙ͢Δ͜ͱͰ্هཁ݅Λຬͨͨ͠ • docker pull ghcr.io/k1low/runn:latest 35 ࣮ࡍͷϢʔεέʔε ૝ఆ֎ͷϢʔεέʔε3: ϦάϨογϣϯςετ

Slide 36

Slide 36 text

• ૝ఆ֎ͷϢʔεέʔεΛ౿·͑ͯɺ͋ΒͨΊͯ • runn ( means "Run N". is pronounced /rʌ́n én/. ) is a package/tool for running operations following a scenario. • ʮΦϖϨʔγϣϯࣗಈԽʯΛπʔϧͷλʔήοτͱ͢ΔͷͰ͋Ε͹ɺʢ౰ॳ͔Β͢ Δͱʣ૝ఆ֎ͳϢʔεέʔε΋෇ਵͨ͠৽ػೳ΋ʮਖ਼౰ͳྲྀΕʯͱ΋͍͑Δ + γφϦΦ࣮ߦϩάͷه࿥ػೳ … ࣮ߦՄೳͳૢ࡞खॱॻͱͯ͠ + SSH Runner … αʔόΦϖϨʔγϣϯࣗಈԽͷखஈͱͯ͠ • ʮ1όΠφϦͰಋೖ͕༰қʯʮՄಡੑߴ͍γφϦΦϑΝΠϧʯͷ2ͭ͸ݫक 36 ࣮ࡍͷϢʔεέʔε ʮΦϖϨʔγϣϯࣗಈԽʯͱ͍͏λʔήοτ

Slide 37

Slide 37 text

͓ΘΓʹ 37

Slide 38

Slide 38 text

• APIγφϦΦςεςΟϯάπʔϧ/ΦϖϨʔγϣϯࣗಈԽπʔϧͱͯ͠runnΛ঺հ • πʔϧ։ൃऀͱͯ͠૝ఆ͍ͯͨ͠Ϣʔεέʔεͱ࣮ࡍͷϢʔεέʔε͸ɺͣΕΔ͜ͱ ͕͋Δʢ͓ͦΒ͘ྑ͍৔߹΋ѱ͍৔߹΋͋Δʣ • ։ൃऀͱͯ͠͸ɺ͢΂ͯϑΟʔυόοΫͱͯ͠ड͚ࢭΊ͍ͨͱࢥ͍ͬͯΔ • ੈͷπʔϧ։ൃऀ͸େମͦ͏ࢥ͍ͬͯΔʢͨͿΜʣ • ʢ࣌ʑݫ͍͜͠ͱ΋͋Δʢͦͷͱ͖͸͝ΊΜͳ͍͞ʣʣ • ϑΟʔυόοΫ͸خ͍͠ʂ 38 ͓ΘΓʹ ·ͱΊ

Slide 39

Slide 39 text

39 ͓ΘΓʹ ࠓޙ • ࠓrunnʹ଍Γͳ͍΋ͷͱͯ͠͸ʮ೔ຊޠͷ৘ใʯͩͱߟ͍͑ͯΔ • runnͷػೳʹରͯ͠গͳ͍ • ࡞ऀ͕ॻ͍͍ͯͳ͍ͷͰɺͦΕ͸ͦ͏ • ݱࡏʮrunn cookbookʯΛZennͷbookͱͯ͠४උத • ϨγϐΛඞཁʹԠͯ͡ਵ࣌௥Ճ͍ͯ͘͠ํࣜ • ΋͠ڵຯ͕Ͱ·ͨ͠Βੋඇࢼ͠ʹ࢖ͬͯΈ͍ͯͩ͘͞ʂ • macOSͳΒ brew install k1LoW/tap/runn • LinuxͳΒ deb΍RPM΍apkύοέʔδ • docker pull ghcr.io/k1low/runn:latest

Slide 40

Slide 40 text

Thank You! Thank You! 40