クラウド上のモバイル端末・シミュレータを活用した大規模並列テスト / Large Parallel Testing with Real Devices or Simulators on Cloud

by Kuniwak

Slide 1

Slide 1 text

Ϋϥ΢υ্ͷϞόΠϧ୺຤Λ  ׆༻ͨ͠େن໛ฒྻςετͷ  ࣮ߦ؀ڥʹ͍ͭͯ γεςϜςετࣗಈԽΧϯϑΝϨϯε ,VOJXBL !PSHB@DIFN

Slide 2

Slide 2 text

"CPVUNF

Slide 3

Slide 3 text

,VOJXBL w ॴଐˠ w HJUIVCDPN,VOJXBL w RJJUBDPN,VOJXBL w ೰ΊΔςετϚϯ

Slide 4

Slide 4 text

ൃදͰ఻͍͑ͨ͜ͱ

Slide 5

Slide 5 text

-JOU΍୯ମςετͰൃݟͰ͖ͳ͍όάΛɺ  อक༰қͳςετͰݟ͚ͭΔج൫ͱͯ͠  ʮEFWGBSNʯ͕ศརͰ͢

Slide 6

Slide 6 text

$*ʹࡌ͍ͤͨ΋ͷ EFWGBSNͷ঺հ ӡ༻։࢝࣌఺Ͱͷײ૝

Slide 7

Slide 7 text

$*ʹࡌ͍ͤͨ΋ͷ EFWGBSNͷ঺հ ӡ༻։࢝࣌఺Ͱͷײ૝ ͱͯ΋ॏཁ

Slide 8

Slide 8 text

ϞόΠϧήʔϜΛ  ແ࡞ҝʹૢ࡞ͯ͠  Ϋϥογϡ΍  ੑೳྼԽΛ  ݕग़͢Δςετ ࠨҠಈ ӈҠಈ δϟϯϓ ແ࡞ҝʹ࣮ߦ

Slide 9

Slide 9 text

͜ͷςετͷҐஔ෇͚͔Β  ཧ૝తͳ༻๏༻ྔΛઆ໌͠·͢ ʮಉ͡ํ๏Ͱ΍ͬͨͷʹશવμϝʯΛආ͚ΔͨΊʹ

Slide 10

Slide 10 text

υϝΠϯݻ༗ͷ݁߹ϨΠϠʔ  ϞϯΩʔςετ શ෦ΛҰ౓ʹઆ໌͢Δͱࠞཚ͢Δͱࢥ͏ͷͰɺ  ॱ൪ʹઆ໌͍͖ͯ͠·͢

Slide 11

Slide 11 text

υϝΠϯݻ༗ͷ݁߹ϨΠϠʔ  ϞϯΩʔςετ ·ͣ͸ʮ݁߹ϨΠϠʔʯʹ͍ͭͯͰɺ  ͜͜ͰԿΛݟ͚ͭΔ΂͖͔ΛΈͯΈ·͠ΐ͏

Slide 12

Slide 12 text

όάൃݟ͸ଟ૚͕جຊ ݁߹ϨΠϠʔ w ߏจϋΠϥΠτ w -JOU࣮ߦ w ੩తܕݕࠪ w ୯ମςετ w ݁߹ςετ w &&ςετ ͍Ζ͍Ζͳ૚ʹόάൃݟͷखஈ͕͋ΓɺͲͷ૚Ͱ  ͲͷόάΛݟ͚ͭΔ͔͕ͱͯ΋ॏཁɿ w ຊ൪؀ڥͰͷ  ΤϥʔϨϙʔλʔ w ͓໰͍߹Θͤ

Slide 13

Slide 13 text

ߏจϋΠϥΠτ -JOU࣮ߦ ੩తܕݕࠪ ୯ମςετ ݁߹ςετ &&ςετ ຊ൪؀ڥͰͷΤϥʔϨϙʔλʔ ଟ૚ͷόάൃݟखஈΛʮ͋ΔنଇʯͰฒ΂ͨ΋ͷ

Slide 14

Slide 14 text

ߏจϋΠϥΠτ ੩తܕݕࠪ ୯ମςετ ݁߹ςετ &&ςετ όά9Λݟ͚ͭͨ৔ॴ όά9 ຊ൪؀ڥͰͷΤϥʔϨϙʔλʔ -JOU࣮ߦ

Slide 15

Slide 15 text

ߏจϋΠϥΠτ -JOU࣮ߦ ੩తܕݕࠪ ୯ମςετ ݁߹ςετ &&ςετ ຊ൪؀ڥͰͷΤϥʔϨϙʔλʔ όά" खલͷํͰࢭ·Δͱ ॴཁ͕࣌ؒ୹͍ όά#

Slide 16

Slide 16 text

ߏจϋΠϥΠτ -JOU࣮ߦ ੩తܕݕࠪ ୯ମςετ ݁߹ςετ &&ςετ ຊ൪؀ڥͰͷΤϥʔϨϙʔλʔ όά" खલͷํͰࢭ·Δͱ ॴཁ͕࣌ؒ୹͍ όά# όά͸ͳΔ΂͘खલͰ  ൃݟ͢Δͷ͕ޮՌత

Slide 17

Slide 17 text

ߏจϋΠϥΠτ -JOU࣮ߦ ੩తܕݕࠪ ୯ମςετ ݁߹ςετ &&ςετ ຊ൪؀ڥͰͷΤϥʔϨϙʔλʔ όά" όά# ಛघͳόά͸ޙͷํͰ  ͔͠ݟ͚ͭΒΕͳ͍

Slide 18

Slide 18 text

ߏจϋΠϥΠτ -JOU࣮ߦ ੩తܕݕࠪ ୯ମςετ ݁߹ςετ &&ςετ ຊ൪؀ڥͰͷΤϥʔϨϙʔλʔ όά" ಛघͳόά͸ޙͷํͰ  ͔͠ݟ͚ͭΒΕͳ͍ όά# खલͰऔΓ͜΅ͨ͠  ΋ͷ͸ԼͰड͚ࢭΊΔ

Slide 19

Slide 19 text

ߏจϋΠϥΠτ -JOU࣮ߦ ੩తܕݕࠪ ୯ମςετ ݁߹ςετ &&ςετ ຊ൪؀ڥͰͷΤϥʔϨϙʔλʔ ͜ͷൃදͰൃݟ͍ͨ͠  όά͸ɺ୯ମςετͰ  औΓ͜΅͞ΕΔ͕  ΤϥʔϨϙʔτΑΓ  ૣ͘ݟ͚ͭΒΕΔόά

Slide 20

Slide 20 text

-JOU΍୯ମςετͰ  όάΛݟ্͚ͭͨͰ  ซ༻͢Δͷ͕େલఏ ཁ͢Δʹ

Slide 21

Slide 21 text

݁߹ςετ͸อक͕ͱͯ΋େม ͨͩ͠

Slide 22

Slide 22 text

݁߹͢Δ΄Ͳɺࢼ͞ͳ͍ͱ  ͍͚ͳ͍૊Έ߹Θ͕ͤ૿͑Δ ݁߹͢Δ΄Ͳɺݸʑͷཁૉͷ  มߋʹରͯ͠ςετ͕੬͘ͳΔ ܽ఺ ܽ఺

Slide 23

Slide 23 text

υϝΠϯݻ༗ͷ݁߹ϨΠϠʔ  ϞϯΩʔςετ ͦ͜Ͱ

Slide 24

Slide 24 text

ಛఆͷૢ࡞खॱΛఆΊͣɺ  ແ࡞ҝʹૢ࡞͢Δςετ ϞϯΩʔςετͱ͸

Slide 25

Slide 25 text

ແ࡞ҝʹૢ࡞͢ΔͷͰ૊Έ߹Θ͕ͤ ૿͑ͯ΋ίʔυ͸͋·Γ૿͑ͳ͍ Ұൠతͳੑ࣭͔֬͠ೝ͠ͳ͍ͷͰɺ  ݸʑͷཁૉ͕มߋ͞Εͯ΋յΕͳ͍ ར఺ ར఺

Slide 26

Slide 26 text

ήʔϜͰ͸ແ࡞ҝʹૢ࡞͢Δ ͚ͩͰ͸ෆे෼ͳ͜ͱ͕ଟ͍ ͨͩ͠

Slide 27

Slide 27 text

͋Δॱ൪Ͱૢ࡞͢Δͱ  ඞࡴٕ͕ൃಈ͢ΔΑ͏ͳ  ΞΫγϣϯ͕͋Δͱ͖ɺ  ແ࡞ҝͩͱΞΫγϣϯ͸  ΄ͱΜͲൃੜ͠ͳ͍ ΞΫγϣϯҙຯΛ΋ͬͨ࿈ଓૢ࡞ʹ  Α࣮ͬͯߦ͞ΕΔΠϕϯτ ΞΫγϣϯ

Slide 28

Slide 28 text

ՄೳͳΞΫγϣϯΛ  ެฏʹൃੜͤ͞ͳ͍ͱ  ςετ͕ෆ׬શʹͳΔ ͭ·Γ

Slide 29

Slide 29 text

υϝΠϯݻ༗ͷ݁߹ϨΠϠʔ  ϞϯΩʔςετ ͦ͜Ͱ

Slide 30

Slide 30 text

 ͭʹߜΓࠐΉ ⁞ՄೳͳΞΫγϣϯΛ  ໰͍߹Θͤ ⁠ΞΫγϣϯΛ࣮ߦ ඞࡴٕ ࠨӈҠಈ δϟϯϓ ඞࡴٕ ࠨӈҠಈ ඞࡴٕ ࣮ߦ ϦλΠΞ

Slide 31

Slide 31 text

ެฏͳΞΫγϣϯ͕Մೳʹͳͬͨ ͭ·Γ

Slide 32

Slide 32 text

͜͜·Ͱͷ·ͱΊ -JOU΍୯ମςετͰ͸  ݟ͚ͭΒΕͳ͍όάΛ  อक༰қͳςετͰݟ͚͍ͭͨ

Slide 33

Slide 33 text

$*ʹࡌͤΔࡍͷ՝୊

Slide 34

Slide 34 text

Ͳ͜ͰςετΛಈ͔͔͢ ΞϓϦΛͲ͔͜Ͱىಈ͢Δඞཁ͕͋Δ͕ɺ  ࣮ػݻ༗ͷ໰୊Λݟ͚͍ͭͨΘ͚Ͱ͸ͳ͍ ϞϯΩʔςετ͸େྔʹฒྻ࣮ߦ͢Δ΄Ͳ ୹࣌ؒͰଟ͘ͷ໰୊Λݟ͚ͭΒΕΔͷͰ  ߴ଎Ͱ҆ՁͳγϛϡϨʔλΛ࢖͑ΔͳΒ  ͦ͏͍ͨ͠

Slide 35

Slide 35 text

γϛϡϨʔλΫϥ΢υʜʁ γϛϡϨʔλΛΫϥ΢υͰ࣮ߦ͢Δ  αʔϏε͸Θ͔ͣʹ͋Δ͕࢖͍ͮΒ͍ʜ ࢓ํͳ͍ͷͰ࣮ػΛΫϥ΢υͰ࣮ߦͰ͖Δ  σόΠεϑΝʔϜ͕ୈҰީิʹͳΔ

Slide 36

Slide 36 text

طଘͷσόΠεϑΝʔϜ w "84%FWJDF'BSN w )FBETQJO w 3FNPUF5FTU,JU w 74"QQ$FOUFS w 'JSFCBTF5FTU-BC w 4BVDF-BCT  3FBM%FWJDF$MPVE w

Slide 37

Slide 37 text

ͨ͘͞Μ͋Γ͗ͯ͢  ͲΕΛબ΂͹ʜʂʁ

Slide 38

Slide 38 text

ࠓճ͸"84%FWJDF'BSN "84%FWJDF'BSN͸NJOͰ  ͍ͭͰ΋ฒྻ਺Λ্͛Լ͛Ͱ͖Δ ଞ͸୆਺͕ܾ·ͬͨظؒܖ໿͕ଟ͘ɺ  $*த͚ͩฒྻ਺Λ֬อ͢Δͱ͍ͬͨ  ࢖͍ํͱ૬ੑ͕ѱ͍ IUUQTBXTBNB[PODPNKQEFWJDFGBSNQSJDJOH

Slide 39

Slide 39 text

ͨͩ͠ෆ҆ཁૉ΋ʜ ւ֎੡ͷ୺຤͕ଟ͘ɺ೔ຊ੡ͷ୺຤͸  ͋·Γἧ͍ͬͯͳ͍ কདྷతʹࠃ಺ͷ୺຤͕ଟ͘  ैྔ՝ۚͷσόΠεϑΝʔϜ͕ग़ͨΒ  ৐Γ׵͑ΔՄೳੑ͕͋Δ

Slide 40

Slide 40 text

ݱঢ়ͷํ਑ ͍ͭผͷγϛϡϨʔλΫϥ΢υ΍  σόΠεΫϥ΢υ΁Ҡߦ͢Δ͔Θ͔Βͳ͍ ͜ͷ͕͍͖࣌ͭͯ΋ɺ  Ҡߦίετͷগͳ͍ঢ়ଶΛอ͍ͪͨ

Slide 41

Slide 41 text

ͦ͜ͰEFWGBSN γϛϡϨʔλσόΠεΫϥ΢υͷ  ੾Γସָ͕͑ͰɺσόΠεΫϥ΢υ্Ͱ  ΞϓϦͷىಈঢ়گΛ؅ཧ͢Δπʔϧ

Slide 42

Slide 42 text

IUUQTHJUIVCDPN%F/"EFWGBSN

Slide 43

Slide 43 text

$*ʹࡌ͍ͤͨ΋ͷ EFWGBSNͷ঺հ ӡ༻։࢝࣌఺Ͱͷײ૝

Slide 44

Slide 44 text

EFWGBSNͰͰ͖Δ͜ͱ w ࢦఆͨ͠σόΠεΫϥ΢υ04ػछ  ୆਺ΞϓϦҾ਺ͰΞϓϦΛىಈ w ΞϓϦ͕Ϋϥογϡͨ͠ΒίϚϯυΛ  ҟৗऴྃɺࢦఆ࣌ؒੜଘͨ͠Βਖ਼ৗऴྃ w ݱࡏ͸"84%FWJDF'BSNͷΈͷରԠ

Slide 45

Slide 45 text

͑ɺͦΕ͚ͩʜ

Slide 46

Slide 46 text

ཧ༝͕ʜ͋Γ·͢ʜ

Slide 47

Slide 47 text

EFWGBSNͷࢥ૝ ࣮ػɾγϛϡϨʔλɾσόΠεΫϥ΢υΛ  ໰Θͣɺಉ͡ΠϯλʔϑΣʔεͰ  ىಈɾऴྃ͢Δ͜ͱͷΈʹઐ೦͢Δ ಛఆͷ৚݅Ͱ͔͠αϙʔτͰ͖ͳ͍ػೳ͸  αϙʔτ͠ͳ͍ʢػೳͷڞ௨ू߹ओٛʣ

Slide 48

Slide 48 text

$ devfarm run-ios --os-version 12.0 \  --device 'apple iphone xs' \  --instance-group 'example' \  --platform 'aws-device-farm' \  --ipa 'path/to/app.ipa' \  --args-json '["-ARG1", "HELLO_DEVFARM"]' \  --lifetime-sec 900 & platform status aws-device-farm launching J04ΞϓϦΛ෼࣮ؒߦʢΫϥογϡ͠ͳ͔ͬͨΒਖ਼ৗऴྃʣ

Slide 49

Slide 49 text

$ devfarm status --instance-group 'example'  platform device os state note aws-device-farm apple iphone xs ios ACTIVE ىಈ͍ͯ͠ΔΞϓϦͷ֬ೝ

Slide 50

Slide 50 text

$ devfarm halt --instance-group 'example'  platform status aws-device-farm halting ΞϓϦͷڧ੍ऴྃ

Slide 51

Slide 51 text

instance_groups: : # for Android - platform: # required android: # required device: # required apk: # required app_id: # required intent_extras: [] # optional lifetime_sec: 900 # required $ devfarm run-all planfile.yml ෳ਺ΞϓϦͷҰׅىಈ

Slide 52

Slide 52 text

"84%FWJDF'BSNͰ୆ฒྻ࣮ߦ͍ͯ͠Δ༷ࢠ

Slide 53

Slide 53 text

EFWGBSNͱ͸ ࣮ػͱγϛϡϨʔλͷ੾Γସָ͕͑Ͱɺ  σόΠεΫϥ΢υ্ͰΞϓϦͷىಈঢ়گΛ ؅ཧ͢Δπʔϧ $-*ͳͷͰ$*αʔϏε͔Β࣮ߦ͠΍͍͢

Slide 54

Slide 54 text

ฐࣾͷEFWGBSNར༻ EFWGBSNʹ͸ςετ࣮ߦΛ։࢝͢Δ  ࢓૊ΈΛ༻ҙ͠ͳ͍ํ਑ ઐ༻ʹϏϧυͨ͠ήʔϜΞϓϦͷىಈͱ  ಉ࣌ʹϞϯΩʔςετΛ։͍࢝ͤͯ͞Δ

Slide 55

Slide 55 text

$*ʹࡌ͍ͤͨ΋ͷ EFWGBSNͷ঺հ ӡ༻։࢝௚લ࣌఺Ͱͷײ૝

Slide 56

Slide 56 text

"OESPJEͷηοτΞοϓ͕ෆ҆ఆʜ "84%FWJDF'BSNଆͱͷ੾Γ෼͚͕  Ͱ͖͍ͯͳ͍͕ɺ"OESPJE͕ͷ  ֬཰Ͱ͏·͘ىಈͯ͘͠Εͳ͍ʜ ݱࡏ"84΁໰͍߹Θͤதʜ ࠷ѱπʔϧଆͰSFUSZରԠʹͳΔ͔΋

Slide 57

Slide 57 text

୆ηοτΞοϓʹࣦഊ༷ͨ͠ࢠ

Slide 58

Slide 58 text

ଟ͘ͷσόΠεϑΝʔϜ͕  ಛఆͷϑϨʔϜϫʔΫʹಛԽ͗͢͠ "QQJVN΍9$6*5FTUͳͲͷಛఆͷ  ϑϨʔϜϫʔΫͷ࣮ߦํ๏ʹґଘ͍ͯ͠Δ ࠓճ͸ىಈ؂ࢹऴ͚ྃͩΛ͔͕ͤͨͬͨ͞  ͍Ζ͍Ζͱେมͩͬͨʜ

Slide 59

Slide 59 text

ྫ"84%FWJDF'BSNͰඞཁͳ΋ͷͱҾ਺ͱͷؔ܎

Slide 60

Slide 60 text

Run all by ./test/health-check/planfile.yml: searching to skip creating AWS Device Farm projects finding the AWS Device Farm project: "devfarm-health_check" the AWS Device Farm project was found skipped to create an AWS Device Farm project (because the project already exists) searching AWS Device Farm device the AWS Device Farm device found searching AWS Device Farm device pool to skip creation listing AWS Device Farm device pools found the AWS Device Farm device pool skipping to create AWS Device Farm device pool (because already exists) generating AWS Device Farm custom test spec AWS Device Farm custom test spec was successfully generated uploading AWS Device Farm custom test spec searching AWS Device Farm upload to skip upload listing AWS Device Farm uploads found AWS Device Farm upload found completed AWS Device Farm upload generating AWS Device Farm test package searching AWS Device Farm upload to skip upload listing AWS Device Farm uploads ྫ࣮ߦϩά͔Β࡯ͤΒΕΔେม͞ ·࣮ͩߦ࢝·ͬͯͳ͍ʜ

Slide 61

Slide 61 text

ͦΕͰ΋ɺςετͷ஌໊౓Λ্͛Δ  ઌۦ͚͔ͭීٴͷج൫ͱͳΓ͍ͨ ىಈ؂ࢹऴྃΛඪ४Ͱ҆ఆͯ͠αϙʔτͯ͘͠ΕΔ  σόΠεϑΝʔϜ͕૿͑ΔͨΊʹ͸஌໊౓͕ඞཁ υϝΠϯݻ༗ɾ݁߹ϨΠϠʔɾϞϯΩʔςετ͸  อक༰қͰޮՌతͳςετͳͷͰɺ  ͜ΕΛ͖͔͚ͬʹ޿·ͬͯ͘ΕͨΒخ͍͠

Slide 62

Slide 62 text

·ͱΊ EFWGBSN͸-JOU΍୯ମςετͰ  ݟ͚ͭΒΕͳ͍όάΛอक༰қͳ  ςετͰݟ͚ͭΔͨΊͷ044Ͱ͢

Slide 63

Slide 63 text

'"2ͳͥ"QQJVNΛ࢖Θͳ͍ͷʁ কདྷతʹΑΓݡ͍ΦʔτύΠϩοτʹ͢Δ  Մೳੑ͕͋ͬͯ"QQJVNͩͱ΍ΓͮΒ͍ ىಈ؂ࢹऴྃͷͨΊʹ"QQJVNΛ  ಺෦తʹ࢖͏ࢼΈ΋͕͋ͬͨ  "84%FWJDF'BSN্ͩͱσόοάࠔ೉