クラウド上のモバイル端末・シミュレータを活用した大規模並列テスト / Large Parallel Testing with Real Devices or Simulators on Cloud

151a0b14f5914e786e2e104cfb3a9b2f?s=47 Kuniwak
November 30, 2019

クラウド上のモバイル端末・シミュレータを活用した大規模並列テスト / Large Parallel Testing with Real Devices or Simulators on Cloud

AWS Device Farm やコンテナ化した Android Emulator をクラウド上で複数起動し、並列に自動テストを実行するノウハウと経験則をお話します。また、この自動テストではテストシナリオの保守コストを下げるためにシナリオを必要としないテストエージェントを活用しています。この種のテストエージェントによって、クラッシュやメモリリーク、パフォーマンスイシューを自動で見つける方法を解説します。

システムテスト自動化カンファレンス2019
https://testautomationresearch.connpass.com/event/144768/

151a0b14f5914e786e2e104cfb3a9b2f?s=128

Kuniwak

November 30, 2019
Tweet

Transcript

  1. 12.

    όάൃݟ͸ଟ૚͕جຊ ݁߹ϨΠϠʔ w ߏจϋΠϥΠτ w -JOU࣮ߦ w ੩తܕݕࠪ w ୯ମςετ

    w ݁߹ςετ w &&ςετ ͍Ζ͍Ζͳ૚ʹόάൃݟͷखஈ͕͋ΓɺͲͷ૚Ͱ
 ͲͷόάΛݟ͚ͭΔ͔͕ͱͯ΋ॏཁɿ w ຊ൪؀ڥͰͷ
 ΤϥʔϨϙʔλʔ w ͓໰͍߹Θͤ
  2. 36.

    طଘͷσόΠεϑΝʔϜ w "84%FWJDF'BSN w )FBETQJO w 3FNPUF5FTU,JU w 74"QQ$FOUFS w

    'JSFCBTF5FTU-BC w 4BVDF-BCT
 3FBM%FWJDF$MPVE w 
  3. 48.

    $ devfarm run-ios --os-version 12.0 \
 --device 'apple iphone xs'

    \
 --instance-group 'example' \
 --platform 'aws-device-farm' \
 --ipa 'path/to/app.ipa' \
 --args-json '["-ARG1", "HELLO_DEVFARM"]' \
 --lifetime-sec 900 & platform status aws-device-farm launching J04ΞϓϦΛ෼࣮ؒߦʢΫϥογϡ͠ͳ͔ͬͨΒਖ਼ৗऴྃʣ
  4. 49.

    $ devfarm status --instance-group 'example'
 platform device os state note

    aws-device-farm apple iphone xs ios ACTIVE ىಈ͍ͯ͠ΔΞϓϦͷ֬ೝ
  5. 51.

    instance_groups: <group>: # for Android - platform: <platform> # required

    android: <version> # required device: <decice> # required apk: <filepath> # required app_id: <app_id> # required intent_extras: [] # optional lifetime_sec: 900 # required $ devfarm run-all planfile.yml ෳ਺ΞϓϦͷҰׅىಈ
  6. 60.

    Run all by ./test/health-check/planfile.yml: searching to skip creating AWS Device

    Farm projects finding the AWS Device Farm project: "devfarm-health_check" the AWS Device Farm project was found skipped to create an AWS Device Farm project (because the project already exists) searching AWS Device Farm device the AWS Device Farm device found searching AWS Device Farm device pool to skip creation listing AWS Device Farm device pools found the AWS Device Farm device pool skipping to create AWS Device Farm device pool (because already exists) generating AWS Device Farm custom test spec AWS Device Farm custom test spec was successfully generated uploading AWS Device Farm custom test spec searching AWS Device Farm upload to skip upload listing AWS Device Farm uploads found AWS Device Farm upload found completed AWS Device Farm upload generating AWS Device Farm test package searching AWS Device Farm upload to skip upload listing AWS Device Farm uploads ྫ࣮ߦϩά͔Β࡯ͤΒΕΔେม͞ ·࣮ͩߦ࢝·ͬͯͳ͍ʜ