• DeNA’s new service (released in 2016) • Video game platform for PCs. • Runs on Windows • The games are made of Unity, Unreal Engine, etc. • AndApp client is made with Chromium What is AndApp? http://www.andapp.jp/
1. The user starts the game app (Unity) 2. He/she pushes the purchase button on the game app 3. Credit card form window (AndApp client) opens 4. He/she inputs credit card number on the AndApp client Control the Unity app Control the AndApp client How users purchase an item
Problem about controlling the Unity app • Appium does not offer any way to identify an element. • You can click the point at (x:100, y:200) using Appium, but you cannot click the button by specifying the ID. • (Appium fork, Selenium-Unity-Driver offers the way to identify element on Unity app, but it is not available for Windows apps.) Click the button with id: “ok” ! Click the button at the point of (x: 100, y: 200) ! ⭕ ❌
Confusion between two apps • You cannot identify an element on the Unity app. • But you can identify an element on the Chromium based app Controlling different apps is too confusing!
Complicated procedures • We have to control multiple apps at the same time • AndApp client has multiple windows • Multiple window multiple app testing is more difficult than browser testing
Screenplay pattern •A design pattern for E2E tests using Page, Actor, Ability, Task, Action, and Question. •Useful for complicated web app tests •I learned a lot from screenplay pattern! •We did not adopt whole the features
We adopted some parts of screenplay pattern • Our framework uses Task, Action, and Page. • Our framework does not use Actor, Ability, and Question because they are not suitable for our test cases. • Our framework uses Operation that the screenplay pattern does not use. • We used RSpec, but other testing frameworks are also okay Task Action Page Task is made up of actions Action interacts with a page Operation Operation is made up of tasks
Unstable tests When AndApp client dev team changed the client’s behavior, payment dev team’s test fails. Payment dev team AndApp client dev team We changed the behaviour of AndApp client. Hey, our test failed because of other team’s development! AndApp is developed under multiple teams, so it is almost impossible to follow the changes made by other teams.
When a test fails… We have to identify the reason and fix it. However, identifying the reason of the failure is not an easy task We started recording AndApp client’s and the Unity app’s logs, but it was not very useful enough to identify the reason.
Solution for identifying the reasons for the failures We started recording videos that capture the screen of the Windows machine From the video, I found that AndApp clinent’s UI has been changed and this is the reason for the failure!
1. Jenkins job starts 2. It starts recording when each test case begins, it finishes recording when the test case finishes. 3. After the tests, it uploads all the video files to the shared storage 4. It notify us with Slack
Sum up • We Used Selenium for the Chromium based app, Appium for the Unity app • We implemented the same interface methods using polymorphism • We developed the test framework adopting some features from screenplay pattern • We started recording the video to identify the reasons for failures
katryo
atinfinity
h0r15h0
zakky21
clown0082
aridlehoover
koic
texmeijin
tsutsui
codemountains
chinen
bells17
fadis
eileencodes
addyosmani
michaelherold
notwaldorf
jasonvnalue
csswizardry
lara
gr2m
kevinliebholz
jcasabona
bermonpainter
vanstee