Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Automation Tests at Babylon
Search
Ilya Puchka
October 03, 2018
Programming
0
88
Automation Tests at Babylon
Ilya Puchka
October 03, 2018
Tweet
Share
More Decks by Ilya Puchka
See All by Ilya Puchka
Dependency Injection in Swift
ilyapuchka
2
1.4k
Other Decks in Programming
See All in Programming
見せてあげますよ、「本物のLaravel批判」ってやつを。
77web
4
2.7k
Snowflake x dbtで作るセキュアでアジャイルなデータ基盤
tsoshiro
2
500
JaSST 24 九州:ワークショップ(は除く)実践!マインドマップを活用したソフトウェアテスト+活用事例
satohiroyuki
0
450
アジャイルを支えるテストアーキテクチャ設計/Test Architecting for Agile
goyoki
9
3.1k
AWS Lambdaから始まった Serverlessの「熱」とキャリアパス / It started with AWS Lambda Serverless “fever” and career path
seike460
PRO
0
130
シェーダーで魅せるMapLibreの動的ラスタータイル
satoshi7190
1
440
ActiveSupport::Notifications supporting instrumentation of Rails apps with OpenTelemetry
ymtdzzz
1
180
Macとオーディオ再生 2024/11/02
yusukeito
0
330
PLoP 2024: The evolution of the microservice architecture pattern language
cer
PRO
0
2.4k
OpenTelemetryでRailsのパフォーマンス分析を始めてみよう(KoR2024)
ymtdzzz
5
2k
ふかぼれ!CSSセレクターモジュール / Fukabore! CSS Selectors Module
petamoriken
0
120
レガシーシステムにどう立ち向かうか 複雑さと理想と現実/vs-legacy
suzukihoge
14
2k
Featured
See All Featured
No one is an island. Learnings from fostering a developers community.
thoeni
19
3k
Learning to Love Humans: Emotional Interface Design
aarron
273
40k
We Have a Design System, Now What?
morganepeng
50
7.2k
Why You Should Never Use an ORM
jnunemaker
PRO
54
9k
What’s in a name? Adding method to the madness
productmarketing
PRO
22
3.1k
Typedesign – Prime Four
hannesfritz
40
2.4k
Ruby is Unlike a Banana
tanoku
96
11k
Designing on Purpose - Digital PM Summit 2013
jponch
115
7k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
246
1.3M
Scaling GitHub
holman
458
140k
StorybookのUI Testing Handbookを読んだ
zakiyama
26
5.2k
How STYLIGHT went responsive
nonsquared
95
5.2k
Transcript
Automation Tests at Babylon
What we will talk about: — What issues do we
face in UI tests — How do we solve them — BDD-style UI tests — Tooling
What we will not talk about: — Why — How
— What to test
What issues do we face? — unexpected failures — performance
— scenarios hard to automate & hard to test manually
Content stability
External system actions required — start/complete appointment — subscribe prescription
— answer patient question — etc
Network stability* * https://techcrunch.com/2016/07/13/facebook-lifts- the-veil-on-its-mobile-device-lab
Performance Repeated test steps — register new user — open
Appointments tab — enter your symptoms — book an appointment — add payment method — confirm payment
large tests, faster test suite vs focused tests, slower test
suite
How do we solve that?
Control the World** — Control the network — Control the
initial state of the app ** C. Montgomery Burns' Handbook of World Domination By Matt Groening.
How to control the network control == own data
How to control the network — from inside the app
— from inside the test runner
From inside the app - in-app stubs — straightforward ✅
— integrates in application code — not very flexible ❌
From inside test runner - local web server — harder
to setup ❌ — integrates in test runner ✅ — very flexible ✅
Local server setup In the app: — switch environment
Local server setup In the app: — allow local networking
in the app
Local server setup In the test: — web server implementation
(Ambassador, GCDWebServer, swifter, Kitura, SwiftNIO, ...) — request interception
Local server setup In the test: — web server implementation
(Ambassador, GCDWebServer, swifter, Kitura, SwiftNIO, ...) — request interception
Local server edge cases — Parallel testing — Absolute urls
in responses — Overriding response size
Local web server use cases — Logging — Snapshots —
Full mocks
Network session recording — Record and replay using mock URLSession
(Vinyl, DVR) — detect recording mode — configure matching strategy — create URLSession mock
Network session recording — Record random test input — match
full request — record random body data in a file
Network session recording — Override current date
Mocked backend — mock everything ❌ — a lot of
boilerplate code to write — total control ✅
Network stability Content stability ✅
Tests performance
Tests performance Shortcuts via launch arguments — Special feature toggles
— User defaults overrides — Memory storage
Tests performance API calls from tests — to register a
new user — to add a family member — to book an appointment — etc
BDD-style tests
None
None
Tests building blocks — Page Objects — Step Definers —
Features (XCTestCase)
None
Links Web servers envoy/Ambassador httpswift/swifter swisspol/GCDWebServer Snapshots Velhotes/Vinyl venmo/DVR BDD
tests net-a-porter-mobile/XCTest-Gherkin Ahmed-Ali/Cucumberish Other tools TitouanVanBelle/XCTestHTMLReport