Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
Automation Tests at Babylon
Ilya Puchka
October 03, 2018
Programming
0
72
Automation Tests at Babylon
Ilya Puchka
October 03, 2018
Tweet
Share
More Decks by Ilya Puchka
See All by Ilya Puchka
Dependency Injection in Swift
ilyapuchka
2
980
Other Decks in Programming
See All in Programming
Language Summit 2022: WebAssembly: Python in the browser and beyond
tiran
2
310
Named Document って何?
harunakano
0
370
UI State Modeling 어떤게 좋을까?
laco2951
1
220
実録mruby組み込み体験
coe401_
0
100
バンドル最適化マニアクス at tfconf
mizchi
3
2.1k
Reactでアプリケーションを構築する多様化
sakito
4
3.3k
WindowsコンテナDojo:第2回 Windowsコンテナアプリのビルド、公開、デプロイ
oniak3ibm
PRO
0
140
Android Architecture Design With Koin
agiuliani
0
230
ebpfとWASMに思いを馳せる2022 / techfeed-conference-2022-ebpf-wasm-amsy810
masayaaoyama
0
610
Managing gRPC with Wire
oldergod
2
150
Unity Localization で多言語対応実装しよう / xrdnk-yokohamaunity-lt10-20220513
xrdnk
0
110
Jetpack Compose 頑張らないPreviewParameterProvider
horie23
0
100
Featured
See All Featured
Ruby is Unlike a Banana
tanoku
91
9.2k
The Art of Programming - Codeland 2020
erikaheidi
31
5.8k
The Power of CSS Pseudo Elements
geoffreycrofte
46
3.9k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
37
3.2k
Embracing the Ebb and Flow
colly
73
3.3k
Debugging Ruby Performance
tmm1
65
10k
Agile that works and the tools we love
rasmusluckow
319
19k
No one is an island. Learnings from fostering a developers community.
thoeni
9
1.1k
How GitHub Uses GitHub to Build GitHub
holman
465
280k
Documentation Writing (for coders)
carmenhchung
48
2.5k
BBQ
matthewcrist
74
7.9k
The Web Native Designer (August 2011)
paulrobertlloyd
74
1.9k
Transcript
Automation Tests at Babylon
What we will talk about: — What issues do we
face in UI tests — How do we solve them — BDD-style UI tests — Tooling
What we will not talk about: — Why — How
— What to test
What issues do we face? — unexpected failures — performance
— scenarios hard to automate & hard to test manually
Content stability
External system actions required — start/complete appointment — subscribe prescription
— answer patient question — etc
Network stability* * https://techcrunch.com/2016/07/13/facebook-lifts- the-veil-on-its-mobile-device-lab
Performance Repeated test steps — register new user — open
Appointments tab — enter your symptoms — book an appointment — add payment method — confirm payment
large tests, faster test suite vs focused tests, slower test
suite
How do we solve that?
Control the World** — Control the network — Control the
initial state of the app ** C. Montgomery Burns' Handbook of World Domination By Matt Groening.
How to control the network control == own data
How to control the network — from inside the app
— from inside the test runner
From inside the app - in-app stubs — straightforward ✅
— integrates in application code — not very flexible ❌
From inside test runner - local web server — harder
to setup ❌ — integrates in test runner ✅ — very flexible ✅
Local server setup In the app: — switch environment
Local server setup In the app: — allow local networking
in the app
Local server setup In the test: — web server implementation
(Ambassador, GCDWebServer, swifter, Kitura, SwiftNIO, ...) — request interception
Local server setup In the test: — web server implementation
(Ambassador, GCDWebServer, swifter, Kitura, SwiftNIO, ...) — request interception
Local server edge cases — Parallel testing — Absolute urls
in responses — Overriding response size
Local web server use cases — Logging — Snapshots —
Full mocks
Network session recording — Record and replay using mock URLSession
(Vinyl, DVR) — detect recording mode — configure matching strategy — create URLSession mock
Network session recording — Record random test input — match
full request — record random body data in a file
Network session recording — Override current date
Mocked backend — mock everything ❌ — a lot of
boilerplate code to write — total control ✅
Network stability Content stability ✅
Tests performance
Tests performance Shortcuts via launch arguments — Special feature toggles
— User defaults overrides — Memory storage
Tests performance API calls from tests — to register a
new user — to add a family member — to book an appointment — etc
BDD-style tests
None
None
Tests building blocks — Page Objects — Step Definers —
Features (XCTestCase)
None
Links Web servers envoy/Ambassador httpswift/swifter swisspol/GCDWebServer Snapshots Velhotes/Vinyl venmo/DVR BDD
tests net-a-porter-mobile/XCTest-Gherkin Ahmed-Ali/Cucumberish Other tools TitouanVanBelle/XCTestHTMLReport