Automation Tests at Babylon

Slide 1

Slide 1 text

Slide 2

Slide 2 text

What we will talk about: — What issues do we face in UI tests — How do we solve them — BDD-style UI tests — Tooling

Slide 3

Slide 3 text

What we will not talk about: — Why — How — What to test

Slide 4

Slide 4 text

What issues do we face? — unexpected failures — performance — scenarios hard to automate & hard to test manually

Slide 5

Slide 5 text

Content stability

Slide 6

Slide 6 text

External system actions required — start/complete appointment — subscribe prescription — answer patient question — etc

Slide 7

Slide 7 text

Network stability* * https://techcrunch.com/2016/07/13/facebook-lifts- the-veil-on-its-mobile-device-lab

Slide 8

Slide 8 text

Performance Repeated test steps — register new user — open Appointments tab — enter your symptoms — book an appointment — add payment method — conﬁrm payment

Slide 9

Slide 9 text

large tests, faster test suite vs focused tests, slower test suite

Slide 10

Slide 10 text

How do we solve that?

Slide 11

Slide 11 text

Control the World** — Control the network — Control the initial state of the app ** C. Montgomery Burns' Handbook of World Domination By Matt Groening.

Slide 12

Slide 12 text

How to control the network control == own data

Slide 13

Slide 13 text

How to control the network — from inside the app — from inside the test runner

Slide 14

Slide 14 text

From inside the app - in-app stubs — straightforward ✅ — integrates in application code — not very ﬂexible ❌

Slide 15

Slide 15 text

From inside test runner - local web server — harder to setup ❌ — integrates in test runner ✅ — very ﬂexible ✅

Slide 16

Slide 16 text

Local server setup In the app: — switch environment

Slide 17

Slide 17 text

Local server setup In the app: — allow local networking in the app

Slide 18

Slide 18 text

Local server setup In the test: — web server implementation (Ambassador, GCDWebServer, swifter, Kitura, SwiftNIO, ...) — request interception

Slide 19

Slide 19 text

Local server setup In the test: — web server implementation (Ambassador, GCDWebServer, swifter, Kitura, SwiftNIO, ...) — request interception

Slide 20

Slide 20 text

Local server edge cases — Parallel testing — Absolute urls in responses — Overriding response size

Slide 21

Slide 21 text

Local web server use cases — Logging — Snapshots — Full mocks

Slide 22

Slide 22 text

Network session recording — Record and replay using mock URLSession (Vinyl, DVR) — detect recording mode — conﬁgure matching strategy — create URLSession mock

Slide 23

Slide 23 text

Network session recording — Record random test input — match full request — record random body data in a ﬁle

Slide 24

Slide 24 text

Network session recording — Override current date

Slide 25

Slide 25 text

Mocked backend — mock everything ❌ — a lot of boilerplate code to write — total control ✅

Slide 26

Slide 26 text

Network stability Content stability ✅

Slide 27

Slide 27 text

Tests performance

Slide 28

Slide 28 text

Tests performance Shortcuts via launch arguments — Special feature toggles — User defaults overrides — Memory storage

Slide 29

Slide 29 text

Tests performance API calls from tests — to register a new user — to add a family member — to book an appointment — etc

Slide 30

Slide 30 text

BDD-style tests

Slide 31

Slide 31 text

No content

Slide 32

Slide 32 text

No content

Slide 33

Slide 33 text

Tests building blocks — Page Objects — Step Deﬁners — Features (XCTestCase)

Slide 34

Slide 34 text

No content

Slide 35

Slide 35 text

Links Web servers envoy/Ambassador httpswift/swifter swisspol/GCDWebServer Snapshots Velhotes/Vinyl venmo/DVR BDD tests net-a-porter-mobile/XCTest-Gherkin Ahmed-Ali/Cucumberish Other tools TitouanVanBelle/XCTestHTMLReport