Automation Tests at Babylon

Transcript

1. Automation Tests at Babylon
2. What we will talk about: — What issues do we face in UI tests — How do we solve them — BDD-style UI tests — Tooling
3. What we will not talk about: — Why — How — What to test
4. What issues do we face? — unexpected failures — performance — scenarios hard to automate & hard to test manually
5. Content stability
6. External system actions required — start/complete appointment — subscribe prescription — answer patient question — etc
7. Network stability* * https://techcrunch.com/2016/07/13/facebook-lifts- the-veil-on-its-mobile-device-lab
8. Performance Repeated test steps — register new user — open Appointments tab — enter your symptoms — book an appointment — add payment method — confirm payment
9. large tests, faster test suite vs focused tests, slower test suite
10. How do we solve that?
11. Control the World** — Control the network — Control the initial state of the app ** C. Montgomery Burns' Handbook of World Domination By Matt Groening.
12. How to control the network control == own data
13. How to control the network — from inside the app — from inside the test runner
14. From inside the app - in-app stubs — straightforward ✅ — integrates in application code — not very flexible ❌
15. From inside test runner - local web server — harder to setup ❌ — integrates in test runner ✅ — very flexible ✅
16. Local server setup In the app: — switch environment
17. Local server setup In the app: — allow local networking in the app
18. Local server setup In the test: — web server implementation (Ambassador, GCDWebServer, swifter, Kitura, SwiftNIO, ...) — request interception
19. Local server setup In the test: — web server implementation (Ambassador, GCDWebServer, swifter, Kitura, SwiftNIO, ...) — request interception
20. Local server edge cases — Parallel testing — Absolute urls in responses — Overriding response size
21. Local web server use cases — Logging — Snapshots — Full mocks
22. Network session recording — Record and replay using mock URLSession (Vinyl, DVR) — detect recording mode — configure matching strategy — create URLSession mock
23. Network session recording — Record random test input — match full request — record random body data in a file
24. Network session recording — Override current date
25. Mocked backend — mock everything ❌ — a lot of boilerplate code to write — total control ✅
26. Network stability Content stability ✅
27. Tests performance
28. Tests performance Shortcuts via launch arguments — Special feature toggles — User defaults overrides — Memory storage
29. Tests performance API calls from tests — to register a new user — to add a family member — to book an appointment — etc
30. BDD-style tests
33. Tests building blocks — Page Objects — Step Definers — Features (XCTestCase)
35. Links Web servers envoy/Ambassador httpswift/swifter swisspol/GCDWebServer Snapshots Velhotes/Vinyl venmo/DVR BDD tests net-a-porter-mobile/XCTest-Gherkin Ahmed-Ali/Cucumberish Other tools TitouanVanBelle/XCTestHTMLReport