Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Automation Tests at Babylon

Automation Tests at Babylon

9763301caabd158f2f22c5a8e323e32a?s=128

Ilya Puchka

October 03, 2018
Tweet

More Decks by Ilya Puchka

Other Decks in Programming

Transcript

  1. Automation Tests at Babylon

  2. What we will talk about: — What issues do we

    face in UI tests — How do we solve them — BDD-style UI tests — Tooling
  3. What we will not talk about: — Why — How

    — What to test
  4. What issues do we face? — unexpected failures — performance

    — scenarios hard to automate & hard to test manually
  5. Content stability

  6. External system actions required — start/complete appointment — subscribe prescription

    — answer patient question — etc
  7. Network stability* * https://techcrunch.com/2016/07/13/facebook-lifts- the-veil-on-its-mobile-device-lab

  8. Performance Repeated test steps — register new user — open

    Appointments tab — enter your symptoms — book an appointment — add payment method — confirm payment
  9. large tests, faster test suite vs focused tests, slower test

    suite
  10. How do we solve that?

  11. Control the World** — Control the network — Control the

    initial state of the app ** C. Montgomery Burns' Handbook of World Domination By Matt Groening.
  12. How to control the network control == own data

  13. How to control the network — from inside the app

    — from inside the test runner
  14. From inside the app - in-app stubs — straightforward ✅

    — integrates in application code — not very flexible ❌
  15. From inside test runner - local web server — harder

    to setup ❌ — integrates in test runner ✅ — very flexible ✅
  16. Local server setup In the app: — switch environment

  17. Local server setup In the app: — allow local networking

    in the app
  18. Local server setup In the test: — web server implementation

    (Ambassador, GCDWebServer, swifter, Kitura, SwiftNIO, ...) — request interception
  19. Local server setup In the test: — web server implementation

    (Ambassador, GCDWebServer, swifter, Kitura, SwiftNIO, ...) — request interception
  20. Local server edge cases — Parallel testing — Absolute urls

    in responses — Overriding response size
  21. Local web server use cases — Logging — Snapshots —

    Full mocks
  22. Network session recording — Record and replay using mock URLSession

    (Vinyl, DVR) — detect recording mode — configure matching strategy — create URLSession mock
  23. Network session recording — Record random test input — match

    full request — record random body data in a file
  24. Network session recording — Override current date

  25. Mocked backend — mock everything ❌ — a lot of

    boilerplate code to write — total control ✅
  26. Network stability Content stability ✅

  27. Tests performance

  28. Tests performance Shortcuts via launch arguments — Special feature toggles

    — User defaults overrides — Memory storage
  29. Tests performance API calls from tests — to register a

    new user — to add a family member — to book an appointment — etc
  30. BDD-style tests

  31. None
  32. None
  33. Tests building blocks — Page Objects — Step Definers —

    Features (XCTestCase)
  34. None
  35. Links Web servers envoy/Ambassador httpswift/swifter swisspol/GCDWebServer Snapshots Velhotes/Vinyl venmo/DVR BDD

    tests net-a-porter-mobile/XCTest-Gherkin Ahmed-Ali/Cucumberish Other tools TitouanVanBelle/XCTestHTMLReport