Save 37% off PRO during our Black Friday Sale! »

Having Our Cake and Eating It

5358df52bd2ef4f57da1b1cc8634cfd9?s=47 Nat Pryce
August 09, 2017

Having Our Cake and Eating It

End-to-end functional tests that can run in milliseconds.

Presented at the Agile and Open Source meetup in London.

5358df52bd2ef4f57da1b1cc8634cfd9?s=128

Nat Pryce

August 09, 2017
Tweet

Transcript

  1. End-to-end functional tests that can run in milliseconds Having Our

    Cake & Eating It Nat Pryce
  2. A familiar story...

  3. We needed a new approach Developers responsible for all test

    automation QA responsible for manual, exploratory & qualitative testing Automated functional tests written in terms of the domain model, not the user interface
  4. The Hexagonal architecture AKA Ports & Adaptors HTTP service f

    Adaptor layer maps between technical domain & application domain model, but performs no business logic.
  5. The domain model can be tested in isolation Test In-memory

    storage
  6. Invert the Hexagonal architecture to run the same tests at

    a wider scale HTTP service Test driver f -1 Test f
  7. Virtual Machine Virtual Machine Virtual Machine Similarly, at yet wider

    scales... Test driver f -1 Test Load balancer f Firewall
  8. Service cluster Service cluster Front-end cluster Front-end cluster Service cluster

    ...and wider Test driver (g∘f) -1 Test Front-end cluster Web browser f g
  9. Data centre Data centre Data centre ...and wider Test driver

    Test f (g∘f) -1 g Web browser CDN
  10. Scenario Turn the Screenplay metaphor up to eleven! Actor Role

    Tasks Domain Model Behind the scenes Visible in test code System under test Production System Interface Plays Acts upon Acts upon Creates Creates Performs Defined in terms of Defines
  11. Example @Test fun `An editor can request an amendment`() {

    val scenario = theProduction.newScenario() val alice = scenario.newAuthor() val ed = scenario.newEditorialStaffMember() val submissionDetails = exampleSubmission.withSingleAuthor(alice) val initialSubmission = alice.hasSubmitted(submissionDetails) val amendment = ed.canRequestAmendmentOf(initialSubmission) alice.canReview(amendment, expectedContents = submissionDetails.withStatus(IN_PROGRESS)) alice.canUpdate(amendment, SetTitle("a new title"), SubmitToPeerReview) ed.canSeeInThePeerReviewSystem(amendment, status = RECEIVED) }
  12. Some benefits

  13. Can trade off speed against scale The same test code

    can run... • Quickly against the domain model • More slowly against a deployed system
  14. Tests are fast to write • Concise • Type safe

    • Null safe • Play well with the IDE: navigation, auto-complete, refactoring, etc.
  15. Tests are easy to read • Concise • No user-interface

    details • No technical details
  16. No duplication • within test code • between test &

    production code
  17. Some challenges

  18. 1. Unfamiliar: How to approach TDD? Inside-out? Outside-in?

  19. 1. Unfamiliar: How to approach TDD? Inside-out? Outside-in? Both!

  20. 2. Combinatorial explosion of mappings Add Author Remove Author Set

    Affiliation Set Title & Abstract Submit to Review Domain Model Service API Browser (with JS) Browser (no JS)
  21. 2. Apply the Lingua Franca Pattern! Add Author Remove Author

    Set Affiliation Set Title & Abstract Submit to Review Domain Model Service API Browser (with JS) Browser (no JS) User Actions
  22. 3. Multiple possible mappings Can map from action to user

    action in multiple ways • Different ways to navigate through the user interface • Different ways to upload files • Keyboard vs mouse input • JavaScript vs no JavaScript Approaches: • Choose randomly on each run ◦ A form of property test • Ensure equivalence with unit tests and exercise only one in the functional tests
  23. 4. Is it worth the effort?

  24. Q: Why system tests? What about the testing pyramid?

  25. The Funnel o' Feedback "Developer UX" Type checking Unit tests

    Connector tests Service tests System tests Manual tests Refactor to prefer fast feedback and exhaustive coverage
  26. Test-drive at the system scale

  27. To test the system we need to... know what the

    system is doing know when it has stopped doing it know when the system has failed explain what has gone wrong restore the system to a good state
  28. To test the system we need to... support know what

    the system is doing know when it has stopped doing it know when the system has failed explain what has gone wrong restore the system to a good state
  29. The End