Slide 1

Slide 1 text

End-to-end functional tests that can run in milliseconds Having Our Cake & Eating It Nat Pryce

Slide 2

Slide 2 text

A familiar story...

Slide 3

Slide 3 text

We needed a new approach Developers responsible for all test automation QA responsible for manual, exploratory & qualitative testing Automated functional tests written in terms of the domain model, not the user interface

Slide 4

Slide 4 text

The Hexagonal architecture AKA Ports & Adaptors HTTP service f Adaptor layer maps from technical domain to application domain, but does no business logic.

Slide 5

Slide 5 text

Invert the Hexagonal architecture HTTP service Test driver f -1 Test f

Slide 6

Slide 6 text

Virtual Machine Virtual Machine Virtual Machine Similarly, at wider scales... Test driver f -1 Test Load balancer f Firewall

Slide 7

Slide 7 text

Service cluster Service cluster Front-end cluster Front-end cluster Service cluster ...and wider Test driver (g∘f) -1 Test Front-end cluster Web browser f g

Slide 8

Slide 8 text

Data centre Data centre Data centre ...and wider Test driver Test f (g∘f) -1 g Web browser CDN

Slide 9

Slide 9 text

Scenario Turn the Screenplay metaphor up to eleven! Actor Role Tasks Domain Model Behind the scenes Visible in test code System under test Production System Interface Plays Acts upon Acts upon Creates Involves Performs Defined in terms of Defines

Slide 10

Slide 10 text

Example @Test fun `An editor can request an amendment`() { val scenario = theProduction.newScenario() val alice = scenario.newAuthor() val ed = scenario.newEditorialStaffMember() val submissionDetails = exampleSubmission.withSingleAuthor(alice) val initialSubmission = alice.hasSubmitted(submissionDetails) val amendment = ed.canRequestAmendmentOf(initialSubmission) alice.canReview(amendment, expectedContents = submissionDetails.withStatus(IN_PROGRESS)) alice.canUpdate(amendment, SetTitle("a new title"), SubmitToPeerReview) ed.canSeeInThePeerReviewSystem(amendment, status = RECEIVED) }

Slide 11

Slide 11 text

Benefits Same test code can run quickly against the domain model, or more slowly against different deployments of the system Tests are quick to write ● Concise ● Type safe ● IDE support: navigation, auto-complete, refactoring, etc. Tests are easy to read ● Concise ● No user-interface and technical details No duplication in test code or between test & production code

Slide 12

Slide 12 text

Some challenges

Slide 13

Slide 13 text

1. Unfamiliar: How to approach TDD? Inside-out? Outside-in?

Slide 14

Slide 14 text

1. How to approach TDD? Inside-out? Outside-in? Both!

Slide 15

Slide 15 text

2. Combinatorial explosion of mappings Add Author Remove Author Set Affiliation Set Title & Abstract Submit to Review Domain Model Service API Browser (with JS) Browser (no JS)

Slide 16

Slide 16 text

2. Apply the Lingua Franca Pattern! Add Author Remove Author Set Affiliation Set Title & Abstract Submit to Review Domain Model Service API Browser (with JS) Browser (no JS) User Actions

Slide 17

Slide 17 text

3. Multiple possible mappings Can map from action to user action in multiple ways ● Different ways to navigate through the user interface ● Different ways to upload files ● Keyboard vs mouse input ● JavaScript vs no JavaScript Approaches: ● Choose randomly on each run ○ A form of property test ● Ensure equivalence with unit tests and exercise only one in the functional tests

Slide 18

Slide 18 text

Q: Why system tests? What about the testing pyramid?

Slide 19

Slide 19 text

Test-drive at the system scale

Slide 20

Slide 20 text

To test the system we need to... know what the system is doing know when it has stopped doing it know when the system has failed explain what has gone wrong restore the system to a good state

Slide 21

Slide 21 text

To test the system we need to... support know what the system is doing know when it has stopped doing it know when the system has failed explain what has gone wrong restore the system to a good state

Slide 22

Slide 22 text

The End