Having Our Cake and Eating It

End-to-end functional tests that can run in milliseconds.

Presented at the Agile and Open Source meetup in London.

Transcript

1. End-to-end
   functional tests
   that can run in
   milliseconds
   Having Our Cake & Eating It
   Nat Pryce
   

   View Slide

2. A familiar story...
   

   View Slide

3. We needed a new approach
   Developers responsible for all
   test automation
   QA responsible for manual,
   exploratory & qualitative testing
   Automated functional tests
   written in terms of the domain
   model, not the user interface
   

   View Slide

4. The Hexagonal architecture
   AKA Ports & Adaptors
   HTTP
   service
   f Adaptor layer maps between
   technical domain & application
   domain model, but performs
   no business logic.
   

   View Slide

5. The domain model can be tested in isolation
   Test
   In-memory
   storage
   

   View Slide

6. Invert the Hexagonal architecture to
   run the same tests at a wider scale
   HTTP
   service
   Test
   driver
   f -1
   Test
   f
   

   View Slide

7. Virtual Machine
   Virtual Machine
   Virtual Machine
   Similarly, at yet wider scales...
   Test
   driver
   f -1
   Test
   Load balancer
   f
   Firewall
   

   View Slide

8. Service
   cluster
   Service
   cluster
   Front-end
   cluster
   Front-end
   cluster
   Service
   cluster
   ...and wider
   Test
   driver
   (g∘f) -1
   Test
   Front-end
   cluster
   Web
   browser
   f
   g
   

   View Slide

9. Data centre
   Data centre
   Data centre
   ...and wider
   Test
   driver
   Test
   f
   (g∘f) -1 g
   Web
   browser
   CDN
   

   View Slide

10. Scenario
    Turn the Screenplay metaphor up to eleven!
    Actor Role Tasks
    Domain
    Model
    Behind the
    scenes
    Visible in test code System under test
    Production
    System
    Interface
    Plays
    Acts
    upon
    Acts upon
    Creates
    Creates
    Performs Defined in
    terms of
    Defines
    

    View Slide

11. Example
    @Test fun `An editor can request an amendment`() {
    val scenario = theProduction.newScenario()
    val alice = scenario.newAuthor()
    val ed = scenario.newEditorialStaffMember()
    val submissionDetails = exampleSubmission.withSingleAuthor(alice)
    val initialSubmission = alice.hasSubmitted(submissionDetails)
    val amendment = ed.canRequestAmendmentOf(initialSubmission)
    alice.canReview(amendment,
    expectedContents = submissionDetails.withStatus(IN_PROGRESS))
    alice.canUpdate(amendment, SetTitle("a new title"), SubmitToPeerReview)
    ed.canSeeInThePeerReviewSystem(amendment, status = RECEIVED)
    }
    

    View Slide

12. Some benefits
    

    View Slide

13. Can trade off speed against scale
    The same test code can run...
    ● Quickly against the domain
    model
    ● More slowly against a
    deployed system
    

    View Slide

14. Tests are fast to write
    ● Concise
    ● Type safe
    ● Null safe
    ● Play well with the IDE:
    navigation, auto-complete,
    refactoring, etc.
    

    View Slide

15. Tests are easy to read
    ● Concise
    ● No user-interface details
    ● No technical details
    

    View Slide

16. No duplication
    ● within test code
    ● between test &
    production code
    

    View Slide

17. Some challenges
    

    View Slide

18. 1. Unfamiliar: How to approach TDD?
    Inside-out?
    Outside-in?
    

    View Slide

19. 1. Unfamiliar: How to approach TDD?
    Inside-out?
    Outside-in?
    Both!
    

    View Slide

20. 2. Combinatorial explosion of mappings
    Add Author
    Remove Author
    Set Affiliation
    Set Title & Abstract
    Submit to Review
    Domain Model
    Service API
    Browser (with JS)
    Browser (no JS)
    

    View Slide

21. 2. Apply the Lingua Franca Pattern!
    Add Author
    Remove Author
    Set Affiliation
    Set Title & Abstract
    Submit to Review
    Domain Model
    Service API
    Browser (with JS)
    Browser (no JS)
    User Actions
    

    View Slide

22. 3. Multiple possible mappings
    Can map from action to user action in multiple ways
    ● Different ways to navigate through the user interface
    ● Different ways to upload files
    ● Keyboard vs mouse input
    ● JavaScript vs no JavaScript
    Approaches:
    ● Choose randomly on each run
    ○ A form of property test
    ● Ensure equivalence with unit tests and exercise only one in
    the functional tests
    

    View Slide

23. 4. Is it worth the effort?
    

    View Slide

24. Q: Why system tests?
    What about the testing pyramid?
    

    View Slide

25. The Funnel o' Feedback
    "Developer UX"
    Type checking
    Unit tests
    Connector tests
    Service tests
    System tests
    Manual tests
    Refactor to prefer
    fast feedback and
    exhaustive coverage
    

    View Slide

26. Test-drive at the system scale
    

    View Slide

27. To test the system we need to...
    know what the system is doing
    know when it has stopped doing it
    know when the system has failed
    explain what has gone wrong
    restore the system to a good state
    

    View Slide

28. To test the system we need to...
    support
    know what the system is doing
    know when it has stopped doing it
    know when the system has failed
    explain what has gone wrong
    restore the system to a good state
    

    View Slide

29. The End
    

    View Slide