$30 off During Our Annual Pro Sale. View Details »

Having Our Cake and Eating It

Having Our Cake and Eating It

End-to-end functional tests that can run in milliseconds.

Presented at the Agile and Open Source meetup in London.

Nat Pryce

August 09, 2017
Tweet

More Decks by Nat Pryce

Other Decks in Programming

Transcript

  1. End-to-end
    functional tests
    that can run in
    milliseconds
    Having Our Cake & Eating It
    Nat Pryce

    View Slide

  2. A familiar story...

    View Slide

  3. We needed a new approach
    Developers responsible for all
    test automation
    QA responsible for manual,
    exploratory & qualitative testing
    Automated functional tests
    written in terms of the domain
    model, not the user interface

    View Slide

  4. The Hexagonal architecture
    AKA Ports & Adaptors
    HTTP
    service
    f Adaptor layer maps between
    technical domain & application
    domain model, but performs
    no business logic.

    View Slide

  5. The domain model can be tested in isolation
    Test
    In-memory
    storage

    View Slide

  6. Invert the Hexagonal architecture to
    run the same tests at a wider scale
    HTTP
    service
    Test
    driver
    f -1
    Test
    f

    View Slide

  7. Virtual Machine
    Virtual Machine
    Virtual Machine
    Similarly, at yet wider scales...
    Test
    driver
    f -1
    Test
    Load balancer
    f
    Firewall

    View Slide

  8. Service
    cluster
    Service
    cluster
    Front-end
    cluster
    Front-end
    cluster
    Service
    cluster
    ...and wider
    Test
    driver
    (g∘f) -1
    Test
    Front-end
    cluster
    Web
    browser
    f
    g

    View Slide

  9. Data centre
    Data centre
    Data centre
    ...and wider
    Test
    driver
    Test
    f
    (g∘f) -1 g
    Web
    browser
    CDN

    View Slide

  10. Scenario
    Turn the Screenplay metaphor up to eleven!
    Actor Role Tasks
    Domain
    Model
    Behind the
    scenes
    Visible in test code System under test
    Production
    System
    Interface
    Plays
    Acts
    upon
    Acts upon
    Creates
    Creates
    Performs Defined in
    terms of
    Defines

    View Slide

  11. Example
    @Test fun `An editor can request an amendment`() {
    val scenario = theProduction.newScenario()
    val alice = scenario.newAuthor()
    val ed = scenario.newEditorialStaffMember()
    val submissionDetails = exampleSubmission.withSingleAuthor(alice)
    val initialSubmission = alice.hasSubmitted(submissionDetails)
    val amendment = ed.canRequestAmendmentOf(initialSubmission)
    alice.canReview(amendment,
    expectedContents = submissionDetails.withStatus(IN_PROGRESS))
    alice.canUpdate(amendment, SetTitle("a new title"), SubmitToPeerReview)
    ed.canSeeInThePeerReviewSystem(amendment, status = RECEIVED)
    }

    View Slide

  12. Some benefits

    View Slide

  13. Can trade off speed against scale
    The same test code can run...
    ● Quickly against the domain
    model
    ● More slowly against a
    deployed system

    View Slide

  14. Tests are fast to write
    ● Concise
    ● Type safe
    ● Null safe
    ● Play well with the IDE:
    navigation, auto-complete,
    refactoring, etc.

    View Slide

  15. Tests are easy to read
    ● Concise
    ● No user-interface details
    ● No technical details

    View Slide

  16. No duplication
    ● within test code
    ● between test &
    production code

    View Slide

  17. Some challenges

    View Slide

  18. 1. Unfamiliar: How to approach TDD?
    Inside-out?
    Outside-in?

    View Slide

  19. 1. Unfamiliar: How to approach TDD?
    Inside-out?
    Outside-in?
    Both!

    View Slide

  20. 2. Combinatorial explosion of mappings
    Add Author
    Remove Author
    Set Affiliation
    Set Title & Abstract
    Submit to Review
    Domain Model
    Service API
    Browser (with JS)
    Browser (no JS)

    View Slide

  21. 2. Apply the Lingua Franca Pattern!
    Add Author
    Remove Author
    Set Affiliation
    Set Title & Abstract
    Submit to Review
    Domain Model
    Service API
    Browser (with JS)
    Browser (no JS)
    User Actions

    View Slide

  22. 3. Multiple possible mappings
    Can map from action to user action in multiple ways
    ● Different ways to navigate through the user interface
    ● Different ways to upload files
    ● Keyboard vs mouse input
    ● JavaScript vs no JavaScript
    Approaches:
    ● Choose randomly on each run
    ○ A form of property test
    ● Ensure equivalence with unit tests and exercise only one in
    the functional tests

    View Slide

  23. 4. Is it worth the effort?

    View Slide

  24. Q: Why system tests?
    What about the testing pyramid?

    View Slide

  25. The Funnel o' Feedback
    "Developer UX"
    Type checking
    Unit tests
    Connector tests
    Service tests
    System tests
    Manual tests
    Refactor to prefer
    fast feedback and
    exhaustive coverage

    View Slide

  26. Test-drive at the system scale

    View Slide

  27. To test the system we need to...
    know what the system is doing
    know when it has stopped doing it
    know when the system has failed
    explain what has gone wrong
    restore the system to a good state

    View Slide

  28. To test the system we need to...
    support
    know what the system is doing
    know when it has stopped doing it
    know when the system has failed
    explain what has gone wrong
    restore the system to a good state

    View Slide

  29. The End

    View Slide