Upgrade to Pro — share decks privately, control downloads, hide ads and more …

DevTalks: Whose Test Is It Anyway

DevTalks: Whose Test Is It Anyway

This talk is one where I am learning the core message of the talk in clarity by delivering more of it. Dropping the things I layered for "clarity" that turned distraction, now it works as intended.

Whose Test Is It Anyway discusses - with an example of testing when large language model ("AI") is your programmer coming to three conclusions:
- We no longer care whose test it is because AI makes us all testers
- We do our best work when we get people from diverse backgrounds together and source and combine their ideas like I did for having an example for properly testing roman numerals
- Improv in exploratory testing is a great frame for putting together testing, ensembling and programming. Contemporary exploratory testing is not "non-automated whole product exploratory-focused" as some people framed exploratory testing of the past but very much "automated, all levels building up to whole products exploratory-focused" testing.

Maaret Pyhäjärvi

June 26, 2023
Tweet

More Decks by Maaret Pyhäjärvi

Other Decks in Programming

Transcript

  1. @maaretp
    @[email protected]
    by Maaret Pyhäjärvi
    Whose Test Is It
    Anyway?

    View Slide

  2. @maaretp
    @[email protected]
    Social
    Software
    Testing
    Approaches
    Exploratory
    Testing
    Programmatic
    Tests

    View Slide

  3. @maaretp
    @[email protected]
    This is a test.
    Whose test is it?

    View Slide

  4. @maaretp
    @[email protected]
    Improv
    performance
    External
    Imagination

    View Slide

  5. @maaretp
    @[email protected]
    Computer Assisted
    Software Authorship https://github.com/features/copilot/

    View Slide

  6. @maaretp
    @[email protected]
    WE are
    accountable
    1. Legal / Ethical
    2. Intent / Implementation
    3. Domain for the Layman
    4. Domain for the Expert
    5. Reference Implementation
    6. People Filtering
    7. Interesting side effects

    View Slide

  7. @maaretp
    @[email protected]
    Guessing with power to accept
    For Mood:
    https://aboutmonica.com/blog/how-to-make-your-vs-code-sparkle/

    View Slide

  8. @maaretp
    @[email protected]
    Who Am I?
    Guessing with power to accept
    What am I testing again? My
    program or the tool?

    View Slide

  9. @maaretp
    @[email protected]
    Note:
    …pillaging digital content
    without consent, compensation
    and attribution
    Even if legal not ethical.

    View Slide

  10. @maaretp
    @[email protected]
    CTRL+enter for alternatives
    Code Review

    View Slide

  11. @maaretp
    @[email protected]
    Say We Have This…
    Done?

    View Slide

  12. @maaretp
    @[email protected]
    Some Tests
    Done?

    View Slide

  13. @maaretp
    @[email protected]
    Shapes of Approaches
    Asserts
    Approvals

    View Slide

  14. @maaretp
    @[email protected]
    From One to Many
    Asserts
    Approvals

    View Slide

  15. @maaretp
    @[email protected]
    Hypothesis

    View Slide

  16. @maaretp
    @[email protected]
    Note:
    …developer intent. It might
    not do what you wanted it to
    do.

    View Slide

  17. @maaretp
    @[email protected]
    Part I. Review for correctness and conciseness
    Part II. Input -> Output
    Part III. Rules of behavior boundaries
    Part IIII. Coverage
    Part IIIII. Sampling vs wide nets (approvals)
    Part VI. Properties
    Developer intent

    View Slide

  18. @maaretp
    @[email protected]
    Domain rules: 1V à IIII in clock design as
    per orders of King Louis XIV of France
    https://www.amalgamsab.com/iiii-or-is-it-iv.html

    View Slide

  19. @maaretp
    @[email protected]
    https://en.wikipedia.org/wiki/Roman_numerals

    View Slide

  20. View Slide

  21. @maaretp
    @[email protected]
    Upper Boundary?

    View Slide

  22. View Slide

  23. @maaretp
    @[email protected]
    References…
    References…

    View Slide

  24. @maaretp
    @[email protected]
    References…

    View Slide

  25. @maaretp
    @[email protected]
    References…

    View Slide

  26. @maaretp
    @[email protected]
    ApprovalTests
    Done?

    View Slide

  27. @maaretp
    @[email protected]
    Note:
    …business rules. You think you
    know them and yet you don’t.

    View Slide

  28. @maaretp
    @[email protected]
    Part I. Be the resident expert. Ask around.
    Part II. Rules. More rules.
    Part III. Find better experts.
    Part IIII. Disagreeing with boundaries.
    Part IIIII. Oracles.
    Part VI. Find better oracles.
    Part VII. No user would do what users would do.
    Domain

    View Slide

  29. @maaretp
    @[email protected]
    Part I. Dependencies.
    Part II. Interruptions. Both software and hardware.
    Part III. People.
    Part IIII. Scale. Reliability. Performance. Security.
    Environment
    “People are not pure functions; they have all sorts of
    interesting side effects.” - Engineering Management for the
    Rest of Us, Sarah Drasner
    … nor are pure functions if you grow the boundary of what might fail.

    View Slide

  30. @maaretp
    @[email protected]
    WE are
    accountable
    1. Legal / Ethical
    2. Intent / Implementation
    3. Domain for the Layman
    4. Domain for the Expert
    5. Reference Implementation
    6. People Filtering
    7. Interesting side effects

    View Slide

  31. @maaretp
    @[email protected]
    A majority of the
    production failures (77%)
    can be reproduced by a
    unit test.
    https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-yuan.pdf
    Through https://www.slideshare.net/Kevlin/the-error-of-our-ways

    View Slide

  32. @maaretp
    @[email protected]
    Everything that does not
    need to be automated gets
    done while automating.
    * Programming with thinking – quality of thinking matters

    View Slide

  33. @maaretp
    @[email protected]
    Stakeholders happy,
    even delighted
    –Quality Information
    Good Team’s Output
    –Quality Information
    Less than Good
    Team’s Output
    –Quality Information
    Results Gap
    Surprise!
    Results Gap
    on a Team that thinks
    Testers == Testing
    Pick up the pizza boxes…
    ”Find (some of) What Others May Have Missed”

    View Slide

  34. @maaretp
    @[email protected]
    How Would You Test This?
    https://todomvc.com/examples/vanillajs/
    Visual Functional Purpose?

    View Slide

  35. @maaretp
    @[email protected]
    No Todos
    ✓ should hide #main and #footer
    New Todo
    ✓ should allow me to add todo items
    ✓ should clear text input field when an item is added
    ✓ should trim text input
    ✓ should show #main and #footer when items added
    Mark all as completed
    ✓ should allow me to mark all items as completed
    ✓ should allow me to clear the completion state of all items
    ✓ complete all checkbox should update state when items are completed
    Item
    ✓ should allow me to mark items as complete
    ✓ should allow me to un-mark items as complete
    ✓ should allow me to edit an item
    ✓ should show the remove button on hover
    Editing
    ✓ should hide other controls when editing
    ✓ should save edits on enter
    ✓ should save edits on blur
    ✓ should trim entered text
    ✓ should remove the item if an empty text string was entered
    ✓ should cancel edits on escape
    Counter
    ✓ should display the current number of todo items
    Clear completed button
    ✓ should display the number of completed items
    ✓ should remove completed items when clicked
    ✓ should be hidden when there are no items that are completed
    Persistence
    ✓ should persist its data
    Routing
    ✓ should allow me to display active items
    ✓ should allow me to display completed items
    ✓ should allow me to display all items
    ✓ should highlight the currently applied filter
    Developer
    Intent

    View Slide

  36. @maaretp
    @[email protected]
    Driver (Hands)
    Designated
    Navigator (Voice)
    Navigators (Brains)

    View Slide

  37. @maaretp
    @[email protected]
    Testing is too important to
    be left for testers. Testing is
    too important to be left
    without testers.

    View Slide

  38. @maaretp
    @[email protected]
    Agency.
    Pull.
    * sense of control
    * timely for use

    View Slide

  39. @maaretp
    @[email protected]
    Observe Ideate
    Let Go
    Be
    Impressed
    Support
    others
    Do
    Something
    Yes,
    and…
    1. Observe: notice more
    2. Ideate: stop censuring
    3. Let go: trust the process
    4. Be Impressed: react
    5. Support others: make them shine
    6. Do something: experiment culture
    7. Yes, and: combine
    * Metaskills of Improv, Simo Routarinne, proimpro.fi

    View Slide

  40. @maaretp
    @[email protected]
    2020 2016
    MIATPP
    Most Influential Agile Testing
    Professional Person
    #PayToSpeak #TechVoices
    #EnsembleTesting #EnsembleProgramming #StrongStylePairing
    #ExploratoryTesting #TestAutomation
    #ModernAgile
    #AwesomeTesters
    Maaret Pyhäjärvi
    (from Finland)
    Email: [email protected]
    Twitter: @maaretp
    Web: maaretp.com
    Blog: visible-quality.blogspot.fi
    (please connect with me through
    Twitter or LinkedIn)
    2019 - 2022
    https://exploratorytestingacademy.com
    Ohjelmistotestaus ry
    https://techvoices.org

    View Slide