[Talk] Let's Do a Thing and Call It Foo 2023 ed.

Let’s take computer assisted software authorship and consider ourselves held accountable for results. We do a thing of testing, and call it Foo until we have a shared experience.

Foo is what allows us to be accountable, and we learn that there are 5 layers of oracles we can choose to apply, and from our example can take ideas of how to apply those further.

Transcript

1. @maaretp
   @maaretp@mas.to
   by Maaret Pyhäjärvi
   Let’s Do A Thing
   and Call it Foo
   

   View Slide

2. @maaretp
   @maaretp@mas.to
   The terms foobar (/ˈfuːbɑːr/), foo, bar, baz, and others are
   used as metasyntactic variables and placeholder
   names in computer programming or computer-related
   documentation.[1] They have been used to name entities
   such as variables, functions, and commands whose
   exact identity is unimportant and serve
   only to demonstrate a concept.
   https://en.wikipedia.org/wiki/Foobar
   

   View Slide

3. @maaretp
   @maaretp@mas.to
   Stakeholders happy,
   even delighted
   –Quality Information
   Good Team’s Output
   –Quality Information
   Less than Good
   Team’s Output
   –Quality Information
   Results Gap
   Surprise!
   Results Gap
   on a Team that thinks
   Testers == Testing
   Pick up the pizza boxes…
   ”Find (some of) What Others May Have Missed”
   

   View Slide

4. @maaretp
   @maaretp@mas.to
   Computer Assisted
   Software Authorship https://github.com/features/copilot/
   

   View Slide

5. @maaretp
   @maaretp@mas.to
   CTRL+enter for alternatives
   

   View Slide

6. @maaretp
   @maaretp@mas.to
   CTRL+enter for alternatives
   

   View Slide

7. @maaretp
   @maaretp@mas.to
   WE are
   accountable
   1. Intent / Implementation
   2. Domain for the Layman
   3. Domain for the Expert
   4. Reference Implementation
   5. People Filtering
   

   View Slide

8. @maaretp
   @maaretp@mas.to
   Say We Have This…
   Done?
   

   View Slide

9. @maaretp
   @maaretp@mas.to
   Shapes of Approaches
   Asserts
   Approvals
   

   View Slide

10. @maaretp
    @maaretp@mas.to
    From One to Many
    Asserts
    Approvals
    

    View Slide

11. @maaretp
    @maaretp@mas.to
    Domain rules: 1V à IIII in clock design as
    per orders of King Louis XIV of France
    https://www.amalgamsab.com/iiii-or-is-it-iv.html
    

    View Slide

12. @maaretp
    @maaretp@mas.to
    Some Tests
    Done?
    

    View Slide

13. @maaretp
    @maaretp@mas.to
    https://en.wikipedia.org/wiki/Roman_numerals
    

    View Slide

14. @maaretp
    @maaretp@mas.to
    

    View Slide

15. @maaretp
    @maaretp@mas.to
    Upper Boundary?
    

    View Slide

16. @maaretp
    @maaretp@mas.to
    

    View Slide

17. @maaretp
    @maaretp@mas.to
    References…
    

    View Slide

18. @maaretp
    @maaretp@mas.to
    References…
    

    View Slide

19. @maaretp
    @maaretp@mas.to
    References…
    Done?
    

    View Slide

20. @maaretp
    @maaretp@mas.to
    ApprovalTests
    

    View Slide

21. @maaretp
    @maaretp@mas.to
    Hypothesis
    

    View Slide

22. @maaretp
    @maaretp@mas.to
    Done.
    

    View Slide

23. @maaretp
    @maaretp@mas.to
    Answer Key to Some of the Bugs
    • Values over 4k don’t work (without extending to line-on-top notation)
    – Zeros, fractions, large numbers later in scope
    • Decimal values truncated vs. rounded
    • *some of the implementations*
    – Boundary values miscalculated
    – Infinite loop
    • Implementing Classic when expecting Simplified
    • Got someone’s code without license text!
    

    View Slide

24. @maaretp
    @maaretp@mas.to
    Insights from 8 rounds of pair/ensemble testing
    • Unit Exploratory Testing is FAST
    • Concepts: Parametrized tests, pytest.raises, approvaltests
    • Selecting vs. oversampling for finding boundaries
    • Choosing later what is left behind (if dropping tests is worth
    the investment)
    • Both valid and invalid values, balance and role-based heuristics
    • Fooling a domain expert on conflicting domain rules
    surprising how versatile a simple appearing domain is
    

    View Slide

25. @maaretp
    @maaretp@mas.to
    Foo
    Contemporary
    Exploratory Testing
    

    View Slide

26. @maaretp
    @maaretp@mas.to
    A majority of the
    production failures (77%)
    can be reproduced by a
    unit test.
    https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-yuan.pdf
    Through https://www.slideshare.net/Kevlin/the-error-of-our-ways
    

    View Slide

27. @maaretp
    @maaretp@mas.to
    

    View Slide

28. @maaretp
    @maaretp@mas.to
    Everything that does not
    need to be automated gets
    done while automating.
    * Programming with thinking – quality of thinking matters
    

    View Slide

29. @maaretp
    @maaretp@mas.to
    WE are
    accountable
    1. Intent / Implementation
    2. Domain for the Layman
    3. Domain for the Expert
    4. Reference Implementation
    5. People Filtering
    

    View Slide

30. @maaretp
    @maaretp@mas.to
    Find (some of) what
    others may have missed!
    Testing
    … and never be bored.
    

    View Slide

31. @maaretp
    @maaretp@mas.to
    For pair / ensemble testing this with me <3
    Thank You
    Alex, Elisabeth, Bertold, Thomas,
    Mirja, Karen, Moroni, Isidro,
    Benjamin & Testing Dozen #1
    

    View Slide

32. @maaretp
    @maaretp@mas.to
    2020 2016
    MIATPP
    Most Influential Agile Testing
    Professional Person
    #PayToSpeak #TechVoices
    #EnsembleTesting #EnsembleProgramming #StrongStylePairing
    #ExploratoryTesting #TestAutomation
    #ModernAgile
    #AwesomeTesters
    Maaret Pyhäjärvi
    (from Finland)
    Email: maaret@iki.fi
    Twitter: @maaretp
    Web: maaretp.com
    Blog: visible-quality.blogspot.fi
    (please connect with me through
    Twitter or LinkedIn)
    2019 - 2022
    https://exploratorytestingacademy.com
    Ohjelmistotestaus ry
    https://techvoices.org
    

    View Slide