The Secret To Blazing Fast, Rock Solid Mobile Acceptance Tests.

Automated acceptance testing is an essential component of a healthy agile software development process. Unfortunately, attempts to adopt this approach in mobile often result in slow, brittle, and highly complex device tests, based on UI automation. This approach harms confidence in mobile development. Automation used in this way often creates more problems than it solves, leaving development teams and businesses wondering where it all went wrong—often blaming tools rather than their approach. Revisiting fundamentals via a simple example of a mobile app, Paul Stringer helps us re-discover the alternative approach to UI-based testing. We’ll see how the UI approach is flawed. And then we’ll refactor our tests to demonstrate how the majority of business logic can be tested via acceptance tests to be extensive, fast, and reliable. This approach shows not only how to reduce regression cycles from days to minutes but also how we can advance the architecture of mobile software. This improves confidence and allows teams to maintain sustainable progress in an environment of relentless updates and change.

Transcript

1. @paulstringer The Secrets to Blazing-Fast, Rock Solid Mobile Acceptance Testing Starwest, Anaheim, CA 2017.
2. @paulstringer Acceptance Testing - Recap Automated verification Focus on requirements Facilitate collaboration Reduce misunderstandings / assumptions Comprehensible examples Definition of done System level tests slideshare.net/tcmak/atdd-in-practice Cross-function shared understanding
3. @paulstringer TL;DR - Two Kinds of Tests* • Verify your code is correct • Verify you wrote the right code Unit / Functional Acceptance Test or Service Test Search “Uncle Bob Test definitions”
4. @paulstringer Pyramid Definition Programmers Expectations Business Expectations Plumbing Tests Mocks Dependancies System UI ✘ ✘ ✔ ✘ Integration UI ✘ ✘ ✔ ✔ Acceptance Service ✘ ✔ ✘ ✔ Functional Unit ✔ ✘ ✘ ✔ Unit Unit ✔ ✘ ✘ ✔ Micro Unit ✔ ✘ ✘ ✔✔ Test Definitions
5. “One of the weaknesses of TDD as originally described is that it can devolve into a programmer’s technique used to meet a programmer’s needs - with Acceptance Testing there is no ambiguity-this is a technique for enhancing communication with people for whom programming languages are foreign.” Kent Beck Foreword, ATDD by Example
6. @paulstringer “these developers just don’t pay attention to what we ask them to build, it was meant to be so GREAT” “the customer can’t tell us what they want, and they don’t even know what they want!” “terrible app, does anyone even read the reviews?!“
7. User Customer Developer Level of detailed understanding required in software development… ?
8. “Well begun, is half done” Ancient Philosopher Aristotle
9. Mobile Software is Complex
10. Wireframe specifications lack detail
11. @paulstringer Mobile Acceptance Testing Appium Zuchinni UIAutomation Subliminal XCUITest Espresso KIF Frank Calabash
12. @paulstringer UI Automated Acceptance Tests • Fast • Independant • Repeatable • Self Validating • Timely
13. @paulstringer UI Automated Acceptance Tests • Fast • Independant • Repeatable • Self Validating • Timely
14. Throwing robots at a human interface may be the "common sense" approach, but it puts those robots at an immediate disadvantage by forcing them into a human world. Mike Duskis #Starwest Conference 2017
15. @paulstringer martinfowler.com
16. @paulstringer What should be the ‘acceptance criteria’ for our tests? Written by business / QA with sufficient detail Potentially viewed, edited and run by anyone Solid, not brittle (e.g. doesn’t rely on UI) Comprehensive examples and deterministic Rock-solid and blazing fast, running often!
17. " Going back to basics Kent Beck Markus Gärtner Mike Cohn
18. @paulstringer Fully integrated standalone wiki and acceptance testing framework David Parnus Ward Cunningham Micah Martin fitnesse.org Robert Martin
19. “Great software requires collaboration and communication. Fit is a tool for enhancing collaboration in software development. It's an invaluable way to collaborate on complicated problems -and get them right- early in development” Ward Cunningham Framework for Integrated Testing
20. @paulstringer Frameworks for Integrated Test (FIT) • Plan, read, write, execute requirements • Cross platform (Mac, iOS, Ruby, .Net, APIs,) • Useful for teams developing complex software • An alternative UI for your software during testing
21. @paulstringer +# Would it blend? Fully integrated standalone wiki and acceptance testing framework iPhone
22. @paulstringer Anatomy of an acceptance testable mobile app Acceptance Tests <Socket Server> Fixtures Instructions Browser OS Views Controllers System Events Human Responses Updates CI AcceptanceTests.App MyGreat.App #
23. Let’s play a game…
24. [0,1,4,2,8] [ 0 , 1 , 1 , nil, 0, nil, nil, nil, 0 ] INPUT
25. New Human v Human Game TURNS GAME MESSAGE 0,1,4,2,8 PLAYER_ONE_WINS 3,1,4,2,5 PLAYER_ONE_WINS 0,1,3,2,6 PLAYER_ONE_WINS 0,1,2,3,4,6,5,8,7 STALEMATE
26. Blazing-Fast
27. Rock Solid
28. Architecture Using architectural patterns that cleanly separate details such as UI, databases & networks from your core business logic allow independent validation of system behaviour. Change this to a great piece of architecture from California, Wright?
29. @paulstringer Software Architectural Patterns • Clean / Ports and Adaptors • Flux • MVP, MVVM • Lean • Anything S.O.L.I.D.
30. @paulstringer Summary • Fully understanding requirements early saves effort later • UI based testing is inherently slow, fragile and complex • S.O.L.I.D. architecture is an essential ingredient for acceptance testing • Vital for healthy long-term, complex software delivery
31. @paulstringer Which Software is Complex?
32. Creation Deployment Maturity Crazy idea Frenzy Scaling PCs Mobile AR? “Mobile is eating the world” - Ben Evans, Andreeson Horowitz Mobile Software is Complex
33. We’re only just getting started…
34. Thank You Twitter @EqualExperts LinkedIn linkedin.com/company/equal-experts Web www.equalexperts.com Thank You