Acceptance testing in the Land of the Startup

Presentation given at Agile 2011 talking about Acceptance testing in Startups. Looking an examples from the startup Songkick.com.

Transcript

1. Acceptance testing
   in the land of the startup
   @josephwilk
   

   View Slide

2. Context Songkick Questions
   

   View Slide

3. Startup:
   “an organisation dedicated to creating
   something new under conditions
   of extreme uncertainty
   

   View Slide

4. Acceptance tests:
   Customer facing tests with each
   acceptance test representing some
   expected result from the system.
   

   View Slide

5. Acceptance tests
   for startups?
   

   View Slide

6. Acceptance tests
   in extreme uncertainty?
   

   View Slide

7. Cynefin model
   Complex
   Complicated
   Chaotic
   Simple
   Complex
   Probe
   Sense
   Respond
   Emergent
   Complicated
   Sense
   Analyze
   Respond
   Good practice
   Chaotic
   Act
   Sense
   Respond
   Novel
   Simple
   Sense
   Categorise
   Respond
   Best Practice
   Disorder
   

   View Slide

8. Cynefin model
   Complex
   Complicated
   Chaotic
   Simple
   Complex
   Probe
   Sense
   Respond
   Emergent
   Complicated
   Sense
   Analyze
   Respond
   Good practice
   Chaotic
   Act
   Sense
   Respond
   Novel
   Simple
   Sense
   Categorise
   Respond
   Best Practice
   Disorder
   

   View Slide

9. Build
   Measure
   Learn
   Measure
   Learn
   Build Measure
   Learn
   Build
   Measure
   Learn
   Build
   

   View Slide

10. Uncertainty
    Complexity Team size
    

    View Slide

11. View Slide

12. View Slide

13. View Slide

14. Team
    

    View Slide

15. Principles
    Embrace
    change
    Testing
    UX obsessed
    

    View Slide

16. Principles
    Embrace
    change
    Testing
    UX obsessed
    

    View Slide

17. Songkick @ work
    

    View Slide

18. We have no idea what
    we are doing
    

    View Slide

19. Really.
    

    View Slide

20. Kanban
    

    View Slide

21. View Slide

22. View Slide

23. Feature
    

    View Slide

24. Facebook signup
    In order to increase signups
    I want visitors to signup
    through their facebook
    accounts
    10% increase in signups
    Measurable
    MMF
    

    View Slide

25. Successful signup through Facebook
    Successful signup through Facebook
    In order to sign up with as little
    effort as possible
    As a non-member
    I want to signup through my facebook
    account
    Failed signup up through Facebook
    Failed signup up through Facebook
    In order to sign with as little
    effort as possible
    As a non-member
    I want to know what I can do to
    correct errors preventing signup
    

    View Slide

26. Where acceptance
    tests are born
    Product
    Owner
    Developer
    QA
    User
    interaction
    

    View Slide

27. Where acceptance
    tests are born
    Product
    Owner
    Developer
    QA
    Just In Time
    Talking
    User
    interaction
    

    View Slide

28. View Slide

29. Pairing
    

    View Slide

30. Pairing
    

    View Slide

31. Value
    Specification
    Examples
    Feature: Visitor Signs up through facebook
    In order to reduce friction in signin up
    As a non-member
    I want to signup through my facebook account
    Allow users to signup through logging into
    there facebook account. A large percentage of
    people browse the web with their facebook
    accounts login.So this should mean most users
    have to make 1 click to signup.
    See mockups: http://of1-dev-songkick.net/
    facebook/mocks
    Scenario: Successful signup through facebook
    Given ...
    When ...
    Then ...
    

    View Slide

32. View Slide

33. Hacking
    

    View Slide

34. Hacking
    

    View Slide

35. Hacking
    

    View Slide

36. View Slide

37. Anyone can
    deploy
    

    View Slide

38. Two way
    mirrors
    

    View Slide

39. Lessons
    

    View Slide

40. Bring on the pain
    

    View Slide

41. Easy Hard
    Full stack Unit test
    Urm...
    

    View Slide

42. Feature Rot
    

    View Slide

43. Duplicating effort
    We
    already have
    tests for that
    

    View Slide

44. A/B/C/D Testing
    @A
    Scenario: example
    @B
    Scenario: example
    @C
    Scenario: example
    @D
    Scenario: example
    

    View Slide

45. Work In Progress
    Limits
    I’m a
    tester too
    

    View Slide

46. Slipping under the UI
    

    View Slide

47. Done. Done. Done
    “A story isn't done until it is being
    used by real users
    in production and has been validated to be a
    useful part of a product.”
    Joshua Kerievsky
    

    View Slide

48. Love Metrics
    

    View Slide

49. View Slide

50. Questions
    

    View Slide

51. Q. Situation with no
    acceptance testing
    

    View Slide

52. Joe steals your
    lessons
    

    View Slide

53. Thanks!
    Joseph Wilk
    @josephwilk
    @josephwilk
    

    View Slide