Acceptance testing in the Land of the Startup

Presentation given at Agile 2011 talking about Acceptance testing in Startups. Looking an examples from the startup Songkick.com.

Transcript

1. Acceptance testing in the land of the startup @josephwilk
2. Context Songkick Questions
3. Startup: “an organisation dedicated to creating something new under conditions of extreme uncertainty
4. Acceptance tests: Customer facing tests with each acceptance test representing some expected result from the system.
5. Acceptance tests for startups?
6. Acceptance tests in extreme uncertainty?
7. Cynefin model Complex Complicated Chaotic Simple Complex Probe Sense Respond Emergent Complicated Sense Analyze Respond Good practice Chaotic Act Sense Respond Novel Simple Sense Categorise Respond Best Practice Disorder
8. Cynefin model Complex Complicated Chaotic Simple Complex Probe Sense Respond Emergent Complicated Sense Analyze Respond Good practice Chaotic Act Sense Respond Novel Simple Sense Categorise Respond Best Practice Disorder
9. Build Measure Learn Measure Learn Build Measure Learn Build Measure Learn Build
10. Uncertainty Complexity Team size
14. Team
15. Principles Embrace change Testing UX obsessed
16. Principles Embrace change Testing UX obsessed
17. Songkick @ work
18. We have no idea what we are doing
19. Really.
20. Kanban
23. Feature
24. Facebook signup In order to increase signups I want visitors to signup through their facebook accounts 10% increase in signups Measurable MMF
25. Successful signup through Facebook Successful signup through Facebook In order to sign up with as little effort as possible As a non-member I want to signup through my facebook account Failed signup up through Facebook Failed signup up through Facebook In order to sign with as little effort as possible As a non-member I want to know what I can do to correct errors preventing signup
26. Where acceptance tests are born Product Owner Developer QA User interaction
27. Where acceptance tests are born Product Owner Developer QA Just In Time Talking User interaction
29. Pairing
30. Pairing
31. Value Specification Examples Feature: Visitor Signs up through facebook In order to reduce friction in signin up As a non-member I want to signup through my facebook account Allow users to signup through logging into there facebook account. A large percentage of people browse the web with their facebook accounts login.So this should mean most users have to make 1 click to signup. See mockups: http://of1-dev-songkick.net/ facebook/mocks Scenario: Successful signup through facebook Given ... When ... Then ...
33. Hacking
34. Hacking
35. Hacking
37. Anyone can deploy
38. Two way mirrors
39. Lessons
40. Bring on the pain
41. Easy Hard Full stack Unit test Urm...
42. Feature Rot
43. Duplicating effort We already have tests for that
44. A/B/C/D Testing @A Scenario: example @B Scenario: example @C Scenario: example @D Scenario: example
45. Work In Progress Limits I’m a tester too
46. Slipping under the UI
47. Done. Done. Done “A story isn't done until it is being used by real users in production and has been validated to be a useful part of a product.” Joshua Kerievsky
48. Love Metrics
50. Questions
51. Q. Situation with no acceptance testing
52. Joe steals your lessons
53. Thanks! Joseph Wilk @josephwilk @josephwilk