Acceptance Testing: The DIrty Details

Presented at CodeMash 2.0.1.4

Web based acceptance tests are quite valuable, and you should absolutely be writing them. However, there are many different ways that acceptance tests can be written, and choosing the right ones can mean the difference between success and failure for your project. This talk will get into the dirty details of dealing with AJAX, managing test data, keeping tests isolated, and working with your customer or Business Analyst. The result will be cleaner tests that run faster and are easier to write.

Transcript

1. Acceptance Testing
   The Dirty Details
   

   View Slide

2. Kevin Berridge
   @kberridge
   kevinberridge.com
   

   View Slide

3. The Dirty Details
   Browser Automation Testing
   

   View Slide

4. The Dirty Details
   ✤ What and Why!
   ✤ Process!
   ✤ Tool Choice!
   ✤ Choosing Tests!
   ✤ Test Data!
   ✤ Ajax!
   ✤ Test Tips and Tricks
   

   View Slide

5. What
   is Acceptance Testing?
   

   View Slide

6. What is Acceptance Testing?
   Client Developer
   

   View Slide

7. What is Acceptance Testing?
   Full Stack
   UI
   Web
   Code
   Database
   

   View Slide

8. What is Acceptance Testing?
   “As much realism as we can afford”
   -Justin Searls
   

   View Slide

9. What is Acceptance Testing?
   Test Browser Automation! Browser
   

   View Slide

10. View Slide

11. Why
    write Acceptance Tests?
    

    View Slide

12. Why write Acceptance Tests?
    ✤ Most “real” kind of test you can write!
    ✤ Test the entire stack JS, HTML, Domain, and Database!
    ✤ Catch bugs unit tests don’t!
    ✤ Help focus and drive your dev process!
    ✤ New level of confidence!
    ✤ Lessen the load on QA testers!
    ✤ Feedback earlier
    

    View Slide

13. Why
    NOT write Acceptance Tests?
    

    View Slide

14. Why NOT write Acceptance Tests?
    ✤ Time consuming to write!
    ✤ Slow to run!
    ✤ Brittle!
    ✤ False Negatives
    

    View Slide

15. Who
    should write the tests?
    

    View Slide

16. Who should write the tests?
    ✤ Your actual customer!
    ✤ A business analyst!
    ✤ A developer!
    ✤ A tester
    

    View Slide

17. When
    should a test be written?
    

    View Slide

18. When should a test be written?
    ✤ First!
    ✤ Before “done”!
    ✤ After “done”!
    ✤ After released
    

    View Slide

19. How
    should they be written?
    

    View Slide

20. Cucumber
    How should they be written?
    

    View Slide

21. Cucumber: Step Definitions
    How should they be written?
    

    View Slide

22. RSpec
    How should they be written?
    

    View Slide

23. XUnit
    How should they be written?
    

    View Slide

24. Recommendations
    Customer First Cucumber
    

    View Slide

25. Recommendations
    Developer Early Not Cucumber
    

    View Slide

26. Recommendations
    Outside In
    

    View Slide

27. What
    language should tests be written in?
    

    View Slide

28. What language should tests be written in?
    ✤ Ruby!
    ✤ Whatever language you are using!
    ✤ IF it has a good browser automation library
    

    View Slide

29. What
    tests should be written?
    

    View Slide

30. What tests should be written?
    How many features should you test?
    How much of the feature should you test?
    

    View Slide

31. What tests should be written?
    How many features should you test?
    How much of the feature should you test?
    All the “important” ones
    As little as possible
    “Prove the existence of the feature, not the correctness.”
    - Gary Bernhardt
    

    View Slide

32. How
    do we keep tests isolated?
    (and why should we?)
    

    View Slide

33. Test1
    DB R1
    Test2
    R2
    R.All.Count().ShouldBe(2)
    

    View Slide

34. How
    do we keep tests isolated?
    

    View Slide

35. How do we keep tests isolated?
    ✤ “Reset” the database!
    ✤ Clear browser session/cookies
    

    View Slide

36. How do we reset the database?
    ✤ Build the db from scratch between each test run!
    ✤ Wipe the db between each test run!
    ✤ Run each test in a transactional fixture
    

    View Slide

37. How do we wipe the db?
    ✤ Database Cleaner Ruby Gem!
    ✤ Manually:!
    ✤ Drop all FKs!
    ✤ Black list “system data” tables!
    ✤ Truncate all other tables!
    ✤ Re-enable all FKs
    

    View Slide

38. How do we do transactional fixtures?
    Tests Browser Web Server Database
    Transaction
    Managed
    Controlled
    Start transactional fixture
    End transactional fixture
    

    View Slide

39. How do we do transactional fixtures?
    ✤ Full Text Search!
    ✤ Backend Services
    

    View Slide

40. How do we clear session?
    ✤ After each test, call your framework’s

    reset sessions method!
    ✤ Every test has to login
    

    View Slide

41. How
    do we setup test data?
    

    View Slide

42. How do we setup test data?
    Automate the application
    + Robust against change in code and database
    - Slow
    + Very real
    - Verbose
    - Some data can’t be configured through the app
    - Brittle against change in the UI
    

    View Slide

43. How do we setup test data?
    Seed the database
    + Any data can be setup
    + Fast
    - Brittle against change in the DB
    + Tooling: FactoryGirl, any db framework
    - Brittle against change in domain rules
    - Not real
    

    View Slide

44. How do we setup test data?
    Incident Report Case Party
    1. Incident Report creates a Case
    2. Incident Report creates a Case with a Party
    - Brittle against change in domain rules
    3. Incident Report creates a Case with 2 Parties (1 optional)
    4. Incident Report creates a Case with 3 Parties (1 optional)
    

    View Slide

45. How do we setup test data?
    Use the domain
    + Any data can be setup
    + Fast
    - Brittle against change in the domain
    + Robust against change in DB and UI
    - Requires access to domain objects
    - Not terribly real
    

    View Slide

46. How do we setup test data?
    Use the Builder pattern
    new TaskBuilder().Build();
    new TaskBuilder().WithName(“write talk”).Build();
    

    View Slide

47. How do we setup test data?
    Use the Builder pattern
    + More robust against change in the domain
    + Just like using the domain
    - Brittle against some changes in the domain
    - Requires access to the builder objects
    

    View Slide

48. View Slide

49. View Slide

50. How
    do we deal with Ajax?
    

    View Slide

51. How do we deal with Ajax?
    What’s the problem?
    visit tasklistpage!
    click “my first task”!
    fillin “name”, “take out the trash”!
    click “Save”!
    shouldsee “take out the trash”
    opens a dialog
    closes dialog, updates list
    is the dialog ready?
    has the list been updated?
    

    View Slide

52. How do we deal with Ajax?
    Explicit Waits
    visit tasklistpage!
    click “my first task”!
    waitfor “new task dialog”, visible!
    fillin “name”, “take out the trash”!
    click “Save”!
    waitfor “new task dialog”, notvisible!
    shouldsee “take out the trash”
    

    View Slide

53. How do we deal with Ajax?
    Implicit Waits
    visit tasklistpage!
    click “my first task”!
    fillin “name”, “take out the trash”!
    click “Save”!
    shouldsee “take out the trash”
    waits automatically
    waits automatically
    

    View Slide

54. How do we deal with Ajax?
    Frameworks That Wait
    Capybara Coypu
    

    View Slide

55. How do we deal with Ajax?
    Capybara removed all support for explicit waits!
    

    View Slide

56. Test Tips and Tricks
    

    View Slide

57. Robust Locators
    ✤ Locate by label:

    browser.FillIn(“username”).With(“kberridge”);

    browser.FillIn(“password”).With(“not my password”);!
    ✤ Decouple from HTML structure:

    browser.FindCss("div.my-section table tbody tr td input.special-field”);

    browser.FindCss(“.my-section .special-field”);!
    ✤ Write your HTML to be easy to test
    

    View Slide

58. Refactor Away Duplication
    ✤ Control Extensions

    browser.FillInDropdown(“Type”, “Home”);!
    ✤ Page Objects

    var tweetsPage = loginPage.Login(“kberridge”, “not my password”);

    tweetsPage.SendTweet(“This talk is so great! #codemash”);

    Assert.IsTrue(tweetsPage.HasTweet(“This talk is so great! #codemash”));
    

    View Slide

59. Use Your Framework
    ✤ Trust the implicit waits

    browser.ClickLink("Open Dialog");

    browser.FindCss("#my-dialog").Exists().ShouldBeTrue();

    browser.FindField("#my-dialog .somefield").FillInWith("something");!
    ✤ Use scopes

    browser.ClickLink(“Open Dialog”);

    var dialog = browser.FindCss(“#my-dialog”);

    dialog.FillIn(“some field”).With(“something”);
    

    View Slide

60. Get A Fast Build Server
    Really.
    

    View Slide

61. Review
    ✤ Make AT part of YOUR process!
    ✤ Write the right tests!
    ✤ Test Isolation And Test Data!
    ✤ Ajax!
    ✤ Test code “best practices”
    

    View Slide

62. References
    http://vimeo.com/53276460
    - Budgeting Reality, Justin Searls
    https://www.destroyallsoftware.com
    - Destroy All Software, Gary Bernhardt
    

    View Slide

63. Kevin Berridge
    @kberridge
    Argue with me!
    

    View Slide