Acceptance Testing: The DIrty Details

Acceptance Testing: The DIrty Details

Presented at CodeMash 2.0.1.4

Web based acceptance tests are quite valuable, and you should absolutely be writing them. However, there are many different ways that acceptance tests can be written, and choosing the right ones can mean the difference between success and failure for your project. This talk will get into the dirty details of dealing with AJAX, managing test data, keeping tests isolated, and working with your customer or Business Analyst. The result will be cleaner tests that run faster and are easier to write.

35e74c48a612d8a6786f8ab6424b49a1?s=128

Kevin Berridge

January 10, 2014
Tweet

Transcript

  1. Acceptance Testing The Dirty Details

  2. Kevin Berridge @kberridge kevinberridge.com

  3. The Dirty Details Browser Automation Testing

  4. The Dirty Details ✤ What and Why! ✤ Process! ✤

    Tool Choice! ✤ Choosing Tests! ✤ Test Data! ✤ Ajax! ✤ Test Tips and Tricks
  5. What is Acceptance Testing?

  6. What is Acceptance Testing? Client Developer

  7. What is Acceptance Testing? Full Stack UI Web Code Database

  8. What is Acceptance Testing? “As much realism as we can

    afford” -Justin Searls
  9. What is Acceptance Testing? Test Browser Automation! Browser

  10. None
  11. Why write Acceptance Tests?

  12. Why write Acceptance Tests? ✤ Most “real” kind of test

    you can write! ✤ Test the entire stack JS, HTML, Domain, and Database! ✤ Catch bugs unit tests don’t! ✤ Help focus and drive your dev process! ✤ New level of confidence! ✤ Lessen the load on QA testers! ✤ Feedback earlier
  13. Why NOT write Acceptance Tests?

  14. Why NOT write Acceptance Tests? ✤ Time consuming to write!

    ✤ Slow to run! ✤ Brittle! ✤ False Negatives
  15. Who should write the tests?

  16. Who should write the tests? ✤ Your actual customer! ✤

    A business analyst! ✤ A developer! ✤ A tester
  17. When should a test be written?

  18. When should a test be written? ✤ First! ✤ Before

    “done”! ✤ After “done”! ✤ After released
  19. How should they be written?

  20. Cucumber How should they be written?

  21. Cucumber: Step Definitions How should they be written?

  22. RSpec How should they be written?

  23. XUnit How should they be written?

  24. Recommendations Customer First Cucumber

  25. Recommendations Developer Early Not Cucumber

  26. Recommendations Outside In

  27. What language should tests be written in?

  28. What language should tests be written in? ✤ Ruby! ✤

    Whatever language you are using! ✤ IF it has a good browser automation library
  29. What tests should be written?

  30. What tests should be written? How many features should you

    test? How much of the feature should you test?
  31. What tests should be written? How many features should you

    test? How much of the feature should you test? All the “important” ones As little as possible “Prove the existence of the feature, not the correctness.” - Gary Bernhardt
  32. How do we keep tests isolated? (and why should we?)

  33. Test1 DB R1 Test2 R2 R.All.Count().ShouldBe(2)

  34. How do we keep tests isolated?

  35. How do we keep tests isolated? ✤ “Reset” the database!

    ✤ Clear browser session/cookies
  36. How do we reset the database? ✤ Build the db

    from scratch between each test run! ✤ Wipe the db between each test run! ✤ Run each test in a transactional fixture
  37. How do we wipe the db? ✤ Database Cleaner Ruby

    Gem! ✤ Manually:! ✤ Drop all FKs! ✤ Black list “system data” tables! ✤ Truncate all other tables! ✤ Re-enable all FKs
  38. How do we do transactional fixtures? Tests Browser Web Server

    Database Transaction Managed Controlled Start transactional fixture End transactional fixture
  39. How do we do transactional fixtures? ✤ Full Text Search!

    ✤ Backend Services
  40. How do we clear session? ✤ After each test, call

    your framework’s
 reset sessions method! ✤ Every test has to login
  41. How do we setup test data?

  42. How do we setup test data? Automate the application +

    Robust against change in code and database - Slow + Very real - Verbose - Some data can’t be configured through the app - Brittle against change in the UI
  43. How do we setup test data? Seed the database +

    Any data can be setup + Fast - Brittle against change in the DB + Tooling: FactoryGirl, any db framework - Brittle against change in domain rules - Not real
  44. How do we setup test data? Incident Report Case Party

    1. Incident Report creates a Case 2. Incident Report creates a Case with a Party - Brittle against change in domain rules 3. Incident Report creates a Case with 2 Parties (1 optional) 4. Incident Report creates a Case with 3 Parties (1 optional)
  45. How do we setup test data? Use the domain +

    Any data can be setup + Fast - Brittle against change in the domain + Robust against change in DB and UI - Requires access to domain objects - Not terribly real
  46. How do we setup test data? Use the Builder pattern

    new TaskBuilder().Build(); new TaskBuilder().WithName(“write talk”).Build();
  47. How do we setup test data? Use the Builder pattern

    + More robust against change in the domain + Just like using the domain - Brittle against some changes in the domain - Requires access to the builder objects
  48. None
  49. None
  50. How do we deal with Ajax?

  51. How do we deal with Ajax? What’s the problem? visit

    tasklistpage! click “my first task”! fillin “name”, “take out the trash”! click “Save”! shouldsee “take out the trash” opens a dialog closes dialog, updates list is the dialog ready? has the list been updated?
  52. How do we deal with Ajax? Explicit Waits visit tasklistpage!

    click “my first task”! waitfor “new task dialog”, visible! fillin “name”, “take out the trash”! click “Save”! waitfor “new task dialog”, notvisible! shouldsee “take out the trash”
  53. How do we deal with Ajax? Implicit Waits visit tasklistpage!

    click “my first task”! fillin “name”, “take out the trash”! click “Save”! shouldsee “take out the trash” waits automatically waits automatically
  54. How do we deal with Ajax? Frameworks That Wait Capybara

    Coypu
  55. How do we deal with Ajax? Capybara removed all support

    for explicit waits!
  56. Test Tips and Tricks

  57. Robust Locators ✤ Locate by label:
 browser.FillIn(“username”).With(“kberridge”);
 browser.FillIn(“password”).With(“not my password”);!

    ✤ Decouple from HTML structure:
 browser.FindCss("div.my-section table tbody tr td input.special-field”);
 browser.FindCss(“.my-section .special-field”);! ✤ Write your HTML to be easy to test
  58. Refactor Away Duplication ✤ Control Extensions
 browser.FillInDropdown(“Type”, “Home”);! ✤ Page

    Objects
 var tweetsPage = loginPage.Login(“kberridge”, “not my password”);
 tweetsPage.SendTweet(“This talk is so great! #codemash”);
 Assert.IsTrue(tweetsPage.HasTweet(“This talk is so great! #codemash”));
  59. Use Your Framework ✤ Trust the implicit waits
 browser.ClickLink("Open Dialog");


    browser.FindCss("#my-dialog").Exists().ShouldBeTrue();
 browser.FindField("#my-dialog .somefield").FillInWith("something");! ✤ Use scopes
 browser.ClickLink(“Open Dialog”);
 var dialog = browser.FindCss(“#my-dialog”);
 dialog.FillIn(“some field”).With(“something”);
  60. Get A Fast Build Server Really.

  61. Review ✤ Make AT part of YOUR process! ✤ Write

    the right tests! ✤ Test Isolation And Test Data! ✤ Ajax! ✤ Test code “best practices”
  62. References http://vimeo.com/53276460 - Budgeting Reality, Justin Searls https://www.destroyallsoftware.com - Destroy

    All Software, Gary Bernhardt
  63. Kevin Berridge @kberridge Argue with me!