$30 off During Our Annual Pro Sale. View Details »

Acceptance Testing: The DIrty Details

Acceptance Testing: The DIrty Details

Presented at CodeMash 2.0.1.4

Web based acceptance tests are quite valuable, and you should absolutely be writing them. However, there are many different ways that acceptance tests can be written, and choosing the right ones can mean the difference between success and failure for your project. This talk will get into the dirty details of dealing with AJAX, managing test data, keeping tests isolated, and working with your customer or Business Analyst. The result will be cleaner tests that run faster and are easier to write.

Kevin Berridge

January 10, 2014
Tweet

More Decks by Kevin Berridge

Other Decks in Programming

Transcript

  1. Acceptance Testing
    The Dirty Details

    View Slide

  2. Kevin Berridge
    @kberridge
    kevinberridge.com

    View Slide

  3. The Dirty Details
    Browser Automation Testing

    View Slide

  4. The Dirty Details
    ✤ What and Why!
    ✤ Process!
    ✤ Tool Choice!
    ✤ Choosing Tests!
    ✤ Test Data!
    ✤ Ajax!
    ✤ Test Tips and Tricks

    View Slide

  5. What
    is Acceptance Testing?

    View Slide

  6. What is Acceptance Testing?
    Client Developer

    View Slide

  7. What is Acceptance Testing?
    Full Stack
    UI
    Web
    Code
    Database

    View Slide

  8. What is Acceptance Testing?
    “As much realism as we can afford”
    -Justin Searls

    View Slide

  9. What is Acceptance Testing?
    Test Browser Automation! Browser

    View Slide

  10. View Slide

  11. Why
    write Acceptance Tests?

    View Slide

  12. Why write Acceptance Tests?
    ✤ Most “real” kind of test you can write!
    ✤ Test the entire stack JS, HTML, Domain, and Database!
    ✤ Catch bugs unit tests don’t!
    ✤ Help focus and drive your dev process!
    ✤ New level of confidence!
    ✤ Lessen the load on QA testers!
    ✤ Feedback earlier

    View Slide

  13. Why
    NOT write Acceptance Tests?

    View Slide

  14. Why NOT write Acceptance Tests?
    ✤ Time consuming to write!
    ✤ Slow to run!
    ✤ Brittle!
    ✤ False Negatives

    View Slide

  15. Who
    should write the tests?

    View Slide

  16. Who should write the tests?
    ✤ Your actual customer!
    ✤ A business analyst!
    ✤ A developer!
    ✤ A tester

    View Slide

  17. When
    should a test be written?

    View Slide

  18. When should a test be written?
    ✤ First!
    ✤ Before “done”!
    ✤ After “done”!
    ✤ After released

    View Slide

  19. How
    should they be written?

    View Slide

  20. Cucumber
    How should they be written?

    View Slide

  21. Cucumber: Step Definitions
    How should they be written?

    View Slide

  22. RSpec
    How should they be written?

    View Slide

  23. XUnit
    How should they be written?

    View Slide

  24. Recommendations
    Customer First Cucumber

    View Slide

  25. Recommendations
    Developer Early Not Cucumber

    View Slide

  26. Recommendations
    Outside In

    View Slide

  27. What
    language should tests be written in?

    View Slide

  28. What language should tests be written in?
    ✤ Ruby!
    ✤ Whatever language you are using!
    ✤ IF it has a good browser automation library

    View Slide

  29. What
    tests should be written?

    View Slide

  30. What tests should be written?
    How many features should you test?
    How much of the feature should you test?

    View Slide

  31. What tests should be written?
    How many features should you test?
    How much of the feature should you test?
    All the “important” ones
    As little as possible
    “Prove the existence of the feature, not the correctness.”
    - Gary Bernhardt

    View Slide

  32. How
    do we keep tests isolated?
    (and why should we?)

    View Slide

  33. Test1
    DB R1
    Test2
    R2
    R.All.Count().ShouldBe(2)

    View Slide

  34. How
    do we keep tests isolated?

    View Slide

  35. How do we keep tests isolated?
    ✤ “Reset” the database!
    ✤ Clear browser session/cookies

    View Slide

  36. How do we reset the database?
    ✤ Build the db from scratch between each test run!
    ✤ Wipe the db between each test run!
    ✤ Run each test in a transactional fixture

    View Slide

  37. How do we wipe the db?
    ✤ Database Cleaner Ruby Gem!
    ✤ Manually:!
    ✤ Drop all FKs!
    ✤ Black list “system data” tables!
    ✤ Truncate all other tables!
    ✤ Re-enable all FKs

    View Slide

  38. How do we do transactional fixtures?
    Tests Browser Web Server Database
    Transaction
    Managed
    Controlled
    Start transactional fixture
    End transactional fixture

    View Slide

  39. How do we do transactional fixtures?
    ✤ Full Text Search!
    ✤ Backend Services

    View Slide

  40. How do we clear session?
    ✤ After each test, call your framework’s

    reset sessions method!
    ✤ Every test has to login

    View Slide

  41. How
    do we setup test data?

    View Slide

  42. How do we setup test data?
    Automate the application
    + Robust against change in code and database
    - Slow
    + Very real
    - Verbose
    - Some data can’t be configured through the app
    - Brittle against change in the UI

    View Slide

  43. How do we setup test data?
    Seed the database
    + Any data can be setup
    + Fast
    - Brittle against change in the DB
    + Tooling: FactoryGirl, any db framework
    - Brittle against change in domain rules
    - Not real

    View Slide

  44. How do we setup test data?
    Incident Report Case Party
    1. Incident Report creates a Case
    2. Incident Report creates a Case with a Party
    - Brittle against change in domain rules
    3. Incident Report creates a Case with 2 Parties (1 optional)
    4. Incident Report creates a Case with 3 Parties (1 optional)

    View Slide

  45. How do we setup test data?
    Use the domain
    + Any data can be setup
    + Fast
    - Brittle against change in the domain
    + Robust against change in DB and UI
    - Requires access to domain objects
    - Not terribly real

    View Slide

  46. How do we setup test data?
    Use the Builder pattern
    new TaskBuilder().Build();
    new TaskBuilder().WithName(“write talk”).Build();

    View Slide

  47. How do we setup test data?
    Use the Builder pattern
    + More robust against change in the domain
    + Just like using the domain
    - Brittle against some changes in the domain
    - Requires access to the builder objects

    View Slide

  48. View Slide

  49. View Slide

  50. How
    do we deal with Ajax?

    View Slide

  51. How do we deal with Ajax?
    What’s the problem?
    visit tasklistpage!
    click “my first task”!
    fillin “name”, “take out the trash”!
    click “Save”!
    shouldsee “take out the trash”
    opens a dialog
    closes dialog, updates list
    is the dialog ready?
    has the list been updated?

    View Slide

  52. How do we deal with Ajax?
    Explicit Waits
    visit tasklistpage!
    click “my first task”!
    waitfor “new task dialog”, visible!
    fillin “name”, “take out the trash”!
    click “Save”!
    waitfor “new task dialog”, notvisible!
    shouldsee “take out the trash”

    View Slide

  53. How do we deal with Ajax?
    Implicit Waits
    visit tasklistpage!
    click “my first task”!
    fillin “name”, “take out the trash”!
    click “Save”!
    shouldsee “take out the trash”
    waits automatically
    waits automatically

    View Slide

  54. How do we deal with Ajax?
    Frameworks That Wait
    Capybara Coypu

    View Slide

  55. How do we deal with Ajax?
    Capybara removed all support for explicit waits!

    View Slide

  56. Test Tips and Tricks

    View Slide

  57. Robust Locators
    ✤ Locate by label:

    browser.FillIn(“username”).With(“kberridge”);

    browser.FillIn(“password”).With(“not my password”);!
    ✤ Decouple from HTML structure:

    browser.FindCss("div.my-section table tbody tr td input.special-field”);

    browser.FindCss(“.my-section .special-field”);!
    ✤ Write your HTML to be easy to test

    View Slide

  58. Refactor Away Duplication
    ✤ Control Extensions

    browser.FillInDropdown(“Type”, “Home”);!
    ✤ Page Objects

    var tweetsPage = loginPage.Login(“kberridge”, “not my password”);

    tweetsPage.SendTweet(“This talk is so great! #codemash”);

    Assert.IsTrue(tweetsPage.HasTweet(“This talk is so great! #codemash”));

    View Slide

  59. Use Your Framework
    ✤ Trust the implicit waits

    browser.ClickLink("Open Dialog");

    browser.FindCss("#my-dialog").Exists().ShouldBeTrue();

    browser.FindField("#my-dialog .somefield").FillInWith("something");!
    ✤ Use scopes

    browser.ClickLink(“Open Dialog”);

    var dialog = browser.FindCss(“#my-dialog”);

    dialog.FillIn(“some field”).With(“something”);

    View Slide

  60. Get A Fast Build Server
    Really.

    View Slide

  61. Review
    ✤ Make AT part of YOUR process!
    ✤ Write the right tests!
    ✤ Test Isolation And Test Data!
    ✤ Ajax!
    ✤ Test code “best practices”

    View Slide

  62. References
    http://vimeo.com/53276460
    - Budgeting Reality, Justin Searls
    https://www.destroyallsoftware.com
    - Destroy All Software, Gary Bernhardt

    View Slide

  63. Kevin Berridge
    @kberridge
    Argue with me!

    View Slide