Acceptance Testing: The DIrty Details

Presented at CodeMash 2.0.1.4

Web based acceptance tests are quite valuable, and you should absolutely be writing them. However, there are many different ways that acceptance tests can be written, and choosing the right ones can mean the difference between success and failure for your project. This talk will get into the dirty details of dealing with AJAX, managing test data, keeping tests isolated, and working with your customer or Business Analyst. The result will be cleaner tests that run faster and are easier to write.

Transcript

1. Acceptance Testing The Dirty Details
2. Kevin Berridge @kberridge kevinberridge.com
3. The Dirty Details Browser Automation Testing
4. The Dirty Details ✤ What and Why! ✤ Process! ✤ Tool Choice! ✤ Choosing Tests! ✤ Test Data! ✤ Ajax! ✤ Test Tips and Tricks
5. What is Acceptance Testing?
6. What is Acceptance Testing? Client Developer
7. What is Acceptance Testing? Full Stack UI Web Code Database
8. What is Acceptance Testing? “As much realism as we can afford” -Justin Searls
9. What is Acceptance Testing? Test Browser Automation! Browser
11. Why write Acceptance Tests?
12. Why write Acceptance Tests? ✤ Most “real” kind of test you can write! ✤ Test the entire stack JS, HTML, Domain, and Database! ✤ Catch bugs unit tests don’t! ✤ Help focus and drive your dev process! ✤ New level of confidence! ✤ Lessen the load on QA testers! ✤ Feedback earlier
13. Why NOT write Acceptance Tests?
14. Why NOT write Acceptance Tests? ✤ Time consuming to write! ✤ Slow to run! ✤ Brittle! ✤ False Negatives
15. Who should write the tests?
16. Who should write the tests? ✤ Your actual customer! ✤ A business analyst! ✤ A developer! ✤ A tester
17. When should a test be written?
18. When should a test be written? ✤ First! ✤ Before “done”! ✤ After “done”! ✤ After released
19. How should they be written?
20. Cucumber How should they be written?
21. Cucumber: Step Definitions How should they be written?
22. RSpec How should they be written?
23. XUnit How should they be written?
24. Recommendations Customer First Cucumber
25. Recommendations Developer Early Not Cucumber
26. Recommendations Outside In
27. What language should tests be written in?
28. What language should tests be written in? ✤ Ruby! ✤ Whatever language you are using! ✤ IF it has a good browser automation library
29. What tests should be written?
30. What tests should be written? How many features should you test? How much of the feature should you test?
31. What tests should be written? How many features should you test? How much of the feature should you test? All the “important” ones As little as possible “Prove the existence of the feature, not the correctness.” - Gary Bernhardt
32. How do we keep tests isolated? (and why should we?)
33. Test1 DB R1 Test2 R2 R.All.Count().ShouldBe(2)
34. How do we keep tests isolated?
35. How do we keep tests isolated? ✤ “Reset” the database! ✤ Clear browser session/cookies
36. How do we reset the database? ✤ Build the db from scratch between each test run! ✤ Wipe the db between each test run! ✤ Run each test in a transactional fixture
37. How do we wipe the db? ✤ Database Cleaner Ruby Gem! ✤ Manually:! ✤ Drop all FKs! ✤ Black list “system data” tables! ✤ Truncate all other tables! ✤ Re-enable all FKs
38. How do we do transactional fixtures? Tests Browser Web Server Database Transaction Managed Controlled Start transactional fixture End transactional fixture
39. How do we do transactional fixtures? ✤ Full Text Search! ✤ Backend Services
40. How do we clear session? ✤ After each test, call your framework’s
 reset sessions method! ✤ Every test has to login
41. How do we setup test data?
42. How do we setup test data? Automate the application + Robust against change in code and database - Slow + Very real - Verbose - Some data can’t be configured through the app - Brittle against change in the UI
43. How do we setup test data? Seed the database + Any data can be setup + Fast - Brittle against change in the DB + Tooling: FactoryGirl, any db framework - Brittle against change in domain rules - Not real
44. How do we setup test data? Incident Report Case Party 1. Incident Report creates a Case 2. Incident Report creates a Case with a Party - Brittle against change in domain rules 3. Incident Report creates a Case with 2 Parties (1 optional) 4. Incident Report creates a Case with 3 Parties (1 optional)
45. How do we setup test data? Use the domain + Any data can be setup + Fast - Brittle against change in the domain + Robust against change in DB and UI - Requires access to domain objects - Not terribly real
46. How do we setup test data? Use the Builder pattern new TaskBuilder().Build(); new TaskBuilder().WithName(“write talk”).Build();
47. How do we setup test data? Use the Builder pattern + More robust against change in the domain + Just like using the domain - Brittle against some changes in the domain - Requires access to the builder objects
50. How do we deal with Ajax?
51. How do we deal with Ajax? What’s the problem? visit tasklistpage! click “my first task”! fillin “name”, “take out the trash”! click “Save”! shouldsee “take out the trash” opens a dialog closes dialog, updates list is the dialog ready? has the list been updated?
52. How do we deal with Ajax? Explicit Waits visit tasklistpage! click “my first task”! waitfor “new task dialog”, visible! fillin “name”, “take out the trash”! click “Save”! waitfor “new task dialog”, notvisible! shouldsee “take out the trash”
53. How do we deal with Ajax? Implicit Waits visit tasklistpage! click “my first task”! fillin “name”, “take out the trash”! click “Save”! shouldsee “take out the trash” waits automatically waits automatically
54. How do we deal with Ajax? Frameworks That Wait Capybara Coypu
55. How do we deal with Ajax? Capybara removed all support for explicit waits!
56. Test Tips and Tricks
57. Robust Locators ✤ Locate by label:
 browser.FillIn(“username”).With(“kberridge”);
 browser.FillIn(“password”).With(“not my password”);! ✤ Decouple from HTML structure:
 browser.FindCss("div.my-section table tbody tr td input.special-field”);
 browser.FindCss(“.my-section .special-field”);! ✤ Write your HTML to be easy to test
58. Refactor Away Duplication ✤ Control Extensions
 browser.FillInDropdown(“Type”, “Home”);! ✤ Page Objects
 var tweetsPage = loginPage.Login(“kberridge”, “not my password”);
 tweetsPage.SendTweet(“This talk is so great! #codemash”);
 Assert.IsTrue(tweetsPage.HasTweet(“This talk is so great! #codemash”));
59. Use Your Framework ✤ Trust the implicit waits
 browser.ClickLink("Open Dialog");
 browser.FindCss("#my-dialog").Exists().ShouldBeTrue();
 browser.FindField("#my-dialog .somefield").FillInWith("something");! ✤ Use scopes
 browser.ClickLink(“Open Dialog”);
 var dialog = browser.FindCss(“#my-dialog”);
 dialog.FillIn(“some field”).With(“something”);
60. Get A Fast Build Server Really.
61. Review ✤ Make AT part of YOUR process! ✤ Write the right tests! ✤ Test Isolation And Test Data! ✤ Ajax! ✤ Test code “best practices”
62. References http://vimeo.com/53276460 - Budgeting Reality, Justin Searls https://www.destroyallsoftware.com - Destroy All Software, Gary Bernhardt
63. Kevin Berridge @kberridge Argue with me!