Best Practices for Automated Regression Testing

So you wrote a bunch of great unit tests but your site is still broken? Maybe it’s time to start investing in a suite of end-to-end regression tests. In this talk you’ll learn best practices for incorporating Selenium into your QA process while avoiding some common pitfalls.

This talk was given at Beer City Code in Grand Rapids, MI.

Transcript

1. Best Practices for Automated Regression Testing Jace Browning
2. Outline • Terminology • Goals • Example / Demo • Limitations • Recommendations
3. About Me +
4. Terminology
5. Unit Tests
6. Integration Tests
7. System Tests
8. Automated System Tests Automated Integration Tests Automated Unit Tests
9. Exploratory Testing Scripted Testing Automated Testing
10. Goals
11. Production Staging Review Review Review
12. Approval Production Staging Review Review Review
13. Promotion Production Staging Review Review Review
14. Validation Production Staging Review Review Review
15. Detection Production Staging Review Review Review
16. Example
20. WebDriver Basics
21. Finding Elements .payment-details__input cardNumber //*[@id=“app”]/div/div[2]/div/form/div/div/div/div/div[2]/div/label[1]/input
22. Finding Elements .async-button__text “Start Trial” //*[@id="app"]/div/div[2]/div/form/div/footer/div/button/span/span/span
23. Language Bindings from selenium import webdriver driver = webdriver.Firefox() driver.get("http://zapier.com/app/team-account-setup/billing") assert "Start Your 14-Day Free Trial" in driver.html element = driver.find_element_by_name("cardNumber") element.send_keys("4242424242424242") element = driver.find_element_by_link_text("Start Trial") element.click() assert "Invite Your Teammates" in driver.html
24. Abstraction Layers from splinter import Browser browser = Browser("firefox") browser.visit("http://zapier.com/app/team-account-setup/billing") assert "Start Your 14-Day Free Trial" in browser.html browser.fill("cardNumber", "4242424242424242") browser.find_by_text("Start Trial").click() assert "Invite Your Teammates" in browser.html
25. Page Object Model class TeamSetupBilling: URL = "http://zapier.com/app/team-account-setup/billing" def __init__(self): self.browser = Browser("firefox") self.browser.visit(self.URL) def __contains__(self, value): return value in self.browser.html def fill_credit_card_number(self, value): self.browser.fill("cardNumber", value) def click_start_trial(self): self.browser.find_by_text("Start Trial").click()
26. Page Object Model page = TeamSetupBilling() assert "Start Your 14-Day Free Trial" in page page.fill_credit_card_number("4242424242424242") page.click_start_trial() assert "Invite Your Teammates" in page
27. Helper Functions def create_team(team_name, team=None, credit_card=None): if team is None: team = get_team(team_name) if credit_card is None: credit_card = get_credit_card() page = views.TeamSetup() return page.complete_wizard(team, credit_card)
28. Helper Functions helpers.create_team(team_name="Beer City") page = TeamSettings() assert "Beer City" in page page.fill_team_members("jane@beercitycode.com") page.click_invite() assert "Pending Invite" in page
29. Demo
30. Limitations
31. Dependencies • Internal Data • External Services • Browsers
32. Authentication • OAuth endpoints • Two-factor authentication • Administrative features
33. User Experience • Designs • Animations / Rendering / Responsiveness • Experiments
34. Maintenance • Upkeep • Monitoring • Coverage
35. Recommendations
36. Testability • Named inputs • Scoped CSS • IDs where necessary
37. Cheats • Security bypasses • Test credit cards • Exposed internals
38. Cleanup • Identifiable test users • Filter from metrics • Delete old accounts
39. Roadmap 1. Sign up / Log in 2. Run after every deploy 3. Low-coverge, High-impact 4. Business critical 5. Postmortems / Retrospectives
40. Questions? @jacebrowning zapier.com