Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Best Practices for Automated Regression Testing

Best Practices for Automated Regression Testing

So you wrote a bunch of great unit tests but your site is still broken? Maybe it’s time to start investing in a suite of end-to-end regression tests. In this talk you’ll learn best practices for incorporating Selenium into your QA process while avoiding some common pitfalls.

This talk was given at Beer City Code in Grand Rapids, MI.

Eff2cdaa8474520ed29ebc38031fab87?s=128

Jace Browning

June 23, 2018
Tweet

Transcript

  1. Best Practices for Automated Regression Testing Jace Browning

  2. Outline • Terminology • Goals • Example / Demo •

    Limitations • Recommendations
  3. About Me +

  4. Terminology

  5. Unit Tests

  6. Integration Tests

  7. System Tests

  8. Automated System Tests Automated Integration Tests Automated Unit Tests

  9. Exploratory Testing Scripted Testing Automated Testing

  10. Goals

  11. Production Staging Review Review Review

  12. Approval Production Staging Review Review Review

  13. Promotion Production Staging Review Review Review

  14. Validation Production Staging Review Review Review

  15. Detection Production Staging Review Review Review

  16. Example

  17. None
  18. None
  19. None
  20. WebDriver Basics

  21. Finding Elements .payment-details__input cardNumber //*[@id=“app”]/div/div[2]/div/form/div/div/div/div/div[2]/div/label[1]/input

  22. Finding Elements .async-button__text “Start Trial” //*[@id="app"]/div/div[2]/div/form/div/footer/div/button/span/span/span

  23. Language Bindings from selenium import webdriver driver = webdriver.Firefox() driver.get("http://zapier.com/app/team-account-setup/billing")

    assert "Start Your 14-Day Free Trial" in driver.html element = driver.find_element_by_name("cardNumber") element.send_keys("4242424242424242") element = driver.find_element_by_link_text("Start Trial") element.click() assert "Invite Your Teammates" in driver.html
  24. Abstraction Layers from splinter import Browser browser = Browser("firefox") browser.visit("http://zapier.com/app/team-account-setup/billing")

    assert "Start Your 14-Day Free Trial" in browser.html browser.fill("cardNumber", "4242424242424242") browser.find_by_text("Start Trial").click() assert "Invite Your Teammates" in browser.html
  25. Page Object Model class TeamSetupBilling: URL = "http://zapier.com/app/team-account-setup/billing" def __init__(self):

    self.browser = Browser("firefox") self.browser.visit(self.URL) def __contains__(self, value): return value in self.browser.html def fill_credit_card_number(self, value): self.browser.fill("cardNumber", value) def click_start_trial(self): self.browser.find_by_text("Start Trial").click()
  26. Page Object Model page = TeamSetupBilling() assert "Start Your 14-Day

    Free Trial" in page page.fill_credit_card_number("4242424242424242") page.click_start_trial() assert "Invite Your Teammates" in page
  27. Helper Functions def create_team(team_name, team=None, credit_card=None): if team is None:

    team = get_team(team_name) if credit_card is None: credit_card = get_credit_card() page = views.TeamSetup() return page.complete_wizard(team, credit_card)
  28. Helper Functions helpers.create_team(team_name="Beer City") page = TeamSettings() assert "Beer City"

    in page page.fill_team_members("jane@beercitycode.com") page.click_invite() assert "Pending Invite" in page
  29. Demo

  30. Limitations

  31. Dependencies • Internal Data • External Services • Browsers

  32. Authentication • OAuth endpoints • Two-factor authentication • Administrative features

  33. User Experience • Designs • Animations / Rendering / Responsiveness

    • Experiments
  34. Maintenance • Upkeep • Monitoring • Coverage

  35. Recommendations

  36. Testability • Named inputs • Scoped CSS • IDs where

    necessary
  37. Cheats • Security bypasses • Test credit cards • Exposed

    internals
  38. Cleanup • Identifiable test users • Filter from metrics •

    Delete old accounts
  39. Roadmap 1. Sign up / Log in 2. Run after

    every deploy 3. Low-coverge, High-impact 4. Business critical 5. Postmortems / Retrospectives
  40. Questions? @jacebrowning zapier.com