Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Best Practices for Automated Regression Testing

Best Practices for Automated Regression Testing

So you wrote a bunch of great unit tests but your site is still broken? Maybe it’s time to start investing in a suite of end-to-end regression tests. In this talk you’ll learn best practices for incorporating Selenium into your QA process while avoiding some common pitfalls.

This talk was given at Beer City Code in Grand Rapids, MI.

Jace Browning

June 23, 2018
Tweet

More Decks by Jace Browning

Other Decks in Programming

Transcript

  1. Best Practices
    for Automated
    Regression Testing
    Jace Browning

    View Slide

  2. Outline
    • Terminology

    • Goals

    • Example / Demo

    • Limitations

    • Recommendations

    View Slide

  3. About Me
    +

    View Slide

  4. Terminology

    View Slide

  5. Unit Tests

    View Slide

  6. Integration Tests

    View Slide

  7. System Tests

    View Slide

  8. Automated
    System Tests
    Automated
    Integration Tests
    Automated
    Unit Tests

    View Slide

  9. Exploratory Testing
    Scripted Testing
    Automated
    Testing

    View Slide

  10. Goals

    View Slide

  11. Production
    Staging
    Review
    Review
    Review

    View Slide

  12. Approval
    Production
    Staging
    Review
    Review
    Review

    View Slide

  13. Promotion
    Production
    Staging
    Review
    Review
    Review

    View Slide

  14. Validation
    Production
    Staging
    Review
    Review
    Review

    View Slide

  15. Detection
    Production
    Staging
    Review
    Review
    Review

    View Slide

  16. Example

    View Slide

  17. View Slide

  18. View Slide

  19. View Slide

  20. WebDriver Basics

    View Slide

  21. Finding Elements
    .payment-details__input
    cardNumber
    //*[@id=“app”]/div/div[2]/div/form/div/div/div/div/div[2]/div/label[1]/input

    View Slide

  22. Finding Elements
    .async-button__text
    “Start Trial”
    //*[@id="app"]/div/div[2]/div/form/div/footer/div/button/span/span/span

    View Slide

  23. Language Bindings
    from selenium import webdriver
    driver = webdriver.Firefox()
    driver.get("http://zapier.com/app/team-account-setup/billing")
    assert "Start Your 14-Day Free Trial" in driver.html
    element = driver.find_element_by_name("cardNumber")
    element.send_keys("4242424242424242")
    element = driver.find_element_by_link_text("Start Trial")
    element.click()
    assert "Invite Your Teammates" in driver.html

    View Slide

  24. Abstraction Layers
    from splinter import Browser
    browser = Browser("firefox")
    browser.visit("http://zapier.com/app/team-account-setup/billing")
    assert "Start Your 14-Day Free Trial" in browser.html
    browser.fill("cardNumber", "4242424242424242")
    browser.find_by_text("Start Trial").click()
    assert "Invite Your Teammates" in browser.html

    View Slide

  25. Page Object Model
    class TeamSetupBilling:
    URL = "http://zapier.com/app/team-account-setup/billing"
    def __init__(self):
    self.browser = Browser("firefox")
    self.browser.visit(self.URL)
    def __contains__(self, value):
    return value in self.browser.html
    def fill_credit_card_number(self, value):
    self.browser.fill("cardNumber", value)
    def click_start_trial(self):
    self.browser.find_by_text("Start Trial").click()

    View Slide

  26. Page Object Model
    page = TeamSetupBilling()
    assert "Start Your 14-Day Free Trial" in page
    page.fill_credit_card_number("4242424242424242")
    page.click_start_trial()
    assert "Invite Your Teammates" in page

    View Slide

  27. Helper Functions
    def create_team(team_name, team=None, credit_card=None):
    if team is None:
    team = get_team(team_name)
    if credit_card is None:
    credit_card = get_credit_card()
    page = views.TeamSetup()
    return page.complete_wizard(team, credit_card)

    View Slide

  28. Helper Functions
    helpers.create_team(team_name="Beer City")
    page = TeamSettings()
    assert "Beer City" in page
    page.fill_team_members("[email protected]")
    page.click_invite()
    assert "Pending Invite" in page

    View Slide

  29. Demo

    View Slide

  30. Limitations

    View Slide

  31. Dependencies
    • Internal Data

    • External Services

    • Browsers

    View Slide

  32. Authentication
    • OAuth endpoints

    • Two-factor authentication

    • Administrative features

    View Slide

  33. User Experience
    • Designs

    • Animations / Rendering / Responsiveness

    • Experiments

    View Slide

  34. Maintenance
    • Upkeep

    • Monitoring

    • Coverage

    View Slide

  35. Recommendations

    View Slide

  36. Testability
    • Named inputs

    • Scoped CSS

    • IDs where necessary

    View Slide

  37. Cheats
    • Security bypasses

    • Test credit cards

    • Exposed internals

    View Slide

  38. Cleanup
    • Identifiable test users

    • Filter from metrics

    • Delete old accounts

    View Slide

  39. Roadmap
    1. Sign up / Log in

    2. Run after every deploy

    3. Low-coverge, High-impact

    4. Business critical

    5. Postmortems / Retrospectives

    View Slide

  40. Questions?
    @jacebrowning
    zapier.com

    View Slide