Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Best Practices for Automated Regression Testing

Best Practices for Automated Regression Testing

So you wrote a bunch of great unit tests but your site is still broken? Maybe it’s time to start investing in a suite of end-to-end regression tests. In this talk you’ll learn best practices for incorporating Selenium into your QA process while avoiding some common pitfalls.

This talk was given at Beer City Code in Grand Rapids, MI.

Jace Browning

June 23, 2018
Tweet

More Decks by Jace Browning

Other Decks in Programming

Transcript

  1. Best Practices
    for Automated
    Regression Testing
    Jace Browning

    View full-size slide

  2. Outline
    • Terminology

    • Goals

    • Example / Demo

    • Limitations

    • Recommendations

    View full-size slide

  3. Integration Tests

    View full-size slide

  4. System Tests

    View full-size slide

  5. Automated
    System Tests
    Automated
    Integration Tests
    Automated
    Unit Tests

    View full-size slide

  6. Exploratory Testing
    Scripted Testing
    Automated
    Testing

    View full-size slide

  7. Production
    Staging
    Review
    Review
    Review

    View full-size slide

  8. Approval
    Production
    Staging
    Review
    Review
    Review

    View full-size slide

  9. Promotion
    Production
    Staging
    Review
    Review
    Review

    View full-size slide

  10. Validation
    Production
    Staging
    Review
    Review
    Review

    View full-size slide

  11. Detection
    Production
    Staging
    Review
    Review
    Review

    View full-size slide

  12. WebDriver Basics

    View full-size slide

  13. Finding Elements
    .payment-details__input
    cardNumber
    //*[@id=“app”]/div/div[2]/div/form/div/div/div/div/div[2]/div/label[1]/input

    View full-size slide

  14. Finding Elements
    .async-button__text
    “Start Trial”
    //*[@id="app"]/div/div[2]/div/form/div/footer/div/button/span/span/span

    View full-size slide

  15. Language Bindings
    from selenium import webdriver
    driver = webdriver.Firefox()
    driver.get("http://zapier.com/app/team-account-setup/billing")
    assert "Start Your 14-Day Free Trial" in driver.html
    element = driver.find_element_by_name("cardNumber")
    element.send_keys("4242424242424242")
    element = driver.find_element_by_link_text("Start Trial")
    element.click()
    assert "Invite Your Teammates" in driver.html

    View full-size slide

  16. Abstraction Layers
    from splinter import Browser
    browser = Browser("firefox")
    browser.visit("http://zapier.com/app/team-account-setup/billing")
    assert "Start Your 14-Day Free Trial" in browser.html
    browser.fill("cardNumber", "4242424242424242")
    browser.find_by_text("Start Trial").click()
    assert "Invite Your Teammates" in browser.html

    View full-size slide

  17. Page Object Model
    class TeamSetupBilling:
    URL = "http://zapier.com/app/team-account-setup/billing"
    def __init__(self):
    self.browser = Browser("firefox")
    self.browser.visit(self.URL)
    def __contains__(self, value):
    return value in self.browser.html
    def fill_credit_card_number(self, value):
    self.browser.fill("cardNumber", value)
    def click_start_trial(self):
    self.browser.find_by_text("Start Trial").click()

    View full-size slide

  18. Page Object Model
    page = TeamSetupBilling()
    assert "Start Your 14-Day Free Trial" in page
    page.fill_credit_card_number("4242424242424242")
    page.click_start_trial()
    assert "Invite Your Teammates" in page

    View full-size slide

  19. Helper Functions
    def create_team(team_name, team=None, credit_card=None):
    if team is None:
    team = get_team(team_name)
    if credit_card is None:
    credit_card = get_credit_card()
    page = views.TeamSetup()
    return page.complete_wizard(team, credit_card)

    View full-size slide

  20. Helper Functions
    helpers.create_team(team_name="Beer City")
    page = TeamSettings()
    assert "Beer City" in page
    page.fill_team_members("[email protected]")
    page.click_invite()
    assert "Pending Invite" in page

    View full-size slide

  21. Dependencies
    • Internal Data

    • External Services

    • Browsers

    View full-size slide

  22. Authentication
    • OAuth endpoints

    • Two-factor authentication

    • Administrative features

    View full-size slide

  23. User Experience
    • Designs

    • Animations / Rendering / Responsiveness

    • Experiments

    View full-size slide

  24. Maintenance
    • Upkeep

    • Monitoring

    • Coverage

    View full-size slide

  25. Recommendations

    View full-size slide

  26. Testability
    • Named inputs

    • Scoped CSS

    • IDs where necessary

    View full-size slide

  27. Cheats
    • Security bypasses

    • Test credit cards

    • Exposed internals

    View full-size slide

  28. Cleanup
    • Identifiable test users

    • Filter from metrics

    • Delete old accounts

    View full-size slide

  29. Roadmap
    1. Sign up / Log in

    2. Run after every deploy

    3. Low-coverge, High-impact

    4. Business critical

    5. Postmortems / Retrospectives

    View full-size slide

  30. Questions?
    @jacebrowning
    zapier.com

    View full-size slide