Tips From Our Codebase To Help You Write Reliable Selenium Tests

Transcript

1. Tips From Our Codebase To Help You Write Reliable Selenium Tests Santiago Suarez Ordoñez sso@saucelabs.com @santiycr
2. Agenda Selenium 1 tips • Implicit waits • Reasonable timeouts (over 90s) • Ignore Open and WaitForPageToLoad failures • Generate application states with test handles • Wait for the right events (never ever use static pauses) • Don't use verifications Selenium 2 tips • Implicit waits • Avoid using complex locators Sauce tips • Report pass/fail status automatically • Use the new Sauce Connect
3. Implicit waits import time, re from selenium import selenium class implicitWaitSelenium(selenium): """ Extending the regular selenium class to add implicit waits """ def __init__(self, *args): self.implicit_wait = 30 super(implicitWaitSelenium, self).__init__(*args) def do_command(self, verb, args, implicit_wait=None): if implicit_wait is None: implicit_wait = self.implicit_wait try: return super(implicitWaitSelenium, self).do_command(verb, args) except Exception, e: if (re.match("ERROR: Element .* not found", str(e)) and implicit_wait > 0): time.sleep(1) return self.do_command(verb, args, implicit_wait-1) raise Exception(e) https://gist.github.com/1107375
4. Reasonable timeouts (over 90s) Always set: selenium.setTimeout(90000) Or use: selenium.waitForPageToLoad(90000)
5. Ignore Open and WaitForPage failures class SeleniumTestCase(TestCase): # Let us just call self.click or self.type instead of self.selenium.type def __getattr__(self, name): if hasattr(self, 'selenium'): return getattr(self.selenium, name) raise AttributeError(name) # Ignore open commands that fail (same needs to be don for wait_for_page_to_load) def open(self, url, ignoreResponseCode=True): try: self.selenium.open(url, ignoreResponseCode) except Exception, e: print "Open failed on %s. Exception: %s" % (url, str(e)) pass #sometimes open appears to work fine but flakes out. ignore those # Report pass/fail status to Sauce Labs def tearDown(self): passed = {'passed': self._exc_info() == (None, None, None)} self.set_context("sauce: job-info=%s" % json.dumps(passed)) self.stop() https://gist.github.com/1107711
6. Special test handle: def magic_login(username): sel.open("/test/magic-login/" + username) assert sel.get_title() == "logged in as " + username don't do: selenium.open("/login") selenium.type("username", "santi") selenium.type("password", "bleh") selenium.click("submit") selenium.wait_for_page_to_load(90000) do: magic_login("santi") Generate app states using test handles
7. Wait for the right events (never ever use static pauses) Never use: sleep(10) Always use: selenium.waitForCondition("...") or: for i in range(60): if selenium.is_element_present("bleh"): break else: sleep(1)
8. Don't use verifications • Never verify*, always assert*. • Short tests are better • Testing for multiple conditions in a single workflow gives you hard to read results • Make tests faster with other tricks. Don't cram them in a single workflow.
9. Selenium 2: Implicit waits http://seleniumhq.org/docs/04_webdriver_advanced.html#explicit- and-implicit-waits driver.implicitly_wait(60)
10. Avoid using complex locators Instead of using: e = driver.find_element_by_css_selector("#actions .delete-links:nth-child(2)") e.click() Use: actions = driver.find_element_by_id("actions") delete_links = actions.find_elements_by_class("delete-links") delete_links[3].click() This keeps logic in your code, not your locators.
11. Report pass/fail to Sauce: Se1 https://gist.github.com/1003731 def tearDown(self): passed = {'passed': self._exc_info() == (None, None, None)} self.selenium.set_context("sauce: job-info=%s" % json.dumps (passed)) self.selenium.stop()
12. Report pass/fail to Sauce: Se2 https://gist.github.com/1107750 class Selenium2TestCase(TestCase): def report_pass_fail(self): base64string = base64.encodestring('%s:%s' % (config['username'], config['access-key']))[:-1] result = json.dumps({'passed': self._exc_info() == (None, None, None)}) connection = httplib.HTTPConnection(self.config['host']) connection.request('PUT', '/rest/v1/%s/jobs/%s' % (self.config['username'], self.driver.session_id), result, headers={"Authorization": "Basic %s" % base64string}) result = connection.getresponse() return result.status == 200 def tearDown(self): self.report_pass_fail() self.driver.quit()
13. Use the latest Sauce Connect http://saucelabs.com/docs/sauce-connect
14. Q&A
15. Thanks! Santiago Suarez Ordoñez sso@saucelabs.com @santiycr