Selenium: Less Testing, More Coding

Slide 1

Slide 1 text

SELENIUM Less Testing, More Coding Jonathan Lipps (@jlipps) SDEC :: October 16, 2012

Slide 2

Slide 2 text

Introduction Who am I? Real subtitle: “Better testing with the fun of coding” Contributions of Sauce Labs colleagues @admc, @santiycr Assumptions and goals of this talk

Slide 3

Slide 3 text

Talk Outline Automated testing discussion Selenium & its place in the testing stack How Selenium works & demo Selenium gotchas & best practices Sauce Labs & demo Q & A

Slide 4

Slide 4 text

We all believe in automation, right? QA is necessary, but manual QA sucks Risk of unintended consequences grows with app complexity Cheap to run automated tests on every commit Writing automated tests is writing code Opens up possibility of Development Cycle of Optimal Happiness

Slide 5

Slide 5 text

The Development Cycle of Optimal Happiness

Slide 6

Slide 6 text

Automated Testing Landscape Unit tests Tests of speciﬁc methods/modules Integration points are mocked for speed/simplicity Integration / end-to-end / functional tests Tests of entire systems or integration points System is a functional black box from test’s perspective

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

The Challenge of Many Platforms Integration tests are better the closer they are to reality “Reality” in web environments is hard to simulate People clicking mouse buttons in browsers Different OSes, different browsers, different devices Takeaway: integration tests are more expensive

Slide 10

Slide 10 text

The Testing Stack of Optimal Happiness

Slide 11

Slide 11 text

The Place of Selenium Integration tests are a big hammer Get as much mileage as you can out of lighter-weight solutions ﬁrst Debugging Selenium tests is more painful than debugging unit tests Nothing beats Selenium for uncovering cross-platform issues and legitimate stability concerns

Slide 12

Slide 12 text

How Selenium Works Client/server architecture Scripts speak Selenium API to the Selenium Server Selenium Server sends commands to browser Drivers, returns responses to script Independent Drivers for each browser/OS Bindings for speaking Selenium API to the server available in any programming language

Slide 13

Slide 13 text

How Selenium Works Language bindings Python Nose, unittest, selenium.driver C# NUnit, OpenQA.Selenium Ruby RSpec, Cucumber, selenium-webdriver gem Node.js Wd, Mocha, Vows PHP PHPUnit, Sausage, PHPWebDriver Java JUnit, TestNG, org.openqa.selenium.WebDriver

Slide 14

Slide 14 text

How Selenium Works

Slide 15

Slide 15 text

A Tale of Two Versions Selenium 1 (Selenium Remote Control) Selenium 2 (WebDriver) ^--- use this one

Slide 16

Slide 16 text

Selenium Demo

Slide 17

Slide 17 text

Selenium Best Practices Use spinAsserts Avoid brittle locators Run tests in parallel to the max Use state generators to save time

Slide 18

Slide 18 text

spinAsserts Selenium checks page state when you ask it This might be before an action’s result has changed the page state, e.g. for AJAX requests Selenium will return information which results in a test failure, but wouldn’t have if it had waited longer Solution: “spin” for responses

Slide 19

Slide 19 text

spinAsserts

Slide 20

Slide 20 text

spinAsserts

Slide 21

Slide 21 text

Avoid brittle locators Many different “locator strategies” in Selenium CSS, XPath, ID, link text, tag name, input name Don’t use locators which are likely to break during refactoring of your HTML Use sensible semantics for IDs and classes

Slide 22

Slide 22 text

Parallelism You don’t want to run Selenium tests serially Challenges of parallelism Tests need to be logically independent Tests can’t assume any state Beware of DoS-ing your test server Infrastructure / sysadmin costs

Slide 23

Slide 23 text

Parallelism: Bad

Slide 24

Slide 24 text

Parallelism: Good

Slide 25

Slide 25 text

State Generators

Slide 26

Slide 26 text

Parallelism to the max! Selenium Grid Open source Your machines, your VMs, your browsers, your Selenium Sauce Labs http://saucelabs.com

Slide 27

Slide 27 text

Sauce Labs PaaS analog of Selenium Grid All OSes and browsers that Selenium supports Infrastructure advantages No need to set up or maintain OSes, browsers, or Selenium No need to worry about scaling according to your growth Mobile is hard! Security advantages

Slide 28

Slide 28 text

Sauce Labs

Slide 29

Slide 29 text

Sauce Connect

Slide 30

Slide 30 text

Sauce Labs Feature advantages Videos of every test Automatic screenshots taken throughout test Log of Selenium commands and responses Breakpoints: jump into the browser when something goes wrong Team account management REST API etc...

Slide 31

Slide 31 text

Sauce Labs Demo

Slide 32

Slide 32 text

Sauce Labs Also check out: SeBuilder http://sebuilder.com Sauce for Mac http://saucelabs.com/downloads

Slide 33

Slide 33 text

Recap Selenium: tool for scripting browsers to act like humans Great for integration testing (which should be used judiciously as a part of every build) Be aware of common pitfalls and use Selenium best practices Sauce Labs: pain-free Selenium in the cloud Coding is better and funner than manual QA!

Slide 34

Slide 34 text

PS: If you haven’t automated yet It’s OK. Don’t be embarrassed. We’ve all been there. There’s still time... Do it now! Start small, demand value as you build test coverage. Focus on the money path. Never let bugs prevent someone from giving you $$. Be careful: it’s addictive!

Slide 35

Slide 35 text

Questions?

Slide 36

Slide 36 text

Thank you! If you like the Twitters: @jlipps Demo code: github.com/jlipps/sauce-php-demo PHP Library: github.com/jlipps/sausage