Selenium: Less Testing, More Coding

A comprehensive introduction to Selenium and its place in the automated testing stack. Not really *less* testing, but less *painful* testing. Also, a guide to Sauce Labs.

Transcript

1. SELENIUM Less Testing, More Coding Jonathan Lipps (@jlipps) SDEC :: October 16, 2012
2. Introduction Who am I? Real subtitle: “Better testing with the fun of coding” Contributions of Sauce Labs colleagues @admc, @santiycr Assumptions and goals of this talk
3. Talk Outline Automated testing discussion Selenium & its place in the testing stack How Selenium works & demo Selenium gotchas & best practices Sauce Labs & demo Q & A
4. We all believe in automation, right? QA is necessary, but manual QA sucks Risk of unintended consequences grows with app complexity Cheap to run automated tests on every commit Writing automated tests is writing code Opens up possibility of Development Cycle of Optimal Happiness
5. The Development Cycle of Optimal Happiness
6. Automated Testing Landscape Unit tests Tests of specific methods/modules Integration points are mocked for speed/simplicity Integration / end-to-end / functional tests Tests of entire systems or integration points System is a functional black box from test’s perspective
9. The Challenge of Many Platforms Integration tests are better the closer they are to reality “Reality” in web environments is hard to simulate People clicking mouse buttons in browsers Different OSes, different browsers, different devices Takeaway: integration tests are more expensive
10. The Testing Stack of Optimal Happiness
11. The Place of Selenium Integration tests are a big hammer Get as much mileage as you can out of lighter-weight solutions first Debugging Selenium tests is more painful than debugging unit tests Nothing beats Selenium for uncovering cross-platform issues and legitimate stability concerns
12. How Selenium Works Client/server architecture Scripts speak Selenium API to the Selenium Server Selenium Server sends commands to browser Drivers, returns responses to script Independent Drivers for each browser/OS Bindings for speaking Selenium API to the server available in any programming language
13. How Selenium Works Language bindings Python Nose, unittest, selenium.driver C# NUnit, OpenQA.Selenium Ruby RSpec, Cucumber, selenium-webdriver gem Node.js Wd, Mocha, Vows PHP PHPUnit, Sausage, PHPWebDriver Java JUnit, TestNG, org.openqa.selenium.WebDriver
14. How Selenium Works
15. A Tale of Two Versions Selenium 1 (Selenium Remote Control) Selenium 2 (WebDriver) ^--- use this one
16. Selenium Demo
17. Selenium Best Practices Use spinAsserts Avoid brittle locators Run tests in parallel to the max Use state generators to save time
18. spinAsserts Selenium checks page state when you ask it This might be before an action’s result has changed the page state, e.g. for AJAX requests Selenium will return information which results in a test failure, but wouldn’t have if it had waited longer Solution: “spin” for responses
19. spinAsserts
20. spinAsserts
21. Avoid brittle locators Many different “locator strategies” in Selenium CSS, XPath, ID, link text, tag name, input name Don’t use locators which are likely to break during refactoring of your HTML Use sensible semantics for IDs and classes
22. Parallelism You don’t want to run Selenium tests serially Challenges of parallelism Tests need to be logically independent Tests can’t assume any state Beware of DoS-ing your test server Infrastructure / sysadmin costs
23. Parallelism: Bad
24. Parallelism: Good
25. State Generators
26. Parallelism to the max! Selenium Grid Open source Your machines, your VMs, your browsers, your Selenium Sauce Labs http://saucelabs.com
27. Sauce Labs PaaS analog of Selenium Grid All OSes and browsers that Selenium supports Infrastructure advantages No need to set up or maintain OSes, browsers, or Selenium No need to worry about scaling according to your growth Mobile is hard! Security advantages
28. Sauce Labs
29. Sauce Connect
30. Sauce Labs Feature advantages Videos of every test Automatic screenshots taken throughout test Log of Selenium commands and responses Breakpoints: jump into the browser when something goes wrong Team account management REST API etc...
31. Sauce Labs Demo
32. Sauce Labs Also check out: SeBuilder http://sebuilder.com Sauce for Mac http://saucelabs.com/downloads
33. Recap Selenium: tool for scripting browsers to act like humans Great for integration testing (which should be used judiciously as a part of every build) Be aware of common pitfalls and use Selenium best practices Sauce Labs: pain-free Selenium in the cloud Coding is better and funner than manual QA!
34. PS: If you haven’t automated yet It’s OK. Don’t be embarrassed. We’ve all been there. There’s still time... Do it now! Start small, demand value as you build test coverage. Focus on the money path. Never let bugs prevent someone from giving you $$. Be careful: it’s addictive!
35. Questions?
36. Thank you! If you like the Twitters: @jlipps Demo code: github.com/jlipps/sauce-php-demo PHP Library: github.com/jlipps/sausage