Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Advanced Techniques for Web Functional Testing

Advanced Techniques for Web Functional Testing

Presented at PyCon, Montréal 2014.
The video: https://www.youtube.com/watch?v=St9cL47_1GI

7617093fdae20c6fe4b0f7fe954a8aad?s=128

Julien Phalip

April 12, 2014
Tweet

Transcript

  1. ~ Julien Phalip Advanced Techniques for Web Functional Testing April

    12th, 2014 PYCON 2014 - MONTRÉAL
  2. CONNECTED PERSONAL OBJECTS 5/2012 In brief: about me... ‣ Happy

    Python programmer since 2007. ‣ Django core committer since 2011. ‣ @julienphalip
  3. CONNECTED PERSONAL OBJECTS 5/2012 www.nurun.com

  4. CONNECTED PERSONAL OBJECTS 5/2012 Definitions ‣ Unit testing Ensure that

    small parts work in isolation. ‣ Integration testing Ensure that those parts work well together. ‣ Functional testing Ensure that the application works, from a user’s perspective.
  5. Browser automators

  6. CONNECTED PERSONAL OBJECTS 5/2012 Why use browser automators? ‣ Closer

    to real Web user environment. ‣ Media & static files (CSS/images/etc.) implicitly get loaded. ‣ Javascript code & Ajax calls implicitly get executed. ‣ User interactions can be tested. ‣ Browser compatibility can be tested. ‣ It’s fun!
  7. CONNECTED PERSONAL OBJECTS 5/2012 Selenium ‣ WebDriver: API to become

    W3C standard (currently in draft) ‣ Works across multiple browsers (Firefox, Chrome, IE, Safari, Opera) ‣ Client libraries in Java, C#, .NET, Ruby, PHP, Perl, Javascript and Python. ‣ Easy to install: pip install selenium
  8. CONNECTED PERSONAL OBJECTS 5/2012 DEMO Django core test suite

  9. CONNECTED PERSONAL OBJECTS 5/2012 Testing responsive sites class MyResponsiveTests(TestCase): def

    test_responsive(self): # Mobile self.driver.set_window_size(600, 600) ... # Desktop self.driver.set_window_size(1200, 800) ...
  10. CONNECTED PERSONAL OBJECTS 5/2012 DEMO Testing responsive sites

  11. CONNECTED PERSONAL OBJECTS 5/2012 Testing visuals with Needle ‣ https://github.com/bfirsh/needle

    ‣ Based on Selenium. ‣ Catch regressions with styles, typography, SVG, images, etc. by comparing screenshots.
  12. CONNECTED PERSONAL OBJECTS 5/2012 Needle: Example

  13. CONNECTED PERSONAL OBJECTS 5/2012 Needle: Example class BannerTests(NeedleTestCase): def test_banner(self):

    self.driver.get(‘http://us.pycon.org/’) self.assertScreenshot( '.banner', # CSS selector 'banner' # Filename (PNG) )
  14. CONNECTED PERSONAL OBJECTS 5/2012 DEMO Testing visuals with Needle (CSS

    & SVG)
  15. CONNECTED PERSONAL OBJECTS 5/2012 Needle Tips ‣ Baseline screenshots ==

    Fixtures ‣ Store them in your repository ‣ Ensure consistency between environments (browser version, fonts, etc.) to avoid false positives.
  16. Continuous integration

  17. CONNECTED PERSONAL OBJECTS 5/2012 ‣ Install browser: $ wget https://ftp.mozilla.org/pub/mozilla.org/

    firefox/releases/27.0/linux-x86_64/en-US/ firefox-27.0.tar.bz2 $ tar jxpvf firefox-27.0.tar.bz2 $ sudo ln -s "`pwd`/firefox/firefox" /usr/bin/firefox ‣ Install fonts: $ sudo apt-get install -y xfonts-100dpi xfonts-75dpi xfonts-scalable xfonts-cyrillic ‣ Install Xvfb: $ sudo apt-get install -y xvfb Headless mode ( 1/4 )
  18. CONNECTED PERSONAL OBJECTS 5/2012 ‣ Either with the Xvfb command:

    $ Xvfb -ac :99 2>/dev/null & $ export DISPLAY=:99 ‣ ... or with Jenkins: Xvfb plugin. ‣ ... or with Travis: before_install: - "sh -e /etc/init.d/xvfb start" - "export DISPLAY=:99.0" Headless mode ( 2/4 )
  19. CONNECTED PERSONAL OBJECTS 5/2012 ‣ ... or with PyVirtualDisplay: from

    pyvirtualdisplay import Display class MyTests(TestCase): def setUp(self): self.display = Display( 'xvfb', visible=1,size=(1280, 1024)) self.display.start() def tearDown(self): self.display.stop() Headless mode ( 3/4 )
  20. CONNECTED PERSONAL OBJECTS 5/2012 ‣ ... or with PhantomJS WebDriver:

    from selenium.webdriver import PhantomJS class MyTests(TestCase): def setUp(self): self.driver = PhantomJS() Headless mode ( 4/4 )
  21. CONNECTED PERSONAL OBJECTS 5/2012 ‣ Runs remotely in the cloud

    (no local browser needed). ‣ Gives access to multiple browsers (IE, FF, Opera, Chrome, Safari...) and multiple platforms (Windows, Linux, OSX, Android, iPad, iPhone). ‣ Allows tests to be run in parallel. ‣ Keeps video recordings of all sessions. Sauce Labs
  22. CONNECTED PERSONAL OBJECTS 5/2012 DEMO SauceLabs

  23. CONNECTED PERSONAL OBJECTS 5/2012

  24. CONNECTED PERSONAL OBJECTS 5/2012

  25. Speed and robustness

  26. CONNECTED PERSONAL OBJECTS 5/2012 Optimizing for speed ‣ Use PhantomJS

    (headless mode) ‣ Use parallelization (e.g. nosetests --processes=10) ‣ Bypass common user workflows with fake views or with test fixtures.
  27. CONNECTED PERSONAL OBJECTS 5/2012 Optimizing for robustness ‣ Pin down

    versions for your whole software stack (incl. browser) ‣ Master the use of explicit waits (esp. with heavy JS/Ajax apps) ‣ Use lowest common denominator of WebDriver API implementations (for cross-browser testing)
  28. CONNECTED PERSONAL OBJECTS 5/2012 ‣ The Selenium Guidebook http://seleniumguidebook.com/ ‣

    Selenium Testing Tools Cookbook http://www.packtpub.com/recipes-to- master-selenium-2-testing-tools-cookbook/ book More tips...
  29. Epilogue...

  30. CONNECTED PERSONAL OBJECTS 5/2012 A word of caution... ‣ Integration

    & functional tests are slow. Use them with moderation. ‣ Use Selenium only for what a dummy test client may not already achieve (interactive user interfaces, visuals, complex user workflows).
  31. CONNECTED PERSONAL OBJECTS 5/2012 Integration & functional tests are important.

    ‣ Increase your confidence in your code. ‣ Increase your test coverage. ‣ Test the integration frontend/backend. ‣ Ensure that the user interface works and looks as expected. ‣ Have fun in the process!
  32. CONNECTED PERSONAL OBJECTS 5/2012 Photo credits ‣ http://www.flickr.com/photos/t0msk/3148160756/ ‣ http://www.flickr.com/photos/loozrboy/7311771718/

    ‣ http://www.flickr.com/photos/randomskk/3057595640/
  33. @julienphalip http:/ /julienphalip.com http:/ /nurun.com Thank you!