Slide 1

Slide 1 text

~ Julien Phalip Advanced Techniques for Web Functional Testing April 12th, 2014 PYCON 2014 - MONTRÉAL

Slide 2

Slide 2 text

CONNECTED PERSONAL OBJECTS 5/2012 In brief: about me... ‣ Happy Python programmer since 2007. ‣ Django core committer since 2011. ‣ @julienphalip

Slide 3

Slide 3 text

CONNECTED PERSONAL OBJECTS 5/2012 www.nurun.com

Slide 4

Slide 4 text

CONNECTED PERSONAL OBJECTS 5/2012 Definitions ‣ Unit testing Ensure that small parts work in isolation. ‣ Integration testing Ensure that those parts work well together. ‣ Functional testing Ensure that the application works, from a user’s perspective.

Slide 5

Slide 5 text

Browser automators

Slide 6

Slide 6 text

CONNECTED PERSONAL OBJECTS 5/2012 Why use browser automators? ‣ Closer to real Web user environment. ‣ Media & static files (CSS/images/etc.) implicitly get loaded. ‣ Javascript code & Ajax calls implicitly get executed. ‣ User interactions can be tested. ‣ Browser compatibility can be tested. ‣ It’s fun!

Slide 7

Slide 7 text

CONNECTED PERSONAL OBJECTS 5/2012 Selenium ‣ WebDriver: API to become W3C standard (currently in draft) ‣ Works across multiple browsers (Firefox, Chrome, IE, Safari, Opera) ‣ Client libraries in Java, C#, .NET, Ruby, PHP, Perl, Javascript and Python. ‣ Easy to install: pip install selenium

Slide 8

Slide 8 text

CONNECTED PERSONAL OBJECTS 5/2012 DEMO Django core test suite

Slide 9

Slide 9 text

CONNECTED PERSONAL OBJECTS 5/2012 Testing responsive sites class MyResponsiveTests(TestCase): def test_responsive(self): # Mobile self.driver.set_window_size(600, 600) ... # Desktop self.driver.set_window_size(1200, 800) ...

Slide 10

Slide 10 text

CONNECTED PERSONAL OBJECTS 5/2012 DEMO Testing responsive sites

Slide 11

Slide 11 text

CONNECTED PERSONAL OBJECTS 5/2012 Testing visuals with Needle ‣ https://github.com/bfirsh/needle ‣ Based on Selenium. ‣ Catch regressions with styles, typography, SVG, images, etc. by comparing screenshots.

Slide 12

Slide 12 text

CONNECTED PERSONAL OBJECTS 5/2012 Needle: Example

Slide 13

Slide 13 text

CONNECTED PERSONAL OBJECTS 5/2012 Needle: Example class BannerTests(NeedleTestCase): def test_banner(self): self.driver.get(‘http://us.pycon.org/’) self.assertScreenshot( '.banner', # CSS selector 'banner' # Filename (PNG) )

Slide 14

Slide 14 text

CONNECTED PERSONAL OBJECTS 5/2012 DEMO Testing visuals with Needle (CSS & SVG)

Slide 15

Slide 15 text

CONNECTED PERSONAL OBJECTS 5/2012 Needle Tips ‣ Baseline screenshots == Fixtures ‣ Store them in your repository ‣ Ensure consistency between environments (browser version, fonts, etc.) to avoid false positives.

Slide 16

Slide 16 text

Continuous integration

Slide 17

Slide 17 text

CONNECTED PERSONAL OBJECTS 5/2012 ‣ Install browser: $ wget https://ftp.mozilla.org/pub/mozilla.org/ firefox/releases/27.0/linux-x86_64/en-US/ firefox-27.0.tar.bz2 $ tar jxpvf firefox-27.0.tar.bz2 $ sudo ln -s "`pwd`/firefox/firefox" /usr/bin/firefox ‣ Install fonts: $ sudo apt-get install -y xfonts-100dpi xfonts-75dpi xfonts-scalable xfonts-cyrillic ‣ Install Xvfb: $ sudo apt-get install -y xvfb Headless mode ( 1/4 )

Slide 18

Slide 18 text

CONNECTED PERSONAL OBJECTS 5/2012 ‣ Either with the Xvfb command: $ Xvfb -ac :99 2>/dev/null & $ export DISPLAY=:99 ‣ ... or with Jenkins: Xvfb plugin. ‣ ... or with Travis: before_install: - "sh -e /etc/init.d/xvfb start" - "export DISPLAY=:99.0" Headless mode ( 2/4 )

Slide 19

Slide 19 text

CONNECTED PERSONAL OBJECTS 5/2012 ‣ ... or with PyVirtualDisplay: from pyvirtualdisplay import Display class MyTests(TestCase): def setUp(self): self.display = Display( 'xvfb', visible=1,size=(1280, 1024)) self.display.start() def tearDown(self): self.display.stop() Headless mode ( 3/4 )

Slide 20

Slide 20 text

CONNECTED PERSONAL OBJECTS 5/2012 ‣ ... or with PhantomJS WebDriver: from selenium.webdriver import PhantomJS class MyTests(TestCase): def setUp(self): self.driver = PhantomJS() Headless mode ( 4/4 )

Slide 21

Slide 21 text

CONNECTED PERSONAL OBJECTS 5/2012 ‣ Runs remotely in the cloud (no local browser needed). ‣ Gives access to multiple browsers (IE, FF, Opera, Chrome, Safari...) and multiple platforms (Windows, Linux, OSX, Android, iPad, iPhone). ‣ Allows tests to be run in parallel. ‣ Keeps video recordings of all sessions. Sauce Labs

Slide 22

Slide 22 text

CONNECTED PERSONAL OBJECTS 5/2012 DEMO SauceLabs

Slide 23

Slide 23 text

CONNECTED PERSONAL OBJECTS 5/2012

Slide 24

Slide 24 text

CONNECTED PERSONAL OBJECTS 5/2012

Slide 25

Slide 25 text

Speed and robustness

Slide 26

Slide 26 text

CONNECTED PERSONAL OBJECTS 5/2012 Optimizing for speed ‣ Use PhantomJS (headless mode) ‣ Use parallelization (e.g. nosetests --processes=10) ‣ Bypass common user workflows with fake views or with test fixtures.

Slide 27

Slide 27 text

CONNECTED PERSONAL OBJECTS 5/2012 Optimizing for robustness ‣ Pin down versions for your whole software stack (incl. browser) ‣ Master the use of explicit waits (esp. with heavy JS/Ajax apps) ‣ Use lowest common denominator of WebDriver API implementations (for cross-browser testing)

Slide 28

Slide 28 text

CONNECTED PERSONAL OBJECTS 5/2012 ‣ The Selenium Guidebook http://seleniumguidebook.com/ ‣ Selenium Testing Tools Cookbook http://www.packtpub.com/recipes-to- master-selenium-2-testing-tools-cookbook/ book More tips...

Slide 29

Slide 29 text

Epilogue...

Slide 30

Slide 30 text

CONNECTED PERSONAL OBJECTS 5/2012 A word of caution... ‣ Integration & functional tests are slow. Use them with moderation. ‣ Use Selenium only for what a dummy test client may not already achieve (interactive user interfaces, visuals, complex user workflows).

Slide 31

Slide 31 text

CONNECTED PERSONAL OBJECTS 5/2012 Integration & functional tests are important. ‣ Increase your confidence in your code. ‣ Increase your test coverage. ‣ Test the integration frontend/backend. ‣ Ensure that the user interface works and looks as expected. ‣ Have fun in the process!

Slide 32

Slide 32 text

CONNECTED PERSONAL OBJECTS 5/2012 Photo credits ‣ http://www.flickr.com/photos/t0msk/3148160756/ ‣ http://www.flickr.com/photos/loozrboy/7311771718/ ‣ http://www.flickr.com/photos/randomskk/3057595640/

Slide 33

Slide 33 text

@julienphalip http:/ /julienphalip.com http:/ /nurun.com Thank you!