Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Integration and Functional Testing with LiveServerTestCase, Selenium and More.

Julien Phalip
September 05, 2013

Integration and Functional Testing with LiveServerTestCase, Selenium and More.

Talk given at DjangoCon US 2013. Chicago, IL.
The video: http://www.youtube.com/watch?v=BSYg9-sxSjM

Julien Phalip

September 05, 2013
Tweet

More Decks by Julien Phalip

Other Decks in Programming

Transcript

  1. CONNECTED PERSONAL OBJECTS 5/2012 About me... ‣ Worked with Django

    since 2007 (version 0.96). ‣ Django core committer since 2011.
  2. CONNECTED PERSONAL OBJECTS 5/2012 Definitions (subject to debate) ‣ Unit

    testing Ensure that small parts work in isolation.
  3. CONNECTED PERSONAL OBJECTS 5/2012 Definitions (subject to debate) ‣ Unit

    testing Ensure that small parts work in isolation. ‣ Integration testing Ensure that those parts work well together.
  4. CONNECTED PERSONAL OBJECTS 5/2012 Definitions (subject to debate) ‣ Unit

    testing Ensure that small parts work in isolation. ‣ Integration testing Ensure that those parts work well together. ‣ Functional testing Ensure that the application works, from a user’s perspective.
  5. CONNECTED PERSONAL OBJECTS 5/2012 Testing in Django (the traditional way)

    from django.test import TestCase class MyTests(TestCase): def test_hello(self): r = self.client.get('/hello') self.assertTrue('Hello' in r.content)
  6. CONNECTED PERSONAL OBJECTS 5/2012 Pretty good but incomplete... ‣ Does

    not exercise the full http protocol. ‣ Does not allow to test the user interface.
  7. CONNECTED PERSONAL OBJECTS 5/2012 django.test.LiveServerTestCase “Real” http client Database LiveServerTestCase

    http Application code DJANGO THREAD #2 DJANGO THREAD #1 Live http server wsgi
  8. CONNECTED PERSONAL OBJECTS 5/2012 django.test.LiveServerTestCase “Real” http client Database LiveServerTestCase

    http Application code DJANGO THREAD #2 DJANGO THREAD #1 Live http server wsgi
  9. CONNECTED PERSONAL OBJECTS 5/2012 django.test.LiveServerTestCase “Real” http client Database LiveServerTestCase

    http Application code DJANGO THREAD #2 DJANGO THREAD #1 Live http server wsgi
  10. CONNECTED PERSONAL OBJECTS 5/2012 django.test.LiveServerTestCase “Real” http client Database LiveServerTestCase

    http Application code DJANGO THREAD #2 DJANGO THREAD #1 Live http server wsgi
  11. CONNECTED PERSONAL OBJECTS 5/2012 django.test.LiveServerTestCase “Real” http client Database LiveServerTestCase

    http Application code DJANGO THREAD #2 DJANGO THREAD #1 Live http server wsgi
  12. CONNECTED PERSONAL OBJECTS 5/2012 django.test.LiveServerTestCase “Real” http client Database LiveServerTestCase

    http Application code DJANGO THREAD #2 DJANGO THREAD #1 Live http server wsgi
  13. CONNECTED PERSONAL OBJECTS 5/2012 Some important details ‣ Live server

    listens to localhost:8081 by default ‣ Address can be overridden: ./manage.py test --liveserver=localhost:8082 ./manage.py test --liveserver=localhost:9000-9200 ‣ Address accessible from the test’s code: self.live_server_url
  14. CONNECTED PERSONAL OBJECTS 5/2012 Testing with python-requests ‣ Testing REST

    APIs: import requests class MyTests(LiveServerTestCase): def test_api_user(self): r = requests.get( self.live_server_url + '/api/user/1/') self.assertEqual(r.status_code, 200) self.assertEqual(r.json, {'username': 'john.doe', 'email': '[email protected]'})
  15. CONNECTED PERSONAL OBJECTS 5/2012 Testing with python-requests ‣ Testing OAUTH

    workflows: import requests from requests_oauthlib import OAuth2 from oauthlib.oauth2 import BackendApplicationClient class MyTests(LiveServerTestCase): def test_oauth(self): backend = BackendApplicationClient('mybackend') oauth2 = OAuth2(client=backend, token=TOKEN) r = requests.get( self.live_server_url + '/oauth', auth=oauth2) self.assertEqual(r.status_code, 200) self.assertTrue('Hello' in r.content)
  16. CONNECTED PERSONAL OBJECTS 5/2012 Why use browser automators? ‣ Closer

    to real Web user environment. ‣ Media & static files (CSS/images/etc.) implicitly get loaded.
  17. CONNECTED PERSONAL OBJECTS 5/2012 Why use browser automators? ‣ Closer

    to real Web user environment. ‣ Media & static files (CSS/images/etc.) implicitly get loaded. ‣ Javascript code & Ajax calls implicitly get executed.
  18. CONNECTED PERSONAL OBJECTS 5/2012 Why use browser automators? ‣ Closer

    to real Web user environment. ‣ Media & static files (CSS/images/etc.) implicitly get loaded. ‣ Javascript code & Ajax calls implicitly get executed. ‣ User interactions can be tested.
  19. CONNECTED PERSONAL OBJECTS 5/2012 Why use browser automators? ‣ Closer

    to real Web user environment. ‣ Media & static files (CSS/images/etc.) implicitly get loaded. ‣ Javascript code & Ajax calls implicitly get executed. ‣ User interactions can be tested. ‣ Browser compatibility can be tested.
  20. CONNECTED PERSONAL OBJECTS 5/2012 Why use browser automators? ‣ Closer

    to real Web user environment. ‣ Media & static files (CSS/images/etc.) implicitly get loaded. ‣ Javascript code & Ajax calls implicitly get executed. ‣ User interactions can be tested. ‣ Browser compatibility can be tested. ‣ It’s fun!
  21. CONNECTED PERSONAL OBJECTS 5/2012 Selenium ‣ API to become W3C

    standard (currently in draft) ‣ Client libraries in Java, C#, .NET, Ruby, PHP, Perl, Javascript and Python.
  22. CONNECTED PERSONAL OBJECTS 5/2012 Selenium ‣ API to become W3C

    standard (currently in draft) ‣ Client libraries in Java, C#, .NET, Ruby, PHP, Perl, Javascript and Python. ‣ Works across multiple browsers (Firefox, Chrome, IE, Safari, Opera)
  23. CONNECTED PERSONAL OBJECTS 5/2012 Selenium ‣ API to become W3C

    standard (currently in draft) ‣ Client libraries in Java, C#, .NET, Ruby, PHP, Perl, Javascript and Python. ‣ Works across multiple browsers (Firefox, Chrome, IE, Safari, Opera) ‣ Easy to install: pip install selenium
  24. CONNECTED PERSONAL OBJECTS 5/2012 from django.test import LiveServerTestCase from selenium.webdriver.firefox.webdriver

    import \ WebDriver class MyTests(LiveServerTestCase): def setUp(self): self.driver = WebDriver() super(MyTests, self).setUp() def tearDown(self): self.driver.quit() super(MyTests, self).tearDown()
  25. CONNECTED PERSONAL OBJECTS 5/2012 from django.test import LiveServerTestCase from selenium.webdriver.firefox.webdriver

    import \ WebDriver class MyTests(LiveServerTestCase): def setUp(self): self.driver = WebDriver() super(MyTests, self).setUp() def tearDown(self): self.driver.quit() super(MyTests, self).tearDown()
  26. CONNECTED PERSONAL OBJECTS 5/2012 from django.test import LiveServerTestCase from selenium.webdriver.firefox.webdriver

    import \ WebDriver class MyTests(LiveServerTestCase): def test_hello(self): self.driver.get( self.live_server_url + '/hello') e = self.driver.find_element_by_id('btn') e.click() ... ...
  27. CONNECTED PERSONAL OBJECTS 5/2012 Testing visuals with Needle ‣ https://github.com/bfirsh/needle

    ‣ Based on Selenium. ‣ Tests visuals by comparing screenshots.
  28. CONNECTED PERSONAL OBJECTS 5/2012 from needle.cases import NeedleTestCase from from

    selenium.webdriver import firefox class MyTests(NeedleTestCase, LiveServerTestCase): driver_command_executor = firefox.ExtensionConnection( "127.0.0.1", firefox.FirefoxProfile()) def test_homepage_menu(self): self.driver.get(self.live_server_url) self.assertScreenshot('ul.menu', 'menu') Testing visuals with Needle
  29. CONNECTED PERSONAL OBJECTS 5/2012 ====================================================================== FAIL: test_homepage_menu (myapp.tests.MyTests) ---------------------------------------------------------------------- Traceback

    (most recent call last): File "myapp/tests.py", line 115, in test_homepage_menu self.assertScreenshot('ul.menu', 'menu') File "/.virtualenvs/myproject/src/needle/needle/cases.py", line 99, in assertScreenshot pass File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/ python2.6/contextlib.py", line 23, in __exit__ self.gen.next() File "/.virtualenvs/myproject/src/needle/needle/cases.py", line 160, in compareScreenshot % (filename, distance)) AssertionError: The saved screenshot for 'menu' did not match the screenshot captured (by a distance of 26.17) Testing visuals with Needle
  30. CONNECTED PERSONAL OBJECTS 5/2012 Test coverage ‣ Yes, it does

    work too. $ coverage run manage.py test \ myapp.SeleniumTests $ coverage report
  31. CONNECTED PERSONAL OBJECTS 5/2012 Headless mode for continuous integration ‣

    Install a browser and fonts: $ sudo apt-get install -y firefox xfonts-100dpi xfonts-75dpi xfonts- scalable xfonts-cyrillic
  32. CONNECTED PERSONAL OBJECTS 5/2012 Headless mode for continuous integration ‣

    Install a browser and fonts: $ sudo apt-get install -y firefox xfonts-100dpi xfonts-75dpi xfonts- scalable xfonts-cyrillic ‣ Install Xvfb: $ sudo apt-get install -y xvfb
  33. CONNECTED PERSONAL OBJECTS 5/2012 ‣ Either with the Xvfb command:

    $ Xvfb -ac :99 2>/dev/null & $ DISPLAY=:99 ./manage.py test Headless mode for continuous integration
  34. CONNECTED PERSONAL OBJECTS 5/2012 ‣ Either with the Xvfb command:

    $ Xvfb -ac :99 2>/dev/null & $ DISPLAY=:99 ./manage.py test ‣ ... or with Jenkins: Xvfb plugin. Headless mode for continuous integration
  35. CONNECTED PERSONAL OBJECTS 5/2012 ‣ Either with the Xvfb command:

    $ Xvfb -ac :99 2>/dev/null & $ DISPLAY=:99 ./manage.py test ‣ ... or with Jenkins: Xvfb plugin. ‣ ... or with Travis: before_install: - "export DISPLAY=:99.0" - "sh -e /etc/init.d/xvfb start" Headless mode for continuous integration
  36. CONNECTED PERSONAL OBJECTS 5/2012 ‣ ... or with PyVirtualDisplay: from

    pyvirtualdisplay import Display class MyTests(LiveServerTestCase) def setUp(self): self.display = Display( 'xvfb', visible=1,size=(1280, 1024)) self.display.start() def tearDown(self): self.display.stop() Headless mode for continuous integration
  37. CONNECTED PERSONAL OBJECTS 5/2012 ‣ Runs remotely in the cloud

    (no local browser needed). ‣ Gives access to multiple browsers (IE, FF, Opera, Chrome, Safari...) and multiple platforms (Windows, Linux, OSX, Android, iPad, iPhone). Sauce Labs
  38. CONNECTED PERSONAL OBJECTS 5/2012 ‣ Runs remotely in the cloud

    (no local browser needed). ‣ Gives access to multiple browsers (IE, FF, Opera, Chrome, Safari...) and multiple platforms (Windows, Linux, OSX, Android, iPad, iPhone). ‣ Allows tests to be run in parallel. Sauce Labs
  39. CONNECTED PERSONAL OBJECTS 5/2012 Ghost.py from ghost import Ghost class

    MyTests(LiveServerTestCase): def setUp(self): self.ghost = Ghost() super(MyTests, self).setUp() def test_hello(self): self.ghost.open(self.live_server_url + '/hello') self.assertTrue('Hello' in self.ghost.content) ‣ http://jeanphix.me/Ghost.py/
  40. CONNECTED PERSONAL OBJECTS 5/2012 CasperJS from casper.tests import CasperTestCase class

    MyTests(CasperTestCase): def test_something(self): self.assertTrue(self.casper('test.js')) ‣ http://casperjs.org/ ‣ https://github.com/dobarkod/django-casper
  41. CONNECTED PERSONAL OBJECTS 5/2012 Splinter from splinter import Browser class

    MyTests(LiveServerTestCase): def setUp(self): self.browser = Browser() super(MyTests, self).setUp() def test_hello(self): self.browser.visit(self.live_server_url + '/hello') self.assertTrue( self.browser.is_text_present('Hello')) ‣ http://splinter.cobrateam.info
  42. CONNECTED PERSONAL OBJECTS 5/2012 A word of caution... ‣ Selenium

    tests can be a little flaky. ‣ Integration & functional tests are slow. Use them with moderation.
  43. CONNECTED PERSONAL OBJECTS 5/2012 A word of caution... ‣ Selenium

    tests can be a little flaky. ‣ Integration & functional tests are slow. Use them with moderation. ‣ Use LiveServerTestCase or Selenium only for what the dummy client cannot already achieve.
  44. CONNECTED PERSONAL OBJECTS 5/2012 Integration & functional tests are important.

    ‣ Increase your confidence in your code. ‣ Increase your test coverage.
  45. CONNECTED PERSONAL OBJECTS 5/2012 Integration & functional tests are important.

    ‣ Increase your confidence in your code. ‣ Increase your test coverage. ‣ Test the integration frontend/backend.
  46. CONNECTED PERSONAL OBJECTS 5/2012 Integration & functional tests are important.

    ‣ Increase your confidence in your code. ‣ Increase your test coverage. ‣ Test the integration frontend/backend. ‣ Ensure the user interface works.
  47. CONNECTED PERSONAL OBJECTS 5/2012 Integration & functional tests are important.

    ‣ Increase your confidence in your code. ‣ Increase your test coverage. ‣ Test the integration frontend/backend. ‣ Ensure the user interface works. ‣ Have fun in the process!
  48. CONNECTED PERSONAL OBJECTS 5/2012 Photo credits ‣ http://www.flickr.com/photos/t0msk/3148160756/ ‣ http://www.flickr.com/photos/loozrboy/7311771718/

    ‣ http://www.flickr.com/photos/johnonolan/5836211096/ ‣ http://www.flickr.com/photos/randomskk/3057595640/