Upgrade to Pro — share decks privately, control downloads, hide ads and more …

WebDriver Internals

jarib
February 28, 2012

WebDriver Internals

Slides from my presentation at SeleniumCamp 2012 in Kiev

jarib

February 28, 2012
Tweet

More Decks by jarib

Other Decks in Programming

Transcript

  1. WebDriver Internals SeleniumCamp 2012

  2. Who am I? • Senior Test Engineer at FINN.no •

    Engineering Productivity team • Norway’s largest online marketplace • 900 million page views / month • 4 million unique users / month
  3. Who am I? @jarib http://github.com/jarib [email protected]finn.no

  4. Who am I?

  5. Browser automation 2003 2004 2005 2006 2007 2008 2009 2010

    2002 2011
  6. Browser automation Watir 2003 2004 2005 2006 2007 2008 2009

    2010 2002 2011
  7. Watir • Nice API • Native control over IE (using

    COM) + - • Ruby only • IE only (thus Windows only) • Internals not pretty
  8. Watir

  9. Browser automation Watir 2003 2004 2005 2006 2007 2008 2009

    2010 2002 2011
  10. Browser automation Watir Selenium 2003 2004 2005 2006 2007 2008

    2009 2010 2002 2011
  11. Selenium • Jason Huggins @ Thoughtworks • Needed to test

    in-house Time and Expenses app • Open source release in 2004
  12. Selenium Table-based syntax, inspired by FIT

  13. Selenium Table-based syntax, inspired by FIT "Selenium Core"

  14. Selenium Pure JavaScript Cross-domain policy prevents moving from one domain

    to the other ==
  15. Selenium Solution: HTTP server

  16. Selenium "Selenium Remote Control"

  17. Selenium • Languages: Java, Python, C#, Ruby ++ • Core

    written in JavaScript • Supports "all" browsers + - • Restricted by JS sandbox • Bloated, procedural API • Complicated architecture
  18. Selenium public void testNew() throws Exception { selenium.open("/"); selenium.type("q", "selenium

    rc"); selenium.click("btnG"); selenium.waitForPageToLoad("30000"); assertTrue(selenium.isTextPresent("Results * for selenium rc")); }
  19. Selenium

  20. Selenium WTF!?

  21. Selenium selenium.open "/" selenium.type 'q', 'selenium' puts selenium.title POST /selenium-server/driver/?cmd=open&1=%2F&sessionId=24f8bcc

    POST /selenium-server/driver/?cmd=type&1=q&2=selenium&sessionId=24f8bcc POST /selenium-server/driver/?cmd=getTitle&sessionId=24f8bcc
  22. Selenium selenium.type 'q', 'selenium' POST /selenium-server/driver/?cmd=type&1=q&2=selenium command name

  23. Selenium selenium.type 'q', 'selenium' POST /selenium-server/driver/?cmd=type&1=q&2=selenium argument 1

  24. Selenium selenium.type 'q', 'selenium' POST /selenium-server/driver/?cmd=type&1=q&2=selenium argument 2

  25. Browser automation Watir Selenium 2003 2004 2005 2006 2007 2008

    2009 2010 2002 2011
  26. Browser automation Watir Selenium WebDriver 2003 2004 2005 2006 2007

    2008 2009 2010 2002 2011
  27. WebDriver • Benefit of hindsight • design goal: small, object-oriented

    API • native code when needed, JavaScript elsewhere • still leverage HTTP as a transport • Language support (Java, Ruby, Python, C#) • Browser support • Firefox, Chrome, Opera, IE, iPhone, Android
  28. WebDriver

  29. Selenium

  30. Browser automation Watir Selenium WebDriver 2003 2004 2005 2006 2007

    2008 2009 2010 2002 2011
  31. Browser automation Watir Selenium WebDriver 2003 2004 2005 2006 2007

    2008 2009 2010 2002 2011
  32. Browser automation Watir Selenium WebDriver Selenium 2 Watir-Webdriver 2003 2004

    2005 2006 2007 2008 2009 2010 2002 2011
  33. Architectural themes • Emulate the user • Prove that the

    drivers work • You shouldn't need to understand everything • Every API call is an RPC
  34. Architectural themes Emulate the user

  35. Architectural themes • Designed to accurately emulate user interaction with

    a web application • Use native events where possible... • ...but make it easy to do cross-browser • API should match user interactions • e.g., no fireEvent() Emulate the user
  36. Architectural themes Prove the drivers work

  37. Architectural themes • Extensive automated test suite • Mostly integration

    tests • Strong culture for adding tests when fixing bugs • Challening CI setup • 6 browsers on 3 OSes with 4 languages • 72 builds per commit Prove the drivers work
  38. Architectural themes You shouldn't need to understand how everything works

  39. Architectural themes • Lots of languages / technologies in use

    • Architecture should allow developers to focus their talents where they'll be most productive You shouldn't need to understand how everything works
  40. Architectural themes Every API call is an RPC

  41. Architectural themes • Need to communicate with external browser process

    • Performance at the mercy of network latency • Introduces tension in the API design • Coarseness (== improved performance) vs expressiveness and ease of use Every API call is an RPC
  42. WebDriver Internals WebDriver API WebDriver SPI JSON wire protocol Browser

  43. API user facing

  44. API user facing SPI implementor facing

  45. driver.findElement(By.name("q")).sendKeys("webdriver"); API command –> { command : "findElement", parameters: {

    using: "name", value: "q" } } response <– { status: 0, value : <opaque element identifier> } command –> { command: "sendKeys", parameters: { element: <opaque element identifier>, value: ["webdriver"] } } response <– { status: 0 } SPI
  46. API command –> { command : "findElement", parameters: { using:

    "name", value: "q" } } response <– { status: 0, value : <opaque element identifier> } command –> { command: "sendKeys", parameters: { element: <opaque element identifier>, value: ["webdriver"] } } response <– { status: 0 } driver.findElement(By.name("q")).sendKeys("webdriver"); SPI
  47. API command –> { command : "findElement", parameters: { using:

    "name", value: "q" } } response <– { status: 0, value : <opaque element identifier> } command –> { command: "sendKeys", parameters: { element: <opaque element identifier>, value: ["webdriver"] } } response <– { status: 0 } driver.findElement(By.name("q")).sendKeys("webdriver"); SPI
  48. API object oriented SPI procedural

  49. API object oriented SPI procedural WebDriver driver = new FirefoxDriver();

    WebElement input = driver.findElement(By.name("q")); input.sendKeys("webdriver"); input.getAttribute("value"); newSession(browser: 'firefox') findElement(using: 'name', value: 'q') sendKeysToElement(id: '00cbe31e', value: ['webdriver']) getElementAttribute(id: '00cbe31e', name: 'value')
  50. API object oriented, user facing SPI procedural, implementor facing

  51. API SPI wire protocol browser automation atoms, native code

  52. API SPI browser automation standard reference implementation wire protocol browser

    automation atoms, native code
  53. Wire protocol • Reference implementation of the Service Provider Interface

    (SPI) • In spec terms, not a requirement for implementors • Though comes with some benefits • e.g. existing clients in multiple languages
  54. Wire protocol Language binding Java Ruby C# Python Browser Firefox

    Opera Chrome IE
  55. Wire protocol JSON HTTP Language binding Java Ruby C# Python

    Browser Firefox Opera Chrome IE
  56. Wire protocol JSON HTTP Language binding Java Ruby C# Python

    Browser Firefox Opera Chrome IE server client
  57. Wire protocol Machine A JSON HTTP Language binding Java Ruby

    C# Python Browser Firefox Opera Chrome IE
  58. Wire protocol JSON HTTP Machine B Remote WebDriver Server JSON

    HTTP Browser Firefox Opera Chrome IE Machine A Language bindings Java Ruby Python C#
  59. Wire protocol Selenium 2 Grid Hub JSON HTTP Grid Node

    JSON HTTP Language bindings Java Ruby Python C# Grid Node Grid Node CI server
  60. Wire protocol Maps SPI commands to RESTish HTTP resources POST

    /session POST /session/:sessionId/element POST /session/:sessionId/element/:id/value GET /session/:sessionId/element/:id/attribute/:name newSession(browser: 'firefox') findElement(sessionId: 'c688f8e4', using: 'name', value: 'q') sendKeysToElement(id: '00cbe31e', value: ['webdriver']) getElementAttribute(id: '00cbe31e', name: 'value')
  61. Wire protocol Maps SPI commands to RESTish HTTP resources POST

    /session POST /session/c688f8e4/element POST /session/c688f8e4/element/00cbe31e/value GET /session/c688f8e4/element/00cbe31e/attribute/value newSession(browser: 'firefox') findElement(sessionId: 'c688f8e4', using: 'name', value: 'q') sendKeysToElement(id: '00cbe31e', value: ['webdriver']) getElementAttribute(id: '00cbe31e', name: 'value')
  62. Wire protocol http://code.google.com/p/selenium/wiki/JsonWireProtocol

  63. Automation Atoms • Shared JavaScript library • Google Closure Compiler

    / Library • Advanced compilation
  64. Automation Atoms JavaScript Atoms (built on Google Closure Library) Google

    Closure Compiler Firefox extension (atoms.js) ChromeDriver (atoms.h) IE driver (atoms.h) Opera driver (OperaAtoms.java)
  65. Automation Atoms /** * @param {!Element} elem The element to

    consider. * @return {string} visible text. */ bot.dom.getVisibleText = function(elem) { var lines = []; bot.dom.appendVisibleTextLinesFromElement_(elem, lines); lines = goog.array.map( lines, bot.dom.trimExcludingNonBreakingSpaceCharacters_); var joined = lines.join('\n'); var trimmed = bot.dom.trimExcludingNonBreakingSpaceCharacters_(joined); // Replace non-breakable spaces with regular ones. return trimmed.replace(/\xa0/g, ' '); };
  66. Drivers client JS HTTPD wire protocol Firefox Dispatcher / Command

    Processor JS atoms XPCOM
  67. Drivers client wire protocol chromedriver mongoose.c HTTPD JS atoms Chrome

    Automation Proxy IPC
  68. Drivers client wire protocol IEDriver.dll mongoose.c HTTPD JS atoms IE

    COM APIs COM
  69. Drivers client wire protocol OperaDriver Jetty HTTPD JS atoms Opera

    Scope (protobuf)
  70. Demo Goal: Implement the ability to maximize the browser window

    Step 1: Ruby + Firefox Step 2: Java client + Remote Server driver.manage().window().maximize();
  71. None
  72. Getting involved • Ask for help and input early •

    Keep the change small • Write tests
  73. Getting involved • selenium-developers mailing list • #selenium on irc.freenode.net

    • http://code.google.com/p/selenium/ • http://code.google.com/p/selenium/issues/list • Look for the GettingInvolved label
  74. Questions?