WebDriver Internals

Slides from my presentation at SeleniumCamp 2012 in Kiev

Transcript

1. WebDriver Internals SeleniumCamp 2012
2. Who am I? • Senior Test Engineer at FINN.no • Engineering Productivity team • Norway’s largest online marketplace • 900 million page views / month • 4 million unique users / month
3. Who am I? @jarib http://github.com/jarib jari@finn.no
4. Who am I?
5. Browser automation 2003 2004 2005 2006 2007 2008 2009 2010 2002 2011
6. Browser automation Watir 2003 2004 2005 2006 2007 2008 2009 2010 2002 2011
7. Watir • Nice API • Native control over IE (using COM) + - • Ruby only • IE only (thus Windows only) • Internals not pretty
8. Watir
9. Browser automation Watir 2003 2004 2005 2006 2007 2008 2009 2010 2002 2011
10. Browser automation Watir Selenium 2003 2004 2005 2006 2007 2008 2009 2010 2002 2011
11. Selenium • Jason Huggins @ Thoughtworks • Needed to test in-house Time and Expenses app • Open source release in 2004
12. Selenium Table-based syntax, inspired by FIT
13. Selenium Table-based syntax, inspired by FIT "Selenium Core"
14. Selenium Pure JavaScript Cross-domain policy prevents moving from one domain to the other ==
15. Selenium Solution: HTTP server
16. Selenium "Selenium Remote Control"
17. Selenium • Languages: Java, Python, C#, Ruby ++ • Core written in JavaScript • Supports "all" browsers + - • Restricted by JS sandbox • Bloated, procedural API • Complicated architecture
18. Selenium public void testNew() throws Exception { selenium.open("/"); selenium.type("q", "selenium rc"); selenium.click("btnG"); selenium.waitForPageToLoad("30000"); assertTrue(selenium.isTextPresent("Results * for selenium rc")); }
19. Selenium
20. Selenium WTF!?
21. Selenium selenium.open "/" selenium.type 'q', 'selenium' puts selenium.title POST /selenium-server/driver/?cmd=open&1=%2F&sessionId=24f8bcc POST /selenium-server/driver/?cmd=type&1=q&2=selenium&sessionId=24f8bcc POST /selenium-server/driver/?cmd=getTitle&sessionId=24f8bcc
22. Selenium selenium.type 'q', 'selenium' POST /selenium-server/driver/?cmd=type&1=q&2=selenium command name
23. Selenium selenium.type 'q', 'selenium' POST /selenium-server/driver/?cmd=type&1=q&2=selenium argument 1
24. Selenium selenium.type 'q', 'selenium' POST /selenium-server/driver/?cmd=type&1=q&2=selenium argument 2
25. Browser automation Watir Selenium 2003 2004 2005 2006 2007 2008 2009 2010 2002 2011
26. Browser automation Watir Selenium WebDriver 2003 2004 2005 2006 2007 2008 2009 2010 2002 2011
27. WebDriver • Benefit of hindsight • design goal: small, object-oriented API • native code when needed, JavaScript elsewhere • still leverage HTTP as a transport • Language support (Java, Ruby, Python, C#) • Browser support • Firefox, Chrome, Opera, IE, iPhone, Android
28. WebDriver
29. Selenium
30. Browser automation Watir Selenium WebDriver 2003 2004 2005 2006 2007 2008 2009 2010 2002 2011
31. Browser automation Watir Selenium WebDriver 2003 2004 2005 2006 2007 2008 2009 2010 2002 2011
32. Browser automation Watir Selenium WebDriver Selenium 2 Watir-Webdriver 2003 2004 2005 2006 2007 2008 2009 2010 2002 2011
33. Architectural themes • Emulate the user • Prove that the drivers work • You shouldn't need to understand everything • Every API call is an RPC
34. Architectural themes Emulate the user
35. Architectural themes • Designed to accurately emulate user interaction with a web application • Use native events where possible... • ...but make it easy to do cross-browser • API should match user interactions • e.g., no fireEvent() Emulate the user
36. Architectural themes Prove the drivers work
37. Architectural themes • Extensive automated test suite • Mostly integration tests • Strong culture for adding tests when fixing bugs • Challening CI setup • 6 browsers on 3 OSes with 4 languages • 72 builds per commit Prove the drivers work
38. Architectural themes You shouldn't need to understand how everything works
39. Architectural themes • Lots of languages / technologies in use • Architecture should allow developers to focus their talents where they'll be most productive You shouldn't need to understand how everything works
40. Architectural themes Every API call is an RPC
41. Architectural themes • Need to communicate with external browser process • Performance at the mercy of network latency • Introduces tension in the API design • Coarseness (== improved performance) vs expressiveness and ease of use Every API call is an RPC
42. WebDriver Internals WebDriver API WebDriver SPI JSON wire protocol Browser
43. API user facing
44. API user facing SPI implementor facing
45. driver.findElement(By.name("q")).sendKeys("webdriver"); API command –> { command : "findElement", parameters: { using: "name", value: "q" } } response <– { status: 0, value : <opaque element identifier> } command –> { command: "sendKeys", parameters: { element: <opaque element identifier>, value: ["webdriver"] } } response <– { status: 0 } SPI
46. API command –> { command : "findElement", parameters: { using: "name", value: "q" } } response <– { status: 0, value : <opaque element identifier> } command –> { command: "sendKeys", parameters: { element: <opaque element identifier>, value: ["webdriver"] } } response <– { status: 0 } driver.findElement(By.name("q")).sendKeys("webdriver"); SPI
47. API command –> { command : "findElement", parameters: { using: "name", value: "q" } } response <– { status: 0, value : <opaque element identifier> } command –> { command: "sendKeys", parameters: { element: <opaque element identifier>, value: ["webdriver"] } } response <– { status: 0 } driver.findElement(By.name("q")).sendKeys("webdriver"); SPI
48. API object oriented SPI procedural
49. API object oriented SPI procedural WebDriver driver = new FirefoxDriver(); WebElement input = driver.findElement(By.name("q")); input.sendKeys("webdriver"); input.getAttribute("value"); newSession(browser: 'firefox') findElement(using: 'name', value: 'q') sendKeysToElement(id: '00cbe31e', value: ['webdriver']) getElementAttribute(id: '00cbe31e', name: 'value')
50. API object oriented, user facing SPI procedural, implementor facing
51. API SPI wire protocol browser automation atoms, native code
52. API SPI browser automation standard reference implementation wire protocol browser automation atoms, native code
53. Wire protocol • Reference implementation of the Service Provider Interface (SPI) • In spec terms, not a requirement for implementors • Though comes with some benefits • e.g. existing clients in multiple languages
54. Wire protocol Language binding Java Ruby C# Python Browser Firefox Opera Chrome IE
55. Wire protocol JSON HTTP Language binding Java Ruby C# Python Browser Firefox Opera Chrome IE
56. Wire protocol JSON HTTP Language binding Java Ruby C# Python Browser Firefox Opera Chrome IE server client
57. Wire protocol Machine A JSON HTTP Language binding Java Ruby C# Python Browser Firefox Opera Chrome IE
58. Wire protocol JSON HTTP Machine B Remote WebDriver Server JSON HTTP Browser Firefox Opera Chrome IE Machine A Language bindings Java Ruby Python C#
59. Wire protocol Selenium 2 Grid Hub JSON HTTP Grid Node JSON HTTP Language bindings Java Ruby Python C# Grid Node Grid Node CI server
60. Wire protocol Maps SPI commands to RESTish HTTP resources POST /session POST /session/:sessionId/element POST /session/:sessionId/element/:id/value GET /session/:sessionId/element/:id/attribute/:name newSession(browser: 'firefox') findElement(sessionId: 'c688f8e4', using: 'name', value: 'q') sendKeysToElement(id: '00cbe31e', value: ['webdriver']) getElementAttribute(id: '00cbe31e', name: 'value')
61. Wire protocol Maps SPI commands to RESTish HTTP resources POST /session POST /session/c688f8e4/element POST /session/c688f8e4/element/00cbe31e/value GET /session/c688f8e4/element/00cbe31e/attribute/value newSession(browser: 'firefox') findElement(sessionId: 'c688f8e4', using: 'name', value: 'q') sendKeysToElement(id: '00cbe31e', value: ['webdriver']) getElementAttribute(id: '00cbe31e', name: 'value')
62. Wire protocol http://code.google.com/p/selenium/wiki/JsonWireProtocol
63. Automation Atoms • Shared JavaScript library • Google Closure Compiler / Library • Advanced compilation
64. Automation Atoms JavaScript Atoms (built on Google Closure Library) Google Closure Compiler Firefox extension (atoms.js) ChromeDriver (atoms.h) IE driver (atoms.h) Opera driver (OperaAtoms.java)
65. Automation Atoms /** * @param {!Element} elem The element to consider. * @return {string} visible text. */ bot.dom.getVisibleText = function(elem) { var lines = []; bot.dom.appendVisibleTextLinesFromElement_(elem, lines); lines = goog.array.map( lines, bot.dom.trimExcludingNonBreakingSpaceCharacters_); var joined = lines.join('\n'); var trimmed = bot.dom.trimExcludingNonBreakingSpaceCharacters_(joined); // Replace non-breakable spaces with regular ones. return trimmed.replace(/\xa0/g, ' '); };
66. Drivers client JS HTTPD wire protocol Firefox Dispatcher / Command Processor JS atoms XPCOM
67. Drivers client wire protocol chromedriver mongoose.c HTTPD JS atoms Chrome Automation Proxy IPC
68. Drivers client wire protocol IEDriver.dll mongoose.c HTTPD JS atoms IE COM APIs COM
69. Drivers client wire protocol OperaDriver Jetty HTTPD JS atoms Opera Scope (protobuf)
70. Demo Goal: Implement the ability to maximize the browser window Step 1: Ruby + Firefox Step 2: Java client + Remote Server driver.manage().window().maximize();
72. Getting involved • Ask for help and input early • Keep the change small • Write tests
73. Getting involved • selenium-developers mailing list • #selenium on irc.freenode.net • http://code.google.com/p/selenium/ • http://code.google.com/p/selenium/issues/list • Look for the GettingInvolved label
74. Questions?