WebDriver Internals Test Automation Bazaar 2012 

Who am I? • Senior Test Engineer at FINN.no • Engineering Productivity team • Norway’s largest online marketplace • 900 million page views / month • 4 million unique users / month 

Who am I? @jarib http://github.com/jarib jari@finn.no 

Who am I? 

Browser automation Watir Selenium WebDriver 2003 2004 2005 2006 2007 2008 2009 2010 2002 2011 

Browser automation Watir Selenium WebDriver 2003 2004 2005 2006 2007 2008 2009 2010 2002 2011 

Browser automation Watir Selenium WebDriver Selenium 2 Watir-Webdriver 2003 2004 2005 2006 2007 2008 2009 2010 2002 2011 

WebDriver • Selenium 2: benefit of hindsight • design goal: small, object-oriented API • native code when needed, JavaScript elsewhere • still leverage HTTP as a transport • Language support • Java, Ruby, Python, C#, JavaScript, (PHP, Perl) • Browser support • Firefox, Chrome, Opera, IE, Safari, iOS, Android, HtmlUnit 

Selenium 

WebDriver 

Architectural themes • Emulate the user • Prove that the drivers work • You shouldn't need to understand everything • Every API call is an RPC 

Architectural themes Emulate the user 

Architectural themes • Designed to accurately emulate user interaction with a web application • Use native events where possible... • ...but make it easy to do cross-browser • API should match user interactions • e.g. no #fire_event Emulate the user 

Architectural themes Prove the drivers work 

Architectural themes • Extensive automated test suite • Mostly integration tests • Strong culture for adding tests when fixing bugs • Challening CI setup • 6 browsers on 3 OSes with 4 languages • 72 builds per commit Prove the drivers work 

Architectural themes You shouldn't need to understand how everything works 

Architectural themes • Lots of languages / technologies in use • Architecture should allow developers to focus their talents where they'll be most productive You shouldn't need to understand how everything works 

Architectural themes Every API call is an RPC 

Architectural themes • Need to communicate with external browser process • Performance at the mercy of network latency • Introduces tension in the API design • Coarseness (== improved performance) • Expressiveness, ease of use Every API call is an RPC 

Future • Browser automation standard • Make browser vendors to care • Delete implementations from the Selenium tree 

WebDriver Internals WebDriver API WebDriver SPI JSON wire protocol Browser 

API user facing 

API user facing SPI implementor facing 

API command –> { name: "findElement", parameters: { using: "name", value: "q" } } response <– { status: 0, value : } command –> { name: "sendKeys", parameters: { element: , value: ["webdriver"] } } response <– { status: 0 } SPI driver.find_element(name: 'q').send_keys('webdriver') 

API command –> { name: "findElement", parameters: { using: "name", value: "q" } } response <– { status: 0, value : } command –> { name: "sendKeys", parameters: { element: , value: ["webdriver"] } } response <– { status: 0 } SPI driver.find_element(name: 'q').send_keys('webdriver') 

driver.find_element(name: 'q').send_keys('webdriver') API command –> { name: "findElement", parameters: { using: "name", value: "q" } } response <– { status: 0, value : } command –> { name: "sendKeys", parameters: { element: , value: ["webdriver"] } } response <– { status: 0 } SPI 

API object oriented SPI procedural newSession(browser: 'firefox') findElement(using: 'name', value: 'q') sendKeysToElement(id: '00cbe31e', value: ['webdriver']) getElementAttribute(id: '00cbe31e', name: 'value') driver = Selenium::WebDriver.for :firefox input = driver.find_element(name: 'q') input.send_keys('webdriver') puts input.attribute('value') 

API object oriented, user facing SPI procedural, implementor facing 

API SPI wire protocol browser automation atoms, native code 

API SPI browser automation standard reference implementation wire protocol browser automation atoms, native code 

Wire protocol • Reference implementation of the Service Provider Interface (SPI) • In spec terms, non-normative (not a requirement for implementors) • Though comes with some benefits • e.g. existing clients in multiple languages 

Wire protocol Language binding Java Ruby C# Python Browser Firefox Opera Chrome IE 

Wire protocol JSON HTTP Language binding Java Ruby C# Python Browser Firefox Opera Chrome IE 

Wire protocol JSON HTTP Language binding Java Ruby C# Python Browser Firefox Opera Chrome IE server client 

Wire protocol Machine A JSON HTTP Language binding Java Ruby C# Python Browser Firefox Opera Chrome IE 

Wire protocol JSON HTTP Machine B Remote WebDriver Server JSON HTTP Browser Firefox Opera Chrome IE Machine A Language bindings Java Ruby Python C# 

Wire protocol Selenium 2 Grid Hub JSON HTTP Grid Node JSON HTTP Language bindings Java Ruby Python C# Grid Node Grid Node CI server 

Wire protocol Maps SPI commands to RESTish HTTP resources POST /session POST /session/:sessionId/element POST /session/:sessionId/element/:id/value GET /session/:sessionId/element/:id/attribute/:name newSession(browser: 'firefox') findElement(sessionId: 'c688f8e4', using: 'name', value: 'q') sendKeysToElement(id: '00cbe31e', value: ['webdriver']) getElementAttribute(id: '00cbe31e', name: 'value') 

Wire protocol Maps SPI commands to RESTish HTTP resources POST /session POST /session/c688f8e4/element POST /session/c688f8e4/element/00cbe31e/value GET /session/c688f8e4/element/00cbe31e/attribute/value newSession(browser: 'firefox') findElement(sessionId: 'c688f8e4', using: 'name', value: 'q') sendKeysToElement(id: '00cbe31e', value: ['webdriver']) getElementAttribute(id: '00cbe31e', name: 'value') 

Wire protocol http://code.google.com/p/selenium/wiki/JsonWireProtocol 

Automation Atoms • Shared JavaScript library • Google Closure Compiler / Library • Advanced compilation 

Automation Atoms JavaScript Atoms (built on Google Closure Library) Google Closure Compiler Firefox extension (atoms.js) ChromeDriver (atoms.h) IE driver (atoms.h) Opera driver (OperaAtoms.java) 

Automation Atoms /** * @param {!Element} elem The element to consider. * @return {string} visible text. */ bot.dom.getVisibleText = function(elem) { var lines = []; bot.dom.appendVisibleTextLinesFromElement_(elem, lines); lines = goog.array.map( lines, bot.dom.trimExcludingNonBreakingSpaceCharacters_); var joined = lines.join('\n'); var trimmed = bot.dom.trimExcludingNonBreakingSpaceCharacters_(joined); // Replace non-breakable spaces with regular ones. return trimmed.replace(/\xa0/g, ' '); }; 

Drivers client JS HTTPD wire protocol Firefox Dispatcher / Command Processor JS atoms XPCOM 

Drivers client wire protocol chromedriver mongoose.c HTTPD JS atoms Chrome Automation Proxy IPC 

Drivers client wire protocol IEDriver.dll mongoose.c HTTPD JS atoms IE COM APIs COM 

Drivers client wire protocol OperaDriver Jetty HTTPD JS atoms Opera Scope (protobuf) 

watir-webdriver • Watir implementation on WebDriver • Started 2009 • http://watirwebdriver.com/ • http://github.com/watir/watir-webdriver 

watir-webdriver selenium-webdriver driver = Selenium::WebDriver.for :firefox browser = Watir::Browser.new(driver) 

selenium-webdriver watir-webdriver >> driver.find_element(:id => "country") => # >> browser.select_list(:id => "country") => #"country", :ta >> d.text_field(:name => "user") => #

Watir::Anchor Watir::Applet Watir::Area Watir::Audio Watir::BR Watir::Base Watir::BaseFont Watir::Body Watir::Button Watir::Canvas Watir::CheckBox Watir::Command Watir::DList Watir::DataList Watir::Details Watir::Device Watir::Directory Watir::Div Watir::Embed Watir::FieldSet Watir::FileField Watir::Font Watir::Form Watir::Frame Watir::FrameSet Watir::HR Watir::HTMLElement Watir::Head Watir::Heading Watir::Hidden Watir::Html Watir::IFrame Watir::Image Watir::Input Watir::Keygen Watir::LI Watir::Label Watir::Legend Watir::Map Watir::Marquee Watir::Media Watir::Menu Watir::Meta Watir::Meter Watir::Mod Watir::OList Watir::Object Watir::OptGroup Watir::Option Watir::Output Watir::Paragraph Watir::Param Watir::Pre Watir::Progress Watir::Quote Watir::Radio Watir::Script Watir::Select Watir::Source Watir::Span Watir::Style Watir::Table Watir::TableCaption Watir::TableCell Watir::TableCol Watir::TableDataCel Watir::TableHeaderC Watir::TableRow Watir::TableSection Watir::TextArea Watir::TextField Watir::Time Watir::Title Watir::Track Watir::UList Watir::Unknown Watir::Video 

Ruby code generated from the HTML spec module Watir class Meter < HTMLElement attributes( :float => [:value, :min, :max, :low, :high, :optimum], :html_element => [:form], :list => [:labels] ) end end 

Voter turnout: 75% >> meter = browser.meter(:id => "turnout") #=> #> meter.value #=> 0.75 >> meter.value.class #=> Float 

Comparison with Watir 1 • Supports all browsers available in WebDriver • Mostly compatible API-wise. Some major changes: • 0-indexed instead of 1-indexed • All HTML elements supported • Revised table API • New window switching API • Collection methods can take locator arguments https://github.com/watir/watir-webdriver/wiki/Comparison-with-Watir-1.X 

Demo Goal: Implement the ability to maximize the browser window Step 1: selenium-webdriver Firefox Step 2: watir-webdriver browser.window.maximize 

Getting involved • Ask for help and input early • Keep the change small • Write tests 

Getting involved • Mailing lists: • wtr-development • selenium-developers • IRC: • irc.freenode.net (#watir or #selenium) • Code: • http://code.google.com/p/selenium/ • https://github.com/watir/watir-webdriver 

Questions?