Remotely Control this Browser: WebDriver and the Path to an Interoperable Web

Notes: http://www.erranderr.com/blog/presentations/webdriver_osb2017

Session: http://opensourcebridge.org/sessions/1997

You can write simple code to automate your browser and simulate user interaction. Moreover, thanks to the people behind a W3C standard called WebDriver, some day soon you can expect to write just one script in your favourite programming language to automate all major browsers in the same way.

The WebDriver protocol can be used for testing web applications, reporting and reproducing web-compatibility bugs, gathering browser data and whatever else you can think to do with browser automation. It’s closely tied to Selenium, a well-known, open-source browser automation tool.

This talk explains why WebDriver is becoming a standard and how you can work on it with us. Using Firefox as our example, we explore why browser automation is tricky to get right. How do WebDriver commands get to Firefox? How does the browser figure out the correct response? What does “correct” even mean here?

Today, all major browsers work with WebDriver to varying degrees. As part of our mission to make the web interoperable, Mozilla is leading the way to fully implementing the WebDriver standard and we welcome your contributions.

Transcript

1. Remotely Control This Browser WebDriver and the Path to an Interoperable Web Maja Frydrychowicz Open Source Bridge 2017 1
2. Hello, I'm Maja. So ware Engineer at Mozilla @impossibus Excerpt from https://xkcd.com/1629/ 2
3. Example of Remote Control 3
4. Web Compatibility 4
5. Pain "lol web development" -@notwaldorf 5
6. Web Predictability 6
7. Web Standards ➔ Interoperable 7
8. Interoperability open instead of fragmented 8
9. Tools for a healthy web 9
10. Coming up 1. Why browser automation? 2. What is WebDriver? Selenium? 3. How does WebDriver work in Firefox? 4. Hey, let's add a WebDriver command to Firefox! 10
11. Welcome to browser automation 11 . 1
12. Example session.get("https://www.mozilla.org/") link = session.find_element_by_id("participate") link.click() 11 . 2
13. Turned off by default Visual cue in URL bar 11 . 3
14. Why automate? Testing... 11 . 4
15. Beyond Testing 11 . 5
16. How? WebDriver! 11 . 6
17. WebDriver Client Sends Commands to Browser session.get("https://www.mozilla.org/") link = session.find_element_by_id("participate") link.click() 11 . 7
18. Example: Talking to Firefox Automation script: pip install selenium export PATH=$PATH:/path/to/geckodriver from selenium import webdriver # e.g. check the web page title session = webdriver.Firefox() session.get("http://www.python.org") assert "Python" in session.title session.close() 11 . 8
19. ✨open source project to web standard!✨ Selenium WebDriver ➔ W3C WebDriver Standard 11 . 9
20. Each WebDriver command is an HTTP request with a JSON payload POST /session/<id>/window/maximize POST /session/<id>/element GET /session/<id>/title GET /session/<id>/screenshot 11 . 10
21. WebDriver Protocol Request: please navigate to mozilla.org Response: command was successful! POST /session/0152f...a9e8/url {"url": "https://www.mozilla.org/"} Status: 200 Data: {} 11 . 11
22. Spec Implementation Status How many WebDriver commands work in each browser? 11 . 12
23. Aside: Web Platform Tests testthewebforward.org 11 . 13
24. WebDriver in Firefox 12 . 1
25. geckodriver + Marionette 12 . 2
26. Firefox: Why not JSON over HTTP? Based on existing remote debugging protocol instead. 12 . 3
27. You've started a WebDriver session. What happens when you send the "Get Title" command? print session.title 12 . 4
28. Processing a Command (1) Client Send HTTP request to geckodriver GET /session/<session id>/title 12 . 5
29. Processing a Command (2) geckodriver Translate and send to Marionette The result is a message in the format : 30:[0, 21, "getTitle", null] 12 . 6
30. Quick peek inside geckodriver (Get, "/session/{sessionId}/title", Route::GetTitle) GetTitle => (Some("getTitle"), None) impl ToJson for MarionetteCommand { fn to_json(&self) -> Json { Json::Array(vec![Json::U64(0), self.id.to_json(), self.name.to_json(), self.params.to_json()]) } } 12 . 7
31. Processing a Command (3) Marionette Parse stream of incoming packets (e.g. from geckodriver) Request parsed into JSON: type, id, method, params [0,21,"getTitle",{}] 12 . 8
32. Processing a Command (4) Marionette Dispatch cmd ("getTitle") to the right driver method execute(cmd) { let fn = this.driver.commands[cmd.name]; if (typeof fn == "undefined") { throw new UnknownCommandError(cmd.name); } let rv = yield fn.bind(this.driver)(cmd, resp); // send resp } 12 . 9
33. Processing a Command (5) Marionette Perform browser interaction to compute response. Example response: type, id, error, result function getTitle() { return curContainer.frame.top.document.title; } [1,21,null,{"value":"Cats in Boxes"}] 12 . 10
34. Processing a Command (6) Finally, the response is sent back, translated by geckodriver to JSON-over-HTTP: Status: 200 Data: {"value":"Cats in Boxes"} 12 . 11
35. Implementing a Command 13 . 1
36. Example: The "Actions" Command POST /session/<id>/actions {"actions": [{ "parameters": {"pointerType": "mouse"}, "type": "pointer", "id": "myMouse", "actions": [ {"type": "pointerMove", "y": 10, "x": 80, "duration": 50}, {"type": "pointerDown", "button": 0}, {"type": "pointerUp", "button": 0} ] }]} 13 . 2
37. Read the spec https://www.w3.org/TR/webdriver/ 13 . 3
38. Read the spec Spec describes what should happen and o en how (at a high level) Not set in stone: clarify and discuss as we implement 13 . 4
39. Write web-platform tests Examples: Invalid payload is rejected with the correct kind of error. pointerDown followed by pointerUp results in expected events. pointerMove ends up at correct coordinates. 13 . 5
40. Add to marionette + geckodriver First, parse and validate the command payload 13 . 6
41. Control the order of DOM events action.dispatch = function (chain, ...) { let chainEvents = Task.spawn(function* () { for (let tickActs of chain) { yield action.dispatchTickActions( tickActs, action.computeTickDuration(tickActs), ...); } }); return chainEvents; }; 13 . 7
42. What next? 14 . 1
43. Ways to contribute 1. Use Selenium >3.3.1 with geckodriver + Firefox Nightly then file new issues on Github 2. Test-drive documentation, help us clarify/update 14 . 2
44. Ways to contribute 4. Write spec tests (Python, pytest, JS) 5. Work on bugs in spec implementation (JS, Rust) 6. Tackle some Mozilla test-harness bugs (Python) 14 . 3
45. Remotely control this browser to make a better Web ❤ 14 . 4
46. Thanks! I'm Maja Frydrychowicz. Contact: IRC: maja_zf on irc.mozilla.org and irc.w3.org @impossibus 14 . 5
47. Resources #ateam channel on irc.mozilla.org https://wiki.mozilla.org/User:Mjzffr/New_Contributors http://www.erranderr.com/blog/webdriver-ontology.html http://vakila.github.io/blog/marionette-act-i-automation/ https://developer.mozilla.org/en-US/docs/Mozilla/QA/Marionette/WebDriver https://github.com/mozilla/geckodriver https://github.com/mozilla/webdriver-rust https://github.com/w3c/web-platform-tests/tree/master/webdriver 14 . 6