The Mobile JSON Wire Protocol

The JSON Wire Protocol (JSONWP) is the version of the WebDriver spec currently implemented by all the Selenium clients. It defines an HTTP API that models the basic objects of web automation---sessions, elements, etc. The JSON Wire Protocol is the magic that powers Selenium’s client/server architecture, enables services like Selenium Grid or Sauce Labs to work, and gives you the ability to write your test scripts in any language.

The JSONWP has served Selenium faithfully for a number of years, but the future of automated testing lies beyond the borders of the web browser. Mobile automation is an essential ingredient in any build, and tools like Appium or Selendroid have made it possible to run tests against mobile apps using the JSONWP. The JSONWP’s current incarnation isn’t enough to automate all the new behaviors that mobile apps support, however. Complex gestures, multiple device orientations, airplane mode, and the ability to use both native and web contexts, for example, are all essential to mobile automation.

For this reason the leaders of the Selenium project, in concert with other Selenium-based projects like Appium and Selendroid, met to discuss the future of the JSONWP. We’ve been working on its next version, called the "Mobile JSON Wire Protocol" (MJSONWP). Appium and Selendroid already implement much of the MJSONWP spec. In this talk I’ll dive into the specifics of the MJSONWP extensions, how they relate to the original JSONWP, and how the Selenium clients have begun to implement them.

Finally, I will talk about the future of the MJSONWP and how it’s related to the current and future versions of the WebDriver spec. I’ll share how you can get help with the creation of the MJSONWP, and discuss issues with the authors of the new spec before the API is set in stone. We need the help of everyone who’s involved in mobile automation to come up with the best and most future-proof version of the MJSONWP. Ultimately, your understanding of how Selenium works will be improved, and you’ll have a much better handle on how projects like Appium and Selenium work together to make sure you have the best automation methods available.

Transcript

1. The Mobile JSON Wire Protocol Architecting the Future of Mobile Automation Presented by Jonathan Lipps, Director of Engineering, Sauce Labs, Inc. 9/9/2015
2. © Sauce Labs, Inc. I’m Jonathan Lipps Find me online @jlipps @AppiumDevs @saucelabs http://saucelabs.com http://appium.io http://jonathanlipps.com I manage an engineering team at Sauce Labs and take care of the Appium project. I’m involved with moving the spec for mobile automation forward.
3. Agenda The JSONWP: Why? What’s the motivation behind the JSON Wire Protocol? HTTP Protocols: A Brief Intro How HTTP protocols (like the JSONWP) work The JSONWP: How it works A detailed look at how the spec is put together The Mobile JSONWP Extending the JSONWP for a mobile future The WebDriver Spec Double spec, what does it mean??
4. The JSONWP is… The abstract specification of how automation behaviors are mapped to Selenium server HTTP requests and responses
5. The JSONWP: Why? What’s the motivation behind the JSONWP?
6. The JSONWP: Why We want a client/server implementation • Write tests in any language • Possibility of Selenium Grid, or Sauce Labs • Possibility of multiple server implementations (“drivers”), maintained by different groups
7. The JSONWP: Why Client/server implementation requires a communication spec • HTTP / REST is a good base for a spec • Every language worth writing tests in has an HTTP library • HTTP servers easy to write in most languages as well
8. The JSONWP: Why Anyone can implement an open protocol • The goal of Selenium was always to go away • Chromedriver, IEDriver, any future diver can be stitched seamlessly together
9. HTTP Protocols: A Brief Intro How HTTP protocols (like the JSONWP) work
10. HTTP CLIENT HTTP SERVER REQ VERB REQ ROUTE REQ BODY RES STATUS RES BODY ACTION HTTP Protocols
11. HTTP Protocols HTTP Request • HTTP Verb: general kind of thing you want to do • GET, POST, etc… • HTTP Route (or endpoint): address for the action/resource • HTTP Request Body: more (text) details to send to the server HTTP Response • HTTP Status: generally what happened • 200, 404, 500, etc… • HTTP Response Body: more (text) response details
12. The JSWONP: How it Works A detailed look at how the spec is put together
13. The JSONWP: How it works https://code.google.com/p/selenium/wiki/JsonWireProtocol
14. The JSONWP: How it works HTTP Verbs • GET: retrieve information from the server • get the status of the server • get the text of an element • POST: make something happen, or get info with side effects • start a session • find an element • click • DELETE: remove a resource • only used with ending a session
15. The JSONWP: How it works HTTP Routes • Many routes • Object-oriented structure encoded in routes • GET /status • POST /session • GET /sessions
16. The JSONWP: How it works HTTP Response Status Codes • 200 your action completed successfully • 500 something went wrong • 404 you tried to access a route or resource that wasn’t there • 501 this server doesn’t implement that action
17. The JSONWP: How it works HTTP Request and Response Bodies • Everything is JSON text! • Every route describes the JSON it expects in the request • POST /session • {“desiredCapabilities”: {“browserName”: “safari”}} • JSONWP responses have a status and a value • {“status”: 0, “value”: {“ELEMENT”: “1234”}}
18. The JSONWP: How it works JSONWP Status Name Description 0 Success The action completed successfully 7 NoSuchElement Could not find an element matching your requirements 33 SessionNotCreatedException A session could not be created
19. The JSONWP: How it works HTTP is stateless; your tests are not • Client and server need to keep track of objects across multiple requests/responses • Sessions, elements, windows, frames, contexts… • The server assigns IDs to these and shares them with the client. They can then be used in subsequent requests. • POST /session/:sessionId/element • {“status”: 0, “value”: {“ELEMENT”: “1234”}} • POST /session/:sessionId/element/:elementId/click
20. The JSONWP: How it works JSONWP clients • The purpose of the client is to provide an idiomatic interface in a given language, hiding protocol details • The server does not know (or care) what client you are using —it only cares about the protocol • The server cannot reliably reverse-engineer your test code based on a session
21. The JSONWP: How it works driver = webdriver.remote(…) driver.set_implicit_wait_timeout(10000) driver.url = “http://google.com” if driver.title == “google”: driver.quit() else: # do some other stuff
22. The JSONWP: How it works Advanced JSONWP • We have only scratched the surface • Read the doc to learn more! • Types of JSON request/response objects, complex error handling, unicode for non-text keyboard input, etc…
23. The Mobile JSONWP Extending the JSONWP for a mobile future
24. The JSONWP: How it works https://github.com/SeleniumHQ/mobile-spec/blob/master/spec-draft.md
25. The MJSONWP What is the MJSONWP? • Mobile-specific extensions to the JSONWP • New routes defining new automation behaviors • New locator strategies • …and that’s it!
26. The MJSONWP Why do we need a MJSONWP? • Mobile automation is a thing now, if you haven’t heard • Mobile automation can and should build off of the excellent architectural choices made by Selenium/WebDriver • No need to re-invent a protocol • The JSONWP is not complete enough for mobile automation • Projects should compete on reliability, features, etc… not the spec they use
27. The MJSONWP History of the MJSONWP • 1-day meeting in London, 8/2013 • Participants included: Simon Stewart, David Burns, myself, Santi Suarez Ordoñez, François Reynaud, Dominik Dary • Since then other Selenium and Appium community members have assisted in moving things forward, for example Luke Inman-Semerau, Matthew Edwards, and Malini Das
28. The MJSONWP Status of the MJSONWP • Very much a draft • Lot of good ideas, some of them will change • The automation “backends” do not always give us enough control to actually implement the spec • Appium, Selendroid, and ios-driver have implemented all or some of the MJSONWP, server-side • Clients getting updated as well
29. The MJSONWP: Extensions in Detail New desired capability structure • The world of mobile is much more complex than the world of browsers • We need new ways to specify new combinations of device, OS, version, and even the specific choice of automation mechanism. • Also… hello, apps!
30. The MJSONWP: Extensions in Detail New desired capability structure • deviceName — the kind of device you want to automate • platformName — the OS you want to automate • platformVersion — the version of that OS • automationName — the automation mechanism • app — the path to the app you want to automate • browserName — the name of a mobile browser to automate
31. The MJSONWP: Extensions in Detail New desired capability structure { platformName: “iOS”, platformVersion: “8.4”, deviceName: “iPhone 6”, app: “/path/to/my.app” }
32. The MJSONWP: Extensions in Detail New desired capability structure { platformName: “iOS”, platformVersion: “8.4”, deviceName: “iPhone 6”, browserName: “Safari” }
33. The MJSONWP: Extensions in Detail New desired capability structure { platformName: “Android”, platformVersion: “4.4”, deviceName: “Nexus S”, app: “/path/to/my.apk”, automationName: “Selendroid” }
34. The MJSONWP: Extensions in Detail Updated locator strategies • class name — the native UI class (not HTML class) • id — the native resource id • xpath — same as before, but applied to an XML document representing a native UI hierarchy
35. The MJSONWP: Extensions in Detail New locator strategies • accessibility id — the string used for making a UI object accessible • -android uiautomator — use the Android UiAutomator selector API to find an element • -ios uiautomation — use the iOS UIAutomation selector API to find an element
36. The MJSONWP: Extensions in Detail Network connection • Allows modifying the network connectivity state of the device • Airplane mode, data mode, wifi mode • GET /session/:sessionId/network_connection • returns a NetworkConnection value • POST /session/:sessionId/network_connection • expects a NetworkConnection value
37. The MJSONWP: Extensions in Detail Network connection • NetworkConnection • {“type”: <int>} • Here, the value of type is an integer that represents any combination of network connection states
38. The JSONWP: How it works Data Wifi Airplane “Type” Binary Off Off Off 0 000 Off Off On 1 001 Off On Off 2 010 Off On On 3 011 On Off Off 4 100 On Off On 5 101 On On Off 6 110 On On On 7 111
39. The MJSONWP: Extensions in Detail Rotation • More general than the current “orientation” route • Arbitrary movement of device in a 3d space • Automation tools do not yet provide ability to simulate arbitrary movement/position input • GET /session/:sessionId/rotation • returns a DeviceRotation value • POST /session/:sessionId/rotation • expects a DeviceRotation value
40. The MJSONWP: Extensions in Detail Rotation • DeviceRotation • {“x”: <float>, “y”: <float>, “z”: <float>} • Here, the values of x, y, and z represent the angle that the device is rotated about each of those axes in 3d space
41. The MJSONWP: Extensions in Detail
42. The MJSONWP: Extensions in Detail Contexts • Apps, unlike browsers, can have multiple contexts • Hybrid apps have a ‘native’ context and one or more ‘web’ contexts • GET /session/:sessionId/contexts • returns a list of available context names • GET /session/:sessionId/context • returns the current context name • POST /session/:sessionId/context • change the current context to the name provided
43. The MJSONWP: Extensions in Detail Contexts • Some commands can only work in certain contexts • GET /session/:sessionId/title • InvalidContentException • NoSuchContext
44. The MJSONWP What’s left to do? • Create a set of mobile apps to test with • Create a set of reference test scripts that can be used to validate any MJSONWP server implementation • Define XML schemas for Android/iOS UIs • Probably lots more once we dig in further!
45. The WebDriver Spec Double spec, what does it mean??
46. The WebDriver Spec https://w3c.github.io/webdriver/webdriver-spec.html
47. The WebDriver Spec Parallel evolution • The MJSONWP is based off of the JSONWP • The JSONWP will be superseded by the WebDriver spec • The MJSONWP will have to take the WebDriver spec into account sooner or later
48. The WebDriver Spec A period of transition • The WebDriver spec is subtly different in many ways than the JSONWP • (Sometimes) different routes • Different error handling • Different request/response expectations • Client and server implementations will undergo a (hopefully not too long) process of supporting the new spec • The MJSONWP will eventually merge into the WebDriver spec
49. The WebDriver Spec The end goal • No more Selenium • No more Appium, Selendroid, ios-driver, etc… • Just the protocol and the vendors’ implementations of it • Browser vendors have (mostly) agreed on how to render a website given the same HTML, CSS, etc… • Browser vendors are getting close to doing this for automation • Let’s make the same thing happen for mobile!
50. The WebDriver Spec How to get involved • Subscribe to SeleniumHQ/mobile-spec and chime in on issues • Look for TODOs and make your own proposals • Let us know what mobile behaviors the MJSONWP should be extended to work with • Help client and server implementations support the MJSONWP • Help us find conceptual gaps in the spec
51. Questions? @jlipps @saucelabs
52. Thank you! @jlipps @saucelabs