The Mobile JSON Wire Protocol

174ae1c2a863b7daf240a86da84671bd?s=47 Jonathan Lipps
September 09, 2015

The Mobile JSON Wire Protocol

The JSON Wire Protocol (JSONWP) is the version of the WebDriver spec currently implemented by all the Selenium clients. It defines an HTTP API that models the basic objects of web automation---sessions, elements, etc. The JSON Wire Protocol is the magic that powers Selenium’s client/server architecture, enables services like Selenium Grid or Sauce Labs to work, and gives you the ability to write your test scripts in any language.

The JSONWP has served Selenium faithfully for a number of years, but the future of automated testing lies beyond the borders of the web browser. Mobile automation is an essential ingredient in any build, and tools like Appium or Selendroid have made it possible to run tests against mobile apps using the JSONWP. The JSONWP’s current incarnation isn’t enough to automate all the new behaviors that mobile apps support, however. Complex gestures, multiple device orientations, airplane mode, and the ability to use both native and web contexts, for example, are all essential to mobile automation.

For this reason the leaders of the Selenium project, in concert with other Selenium-based projects like Appium and Selendroid, met to discuss the future of the JSONWP. We’ve been working on its next version, called the "Mobile JSON Wire Protocol" (MJSONWP). Appium and Selendroid already implement much of the MJSONWP spec. In this talk I’ll dive into the specifics of the MJSONWP extensions, how they relate to the original JSONWP, and how the Selenium clients have begun to implement them.

Finally, I will talk about the future of the MJSONWP and how it’s related to the current and future versions of the WebDriver spec. I’ll share how you can get help with the creation of the MJSONWP, and discuss issues with the authors of the new spec before the API is set in stone. We need the help of everyone who’s involved in mobile automation to come up with the best and most future-proof version of the MJSONWP. Ultimately, your understanding of how Selenium works will be improved, and you’ll have a much better handle on how projects like Appium and Selenium work together to make sure you have the best automation methods available.

174ae1c2a863b7daf240a86da84671bd?s=128

Jonathan Lipps

September 09, 2015
Tweet

Transcript

  1. The Mobile JSON Wire Protocol Architecting the Future of Mobile

    Automation Presented by Jonathan Lipps, Director of Engineering, Sauce Labs, Inc. 9/9/2015
  2. © Sauce Labs, Inc. I’m Jonathan Lipps Find me online

    @jlipps @AppiumDevs @saucelabs http://saucelabs.com http://appium.io http://jonathanlipps.com I manage an engineering team at Sauce Labs and take care of the Appium project. I’m involved with moving the spec for mobile automation forward.
  3. Agenda The JSONWP: Why? What’s the motivation behind the JSON

    Wire Protocol? HTTP Protocols: A Brief Intro How HTTP protocols (like the JSONWP) work The JSONWP: How it works A detailed look at how the spec is put together The Mobile JSONWP Extending the JSONWP for a mobile future The WebDriver Spec Double spec, what does it mean??
  4. The JSONWP is… The abstract specification of how automation behaviors

    are mapped to Selenium server HTTP requests and responses
  5. The JSONWP: Why? What’s the motivation behind the JSONWP?

  6. The JSONWP: Why We want a client/server implementation • Write

    tests in any language • Possibility of Selenium Grid, or Sauce Labs • Possibility of multiple server implementations (“drivers”), maintained by different groups
  7. The JSONWP: Why Client/server implementation requires a communication spec •

    HTTP / REST is a good base for a spec • Every language worth writing tests in has an HTTP library • HTTP servers easy to write in most languages as well
  8. The JSONWP: Why Anyone can implement an open protocol •

    The goal of Selenium was always to go away • Chromedriver, IEDriver, any future diver can be stitched seamlessly together
  9. HTTP Protocols: A Brief Intro How HTTP protocols (like the

    JSONWP) work
  10. HTTP CLIENT HTTP SERVER REQ VERB REQ ROUTE REQ BODY

    RES STATUS RES BODY ACTION HTTP Protocols
  11. HTTP Protocols HTTP Request • HTTP Verb: general kind of

    thing you want to do • GET, POST, etc… • HTTP Route (or endpoint): address for the action/resource • HTTP Request Body: more (text) details to send to the server HTTP Response • HTTP Status: generally what happened • 200, 404, 500, etc… • HTTP Response Body: more (text) response details
  12. The JSWONP: How it Works A detailed look at how

    the spec is put together
  13. The JSONWP: How it works https://code.google.com/p/selenium/wiki/JsonWireProtocol

  14. The JSONWP: How it works HTTP Verbs • GET: retrieve

    information from the server • get the status of the server • get the text of an element • POST: make something happen, or get info with side effects • start a session • find an element • click • DELETE: remove a resource • only used with ending a session
  15. The JSONWP: How it works HTTP Routes • Many routes

    • Object-oriented structure encoded in routes • GET /status • POST /session • GET /sessions
  16. The JSONWP: How it works HTTP Response Status Codes •

    200 your action completed successfully • 500 something went wrong • 404 you tried to access a route or resource that wasn’t there • 501 this server doesn’t implement that action
  17. The JSONWP: How it works HTTP Request and Response Bodies

    • Everything is JSON text! • Every route describes the JSON it expects in the request • POST /session • {“desiredCapabilities”: {“browserName”: “safari”}} • JSONWP responses have a status and a value • {“status”: 0, “value”: {“ELEMENT”: “1234”}}
  18. The JSONWP: How it works JSONWP Status Name Description 0

    Success The action completed successfully 7 NoSuchElement Could not find an element matching your requirements 33 SessionNotCreatedException A session could not be created
  19. The JSONWP: How it works HTTP is stateless; your tests

    are not • Client and server need to keep track of objects across multiple requests/responses • Sessions, elements, windows, frames, contexts… • The server assigns IDs to these and shares them with the client. They can then be used in subsequent requests. • POST /session/:sessionId/element • {“status”: 0, “value”: {“ELEMENT”: “1234”}} • POST /session/:sessionId/element/:elementId/click
  20. The JSONWP: How it works JSONWP clients • The purpose

    of the client is to provide an idiomatic interface in a given language, hiding protocol details • The server does not know (or care) what client you are using —it only cares about the protocol • The server cannot reliably reverse-engineer your test code based on a session
  21. The JSONWP: How it works driver = webdriver.remote(…) driver.set_implicit_wait_timeout(10000) driver.url

    = “http://google.com” if driver.title == “google”: driver.quit() else: # do some other stuff
  22. The JSONWP: How it works Advanced JSONWP • We have

    only scratched the surface • Read the doc to learn more! • Types of JSON request/response objects, complex error handling, unicode for non-text keyboard input, etc…
  23. The Mobile JSONWP Extending the JSONWP for a mobile future

  24. The JSONWP: How it works https://github.com/SeleniumHQ/mobile-spec/blob/master/spec-draft.md

  25. The MJSONWP What is the MJSONWP? • Mobile-specific extensions to

    the JSONWP • New routes defining new automation behaviors • New locator strategies • …and that’s it!
  26. The MJSONWP Why do we need a MJSONWP? • Mobile

    automation is a thing now, if you haven’t heard • Mobile automation can and should build off of the excellent architectural choices made by Selenium/WebDriver • No need to re-invent a protocol • The JSONWP is not complete enough for mobile automation • Projects should compete on reliability, features, etc… not the spec they use
  27. The MJSONWP History of the MJSONWP • 1-day meeting in

    London, 8/2013 • Participants included: Simon Stewart, David Burns, myself, Santi Suarez Ordoñez, François Reynaud, Dominik Dary • Since then other Selenium and Appium community members have assisted in moving things forward, for example Luke Inman-Semerau, Matthew Edwards, and Malini Das
  28. The MJSONWP Status of the MJSONWP • Very much a

    draft • Lot of good ideas, some of them will change • The automation “backends” do not always give us enough control to actually implement the spec • Appium, Selendroid, and ios-driver have implemented all or some of the MJSONWP, server-side • Clients getting updated as well
  29. The MJSONWP: Extensions in Detail New desired capability structure •

    The world of mobile is much more complex than the world of browsers • We need new ways to specify new combinations of device, OS, version, and even the specific choice of automation mechanism. • Also… hello, apps!
  30. The MJSONWP: Extensions in Detail New desired capability structure •

    deviceName — the kind of device you want to automate • platformName — the OS you want to automate • platformVersion — the version of that OS • automationName — the automation mechanism • app — the path to the app you want to automate • browserName — the name of a mobile browser to automate
  31. The MJSONWP: Extensions in Detail New desired capability structure {

    platformName: “iOS”, platformVersion: “8.4”, deviceName: “iPhone 6”, app: “/path/to/my.app” }
  32. The MJSONWP: Extensions in Detail New desired capability structure {

    platformName: “iOS”, platformVersion: “8.4”, deviceName: “iPhone 6”, browserName: “Safari” }
  33. The MJSONWP: Extensions in Detail New desired capability structure {

    platformName: “Android”, platformVersion: “4.4”, deviceName: “Nexus S”, app: “/path/to/my.apk”, automationName: “Selendroid” }
  34. The MJSONWP: Extensions in Detail Updated locator strategies • class

    name — the native UI class (not HTML class) • id — the native resource id • xpath — same as before, but applied to an XML document representing a native UI hierarchy
  35. The MJSONWP: Extensions in Detail New locator strategies • accessibility

    id — the string used for making a UI object accessible • -android uiautomator — use the Android UiAutomator selector API to find an element • -ios uiautomation — use the iOS UIAutomation selector API to find an element
  36. The MJSONWP: Extensions in Detail Network connection • Allows modifying

    the network connectivity state of the device • Airplane mode, data mode, wifi mode • GET /session/:sessionId/network_connection • returns a NetworkConnection value • POST /session/:sessionId/network_connection • expects a NetworkConnection value
  37. The MJSONWP: Extensions in Detail Network connection • NetworkConnection •

    {“type”: <int>} • Here, the value of type is an integer that represents any combination of network connection states
  38. The JSONWP: How it works Data Wifi Airplane “Type” Binary

    Off Off Off 0 000 Off Off On 1 001 Off On Off 2 010 Off On On 3 011 On Off Off 4 100 On Off On 5 101 On On Off 6 110 On On On 7 111
  39. The MJSONWP: Extensions in Detail Rotation • More general than

    the current “orientation” route • Arbitrary movement of device in a 3d space • Automation tools do not yet provide ability to simulate arbitrary movement/position input • GET /session/:sessionId/rotation • returns a DeviceRotation value • POST /session/:sessionId/rotation • expects a DeviceRotation value
  40. The MJSONWP: Extensions in Detail Rotation • DeviceRotation • {“x”:

    <float>, “y”: <float>, “z”: <float>} • Here, the values of x, y, and z represent the angle that the device is rotated about each of those axes in 3d space
  41. The MJSONWP: Extensions in Detail

  42. The MJSONWP: Extensions in Detail Contexts • Apps, unlike browsers,

    can have multiple contexts • Hybrid apps have a ‘native’ context and one or more ‘web’ contexts • GET /session/:sessionId/contexts • returns a list of available context names • GET /session/:sessionId/context • returns the current context name • POST /session/:sessionId/context • change the current context to the name provided
  43. The MJSONWP: Extensions in Detail Contexts • Some commands can

    only work in certain contexts • GET /session/:sessionId/title • InvalidContentException • NoSuchContext
  44. The MJSONWP What’s left to do? • Create a set

    of mobile apps to test with • Create a set of reference test scripts that can be used to validate any MJSONWP server implementation • Define XML schemas for Android/iOS UIs • Probably lots more once we dig in further!
  45. The WebDriver Spec Double spec, what does it mean??

  46. The WebDriver Spec https://w3c.github.io/webdriver/webdriver-spec.html

  47. The WebDriver Spec Parallel evolution • The MJSONWP is based

    off of the JSONWP • The JSONWP will be superseded by the WebDriver spec • The MJSONWP will have to take the WebDriver spec into account sooner or later
  48. The WebDriver Spec A period of transition • The WebDriver

    spec is subtly different in many ways than the JSONWP • (Sometimes) different routes • Different error handling • Different request/response expectations • Client and server implementations will undergo a (hopefully not too long) process of supporting the new spec • The MJSONWP will eventually merge into the WebDriver spec
  49. The WebDriver Spec The end goal • No more Selenium

    • No more Appium, Selendroid, ios-driver, etc… • Just the protocol and the vendors’ implementations of it • Browser vendors have (mostly) agreed on how to render a website given the same HTML, CSS, etc… • Browser vendors are getting close to doing this for automation • Let’s make the same thing happen for mobile!
  50. The WebDriver Spec How to get involved • Subscribe to

    SeleniumHQ/mobile-spec and chime in on issues • Look for TODOs and make your own proposals • Let us know what mobile behaviors the MJSONWP should be extended to work with • Help client and server implementations support the MJSONWP • Help us find conceptual gaps in the spec
  51. Questions? @jlipps @saucelabs

  52. Thank you! @jlipps @saucelabs