Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Mobile JSON Wire Protocol

Jonathan Lipps
September 09, 2015

The Mobile JSON Wire Protocol

The JSON Wire Protocol (JSONWP) is the version of the WebDriver spec currently implemented by all the Selenium clients. It defines an HTTP API that models the basic objects of web automation---sessions, elements, etc. The JSON Wire Protocol is the magic that powers Selenium’s client/server architecture, enables services like Selenium Grid or Sauce Labs to work, and gives you the ability to write your test scripts in any language.

The JSONWP has served Selenium faithfully for a number of years, but the future of automated testing lies beyond the borders of the web browser. Mobile automation is an essential ingredient in any build, and tools like Appium or Selendroid have made it possible to run tests against mobile apps using the JSONWP. The JSONWP’s current incarnation isn’t enough to automate all the new behaviors that mobile apps support, however. Complex gestures, multiple device orientations, airplane mode, and the ability to use both native and web contexts, for example, are all essential to mobile automation.

For this reason the leaders of the Selenium project, in concert with other Selenium-based projects like Appium and Selendroid, met to discuss the future of the JSONWP. We’ve been working on its next version, called the "Mobile JSON Wire Protocol" (MJSONWP). Appium and Selendroid already implement much of the MJSONWP spec. In this talk I’ll dive into the specifics of the MJSONWP extensions, how they relate to the original JSONWP, and how the Selenium clients have begun to implement them.

Finally, I will talk about the future of the MJSONWP and how it’s related to the current and future versions of the WebDriver spec. I’ll share how you can get help with the creation of the MJSONWP, and discuss issues with the authors of the new spec before the API is set in stone. We need the help of everyone who’s involved in mobile automation to come up with the best and most future-proof version of the MJSONWP. Ultimately, your understanding of how Selenium works will be improved, and you’ll have a much better handle on how projects like Appium and Selenium work together to make sure you have the best automation methods available.

Jonathan Lipps

September 09, 2015
Tweet

More Decks by Jonathan Lipps

Other Decks in Technology

Transcript

  1. The Mobile JSON Wire Protocol
    Architecting the Future of Mobile Automation
    Presented by Jonathan Lipps, Director of Engineering, Sauce Labs, Inc.
    9/9/2015

    View Slide

  2. © Sauce Labs, Inc.
    I’m Jonathan Lipps Find me online
    @jlipps
    @AppiumDevs
    @saucelabs
    http://saucelabs.com
    http://appium.io
    http://jonathanlipps.com
    I manage an engineering team at Sauce
    Labs and take care of the Appium
    project. I’m involved with moving the
    spec for mobile automation forward.

    View Slide

  3. Agenda
    The JSONWP: Why?
    What’s the motivation behind the JSON
    Wire Protocol?
    HTTP Protocols: A Brief Intro
    How HTTP protocols (like the
    JSONWP) work
    The JSONWP: How it works
    A detailed look at how the spec is put
    together
    The Mobile JSONWP
    Extending the JSONWP for a mobile
    future
    The WebDriver Spec
    Double spec, what does it mean??

    View Slide

  4. The JSONWP is…
    The abstract specification of how automation behaviors are
    mapped to Selenium server HTTP requests and responses

    View Slide

  5. The JSONWP: Why?
    What’s the motivation behind the JSONWP?

    View Slide

  6. The JSONWP: Why
    We want a client/server implementation
    • Write tests in any language
    • Possibility of Selenium Grid, or Sauce Labs
    • Possibility of multiple server implementations (“drivers”),
    maintained by different groups

    View Slide

  7. The JSONWP: Why
    Client/server implementation requires a communication spec
    • HTTP / REST is a good base for a spec
    • Every language worth writing tests in has an HTTP library
    • HTTP servers easy to write in most languages as well

    View Slide

  8. The JSONWP: Why
    Anyone can implement an open protocol
    • The goal of Selenium was always to go away
    • Chromedriver, IEDriver, any future diver can be stitched
    seamlessly together

    View Slide

  9. HTTP Protocols: A Brief Intro
    How HTTP protocols (like the JSONWP) work

    View Slide

  10. HTTP
    CLIENT
    HTTP
    SERVER
    REQ VERB
    REQ ROUTE
    REQ BODY
    RES STATUS
    RES BODY
    ACTION
    HTTP Protocols

    View Slide

  11. HTTP Protocols
    HTTP Request
    • HTTP Verb: general kind of thing you want to do
    • GET, POST, etc…
    • HTTP Route (or endpoint): address for the action/resource
    • HTTP Request Body: more (text) details to send to the server
    HTTP Response
    • HTTP Status: generally what happened
    • 200, 404, 500, etc…
    • HTTP Response Body: more (text) response details

    View Slide

  12. The JSWONP: How it Works
    A detailed look at how the spec is put together

    View Slide

  13. The JSONWP: How it works
    https://code.google.com/p/selenium/wiki/JsonWireProtocol

    View Slide

  14. The JSONWP: How it works
    HTTP Verbs
    • GET: retrieve information from the server
    • get the status of the server
    • get the text of an element
    • POST: make something happen, or get info with side effects
    • start a session
    • find an element
    • click
    • DELETE: remove a resource
    • only used with ending a session

    View Slide

  15. The JSONWP: How it works
    HTTP Routes
    • Many routes
    • Object-oriented structure encoded in routes
    • GET /status
    • POST /session
    • GET /sessions

    View Slide

  16. The JSONWP: How it works
    HTTP Response Status Codes
    • 200 your action completed successfully
    • 500 something went wrong
    • 404 you tried to access a route or resource that wasn’t there
    • 501 this server doesn’t implement that action

    View Slide

  17. The JSONWP: How it works
    HTTP Request and Response Bodies
    • Everything is JSON text!
    • Every route describes the JSON it expects in the request
    • POST /session
    • {“desiredCapabilities”: {“browserName”: “safari”}}
    • JSONWP responses have a status and a value
    • {“status”: 0, “value”: {“ELEMENT”: “1234”}}

    View Slide

  18. The JSONWP: How it works
    JSONWP Status Name Description
    0 Success The action completed successfully
    7 NoSuchElement
    Could not find an element matching your
    requirements
    33 SessionNotCreatedException A session could not be created

    View Slide

  19. The JSONWP: How it works
    HTTP is stateless; your tests are not
    • Client and server need to keep track of objects across
    multiple requests/responses
    • Sessions, elements, windows, frames, contexts…
    • The server assigns IDs to these and shares them with the
    client. They can then be used in subsequent requests.
    • POST /session/:sessionId/element
    • {“status”: 0, “value”: {“ELEMENT”: “1234”}}
    • POST /session/:sessionId/element/:elementId/click

    View Slide

  20. The JSONWP: How it works
    JSONWP clients
    • The purpose of the client is to provide an idiomatic interface in
    a given language, hiding protocol details
    • The server does not know (or care) what client you are using
    —it only cares about the protocol
    • The server cannot reliably reverse-engineer your test code
    based on a session

    View Slide

  21. The JSONWP: How it works
    driver = webdriver.remote(…)
    driver.set_implicit_wait_timeout(10000)
    driver.url = “http://google.com”
    if driver.title == “google”:
    driver.quit()
    else:
    # do some other stuff

    View Slide

  22. The JSONWP: How it works
    Advanced JSONWP
    • We have only scratched the surface
    • Read the doc to learn more!
    • Types of JSON request/response objects, complex error
    handling, unicode for non-text keyboard input, etc…

    View Slide

  23. The Mobile JSONWP
    Extending the JSONWP for a mobile future

    View Slide

  24. The JSONWP: How it works
    https://github.com/SeleniumHQ/mobile-spec/blob/master/spec-draft.md

    View Slide

  25. The MJSONWP
    What is the MJSONWP?
    • Mobile-specific extensions to the JSONWP
    • New routes defining new automation behaviors
    • New locator strategies
    • …and that’s it!

    View Slide

  26. The MJSONWP
    Why do we need a MJSONWP?
    • Mobile automation is a thing now, if you haven’t heard
    • Mobile automation can and should build off of the excellent
    architectural choices made by Selenium/WebDriver
    • No need to re-invent a protocol
    • The JSONWP is not complete enough for mobile automation
    • Projects should compete on reliability, features, etc… not the
    spec they use

    View Slide

  27. The MJSONWP
    History of the MJSONWP
    • 1-day meeting in London, 8/2013
    • Participants included: Simon Stewart, David Burns, myself,
    Santi Suarez Ordoñez, François Reynaud, Dominik Dary
    • Since then other Selenium and Appium community members
    have assisted in moving things forward, for example Luke
    Inman-Semerau, Matthew Edwards, and Malini Das

    View Slide

  28. The MJSONWP
    Status of the MJSONWP
    • Very much a draft
    • Lot of good ideas, some of them will change
    • The automation “backends” do not always give us enough
    control to actually implement the spec
    • Appium, Selendroid, and ios-driver have implemented all or
    some of the MJSONWP, server-side
    • Clients getting updated as well

    View Slide

  29. The MJSONWP: Extensions in Detail
    New desired capability structure
    • The world of mobile is much more complex than the world of
    browsers
    • We need new ways to specify new combinations of device,
    OS, version, and even the specific choice of automation
    mechanism.
    • Also… hello, apps!

    View Slide

  30. The MJSONWP: Extensions in Detail
    New desired capability structure
    • deviceName — the kind of device you want to automate
    • platformName — the OS you want to automate
    • platformVersion — the version of that OS
    • automationName — the automation mechanism
    • app — the path to the app you want to automate
    • browserName — the name of a mobile browser to automate

    View Slide

  31. The MJSONWP: Extensions in Detail
    New desired capability structure
    {
    platformName: “iOS”,
    platformVersion: “8.4”,
    deviceName: “iPhone 6”,
    app: “/path/to/my.app”
    }

    View Slide

  32. The MJSONWP: Extensions in Detail
    New desired capability structure
    {
    platformName: “iOS”,
    platformVersion: “8.4”,
    deviceName: “iPhone 6”,
    browserName: “Safari”
    }

    View Slide

  33. The MJSONWP: Extensions in Detail
    New desired capability structure
    {
    platformName: “Android”,
    platformVersion: “4.4”,
    deviceName: “Nexus S”,
    app: “/path/to/my.apk”,
    automationName: “Selendroid”
    }

    View Slide

  34. The MJSONWP: Extensions in Detail
    Updated locator strategies
    • class name — the native UI class (not HTML class)
    • id — the native resource id
    • xpath — same as before, but applied to an XML document
    representing a native UI hierarchy

    View Slide

  35. The MJSONWP: Extensions in Detail
    New locator strategies
    • accessibility id — the string used for making a UI object
    accessible
    • -android uiautomator — use the Android UiAutomator
    selector API to find an element
    • -ios uiautomation — use the iOS UIAutomation selector API
    to find an element

    View Slide

  36. The MJSONWP: Extensions in Detail
    Network connection
    • Allows modifying the network connectivity state of the device
    • Airplane mode, data mode, wifi mode
    • GET /session/:sessionId/network_connection
    • returns a NetworkConnection value
    • POST /session/:sessionId/network_connection
    • expects a NetworkConnection value

    View Slide

  37. The MJSONWP: Extensions in Detail
    Network connection
    • NetworkConnection
    • {“type”: }
    • Here, the value of type is an integer that represents any
    combination of network connection states

    View Slide

  38. The JSONWP: How it works
    Data Wifi Airplane “Type” Binary
    Off Off Off 0 000
    Off Off On 1 001
    Off On Off 2 010
    Off On On 3 011
    On Off Off 4 100
    On Off On 5 101
    On On Off 6 110
    On On On 7 111

    View Slide

  39. The MJSONWP: Extensions in Detail
    Rotation
    • More general than the current “orientation” route
    • Arbitrary movement of device in a 3d space
    • Automation tools do not yet provide ability to simulate arbitrary
    movement/position input
    • GET /session/:sessionId/rotation
    • returns a DeviceRotation value
    • POST /session/:sessionId/rotation
    • expects a DeviceRotation value

    View Slide

  40. The MJSONWP: Extensions in Detail
    Rotation
    • DeviceRotation
    • {“x”: , “y”: , “z”: }
    • Here, the values of x, y, and z represent the angle that the
    device is rotated about each of those axes in 3d space

    View Slide

  41. The MJSONWP: Extensions in Detail

    View Slide

  42. The MJSONWP: Extensions in Detail
    Contexts
    • Apps, unlike browsers, can have multiple contexts
    • Hybrid apps have a ‘native’ context and one or more ‘web’ contexts
    • GET /session/:sessionId/contexts
    • returns a list of available context names
    • GET /session/:sessionId/context
    • returns the current context name
    • POST /session/:sessionId/context
    • change the current context to the name provided

    View Slide

  43. The MJSONWP: Extensions in Detail
    Contexts
    • Some commands can only work in certain contexts
    • GET /session/:sessionId/title
    • InvalidContentException
    • NoSuchContext

    View Slide

  44. The MJSONWP
    What’s left to do?
    • Create a set of mobile apps to test with
    • Create a set of reference test scripts that can be used to
    validate any MJSONWP server implementation
    • Define XML schemas for Android/iOS UIs
    • Probably lots more once we dig in further!

    View Slide

  45. The WebDriver Spec
    Double spec, what does it mean??

    View Slide

  46. The WebDriver Spec
    https://w3c.github.io/webdriver/webdriver-spec.html

    View Slide

  47. The WebDriver Spec
    Parallel evolution
    • The MJSONWP is based off of the JSONWP
    • The JSONWP will be superseded by the WebDriver spec
    • The MJSONWP will have to take the WebDriver spec into
    account sooner or later

    View Slide

  48. The WebDriver Spec
    A period of transition
    • The WebDriver spec is subtly different in many ways than the
    JSONWP
    • (Sometimes) different routes
    • Different error handling
    • Different request/response expectations
    • Client and server implementations will undergo a (hopefully
    not too long) process of supporting the new spec
    • The MJSONWP will eventually merge into the WebDriver spec

    View Slide

  49. The WebDriver Spec
    The end goal
    • No more Selenium
    • No more Appium, Selendroid, ios-driver, etc…
    • Just the protocol and the vendors’ implementations of it
    • Browser vendors have (mostly) agreed on how to render a
    website given the same HTML, CSS, etc…
    • Browser vendors are getting close to doing this for automation
    • Let’s make the same thing happen for mobile!

    View Slide

  50. The WebDriver Spec
    How to get involved
    • Subscribe to SeleniumHQ/mobile-spec and chime in on
    issues
    • Look for TODOs and make your own proposals
    • Let us know what mobile behaviors the MJSONWP should be
    extended to work with
    • Help client and server implementations support the
    MJSONWP
    • Help us find conceptual gaps in the spec

    View Slide

  51. Questions?
    @jlipps
    @saucelabs

    View Slide

  52. Thank you!
    @jlipps
    @saucelabs

    View Slide