The Mobile JSON Wire Protocol

The JSON Wire Protocol (JSONWP) is the version of the WebDriver spec currently implemented by all the Selenium clients. It defines an HTTP API that models the basic objects of web automation---sessions, elements, etc. The JSON Wire Protocol is the magic that powers Selenium’s client/server architecture, enables services like Selenium Grid or Sauce Labs to work, and gives you the ability to write your test scripts in any language.

The JSONWP has served Selenium faithfully for a number of years, but the future of automated testing lies beyond the borders of the web browser. Mobile automation is an essential ingredient in any build, and tools like Appium or Selendroid have made it possible to run tests against mobile apps using the JSONWP. The JSONWP’s current incarnation isn’t enough to automate all the new behaviors that mobile apps support, however. Complex gestures, multiple device orientations, airplane mode, and the ability to use both native and web contexts, for example, are all essential to mobile automation.

For this reason the leaders of the Selenium project, in concert with other Selenium-based projects like Appium and Selendroid, met to discuss the future of the JSONWP. We’ve been working on its next version, called the "Mobile JSON Wire Protocol" (MJSONWP). Appium and Selendroid already implement much of the MJSONWP spec. In this talk I’ll dive into the specifics of the MJSONWP extensions, how they relate to the original JSONWP, and how the Selenium clients have begun to implement them.

Finally, I will talk about the future of the MJSONWP and how it’s related to the current and future versions of the WebDriver spec. I’ll share how you can get help with the creation of the MJSONWP, and discuss issues with the authors of the new spec before the API is set in stone. We need the help of everyone who’s involved in mobile automation to come up with the best and most future-proof version of the MJSONWP. Ultimately, your understanding of how Selenium works will be improved, and you’ll have a much better handle on how projects like Appium and Selenium work together to make sure you have the best automation methods available.

Transcript

1. The Mobile JSON Wire Protocol
   Architecting the Future of Mobile Automation
   Presented by Jonathan Lipps, Director of Engineering, Sauce Labs, Inc.
   9/9/2015
   

   View Slide

2. © Sauce Labs, Inc.
   I’m Jonathan Lipps Find me online
   @jlipps
   @AppiumDevs
   @saucelabs
   http://saucelabs.com
   http://appium.io
   http://jonathanlipps.com
   I manage an engineering team at Sauce
   Labs and take care of the Appium
   project. I’m involved with moving the
   spec for mobile automation forward.
   

   View Slide

3. Agenda
   The JSONWP: Why?
   What’s the motivation behind the JSON
   Wire Protocol?
   HTTP Protocols: A Brief Intro
   How HTTP protocols (like the
   JSONWP) work
   The JSONWP: How it works
   A detailed look at how the spec is put
   together
   The Mobile JSONWP
   Extending the JSONWP for a mobile
   future
   The WebDriver Spec
   Double spec, what does it mean??
   

   View Slide

4. The JSONWP is…
   The abstract specification of how automation behaviors are
   mapped to Selenium server HTTP requests and responses
   

   View Slide

5. The JSONWP: Why?
   What’s the motivation behind the JSONWP?
   

   View Slide

6. The JSONWP: Why
   We want a client/server implementation
   • Write tests in any language
   • Possibility of Selenium Grid, or Sauce Labs
   • Possibility of multiple server implementations (“drivers”),
   maintained by different groups
   

   View Slide

7. The JSONWP: Why
   Client/server implementation requires a communication spec
   • HTTP / REST is a good base for a spec
   • Every language worth writing tests in has an HTTP library
   • HTTP servers easy to write in most languages as well
   

   View Slide

8. The JSONWP: Why
   Anyone can implement an open protocol
   • The goal of Selenium was always to go away
   • Chromedriver, IEDriver, any future diver can be stitched
   seamlessly together
   

   View Slide

9. HTTP Protocols: A Brief Intro
   How HTTP protocols (like the JSONWP) work
   

   View Slide

10. HTTP
    CLIENT
    HTTP
    SERVER
    REQ VERB
    REQ ROUTE
    REQ BODY
    RES STATUS
    RES BODY
    ACTION
    HTTP Protocols
    

    View Slide

11. HTTP Protocols
    HTTP Request
    • HTTP Verb: general kind of thing you want to do
    • GET, POST, etc…
    • HTTP Route (or endpoint): address for the action/resource
    • HTTP Request Body: more (text) details to send to the server
    HTTP Response
    • HTTP Status: generally what happened
    • 200, 404, 500, etc…
    • HTTP Response Body: more (text) response details
    

    View Slide

12. The JSWONP: How it Works
    A detailed look at how the spec is put together
    

    View Slide

13. The JSONWP: How it works
    https://code.google.com/p/selenium/wiki/JsonWireProtocol
    

    View Slide

14. The JSONWP: How it works
    HTTP Verbs
    • GET: retrieve information from the server
    • get the status of the server
    • get the text of an element
    • POST: make something happen, or get info with side effects
    • start a session
    • find an element
    • click
    • DELETE: remove a resource
    • only used with ending a session
    

    View Slide

15. The JSONWP: How it works
    HTTP Routes
    • Many routes
    • Object-oriented structure encoded in routes
    • GET /status
    • POST /session
    • GET /sessions
    

    View Slide

16. The JSONWP: How it works
    HTTP Response Status Codes
    • 200 your action completed successfully
    • 500 something went wrong
    • 404 you tried to access a route or resource that wasn’t there
    • 501 this server doesn’t implement that action
    

    View Slide

17. The JSONWP: How it works
    HTTP Request and Response Bodies
    • Everything is JSON text!
    • Every route describes the JSON it expects in the request
    • POST /session
    • {“desiredCapabilities”: {“browserName”: “safari”}}
    • JSONWP responses have a status and a value
    • {“status”: 0, “value”: {“ELEMENT”: “1234”}}
    

    View Slide

18. The JSONWP: How it works
    JSONWP Status Name Description
    0 Success The action completed successfully
    7 NoSuchElement
    Could not find an element matching your
    requirements
    33 SessionNotCreatedException A session could not be created
    

    View Slide

19. The JSONWP: How it works
    HTTP is stateless; your tests are not
    • Client and server need to keep track of objects across
    multiple requests/responses
    • Sessions, elements, windows, frames, contexts…
    • The server assigns IDs to these and shares them with the
    client. They can then be used in subsequent requests.
    • POST /session/:sessionId/element
    • {“status”: 0, “value”: {“ELEMENT”: “1234”}}
    • POST /session/:sessionId/element/:elementId/click
    

    View Slide

20. The JSONWP: How it works
    JSONWP clients
    • The purpose of the client is to provide an idiomatic interface in
    a given language, hiding protocol details
    • The server does not know (or care) what client you are using
    —it only cares about the protocol
    • The server cannot reliably reverse-engineer your test code
    based on a session
    

    View Slide

21. The JSONWP: How it works
    driver = webdriver.remote(…)
    driver.set_implicit_wait_timeout(10000)
    driver.url = “http://google.com”
    if driver.title == “google”:
    driver.quit()
    else:
    # do some other stuff
    

    View Slide

22. The JSONWP: How it works
    Advanced JSONWP
    • We have only scratched the surface
    • Read the doc to learn more!
    • Types of JSON request/response objects, complex error
    handling, unicode for non-text keyboard input, etc…
    

    View Slide

23. The Mobile JSONWP
    Extending the JSONWP for a mobile future
    

    View Slide

24. The JSONWP: How it works
    https://github.com/SeleniumHQ/mobile-spec/blob/master/spec-draft.md
    

    View Slide

25. The MJSONWP
    What is the MJSONWP?
    • Mobile-specific extensions to the JSONWP
    • New routes defining new automation behaviors
    • New locator strategies
    • …and that’s it!
    

    View Slide

26. The MJSONWP
    Why do we need a MJSONWP?
    • Mobile automation is a thing now, if you haven’t heard
    • Mobile automation can and should build off of the excellent
    architectural choices made by Selenium/WebDriver
    • No need to re-invent a protocol
    • The JSONWP is not complete enough for mobile automation
    • Projects should compete on reliability, features, etc… not the
    spec they use
    

    View Slide

27. The MJSONWP
    History of the MJSONWP
    • 1-day meeting in London, 8/2013
    • Participants included: Simon Stewart, David Burns, myself,
    Santi Suarez Ordoñez, François Reynaud, Dominik Dary
    • Since then other Selenium and Appium community members
    have assisted in moving things forward, for example Luke
    Inman-Semerau, Matthew Edwards, and Malini Das
    

    View Slide

28. The MJSONWP
    Status of the MJSONWP
    • Very much a draft
    • Lot of good ideas, some of them will change
    • The automation “backends” do not always give us enough
    control to actually implement the spec
    • Appium, Selendroid, and ios-driver have implemented all or
    some of the MJSONWP, server-side
    • Clients getting updated as well
    

    View Slide

29. The MJSONWP: Extensions in Detail
    New desired capability structure
    • The world of mobile is much more complex than the world of
    browsers
    • We need new ways to specify new combinations of device,
    OS, version, and even the specific choice of automation
    mechanism.
    • Also… hello, apps!
    

    View Slide

30. The MJSONWP: Extensions in Detail
    New desired capability structure
    • deviceName — the kind of device you want to automate
    • platformName — the OS you want to automate
    • platformVersion — the version of that OS
    • automationName — the automation mechanism
    • app — the path to the app you want to automate
    • browserName — the name of a mobile browser to automate
    

    View Slide

31. The MJSONWP: Extensions in Detail
    New desired capability structure
    {
    platformName: “iOS”,
    platformVersion: “8.4”,
    deviceName: “iPhone 6”,
    app: “/path/to/my.app”
    }
    

    View Slide

32. The MJSONWP: Extensions in Detail
    New desired capability structure
    {
    platformName: “iOS”,
    platformVersion: “8.4”,
    deviceName: “iPhone 6”,
    browserName: “Safari”
    }
    

    View Slide

33. The MJSONWP: Extensions in Detail
    New desired capability structure
    {
    platformName: “Android”,
    platformVersion: “4.4”,
    deviceName: “Nexus S”,
    app: “/path/to/my.apk”,
    automationName: “Selendroid”
    }
    

    View Slide

34. The MJSONWP: Extensions in Detail
    Updated locator strategies
    • class name — the native UI class (not HTML class)
    • id — the native resource id
    • xpath — same as before, but applied to an XML document
    representing a native UI hierarchy
    

    View Slide

35. The MJSONWP: Extensions in Detail
    New locator strategies
    • accessibility id — the string used for making a UI object
    accessible
    • -android uiautomator — use the Android UiAutomator
    selector API to find an element
    • -ios uiautomation — use the iOS UIAutomation selector API
    to find an element
    

    View Slide

36. The MJSONWP: Extensions in Detail
    Network connection
    • Allows modifying the network connectivity state of the device
    • Airplane mode, data mode, wifi mode
    • GET /session/:sessionId/network_connection
    • returns a NetworkConnection value
    • POST /session/:sessionId/network_connection
    • expects a NetworkConnection value
    

    View Slide

37. The MJSONWP: Extensions in Detail
    Network connection
    • NetworkConnection
    • {“type”: }
    • Here, the value of type is an integer that represents any
    combination of network connection states
    

    View Slide

38. The JSONWP: How it works
    Data Wifi Airplane “Type” Binary
    Off Off Off 0 000
    Off Off On 1 001
    Off On Off 2 010
    Off On On 3 011
    On Off Off 4 100
    On Off On 5 101
    On On Off 6 110
    On On On 7 111
    

    View Slide

39. The MJSONWP: Extensions in Detail
    Rotation
    • More general than the current “orientation” route
    • Arbitrary movement of device in a 3d space
    • Automation tools do not yet provide ability to simulate arbitrary
    movement/position input
    • GET /session/:sessionId/rotation
    • returns a DeviceRotation value
    • POST /session/:sessionId/rotation
    • expects a DeviceRotation value
    

    View Slide

40. The MJSONWP: Extensions in Detail
    Rotation
    • DeviceRotation
    • {“x”: , “y”: , “z”: }
    • Here, the values of x, y, and z represent the angle that the
    device is rotated about each of those axes in 3d space
    

    View Slide

41. The MJSONWP: Extensions in Detail
    

    View Slide

42. The MJSONWP: Extensions in Detail
    Contexts
    • Apps, unlike browsers, can have multiple contexts
    • Hybrid apps have a ‘native’ context and one or more ‘web’ contexts
    • GET /session/:sessionId/contexts
    • returns a list of available context names
    • GET /session/:sessionId/context
    • returns the current context name
    • POST /session/:sessionId/context
    • change the current context to the name provided
    

    View Slide

43. The MJSONWP: Extensions in Detail
    Contexts
    • Some commands can only work in certain contexts
    • GET /session/:sessionId/title
    • InvalidContentException
    • NoSuchContext
    

    View Slide

44. The MJSONWP
    What’s left to do?
    • Create a set of mobile apps to test with
    • Create a set of reference test scripts that can be used to
    validate any MJSONWP server implementation
    • Define XML schemas for Android/iOS UIs
    • Probably lots more once we dig in further!
    

    View Slide

45. The WebDriver Spec
    Double spec, what does it mean??
    

    View Slide

46. The WebDriver Spec
    https://w3c.github.io/webdriver/webdriver-spec.html
    

    View Slide

47. The WebDriver Spec
    Parallel evolution
    • The MJSONWP is based off of the JSONWP
    • The JSONWP will be superseded by the WebDriver spec
    • The MJSONWP will have to take the WebDriver spec into
    account sooner or later
    

    View Slide

48. The WebDriver Spec
    A period of transition
    • The WebDriver spec is subtly different in many ways than the
    JSONWP
    • (Sometimes) different routes
    • Different error handling
    • Different request/response expectations
    • Client and server implementations will undergo a (hopefully
    not too long) process of supporting the new spec
    • The MJSONWP will eventually merge into the WebDriver spec
    

    View Slide

49. The WebDriver Spec
    The end goal
    • No more Selenium
    • No more Appium, Selendroid, ios-driver, etc…
    • Just the protocol and the vendors’ implementations of it
    • Browser vendors have (mostly) agreed on how to render a
    website given the same HTML, CSS, etc…
    • Browser vendors are getting close to doing this for automation
    • Let’s make the same thing happen for mobile!
    

    View Slide

50. The WebDriver Spec
    How to get involved
    • Subscribe to SeleniumHQ/mobile-spec and chime in on
    issues
    • Look for TODOs and make your own proposals
    • Let us know what mobile behaviors the MJSONWP should be
    extended to work with
    • Help client and server implementations support the
    MJSONWP
    • Help us find conceptual gaps in the spec
    

    View Slide

51. Questions?
    @jlipps
    @saucelabs
    

    View Slide

52. Thank you!
    @jlipps
    @saucelabs
    

    View Slide