The Mobile JSON Wire Protocol

Slide 1

Slide 1 text

The Mobile JSON Wire Protocol Architecting the Future of Mobile Automation Presented by Jonathan Lipps, Director of Engineering, Sauce Labs, Inc. 9/9/2015

Slide 2

Slide 2 text

© Sauce Labs, Inc. I’m Jonathan Lipps Find me online @jlipps @AppiumDevs @saucelabs http://saucelabs.com http://appium.io http://jonathanlipps.com I manage an engineering team at Sauce Labs and take care of the Appium project. I’m involved with moving the spec for mobile automation forward.

Slide 3

Slide 3 text

Agenda The JSONWP: Why? What’s the motivation behind the JSON Wire Protocol? HTTP Protocols: A Brief Intro How HTTP protocols (like the JSONWP) work The JSONWP: How it works A detailed look at how the spec is put together The Mobile JSONWP Extending the JSONWP for a mobile future The WebDriver Spec Double spec, what does it mean??

Slide 4

Slide 4 text

The JSONWP is… The abstract specification of how automation behaviors are mapped to Selenium server HTTP requests and responses

Slide 5

Slide 5 text

The JSONWP: Why? What’s the motivation behind the JSONWP?

Slide 6

Slide 6 text

The JSONWP: Why We want a client/server implementation • Write tests in any language • Possibility of Selenium Grid, or Sauce Labs • Possibility of multiple server implementations (“drivers”), maintained by different groups

Slide 7

Slide 7 text

The JSONWP: Why Client/server implementation requires a communication spec • HTTP / REST is a good base for a spec • Every language worth writing tests in has an HTTP library • HTTP servers easy to write in most languages as well

Slide 8

Slide 8 text

The JSONWP: Why Anyone can implement an open protocol • The goal of Selenium was always to go away • Chromedriver, IEDriver, any future diver can be stitched seamlessly together

Slide 9

Slide 9 text

HTTP Protocols: A Brief Intro How HTTP protocols (like the JSONWP) work

Slide 10

Slide 10 text

HTTP CLIENT HTTP SERVER REQ VERB REQ ROUTE REQ BODY RES STATUS RES BODY ACTION HTTP Protocols

Slide 11

Slide 11 text

HTTP Protocols HTTP Request • HTTP Verb: general kind of thing you want to do • GET, POST, etc… • HTTP Route (or endpoint): address for the action/resource • HTTP Request Body: more (text) details to send to the server HTTP Response • HTTP Status: generally what happened • 200, 404, 500, etc… • HTTP Response Body: more (text) response details

Slide 12

Slide 12 text

The JSWONP: How it Works A detailed look at how the spec is put together

Slide 13

Slide 13 text

The JSONWP: How it works https://code.google.com/p/selenium/wiki/JsonWireProtocol

Slide 14

Slide 14 text

The JSONWP: How it works HTTP Verbs • GET: retrieve information from the server • get the status of the server • get the text of an element • POST: make something happen, or get info with side effects • start a session • find an element • click • DELETE: remove a resource • only used with ending a session

Slide 15

Slide 15 text

The JSONWP: How it works HTTP Routes • Many routes • Object-oriented structure encoded in routes • GET /status • POST /session • GET /sessions

Slide 16

Slide 16 text

The JSONWP: How it works HTTP Response Status Codes • 200 your action completed successfully • 500 something went wrong • 404 you tried to access a route or resource that wasn’t there • 501 this server doesn’t implement that action

Slide 17

Slide 17 text

The JSONWP: How it works HTTP Request and Response Bodies • Everything is JSON text! • Every route describes the JSON it expects in the request • POST /session • {“desiredCapabilities”: {“browserName”: “safari”}} • JSONWP responses have a status and a value • {“status”: 0, “value”: {“ELEMENT”: “1234”}}

Slide 18

Slide 18 text

The JSONWP: How it works JSONWP Status Name Description 0 Success The action completed successfully 7 NoSuchElement Could not find an element matching your requirements 33 SessionNotCreatedException A session could not be created

Slide 19

Slide 19 text

The JSONWP: How it works HTTP is stateless; your tests are not • Client and server need to keep track of objects across multiple requests/responses • Sessions, elements, windows, frames, contexts… • The server assigns IDs to these and shares them with the client. They can then be used in subsequent requests. • POST /session/:sessionId/element • {“status”: 0, “value”: {“ELEMENT”: “1234”}} • POST /session/:sessionId/element/:elementId/click

Slide 20

Slide 20 text

The JSONWP: How it works JSONWP clients • The purpose of the client is to provide an idiomatic interface in a given language, hiding protocol details • The server does not know (or care) what client you are using —it only cares about the protocol • The server cannot reliably reverse-engineer your test code based on a session

Slide 21

Slide 21 text

The JSONWP: How it works driver = webdriver.remote(…) driver.set_implicit_wait_timeout(10000) driver.url = “http://google.com” if driver.title == “google”: driver.quit() else: # do some other stuff

Slide 22

Slide 22 text

The JSONWP: How it works Advanced JSONWP • We have only scratched the surface • Read the doc to learn more! • Types of JSON request/response objects, complex error handling, unicode for non-text keyboard input, etc…

Slide 23

Slide 23 text

The Mobile JSONWP Extending the JSONWP for a mobile future

Slide 24

Slide 24 text

The JSONWP: How it works https://github.com/SeleniumHQ/mobile-spec/blob/master/spec-draft.md

Slide 25

Slide 25 text

The MJSONWP What is the MJSONWP? • Mobile-specific extensions to the JSONWP • New routes defining new automation behaviors • New locator strategies • …and that’s it!

Slide 26

Slide 26 text

The MJSONWP Why do we need a MJSONWP? • Mobile automation is a thing now, if you haven’t heard • Mobile automation can and should build off of the excellent architectural choices made by Selenium/WebDriver • No need to re-invent a protocol • The JSONWP is not complete enough for mobile automation • Projects should compete on reliability, features, etc… not the spec they use

Slide 27

Slide 27 text

The MJSONWP History of the MJSONWP • 1-day meeting in London, 8/2013 • Participants included: Simon Stewart, David Burns, myself, Santi Suarez Ordoñez, François Reynaud, Dominik Dary • Since then other Selenium and Appium community members have assisted in moving things forward, for example Luke Inman-Semerau, Matthew Edwards, and Malini Das

Slide 28

Slide 28 text

The MJSONWP Status of the MJSONWP • Very much a draft • Lot of good ideas, some of them will change • The automation “backends” do not always give us enough control to actually implement the spec • Appium, Selendroid, and ios-driver have implemented all or some of the MJSONWP, server-side • Clients getting updated as well

Slide 29

Slide 29 text

The MJSONWP: Extensions in Detail New desired capability structure • The world of mobile is much more complex than the world of browsers • We need new ways to specify new combinations of device, OS, version, and even the specific choice of automation mechanism. • Also… hello, apps!

Slide 30

Slide 30 text

The MJSONWP: Extensions in Detail New desired capability structure • deviceName — the kind of device you want to automate • platformName — the OS you want to automate • platformVersion — the version of that OS • automationName — the automation mechanism • app — the path to the app you want to automate • browserName — the name of a mobile browser to automate

Slide 31

Slide 31 text

The MJSONWP: Extensions in Detail New desired capability structure { platformName: “iOS”, platformVersion: “8.4”, deviceName: “iPhone 6”, app: “/path/to/my.app” }

Slide 32

Slide 32 text

The MJSONWP: Extensions in Detail New desired capability structure { platformName: “iOS”, platformVersion: “8.4”, deviceName: “iPhone 6”, browserName: “Safari” }

Slide 33

Slide 33 text

The MJSONWP: Extensions in Detail New desired capability structure { platformName: “Android”, platformVersion: “4.4”, deviceName: “Nexus S”, app: “/path/to/my.apk”, automationName: “Selendroid” }

Slide 34

Slide 34 text

The MJSONWP: Extensions in Detail Updated locator strategies • class name — the native UI class (not HTML class) • id — the native resource id • xpath — same as before, but applied to an XML document representing a native UI hierarchy

Slide 35

Slide 35 text

The MJSONWP: Extensions in Detail New locator strategies • accessibility id — the string used for making a UI object accessible • -android uiautomator — use the Android UiAutomator selector API to find an element • -ios uiautomation — use the iOS UIAutomation selector API to find an element

Slide 36

Slide 36 text

The MJSONWP: Extensions in Detail Network connection • Allows modifying the network connectivity state of the device • Airplane mode, data mode, wifi mode • GET /session/:sessionId/network_connection • returns a NetworkConnection value • POST /session/:sessionId/network_connection • expects a NetworkConnection value

Slide 37

Slide 37 text

The MJSONWP: Extensions in Detail Network connection • NetworkConnection • {“type”: } • Here, the value of type is an integer that represents any combination of network connection states

Slide 38

Slide 38 text

The JSONWP: How it works Data Wifi Airplane “Type” Binary Off Off Off 0 000 Off Off On 1 001 Off On Off 2 010 Off On On 3 011 On Off Off 4 100 On Off On 5 101 On On Off 6 110 On On On 7 111

Slide 39

Slide 39 text

The MJSONWP: Extensions in Detail Rotation • More general than the current “orientation” route • Arbitrary movement of device in a 3d space • Automation tools do not yet provide ability to simulate arbitrary movement/position input • GET /session/:sessionId/rotation • returns a DeviceRotation value • POST /session/:sessionId/rotation • expects a DeviceRotation value

Slide 40

Slide 40 text

The MJSONWP: Extensions in Detail Rotation • DeviceRotation • {“x”: , “y”: , “z”: } • Here, the values of x, y, and z represent the angle that the device is rotated about each of those axes in 3d space

Slide 41

Slide 41 text

The MJSONWP: Extensions in Detail

Slide 42

Slide 42 text

The MJSONWP: Extensions in Detail Contexts • Apps, unlike browsers, can have multiple contexts • Hybrid apps have a ‘native’ context and one or more ‘web’ contexts • GET /session/:sessionId/contexts • returns a list of available context names • GET /session/:sessionId/context • returns the current context name • POST /session/:sessionId/context • change the current context to the name provided

Slide 43

Slide 43 text

The MJSONWP: Extensions in Detail Contexts • Some commands can only work in certain contexts • GET /session/:sessionId/title • InvalidContentException • NoSuchContext

Slide 44

Slide 44 text

The MJSONWP What’s left to do? • Create a set of mobile apps to test with • Create a set of reference test scripts that can be used to validate any MJSONWP server implementation • Define XML schemas for Android/iOS UIs • Probably lots more once we dig in further!

Slide 45

Slide 45 text

The WebDriver Spec Double spec, what does it mean??

Slide 46

Slide 46 text

The WebDriver Spec https://w3c.github.io/webdriver/webdriver-spec.html

Slide 47

Slide 47 text

The WebDriver Spec Parallel evolution • The MJSONWP is based off of the JSONWP • The JSONWP will be superseded by the WebDriver spec • The MJSONWP will have to take the WebDriver spec into account sooner or later

Slide 48

Slide 48 text

The WebDriver Spec A period of transition • The WebDriver spec is subtly different in many ways than the JSONWP • (Sometimes) different routes • Different error handling • Different request/response expectations • Client and server implementations will undergo a (hopefully not too long) process of supporting the new spec • The MJSONWP will eventually merge into the WebDriver spec

Slide 49

Slide 49 text

The WebDriver Spec The end goal • No more Selenium • No more Appium, Selendroid, ios-driver, etc… • Just the protocol and the vendors’ implementations of it • Browser vendors have (mostly) agreed on how to render a website given the same HTML, CSS, etc… • Browser vendors are getting close to doing this for automation • Let’s make the same thing happen for mobile!

Slide 50

Slide 50 text

The WebDriver Spec How to get involved • Subscribe to SeleniumHQ/mobile-spec and chime in on issues • Look for TODOs and make your own proposals • Let us know what mobile behaviors the MJSONWP should be extended to work with • Help client and server implementations support the MJSONWP • Help us find conceptual gaps in the spec