RESTfulness for the Rest of Us

REST for Fun and Proﬁt Modelling the Graph hypr Amuse-Bouche
Yggdrasil . . . and Finally RESTfulness for the Rest of Us Developing a Hypermedia Type Christopher Harrison [email protected] Human Genetics Informatics · Wellcome Trust Sanger Institute Tuesday 21st July, 2015

Yggdrasil . . . and Finally The Path to RESTlightenment The Richardson Maturity Model[1]: Level 0 Single source/sink interface Level 1 Separate resources Level 2 Semantic use of HTTP Level 3 Hypermedia driven

Yggdrasil . . . and Finally Ye Olˇ Bla¨-Boxen of Yore (e.g., Cherwell) • Single endpoint • Communication is opaque and context dependent • Uses HTTP as an RPC tunnel (i.e., just for transport) • e.g., SOAP (without WSDL. . . which doesn’t improve the situation much!)

Yggdrasil . . . and Finally HTTP Semantics • Stateless protocol • Transactional, request-response interface • Well-deﬁned and delineated[2]: • Request methods • Response status codes • Addressable/identiﬁable resources • Caching mechanism

Yggdrasil . . . and Finally RESTful HTTP Methods Method Nullipotent Idempotent GET - Retrieve the resource POST Create subordinate data under the resource address PUT Upsert (create or update) data at the address DELETE Delete the resource n.b., Other methods are available

Yggdrasil . . . and Finally Hypermedia as the Engine of Application State • Single entrypoint • Resources linked together • Link relations facilitate self-documentation • Discoverable graph-based information system This is real RESTfulness, per Fielding’s deﬁnition[3]

Yggdrasil . . . and Finally Impedance Mismatch RESTful API

Yggdrasil . . . and Finally Impedance Mismatch RESTful API DBI

Yggdrasil . . . and Finally General Structure

Yggdrasil . . . and Finally Departure from Graph Theory • The graph is necessarily connected∗ • Each vertex (resource) can have arbitrary, directed edges (links) to others vertices • Edges must be represented within their source vertex • Edges are named (link relations) and form a many-to-many relationship • Each vertex ought to have a loop (link to self). . . which, FWIW, gives us a category ∗With respect to some deﬁned entrypoint

Yggdrasil . . . and Finally Collections of Collections of. . .

Yggdrasil . . . and Finally Normalisation, Symlinking and Canonicalisation

Yggdrasil . . . and Finally Conditional Representation Need to be able to ﬁlter state and collections in useful ways. This can be done at the request level using the IRI query component: Selective Reducing to just what the user cares about Conditional Using a serialisable querying language (similar to LDAP search ﬁlters) Partitional For result pagination or arbitrary, contiguous slicing Useful to persist conditional representations (cf. SQL views), which are themselves open to additional conditions

Yggdrasil . . . and Finally Ontology Driven • How can a software agent explore a graph meaningfully without suﬃcient intelligence guiding it? • Regardless of possibility, the model is still useful for other consumers (e.g., web crawlers, you and me) • However, given a suﬃciently rich knowledge base, a client could purposefully navigate and manipulate a graph

Yggdrasil . . . and Finally Example Command manager of foo becomes quux Ontology • manager is type Person • Person has mutable property manager • become is an idempotent update

Yggdrasil . . . and Finally Open Question: Procedural-Graph Interface? • The impedance mismatch between relational databases and object-orientated programming led to the development of ORM (object-relational mapping) tools • There is a similar—and demonstrable—mismatch between a graph database and remote procedure calling. Does it therefore make sense to develop an analogue: PGI ("piggy")? • The ontology driven model satisﬁes this—albeit somewhat non-deterministically—but can we do better?

Yggdrasil . . . and Finally A Hypermedia Type h y p representations

Yggdrasil . . . and Finally A Hypermedia Type hypr y p representations

Yggdrasil . . . and Finally A Hypermedia Type hypr yields p representations

Yggdrasil . . . and Finally A Hypermedia Type hypr yields pluripotent representations

Yggdrasil . . . and Finally Representing the Graph • Need a stable representation format that covers our model • Any new format needs to be “human friendly”—i.e., transparent and non-verbose—to drive adoption • Otherwise, expand on the best ideas from existing formats and representations (i.e., don’t reinvent too many wheels!)

Yggdrasil . . . and Finally Generic Serialisation? • JSON serialisation is [currently] baked into the speciﬁcation • Should hypr be more general? • Other potential serialisation targets: • YAML • XML • HTML5 extensions • Driven by the Accept request header

Yggdrasil . . . and Finally Basic Representation • All resources have a set of relational links (minimally, one to themselves) • Resources may also contain arbitrary, key-value state • Links can be associated with: • State elements • Conditional representations (e.g., next/previous) • Arbitrary hypr resources • hypr Collections • Arbitrary foreign resources

Yggdrasil . . . and Finally Progressive Enhancement • To facilitate “friendliness”, the minimal representation should be very bare, but still functional • Optional augmentations may be added to the representation, at increasing degree, to improve its ﬁdelity (e.g., gradual typing) • More annotation gives the machine a better chance of making meaningful use of the data, but this is at the developer’s discretion

Yggdrasil . . . and Finally Type Checking and Structural Inference • JSON has very bare primitive types and, importantly, no mechanism for signalling IRIs • We provide a type annotation system that allows JSON’s primitives to be subtyped (e.g., numeric can be subtyped by set, strings can be designated as IRIs, etc.) • Type annotations can also declare quantiﬁcation, optionality and mutability—amongst other things—which a machine can use meaningfully • The structure of the representation implies what could be done with it, without checking the Allow header. For example, an embedded, mutable collection implies that the resource (probably) allows POST requests

Yggdrasil . . . and Finally Recursive Structure • hypr collections can be minimally or maximally embedded within a representation Minimal Enough to dereference each collection item Maximal Each collection item’s links and state • Collection items are necessarily of the same class, so any type annotations are lifted out (rather than redundantly repeated) • Embedding depth is at the server’s control; bottoming out with a minimal representation

Yggdrasil . . . and Finally Contract Programming • An interface with a set of assertions that both parties (i.e., client and server) must abide by during any transaction • Using the Fielding constraint of “Code on Demand”, we can (optionally) specify a programmatic contract that both the client and server must validate to accept any input; i.e., predication on state • Contract code must conform to a standard interface and, preferably be pure (or at least sandboxed) for security • This gives us even more control over the data, beyond type checking, to help avoid invalid state transitions

Yggdrasil . . . and Finally Standardisation Route http://xkcd.com/927 • Still an internal speciﬁcation; open to tweaks • To be publicly released as application/vnd.hypr • Ultimate goal is formal standardisation (e.g., via the IETF), with the application/hypr designation

Yggdrasil . . . and Finally Minimal { "links": { "self": "/path/to/resource" } }

Yggdrasil . . . and Finally Simple, Untyped State with Linked Data { "links": { "self": "/people/foo", "manager": "/people/bar" }, "state": { "name": "Joe Bloggs", "mail": "[email protected]", "manager": "President Business" } }

Yggdrasil . . . and Finally Typed State { "links": { "self": "/people/foo" }, "state": { "name": { "value": "Joe Bloggs", "type": { "primitive": "text", "label": "Name" } } } }

Yggdrasil . . . and Finally Type Zoo Text: { "primitive": "text" } Temporal: { "primitive": "text", "subtype": "datetime" } Raw data, by media type: { "primitive": "text", "subtype": "image/png;base64" } Pattern match, by regular expression: { "primitive": "text", "subtype": "/^regex(p?)$" } Numeric: { "primitive": "number" } Numeric, by set, interval and stepping: { "primitive": "number", "subtype": "int[1,)/2" } Boolean: { "primitive": "bool" } Enumeration: { "primitive": { "l": "Left", "r": "Right" } }

Yggdrasil . . . and Finally Type Annotations • Human-readable description of data • Expected number of data items, using regular expression curly-brace format and shortcuts • Mutability • Default (scalar) value For example: { "primitive": "text", "label": "Your Message", "mutable": true, "quantity": "?", "default": "My hovercraft is full of eels!" }

Yggdrasil . . . and Finally Simple Collection { "links": { "self": "/people", "collection": "/people/{id}" }, "state": { "collection": ["foo", "bar", "quux"] } }

Yggdrasil . . . and Finally Embedded Collection { "links": { "self": "/people", "collection": "/people/{id}" }, "state": { "collection": { "value": [ { "links": { "self": "/people/foo" }, "state": { "id": "foo" } } ], "type": { "primitive": "collection", "subtype": { "id": { "primitive": "text" } } } } } }

Yggdrasil . . . and Finally Paginated, Conditional Simple Collection { "links": { "self": "/people?q=(dept=sales)&slice=5:10", "prev": "/people?q=(dept=sales)&slice=:5", "next": "/people?q=(dept=sales)&slice=10:15", "base": "/people", "people": "/people/{id}" }, "state": { "people": ["foo", "bar", "baz", "quux", "xyzzy"] } }

Yggdrasil . . . and Finally Yggdrasil Reference Implementation • Server framework implementation; currently envisioned as Connect/Express middleware for Node.js • Query parsers and primitive type checkers are complete, per the ﬁrst draft of the spec • Development questions: • Repurpose as a DBMS, in something more suitable (e.g., C, Rust, Haskell, Erlang/Elixir), with a companion interface for Node.js (and others; e.g., Python/Flask)? • Full DBMS functionality, or a wrapper over a mature graph database (e.g., Neo4j, etc.)? • Stick with Node.js and use the same code to also produce a client framework?

Yggdrasil . . . and Finally Repositories On RESTfulness https://github.com/wtsi-hgi/hgi-web/wiki/Hypermedia Yggdrasil https://github.com/wtsi-hgi/yggdrasil

Yggdrasil . . . and Finally Select References Martin Fowler. Richardson security model. http://martinfowler.com/articles/ richardsonMaturityModel.html, 2010. Hypertext transfer protocol (HTTP/1.1): Semantics and content. RFC 7231, IETF, 2014. Roy Fielding. Architectural Styles and the Design of Network-Based Software Architectures. PhD thesis, University of California, Irvine, 2000.

Yggdrasil . . . and Finally Acknowledgements Thanks to: • Josh Randall • Irina Colgiu • Emyr James • John Constable • Simon Fraser • Jon Nicholson • Pete Clapham Developed from: • Collection+JSON Amundsen (and Richardson) • HAL Kelly • Siren Wiber • JSON-LD and Hydra Sporny, Lathaler et al. Questions?

RESTfulness for the Rest of Us

RESTfulness for the Rest of Us

More Decks by Christopher Harrison

Other Decks in Programming

Featured

Transcript