JSON Schema Draft-8 - To $vocabularies and Beyond

Fund more of these talks: https://opencollective.com/json-schema
Tip this talk: https://ko-fi.com/relequestual

NOTE: Some of the examples found in these slides are inaccurate or incorrect as of publication of draft 2019-09 (formally draft-8). Please check with the JSON Schema project (via our open slack) as these slides will not be updated.

JSON Schema has been around for quite a while. Back in 2015, a small group decided to pick up development from draft-4, publishing a futher 4 drafts (draft-8 pending).

JSON Schema is primerily used for validation, but during the period between draft-4 and rebooting the project, a number of other use cases developed, including form validation and code generation.

Each library that extends the functionality of JSON Schema by adding their own key words, creates their own extensions, which only work with that specific library, or function differently for similar libraries.

JSON Schema draft-8 adds a new concept. Vocabularies. Allowing groups to specify new sets of keywords, and schemas to identify as using a set of keywords.

Learn about these and other draft-8 keywords, such as `unevaluatedProperties`, which can "see through" conditional applicators, a common user wish.

Transcript

1. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev JSON Schema Draft-8 – To $vocabularies and Beyond ! Ben Hutton " @relequestual on the internet # JSON Schema core $ opencollective.com/json-schema ko-fi.com/relequestual % ASC 2019 - API Specification Conference 2019
2. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev Actually… It’s draft `2019-09` now
3. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev Vocabularies
4. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev Vocabularies • A "Vocabulary" vs a "meta-schema" • '$vocabularies' created for "Vocabulary" authors • Construction of new and extension of existing vocabularies • Keys are URIs which identify the vocabulary • Values are Boolean which identifies required or optional support • URI for a Vocabulary and associated meta-schema will always be different
5. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev Vocabularies – An Example Value indicates if support is required or optional
6. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev Vocabularies – An Example Keys are URIs As with "$id", these are not necessarily network addressable. "The nature of the retrievable resource is currently undefined, and reserved for future use."
7. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev "$ref" is an applicator
8. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev "$ref", then and now "$ref" not allowed to play with others (All other properties ignored) "$ref" is grown up now and can play with others (Other properties not ignored) You still shouldn't duplicate keys of an object. That's JSON.
9. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev Annotation Collection Annotations, Errors, and Output
10. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev Annotations, Errors, and Output 'draft-7' users: "I can see validation fails, but I'd like more detailed information." "I have an error message, but I can't see where it comes from." "I'd like to apply the default values to the data." "Can I get some information on why validation passed?" Us: "errr, that would depend on the implementation…. Sorry."
11. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev Annotations, Errors, and Output 'draft 2019-09': Standard Output • Flag - A boolean which simply indicates the overall validation result with no further details. • Basic - Provides validation information in a flat list structure. • Detailed - Provides validation information in a condensed hierarchical structure based on the structure of the schema. • Verbose - Provides validation information in an uncondensed hierarchical structure that matches the exact structure of the schema.
12. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev Annotations, Errors, and Output 'draft 2019-09': Standard Output But won't that require a bit of extra work for implementations? Use of 'unevaluated*' keywords will require annotation collection anyway, or throwing an error (more on that later)
13. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev Annotations, Errors, and Output 'draft 2019-09': Standard Output – Flag
14. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev Annotations, Errors, and Output 'draft 2019-09': Standard Output – Basic Contains: • overall 'valid' assertion • 'errors' array Locations are JSON Pointers
15. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev Annotations, Errors, and Output 'draft 2019-09': Standard Output – Detailed and Verbose • More detailed • Hierarchical • "Detailed" contains only what is most likely to be useful • "Verbose" returns all results • (including for example annotations for failed validations)
16. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev "$anchor"
17. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev "$anchor" • Embedding a schema inside another schema is a common concern • Creating a single schema out of many small ones • "$id" is used at the root of a schema to identify itself • "$id" previously could be a plain name fragment in subschemas • Now use "$anchor" for plain name fragment subschema identification • This allows transcluded schemas with relative references to continue to function as expected, rather than potentially breaking. • Further reading in the spec: "canonical URI" and Core Appendix A http://json-schema.org/draft/2019-09/json-schema-core.html#idExamples
18. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev "$recursive*" One for meta-schema authors
19. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev "$recursiveRef" and "$recursiveAnchor" • A recursive schema is one that has a reference to its own root • New keywords allow the construction of extensible recursive schemas.
20. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev "$recursiveRef" and "$recursiveAnchor" Base schema "tree" uses "$recusiveRef" with target of "#" (document root). The schema on its own results in "$recursiveRef" working the same as "$ref" http://json-schema.org/draft/2019-09/json-schema-core.html#rfc.appendix.C In evaluating "strict-tree" (right), the referenced schema "tree" (left) uses "$recursiveRef" on line 11. The dynamic scope path is checked; the outermost schema which defines "$recursiveAnchor" is now the target ("strict-tree"). Being able to prohibit unknown keywords is a common question. Previously, you would need to duplicate the whole meta-schema. Now it's EASY!
21. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev "unevaluated*"
22. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev "unevaluatedProperties" • Value must be a Schema • Value Schema is applied to child values of instance names that have not been evaluated (annotation collection) • "seeing through" applicator keywords • Annotation collection support required, or must raise an error
23. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev "unevaluatedProperties" Instance data is of a patient, which is made up of schemas "patient" and "person". These could be external references. "unevaluatedProperties" is similar to "additionalProperties" Value schema is applied based on the annotation collection of other keywords (which must be done first). "name" and "age from "person, and "patientID" and "patientDetails" from patient, will have annotations. Other keys in the instance object will cause validation failure
24. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev "unevaluatedItems" • Similar to "unevaluatedProperties" • Value Schema is applied to child values in the array which do not have annotation results • All other keywords must be evaluated first • Annotation collection support required, or must raise an error • Only applies when "items" is an array as opposed to a single Schema (Otherwise must be ignored)
25. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev "unevaluatedItems" This schema represents a function call stored in something (Not saying this is the best idea in the world, but I had to invent an example) "$defs" defines an "httpError" "httpError" is an array, minimum 3 items. The first and third items must be a string, and the second item must be a number. Want to extend the function call array to take additional items which must be objects. Subschema applies to all unevaluated items of the applicable array. Relies on annotation collection!
26. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev Beyond? What's next for JSON Schema?
27. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev Beyond draft-8 draft 2019-09 opencollective.com/json-schema ! "
28. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev Beyond draft-8 draft 2019-09 • opencollective.com/json-schema • Update test suite (including output) • Educational resource restructuring! • Official linter tooling • One (or two) more SMALL updates (promise) • Define a clear path to standardisation adoption • Multiple possible paths are available
29. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev Thank you all • Previous and current team • Contributors and community • Implementation developers • all of you
30. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev JSON Schema Draft-8 – To $vocabularies and Beyond Thank you sponsors! ! Ben Hutton " @relequestual on the internet # JSON Schema core $ opencollective.com/json-schema -----> %& ko-fi.com/relequestual ' ASC 2019 - API Specification Conference 2019 AsyncAPI Initiative Stoplight JSON Buddy