JSON Schema Draft-8 - To $vocabularies and Beyond

Fund more of these talks: https://opencollective.com/json-schema
Tip this talk: https://ko-fi.com/relequestual

NOTE: Some of the examples found in these slides are inaccurate or incorrect as of publication of draft 2019-09 (formally draft-8). Please check with the JSON Schema project (via our open slack) as these slides will not be updated.

JSON Schema has been around for quite a while. Back in 2015, a small group decided to pick up development from draft-4, publishing a futher 4 drafts (draft-8 pending).

JSON Schema is primerily used for validation, but during the period between draft-4 and rebooting the project, a number of other use cases developed, including form validation and code generation.

Each library that extends the functionality of JSON Schema by adding their own key words, creates their own extensions, which only work with that specific library, or function differently for similar libraries.

JSON Schema draft-8 adds a new concept. Vocabularies. Allowing groups to specify new sets of keywords, and schemas to identify as using a set of keywords.

Learn about these and other draft-8 keywords, such as `unevaluatedProperties`, which can "see through" conditional applicators, a common user wish.

Transcript

1. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev
   JSON Schema
   Draft-8 – To $vocabularies and Beyond
   ! Ben Hutton
   " @relequestual on the internet
   # JSON Schema core
   $ opencollective.com/json-schema
   ko-fi.com/relequestual
   % ASC 2019 - API Specification Conference 2019
   

   View Slide

2. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev
   Actually…
   It’s draft `2019-09` now
   

   View Slide

3. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev
   Vocabularies
   

   View Slide

4. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev
   Vocabularies
   • A "Vocabulary" vs a "meta-schema"
   • '$vocabularies' created for "Vocabulary" authors
   • Construction of new and extension of existing vocabularies
   • Keys are URIs which identify the vocabulary
   • Values are Boolean which identifies required or optional support
   • URI for a Vocabulary and associated meta-schema will always be different
   

   View Slide

5. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev
   Vocabularies – An Example
   Value indicates if
   support is required or
   optional
   

   View Slide

6. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev
   Vocabularies – An Example
   Keys are URIs
   As with "$id", these
   are not necessarily
   network addressable.
   "The nature of the
   retrievable resource is
   currently undefined,
   and reserved for future
   use."
   

   View Slide

7. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev
   "$ref" is an applicator
   

   View Slide

8. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev
   "$ref", then and now
   "$ref" not allowed to play with others
   (All other properties ignored)
   "$ref" is grown up now and can play with others
   (Other properties not ignored)
   You still shouldn't duplicate keys of an object. That's JSON.
   

   View Slide

9. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev
   Annotation Collection
   Annotations, Errors, and Output
   

   View Slide

10. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev
    Annotations, Errors, and Output
    'draft-7' users:
    "I can see validation fails, but I'd like more detailed information."
    "I have an error message, but I can't see where it comes from."
    "I'd like to apply the default values to the data."
    "Can I get some information on why validation passed?"
    Us: "errr, that would depend on the implementation…. Sorry."
    

    View Slide

11. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev
    Annotations, Errors, and Output
    'draft 2019-09': Standard Output
    • Flag - A boolean which simply indicates the overall validation result
    with no further details.
    • Basic - Provides validation information in a flat list structure.
    • Detailed - Provides validation information in a condensed hierarchical
    structure based on the structure of the schema.
    • Verbose - Provides validation information in an uncondensed
    hierarchical structure that matches the exact structure of the schema.
    

    View Slide

12. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev
    Annotations, Errors, and Output
    'draft 2019-09': Standard Output
    But won't that require a bit of extra work for implementations?
    Use of 'unevaluated*' keywords will require annotation collection
    anyway, or throwing an error (more on that later)
    

    View Slide

13. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev
    Annotations, Errors, and Output
    'draft 2019-09': Standard Output – Flag
    

    View Slide

14. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev
    Annotations, Errors, and Output
    'draft 2019-09': Standard
    Output – Basic
    Contains:
    • overall 'valid' assertion
    • 'errors' array
    Locations are JSON Pointers
    

    View Slide

15. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev
    Annotations, Errors, and Output
    'draft 2019-09': Standard Output – Detailed and Verbose
    • More detailed
    • Hierarchical
    • "Detailed" contains only what is most likely to be useful
    • "Verbose" returns all results
    • (including for example annotations for failed validations)
    

    View Slide

16. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev
    "$anchor"
    

    View Slide

17. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev
    "$anchor"
    • Embedding a schema inside another schema is a common concern
    • Creating a single schema out of many small ones
    • "$id" is used at the root of a schema to identify itself
    • "$id" previously could be a plain name fragment in subschemas
    • Now use "$anchor" for plain name fragment subschema identification
    • This allows transcluded schemas with relative references to continue
    to function as expected, rather than potentially breaking.
    • Further reading in the spec: "canonical URI" and Core Appendix A
    http://json-schema.org/draft/2019-09/json-schema-core.html#idExamples
    

    View Slide

18. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev
    "$recursive*"
    One for meta-schema authors
    

    View Slide

19. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev
    "$recursiveRef" and "$recursiveAnchor"
    • A recursive schema is one that has a reference to its own root
    • New keywords allow the construction of extensible recursive
    schemas.
    

    View Slide

20. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev
    "$recursiveRef" and "$recursiveAnchor"
    Base schema "tree" uses "$recusiveRef"
    with target of "#" (document root).
    The schema on its own results in
    "$recursiveRef" working the same as "$ref"
    http://json-schema.org/draft/2019-09/json-schema-core.html#rfc.appendix.C
    In evaluating "strict-tree" (right), the referenced schema "tree" (left) uses "$recursiveRef" on line 11.
    The dynamic scope path is checked; the outermost schema which defines "$recursiveAnchor" is now the
    target ("strict-tree").
    Being able to prohibit unknown keywords is a common question. Previously, you would need to duplicate the
    whole meta-schema. Now it's EASY!
    

    View Slide

21. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev
    "unevaluated*"
    

    View Slide

22. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev
    "unevaluatedProperties"
    • Value must be a Schema
    • Value Schema is applied to child values of instance names that have
    not been evaluated (annotation collection)
    • "seeing through" applicator keywords
    • Annotation collection support required, or must raise an error
    

    View Slide

23. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev
    "unevaluatedProperties"
    Instance data is of a patient, which is made up of
    schemas "patient" and "person". These could be
    external references.
    "unevaluatedProperties" is similar to
    "additionalProperties"
    Value schema is applied based on the annotation
    collection of other keywords (which must be done
    first).
    "name" and "age from "person, and "patientID"
    and "patientDetails" from patient, will have
    annotations.
    Other keys in the instance object will cause
    validation failure
    

    View Slide

24. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev
    "unevaluatedItems"
    • Similar to "unevaluatedProperties"
    • Value Schema is applied to child values in the array which do not have
    annotation results
    • All other keywords must be evaluated first
    • Annotation collection support required, or must raise an error
    • Only applies when "items" is an array as opposed to a single Schema
    (Otherwise must be ignored)
    

    View Slide

25. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev
    "unevaluatedItems" This schema represents a function call
    stored in something
    (Not saying this is the best idea in the
    world, but I had to invent an example)
    "$defs" defines an "httpError"
    "httpError" is an array, minimum 3
    items.
    The first and third items must be a
    string, and the second item must be a
    number.
    Want to extend the function call array
    to take additional items which must be
    objects.
    Subschema applies to all unevaluated
    items of the applicable array.
    Relies on annotation collection!
    

    View Slide

26. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev
    Beyond?
    What's next for JSON Schema?
    

    View Slide

27. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev
    Beyond draft-8 draft 2019-09
    opencollective.com/json-schema
    !
    "
    

    View Slide

28. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev
    Beyond draft-8 draft 2019-09
    • opencollective.com/json-schema
    • Update test suite (including output)
    • Educational resource restructuring!
    • Official linter tooling
    • One (or two) more SMALL updates (promise)
    • Define a clear path to standardisation adoption
    • Multiple possible paths are available
    

    View Slide

29. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev
    Thank you all
    • Previous and current team
    • Contributors and community
    • Implementation developers
    • all of you
    

    View Slide

30. #ASC2019 – Ben Hutton @relequestual – JSON Schema core – jsonschema.dev
    JSON Schema
    Draft-8 – To $vocabularies and Beyond
    Thank you sponsors!
    ! Ben Hutton
    " @relequestual on the internet
    # JSON Schema core
    $ opencollective.com/json-schema -----> %&
    ko-fi.com/relequestual
    ' ASC 2019 - API Specification Conference 2019
    AsyncAPI Initiative Stoplight
    JSON Buddy
    

    View Slide