So you think you understand JSON Schema? - ASC 2021

Recording: https://www.youtube.com/watch?v=vMG0NCDifI0

With OpenAPI 3.1 fully adopting JSON Schema draft 2020-12, and AsyncAPI growth and use of JSON Schema, it's never been more important to really understand JSON Schema.

Draft 2019-09 and 2020-12 have introduced the notion of Dialects and Vocabularies, which opens up a standardised approach to extensions.

In this session you'll learn some key fundamentals, intracacies that even catch out the experienced, and how to develop your own interoperable JSON Schema Vocabulary.

If you're responsible for your organisations use of OpenAPI, AsyncAPI, JSON Schema, or any other technology that uses JSON Schema, this is your essential session for the year, brought to you by the JSON Schema project lead and publisher of the latest draft.

Transcript

1. ASC 2021 – Ben Hutton @relequestual / benhutton.me – JSON Schema / Postman So you think you know JSON Schema? 🧔 Ben Hutton 💻 @relequestual on the internet 👨🚀 JSON Schema Specification Lead @ Postman! 👨💻 benhutton.me 🙌 opencollective.com/json-schema 🎤 API Specifications Conference 2021
2. ASC 2021 – Ben Hutton @relequestual / benhutton.me – JSON Schema / Postman What we will discover • Validation is permissive and constraints based • Subschemas all the way down • How to identify Applicability, Assertion, and Annotations • URIs are different to URLs, and it matters • URI resolution is probably something you already know
3. ASC 2021 – Ben Hutton @relequestual / benhutton.me – JSON Schema / Postman JSON Schema!
4. ASC 2021 – Ben Hutton @relequestual / benhutton.me – JSON Schema / Postman What’s a JSON Schema again?
5. ASC 2021 – Ben Hutton @relequestual / benhutton.me – JSON Schema / Postman A JSON Schema is…. • A vocabulary that allows you to annotate and validate JSON documents. • An IETF "personal draft" document specification • JSON! • Must be a Boolean or an Object
6. ASC 2021 – Ben Hutton @relequestual / benhutton.me – JSON Schema / Postman A JSON Schema is a Boolean or an Object Always passes validation Always fails validation An empty object. No constraints. Boolean Object `not` inverts the assertion
7. ASC 2021 – Ben Hutton @relequestual / benhutton.me – JSON Schema / Postman Validation is permissive and constraints based
8. ASC 2021 – Ben Hutton @relequestual / benhutton.me – JSON Schema / Postman Question: `oneOf` not validating correctly? Validation is permissive and constraints based What's the validation result? Fail, because the instance is valid according to both subschemas. Subschema at `oneOf[1]` does not prevent the data in the instance. JSON Schema is constraints based. Anything which is not constrained, is allowed. JSON Schema is permissive.
9. ASC 2021 – Ben Hutton @relequestual / benhutton.me – JSON Schema / Postman Question: `oneOf` not validating correctly? Validation is permissive and constraints based "The issue I am facing is that both of the `metricsGroups` arrays in my believed to be valid data validate against both of the sub-schemas - this then invalidates the data due to the use of the `oneOf` keyword." Problem? 👏 Correctly identified! Cause? 🤔 Unconstrained properties Solution? 👨💻 Add `"additionalProperties": false` to each of the subschema values in the `oneOf` array.
10. ASC 2021 – Ben Hutton @relequestual / benhutton.me – JSON Schema / Postman Question: `oneOf` not validating correctly? Validation is permissive and constraints based “The issue I am facing is that both of the `metricsGroup` arrays in my believed to be valid data validate against both of the sub-schemas - this then invalidates the data due to the use of the `oneOf` keyword.” Solution? 👨💻 Add to each of the subschema values in the `oneOf` array. Why? 🧐 The `properties` keyword only validates the VALUES, not the keys. `additionalProperties` - Takes a Schema for its value - The Schema value is applied to the VALUE of each key that is not defined in the `properties`* keyword - Restricts the allowed keys in the object in this case - Schema could be a value other than `false` * And that does not match those defined in `patternProperties`
11. ASC 2021 – Ben Hutton @relequestual / benhutton.me – JSON Schema / Postman Subschemas all the way down
12. ASC 2021 – Ben Hutton @relequestual / benhutton.me – JSON Schema / Postman Question: How do I reuse JSON Schema? Subschemas all the way down "I am needing to use a sub schema multiple times in my JSON file. How do I use the `Object1` subschema for an unknown number of objects?"
13. ASC 2021 – Ben Hutton @relequestual / benhutton.me – JSON Schema / Postman Question: How do I reuse JSON Schema? Subschemas all the way down "I am needing to use a sub schema multiple times in my JSON file. How do I use the `Object1` subschema for an unknown number of objects?" Solution? 👨💻 Use the `additionalProperties` keyword rather than `properties` to apply the subschema value to all object values. 🧐 Subschema "Object1" is moved under `additionalProperties` Why? 🧐 `additionalProperties` takes a JSON Schema for its value.
14. ASC 2021 – Ben Hutton @relequestual / benhutton.me – JSON Schema / Postman Question: How do I use if/then/else? Subschemas all the way down "I have recently discovered the if-then-else keywords that are available in JSON Schema draft-07. How do I use the `else` keyword?" Solution? 👨💻 Add a similar subschema to `then` as you have used for your `if` schema value Why? 🧐 `then` takes a JSON Schema as its value.
15. ASC 2021 – Ben Hutton @relequestual / benhutton.me – JSON Schema / Postman Applicability, Assertion, and Annotations
16. ASC 2021 – Ben Hutton @relequestual / benhutton.me – JSON Schema / Postman Question: How do I use if/then/else? Applicability, Assertion, and Annotation Solution? 👨💻 Add a similar subschema to `then` as you have used for your `if` schema value How is this applicable? - `if` subschema is Applied - `if` is an Applicator keyword - Determines another subschema to apply based on the Assertion result of its schema value.
17. ASC 2021 – Ben Hutton @relequestual / benhutton.me – JSON Schema / Postman Question: What happens if `default` is an object? Applicability, Assertion, and Annotations "If I have a `default` set for an object, what happens if I don't provide one property in the instance JSON?" Answer? 👨💻 Nothing, in both cases. Why? 🧐 `default` is an annotation only keyword.
18. ASC 2021 – Ben Hutton @relequestual / benhutton.me – JSON Schema / Postman URIs are not URLs
19. ASC 2021 – Ben Hutton @relequestual / benhutton.me – JSON Schema / Postman Question: $id $ref not working in JSON Schema URIs are not URLs "I have defined multiple JSON Schema files. References from one file to the another do not work. This library tries to make a network request, but the specification says that $ids are URIs and not necessarily network addressable. I get a file not found error." Answer? 👨💻 If the Schemas are not network addressable, you need to tell the implementation where to find them. Why? 🧐 JSON Schema uses URIs, which may not be network addressable. - Do not rely on a network connection - Do not assume any URIs are network addressable Network Addressable? The URI can be accessed as a URL across the local network or external network (the internet).
20. ASC 2021 – Ben Hutton @relequestual / benhutton.me – JSON Schema / Postman URI resolution is not hard
21. ASC 2021 – Ben Hutton @relequestual / benhutton.me – JSON Schema / Postman URI resolution is not hard 1. http://a/b/c/d 2. http://c/d 3. Error 4. c://d https://twitter.com/relequestual/status/1435956665057046530
22. ASC 2021 – Ben Hutton @relequestual / benhutton.me – JSON Schema / Postman URI resolution is not hard https://twitter.com/relequestual/status/1435956665057046530
23. ASC 2021 – Ben Hutton @relequestual / benhutton.me – JSON Schema / Postman URI resolution is not hard 😬 https://twitter.com/relequestual/status/1435956665057046530
24. ASC 2021 – Ben Hutton @relequestual / benhutton.me – JSON Schema / Postman URI resolution is not hard https://twitter.com/relequestual/status/1435956665057046530 Maybe not so easy!
25. ASC 2021 – Ben Hutton @relequestual / benhutton.me – JSON Schema / Postman URI resolution is not hard
26. ASC 2021 – Ben Hutton @relequestual / benhutton.me – JSON Schema / Postman URI resolution is probably something you already know (In most cases)
27. ASC 2021 – Ben Hutton @relequestual / benhutton.me – JSON Schema / Postman Question: Error validating using $ref? URI resolution is probably something you already know "error Could not resolve schema reference '#/definitions/categoryList'. Path properties.publishing.items.properties.territory', line 18." Answer? 👨💻 The full path is required for referencing `categoryList`, which is `properties.publishing.definitions.categoryList`. Why? 🧐 URI fragments are either plain names or JSON Pointers. JSON Pointers are resolved from the resource root (usually the document root), not the closest parent schema. - Relative URI resolution - Against the Base URI - Base URI as defined by RFC3986 FYI: `definitions` in draft-07 is now `$defs` from draft 2019-09
28. ASC 2021 – Ben Hutton @relequestual / benhutton.me – JSON Schema / Postman Question: Can a property inside `allOf` $ref another `allOf` property? URI resolution is probably something you already know "Is "#/allOf/1/properties/body/properties/foo" a valid $ref? I get an error that it can't be resolved." The error: can't resolve reference #/allOf/1/properties/body/properties/foo from id http://example.org/foo.schema.json" Answer? 👨💻 "$ref" would need to be "#/properties/foo" Why? 🧐 `$id` changes the Base URI. Relative references resolve against the Base URI of the current resource, not the document root. - Relative URI resolution - Against the closest Base URI - $id identifies an "embedded resource" - $id may be omitted - Base URI as defined by RFC3986
29. ASC 2021 – Ben Hutton @relequestual / benhutton.me – JSON Schema / Postman Determining the Base URI as per RFC 3986 Understanding how the Base URI is determined hinges on understanding this graphic! https://datatracker.ietf.org/doc/html/rfc3986#section-5.1 Closest $id Base URI from another specification HTTP request or file location* Implementer's choice!
30. ASC 2021 – Ben Hutton @relequestual / benhutton.me – JSON Schema / Postman What we discovered • Validation is permissive and constraints based • Subschemas all the way down • How to identify Applicability, Assertion, and Annotations • URIs are different to URLs, and it matters • URI resolution is probably something you already know • JSON Schema Debugging is easy when you know how
31. ASC 2021 – Ben Hutton @relequestual / benhutton.me – JSON Schema / Postman Where do I go?
32. ASC 2021 – Ben Hutton @relequestual / benhutton.me – JSON Schema / Postman Where to go Basics: json-schema.org/learn More info: json-schema.org/understanding-json-schema Playgrounds: jsonschema.dev (draft-07 only for now) json-schema.hyperjump.io (04 – 2020-12) Support: stackoverflow.com (and tag with `jsonschema`) Community: json-schema.org/slack Twitter: @jsonschema Elevating the community: github.com/json-schema-org/community
33. ASC 2021 – Ben Hutton @relequestual / benhutton.me – JSON Schema / Postman Thank you all • Previous and current team • Contributors and community • Implementation developers Pictures thanks to @bit_loom
34. ASC 2021 – Ben Hutton @relequestual / benhutton.me – JSON Schema / Postman So you think you know JSON Schema? Thank you sponsors! AsyncAPI Initiative 🧔 Ben Hutton 💻 @relequestual on the internet 👨🚀 JSON Schema Specification Lead @ Postman! 👨💻 benhutton.me 🙌 opencollective.com/json-schema 🎤 API Specifications Conference 2021 Retool Stoplight apideck Airbnb Graphics from @bit_loom