Validating JSON with JSON Schema

Validating JSON with JSON Schema

My talk from Berlin PHP Usergroup, December 2013, about the benefits of using JSON schema for validating complex JSON data

5fcfa864ae04004e92416a5ea05d5ffa?s=128

Anne-Julia Seitz

December 03, 2013
Tweet

Transcript

  1. Validating your JSON

  2. About me → Anne-Julia Scheuermann → Creating Web-Applications since 2004

    → Developing information literacy solutions for
  3. Short Story API DB JSON

  4. Short Story API DB JSON

  5. Custom validation

  6. Custom validation protected function isValidUnformatted(array $data) { if (empty($data['data'])) {

    return false; // no data sent } $data = $data['data']; $source = $data['source']; $style = empty($data['style']) ? null : $data['style']; $structure = $this->getStructure($source, $style); $allowedTopLevelFields = $this->getOtherValidFields(); $ignoredDataFields = $this->getIgnoredFields(); // source if (!empty($structure['source_data'])) { if (empty($data[$source])) { $this->errors[] = 'needed source data missing'; return false; } $notAllowed = array_diff_key($data[$source], array_flip($structure['source_data'])); // +300 lines of code more
  7. Many lines of complex code

  8. PHPMD warnings /!\ /** * @param array $data * *

    @Todo Fix suppressed warnings * @SuppressWarnings( * PHPMD.CyclomaticComplexity, * PHPMD.NPathComplexity * ) * @return bool */ protected function isValidUnformatted(array $data) { //...
  9. Validation only on server side API DB JSON

  10. Other technologies have solved this

  11. DTD for SGML <!DOCTYPE HTML PUBLIC ”-//W3C//DTD HTML 4.01//EN” ”http://www.w3.org/TR/html4/strict.dtd”>

    <!--=================== Generic Attributes ===============================--> <!ENTITY % coreattrs ”id ID #IMPLIED -- document-wide unique id -- class CDATA #IMPLIED -- space-separated list of classes -- style %StyleSheet; #IMPLIED -- associated style info -- title %Text; #IMPLIED -- advisory title --” >
  12. XSD for XML (XHTML) <?xml version=”1.0”?> <xs:schema xmlns:xs=”http://www.w3.org/2001/XMLSchema”> <xs:element name=”note”>

    <xs:complexType> <xs:sequence> <xs:element name=”to” type=”xs:string”/> <xs:element name=”from” type=”xs:string”/> <xs:element name=”body” type=”xs:string”/> </xs:sequence> </xs:complexType> </xs:element> </xs:schema>
  13. Short Story API DB JSON

  14. JSON Schema → http://json-schema.org/

  15. Hello World! 42 ”42” {} true

  16. JSON Schema specification Validation Core

  17. JSON Schema specification Validation Core

  18. type {“type”: “string”} {“type”: “integer”} {“type”: “array”} {“type”: “object”} {“type”:

    “boolean”} {“type”: “null”} {“type”: “number”}
  19. string

  20. string ”Hello World!” ”42” {”type”: ”string”} 42

  21. ”php” string length { ”type”: ”string”, ”minLength”: 2, ”maxLength”: 3

    } ”i <? php” ”p”
  22. string pattern { ”type”: ”string”, ”pattern”: ”^and.+$” } ”iphone” ”android”

  23. numeric

  24. integer -1 42 0.5 “42” { ”type”: ”integer” }

  25. number -1 42 0.5 2.99792458e8 { ”type”: ”number” }

  26. number range 10 11 40 { ”type”: ”number”, “minimum”: 10,

    “maximum”: 40, “exclusiveMinimum”: true, “exclusiveMaximum”: false }
  27. array

  28. array [1, 2, 3, 4, 5] [1, “2”, {“3”:”4 and

    5”}, [42]] {”type”: ”array”} {“this”: “is an object”}
  29. array items [] [1, 2, 3, 4, 5] [1, 2,

    3, 4, “5”] { ”type”: ”array”, “items”: {“type”: “number”} }
  30. array length [] [1] [1, 2] { ”type”: ”array”, “minItems”:

    1, “maxItems”: 2 } [1, 2, 3]
  31. object

  32. object { ”key”: ”value”, ”anotherKey”: ”anotherValue” } {”type”: ”object”} ”42”

    [“hello”, “world”]
  33. {”author”: ”A. A. Milne”, ”year”: 1882} properties { ”type”: ”object”,

    ”properties”: { ”author”: {”type”: ”string”}, ”year”: {”type”: ”number”} } } {”year”: ”1882”} {”author”: ”A. A. Milne”} {}
  34. {”author”: ”A. A Milne”, ”title”: 1882} required properties { ”type”:

    ”object”, ”properties”: { ”author”: {”type”: ”string”}, ”title”: {”type”: ”string”}}, ”required”: [”title”] } {} {”author”: ”A. A Milne”, ”title”: ”Winni the Pooh”, ”year”: 1926} {”author”: “A. A Milne”}
  35. additional properties { ”type”: ”object”, ”properties”: { ”author”: {”type”: ”string”},

    ”title”: {”type”: ”string”} }, ”additionalProperties”: false } {”title”: []} {} {”author”: ”A. A. Milne”, ”title”: ”Winni the Pooh”, ”year”: 1926} {”author”: ”A. A. Milne”, ”title”: ”Winni the Pooh”}
  36. {”a”: ”aa”, “b”: “bb”} number of properties { ”type”: ”object”,

    ”minProperties”: 1, ”maxProperties”: 2 } {} {”a”: ”aa”} {”a”: ”aa”, “b”: “bb”, “c”: “cc”}
  37. dependencies

  38. {”name”: ”Alan”, “birthday”: “1882-01-18”} property dependencies { ”type”: ”object”, ”properties”:

    {“name”: {“type”: “string”}, “birthday”: {“type”: “string”}}, ”dependencies”: { “birthday”: [“name”]} } {”name”: ”Alan”} {“birthday”: “1882-01-18”}
  39. {”name”: ”Alan”, “birthday”: “1882-01-18”} schema dependencies { ”type”: ”object”, ”properties”:

    { “birthday”: {“type”: “string”}}, ”dependencies”: { “birthday”: { “properties”: {“name”: {“type”: “string”}}}} } {”name”: ”Alan”} {“birthday”: “1882-01-18”}
  40. {”PHPUnit”: ”testing”, “PHPCS”: “styling”} pattern properties { ”type”: ”object”, ”patternProperties”:

    { “^PHP”: {“type”: “string”}}, “additionalProperties”: {“type”: “integer”} } {”BeHat”: ”accepting”} {“PHPLOC”: 9001} {“LOC”: 9001}
  41. boolean

  42. boolean true false {”type”: ”boolean”} “true” 0

  43. null

  44. null null false {”type”: ”null”} “” 0

  45. More features

  46. “one” “eleven” 1 -11 Combining with oneOf, anyOf, allOf {

    ”anyOf”: [ {“type”: “string”, “maxLength”: 5}, {“type”: “number”, “minimum”: 0} ] }
  47. not “42” {“key”: 42} 42 { ”not”: {“type”: “number”} }

  48. available formats: email, date-time, hostname, ipv4, ipv6, uri ”ab@mail.de” ”ab@mail.”

    built-in formats { ”format”: ”email” }
  49. JSON Schema specification Validation Core

  50. title, description & default { ”title”: ”Expressive title”, ”description”: “Take

    time to explain”, ”default”: “Validators ignore this keyword” }
  51. $schema { “$schema”: “http://json-schema.org/draft-04/schema#”, ”title”: ”Expressive title”, ”description”: “Take time

    to explain”, ”default”: “Validators ignore this keyword” }
  52. definitions { “$schema”: “http://json-schema.org/draft-04/schema#”, “definitions”: { "address": { "type": "object",

    "properties": { "street_address": { "type": "string" }, "city": { "type": "string" }, "state": { "type": "string" } }, "required": ["street_address", "city", "state"] } } }
  53. $ref & definitions { “$schema”: “http://json-schema.org/draft-04/schema#”, “definitions”: { "address": {

    "type": "object", "properties": { "street_address": { "type": "string" }, "city": { "type": "string" }, "state": { "type": "string" } }, "required": ["street_address", "city", "state"] } } "type": "object", "properties": { "billing_address": { "$ref": "#/definitions/address" }, "shipping_address": { "$ref": "#/definitions/address" } } }
  54. $ref & definitions { “$schema”: “http://json-schema.org/draft-04/schema#”, “description”: “This schema defines

    an address”, "type": "object", "properties": { "city": { "type": "string" }, "state": { "type": "string" } }, "required": ["street_address", "city", "state"] } { “$schema”: “http://json-schema.org/draft-04/schema#”, “description”: “This schema describes an order”, "type": "object", "properties": { "billing_address": { "$ref": "address.json#" }, "shipping_address": { "$ref": "address.json#" } } } address.json order.json
  55. Breathe!

  56. Validation Libraries

  57. Validation Libraries geraintluff/jsv4-php 11 commits, 1 contributor hasbridge/php-json-schema 28 commits,

    2 contributors, not yet feature complete justinrainbow/json-schema 202 commits, 21 contributors
  58. Integration of schema validation

  59. Preparing validation with json-schema // Get the schema as object

    $retriever = new JsonSchema\Uri\UriRetriever; $schema = $retriever->retrieve('file://' . realpath('schema.json')); // Resolve references $refResolver = new JsonSchema\RefResolver($retriever); $refResolver->resolve($schema, 'file://' . __DIR__);
  60. Validation with json-schema /** * Validate json against schema *

    * @param string $data * @param string $schema * * @return bool */ public function isValid($data, $schema) { $data = json_decode($data); $this->validator->check($data, $schema); $this->errors = $this->validator->getErrors(); return $this->validator->isValid(); }
  61. The benefits of using JSON Schema

  62. Less code /** * Validate json against schema * *

    @param string $data * @param string $schema * * @return bool */ public function isValid($data, $schema) { $data = json_decode($data); $this->validator->check($data, $schema); $this->errors = $this->validator->getErrors(); return $this->validator->isValid(); }
  63. No custom validation

  64. Complexity Complexity of validation and document structure is independent

  65. Validation on client and server side API DB JSON

  66. Implementation in many languages PHP, JavaScript, Java, Python, Ruby, Perl,

    .NET, C, Haskell, Erlang, Go
  67. Better user experience Clear, human- and machine-readable documentation of our

    data structure
  68. Thank You! • http://json-schema.org/ • http://tools.ietf.org/html/draft-zyp-json-schema-04 • https://en.wikipedia.org/wiki/JSON • http://spacetelescope.github.io/understanding-json-schema/

    • https://packagist.org/packages/justinrainbow/json-schema • http://getcomposer.org
  69. Questions?