Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Validating JSON with JSON Schema

Validating JSON with JSON Schema

My talk from Berlin PHP Usergroup, December 2013, about the benefits of using JSON schema for validating complex JSON data

Anne-Julia Seitz

December 03, 2013
Tweet

More Decks by Anne-Julia Seitz

Other Decks in Technology

Transcript

  1. Validating your JSON

    View full-size slide

  2. About me
    → Anne-Julia Scheuermann
    → Creating Web-Applications since 2004
    → Developing information literacy solutions for

    View full-size slide

  3. Short Story
    API DB
    JSON

    View full-size slide

  4. Short Story
    API DB
    JSON

    View full-size slide

  5. Custom validation

    View full-size slide

  6. Custom validation
    protected function isValidUnformatted(array $data) {
    if (empty($data['data'])) {
    return false; // no data sent
    }
    $data = $data['data'];
    $source = $data['source'];
    $style = empty($data['style']) ? null : $data['style'];
    $structure = $this->getStructure($source, $style);
    $allowedTopLevelFields = $this->getOtherValidFields();
    $ignoredDataFields = $this->getIgnoredFields();
    // source
    if (!empty($structure['source_data'])) {
    if (empty($data[$source])) {
    $this->errors[] = 'needed source data missing';
    return false;
    }
    $notAllowed = array_diff_key($data[$source], array_flip($structure['source_data']));
    // +300 lines of code more

    View full-size slide

  7. Many lines of complex code

    View full-size slide

  8. PHPMD warnings /!\
    /**
    * @param array $data
    *
    * @Todo Fix suppressed warnings
    * @SuppressWarnings(
    * PHPMD.CyclomaticComplexity,
    * PHPMD.NPathComplexity
    * )
    * @return bool
    */
    protected function isValidUnformatted(array $data) { //...

    View full-size slide

  9. Validation only on server side
    API DB
    JSON

    View full-size slide

  10. Other technologies have solved this

    View full-size slide

  11. DTD for SGML
    ”http://www.w3.org/TR/html4/strict.dtd”>

    ”id ID #IMPLIED -- document-wide unique id --
    class CDATA #IMPLIED -- space-separated list of classes --
    style %StyleSheet; #IMPLIED -- associated style info --
    title %Text; #IMPLIED -- advisory title --”
    >

    View full-size slide

  12. XSD for XML (XHTML)












    View full-size slide

  13. Short Story
    API DB
    JSON

    View full-size slide

  14. JSON Schema
    → http://json-schema.org/

    View full-size slide

  15. Hello World!
    42
    ”42”
    {}
    true

    View full-size slide

  16. JSON Schema specification
    Validation
    Core

    View full-size slide

  17. JSON Schema specification
    Validation
    Core

    View full-size slide

  18. type
    {“type”: “string”}
    {“type”: “integer”}
    {“type”: “array”}
    {“type”: “object”}
    {“type”: “boolean”}
    {“type”: “null”}
    {“type”: “number”}

    View full-size slide

  19. string
    ”Hello World!”
    ”42”
    {”type”: ”string”}
    42

    View full-size slide

  20. ”php”
    string length
    {
    ”type”: ”string”,
    ”minLength”: 2,
    ”maxLength”: 3
    }
    ”i php”
    ”p”

    View full-size slide

  21. string pattern
    {
    ”type”: ”string”,
    ”pattern”: ”^and.+$”
    }
    ”iphone”
    ”android”

    View full-size slide

  22. integer
    -1
    42
    0.5
    “42”
    {
    ”type”: ”integer”
    }

    View full-size slide

  23. number
    -1
    42
    0.5
    2.99792458e8
    {
    ”type”: ”number”
    }

    View full-size slide

  24. number range
    10
    11
    40
    {
    ”type”: ”number”,
    “minimum”: 10,
    “maximum”: 40,
    “exclusiveMinimum”: true,
    “exclusiveMaximum”: false
    }

    View full-size slide

  25. array
    [1, 2, 3, 4, 5]
    [1, “2”, {“3”:”4 and 5”}, [42]]
    {”type”: ”array”}
    {“this”: “is an object”}

    View full-size slide

  26. array items
    []
    [1, 2, 3, 4, 5]
    [1, 2, 3, 4, “5”]
    {
    ”type”: ”array”,
    “items”: {“type”: “number”}
    }

    View full-size slide

  27. array length
    []
    [1]
    [1, 2]
    {
    ”type”: ”array”,
    “minItems”: 1,
    “maxItems”: 2
    }
    [1, 2, 3]

    View full-size slide

  28. object
    {
    ”key”: ”value”,
    ”anotherKey”: ”anotherValue”
    }
    {”type”: ”object”}
    ”42”
    [“hello”, “world”]

    View full-size slide

  29. {”author”: ”A. A. Milne”, ”year”: 1882}
    properties
    { ”type”: ”object”,
    ”properties”: {
    ”author”: {”type”: ”string”},
    ”year”: {”type”: ”number”}
    }
    }
    {”year”: ”1882”}
    {”author”: ”A. A. Milne”}
    {}

    View full-size slide

  30. {”author”: ”A. A Milne”, ”title”: 1882}
    required properties
    { ”type”: ”object”,
    ”properties”: {
    ”author”: {”type”: ”string”},
    ”title”: {”type”: ”string”}},
    ”required”: [”title”]
    }
    {}
    {”author”: ”A. A Milne”, ”title”: ”Winni the Pooh”, ”year”: 1926}
    {”author”: “A. A Milne”}

    View full-size slide

  31. additional properties
    { ”type”: ”object”,
    ”properties”: {
    ”author”: {”type”: ”string”},
    ”title”: {”type”: ”string”}
    },
    ”additionalProperties”: false
    }
    {”title”: []}
    {}
    {”author”: ”A. A. Milne”, ”title”: ”Winni the Pooh”, ”year”: 1926}
    {”author”: ”A. A. Milne”, ”title”: ”Winni the Pooh”}

    View full-size slide

  32. {”a”: ”aa”, “b”: “bb”}
    number of properties
    {
    ”type”: ”object”,
    ”minProperties”: 1,
    ”maxProperties”: 2
    }
    {}
    {”a”: ”aa”}
    {”a”: ”aa”, “b”: “bb”, “c”: “cc”}

    View full-size slide

  33. dependencies

    View full-size slide

  34. {”name”: ”Alan”, “birthday”: “1882-01-18”}
    property dependencies
    { ”type”: ”object”,
    ”properties”: {“name”: {“type”: “string”},
    “birthday”: {“type”: “string”}},
    ”dependencies”: {
    “birthday”: [“name”]}
    }
    {”name”: ”Alan”}
    {“birthday”: “1882-01-18”}

    View full-size slide

  35. {”name”: ”Alan”, “birthday”: “1882-01-18”}
    schema dependencies
    { ”type”: ”object”,
    ”properties”: { “birthday”: {“type”: “string”}},
    ”dependencies”: {
    “birthday”: {
    “properties”: {“name”: {“type”: “string”}}}}
    }
    {”name”: ”Alan”}
    {“birthday”: “1882-01-18”}

    View full-size slide

  36. {”PHPUnit”: ”testing”, “PHPCS”: “styling”}
    pattern properties
    { ”type”: ”object”,
    ”patternProperties”: {
    “^PHP”: {“type”: “string”}},
    “additionalProperties”: {“type”: “integer”}
    }
    {”BeHat”: ”accepting”}
    {“PHPLOC”: 9001}
    {“LOC”: 9001}

    View full-size slide

  37. boolean
    true
    false
    {”type”: ”boolean”}
    “true”
    0

    View full-size slide

  38. null
    null
    false
    {”type”: ”null”}
    “”
    0

    View full-size slide

  39. More features

    View full-size slide

  40. “one”
    “eleven”
    1
    -11
    Combining with oneOf, anyOf, allOf
    {
    ”anyOf”: [
    {“type”: “string”, “maxLength”: 5},
    {“type”: “number”, “minimum”: 0}
    ]
    }

    View full-size slide

  41. not
    “42”
    {“key”: 42}
    42
    {
    ”not”: {“type”: “number”}
    }

    View full-size slide

  42. available formats: email, date-time, hostname, ipv4, ipv6, uri
    [email protected]
    ”ab@mail.”
    built-in formats
    {
    ”format”: ”email”
    }

    View full-size slide

  43. JSON Schema specification
    Validation
    Core

    View full-size slide

  44. title, description & default
    {
    ”title”: ”Expressive title”,
    ”description”: “Take time to explain”,
    ”default”: “Validators ignore this keyword”
    }

    View full-size slide

  45. $schema
    {
    “$schema”: “http://json-schema.org/draft-04/schema#”,
    ”title”: ”Expressive title”,
    ”description”: “Take time to explain”,
    ”default”: “Validators ignore this keyword”
    }

    View full-size slide

  46. definitions
    {
    “$schema”: “http://json-schema.org/draft-04/schema#”,
    “definitions”: {
    "address": {
    "type": "object",
    "properties": {
    "street_address": { "type": "string" },
    "city": { "type": "string" },
    "state": { "type": "string" }
    },
    "required": ["street_address", "city", "state"]
    }
    }
    }

    View full-size slide

  47. $ref & definitions
    {
    “$schema”: “http://json-schema.org/draft-04/schema#”,
    “definitions”: {
    "address": {
    "type": "object",
    "properties": {
    "street_address": { "type": "string" },
    "city": { "type": "string" },
    "state": { "type": "string" }
    },
    "required": ["street_address", "city", "state"]
    }
    }
    "type": "object",
    "properties": {
    "billing_address": { "$ref": "#/definitions/address" },
    "shipping_address": { "$ref": "#/definitions/address" }
    }
    }

    View full-size slide

  48. $ref & definitions
    {
    “$schema”: “http://json-schema.org/draft-04/schema#”,
    “description”: “This schema defines an address”,
    "type": "object",
    "properties": {
    "city": { "type": "string" },
    "state": { "type": "string" }
    },
    "required": ["street_address", "city", "state"]
    }
    {
    “$schema”: “http://json-schema.org/draft-04/schema#”,
    “description”: “This schema describes an order”,
    "type": "object",
    "properties": {
    "billing_address": { "$ref": "address.json#" },
    "shipping_address": { "$ref": "address.json#" }
    }
    }
    address.json
    order.json

    View full-size slide

  49. Validation Libraries

    View full-size slide

  50. Validation Libraries
    geraintluff/jsv4-php
    11 commits, 1 contributor
    hasbridge/php-json-schema
    28 commits, 2 contributors, not yet feature complete
    justinrainbow/json-schema
    202 commits, 21 contributors

    View full-size slide

  51. Integration of schema validation

    View full-size slide

  52. Preparing validation with json-schema
    // Get the schema as object
    $retriever = new JsonSchema\Uri\UriRetriever;
    $schema = $retriever->retrieve('file://' . realpath('schema.json'));
    // Resolve references
    $refResolver = new JsonSchema\RefResolver($retriever);
    $refResolver->resolve($schema, 'file://' . __DIR__);

    View full-size slide

  53. Validation with json-schema
    /**
    * Validate json against schema
    *
    * @param string $data
    * @param string $schema
    *
    * @return bool
    */
    public function isValid($data, $schema)
    {
    $data = json_decode($data);
    $this->validator->check($data, $schema);
    $this->errors = $this->validator->getErrors();
    return $this->validator->isValid();
    }

    View full-size slide

  54. The benefits of using JSON Schema

    View full-size slide

  55. Less code
    /**
    * Validate json against schema
    *
    * @param string $data
    * @param string $schema
    *
    * @return bool
    */
    public function isValid($data, $schema)
    {
    $data = json_decode($data);
    $this->validator->check($data, $schema);
    $this->errors = $this->validator->getErrors();
    return $this->validator->isValid();
    }

    View full-size slide

  56. No custom validation

    View full-size slide

  57. Complexity
    Complexity of validation and
    document structure
    is independent

    View full-size slide

  58. Validation on client and server side
    API DB
    JSON

    View full-size slide

  59. Implementation in many languages
    PHP, JavaScript, Java,
    Python, Ruby, Perl, .NET,
    C, Haskell, Erlang, Go

    View full-size slide

  60. Better user experience
    Clear, human- and machine-readable
    documentation of our data structure

    View full-size slide

  61. Thank You!
    ● http://json-schema.org/
    ● http://tools.ietf.org/html/draft-zyp-json-schema-04
    ● https://en.wikipedia.org/wiki/JSON
    ● http://spacetelescope.github.io/understanding-json-schema/
    ● https://packagist.org/packages/justinrainbow/json-schema
    ● http://getcomposer.org

    View full-size slide