Data (De)Serialization 101

Slide 1

Slide 1 text

SensioLabs Data (De)Serialization 101

Slide 2

Slide 2 text

Hugo Hamon Head of training at SensioLabs France Book author Speaker at Conferences Symfony contributor Travel addict @hhamon

Slide 3

Slide 3 text

Lukas Kahwe Smith Developer & patern at Liip Switzerland Co-RM for PHP 5.3 Co-lead of Symfony CMF / PHPCR Symfony core team member Ultimate frisbee lover @lsmith

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

http://live.symfony.com/

Slide 6

Slide 6 text

SensioLabs Data (De)Serialization??? https://www.flickr.com/photos/zigazou76

Slide 7

Slide 7 text

« Serialization is the process of translating data structures or object state into a format that can be stored and reconstructed later in the same or another computer environment. » -- Wikipedia.

Slide 8

Slide 8 text

Examples: HTTP Messages Data encoding (base 64) XML / SOAP JSON YAML CSV…

Slide 9

Slide 9 text

Most Common Usages: Data Storage (file, memory, database) RESTful APIs SOAP Web Services Distributing Objects (Java RMI) Remote Procedure Call (RPC)

Slide 10

Slide 10 text

Data Serialization with PHP

Slide 11

Slide 11 text

Serializing Data Structures with PHP

Slide 12

Slide 12 text

serialize(18); serialize(12.50); serialize(null); serialize(true); serialize(false); serialize('John Smith'); serialize([ 'a', 'b' ]); serialize(new stdClass());

Slide 13

Slide 13 text

i:18; d:12.5; N; b:1; b:0; s:10:"John Smith"; a:3:{i:0;s:1:"a";i:1;s:1:"b";} O:8:"stdClass":0:{}

Slide 14

Slide 14 text

$a = unserialize('i:18;'); $b = unserialize('d:12.5;'); $c = unserialize('b:1;'); $d = unserialize('b:0;'); $e = unserialize('N:;'); $f = unserialize('s:10:"John Smith";'); $g = unserialize('a:2:{i:0;s:1:"a";i:1;s:1:"b";}'); $h = unserialize('O:8:"stdClass":0:{}');

Slide 15

Slide 15 text

Object (de)Serialization Handling

Slide 16

Slide 16 text

__sleep() __wakeup()

Slide 17

Slide 17 text

namespace Database; class Connection { private $link; private $dsn; private $user; private $pwd; public function __construct($dsn, $username, $password) { $this->dsn = $dsn; $this->user = $username; $this->pwd = $password; } private function connect() { if (!$this->link instanceof \PDO) { $this->link = new \PDO($this->dsn, $this->user, $this->pwd); } } }

Slide 18

Slide 18 text

class Connection { // … public function __sleep() { return [ 'dsn', 'user', 'pwd' ]; } public function __wakeup() { $this->connect(); } }

Slide 19

Slide 19 text

use Database\Connection; $dsn = 'mysql:host=localhost;dbname=test'; $usr = 'root'; $pwd = ''; $db = new Connection($dsn, $usr, $pwd); $db->query('SELECT ...'); $serialized = serialize($db); $db = unserialize($serialized); $db->query('SELECT ...');

Slide 20

Slide 20 text

Serializable Interface

Slide 21

Slide 21 text

class Connection implements \Serializable { public function serialize() { return serialize([ 'dsn' => $this->dsn, 'user' => $this->user, 'password' => $this->pwd, ]); } }

Slide 22

Slide 22 text

class Connection implements \Serializable { public function unserialize($data) { $data = unserialize($data); $this->dsn = $data['dsn']; $this->user = $data['user']; $this->pwd = $data['password']; $this->connect(); } }

Slide 23

Slide 23 text

What about JSON as serialization format?

Slide 24

Slide 24 text

json_encode() json_decode() JsonSerializable

Slide 25

Slide 25 text

class ArrayValue implements JsonSerializable { public function __construct(array $array) { $this->array = $array; } public function jsonSerialize() { return $this->array; } } json_encode(new ArrayValue([1, 2, 3]));

Slide 26

Slide 26 text

Serialization is a very complex task…

Slide 27

Slide 27 text

The Symfony Serializer

Slide 28

Slide 28 text

«The Serializer component is meant to be used to turn objects into a specific format (XML, JSON, YAML, ...) and the other way around. » -- Symfony.com

Slide 29

Slide 29 text

http://symfony.com/doc/current/components/serializer.html

Slide 30

Slide 30 text

class Serializer { final function serialize($data, $format, array $context = []) final function deserialize($data, $type, $format, array $context = []); function normalize($data, $format = null, array $context = []) function denormalize($data, $type, $format = null, array $context = []); function supportsNormalization($data, $format = null); function supportsDenormalization($data, $type, $format = null) final function encode($data, $format, array $context = []); final function decode($data, $format, array $context = []); function supportsEncoding($format); function supportsDecoding($format); } The Serializer Public API

Slide 31

Slide 31 text

use Symfony\Component\Serializer\Serializer; use Symfony\Component\Serializer\Normalizer; use Symfony\Component\Serializer\Encoder; // Setup the normalizers $normalizers[] = new Normalizer\PropertyNormalizer(); // Setup the encoders $encoders[] = new Encoder\JsonEncoder(); $encoders[] = new Encoder\XmlEncoder(); // Setup the serializer $serializer = new Serializer($normalizers, $encoders); // Use the serializer $data = $serializer->serialize($object, 'json'); $object = $serializer->deserialize($data, 'Acme\\User', 'json');

Slide 32

Slide 32 text

Normalizers / Denormalizers Name Goal Property Normalizes public / private properties to an associative array. GetSetMethod Normalizes properties by calling getter, isser & setter methods. Object Normalizes objects with the PropertyAccess component. Custom Normalizes an object by delegating serialization to it. Array Denormalizes array of objects (as of 2.8).

Slide 33

Slide 33 text

Encoders / Decoders Name Goal JsonEncoder Encodes & decodes an array from/to JSON. XmlEncoder Encodes & decodes an array from/to XML. ChainEncoder Chains multiple encoders. ChainDecoder Chains multiple decoders.

Slide 34

Slide 34 text

Serializer Basic Usages

Slide 35

Slide 35 text

class Movie { private $id; private $title; private $slug; private $description; private $duration; private $releaseDate; private $storageKey; }

Slide 36

Slide 36 text

$movie = new Movie(); $movie->setTitle('Seven'); $movie->setSlug('seven'); $movie->setDescription('A brilliant…'); $movie->setDuration(130); $movie->setReleaseDate('1996-01-31'); Serializing an Object

Slide 37

Slide 37 text

$data = $serializer->serialize( $movie, 'json' ); $movie = $serializer->deserialize( $data, 'Movie', 'json' );

Slide 38

Slide 38 text

Properties Serialization

Slide 39

Slide 39 text

{ "id":null, "title":"Seven", "slug":"seven", "description":"A … thriller!", "duration":130, "releaseDate":"1996-01-31", "storageKey":null } JSON Serialization

Slide 40

Slide 40 text

Seven seven A … thriller! 130 1996-01-31 XML Serialization

Slide 41

Slide 41 text

String Deserialization

Slide 42

Slide 42 text

$data = <<deserialize($data, 'Movie', 'json'); print_r($movie); JSON Deserialization

Slide 43

Slide 43 text

$data = << Seven seven A … thriller! 130 1996-01-31 DATA; $movie = $serializer->deserialize($data, 'Movie', 'xml'); print_r($movie); XML Deserialization

Slide 44

Slide 44 text

Movie Object ( [id:Movie:private] => [title:Movie:private] => Seven [slug:Movie:private] => seven [description:Movie:private] => A … thriller! [duration:Movie:private] => 130 [releaseDate:Movie:private] => 1996-01-31 [storageKey:Movie:private] => ) String Deserialization

Slide 45

Slide 45 text

class Movie { // ... function __construct($id = null, $title = null, $slug = null) { $this->id = $id; $this->title = $title; $this->slug = $slug; } } Constructor Initialization Constructor arguments must match properties names.

Slide 46

Slide 46 text

Going Further with the Serializer

Slide 47

Slide 47 text

Getter, Hasser & Isser Methods Normalizer

Slide 48

Slide 48 text

// Setup the normalizers $normalizers[] = new Normalizer\ObjectNormalizer(); $normalizers[] = new Normalizer\GetSetMethodNormalizer(); $normalizers[] = new Normalizer\PropertyNormalizer(); // Setup the encoders $encoders[] = new Encoder\JsonEncoder(); $encoders[] = new Encoder\XmlEncoder(); // Setup the serializer $serializer = new Serializer($normalizers, $encoders); // Use the serializer $serializer->serialize($object, 'json'); $serializer->deserialize($data, 'Acme\User','json'); The object normalizer can invoke « hasser » methods.

Slide 49

Slide 49 text

class Movie { public function getId() { return $this->id; } public function getTitle() { return $this->title; } public function hasGenre() { return false; } // ... public function isReleased() { return new \DateTime($this->releaseDate) <= new \DateTime(); } } The normalizer invokes getter & isser methods.

Slide 50

Slide 50 text

{ "id":null, "title":"Seven", "slug":"seven", "description":"A … thriller!", "duration":130, "releaseDate":"1996-01-31", "storageKey":null, "genre":false, "released":true, } JSON Serialization

Slide 51

Slide 51 text

Seven seven A … thriller! 130 1996-01-31 0 1 XML Serialization

Slide 52

Slide 52 text

Ignoring Attributes

Slide 53

Slide 53 text

$normalizer = new GetSetMethodNormalizer(); $normalizer->setIgnoredAttributes([ 'storageKey' ]); Seven seven A … thriller! 130 1996-01-31 1

Slide 54

Slide 54 text

Converting properties names to underscore case.

Slide 55

Slide 55 text

$converter = new CamelCaseToSnakeCaseNameConverter(); $normalizer = new GetSetMethodNormalizer(null, $converter); Seven seven A … thriller! 130 1996-01-31 1

Slide 56

Slide 56 text

Customizing all serialized properties names.

Slide 57

Slide 57 text

class PrefixNameConverter implements NameConverterInterface { private $prefix; public function __construct($prefix) { $this->prefix = $prefix; } public function normalize($propertyName) { return $this->prefix.'_'.$propertyName; } public function denormalize($propertyName) { if ($this->prefix.'_' === substr($propertyName, 0, count($this->prefix))) { return substr($propertyName, count($this->prefix)); } return $propertyName; } } The NameConverterInterface has been introduced in 2.7.

Slide 58

Slide 58 text

$converter = new PrefixNameConverter('movie'); $normalizer = new GetSetMethodNormalizer(null, $converter); Seven seven A … thriller! 130 1996-01-31 1

Slide 59

Slide 59 text

Changing the XML root name.

Slide 60

Slide 60 text

$serializer->serialize($movie, 'xml', [ 'xml_root_node_name' => 'movie', ]); Seven ...

Slide 61

Slide 61 text

Deserializing into an existing object.

Slide 62

Slide 62 text

$data = << 130 1996-01-31 DATA; $movie1 = new Movie(1234, 'Seven', 'seven'); $movie2 = $serializer->deserialize($data, 'Movie', 'xml', [ 'xml_root_node_name' => 'movie', 'object_to_populate' => $movie1, ]);

Slide 63

Slide 63 text

Movie Object ( [id:Movie:private] => 1234 [title:Movie:private] => Seven [slug:Movie:private] => seven [description:Movie:private] => [duration:Movie:private] => 130 [releaseDate:Movie:private] => 1996-01-31 [storageKey:Movie:private] => [genre:Movie:private] => ) The « description » property remains empty while « duration » and « releaseDate » properties are set.

Slide 64

Slide 64 text

Serializer Advanced Features

Slide 65

Slide 65 text

Serializing More Complex Object Graphs.

Slide 66

Slide 66 text

class Movie { /** @var Genre */ private $genre; /** @var Directors[] */ private $directors; /** * Each role keeps a reference to that Movie object * and a reference to an Actor object playing that * role in the movie. * * @var Role[] */ private $roles; }

Slide 67

Slide 67 text

One to One Unidirectional Relationship

Slide 68

Slide 68 text

$genre = new Genre(42, 'Thriller', 'thriller'); $movie = new Movie(1234, 'Seven', 'seven'); $movie->setGenre($genre); $movie->setStorageKey('movie-42-1234'); $movie->setDuration(130); $movie->setDescription('A brilliant thriller!'); $movie->setReleaseDate('1996-01-31'); echo $serializer->serialize($movie, 'xml', [ 'xml_root_node_name' => 'movie', ]);

Slide 69

Slide 69 text

42 thriller Thriller 1234 Seven 130 1 seven A brilliant thriller! 1996-01-31

Slide 70

Slide 70 text

Slide 71

Slide 71 text

One to Many Unidirectional Relationship

Slide 72

Slide 72 text

$fincher = new Director(); $fincher->setId(973463); $fincher->setName('David Fincher'); $fincher->setBirthday('1962-05-10'); $kopelson = new Director(); $kopelson->setId(783237); $kopelson->setName('Arnold Kopelson'); $kopelson->setBirthday('1935-02-14'); $movie = new Movie(1234, 'Seven', 'seven'); $movie->addDirector($fincher); $movie->addDirector($kopelson);

Slide 73

Slide 73 text

973463 David Fincher 1962-05-10 783237 Arnold Kopelson 1935-02-14

Slide 74

Slide 74 text

{ "genre":{ "id":42, "slug":"thriller", "title":"Thriller" }, "id":1234, "title":"Seven", "duration":130, "released":true, "slug":"seven", "description":"A brilliant thriller!", "release_date":"1996-01-31", "directors":[ { "id":973463, "name":"David Fincher", "birthday":"1962-05-10", "deathday":null }, { "id":783237, "name":"Arnold Kopelson", "birthday":"1935-02-14", "deathday":null } ] }

Slide 75

Slide 75 text

Many to Many Bidirectional Relationship

Slide 76

Slide 76 text

class Role { private $id; private $character; private $movie; private $actor; function __construct($id, Movie $movie, Actor $actor, $character) { $this->id = $id; $this->movie = $movie; $this->actor = $actor; $this->character = $character; } } The « Role » instance keeps a reference to the « Movie » that also keeps references to « roles » played by actors.

Slide 77

Slide 77 text

$movie = new Movie(1234, 'Seven', 'seven'); // ... $pitt = new Actor(); $pitt->setId(328470); $pitt->setName('Brad Pitt'); $pitt->setBirthday('1963-12-18'); $freeman = new Actor(); $freeman->setId(329443); $freeman->setName('Morgan Freeman'); $freeman->setBirthday('1937-06-01'); $mills = new Role(233, $movie, $pitt, 'David Mills'); $sommerset = new Role(328, $movie, $freeman, 'William Sommerset'); $movie->addRole($mills); $movie->addRole($sommerset); $serializer->serialize($movie, 'json');

Slide 78

Slide 78 text

PHP Fatal error: Uncaught exception 'Symfony\Component\Serializer \Exception \CircularReferenceException' with message 'A circular reference has been detected (configured limit: 1).' in /Volumes/Development/Sites/ Serializer/vendor/symfony/serializer/ Normalizer/AbstractNormalizer.php:221

Slide 79

Slide 79 text

Handling Circular References

Slide 80

Slide 80 text

$normalizer = new ObjectNormalizer(null, $converter); $normalizer->setIgnoredAttributes([ 'storageKey' ]); // Return the object unique identifier instead of the // instance to stop a potential infinite serialization loop. $normalizer->setCircularReferenceHandler(function ($object) { return $object->getId(); }); Handling Circular References Circular references support has been introduced in Symfony 2.6.

Slide 81

Slide 81 text

{ ... "roles":[ { "actor":{ "id":328470, "name":"Brad Pitt", "birthday":"1963-12-18", "deathday":null }, "character":"David Mills", "id":233163, "movie":1234 }, ... ] }

Slide 82

Slide 82 text

Using Callback Normalizers.

Slide 83

Slide 83 text

$movie = new Movie(1234, 'Seven', 'seven'); $movie->setReleaseDate(new \DateTime('1996-01-31')); $pitt = new Actor(); $pitt->setBirthday(new \DateTime('1963-12-18')); $fincher = new Director(); $fincher->setBirthday(new \DateTime('1962-05-10')); $serializer->serialize($movie, 'json'); Actors, Directors and Movies now store date representations as « DateTime » objects. These instances must be serialized too.

Slide 84

Slide 84 text

0 0 Europe/Paris FR 48.86666 2.33333 3600 823042800 Without custom serializer to handle « DateTime » instance, the Serializer serializes any date object as follows:

Slide 85

Slide 85 text

$normalizer = new Normalizer\ObjectNormalizer(...); $callback = function ($dateTime) { return $dateTime instanceof \DateTime ? $dateTime->format('Y-m-d') : ''; }; $normalizer->setCallbacks([ 'releaseDate' => $callback, 'birthday' => $callback, 'deathday' => $callback, ]); The built-in normalizers allow to set PHP callbacks to handle custom serialization steps.

Slide 86

Slide 86 text

1996-01-31 973463 David Fincher 1962-05-10 783237 Arnold Kopelson 1935-02-14

Slide 87

Slide 87 text

Slide 88

Slide 88 text

Using the Custom Normalizer.

Slide 89

Slide 89 text

Adding the Custom Normalizer The built-in « Custom » normalizer is responsible for automatically calling the « normalize() » and « denormalize() » methods of your objects if they implement the corresponding interface. $normalizers[] = new Normalizer\CustomNormalizer();

Slide 90

Slide 90 text

use Symfony\Component\Serializer\Normalizer\NormalizableInterface; use Symfony\Component\Serializer\Normalizer\NormalizerInterface; class Role implements NormalizableInterface { private $id; private $character; private $movie; private $actor; function normalize(NormalizerInterface $normalizer, $format = null, array $context = []) { return [ 'id' => $this->id, 'character' => $this->character, 'actor' => $this->actor, ]; } }

Slide 91

Slide 91 text

SerializerAwareInterface

Slide 92

Slide 92 text

class ArrayDenormalizer implements DenormalizerInterface, SerializerAwareInterface { … public function supportsDenormalization($data, $type, $format = null) { return substr($class, -2) === '[]' && $this->serializer ->supportsDenormalization($data, substr($class, 0, -2), $format); } public function denormalize($data, $class, $format = null, array $context = []) { … $serializer = $this->serializer; $class = substr($class, 0, -2); return array_map( function ($data) use ($serializer, $class, $format, $context) { return $serializer->denormalize($data, $class, $format, $context) }, $data ); } }

Slide 93

Slide 93 text

Serialization Groups

Slide 94

Slide 94 text

Annotation Conﬁguration use Symfony\Component\Serializer\Annotation\Groups; class Movie { /** @Groups({"admins"}) */ private $id; /** @Groups({"admins", "publishers", "users" }) */ private $title; /** @Groups({"admins", "publishers" }) */ private $slug; /** @Groups({"admins", "publishers", "users" }) */ private $releaseDate; /** @Groups({ "admins", "publishers", "users" }) */ public function isReleased() { return new $this->releaseDate <= new \DateTime(); } }

Slide 95

Slide 95 text

Movie: attributes: id: groups: [ admins ] title: groups: [ admins, publishers, users ] slug: groups: [ admins, publishers ] releaseDate: groups: [ admins, publishers, users ] released: groups: [ admins, publishers, users ] YAML Conﬁguration

Slide 96

Slide 96 text

admins admins publishers users admins publishers XML Conﬁguration

Slide 97

Slide 97 text

$serializer->serialize($movie, 'xml', [ 'xml_root_node_name' => 'movie', 'groups' => [ 'users' ], ]); $serializer->deserialize($movie, 'Movie', 'xml', [ 'xml_root_node_name' => 'movie', 'groups' => [ 'users' ], ]); Serialization Groups Groups also work when deserializing

Slide 98

Slide 98 text

New Features in Symfony 2.8

Slide 99

Slide 99 text

class Invoice { /* @var Customer */ private $customer; public function setCustomer(Customer $customer) { return $this->customer = $customer; } } $invoiceArray = [ ‘customer’ => [ /* field data of a customer */] ]; $invoice = $normalizer->denormalize($invoiceArray, Invoice::class, null); Denormalizing with Type Hinting See h%ps://github.com/symfony/symfony/pull/14844 Symfony 2.8 integrates h%ps://github.com/dunglas/php-‐property-‐info to be able to read docblocks, typehints, Doctrine mappings etc via one API

Slide 100

Slide 100 text

Going Further with Data Serialization

Slide 101

Slide 101 text

JMS Serializer Library •  Yaml / XML / JSON Serialization •  Advanced Serialization Mapping •  Handle Circular References gracefully •  Advanced Metadata Configuration •  Integrates with Doctrine / Symfony / Zend Framework •  Versioning support •  Extensible at will http://jmsyst.com/libs/serializer

Slide 102

Slide 102 text

use JMS\Serializer\Annotation as Serializer; /** * @Serializer\ExclusionPolicy("ALL") * @Serializer\XmlRoot("movie") */ class Movie { /** * @Serializer\Type("integer") * @Serializer\Expose * @Serializer\XmlAttribute */ private $id; /** * @Serializer\Type("string") * @Serializer\Expose */ private $title; /** * @Serializer\Expose * @Serializer\SerializedName("dateOfRelease") * @Serializer\Type("DateTime<'Y-m-d'>") */ private $releaseDate; } Basic Mapping

Slide 103

Slide 103 text

Accessors and Virtual Properties class Movie { /** * @Serializer\Accessor( * getter="getAverageRating", * setter="setAverageRating" * ) * @Serializer\Expose */ private $rating; /** * @Serializer\VirtualProperty * @Serializer\SerializedName("lastRate") */ public function getLastRatingMark() { return $this->ratings->getLast(); } }

Slide 104

Slide 104 text

Serialization Groups Support class Movie { /** * @Serializer\Groups({"admin", "archives"}) * @Serializer\Expose */ private $internalStorageKey; }

Slide 105

Slide 105 text

Versionning Support class Movie { /** * @Serializer\Until("0.9.10") * @Serializer\SerializedName("synopsis") * @Serializer\Expose */ private $summary; /** * @Serializer\Since("0.9.11") * @Serializer\Expose */ private $synopsis; }

Slide 106

Slide 106 text

Nested Object Graphs Support class Movie { /** * @Serializer\Type("IMDB\Model\Genre") * @Serializer\Expose */ private $genre; /** * @Serializer\Type("ArrayCollection") * @Serializer\Expose */ private $directors; /** * @Serializer\Type("ArrayCollection") * @Serializer\Expose */ private $characters; }

Slide 107

Slide 107 text

HATEOAS Libray • Easy way to serialize actions on entities • Embed hyperlinks in JSON / XML strings • Make a REST API easily browsable • Works perfectly with JMS Serializer • Perfect integration with Symfony • Annotation Metadata Mapping Support https://github.com/willdurand/Hateoas

Slide 108

Slide 108 text

An Example of Usage /** * @Hateoas\Relation("self", href = "expr('/api/movies/' ~ object.getId())") * * @Hateoas\Relation( * name="genre", * href = "expr('/api/genres/' ~ object.getGenre().getId())", * embedded = "expr(object.getGenre())", * exclusion = @Hateoas\Exclusion( * excludeIf = "expr(object.getGenre().getName() ==='pornography')" * ) * ) */ class Movie { // ... }

Slide 109

Slide 109 text

HATEOAS with JSON { "id": 42, "title": "Seven", ... "_links": { "self": { "href": "/api/movies/42" }, "genre": { "href": "/api/genres/21" } }, "_embedded": { "genre": { "id": 21, "name": "Thriller" } } }

Slide 110

Slide 110 text

HATEOAS with XML ...

Slide 111

Slide 111 text

Some Final Thoughts & Advices •  Serializing data is « mostly » easy to achieve! •  Deserializing is not easy at all! •  For very simple use cases, use PHP native tools •  For more advanced cases, use the Symfony Serializer! •  For real advanced (de)serialization, use the JMS Serializer!

Slide 112

Slide 112 text

SensioLabs h%ps://www.ﬂickr.com/photos/31843304@N02/ Thank You for listening! hugo.hamon@sensiolabs.com - @hhamon