Advanced Serializing in PHP: Speed it up!

Building an API for the biggest retailer of Switzerland has some challenges. We are an API with a lot of extremely complex data. Our bottleneck is and always has been serializing. It has been a long journey for us to get the most performance out of this. We are proud of what we’ve accomplished.

Let me first share what advanced serialization is, and what tools there are out there to solve any needs you may have, we tried them all.

Then I will share our personal journey trying them out. And in the end: A colleague wrote php-to-go so that we could GO beyond PHP, using PHP and GoLang together, for an amazing performance boost.

(Spoiler alert: We found a solution in PHP that we ended up with, and open sourced)

Transcript

1. @michellesanver Advanced serialization in PHP: Speed it up! PHPBenelux 2020
2. @michellesanver WIIIIE \o/ “Learn the most by sharing Your knowledge with others” - @coderabbi
3. @michellesanver Michelle Sanver Colour & Code addict
4. @michellesanver
5. @michellesanver Accent!?!?!?
7. @michellesanver This talk is “open source”
8. @michellesanver Beginner friendly talk Advanced serialization is for everyone!
9. @michellesanver Agenda – Biggest retailer of Switzerland has complex data needs! – Advanced serialization in PHP – Using GoLang to GO beyond PHP and speed it up! – Why most PHP serialization libraries are slow for complex data – How we solved the problem in PHP: A 55% overall performance gain, over using JMS. Including a small demo!
10. @michellesanver Our project, a big retailer with complex data
11. DEV: Rae Knowler DEV: Tobias Schultze DEV: Christian Riesen DEV: Thereza Scherrer DEV: Martin Janser DEV: Emanuele Panzeri DEV: Michelle Sanver “Cloud Tamer”: Chregu PO: Timur Erdag PO: Colin Frei SM: Léo Davesne DEV: David Buchmann Team: 8 developers, 2 PO’s 1 SM, and… Chregu @michellesanver
12. @michellesanver A retailer API Biggest retailer of Switzerland
13. @michellesanver REST Endpoints ElasticSearch MySQL The Data Providers Serializing Importing Mapping Beautiful JSON
14. @michellesanver
15. @michellesanver REST Controllers ElasticSearch MySQL The Data Providers Serializing Importing Mapping Beautiful JSON
16. @michellesanver Huge Technology Stack
17. @michellesanver API: Lots of data “Out of memory exception on 100 products… But I’m not gonna wait for 50”
18. @michellesanver Blackfire Profile your application!
19. @michellesanver
20. @michellesanver
21. @michellesanver Advanced Serialization
22. @michellesanver Simple Serialization One format to another
23. @michellesanver json_encode / json_decode Use safe php! https://github.com/thecodingmachine/safe
24. @michellesanver
25. @michellesanver
26. @michellesanver Where are the tags? Not there, because protected.
27. @michellesanver Dealing with versions Removing fields, renaming, different structure, you name it.
28. @michellesanver
29. @michellesanver
30. @michellesanver Version 1
31. @michellesanver Version 2+
32. @michellesanver Different views (groups) The common “detail” vs “list” views, handled by one set of data!
33. @michellesanver
34. @michellesanver Version 2+ Detail view
35. @michellesanver Version 2+ API view
36. @michellesanver Virtual properties Properties with a punch
37. @michellesanver
38. @michellesanver
39. @michellesanver
40. @michellesanver
41. @michellesanver Version 1+2
42. @michellesanver Version 3+
43. @michellesanver Advanced serialization One set of data, variable output!
44. @michellesanver Still… Advanced Serialization is slow
45. @michellesanver We tried lots of options And everything is great
46. @michellesanver We tried lots of options And everything is great
47. @michellesanver We tried lots of options And everything sucks
48. @michellesanver JMS Serializer I would say it’s a standard?
49. @michellesanver Symfony Serializer It’s great and all… Use it!
50. @michellesanver Better Serializer Maybe it’d be better, if it worked.
51. @michellesanver Use these serializers! Really, they’re great! https://github.com/php-serializers/ivory-serializer-benchmark
52. @michellesanver Let’s GO beyond PHP!
53. @michellesanver Hire someone amazing They make you GO far!
54. @michellesanver Sheriff https://github.com/liip/sheriff An “advanced” serializer in GO
55. @michellesanver https://github.com/liip/sheriff
56. @michellesanver PHP to GO: Generating GO structs from PHP models https://github.com/mweibel/php-to-go
57. @michellesanver #dpc19 <?php $srcGlob = './models/*.php'; $targetDirectory = getenv('GOPATH').'/src/github.com/ mweibel/php-to-go-tests'; $packageName = 'models'; $ignoredFiles = []; $ignoredPropertyNames = []; $verbose = true; $generator = new PHPToGo\CodeGenerator($srcGlob, $targetDirectory, $packageName, $ignoredFiles, $ignoredPropertyNames, $verbose); $generator->generate();
58. @michellesanver https://github.com/liip/sheriff
59. @michellesanver #dpc19 GoRidge: High-performance PHP-to-Golang IPC bridge https://github.com/spiral/goridge
60. @michellesanver #dpc19 <?php use Spiral\Goridge; require "vendor/autoload.php"; $rpc = new Goridge\RPC(new Goridge\SocketRelay("127.0.0.1", 6001)); $product = $rpc->call("App.Serialize", “Products"); https://github.com/spiral/goridge
61. @michellesanver #dpc19 Go + PHP = Crazy love? Sometimes crazy is good ;)
62. @michellesanver But PHP had no fair chance Go was great!
63. @michellesanver Liip Serializer: Speed it up!
64. @michellesanver Model Parser SerializerGenerator DeserializerGenerator
65. @michellesanver Liip Serializer: Demo!
66. @michellesanver Github Repositories Liip Serializer: liip/serializer Meta data parser: liip/metadata-parser JMS Adapter: liip/serializer-jms-adapter Demo code: michellesanver/liip-serializer-demo
67. @michellesanver Final Words
68. @michellesanver Serialization is cool You can do a lot with your data, with an advanced serializer alone
69. @michellesanver You (probably)… Don’t need Liip/Serializer Options out there are fast, performant, and easy to use for 99% of the cases.
70. @michellesanver Monitor performance Find your bottlenecks and fix them!
71. @michellesanver Language limitions... Know your technology and don’t be afraid to GO out of your comfort zone.
72. @michellesanver Cool stuff you do Open source it!
73. @michellesanver Hire someone not like you They will look at the problems with new eyes
74. @michellesanver Dare to experiment Stay curious & Ask questions
75. @michellesanver Different people and views Experiment often Stretch your limitations
76. @michellesanver We can make the impossible, possible Together
77. @michellesanver Thank you By the way, we’re hiring ;) Michelle Sanver michelle@liip.ch https://joind.in/talk/02645