Advanced Serializing in PHP: Speed it up!

Advanced Serializing in PHP: Speed it up!

Building an API for the biggest retailer of Switzerland has some challenges. We are an API with a lot of extremely complex data. Our bottleneck is and always has been serializing. It has been a long journey for us to get the most performance out of this. We are proud of what we’ve accomplished.

Let me first share what advanced serialization is, and what tools there are out there to solve any needs you may have, we tried them all.

Then I will share our personal journey trying them out. And in the end: A colleague wrote php-to-go so that we could GO beyond PHP, using PHP and GoLang together, for an amazing performance boost.

(Spoiler alert: We found a solution in PHP that we ended up with, and open sourced)

01da6d807a29ad6d49801c0157518148?s=128

Michelle Sanver

January 24, 2020
Tweet

Transcript

  1. @michellesanver Advanced serialization in PHP: Speed it up! PHPBenelux 2020

  2. @michellesanver WIIIIE \o/ “Learn the most by sharing Your knowledge

    with others” - @coderabbi
  3. @michellesanver Michelle Sanver Colour & Code addict

  4. @michellesanver

  5. @michellesanver Accent!?!?!?

  6. None
  7. @michellesanver This talk is “open source”

  8. @michellesanver Beginner friendly talk Advanced serialization is for everyone!

  9. @michellesanver Agenda – Biggest retailer of Switzerland has complex data

    needs! – Advanced serialization in PHP – Using GoLang to GO beyond PHP and speed it up! – Why most PHP serialization libraries are slow for complex data – How we solved the problem in PHP: A 55% overall performance gain, over using JMS. Including a small demo!
  10. @michellesanver Our project, a big retailer with complex data

  11. DEV: Rae Knowler DEV: Tobias Schultze DEV: Christian Riesen DEV:

    Thereza Scherrer DEV: Martin Janser DEV: Emanuele Panzeri DEV: Michelle Sanver “Cloud Tamer”: Chregu PO: Timur Erdag PO: Colin Frei SM: Léo Davesne DEV: David Buchmann Team: 8 developers, 2 PO’s 1 SM, and… Chregu @michellesanver
  12. @michellesanver A retailer API Biggest retailer of Switzerland

  13. @michellesanver REST Endpoints ElasticSearch MySQL The Data Providers Serializing Importing

    Mapping Beautiful JSON
  14. @michellesanver

  15. @michellesanver REST Controllers ElasticSearch MySQL The Data Providers Serializing Importing

    Mapping Beautiful JSON
  16. @michellesanver Huge Technology Stack

  17. @michellesanver API: Lots of data “Out of memory exception on

    100 products… But I’m not gonna wait for 50”
  18. @michellesanver Blackfire Profile your application!

  19. @michellesanver

  20. @michellesanver

  21. @michellesanver Advanced Serialization

  22. @michellesanver Simple Serialization One format to another

  23. @michellesanver json_encode / json_decode Use safe php! https://github.com/thecodingmachine/safe

  24. @michellesanver

  25. @michellesanver

  26. @michellesanver Where are the tags? Not there, because protected.

  27. @michellesanver Dealing with versions Removing fields, renaming, different structure, you

    name it.
  28. @michellesanver

  29. @michellesanver

  30. @michellesanver Version 1

  31. @michellesanver Version 2+

  32. @michellesanver Different views (groups) The common “detail” vs “list” views,

    handled by one set of data!
  33. @michellesanver

  34. @michellesanver Version 2+ Detail view

  35. @michellesanver Version 2+ API view

  36. @michellesanver Virtual properties Properties with a punch

  37. @michellesanver

  38. @michellesanver

  39. @michellesanver

  40. @michellesanver

  41. @michellesanver Version 1+2

  42. @michellesanver Version 3+

  43. @michellesanver Advanced serialization One set of data, variable output!

  44. @michellesanver Still… Advanced Serialization is slow

  45. @michellesanver We tried lots of options And everything is great

  46. @michellesanver We tried lots of options And everything is great

  47. @michellesanver We tried lots of options And everything sucks

  48. @michellesanver JMS Serializer I would say it’s a standard?

  49. @michellesanver Symfony Serializer It’s great and all… Use it!

  50. @michellesanver Better Serializer Maybe it’d be better, if it worked.

  51. @michellesanver Use these serializers! Really, they’re great! https://github.com/php-serializers/ivory-serializer-benchmark

  52. @michellesanver Let’s GO beyond PHP!

  53. @michellesanver Hire someone amazing They make you GO far!

  54. @michellesanver Sheriff https://github.com/liip/sheriff An “advanced” serializer in GO

  55. @michellesanver https://github.com/liip/sheriff

  56. @michellesanver PHP to GO: Generating GO structs from PHP models

    https://github.com/mweibel/php-to-go
  57. @michellesanver #dpc19 <?php $srcGlob = './models/*.php'; $targetDirectory = getenv('GOPATH').'/src/github.com/ mweibel/php-to-go-tests';

    $packageName = 'models'; $ignoredFiles = []; $ignoredPropertyNames = []; $verbose = true; $generator = new PHPToGo\CodeGenerator($srcGlob, $targetDirectory, $packageName, $ignoredFiles, $ignoredPropertyNames, $verbose); $generator->generate();
  58. @michellesanver https://github.com/liip/sheriff

  59. @michellesanver #dpc19 GoRidge: High-performance PHP-to-Golang IPC bridge https://github.com/spiral/goridge

  60. @michellesanver #dpc19 <?php use Spiral\Goridge; require "vendor/autoload.php"; $rpc = new

    Goridge\RPC(new Goridge\SocketRelay("127.0.0.1", 6001)); $product = $rpc->call("App.Serialize", “Products"); https://github.com/spiral/goridge
  61. @michellesanver #dpc19 Go + PHP = Crazy love? Sometimes crazy

    is good ;)
  62. @michellesanver But PHP had no fair chance Go was great!

  63. @michellesanver Liip Serializer: Speed it up!

  64. @michellesanver Model Parser SerializerGenerator DeserializerGenerator

  65. @michellesanver Liip Serializer: Demo!

  66. @michellesanver Github Repositories Liip Serializer: liip/serializer Meta data parser: liip/metadata-parser

    JMS Adapter: liip/serializer-jms-adapter Demo code: michellesanver/liip-serializer-demo
  67. @michellesanver Final Words

  68. @michellesanver Serialization is cool You can do a lot with

    your data, with an advanced serializer alone
  69. @michellesanver You (probably)… Don’t need Liip/Serializer Options out there are

    fast, performant, and easy to use for 99% of the cases.
  70. @michellesanver Monitor performance Find your bottlenecks and fix them!

  71. @michellesanver Language limitions... Know your technology and don’t be afraid

    to GO out of your comfort zone.
  72. @michellesanver Cool stuff you do Open source it!

  73. @michellesanver Hire someone not like you They will look at

    the problems with new eyes
  74. @michellesanver Dare to experiment Stay curious & Ask questions

  75. @michellesanver Different people and views Experiment often Stretch your limitations

  76. @michellesanver We can make the impossible, possible Together

  77. @michellesanver Thank you By the way, we’re hiring ;) Michelle

    Sanver michelle@liip.ch https://joind.in/talk/02645