Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Advanced Serializing in PHP: Speed it up!

Advanced Serializing in PHP: Speed it up!

Building an API for the biggest retailer of Switzerland has some challenges. We are an API with a lot of extremely complex data. Our bottleneck is and always has been serializing. It has been a long journey for us to get the most performance out of this. We are proud of what we’ve accomplished.

Let me first share what advanced serialization is, and what tools there are out there to solve any needs you may have, we tried them all.

Then I will share our personal journey trying them out. And in the end: A colleague wrote php-to-go so that we could GO beyond PHP, using PHP and GoLang together, for an amazing performance boost.

(Spoiler alert: We found a solution in PHP that we ended up with, and open sourced)

Michelle Sanver

January 24, 2020
Tweet

More Decks by Michelle Sanver

Other Decks in Programming

Transcript

  1. @michellesanver
    Advanced
    serialization in PHP:
    Speed it up!
    PHPBenelux 2020

    View full-size slide

  2. @michellesanver
    WIIIIE \o/
    “Learn the most by sharing
    Your knowledge with others”
    - @coderabbi

    View full-size slide

  3. @michellesanver
    Michelle Sanver
    Colour & Code addict

    View full-size slide

  4. @michellesanver

    View full-size slide

  5. @michellesanver
    Accent!?!?!?

    View full-size slide

  6. @michellesanver
    This talk is “open source”

    View full-size slide

  7. @michellesanver
    Beginner friendly talk
    Advanced serialization is for everyone!

    View full-size slide

  8. @michellesanver
    Agenda
    – Biggest retailer of Switzerland has complex data needs!
    – Advanced serialization in PHP
    – Using GoLang to GO beyond PHP and speed it up!
    – Why most PHP serialization libraries are slow for complex data
    – How we solved the problem in PHP: A 55% overall performance
    gain, over using JMS. Including a small demo!

    View full-size slide

  9. @michellesanver
    Our project, a big
    retailer with complex
    data

    View full-size slide

  10. DEV: Rae Knowler DEV: Tobias Schultze DEV: Christian Riesen DEV: Thereza Scherrer DEV: Martin Janser
    DEV: Emanuele Panzeri DEV: Michelle Sanver “Cloud Tamer”: Chregu
    PO: Timur Erdag PO: Colin Frei SM: Léo Davesne
    DEV: David Buchmann
    Team: 8 developers, 2 PO’s 1 SM, and… Chregu @michellesanver

    View full-size slide

  11. @michellesanver
    A retailer API
    Biggest retailer of Switzerland

    View full-size slide

  12. @michellesanver
    REST Endpoints
    ElasticSearch
    MySQL
    The Data Providers
    Serializing
    Importing
    Mapping
    Beautiful
    JSON

    View full-size slide

  13. @michellesanver

    View full-size slide

  14. @michellesanver
    REST Controllers
    ElasticSearch
    MySQL
    The Data Providers
    Serializing
    Importing
    Mapping
    Beautiful
    JSON

    View full-size slide

  15. @michellesanver
    Huge Technology Stack

    View full-size slide

  16. @michellesanver
    API: Lots of data
    “Out of memory exception on 100 products…
    But I’m not gonna wait for 50”

    View full-size slide

  17. @michellesanver
    Blackfire
    Profile your
    application!

    View full-size slide

  18. @michellesanver

    View full-size slide

  19. @michellesanver

    View full-size slide

  20. @michellesanver
    Advanced
    Serialization

    View full-size slide

  21. @michellesanver
    Simple Serialization
    One format to another

    View full-size slide

  22. @michellesanver
    json_encode / json_decode
    Use safe php!
    https://github.com/thecodingmachine/safe

    View full-size slide

  23. @michellesanver

    View full-size slide

  24. @michellesanver

    View full-size slide

  25. @michellesanver
    Where are the tags?
    Not there, because protected.

    View full-size slide

  26. @michellesanver
    Dealing with versions
    Removing fields, renaming, different
    structure, you name it.

    View full-size slide

  27. @michellesanver

    View full-size slide

  28. @michellesanver

    View full-size slide

  29. @michellesanver
    Version 1

    View full-size slide

  30. @michellesanver
    Version 2+

    View full-size slide

  31. @michellesanver
    Different views (groups)
    The common “detail” vs “list” views, handled
    by one set of data!

    View full-size slide

  32. @michellesanver

    View full-size slide

  33. @michellesanver
    Version 2+
    Detail view

    View full-size slide

  34. @michellesanver
    Version 2+
    API view

    View full-size slide

  35. @michellesanver
    Virtual properties
    Properties with a punch

    View full-size slide

  36. @michellesanver

    View full-size slide

  37. @michellesanver

    View full-size slide

  38. @michellesanver

    View full-size slide

  39. @michellesanver

    View full-size slide

  40. @michellesanver
    Version 1+2

    View full-size slide

  41. @michellesanver
    Version 3+

    View full-size slide

  42. @michellesanver
    Advanced serialization
    One set of data, variable output!

    View full-size slide

  43. @michellesanver
    Still…
    Advanced
    Serialization
    is slow

    View full-size slide

  44. @michellesanver
    We tried lots of options
    And everything is great

    View full-size slide

  45. @michellesanver
    We tried lots of options
    And everything is great

    View full-size slide

  46. @michellesanver
    We tried lots of options
    And everything sucks

    View full-size slide

  47. @michellesanver
    JMS Serializer
    I would say it’s a standard?

    View full-size slide

  48. @michellesanver
    Symfony Serializer
    It’s great and all… Use it!

    View full-size slide

  49. @michellesanver
    Better Serializer
    Maybe it’d be better, if it worked.

    View full-size slide

  50. @michellesanver
    Use these serializers!
    Really, they’re great!
    https://github.com/php-serializers/ivory-serializer-benchmark

    View full-size slide

  51. @michellesanver
    Let’s GO beyond
    PHP!

    View full-size slide

  52. @michellesanver
    Hire someone amazing
    They make you GO far!

    View full-size slide

  53. @michellesanver
    Sheriff
    https://github.com/liip/sheriff
    An “advanced” serializer in GO

    View full-size slide

  54. @michellesanver
    https://github.com/liip/sheriff

    View full-size slide

  55. @michellesanver
    PHP to GO: Generating GO structs
    from PHP models
    https://github.com/mweibel/php-to-go

    View full-size slide

  56. @michellesanver #dpc19
    $srcGlob = './models/*.php';
    $targetDirectory = getenv('GOPATH').'/src/github.com/
    mweibel/php-to-go-tests';
    $packageName = 'models';
    $ignoredFiles = [];
    $ignoredPropertyNames = [];
    $verbose = true;
    $generator = new PHPToGo\CodeGenerator($srcGlob,
    $targetDirectory, $packageName, $ignoredFiles,
    $ignoredPropertyNames, $verbose);
    $generator->generate();

    View full-size slide

  57. @michellesanver
    https://github.com/liip/sheriff

    View full-size slide

  58. @michellesanver #dpc19
    GoRidge: High-performance
    PHP-to-Golang IPC bridge
    https://github.com/spiral/goridge

    View full-size slide

  59. @michellesanver #dpc19
    use Spiral\Goridge;
    require "vendor/autoload.php";
    $rpc = new Goridge\RPC(new Goridge\SocketRelay("127.0.0.1", 6001));
    $product = $rpc->call("App.Serialize", “Products");
    https://github.com/spiral/goridge

    View full-size slide

  60. @michellesanver #dpc19
    Go + PHP = Crazy love?
    Sometimes crazy is good ;)

    View full-size slide

  61. @michellesanver
    But PHP had no fair chance
    Go was great!

    View full-size slide

  62. @michellesanver
    Liip Serializer:
    Speed it up!

    View full-size slide

  63. @michellesanver
    Model
    Parser
    SerializerGenerator
    DeserializerGenerator

    View full-size slide

  64. @michellesanver
    Liip Serializer:
    Demo!

    View full-size slide

  65. @michellesanver
    Github Repositories
    Liip Serializer: liip/serializer
    Meta data parser: liip/metadata-parser
    JMS Adapter: liip/serializer-jms-adapter
    Demo code: michellesanver/liip-serializer-demo

    View full-size slide

  66. @michellesanver
    Final Words

    View full-size slide

  67. @michellesanver
    Serialization is cool
    You can do a lot with your data, with an
    advanced serializer alone

    View full-size slide

  68. @michellesanver
    You (probably)…
    Don’t need Liip/Serializer
    Options out there are fast, performant,
    and easy to use for 99% of the cases.

    View full-size slide

  69. @michellesanver
    Monitor performance
    Find your bottlenecks and fix them!

    View full-size slide

  70. @michellesanver
    Language limitions...
    Know your technology and don’t be afraid
    to GO out of your comfort zone.

    View full-size slide

  71. @michellesanver
    Cool stuff you do
    Open source it!

    View full-size slide

  72. @michellesanver
    Hire someone not like you
    They will look at the problems with new
    eyes

    View full-size slide

  73. @michellesanver
    Dare to experiment
    Stay curious & Ask questions

    View full-size slide

  74. @michellesanver
    Different
    people
    and views
    Experiment
    often
    Stretch your
    limitations

    View full-size slide

  75. @michellesanver
    We can make the
    impossible, possible
    Together

    View full-size slide

  76. @michellesanver
    Thank you
    By the way, we’re hiring ;)
    Michelle Sanver
    [email protected]
    https://joind.in/talk/02645

    View full-size slide