Advanced Serializing in PHP: Speed it up!

Building an API for the biggest retailer of Switzerland has some challenges. We are an API with a lot of extremely complex data. Our bottleneck is and always has been serializing. It has been a long journey for us to get the most performance out of this. We are proud of what we’ve accomplished.

Let me first share what advanced serialization is, and what tools there are out there to solve any needs you may have, we tried them all.

Then I will share our personal journey trying them out. And in the end: A colleague wrote php-to-go so that we could GO beyond PHP, using PHP and GoLang together, for an amazing performance boost.

(Spoiler alert: We found a solution in PHP that we ended up with, and open sourced)

Transcript

1. @michellesanver
   Advanced
   serialization in PHP:
   Speed it up!
   PHPBenelux 2020
   

   View Slide

2. @michellesanver
   WIIIIE \o/
   “Learn the most by sharing
   Your knowledge with others”
   - @coderabbi
   

   View Slide

3. @michellesanver
   Michelle Sanver
   Colour & Code addict
   

   View Slide

4. @michellesanver
   

   View Slide

5. @michellesanver
   Accent!?!?!?
   

   View Slide

6. View Slide

7. @michellesanver
   This talk is “open source”
   

   View Slide

8. @michellesanver
   Beginner friendly talk
   Advanced serialization is for everyone!
   

   View Slide

9. @michellesanver
   Agenda
   – Biggest retailer of Switzerland has complex data needs!
   – Advanced serialization in PHP
   – Using GoLang to GO beyond PHP and speed it up!
   – Why most PHP serialization libraries are slow for complex data
   – How we solved the problem in PHP: A 55% overall performance
   gain, over using JMS. Including a small demo!
   

   View Slide

10. @michellesanver
    Our project, a big
    retailer with complex
    data
    

    View Slide

11. DEV: Rae Knowler DEV: Tobias Schultze DEV: Christian Riesen DEV: Thereza Scherrer DEV: Martin Janser
    DEV: Emanuele Panzeri DEV: Michelle Sanver “Cloud Tamer”: Chregu
    PO: Timur Erdag PO: Colin Frei SM: Léo Davesne
    DEV: David Buchmann
    Team: 8 developers, 2 PO’s 1 SM, and… Chregu @michellesanver
    

    View Slide

12. @michellesanver
    A retailer API
    Biggest retailer of Switzerland
    

    View Slide

13. @michellesanver
    REST Endpoints
    ElasticSearch
    MySQL
    The Data Providers
    Serializing
    Importing
    Mapping
    Beautiful
    JSON
    

    View Slide

14. @michellesanver
    

    View Slide

15. @michellesanver
    REST Controllers
    ElasticSearch
    MySQL
    The Data Providers
    Serializing
    Importing
    Mapping
    Beautiful
    JSON
    

    View Slide

16. @michellesanver
    Huge Technology Stack
    

    View Slide

17. @michellesanver
    API: Lots of data
    “Out of memory exception on 100 products…
    But I’m not gonna wait for 50”
    

    View Slide

18. @michellesanver
    Blackfire
    Profile your
    application!
    

    View Slide

19. @michellesanver
    

    View Slide

20. @michellesanver
    

    View Slide

21. @michellesanver
    Advanced
    Serialization
    

    View Slide

22. @michellesanver
    Simple Serialization
    One format to another
    

    View Slide

23. @michellesanver
    json_encode / json_decode
    Use safe php!
    https://github.com/thecodingmachine/safe
    

    View Slide

24. @michellesanver
    

    View Slide

25. @michellesanver
    

    View Slide

26. @michellesanver
    Where are the tags?
    Not there, because protected.
    

    View Slide

27. @michellesanver
    Dealing with versions
    Removing fields, renaming, different
    structure, you name it.
    

    View Slide

28. @michellesanver
    

    View Slide

29. @michellesanver
    

    View Slide

30. @michellesanver
    Version 1
    

    View Slide

31. @michellesanver
    Version 2+
    

    View Slide

32. @michellesanver
    Different views (groups)
    The common “detail” vs “list” views, handled
    by one set of data!
    

    View Slide

33. @michellesanver
    

    View Slide

34. @michellesanver
    Version 2+
    Detail view
    

    View Slide

35. @michellesanver
    Version 2+
    API view
    

    View Slide

36. @michellesanver
    Virtual properties
    Properties with a punch
    

    View Slide

37. @michellesanver
    

    View Slide

38. @michellesanver
    

    View Slide

39. @michellesanver
    

    View Slide

40. @michellesanver
    

    View Slide

41. @michellesanver
    Version 1+2
    

    View Slide

42. @michellesanver
    Version 3+
    

    View Slide

43. @michellesanver
    Advanced serialization
    One set of data, variable output!
    

    View Slide

44. @michellesanver
    Still…
    Advanced
    Serialization
    is slow
    

    View Slide

45. @michellesanver
    We tried lots of options
    And everything is great
    

    View Slide

46. @michellesanver
    We tried lots of options
    And everything is great
    

    View Slide

47. @michellesanver
    We tried lots of options
    And everything sucks
    

    View Slide

48. @michellesanver
    JMS Serializer
    I would say it’s a standard?
    

    View Slide

49. @michellesanver
    Symfony Serializer
    It’s great and all… Use it!
    

    View Slide

50. @michellesanver
    Better Serializer
    Maybe it’d be better, if it worked.
    

    View Slide

51. @michellesanver
    Use these serializers!
    Really, they’re great!
    https://github.com/php-serializers/ivory-serializer-benchmark
    

    View Slide

52. @michellesanver
    Let’s GO beyond
    PHP!
    

    View Slide

53. @michellesanver
    Hire someone amazing
    They make you GO far!
    

    View Slide

54. @michellesanver
    Sheriff
    https://github.com/liip/sheriff
    An “advanced” serializer in GO
    

    View Slide

55. @michellesanver
    https://github.com/liip/sheriff
    

    View Slide

56. @michellesanver
    PHP to GO: Generating GO structs
    from PHP models
    https://github.com/mweibel/php-to-go
    

    View Slide

57. @michellesanver #dpc19
    $srcGlob = './models/*.php';
    $targetDirectory = getenv('GOPATH').'/src/github.com/
    mweibel/php-to-go-tests';
    $packageName = 'models';
    $ignoredFiles = [];
    $ignoredPropertyNames = [];
    $verbose = true;
    $generator = new PHPToGo\CodeGenerator($srcGlob,
    $targetDirectory, $packageName, $ignoredFiles,
    $ignoredPropertyNames, $verbose);
    $generator->generate();
    

    View Slide

58. @michellesanver
    https://github.com/liip/sheriff
    

    View Slide

59. @michellesanver #dpc19
    GoRidge: High-performance
    PHP-to-Golang IPC bridge
    https://github.com/spiral/goridge
    

    View Slide

60. @michellesanver #dpc19
    use Spiral\Goridge;
    require "vendor/autoload.php";
    $rpc = new Goridge\RPC(new Goridge\SocketRelay("127.0.0.1", 6001));
    $product = $rpc->call("App.Serialize", “Products");
    https://github.com/spiral/goridge
    

    View Slide

61. @michellesanver #dpc19
    Go + PHP = Crazy love?
    Sometimes crazy is good ;)
    

    View Slide

62. @michellesanver
    But PHP had no fair chance
    Go was great!
    

    View Slide

63. @michellesanver
    Liip Serializer:
    Speed it up!
    

    View Slide

64. @michellesanver
    Model
    Parser
    SerializerGenerator
    DeserializerGenerator
    

    View Slide

65. @michellesanver
    Liip Serializer:
    Demo!
    

    View Slide

66. @michellesanver
    Github Repositories
    Liip Serializer: liip/serializer
    Meta data parser: liip/metadata-parser
    JMS Adapter: liip/serializer-jms-adapter
    Demo code: michellesanver/liip-serializer-demo
    

    View Slide

67. @michellesanver
    Final Words
    

    View Slide

68. @michellesanver
    Serialization is cool
    You can do a lot with your data, with an
    advanced serializer alone
    

    View Slide

69. @michellesanver
    You (probably)…
    Don’t need Liip/Serializer
    Options out there are fast, performant,
    and easy to use for 99% of the cases.
    

    View Slide

70. @michellesanver
    Monitor performance
    Find your bottlenecks and fix them!
    

    View Slide

71. @michellesanver
    Language limitions...
    Know your technology and don’t be afraid
    to GO out of your comfort zone.
    

    View Slide

72. @michellesanver
    Cool stuff you do
    Open source it!
    

    View Slide

73. @michellesanver
    Hire someone not like you
    They will look at the problems with new
    eyes
    

    View Slide

74. @michellesanver
    Dare to experiment
    Stay curious & Ask questions
    

    View Slide

75. @michellesanver
    Different
    people
    and views
    Experiment
    often
    Stretch your
    limitations
    

    View Slide

76. @michellesanver
    We can make the
    impossible, possible
    Together
    

    View Slide

77. @michellesanver
    Thank you
    By the way, we’re hiring ;)
    Michelle Sanver
    michelle@liip.ch
    https://joind.in/talk/02645
    

    View Slide