What I learned trying to make Symfony and API Platform 50% faster

Working on a big symfony project with a quite common stack (API with APIP, lots of doctrine entities and all config in YAML files), we experienced performances issues in both dev and prod mode.

I took my headband lamp (hi blackfire) and dug. I learned A LOT and discovered quite a few issues that results to a lot of pull requests that got merged.

This talk will be a feedback of the process used and the lessons learned. It will be both practical and theoric.

Transcript

1. What I learned trying to
   make Symfony and API
   Platform 50% faster
   SymfonyWorld Online 2020 – Bastien Jaillot – @bastnic
   Architect @JoliCode
   

   View Slide

2. Such a loss of performance that
   the Developer eXperience (DX)
   has become very unpleasant.
   

   View Slide

3. Normal! (?)
   - Docker on Mac
   - API Platform
   - a lot of Doctrine entities
   - autowire / autoconfigure
   

   View Slide

4. Stop blaming yourself
   and REALLY analyze what's going
   on ...
   

   View Slide

5. Performance nice again!
   - Use of Symfony binary on mac envs
   - APIs mocks
   - There's more than just our Symfony application, itself well optimized on paper.
   So we're going to try to optimize what's under the hood :
   Symfony, API Platform, Doctrine, bundles...
   15 Pull Requests merged and 2 blog posts published… (and 2 talks )
   .
   

   View Slide

6. View Slide

7. Methodology
   

   View Slide

8. Fabien Potencier, https://symfony.com/book
   

   View Slide

9. Gives a complete and detailed view of your application and what actually happens while
   running:
   - views "call graph" and "timeline"
   - perfect and easy integration with docker compose
   - profiling from CLI and FPM
   - a "player" for lazy people to automate everything
   Disclaimer: for the purpose of the demo of this talk, Blackfire gave me
   a temporary entreprise licence
   

   View Slide

10. Other tools
    - echo
    - var_dump
    - dump
    - dd
    - xdebug
    - … tracing tools (tideways, etc.)
    

    View Slide

11. View Slide

12. Baseline prod
    BLACKFIRE_EXTERNAL_ID=init-prod \
    php blackfire-player.phar run .blackfire.bkf \
    --blackfire-env="bjaillot-laptop"
    Only one command run:
    - env prod, warmup done
    - multiple routes tested
    - direct access to blackfire profiles
    /!\ project less critical than the one that
    started this
    

    View Slide

13. Futures analysis
    BLACKFIRE_EXTERNAL_ID= \
    BLACKFIRE_EXTERNAL_PARENT_ID= init-prod \
    php blackfire-player.phar run .blackfire.bkf \
    --blackfire-env="bjaillot-laptop"
    Give directly the comparison profiles for
    each of my routes against the baseline.
    

    View Slide

14. But really?
    - update everything
    - profile
    - read the code in vendors/
    - if we have an idea for a patch, we modify the files in vendors
    - otherwise, you write a ticket to talk about your concern.
    - profile again
    - is performance improving or not? If not, back to step 3
    - pull the project, patch, submit a PR
    

    View Slide

15. Performance? What is costly?
    1. useless executions, the fastest code is the one that is not written, or at least
    does not run. Hints:
    a. abnormal cardinalities
    b. call to unused services
    c. IOs, even with optimized PHP at best, a network call will always be
    slow
    2. The execution itself
    

    View Slide

16. How is a query broken down in the
    PROD environment?
    

    View Slide

17. (without warmup)
    (with warmup)
    

    View Slide

18. View Slide

19. View Slide

20. Make normalizers “cacheable”
    https://symfony.com/blog/new-in-symfony-4-1-faster-serializer
    

    View Slide

21. View Slide

22. Symfony#35252
    Use of isset on a nullable value. isset always
    returns false in that case, causes the computation.
    Remplacement par array_key_exists et voilà!
    What makes you think that? The cardinality of the
    "heavy" code called, I didn't have thousands of
    names to convert, so the cache wasn't efficient.
    

    View Slide

23. Symfony#35079
    Optimized cached generated by warmup was not
    memoized, so the call to cache backend happened
    all the time.
    On a large volume of data,
    it's a 10% gain free of charge...
    What makes me think that? Cardinality + the
    number of calls to apcu_fetch.
    

    View Slide

24. ApiPlatform#3317
    Another cardinality that seems off limit. It reveals
    that something must be cached.
    //
    

    View Slide

25. Main path dependencies: LAZY
    Some classes on the critical path deserve your
    attention, at the very least:
    - EventListener / Subscriber
    - Normalizers
    Because they depend on the content of the
    request, and not something we can build upfront:
    they need to be booted to know if they are needed
    for the current req. If they have dependencies,
    they will be loaded.
    This slows down ALL requests.
    

    View Slide

26. Prod: results
    * on an existing application chosen for its total lack of major interest.
    API entities list API single entity
    Baseline 161ms 121ms
    Final 119ms 98ms
    Variations -26% -19%
    

    View Slide

27. View Slide

28. /!\ REMINDER /!\
    in prod, do not forget
    to PRELOAD
    (Symfony provides the preload script to include)
    *PHP 7.4 compatible only
    

    View Slide

29. How is a query broken down in the
    DEV environment?
    

    View Slide

30. View Slide

31. Conditions that trigger a rebuild
    - any modification on Kernel.php, composer dependencies, config files, or
    CompilerPass classes
    - modification on class signature, phpdoc, public / protected properties or method
    signature
    - add / remove file in a folder monitored by autoconfig / autowire
    - and many other specificities (routes, translations, templates, etc.)
    

    View Slide

32. Dev: where are we?
    * on an existing application chosen for its total lack of major interest.
    API entities list API single entity
    Baseline 838ms 839ms
    Final optim prod 822ms 826ms
    Variations almost nothing :( almost nothing :(
    

    View Slide

33. View Slide

34. View Slide

35. View Slide

36. View Slide

37. doctrine/annotations#301
    Doctrine has its own file watcher, based on
    filemtime. But for each file, it will look for
    all its interfaces, parent classes, traits, and
    retrieve their last modification date.
    I use a lot of traits and interfaces,
    which makes it costly.
    We add a static cache.
    

    View Slide

38. symfony/recipes#405
    The Kernel generated by default at the time was very suboptimal. It indicated a lot of possible
    folders / files, multiplying the paths to check at each request.
    Check your Kernel.php
    

    View Slide

39. The default Kernel used to browse too many folders, in GLOB and on several extensions
    (php,xml,yaml,yml), now it only looks for a few files, and only in YAML.
    As a result : use of highly optimized code, and on less files / folders.
    

    View Slide

40. Symfony#35009
    

    View Slide

41. https://github.com/symfony/symfony/commit/d7a06790119b04e4f3e96f36dbf68e70ec5964ae
    

    View Slide

42. https://github.com/symfony/symfony/commit/d7a06790119b04e4f3e96f36dbf68e70ec5964ae
    ¯\_(ツ)_/¯
    

    View Slide

43. View Slide

44. Symfony#35109
    In dev, the Yaml/XML validator and serialization are
    read at runtime.
    While if they are modified, a build is forced anyway.
    A warmer cache phase is added.
    This is a tip seen on Api Platform and backported in
    Symfony. Thanks Teoh
    

    View Slide

45. Doctrine-bundle#1115
    Doctrine Bundle validates the schema of your entities at
    each request. My schema has been frozen for month
    and there is a CI task to check it anyway.
    I added an option to disable it,
    thus avoiding the loading of unnecessary metadata.
    

    View Slide

46. Dev: results
    WITHOUT the Alpine modification, which alone accounts for 60%, which would have made
    the baseline 2s
    API entities list API single entity
    Baseline 838ms 839ms
    Post-prod 822ms 826ms
    Final optim dev 482ms 427ms
    Variations -42% -49%
    

    View Slide

47. Watchers are expensive
    But they allow autowire /
    autoconfigure and simplified
    configuration that I'm not willing
    to sacrifice!
    

    View Slide

48. So you can have a much more pleasant performance in dev
    while having a top notch Dx!
    (optimized)
    (optimized, and
    can be disabled)
    (optimized) (optimized)
    

    View Slide

49. Strangebuzz/cache-warmer
    https://www.strangebuzz.com/en/blog/introducing-cw-a-cache-watcher-for-symfony
    Principle: watcher in go which relaunches the rebuild of the kernel in the background.
    Double gains:
    - you are not waiting for the cache rebuild when calling the API
    - we could remove the watchers since they are no longer useful.
    PS : also achievable from PHPStorm, gulp and co.
    

    View Slide

50. So you can have production performance in dev
    while having a top notch Dx!
    (optimized)
    (optimized, and
    can be disabled)
    (optimized)
    

    View Slide

51. Takeaways
    

    View Slide

52. We no longer hesitate to read
    code in vendor/
    (the blackfire trace can help you know where to look)
    

    View Slide

53. Rubber ducks
    

    View Slide

54. View Slide

55. Ticket / Pull request / Patience
    - Having a merged PR in large projects is complicated and requires a significant
    investment.
    - you have to be patient: your case is not the only one, and it has to be suitable for
    everyone, many of which you don't even know.
    - quality++ / test++ / documentation: nothing should be left to chance
    - performance is a feature, so could be released in a long time
    

    View Slide

56. Performance is addictive
    ● the time saved by my team on this project thanks to the performance gains in the
    development environment is equivalent to several hundred years of blackfire license.
    ● on an open source project, everyone benefits from everyone's gains
    ● these advices can be applied to ALL projects (I would do a pass on Sylius, Drupal...)
    ● I can help you, ping me by Mail / Github / Slack / Twitter / Postcard / Carrier Pigeon
    

    View Slide

57. Just Do It!
    

    View Slide

58. Keep your stack up to date!
    

    View Slide

59. Thanks.
    SymfonyWorld Online 2020 – Bastien Jaillot – @bastnic
    

    View Slide