SymfonyLive San Francisco 2015 - Blackfire.io: a Symfony project like no other

Blackfire.io is a Symfony project. But it's only the tip of the iceberg. By its very nature,

Transcript

1. Blackfire.io 

   A Symfony project like no
   other
   Tugdual Saunier
   http://www.jyotiumbrellas.com/wp-content/uploads/2013/06/Red-umbrella.jpg
   

   View Slide

2. blackfire.io @blackfireio #blackfireio
   2
   Tugdual Saunier
   @tucksaun
   Shipping guy.

   Loves Dev, Ops and
   automation.
   London, UK
   Blackfire.io/SensioLabs
   

   View Slide

3. Blackfire?
   

   View Slide

4. blackfire.io @blackfireio #blackfireio
   4
   SaaS-delivered Solution
   Great user experience
   No need to change your code
   Profile comparisons
   Works in dev & prod (no overhead)
   Works for the Web and the CLI
   Freemium for individual developers
   Automation
   

   View Slide

5. Storage
   Server
   Probe
   Agent
   6. Aggregated profile
   1. Get signature
   2. Trigger 5. Send profile
   PHP/HHVM 7. Store profile
   Browser
   3. Check signature
   Companion 4. Collect profile
   App Container
   

   View Slide

6. blackfire.io @blackfireio #blackfireio
   In a year, we rolled out
   220 prod releases of the website
   240 releases for the Companion
   260 releases for the Probe
   350 releases for the Agent
   in a fully automated way
   Numbers
   6
   

   View Slide

7. Challenge #
   We knew we would
   face a lot of Challenges
   1
   

   View Slide

8. Challenge #
   We knew we would
   face a lot of Challenges
   1
   Solution: solve Challenge #2
   

   View Slide

9. Challenge #
   Symfony at Scale
   2
   

   View Slide

10. blackfire.io @blackfireio #blackfireio
    No need to worry about them in urgency,
    Gives us more capacity to deal with other challenges,
    We knew we would face more challenges,
    Handle Symfony challenges before issues arise
    9
    

    View Slide

11. blackfire.io @blackfireio #blackfireio
    Move heavy workload to background workers
    Move assets building to a Jenkins job publishing on S3
    Amazon RDS / ElastiCache
    Reusing recipes working on other projects
    10
    

    View Slide

12. blackfire.io @blackfireio #blackfireio
    Deployment automated very early
    Monitoring, measuring and graphing everything,
    Pushing all the logs into EHK
    Sending all errors to Sentry (and JS too!)
    Reusing recipes working on other projects
    11
    

    View Slide

13. blackfire.io @blackfireio #blackfireio
    Not hesitating to change the model :
    136 migrations so far;

    No table in common with first day model;

    User class is probably the only one who has not been changed since the
    public announcement.
    Not afraid to denormalize model for reads
    Allowed us to easily move to ES for reads, 

    Allowed us to remove the complex models burden of frontend.
    Reusing recipes working on other projects
    12
    

    View Slide

14. Challenge #
    Blackfire Stack
    3
    

    View Slide

15. blackfire.io @blackfireio #blackfireio
    Symfony :

    Only the tip of the iceberg
    http://alpha.wallhaven.cc/wallpaper/169813
    14
    

    View Slide

16. blackfire.io @blackfireio #blackfireio
    PHP
    JavaScript
    Golang
    C
    And a lot of glue in several languages (Bash, Ruby, …)
    Languages used by Blackfire stack
    15
    

    View Slide

17. blackfire.io @blackfireio #blackfireio
    http://2.bp.blogspot.com/-yOUb4YqkIHc/VLUVU1gL3sI/AAAAAAAADXE/TDduW2hzsCk/s1600/Fotolia_64365698_Subscription_XXL.jpg
    16
    My Awesome
    Project
    Regular Symfony
    project stack
    

    View Slide

18. blackfire.io @blackfireio #blackfireio
    http://2.bp.blogspot.com/-yOUb4YqkIHc/VLUVU1gL3sI/AAAAAAAADXE/TDduW2hzsCk/s1600/Fotolia_64365698_Subscription_XXL.jpg
    17
    Blackfire.io
    stack
    

    View Slide

19. Challenge #
    Discover Releasing
    4
    

    View Slide

20. blackfire.io @blackfireio #blackfireio
    Compiling
    Packaging binaries and scripts
    Pushing package somewhere
    Updating release channels (Webstore, repositories, you own
    doc, …)
    Releasing
    19
    For every platform
    

    View Slide

21. blackfire.io @blackfireio #blackfireio
    Companion
    https://static.pexels.com/photos/9002/night.jpg
    20
    

    View Slide

22. blackfire.io @blackfireio #blackfireio
    Binary packaging
    https://www.flickr.com/photos/oli4/16072036064/
    21
    

    View Slide

23. blackfire.io @blackfireio #blackfireio
    PHP is our friend
    http://altion-days.blogspot.co.uk/2014/06/photos-portraits-of-zoo-animals.html
    22
    

    View Slide

24. blackfire.io @blackfireio #blackfireio
    Required to stick to old
    distributions
    https://www.flickr.com/photos/garibaldi/463424726/
    23
    

    View Slide

25. blackfire.io @blackfireio #blackfireio
    Even older
    https://www.flickr.com/photos/misterbzzz/16510261441/
    24
    

    View Slide

26. blackfire.io @blackfireio #blackfireio
    25
    #ifdef&__linux!
    #ifndef&__GLIBC_COMPAT_SYMBOL_H__!
    #define&__GLIBC_COMPAT_SYMBOL_H__&1!
    !!!#ifdef&__amd64__!
    !!!!!!#define&GLIBC_COMPAT_SYMBOL(FFF)&__asm__(".symver&"FFF&","FFF&"@GLIBC_2.2.5");!
    !!!#else!
    !!!!!!#define&GLIBC_COMPAT_SYMBOL(FFF)&__asm__(".symver&"FFF&","FFF&"@GLIBC_2.0");!
    !!!#endif&/*__amd64__*/!
    #endif&/*__GLIBC_COMPAT_SYMBOL_H__*/!
    #ifndef&__SL_LIB_C_COMPAT_LAYER__!
    #define&__SL_LIB_C_COMPAT_LAYER__&1!
    !!!!/*!
    66666*6List6here6ALL6libc6symbol6that6should6be6downgraded!
    66666*6To6find6linked6libc6versions,6make6and6use:6*/!
    !!!!!!!!//66objdump6Mx6.libs/*.{o,so}6**\/.libs/*.o6|6grep6"GLIBC"!
    !!!!GLIBC_COMPAT_SYMBOL(memcpy);!
    #endif&/*__SP_LIB_C_COMPAT_LAYER__*/!
    #endif&/*__linux*/
    

    View Slide

27. blackfire.io @blackfireio #blackfireio
    Looks scary?
    Hell yeah!
    https://www.flickr.com/photos/al-3x/3783471693/
    26
    

    View Slide

28. Challenge #
    Support a wide diversity
    of users platforms
    5
    

    View Slide

29. blackfire.io @blackfireio #blackfireio
    Docker
    Our newest best friend
    https://www.flickr.com/photos/tony_brown/8057860059/
    28
    

    View Slide

30. Challenge #
    Continuous Releasing
    6
    

    View Slide

31. blackfire.io @blackfireio #blackfireio
    Jenkins
    Our best friend
    http://wallvan.com/child-and-dog-friendship-wallpaper-hd.html
    30
    

    View Slide

32. blackfire.io @blackfireio #blackfireio
    Jenkins
    Our best friend
    http://wallvan.com/child-and-dog-friendship-wallpaper-hd.html
    30
    

    View Slide

33. blackfire.io @blackfireio #blackfireio
    Automation
    https://www.flickr.com/photos/auto-clasico-mallorca/6946157199/
    31
    

    View Slide

34. blackfire.io @blackfireio #blackfireio
    Automation
    https://www.flickr.com/photos/auto-clasico-mallorca/6946157199/
    31
    

    View Slide

35. blackfire.io @blackfireio #blackfireio
    One repository
    Several repositories
    https://www.flickr.com/photos/big-oki/3894408367/
    32
    

    View Slide

36. blackfire.io @blackfireio #blackfireio
    One repository
    Several repositories
    https://www.flickr.com/photos/big-oki/3894408367/
    32
    

    View Slide

37. blackfire.io @blackfireio #blackfireio
    Release on merge into prod
    https://www.flickr.com/photos/pixelgewitter/7753198150/
    33
    

    View Slide

38. Challenge #
    Testing
    8
    

    View Slide

39. blackfire.io @blackfireio #blackfireio
    PHPUnit to test Go and C?
    https://www.flickr.com/photos/10846376@N00/8064965917/
    35
    

    View Slide

40. blackfire.io @blackfireio #blackfireio
    We believe short time to feedback is essential to make a tool
    actually useful to developers.
    At first, only a single do-everything Jenkins build for PR
    To decrease time to feedback we split it
    Which also allows to have the tests running for the whole stack
    whilst it would have previously stop at first fail.
    Reducing time to feedback
    36
    

    View Slide

41. blackfire.io @blackfireio #blackfireio
    We believe short time to feedback is essential to make a tool
    actually useful to developers.
    At first, only a single do-everything Jenkins build for PR
    To decrease time to feedback we split it
    Which also allows to have the tests running for the whole stack
    whilst it would have previously stop at first fail.
    Reducing time to feedback
    36
    

    View Slide

42. blackfire.io @blackfireio #blackfireio
    In a year, we rolled out
    220 prod releases of the website
    240 releases for the Companion
    260 releases for the Probe
    350 releases for the Agent
    in a fully automated way
    Numbers
    37
    

    View Slide

43. blackfire.io @blackfireio #blackfireio
    Continuous Improvement
    Nothing can be done at once
    Constantly improve and refine
    Accept new challenges
    http://www.freewallpaperfullhd.com/space-shuttle-launch-wallpaper-full-hd-2560x1600/
    38
    

    View Slide

44. blackfire.io @blackfireio #blackfireio
    Team spirit
    https://www.flickr.com/photos/devilducmike/12140016406
    39
    

    View Slide

45. Want to know more?
    http:/
    /blog.blackfire.io
    We will release more detailed
    articles (soon)
    

    View Slide

46. Questions ?
    @blackfireio #blackfireio
    

    View Slide

47. blackfire.io @blackfireio #blackfireio
    https://www.flickr.com/photos/nesto/27992968/
    @blackfireio #blackfireio
    

    View Slide