Top to Bottom PHP Profiling

PHP applications are becoming more complex than ever before. We've moved away from the simple LAMP stack to having many different layers and pieces that our application interacts with. There are many places where things can slow down or break. We'll discuss the different parts of a PHP application including the browser, php application, database, and the server. We'll show different tools for each part and how to use them. By the end of this presentation you should have a whole arsenal of tools to help you profile & understand your application much better.

Transcript

1. Profiling PHP
   Applications
   Justin Carmony - Lone Star PHP `14
   @JustinCarmony
   

   View Slide

2. About Me
   •Director of Development

   @ Deseret Digital Media
   
   •President of the Utah PHP Usergroup
   
   •Make (and Break) Web Stuff ~10 years
   

   View Slide

3. Experience
   •Lots of work with “Middle-Scale” websites.
   
   •Lucky to Work on Fun Projects
   
   •Lots of Web Service Stuff
   
   •Learned most of this from other really smart people.
   

   View Slide

4. Lets Start With a Story!
   

   View Slide

5. You Work for an
   Awesome Tech Company
   

   View Slide

6. Team Is Working Hard to
   Build New Things!
   

   View Slide

7. You launch your
   awesome product!
   

   View Slide

8. A Few More Features…
   

   View Slide

9. … and next thing you know…
   

   View Slide

10. View Slide

11. Awesome Job
    Team, We Rock!
    

    View Slide

12. We Need !
    Real-Time XYZ Feature!
    ASAP!
    $%!
    

    View Slide

13. View Slide

14. “I fixed it by turning

    off caching…”
    - Dev @ 80th Hour This Week
    

    View Slide

15. “I’m sure this
    will work…”
    

    View Slide

16. View Slide

17. View Slide

18. View Slide

19. “Our servers are melting!”
    

    View Slide

20. View Slide

21. View Slide

22. View Slide

23. View Slide

24. View Slide

25. View Slide

26. Let’s Learn How to Profile
    

    View Slide

27. We’ll Use An Example
    and an awesome one at that...
    

    View Slide

28. View Slide

29. We Have Similar Jobs
    Awesome, huh?
    

    View Slide

30. MythBusters
    • They get a Myth
    
    • Break it down to its different
    parts
    
    • They Conduct a Bunch of Tests
    
    • They Draw Conclusions
    
    • and have a Fun Time
    

    View Slide

31. View Slide

32. Developers
    • We get Myths

    (Hey, this is slow)
    
    • We Break it down to the
    different pieces
    
    • We Conduct a Bunch of Tests.
    
    • We Draw Conclusions

    (and normally make changes)
    
    • and Have a Fun Time...
    

    View Slide

33. View Slide

34. So Lets Get Started!
    php /lets/get/started.php now
    

    View Slide

35. First Off, Some
    PHP Myths
    and there are a lot of them!
    

    View Slide

36. Common Performance 

    Claims (Myths)
    • echo instead of print()
    
    • echo commas, not periods
    
    • don’t use custom functions/
    classes
    
    • pass by reference instead of by
    value
    
    • single quotes, not double
    quotes
    
    • include instead of include_once
    
    • === faster than ==
    
    • for faster than foreach
    Busted
    

    View Slide

37. These
    
    “Performance Tips” 

    Rarely Make a Difference
    

    View Slide

38. Thats Like Debating the Windshield Wipers’s
    
    Effect on this Car’s Performance
    

    View Slide

39. Immaterial
    

    View Slide

40. Intuition Based 

    Optimizations
    i.e. how to not effectively
    
    improve performance
    Phrase Coined by Dave Smith
    
    http://thesmithfam.org/blog/2011/02/
    

    View Slide

41. WRONG
    90% of the time
    Your Intuitions are
    

    View Slide

42. Example:
    Fuel Efficiency Myths
    

    View Slide

43. MythBusters Tested
    • Keeping Your AC Off vs Windows Down
    
    • Idling Better than Stop/Start
    
    • Magical Aerodynamics
    
    • Dirt-Free Filters
    
    • Special Fuel Additives to Slow Burn
    

    View Slide

44. BUSTED
    They Didn’t Make a Difference,
    
    Or Worsened Fuel Efficiency
    All Were
    

    View Slide

45. After Testing, Tory & Grant Used
    
    33% More Fuel While Driving Angry
    Driving
    Angry?
    Tory & Grant used 33% more
    
    fuel while driving angry.
    
    

    View Slide

46. Effective Profiling
    is based on
    Actual Results
    

    View Slide

47. Two
    “Profiling”
    Modes
    

    View Slide

48. Normal Profiling
    Lets Try this Configuration Change
    
    and Measure the Performance Difference
    

    View Slide

49. Emergency Profiling
    Aaaaaaaaaaaaah the Website is Down!!!!
    
    Why is it so slow??? Fix it!!!!!!
    

    View Slide

50. Same Techniques
    Apply to Both
    • Some you’ll want to chose first depending on
    your situation.
    
    • You’ll want to be careful when profiling in
    Production, you can make things worse.
    

    View Slide

51. The Full
    Picture
    

    View Slide

52. Browser
    Web
    Server
    Static Files
    PHP App
    Server
    OS
    Hardware
    Web Services
    & Resources Cache
    Database
    Data Store
    

    View Slide

53. Don’t Panic!
    

    View Slide

54. Goal: Find Bottlenecks
    

    View Slide

55. Tools!
    

    View Slide

56. Browser
    Web
    Server
    Static Files
    PHP App
    Server
    OS
    Hardware
    Web Services
    & Cache
    Database
    Data Store
    

    View Slide

57. Browser Dev Tools
    

    View Slide

58. Google Page Speed
    Web Site & Browser Plugin
    

    View Slide

59. View Slide

60. View Slide

61. HTTP Debugging
    Lower Level Fun Stuff
    

    View Slide

62. WireShark
    Windows / Mac / Linux
    

    View Slide

63. View Slide

64. Charles

    Web Debugging Proxy
    Windows / Mac / Linux
    

    View Slide

65. View Slide

66. Other Tools
    • YSlow

    http://developer.yahoo.com/yslow/
    
    • Pingdom Tools

    http://tools.pingdom.com/
    
    • Windows HTTP Debugger: Fiddler2
    

    View Slide

67. Browser
    Web
    Server
    Static Files
    PHP App
    Server
    OS
    Hardware
    Web Services
    & Cache
    Database
    Data Store
    

    View Slide

68. apachetop
    apachetop -f /var/log/apache/access.log
    

    View Slide

69. View Slide

70. Apachetop works
    with nginx
    

    View Slide

71. Apache’s mod_status
    
    SetHandler server-status
    !
    Order Deny,Allow
    Deny from all
    Allow from 25.131.42.122
    
    !
    # ExtendedStatus On
    

    View Slide

72. View Slide

73. siege
    

    View Slide

74. siege
    • Command Line Tool
    
    • Run Concurrent HTTP Requests
    
    • Runs on Linux & Mac OS
    
    • Windows Users: Run a VM
    
    • Great Way to Test End Result
    

    View Slide

75. • Create txt file with lists of URLs to hit
    
    • Run Command:

    siege -c 10 -r 10 -f urls.txt
    
    • c = concurrent

    r = # of requests

    f = path to URL file
    siege
    

    View Slide

76. View Slide

77. View Slide

78. Other Tools
    • nginx stub status (similar to mod_status)

    http://wiki.nginx.org/HttpStubStatusModule
    
    • Apache Bench & http_load -- CLI
    
    • DDM crawlr

    https://github.com/deseretdigital/crawlr
    
    • JMeter -- Java Desktop App
    

    View Slide

79. Browser
    Web
    Server
    Static Files
    PHP App
    Server
    OS
    Hardware
    Web Services
    & Resources Cache
    Database
    Data Store
    

    View Slide

80. XHProf / XHGui
    My Favorite PHP Profiler
    

    View Slide

81. View Slide

82. XHProf
    • Developed by Facebook
    
    • Works well for both Development 

    & Production* Profiling
    
    • pecl extension
    
    • Decent UI for viewing Results
    http://mirror.facebook.net/facebook/xhprof/doc.html
    * - Use Sampling & Special CPU Flags for Production
    

    View Slide

83. XHGui 1.0
    • Better GUI
    
    • Easy to Setup
    
    • Built In Sampling
    
    • Advanced Configuration
    
    • MySQL Backend
    https://github.com/preinheimer/xhprof
    

    View Slide

84. XHGui 2.0
    • Rewrite of XHGUI 1.0
    
    • MongoDB Backend
    
    • Better UI
    
    • Use this unless Mongo won’t work
    in your setup.
    
    • I recommend using this!
    https://github.com/perftools/xhgui
    

    View Slide

85. Demo Time
    

    View Slide

86. Xdebug Profiler
    • Install Xdebug
    
    • Enable Xdebug Profiling
    
    • Outputs a Cachegrind Log
    
    • Use KCachegrind / WinCachegrind / Webgrind to view
    
    • MacCallGrind for $40
    

    View Slide

87. Enabling Xdebug
    Profiling
    xdebug.profiler_enable=1 

    xdebug.profiler_output_dir=/tmp 

    xdebug.profiler_output_name=cachegrind.out.%p
    Xdebug Profiling Not for Production
    (unless you have a 100TB HDD laying around)
    

    View Slide

88. Webgrind
    

    View Slide

89. Timing Points
    • Most Frameworks have Built-In Profiling / Timing Points
    
    • Most ORMs have them as well
    
    • You can do them yourself
    
    • A must for Database Queries
    

    View Slide

90. Timing Points
    $start = microtime(true); // true to get as float
    
    !
    /* Do your Crazy Code, i.e. query */
    
    !
    $end = microtime(true);
    
    !
    $time = round($end - $start, 4);
    

    View Slide

91. Other Tools
    • Inclued

    http://php.net/manual/en/book.inclued.php
    
    • Memtrack

    http://php.net/manual/en/book.memtrack.php
    

    View Slide

92. Browser
    Web
    Server
    Static Files
    PHP App
    Server
    OS
    Hardware
    Web Services
    & Cache
    Database
    Data Store
    

    View Slide

93. Databases
    • Typically, First Thing to Slow Down
    
    • Things to that will Kill the DB:
    
    • Missing Indexes
    
    • Nested Queries
    
    • Large Joins
    
    • Locked Queries
    

    View Slide

94. Jet Profiler
    • MySQL Profiler
    
    • Free Version (Okay) & Paid (Awesome)
    
    • Not Cheap ($399)
    
    • But Worth It
    
    • Analytics Over Time
    

    View Slide

95. Jet Profiler
    

    View Slide

96. MySQL Commands
    • explain
    
    • show processlist
    
    • show variables
    
    • show status
    

    View Slide

97. mtop
    

    View Slide

98. Browser
    Web
    Server
    Static Files
    PHP App
    Server
    OS
    Hardware
    Web Services
    & Cache
    Database
    Data Store
    

    View Slide

99. vmstat
    command: vmstat 1
    

    View Slide

100. View Slide

101. vmstat
     • swap > 0 means swapping

     Memory Issue
     
     • cpu sys + us really high

     CPU / Code / PHP Problem
     
     • cpu wa > 10

     Disk IO Problem*
     * - Technically could be Network IO as well, but typically
     
     one of the last and more rare bottlenecks to hit
     

     View Slide

102. vnstat
     command: vnstat -l -i eth0
     

     View Slide

103. top
     

     View Slide

104. htop
     

     View Slide

105. strace
     

     View Slide

106. iotop
     

     View Slide

107. Other Tools
     • netstat
     
     • iostat
     
     • mpstat
     
     • pidstat
     
     • (on Debian, install via sysstat package)
     

     View Slide

108. Other Tools
     • grep, awk, sed
     
     • IPs Connected:

     netstat -plan | grep :80 | awk '{print $5}' | sed 's/::ffff://g' | awk -F:
     '{print $1}' | sort | uniq -c | sort -n
     

     View Slide

109. Monitoring
     

     View Slide

110. Graphite & StatsD
     

     View Slide

111. Munin
     http://munin-monitoring.org/
     

     View Slide

112. Wormly
     http://wormly.com/
     

     View Slide

113. Whew, Lots of Tools
     and a lot more out there not in this talk
     
     Find any cool ones, let me know!
     

     View Slide

114. Normal Profiling
     • Start with XHProf/XHGui and FireBug
     
     • Avoid Premature Optimization
     
     • Complicated Change
     
     • Little Reward
     
     • Use siege, or alternative, to simulate load.
     

     View Slide

115. Emergency Profiling
     • Start with OS Level Testing Tools (htop, vmstat, vnstat)
     to check Server Performance
     
     • Determine which Resource(s) are being over utilized
     
     • Finding the bottleneck is key
     

     View Slide

116. Emergency Profiling
     • What Changed?
     
     • Increased Traffic?
     
     • New Feature?
     
     • Something Failed/Down?
     
     • Don’t Panic & Start Wildly Guessing
     

     View Slide

117. Few Final
     Thoughts
     

     View Slide

118. The Better You
     Understand
     The Problem
     The Better You Can
     Fix It
     

     View Slide

119. Don’t
     Put Off Profiling
     Until there is an
     Emergency
     

     View Slide

120. “Throwing Hardware”
     at it typically
     
     Masks the Problem
     

     View Slide

121. Ask Others for Ideas
     !
     aka
     Brainstorming
     

     View Slide

122. Good Luck!
     

     View Slide

123. Checkout Dustin
     Whittle’s Talk
     https://speakerdeck.com/dustinwhittle/performance-testing-crash-course-1
     

     View Slide

124. Questions?
     

     View Slide

125. Thanks!
     Twitter: JustinCarmony
     
     !
     IRC: carmony
     
     #uphpu #salt #phpmentoring
     
     !
     Website:
     
     http://www.justincarmony.com/blog
     
     !
     Email:
     
     justin@justincarmony.com
     

     View Slide