Profiling PHP applications

It's nothing new that speed is important for the success of any web application. Only a few hundred milliseconds may lie between a user leaving your site or staying. Unfortunately performance problems are oftentimes hard to fix and even harder to pinpoint. In this talk I will show you how we at ResearchGate measure web application performance, which means not only timing how long the PHP backend took to deliver a page, but also tracking the speed the users actually perceives in the browser. After that you will see how you can track down and analyze any problems you found through measuring with the help of tools like Xdebug, XHProf and the Symfony Debug Toolbar. And if you still need to get faster after optimizing and fixing all these issues, I'll introduce you to some tricks, techniques and patterns to even further decrease load times.

Transcript

1. My site is slow,
   what can I do?
   @BastianHofmann
   Profiling PHP applications
   

   View Slide

2. This talk is all
   about...
   

   View Slide

3. View Slide

4. Speed of your web
   application
   

   View Slide

5. We'll talk about...
   

   View Slide

6. Why it matters
   

   View Slide

7. How to measure it
   

   View Slide

8. How to find out
   where the problems
   are
   

   View Slide

9. Before we start
   

   View Slide

10. A few words
    about me
    

    View Slide

11. View Slide

12. View Slide

13. View Slide

14. View Slide

15. View Slide

16. View Slide

17. Questions? Ask
    

    View Slide

18. http://speakerdeck.com/u/bastianhofmann
    

    View Slide

19. View Slide

20. Why should you
    care?
    

    View Slide

21. It is really important
    

    View Slide

22. View Slide

23. View Slide

24. Every ms counts
    

    View Slide

25. View Slide

26. View Slide

27. View Slide

28. So what is
    pagespeed?
    

    View Slide

29. Server
    

    View Slide

30. Your PHP application
    Request
    Response
    

    View Slide

31. Your PHP application
    Request
    Response
    Load balancer
    

    View Slide

32. Your PHP application
    Request
    Response
    Load balancer
    

    View Slide

33. web server db
    http service
    http service
    cache
    user request
    

    View Slide

34. But there is more
    

    View Slide

35. View Slide

36. View Slide

37. View Slide

38. View Slide

39. http://www.stevesouders.com/blog/2012/02/10/the-performance-golden-rule/
    

    View Slide

40. So what happens
    on your server is
    really important
    

    View Slide

41. But the rest is as
    well
    

    View Slide

42. Step 1
    

    View Slide

43. Measuring
    

    View Slide

44. View Slide

45. View Slide

46. View Slide

47. View Slide

48. View Slide

49. View Slide

50. View Slide

51. So you need to
    measure at your
    users side
    

    View Slide

52. http://www.soasta.com/
    

    View Slide

53. https://www.librato.com
    

    View Slide

54. Navigation Timing
    API
    https://developer.mozilla.org/en-US/docs/
    Navigation_timing
    

    View Slide

55. View Slide

56. Resource Timing API
    http://www.w3.org/TR/resource-timing/
    

    View Slide

57. View Slide

58. Getting it back to
    the server
    

    View Slide

59. Tracking request
    

    View Slide

60. https://tracking.example.com/?
    page=profile&backend=123&co
    mplete=890&domReady=580
    

    View Slide

61. logstash
    http://logstash.net/
    

    View Slide

62. input filter output
    

    View Slide

63. Very rich plugin
    system
    

    View Slide

64. browser
    JS: boomerang
    logstash
    trackingServer
    access log
    requests
    tracking image
    with timing
    information as
    query
    parameters
    

    View Slide

65. Graphing it
    

    View Slide

66. Graphite
    http://graphite.wikidot.com/
    

    View Slide

67. View Slide

68. browser
    JS: boomerang
    logstash
    trackingServer
    access log
    requests
    tracking image
    with timing
    information as
    query
    parameters
    graphite
    statsd
    

    View Slide

69. input {

    file {

    type => "pagespeed-access"

    path => [ "/var/log/nginx/
    access_log/monitoring-access.log" ]

    }

    }
    

    View Slide

70. filter{

    grok {

    type => "pagespeed-access"

    pattern => "^.*\s\"[A-Z]+\s[^\?\s]+
    \?page=%{DATA:page}\&connectTime=%
    {NUMBER:connectTime}...)?\sHTTP\/\d\.\d\".*
    $"

    }

    grok {

    type => "pagespeed-access"

    match => ["page", "^(profile|
    home|...)\.logged(In|Out)$"]

    exclude_tags =>
    ["_grokparsefailure"]

    }

    }
    

    View Slide

71. output {
    statsd {
    type => "pagespeed-access"
    exclude_tags =>
    ["_grokparsefailure"]
    host => "localhost"
    port => 8126
    namespace => "pagespeed"
    sender => ""
    timing => [
    "%{page}.connect", "%
    {connectTime}", ...
    ]
    }
    }
    

    View Slide

72. View Slide

73. View Slide

74. Can we measure
    more?
    

    View Slide

75. From within your
    PHP app
    

    View Slide

76. DB queries
    

    View Slide

77. HTTP calls
    

    View Slide

78. Rendering time
    

    View Slide

79. …
    

    View Slide

80. More fine grained
    metrics
    

    View Slide

81. By pages
    

    View Slide

82. By browser
    

    View Slide

83. By country
    

    View Slide

84. Logged in / out
    

    View Slide

85. Heavy users
    

    View Slide

86. Slow query logs
    

    View Slide

87. Load balancers
    

    View Slide

88. web server db
    http service
    http service
    cache
    user request
    

    View Slide

89. input {

    file {

    type => "haproxy-http-log"

    path => [ "/var/log/
    haproxy-http.log*" ]

    }

    }
    

    View Slide

90. filter {

    grok {

    type => "haproxy-http-log"

    pattern => "%{HAPROXYHTTP}" 

    }

    mutate {

    type => "haproxy-http-log"

    gsub => [

    "server_name", "\.", "_",

    "client_ip", "\.", "_"

    ]

    }

    }
    

    View Slide

91. output {

    statsd {

    type => "haproxy-http-log"

    exclude_tags => ["_grokparsefailure"]

    host => "localhost"

    port => 8125

    namespace => "lb"

    sender => ""

    increment => [

    "haproxy.%{backend_name}.%{server_name}.%
    {client_ip}.hits",

    "haproxy.%{backend_name}.%{server_name}.%
    {client_ip}.responses.%{http_status_code}"

    ]

    timing => [

    "haproxy.%{backend_name}.%{server_name}.%
    {client_ip}.time_request", "%{time_request}",

    "haproxy.%{backend_name}.%{server_name}.%
    {client_ip}.time_backend_connect", "%{time_backend_connect}",

    "haproxy.%{backend_name}.%{server_name}.%
    {client_ip}.time_backend_response", "%{time_backend_response}",

    "haproxy.%{backend_name}.%{server_name}.%
    {client_ip}.time_queue", "%{time_queue}",

    "haproxy.%{backend_name}.%{server_name}.%
    {client_ip}.time_duration", "%{time_duration}"

    ]

    }

    

    View Slide

92. browser
    JS: boomerang
    logstash
    trackingServer
    access log
    requests
    tracking image
    with timing
    information as
    query
    parameters
    graphite
    statsd
    logstash
    load balancer
    access log
    logstash
    service
    access log
    

    View Slide

93. Define goals
    

    View Slide

94. Step 2
    

    View Slide

95. How to find out
    where the problems
    are
    

    View Slide

96. Profiling
    

    View Slide

97. View Slide

98. xdebug.profiler_enable_trigger = 1
    http://url?XDEBUG_PROFILE
    

    View Slide

99. Webgrind
    https://github.com/jokkedk/webgrind
    

    View Slide

100. View Slide

101. Use it locally on
     your dev machine
     

     View Slide

102. XHProf
     https://github.com/facebook/xhprof
     

     View Slide

103. Use it in production
     for a subset of
     requests
     

     View Slide

104. XHGUI
     https://github.com/perftools/xhgui
     

     View Slide

105. View Slide

106. View Slide

107. View Slide

108. View Slide

109. php-meminfo
     https://github.com/BitOne/php-meminfo
     

     View Slide

110. Symfony Debug
     Toolbar
     

     View Slide

111. View Slide

112. View Slide

113. Extend it
     http://symfony.com/doc/current/cookbook/profiler/
     data_collector.html
     

     View Slide

114. View Slide

115. View Slide

116. View Slide

117. View Slide

118. View Slide

119. View Slide

120. Use it in production
     for a subset of
     requests
     

     View Slide

121. https://qafoolabs.com/
     

     View Slide

122. https://blackfire.io/
     

     View Slide

123. http://newrelic.com/
     

     View Slide

124. Step 3
     

     View Slide

125. Fix it
     

     View Slide

126. That's something
     you have to do
     

     View Slide

127. View Slide

128. Keep up to date
     

     View Slide

129. http://www.perfplanet.com/
     

     View Slide

130. Some hints on
     better
     performance
     

     View Slide

131. The obvious
     

     View Slide

132. GZIP
     

     View Slide

133. CDNs
     

     View Slide

134. DB Indexes
     

     View Slide

135. Minify JS
     https://github.com/mishoo/UglifyJS
     

     View Slide

136. Minify CSS
     http://sass-lang.com/
     

     View Slide

137. Concatenate JS/
     CSS/...
     

     View Slide

138. Correct caching
     headers
     

     View Slide

139. Opcache
     

     View Slide

140. Data Caching
     

     View Slide

141. APCU
     

     View Slide

142. memcached
     

     View Slide

143. PHP 5.6
     https://blog.asmallorange.com/2013/08/php-roadmap-
     performance/
     

     View Slide

144. The not so well
     known
     

     View Slide

145. SPDY/ HTTP2
     

     View Slide

146. Minimize redirects
     

     View Slide

147. Check image
     compression
     https://github.com/gruntjs/grunt-contrib-imagemin
     

     View Slide

148. Compress HTML
     

     View Slide

149. Check YSlow,
     Pagespeed
     

     View Slide

150. DNS prefetch
     

     View Slide

151. href="//host_name_to_prefetch.com">
     

     View Slide

152. Move logic to
     async workers
     

     View Slide

153. The "crazy" stuff
     

     View Slide

154. Varnish
     

     View Slide

155. ESI
     

     View Slide

156. Profile Publications Publication
     Publication
     Publication
     AboutMe
     LeftColumn Image
     Menu
     Institution
     

     View Slide

157. Profile Publications Publication
     Publication
     Publication
     AboutMe
     LeftColumn Image
     Menu
     
     Institution
     

     View Slide

158. Load content
     asynchronously
     

     View Slide

159. Profile Publications Publication
     Publication
     Publication
     AboutMe
     LeftColumn Image
     Menu
     Institution
     

     View Slide

160. Profile Publications Publication
     Publication
     Publication
     AboutMe
     LeftColumn Image
     Menu
     
     loadWidget('/aboutMe', function(w) {<br/>w.render({ replace : '#placeholder' });<br/>})
     Institution
     

     View Slide

161. Cache ajax
     requests
     

     View Slide

162. Flush content early
     

     View Slide

163. Move logic to
     shutdown handlers
     

     View Slide

164. http://www.php.net/manual/en/function.register-shutdown-
     function.php
     

     View Slide

165. http://www.php.net/manual/en/function.fastcgi-finish-
     request.php
     

     View Slide

166. Promises / Futures
     https://github.com/facebook/libphutil
     

     View Slide

167. $future  =  new  HTTPFuture(  
         'http://www.example.com/'  
     );  
     list($status,  $body,  $headers)  =    
       $future-­‐>resolve();
     

     View Slide

168. pushState
     

     View Slide

169. Bigpipe
     

     View Slide

170. View Slide

171. Profile Menu
     Header
     LeftColumn
     RightColumn
     

     View Slide

172. View Slide

173. View Slide

174. View Slide

175. View Slide

176. Remember
     

     View Slide

177. Speed matters
     

     View Slide

178. http://twitter.com/BastianHofmann
     http://lanyrd.com/people/BastianHofmann
     http://speakerdeck.com/u/bastianhofmann
     mail@bastianhofmann.de
     

     View Slide