My site is slow, what can I do? - Profiling PHP applications

It's nothing new that speed is important for the success of any web application. Only a few hundred milliseconds may lie between a user leaving your site or staying. Unfortunately performance problems are oftentimes hard to fix and even harder to pinpoint.
In this talk I will show you how we at ResearchGate measure web application performance, which means not only timing how long the PHP backend took to deliver a page, but also tracking the speed the users actually perceives in the browser. After that you will see how you can track down and analyze any problems you found through measuring with the help of tools like Xdebug, XHProf, the Qafoo Profiler and the Symfony Debug Toolbar. And if you still need to get faster after optimizing and fixing all these issues, I'll introduce you to some tricks, techniques and patterns to even further decrease load times.

Transcript

1. My site is slow,
   what can I do?
   Profiling PHP
   @BastianHofmann
   

   View Slide

2. This talk is all
   about...
   

   View Slide

3. View Slide

4. Speed of your web
   application
   

   View Slide

5. We'll talk about...
   

   View Slide

6. Why it matters
   

   View Slide

7. How to measure it
   

   View Slide

8. How to find out
   where the problems
   are
   

   View Slide

9. Before we start
   

   View Slide

10. A few words
    about me
    

    View Slide

11. View Slide

12. View Slide

13. Questions? Ask
    

    View Slide

14. http://speakerdeck.com/u/bastianhofmann
    

    View Slide

15. View Slide

16. View Slide

17. View Slide

18. Why should you
    care?
    

    View Slide

19. It is really important
    

    View Slide

20. View Slide

21. View Slide

22. Every ms counts
    

    View Slide

23. View Slide

24. View Slide

25. View Slide

26. So what is
    pagespeed?
    

    View Slide

27. Server
    

    View Slide

28. Your PHP application
    Request
    Response
    

    View Slide

29. Your PHP application
    Request
    Response
    Load balancer
    

    View Slide

30. Your PHP application
    Request
    Response
    Load balancer
    

    View Slide

31. web server db
    http service
    http service
    cache
    user request
    

    View Slide

32. But there is more
    

    View Slide

33. View Slide

34. View Slide

35. View Slide

36. View Slide

37. http://www.stevesouders.com/blog/2012/02/10/the-performance-golden-rule/
    

    View Slide

38. So what happens
    on your server is
    really important
    

    View Slide

39. But the rest is as
    well
    

    View Slide

40. Step 1
    

    View Slide

41. Measuring
    

    View Slide

42. View Slide

43. View Slide

44. View Slide

45. View Slide

46. View Slide

47. View Slide

48. View Slide

49. So you need to
    measure at your
    users side
    

    View Slide

50. Navigation Timing
    API
    https://developer.mozilla.org/en-US/docs/
    Navigation_timing
    

    View Slide

51. View Slide

52. Resource Timing API
    http://www.w3.org/TR/resource-timing/
    

    View Slide

53. View Slide

54. For older browsers
    you have to do it
    yourself
    

    View Slide

55. https://github.com/lognormal/boomerang
    

    View Slide

56. View Slide

57. Getting it back to
    the server
    

    View Slide

58. Tracking request
    

    View Slide

59. https://tracking.example.com/?
    page=profile&backend=123&co
    mplete=890&domReady=580
    

    View Slide

60. logstash
    http://logstash.net/
    

    View Slide

61. input filter output
    

    View Slide

62. Very rich plugin
    system
    

    View Slide

63. browser
    JS: boomerang
    logstash
    trackingServer
    access log
    requests
    tracking image
    with timing
    information as
    query
    parameters
    

    View Slide

64. Graphing it
    

    View Slide

65. Graphite
    http://graphite.wikidot.com/
    

    View Slide

66. View Slide

67. browser
    JS: boomerang
    logstash
    trackingServer
    access log
    requests
    tracking image
    with timing
    information as
    query
    parameters
    graphite
    statsd
    

    View Slide

68. input {

    file {

    type => "pagespeed-access"

    path => [ "/var/log/nginx/
    access_log/monitoring-access.log" ]

    }

    }
    

    View Slide

69. filter{

    grok {

    type => "pagespeed-access"

    pattern => "^.*\s\"[A-Z]+\s[^\?\s]+
    \?page=%{DATA:page}\&connectTime=%
    {NUMBER:connectTime}...)?\sHTTP\/\d\.\d\".*
    $"

    }

    grok {

    type => "pagespeed-access"

    match => ["page", "^(profile|
    home|...)\.logged(In|Out)$"]

    exclude_tags =>
    ["_grokparsefailure"]

    }

    }
    

    View Slide

70. output {
    statsd {
    type => "pagespeed-access"
    exclude_tags =>
    ["_grokparsefailure"]
    host => "localhost"
    port => 8126
    namespace => "pagespeed"
    sender => ""
    timing => [
    "%{page}.connect", "%
    {connectTime}", ...
    ]
    }
    }
    

    View Slide

71. View Slide

72. Can we measure
    more?
    

    View Slide

73. Load balancer
    

    View Slide

74. input {

    file {

    type => "haproxy-http-log"

    path => [ "/var/log/
    haproxy-http.log*" ]

    }

    }
    

    View Slide

75. filter {

    grok {

    type => "haproxy-http-log"

    pattern => "%{HAPROXYHTTP}" 

    }

    mutate {

    type => "haproxy-http-log"

    gsub => [

    "server_name", "\.", "_",

    "client_ip", "\.", "_"

    ]

    }

    }
    

    View Slide

76. output {

    statsd {

    type => "haproxy-http-log"

    exclude_tags => ["_grokparsefailure"]

    host => "localhost"

    port => 8125

    namespace => "lb"

    sender => ""

    increment => [

    "haproxy.%{backend_name}.%{server_name}.%
    {client_ip}.hits",

    "haproxy.%{backend_name}.%{server_name}.%
    {client_ip}.responses.%{http_status_code}"

    ]

    timing => [

    "haproxy.%{backend_name}.%{server_name}.%
    {client_ip}.time_request", "%{time_request}",

    "haproxy.%{backend_name}.%{server_name}.%
    {client_ip}.time_backend_connect", "%{time_backend_connect}",

    "haproxy.%{backend_name}.%{server_name}.%
    {client_ip}.time_backend_response", "%{time_backend_response}",

    "haproxy.%{backend_name}.%{server_name}.%
    {client_ip}.time_queue", "%{time_queue}",

    "haproxy.%{backend_name}.%{server_name}.%
    {client_ip}.time_duration", "%{time_duration}"

    ]

    }

    

    View Slide

77. browser
    JS: boomerang
    logstash
    trackingServer
    access log
    requests
    tracking image
    with timing
    information as
    query
    parameters
    graphite
    statsd
    logstash
    load balancer
    access log
    logstash
    service
    access log
    

    View Slide

78. From within your
    PHP app
    

    View Slide

79. From your JS
    frontend
    

    View Slide

80. Slow query logs
    

    View Slide

81. More fine grained
    metrics
    

    View Slide

82. By pages
    

    View Slide

83. By browser
    

    View Slide

84. By country
    

    View Slide

85. Logged in / out
    

    View Slide

86. Heavy users
    

    View Slide

87. Define goals
    

    View Slide

88. Anomaly detection
    

    View Slide

89. http://codeascraft.com/2013/06/11/introducing-kale/
    

    View Slide

90. Step 2
    

    View Slide

91. How to find out
    where the problems
    are
    

    View Slide

92. Profiling
    

    View Slide

93. View Slide

94. xdebug.profiler_enable_trigger = 1
    http://url?XDEBUG_PROFILE
    

    View Slide

95. Webgrind
    https://github.com/jokkedk/webgrind
    

    View Slide

96. View Slide

97. Use it locally on
    your dev machine
    

    View Slide

98. XHProf
    https://github.com/facebook/xhprof
    

    View Slide

99. Use it in production
    for a subset of
    requests
    

    View Slide

100. XHGUI
     https://github.com/perftools/xhgui
     

     View Slide

101. View Slide

102. View Slide

103. View Slide

104. View Slide

105. php-meminfo
     https://github.com/BitOne/php-meminfo
     

     View Slide

106. Blackfire
     https://blackfire.io/
     

     View Slide

107. Tideways
     https://tidyways.io/
     

     View Slide

108. View Slide

109. View Slide

110. New Relic
     http://newrelic.com/
     

     View Slide

111. View Slide

112. Symfony Debug
     Toolbar
     

     View Slide

113. View Slide

114. View Slide

115. Extend it
     http://symfony.com/doc/current/cookbook/profiler/
     data_collector.html
     

     View Slide

116. View Slide

117. View Slide

118. View Slide

119. View Slide

120. View Slide

121. View Slide

122. Aggregate it
     

     View Slide

123. View Slide

124. Step 3
     

     View Slide

125. Fix it
     

     View Slide

126. That's something
     you have to do
     

     View Slide

127. View Slide

128. Some hints on
     better
     performance
     

     View Slide

129. GZIP
     

     View Slide

130. CDNs
     

     View Slide

131. DNS provider
     

     View Slide

132. Test them
     

     View Slide

133. DB Indexes
     

     View Slide

134. Minify JS
     https://github.com/mishoo/UglifyJS
     

     View Slide

135. Minify CSS
     http://sass-lang.com/
     

     View Slide

136. Concatenate JS/
     CSS/...
     

     View Slide

137. Correct caching
     headers
     

     View Slide

138. Opcache
     

     View Slide

139. Data Caching
     

     View Slide

140. APCU
     

     View Slide

141. memcached
     

     View Slide

142. PHP 5.6
     https://blog.asmallorange.com/2013/08/php-roadmap-
     performance/
     

     View Slide

143. Are you using your
     libraries correctly?
     

     View Slide

144. Symfony
     

     View Slide

145. Doctrine
     

     View Slide

146. DI container
     

     View Slide

147. SPDY/HTTP 2.0
     

     View Slide

148. Minimize redirects
     

     View Slide

149. Check image
     compression
     https://github.com/gruntjs/grunt-contrib-imagemin
     

     View Slide

150. Compress HTML
     

     View Slide

151. Check YSlow,
     Pagespeed
     

     View Slide

152. DNS prefetch
     

     View Slide

153. href="//host_name_to_prefetch.com">
     

     View Slide

154. href="//host_name_to_prefetch.com">
     

     View Slide

155. Move logic to
     async workers
     

     View Slide

156. Precompute
     expensive stuff
     

     View Slide

157. Varnish
     

     View Slide

158. ESI
     

     View Slide

159. Profile Publications Publication
     Publication
     Publication
     AboutMe
     LeftColumn Image
     Menu
     Institution
     

     View Slide

160. Profile Publications Publication
     Publication
     Publication
     AboutMe
     LeftColumn Image
     Menu
     
     Institution
     

     View Slide

161. Load content
     asynchronously
     

     View Slide

162. Profile Publications Publication
     Publication
     Publication
     AboutMe
     LeftColumn Image
     Menu
     Institution
     

     View Slide

163. Profile Publications Publication
     Publication
     Publication
     AboutMe
     LeftColumn Image
     Menu
     
     loadWidget('/aboutMe', function(w) {<br/>w.render({ replace : '#placeholder' });<br/>})
     Institution
     

     View Slide

164. Flush content early
     

     View Slide

165. Move logic to
     shutdown handlers
     

     View Slide

166. http://www.php.net/manual/en/function.register-shutdown-
     function.php
     

     View Slide

167. http://www.php.net/manual/en/function.fastcgi-finish-
     request.php
     

     View Slide

168. Promises / Futures
     https://github.com/facebook/libphutil
     

     View Slide

169. $future  =  new  HTTPFuture(  
         'http://www.example.com/'  
     );  
     list($status,  $body,  $headers)  =    
       $future-­‐>resolve();
     

     View Slide

170. pushState
     

     View Slide

171. Bigpipe
     

     View Slide

172. View Slide

173. Profile Menu
     Header
     LeftColumn
     RightColumn
     

     View Slide

174. View Slide

175. View Slide

176. View Slide

177. View Slide

178. Generate code
     

     View Slide

179. TSL early
     termination
     

     View Slide

180. Keep up to date
     

     View Slide

181. http://www.perfplanet.com/
     

     View Slide

182. Regardless of what
     you do:
     Measure it
     

     View Slide

183. Remember
     

     View Slide

184. Speed matters
     

     View Slide

185. https://joind.in/14255
     

     View Slide

186. http://twitter.com/BastianHofmann
     http://lanyrd.com/people/BastianHofmann
     http://speakerdeck.com/u/bastianhofmann
     mail@bastianhofmann.de
     

     View Slide