Rails Performance Tips

my presentation in RubyConf Au 2016

Transcript

1. RAILS PERFORMANCE TIPS Richard Huang @flyerhzm
2. WHERE TO START
3. 80-90% OF THE END-USER RESPONSE TIME IS SPENT ON THE FRONTEND High Performance Web Sites by Steve Souders
4. FRONTEND PERFORMANCE TIPS
5. Keith and Mario’s Guide to Fast Websites From RubyConf Au 2013
6. 1. AS LESS REQUESTS AS POSSIBLE
7. CONCATENATE CSS & JAVASCRIPT Browser Server application.html a.css b.css c.css a.js b.js c.js Before
8. CONCATENATE CSS & JAVASCRIPT Browser Server application.html application.css application.js After
9. USE CSS SPRITE Browser Server application.html a.jpg b.jpg c.jpg Before
10. USE CSS SPRITE Browser Server application.html sprite.jpg After
11. 2. AS SMALL PAYLOADS AS POSSIBLE
12. MINIFY AND GZIP application.css (original) => application.css (minified) => application.css.gz application.js (original) => application.js (minified) => application.js.gz
13. COMPRESS IMAGES
14. COOKIE-LESS DOMAINS H o s t : x i n m i n l a b s . c o m R e f e r e r : h t t p s : / / x i n m i n l a b s . c o m C o o k i e : r a c k . s e s s i o n = B A h 7 E U k i D 3 N l c 3 N p b 2 5 f a W Q G O g Z F V E k i R T J j M j I 3 N j E z Y z Q 3 M T Z l M m Y z Z D k y % 0 A M T E z O T h k Z D d m Z j I y Y j N i O D B l Y W I 3 Z m E x Y z U 1 M j R h Z j Y w N 2 F h M T B i M G U w M 2 Y G % 0 A O w B G S S I J Y 3 N y Z g Y 7 A E Z J I i U x Y z l h M z h l Z D B l M 2 V h N G J h N m F l Z D M 1 M z F j Y j R i % 0 A Z W Y x O Q Y 7 A E Z J I g 1 0 c m F j a 2 l u Z w Y 7 A E Z 7 B 0 k i F E h U V F B f V V N F U l 9 B R 0 V O V A Y 7 % 0 A A F R J I i 1 m Y 2 M 3 Z j A 2 O G U 5 N 2 Q 5 N z A 2 Y T g w M D d m O T g 3 M j E w M j J j O T h i M j d l Y T I w % 0 A B j s A R k k i G U h U V F B f Q U N D R V B U X 0 x B T k d V Q U d F B j s A V E k i L W R i Y 2 M y M T Z i M T B j % 0 A M 2 I 1 M m J k N T g 3 Z T B j O D I w N T Z i M m I y Y m E 5 Y j J k N D Y G O w B G S S I R Y W N j Z X N z X 3 R v % 0 A a 2 V u B j s A V E k i d T A w R D I 4 M D A w M D A w Z n J F U S F B U U 1 B U U g x c V 9 M Y 2 J R b G 0 y R X h 5 % 0 A d 3 p q d m 1 P V j N M Z G h N Q S 5 D S F 8 x a V 9 4 Z D M 0 X 1 c 5 c z A w T D g z a H h J V 0 R a Q T B W N W 8 2 % 0 A U F R 1 c l E 5 c T d U W U t J O T Z m U V B o S 2 N f U j E 3 e k l s Q X p 3 N H c G O w B U S S I K Z W 1 h a W w G % 0 A O w B U S S I c Z m x 5 Z X J o e m 0 r c H J v Z E B n b W F p b C 5 j b 2 0 G O w B U S S I L a W R f d X J s B j s A % 0 A V E k i S m h 0 d H B z O i 8 v b G 9 n a W 4 u c 2 F s Z X N m b 3 J j Z S 5 j b 2 0 v a W Q v M D B E M j g w M D A w % 0 A M D B m c k V R R U F Z L z A w N T I 4 M D A w M D A x Z H l r N k F B Q Q Y 7 A F R J I h F p b n N 0 Y W 5 j Z V 9 1 % 0 A c m w G O w B U S S I j a H R 0 c H M 6 L y 9 s a X R h L m 1 5 L n N h b G V z Z m 9 y Y 2 U u Y 2 9 t B j s A V E k i % 0 A D X V z Z X J u Y W 1 l B j s A V E k i H G Z s e W V y a H p t K 3 B y b 2 R A Z 2 1 h a W w u Y 2 Before
15. COOKIE-LESS DOMAINS H o s t : a s s e t s . x i n m i n l a b s . c o m R e f e r e r : h t t p s : / / x i n m i n l a b s . c o m After
16. 3. AS FAST RESOURCES AS POSSIBLE
17. USE CDN
18. 4. OTHER
19. JAVASCRIPT AT THE BOTTOM
20. BACKEND PERFORMANCE TIPS
21. http://www.speedawarenessmonth.com/when-8020- becomes-2080/
22. HOW Don't guess! Don't guess! Don't guess! Find out the bottleneck
23. MONITOR New Relic Skylight
24. MONITOR
25. MONITOR
26. MONITOR
27. REPRODUCE Simulate the environment (postgres, memcached, etc.) Simulate the data
28. REPRODUCE New Relic (Developer Mode) rack-mini-profiler
29. REPRODUCE
30. REPRODUCE
31. REPRODUCE
32. REPRODUCE
33. Users Production Database Production Production Production Production Production New Relic Developer Mode
34. FIX
35. BENCHMARK rails-per est ab / siege
36. RAILS-PERFTEST r e q u i r e ' t e s t _ h e l p e r ' r e q u i r e ' r a i l s / p e r f o r m a n c e _ t e s t _ h e l p ' c l a s s H o m e p a g e T e s t < A c t i o n D i s p a t c h : : P e r f o r m a n c e T e s t # R e f e r t o t h e d o c u m e n t a t i o n f o r a l l a v a i l a b l e o p t i o n s # s e l f . p r o f i l e _ o p t i o n s = { r u n s : 5 , # m e t r i c s : [ : w a l l _ t i m e , : m e m o r y ] , # o u t p u t : ' t m p / p e r f o r m a n c e ' , # f o r m a t s : [ : f l a t ] } t e s t " h o m e p a g e " d o g e t ' / ' e n d e n d
37. RAILS-PERFTEST B r o w s i n g T e s t # t e s t _ h o m e p a g e ( 5 8 m s w a r m u p ) p r o c e s s _ t i m e : 6 3 m s m e m o r y : 8 3 2 . 1 3 K B o b j e c t s : 7 , 8 8 2
38. AB / SIEGE Monitor with New Relic
39. SUMMARIZE Monitor Find out bottleneck Reproduce Fix Benchmark Deploy Monitor
40. 1. AS LESS REQUESTS AS POSSIBLE
41. N+1 QUERY # c o n t r o l l e r @ c o m m e n t s = C o m m e n t . l i m i t ( 1 0 ) # v i e w < % @ c o m m e n t s . e a c h d o | c o m m e n t | % > < % = c o m m e n t . u s e r . u s e r n a m e % > S a i d : < % = c o m m e n t . b o d y % > < % e n d % > Before
42. N+1 QUERY App Server DB Server SELECT "comments".* FROM "comments" LIMIT 10 SELECT "users".* FROM "users" WHERE "users"."id" = 6 LIMIT 1 SELECT "users".* FROM "users" WHERE "users"."id" = 64 LIMIT 1 ...... SELECT "users".* FROM "users" WHERE "users"."id" = 19 LIMIT 1 Before
43. N+1 QUERY Newrelic response time is 12.3ms Before
44. N+1 QUERY # c o n t r o l l e r @ c o m m e n t s = C o m m e n t . i n c l u d e s ( : u s e r ) . l i m i t ( 1 0 ) # v i e w < % @ c o m m e n t s . e a c h d o | c o m m e n t | % > < % = c o m m e n t . u s e r . u s e r n a m e % > S a i d : < % = c o m m e n t . b o d y % > < % e n d % > After
45. N+1 QUERY App Server DB Server SELECT "comments".* FROM "comments" LIMIT 10 SELECT "users".* FROM "users" WHERE "users"."id" IN (6, 64, 17, 56, 71, 2, 75, 73, 18, 19) After
46. N+1 QUERY Newrelic response time is 6.81ms After
47. N+1 QUERY 12.3ms vs 6.81ms -45% 3.88ms vs 0.621ms -84%
48. N+1 QUERY flyerhzm/bullet
49. COUNTER CACHE # c o n t r o l l e r @ p o s t s = P o s t . l i m i t ( 1 0 ) # v i e w < % @ p o s t s . e a c h d o | p o s t | % > < % = p o s t . t i t l e % > h a s < % = p o s t . c o m m e n t . s i z e % > c o m m e n t s . < % e n d % > Before
50. COUNTER CACHE App Server DB Server SELECT "posts".* FROM "posts" LIMIT 10 SELECT COUNT(*) FROM "comments" WHERE "comments"."post_id" = 1 SELECT COUNT(*) FROM "comments" WHERE "comments"."post_id" = 2 ...... SELECT COUNT(*) FROM "comments" WHERE "comments"."post_id" = 10 Before
51. COUNTER CACHE Newrelic response time is 12.3ms Before
52. COUNTER CACHE # m i g r a t i o n a d d _ c o l u m n : p o s t s , : c o m m e n t s _ c o u n t , : i n t e g e r P o s t . a l l . e a c h d o | p o s t | P o s t . r e s e t _ c o u n t e r s ( p o s t . i d , : c o m m e n t s ) e n d # m o d e l c l a s s C o m m e n t < A c t i v e R e c o r d : : B a s e b e l o n g s _ t o : p o s t , c o u n t e r _ c a c h e : t r u e e n d After
53. COUNTER CACHE App Server DB Server SELECT "posts".* FROM "posts" LIMIT 10 After
54. COUNTER CACHE Newrelic response time is 7.04ms After
55. COUNTER CACHE 12.3ms vs 7.04ms -43% 6.434ms vs 0.511ms -92%
56. COUNTER CACHE flyerhzm/bullet
57. BONUS magnusvk/counter_culture
58. USE GROUP # m o d e l c l a s s C o m m e n t b e l o n g s _ t o : p o s t s c o p e : a p p r o v e d , - > { w h e r e ( a p p r o v e d : t r u e ) } e n d c l a s s P o s t h a s _ m a n y : c o m m e n t s d e f a v e r a g e _ r a t i n g c o m m e n t s . a p p r o v e d . a v e r a g e ( : r a t i n g ) e n d e n d Before
59. USE GROUP # c o n t r o l l e r @ p o s t s = P o s t . l i m i t ( 1 0 ) # v i e w < % @ p o s t s . e a c h d o | p o s t | % > < % = p o s t . t i t l e % > a v e r a g e r a t i n g i s < % = p o s t . a v e r a g e _ r a t i n g % > s t a r s < % e n d % > Before
60. USE GROUP App Server DB Server SELECT "posts".* FROM "posts" LIMIT 10 SELECT AVG("comments"."rating") FROM "comments" WHERE "comments"."post_id" = 1 AND "comments"."approved" = "t" SELECT AVG("comments"."rating") FROM "comments" WHERE "comments"."post_id" = 2 AND "comments"."approved" = "t" ...... SELECT AVG("comments"."rating") FROM "comments" WHERE "comments"."post_id" = 10 AND "comments"."approved" = "t" Before
61. USE GROUP Newrelic response time is 17.4ms Before
62. USE GROUP # G e m f i l e g e m ' e a g e r _ g r o u p ' # m o d e l c l a s s P o s t < A c t i v e R e c o r d : : B a s e h a s _ m a n y : c o m m e n t s d e f i n e _ e a g e r _ g r o u p : a v e r a g e _ r a t i n g , : c o m m e n t s , : a v e r a g e , : r a t i n g , - > { a p p r o v e d } e n d # c o n t r o l l e r @ p o s t s = P o s t . e a g e r _ g r o u p ( : a v e r a g e _ r a t i n g ) . l i m i t ( 1 0 ) After
63. USE GROUP App Server DB Server SELECT "posts".* FROM "posts" LIMIT 10 SELECT AVG("comments"."rating") AS average_rating, post_id AS post_id FROM "comments" WHERE "comments"."approved" = "t" AND "comments"."post_id" IN (1, 2, 3, 4, 5, 6, 7, 8, 9, 10) GROUP BY "comments"."post_id" After
64. USE GROUP Newrelic response time is 7.48ms After
65. USE GROUP 17.4ms vs 7.48ms -57% 6.62ms vs 1.55ms -77%
66. USE GROUP xinminlabs/eager_group
67. BONUS Postgresql Materialized View
68. MULTI INSERTS # c o n t r o l l e r 1 0 . t i m e s d o P o s t . c r e a t e t i t l e : F a k e r : : L o r e m . s e n t e n c e , b o d y : F a k e r : : H i p s t e r . p a r a g r a p h , u s e r _ i d : r a n d ( 1 0 0 ) + 1 e n d Before
69. MULTI INSERTS App Server DB Server BEGIN INSERT INTO "posts" ("title", "body", "user_id", "created_at", "updated_at") VALUES ($1, $2, $3, $4, $5) RETURNING "id" COMMIT ...... BEGIN INSERT INTO "posts" ("title", "body", "user_id", "created_at", "updated_at") VALUES ($1, $2, $3, $4, $5) RETURNING "id" COMMIT Before
70. MULTI INSERTS Newrelic response time is 35.4ms Before
71. MULTI INSERTS # G e m f i l e g e m ' a c t i v e r e c o r d - i m p o r t ' # c o n t r o l l e r p o s t s = [ ] 1 0 . t i m e s d o p o s t s < < P o s t . n e w ( t i t l e : F a k e r : : L o r e m . s e n t e n c e , b o d y : F a k e r : : H i p s t e r . p a r a g r a p h , u s e r _ i d : r a n d ( 1 0 0 ) + 1 ) e n d P o s t . i m p o r t p o s t s After
72. MULTI INSERTS App Server DB Server Class Create Many Without Validations Or Callbacks (3.6ms) INSERT INTO "posts" ("id","title","body","user_id") VALUES (nextval('public.posts_id_seq'),'title','body',31), (nextval('public.posts_id_seq'),'title','body',80), (nextval('public.posts_id_seq'),'title','body',73), (nextval('public.posts_id_seq'),'title','body',56), (nextval('public.posts_id_seq'),'title','body',66), (nextval('public.posts_id_seq'),'title','body',19), (nextval('public.posts_id_seq'),'title','body',11), (nextval('public.posts_id_seq'),'title','body',90), (nextval('public.posts_id_seq'),'title','body',90), (nextval('public.posts_id_seq'),'title','body',9) RETURNING id After
73. MULTI INSERTS Newrelic response time is 11.6ms After
74. MULTI INSERTS 35.4ms vs 11.6ms -67% 19.39ms vs 2.39ms -88%
75. MULTI INSERTS zdennis/activerecord-import
76. MULTI UPDATES P o s t . w h e r e ( " c r e a t e d _ a t < ? " , 1 0 . y e a r s . a g o ) . u p d a t e _ a l l ( a r c h i v e : t r u e )
77. MULTI DELETES P o s t . w h e r e ( " c r e a t e d _ a t < ? " , 1 0 . y e a r s . a g o ) . d e s t r o y _ a l l P o s t . w h e r e ( " c r e a t e d _ a t < ? " , 1 0 . y e a r s . a g o ) . d e l e t e _ a l l
78. 2. AS SMALL PAYLOADS AS POSSIBLE
79. SELECT DATA YOU REALLY NEED # c o n t r o l l e r @ p o s t s = P o s t . l i m i t ( 1 0 ) # v i e w < % @ p o s t s . e a c h d o | p o s t | % > < % = p o s t . t i t l e % > < % e n d % > Before
80. SELECT DATA YOU REALLY NEED App Server DB Server SELECT "posts".* FROM "posts" LIMIT 10 Before
81. SELECT DATA YOU REALLY NEED Newrelic response time is 4.86ms Before
82. SELECT DATA YOU REALLY NEED # c o n t r o l l e r @ p o s t s = P o s t . s e l e c t ( ' t i t l e ' ) . l i m i t ( 1 0 ) After
83. SELECT DATA YOU REALLY NEED App Server DB Server SELECT "posts"."title" FROM "posts" LIMIT 10 After
84. SELECT DATA YOU REALLY NEED Newrelic response time is 4.54ms After
85. SELECT DATA YOU REALLY NEED 4.86ms vs 4.54ms -7% 0.563ms vs 0.421ms -25%
86. 3. AS FAST RESOURCES AS POSSIBLE
87. USE CACHE # c o n t r o l l e r @ p o s t s = P o s t . l i m i t ( 1 0 ) # v i e w < % @ p o s t s . e a c h d o | p o s t | % > < % = p o s t . u s e r . u s e r n a m e % > s a i d < % = p o s t . t i t l e % > < % e n d % > Before
88. USE CACHE App Server DB Server SELECT "posts".* FROM "posts" LIMIT 10 SELECT "users".* FROM "users" WHERE "users"."id" = 50 SELECT "users".* FROM "users" WHERE "users"."id" = 97 ...... SELECT "users".* FROM "users" WHERE "users"."id" = 9 Before
89. USE CACHE Newrelic response time is 12.2ms Before
90. USE CACHE # G e m f i l e g e m ' d a l l i ' # c o n f i g / a p p l i c a t i o n . r b c o n f i g . c a c h e _ s t o r e = : m e m _ c a c h e _ s t o r e # v i e w < % @ p o s t s . e a c h d o | p o s t | % > < % c a c h e p o s t d o % > < % = p o s t . u s e r . u s e r n a m e % > s a i d < % = p o s t . t i t l e % > < % e n d % > < % e n d % > After
91. USE CACHE App Servers Database Memcached SELECT "posts".* FROM "posts" LIMIT 10 Post #50 Cache Page Post #97 Cache Page ...... Post #9 Cache Page After
92. USE CACHE Newrelic response time is 9.97ms After
93. USE CACHE 12.2ms vs 9.97ms -18% 8.11ms vs 5.79ms -29%
94. BONUS R a i l s . c a c h e . r e a d _ m u l t i https://github.com/n8/multi_fetch_fragments https://github.com/hooopo/second_level_cache
95. USE ELASTICSEARCH FOR SEARCH
96. USE REDIS
97. USE SSD
98. 4. OTHER
99. USE INDEX # c o n t r o l l e r @ p o s t s = P o s t . o r d e r ( ' c r e a t e d _ a t d e s c ' ) . l i m i t ( 1 0 ) # v i e w < % @ p o s t s . e a c h d o | p o s t | % > < % = p o s t . t i t l e % > < % e n d % > Before
100. USE INDEX S E L E C T " p o s t s " . * F R O M " p o s t s " O R D E R B Y c r e a t e d _ a t d e s c L I M I T 1 0 Before
101. USE INDEX Newrelic response time is 21.5ms Before
102. USE INDEX # m i g r a t i o n d e f c h a n g e a d d _ i n d e x : p o s t s , : c r e a t e d _ a t e n d After
103. USE INDEX Newrelic response time is 4.95ms After
104. USE INDEX 21.5ms vs 4.95ms -77% 17.3ms vs 0.595ms -97%
105. USE INDEX plentz/lol_dba
106. OPTIMIZE JSON RENDERING # c o n t r o l l e r @ p o s t s = P o s t . l i m i t ( 1 0 ) r e n d e r j s o n : @ p o s t s Before
107. OPTIMIZE JSON RENDERING Newrelic response time is 6.28ms Before
108. OPTIMIZE JSON RENDERING # G e m f i l e g e m ' o j ' g e m ' o j _ m i m i c _ j s o n ' After
109. OPTIMIZE JSON RENDERING Newrelic response time is 5.76ms After
110. OPTIMIZE JSON RENDERING 6.28ms vs 5.76ms -8% 3.92ms vs 3.28ms -16%
111. OPTIMIZE JSON RENDERING ohler55/oj
112. MEMORY OPTIMIZE
113. https://github.com/rails/rails/pull/20946 shave off 1,114 string objects on every request
114. https://github.com/rails/rails/pull/21057 shave off 34,299 objects on every request
115. FIND IN BATCH P e r s o n . w h e r e ( " a g e > 2 1 " ) . e a c h d o | p e r s o n | p e r s o n . p a r t y _ a l l _ n i g h t ! e n d => P e r s o n . w h e r e ( " a g e > 2 1 " ) . f i n d _ e a c h d o | p e r s o n | p e r s o n . p a r t y _ a l l _ n i g h t ! e n d
116. FIND IN BATCH Send multiple sql requests But reduce memory usage
117. NO TIP IS ALWAYS TRUE
118. REVIEW Frontend performance tuning does first Backend performance tuning is important
119. REVIEW As less requests as possible As small payloads as possible As fast resources as possible Other
120. REVIEW Monitor Find out bottleneck Reproduce Fix Benchmark Deploy Monitor
121. DEMOS CODE https://github.com/xinminlabs/rails-performance-tips-code
122. THANK YOU Richard Huang @flyerhzm