Rails Performance Tips

Transcript

1. RAILS PERFORMANCE TIPS Richard Huang @flyerhzm

2. WHERE TO START

3. 80-90% OF THE END-USER RESPONSE TIME IS SPENT ON THE

   FRONTEND High Performance Web Sites by Steve Souders

4. FRONTEND PERFORMANCE TIPS

5. Keith and Mario’s Guide to Fast Websites From RubyConf Au

   2013

6. 1. AS LESS REQUESTS AS POSSIBLE

7. CONCATENATE CSS & JAVASCRIPT Browser Server application.html a.css b.css c.css

   a.js b.js c.js Before

8. CONCATENATE CSS & JAVASCRIPT Browser Server application.html application.css application.js After

9. USE CSS SPRITE Browser Server application.html a.jpg b.jpg c.jpg Before

10. USE CSS SPRITE Browser Server application.html sprite.jpg After

11. 2. AS SMALL PAYLOADS AS POSSIBLE

12. MINIFY AND GZIP application.css (original) => application.css (minified) => application.css.gz

    application.js (original) => application.js (minified) => application.js.gz

13. COMPRESS IMAGES

14. COOKIE-LESS DOMAINS H o s t : x i n

    m i n l a b s . c o m R e f e r e r : h t t p s : / / x i n m i n l a b s . c o m C o o k i e : r a c k . s e s s i o n = B A h 7 E U k i D 3 N l c 3 N p b 2 5 f a W Q G O g Z F V E k i R T J j M j I 3 N j E z Y z Q 3 M T Z l M m Y z Z D k y % 0 A M T E z O T h k Z D d m Z j I y Y j N i O D B l Y W I 3 Z m E x Y z U 1 M j R h Z j Y w N 2 F h M T B i M G U w M 2 Y G % 0 A O w B G S S I J Y 3 N y Z g Y 7 A E Z J I i U x Y z l h M z h l Z D B l M 2 V h N G J h N m F l Z D M 1 M z F j Y j R i % 0 A Z W Y x O Q Y 7 A E Z J I g 1 0 c m F j a 2 l u Z w Y 7 A E Z 7 B 0 k i F E h U V F B f V V N F U l 9 B R 0 V O V A Y 7 % 0 A A F R J I i 1 m Y 2 M 3 Z j A 2 O G U 5 N 2 Q 5 N z A 2 Y T g w M D d m O T g 3 M j E w M j J j O T h i M j d l Y T I w % 0 A B j s A R k k i G U h U V F B f Q U N D R V B U X 0 x B T k d V Q U d F B j s A V E k i L W R i Y 2 M y M T Z i M T B j % 0 A M 2 I 1 M m J k N T g 3 Z T B j O D I w N T Z i M m I y Y m E 5 Y j J k N D Y G O w B G S S I R Y W N j Z X N z X 3 R v % 0 A a 2 V u B j s A V E k i d T A w R D I 4 M D A w M D A w Z n J F U S F B U U 1 B U U g x c V 9 M Y 2 J R b G 0 y R X h 5 % 0 A d 3 p q d m 1 P V j N M Z G h N Q S 5 D S F 8 x a V 9 4 Z D M 0 X 1 c 5 c z A w T D g z a H h J V 0 R a Q T B W N W 8 2 % 0 A U F R 1 c l E 5 c T d U W U t J O T Z m U V B o S 2 N f U j E 3 e k l s Q X p 3 N H c G O w B U S S I K Z W 1 h a W w G % 0 A O w B U S S I c Z m x 5 Z X J o e m 0 r c H J v Z E B n b W F p b C 5 j b 2 0 G O w B U S S I L a W R f d X J s B j s A % 0 A V E k i S m h 0 d H B z O i 8 v b G 9 n a W 4 u c 2 F s Z X N m b 3 J j Z S 5 j b 2 0 v a W Q v M D B E M j g w M D A w % 0 A M D B m c k V R R U F Z L z A w N T I 4 M D A w M D A x Z H l r N k F B Q Q Y 7 A F R J I h F p b n N 0 Y W 5 j Z V 9 1 % 0 A c m w G O w B U S S I j a H R 0 c H M 6 L y 9 s a X R h L m 1 5 L n N h b G V z Z m 9 y Y 2 U u Y 2 9 t B j s A V E k i % 0 A D X V z Z X J u Y W 1 l B j s A V E k i H G Z s e W V y a H p t K 3 B y b 2 R A Z 2 1 h a W w u Y 2 Before

15. COOKIE-LESS DOMAINS H o s t : a s s

    e t s . x i n m i n l a b s . c o m R e f e r e r : h t t p s : / / x i n m i n l a b s . c o m After

16. 3. AS FAST RESOURCES AS POSSIBLE

17. USE CDN

18. 4. OTHER

19. JAVASCRIPT AT THE BOTTOM

20. BACKEND PERFORMANCE TIPS

21. http://www.speedawarenessmonth.com/when-8020- becomes-2080/

22. HOW Don't guess! Don't guess! Don't guess! Find out the

    bottleneck

23. MONITOR New Relic Skylight

24. MONITOR

25. MONITOR

26. MONITOR

27. REPRODUCE Simulate the environment (postgres, memcached, etc.) Simulate the data

28. REPRODUCE New Relic (Developer Mode) rack-mini-profiler

29. REPRODUCE

30. REPRODUCE

31. REPRODUCE

32. REPRODUCE

33. Users Production Database Production Production Production Production Production New Relic

    Developer Mode

34. FIX

35. BENCHMARK rails-per est ab / siege

36. RAILS-PERFTEST r e q u i r e ' t

    e s t _ h e l p e r ' r e q u i r e ' r a i l s / p e r f o r m a n c e _ t e s t _ h e l p ' c l a s s H o m e p a g e T e s t < A c t i o n D i s p a t c h : : P e r f o r m a n c e T e s t # R e f e r t o t h e d o c u m e n t a t i o n f o r a l l a v a i l a b l e o p t i o n s # s e l f . p r o f i l e _ o p t i o n s = { r u n s : 5 , # m e t r i c s : [ : w a l l _ t i m e , : m e m o r y ] , # o u t p u t : ' t m p / p e r f o r m a n c e ' , # f o r m a t s : [ : f l a t ] } t e s t " h o m e p a g e " d o g e t ' / ' e n d e n d

37. RAILS-PERFTEST B r o w s i n g T

    e s t # t e s t _ h o m e p a g e ( 5 8 m s w a r m u p ) p r o c e s s _ t i m e : 6 3 m s m e m o r y : 8 3 2 . 1 3 K B o b j e c t s : 7 , 8 8 2

38. AB / SIEGE Monitor with New Relic

39. SUMMARIZE Monitor Find out bottleneck Reproduce Fix Benchmark Deploy Monitor

40. 1. AS LESS REQUESTS AS POSSIBLE

41. N+1 QUERY # c o n t r o l

    l e r @ c o m m e n t s = C o m m e n t . l i m i t ( 1 0 ) # v i e w < % @ c o m m e n t s . e a c h d o | c o m m e n t | % > < % = c o m m e n t . u s e r . u s e r n a m e % > S a i d : < % = c o m m e n t . b o d y % > < % e n d % > Before

42. N+1 QUERY App Server DB Server SELECT "comments".* FROM "comments"

    LIMIT 10 SELECT "users".* FROM "users" WHERE "users"."id" = 6 LIMIT 1 SELECT "users".* FROM "users" WHERE "users"."id" = 64 LIMIT 1 ...... SELECT "users".* FROM "users" WHERE "users"."id" = 19 LIMIT 1 Before

43. N+1 QUERY Newrelic response time is 12.3ms Before

44. N+1 QUERY # c o n t r o l

    l e r @ c o m m e n t s = C o m m e n t . i n c l u d e s ( : u s e r ) . l i m i t ( 1 0 ) # v i e w < % @ c o m m e n t s . e a c h d o | c o m m e n t | % > < % = c o m m e n t . u s e r . u s e r n a m e % > S a i d : < % = c o m m e n t . b o d y % > < % e n d % > After

45. N+1 QUERY App Server DB Server SELECT "comments".* FROM "comments"

    LIMIT 10 SELECT "users".* FROM "users" WHERE "users"."id" IN (6, 64, 17, 56, 71, 2, 75, 73, 18, 19) After

46. N+1 QUERY Newrelic response time is 6.81ms After

47. N+1 QUERY 12.3ms vs 6.81ms -45% 3.88ms vs 0.621ms -84%

48. N+1 QUERY flyerhzm/bullet

49. COUNTER CACHE # c o n t r o l

    l e r @ p o s t s = P o s t . l i m i t ( 1 0 ) # v i e w < % @ p o s t s . e a c h d o | p o s t | % > < % = p o s t . t i t l e % > h a s < % = p o s t . c o m m e n t . s i z e % > c o m m e n t s . < % e n d % > Before

50. COUNTER CACHE App Server DB Server SELECT "posts".* FROM "posts"

    LIMIT 10 SELECT COUNT(*) FROM "comments" WHERE "comments"."post_id" = 1 SELECT COUNT(*) FROM "comments" WHERE "comments"."post_id" = 2 ...... SELECT COUNT(*) FROM "comments" WHERE "comments"."post_id" = 10 Before

51. COUNTER CACHE Newrelic response time is 12.3ms Before

52. COUNTER CACHE # m i g r a t i

    o n a d d _ c o l u m n : p o s t s , : c o m m e n t s _ c o u n t , : i n t e g e r P o s t . a l l . e a c h d o | p o s t | P o s t . r e s e t _ c o u n t e r s ( p o s t . i d , : c o m m e n t s ) e n d # m o d e l c l a s s C o m m e n t < A c t i v e R e c o r d : : B a s e b e l o n g s _ t o : p o s t , c o u n t e r _ c a c h e : t r u e e n d After

53. COUNTER CACHE App Server DB Server SELECT "posts".* FROM "posts"

    LIMIT 10 After

54. COUNTER CACHE Newrelic response time is 7.04ms After

55. COUNTER CACHE 12.3ms vs 7.04ms -43% 6.434ms vs 0.511ms -92%

56. COUNTER CACHE flyerhzm/bullet

57. BONUS magnusvk/counter_culture

58. USE GROUP # m o d e l c l

    a s s C o m m e n t b e l o n g s _ t o : p o s t s c o p e : a p p r o v e d , - > { w h e r e ( a p p r o v e d : t r u e ) } e n d c l a s s P o s t h a s _ m a n y : c o m m e n t s d e f a v e r a g e _ r a t i n g c o m m e n t s . a p p r o v e d . a v e r a g e ( : r a t i n g ) e n d e n d Before

59. USE GROUP # c o n t r o l

    l e r @ p o s t s = P o s t . l i m i t ( 1 0 ) # v i e w < % @ p o s t s . e a c h d o | p o s t | % > < % = p o s t . t i t l e % > a v e r a g e r a t i n g i s < % = p o s t . a v e r a g e _ r a t i n g % > s t a r s < % e n d % > Before

60. USE GROUP App Server DB Server SELECT "posts".* FROM "posts"

    LIMIT 10 SELECT AVG("comments"."rating") FROM "comments" WHERE "comments"."post_id" = 1 AND "comments"."approved" = "t" SELECT AVG("comments"."rating") FROM "comments" WHERE "comments"."post_id" = 2 AND "comments"."approved" = "t" ...... SELECT AVG("comments"."rating") FROM "comments" WHERE "comments"."post_id" = 10 AND "comments"."approved" = "t" Before

61. USE GROUP Newrelic response time is 17.4ms Before

62. USE GROUP # G e m f i l e

    g e m ' e a g e r _ g r o u p ' # m o d e l c l a s s P o s t < A c t i v e R e c o r d : : B a s e h a s _ m a n y : c o m m e n t s d e f i n e _ e a g e r _ g r o u p : a v e r a g e _ r a t i n g , : c o m m e n t s , : a v e r a g e , : r a t i n g , - > { a p p r o v e d } e n d # c o n t r o l l e r @ p o s t s = P o s t . e a g e r _ g r o u p ( : a v e r a g e _ r a t i n g ) . l i m i t ( 1 0 ) After

63. USE GROUP App Server DB Server SELECT "posts".* FROM "posts"

    LIMIT 10 SELECT AVG("comments"."rating") AS average_rating, post_id AS post_id FROM "comments" WHERE "comments"."approved" = "t" AND "comments"."post_id" IN (1, 2, 3, 4, 5, 6, 7, 8, 9, 10) GROUP BY "comments"."post_id" After

64. USE GROUP Newrelic response time is 7.48ms After

65. USE GROUP 17.4ms vs 7.48ms -57% 6.62ms vs 1.55ms -77%

66. USE GROUP xinminlabs/eager_group

67. BONUS Postgresql Materialized View

68. MULTI INSERTS # c o n t r o l

    l e r 1 0 . t i m e s d o P o s t . c r e a t e t i t l e : F a k e r : : L o r e m . s e n t e n c e , b o d y : F a k e r : : H i p s t e r . p a r a g r a p h , u s e r _ i d : r a n d ( 1 0 0 ) + 1 e n d Before

69. MULTI INSERTS App Server DB Server BEGIN INSERT INTO "posts"

    ("title", "body", "user_id", "created_at", "updated_at") VALUES ($1, $2, $3, $4, $5) RETURNING "id" COMMIT ...... BEGIN INSERT INTO "posts" ("title", "body", "user_id", "created_at", "updated_at") VALUES ($1, $2, $3, $4, $5) RETURNING "id" COMMIT Before

70. MULTI INSERTS Newrelic response time is 35.4ms Before

71. MULTI INSERTS # G e m f i l e

    g e m ' a c t i v e r e c o r d - i m p o r t ' # c o n t r o l l e r p o s t s = [ ] 1 0 . t i m e s d o p o s t s < < P o s t . n e w ( t i t l e : F a k e r : : L o r e m . s e n t e n c e , b o d y : F a k e r : : H i p s t e r . p a r a g r a p h , u s e r _ i d : r a n d ( 1 0 0 ) + 1 ) e n d P o s t . i m p o r t p o s t s After

72. MULTI INSERTS App Server DB Server Class Create Many Without

    Validations Or Callbacks (3.6ms) INSERT INTO "posts" ("id","title","body","user_id") VALUES (nextval('public.posts_id_seq'),'title','body',31), (nextval('public.posts_id_seq'),'title','body',80), (nextval('public.posts_id_seq'),'title','body',73), (nextval('public.posts_id_seq'),'title','body',56), (nextval('public.posts_id_seq'),'title','body',66), (nextval('public.posts_id_seq'),'title','body',19), (nextval('public.posts_id_seq'),'title','body',11), (nextval('public.posts_id_seq'),'title','body',90), (nextval('public.posts_id_seq'),'title','body',90), (nextval('public.posts_id_seq'),'title','body',9) RETURNING id After

73. MULTI INSERTS Newrelic response time is 11.6ms After

74. MULTI INSERTS 35.4ms vs 11.6ms -67% 19.39ms vs 2.39ms -88%

75. MULTI INSERTS zdennis/activerecord-import

76. MULTI UPDATES P o s t . w h e

    r e ( " c r e a t e d _ a t < ? " , 1 0 . y e a r s . a g o ) . u p d a t e _ a l l ( a r c h i v e : t r u e )

77. MULTI DELETES P o s t . w h e

    r e ( " c r e a t e d _ a t < ? " , 1 0 . y e a r s . a g o ) . d e s t r o y _ a l l P o s t . w h e r e ( " c r e a t e d _ a t < ? " , 1 0 . y e a r s . a g o ) . d e l e t e _ a l l

78. 2. AS SMALL PAYLOADS AS POSSIBLE

79. SELECT DATA YOU REALLY NEED # c o n t

    r o l l e r @ p o s t s = P o s t . l i m i t ( 1 0 ) # v i e w < % @ p o s t s . e a c h d o | p o s t | % > < % = p o s t . t i t l e % > < % e n d % > Before

80. SELECT DATA YOU REALLY NEED App Server DB Server SELECT

    "posts".* FROM "posts" LIMIT 10 Before

81. SELECT DATA YOU REALLY NEED Newrelic response time is 4.86ms

    Before

82. SELECT DATA YOU REALLY NEED # c o n t

    r o l l e r @ p o s t s = P o s t . s e l e c t ( ' t i t l e ' ) . l i m i t ( 1 0 ) After

83. SELECT DATA YOU REALLY NEED App Server DB Server SELECT

    "posts"."title" FROM "posts" LIMIT 10 After

84. SELECT DATA YOU REALLY NEED Newrelic response time is 4.54ms

    After

85. SELECT DATA YOU REALLY NEED 4.86ms vs 4.54ms -7% 0.563ms

    vs 0.421ms -25%

86. 3. AS FAST RESOURCES AS POSSIBLE

87. USE CACHE # c o n t r o l

    l e r @ p o s t s = P o s t . l i m i t ( 1 0 ) # v i e w < % @ p o s t s . e a c h d o | p o s t | % > < % = p o s t . u s e r . u s e r n a m e % > s a i d < % = p o s t . t i t l e % > < % e n d % > Before

88. USE CACHE App Server DB Server SELECT "posts".* FROM "posts"

    LIMIT 10 SELECT "users".* FROM "users" WHERE "users"."id" = 50 SELECT "users".* FROM "users" WHERE "users"."id" = 97 ...... SELECT "users".* FROM "users" WHERE "users"."id" = 9 Before

89. USE CACHE Newrelic response time is 12.2ms Before

90. USE CACHE # G e m f i l e

    g e m ' d a l l i ' # c o n f i g / a p p l i c a t i o n . r b c o n f i g . c a c h e _ s t o r e = : m e m _ c a c h e _ s t o r e # v i e w < % @ p o s t s . e a c h d o | p o s t | % > < % c a c h e p o s t d o % > < % = p o s t . u s e r . u s e r n a m e % > s a i d < % = p o s t . t i t l e % > < % e n d % > < % e n d % > After

91. USE CACHE App Servers Database Memcached SELECT "posts".* FROM "posts"

    LIMIT 10 Post #50 Cache Page Post #97 Cache Page ...... Post #9 Cache Page After

92. USE CACHE Newrelic response time is 9.97ms After

93. USE CACHE 12.2ms vs 9.97ms -18% 8.11ms vs 5.79ms -29%

94. BONUS R a i l s . c a c

    h e . r e a d _ m u l t i https://github.com/n8/multi_fetch_fragments https://github.com/hooopo/second_level_cache

95. USE ELASTICSEARCH FOR SEARCH

96. USE REDIS

97. USE SSD

98. 4. OTHER

99. USE INDEX # c o n t r o l

    l e r @ p o s t s = P o s t . o r d e r ( ' c r e a t e d _ a t d e s c ' ) . l i m i t ( 1 0 ) # v i e w < % @ p o s t s . e a c h d o | p o s t | % > < % = p o s t . t i t l e % > < % e n d % > Before

100. USE INDEX S E L E C T " p

     o s t s " . * F R O M " p o s t s " O R D E R B Y c r e a t e d _ a t d e s c L I M I T 1 0 Before

101. USE INDEX Newrelic response time is 21.5ms Before

102. USE INDEX # m i g r a t i

     o n d e f c h a n g e a d d _ i n d e x : p o s t s , : c r e a t e d _ a t e n d After

103. USE INDEX Newrelic response time is 4.95ms After

104. USE INDEX 21.5ms vs 4.95ms -77% 17.3ms vs 0.595ms -97%

105. USE INDEX plentz/lol_dba

106. OPTIMIZE JSON RENDERING # c o n t r o

     l l e r @ p o s t s = P o s t . l i m i t ( 1 0 ) r e n d e r j s o n : @ p o s t s Before

107. OPTIMIZE JSON RENDERING Newrelic response time is 6.28ms Before

108. OPTIMIZE JSON RENDERING # G e m f i l

     e g e m ' o j ' g e m ' o j _ m i m i c _ j s o n ' After

109. OPTIMIZE JSON RENDERING Newrelic response time is 5.76ms After

110. OPTIMIZE JSON RENDERING 6.28ms vs 5.76ms -8% 3.92ms vs 3.28ms

     -16%

111. OPTIMIZE JSON RENDERING ohler55/oj

112. MEMORY OPTIMIZE

113. https://github.com/rails/rails/pull/20946 shave off 1,114 string objects on every request

114. https://github.com/rails/rails/pull/21057 shave off 34,299 objects on every request

115. FIND IN BATCH P e r s o n .

     w h e r e ( " a g e > 2 1 " ) . e a c h d o | p e r s o n | p e r s o n . p a r t y _ a l l _ n i g h t ! e n d => P e r s o n . w h e r e ( " a g e > 2 1 " ) . f i n d _ e a c h d o | p e r s o n | p e r s o n . p a r t y _ a l l _ n i g h t ! e n d

116. FIND IN BATCH Send multiple sql requests But reduce memory

     usage

117. NO TIP IS ALWAYS TRUE

118. REVIEW Frontend performance tuning does first Backend performance tuning is

     important

119. REVIEW As less requests as possible As small payloads as

     possible As fast resources as possible Other

120. REVIEW Monitor Find out bottleneck Reproduce Fix Benchmark Deploy Monitor

121. DEMOS CODE https://github.com/xinminlabs/rails-performance-tips-code

122. THANK YOU Richard Huang @flyerhzm