Rails Performance Tips

13c60e2a5806406b537daa542d3495da?s=47 flyerhzm
February 12, 2016

Rails Performance Tips

my presentation in RubyConf Au 2016

13c60e2a5806406b537daa542d3495da?s=128

flyerhzm

February 12, 2016
Tweet

Transcript

  1. RAILS PERFORMANCE TIPS Richard Huang @flyerhzm

  2. WHERE TO START

  3. 80-90% OF THE END-USER RESPONSE TIME IS SPENT ON THE

    FRONTEND High Performance Web Sites by Steve Souders
  4. FRONTEND PERFORMANCE TIPS

  5. Keith and Mario’s Guide to Fast Websites From RubyConf Au

    2013
  6. 1. AS LESS REQUESTS AS POSSIBLE

  7. CONCATENATE CSS & JAVASCRIPT Browser Server application.html a.css b.css c.css

    a.js b.js c.js Before
  8. CONCATENATE CSS & JAVASCRIPT Browser Server application.html application.css application.js After

  9. USE CSS SPRITE Browser Server application.html a.jpg b.jpg c.jpg Before

  10. USE CSS SPRITE Browser Server application.html sprite.jpg After

  11. 2. AS SMALL PAYLOADS AS POSSIBLE

  12. MINIFY AND GZIP application.css (original) => application.css (minified) => application.css.gz

    application.js (original) => application.js (minified) => application.js.gz
  13. COMPRESS IMAGES

  14. COOKIE-LESS DOMAINS H o s t : x i n

    m i n l a b s . c o m R e f e r e r : h t t p s : / / x i n m i n l a b s . c o m C o o k i e : r a c k . s e s s i o n = B A h 7 E U k i D 3 N l c 3 N p b 2 5 f a W Q G O g Z F V E k i R T J j M j I 3 N j E z Y z Q 3 M T Z l M m Y z Z D k y % 0 A M T E z O T h k Z D d m Z j I y Y j N i O D B l Y W I 3 Z m E x Y z U 1 M j R h Z j Y w N 2 F h M T B i M G U w M 2 Y G % 0 A O w B G S S I J Y 3 N y Z g Y 7 A E Z J I i U x Y z l h M z h l Z D B l M 2 V h N G J h N m F l Z D M 1 M z F j Y j R i % 0 A Z W Y x O Q Y 7 A E Z J I g 1 0 c m F j a 2 l u Z w Y 7 A E Z 7 B 0 k i F E h U V F B f V V N F U l 9 B R 0 V O V A Y 7 % 0 A A F R J I i 1 m Y 2 M 3 Z j A 2 O G U 5 N 2 Q 5 N z A 2 Y T g w M D d m O T g 3 M j E w M j J j O T h i M j d l Y T I w % 0 A B j s A R k k i G U h U V F B f Q U N D R V B U X 0 x B T k d V Q U d F B j s A V E k i L W R i Y 2 M y M T Z i M T B j % 0 A M 2 I 1 M m J k N T g 3 Z T B j O D I w N T Z i M m I y Y m E 5 Y j J k N D Y G O w B G S S I R Y W N j Z X N z X 3 R v % 0 A a 2 V u B j s A V E k i d T A w R D I 4 M D A w M D A w Z n J F U S F B U U 1 B U U g x c V 9 M Y 2 J R b G 0 y R X h 5 % 0 A d 3 p q d m 1 P V j N M Z G h N Q S 5 D S F 8 x a V 9 4 Z D M 0 X 1 c 5 c z A w T D g z a H h J V 0 R a Q T B W N W 8 2 % 0 A U F R 1 c l E 5 c T d U W U t J O T Z m U V B o S 2 N f U j E 3 e k l s Q X p 3 N H c G O w B U S S I K Z W 1 h a W w G % 0 A O w B U S S I c Z m x 5 Z X J o e m 0 r c H J v Z E B n b W F p b C 5 j b 2 0 G O w B U S S I L a W R f d X J s B j s A % 0 A V E k i S m h 0 d H B z O i 8 v b G 9 n a W 4 u c 2 F s Z X N m b 3 J j Z S 5 j b 2 0 v a W Q v M D B E M j g w M D A w % 0 A M D B m c k V R R U F Z L z A w N T I 4 M D A w M D A x Z H l r N k F B Q Q Y 7 A F R J I h F p b n N 0 Y W 5 j Z V 9 1 % 0 A c m w G O w B U S S I j a H R 0 c H M 6 L y 9 s a X R h L m 1 5 L n N h b G V z Z m 9 y Y 2 U u Y 2 9 t B j s A V E k i % 0 A D X V z Z X J u Y W 1 l B j s A V E k i H G Z s e W V y a H p t K 3 B y b 2 R A Z 2 1 h a W w u Y 2 Before
  15. COOKIE-LESS DOMAINS H o s t : a s s

    e t s . x i n m i n l a b s . c o m R e f e r e r : h t t p s : / / x i n m i n l a b s . c o m After
  16. 3. AS FAST RESOURCES AS POSSIBLE

  17. USE CDN

  18. 4. OTHER

  19. JAVASCRIPT AT THE BOTTOM

  20. BACKEND PERFORMANCE TIPS

  21. http://www.speedawarenessmonth.com/when-8020- becomes-2080/

  22. HOW Don't guess! Don't guess! Don't guess! Find out the

    bottleneck
  23. MONITOR New Relic Skylight

  24. MONITOR

  25. MONITOR

  26. MONITOR

  27. REPRODUCE Simulate the environment (postgres, memcached, etc.) Simulate the data

  28. REPRODUCE New Relic (Developer Mode) rack-mini-profiler

  29. REPRODUCE

  30. REPRODUCE

  31. REPRODUCE

  32. REPRODUCE

  33. Users Production Database Production Production Production Production Production New Relic

    Developer Mode
  34. FIX

  35. BENCHMARK rails-per est ab / siege

  36. RAILS-PERFTEST r e q u i r e ' t

    e s t _ h e l p e r ' r e q u i r e ' r a i l s / p e r f o r m a n c e _ t e s t _ h e l p ' c l a s s H o m e p a g e T e s t < A c t i o n D i s p a t c h : : P e r f o r m a n c e T e s t # R e f e r t o t h e d o c u m e n t a t i o n f o r a l l a v a i l a b l e o p t i o n s # s e l f . p r o f i l e _ o p t i o n s = { r u n s : 5 , # m e t r i c s : [ : w a l l _ t i m e , : m e m o r y ] , # o u t p u t : ' t m p / p e r f o r m a n c e ' , # f o r m a t s : [ : f l a t ] } t e s t " h o m e p a g e " d o g e t ' / ' e n d e n d
  37. RAILS-PERFTEST B r o w s i n g T

    e s t # t e s t _ h o m e p a g e ( 5 8 m s w a r m u p ) p r o c e s s _ t i m e : 6 3 m s m e m o r y : 8 3 2 . 1 3 K B o b j e c t s : 7 , 8 8 2
  38. AB / SIEGE Monitor with New Relic

  39. SUMMARIZE Monitor Find out bottleneck Reproduce Fix Benchmark Deploy Monitor

  40. 1. AS LESS REQUESTS AS POSSIBLE

  41. N+1 QUERY # c o n t r o l

    l e r @ c o m m e n t s = C o m m e n t . l i m i t ( 1 0 ) # v i e w < % @ c o m m e n t s . e a c h d o | c o m m e n t | % > < % = c o m m e n t . u s e r . u s e r n a m e % > S a i d : < % = c o m m e n t . b o d y % > < % e n d % > Before
  42. N+1 QUERY App Server DB Server SELECT "comments".* FROM "comments"

    LIMIT 10 SELECT "users".* FROM "users" WHERE "users"."id" = 6 LIMIT 1 SELECT "users".* FROM "users" WHERE "users"."id" = 64 LIMIT 1 ...... SELECT "users".* FROM "users" WHERE "users"."id" = 19 LIMIT 1 Before
  43. N+1 QUERY Newrelic response time is 12.3ms Before

  44. N+1 QUERY # c o n t r o l

    l e r @ c o m m e n t s = C o m m e n t . i n c l u d e s ( : u s e r ) . l i m i t ( 1 0 ) # v i e w < % @ c o m m e n t s . e a c h d o | c o m m e n t | % > < % = c o m m e n t . u s e r . u s e r n a m e % > S a i d : < % = c o m m e n t . b o d y % > < % e n d % > After
  45. N+1 QUERY App Server DB Server SELECT "comments".* FROM "comments"

    LIMIT 10 SELECT "users".* FROM "users" WHERE "users"."id" IN (6, 64, 17, 56, 71, 2, 75, 73, 18, 19) After
  46. N+1 QUERY Newrelic response time is 6.81ms After

  47. N+1 QUERY 12.3ms vs 6.81ms -45% 3.88ms vs 0.621ms -84%

  48. N+1 QUERY flyerhzm/bullet

  49. COUNTER CACHE # c o n t r o l

    l e r @ p o s t s = P o s t . l i m i t ( 1 0 ) # v i e w < % @ p o s t s . e a c h d o | p o s t | % > < % = p o s t . t i t l e % > h a s < % = p o s t . c o m m e n t . s i z e % > c o m m e n t s . < % e n d % > Before
  50. COUNTER CACHE App Server DB Server SELECT "posts".* FROM "posts"

    LIMIT 10 SELECT COUNT(*) FROM "comments" WHERE "comments"."post_id" = 1 SELECT COUNT(*) FROM "comments" WHERE "comments"."post_id" = 2 ...... SELECT COUNT(*) FROM "comments" WHERE "comments"."post_id" = 10 Before
  51. COUNTER CACHE Newrelic response time is 12.3ms Before

  52. COUNTER CACHE # m i g r a t i

    o n a d d _ c o l u m n : p o s t s , : c o m m e n t s _ c o u n t , : i n t e g e r P o s t . a l l . e a c h d o | p o s t | P o s t . r e s e t _ c o u n t e r s ( p o s t . i d , : c o m m e n t s ) e n d # m o d e l c l a s s C o m m e n t < A c t i v e R e c o r d : : B a s e b e l o n g s _ t o : p o s t , c o u n t e r _ c a c h e : t r u e e n d After
  53. COUNTER CACHE App Server DB Server SELECT "posts".* FROM "posts"

    LIMIT 10 After
  54. COUNTER CACHE Newrelic response time is 7.04ms After

  55. COUNTER CACHE 12.3ms vs 7.04ms -43% 6.434ms vs 0.511ms -92%

  56. COUNTER CACHE flyerhzm/bullet

  57. BONUS magnusvk/counter_culture

  58. USE GROUP # m o d e l c l

    a s s C o m m e n t b e l o n g s _ t o : p o s t s c o p e : a p p r o v e d , - > { w h e r e ( a p p r o v e d : t r u e ) } e n d c l a s s P o s t h a s _ m a n y : c o m m e n t s d e f a v e r a g e _ r a t i n g c o m m e n t s . a p p r o v e d . a v e r a g e ( : r a t i n g ) e n d e n d Before
  59. USE GROUP # c o n t r o l

    l e r @ p o s t s = P o s t . l i m i t ( 1 0 ) # v i e w < % @ p o s t s . e a c h d o | p o s t | % > < % = p o s t . t i t l e % > a v e r a g e r a t i n g i s < % = p o s t . a v e r a g e _ r a t i n g % > s t a r s < % e n d % > Before
  60. USE GROUP App Server DB Server SELECT "posts".* FROM "posts"

    LIMIT 10 SELECT AVG("comments"."rating") FROM "comments" WHERE "comments"."post_id" = 1 AND "comments"."approved" = "t" SELECT AVG("comments"."rating") FROM "comments" WHERE "comments"."post_id" = 2 AND "comments"."approved" = "t" ...... SELECT AVG("comments"."rating") FROM "comments" WHERE "comments"."post_id" = 10 AND "comments"."approved" = "t" Before
  61. USE GROUP Newrelic response time is 17.4ms Before

  62. USE GROUP # G e m f i l e

    g e m ' e a g e r _ g r o u p ' # m o d e l c l a s s P o s t < A c t i v e R e c o r d : : B a s e h a s _ m a n y : c o m m e n t s d e f i n e _ e a g e r _ g r o u p : a v e r a g e _ r a t i n g , : c o m m e n t s , : a v e r a g e , : r a t i n g , - > { a p p r o v e d } e n d # c o n t r o l l e r @ p o s t s = P o s t . e a g e r _ g r o u p ( : a v e r a g e _ r a t i n g ) . l i m i t ( 1 0 ) After
  63. USE GROUP App Server DB Server SELECT "posts".* FROM "posts"

    LIMIT 10 SELECT AVG("comments"."rating") AS average_rating, post_id AS post_id FROM "comments" WHERE "comments"."approved" = "t" AND "comments"."post_id" IN (1, 2, 3, 4, 5, 6, 7, 8, 9, 10) GROUP BY "comments"."post_id" After
  64. USE GROUP Newrelic response time is 7.48ms After

  65. USE GROUP 17.4ms vs 7.48ms -57% 6.62ms vs 1.55ms -77%

  66. USE GROUP xinminlabs/eager_group

  67. BONUS Postgresql Materialized View

  68. MULTI INSERTS # c o n t r o l

    l e r 1 0 . t i m e s d o P o s t . c r e a t e t i t l e : F a k e r : : L o r e m . s e n t e n c e , b o d y : F a k e r : : H i p s t e r . p a r a g r a p h , u s e r _ i d : r a n d ( 1 0 0 ) + 1 e n d Before
  69. MULTI INSERTS App Server DB Server BEGIN INSERT INTO "posts"

    ("title", "body", "user_id", "created_at", "updated_at") VALUES ($1, $2, $3, $4, $5) RETURNING "id" COMMIT ...... BEGIN INSERT INTO "posts" ("title", "body", "user_id", "created_at", "updated_at") VALUES ($1, $2, $3, $4, $5) RETURNING "id" COMMIT Before
  70. MULTI INSERTS Newrelic response time is 35.4ms Before

  71. MULTI INSERTS # G e m f i l e

    g e m ' a c t i v e r e c o r d - i m p o r t ' # c o n t r o l l e r p o s t s = [ ] 1 0 . t i m e s d o p o s t s < < P o s t . n e w ( t i t l e : F a k e r : : L o r e m . s e n t e n c e , b o d y : F a k e r : : H i p s t e r . p a r a g r a p h , u s e r _ i d : r a n d ( 1 0 0 ) + 1 ) e n d P o s t . i m p o r t p o s t s After
  72. MULTI INSERTS App Server DB Server Class Create Many Without

    Validations Or Callbacks (3.6ms) INSERT INTO "posts" ("id","title","body","user_id") VALUES (nextval('public.posts_id_seq'),'title','body',31), (nextval('public.posts_id_seq'),'title','body',80), (nextval('public.posts_id_seq'),'title','body',73), (nextval('public.posts_id_seq'),'title','body',56), (nextval('public.posts_id_seq'),'title','body',66), (nextval('public.posts_id_seq'),'title','body',19), (nextval('public.posts_id_seq'),'title','body',11), (nextval('public.posts_id_seq'),'title','body',90), (nextval('public.posts_id_seq'),'title','body',90), (nextval('public.posts_id_seq'),'title','body',9) RETURNING id After
  73. MULTI INSERTS Newrelic response time is 11.6ms After

  74. MULTI INSERTS 35.4ms vs 11.6ms -67% 19.39ms vs 2.39ms -88%

  75. MULTI INSERTS zdennis/activerecord-import

  76. MULTI UPDATES P o s t . w h e

    r e ( " c r e a t e d _ a t < ? " , 1 0 . y e a r s . a g o ) . u p d a t e _ a l l ( a r c h i v e : t r u e )
  77. MULTI DELETES P o s t . w h e

    r e ( " c r e a t e d _ a t < ? " , 1 0 . y e a r s . a g o ) . d e s t r o y _ a l l P o s t . w h e r e ( " c r e a t e d _ a t < ? " , 1 0 . y e a r s . a g o ) . d e l e t e _ a l l
  78. 2. AS SMALL PAYLOADS AS POSSIBLE

  79. SELECT DATA YOU REALLY NEED # c o n t

    r o l l e r @ p o s t s = P o s t . l i m i t ( 1 0 ) # v i e w < % @ p o s t s . e a c h d o | p o s t | % > < % = p o s t . t i t l e % > < % e n d % > Before
  80. SELECT DATA YOU REALLY NEED App Server DB Server SELECT

    "posts".* FROM "posts" LIMIT 10 Before
  81. SELECT DATA YOU REALLY NEED Newrelic response time is 4.86ms

    Before
  82. SELECT DATA YOU REALLY NEED # c o n t

    r o l l e r @ p o s t s = P o s t . s e l e c t ( ' t i t l e ' ) . l i m i t ( 1 0 ) After
  83. SELECT DATA YOU REALLY NEED App Server DB Server SELECT

    "posts"."title" FROM "posts" LIMIT 10 After
  84. SELECT DATA YOU REALLY NEED Newrelic response time is 4.54ms

    After
  85. SELECT DATA YOU REALLY NEED 4.86ms vs 4.54ms -7% 0.563ms

    vs 0.421ms -25%
  86. 3. AS FAST RESOURCES AS POSSIBLE

  87. USE CACHE # c o n t r o l

    l e r @ p o s t s = P o s t . l i m i t ( 1 0 ) # v i e w < % @ p o s t s . e a c h d o | p o s t | % > < % = p o s t . u s e r . u s e r n a m e % > s a i d < % = p o s t . t i t l e % > < % e n d % > Before
  88. USE CACHE App Server DB Server SELECT "posts".* FROM "posts"

    LIMIT 10 SELECT "users".* FROM "users" WHERE "users"."id" = 50 SELECT "users".* FROM "users" WHERE "users"."id" = 97 ...... SELECT "users".* FROM "users" WHERE "users"."id" = 9 Before
  89. USE CACHE Newrelic response time is 12.2ms Before

  90. USE CACHE # G e m f i l e

    g e m ' d a l l i ' # c o n f i g / a p p l i c a t i o n . r b c o n f i g . c a c h e _ s t o r e = : m e m _ c a c h e _ s t o r e # v i e w < % @ p o s t s . e a c h d o | p o s t | % > < % c a c h e p o s t d o % > < % = p o s t . u s e r . u s e r n a m e % > s a i d < % = p o s t . t i t l e % > < % e n d % > < % e n d % > After
  91. USE CACHE App Servers Database Memcached SELECT "posts".* FROM "posts"

    LIMIT 10 Post #50 Cache Page Post #97 Cache Page ...... Post #9 Cache Page After
  92. USE CACHE Newrelic response time is 9.97ms After

  93. USE CACHE 12.2ms vs 9.97ms -18% 8.11ms vs 5.79ms -29%

  94. BONUS R a i l s . c a c

    h e . r e a d _ m u l t i https://github.com/n8/multi_fetch_fragments https://github.com/hooopo/second_level_cache
  95. USE ELASTICSEARCH FOR SEARCH

  96. USE REDIS

  97. USE SSD

  98. 4. OTHER

  99. USE INDEX # c o n t r o l

    l e r @ p o s t s = P o s t . o r d e r ( ' c r e a t e d _ a t d e s c ' ) . l i m i t ( 1 0 ) # v i e w < % @ p o s t s . e a c h d o | p o s t | % > < % = p o s t . t i t l e % > < % e n d % > Before
  100. USE INDEX S E L E C T " p

    o s t s " . * F R O M " p o s t s " O R D E R B Y c r e a t e d _ a t d e s c L I M I T 1 0 Before
  101. USE INDEX Newrelic response time is 21.5ms Before

  102. USE INDEX # m i g r a t i

    o n d e f c h a n g e a d d _ i n d e x : p o s t s , : c r e a t e d _ a t e n d After
  103. USE INDEX Newrelic response time is 4.95ms After

  104. USE INDEX 21.5ms vs 4.95ms -77% 17.3ms vs 0.595ms -97%

  105. USE INDEX plentz/lol_dba

  106. OPTIMIZE JSON RENDERING # c o n t r o

    l l e r @ p o s t s = P o s t . l i m i t ( 1 0 ) r e n d e r j s o n : @ p o s t s Before
  107. OPTIMIZE JSON RENDERING Newrelic response time is 6.28ms Before

  108. OPTIMIZE JSON RENDERING # G e m f i l

    e g e m ' o j ' g e m ' o j _ m i m i c _ j s o n ' After
  109. OPTIMIZE JSON RENDERING Newrelic response time is 5.76ms After

  110. OPTIMIZE JSON RENDERING 6.28ms vs 5.76ms -8% 3.92ms vs 3.28ms

    -16%
  111. OPTIMIZE JSON RENDERING ohler55/oj

  112. MEMORY OPTIMIZE

  113. https://github.com/rails/rails/pull/20946 shave off 1,114 string objects on every request

  114. https://github.com/rails/rails/pull/21057 shave off 34,299 objects on every request

  115. FIND IN BATCH P e r s o n .

    w h e r e ( " a g e > 2 1 " ) . e a c h d o | p e r s o n | p e r s o n . p a r t y _ a l l _ n i g h t ! e n d => P e r s o n . w h e r e ( " a g e > 2 1 " ) . f i n d _ e a c h d o | p e r s o n | p e r s o n . p a r t y _ a l l _ n i g h t ! e n d
  116. FIND IN BATCH Send multiple sql requests But reduce memory

    usage
  117. NO TIP IS ALWAYS TRUE

  118. REVIEW Frontend performance tuning does first Backend performance tuning is

    important
  119. REVIEW As less requests as possible As small payloads as

    possible As fast resources as possible Other
  120. REVIEW Monitor Find out bottleneck Reproduce Fix Benchmark Deploy Monitor

  121. DEMOS CODE https://github.com/xinminlabs/rails-performance-tips-code

  122. THANK YOU Richard Huang @flyerhzm