Slide 1

Slide 1 text

RAILS PERFORMANCE TIPS Richard Huang @flyerhzm

Slide 2

Slide 2 text

WHERE TO START

Slide 3

Slide 3 text

80-90% OF THE END-USER RESPONSE TIME IS SPENT ON THE FRONTEND High Performance Web Sites by Steve Souders

Slide 4

Slide 4 text

FRONTEND PERFORMANCE TIPS

Slide 5

Slide 5 text

Keith and Mario’s Guide to Fast Websites From RubyConf Au 2013

Slide 6

Slide 6 text

1. AS LESS REQUESTS AS POSSIBLE

Slide 7

Slide 7 text

CONCATENATE CSS & JAVASCRIPT Browser Server application.html a.css b.css c.css a.js b.js c.js Before

Slide 8

Slide 8 text

CONCATENATE CSS & JAVASCRIPT Browser Server application.html application.css application.js After

Slide 9

Slide 9 text

USE CSS SPRITE Browser Server application.html a.jpg b.jpg c.jpg Before

Slide 10

Slide 10 text

USE CSS SPRITE Browser Server application.html sprite.jpg After

Slide 11

Slide 11 text

2. AS SMALL PAYLOADS AS POSSIBLE

Slide 12

Slide 12 text

MINIFY AND GZIP application.css (original) => application.css (minified) => application.css.gz application.js (original) => application.js (minified) => application.js.gz

Slide 13

Slide 13 text

COMPRESS IMAGES

Slide 14

Slide 14 text

COOKIE-LESS DOMAINS H o s t : x i n m i n l a b s . c o m R e f e r e r : h t t p s : / / x i n m i n l a b s . c o m C o o k i e : r a c k . s e s s i o n = B A h 7 E U k i D 3 N l c 3 N p b 2 5 f a W Q G O g Z F V E k i R T J j M j I 3 N j E z Y z Q 3 M T Z l M m Y z Z D k y % 0 A M T E z O T h k Z D d m Z j I y Y j N i O D B l Y W I 3 Z m E x Y z U 1 M j R h Z j Y w N 2 F h M T B i M G U w M 2 Y G % 0 A O w B G S S I J Y 3 N y Z g Y 7 A E Z J I i U x Y z l h M z h l Z D B l M 2 V h N G J h N m F l Z D M 1 M z F j Y j R i % 0 A Z W Y x O Q Y 7 A E Z J I g 1 0 c m F j a 2 l u Z w Y 7 A E Z 7 B 0 k i F E h U V F B f V V N F U l 9 B R 0 V O V A Y 7 % 0 A A F R J I i 1 m Y 2 M 3 Z j A 2 O G U 5 N 2 Q 5 N z A 2 Y T g w M D d m O T g 3 M j E w M j J j O T h i M j d l Y T I w % 0 A B j s A R k k i G U h U V F B f Q U N D R V B U X 0 x B T k d V Q U d F B j s A V E k i L W R i Y 2 M y M T Z i M T B j % 0 A M 2 I 1 M m J k N T g 3 Z T B j O D I w N T Z i M m I y Y m E 5 Y j J k N D Y G O w B G S S I R Y W N j Z X N z X 3 R v % 0 A a 2 V u B j s A V E k i d T A w R D I 4 M D A w M D A w Z n J F U S F B U U 1 B U U g x c V 9 M Y 2 J R b G 0 y R X h 5 % 0 A d 3 p q d m 1 P V j N M Z G h N Q S 5 D S F 8 x a V 9 4 Z D M 0 X 1 c 5 c z A w T D g z a H h J V 0 R a Q T B W N W 8 2 % 0 A U F R 1 c l E 5 c T d U W U t J O T Z m U V B o S 2 N f U j E 3 e k l s Q X p 3 N H c G O w B U S S I K Z W 1 h a W w G % 0 A O w B U S S I c Z m x 5 Z X J o e m 0 r c H J v Z E B n b W F p b C 5 j b 2 0 G O w B U S S I L a W R f d X J s B j s A % 0 A V E k i S m h 0 d H B z O i 8 v b G 9 n a W 4 u c 2 F s Z X N m b 3 J j Z S 5 j b 2 0 v a W Q v M D B E M j g w M D A w % 0 A M D B m c k V R R U F Z L z A w N T I 4 M D A w M D A x Z H l r N k F B Q Q Y 7 A F R J I h F p b n N 0 Y W 5 j Z V 9 1 % 0 A c m w G O w B U S S I j a H R 0 c H M 6 L y 9 s a X R h L m 1 5 L n N h b G V z Z m 9 y Y 2 U u Y 2 9 t B j s A V E k i % 0 A D X V z Z X J u Y W 1 l B j s A V E k i H G Z s e W V y a H p t K 3 B y b 2 R A Z 2 1 h a W w u Y 2 Before

Slide 15

Slide 15 text

COOKIE-LESS DOMAINS H o s t : a s s e t s . x i n m i n l a b s . c o m R e f e r e r : h t t p s : / / x i n m i n l a b s . c o m After

Slide 16

Slide 16 text

3. AS FAST RESOURCES AS POSSIBLE

Slide 17

Slide 17 text

USE CDN

Slide 18

Slide 18 text

4. OTHER

Slide 19

Slide 19 text

JAVASCRIPT AT THE BOTTOM

Slide 20

Slide 20 text

BACKEND PERFORMANCE TIPS

Slide 21

Slide 21 text

http://www.speedawarenessmonth.com/when-8020- becomes-2080/

Slide 22

Slide 22 text

HOW Don't guess! Don't guess! Don't guess! Find out the bottleneck

Slide 23

Slide 23 text

MONITOR New Relic Skylight

Slide 24

Slide 24 text

MONITOR

Slide 25

Slide 25 text

MONITOR

Slide 26

Slide 26 text

MONITOR

Slide 27

Slide 27 text

REPRODUCE Simulate the environment (postgres, memcached, etc.) Simulate the data

Slide 28

Slide 28 text

REPRODUCE New Relic (Developer Mode) rack-mini-profiler

Slide 29

Slide 29 text

REPRODUCE

Slide 30

Slide 30 text

REPRODUCE

Slide 31

Slide 31 text

REPRODUCE

Slide 32

Slide 32 text

REPRODUCE

Slide 33

Slide 33 text

Users Production Database Production Production Production Production Production New Relic Developer Mode

Slide 34

Slide 34 text

FIX

Slide 35

Slide 35 text

BENCHMARK rails-per est ab / siege

Slide 36

Slide 36 text

RAILS-PERFTEST r e q u i r e ' t e s t _ h e l p e r ' r e q u i r e ' r a i l s / p e r f o r m a n c e _ t e s t _ h e l p ' c l a s s H o m e p a g e T e s t < A c t i o n D i s p a t c h : : P e r f o r m a n c e T e s t # R e f e r t o t h e d o c u m e n t a t i o n f o r a l l a v a i l a b l e o p t i o n s # s e l f . p r o f i l e _ o p t i o n s = { r u n s : 5 , # m e t r i c s : [ : w a l l _ t i m e , : m e m o r y ] , # o u t p u t : ' t m p / p e r f o r m a n c e ' , # f o r m a t s : [ : f l a t ] } t e s t " h o m e p a g e " d o g e t ' / ' e n d e n d

Slide 37

Slide 37 text

RAILS-PERFTEST B r o w s i n g T e s t # t e s t _ h o m e p a g e ( 5 8 m s w a r m u p ) p r o c e s s _ t i m e : 6 3 m s m e m o r y : 8 3 2 . 1 3 K B o b j e c t s : 7 , 8 8 2

Slide 38

Slide 38 text

AB / SIEGE Monitor with New Relic

Slide 39

Slide 39 text

SUMMARIZE Monitor Find out bottleneck Reproduce Fix Benchmark Deploy Monitor

Slide 40

Slide 40 text

1. AS LESS REQUESTS AS POSSIBLE

Slide 41

Slide 41 text

N+1 QUERY # c o n t r o l l e r @ c o m m e n t s = C o m m e n t . l i m i t ( 1 0 ) # v i e w < % @ c o m m e n t s . e a c h d o | c o m m e n t | % > < % = c o m m e n t . u s e r . u s e r n a m e % > S a i d : < % = c o m m e n t . b o d y % > < % e n d % > Before

Slide 42

Slide 42 text

N+1 QUERY App Server DB Server SELECT "comments".* FROM "comments" LIMIT 10 SELECT "users".* FROM "users" WHERE "users"."id" = 6 LIMIT 1 SELECT "users".* FROM "users" WHERE "users"."id" = 64 LIMIT 1 ...... SELECT "users".* FROM "users" WHERE "users"."id" = 19 LIMIT 1 Before

Slide 43

Slide 43 text

N+1 QUERY Newrelic response time is 12.3ms Before

Slide 44

Slide 44 text

N+1 QUERY # c o n t r o l l e r @ c o m m e n t s = C o m m e n t . i n c l u d e s ( : u s e r ) . l i m i t ( 1 0 ) # v i e w < % @ c o m m e n t s . e a c h d o | c o m m e n t | % > < % = c o m m e n t . u s e r . u s e r n a m e % > S a i d : < % = c o m m e n t . b o d y % > < % e n d % > After

Slide 45

Slide 45 text

N+1 QUERY App Server DB Server SELECT "comments".* FROM "comments" LIMIT 10 SELECT "users".* FROM "users" WHERE "users"."id" IN (6, 64, 17, 56, 71, 2, 75, 73, 18, 19) After

Slide 46

Slide 46 text

N+1 QUERY Newrelic response time is 6.81ms After

Slide 47

Slide 47 text

N+1 QUERY 12.3ms vs 6.81ms -45% 3.88ms vs 0.621ms -84%

Slide 48

Slide 48 text

N+1 QUERY flyerhzm/bullet

Slide 49

Slide 49 text

COUNTER CACHE # c o n t r o l l e r @ p o s t s = P o s t . l i m i t ( 1 0 ) # v i e w < % @ p o s t s . e a c h d o | p o s t | % > < % = p o s t . t i t l e % > h a s < % = p o s t . c o m m e n t . s i z e % > c o m m e n t s . < % e n d % > Before

Slide 50

Slide 50 text

COUNTER CACHE App Server DB Server SELECT "posts".* FROM "posts" LIMIT 10 SELECT COUNT(*) FROM "comments" WHERE "comments"."post_id" = 1 SELECT COUNT(*) FROM "comments" WHERE "comments"."post_id" = 2 ...... SELECT COUNT(*) FROM "comments" WHERE "comments"."post_id" = 10 Before

Slide 51

Slide 51 text

COUNTER CACHE Newrelic response time is 12.3ms Before

Slide 52

Slide 52 text

COUNTER CACHE # m i g r a t i o n a d d _ c o l u m n : p o s t s , : c o m m e n t s _ c o u n t , : i n t e g e r P o s t . a l l . e a c h d o | p o s t | P o s t . r e s e t _ c o u n t e r s ( p o s t . i d , : c o m m e n t s ) e n d # m o d e l c l a s s C o m m e n t < A c t i v e R e c o r d : : B a s e b e l o n g s _ t o : p o s t , c o u n t e r _ c a c h e : t r u e e n d After

Slide 53

Slide 53 text

COUNTER CACHE App Server DB Server SELECT "posts".* FROM "posts" LIMIT 10 After

Slide 54

Slide 54 text

COUNTER CACHE Newrelic response time is 7.04ms After

Slide 55

Slide 55 text

COUNTER CACHE 12.3ms vs 7.04ms -43% 6.434ms vs 0.511ms -92%

Slide 56

Slide 56 text

COUNTER CACHE flyerhzm/bullet

Slide 57

Slide 57 text

BONUS magnusvk/counter_culture

Slide 58

Slide 58 text

USE GROUP # m o d e l c l a s s C o m m e n t b e l o n g s _ t o : p o s t s c o p e : a p p r o v e d , - > { w h e r e ( a p p r o v e d : t r u e ) } e n d c l a s s P o s t h a s _ m a n y : c o m m e n t s d e f a v e r a g e _ r a t i n g c o m m e n t s . a p p r o v e d . a v e r a g e ( : r a t i n g ) e n d e n d Before

Slide 59

Slide 59 text

USE GROUP # c o n t r o l l e r @ p o s t s = P o s t . l i m i t ( 1 0 ) # v i e w < % @ p o s t s . e a c h d o | p o s t | % > < % = p o s t . t i t l e % > a v e r a g e r a t i n g i s < % = p o s t . a v e r a g e _ r a t i n g % > s t a r s < % e n d % > Before

Slide 60

Slide 60 text

USE GROUP App Server DB Server SELECT "posts".* FROM "posts" LIMIT 10 SELECT AVG("comments"."rating") FROM "comments" WHERE "comments"."post_id" = 1 AND "comments"."approved" = "t" SELECT AVG("comments"."rating") FROM "comments" WHERE "comments"."post_id" = 2 AND "comments"."approved" = "t" ...... SELECT AVG("comments"."rating") FROM "comments" WHERE "comments"."post_id" = 10 AND "comments"."approved" = "t" Before

Slide 61

Slide 61 text

USE GROUP Newrelic response time is 17.4ms Before

Slide 62

Slide 62 text

USE GROUP # G e m f i l e g e m ' e a g e r _ g r o u p ' # m o d e l c l a s s P o s t < A c t i v e R e c o r d : : B a s e h a s _ m a n y : c o m m e n t s d e f i n e _ e a g e r _ g r o u p : a v e r a g e _ r a t i n g , : c o m m e n t s , : a v e r a g e , : r a t i n g , - > { a p p r o v e d } e n d # c o n t r o l l e r @ p o s t s = P o s t . e a g e r _ g r o u p ( : a v e r a g e _ r a t i n g ) . l i m i t ( 1 0 ) After

Slide 63

Slide 63 text

USE GROUP App Server DB Server SELECT "posts".* FROM "posts" LIMIT 10 SELECT AVG("comments"."rating") AS average_rating, post_id AS post_id FROM "comments" WHERE "comments"."approved" = "t" AND "comments"."post_id" IN (1, 2, 3, 4, 5, 6, 7, 8, 9, 10) GROUP BY "comments"."post_id" After

Slide 64

Slide 64 text

USE GROUP Newrelic response time is 7.48ms After

Slide 65

Slide 65 text

USE GROUP 17.4ms vs 7.48ms -57% 6.62ms vs 1.55ms -77%

Slide 66

Slide 66 text

USE GROUP xinminlabs/eager_group

Slide 67

Slide 67 text

BONUS Postgresql Materialized View

Slide 68

Slide 68 text

MULTI INSERTS # c o n t r o l l e r 1 0 . t i m e s d o P o s t . c r e a t e t i t l e : F a k e r : : L o r e m . s e n t e n c e , b o d y : F a k e r : : H i p s t e r . p a r a g r a p h , u s e r _ i d : r a n d ( 1 0 0 ) + 1 e n d Before

Slide 69

Slide 69 text

MULTI INSERTS App Server DB Server BEGIN INSERT INTO "posts" ("title", "body", "user_id", "created_at", "updated_at") VALUES ($1, $2, $3, $4, $5) RETURNING "id" COMMIT ...... BEGIN INSERT INTO "posts" ("title", "body", "user_id", "created_at", "updated_at") VALUES ($1, $2, $3, $4, $5) RETURNING "id" COMMIT Before

Slide 70

Slide 70 text

MULTI INSERTS Newrelic response time is 35.4ms Before

Slide 71

Slide 71 text

MULTI INSERTS # G e m f i l e g e m ' a c t i v e r e c o r d - i m p o r t ' # c o n t r o l l e r p o s t s = [ ] 1 0 . t i m e s d o p o s t s < < P o s t . n e w ( t i t l e : F a k e r : : L o r e m . s e n t e n c e , b o d y : F a k e r : : H i p s t e r . p a r a g r a p h , u s e r _ i d : r a n d ( 1 0 0 ) + 1 ) e n d P o s t . i m p o r t p o s t s After

Slide 72

Slide 72 text

MULTI INSERTS App Server DB Server Class Create Many Without Validations Or Callbacks (3.6ms) INSERT INTO "posts" ("id","title","body","user_id") VALUES (nextval('public.posts_id_seq'),'title','body',31), (nextval('public.posts_id_seq'),'title','body',80), (nextval('public.posts_id_seq'),'title','body',73), (nextval('public.posts_id_seq'),'title','body',56), (nextval('public.posts_id_seq'),'title','body',66), (nextval('public.posts_id_seq'),'title','body',19), (nextval('public.posts_id_seq'),'title','body',11), (nextval('public.posts_id_seq'),'title','body',90), (nextval('public.posts_id_seq'),'title','body',90), (nextval('public.posts_id_seq'),'title','body',9) RETURNING id After

Slide 73

Slide 73 text

MULTI INSERTS Newrelic response time is 11.6ms After

Slide 74

Slide 74 text

MULTI INSERTS 35.4ms vs 11.6ms -67% 19.39ms vs 2.39ms -88%

Slide 75

Slide 75 text

MULTI INSERTS zdennis/activerecord-import

Slide 76

Slide 76 text

MULTI UPDATES P o s t . w h e r e ( " c r e a t e d _ a t < ? " , 1 0 . y e a r s . a g o ) . u p d a t e _ a l l ( a r c h i v e : t r u e )

Slide 77

Slide 77 text

MULTI DELETES P o s t . w h e r e ( " c r e a t e d _ a t < ? " , 1 0 . y e a r s . a g o ) . d e s t r o y _ a l l P o s t . w h e r e ( " c r e a t e d _ a t < ? " , 1 0 . y e a r s . a g o ) . d e l e t e _ a l l

Slide 78

Slide 78 text

2. AS SMALL PAYLOADS AS POSSIBLE

Slide 79

Slide 79 text

SELECT DATA YOU REALLY NEED # c o n t r o l l e r @ p o s t s = P o s t . l i m i t ( 1 0 ) # v i e w < % @ p o s t s . e a c h d o | p o s t | % > < % = p o s t . t i t l e % > < % e n d % > Before

Slide 80

Slide 80 text

SELECT DATA YOU REALLY NEED App Server DB Server SELECT "posts".* FROM "posts" LIMIT 10 Before

Slide 81

Slide 81 text

SELECT DATA YOU REALLY NEED Newrelic response time is 4.86ms Before

Slide 82

Slide 82 text

SELECT DATA YOU REALLY NEED # c o n t r o l l e r @ p o s t s = P o s t . s e l e c t ( ' t i t l e ' ) . l i m i t ( 1 0 ) After

Slide 83

Slide 83 text

SELECT DATA YOU REALLY NEED App Server DB Server SELECT "posts"."title" FROM "posts" LIMIT 10 After

Slide 84

Slide 84 text

SELECT DATA YOU REALLY NEED Newrelic response time is 4.54ms After

Slide 85

Slide 85 text

SELECT DATA YOU REALLY NEED 4.86ms vs 4.54ms -7% 0.563ms vs 0.421ms -25%

Slide 86

Slide 86 text

3. AS FAST RESOURCES AS POSSIBLE

Slide 87

Slide 87 text

USE CACHE # c o n t r o l l e r @ p o s t s = P o s t . l i m i t ( 1 0 ) # v i e w < % @ p o s t s . e a c h d o | p o s t | % > < % = p o s t . u s e r . u s e r n a m e % > s a i d < % = p o s t . t i t l e % > < % e n d % > Before

Slide 88

Slide 88 text

USE CACHE App Server DB Server SELECT "posts".* FROM "posts" LIMIT 10 SELECT "users".* FROM "users" WHERE "users"."id" = 50 SELECT "users".* FROM "users" WHERE "users"."id" = 97 ...... SELECT "users".* FROM "users" WHERE "users"."id" = 9 Before

Slide 89

Slide 89 text

USE CACHE Newrelic response time is 12.2ms Before

Slide 90

Slide 90 text

USE CACHE # G e m f i l e g e m ' d a l l i ' # c o n f i g / a p p l i c a t i o n . r b c o n f i g . c a c h e _ s t o r e = : m e m _ c a c h e _ s t o r e # v i e w < % @ p o s t s . e a c h d o | p o s t | % > < % c a c h e p o s t d o % > < % = p o s t . u s e r . u s e r n a m e % > s a i d < % = p o s t . t i t l e % > < % e n d % > < % e n d % > After

Slide 91

Slide 91 text

USE CACHE App Servers Database Memcached SELECT "posts".* FROM "posts" LIMIT 10 Post #50 Cache Page Post #97 Cache Page ...... Post #9 Cache Page After

Slide 92

Slide 92 text

USE CACHE Newrelic response time is 9.97ms After

Slide 93

Slide 93 text

USE CACHE 12.2ms vs 9.97ms -18% 8.11ms vs 5.79ms -29%

Slide 94

Slide 94 text

BONUS R a i l s . c a c h e . r e a d _ m u l t i https://github.com/n8/multi_fetch_fragments https://github.com/hooopo/second_level_cache

Slide 95

Slide 95 text

USE ELASTICSEARCH FOR SEARCH

Slide 96

Slide 96 text

USE REDIS

Slide 97

Slide 97 text

USE SSD

Slide 98

Slide 98 text

4. OTHER

Slide 99

Slide 99 text

USE INDEX # c o n t r o l l e r @ p o s t s = P o s t . o r d e r ( ' c r e a t e d _ a t d e s c ' ) . l i m i t ( 1 0 ) # v i e w < % @ p o s t s . e a c h d o | p o s t | % > < % = p o s t . t i t l e % > < % e n d % > Before

Slide 100

Slide 100 text

USE INDEX S E L E C T " p o s t s " . * F R O M " p o s t s " O R D E R B Y c r e a t e d _ a t d e s c L I M I T 1 0 Before

Slide 101

Slide 101 text

USE INDEX Newrelic response time is 21.5ms Before

Slide 102

Slide 102 text

USE INDEX # m i g r a t i o n d e f c h a n g e a d d _ i n d e x : p o s t s , : c r e a t e d _ a t e n d After

Slide 103

Slide 103 text

USE INDEX Newrelic response time is 4.95ms After

Slide 104

Slide 104 text

USE INDEX 21.5ms vs 4.95ms -77% 17.3ms vs 0.595ms -97%

Slide 105

Slide 105 text

USE INDEX plentz/lol_dba

Slide 106

Slide 106 text

OPTIMIZE JSON RENDERING # c o n t r o l l e r @ p o s t s = P o s t . l i m i t ( 1 0 ) r e n d e r j s o n : @ p o s t s Before

Slide 107

Slide 107 text

OPTIMIZE JSON RENDERING Newrelic response time is 6.28ms Before

Slide 108

Slide 108 text

OPTIMIZE JSON RENDERING # G e m f i l e g e m ' o j ' g e m ' o j _ m i m i c _ j s o n ' After

Slide 109

Slide 109 text

OPTIMIZE JSON RENDERING Newrelic response time is 5.76ms After

Slide 110

Slide 110 text

OPTIMIZE JSON RENDERING 6.28ms vs 5.76ms -8% 3.92ms vs 3.28ms -16%

Slide 111

Slide 111 text

OPTIMIZE JSON RENDERING ohler55/oj

Slide 112

Slide 112 text

MEMORY OPTIMIZE

Slide 113

Slide 113 text

https://github.com/rails/rails/pull/20946 shave off 1,114 string objects on every request

Slide 114

Slide 114 text

https://github.com/rails/rails/pull/21057 shave off 34,299 objects on every request

Slide 115

Slide 115 text

FIND IN BATCH P e r s o n . w h e r e ( " a g e > 2 1 " ) . e a c h d o | p e r s o n | p e r s o n . p a r t y _ a l l _ n i g h t ! e n d => P e r s o n . w h e r e ( " a g e > 2 1 " ) . f i n d _ e a c h d o | p e r s o n | p e r s o n . p a r t y _ a l l _ n i g h t ! e n d

Slide 116

Slide 116 text

FIND IN BATCH Send multiple sql requests But reduce memory usage

Slide 117

Slide 117 text

NO TIP IS ALWAYS TRUE

Slide 118

Slide 118 text

REVIEW Frontend performance tuning does first Backend performance tuning is important

Slide 119

Slide 119 text

REVIEW As less requests as possible As small payloads as possible As fast resources as possible Other

Slide 120

Slide 120 text

REVIEW Monitor Find out bottleneck Reproduce Fix Benchmark Deploy Monitor

Slide 121

Slide 121 text

DEMOS CODE https://github.com/xinminlabs/rails-performance-tips-code

Slide 122

Slide 122 text

THANK YOU Richard Huang @flyerhzm