RubyConfBy 2018 talk about WebPerformance with Rails 5.2

A talk I gave at https://rubyconference.by in Minsk.

Transcript

1. W E B P E R F W I T H R A I L S 5 . 2
   S T E FA N W I N T E R M E Y E R
   Mt. Everest
   

   View Slide

2. Questions? Anytime!
   It’s going to be a bumpy ride.
   Slides: https://speakerdeck.com/wintermeyer
   

   View Slide

3. What's the impact of slow sites?
   Lower conversions and engagement, higher bounce rates...
   Ilya Grigorik @igrigorik
   Make The Web Faster, Google
   

   View Slide

4. Yo ho ho and a few billion pages of RUM
   How speed affects bounce rate
   @igrigorik
   

   View Slide

5. Usability Engineering 101
   Delay User reaction
   0 - 100 ms Instant
   100 - 300 ms Feels sluggish
   300 - 1000 ms Machine is working...
   1 s+ Mental context switch
   10 s+ I'll come back later...
   Stay under 250
   ms to feel "fast".
   Stay under 1000
   ms to keep users
   attention.
   @igrigorik
   

   View Slide

6. Web Search Delay
   Experiment
   Type of Delay Delay (ms)
   Duration
   (weeks)
   Impact on Avg.
   Daily Searches
   Pre-header 100 4 -0.20 %
   Pre-header 200 6 -0.59%
   Post-header 400 6 0.59%
   Post-ads 200 4 0.30%
   Source: https://www.igvita.com/slides/2012/webperf-crash-course.pdf
   

   View Slide

7. For many, mobile is the one and only internet
   device!
   Country Mobile-only users
   Egypt 70%
   India 59%
   South Africa 57%
   Indonesia 44%
   United States 25%
   onDevice Research
   @igrigorik
   

   View Slide

8. < 1.000 ms Page Loading Time on
   3G ist der Mount Everest.
   

   View Slide

9. The (short) life of our 1000 ms budget
   3G (200 ms
   RTT)
   4G(80 ms RTT)
   Control plane (200-2500 ms) (50-100 ms)
   DNS lookup 200 ms 80 ms
   TCP Connection 200 ms 80 ms
   TLS handshake (200-400 ms) (80-160 ms)
   HTTP request 200 ms 80 ms
   Leftover budget 0-400 ms 500-760 ms
   Network overhead
   of one HTTP
   request!
   @igrigorik
   

   View Slide

10. Some WebPerf Problems
    can’t be fixed within Rails.
    If your page initially loads
    3 MB of JavaScript it will
    never be fast.
    

    View Slide

11. Webpage Rendering
    Basics
    

    View Slide

12. Network
    HTML
    DOM
    CSS
    CSSOM JavaScript
    Render tree
    Layout
    Paint
    

    View Slide

13. Network
    HTML
    DOM
    CSS
    CSSOM JavaScript
    Render tree
    Layout
    Paint
    

    View Slide

14. Network
    HTML
    DOM
    CSS
    CSSOM JavaScript
    Render tree
    Layout
    Paint
    

    View Slide

15. Network
    HTML
    DOM
    CSS
    CSSOM JavaScript
    Render tree
    Layout
    Paint
    

    View Slide

16. Network
    HTML
    DOM
    CSS
    CSSOM JavaScript
    Render tree
    Layout
    Paint
    

    View Slide

17. This rendering process
    takes a minimum of 100 ms
    which we have to subtract
    from the 1,000 ms.
    

    View Slide

18. Download a file with
    HTTP 1.1 over TCP
    

    View Slide

19. Latency
    client
    Zeit
    0 ms
    80 ms
    160 ms
    240 ms
    320 ms
    10 TCP Segmente (14.600 Bytes)
    20 TCP Segmente (29.200 Bytes)
    40 TCP Segmente (15.592 Bytes)
    server
    SYN
    ACK
    ACK
    GET
    SYN,ACK
    ACK
    ACK
    

    View Slide

20. TCP Slow-Start
    KB
    0
    55
    110
    165
    220
    Roundtrip
    1. 2. 3. 4.
    214KB
    100KB
    43KB
    14KB
    114KB
    57KB
    29KB
    14KB
    

    View Slide

21. HTTP 2 you can parallel
    download multiple files over
    the same TCP connection.
    Plus better header compression. Plus push, …
    More on that later.
    

    View Slide

22. Waterfall
    www.webpagetest.org
    https://rubyconference.by
    

    View Slide

23. 3G Run => https://www.webpagetest.org/result/
    180412_BK_4b5755e7d0717d0f218a88edf5a691a9/
    

    View Slide

24. View Slide

25. View Slide

26. View Slide

27. LTE Run => https://www.webpagetest.org/result/
    180412_6W_0697a100d965b385b53917b77d51d081/
    

    View Slide

28. View Slide

29. View Slide

30. LTE run => https://www.webpagetest.org/result/
    180316_5B_de436fb724593d7b746b9c1f89ff3c2d/
    

    View Slide

31. View Slide

32. View Slide

33. View Slide

34. Example Online Shop
    

    View Slide

35. $ rails new shop
    $ cd shop
    $ rails g scaffold Category name
    $ rails g scaffold Product category:references
    name description
    price:decimal{8,2}
    $ rails g scaffold User email first_name
    last_name password_digest
    $ rails g scaffold Review user:references
    product:references
    rating:integer
    $ rails db:migrate
    

    View Slide

36. Shop domain model
    Category
    name string
    Product
    description text
    name string
    price decimal (8,2)
    Review
    rating integer
    User
    email string
    first_name string
    last_name string
    password_digest string
    

    View Slide

37. app/models/product.rb:
    class Product < ApplicationRecord
    belongs_to :category
    has_many :reviews
    def number_of_stars
    if reviews.any?
    reviews.average(:rating).round
    else
    nil
    end
    end
    end
    

    View Slide

38. db/seeds.rb:
    Category.create(name: "A")
    Category.create(name: "B")
    Category.create(name: "C")
    100.times do
    Product.create(name: Faker::Food.dish,
    description: Faker::Food.description,
    category: Category.all.sample,
    price: rand(20))
    end
    50.times do
    user = User.create(first_name: Faker::Name.first_name,
    last_name: Faker::Name.last_name)
    products = Product.all
    3.times do
    Review.create(user: user,
    product: products.sample,
    rating: rand(5))
    end
    end
    

    View Slide

39. app/views/products/index.html.erb:
    
    […]
    
    <% @products.each do |product| %>
    
    <%= product.category.name %>
    <%= product.name %>
    <%= product.description %>
    <%= number_to_currency(product.price) %>
    
    <% product.number_of_stars.to_i.times do %>
    

    View Slide

40. View Slide

41. Completed 200 OK in
    497ms (Views: 460.4ms |
    ActiveRecord: 34.1ms)
    Development env.
    

    View Slide

42. The (short) life of our 1000 ms budget
    3G (200 ms
    RTT)
    4G(80 ms RTT)
    Control plane (200-2500 ms) (50-100 ms)
    DNS lookup 200 ms 80 ms
    TCP Connection 200 ms 80 ms
    TLS handshake (200-400 ms) (80-160 ms)
    HTTP request 200 ms 80 ms
    Leftover budget 0-400 ms 500-760 ms
    Network overhead
    of one HTTP
    request!
    @igrigorik
    497ms
    And we don’t have product images yet.
    

    View Slide

43. Development environment?
    Are you crazy?!

    

    Hold your horses!
    I’ll switch to production
    environment later.
    

    View Slide

44. Fragment Caching
    

    View Slide

45. $ rails dev:cache
    Development mode is now being cached.
    By default caching is disabled in dev env.
    

    View Slide

46. 1. Step
    Single Row
    

    View Slide

47. View Slide

48. View Slide

49. app/views/products/index.html.erb:
    
    <% @products.each do |product| %>
    <% cache product do %>
    
    <%= product.category.name %>
    <%= product.name %>
    <%= product.description %>
    <%= number_to_currency(product.price) %>
    
    <% product.number_of_stars.to_i.times do %>
    
    <% end %>
    
    [...]
    
    <% end %>
    <% end %>
    
    

    View Slide

50. app/views/products/index.html.erb:
    
    <% @products.each do |product| %>
    <% cache product do %>
    
    <%= product.category.name %>
    <%= product.name %>
    <%= product.description %>
    <%= number_to_currency(product.price) %>
    
    <% product.number_of_stars.to_i.times do %>
    
    <% end %>
    
    [...]
    
    <% end %>
    <% end %>
    
    

    View Slide

51. app/models/review.rb:
    class Review < ApplicationRecord
    belongs_to :user
    belongs_to :product, touch: true
    end
    Product
    description text
    name string
    price decimal (8,2)
    Review
    rating integer
    User
    email string
    first_name string
    last_name string
    password_digest string
    

    View Slide

52. Total Views Activerecord
    Vanilla 497ms 460 ms 34 ms
    Fragment Cache Row 79 ms 74,6 ms 1 ms
    

    View Slide

53. 2. Step
    The Complete Table
    => Russian Doll
    

    View Slide

54. View Slide

55. View Slide

56. app/views/products/index.html.erb:
    
    <% cache @products do %>
    <% @products.each do |product| %>
    <% cache product do %>
    
    <%= product.category.name %>
    <%= product.name %>
    <%= product.description %>
    <%= number_to_currency(product.price) %>
    
    <% product.number_of_stars.to_i.times do %>
    
    <% end %>
    
    […]
    
    <% end %>
    <% end %>
    <% end %>
    
    

    View Slide

57. app/views/products/index.html.erb:
    
    <% cache @products do %>
    <% @products.each do |product| %>
    <% cache product do %>
    
    <%= product.category.name %>
    <%= product.name %>
    <%= product.description %>
    <%= number_to_currency(product.price) %>
    
    <% product.number_of_stars.to_i.times do %>
    
    <% end %>
    
    […]
    
    <% end %>
    <% end %>
    <% end %>
    
    

    View Slide

58. Total Views Activerecord
    Vanilla 497ms 460 ms 34 ms
    Fragment Cache Row 79 ms 74,6 ms 1 ms
    Fragment Cache Table 53 ms 49,5 ms 0,6 ms
    

    View Slide

59. Use the Database!
    

    View Slide

60. app/models/product.rb:
    class Product < ApplicationRecord
    belongs_to :category
    has_many :reviews
    def number_of_stars
    if reviews.any?
    reviews.average(:rating).round
    else
    nil
    end
    end
    end
    

    View Slide

61. app/models/product.rb:
    class Product < ApplicationRecord
    belongs_to :category
    has_many :reviews
    end
    $ rails g migration AddNumberOfStarsToProduct
    number_of_stars:integer
    $ rails db:migrate
    

    View Slide

62. app/models/product.rb:
    class Review < ApplicationRecord
    belongs_to :user
    belongs_to :product, touch: true
    after_create :recalculate_product_rating
    after_destroy :recalculate_product_rating
    private
    def recalculate_product_rating
    rating = product.reviews.average(:rating).round
    if rating != self.product.number_of_stars
    product.update_attribute(:number_of_stars, rating)
    end
    end
    end
    

    View Slide

63. Total Views Activerecord
    Vanilla 497ms 460 ms 34 ms
    Fragment Cache Row 79 ms 74,6 ms 1 ms
    Fragment Cache Table 53 ms 49,5 ms 0,6 ms
    Plus Database Improvents 40 ms 39 ms 0,5 ms
    Production Env. 35 ms 34 ms 0,5 ms
    

    View Slide

64. Warning: 

    Fragment Caching is
    slower the 1st request!
    Why? Rails checks if the Fragment Cache exists.
    When it doesn’t it renders the view and writes the cache
    which is time consuming.
    

    View Slide

65. Need More Speed?
    Have a look at http://phoenixframework.org
    Phoenix takes 5 ms for the same page.
    BTW: without caching
    

    View Slide

66. HTTP Caching
    

    View Slide

67. Web browsers and proxies
    don‘t want to fetch identical
    resources multiple times.
    

    View Slide

68. The idea of Etags and
    Last-Modified
    

    View Slide

69. Web browser:
    „My user wants to fetch xyz.html.
    I cached a copy last week.
    Is that still good?“
    

    View Slide

70. Web server:
    „xyz.html hasn‘t changed since
    last week.
    Go a head with your copy!“
    aka 304 Not Modified
    

    View Slide

71. > curl -I http://0.0.0.0:3000/products
    HTTP/1.1 200 OK
    Content-Type: text/html; charset=utf-8
    X-Ua-Compatible: IE=Edge
    Etag: "9a779b80e4b0ac3c60d29807e302deb7"
    [...]
    > curl -I http://0.0.0.0:3000/products
    HTTP/1.1 200 OK
    Content-Type: text/html; charset=utf-8
    X-Ua-Compatible: IE=Edge
    Etag: "fa8fc1e981833a6885b583d351c4d823"
    

    View Slide

72. > curl -I http://0.0.0.0:3000/products
    HTTP/1.1 200 OK
    Content-Type: text/html; charset=utf-8
    X-Ua-Compatible: IE=Edge
    Etag: "9a779b80e4b0ac3c60d29807e302deb7"
    [...]
    > curl -I http://0.0.0.0:3000/products
    HTTP/1.1 200 OK
    Content-Type: text/html; charset=utf-8
    X-Ua-Compatible: IE=Edge
    Etag: "fa8fc1e981833a6885b583d351c4d823"
    

    View Slide

73. Set the Etag
    class ProductsController <
    ApplicationController
    # GET /products
    def index
    @products = Product.all
    fresh_when :etag => @products
    end
    [...]

    

    View Slide

74. > curl -I http://0.0.0.0:3000/products -c
    cookies.txt
    HTTP/1.1 200 OK
    Etag: "4d348810e69400799e2ab684c0ef4777"
    > curl -I http://0.0.0.0:3000/products -b
    cookies.txt
    HTTP/1.1 200 OK
    Etag: "4d348810e69400799e2ab684c0ef4777"
    The cookie is needed for the CSRF-Token.
    

    View Slide

75. > curl -I http://0.0.0.0:3000/products -b
    cookies.txt --header 'If-None-Match:
    "4d348810e69400799e2ab684c0ef4777"'
    HTTP/1.1 304 Not Modified
    Etag: "4d348810e69400799e2ab684c0ef4777"
    304!
    

    View Slide

76. Win-Win of a 304
    • The Browser doesn’t have to
    download everything.
    • The Server doesn’t have to
    render the view which is the
    most time consuming bit.
    

    View Slide

77. Not good enough?
    

    View Slide

78. Writing the initial cache
    wastes a lot of time.
    

    View Slide

79. Let‘s preheat the cache
    in off business hours!
    Cron is your friend.
    

    View Slide

80. Use the night to
    preheat your cache.
    And don‘t be afraid
    of brute force!
    

    View Slide

81. A U T O B A H N
    

    View Slide

82. The fastest page is
    delivered by Nginx
    without ever contacting
    Ruby on Rails.
    

    View Slide

83. ├── Gemfile
    ├── [...]
    ├── public
    │ ├── 404.html
    │ ├── 422.html
    │ ├── 500.html
    │ ├── favicon.ico
    │ └── robots.txt
    ├── [...]
    That‘s already
    done for the
    files in the
    public directory.
    

    View Slide

84. Is there an easy way to
    save complete pages
    which are rendered by
    the Rails framework?
    

    View Slide

85. Add caches_page to your
    controller to save views as static
    gz files in your public directory:
    caches_page :index, :show,
    :gzip => :true
    Add gem actionpack-page_caching for Rails 5.2
    

    View Slide

86. Brute Force is your friend!
    During the night the server has a
    hard time to stay awake any way.
    

    View Slide

87. Tricky part:
    How to delete out of date
    gz files?
    

    View Slide

88. after_update :expire_cache
    before_destroy :expire_cache
    private
    def expire_cache
    ActionController::Base.expire_page(Rails.application.routes.url_h
    elpers.company_path(self))
    ActionController::Base.expire_page(Rails.application.routes.url_h
    elpers.companies_path)
    end
    app/models/product.rb
    

    View Slide

89. caches_page
    vs.
    !current_user.nil?
    ???
    

    View Slide

90. caches_page is good to cache
    customized user content too.
    It just takes more thinking.
    

    View Slide

91. Let us assume a user base
    of 10,000,000 people.
    

    View Slide

92. /tmp ᐅ wget http://www.railsconf.com/2013/talks
    --2013-04-27 21:04:24-- http://www.railsconf.com/2013/talks
    Resolving www.railsconf.com... 107.20.162.205
    Connecting to www.railsconf.com|107.20.162.205|:80... connected.
    HTTP request sent, awaiting response... 200 OK
    Length: unspecified [text/html]
    Saving to: ‘talks’
    [ <=> ] 74,321 258KB/
    s in 0.3s
    2013-04-27 21:04:25 (258 KB/s) - ‘talks’ saved [74321]
    /tmp ᐅ du -hs talks
    76K talks
    /tmp ᐅ gzip talks
    /tmp ᐅ du -hs talks.gz
    28K talks.gz
    /tmp ᐅ
    

    View Slide

93. /tmp ᐅ du -hs talks.gz
    28K talks.gz
    28K * 10,000,000 = 0,26 TB
    

    View Slide

94. 28K * 10,000,000 = 0,26 TB
    Harddrive space is cheap.
    By saving the files non-gz and using
    a data deduplication file system you
    just need 5-10% of the 0,26 TB.
    Nginx can gzip the files on the fly.
    

    View Slide

95. Nginx will happily read a
    cookie and find the pre-
    rendered page in a given
    directory structure.
    

    View Slide

96. To setup a complex page_cache
    system is a lot of work. You have to
    tackle not only Rails but Nginx too.
    It does increase the snappiness of
    your application but might not be
    worth the effort for small systems.
    

    View Slide

97. HTTP/1.1 vs. HTTP/2
    

    View Slide

98. HTTP/2 provides an average
    WebPerformance improvement
    of 20%.
    It’s a no-brainer.
    You have to use it!
    

    View Slide

99. CDNs, HTTP/2 and Rails 5.2
    

    View Slide

100. Long story short:

     In most cases where a CDN made
     sense with HTTP/1.1 it doesn’t
     make sense any more.
     Just deliver everything from your
     Rails server!
     

     View Slide

101. Active Storage
     

     View Slide

102. Active Storage should be used to
     minimize image file size and to
     deliver different formats to
     different browsers.

     

     JPEG, PNG or WebP
     

     View Slide

103. https://caniuse.com/#feat=webp
     

     View Slide

104. Apache vs. Nginx
     

     View Slide

105. It doesn’t matter!
     Just use your favorite
     one.
     

     View Slide

106. Brotli vs. gzip
     

     View Slide

107. Always offer both!
     Brotli can be used to
     save bandwidth and
     CPU-Resources.
     

     View Slide

108. Heroku vs. Bare Metal
     

     View Slide

109. Heroku is good for a quick start
     but has never been a good choice
     for good WebPerformance. Bare
     Metal is the way to go if you need
     maximum WebPerformance.

     

     BTW: It’s cheaper too.
     

     View Slide

110. P R E L O A D I N G U N D
     P R E F E T C H I N G
     

     View Slide

111. P R E L O A D I N G U N D P R E F E T C H I N G
     DNS pre-resolution
     TCP pre-connect
     prefresh
     preloader
     

     View Slide

112. M A N U A L D N S - P R E F E T C H
     
     http://www.chromium.org/developers/design-documents/dns-prefetching
     „Most common names like google.com and yahoo.com are resolved so
     often that most local ISP's name resolvers can answer in closer to 80-120ms.
     If the domain name in question is an uncommon name, then a query may
     have to go through numerous resolvers up and down the hierarchy, and the
     delay can average closer to 200-300ms.“
     

     View Slide

113. P R E F E T C H
     
     http://www.whatwg.org/specs/web-apps/current-work/#link-type-prefetch
     „The prefetch keyword indicates that preemptively fetching and caching the
     specified resource is likely to be beneficial, as it is highly likely that the user
     will require this resource.“
     T I P P : " A C C E P T - R A N G E S : B Y T E S “ H E A D E R
     

     View Slide

114. Y O U C A N T E L L N G I N X T O
     P U S H T H O S E F I L E S V I A H T T P / 2 .
     

     View Slide

115. The Most
     Important Tool?
     

     View Slide

116. Set a Time Budget!
     If you run out of your time budget you have to
     cancel features on your website.
     

     View Slide

117. Is WebPerformance really so hard?
     

     View Slide

118. The WebPerf Bible. => https://hpbn.co
     

     View Slide

119. https://amzn.to/2qProVC
     

     View Slide

120. @wintermeyer
     

     View Slide

121. sw@wintermeyer-consulting.de
     last name | twitter | github
     e-mail
     

     View Slide