Upgrade to Pro — share decks privately, control downloads, hide ads and more …

How to build a high performance feed in Rails

Carlos Donderis
October 22, 2018
530

How to build a high performance feed in Rails

Carlos Donderis

October 22, 2018
Tweet

Transcript

  1. Agenda - Introduction / 自己紹介 - About activity feeds /

    アクティビティフィードについて - Architecture / アーキテクチャ - Rails implementation / Rails実装
  2. Carlos Donderis Sansan エンジニア • 10年間Rubyist • Crystal, Python, Javascriptと遊ぶ

    • ElixirとGoを学んでいます。 • 趣味:写真、空手、居合 Introduction / 自己紹介
  3. Architecture / アーキテクチャ 配信は何ですか? 英語:Fan-out “In message-oriented middleware solutions, fan-out

    is a messaging pattern used to model an information exchange that implies the delivery (or spreading) of a message to one or multiple destinations possibly in parallel, and not halting the process that executes the messaging to wait for any response to that message.” Wikipedia: https://en.wikipedia.org/wiki/Fan-out_(software)
  4. 書き込み時の配信: メリット - 読み込みは早い - データを非正規化できる場合、実装が簡単 デメリット - DBの書き込み負荷 -

    配信は時間がかかります。 (realtimeは厳しい) - 投稿の変更ができる場合、データ更新が大変 - 無駄データが多い(inactiveユーザーなど) Architecture / アーキテクチャ
  5. モデル実装 class User < ApplicationRecord has_many :posts, dependent: :destroy has_many

    :connections, dependent: :destroy has_many :follower_connections, foreign_key: :following_id, class_name: 'Connection' has_many :followers, through: :follower_connections, source: :follower has_many :following_connections, foreign_key: :follower_id, class_name: 'Connection' has_many :following, through: :following_connections, source: :following end class Post < ApplicationRecord belongs_to :user end class Connection < ApplicationRecord belongs_to :follower, class_name: 'User', foreign_key: :follower_id belongs_to :following, class_name: 'User', foreign_key: :following_id end
  6. コントローラー実装 Class PostsController < ApplicationController before_action :auth_user def index @posts

    = @user.fetch_feed end end class User < ApplicationRecord def fetch_feed Post.eager_load(:user) .where(user: self.following + [self]) .order('posts.created_at DESC') end end
  7. クエリー user = User.first # => User Load (0.3ms) SELECT

    `users`.* FROM `users` ORDER BY `users`.`id` ASC LIMIT 1 user.followers # => SELECT `users`.* FROM `users` INNER JOIN `connections` ON `users`.`id` = `connections`.`follower_id` WHERE `connections`.`following_id` = 4 user.following # => SELECT `users`.* FROM `users` INNER JOIN `connections` ON `users`.`id` = `connections`.`following_id` WHERE `connections`.`follower_id` = 4 user.fetch_feed # => User Load (0.4ms) SELECT `users`.* FROM `users` INNER JOIN `connections` ON `users`.`id` = `connections`.`following_id` WHERE `connections`.`follower_id` = 4 # => SQL (0.7ms) SELECT ..... FROM `posts` LEFT OUTER JOIN `users` ON `users`.`id` = `posts`.`user_id` WHERE `posts`.`user_id` IN (5, 4) ORDER BY posts.created_at DESC LIMIT 11
  8. 読み込み時配信の敵はこちら: JOIN, INNER JOIN, LEFT OUTER JOIN ... データが増えるJOINは重くなります! けど

    少し改善をしたらしばらく問題がありません。 - DB Index - Pagination - Caching - ...
  9. モデル実装 class Post < ApplicationRecord belongs_to :user after_create_commit :deliver_to_followers def

    cache_key_for(id:) "cached_post/#{id}" end def id_from_cache_key(key:) key.split('/').last end def deliver_to_followers (user.followers + [user]).each do |follower| follower.add_to_cache(post: self) end end end
  10. モデル実装 class User < ApplicationRecord def fetch_feed cache_keys = redis.zrevrange(cache_key,

    0, -1).map { |id| Post.cache_key_for(id: id) } return fetch_read_feed unless cache_keys.any? # Fall back to slow feed if there is no cache Rails.cache.fetch_multi(*cache_keys, expires: 1.minute, race_condition_ttl: 5.seconds) do |key| id = Post.id_from_cache_key(key: key) Post.find_by(id: id) end.values end def fetch_read_feed Post.eager_load(:user) .where(user: self.following + [self]) .order('posts.created_at DESC') end def add_to_cache(post:) redis.zadd(cache_key, post.created_at.to_i, post.id) end end
  11. クエリー user = User.first # => User Load (0.3ms) SELECT

    `users`.* FROM `users` ORDER BY `users`.`id` ASC LIMIT 1 user.fetch_feed # => Post Load (5.9ms) SELECT `posts`.* FROM `posts` WHERE `posts`.`id` = 21 LIMIT 1 # => Post Load (0.7ms) SELECT `posts`.* FROM `posts` WHERE `posts`.`id` = 20 LIMIT 1 # => Post Load (0.6ms) SELECT `posts`.* FROM `posts` WHERE `posts`.`id` = 19 LIMIT 1 # => Post Load (0.3ms) SELECT `posts`.* FROM `posts` WHERE `posts`.`id` = 18 LIMIT 1 # => Post Load (0.3ms) SELECT `posts`.* FROM `posts` WHERE `posts`.`id` = 17 LIMIT 1 # => Post Load (0.5ms) SELECT `posts`.* FROM `posts` WHERE `posts`.`id` = 16 LIMIT 1 # => Post Load (0.4ms) SELECT `posts`.* FROM `posts` WHERE `posts`.`id` = 15 LIMIT 1 # => Post Load (0.3ms) SELECT `posts`.* FROM `posts` WHERE `posts`.`id` = 14 LIMIT 1
  12. クエリー user = User.first # => User Load (0.3ms) SELECT

    `users`.* FROM `users` ORDER BY `users`.`id` ASC LIMIT 1 user.fetch_feed # => Post Load (5.9ms) SELECT `posts`.* FROM `posts` WHERE `posts`.`id` = 21 LIMIT 1 # => Post Load (0.7ms) SELECT `posts`.* FROM `posts` WHERE `posts`.`id` = 20 LIMIT 1 # => Post Load (0.6ms) SELECT `posts`.* FROM `posts` WHERE `posts`.`id` = 19 LIMIT 1 JOINがなくなりました!
  13. Summary / まとめ - フィードはなんですか? - 配信のやりかた: - 書き込み時の配信 -

    読み込み時の配信 - ミクス - Railsで簡単な実装 - Redis - RailsのCache