Scaling Rails for Black Friday / Cyber Monday at Shopify

Scaling Rails for Black Friday / Cyber Monday at Shopify

Talk given at ConFoo 2015 on February 18th, 2015 and RailsConf 2015.

85b03650a2ec5235376b0b983a49511a?s=128

Christian Joudrey

February 18, 2015
Tweet

Transcript

  1. 3.
  2. 4.
  3. 5.
  4. 6.
  5. 7.
  6. 11.
  7. 12.
  8. 17.
  9. 22.
  10. 23.
  11. 27.

    class PostsController < ApplicationController def index response_cache do @posts =

    @shop.posts.paginate(params[:page]) respond_with(@posts) end end def cache_key_data { shop_id: @shop.id, path: request.path, format: request.format, params: params.slice(:page), shop_version: @shop.version } end end
  12. 28.

    md5( { shop_id: 1, path: '/posts', format: 'text/html', params: {

    page: 2 }, shop_version: 123 }.to_s ) GET /posts?page=2
  13. 29.
  14. 30.
  15. 36.

    class Product < ActiveRecord::Base include IdentityCache has_many :images cache_has_many :images,

    :embed => true end product = Product.fetch(id) images = product.fetch_images
  16. 37.

    class Product < ActiveRecord::Base include IdentityCache cache_index :shop_id, :handle, :unique

    => true end Product.fetch_by_shop_id_and_handle(shop_id, handle)
  17. 38.
  18. 39.
  19. 42.
  20. 45.
  21. 47.
  22. 51.
  23. 52.
  24. 58.

    # User@Host: shopify[shopify] @ [127.0.0.1] # Thread_id: 264419969 Schema: shopify

    Last_errno: 0 Killed: 0 # Query_time: 0.150491 Lock_time: 0.000057 Rows_sent: 1 Rows_examined: 147841 Rows_affected: 0 Rows_read: 147841 # Bytes_sent: 1214 Tmp_tables: 0 Tmp_disk_tables: 0 Tmp_table_sizes: 0 # InnoDB_trx_id: FF7021AAA # QC_Hit: No Full_scan: No Full_join: No Tmp_table: No Tmp_table_on_disk: No # Filesort: Yes Filesort_on_disk: No Merge_passes: 0 # InnoDB_IO_r_ops: 0 InnoDB_IO_r_bytes: 0 InnoDB_IO_r_wait: 0.000000 # InnoDB_rec_lock_wait: 0.000000 InnoDB_queue_wait: 0.000000 # InnoDB_pages_distinct: 475 SET timestamp=1393385020; SELECT `discounts`.* FROM `discounts` WHERE `discounts`.`shop_id` = 1745470 AND `discounts`.`status` = 'enabled' ORDER BY ISNULL(ends_at) DESC, ends_at DESC LIMIT 1
  25. 61.

    https://gist.github.com/mnutt/566725 Complete 200 OK in 100ms (Views: 60ms | ActiveRecord:

    40ms | request_id=bc12813bce...) log_process_action ActionController::Instrumentation step 2
  26. 62.

    https://github.com/basecamp/marginalia User Load (0.3ms) SELECT `users`.* FROM `users` WHERE `users`.`id`

    = 1 LIMIT 1 /*application:Shopify, controller:users,action:show, request_id:bc12813bce...*/ basecamp/marginalia step 3
  27. 63.

    # User@Host: shopify[shopify] @ [127.0.0.1] # Thread_id: 264419969 Schema: shopify

    Last_errno: 0 Killed: 0 # Query_time: 0.150491 Lock_time: 0.000057 Rows_sent: 1 Rows_examined: 147841 Rows_affected: 0 Rows_read: 147841 # Bytes_sent: 1214 Tmp_tables: 0 Tmp_disk_tables: 0 Tmp_table_sizes: 0 # InnoDB_trx_id: FF7021AAA # QC_Hit: No Full_scan: No Full_join: No Tmp_table: No Tmp_table_on_disk: No # Filesort: Yes Filesort_on_disk: No Merge_passes: 0 # InnoDB_IO_r_ops: 0 InnoDB_IO_r_bytes: 0 InnoDB_IO_r_wait: 0.000000 # InnoDB_rec_lock_wait: 0.000000 InnoDB_queue_wait: 0.000000 # InnoDB_pages_distinct: 475 SET timestamp=1393385020; SELECT `discounts`.* FROM `discounts` WHERE `discounts`.`shop_id` = 1745470 AND `discounts`.`status` = 'enabled' ORDER BY ISNULL(ends_at) DESC, ends_at DESC LIMIT 1 /*application:Shopify,controller:orders,action:pay, request_id:bc12813bce...*/ profit!
  28. 68.

    class NewIndexOnOrders < ActiveRecord::Migration def self.up Lhm.change_table :orders do |m|

    m.add_index [:shop_id, :customer_id] end end def self.down # end end
  29. 74.

    A resilient system is one that functions with one or

    more components being unavailable or unacceptably slow. -
  30. 82.

    shard 2 shard 3 shard 1 shop 4, 5, 6

    shop 7, 8, 9 shop 1, 2, 3
  31. 83.

    rails request shard 2 shard 3 shard 1 shop 4,

    5, 6 shop 7, 8, 9 shop 1, 2, 3
  32. 84.

    rails request shard 2 shard 3 shard 1 shop 4,

    5, 6 shop 7, 8, 9 shop 1, 2, 3
  33. 89.

    rails request shard 2 shard 3 shard 1 shop 4,

    5, 6 shop 7, 8, 9 shop 1, 2, 3
  34. 91.

    Shopify Shipping rate providers Payment gateways Fulfillment services (FedEX, UPS,

    USPS, etc..) (Stripe, PayPal, etc..) (Shipwire, etc…) Internal services (MySQL, Memcached, etc..)