Tweet
Tweet

Slide 1

Slide 1 text

tips and tricks from Shopify’s codebase

Slide 2

Slide 2 text

No content

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

measure it! if it moves...

Slide 6

Slide 6 text

statsd

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

Liquid::Template.extend StatsD::Instrument ! Liquid::Template.statsd_measure :render, 'Liquid.Template.render'

Slide 9

Slide 9 text

PaymentProcessingJob.statsd_count :perform, 'PaymentProcessingJob.processed'

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

slow queries

Slide 13

Slide 13 text

# User@Host: shopify[shopify] @ [127.0.0.1] # Thread_id: 264419969 Schema: shopify Last_errno: 0 Killed: 0 # Query_time: 0.150491 Lock_time: 0.000057 Rows_sent: 1 Rows_examined: 147841 Rows_affected: 0 Rows_read: 147841 # Bytes_sent: 1214 Tmp_tables: 0 Tmp_disk_tables: 0 Tmp_table_sizes: 0 # InnoDB_trx_id: FF7021AAA # QC_Hit: No Full_scan: No Full_join: No Tmp_table: No Tmp_table_on_disk: No # Filesort: Yes Filesort_on_disk: No Merge_passes: 0 # InnoDB_IO_r_ops: 0 InnoDB_IO_r_bytes: 0 InnoDB_IO_r_wait: 0.000000 # InnoDB_rec_lock_wait: 0.000000 InnoDB_queue_wait: 0.000000 # InnoDB_pages_distinct: 475 SET timestamp=1393385020; SELECT `discounts`.* FROM `discounts` WHERE `discounts`.`shop_id` = 1745470 AND `discounts`.`status` = 'enabled' ORDER BY ISNULL(ends_at) DESC, ends_at DESC LIMIT 1

Slide 14

Slide 14 text

No content

Slide 15

Slide 15 text

# User@Host: shopify[shopify] @ [127.0.0.1] # Thread_id: 264419969 Schema: shopify Last_errno: 0 Killed: 0 # Query_time: 0.150491 Lock_time: 0.000057 Rows_sent: 1 Rows_examined: 147841 Rows_affected: 0 Rows_read: 147841 # Bytes_sent: 1214 Tmp_tables: 0 Tmp_disk_tables: 0 Tmp_table_sizes: 0 # InnoDB_trx_id: FF7021AAA # QC_Hit: No Full_scan: No Full_join: No Tmp_table: No Tmp_table_on_disk: No # Filesort: Yes Filesort_on_disk: No Merge_passes: 0 # InnoDB_IO_r_ops: 0 InnoDB_IO_r_bytes: 0 InnoDB_IO_r_wait: 0.000000 # InnoDB_rec_lock_wait: 0.000000 InnoDB_queue_wait: 0.000000 # InnoDB_pages_distinct: 475 SET timestamp=1393385020; SELECT `discounts`.* FROM `discounts` WHERE `discounts`.`shop_id` = 1745470 AND `discounts`.`status` = 'enabled' ORDER BY ISNULL(ends_at) DESC, ends_at DESC LIMIT 1 /*application:Shopify,controller:orders,action:pay*/

Slide 16

Slide 16 text

determining root cause

Slide 17

Slide 17 text

https://github.com/snormore/nginx-x-rid-header nginx request_id header proxy_set_header X-Request-ID "$request_id"; log_format main '... $request_id' step 1

Slide 18

Slide 18 text

https://gist.github.com/mnutt/566725 Complete 200 OK in 100ms (Views: 60ms | ActiveRecord: 40ms | request_id=bc12813bce...) log_process_action ActionController::Instrumentation step 2

Slide 19

Slide 19 text

https://github.com/basecamp/marginalia User Load (0.3ms) SELECT `users`.* FROM `users` WHERE `users`.`id` = 1 LIMIT 1 /*application:Shopify, controller:users,action:show, request_id:bc12813bce...*/ basecamp/marginalia step 3

Slide 20

Slide 20 text

# User@Host: shopify[shopify] @ [127.0.0.1] # Thread_id: 264419969 Schema: shopify Last_errno: 0 Killed: 0 # Query_time: 0.150491 Lock_time: 0.000057 Rows_sent: 1 Rows_examined: 147841 Rows_affected: 0 Rows_read: 147841 # Bytes_sent: 1214 Tmp_tables: 0 Tmp_disk_tables: 0 Tmp_table_sizes: 0 # InnoDB_trx_id: FF7021AAA # QC_Hit: No Full_scan: No Full_join: No Tmp_table: No Tmp_table_on_disk: No # Filesort: Yes Filesort_on_disk: No Merge_passes: 0 # InnoDB_IO_r_ops: 0 InnoDB_IO_r_bytes: 0 InnoDB_IO_r_wait: 0.000000 # InnoDB_rec_lock_wait: 0.000000 InnoDB_queue_wait: 0.000000 # InnoDB_pages_distinct: 475 SET timestamp=1393385020; SELECT `discounts`.* FROM `discounts` WHERE `discounts`.`shop_id` = 1745470 AND `discounts`.`status` = 'enabled' ORDER BY ISNULL(ends_at) DESC, ends_at DESC LIMIT 1 /*application:Shopify,controller:orders,action:pay, request_id:bc12813bce...*/ profit!

Slide 21

Slide 21 text

access.log rails.log slow_query.log profit! (2)

Slide 22

Slide 22 text

bonus! background jobs

Slide 23

Slide 23 text

schema migration with zero downtime

Slide 24

Slide 24 text

No content

Slide 25

Slide 25 text

orders 20140520_orders

Slide 26

Slide 26 text

insert/delete/update triggers

Slide 27

Slide 27 text

INSERT INTO ... SELECT ... insert/delete/update triggers

Slide 28

Slide 28 text

async caveat

Slide 29

Slide 29 text

tracking exceptions

Slide 30

Slide 30 text

No content

Slide 31

Slide 31 text

No content

Slide 32

Slide 32 text

No content

Slide 33

Slide 33 text

No content

Slide 34

Slide 34 text

thanks! :)