Talk given at Montreal.rb on May 20th, 2014
tips and tricksfrom Shopify’s codebase
View Slide
measure it!if it moves...
statsd
Liquid::Template.extend StatsD::Instrument!Liquid::Template.statsd_measure :render,'Liquid.Template.render'
PaymentProcessingJob.statsd_count :perform,'PaymentProcessingJob.processed'
slow queries
# [email protected]: shopify[shopify] @ [127.0.0.1]# Thread_id: 264419969 Schema: shopify Last_errno: 0 Killed: 0# Query_time: 0.150491 Lock_time: 0.000057 Rows_sent: 1 Rows_examined:147841 Rows_affected: 0 Rows_read: 147841# Bytes_sent: 1214 Tmp_tables: 0 Tmp_disk_tables: 0 Tmp_table_sizes: 0# InnoDB_trx_id: FF7021AAA# QC_Hit: No Full_scan: No Full_join: No Tmp_table: NoTmp_table_on_disk: No# Filesort: Yes Filesort_on_disk: No Merge_passes: 0# InnoDB_IO_r_ops: 0 InnoDB_IO_r_bytes: 0 InnoDB_IO_r_wait: 0.000000# InnoDB_rec_lock_wait: 0.000000 InnoDB_queue_wait: 0.000000# InnoDB_pages_distinct: 475SET timestamp=1393385020;SELECT `discounts`.* FROM `discounts` WHERE `discounts`.`shop_id` =1745470 AND `discounts`.`status` = 'enabled' ORDER BY ISNULL(ends_at) DESC,ends_at DESC LIMIT 1
# [email protected]: shopify[shopify] @ [127.0.0.1]# Thread_id: 264419969 Schema: shopify Last_errno: 0 Killed: 0# Query_time: 0.150491 Lock_time: 0.000057 Rows_sent: 1 Rows_examined:147841 Rows_affected: 0 Rows_read: 147841# Bytes_sent: 1214 Tmp_tables: 0 Tmp_disk_tables: 0 Tmp_table_sizes: 0# InnoDB_trx_id: FF7021AAA# QC_Hit: No Full_scan: No Full_join: No Tmp_table: NoTmp_table_on_disk: No# Filesort: Yes Filesort_on_disk: No Merge_passes: 0# InnoDB_IO_r_ops: 0 InnoDB_IO_r_bytes: 0 InnoDB_IO_r_wait: 0.000000# InnoDB_rec_lock_wait: 0.000000 InnoDB_queue_wait: 0.000000# InnoDB_pages_distinct: 475SET timestamp=1393385020;SELECT `discounts`.* FROM `discounts` WHERE `discounts`.`shop_id` =1745470 AND `discounts`.`status` = 'enabled' ORDER BY ISNULL(ends_at) DESC,ends_at DESC LIMIT 1 /*application:Shopify,controller:orders,action:pay*/
determiningroot cause
https://github.com/snormore/nginx-x-rid-headernginx request_id headerproxy_set_header X-Request-ID "$request_id";log_format main '... $request_id'step 1
https://gist.github.com/mnutt/566725Complete 200 OK in 100ms (Views: 60ms |ActiveRecord: 40ms | request_id=bc12813bce...)log_process_actionActionController::Instrumentationstep 2
https://github.com/basecamp/marginaliaUser Load (0.3ms) SELECT `users`.* FROM `users`WHERE `users`.`id` = 1 LIMIT 1/*application:Shopify,controller:users,action:show,request_id:bc12813bce...*/basecamp/marginaliastep 3
# [email protected]: shopify[shopify] @ [127.0.0.1]# Thread_id: 264419969 Schema: shopify Last_errno: 0 Killed: 0# Query_time: 0.150491 Lock_time: 0.000057 Rows_sent: 1 Rows_examined:147841 Rows_affected: 0 Rows_read: 147841# Bytes_sent: 1214 Tmp_tables: 0 Tmp_disk_tables: 0 Tmp_table_sizes: 0# InnoDB_trx_id: FF7021AAA# QC_Hit: No Full_scan: No Full_join: No Tmp_table: NoTmp_table_on_disk: No# Filesort: Yes Filesort_on_disk: No Merge_passes: 0# InnoDB_IO_r_ops: 0 InnoDB_IO_r_bytes: 0 InnoDB_IO_r_wait: 0.000000# InnoDB_rec_lock_wait: 0.000000 InnoDB_queue_wait: 0.000000# InnoDB_pages_distinct: 475SET timestamp=1393385020;SELECT `discounts`.* FROM `discounts` WHERE `discounts`.`shop_id` =1745470 AND `discounts`.`status` = 'enabled' ORDER BY ISNULL(ends_at) DESC,ends_at DESC LIMIT 1 /*application:Shopify,controller:orders,action:pay,request_id:bc12813bce...*/profit!
access.lograils.logslow_query.logprofit! (2)
bonus!background jobs
schema migrationwith zero downtime
orders 20140520_orders
insert/delete/update triggers
INSERT INTO ... SELECT ...insert/delete/update triggers
asynccaveat
tracking exceptions
thanks! :)