Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Tips and Tricks from Shopify's codebase
Search
Christian Joudrey
May 23, 2014
Programming
2
560
Tips and Tricks from Shopify's codebase
Talk given at Montreal.rb on May 20th, 2014
Christian Joudrey
May 23, 2014
Tweet
Share
More Decks by Christian Joudrey
See All by Christian Joudrey
Writing NES games! with assembly!!
cjoudrey
1
740
Developing at Scale
cjoudrey
3
480
Scaling Rails for Black Friday / Cyber Monday at Shopify
cjoudrey
6
5.9k
Scaling Shopify
cjoudrey
3
530
#pairwithme
cjoudrey
3
250
Two-factor authentication
cjoudrey
4
390
Automate your Infrastructure with Chef
cjoudrey
9
610
Other Decks in Programming
See All in Programming
Kiroで始めるAI-DLC
kaonash
2
490
AI時代に学習する意味はあるのか?
tomoyakamaji
0
100
未来を拓くAI技術〜エージェント開発とAI駆動開発〜
leveragestech
2
200
オープンセミナー2025@広島LT技術ブログを続けるには
satoshi256kbyte
0
150
Microsoft Orleans, Daprのアクターモデルを使い効率的に開発、デプロイを行うためのSekibanの試行錯誤 / Sekiban: Exploring Efficient Development and Deployment with Microsoft Orleans and Dapr Actor Models
tomohisa
0
220
Ruby Parser progress report 2025
yui_knk
1
250
Nuances on Kubernetes - RubyConf Taiwan 2025
envek
0
220
The Past, Present, and Future of Enterprise Java
ivargrimstad
0
280
Oracle Database Technology Night 92 Database Connection control FAN-AC
oracle4engineer
PRO
1
360
サーバーサイドのビルド時間87倍高速化
plaidtech
PRO
0
680
実用的なGOCACHEPROG実装をするために / golang.tokyo #40
mazrean
1
120
[FEConf 2025] 모노레포 절망편, 14개 레포로 부활하기까지 걸린 1년
mmmaxkim
0
1.4k
Featured
See All Featured
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
23
1.4k
GraphQLの誤解/rethinking-graphql
sonatard
71
11k
GraphQLとの向き合い方2022年版
quramy
49
14k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
46
7.6k
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
131
19k
Typedesign – Prime Four
hannesfritz
42
2.8k
Automating Front-end Workflow
addyosmani
1370
200k
Side Projects
sachag
455
43k
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
33
2.4k
Embracing the Ebb and Flow
colly
87
4.8k
Transcript
tips and tricks from Shopify’s codebase
None
None
None
measure it! if it moves...
statsd
None
Liquid::Template.extend StatsD::Instrument ! Liquid::Template.statsd_measure :render, 'Liquid.Template.render'
PaymentProcessingJob.statsd_count :perform, 'PaymentProcessingJob.processed'
None
None
slow queries
# User@Host: shopify[shopify] @ [127.0.0.1] # Thread_id: 264419969 Schema: shopify
Last_errno: 0 Killed: 0 # Query_time: 0.150491 Lock_time: 0.000057 Rows_sent: 1 Rows_examined: 147841 Rows_affected: 0 Rows_read: 147841 # Bytes_sent: 1214 Tmp_tables: 0 Tmp_disk_tables: 0 Tmp_table_sizes: 0 # InnoDB_trx_id: FF7021AAA # QC_Hit: No Full_scan: No Full_join: No Tmp_table: No Tmp_table_on_disk: No # Filesort: Yes Filesort_on_disk: No Merge_passes: 0 # InnoDB_IO_r_ops: 0 InnoDB_IO_r_bytes: 0 InnoDB_IO_r_wait: 0.000000 # InnoDB_rec_lock_wait: 0.000000 InnoDB_queue_wait: 0.000000 # InnoDB_pages_distinct: 475 SET timestamp=1393385020; SELECT `discounts`.* FROM `discounts` WHERE `discounts`.`shop_id` = 1745470 AND `discounts`.`status` = 'enabled' ORDER BY ISNULL(ends_at) DESC, ends_at DESC LIMIT 1
None
# User@Host: shopify[shopify] @ [127.0.0.1] # Thread_id: 264419969 Schema: shopify
Last_errno: 0 Killed: 0 # Query_time: 0.150491 Lock_time: 0.000057 Rows_sent: 1 Rows_examined: 147841 Rows_affected: 0 Rows_read: 147841 # Bytes_sent: 1214 Tmp_tables: 0 Tmp_disk_tables: 0 Tmp_table_sizes: 0 # InnoDB_trx_id: FF7021AAA # QC_Hit: No Full_scan: No Full_join: No Tmp_table: No Tmp_table_on_disk: No # Filesort: Yes Filesort_on_disk: No Merge_passes: 0 # InnoDB_IO_r_ops: 0 InnoDB_IO_r_bytes: 0 InnoDB_IO_r_wait: 0.000000 # InnoDB_rec_lock_wait: 0.000000 InnoDB_queue_wait: 0.000000 # InnoDB_pages_distinct: 475 SET timestamp=1393385020; SELECT `discounts`.* FROM `discounts` WHERE `discounts`.`shop_id` = 1745470 AND `discounts`.`status` = 'enabled' ORDER BY ISNULL(ends_at) DESC, ends_at DESC LIMIT 1 /*application:Shopify,controller:orders,action:pay*/
determining root cause
https://github.com/snormore/nginx-x-rid-header nginx request_id header proxy_set_header X-Request-ID "$request_id"; log_format main '...
$request_id' step 1
https://gist.github.com/mnutt/566725 Complete 200 OK in 100ms (Views: 60ms | ActiveRecord:
40ms | request_id=bc12813bce...) log_process_action ActionController::Instrumentation step 2
https://github.com/basecamp/marginalia User Load (0.3ms) SELECT `users`.* FROM `users` WHERE `users`.`id`
= 1 LIMIT 1 /*application:Shopify, controller:users,action:show, request_id:bc12813bce...*/ basecamp/marginalia step 3
# User@Host: shopify[shopify] @ [127.0.0.1] # Thread_id: 264419969 Schema: shopify
Last_errno: 0 Killed: 0 # Query_time: 0.150491 Lock_time: 0.000057 Rows_sent: 1 Rows_examined: 147841 Rows_affected: 0 Rows_read: 147841 # Bytes_sent: 1214 Tmp_tables: 0 Tmp_disk_tables: 0 Tmp_table_sizes: 0 # InnoDB_trx_id: FF7021AAA # QC_Hit: No Full_scan: No Full_join: No Tmp_table: No Tmp_table_on_disk: No # Filesort: Yes Filesort_on_disk: No Merge_passes: 0 # InnoDB_IO_r_ops: 0 InnoDB_IO_r_bytes: 0 InnoDB_IO_r_wait: 0.000000 # InnoDB_rec_lock_wait: 0.000000 InnoDB_queue_wait: 0.000000 # InnoDB_pages_distinct: 475 SET timestamp=1393385020; SELECT `discounts`.* FROM `discounts` WHERE `discounts`.`shop_id` = 1745470 AND `discounts`.`status` = 'enabled' ORDER BY ISNULL(ends_at) DESC, ends_at DESC LIMIT 1 /*application:Shopify,controller:orders,action:pay, request_id:bc12813bce...*/ profit!
access.log rails.log slow_query.log profit! (2)
bonus! background jobs
schema migration with zero downtime
None
orders 20140520_orders
insert/delete/update triggers
INSERT INTO ... SELECT ... insert/delete/update triggers
async caveat
tracking exceptions
None
None
None
None
thanks! :)