Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Tips and Tricks from Shopify's codebase
Search
Christian Joudrey
May 23, 2014
Programming
2
550
Tips and Tricks from Shopify's codebase
Talk given at Montreal.rb on May 20th, 2014
Christian Joudrey
May 23, 2014
Tweet
Share
More Decks by Christian Joudrey
See All by Christian Joudrey
Writing NES games! with assembly!!
cjoudrey
1
670
Developing at Scale
cjoudrey
3
460
Scaling Rails for Black Friday / Cyber Monday at Shopify
cjoudrey
6
5.6k
Scaling Shopify
cjoudrey
3
520
#pairwithme
cjoudrey
3
230
Two-factor authentication
cjoudrey
4
380
Automate your Infrastructure with Chef
cjoudrey
9
570
Other Decks in Programming
See All in Programming
Folding Cheat Sheet #7
philipschwarz
PRO
0
150
Google's Recipe for Scaling (Web) Security – LocoMocoSec 2024
lweichselbaum
0
170
初心者がおさえておきたいAWS CDKのベストプラクティス 2024
konokenj
15
7.3k
小さな開発会社を作った理由
polidog
0
1.9k
Introduction to GitOps
hwchiu
0
110
英語
s_shimotori
1
220
Jetpack for KMP
fornewid
1
290
今こそ始める、CDKコンストラクトライブラリ開発 ― 入門から実践まで
tmokmss
1
930
Trial
cairolibrary720
1
130
DDDを志して3年経ったら「DDDの皮を被ったクリーンアーキテクチャ」になった話【デブサミ2024夏】
texmeijin
1
620
Namespace on read
tagomoris
2
370
From Spring Boot 2 to Spring Boot 3 with Java 22 and Jakarta EE
ivargrimstad
0
1.9k
Featured
See All Featured
How GitHub Uses GitHub to Build GitHub
holman
471
290k
WebSockets: Embracing the real-time Web
robhawkes
59
7.2k
What's new in Ruby 2.0
geeforr
338
31k
Why Our Code Smells
bkeepers
PRO
332
56k
Stop Working from a Prison Cell
hatefulcrawdad
266
20k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
149
45k
Principles of Awesome APIs and How to Build Them.
keavy
124
16k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
502
140k
Rebuilding a faster, lazier Slack
samanthasiow
78
8.5k
Making the Leap to Tech Lead
cromwellryan
127
8.7k
RailsConf 2023
tenderlove
16
720
Being A Developer After 40
akosma
72
580k
Transcript
tips and tricks from Shopify’s codebase
None
None
None
measure it! if it moves...
statsd
None
Liquid::Template.extend StatsD::Instrument ! Liquid::Template.statsd_measure :render, 'Liquid.Template.render'
PaymentProcessingJob.statsd_count :perform, 'PaymentProcessingJob.processed'
None
None
slow queries
# User@Host: shopify[shopify] @ [127.0.0.1] # Thread_id: 264419969 Schema: shopify
Last_errno: 0 Killed: 0 # Query_time: 0.150491 Lock_time: 0.000057 Rows_sent: 1 Rows_examined: 147841 Rows_affected: 0 Rows_read: 147841 # Bytes_sent: 1214 Tmp_tables: 0 Tmp_disk_tables: 0 Tmp_table_sizes: 0 # InnoDB_trx_id: FF7021AAA # QC_Hit: No Full_scan: No Full_join: No Tmp_table: No Tmp_table_on_disk: No # Filesort: Yes Filesort_on_disk: No Merge_passes: 0 # InnoDB_IO_r_ops: 0 InnoDB_IO_r_bytes: 0 InnoDB_IO_r_wait: 0.000000 # InnoDB_rec_lock_wait: 0.000000 InnoDB_queue_wait: 0.000000 # InnoDB_pages_distinct: 475 SET timestamp=1393385020; SELECT `discounts`.* FROM `discounts` WHERE `discounts`.`shop_id` = 1745470 AND `discounts`.`status` = 'enabled' ORDER BY ISNULL(ends_at) DESC, ends_at DESC LIMIT 1
None
# User@Host: shopify[shopify] @ [127.0.0.1] # Thread_id: 264419969 Schema: shopify
Last_errno: 0 Killed: 0 # Query_time: 0.150491 Lock_time: 0.000057 Rows_sent: 1 Rows_examined: 147841 Rows_affected: 0 Rows_read: 147841 # Bytes_sent: 1214 Tmp_tables: 0 Tmp_disk_tables: 0 Tmp_table_sizes: 0 # InnoDB_trx_id: FF7021AAA # QC_Hit: No Full_scan: No Full_join: No Tmp_table: No Tmp_table_on_disk: No # Filesort: Yes Filesort_on_disk: No Merge_passes: 0 # InnoDB_IO_r_ops: 0 InnoDB_IO_r_bytes: 0 InnoDB_IO_r_wait: 0.000000 # InnoDB_rec_lock_wait: 0.000000 InnoDB_queue_wait: 0.000000 # InnoDB_pages_distinct: 475 SET timestamp=1393385020; SELECT `discounts`.* FROM `discounts` WHERE `discounts`.`shop_id` = 1745470 AND `discounts`.`status` = 'enabled' ORDER BY ISNULL(ends_at) DESC, ends_at DESC LIMIT 1 /*application:Shopify,controller:orders,action:pay*/
determining root cause
https://github.com/snormore/nginx-x-rid-header nginx request_id header proxy_set_header X-Request-ID "$request_id"; log_format main '...
$request_id' step 1
https://gist.github.com/mnutt/566725 Complete 200 OK in 100ms (Views: 60ms | ActiveRecord:
40ms | request_id=bc12813bce...) log_process_action ActionController::Instrumentation step 2
https://github.com/basecamp/marginalia User Load (0.3ms) SELECT `users`.* FROM `users` WHERE `users`.`id`
= 1 LIMIT 1 /*application:Shopify, controller:users,action:show, request_id:bc12813bce...*/ basecamp/marginalia step 3
# User@Host: shopify[shopify] @ [127.0.0.1] # Thread_id: 264419969 Schema: shopify
Last_errno: 0 Killed: 0 # Query_time: 0.150491 Lock_time: 0.000057 Rows_sent: 1 Rows_examined: 147841 Rows_affected: 0 Rows_read: 147841 # Bytes_sent: 1214 Tmp_tables: 0 Tmp_disk_tables: 0 Tmp_table_sizes: 0 # InnoDB_trx_id: FF7021AAA # QC_Hit: No Full_scan: No Full_join: No Tmp_table: No Tmp_table_on_disk: No # Filesort: Yes Filesort_on_disk: No Merge_passes: 0 # InnoDB_IO_r_ops: 0 InnoDB_IO_r_bytes: 0 InnoDB_IO_r_wait: 0.000000 # InnoDB_rec_lock_wait: 0.000000 InnoDB_queue_wait: 0.000000 # InnoDB_pages_distinct: 475 SET timestamp=1393385020; SELECT `discounts`.* FROM `discounts` WHERE `discounts`.`shop_id` = 1745470 AND `discounts`.`status` = 'enabled' ORDER BY ISNULL(ends_at) DESC, ends_at DESC LIMIT 1 /*application:Shopify,controller:orders,action:pay, request_id:bc12813bce...*/ profit!
access.log rails.log slow_query.log profit! (2)
bonus! background jobs
schema migration with zero downtime
None
orders 20140520_orders
insert/delete/update triggers
INSERT INTO ... SELECT ... insert/delete/update triggers
async caveat
tracking exceptions
None
None
None
None
thanks! :)