Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
大規模Mastodonインスタンスを運用するコツ / Inside Pawoo Mastodon infrastructure
Search
Harukasan
PRO
April 27, 2017
Technology
0
2.9k
大規模Mastodonインスタンスを運用するコツ / Inside Pawoo Mastodon infrastructure
実際運用してみてわかった、大規模Mastodonインスタンスを運用するコツ
2017-04-25 at pixiv night!
Harukasan
PRO
April 27, 2017
Tweet
Share
More Decks by Harukasan
See All by Harukasan
20240401 新卒研修 - ピクシブにおける技術領域
harukasan
PRO
1
560
ピクシブのコンテンツ配信基盤技術 / pixiv TECH SALON
harukasan
PRO
5
5.1k
Goにおける画像ファイル処理 / golang.tokyo #19
harukasan
PRO
7
6.2k
WebRTC動画をトランスコードする / Transcoding video streams from WebRTC
harukasan
PRO
5
1.4k
ImageFluxを支えるリモート開発 / 20171202
harukasan
PRO
2
1.7k
YAPC::Fukuoka 前夜祭LT / Yet Another Pawoo Commit logs
harukasan
PRO
0
2.8k
YAPC::Fukuoka lunch session
harukasan
PRO
1
2.9k
マストドン会議: Pawoo / Mastodon Kaigi2
harukasan
PRO
2
410
JPEG streaming in Go
harukasan
PRO
2
2.5k
Other Decks in Technology
See All in Technology
「できる!」を増やすGitHub Copilot活用法 / How to use GitHub Copilot to expand your possibilities
sansan_randd
1
220
PHP 9 に備えよ - 動的プロパティ、どうすればいぃ?
taisukearase
0
140
5分で分かる(かもしれない) Vector engine for OpenSearch Serverless
tsukuboshi
0
260
IaCからAWSに入門した初心者が CloudFormationを通して考えた「AWS操作」の使い分け
maimyyym
3
660
Secrets of a PowerShell "Guru"
guyrleech
1
120
RailsConf 2024 Keynote "Startups on Rails in 2024"
irinanazarova
0
520
Oracle Base Database Service 技術詳細
oracle4engineer
PRO
5
38k
データ分析力を高めるSQL研修サービス『SQL Everyone』
hikarut
1
350
QAエンジニアが伝えたい品質保証の羅針盤 / Compass for Quality Assurance
mii3king
1
310
Documentação de Produtos: Artefatos essenciais na prática
rigolon
1
270
類似ロジック実装をiOS/Android間で合わせる道標No.1
fumiyasac0921
1
220
Prisma ORMを2年運用して培ったノウハウを共有する
tockn
18
4.7k
Featured
See All Featured
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
275
13k
Web Components: a chance to create the future
zenorocha
306
41k
Into the Great Unknown - MozCon
thekraken
15
1.1k
Debugging Ruby Performance
tmm1
70
11k
Stop Working from a Prison Cell
hatefulcrawdad
266
19k
ParisWeb 2013: Learning to Love: Crash Course in Emotional UX Design
dotmariusz
104
6.7k
We Have a Design System, Now What?
morganepeng
43
6.8k
Building Flexible Design Systems
yeseniaperezcruz
320
37k
Robots, Beer and Maslow
schacon
PRO
155
7.9k
Fantastic passwords and where to find them - at NoRuKo
philnash
39
2.5k
Statistics for Hackers
jakevdp
790
220k
Intergalactic Javascript Robots from Outer Space
tanoku
266
26k
Transcript
@harukasan / Shunsuke Michii ࣮ࡍӡ༻ͯ͠ΈͯΘ͔ͬͨɺେن.BTUPEPOΠϯελϯεΛӡ༻͢Δίπ
@Harukasan Shunsuke MICHII Lead engineer of pixiv Inc., Product manager
of ImageFlux, Infrastructure engineer, Kibana, Gopher
None
࣮ࡍӡ༻ͯ͠ΈͯΘ͔ͬͨɺେن.BTUPEPOΠϯελϯεΛӡ༻͢Δίπ
None
Mastodon
͍͖ͭͬ͞v1.2.2Λͩͨ͠
github.com/pixiv/mastodon:pawoo
Ruby on Rails node.js PostgreSQL Redis SendGrid Sidekiq ΫϥΠΞϯτ Websocket
(streaming) HTTP SMTP ϞμϯͳRuby on RailsΞϓϦέʔγϣϯ nginx
Ruby on Rails node.js PostgreSQL Redis SendGrid Sidekiq ΫϥΠΞϯτ Websocket
(streaming) HTTP SMTP ϞμϯͳRuby on RailsΞϓϦέʔγϣϯ nginx RESTͬΆ͍API
Ruby on Rails node.js PostgreSQL Redis SendGrid Sidekiq ΫϥΠΞϯτ Websocket
(streaming) HTTP SMTP ϞμϯͳRuby on RailsΞϓϦέʔγϣϯ nginx RESTͬΆ͍API ετϦʔϛϯάAPI
10࣌ؒͰϦϦʔε
Pawooͷߏ
None
AWSͰγϡοͱΔ • ͕࣌ؒͳ͍͚Ͳ࠷༏ઌ҆ఆੑ • RDS/ElasticaceͷϚϧνAZߏ • ϝσΟΞS3ʹอଘ • EC2ฒͯALBͰϩʔυόϥϯε •
CloudFrontͰྲྀ͢ʢ͏Ίͨʣ • ظؒݶఆαΠτΛཱͯΔ࣌ͷϊϋΛྲྀ༻
ΠϯελϯεΛฒΔ • ͯ͢ͷΠϯελϯεಉҰߏ • γϡοͱͭͬͯ͘AMIͰ૿͢ • ԣʹฒͯεέʔϧΞτ • ͦͷ͏ͪΦʔτεέʔϦϯά͍ͨ͠
Docker͔Β͕͢ • શ෦systemdαʔϏεʹஔ͖͑ͨ • ͦͷ··͏ͷͪΐͬͱ͍͜͠ͱ͕ଟ͍ • σϓϩΠͷ͜ͱΒͳΜΒߟ͍͑ͯΔͱͪΐͬͱ༨༟͕ͳ͍ • DockerͰ͙͢ΔͳΒϗοτσϓϩΠͱ͔ϦιʔεͷΈΛ ͋Β͔͡Ί༻ҙ͓ͯ͘͠ඞཁ͕͋Γͦ͏ʢͦͷ͏ͪΓ͍ͨʣ
ͦͷͷ͏ͪʹϦϦʔε͞Ε͕ͨ ͙͢ʹΤϥʔΛฦ͠͡ΊΔ ʢ͜͜ΒลͰνϟοτΛݟ࢝ΊΔʣ
nginx • ϦΫΤετόοϑΝϦϯάͷͨΊʹPumaͷલʹnginxΛ͓͘ • Τϥʔ͕ग़͍ͯͨͷͰͳ͓ͨ͠ • worker_rlimit_nofile: limitΛ૿͓ͯ͘͠ • worker_connections:
ίωΫγϣϯͨ͘͞ΜΔͷͰ • ϩάϩʔςʔτͷઃఆ͕ೖͬͯͳ͔ͬͨͷͰೖΕͨ • όϥϯγϯάEC2ͷnginx͡Όͳͯ͘ALBͰΔ
ৄ͘͜͠ͷຊʹ
PostgreSQL • PostgreSQLίωΫγϣϯ͝ͱʹfork͢Δ ʢฐࣾͯ͢MySQLͩͬͨͷͰݟͳ͠ʣ • ίωΫγϣϯϓʔϧΛ͏·͍۩߹ͷʹઃఆ͢Δ • ΠϯελϯεΛεέʔϧΞοϓͯ͠ରԠ • ίʔυʹ࠷దԽΛೖΕͯΠϯελϯεαΠζখͨ͘͞͠Γͨ͠
• Մ༻ੑͱόοΫΞοϓRDSʹ͓ͤ • PgBouncerͬͨํ͕͍͍͔ʁͱࢥ͍ͳ͕Βාͯͬͯ͘ͳ͍
͜͜ͰεέʔϧΞοϓ
None
ݟٻΉ
Ruby on Rails node.js PostgreSQL Redis SendGrid Sidekiq ΫϥΠΞϯτ Websocket
(streaming) HTTP SMTP ϞμϯͳRuby on RailsΞϓϦέʔγϣϯ nginx
Ruby on Rails node.js PostgreSQL Redis SendGrid Sidekiq ΫϥΠΞϯτ Websocket
(streaming) HTTP SMTP nginx ϢʔβʔΠϯλʔϑΣʔε ϞμϯͳRuby on RailsΞϓϦέʔγϣϯ
Ruby on Rails node.js PostgreSQL Redis SendGrid Sidekiq ΫϥΠΞϯτ Websocket
(streaming) HTTP SMTP nginx ϞμϯͳRuby on RailsΞϓϦέʔγϣϯ ϝΠϯϩδοΫ
Sidekiq • MastodonͷϝοηʔδύογϯάSidekiqͰߦΘΕΔ • τΡʔτͷ৴ • ετϦʔϛϯά • ϦϞʔτΠϯελϯεͷ৴ •
1τΡʔτ͝ͱʹϑΥϩϫʔͷ͚ͩenqueue͞ΕΔ • ϑΥϩϫʔ͕ଟ͍Ϣʔβʔͩͱ͍
ීஈͷΩϡʔͷ༷ࢠʢͪΐͬͱલʣ
ͨ·ʹ͜͏ͳΔ
ͨ·ʹ͜͏ͳΔ
Ωϡʔͷछྨ default pull push mail • ΄͔ͷશ෦ͷδϣϒ • ϦϞʔτΠϯελϯεͷ৴ •
ϝʔϧͷૹ৴
Sidekiq٧·Δ • σϑΥϧτͩͱͯ͢ͷΩϡʔΛ̍ͭͷSidekiqϓϩηεͰॲཧ͢Δ • ϦϞʔτΠϯελϯε͕٧·Δͱpull/pushͷΩϡʔ͕٧·Δ • pull/pushͷ͍ͤͰλΠϜϥΠϯ͕Ԇ͢Δʂ • ͜ͷ··ͩͱαʔϏεͱͯ͠ऴΘͬͯ͠·͏ͷͰɺϓϩηεΛ͚Δ •
Sidekiq1ϓϩηεͰ1ίΞʢ100%ʣ·Ͱ͔ͬͯ͘͠Εͳ͍ • ϓϩηεΛͨ͘͞ΜཱͯͯΧόʔ
ίϚϯυϥΠϯҾͰࢦఆ͢Δ͚ͩ [Service] Type=simple User=mastodon WorkingDirectory=/home/mastodon/live Environment="RAILS_ENV=production" Environment="DB_POOL=40" ExecStart=/usr/local/rbenv/shims/bundle exec sidekiq
-c 40 -q default TimeoutSec=15 Restart=always
SidekiqϓϩηεΛ͚Δ sidekiq-default-1.service sidekiq-default-2.service sidekiq-default-3.service sidekiq-pullpush.service sidekiq-mail.service 1Πϯελϯεʹ͜Εཱ͚ͩͯͯΔɿ σϑΥϧτ͚ͩͰ1,000εϨουҎ্ 40 40
40 10 5 εϨου
Sidekiqͷϓϩηεը໘͕ ͍ͩͿݟͮΒ͍
Sidekiq٧·Δͷ՝ • ϦϞʔτΠϯελϯε͕٧·Δͱ૬มΘΒͣpull/push͕٧·Δ • ϓϩηεΛཱͯͯΧόʔͯ͠Δ͚Ͳɺ΄͔ͷΠϯελϯε͔Β͢Δͱ ٸʹϦΫΤετ͕૿͑ͯͭΒͦ͏…… • ͍Πϯελϯεͱ͔େنΠϯελϯεͷॲཧΛͲ͏ʹ͔͍ͨ͠ • ͦͦτΡʔτ͋ͨΓͷδϣϒ͕N+1ʹͳͬͯΔͷ͕͍
ղܾͯ͠ͳ͍͚Ͳ·ͱΊ • AWSͬͯγϡοͱཱͯͯӡ༻։࢝ • ϚετυϯͷϝΠϯSidekiqδϣϒ • SidekiqͷΩϡʔΛ͏·͚͘͞ΔͱΑ͍ • SidekiqϓϩηεΛεϙοτΠϯελϯεͰ૿͢Α͏ʹ͍ͨ͠ •
Ͳ͔ͬͪͱ͍͏ͱ࠷దԽͨ͠ํ͕Αͦ͞͏