Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Perl in Mercari YAPC::Okinawa 2018 ONNASON
Search
kazeburo
March 03, 2018
Technology
8
5.6k
Perl in Mercari YAPC::Okinawa 2018 ONNASON
Perl in Mercari
YAPC::Okinawa 2018 ONNASON スペシャルセッション
kazeburo
March 03, 2018
Tweet
Share
More Decks by kazeburo
See All by kazeburo
クラウド開発の舞台裏とSRE文化の醸成 / SRE NEXT 2025 Lunch Session
kazeburo
1
1.1k
さくらのクラウド 開発の挑戦とその舞台裏
kazeburo
1
990
[SRE kaigi 2025] ガバメントクラウドに向けた開発と変化するSRE組織のあり方 / Development for Government Cloud and the Evolving Role of SRE Teams
kazeburo
4
3.5k
[さくらのTech Day] ガバメントクラウド開発と変化と成長する組織 / sakura techday, Develop govcloud and the team
kazeburo
0
7.1k
ガバメントクラウド開発と変化と成長する組織 / Organizational change and growth in developing a government cloud
kazeburo
4
3.2k
DNS水責め攻撃と監視 / DNS water torture attack Monitoring and SLO
kazeburo
5
4.4k
DBやめてみた / DNS water torture attack and countermeasures
kazeburo
13
14k
IaaSにおけるPlatform Engineeringとこれから / Platform engineering in IaaS
kazeburo
2
1.5k
高信頼IaaSを実現するDevOps / DevOps for Highly Reliable IaaS
kazeburo
1
730
Other Decks in Technology
See All in Technology
Kubernetes における cgroup v2 でのOut-Of-Memory 問題の解決
pfn
PRO
0
420
AI時代に非連続な成長を実現するエンジニアリング戦略
sansantech
PRO
3
880
『FailNet~やらかし共有SNS~』エレベーターピッチ
yokomachi
1
190
ZOZOマッチのアーキテクチャと技術構成
zozotech
PRO
2
680
実践アプリケーション設計 ①データモデルとドメインモデル
recruitengineers
PRO
5
1.3k
個人CLAUDE.md紹介と設定から学んだこと/introduce-my-claude-md
shibayu36
0
110
小さなチーム 大きな仕事 - 個人開発でAIをフル活用する
himaratsu
0
150
Vault meets Kubernetes
mochizuki875
0
150
Microsoft Fabric のネットワーク保護のアップデートについて
ryomaru0825
1
120
MCPで変わる Amebaデザインシステム「Spindle」の開発
spindle
PRO
1
1k
ZOZOTOWNフロントエンドにおけるディレクトリの分割戦略
zozotech
PRO
18
6k
トヨタ生産方式(TPS)入門
recruitengineers
PRO
5
1.4k
Featured
See All Featured
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
53
2.9k
Writing Fast Ruby
sferik
628
62k
Build The Right Thing And Hit Your Dates
maggiecrowley
37
2.8k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
194
16k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
667
120k
BBQ
matthewcrist
89
9.8k
Typedesign – Prime Four
hannesfritz
42
2.8k
Why You Should Never Use an ORM
jnunemaker
PRO
59
9.5k
Measuring & Analyzing Core Web Vitals
bluesmoon
9
570
Become a Pro
speakerdeck
PRO
29
5.5k
How to Ace a Technical Interview
jacobian
279
23k
[RailsConf 2023] Rails as a piece of cake
palkan
56
5.8k
Transcript
Perl in Mercari 2018 @kazeburo YAPC::Okinawa 2018 ONNASON
େͳ͓Βͤ
memcached UDPϦϑϨΫγϣϯDDoS • Πϯλʔωοτʹެ։͞Ε͍ͯΔmemcached 11211/UDPΛར༻ͯ͠UDP ϦϑϨ Ϋγϣϯ͕ߦ͑Δ • github.com ʹ1.3TbpsͷDDoS߈ܸ͕ൃੜ
(2/28) • ͘͞ΒVPS11211/UDPΛϑΟϧλϦϯά • [ॏཁ] memcachedΛάϩʔόϧʹެ։͠ͳ͍ • UDPΛແޮԽ͢Δɻmemcached -U 0Ͱىಈ memcached-1.5.6ͰσϑΥϧτແޮʹͳΓ·ͨ͠
None
Me • խ • PAUSE:KAZEBURO • גࣜձࣾϝϧΧϦ ϓϦϯγύϧΤϯδχΞ Site Reliability
Engineering (SRE) νʔϜ • BASE, Inc ٕज़ΞυόΠβʔ
None
ϝϧΧϦ • ࠃ࠷େڃͷϑϦϚΞϓϦ • 3Ͱ؆୯ʹग़ 1) ࣸਅΛࡱΔ 2) ใΛهೖ 3)
ग़ϘλϯΛԡ͢ • ҆৺҆શͳܾࡁɾऔҾ • ΤεΫϩʔ(͓ۚͷΓͱΓ͕ࣾؒʹհࡏ) • ಗ໊ૹ
KPI μϯϩʔυ GMV(૯औҾֹ) 1ԯDLҎ্(JP+US+UK) ݄ؒ100ԯԁҎ্ ग़ 1100ສҎ্
ϝϧΧϦΛࢧ͑Δ αʔόαΠυͷ ϓϩάϥϛϯάݴޠ
PHP 7
Golang https://github.com/golang-samples/gopher-vector
ϝϧΧϦΛࢧ͑Δϓϩάϥϛϯάݴޠ • ϝϧΧϦͷAPI΄΅શͯPHPʹΑΓ࣮ • ϚΠΫϩαʔϏε/ϛυϧΣΞͰ Golang ͕த৺ • AI/MLͷͰPythonΘΕΔ
1FSMϝΠϯϦʔϜͳݴޠͰͳ͍
͕͔ͩ͠͠ খ͞ͳ Perl εΫϦϓτ͕αʔϏεͷ҆ఆՔಇΛࢧ͑ ཪํͱͯ͠େنͳτϥϑΟοΫΛॲཧ͍ͯ͠Δɻ
slacklog
slacklog = cronlog + Slack notification
cronlog • cronlog kazuho͞Μ࡞ͷPerlεΫϦϓτɻҾͱͯ͠༩͑ΒΕͨίϚϯ υΛ࣮ߦ͠ɺͦͷऴྃίʔυ͕ඇθϩͷ߹ͷΈɺίϚϯυͷग़ྗ (stdout, stderr)Λඪ४ग़ྗʹૹΔ • cronͰར༻͢Δ͜ͱͰΤϥʔ࣌ͷΈMAILTOʹϝʔϧૹ৴ •
`5 * * * * cronlog -- ping -n 5 my-server 2>&1` • ʮࢹͱܧଓతͳςετʯͱ͍͏ݴ༿ΛੜΈग़ͨ͠
slacklog • ίϚϯυͷऴྃίʔυ͕ඇθϩ࣌ʹඪ४ग़ྗͷૹ৴ʹՃ͑ɺslackͷ௨Λߦ͏ • @channel ΛՃ͢Δ͔ࢦఆՄೳ • ىಈͱऴྃ࣌ʹSlackͷ௨ɺsyslogه • hostname,pid
ͷՃ • slackͷ௨ slackboard Λར༻ • ϝϧΧϦͷ΄΅શͯͷcronͰར༻͍ͯ͠Δ
slacklog use slackboard App Batch MySQL App Batch MySQL App
Batch MySQL JP US UK slacklog slacklog slacklog slacklog slacklog slacklog slacklog slacklog slacklog slackborad slackborad slackborad JP US EU HTTP Load Balancer MySQL slacklog slacklog slacklog SlackͷAPI keyͷҰݩཧ Buffering/Queueing Slack https://github.com/cubicdaiya/slackboard
࣮ߦྫ $ slacklog -c test --log-channel test -- perl -E
'$|=1;say "Hello";sleep 3;die' [2018-02-25T01:09:51][21066] *hostname* starting: perl -E $|=1;say "Hello";sleep 3;die [2018-02-25T01:09:51][21066] Hello [2018-02-25T01:09:54][21066] Died at -e line 1. [2018-02-25T01:09:54][21066] command exited with code:255 $
࣮ߦྫ(slack) ىಈऴྃϩά ίϚϯυͷ STDOUT STDERR
ಋೖޮՌ • batchͷΤϥʔͷݕग़ͱ௨ͷଈ࣌ੑΞοϓ • ϝʔϧ͍ɺ͘͠ΈΒΕͯͳ͍ • αʔόαΠυΤϯδχΞͷslack channelʹૹ৴ɻ࣭ͷ্ • ϩάΛ/dev/nullʹࣺͯͳ͍ӡ༻͕Մೳʹ
ಋೖޮՌ • ʮ͋ͷcronىಈ͍ͯ͠·͔͢ʁʯʮࡢϦϦʔεͨ͠batchɺਖ਼͘͠ಈ͍ ͯ·͔͢ʁʯͷΑ͏ͳΤϯδχΞ͔Βͷ֬ೝґཔͷݮ • τΠϧͷ໓
Ӿཡ ཤྺ SERVICE
Ӿཡཤྺ • ͓٬༷ͷӾཡͷཤྺΛαʔόʹ֨ೲ • UX্ͱͱʹɺTimeline Personalizationͷσʔλ ͱͯ͠׆༻ • ͷӾཡઍreq/sec PerlͰߏங͞Εͨ
ServiceͰॲཧ
App App App App App NGINX API(Perl) Q4M Worker (Perl)
browse_item_service NGINX API(Perl) Q4M Worker (Perl) browse_item_service MySQL MySQL memcached memcached memcached "ߋ৽" "ࢀর" cluster (2) DBߋ৽&ཤྺऔಘ (3) cache࡞ (1) enqueue (1) cacheऔಘ/missͨ͠ΒDB App App App App App memcached memcached memcached NGINX API(Perl) Q4M Worker (Perl) browse_item_service DNS-RR by consul
CPAN modules • API • Plack, Gazelle • Worker •
Parallel::Prefork • ڞ௨ • JSON::XS, DBD::mysql, Cache::Memcached::Fast::Safe Web Application Frameworkະ༻ Plack::(Request|Response)ͬͯͳ͍ XS moduleΛ׆༻্ͯ͠
Performance • API/WorkerScale Out͕Մೳ • GazelleʹΑΔAPIαʔόेʹߴ • ϘτϧωοΫ MySQL ͷϨϓϦέʔγϣϯԆ
• ௨ৗͷϨϓϦέʔγϣϯͰઍ transaction/s ͰԆൃੜ • MySQL 5.7ͷ Multi Thread Slave Λར༻ͯ͠ 18,000 t/s ͕ݶք • Sharding ΛॳظઃܭʹΈࠐΈ
Performance (newRelic) ͝ͱͷPVΛܭଌ ͢Δ Service (Golang) Avg time: 3 msec
Ӿཡཤྺ Service (Perl) Avg time: 4 msec PHP DBͳͲ
http://tech.mercari.com/entry/2018/02/26/110237
Կނ Perl?
ͳͥ Perl ? • kazeburo͕ॻ͍͔ͨΒ • System Call friendly ‣
OS ProcessɺNetwork ͷجૅΛֶͿͷʹద͍ͯ͠Δ • (Core Module ͷΈͰ࡞Ε) Deploy ͕༰қ • PerlͰഓΘΕٕͨज़ͷܧঝ ‣ mixi, livedoor (LINE), DeNA, ͯͳ ͳͲେنWebαʔϏεΛࢧ͑ͯ(͍Δ|͖ͨ)ݴޠ
͜Ε͔Β • ϚΠΫϩαʔϏεɾίϯςφத৺ͷInfrustructure • Serverlessɺk8sʹΑΔࣗಈԽ͞Εͨ Infrastructure • OSnetworkͳͲγεςϜΤϯδχΞϦϯάͷ͕ࣝ։ൃऀ͔Βԕ͘ͳΔ • Golang
த৺ɺPerl ΠϯλϓϦλ͕ͳ͍ڥ
͜Ε͔Β • SREݚमͰPerlΛ৮ͬͯΒ͏ • ৽ଔ͚ʹ1ϲ݄͔Β1ϲ݄ఔͷSREݚमΛߦ͍ͬͯΔ • ͦͷதͰͻͱͭPerlΛͬͨΈʹ৮ΕΔػձΛͭ͘Δ • slacklogͷىಈϩάΛSlackʹૹΔػೳSREݚमͰ࡞ͬͯΒͬͨͷ •
OSɾFile SystemɾProcessɾNetworkͱ͍ͬͨϨΠϠʔͷڵຯɾؔ৺ UNIX ֶʹ৮ΕΔػձ • ٕज़ͷఈ্͛ʹඞཁͳݴޠͷҰͭ
Ҏ্ careers.mercari.com/job/