Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
Perl in Mercari YAPC::Okinawa 2018 ONNASON
kazeburo
March 03, 2018
Technology
8
5k
Perl in Mercari YAPC::Okinawa 2018 ONNASON
Perl in Mercari
YAPC::Okinawa 2018 ONNASON スペシャルセッション
kazeburo
March 03, 2018
Tweet
Share
More Decks by kazeburo
See All by kazeburo
DNS権威サーバのクラウドサービス向けに行われた攻撃および対策 / DNS Pseudo-Random Subdomain Attack and mitigations
kazeburo
4
850
sacloudns
kazeburo
2
220
「orchestratorとGTID運用を支える監視」の勉強 / Monitoring orchestrator and GTID operation
kazeburo
2
1k
最近の監視(仮)/Recent system monitoring with mackerel
kazeburo
3
3.8k
Mercari Item Search: Behind The Scenes (20min)
kazeburo
3
2.6k
成長し続けるインフラストラクチャとメルカリの挑戦/mercari infrastructure and software
kazeburo
23
4.6k
Microservices on Multi-Cloud
kazeburo
21
5.4k
logrotate殺プロセス事件 YAPC::Okinawa 2018 前夜祭 LT/Mystery of logrotate's death
kazeburo
9
11k
インフラチームからSREへ / SRE in Mercari Developers Summit 2018
kazeburo
26
9k
Other Decks in Technology
See All in Technology
PHPのimmutable arrayとは
hnw
1
130
20230121_BuriKaigi
oyakata2438
0
120
FlexScan HD2452Wの 後継を探して
tring
0
240
IoT から見る AWS re:invent 2022 ― AWSのIoTの歴史を添えて/Point of view the AWS re:invent 2022 with IoT - with a history of IoT in AWS
ma2shita
0
150
LINE iOSエンジニアの日々 / LINE iOS Engineer Days
line_developers
PRO
1
130
20230117_JAWS-UG_朝会_41_LT資料
tsumita
0
300
証明書って何だっけ? 〜AWSの中間CA移行に備える〜
minorun365
3
2k
CUEとKubernetesカスタムオペレータを用いた新しいネットワークコントローラをつくってみた
hrk091
0
210
Technologies for developing editors / Webエディタ開発を支える技術
shuta13
1
220
TypeScriptは10年でこんなに進化しました / TechFeed Experts Night 11
okunokentaro
5
1.1k
DID/VCを用いた自己主権型アイデンティティの実現
sbtechnight
0
370
20230123_FinJAWS
takuyay0ne
0
100
Featured
See All Featured
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
109
16k
Put a Button on it: Removing Barriers to Going Fast.
kastner
56
2.5k
Docker and Python
trallard
30
1.9k
Large-scale JavaScript Application Architecture
addyosmani
499
110k
Teambox: Starting and Learning
jrom
124
7.9k
The Invisible Customer
myddelton
113
12k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
32
6.7k
4 Signs Your Business is Dying
shpigford
171
20k
Atom: Resistance is Futile
akmur
256
24k
The Straight Up "How To Draw Better" Workshop
denniskardys
226
130k
How STYLIGHT went responsive
nonsquared
89
4.2k
Designing for humans not robots
tammielis
245
24k
Transcript
Perl in Mercari 2018 @kazeburo YAPC::Okinawa 2018 ONNASON
େͳ͓Βͤ
memcached UDPϦϑϨΫγϣϯDDoS • Πϯλʔωοτʹެ։͞Ε͍ͯΔmemcached 11211/UDPΛར༻ͯ͠UDP ϦϑϨ Ϋγϣϯ͕ߦ͑Δ • github.com ʹ1.3TbpsͷDDoS߈ܸ͕ൃੜ
(2/28) • ͘͞ΒVPS11211/UDPΛϑΟϧλϦϯά • [ॏཁ] memcachedΛάϩʔόϧʹެ։͠ͳ͍ • UDPΛແޮԽ͢Δɻmemcached -U 0Ͱىಈ memcached-1.5.6ͰσϑΥϧτແޮʹͳΓ·ͨ͠
None
Me • խ • PAUSE:KAZEBURO • גࣜձࣾϝϧΧϦ ϓϦϯγύϧΤϯδχΞ Site Reliability
Engineering (SRE) νʔϜ • BASE, Inc ٕज़ΞυόΠβʔ
None
ϝϧΧϦ • ࠃ࠷େڃͷϑϦϚΞϓϦ • 3Ͱ؆୯ʹग़ 1) ࣸਅΛࡱΔ 2) ใΛهೖ 3)
ग़ϘλϯΛԡ͢ • ҆৺҆શͳܾࡁɾऔҾ • ΤεΫϩʔ(͓ۚͷΓͱΓ͕ࣾؒʹհࡏ) • ಗ໊ૹ
KPI μϯϩʔυ GMV(૯औҾֹ) 1ԯDLҎ্(JP+US+UK) ݄ؒ100ԯԁҎ্ ग़ 1100ສҎ্
ϝϧΧϦΛࢧ͑Δ αʔόαΠυͷ ϓϩάϥϛϯάݴޠ
PHP 7
Golang https://github.com/golang-samples/gopher-vector
ϝϧΧϦΛࢧ͑Δϓϩάϥϛϯάݴޠ • ϝϧΧϦͷAPI΄΅શͯPHPʹΑΓ࣮ • ϚΠΫϩαʔϏε/ϛυϧΣΞͰ Golang ͕த৺ • AI/MLͷͰPythonΘΕΔ
1FSMϝΠϯϦʔϜͳݴޠͰͳ͍
͕͔ͩ͠͠ খ͞ͳ Perl εΫϦϓτ͕αʔϏεͷ҆ఆՔಇΛࢧ͑ ཪํͱͯ͠େنͳτϥϑΟοΫΛॲཧ͍ͯ͠Δɻ
slacklog
slacklog = cronlog + Slack notification
cronlog • cronlog kazuho͞Μ࡞ͷPerlεΫϦϓτɻҾͱͯ͠༩͑ΒΕͨίϚϯ υΛ࣮ߦ͠ɺͦͷऴྃίʔυ͕ඇθϩͷ߹ͷΈɺίϚϯυͷग़ྗ (stdout, stderr)Λඪ४ग़ྗʹૹΔ • cronͰར༻͢Δ͜ͱͰΤϥʔ࣌ͷΈMAILTOʹϝʔϧૹ৴ •
`5 * * * * cronlog -- ping -n 5 my-server 2>&1` • ʮࢹͱܧଓతͳςετʯͱ͍͏ݴ༿ΛੜΈग़ͨ͠
slacklog • ίϚϯυͷऴྃίʔυ͕ඇθϩ࣌ʹඪ४ग़ྗͷૹ৴ʹՃ͑ɺslackͷ௨Λߦ͏ • @channel ΛՃ͢Δ͔ࢦఆՄೳ • ىಈͱऴྃ࣌ʹSlackͷ௨ɺsyslogه • hostname,pid
ͷՃ • slackͷ௨ slackboard Λར༻ • ϝϧΧϦͷ΄΅શͯͷcronͰར༻͍ͯ͠Δ
slacklog use slackboard App Batch MySQL App Batch MySQL App
Batch MySQL JP US UK slacklog slacklog slacklog slacklog slacklog slacklog slacklog slacklog slacklog slackborad slackborad slackborad JP US EU HTTP Load Balancer MySQL slacklog slacklog slacklog SlackͷAPI keyͷҰݩཧ Buffering/Queueing Slack https://github.com/cubicdaiya/slackboard
࣮ߦྫ $ slacklog -c test --log-channel test -- perl -E
'$|=1;say "Hello";sleep 3;die' [2018-02-25T01:09:51][21066] *hostname* starting: perl -E $|=1;say "Hello";sleep 3;die [2018-02-25T01:09:51][21066] Hello [2018-02-25T01:09:54][21066] Died at -e line 1. [2018-02-25T01:09:54][21066] command exited with code:255 $
࣮ߦྫ(slack) ىಈऴྃϩά ίϚϯυͷ STDOUT STDERR
ಋೖޮՌ • batchͷΤϥʔͷݕग़ͱ௨ͷଈ࣌ੑΞοϓ • ϝʔϧ͍ɺ͘͠ΈΒΕͯͳ͍ • αʔόαΠυΤϯδχΞͷslack channelʹૹ৴ɻ࣭ͷ্ • ϩάΛ/dev/nullʹࣺͯͳ͍ӡ༻͕Մೳʹ
ಋೖޮՌ • ʮ͋ͷcronىಈ͍ͯ͠·͔͢ʁʯʮࡢϦϦʔεͨ͠batchɺਖ਼͘͠ಈ͍ ͯ·͔͢ʁʯͷΑ͏ͳΤϯδχΞ͔Βͷ֬ೝґཔͷݮ • τΠϧͷ໓
Ӿཡ ཤྺ SERVICE
Ӿཡཤྺ • ͓٬༷ͷӾཡͷཤྺΛαʔόʹ֨ೲ • UX্ͱͱʹɺTimeline Personalizationͷσʔλ ͱͯ͠׆༻ • ͷӾཡઍreq/sec PerlͰߏங͞Εͨ
ServiceͰॲཧ
App App App App App NGINX API(Perl) Q4M Worker (Perl)
browse_item_service NGINX API(Perl) Q4M Worker (Perl) browse_item_service MySQL MySQL memcached memcached memcached "ߋ৽" "ࢀর" cluster (2) DBߋ৽&ཤྺऔಘ (3) cache࡞ (1) enqueue (1) cacheऔಘ/missͨ͠ΒDB App App App App App memcached memcached memcached NGINX API(Perl) Q4M Worker (Perl) browse_item_service DNS-RR by consul
CPAN modules • API • Plack, Gazelle • Worker •
Parallel::Prefork • ڞ௨ • JSON::XS, DBD::mysql, Cache::Memcached::Fast::Safe Web Application Frameworkະ༻ Plack::(Request|Response)ͬͯͳ͍ XS moduleΛ׆༻্ͯ͠
Performance • API/WorkerScale Out͕Մೳ • GazelleʹΑΔAPIαʔόेʹߴ • ϘτϧωοΫ MySQL ͷϨϓϦέʔγϣϯԆ
• ௨ৗͷϨϓϦέʔγϣϯͰઍ transaction/s ͰԆൃੜ • MySQL 5.7ͷ Multi Thread Slave Λར༻ͯ͠ 18,000 t/s ͕ݶք • Sharding ΛॳظઃܭʹΈࠐΈ
Performance (newRelic) ͝ͱͷPVΛܭଌ ͢Δ Service (Golang) Avg time: 3 msec
Ӿཡཤྺ Service (Perl) Avg time: 4 msec PHP DBͳͲ
http://tech.mercari.com/entry/2018/02/26/110237
Կނ Perl?
ͳͥ Perl ? • kazeburo͕ॻ͍͔ͨΒ • System Call friendly ‣
OS ProcessɺNetwork ͷجૅΛֶͿͷʹద͍ͯ͠Δ • (Core Module ͷΈͰ࡞Ε) Deploy ͕༰қ • PerlͰഓΘΕٕͨज़ͷܧঝ ‣ mixi, livedoor (LINE), DeNA, ͯͳ ͳͲେنWebαʔϏεΛࢧ͑ͯ(͍Δ|͖ͨ)ݴޠ
͜Ε͔Β • ϚΠΫϩαʔϏεɾίϯςφத৺ͷInfrustructure • Serverlessɺk8sʹΑΔࣗಈԽ͞Εͨ Infrastructure • OSnetworkͳͲγεςϜΤϯδχΞϦϯάͷ͕ࣝ։ൃऀ͔Βԕ͘ͳΔ • Golang
த৺ɺPerl ΠϯλϓϦλ͕ͳ͍ڥ
͜Ε͔Β • SREݚमͰPerlΛ৮ͬͯΒ͏ • ৽ଔ͚ʹ1ϲ݄͔Β1ϲ݄ఔͷSREݚमΛߦ͍ͬͯΔ • ͦͷதͰͻͱͭPerlΛͬͨΈʹ৮ΕΔػձΛͭ͘Δ • slacklogͷىಈϩάΛSlackʹૹΔػೳSREݚमͰ࡞ͬͯΒͬͨͷ •
OSɾFile SystemɾProcessɾNetworkͱ͍ͬͨϨΠϠʔͷڵຯɾؔ৺ UNIX ֶʹ৮ΕΔػձ • ٕज़ͷఈ্͛ʹඞཁͳݴޠͷҰͭ
Ҏ্ careers.mercari.com/job/