Perl in Mercari 2018 @kazeburo
 YAPC::Okinawa 2018 ONNASON

େ੾ͳ͓஌Βͤ

memcached UDPϦϑϨΫγϣϯDDoS • Πϯλʔωοτʹެ։͞Ε͍ͯΔmemcached 11211/UDPΛར༻ͯ͠UDP ϦϑϨ Ϋγϣϯ͕ߦ͑Δ • github.com ʹ1.3TbpsͷDDoS߈ܸ͕ൃੜ (2/28) • ͘͞ΒVPS͸11211/UDPΛϑΟϧλϦϯά • [ॏཁ] memcachedΛάϩʔόϧʹެ։͠ͳ͍ • UDPΛແޮԽ͢Δɻmemcached -U 0Ͱىಈ
 memcached-1.5.6Ͱ͸σϑΥϧτແޮʹͳΓ·ͨ͠

No content

Me • ௕໺խ޿ • PAUSE:KAZEBURO • גࣜձࣾϝϧΧϦ
 ϓϦϯγύϧΤϯδχΞ
 Site Reliability Engineering (SRE) νʔϜ • BASE, Inc ٕज़ΞυόΠβʔ

No content

ϝϧΧϦ • ࠃ಺࠷େڃͷϑϦϚΞϓϦ • 3෼Ͱ؆୯ʹग़඼ 1) ࣸਅΛࡱΔ 2) ঎඼৘ใΛهೖ 3) ग़඼ϘλϯΛԡ͢ • ҆৺҆શͳܾࡁɾऔҾ • ΤεΫϩʔ(͓ۚͷ΍ΓͱΓ͸౰͕ࣾؒʹհࡏ) • ಗ໊഑ૹ

KPI μ΢ϯϩʔυ਺ GMV(૯औҾֹ) 1ԯDLҎ্(JP+US+UK) ݄ؒ100ԯԁҎ্ ग़඼਺ 1೔100ສ඼Ҏ্

ϝϧΧϦΛࢧ͑Δ αʔόαΠυͷ ϓϩάϥϛϯάݴޠ

PHP 7

Golang https://github.com/golang-samples/gopher-vector

ϝϧΧϦΛࢧ͑Δϓϩάϥϛϯάݴޠ • ϝϧΧϦͷAPI͸΄΅શͯPHPʹΑΓ࣮૷ • ϚΠΫϩαʔϏε/ϛυϧ΢ΣΞͰ͸ Golang ͕த৺ • AI/MLͷ෼໺Ͱ͸Python΋࢖ΘΕΔ

1FSM͸ϝΠϯϦʔϜͳݴޠͰ͸ͳ͍

͕͔ͩ͠͠ খ͞ͳ Perl εΫϦϓτ͕αʔϏεͷ҆ఆՔಇΛࢧ͑ ཪํͱͯ͠େن໛ͳτϥϑΟοΫΛॲཧ͍ͯ͠Δɻ

slacklog

slacklog = cronlog + Slack notification

cronlog • cronlog ͸kazuho͞Μ࡞੒ͷPerlεΫϦϓτɻҾ਺ͱͯ͠༩͑ΒΕͨίϚϯ υΛ࣮ߦ͠ɺͦͷऴྃίʔυ͕ඇθϩͷ৔߹ͷΈɺίϚϯυͷग़ྗ (stdout, stderr)Λඪ४ग़ྗʹૹΔ • cronͰར༻͢Δ͜ͱͰΤϥʔ࣌ͷΈMAILTOʹϝʔϧૹ৴ • `5 * * * * cronlog -- ping -n 5 my-server 2>&1` • ʮ؂ࢹͱ͸ܧଓతͳςετʯͱ͍͏ݴ༿ΛੜΈग़ͨ͠

slacklog • ίϚϯυͷऴྃίʔυ͕ඇθϩ࣌ʹඪ४ग़ྗ΁ͷૹ৴ʹՃ͑ɺslack΁ͷ௨஌Λߦ͏ • @channel Λ௥Ճ͢Δ͔ࢦఆՄೳ • ىಈͱऴྃ࣌ʹSlack΁ͷ௨஌ɺsyslog΁΋ه࿥ • hostname,pid ͷ෇Ճ • slack΁ͷ௨஌͸ slackboard Λར༻ • ϝϧΧϦͷ΄΅શͯͷcronͰར༻͍ͯ͠Δ

slacklog use slackboard App Batch MySQL App Batch MySQL App Batch MySQL JP US UK slacklog slacklog slacklog slacklog slacklog slacklog slacklog slacklog slacklog slackborad slackborad slackborad JP US EU HTTP Load Balancer MySQL slacklog slacklog slacklog SlackͷAPI keyͷҰݩ؅ཧ Buffering/Queueing Slack https://github.com/cubicdaiya/slackboard

࣮ߦྫ $ slacklog -c test --log-channel test -- perl -E '$|=1;say "Hello";sleep 3;die' [2018-02-25T01:09:51][21066] *hostname* starting: perl -E $|=1;say "Hello";sleep 3;die [2018-02-25T01:09:51][21066] Hello [2018-02-25T01:09:54][21066] Died at -e line 1. [2018-02-25T01:09:54][21066] command exited with code:255 $

࣮ߦྫ(slack) ىಈऴྃϩά ίϚϯυͷ STDOUT STDERR

ಋೖޮՌ • batchͷΤϥʔͷݕग़ͱ௨஌ͷଈ࣌ੑΞοϓ • ϝʔϧ͸஗͍ɺ΋͘͠͸ΈΒΕͯͳ͍ • αʔόαΠυΤϯδχΞͷslack channelʹ௚઀ૹ৴ɻ඼࣭ͷ޲্ • ϩάΛ/dev/nullʹࣺͯͳ͍ӡ༻͕Մೳʹ

ಋೖޮՌ • ʮ͋ͷcronىಈ͍ͯ͠·͔͢ʁʯʮࡢ೔ϦϦʔεͨ͠batchɺਖ਼͘͠ಈ͍ ͯ·͔͢ʁʯͷΑ͏ͳΤϯδχΞ͔Βͷ֬ೝґཔͷ࡟ݮ • τΠϧͷ๾໓

঎඼ Ӿཡ ཤྺ SERVICE

Ӿཡཤྺ • ͓٬༷ͷ঎඼ӾཡͷཤྺΛαʔόʹ֨ೲ • UX޲্ͱͱ΋ʹɺTimeline Personalizationͷσʔλ ͱͯ͠΋׆༻ • ঎඼ͷӾཡ͸਺ઍreq/sec
 PerlͰߏங͞Εͨ ServiceͰॲཧ

App App App App App NGINX API(Perl) Q4M Worker (Perl) browse_item_service NGINX API(Perl) Q4M Worker (Perl) browse_item_service MySQL MySQL memcached memcached memcached "ߋ৽" "ࢀর" cluster (2) DBߋ৽&ཤྺऔಘ (3) cache࡞੒ (1) enqueue (1) cacheऔಘ/missͨ͠ΒDB App App App App App memcached memcached memcached NGINX API(Perl) Q4M Worker (Perl) browse_item_service DNS-RR by consul

CPAN modules • API • Plack, Gazelle • Worker • Parallel::Prefork • ڞ௨ • JSON::XS, DBD::mysql, Cache::Memcached::Fast::Safe Web Application Framework͸ະ࢖༻ Plack::(Request|Response)΋࢖ͬͯͳ͍ XS moduleΛ׆༻ͯ͠଎౓޲্

Performance • API/Worker͸Scale Out͕Մೳ • GazelleʹΑΔAPIαʔό͸े෼ʹߴ଎ • ϘτϧωοΫ͸ MySQL ͷϨϓϦέʔγϣϯ஗Ԇ • ௨ৗͷϨϓϦέʔγϣϯͰ͸਺ઍ transaction/s Ͱ஗Ԇൃੜ • MySQL 5.7ͷ Multi Thread Slave Λར༻ͯ͠΋ 18,000 t/s ͕ݶք • Sharding Λॳظઃܭʹ૊ΈࠐΈ

Performance (newRelic) ঎඼͝ͱͷPVΛܭଌ ͢Δ Service (Golang) Avg time: 3 msec Ӿཡཤྺ Service (Perl) Avg time: 4 msec PHP DBͳͲ

http://tech.mercari.com/entry/2018/02/26/110237

Կނ Perl?

ͳͥ Perl ? • kazeburo͕ॻ͍͔ͨΒ • System Call friendly ‣ OS ΍ ProcessɺNetwork ͷجૅΛֶͿͷʹద͍ͯ͠Δ • (Core Module ͷΈͰ࡞Ε͹) Deploy ͕༰қ • PerlͰഓΘΕٕͨज़ͷܧঝ ‣ mixi, livedoor (LINE), DeNA, ͸ͯͳ ͳͲେن໛WebαʔϏεΛࢧ͑ͯ(͍Δ|͖ͨ)ݴޠ

͜Ε͔Β • ϚΠΫϩαʔϏεɾίϯςφத৺ͷInfrustructure΁ • Serverlessɺk8sʹΑΔࣗಈԽ͞Εͨ Infrastructure • OS΍networkͳͲγεςϜΤϯδχΞϦϯάͷ஌͕ࣝ։ൃऀ͔Βԕ͘ͳΔ • Golang த৺ɺPerl ΠϯλϓϦλ͕ͳ͍؀ڥ

͜Ε͔Β • SREݚमͰPerlΛ৮ͬͯ΋Β͏ • ৽ଔ޲͚ʹ1ϲ݄͔Β1ϲ݄൒ఔ౓ͷSREݚमΛߦ͍ͬͯΔ • ͦͷதͰͻͱͭ͸PerlΛ࢖ͬͨ࢓૊Έʹ৮ΕΔػձΛͭ͘Δ • slacklogͷىಈϩάΛSlackʹૹΔػೳ͸SREݚमͰ࡞ͬͯ΋Βͬͨ΋ͷ • OSɾFile SystemɾProcessɾNetworkͱ͍ͬͨ௿ϨΠϠʔ΁ͷڵຯɾؔ৺
 UNIX ఩ֶʹ৮ΕΔػձ • ٕज़ͷఈ্͛ʹඞཁͳݴޠͷҰͭ

Ҏ্ careers.mercari.com/job/