Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Perl in Mercari YAPC::Okinawa 2018 ONNASON

700669515ee872152d8b9403c2a0cf8c?s=47 kazeburo
March 03, 2018

Perl in Mercari YAPC::Okinawa 2018 ONNASON

Perl in Mercari
YAPC::Okinawa 2018 ONNASON スペシャルセッション

700669515ee872152d8b9403c2a0cf8c?s=128

kazeburo

March 03, 2018
Tweet

Transcript

  1. Perl in Mercari 2018 @kazeburo
 YAPC::Okinawa 2018 ONNASON

  2. େ੾ͳ͓஌Βͤ

  3. memcached UDPϦϑϨΫγϣϯDDoS • Πϯλʔωοτʹެ։͞Ε͍ͯΔmemcached 11211/UDPΛར༻ͯ͠UDP ϦϑϨ Ϋγϣϯ͕ߦ͑Δ • github.com ʹ1.3TbpsͷDDoS߈ܸ͕ൃੜ

    (2/28) • ͘͞ΒVPS͸11211/UDPΛϑΟϧλϦϯά • [ॏཁ] memcachedΛάϩʔόϧʹެ։͠ͳ͍ • UDPΛແޮԽ͢Δɻmemcached -U 0Ͱىಈ
 memcached-1.5.6Ͱ͸σϑΥϧτແޮʹͳΓ·ͨ͠
  4. None
  5. Me • ௕໺խ޿ • PAUSE:KAZEBURO • גࣜձࣾϝϧΧϦ
 ϓϦϯγύϧΤϯδχΞ
 Site Reliability

    Engineering (SRE) νʔϜ • BASE, Inc ٕज़ΞυόΠβʔ
  6. None
  7. ϝϧΧϦ • ࠃ಺࠷େڃͷϑϦϚΞϓϦ • 3෼Ͱ؆୯ʹग़඼ 1) ࣸਅΛࡱΔ 2) ঎඼৘ใΛهೖ 3)

    ग़඼ϘλϯΛԡ͢ • ҆৺҆શͳܾࡁɾऔҾ • ΤεΫϩʔ(͓ۚͷ΍ΓͱΓ͸౰͕ࣾؒʹհࡏ) • ಗ໊഑ૹ
  8. KPI μ΢ϯϩʔυ਺ GMV(૯औҾֹ) 1ԯDLҎ্(JP+US+UK) ݄ؒ100ԯԁҎ্ ग़඼਺ 1೔100ສ඼Ҏ্

  9. ϝϧΧϦΛࢧ͑Δ αʔόαΠυͷ ϓϩάϥϛϯάݴޠ

  10. PHP 7

  11. Golang https://github.com/golang-samples/gopher-vector

  12. ϝϧΧϦΛࢧ͑Δϓϩάϥϛϯάݴޠ • ϝϧΧϦͷAPI͸΄΅શͯPHPʹΑΓ࣮૷ • ϚΠΫϩαʔϏε/ϛυϧ΢ΣΞͰ͸ Golang ͕த৺ • AI/MLͷ෼໺Ͱ͸Python΋࢖ΘΕΔ

  13. 1FSM͸ϝΠϯϦʔϜͳݴޠͰ͸ͳ͍

  14. ͕͔ͩ͠͠ খ͞ͳ Perl εΫϦϓτ͕αʔϏεͷ҆ఆՔಇΛࢧ͑ ཪํͱͯ͠େن໛ͳτϥϑΟοΫΛॲཧ͍ͯ͠Δɻ

  15. slacklog

  16. slacklog = cronlog + Slack notification

  17. cronlog • cronlog ͸kazuho͞Μ࡞੒ͷPerlεΫϦϓτɻҾ਺ͱͯ͠༩͑ΒΕͨίϚϯ υΛ࣮ߦ͠ɺͦͷऴྃίʔυ͕ඇθϩͷ৔߹ͷΈɺίϚϯυͷग़ྗ (stdout, stderr)Λඪ४ग़ྗʹૹΔ • cronͰར༻͢Δ͜ͱͰΤϥʔ࣌ͷΈMAILTOʹϝʔϧૹ৴ •

    `5 * * * * cronlog -- ping -n 5 my-server 2>&1` • ʮ؂ࢹͱ͸ܧଓతͳςετʯͱ͍͏ݴ༿ΛੜΈग़ͨ͠
  18. slacklog • ίϚϯυͷऴྃίʔυ͕ඇθϩ࣌ʹඪ४ग़ྗ΁ͷૹ৴ʹՃ͑ɺslack΁ͷ௨஌Λߦ͏ • @channel Λ௥Ճ͢Δ͔ࢦఆՄೳ • ىಈͱऴྃ࣌ʹSlack΁ͷ௨஌ɺsyslog΁΋ه࿥ • hostname,pid

    ͷ෇Ճ • slack΁ͷ௨஌͸ slackboard Λར༻ • ϝϧΧϦͷ΄΅શͯͷcronͰར༻͍ͯ͠Δ
  19. slacklog use slackboard App Batch MySQL App Batch MySQL App

    Batch MySQL JP US UK slacklog slacklog slacklog slacklog slacklog slacklog slacklog slacklog slacklog slackborad slackborad slackborad JP US EU HTTP Load Balancer MySQL slacklog slacklog slacklog SlackͷAPI keyͷҰݩ؅ཧ Buffering/Queueing Slack https://github.com/cubicdaiya/slackboard
  20. ࣮ߦྫ $ slacklog -c test --log-channel test -- perl -E

    '$|=1;say "Hello";sleep 3;die' [2018-02-25T01:09:51][21066] *hostname* starting: perl -E $|=1;say "Hello";sleep 3;die [2018-02-25T01:09:51][21066] Hello [2018-02-25T01:09:54][21066] Died at -e line 1. [2018-02-25T01:09:54][21066] command exited with code:255 $
  21. ࣮ߦྫ(slack) ىಈऴྃϩά ίϚϯυͷ STDOUT STDERR

  22. ಋೖޮՌ • batchͷΤϥʔͷݕग़ͱ௨஌ͷଈ࣌ੑΞοϓ • ϝʔϧ͸஗͍ɺ΋͘͠͸ΈΒΕͯͳ͍ • αʔόαΠυΤϯδχΞͷslack channelʹ௚઀ૹ৴ɻ඼࣭ͷ޲্ • ϩάΛ/dev/nullʹࣺͯͳ͍ӡ༻͕Մೳʹ

  23. ಋೖޮՌ • ʮ͋ͷcronىಈ͍ͯ͠·͔͢ʁʯʮࡢ೔ϦϦʔεͨ͠batchɺਖ਼͘͠ಈ͍ ͯ·͔͢ʁʯͷΑ͏ͳΤϯδχΞ͔Βͷ֬ೝґཔͷ࡟ݮ • τΠϧͷ๾໓

  24. ঎඼ Ӿཡ ཤྺ SERVICE

  25. Ӿཡཤྺ • ͓٬༷ͷ঎඼ӾཡͷཤྺΛαʔόʹ֨ೲ • UX޲্ͱͱ΋ʹɺTimeline Personalizationͷσʔλ ͱͯ͠΋׆༻ • ঎඼ͷӾཡ͸਺ઍreq/sec
 PerlͰߏங͞Εͨ

    ServiceͰॲཧ
  26. App App App App App NGINX API(Perl) Q4M Worker (Perl)

    browse_item_service NGINX API(Perl) Q4M Worker (Perl) browse_item_service MySQL MySQL memcached memcached memcached "ߋ৽" "ࢀর" cluster (2) DBߋ৽&ཤྺऔಘ (3) cache࡞੒ (1) enqueue (1) cacheऔಘ/missͨ͠ΒDB App App App App App memcached memcached memcached NGINX API(Perl) Q4M Worker (Perl) browse_item_service DNS-RR by consul
  27. CPAN modules • API • Plack, Gazelle • Worker •

    Parallel::Prefork • ڞ௨ • JSON::XS, DBD::mysql, Cache::Memcached::Fast::Safe Web Application Framework͸ະ࢖༻ Plack::(Request|Response)΋࢖ͬͯͳ͍ XS moduleΛ׆༻ͯ͠଎౓޲্
  28. Performance • API/Worker͸Scale Out͕Մೳ • GazelleʹΑΔAPIαʔό͸े෼ʹߴ଎ • ϘτϧωοΫ͸ MySQL ͷϨϓϦέʔγϣϯ஗Ԇ

    • ௨ৗͷϨϓϦέʔγϣϯͰ͸਺ઍ transaction/s Ͱ஗Ԇൃੜ • MySQL 5.7ͷ Multi Thread Slave Λར༻ͯ͠΋ 18,000 t/s ͕ݶք • Sharding Λॳظઃܭʹ૊ΈࠐΈ
  29. Performance (newRelic) ঎඼͝ͱͷPVΛܭଌ ͢Δ Service (Golang) Avg time: 3 msec

    Ӿཡཤྺ Service (Perl) Avg time: 4 msec PHP DBͳͲ
  30. http://tech.mercari.com/entry/2018/02/26/110237

  31. Կނ Perl?

  32. ͳͥ Perl ? • kazeburo͕ॻ͍͔ͨΒ • System Call friendly ‣

    OS ΍ ProcessɺNetwork ͷجૅΛֶͿͷʹద͍ͯ͠Δ • (Core Module ͷΈͰ࡞Ε͹) Deploy ͕༰қ • PerlͰഓΘΕٕͨज़ͷܧঝ ‣ mixi, livedoor (LINE), DeNA, ͸ͯͳ ͳͲେن໛WebαʔϏεΛࢧ͑ͯ(͍Δ|͖ͨ)ݴޠ
  33. ͜Ε͔Β • ϚΠΫϩαʔϏεɾίϯςφத৺ͷInfrustructure΁ • Serverlessɺk8sʹΑΔࣗಈԽ͞Εͨ Infrastructure • OS΍networkͳͲγεςϜΤϯδχΞϦϯάͷ஌͕ࣝ։ൃऀ͔Βԕ͘ͳΔ • Golang

    த৺ɺPerl ΠϯλϓϦλ͕ͳ͍؀ڥ
  34. ͜Ε͔Β • SREݚमͰPerlΛ৮ͬͯ΋Β͏ • ৽ଔ޲͚ʹ1ϲ݄͔Β1ϲ݄൒ఔ౓ͷSREݚमΛߦ͍ͬͯΔ • ͦͷதͰͻͱͭ͸PerlΛ࢖ͬͨ࢓૊Έʹ৮ΕΔػձΛͭ͘Δ • slacklogͷىಈϩάΛSlackʹૹΔػೳ͸SREݚमͰ࡞ͬͯ΋Βͬͨ΋ͷ •

    OSɾFile SystemɾProcessɾNetworkͱ͍ͬͨ௿ϨΠϠʔ΁ͷڵຯɾؔ৺
 UNIX ఩ֶʹ৮ΕΔػձ • ٕज़ͷఈ্͛ʹඞཁͳݴޠͷҰͭ
  35. Ҏ্ careers.mercari.com/job/