Perl in Mercari YAPC::Okinawa 2018 ONNASON

700669515ee872152d8b9403c2a0cf8c?s=47 kazeburo
March 03, 2018

Perl in Mercari YAPC::Okinawa 2018 ONNASON

Perl in Mercari
YAPC::Okinawa 2018 ONNASON スペシャルセッション

700669515ee872152d8b9403c2a0cf8c?s=128

kazeburo

March 03, 2018
Tweet

Transcript

  1. 3.

    memcached UDPϦϑϨΫγϣϯDDoS • Πϯλʔωοτʹެ։͞Ε͍ͯΔmemcached 11211/UDPΛར༻ͯ͠UDP ϦϑϨ Ϋγϣϯ͕ߦ͑Δ • github.com ʹ1.3TbpsͷDDoS߈ܸ͕ൃੜ

    (2/28) • ͘͞ΒVPS͸11211/UDPΛϑΟϧλϦϯά • [ॏཁ] memcachedΛάϩʔόϧʹެ։͠ͳ͍ • UDPΛແޮԽ͢Δɻmemcached -U 0Ͱىಈ
 memcached-1.5.6Ͱ͸σϑΥϧτແޮʹͳΓ·ͨ͠
  2. 4.
  3. 6.
  4. 7.

    ϝϧΧϦ • ࠃ಺࠷େڃͷϑϦϚΞϓϦ • 3෼Ͱ؆୯ʹग़඼ 1) ࣸਅΛࡱΔ 2) ঎඼৘ใΛهೖ 3)

    ग़඼ϘλϯΛԡ͢ • ҆৺҆શͳܾࡁɾऔҾ • ΤεΫϩʔ(͓ۚͷ΍ΓͱΓ͸౰͕ࣾؒʹհࡏ) • ಗ໊഑ૹ
  5. 10.
  6. 15.
  7. 19.

    slacklog use slackboard App Batch MySQL App Batch MySQL App

    Batch MySQL JP US UK slacklog slacklog slacklog slacklog slacklog slacklog slacklog slacklog slacklog slackborad slackborad slackborad JP US EU HTTP Load Balancer MySQL slacklog slacklog slacklog SlackͷAPI keyͷҰݩ؅ཧ Buffering/Queueing Slack https://github.com/cubicdaiya/slackboard
  8. 20.

    ࣮ߦྫ $ slacklog -c test --log-channel test -- perl -E

    '$|=1;say "Hello";sleep 3;die' [2018-02-25T01:09:51][21066] *hostname* starting: perl -E $|=1;say "Hello";sleep 3;die [2018-02-25T01:09:51][21066] Hello [2018-02-25T01:09:54][21066] Died at -e line 1. [2018-02-25T01:09:54][21066] command exited with code:255 $
  9. 26.

    App App App App App NGINX API(Perl) Q4M Worker (Perl)

    browse_item_service NGINX API(Perl) Q4M Worker (Perl) browse_item_service MySQL MySQL memcached memcached memcached "ߋ৽" "ࢀর" cluster (2) DBߋ৽&ཤྺऔಘ (3) cache࡞੒ (1) enqueue (1) cacheऔಘ/missͨ͠ΒDB App App App App App memcached memcached memcached NGINX API(Perl) Q4M Worker (Perl) browse_item_service DNS-RR by consul
  10. 27.

    CPAN modules • API • Plack, Gazelle • Worker •

    Parallel::Prefork • ڞ௨ • JSON::XS, DBD::mysql, Cache::Memcached::Fast::Safe Web Application Framework͸ະ࢖༻ Plack::(Request|Response)΋࢖ͬͯͳ͍ XS moduleΛ׆༻ͯ͠଎౓޲্
  11. 28.

    Performance • API/Worker͸Scale Out͕Մೳ • GazelleʹΑΔAPIαʔό͸े෼ʹߴ଎ • ϘτϧωοΫ͸ MySQL ͷϨϓϦέʔγϣϯ஗Ԇ

    • ௨ৗͷϨϓϦέʔγϣϯͰ͸਺ઍ transaction/s Ͱ஗Ԇൃੜ • MySQL 5.7ͷ Multi Thread Slave Λར༻ͯ͠΋ 18,000 t/s ͕ݶք • Sharding Λॳظઃܭʹ૊ΈࠐΈ
  12. 29.

    Performance (newRelic) ঎඼͝ͱͷPVΛܭଌ ͢Δ Service (Golang) Avg time: 3 msec

    Ӿཡཤྺ Service (Perl) Avg time: 4 msec PHP DBͳͲ
  13. 32.

    ͳͥ Perl ? • kazeburo͕ॻ͍͔ͨΒ • System Call friendly ‣

    OS ΍ ProcessɺNetwork ͷجૅΛֶͿͷʹద͍ͯ͠Δ • (Core Module ͷΈͰ࡞Ε͹) Deploy ͕༰қ • PerlͰഓΘΕٕͨज़ͷܧঝ ‣ mixi, livedoor (LINE), DeNA, ͸ͯͳ ͳͲେن໛WebαʔϏεΛࢧ͑ͯ(͍Δ|͖ͨ)ݴޠ