Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Akka Streamsの縮退処理を用いた耐障害性の実現 〜 広告のリアルタイムオークションの事例 〜

Akka Streamsの縮退処理を用いた耐障害性の実現 〜 広告のリアルタイムオークションの事例 〜

「Scala将軍達の後の祭り」での発表資料です。

http://scala-syogun-matsuri.connpass.com/event/28124/

Fef432a66695449f818f1d4e29b2e897?s=128

Ikuo Matsumura

March 26, 2016
Tweet

Transcript

  1. Akka StreamsͷॖୀॲཧΛ༻͍ͨ ଱ো֐ੑͷ࣮ݱ ʙ ޿ࠂͷϦΞϧλΠϜΦʔΫγϣϯͷࣄྫ ʙ

  2. ࣗݾ঺հ @ikuo (Ikuo Matsumura) 2008~ ૊ࠐܥͷ෼ࢄγϛϡϨʔγϣϯͷઃܭ։ൃͳͲ; C++, MATLAB IBM Research

    - Tokyo 2011~ ϨγϐӫཆՁͷࣗಈܭࢉͷઃܭ։ൃͳͲ; Ruby/Rails, Treetop Cookpad, Inc. 2014~ SSP/RTBΦʔΫγϣϯͷઃܭ։ൃͳͲ; Scala/Akka CyberAgent, Inc. CA ProFit-X team
  3. ޿ࠂͷϦΞϧλΠϜΦʔΫγϣϯ (RTB) ߴ͍Մ༻ੑཁ݅ɿࢭΊΒΕͳ͍ ˍ ଴ͬͯ͘Εͳ͍ ෆ࣮֬ͳ؀ڥɿ֎෦ͷଟ਺ͷೖࡳݩ(DSP)ʹৗ࣌઀ଓ ೖࡳ͕શͯἧΘͳ͍͜ͱ͸ڐ༰Ͱ͖Δ

  4. ଟ਺ͷ֎෦઀ଓͷো֐͔ΒγεςϜΛकΓ͍ͨ * Photo by FEMA/Joselyne Augus5no; A home in Gilchrist,

    Texas, designed to resist flood waters survived Hurricane Ike in 2008.
  5. ֎෦઀ଓͷো֐ͱΞϓϩʔν ʮͪ͜Βʯͷҟৗɿ઀ଓͷ಺෦ঢ়ଶͷҟৗ ྫɿHTTP ΫϥΠΞϯτͷActorͷঢ়ଶҟৗ → SupervisorʹΑΔҟৗݕ஌ͱActorͷ࠶ىಈ ʮ͋ͪΒʯͷҟৗɿϨΠςϯγѱԽɾແԠ౴Խ ྫɿ઀ଓઌͷܭࢉࢿݯ΍ଳҬͷṧഭ → ॖୀ

    (fallback)ɿॏཁͰͳ͍ϝοηʔδΛࣺͯͯ ɹɹϝϞϦރׇ (OutOfMemoryError) Λ๷͙
  6. ରࡦ1: BoundedMailboxΛ࢖͏ bounded-mailbox { mailbox-type = "akka.dispatch.NonBlockingBoundedMailbox" } ✅ ᷓΕͨΒϝοηʔδࣺͯΔͷͰϝϞϦރׇ๷͛Δ

  7. ❌ Ձ஋ͷߴ͍ϝοηʔδ͕ࣺͯΒΕΔ ؒʹ߹͏ݟࠐΈͷখ͍͞ೖࡳཁٻ͕࢒ͬͯ͠·͏

  8. ରࡦ2: ༏ઌ౓͖ͭͷBoundedMailboxΛ࢖͏ bounded-mailbox { mailbox-type = "akka.dispatch.BoundedStablePriorityMailbox" mailbox-push-timeout-time = 0s

    } ✅ ᷓΕͨΒϝοηʔδࣺͯΔͷͰϝϞϦރׇ๷͛Δ ✅ Ձ஋ͷߴ͍ϝοηʔδΛ࢒ͤΔ
  9. ❌ ద༻࿙ΕͷActor͕͋ΔͱϝϞϦރׇ๷͛ͳ͍ UnboundedͳϝʔϧϘοΫεͷActor͕͋Γɺ͕ͦ͜ ཯଎ʹͳΔͱϝʔϧϘοΫε͕ᷓΕΔ

  10. ରࡦ3: Akka StreamsΛ࢖͏ ϝοηʔδ͸ִนϊʔυ (bulkhead) Λհͯ͠௨৴ϊʔυ (subscriber) ʹBroadcastɻ ௨৴ϊʔυ͸֎෦ཁٻ׬ྃ·ͰϝοηʔδΛ্ྲྀʹཁٻ͠ͳ͍ɻ

  11. ִนϊʔυ (bulkhead) val bulkhead = Flow .apply[MyMessage] .buffer( bulkheadBufferSize, OverflowStrategy.dropHead

    ) όοϑΝͷ࠷େαΠζΛ௒͑Δͱɺݹ͍ϝοηʔδ ΛࣺͯΔ ( dropHead )
  12. ௨৴ϊʔυ (subscriber) val subscriberA = Sink(ActorSubscriber[MyMessage](actorA)) class ProxyA extends Actor

    with ActorSubscriber { def receive = { case OnNext(msg: MyMessage) => hostConnector .ask(httpReq(msg)) .map(...) .onComplete(request(1)) ... ֎෦ཁٻͷ׬ྃΛ଴͔ͬͯΒ࣍ͷϝοηʔδΛ্ྲྀ (bulkhead) ʹཁٻ͢ΔͷͰɺ֎෦ཁٻͷաఔͷͲ͕͜ ཯଎ʹͳͬͯ΋ϝʔϧϘοΫε͕ᷓΕͳ͍
  13. Akka StreamsΛ࢖͏ ✅ ᷓΕͨΒϝοηʔδࣺͯΔͷͰϝϞϦރׇ๷͛Δ ✅ Ձ஋ͷߴ͍ϝοηʔδΛ࢒ͤΔ ✅ ཯଎ཁҼ͕Ҡಈͯ͠΋࣮֬ʹॖୀͰ͖Δ

  14. CA ProFit-X • 2015೥3݄ʹϦϦʔε • ౰ॳ͔ΒAkka StreamsʹΑΔ ॖୀॲཧΛ૊ࠐΈ • ݱࡏ͸ඵؒ໿1ສ݅ͷϦΞϧλ

    ΠϜΦʔΫγϣϯΛ҆ఆͯ͠ ॲཧ
  15. ·ͱΊ Akka StreamsΛ֎෦઀ଓͷॖୀͷͨΊʹ࢖͏͜ͱ Ͱɺʮ଴ͬͯ͘Εͳ͍ʯ؀ڥͰ֎෦઀ଓͷো֐ʢੑ ೳྼԽ ) ͔ΒγεςϜΛकΔྫΛ঺հ͠·ͨ͠ɻ ॲཧ଎౓ͱ͸ผͷ ଱ো֐ੑ ͷଆ໘ͰAkka

    StreamsΛ ׆༻͢Δྫͷ1ͭͱͯ͠ɺࢀߟʹͳΕ͹޾͍Ͱ͢ɻ