Akka Streamsの縮退処理を用いた耐障害性の実現 〜 広告のリアルタイムオークションの事例 〜

Transcript

1. Akka StreamsͷॖୀॲཧΛ༻͍ͨ ଱ো֐ੑͷ࣮ݱ ʙ ޿ࠂͷϦΞϧλΠϜΦʔΫγϣϯͷࣄྫ ʙ

2. ࣗݾ঺հ @ikuo (Ikuo Matsumura) 2008~ ૊ࠐܥͷ෼ࢄγϛϡϨʔγϣϯͷઃܭ։ൃͳͲ; C++, MATLAB IBM Research

   - Tokyo 2011~ ϨγϐӫཆՁͷࣗಈܭࢉͷઃܭ։ൃͳͲ; Ruby/Rails, Treetop Cookpad, Inc. 2014~ SSP/RTBΦʔΫγϣϯͷઃܭ։ൃͳͲ; Scala/Akka CyberAgent, Inc. CA ProFit-X team

3. ޿ࠂͷϦΞϧλΠϜΦʔΫγϣϯ (RTB) ߴ͍Մ༻ੑཁ݅ɿࢭΊΒΕͳ͍ ˍ ଴ͬͯ͘Εͳ͍ ෆ࣮֬ͳ؀ڥɿ֎෦ͷଟ਺ͷೖࡳݩ(DSP)ʹৗ࣌઀ଓ ೖࡳ͕શͯἧΘͳ͍͜ͱ͸ڐ༰Ͱ͖Δ

4. ଟ਺ͷ֎෦઀ଓͷো֐͔ΒγεςϜΛकΓ͍ͨ * Photo by FEMA/Joselyne Augus5no; A home in Gilchrist,

   Texas, designed to resist flood waters survived Hurricane Ike in 2008.

5. ֎෦઀ଓͷো֐ͱΞϓϩʔν ʮͪ͜Βʯͷҟৗɿ઀ଓͷ಺෦ঢ়ଶͷҟৗ ྫɿHTTP ΫϥΠΞϯτͷActorͷঢ়ଶҟৗ → SupervisorʹΑΔҟৗݕ஌ͱActorͷ࠶ىಈ ʮ͋ͪΒʯͷҟৗɿϨΠςϯγѱԽɾແԠ౴Խ ྫɿ઀ଓઌͷܭࢉࢿݯ΍ଳҬͷṧഭ → ॖୀ

   (fallback)ɿॏཁͰͳ͍ϝοηʔδΛࣺͯͯ ɹɹϝϞϦރׇ (OutOfMemoryError) Λ๷͙

6. ରࡦ1: BoundedMailboxΛ࢖͏ bounded-mailbox { mailbox-type = "akka.dispatch.NonBlockingBoundedMailbox" } ✅ ᷓΕͨΒϝοηʔδࣺͯΔͷͰϝϞϦރׇ๷͛Δ

7. ❌ Ձ஋ͷߴ͍ϝοηʔδ͕ࣺͯΒΕΔ ؒʹ߹͏ݟࠐΈͷখ͍͞ೖࡳཁٻ͕࢒ͬͯ͠·͏

8. ରࡦ2: ༏ઌ౓͖ͭͷBoundedMailboxΛ࢖͏ bounded-mailbox { mailbox-type = "akka.dispatch.BoundedStablePriorityMailbox" mailbox-push-timeout-time = 0s

   } ✅ ᷓΕͨΒϝοηʔδࣺͯΔͷͰϝϞϦރׇ๷͛Δ ✅ Ձ஋ͷߴ͍ϝοηʔδΛ࢒ͤΔ

9. ❌ ద༻࿙ΕͷActor͕͋ΔͱϝϞϦރׇ๷͛ͳ͍ UnboundedͳϝʔϧϘοΫεͷActor͕͋Γɺ͕ͦ͜ ཯଎ʹͳΔͱϝʔϧϘοΫε͕ᷓΕΔ

10. ରࡦ3: Akka StreamsΛ࢖͏ ϝοηʔδ͸ִนϊʔυ (bulkhead) Λհͯ͠௨৴ϊʔυ (subscriber) ʹBroadcastɻ ௨৴ϊʔυ͸֎෦ཁٻ׬ྃ·ͰϝοηʔδΛ্ྲྀʹཁٻ͠ͳ͍ɻ

11. ִนϊʔυ (bulkhead) val bulkhead = Flow .apply[MyMessage] .buffer( bulkheadBufferSize, OverflowStrategy.dropHead

    ) όοϑΝͷ࠷େαΠζΛ௒͑Δͱɺݹ͍ϝοηʔδ ΛࣺͯΔ ( dropHead )

12. ௨৴ϊʔυ (subscriber) val subscriberA = Sink(ActorSubscriber[MyMessage](actorA)) class ProxyA extends Actor

    with ActorSubscriber { def receive = { case OnNext(msg: MyMessage) => hostConnector .ask(httpReq(msg)) .map(...) .onComplete(request(1)) ... ֎෦ཁٻͷ׬ྃΛ଴͔ͬͯΒ࣍ͷϝοηʔδΛ্ྲྀ (bulkhead) ʹཁٻ͢ΔͷͰɺ֎෦ཁٻͷաఔͷͲ͕͜ ཯଎ʹͳͬͯ΋ϝʔϧϘοΫε͕ᷓΕͳ͍

13. Akka StreamsΛ࢖͏ ✅ ᷓΕͨΒϝοηʔδࣺͯΔͷͰϝϞϦރׇ๷͛Δ ✅ Ձ஋ͷߴ͍ϝοηʔδΛ࢒ͤΔ ✅ ཯଎ཁҼ͕Ҡಈͯ͠΋࣮֬ʹॖୀͰ͖Δ

14. CA ProFit-X • 2015೥3݄ʹϦϦʔε • ౰ॳ͔ΒAkka StreamsʹΑΔ ॖୀॲཧΛ૊ࠐΈ • ݱࡏ͸ඵؒ໿1ສ݅ͷϦΞϧλ

    ΠϜΦʔΫγϣϯΛ҆ఆͯ͠ ॲཧ

15. ·ͱΊ Akka StreamsΛ֎෦઀ଓͷॖୀͷͨΊʹ࢖͏͜ͱ Ͱɺʮ଴ͬͯ͘Εͳ͍ʯ؀ڥͰ֎෦઀ଓͷো֐ʢੑ ೳྼԽ ) ͔ΒγεςϜΛकΔྫΛ঺հ͠·ͨ͠ɻ ॲཧ଎౓ͱ͸ผͷ ଱ো֐ੑ ͷଆ໘ͰAkka

    StreamsΛ ׆༻͢Δྫͷ1ͭͱͯ͠ɺࢀߟʹͳΕ͹޾͍Ͱ͢ɻ