Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Akka Streamsの縮退処理を用いた耐障害性の実現 〜 広告のリアルタイムオークションの事例 〜

Akka Streamsの縮退処理を用いた耐障害性の実現 〜 広告のリアルタイムオークションの事例 〜

「Scala将軍達の後の祭り」での発表資料です。

http://scala-syogun-matsuri.connpass.com/event/28124/

Ikuo Matsumura

March 26, 2016
Tweet

More Decks by Ikuo Matsumura

Other Decks in Programming

Transcript

  1. Akka StreamsͷॖୀॲཧΛ༻͍ͨ
    ଱ো֐ੑͷ࣮ݱ
    ʙ ޿ࠂͷϦΞϧλΠϜΦʔΫγϣϯͷࣄྫ ʙ

    View Slide

  2. ࣗݾ঺հ
    @ikuo (Ikuo Matsumura)
    2008~ ૊ࠐܥͷ෼ࢄγϛϡϨʔγϣϯͷઃܭ։ൃͳͲ; C++, MATLAB
    IBM Research - Tokyo
    2011~ ϨγϐӫཆՁͷࣗಈܭࢉͷઃܭ։ൃͳͲ; Ruby/Rails, Treetop
    Cookpad, Inc.
    2014~ SSP/RTBΦʔΫγϣϯͷઃܭ։ൃͳͲ; Scala/Akka
    CyberAgent, Inc.
    CA ProFit-X team

    View Slide

  3. ޿ࠂͷϦΞϧλΠϜΦʔΫγϣϯ (RTB)
    ߴ͍Մ༻ੑཁ݅ɿࢭΊΒΕͳ͍ ˍ ଴ͬͯ͘Εͳ͍
    ෆ࣮֬ͳ؀ڥɿ֎෦ͷଟ਺ͷೖࡳݩ(DSP)ʹৗ࣌઀ଓ
    ೖࡳ͕શͯἧΘͳ͍͜ͱ͸ڐ༰Ͱ͖Δ

    View Slide

  4. ଟ਺ͷ֎෦઀ଓͷো֐͔ΒγεςϜΛकΓ͍ͨ
    * Photo by FEMA/Joselyne Augus5no; A home in Gilchrist, Texas, designed to resist flood
    waters survived Hurricane Ike in 2008.

    View Slide

  5. ֎෦઀ଓͷো֐ͱΞϓϩʔν
    ʮͪ͜Βʯͷҟৗɿ઀ଓͷ಺෦ঢ়ଶͷҟৗ
    ྫɿHTTP ΫϥΠΞϯτͷActorͷঢ়ଶҟৗ
    → SupervisorʹΑΔҟৗݕ஌ͱActorͷ࠶ىಈ
    ʮ͋ͪΒʯͷҟৗɿϨΠςϯγѱԽɾແԠ౴Խ
    ྫɿ઀ଓઌͷܭࢉࢿݯ΍ଳҬͷṧഭ
    → ॖୀ (fallback)ɿॏཁͰͳ͍ϝοηʔδΛࣺͯͯ
    ɹɹϝϞϦރׇ (OutOfMemoryError) Λ๷͙

    View Slide

  6. ରࡦ1: BoundedMailboxΛ࢖͏
    bounded-mailbox {
    mailbox-type = "akka.dispatch.NonBlockingBoundedMailbox"
    }
    ✅ ᷓΕͨΒϝοηʔδࣺͯΔͷͰϝϞϦރׇ๷͛Δ

    View Slide

  7. ❌ Ձ஋ͷߴ͍ϝοηʔδ͕ࣺͯΒΕΔ
    ؒʹ߹͏ݟࠐΈͷখ͍͞ೖࡳཁٻ͕࢒ͬͯ͠·͏

    View Slide

  8. ରࡦ2: ༏ઌ౓͖ͭͷBoundedMailboxΛ࢖͏
    bounded-mailbox {
    mailbox-type = "akka.dispatch.BoundedStablePriorityMailbox"
    mailbox-push-timeout-time = 0s
    }
    ✅ ᷓΕͨΒϝοηʔδࣺͯΔͷͰϝϞϦރׇ๷͛Δ
    ✅ Ձ஋ͷߴ͍ϝοηʔδΛ࢒ͤΔ

    View Slide

  9. ❌ ద༻࿙ΕͷActor͕͋ΔͱϝϞϦރׇ๷͛ͳ͍
    UnboundedͳϝʔϧϘοΫεͷActor͕͋Γɺ͕ͦ͜
    ཯଎ʹͳΔͱϝʔϧϘοΫε͕ᷓΕΔ

    View Slide

  10. ରࡦ3: Akka StreamsΛ࢖͏
    ϝοηʔδ͸ִนϊʔυ (bulkhead) Λհͯ͠௨৴ϊʔυ (subscriber) ʹBroadcastɻ
    ௨৴ϊʔυ͸֎෦ཁٻ׬ྃ·ͰϝοηʔδΛ্ྲྀʹཁٻ͠ͳ͍ɻ

    View Slide

  11. ִนϊʔυ (bulkhead)
    val bulkhead =
    Flow
    .apply[MyMessage]
    .buffer(
    bulkheadBufferSize,
    OverflowStrategy.dropHead
    )
    όοϑΝͷ࠷େαΠζΛ௒͑Δͱɺݹ͍ϝοηʔδ
    ΛࣺͯΔ ( dropHead )

    View Slide

  12. ௨৴ϊʔυ (subscriber)
    val subscriberA = Sink(ActorSubscriber[MyMessage](actorA))
    class ProxyA extends Actor with ActorSubscriber {
    def receive = {
    case OnNext(msg: MyMessage) =>
    hostConnector
    .ask(httpReq(msg))
    .map(...)
    .onComplete(request(1))
    ...
    ֎෦ཁٻͷ׬ྃΛ଴͔ͬͯΒ࣍ͷϝοηʔδΛ্ྲྀ
    (bulkhead) ʹཁٻ͢ΔͷͰɺ֎෦ཁٻͷաఔͷͲ͕͜
    ཯଎ʹͳͬͯ΋ϝʔϧϘοΫε͕ᷓΕͳ͍

    View Slide

  13. Akka StreamsΛ࢖͏
    ✅ ᷓΕͨΒϝοηʔδࣺͯΔͷͰϝϞϦރׇ๷͛Δ
    ✅ Ձ஋ͷߴ͍ϝοηʔδΛ࢒ͤΔ
    ✅ ཯଎ཁҼ͕Ҡಈͯ͠΋࣮֬ʹॖୀͰ͖Δ

    View Slide

  14. CA ProFit-X
    • 2015೥3݄ʹϦϦʔε
    • ౰ॳ͔ΒAkka StreamsʹΑΔ
    ॖୀॲཧΛ૊ࠐΈ
    • ݱࡏ͸ඵؒ໿1ສ݅ͷϦΞϧλ
    ΠϜΦʔΫγϣϯΛ҆ఆͯ͠
    ॲཧ

    View Slide

  15. ·ͱΊ
    Akka StreamsΛ֎෦઀ଓͷॖୀͷͨΊʹ࢖͏͜ͱ
    Ͱɺʮ଴ͬͯ͘Εͳ͍ʯ؀ڥͰ֎෦઀ଓͷো֐ʢੑ
    ೳྼԽ ) ͔ΒγεςϜΛकΔྫΛ঺հ͠·ͨ͠ɻ
    ॲཧ଎౓ͱ͸ผͷ ଱ো֐ੑ ͷଆ໘ͰAkka StreamsΛ
    ׆༻͢Δྫͷ1ͭͱͯ͠ɺࢀߟʹͳΕ͹޾͍Ͱ͢ɻ

    View Slide