Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Implementing DMM API Gateway in Akka Streams and HTTP

Implementing DMM API Gateway in Akka Streams and HTTP

Slides for ScalaMatsuri 2017
http://2017.scalamatsuri.org/index_en.html#schedule

DMM.com Labo Co.,Ltd. Yukio Kakumoto/Masateru Nishimura

Masateru Nishimura

February 25, 2017
Tweet

More Decks by Masateru Nishimura

Other Decks in Programming

Transcript

  1. Implementing
    DMM API Gateway in
    Akka Streams and HTTP
    %..ͷ"1*(BUFXBZΛ"LLB4USFBNTͱ
    "LLB)551Ͱ࡞ΓࠐΜͰΈͨ
    DMM.com labo Co.,Ltd. Yukio Kakumoto / Masateru Nishimura
    Feb. 25, 2017 in ScalaMatsuri 2017

    View Slide

  2. Copyright © since 1998 DMM All Rights Reserved. 2
    ֯ຊ޾ੜ ੢ଜ੓ً
    %..DPNϥϘ γεςϜຊ෦ ϓϥοτϑΥʔϜ։ൃ෦
    About us
    Yukio Kakumoto & Masateru Nishimura
    DMM.com Labo Co.,Ltd.
    We develop and improve our micro
    service platform.

    View Slide

  3. Copyright © since 1998 DMM All Rights Reserved. 3
    %..DPN͸೔ຊΛ୅ද͢Δ&$αΠτͷͻͱͭͰ͢ɻ
    About DMM.com Group
    DMM.com is one of
    Japan's leading
    E-commerce websites.
    The Outline of DMM's website and
    the company:
    · 2.5 billion page view per month.
    · Managing more than 40 services.
    · Members exceeding 20 million.
    · Unit sales of about $ 700 million.
    · Group sales of about $ 1.4 billion.
    (Converted by: 1USD=100JPY)

    View Slide

  4. Copyright © since 1998 DMM All Rights Reserved. 4
    ೥લ͔Β͍Ζ͍Ζ΍͖ͬͯ·͕ͨ͠ɺࠓճɺ"1*(BUFXBZͱೝ
    Մ"1*Λ "LLB4USFBNTBOE)551Ͱ࡞Γ௚͠·ͨ͠ɻ
    Refurbishment history of DMM’s platform
    OAuth 2.0 Connect API (Fuel PHP)
    API Gateway
    (Fuel PHP) Non-blocking API GW
    which was built in
    OAuth2.0 API
    Scala 2.11 /
    Akka Streams
    and HTTP
    Non-blocking API Gateway
    (Java7/Vert.x2)
    Countermeasure for C10K
    www.dmm.com /
    www.dmm.co.jp
    LAMP application
    Typical problems,
    large and monolithic
    state was given by
    business growth
    priority. Another microservices Purchase info API, Settlement agent API etc.. (Fuel PHP)
    Unified frontend shopping cart (FuelPHP/Couchbase)
    Legacy emoney systems
    Legacy shopping carts
    Unified Emoney API (Java7/Vert.x2/In-memory Data Grid)
    new Member API (Java8/Spring)
    Old Member API (LAMP)
    since ‘03〜? ‘14 Q1 ‘15 Q2 ‘16 Q3 ‘17 Q1〜
    Today’s agenda.

    View Slide

  5. Copyright © since 1998 DMM All Rights Reserved. 5
    ຊ೔͓࿩͢Δ಺༰
    Today's agenda
    ● Project overview
    ● Architecture design
    ● Designing Flow Graphs
    ● Implementing Router with Graph API
    ● Testing the Performance of Gateway

    View Slide

  6. Copyright © since 1998 DMM All Rights Reserved. 6
    Project overview
    ϓϩδΣΫτ֓ཁ
    6

    View Slide

  7. Copyright © since 1998 DMM All Rights Reserved. 7
    ೝՄύϑΥʔϚϯεͷڧԽͳͲΛ໨తʹɺ"1*(BUFXBZͱೝՄ
    "1*Λ"LLBͰ౷߹͠·ͨ͠ɻ
    After
    Before
    What we have done
    Backend
    API
    servers
    API-GW
    clients
    (API Gateway)
    (OAuth2.0 API)
    (Unified API Gateway and OAuth2.0 API)
    The purpose of the project
    ・To increase authorization
    performance.
    ・To centralize management of
    API routing and authorizing
    (to do in the future).

    View Slide

  8. Copyright © since 1998 DMM All Rights Reserved. 8
    "1*(BUFXBZ͸ࣾ಺֎ͷଟ਺ͷγεςϜ͔Βͷॲཧͷ૭ޱͱͳΔ
    ॏཁͳγεςϜͰ͢ɻ
    Responsibility of API Gateway
    LB
    LB
    ・Mobile devices
    ・3rd party
    etc.
    Backend
    API
    servers
    ・HTTPS TLS decoder
    ・Ephemeral port buffer
    API Gateway (Today's agenda.)
    ・Routing to backend APIs
    ・Authentication with OAuth2.0

    Actual to
    x,000req/sec
    during peak time.

    Managing of
    ・member
    ・billing
    ・accounting
    etc.

    View Slide

  9. Copyright © since 1998 DMM All Rights Reserved. 9
    Architecture design
    ΞʔΩςΫνϟσβΠϯʹ͍ͭͯ
    9

    View Slide

  10. Copyright © since 1998 DMM All Rights Reserved. 10
    "LLB4USFBNTΛ࣠ʹ"LLB)5514FSWFS $MJFOUΛ࠾༻
    ετϨʔδ͸ .Z42-ɾ3FEJT 4FOUJOFM
    ɾ)BTIJDPSQ7BVMU
    Scala 2.11 on JVM8
    Akka Streams 2.4 and HTTP 10.0 Low-level API
    Original Framework
    Router
    Hashicorp Valut driver
    Endpoint plugins
    HTTP(S) endpoint plugin
    OAuth2.0
    endpoint
    plugin
    ScalikeJDBC
    Rediscala
    Incoming
    plugins
    Outgoing
    plugins
    Akka
    HTTP
    Server
    Akka
    HTTP
    Client
    (HTTP(S) API driver)
    Technology stack
    Backend
    API
    servers
    storages
    Redis®
    Redis Sentinel®
    Hashicorp Vault
    API-GW
    clients

    View Slide

  11. Copyright © since 1998 DMM All Rights Reserved. 11
    શͯͷετϨʔδυϥΠό͸'VUVSFʹରԠ͍ͯ͠·͢ɻ
    Technology stack (storages & drivers)
    ScalikeJDBC (using MySQL)
    • It's Query driver which has SQL interpolation function that is very easy to use.
    • Future-based asynchronous interface.
    Rediscala (using Redis® for cache storage)
    • Supporting failover using Redis Sentinel® (discovery service for Redis®).
    • Future-based asynchronous interface.
    (Temporary solution for taking over from PHP version and running in parallel.)
    Hashicorp Vault with original driver implements Akka HTTP Client
    • For the purpose of keeping secret data.
    (For example DB password, salt for hash function etc.)
    • Unified akka HTTP Clinet because it’s interface is HTTP REST API.
    • Future-based asynchronous interface (via Akka Streams’ mat).

    View Slide

  12. Copyright © since 1998 DMM All Rights Reserved. 12
    ߴ৴པͷ+7.ͱ.BWFOϦϙδτϦͷ๛෋ͳϥΠϒϥϦ܈ʹՃ͑
    "LLB4USFBNTͷόοΫϓϨογϟʔ͕͋Ε͹׬ᘳʂ
    Why did we adopt Akka Streams?
    As API Gateway, stability is the most important, and productivity
    (fulfillment of libraries and drivers etc.) is also important.
    Stability
    • Running on JVM 8 which is a runtime of high achievement and reliability.
    • Using Akka with sufficient operational results for internal messaging.
    • Having protection of internal messaging flow by Akka streams’ back pressure.
    Productivity
    • Various libraries and drivers in several repositories. e.g. Maven, Ivy, etc.
    • Scala features, type safety and excellent syntax.
    Akka Streams is a comprehensive powerful solution!

    View Slide

  13. Copyright © since 1998 DMM All Rights Reserved. 13
    όοΫϓϨογϟʔͱ͸ɺϑϩʔ੍ޚʹ͓͍ͯԼྲྀͷόοϑΝ͕
    ҲΕͳ͍Α͏্ྲྀ΁࿈࠯తʹྲྀྔ৘ใΛ௨஌͢Δ࢓૊Έͷ͜ͱ
    What is back pressure?
    In information terminology, in order to prevent
    the downstream buffer from overflowing in flow control,
    flow information is notified in a chain manner to the upstream.
    Part of supported libraries
    • Akka Streams(Scala)
    • Scalaz Stream(Scala)
    • Node.js (JavaScript)
    • ReactiveX
    • RxJava
    • RxScala
    • RxJS etc.
    Implementation depends on each language, library, and backend queue system.
    Publisher
    (upstream)
    Subscriber
    (downstream)
    For example in Akka streams’ back pressure.
    ‘Dynamic push / pull mode’
    ✉✉✉✉✉
    request(5)

    View Slide

  14. Copyright © since 1998 DMM All Rights Reserved. 14
    "1*(BUFXBZͳͷͰϝϞϦ಺ͷϧʔτઃఆͷϦϩʔυ͕ඞཁɻ
    )JHIMFWFM"1*Ͱ͸ϧʔςΟϯάͷ%4-͕ϋʔυίʔυͷͨΊº
    The use of API Gateway requires
    reloading of route setting in memory
    at an arbitrary timing.
    Implementing this with the Akka
    HTTP High-level API can not satisfy
    the requirement, because routing
    DSL is hard code.
    Low-level API High-level API
    Flexibility Yes.
    Akka HTTP can
    bind original flow
    implimented by
    ourself.
    No.
    Hard-coded routing
    DSL generates flow
    to bind to Akka
    HTTP.
    Difficulty Hard.
    Requireing of
    understanding of
    Akka streams
    Graph DSL.
    Easy.
    Routing DSL is
    Spray-like simpile
    and intuitive
    directive.
    We chose Akka HTTP Low-level API

    View Slide

  15. Copyright © since 1998 DMM All Rights Reserved. 15
    ϧʔτ৘ใΛ)0$0/Ͱهड़͠ɺ)61γάφϧͰࠩ͠ସ͑Δɻ
    ϦϓϨʔεલγεςϜͷઃఆ͕KTPOͷͨΊ্Ґޓ׵ͱΕΔɻ

    HOCON based original routing system
    routes.
    before.json
    (JSON)
    routes.conf
    (HOCON)
    routes.
    before.conf
    (HOCON)
    Copy as is
    (and rename)
    include
    SIGHUP!
    In a common practice,
    when the application
    receives a HUP
    signal, it reloads
    route information.
    HOCON is upward compatible with json.
    By using this, compatibility can be keeping by including the
    pre-replacement configuration as it is.

    View Slide

  16. Copyright © since 1998 DMM All Rights Reserved. 16
    +40/Ͱهड़͍ͯͨ͠ϧʔςΟϯάͷઃఆ͕)0$0/Ͱهड़Ͱ͖
    ΔΑ͏ʹͳΓɺهड़ྔ͕ܹݮ͠·ͨ͠ʂ
    HOCON is a human optimised config : )
    foo.__default__.versions {
    v1.config.outfilter.valid = 200
    v4.config.outfilter.invalid = 500
    ...
    }
    "foo": {
    "__default__": {
    "versions": {
    "v1": {
    "config": {
    "outfilter": {
    "valid": 200
    }
    }
    },
    "v4": {
    "config": {
    "outfilter": {
    "invalid": 500
    }
    }
    }
    },
    ...
    },
    ...
    }
    routes.conf
    HOCON making
    configuration more simple!
    routes.json

    View Slide

  17. Copyright © since 1998 DMM All Rights Reserved. 17
    "UPNJD3FGFSDFOFʹͱͬͨ3PVUFTͷࢀরΛɺ)UUQ3FRVFTUͱϖ
    ΞͰ3PVUJOH'MPX΁ྲྀ͠ࠐΈ·͢ɻ
    Sample code of swapping routes
    trait ApplicationServer extends ActorSystemAware {
    def serverName: String
    // This AtomicReference is updated when SIGHUP is received.
    protected lazy val routesRef: AtomicReference[Routes] = Routes.atomicRefercnce
    protected lazy val serverFlow: Flow[HttpRequest, HttpResponse, _] =
    Flow[HttpRequest]
    // It is necessary to the latest route information.
    .map { req => (req, routesRef.get) }
    .via(RoutingFlow.flow)
    // Start the HTTP server. The started server can stop using return value ‘Http.ServerBindig’.
    protected lazy val serverBinding: Future[Http.ServerBinding] = {
    Global.serverSetting(serverName) match {
    case (host, port, Some(connectionContext))=>
    Http().bindAndHandle(serverFlow, host, port, connectionContext)
    case (host, port, None) =>
    Http().bindAndHandle(serverFlow, host, port)
    }
    }
    }

    View Slide

  18. Copyright © since 1998 DMM All Rights Reserved. 18
    Designing
    Flow Graphs
    'MPX(SBQIͷઃܭʹ͍ͭͯ
    18

    View Slide

  19. Copyright © since 1998 DMM All Rights Reserved. 19
    ϑϩʔͷશମਤ
    Combined Routing flow
    Routing flow diagram
    Partition
    (find
    route)
    Endpoint
    Flow
    e.g.
    ・HTTP
    ・OAuth2.0
    and future
    ・AMQP
    etc.
    Http.
    Incoming
    Connection
    IncomingFiltersFlow
    Inco
    ming
    Filter
    Inco
    ming
    Filter
    ・・・
    OutgoingFiltersFlow
    Outg
    oing
    Filter
    Outg
    oing
    Filter
    ・・・
    Partition
    (continue
    or abort)
    Merge
    AbortFlow
    Merge
    Not
    Found
    Flow
    If
    found
    If not
    found
    If abort
    If continue

    View Slide

  20. Copyright © since 1998 DMM All Rights Reserved. 20
    &OEQPJOUʹΞΫηεෆཁͳͱ͖ͷ୹བྷܦ࿏Λ༻ҙ͍ͯ͠·͢ɻ
    ͜ͷਤͰ੺͘هͨ͠/PU'PVOE'MPXͱ"CPSU'MPXͰ͢ɻ
    Combined Routing flow
    Routing flow diagram (short circuit)
    Partition
    (find
    route)
    Endpoint
    Flow
    e.g.
    ・HTTP
    ・OAuth2.0
    and future
    ・AMQP
    etc.
    Http.
    Incoming
    Connection
    IncomingFiltersFlow
    Inco
    ming
    Filter
    Inco
    ming
    Filter
    ・・・
    OutgoingFiltersFlow
    Outg
    oing
    Filter
    Outg
    oing
    Filter
    ・・・
    Partition
    (continue
    or abort)
    Merge
    AbortFlow
    Merge
    Not
    Found
    Flow
    If
    found
    If not
    found
    If abort
    If continue

    View Slide

  21. Copyright © since 1998 DMM All Rights Reserved. 21
    ৚݅෼ذʹ͍ͭͯ
    Combined Routing flow
    Routing flow diagram (conditional branch)
    Partition
    (find
    route)
    Endpoint
    Flow
    e.g.
    ・HTTP
    ・OAuth2.0
    and future
    ・AMQP
    etc.
    Http.
    Incoming
    Connection
    IncomingFiltersFlow
    Inco
    ming
    Filter
    Inco
    ming
    Filter
    ・・・
    OutgoingFiltersFlow
    Outg
    oing
    Filter
    Outg
    oing
    Filter
    ・・・
    Partition
    (continue
    or abort)
    Merge
    AbortFlow
    Merge
    Not
    Found
    Flow
    If
    found
    If not
    found
    If abort
    If continue

    View Slide

  22. Copyright © since 1998 DMM All Rights Reserved. 22
    ϑϩʔͷ෼ذʹ͸"LLB4USFBNTͷ(SBQI%4-͕࢖͑·͢ɻ
    ৚݅෼ذͳΒ'BOPVUδϟϯΫγϣϯͷ1BSUJUJPOΛ࢖͍·͢ɻ
    Akka Streams’ GraphDSL can be used for expressing
    branch of Flow. (And also for balance, broadcast or etc.)
    In the conditional branching case,
    • The branching uses the Fan-out junction 'Partition' (1 input n outputs).
    Give the Partition’s factory method’s argument a branch judgment function ‘IN => Int’.
    This function is an implementation that returns the index number of downstream according to
    IN contents.
    • Finally, the branch integration uses
    the Fan-in junction 'Merge'
    (n inputs 1 output).
    How to express conditional branch of Flow
    Partition Merge
    Any
    Flows


    View Slide

  23. Copyright © since 1998 DMM All Rights Reserved. 23
    1BSUJUJPOδϟϯΫγϣϯ͸؆୯Ͱ͢ɻ
    ෼ذ൑ఆͷؔ਺ΛҾ਺ʹ౉͚ͩ͢Ͱ࢖͑·͢ɻ
    Sample code of Partition and Merge
    val routingFlowShape = GraphDSL.create() { implicit builder =>
    import GraphDSL.Implicits._
    type Input = (Routes, HttpRequest)
    val input = builder.add(Flow[Input])
    val (found, notFound) = (1, 0)
    val findingRouteP = builder.add(Partition[Input](2, { case (routes, req) =>
    if (routes.isFound(req)) found else notFound
    }))
    val findingRouteM = builder.add(Merge[HttpResponse](2))
    val foundResponse = builder.add(/* Implementation in following slide! :) */???)
    val notFoundResponse = builder.add(Flow[Input])map(_ => HttpResponse(status = NotFound)))
    input ~> findingRouteP.in
    findingRouteP.out(found) ~> foundResponse ~> findingRouteM.in(found)
    findingRouteP.out(notFound) ~> notFoundResponse ~> findingRouteM.in(notFound)
    FlowShape(input.in, findingRouteM.out)
    }
    Defining Partition and Merge.
    Give the factory method’s
    argument a branch judgment
    function and junction width.

    View Slide

  24. Copyright © since 1998 DMM All Rights Reserved. 24
    ੺ࣔͨ͘͠෦෼͸ฐࣾಠࣗઃܭͷGJMUFSQMVHJOͰ͢ɻࣾ಺޲͚
    )551ϔομͷ෇༩΍+40/31$㱻3&45ม׵ͳͲΛߦ͍·͢ɻ
    Combined Routing flow
    Routing flow diagram (fliter plug-in)
    Partition
    (find
    route)
    Endpoint
    Flow
    e.g.
    ・HTTP
    ・OAuth2.0
    and future
    ・AMQP
    etc.
    Http.
    Incoming
    Connection
    IncomingFiltersFlow
    Inco
    ming
    Filter
    Inco
    ming
    Filter
    ・・・
    OutgoingFiltersFlow
    Outg
    oing
    Filter
    Outg
    oing
    Filter
    ・・・
    Partition
    (continue
    or abort)
    Merge
    AbortFlow
    Merge
    Not
    Found
    Flow
    If
    found
    If not
    found
    If abort
    If continue

    View Slide

  25. Copyright © since 1998 DMM All Rights Reserved. 25
    ΤϯυϙΠϯτ΁ͷೖग़ྗ৘ใΛॊೈʹฤू͢Δ֦ுϙΠϯτɻ
    ϧʔτ৘ใʹඥ͍ͮͨϓϥάΠϯઃఆ͔Βಈతʹߏங͠·͢ɻ
    The plug-in modules are for flexible editing and verifying
    the input / output contents to the Endpoint.
    For example,
    • Addition Internal HTTP headers
    • Interconversion of REST and non-REST
    • Verifying authorized with OAuth2.0
    etc.
    The composed result of the plug-in is constructed
    dynamically by the plug-in setting which links to
    the route information.
    For composing and applying filters in order,
    Flow’s IN and OUT type signatures are the same.
    IncomingFiltersFlow
    (OutgoingFiltersFlow is similar to this structure.)
    Source
    (single)
    Verify
    Autho
    rized
    REST
    ⇔non
    REST
    Foo
    Filter
    Add
    Head
    er
    Dynamically
    concatination flows by route info
    using ‘Flow.flatMapConcat(Out1 => Source[Out2, Mat])’.
    HttpRequestContext
    (request, route)
    About incoming (outgoing) filter plug-in

    View Slide

  26. Copyright © since 1998 DMM All Rights Reserved. 26
    ΦϦδφϧͷ)551ϦΫΤετͷ಺༰Λอޢ͠ͳ͕ΒɺBQQMZϝ
    ιουͷॲཧΛద༻͠·͢ɻ
    Sample code of IncomingFilter trait
    // EI type variable is endpoint incoming data. V type variable is voluntary data.
    trait IncomingFilter[EI <: RequestContext, V] extends ActorSystemAware {
    type Acc = (HttpResponse ∨ EI, V) // Applied filter result type signature.
    type IO = (HttpRequestContext, Acc) // IncomingFilter.flow’s In-Out type signature.
    val flow: Flow[IO, IO, _]
    }
    trait AsyncIncomingFilter[EI <: RequestContext, V] extends IncomingFilter {
    protected def apply(route: Route, orig: HttpRequest, acc: Acc): Future[Acc]
    lazy val flow: Flow[IO, IO, _] = Flow[IO].flatMapConcat { case (orig, acc) =>
    Source.fromFuture(apply(orig.route, orig.request, acc)).map((orig, _))
    }
    }
    For the apply method, narrowing down
    the necessary input / output values and
    return the result with type Future[Acc].
    Therefore, knowledge of Akka Streams is
    unnecessary for plug-in implementation.
    Note for:
    The original HttpRequestContext should not be passed to the
    apply method’s return value.
    As a reason, it is routed to the next IncomingFilter through a
    detour path so as not to destroy original.

    View Slide

  27. Copyright © since 1998 DMM All Rights Reserved. 27
    㱹͕MFGUͷ࣌ʹ୹བྷධՁ͢ΔUSBJUɻMFGUͷܕ͸)UUQ3FTQPOTFɻ
    ݁Ռ͕MFGUͩͬͨ৔߹͸ԼྲྀͰ"CPSU'MPX΁సૹ͍ͯ͠·͢ɻ
    Sample code of right-bind IncomingFilter
    trait RightIncomingFilter[EI <: RequestContext, V] extends AsyncIncomingFilter[EI, V] {
    override final def apply(route: Route, orig: HttpRequest, acc: Acc): Future[Acc] = {
    acc match {
    case (∨-(ei), v) => applyRight(route, orig, (ei, v))
    case left => Future.successful(left)
    }
    }
    protected def applyRight(route: Route, orig: HttpRequest, acc: (EI, V)): Future[Acc]
    }
    Rignt-bind implementation.
    ・ To short-circuit when the left comes.
    ・ ∨’s left type is HttpResponse.
    (It is sent downstream to AbortFlow.)

    View Slide

  28. Copyright © since 1998 DMM All Rights Reserved. 28
    ϑΟϧλͷ߹੒͸Ϧετʹ֨ೲͨ͠'MPXΛ৞ΈࠐΉ͜ͱͰੜ੒͠
    ·͢ɻ৞ΈࠐΈͷॳظ஋ ୯Ґݩ
    ͸'MPXBQQMZ<5>Ͱऔಘ͠·͢ɻ
    Sample code of composing IncomingFilters
    object IncomingFilters {
    def flow[EI <: RequestContext, V](): Flow[HttpRequestContext,(HttpResponse ∨ EI, V), _] = {
    type Acc = (HttpResponse ∨ EI, V)
    Flow[HttpRequestContext].flatMapConcat { r =>
    val initFlow = r.route.incomingValue.flow
    val combinedFlow = r.route.infilters[TO, V]
    .map(_.flow)
    .foldLeft(Flow[(HttpRequestContext, Acc)])(_.via(_))
    Source.single(r).via(initFlow).via(combinedFlow)
    }.map { case (_, acc) => acc }
    }
    } Composing filters implimentation.
    ・ Filter composition is generated by folding Flow from the list.
    ・ Identity flow ‘Flow[T, T, _]’ can be taken with ‘Flow.apply[T]’.

    View Slide

  29. Copyright © since 1998 DMM All Rights Reserved. 29
    )551&OUJUZͷղੳ͸"LLB)551ͷ6ONBSTIBMMFSΛ࢖͍·͢ɻ
    ݁Ռ͸'VUVSFʹ͘Δ·ΕΔͷͰ໌ࣔతͳϒϩοΫ͸ෆཁͰ͢ɻ
    Sample code of HTTP method editing
    private def editMethodWithEntityToQuery(destMethod: HttpMethod)(srcReq: HttpRequset): Future[HttpRequest] =
    val srcUri = srcReq.uri
    val srcEntity = srcReq.entity
    Unmarshal(srcEntity).to[FormData].map { formData =>
    val srcQuery = srcUri.query()
    val fields = formData.fields
    val queryBuilder = Query.newBuilder // mutable API
    queryBuilder ++= srcQuery
    queryBuilder ++= fields
    val destQuery = queryBuilder.result
    val destQueryString = destQuery.toString.some.filterNot(_.isEmpty)
    val destHeaders = srcReq.headers.filterNot(_.name == `Content-Type`.name)
    srcReq.copy(
    method = destMethod, headers = destHeaders, entity = HttpEntity.Empty,
    uri = srcUri.copy(rawQueryString = destQueryString)
    )
    } recover { case e: UnsupportedContentTypeException =>
    logger.warn(s"Availed HTTP entity may be not single-part form data in HttpMethodEditing of entity to que
    srcReq
    }
    }
    How to parse object in Akka HTTP?
    Akka HTTP Unmarshaller is useful for parsing HTTP
    entities. Explicit blocking is unnecessary because
    parsing results are wrapped in Future.
    Depending on the type classes, the required
    implementation of Akka HTTP embedded is resolved
    at compilation time.

    View Slide

  30. Copyright © since 1998 DMM All Rights Reserved. 30
    ࣍͸ΤϯυϙΠϯτͷઆ໌Ͱ͢ɻ
    ͪ͜Β΋೚ҙͷ࣮૷ΛઃఆͰ͖ΔΑ͏ʹϓϥάΠϯܗࣜͰ͢ɻ
    Combined Routing flow
    Routing flow diagram (endpoint plug-in)
    Partition
    (find
    route)
    Endpoint
    Flow
    e.g.
    ・HTTP
    ・OAuth2.0
    and future
    ・AMQP
    etc.
    Http.
    Incoming
    Connection
    IncomingFiltersFlow
    Inco
    ming
    Filter
    Inco
    ming
    Filter
    ・・・
    OutgoingFiltersFlow
    Outg
    oing
    Filter
    Outg
    oing
    Filter
    ・・・
    Partition
    (continue
    or abort)
    Merge
    AbortFlow
    Merge
    Not
    Found
    Flow
    If
    found
    If not
    found
    If abort
    If continue

    View Slide

  31. Copyright © since 1998 DMM All Rights Reserved. 31
    ओʹ֎෦௨৴͕໨తͷϓϥάΠϯͰ͢ɻ
    ࠓճ͸ೝՄ"1*΋&OEQPJOUϓϥάΠϯͱ࣮ͯ͠૷͠·ͨ͠ɻ
    As the API Gateway, Endpoint plug-in is a plug-in
    primarily for external communication.
    For example,
    • HTTP / HTTPS client
    • Authorization API's business logic
    • AMQP client (to do in the future)
    • Kafka client (to do in the future)
    • gRPC client (to do in the future)
    etc.
    In this time, OAuth 2.0 API is implemented by this as a external communication.
    About Endpoint plug-in

    View Slide

  32. Copyright © since 1998 DMM All Rights Reserved. 32
    ࠓճ͸"1*(BUFXBZαʔόʔͳͷͰIPTUDPOOFDUJPOQPPMΛ࢖
    ༻ɻϗετຖͷϓʔϧ਺΍44-ͷઃఆ͸BQQMJDBUJPODPOG΁ɻ
    Sample code of Akka HTTP client
    trait HttpClients extends ActorSystemAware {
    // Type parameter [T] means to keep path-through data in Flow’s I/O.
    def forUri[T](apiUrl: Uri): Flow[(HttpRequest, T), (Try[HttpResponse], T), HostConnectionPool] = {
    val host = apiUrl.authority.host.toString
    val port = apiUrl.authority.port
    val hostSettings = resolveHostSpecificSettings(host) | defaultConnectionPoolSettings
    val sslCtxt = Global.clientHttpsContext | Http().defaultClientHttpsContext
    (apiUrl.scheme, port) match {
    case ("http" , 0) => Http().cachedHostConnectionPool[T](host, settings = hostSettings)
    case ("http" , _) => Http().cachedHostConnectionPool[T](host, port, hostSettings)
    case ("https", 0) => Http().cachedHostConnectionPoolHttps[T](host, connectionContext = sslCtxt
    , settings = hostSettings)
    case ("https", _) => Http().cachedHostConnectionPoolHttps[T](host, port, sslCtxt, hostSettings)
    case _ => throw new NoSuchElementException
    }
    // .......
    }
    This time it is an API Gateway server, so we set up endpoints
    host connection pool.
    The number of connection pools and the setting of SSL for each
    host is basically set in application.conf.

    View Slide

  33. Copyright © since 1998 DMM All Rights Reserved. 33
    Implementing Router
    with Graph API
    'MPXHSBQIΛ༻͍ͨϧʔςΟϯάॲཧͷ࣮૷

    View Slide

  34. Copyright © since 1998 DMM All Rights Reserved. 34
    • Receive HttpRequest at the Flow's parameter.
    • According to the setting of routing, make as many Partitions as number
    of routes.
    • Applied plug-in for each route is different and the length of the
    previous Flow by Partition is different.
    Implementation process
    ϧʔτఆٛͱಉ͡਺͚ͩ1BSUJUJPOʹΑΔ෼ذΛ࡞Γ·͢ɻ'MPX
    ͷ௕͞͸ϧʔτຖʹҟͳΓ·͢ɻ

    View Slide

  35. Copyright © since 1998 DMM All Rights Reserved. 35
    • Depending on the requested path, the index (or root) of the Partition is
    specified.
    • Connect partial Graph per sort of plugin to Merge.
    ϦΫΤετ͞ΕͨύεʹԠͯ͡1BSUJUJPOͷࡧҾ ϧʔτͷࡧҾ
    ͕
    ಛఆ͞Εɺ.FSHFʹܨ͛ΒΕ·͢ɻ
    Implementation process

    View Slide

  36. Copyright © since 1998 DMM All Rights Reserved. 36
    First implementation
    val graph = GraphDSL.create() { implicit builder =>
    import GraphDSL.Implicits._
    val orderedListOfRoutes = routes.sequentialRoutes
    val routeP = builder.add(Partition[HttpRequest](routes.size, req => {
    // Routes#route returns index for partition.if Route not found returns 0.
    routes.route(req).map(_._1) getOrElse 0
    }))
    val routeM = builder.add(Merge[Try[HttpResponse]](routes.size))
    orderedListOfRoutes.zipWithIndex.foreach {
    case ((_, _), n) if n == notFoundIndex => routeP.out(n) ~> routeNotFoundFlow ~> routeM.in(n)
    case ((_, routeConfig), i) => {
    val endpointAndOutgoingPluginsFlow = Flow.fromFunction[(HttpResponse ∨ E, EI), (N, V)] ...
    val abortFlow = Flow.fromFunction[(HttpResponse ∨ Any, Any), Try[HttpResponse]] ...

    // format: OFF
    routeP.out(i) ~> incomingPluginsFlow ~> continueOrAbortP.in
    continueOrAbortP.out(continue) ~> endpointAndOutgoingPluginsFlow ~> continueOrAbortM.in(continue)
    continueOrAbortP.out(abort) ~> abortFlow ~> continueOrAbortM.in(abort)
    continueOrAbortM.out ~> routeM.in(i)
    // format: ON
    }
    FlowShape(routeP.in, routeM.out)
    }
    }
    Ordered by index of
    Route.
    Create partial Flow
    as same as number
    of Route.
    1BSUJUJPOͷࡧҾͱ߹͏Α͏ʹϧʔτ͕੔ྻ͞Εͨঢ়ଶͰอ࣋͞Ε
    ͍ͯ·͢ɻ

    View Slide

  37. Copyright © since 1998 DMM All Rights Reserved. 37
    ... ...
    Endpoint
    Flow
    .
    .
    .
    i = index of Partition
    N = number of Route
    i = N
    i = 1
    i = 0
    Partition
    [HttpRequest]
    Route
    NotFound
    Flow
    ... ...
    Incoming
    Plugins
    Flow
    ... ...
    Outgoing
    Plugins
    Flow
    ... ...
    Endpoint
    Flow
    ... ...
    Incoming
    Plugins
    Flow
    ... ...
    Outgoing
    Plugins
    Flow
    Merge
    [Try[HttpResponse]]
    ࠷ॳͷ'MPXͷߏ଄
    First implementation

    View Slide

  38. Copyright © since 1998 DMM All Rights Reserved. 38
    • As the routing configuration size (Partition size) increased, heap size expanded.
    • When the route size exceeds 100, heap size rises 1 GB at the time of start.
    • Performance has been dragged by Full GC, and throughput has also shrunken.
    ϧʔτͷ਺ 1BSUJUJPOͷαΠζ
    ͕େ͖͘ͳΔʹͭΕͯɺىಈ࣌ͷ
    ώʔϓαΠζ͕๲ΒΈɺύϑΥʔϚϯεͷྼԽ͕ݟΒΕ·ͨ͠ɻ
    But ...

    View Slide

  39. Copyright © since 1998 DMM All Rights Reserved. 39
    ... ...
    Endpoint
    Flow
    .
    .
    .
    i = index of Partition
    N = number of Route
    i = N
    i = 1
    i = 0
    Partition
    [HttpRequest]
    Route
    NotFound
    Flow
    ... ...
    Incoming
    Plugins
    Flow
    ... ...
    Outgoing
    Plugins
    Flow
    ... ...
    Endpoint
    Flow
    ... ...
    Incoming
    Plugins
    Flow
    ... ...
    Outgoing
    Plugins
    Flow
    Merge
    [Try[HttpResponse]]
    ࠷ॳͷ'MPXͷߏ଄
    First implementation

    View Slide

  40. Copyright © since 1998 DMM All Rights Reserved. 40
    मਖ਼ޙͷ'MPXͷߏ଄
    i = index of Partition
    i = 1
    i = 0
    Partition
    [HttpRequest]
    Route
    NotFound
    Flow
    ... ...
    Endpoint
    Flow
    ... ...
    Incoming
    Plugins
    Flow
    ... ...
    Outgoing
    Plugins
    Flow
    Merge
    [Try[HttpResponse]]
    Second implementation

    View Slide

  41. Copyright © since 1998 DMM All Rights Reserved. 41
    Second implementation
    lazy val graph = GraphDSL.create() { implicit builder =>
    import GraphDSL.Implicits._
    val request = builder.add(Flow[HttpRequest].map {...
    val (found, notFound) = (1, 0)
    val findingRouteP = builder.add(Partition[HttpRequest](2,{ req =>
    // Routes#route returns Option[Route]
    routes.route(req).map(found) | notFound
    }))
    val findingRouteM = builder.add(Merge[HttpResponse](2))

    // format: OFF
    request ~> findingRouteP.in
    findingRouteP.out(found) ~> incomingPluginsFlow ~> p.in
    continueOrAbortP.out(continue) ~> endpointAndOutgoingPluginsFlow ~> continueOrAbortM.in(continue)
    continueOrAbortP.out(abort) ~> abortFlow ~> continueOrAbortM.in(abort)
    continueOrAbortM.out ~> response ~> findingRouteM.in(found)
    findingRouteP.out(notFound) ~> notFoundResponse ~> findingRouteM.in(notFound)
    // format: ON
    FlowShape(request.in, findingRouteM.out)
    }
    Partition devided
    into two cases.
    1BSUJUJPO͕ϦΫΤετʹରԠ͢Δϧʔτ͕ݟ͔ͭͬͨ৔߹ͱɺͦ
    ͏Ͱͳ͍৔߹ͷ͚̎ͭͩʹͳΔΑ͏ʹमਖ਼͠·ͨ͠ɻ

    View Slide

  42. Copyright © since 1998 DMM All Rights Reserved. 42
    1BSUJUJPOΛখͨ͘͜͞͠ͱͰώʔϓαΠζ͕ݮগ͠·ͨ͠ɻ'BO
    *O0VU͸ଟྔͷϦΫΤετͷॲཧʹ͸޲͍͍ͯͳ͍Α͏Ͱ͢ɻ
    • Removed Partition in Rouing Flow and Graph got smaller.
    • The heap size at startup also has been decreased to 100-200M.
    • Full GC also disappeared and throughput has got improved.
    • Fan In/Fan Out is not suitable for large number of request processing.
    After improvement

    View Slide

  43. Copyright © since 1998 DMM All Rights Reserved. 43
    ϦΫΤετຖʹ'MPXͰ૊·Ε͍ͯΔϓϥάΠϯΛಈతʹܨ͛Δ෦
    ෼͸GMBU.BQ$PODBU͕༗ޮͰͨ͠ɻ
    • However, depending on the result of routing, plug-ins that are organized by Flow
    must be selected and connect dynamically per request.
    • Flow#flatMapConcat was useful for the part where Flow should be dynamically
    connected.
    After improvement
    val incomingPluginsFlow = Flow[(HttpRequest,Route)].flatMapConcat {
    case (request,route) =>
    // flatMapConcat requires SourceShape.
    Source.single((request -> route)).via {
    // Connect all filter(Flow) in this route.
    route.infilters.map{ infilter => infilter.flow}.foldLeft(initalValue)(_.via(_))
    }
    }

    View Slide

  44. Copyright © since 1998 DMM All Rights Reserved. 44
    ϝΠϯͱͳΔ'MPX͸(SBQI%4-Λ࢖ͬͯॻ͍͍ͯ·͢ɻ͜ΕΒΛ
    'MPXͰॻ͘ͱͲ͏ͳΔͰ͠ΐ͏͔
    • We use Graph DSL in a main Flow.
    • DSL is very useful for visualizing that whole processing flow.
    • if this flow written without DSL...
    By the way...
    // format: OFF
    request ~> findingRouteP.in
    findingRouteP.out(found) ~> incomingPluginsFlow ~> p.in
    continueOrAbortP.out(continue) ~> endpointAndOutgoingPluginsFlow ~> continueOrAbortM.in(continue)
    continueOrAbortP.out(abort) ~> abortFlow ~> continueOrAbortM.in(abort)
    continueOrAbortM.out ~> response ~> findingRouteM.in(found)
    findingRouteP.out(notFound) ~> notFoundResponse ~> findingRouteM.in(notFound)
    // format: ON
    FlowShape(request.in, findingRouteM.out)

    View Slide

  45. Copyright © since 1998 DMM All Rights Reserved. 45
    ৚݅෼ذͷͨͼʹ4PVSDF4IBQFΛೖΕࢠʹ͢Δߏ଄ʹͳΓ·͢ɻ
    ࠓճͷΑ͏ͳ୯७ͳϑϩʔͰ͸গ͠؆ܿʹͳΓ·ͨ͠ɻ
    lazy val endpointOrAboutSource: PartialFunction[(HttpResponse ∨ EI, V), Source[HttpResponse,NotUsed]] = {
    case (∨-(r), v) => Source.single((r, v))
    .via(measuredProcessingTimeEndpointFlow).via(outgoingFlow).via(response)
    case (-∨(l), _) => Source.single(l).map(Success(_)).via(response)
    }
    lazy val flow = findingRouteFlow.flatMapConcat {
    case in @ (_, _, None) => Source.single(in).via(notFoundResponse)
    case x => Source.single(x).via(request).via(incomingFlow).flatMapConcat(endpointOrAbortSource)
    }
    Nested
    flatMapConcat
    request ~> findingRouteP.in
    findingRouteP.out(found) ~> incomingPluginsFlow ~> p.in
    continueOrAbortP.out(continue) ~> endpointAndOutgoingPluginsFlow ~> continueOrAbortM.in(continue)
    continueOrAbortP.out(abort) ~> abortFlow ~> continueOrAbortM.in(abort)
    continueOrAbortM.out ~> response ~> findingRouteM.in(found)
    findingRouteP.out(notFound) ~> notFoundResponse ~> findingRouteM.in(notFound)
    FlowShape(request.in, findingRouteM.out)
    Write main Flow without Graph DSL

    View Slide

  46. Copyright © since 1998 DMM All Rights Reserved. 46
    Testing the Performance
    of Gateway
    (BUFXBZͷύϑΥʔϚϯεΛςετ͢Δ

    View Slide

  47. Copyright © since 1998 DMM All Rights Reserved. 47
    ෛՙΫϥΠΞϯτΛ୆༻ҙ͠ɺ(BUFXBZͷഎޙʹ͸ϓϩΩγઌ
    ͱͳΔ8FC"1*Λ໛ͨ͠αʔόΛ༻ҙ͠·ͨ͠ɻ
    Instrumentation
    Web API
    (Mock server)
    CPU: 8 Core
    RAM: 8 GB

    View Slide

  48. Copyright © since 1998 DMM All Rights Reserved. 48
    7FSUYͰ࡞੒͞Εͨ(BUFXBZͱൺֱͯ͠ɺ෇ՃػೳΛצҊͯ͠
    ΋े෼ͳεϧʔϓοτΛಘΔ͜ͱ͕Ͱ͖·ͨ͠ɻ ౰ࣾൺ

    Performance testing result
    • Throughput which created by Akka is comparable to previous Gateway(Vert.x 2) .
    • About 8% of performance was reduced.
    • We obtained sufficient throughput even with considering additional functions.

    View Slide

  49. Copyright © since 1998 DMM All Rights Reserved. 49
    (BUFXBZͷಛੑΛ֬ೝ͢Δҝͷݕূͱͯ͠ɺ(BUFXBZͷഎޙʹ
    ஗͍ΤϯυϙΠϯτɺ଎͍ΤϯυϙΠϯτΛ༻ҙ͠·ͨ͠ɻ
    Testing the characteristics of Gateway
    • We prepared a slow endpoint (emulating processing time 1 second) and a fast
    endpoint (emulating processing time 50 ms) behind Gateway.
    Fast
    Endpoint
    Slow
    Endpoint
    1500 req/sec
    constantry

    View Slide

  50. Copyright © since 1998 DMM All Rights Reserved. 50
    ૒ํͷΤϯυϙΠϯτʹಉ࣌ʹෛՙΛֻ͚ɺ஗͍ΤϯυϙΠϯτ
    ͷӨڹΛड͚ͯશମͷεϧʔϓοτ͕མͪͳ͍͔ݕূ͠·ͨ͠ɻ
    • We loaded both endpoints at the same time and tested whether the overall
    throughput has not decreased because it was affected by the slow endpoint.
    Testing the characteristics of Gateway
    Fast
    Endpoint
    Slow
    Endpoint
    1500 req/sec
    constantry

    View Slide

  51. Copyright © since 1998 DMM All Rights Reserved. 51
    ଎͍ΤϯυϙΠϯτͰ͸ɺෛՙ͕ߴ͘ͳͬͯ΋ߴ͍εϧʔϓοτ
    Λҡ͍࣋ͯ͠·ͨ͠ɻ
    Proxy
    destination
    Throughput
    (req/sec)
    Minimum
    Response
    Time(msec)
    Average
    Response
    Time(msec)
    Maximum
    Response
    Time(msec)
    Fast Endpoint 1490.066 52 88 608
    Slow Endpoint 80.972 1002 25920 600010
    Test Result
    • At the fast endpoint, consistently high throughput was maintained even at high
    loads.

    View Slide

  52. Copyright © since 1998 DMM All Rights Reserved. 52
    ஗͍ΤϯυϙΠϯτͰ͸ෛՙ͕ߴ͘ͳΔʹͭΕλΠϜΞ΢τ͕૿
    Ճ͕ͨ͠ɺΤϯυϙΠϯτ͕ఀࢭ͢Δࣄ͸͋Γ·ͤΜͰͨ͠ɻ
    Test Result
    • At the slow endpoint, timeout increased as load increased.
    • Connecting to the endpoint that has been timed out in 60 seconds.
    • Slow endpoints were not destroyed even when high load was applied.
    Proxy
    destination
    Throughput
    (req/sec)
    Minimum
    Response
    Time(msec)
    Average
    Response
    Time(msec)
    Maximum
    Response
    Time(msec)
    Fast Endpoint 1490.066 52 88 608
    Slow Endpoint 80.972 1002 25920 600010

    View Slide

  53. Copyright © since 1998 DMM All Rights Reserved. 53
    "LLB)551ͱ"LLB4USFBNTʹΑͬͯɺΤϯυϙΠϯτ΁ͷӨڹ
    Λ཈͑ͭͭɺߴ͍ύϑΥʔϚϯεΛಘΔ͜ͱ͕ग़དྷ·ͨ͠ɻ
    • Akka HTTP (and Akka Streams) performed the expected behavior of
    minimizing influence on the endpoint and the gateway itself using
    back-pressure control.
    • While performing backpressure control, it was possible to obtain
    sufficient throughput even compared to before transplantation.
    Summary

    View Slide

  54. Copyright © since 1998 DMM All Rights Reserved. 54
    "LLB)551 )PTU$POOFDUJPO1PPM
    ʹΑΔϓϩΩγ͸ɺ
    (BUFXBZ ϚΠΫϩαʔϏε
    ͷಛੑʹద߹ͯ͠͠·͢ʂ
    • Flow is suitable for constructing a static structure.
    • The proxy by HostConnectionPool is suitable due to the nature of the
    Gateway server.
    • Akka Streams and HTTP are useful for microservice architecture!
    Summary

    View Slide

  55. Any Questions?
    ࣭ٙԠ౴

    View Slide