Implementing DMM API Gateway in Akka Streams and HTTP

Implementing DMM API Gateway in Akka Streams and HTTP

Slides for ScalaMatsuri 2017
http://2017.scalamatsuri.org/index_en.html#schedule

DMM.com Labo Co.,Ltd. Yukio Kakumoto/Masateru Nishimura

F1fa6bfa8c7394643fba587481851b57?s=128

Masateru Nishimura

February 25, 2017
Tweet

Transcript

  1. Implementing DMM API Gateway in Akka Streams and HTTP %..ͷ"1*(BUFXBZΛ"LLB4USFBNTͱ

    "LLB)551Ͱ࡞ΓࠐΜͰΈͨ DMM.com labo Co.,Ltd. Yukio Kakumoto / Masateru Nishimura Feb. 25, 2017 in ScalaMatsuri 2017
  2. Copyright © since 1998 DMM All Rights Reserved. 2 ֯ຊ޾ੜ

    ੢ଜ੓ً %..DPNϥϘ γεςϜຊ෦ ϓϥοτϑΥʔϜ։ൃ෦ About us Yukio Kakumoto & Masateru Nishimura DMM.com Labo Co.,Ltd. We develop and improve our micro service platform.
  3. Copyright © since 1998 DMM All Rights Reserved. 3 %..DPN͸೔ຊΛ୅ද͢Δ&$αΠτͷͻͱͭͰ͢ɻ

    About DMM.com Group DMM.com is one of Japan's leading E-commerce websites. The Outline of DMM's website and the company: · 2.5 billion page view per month. · Managing more than 40 services. · Members exceeding 20 million. · Unit sales of about $ 700 million. · Group sales of about $ 1.4 billion. (Converted by: 1USD=100JPY)
  4. Copyright © since 1998 DMM All Rights Reserved. 4 ೥લ͔Β͍Ζ͍Ζ΍͖ͬͯ·͕ͨ͠ɺࠓճɺ"1*(BUFXBZͱೝ

    Մ"1*Λ "LLB4USFBNTBOE)551Ͱ࡞Γ௚͠·ͨ͠ɻ Refurbishment history of DMM’s platform OAuth 2.0 Connect API (Fuel PHP) API Gateway (Fuel PHP) Non-blocking API GW which was built in OAuth2.0 API Scala 2.11 / Akka Streams and HTTP Non-blocking API Gateway (Java7/Vert.x2) Countermeasure for C10K www.dmm.com / www.dmm.co.jp LAMP application Typical problems, large and monolithic state was given by business growth priority. Another microservices Purchase info API, Settlement agent API etc.. (Fuel PHP) Unified frontend shopping cart (FuelPHP/Couchbase) Legacy emoney systems Legacy shopping carts Unified Emoney API (Java7/Vert.x2/In-memory Data Grid) new Member API (Java8/Spring) Old Member API (LAMP) since ‘03〜? ‘14 Q1 ‘15 Q2 ‘16 Q3 ‘17 Q1〜 Today’s agenda.
  5. Copyright © since 1998 DMM All Rights Reserved. 5 ຊ೔͓࿩͢Δ಺༰

    Today's agenda • Project overview • Architecture design • Designing Flow Graphs • Implementing Router with Graph API • Testing the Performance of Gateway
  6. Copyright © since 1998 DMM All Rights Reserved. 6 Project

    overview ϓϩδΣΫτ֓ཁ 6
  7. Copyright © since 1998 DMM All Rights Reserved. 7 ೝՄύϑΥʔϚϯεͷڧԽͳͲΛ໨తʹɺ"1*(BUFXBZͱೝՄ

    "1*Λ"LLBͰ౷߹͠·ͨ͠ɻ After Before What we have done Backend API servers API-GW clients (API Gateway) (OAuth2.0 API) (Unified API Gateway and OAuth2.0 API) The purpose of the project ・To increase authorization performance. ・To centralize management of API routing and authorizing (to do in the future).
  8. Copyright © since 1998 DMM All Rights Reserved. 8 "1*(BUFXBZ͸ࣾ಺֎ͷଟ਺ͷγεςϜ͔Βͷॲཧͷ૭ޱͱͳΔ

    ॏཁͳγεςϜͰ͢ɻ Responsibility of API Gateway LB LB ・Mobile devices ・3rd party etc. Backend API servers ・HTTPS TLS decoder ・Ephemeral port buffer API Gateway (Today's agenda.) ・Routing to backend APIs ・Authentication with OAuth2.0 … Actual to x,000req/sec during peak time. … Managing of ・member ・billing ・accounting etc.
  9. Copyright © since 1998 DMM All Rights Reserved. 9 Architecture

    design ΞʔΩςΫνϟσβΠϯʹ͍ͭͯ 9
  10. Copyright © since 1998 DMM All Rights Reserved. 10 "LLB4USFBNTΛ࣠ʹ"LLB)5514FSWFS

    $MJFOUΛ࠾༻ ετϨʔδ͸ .Z42-ɾ3FEJT 4FOUJOFM ɾ)BTIJDPSQ7BVMU Scala 2.11 on JVM8 Akka Streams 2.4 and HTTP 10.0 Low-level API Original Framework Router Hashicorp Valut driver Endpoint plugins HTTP(S) endpoint plugin OAuth2.0 endpoint plugin ScalikeJDBC Rediscala Incoming plugins Outgoing plugins Akka HTTP Server Akka HTTP Client (HTTP(S) API driver) Technology stack Backend API servers storages Redis® Redis Sentinel® Hashicorp Vault API-GW clients
  11. Copyright © since 1998 DMM All Rights Reserved. 11 શͯͷετϨʔδυϥΠό͸'VUVSFʹରԠ͍ͯ͠·͢ɻ

    Technology stack (storages & drivers) ScalikeJDBC (using MySQL) • It's Query driver which has SQL interpolation function that is very easy to use. • Future-based asynchronous interface. Rediscala (using Redis® for cache storage) • Supporting failover using Redis Sentinel® (discovery service for Redis®). • Future-based asynchronous interface. (Temporary solution for taking over from PHP version and running in parallel.) Hashicorp Vault with original driver implements Akka HTTP Client • For the purpose of keeping secret data. (For example DB password, salt for hash function etc.) • Unified akka HTTP Clinet because it’s interface is HTTP REST API. • Future-based asynchronous interface (via Akka Streams’ mat).
  12. Copyright © since 1998 DMM All Rights Reserved. 12 ߴ৴པͷ+7.ͱ.BWFOϦϙδτϦͷ๛෋ͳϥΠϒϥϦ܈ʹՃ͑

    "LLB4USFBNTͷόοΫϓϨογϟʔ͕͋Ε͹׬ᘳʂ Why did we adopt Akka Streams? As API Gateway, stability is the most important, and productivity (fulfillment of libraries and drivers etc.) is also important. Stability • Running on JVM 8 which is a runtime of high achievement and reliability. • Using Akka with sufficient operational results for internal messaging. • Having protection of internal messaging flow by Akka streams’ back pressure. Productivity • Various libraries and drivers in several repositories. e.g. Maven, Ivy, etc. • Scala features, type safety and excellent syntax. Akka Streams is a comprehensive powerful solution!
  13. Copyright © since 1998 DMM All Rights Reserved. 13 όοΫϓϨογϟʔͱ͸ɺϑϩʔ੍ޚʹ͓͍ͯԼྲྀͷόοϑΝ͕

    ҲΕͳ͍Α͏্ྲྀ΁࿈࠯తʹྲྀྔ৘ใΛ௨஌͢Δ࢓૊Έͷ͜ͱ What is back pressure? In information terminology, in order to prevent the downstream buffer from overflowing in flow control, flow information is notified in a chain manner to the upstream. Part of supported libraries • Akka Streams(Scala) • Scalaz Stream(Scala) • Node.js (JavaScript) • ReactiveX • RxJava • RxScala • RxJS etc. Implementation depends on each language, library, and backend queue system. Publisher (upstream) Subscriber (downstream) For example in Akka streams’ back pressure. ‘Dynamic push / pull mode’ ✉✉✉✉✉ request(5)
  14. Copyright © since 1998 DMM All Rights Reserved. 14 "1*(BUFXBZͳͷͰϝϞϦ಺ͷϧʔτઃఆͷϦϩʔυ͕ඞཁɻ

    )JHIMFWFM"1*Ͱ͸ϧʔςΟϯάͷ%4-͕ϋʔυίʔυͷͨΊº The use of API Gateway requires reloading of route setting in memory at an arbitrary timing. Implementing this with the Akka HTTP High-level API can not satisfy the requirement, because routing DSL is hard code. Low-level API High-level API Flexibility Yes. Akka HTTP can bind original flow implimented by ourself. No. Hard-coded routing DSL generates flow to bind to Akka HTTP. Difficulty Hard. Requireing of understanding of Akka streams Graph DSL. Easy. Routing DSL is Spray-like simpile and intuitive directive. We chose Akka HTTP Low-level API
  15. Copyright © since 1998 DMM All Rights Reserved. 15 ϧʔτ৘ใΛ)0$0/Ͱهड़͠ɺ)61γάφϧͰࠩ͠ସ͑Δɻ

    ϦϓϨʔεલγεςϜͷઃఆ͕KTPOͷͨΊ্Ґޓ׵ͱΕΔɻ HOCON based original routing system routes. before.json (JSON) routes.conf (HOCON) routes. before.conf (HOCON) Copy as is (and rename) include SIGHUP! In a common practice, when the application receives a HUP signal, it reloads route information. HOCON is upward compatible with json. By using this, compatibility can be keeping by including the pre-replacement configuration as it is.
  16. Copyright © since 1998 DMM All Rights Reserved. 16 +40/Ͱهड़͍ͯͨ͠ϧʔςΟϯάͷઃఆ͕)0$0/Ͱهड़Ͱ͖

    ΔΑ͏ʹͳΓɺهड़ྔ͕ܹݮ͠·ͨ͠ʂ HOCON is a human optimised config : ) foo.__default__.versions { v1.config.outfilter.valid = 200 v4.config.outfilter.invalid = 500 ... } "foo": { "__default__": { "versions": { "v1": { "config": { "outfilter": { "valid": 200 } } }, "v4": { "config": { "outfilter": { "invalid": 500 } } } }, ... }, ... } routes.conf HOCON making configuration more simple! routes.json
  17. Copyright © since 1998 DMM All Rights Reserved. 17 "UPNJD3FGFSDFOFʹͱͬͨ3PVUFTͷࢀরΛɺ)UUQ3FRVFTUͱϖ

    ΞͰ3PVUJOH'MPX΁ྲྀ͠ࠐΈ·͢ɻ Sample code of swapping routes trait ApplicationServer extends ActorSystemAware { def serverName: String // This AtomicReference is updated when SIGHUP is received. protected lazy val routesRef: AtomicReference[Routes] = Routes.atomicRefercnce protected lazy val serverFlow: Flow[HttpRequest, HttpResponse, _] = Flow[HttpRequest] // It is necessary to the latest route information. .map { req => (req, routesRef.get) } .via(RoutingFlow.flow) // Start the HTTP server. The started server can stop using return value ‘Http.ServerBindig’. protected lazy val serverBinding: Future[Http.ServerBinding] = { Global.serverSetting(serverName) match { case (host, port, Some(connectionContext))=> Http().bindAndHandle(serverFlow, host, port, connectionContext) case (host, port, None) => Http().bindAndHandle(serverFlow, host, port) } } }
  18. Copyright © since 1998 DMM All Rights Reserved. 18 Designing

    Flow Graphs 'MPX(SBQIͷઃܭʹ͍ͭͯ 18
  19. Copyright © since 1998 DMM All Rights Reserved. 19 ϑϩʔͷશମਤ

    Combined Routing flow Routing flow diagram Partition (find route) Endpoint Flow e.g. ・HTTP ・OAuth2.0 and future ・AMQP etc. Http. Incoming Connection IncomingFiltersFlow Inco ming Filter Inco ming Filter ・・・ OutgoingFiltersFlow Outg oing Filter Outg oing Filter ・・・ Partition (continue or abort) Merge AbortFlow Merge Not Found Flow If found If not found If abort If continue
  20. Copyright © since 1998 DMM All Rights Reserved. 20 &OEQPJOUʹΞΫηεෆཁͳͱ͖ͷ୹བྷܦ࿏Λ༻ҙ͍ͯ͠·͢ɻ

    ͜ͷਤͰ੺͘هͨ͠/PU'PVOE'MPXͱ"CPSU'MPXͰ͢ɻ Combined Routing flow Routing flow diagram (short circuit) Partition (find route) Endpoint Flow e.g. ・HTTP ・OAuth2.0 and future ・AMQP etc. Http. Incoming Connection IncomingFiltersFlow Inco ming Filter Inco ming Filter ・・・ OutgoingFiltersFlow Outg oing Filter Outg oing Filter ・・・ Partition (continue or abort) Merge AbortFlow Merge Not Found Flow If found If not found If abort If continue
  21. Copyright © since 1998 DMM All Rights Reserved. 21 ৚݅෼ذʹ͍ͭͯ

    Combined Routing flow Routing flow diagram (conditional branch) Partition (find route) Endpoint Flow e.g. ・HTTP ・OAuth2.0 and future ・AMQP etc. Http. Incoming Connection IncomingFiltersFlow Inco ming Filter Inco ming Filter ・・・ OutgoingFiltersFlow Outg oing Filter Outg oing Filter ・・・ Partition (continue or abort) Merge AbortFlow Merge Not Found Flow If found If not found If abort If continue
  22. Copyright © since 1998 DMM All Rights Reserved. 22 ϑϩʔͷ෼ذʹ͸"LLB4USFBNTͷ(SBQI%4-͕࢖͑·͢ɻ

    ৚݅෼ذͳΒ'BOPVUδϟϯΫγϣϯͷ1BSUJUJPOΛ࢖͍·͢ɻ Akka Streams’ GraphDSL can be used for expressing branch of Flow. (And also for balance, broadcast or etc.) In the conditional branching case, • The branching uses the Fan-out junction 'Partition' (1 input n outputs). Give the Partition’s factory method’s argument a branch judgment function ‘IN => Int’. This function is an implementation that returns the index number of downstream according to IN contents. • Finally, the branch integration uses the Fan-in junction 'Merge' (n inputs 1 output). How to express conditional branch of Flow Partition Merge Any Flows … …
  23. Copyright © since 1998 DMM All Rights Reserved. 23 1BSUJUJPOδϟϯΫγϣϯ͸؆୯Ͱ͢ɻ

    ෼ذ൑ఆͷؔ਺ΛҾ਺ʹ౉͚ͩ͢Ͱ࢖͑·͢ɻ Sample code of Partition and Merge val routingFlowShape = GraphDSL.create() { implicit builder => import GraphDSL.Implicits._ type Input = (Routes, HttpRequest) val input = builder.add(Flow[Input]) val (found, notFound) = (1, 0) val findingRouteP = builder.add(Partition[Input](2, { case (routes, req) => if (routes.isFound(req)) found else notFound })) val findingRouteM = builder.add(Merge[HttpResponse](2)) val foundResponse = builder.add(/* Implementation in following slide! :) */???) val notFoundResponse = builder.add(Flow[Input])map(_ => HttpResponse(status = NotFound))) input ~> findingRouteP.in findingRouteP.out(found) ~> foundResponse ~> findingRouteM.in(found) findingRouteP.out(notFound) ~> notFoundResponse ~> findingRouteM.in(notFound) FlowShape(input.in, findingRouteM.out) } Defining Partition and Merge. Give the factory method’s argument a branch judgment function and junction width.
  24. Copyright © since 1998 DMM All Rights Reserved. 24 ੺ࣔͨ͘͠෦෼͸ฐࣾಠࣗઃܭͷGJMUFSQMVHJOͰ͢ɻࣾ಺޲͚

    )551ϔομͷ෇༩΍+40/31$㱻3&45ม׵ͳͲΛߦ͍·͢ɻ Combined Routing flow Routing flow diagram (fliter plug-in) Partition (find route) Endpoint Flow e.g. ・HTTP ・OAuth2.0 and future ・AMQP etc. Http. Incoming Connection IncomingFiltersFlow Inco ming Filter Inco ming Filter ・・・ OutgoingFiltersFlow Outg oing Filter Outg oing Filter ・・・ Partition (continue or abort) Merge AbortFlow Merge Not Found Flow If found If not found If abort If continue
  25. Copyright © since 1998 DMM All Rights Reserved. 25 ΤϯυϙΠϯτ΁ͷೖग़ྗ৘ใΛॊೈʹฤू͢Δ֦ுϙΠϯτɻ

    ϧʔτ৘ใʹඥ͍ͮͨϓϥάΠϯઃఆ͔Βಈతʹߏங͠·͢ɻ The plug-in modules are for flexible editing and verifying the input / output contents to the Endpoint. For example, • Addition Internal HTTP headers • Interconversion of REST and non-REST • Verifying authorized with OAuth2.0 etc. The composed result of the plug-in is constructed dynamically by the plug-in setting which links to the route information. For composing and applying filters in order, Flow’s IN and OUT type signatures are the same. IncomingFiltersFlow (OutgoingFiltersFlow is similar to this structure.) Source (single) Verify Autho rized REST ⇔non REST Foo Filter Add Head er Dynamically concatination flows by route info using ‘Flow.flatMapConcat(Out1 => Source[Out2, Mat])’. HttpRequestContext (request, route) About incoming (outgoing) filter plug-in
  26. Copyright © since 1998 DMM All Rights Reserved. 26 ΦϦδφϧͷ)551ϦΫΤετͷ಺༰Λอޢ͠ͳ͕ΒɺBQQMZϝ

    ιουͷॲཧΛద༻͠·͢ɻ Sample code of IncomingFilter trait // EI type variable is endpoint incoming data. V type variable is voluntary data. trait IncomingFilter[EI <: RequestContext, V] extends ActorSystemAware { type Acc = (HttpResponse ∨ EI, V) // Applied filter result type signature. type IO = (HttpRequestContext, Acc) // IncomingFilter.flow’s In-Out type signature. val flow: Flow[IO, IO, _] } trait AsyncIncomingFilter[EI <: RequestContext, V] extends IncomingFilter { protected def apply(route: Route, orig: HttpRequest, acc: Acc): Future[Acc] lazy val flow: Flow[IO, IO, _] = Flow[IO].flatMapConcat { case (orig, acc) => Source.fromFuture(apply(orig.route, orig.request, acc)).map((orig, _)) } } For the apply method, narrowing down the necessary input / output values and return the result with type Future[Acc]. Therefore, knowledge of Akka Streams is unnecessary for plug-in implementation. Note for: The original HttpRequestContext should not be passed to the apply method’s return value. As a reason, it is routed to the next IncomingFilter through a detour path so as not to destroy original.
  27. Copyright © since 1998 DMM All Rights Reserved. 27 㱹͕MFGUͷ࣌ʹ୹བྷධՁ͢ΔUSBJUɻMFGUͷܕ͸)UUQ3FTQPOTFɻ

    ݁Ռ͕MFGUͩͬͨ৔߹͸ԼྲྀͰ"CPSU'MPX΁సૹ͍ͯ͠·͢ɻ Sample code of right-bind IncomingFilter trait RightIncomingFilter[EI <: RequestContext, V] extends AsyncIncomingFilter[EI, V] { override final def apply(route: Route, orig: HttpRequest, acc: Acc): Future[Acc] = { acc match { case (∨-(ei), v) => applyRight(route, orig, (ei, v)) case left => Future.successful(left) } } protected def applyRight(route: Route, orig: HttpRequest, acc: (EI, V)): Future[Acc] } Rignt-bind implementation. ・ To short-circuit when the left comes. ・ ∨’s left type is HttpResponse. (It is sent downstream to AbortFlow.)
  28. Copyright © since 1998 DMM All Rights Reserved. 28 ϑΟϧλͷ߹੒͸Ϧετʹ֨ೲͨ͠'MPXΛ৞ΈࠐΉ͜ͱͰੜ੒͠

    ·͢ɻ৞ΈࠐΈͷॳظ஋ ୯Ґݩ ͸'MPXBQQMZ<5>Ͱऔಘ͠·͢ɻ Sample code of composing IncomingFilters object IncomingFilters { def flow[EI <: RequestContext, V](): Flow[HttpRequestContext,(HttpResponse ∨ EI, V), _] = { type Acc = (HttpResponse ∨ EI, V) Flow[HttpRequestContext].flatMapConcat { r => val initFlow = r.route.incomingValue.flow val combinedFlow = r.route.infilters[TO, V] .map(_.flow) .foldLeft(Flow[(HttpRequestContext, Acc)])(_.via(_)) Source.single(r).via(initFlow).via(combinedFlow) }.map { case (_, acc) => acc } } } Composing filters implimentation. ・ Filter composition is generated by folding Flow from the list. ・ Identity flow ‘Flow[T, T, _]’ can be taken with ‘Flow.apply[T]’.
  29. Copyright © since 1998 DMM All Rights Reserved. 29 )551&OUJUZͷղੳ͸"LLB)551ͷ6ONBSTIBMMFSΛ࢖͍·͢ɻ

    ݁Ռ͸'VUVSFʹ͘Δ·ΕΔͷͰ໌ࣔతͳϒϩοΫ͸ෆཁͰ͢ɻ Sample code of HTTP method editing private def editMethodWithEntityToQuery(destMethod: HttpMethod)(srcReq: HttpRequset): Future[HttpRequest] = val srcUri = srcReq.uri val srcEntity = srcReq.entity Unmarshal(srcEntity).to[FormData].map { formData => val srcQuery = srcUri.query() val fields = formData.fields val queryBuilder = Query.newBuilder // mutable API queryBuilder ++= srcQuery queryBuilder ++= fields val destQuery = queryBuilder.result val destQueryString = destQuery.toString.some.filterNot(_.isEmpty) val destHeaders = srcReq.headers.filterNot(_.name == `Content-Type`.name) srcReq.copy( method = destMethod, headers = destHeaders, entity = HttpEntity.Empty, uri = srcUri.copy(rawQueryString = destQueryString) ) } recover { case e: UnsupportedContentTypeException => logger.warn(s"Availed HTTP entity may be not single-part form data in HttpMethodEditing of entity to que srcReq } } How to parse object in Akka HTTP? Akka HTTP Unmarshaller is useful for parsing HTTP entities. Explicit blocking is unnecessary because parsing results are wrapped in Future. Depending on the type classes, the required implementation of Akka HTTP embedded is resolved at compilation time.
  30. Copyright © since 1998 DMM All Rights Reserved. 30 ࣍͸ΤϯυϙΠϯτͷઆ໌Ͱ͢ɻ

    ͪ͜Β΋೚ҙͷ࣮૷ΛઃఆͰ͖ΔΑ͏ʹϓϥάΠϯܗࣜͰ͢ɻ Combined Routing flow Routing flow diagram (endpoint plug-in) Partition (find route) Endpoint Flow e.g. ・HTTP ・OAuth2.0 and future ・AMQP etc. Http. Incoming Connection IncomingFiltersFlow Inco ming Filter Inco ming Filter ・・・ OutgoingFiltersFlow Outg oing Filter Outg oing Filter ・・・ Partition (continue or abort) Merge AbortFlow Merge Not Found Flow If found If not found If abort If continue
  31. Copyright © since 1998 DMM All Rights Reserved. 31 ओʹ֎෦௨৴͕໨తͷϓϥάΠϯͰ͢ɻ

    ࠓճ͸ೝՄ"1*΋&OEQPJOUϓϥάΠϯͱ࣮ͯ͠૷͠·ͨ͠ɻ As the API Gateway, Endpoint plug-in is a plug-in primarily for external communication. For example, • HTTP / HTTPS client • Authorization API's business logic • AMQP client (to do in the future) • Kafka client (to do in the future) • gRPC client (to do in the future) etc. In this time, OAuth 2.0 API is implemented by this as a external communication. About Endpoint plug-in
  32. Copyright © since 1998 DMM All Rights Reserved. 32 ࠓճ͸"1*(BUFXBZαʔόʔͳͷͰIPTUDPOOFDUJPOQPPMΛ࢖

    ༻ɻϗετຖͷϓʔϧ਺΍44-ͷઃఆ͸BQQMJDBUJPODPOG΁ɻ Sample code of Akka HTTP client trait HttpClients extends ActorSystemAware { // Type parameter [T] means to keep path-through data in Flow’s I/O. def forUri[T](apiUrl: Uri): Flow[(HttpRequest, T), (Try[HttpResponse], T), HostConnectionPool] = { val host = apiUrl.authority.host.toString val port = apiUrl.authority.port val hostSettings = resolveHostSpecificSettings(host) | defaultConnectionPoolSettings val sslCtxt = Global.clientHttpsContext | Http().defaultClientHttpsContext (apiUrl.scheme, port) match { case ("http" , 0) => Http().cachedHostConnectionPool[T](host, settings = hostSettings) case ("http" , _) => Http().cachedHostConnectionPool[T](host, port, hostSettings) case ("https", 0) => Http().cachedHostConnectionPoolHttps[T](host, connectionContext = sslCtxt , settings = hostSettings) case ("https", _) => Http().cachedHostConnectionPoolHttps[T](host, port, sslCtxt, hostSettings) case _ => throw new NoSuchElementException } // ....... } This time it is an API Gateway server, so we set up endpoints host connection pool. The number of connection pools and the setting of SSL for each host is basically set in application.conf.
  33. Copyright © since 1998 DMM All Rights Reserved. 33 Implementing

    Router with Graph API 'MPXHSBQIΛ༻͍ͨϧʔςΟϯάॲཧͷ࣮૷
  34. Copyright © since 1998 DMM All Rights Reserved. 34 •

    Receive HttpRequest at the Flow's parameter. • According to the setting of routing, make as many Partitions as number of routes. • Applied plug-in for each route is different and the length of the previous Flow by Partition is different. Implementation process ϧʔτఆٛͱಉ͡਺͚ͩ1BSUJUJPOʹΑΔ෼ذΛ࡞Γ·͢ɻ'MPX ͷ௕͞͸ϧʔτຖʹҟͳΓ·͢ɻ
  35. Copyright © since 1998 DMM All Rights Reserved. 35 •

    Depending on the requested path, the index (or root) of the Partition is specified. • Connect partial Graph per sort of plugin to Merge. ϦΫΤετ͞ΕͨύεʹԠͯ͡1BSUJUJPOͷࡧҾ ϧʔτͷࡧҾ ͕ ಛఆ͞Εɺ.FSHFʹܨ͛ΒΕ·͢ɻ Implementation process
  36. Copyright © since 1998 DMM All Rights Reserved. 36 First

    implementation val graph = GraphDSL.create() { implicit builder => import GraphDSL.Implicits._ val orderedListOfRoutes = routes.sequentialRoutes val routeP = builder.add(Partition[HttpRequest](routes.size, req => { // Routes#route returns index for partition.if Route not found returns 0. routes.route(req).map(_._1) getOrElse 0 })) val routeM = builder.add(Merge[Try[HttpResponse]](routes.size)) orderedListOfRoutes.zipWithIndex.foreach { case ((_, _), n) if n == notFoundIndex => routeP.out(n) ~> routeNotFoundFlow ~> routeM.in(n) case ((_, routeConfig), i) => { val endpointAndOutgoingPluginsFlow = Flow.fromFunction[(HttpResponse ∨ E, EI), (N, V)] ... val abortFlow = Flow.fromFunction[(HttpResponse ∨ Any, Any), Try[HttpResponse]] ... … // format: OFF routeP.out(i) ~> incomingPluginsFlow ~> continueOrAbortP.in continueOrAbortP.out(continue) ~> endpointAndOutgoingPluginsFlow ~> continueOrAbortM.in(continue) continueOrAbortP.out(abort) ~> abortFlow ~> continueOrAbortM.in(abort) continueOrAbortM.out ~> routeM.in(i) // format: ON } FlowShape(routeP.in, routeM.out) } } Ordered by index of Route. Create partial Flow as same as number of Route. 1BSUJUJPOͷࡧҾͱ߹͏Α͏ʹϧʔτ͕੔ྻ͞Εͨঢ়ଶͰอ࣋͞Ε ͍ͯ·͢ɻ
  37. Copyright © since 1998 DMM All Rights Reserved. 37 ...

    ... Endpoint Flow . . . i = index of Partition N = number of Route i = N i = 1 i = 0 Partition [HttpRequest] Route NotFound Flow ... ... Incoming Plugins Flow ... ... Outgoing Plugins Flow ... ... Endpoint Flow ... ... Incoming Plugins Flow ... ... Outgoing Plugins Flow Merge [Try[HttpResponse]] ࠷ॳͷ'MPXͷߏ଄ First implementation
  38. Copyright © since 1998 DMM All Rights Reserved. 38 •

    As the routing configuration size (Partition size) increased, heap size expanded. • When the route size exceeds 100, heap size rises 1 GB at the time of start. • Performance has been dragged by Full GC, and throughput has also shrunken. ϧʔτͷ਺ 1BSUJUJPOͷαΠζ ͕େ͖͘ͳΔʹͭΕͯɺىಈ࣌ͷ ώʔϓαΠζ͕๲ΒΈɺύϑΥʔϚϯεͷྼԽ͕ݟΒΕ·ͨ͠ɻ But ...
  39. Copyright © since 1998 DMM All Rights Reserved. 39 ...

    ... Endpoint Flow . . . i = index of Partition N = number of Route i = N i = 1 i = 0 Partition [HttpRequest] Route NotFound Flow ... ... Incoming Plugins Flow ... ... Outgoing Plugins Flow ... ... Endpoint Flow ... ... Incoming Plugins Flow ... ... Outgoing Plugins Flow Merge [Try[HttpResponse]] ࠷ॳͷ'MPXͷߏ଄ First implementation
  40. Copyright © since 1998 DMM All Rights Reserved. 40 मਖ਼ޙͷ'MPXͷߏ଄

    i = index of Partition i = 1 i = 0 Partition [HttpRequest] Route NotFound Flow ... ... Endpoint Flow ... ... Incoming Plugins Flow ... ... Outgoing Plugins Flow Merge [Try[HttpResponse]] Second implementation
  41. Copyright © since 1998 DMM All Rights Reserved. 41 Second

    implementation lazy val graph = GraphDSL.create() { implicit builder => import GraphDSL.Implicits._ val request = builder.add(Flow[HttpRequest].map {... val (found, notFound) = (1, 0) val findingRouteP = builder.add(Partition[HttpRequest](2,{ req => // Routes#route returns Option[Route] routes.route(req).map(found) | notFound })) val findingRouteM = builder.add(Merge[HttpResponse](2)) … // format: OFF request ~> findingRouteP.in findingRouteP.out(found) ~> incomingPluginsFlow ~> p.in continueOrAbortP.out(continue) ~> endpointAndOutgoingPluginsFlow ~> continueOrAbortM.in(continue) continueOrAbortP.out(abort) ~> abortFlow ~> continueOrAbortM.in(abort) continueOrAbortM.out ~> response ~> findingRouteM.in(found) findingRouteP.out(notFound) ~> notFoundResponse ~> findingRouteM.in(notFound) // format: ON FlowShape(request.in, findingRouteM.out) } Partition devided into two cases. 1BSUJUJPO͕ϦΫΤετʹରԠ͢Δϧʔτ͕ݟ͔ͭͬͨ৔߹ͱɺͦ ͏Ͱͳ͍৔߹ͷ͚̎ͭͩʹͳΔΑ͏ʹमਖ਼͠·ͨ͠ɻ
  42. Copyright © since 1998 DMM All Rights Reserved. 42 1BSUJUJPOΛখͨ͘͜͞͠ͱͰώʔϓαΠζ͕ݮগ͠·ͨ͠ɻ'BO

    *O0VU͸ଟྔͷϦΫΤετͷॲཧʹ͸޲͍͍ͯͳ͍Α͏Ͱ͢ɻ • Removed Partition in Rouing Flow and Graph got smaller. • The heap size at startup also has been decreased to 100-200M. • Full GC also disappeared and throughput has got improved. • Fan In/Fan Out is not suitable for large number of request processing. After improvement
  43. Copyright © since 1998 DMM All Rights Reserved. 43 ϦΫΤετຖʹ'MPXͰ૊·Ε͍ͯΔϓϥάΠϯΛಈతʹܨ͛Δ෦

    ෼͸GMBU.BQ$PODBU͕༗ޮͰͨ͠ɻ • However, depending on the result of routing, plug-ins that are organized by Flow must be selected and connect dynamically per request. • Flow#flatMapConcat was useful for the part where Flow should be dynamically connected. After improvement val incomingPluginsFlow = Flow[(HttpRequest,Route)].flatMapConcat { case (request,route) => // flatMapConcat requires SourceShape. Source.single((request -> route)).via { // Connect all filter(Flow) in this route. route.infilters.map{ infilter => infilter.flow}.foldLeft(initalValue)(_.via(_)) } }
  44. Copyright © since 1998 DMM All Rights Reserved. 44 ϝΠϯͱͳΔ'MPX͸(SBQI%4-Λ࢖ͬͯॻ͍͍ͯ·͢ɻ͜ΕΒΛ

    'MPXͰॻ͘ͱͲ͏ͳΔͰ͠ΐ͏͔ • We use Graph DSL in a main Flow. • DSL is very useful for visualizing that whole processing flow. • if this flow written without DSL... By the way... // format: OFF request ~> findingRouteP.in findingRouteP.out(found) ~> incomingPluginsFlow ~> p.in continueOrAbortP.out(continue) ~> endpointAndOutgoingPluginsFlow ~> continueOrAbortM.in(continue) continueOrAbortP.out(abort) ~> abortFlow ~> continueOrAbortM.in(abort) continueOrAbortM.out ~> response ~> findingRouteM.in(found) findingRouteP.out(notFound) ~> notFoundResponse ~> findingRouteM.in(notFound) // format: ON FlowShape(request.in, findingRouteM.out)
  45. Copyright © since 1998 DMM All Rights Reserved. 45 ৚݅෼ذͷͨͼʹ4PVSDF4IBQFΛೖΕࢠʹ͢Δߏ଄ʹͳΓ·͢ɻ

    ࠓճͷΑ͏ͳ୯७ͳϑϩʔͰ͸গ͠؆ܿʹͳΓ·ͨ͠ɻ lazy val endpointOrAboutSource: PartialFunction[(HttpResponse ∨ EI, V), Source[HttpResponse,NotUsed]] = { case (∨-(r), v) => Source.single((r, v)) .via(measuredProcessingTimeEndpointFlow).via(outgoingFlow).via(response) case (-∨(l), _) => Source.single(l).map(Success(_)).via(response) } lazy val flow = findingRouteFlow.flatMapConcat { case in @ (_, _, None) => Source.single(in).via(notFoundResponse) case x => Source.single(x).via(request).via(incomingFlow).flatMapConcat(endpointOrAbortSource) } Nested flatMapConcat request ~> findingRouteP.in findingRouteP.out(found) ~> incomingPluginsFlow ~> p.in continueOrAbortP.out(continue) ~> endpointAndOutgoingPluginsFlow ~> continueOrAbortM.in(continue) continueOrAbortP.out(abort) ~> abortFlow ~> continueOrAbortM.in(abort) continueOrAbortM.out ~> response ~> findingRouteM.in(found) findingRouteP.out(notFound) ~> notFoundResponse ~> findingRouteM.in(notFound) FlowShape(request.in, findingRouteM.out) Write main Flow without Graph DSL
  46. Copyright © since 1998 DMM All Rights Reserved. 46 Testing

    the Performance of Gateway (BUFXBZͷύϑΥʔϚϯεΛςετ͢Δ
  47. Copyright © since 1998 DMM All Rights Reserved. 47 ෛՙΫϥΠΞϯτΛ୆༻ҙ͠ɺ(BUFXBZͷഎޙʹ͸ϓϩΩγઌ

    ͱͳΔ8FC"1*Λ໛ͨ͠αʔόΛ༻ҙ͠·ͨ͠ɻ Instrumentation Web API (Mock server) CPU: 8 Core RAM: 8 GB
  48. Copyright © since 1998 DMM All Rights Reserved. 48 7FSUYͰ࡞੒͞Εͨ(BUFXBZͱൺֱͯ͠ɺ෇ՃػೳΛצҊͯ͠

    ΋े෼ͳεϧʔϓοτΛಘΔ͜ͱ͕Ͱ͖·ͨ͠ɻ ౰ࣾൺ Performance testing result • Throughput which created by Akka is comparable to previous Gateway(Vert.x 2) . • About 8% of performance was reduced. • We obtained sufficient throughput even with considering additional functions.
  49. Copyright © since 1998 DMM All Rights Reserved. 49 (BUFXBZͷಛੑΛ֬ೝ͢Δҝͷݕূͱͯ͠ɺ(BUFXBZͷഎޙʹ

    ஗͍ΤϯυϙΠϯτɺ଎͍ΤϯυϙΠϯτΛ༻ҙ͠·ͨ͠ɻ Testing the characteristics of Gateway • We prepared a slow endpoint (emulating processing time 1 second) and a fast endpoint (emulating processing time 50 ms) behind Gateway. Fast Endpoint Slow Endpoint 1500 req/sec constantry
  50. Copyright © since 1998 DMM All Rights Reserved. 50 ૒ํͷΤϯυϙΠϯτʹಉ࣌ʹෛՙΛֻ͚ɺ஗͍ΤϯυϙΠϯτ

    ͷӨڹΛड͚ͯશମͷεϧʔϓοτ͕མͪͳ͍͔ݕূ͠·ͨ͠ɻ • We loaded both endpoints at the same time and tested whether the overall throughput has not decreased because it was affected by the slow endpoint. Testing the characteristics of Gateway Fast Endpoint Slow Endpoint 1500 req/sec constantry
  51. Copyright © since 1998 DMM All Rights Reserved. 51 ଎͍ΤϯυϙΠϯτͰ͸ɺෛՙ͕ߴ͘ͳͬͯ΋ߴ͍εϧʔϓοτ

    Λҡ͍࣋ͯ͠·ͨ͠ɻ Proxy destination Throughput (req/sec) Minimum Response Time(msec) Average Response Time(msec) Maximum Response Time(msec) Fast Endpoint 1490.066 52 88 608 Slow Endpoint 80.972 1002 25920 600010 Test Result • At the fast endpoint, consistently high throughput was maintained even at high loads.
  52. Copyright © since 1998 DMM All Rights Reserved. 52 ஗͍ΤϯυϙΠϯτͰ͸ෛՙ͕ߴ͘ͳΔʹͭΕλΠϜΞ΢τ͕૿

    Ճ͕ͨ͠ɺΤϯυϙΠϯτ͕ఀࢭ͢Δࣄ͸͋Γ·ͤΜͰͨ͠ɻ Test Result • At the slow endpoint, timeout increased as load increased. • Connecting to the endpoint that has been timed out in 60 seconds. • Slow endpoints were not destroyed even when high load was applied. Proxy destination Throughput (req/sec) Minimum Response Time(msec) Average Response Time(msec) Maximum Response Time(msec) Fast Endpoint 1490.066 52 88 608 Slow Endpoint 80.972 1002 25920 600010
  53. Copyright © since 1998 DMM All Rights Reserved. 53 "LLB)551ͱ"LLB4USFBNTʹΑͬͯɺΤϯυϙΠϯτ΁ͷӨڹ

    Λ཈͑ͭͭɺߴ͍ύϑΥʔϚϯεΛಘΔ͜ͱ͕ग़དྷ·ͨ͠ɻ • Akka HTTP (and Akka Streams) performed the expected behavior of minimizing influence on the endpoint and the gateway itself using back-pressure control. • While performing backpressure control, it was possible to obtain sufficient throughput even compared to before transplantation. Summary
  54. Copyright © since 1998 DMM All Rights Reserved. 54 "LLB)551

    )PTU$POOFDUJPO1PPM ʹΑΔϓϩΩγ͸ɺ (BUFXBZ ϚΠΫϩαʔϏε ͷಛੑʹద߹ͯ͠͠·͢ʂ • Flow is suitable for constructing a static structure. • The proxy by HostConnectionPool is suitable due to the nature of the Gateway server. • Akka Streams and HTTP are useful for microservice architecture! Summary
  55. Any Questions? ࣭ٙԠ౴