Implementing DMM API Gateway in Akka Streams and HTTP

Implementing
DMM API Gateway in
Akka Streams and HTTP
%..ͷ"1*(BUFXBZΛ"LLB4USFBNTͱ
"LLB)551Ͱ࡞ΓࠐΜͰΈͨ
DMM.com labo Co.,Ltd. Yukio Kakumoto / Masateru Nishimura
Feb. 25, 2017 in ScalaMatsuri 2017

View Slide

Copyright © since 1998 DMM All Rights Reserved. 2
֯ຊ޾ੜ ੢ଜ੓ً
%..DPNϥϘ γεςϜຊ෦ ϓϥοτϑΥʔϜ։ൃ෦
About us
Yukio Kakumoto & Masateru Nishimura
DMM.com Labo Co.,Ltd.
We develop and improve our micro
service platform.

View Slide

%..DPN͸೔ຊΛ୅ද͢Δ&$αΠτͷͻͱͭͰ͢ɻ
About DMM.com Group
DMM.com is one of
Japan's leading
E-commerce websites.
The Outline of DMM's website and
the company:
· 2.5 billion page view per month.
· Managing more than 40 services.
· Members exceeding 20 million.
· Unit sales of about $ 700 million.
· Group sales of about $ 1.4 billion.
(Converted by: 1USD=100JPY)

View Slide

೥લ͔Β͍Ζ͍Ζ΍͖ͬͯ·͕ͨ͠ɺࠓճɺ"1*(BUFXBZͱೝ
Մ"1*Λ "LLB4USFBNTBOE)551Ͱ࡞Γ௚͠·ͨ͠ɻ
Refurbishment history of DMM’s platform
OAuth 2.0 Connect API (Fuel PHP)
API Gateway
(Fuel PHP) Non-blocking API GW
which was built in
OAuth2.0 API
Scala 2.11 /
Akka Streams
and HTTP
Non-blocking API Gateway
(Java7/Vert.x2)
Countermeasure for C10K
www.dmm.com /
www.dmm.co.jp
LAMP application
Typical problems,
large and monolithic
state was given by
business growth
priority. Another microservices Purchase info API, Settlement agent API etc.. (Fuel PHP)
Unified frontend shopping cart (FuelPHP/Couchbase)
Legacy emoney systems
Legacy shopping carts
Unified Emoney API (Java7/Vert.x2/In-memory Data Grid)
new Member API (Java8/Spring)
Old Member API (LAMP)
since ‘03〜? ‘14 Q1 ‘15 Q2 ‘16 Q3 ‘17 Q1〜
Today’s agenda.

View Slide

ຊ೔͓࿩͢Δ಺༰
Today's agenda
● Project overview
● Architecture design
● Designing Flow Graphs
● Implementing Router with Graph API
● Testing the Performance of Gateway

View Slide

Project overview
ϓϩδΣΫτ֓ཁ
6

View Slide

ೝՄύϑΥʔϚϯεͷڧԽͳͲΛ໨తʹɺ"1*(BUFXBZͱೝՄ
"1*Λ"LLBͰ౷߹͠·ͨ͠ɻ
After
Before
What we have done
Backend
API
servers
API-GW
clients
(API Gateway)
(OAuth2.0 API)
(Unified API Gateway and OAuth2.0 API)
The purpose of the project
・To increase authorization
performance.
・To centralize management of
API routing and authorizing
(to do in the future).

View Slide

"1*(BUFXBZ͸ࣾ಺֎ͷଟ਺ͷγεςϜ͔Βͷॲཧͷ૭ޱͱͳΔ
ॏཁͳγεςϜͰ͢ɻ
Responsibility of API Gateway
LB
LB
・Mobile devices
・3rd party
etc.
Backend
API
servers
・HTTPS TLS decoder
・Ephemeral port buffer
API Gateway (Today's agenda.)
・Routing to backend APIs
・Authentication with OAuth2.0
…
Actual to
x,000req/sec
during peak time.
…
Managing of
・member
・billing
・accounting
etc.

View Slide

Architecture design
ΞʔΩςΫνϟσβΠϯʹ͍ͭͯ
9

View Slide

"LLB4USFBNTΛ࣠ʹ"LLB)5514FSWFS $MJFOUΛ࠾༻
ετϨʔδ͸ .Z42-ɾ3FEJT 4FOUJOFM
ɾ)BTIJDPSQ7BVMU
Scala 2.11 on JVM8
Akka Streams 2.4 and HTTP 10.0 Low-level API
Original Framework
Router
Hashicorp Valut driver
Endpoint plugins
HTTP(S) endpoint plugin
OAuth2.0
endpoint
plugin
ScalikeJDBC
Rediscala
Incoming
plugins
Outgoing
plugins
Akka
HTTP
Server
Akka
HTTP
Client
(HTTP(S) API driver)
Technology stack
Backend
API
servers
storages
Redis®
Redis Sentinel®
Hashicorp Vault
API-GW
clients

View Slide

શͯͷετϨʔδυϥΠό͸'VUVSFʹରԠ͍ͯ͠·͢ɻ
Technology stack (storages & drivers)
ScalikeJDBC (using MySQL)
• It's Query driver which has SQL interpolation function that is very easy to use.
• Future-based asynchronous interface.
Rediscala (using Redis® for cache storage)
• Supporting failover using Redis Sentinel® (discovery service for Redis®).
• Future-based asynchronous interface.
(Temporary solution for taking over from PHP version and running in parallel.)
Hashicorp Vault with original driver implements Akka HTTP Client
• For the purpose of keeping secret data.
(For example DB password, salt for hash function etc.)
• Unified akka HTTP Clinet because it’s interface is HTTP REST API.
• Future-based asynchronous interface (via Akka Streams’ mat).

View Slide

ߴ৴པͷ+7.ͱ.BWFOϦϙδτϦͷ๛෋ͳϥΠϒϥϦ܈ʹՃ͑
"LLB4USFBNTͷόοΫϓϨογϟʔ͕͋Ε͹׬ᘳʂ
Why did we adopt Akka Streams?
As API Gateway, stability is the most important, and productivity
(fulfillment of libraries and drivers etc.) is also important.
Stability
• Running on JVM 8 which is a runtime of high achievement and reliability.
• Using Akka with sufficient operational results for internal messaging.
• Having protection of internal messaging flow by Akka streams’ back pressure.
Productivity
• Various libraries and drivers in several repositories. e.g. Maven, Ivy, etc.
• Scala features, type safety and excellent syntax.
Akka Streams is a comprehensive powerful solution!

View Slide

όοΫϓϨογϟʔͱ͸ɺϑϩʔ੍ޚʹ͓͍ͯԼྲྀͷόοϑΝ͕
ҲΕͳ͍Α͏্ྲྀ΁࿈࠯తʹྲྀྔ৘ใΛ௨஌͢Δ࢓૊Έͷ͜ͱ
What is back pressure?
In information terminology, in order to prevent
the downstream buffer from overflowing in flow control,
flow information is notified in a chain manner to the upstream.
Part of supported libraries
• Akka Streams(Scala)
• Scalaz Stream(Scala)
• Node.js (JavaScript)
• ReactiveX
• RxJava
• RxScala
• RxJS etc.
Implementation depends on each language, library, and backend queue system.
Publisher
(upstream)
Subscriber
(downstream)
For example in Akka streams’ back pressure.
‘Dynamic push / pull mode’
✉✉✉✉✉
request(5)

View Slide

"1*(BUFXBZͳͷͰϝϞϦ಺ͷϧʔτઃఆͷϦϩʔυ͕ඞཁɻ
)JHIMFWFM"1*Ͱ͸ϧʔςΟϯάͷ%4-͕ϋʔυίʔυͷͨΊº
The use of API Gateway requires
reloading of route setting in memory
at an arbitrary timing.
Implementing this with the Akka
HTTP High-level API can not satisfy
the requirement, because routing
DSL is hard code.
Low-level API High-level API
Flexibility Yes.
Akka HTTP can
bind original flow
implimented by
ourself.
No.
Hard-coded routing
DSL generates flow
to bind to Akka
HTTP.
Difficulty Hard.
Requireing of
understanding of
Akka streams
Graph DSL.
Easy.
Routing DSL is
Spray-like simpile
and intuitive
directive.
We chose Akka HTTP Low-level API

View Slide

ϧʔτ৘ใΛ)0$0/Ͱهड़͠ɺ)61γάφϧͰࠩ͠ସ͑Δɻ
ϦϓϨʔεલγεςϜͷઃఆ͕KTPOͷͨΊ্Ґޓ׵ͱΕΔɻ

HOCON based original routing system
routes.
before.json
(JSON)
routes.conf
(HOCON)
routes.
before.conf
(HOCON)
Copy as is
(and rename)
include
SIGHUP!
In a common practice,
when the application
receives a HUP
signal, it reloads
route information.
HOCON is upward compatible with json.
By using this, compatibility can be keeping by including the
pre-replacement configuration as it is.

View Slide

+40/Ͱهड़͍ͯͨ͠ϧʔςΟϯάͷઃఆ͕)0$0/Ͱهड़Ͱ͖
ΔΑ͏ʹͳΓɺهड़ྔ͕ܹݮ͠·ͨ͠ʂ
HOCON is a human optimised config : )
foo.__default__.versions {
v1.config.outfilter.valid = 200
v4.config.outfilter.invalid = 500
...
}
"foo": {
"__default__": {
"versions": {
"v1": {
"config": {
"outfilter": {
"valid": 200
}
}
},
"v4": {
"config": {
"outfilter": {
"invalid": 500
}
}
}
},
...
},
...
}
routes.conf
HOCON making
configuration more simple!
routes.json

View Slide

"UPNJD3FGFSDFOFʹͱͬͨ3PVUFTͷࢀরΛɺ)UUQ3FRVFTUͱϖ
ΞͰ3PVUJOH'MPX΁ྲྀ͠ࠐΈ·͢ɻ
Sample code of swapping routes
trait ApplicationServer extends ActorSystemAware {
def serverName: String
// This AtomicReference is updated when SIGHUP is received.
protected lazy val routesRef: AtomicReference[Routes] = Routes.atomicRefercnce
protected lazy val serverFlow: Flow[HttpRequest, HttpResponse, _] =
Flow[HttpRequest]
// It is necessary to the latest route information.
.map { req => (req, routesRef.get) }
.via(RoutingFlow.flow)
// Start the HTTP server. The started server can stop using return value ‘Http.ServerBindig’.
protected lazy val serverBinding: Future[Http.ServerBinding] = {
Global.serverSetting(serverName) match {
case (host, port, Some(connectionContext))=>
Http().bindAndHandle(serverFlow, host, port, connectionContext)
case (host, port, None) =>
Http().bindAndHandle(serverFlow, host, port)
}
}
}

View Slide

Designing
Flow Graphs
'MPX(SBQIͷઃܭʹ͍ͭͯ
18

View Slide

ϑϩʔͷશମਤ
Combined Routing flow
Routing flow diagram
Partition
(find
route)
Endpoint
Flow
e.g.
・HTTP
・OAuth2.0
and future
・AMQP
etc.
Http.
Incoming
Connection
IncomingFiltersFlow
Inco
ming
Filter
Inco
ming
Filter
・・・
OutgoingFiltersFlow
Outg
oing
Filter
Outg
oing
Filter
・・・
Partition
(continue
or abort)
Merge
AbortFlow
Merge
Not
Found
Flow
If
found
If not
found
If abort
If continue

View Slide

&OEQPJOUʹΞΫηεෆཁͳͱ͖ͷ୹བྷܦ࿏Λ༻ҙ͍ͯ͠·͢ɻ
͜ͷਤͰ੺͘هͨ͠/PU'PVOE'MPXͱ"CPSU'MPXͰ͢ɻ
Routing flow diagram (short circuit)
Partition
(find
route)
Endpoint
Flow
e.g.
・HTTP
・OAuth2.0
and future
・AMQP
etc.
Http.
Incoming
Connection
IncomingFiltersFlow
Inco
ming
Filter
Inco
ming
Filter
・・・
OutgoingFiltersFlow
Outg
oing
Filter
Outg
oing
Filter
・・・
Partition
(continue
or abort)
Merge
AbortFlow
Merge
Not
Found
Flow
If
found
If not
found
If abort
If continue

View Slide

৚݅෼ذʹ͍ͭͯ
Routing flow diagram (conditional branch)
Partition
(find
route)
Endpoint
Flow
e.g.
・HTTP
・OAuth2.0
and future
・AMQP
etc.
Http.
Incoming
Connection
IncomingFiltersFlow
Inco
ming
Filter
Inco
ming
Filter
・・・
OutgoingFiltersFlow
Outg
oing
Filter
Outg
oing
Filter
・・・
Partition
(continue
or abort)
Merge
AbortFlow
Merge
Not
Found
Flow
If
found
If not
found
If abort
If continue

View Slide

ϑϩʔͷ෼ذʹ͸"LLB4USFBNTͷ(SBQI%4-͕࢖͑·͢ɻ
৚݅෼ذͳΒ'BOPVUδϟϯΫγϣϯͷ1BSUJUJPOΛ࢖͍·͢ɻ
Akka Streams’ GraphDSL can be used for expressing
branch of Flow. (And also for balance, broadcast or etc.)
In the conditional branching case,
• The branching uses the Fan-out junction 'Partition' (1 input n outputs).
Give the Partition’s factory method’s argument a branch judgment function ‘IN => Int’.
This function is an implementation that returns the index number of downstream according to
IN contents.
• Finally, the branch integration uses
the Fan-in junction 'Merge'
(n inputs 1 output).
How to express conditional branch of Flow
Partition Merge
Any
Flows
…
…

View Slide

1BSUJUJPOδϟϯΫγϣϯ͸؆୯Ͱ͢ɻ
෼ذ൑ఆͷؔ਺ΛҾ਺ʹ౉͚ͩ͢Ͱ࢖͑·͢ɻ
Sample code of Partition and Merge
val routingFlowShape = GraphDSL.create() { implicit builder =>
import GraphDSL.Implicits._
type Input = (Routes, HttpRequest)
val input = builder.add(Flow[Input])
val (found, notFound) = (1, 0)
val findingRouteP = builder.add(Partition[Input](2, { case (routes, req) =>
if (routes.isFound(req)) found else notFound
}))
val findingRouteM = builder.add(Merge[HttpResponse](2))
val foundResponse = builder.add(/* Implementation in following slide! :) */???)
val notFoundResponse = builder.add(Flow[Input])map(_ => HttpResponse(status = NotFound)))
input ~> findingRouteP.in
findingRouteP.out(found) ~> foundResponse ~> findingRouteM.in(found)
findingRouteP.out(notFound) ~> notFoundResponse ~> findingRouteM.in(notFound)
FlowShape(input.in, findingRouteM.out)
}
Defining Partition and Merge.
Give the factory method’s
argument a branch judgment
function and junction width.

View Slide

੺ࣔͨ͘͠෦෼͸ฐࣾಠࣗઃܭͷGJMUFSQMVHJOͰ͢ɻࣾ಺޲͚
)551ϔομͷ෇༩΍+40/31$㱻3&45ม׵ͳͲΛߦ͍·͢ɻ
Routing flow diagram (fliter plug-in)
Partition
(find
route)
Endpoint
Flow
e.g.
・HTTP
・OAuth2.0
and future
・AMQP
etc.
Http.
Incoming
Connection
IncomingFiltersFlow
Inco
ming
Filter
Inco
ming
Filter
・・・
OutgoingFiltersFlow
Outg
oing
Filter
Outg
oing
Filter
・・・
Partition
(continue
or abort)
Merge
AbortFlow
Merge
Not
Found
Flow
If
found
If not
found
If abort
If continue

View Slide

ΤϯυϙΠϯτ΁ͷೖग़ྗ৘ใΛॊೈʹฤू͢Δ֦ுϙΠϯτɻ
ϧʔτ৘ใʹඥ͍ͮͨϓϥάΠϯઃఆ͔Βಈతʹߏங͠·͢ɻ
The plug-in modules are for flexible editing and verifying
the input / output contents to the Endpoint.
For example,
• Addition Internal HTTP headers
• Interconversion of REST and non-REST
• Verifying authorized with OAuth2.0
etc.
The composed result of the plug-in is constructed
dynamically by the plug-in setting which links to
the route information.
For composing and applying filters in order,
Flow’s IN and OUT type signatures are the same.
IncomingFiltersFlow
(OutgoingFiltersFlow is similar to this structure.)
Source
(single)
Verify
Autho
rized
REST
⇔non
REST
Foo
Filter
Add
Head
er
Dynamically
concatination flows by route info
using ‘Flow.flatMapConcat(Out1 => Source[Out2, Mat])’.
HttpRequestContext
(request, route)
About incoming (outgoing) filter plug-in

View Slide

ΦϦδφϧͷ)551ϦΫΤετͷ಺༰Λอޢ͠ͳ͕ΒɺBQQMZϝ
ιουͷॲཧΛద༻͠·͢ɻ
Sample code of IncomingFilter trait
// EI type variable is endpoint incoming data. V type variable is voluntary data.
trait IncomingFilter[EI <: RequestContext, V] extends ActorSystemAware {
type Acc = (HttpResponse ∨ EI, V) // Applied filter result type signature.
type IO = (HttpRequestContext, Acc) // IncomingFilter.flow’s In-Out type signature.
val flow: Flow[IO, IO, _]
}
trait AsyncIncomingFilter[EI <: RequestContext, V] extends IncomingFilter {
protected def apply(route: Route, orig: HttpRequest, acc: Acc): Future[Acc]
lazy val flow: Flow[IO, IO, _] = Flow[IO].flatMapConcat { case (orig, acc) =>
Source.fromFuture(apply(orig.route, orig.request, acc)).map((orig, _))
}
}
For the apply method, narrowing down
the necessary input / output values and
return the result with type Future[Acc].
Therefore, knowledge of Akka Streams is
unnecessary for plug-in implementation.
Note for:
The original HttpRequestContext should not be passed to the
apply method’s return value.
As a reason, it is routed to the next IncomingFilter through a
detour path so as not to destroy original.

View Slide

㱹͕MFGUͷ࣌ʹ୹བྷධՁ͢ΔUSBJUɻMFGUͷܕ͸)UUQ3FTQPOTFɻ
݁Ռ͕MFGUͩͬͨ৔߹͸ԼྲྀͰ"CPSU'MPX΁సૹ͍ͯ͠·͢ɻ
Sample code of right-bind IncomingFilter
trait RightIncomingFilter[EI <: RequestContext, V] extends AsyncIncomingFilter[EI, V] {
override final def apply(route: Route, orig: HttpRequest, acc: Acc): Future[Acc] = {
acc match {
case (∨-(ei), v) => applyRight(route, orig, (ei, v))
case left => Future.successful(left)
}
}
protected def applyRight(route: Route, orig: HttpRequest, acc: (EI, V)): Future[Acc]
}
Rignt-bind implementation.
・ To short-circuit when the left comes.
・ ∨’s left type is HttpResponse.
(It is sent downstream to AbortFlow.)

View Slide

ϑΟϧλͷ߹੒͸Ϧετʹ֨ೲͨ͠'MPXΛ৞ΈࠐΉ͜ͱͰੜ੒͠
·͢ɻ৞ΈࠐΈͷॳظ஋ ୯Ґݩ
͸'MPXBQQMZ<5>Ͱऔಘ͠·͢ɻ
Sample code of composing IncomingFilters
object IncomingFilters {
def flow[EI <: RequestContext, V](): Flow[HttpRequestContext,(HttpResponse ∨ EI, V), _] = {
type Acc = (HttpResponse ∨ EI, V)
Flow[HttpRequestContext].flatMapConcat { r =>
val initFlow = r.route.incomingValue.flow
val combinedFlow = r.route.infilters[TO, V]
.map(_.flow)
.foldLeft(Flow[(HttpRequestContext, Acc)])(_.via(_))
Source.single(r).via(initFlow).via(combinedFlow)
}.map { case (_, acc) => acc }
}
} Composing filters implimentation.
・ Filter composition is generated by folding Flow from the list.
・ Identity flow ‘Flow[T, T, _]’ can be taken with ‘Flow.apply[T]’.

View Slide

)551&OUJUZͷղੳ͸"LLB)551ͷ6ONBSTIBMMFSΛ࢖͍·͢ɻ
݁Ռ͸'VUVSFʹ͘Δ·ΕΔͷͰ໌ࣔతͳϒϩοΫ͸ෆཁͰ͢ɻ
Sample code of HTTP method editing
private def editMethodWithEntityToQuery(destMethod: HttpMethod)(srcReq: HttpRequset): Future[HttpRequest] =
val srcUri = srcReq.uri
val srcEntity = srcReq.entity
Unmarshal(srcEntity).to[FormData].map { formData =>
val srcQuery = srcUri.query()
val fields = formData.fields
val queryBuilder = Query.newBuilder // mutable API
queryBuilder ++= srcQuery
queryBuilder ++= fields
val destQuery = queryBuilder.result
val destQueryString = destQuery.toString.some.filterNot(_.isEmpty)
val destHeaders = srcReq.headers.filterNot(_.name == `Content-Type`.name)
srcReq.copy(
method = destMethod, headers = destHeaders, entity = HttpEntity.Empty,
uri = srcUri.copy(rawQueryString = destQueryString)
)
} recover { case e: UnsupportedContentTypeException =>
logger.warn(s"Availed HTTP entity may be not single-part form data in HttpMethodEditing of entity to que
srcReq
}
}
How to parse object in Akka HTTP?
Akka HTTP Unmarshaller is useful for parsing HTTP
entities. Explicit blocking is unnecessary because
parsing results are wrapped in Future.
Depending on the type classes, the required
implementation of Akka HTTP embedded is resolved
at compilation time.

View Slide

࣍͸ΤϯυϙΠϯτͷઆ໌Ͱ͢ɻ
ͪ͜Β΋೚ҙͷ࣮૷ΛઃఆͰ͖ΔΑ͏ʹϓϥάΠϯܗࣜͰ͢ɻ
Routing flow diagram (endpoint plug-in)
Partition
(find
route)
Endpoint
Flow
e.g.
・HTTP
・OAuth2.0
and future
・AMQP
etc.
Http.
Incoming
Connection
IncomingFiltersFlow
Inco
ming
Filter
Inco
ming
Filter
・・・
OutgoingFiltersFlow
Outg
oing
Filter
Outg
oing
Filter
・・・
Partition
(continue
or abort)
Merge
AbortFlow
Merge
Not
Found
Flow
If
found
If not
found
If abort
If continue

View Slide

ओʹ֎෦௨৴͕໨తͷϓϥάΠϯͰ͢ɻ
ࠓճ͸ೝՄ"1*΋&OEQPJOUϓϥάΠϯͱ࣮ͯ͠૷͠·ͨ͠ɻ
As the API Gateway, Endpoint plug-in is a plug-in
primarily for external communication.
For example,
• HTTP / HTTPS client
• Authorization API's business logic
• AMQP client (to do in the future)
• Kafka client (to do in the future)
• gRPC client (to do in the future)
etc.
In this time, OAuth 2.0 API is implemented by this as a external communication.
About Endpoint plug-in

View Slide

ࠓճ͸"1*(BUFXBZαʔόʔͳͷͰIPTUDPOOFDUJPOQPPMΛ࢖
༻ɻϗετຖͷϓʔϧ਺΍44-ͷઃఆ͸BQQMJDBUJPODPOG΁ɻ
Sample code of Akka HTTP client
trait HttpClients extends ActorSystemAware {
// Type parameter [T] means to keep path-through data in Flow’s I/O.
def forUri[T](apiUrl: Uri): Flow[(HttpRequest, T), (Try[HttpResponse], T), HostConnectionPool] = {
val host = apiUrl.authority.host.toString
val port = apiUrl.authority.port
val hostSettings = resolveHostSpecificSettings(host) | defaultConnectionPoolSettings
val sslCtxt = Global.clientHttpsContext | Http().defaultClientHttpsContext
(apiUrl.scheme, port) match {
case ("http" , 0) => Http().cachedHostConnectionPool[T](host, settings = hostSettings)
case ("http" , _) => Http().cachedHostConnectionPool[T](host, port, hostSettings)
case ("https", 0) => Http().cachedHostConnectionPoolHttps[T](host, connectionContext = sslCtxt
, settings = hostSettings)
case ("https", _) => Http().cachedHostConnectionPoolHttps[T](host, port, sslCtxt, hostSettings)
case _ => throw new NoSuchElementException
}
// .......
}
This time it is an API Gateway server, so we set up endpoints
host connection pool.
The number of connection pools and the setting of SSL for each
host is basically set in application.conf.

View Slide

Implementing Router
with Graph API
'MPXHSBQIΛ༻͍ͨϧʔςΟϯάॲཧͷ࣮૷

View Slide

• Receive HttpRequest at the Flow's parameter.
• According to the setting of routing, make as many Partitions as number
of routes.
• Applied plug-in for each route is different and the length of the
previous Flow by Partition is different.
Implementation process
ϧʔτఆٛͱಉ͡਺͚ͩ1BSUJUJPOʹΑΔ෼ذΛ࡞Γ·͢ɻ'MPX
ͷ௕͞͸ϧʔτຖʹҟͳΓ·͢ɻ

View Slide

• Depending on the requested path, the index (or root) of the Partition is
specified.
• Connect partial Graph per sort of plugin to Merge.
ϦΫΤετ͞ΕͨύεʹԠͯ͡1BSUJUJPOͷࡧҾ ϧʔτͷࡧҾ
͕
ಛఆ͞Εɺ.FSHFʹܨ͛ΒΕ·͢ɻ
Implementation process

View Slide

First implementation
val graph = GraphDSL.create() { implicit builder =>
val orderedListOfRoutes = routes.sequentialRoutes
val routeP = builder.add(Partition[HttpRequest](routes.size, req => {
// Routes#route returns index for partition.if Route not found returns 0.
routes.route(req).map(_._1) getOrElse 0
}))
val routeM = builder.add(Merge[Try[HttpResponse]](routes.size))
orderedListOfRoutes.zipWithIndex.foreach {
case ((_, _), n) if n == notFoundIndex => routeP.out(n) ~> routeNotFoundFlow ~> routeM.in(n)
case ((_, routeConfig), i) => {
val endpointAndOutgoingPluginsFlow = Flow.fromFunction[(HttpResponse ∨ E, EI), (N, V)] ...
val abortFlow = Flow.fromFunction[(HttpResponse ∨ Any, Any), Try[HttpResponse]] ...
…
// format: OFF
routeP.out(i) ~> incomingPluginsFlow ~> continueOrAbortP.in
continueOrAbortP.out(continue) ~> endpointAndOutgoingPluginsFlow ~> continueOrAbortM.in(continue)
continueOrAbortP.out(abort) ~> abortFlow ~> continueOrAbortM.in(abort)
continueOrAbortM.out ~> routeM.in(i)
// format: ON
}
FlowShape(routeP.in, routeM.out)
}
}
Ordered by index of
Route.
Create partial Flow
as same as number
of Route.
1BSUJUJPOͷࡧҾͱ߹͏Α͏ʹϧʔτ͕੔ྻ͞Εͨঢ়ଶͰอ࣋͞Ε
͍ͯ·͢ɻ

View Slide

... ...
Endpoint
Flow
.
.
.
i = index of Partition
N = number of Route
i = N
i = 1
i = 0
Partition
[HttpRequest]
Route
NotFound
Flow
... ...
Incoming
Plugins
Flow
... ...
Outgoing
Plugins
Flow
... ...
Endpoint
Flow
... ...
Incoming
Plugins
Flow
... ...
Outgoing
Plugins
Flow
Merge
[Try[HttpResponse]]
࠷ॳͷ'MPXͷߏ଄

View Slide

• As the routing configuration size (Partition size) increased, heap size expanded.
• When the route size exceeds 100, heap size rises 1 GB at the time of start.
• Performance has been dragged by Full GC, and throughput has also shrunken.
ϧʔτͷ਺ 1BSUJUJPOͷαΠζ
͕େ͖͘ͳΔʹͭΕͯɺىಈ࣌ͷ
ώʔϓαΠζ͕๲ΒΈɺύϑΥʔϚϯεͷྼԽ͕ݟΒΕ·ͨ͠ɻ
But ...

View Slide

... ...
Endpoint
Flow
.
.
.
N = number of Route
i = N
i = 1
i = 0
Partition
[HttpRequest]
Route
NotFound
Flow
... ...
Incoming
Plugins
Flow
... ...
Outgoing
Plugins
Flow
... ...
Endpoint
Flow
... ...
Incoming
Plugins
Flow
... ...
Outgoing
Plugins
Flow
Merge
[Try[HttpResponse]]
࠷ॳͷ'MPXͷߏ଄

View Slide

मਖ਼ޙͷ'MPXͷߏ଄
i = 1
i = 0
Partition
[HttpRequest]
Route
NotFound
Flow
... ...
Endpoint
Flow
... ...
Incoming
Plugins
Flow
... ...
Outgoing
Plugins
Flow
Merge
[Try[HttpResponse]]
Second implementation

View Slide

Second implementation
lazy val graph = GraphDSL.create() { implicit builder =>
val request = builder.add(Flow[HttpRequest].map {...
val (found, notFound) = (1, 0)
val findingRouteP = builder.add(Partition[HttpRequest](2,{ req =>
// Routes#route returns Option[Route]
routes.route(req).map(found) | notFound
}))
val findingRouteM = builder.add(Merge[HttpResponse](2))
…
// format: OFF
request ~> findingRouteP.in
findingRouteP.out(found) ~> incomingPluginsFlow ~> p.in
continueOrAbortM.out ~> response ~> findingRouteM.in(found)
// format: ON
FlowShape(request.in, findingRouteM.out)
}
Partition devided
into two cases.
1BSUJUJPO͕ϦΫΤετʹରԠ͢Δϧʔτ͕ݟ͔ͭͬͨ৔߹ͱɺͦ
͏Ͱͳ͍৔߹ͷ͚̎ͭͩʹͳΔΑ͏ʹमਖ਼͠·ͨ͠ɻ

View Slide

1BSUJUJPOΛখͨ͘͜͞͠ͱͰώʔϓαΠζ͕ݮগ͠·ͨ͠ɻ'BO
*O0VU͸ଟྔͷϦΫΤετͷॲཧʹ͸޲͍͍ͯͳ͍Α͏Ͱ͢ɻ
• Removed Partition in Rouing Flow and Graph got smaller.
• The heap size at startup also has been decreased to 100-200M.
• Full GC also disappeared and throughput has got improved.
• Fan In/Fan Out is not suitable for large number of request processing.
After improvement

View Slide

ϦΫΤετຖʹ'MPXͰ૊·Ε͍ͯΔϓϥάΠϯΛಈతʹܨ͛Δ෦
෼͸GMBU.BQ$PODBU͕༗ޮͰͨ͠ɻ
• However, depending on the result of routing, plug-ins that are organized by Flow
must be selected and connect dynamically per request.
• Flow#flatMapConcat was useful for the part where Flow should be dynamically
connected.
After improvement
val incomingPluginsFlow = Flow[(HttpRequest,Route)].flatMapConcat {
case (request,route) =>
// flatMapConcat requires SourceShape.
Source.single((request -> route)).via {
// Connect all filter(Flow) in this route.
route.infilters.map{ infilter => infilter.flow}.foldLeft(initalValue)(_.via(_))
}
}

View Slide

ϝΠϯͱͳΔ'MPX͸(SBQI%4-Λ࢖ͬͯॻ͍͍ͯ·͢ɻ͜ΕΒΛ
'MPXͰॻ͘ͱͲ͏ͳΔͰ͠ΐ͏͔
• We use Graph DSL in a main Flow.
• DSL is very useful for visualizing that whole processing flow.
• if this flow written without DSL...
By the way...
// format: OFF
// format: ON

View Slide

৚݅෼ذͷͨͼʹ4PVSDF4IBQFΛೖΕࢠʹ͢Δߏ଄ʹͳΓ·͢ɻ
ࠓճͷΑ͏ͳ୯७ͳϑϩʔͰ͸গ͠؆ܿʹͳΓ·ͨ͠ɻ
lazy val endpointOrAboutSource: PartialFunction[(HttpResponse ∨ EI, V), Source[HttpResponse,NotUsed]] = {
case (∨-(r), v) => Source.single((r, v))
.via(measuredProcessingTimeEndpointFlow).via(outgoingFlow).via(response)
case (-∨(l), _) => Source.single(l).map(Success(_)).via(response)
}
lazy val flow = findingRouteFlow.flatMapConcat {
case in @ (_, _, None) => Source.single(in).via(notFoundResponse)
case x => Source.single(x).via(request).via(incomingFlow).flatMapConcat(endpointOrAbortSource)
}
Nested
flatMapConcat
Write main Flow without Graph DSL

View Slide

Testing the Performance
of Gateway
(BUFXBZͷύϑΥʔϚϯεΛςετ͢Δ

View Slide

ෛՙΫϥΠΞϯτΛ୆༻ҙ͠ɺ(BUFXBZͷഎޙʹ͸ϓϩΩγઌ
ͱͳΔ8FC"1*Λ໛ͨ͠αʔόΛ༻ҙ͠·ͨ͠ɻ
Instrumentation
Web API
(Mock server)
CPU: 8 Core
RAM: 8 GB

View Slide

7FSUYͰ࡞੒͞Εͨ(BUFXBZͱൺֱͯ͠ɺ෇ՃػೳΛצҊͯ͠
΋े෼ͳεϧʔϓοτΛಘΔ͜ͱ͕Ͱ͖·ͨ͠ɻ ౰ࣾൺ

Performance testing result
• Throughput which created by Akka is comparable to previous Gateway(Vert.x 2) .
• About 8% of performance was reduced.
• We obtained sufficient throughput even with considering additional functions.

View Slide

(BUFXBZͷಛੑΛ֬ೝ͢Δҝͷݕূͱͯ͠ɺ(BUFXBZͷഎޙʹ
஗͍ΤϯυϙΠϯτɺ଎͍ΤϯυϙΠϯτΛ༻ҙ͠·ͨ͠ɻ
Testing the characteristics of Gateway
• We prepared a slow endpoint (emulating processing time 1 second) and a fast
endpoint (emulating processing time 50 ms) behind Gateway.
Fast
Endpoint
Slow
Endpoint
1500 req/sec
constantry

View Slide

૒ํͷΤϯυϙΠϯτʹಉ࣌ʹෛՙΛֻ͚ɺ஗͍ΤϯυϙΠϯτ
ͷӨڹΛड͚ͯશମͷεϧʔϓοτ͕མͪͳ͍͔ݕূ͠·ͨ͠ɻ
• We loaded both endpoints at the same time and tested whether the overall
throughput has not decreased because it was affected by the slow endpoint.
Testing the characteristics of Gateway
Fast
Endpoint
Slow
Endpoint
1500 req/sec
constantry

View Slide

଎͍ΤϯυϙΠϯτͰ͸ɺෛՙ͕ߴ͘ͳͬͯ΋ߴ͍εϧʔϓοτ
Λҡ͍࣋ͯ͠·ͨ͠ɻ
Proxy
destination
Throughput
(req/sec)
Minimum
Response
Time(msec)
Average
Response
Time(msec)
Maximum
Response
Time(msec)
Fast Endpoint 1490.066 52 88 608
Slow Endpoint 80.972 1002 25920 600010
Test Result
• At the fast endpoint, consistently high throughput was maintained even at high
loads.

View Slide

஗͍ΤϯυϙΠϯτͰ͸ෛՙ͕ߴ͘ͳΔʹͭΕλΠϜΞ΢τ͕૿
Ճ͕ͨ͠ɺΤϯυϙΠϯτ͕ఀࢭ͢Δࣄ͸͋Γ·ͤΜͰͨ͠ɻ
Test Result
• At the slow endpoint, timeout increased as load increased.
• Connecting to the endpoint that has been timed out in 60 seconds.
• Slow endpoints were not destroyed even when high load was applied.
Proxy
destination
Throughput
(req/sec)
Minimum
Response
Time(msec)
Average
Response
Time(msec)
Maximum
Response
Time(msec)
Fast Endpoint 1490.066 52 88 608
Slow Endpoint 80.972 1002 25920 600010

View Slide

"LLB)551ͱ"LLB4USFBNTʹΑͬͯɺΤϯυϙΠϯτ΁ͷӨڹ
Λ཈͑ͭͭɺߴ͍ύϑΥʔϚϯεΛಘΔ͜ͱ͕ग़དྷ·ͨ͠ɻ
• Akka HTTP (and Akka Streams) performed the expected behavior of
minimizing influence on the endpoint and the gateway itself using
back-pressure control.
• While performing backpressure control, it was possible to obtain
sufficient throughput even compared to before transplantation.
Summary

View Slide

"LLB)551 )PTU$POOFDUJPO1PPM
ʹΑΔϓϩΩγ͸ɺ
(BUFXBZ ϚΠΫϩαʔϏε
ͷಛੑʹద߹ͯ͠͠·͢ʂ
• Flow is suitable for constructing a static structure.
• The proxy by HostConnectionPool is suitable due to the nature of the
Gateway server.
• Akka Streams and HTTP are useful for microservice architecture!
Summary

View Slide

Any Questions?
࣭ٙԠ౴

View Slide

Implementing DMM API Gateway in Akka Streams and HTTP

Implementing DMM API Gateway in Akka Streams and HTTP

Masateru Nishimura

More Decks by Masateru Nishimura

Other Decks in Programming

Featured

Transcript