Slide 1
Slide 1 text
Armeria
:
LINE
'
s next generation RPC layer
line
.github
.io
/
armeria
이희승
(@
trustin
/イ
·ヒスン
) ·
kojilin
1
Slide 2
Slide 2 text
Agenda
•
What and why
•
Case study
: Sticker
·
theme shop by kojilin
•
Learning more
2
Slide 3
Slide 3 text
What is Armeria
?
• (
A
)
synchronous RPC library
•
Java
8
•
HTTP
/1 · 2
•
Thri
•
with smooth migration in mind
• ... all these becomes
'
why
?'
as well
.
3
Slide 4
Slide 4 text
Why asynchrony
?
•
Synchronous calls do not scale up
.
•
maxConcurrentReq
≤
maxThreads
•
Ine
fficient CPU usage due to contention and context switches
•
Fast
·
Priority requests blocked by others
•
Cascading fluctuation of thread count
•
We need real asynchrony to solve this fundamentally
.
4
Slide 5
Slide 5 text
Is Armeria really asynchronous
?
•
Yes
!
•
maxConcurrentReq
⋙ maxThreads
•
Can serve
·
send multiple reqs simultaneously with only a few threads
.
•
Slow requests do not block others
.
•
Bonus
:
•
Asynchronous DNS lookups
•
Asynchronous connection pool
5
Slide 6
Slide 6 text
Why HTTP
/2?
•
Solves known performance issues with HTTP
/1:
•
Serving multiple requests simultaneously in fewer connections
•
Reduced tra
ffic overhead thanks to binary header compression
•
Retains flexibility and extensibility of HTTP
/1
•
Smooth migration path from HTTP
/1
•
High quality tools
-
cURL
, web browsers
, Wireshark
6
Slide 7
Slide 7 text
Is Armeria
'
s HTTP
/1 · 2 prod
. ready
?
•
Built on Netty
4.1 and its HTTP
/1 · 2 implementation
•
HTTP
/1
•
TechEmpower web framework benchmark all
-
time high
-
ranker
•
Apple
, Twitter
, ...
•
HTTP
/2
•
Google
-
gRPC
•
LINE
-
authentication
, sticker
·
theme shop
, ...
7
Slide 8
Slide 8 text
What is Thri
?
1. Write a
.
t
h
r
i
f
t file that contains service definitions
.
2. Generate your favorite language code using Thri compiler
.
3. Fill the stub
. i
.e
. business logic
n
a
m
e
s
p
a
c
e j
a
v
a c
o
m
.
l
i
n
e
c
o
r
p
.
e
x
a
m
p
l
e
.
t
h
r
i
f
t
s
e
r
v
i
c
e H
e
l
l
o
S
e
r
v
i
c
e {
s
t
r
i
n
g h
e
l
l
o
(
1
:
s
t
r
i
n
g n
a
m
e
)
}
0
1
.
0
2
.
0
3
.
0
4
.
8
Slide 9
Slide 9 text
What is Thri
? (
Cont
'
d
)
c
l
a
s
s M
y
H
e
l
l
o
S
e
r
v
i
c
e i
m
p
l
e
m
e
n
t
s H
e
l
l
o
S
e
r
v
i
c
e
.
A
s
y
n
c
I
f
a
c
e
@
O
v
e
r
r
i
d
e
p
u
b
l
i
c v
o
i
d h
e
l
l
o
(
S
t
r
i
n
g n
a
m
e
, A
s
y
n
c
M
e
t
h
o
d
C
a
l
l
b
a
c
k c
b
) {
c
b
.
o
n
C
o
m
p
l
e
t
e
(
"
H
e
l
l
o
, " + n
a
m
e + '
!
'
)
;
}
}
c
l
i
e
n
t
.
h
e
l
l
o
(
"
w
o
r
l
d
"
, n
e
w A
s
y
n
c
M
e
t
h
o
d
C
a
l
l
b
a
c
k
<
S
t
r
i
n
g
>
(
) {
p
u
b
l
i
c v
o
i
d o
n
C
o
m
p
l
e
t
e
(
S
t
r
i
n
g n
a
m
e
) { .
.
. }
.
.
.
}
)
;
0
1
.
0
2
.
0
3
.
0
4
.
0
5
.
0
6
.
0
7
.
0
8
.
0
9
.
1
0
.
9
Slide 10
Slide 10 text
Wiring up
S
e
r
v
e
r
B
u
i
l
d
e
r b
u
i
l
d
e
r = n
e
w S
e
r
v
e
r
B
u
i
l
d
e
r
(
)
;
b
u
i
l
d
e
r
.
p
o
r
t
(
8
0
8
0
, S
e
s
s
i
o
n
P
r
o
t
o
c
o
l
.
H
T
T
P
)
.
p
o
r
t
(
8
4
3
3
, S
e
s
s
i
o
n
P
r
o
t
o
c
o
l
.
H
T
T
P
S
)
.
s
s
l
C
o
n
t
e
x
t
(
.
.
.
)
;
.
s
e
r
v
i
c
e
A
t
(
"
/
t
h
r
i
f
t
/
h
e
l
l
o
"
,
T
h
r
i
f
t
S
e
r
v
i
c
e
.
o
f
(
n
e
w M
y
H
e
l
l
o
S
e
r
v
i
c
e
(
)
)
)
;
S
e
r
v
e
r s
e
r
v
e
r = b
u
i
l
d
e
r
.
b
u
i
l
d
(
)
;
s
e
r
v
e
r
.
s
t
a
r
t
(
)
;
0
1
.
0
2
.
0
3
.
0
4
.
0
5
.
0
6
.
0
7
.
0
8
.
0
9
.
10
Slide 11
Slide 11 text
Wiring up
(
cont
'
d
)
H
e
l
l
o
S
e
r
v
i
c
e
.
I
f
a
c
e c
l
i
e
n
t = C
l
i
e
n
t
s
.
n
e
w
C
l
i
e
n
t
(
"
t
b
i
n
a
r
y
+
h
t
t
p
:
/
/
1
2
7
.
0
.
0
.
1
:
8
0
8
0
/
t
h
r
i
f
t
/
h
e
l
l
o
"
,
H
e
l
l
o
S
e
r
v
i
c
e
.
I
f
a
c
e
.
c
l
a
s
s
)
;
a
s
s
e
r
t
E
q
u
a
l
s
(
"
H
e
l
l
o
, A
r
m
e
r
i
a
!
"
,
c
l
i
e
n
t
.
h
e
l
l
o
(
"
A
r
m
e
r
i
a
"
)
)
;
0
1
.
0
2
.
0
3
.
0
4
.
0
5
.
0
6
.
11
Slide 12
Slide 12 text
Wait
, but why Thri
?
•
Suitable for complex
& evolving distributed services
:
•
Well
-
defined service definitions
•
Well known schema evolution strategy
•
Interoperability with many languages
•
Faster
·
more compact than text
-
based schemes
•
Many ways to encode
: TBinary
, TCompact
, T
JSON
, TText
, ...
•
We
'
re using it already successfully at LINE
.
12
Slide 13
Slide 13 text
Thri myths
•
Where
'
s my Swagger that produces beautiful docs
?
•
Where
'
s my cURL
·
telnet that sends a test request
?
•
I want to serve static files and webapps on the same port
·
JVM
!
13
Slide 14
Slide 14 text
Automated documentation generator
14
Slide 15
Slide 15 text
Testing from web browser
•
Share the URL to reproduce a Thri call
. 15
Slide 16
Slide 16 text
Wiring up
b
u
i
l
d
e
r
.
s
e
r
v
i
c
e
U
n
d
e
r
(
"
/
d
o
c
s
"
, n
e
w D
o
c
S
e
r
v
i
c
e
(
)
)
;
•
Add a single service and you
'
re done
!
•
Automatically discovers all available Thri services
16
Slide 17
Slide 17 text
But
, how do we migrate legacy apps
?
•
Servlet
/
JSP
-
based REST
/
RPC services
•
Serving static resources
•
Talking to other non
-
Thri services
17
Slide 18
Slide 18 text
Armeria
Ὁ Servlets
!
•
Powered by a real Servlet engine
(
Tomcat
)
•
Takes only the good part from Tomcat
:
•
Tomcat opens
'
zero
'
sockets
.
•
Tomcat gets HTTP
/2 support for free
!
•
Let Armeria serve your RPC calls at its best performance
while serving your rich
·
legacy webapps on the same port
·
JVM
!
18
Slide 19
Slide 19 text
Armeria is protocol
-
agnostic
!
• H
t
t
p
F
i
l
e
S
e
r
v
i
c
e
•
Async HTTP
/1 · 2 client
•
HTTP
/2 even on a cleartext connection
•
No context switches when used in an Armeria service
•
more to come on your demand
!
•
More services
•
More protocols
19
Slide 20
Slide 20 text
Wiring up
b
u
i
l
d
e
r
.
s
e
r
v
i
c
e
U
n
d
e
r
(
"
/
f
i
l
e
s
"
,
H
t
t
p
F
i
l
e
S
e
r
v
i
c
e
.
f
o
r
C
l
a
s
s
P
a
t
h
(
"
/
s
t
a
t
i
c
"
)
)
.
s
e
r
v
i
c
e
U
n
d
e
r
(
"
/
w
e
b
a
p
p
"
,
T
o
m
c
a
t
S
e
r
v
i
c
e
.
f
o
r
F
i
l
e
S
y
s
t
e
m
(
"
m
y
.
w
a
r
"
)
)
;
0
1
.
0
2
.
0
3
.
0
4
.
20
Slide 21
Slide 21 text
Wiring up
(
cont
'
d
)
S
i
m
p
l
e
H
t
t
p
C
l
i
e
n
t c
l
i
e
n
t = C
l
i
e
n
t
s
.
n
e
w
C
l
i
e
n
t
(
"
n
o
n
e
+
h
t
t
p
:
/
/
e
x
a
m
p
l
e
.
c
o
m
/
"
, S
i
m
p
l
e
H
t
t
p
C
l
i
e
n
t
.
c
l
a
s
s
)
;
S
i
m
p
l
e
H
t
t
p
R
e
q
u
e
s
t r
e
q = S
i
m
p
l
e
H
t
t
p
R
e
q
u
e
s
t
B
u
i
l
d
e
r
.
f
o
r
G
e
t
(
"
/
i
n
d
e
x
.
h
t
m
l
"
)
.
h
e
a
d
e
r
(
"
A
u
t
h
o
r
i
z
a
t
i
o
n
"
, "
.
.
.
"
)
.
b
u
i
l
d
(
)
;
F
u
t
u
r
e
<
S
i
m
p
l
e
H
t
t
p
R
e
s
p
o
n
s
e
> f = c
l
i
e
n
t
.
e
x
e
c
u
t
e
(
r
e
q
)
;
S
i
m
p
l
e
H
t
t
p
R
e
s
p
o
n
s
e r
e
s = f
.
g
e
t
(
)
;
0
1
.
0
2
.
0
3
.
0
4
.
0
5
.
0
6
.
0
7
.
0
8
.
0
9
.
21
Slide 22
Slide 22 text
Things we did not achieve
(
yet
)
•
Thri usability
•
Incorporating Thri compiler into build process easily
•
Code generated by Thri compiler isn
'
t beautiful
.
•
Integrated metrics
•
Incorporating metrics into documentation service
•
We have IMON support though
.
•
Redis
, HBase
, Protobuf
, RESTful API
...
22
Slide 23
Slide 23 text
23
Case study
:
Shop
❤ Armeria
Slide 24
Slide 24 text
Overview
24
Slide 25
Slide 25 text
What we will visit
•
SearchFe
(
Search front
-
end
)
•
ShopInternalService
(
known as
'
Shop server
')
•
Shop proxy
• ... and asynchronification
25
Slide 26
Slide 26 text
Why a search API
?
•
Exposing Elasticsearch API directly gives too much control to clients
•
Can
’
t control ranking
•
Exposes potentially dangerous operations without complicated
authentication schemes
•
JDK compatibility
–
Elasticsearch clients should use same JVM as cluster
•
Decouples index concepts
(
localized string fields
)
with query concepts
26
Slide 27
Slide 27 text
Why Armeria
?
•
Need an RPC layer
, may as well use LINE standard
•
Thri is good for performance
, compatibility
, and clearness of API design
•
Can add features to Armeria and apply to all servers that use it
:
•
Documentation service with request debugging
•
Graceful shutdown
•
Health check
27
Slide 28
Slide 28 text
s
t
r
u
c
t S
e
a
r
c
h
R
e
q
u
e
s
t {
/
/ T
h
e u
s
e
r
'
s i
n
p
u
t t
e
x
t q
u
e
r
y
. I
f e
m
p
t
y
, w
i
l
l p
e
r
f
o
r
m n
o t
e
x
t m
a
t
c
h
i
n
g -
/
/ t
h
i
s c
a
n b
e u
s
e
d f
o
r e
x
a
m
p
l
e t
o r
e
t
u
r
n s
h
o
w
c
a
s
e
s
.
1
: o
p
t
i
o
n
a
l s
t
r
i
n
g q
u
e
r
y = "
"
,
/
/ T
h
e u
s
e
r
'
s l
o
c
a
l
e
.
2
: S
h
o
p
.
L
o
c
a
l
e l
o
c
a
l
e = {
"
l
a
n
g
u
a
g
e
"
: "
e
n
"
, "
c
o
u
n
t
r
y
"
: "
U
S
"
}
,
.
.
.
}
s
t
r
u
c
t S
e
a
r
c
h
R
e
s
p
o
n
s
e {
/
/ T
h
e r
e
s
u
l
t
i
n
g p
r
o
d
u
c
t
s f
o
r t
h
e q
u
e
r
y
.
1
: S
h
o
p
.
P
r
o
d
u
c
t
L
i
s
t r
e
s
u
l
t
s
,
.
.
.
}
s
e
r
v
i
c
e S
e
a
r
c
h
F
e {
S
e
a
r
c
h
R
e
s
p
o
n
s
e s
e
a
r
c
h
(
1
: S
e
a
r
c
h
R
e
q
u
e
s
t r
e
q
u
e
s
t
)
} 28
Slide 29
Slide 29 text
@
C
o
m
p
o
n
e
n
t
p
u
b
l
i
c c
l
a
s
s S
e
a
r
c
h
F
e
H
a
n
d
l
e
r i
m
p
l
e
m
e
n
t
s S
e
a
r
c
h
F
e
.
A
s
y
n
c
I
f
a
c
e {
.
.
.
p
r
i
v
a
t
e f
i
n
a
l C
l
i
e
n
t s
e
a
r
c
h
C
l
i
e
n
t
;
.
.
.
@
I
n
j
e
c
t
S
e
a
r
c
h
F
e
H
a
n
d
l
e
r
(
C
l
i
e
n
t s
e
a
r
c
h
C
l
i
e
n
t
, .
.
.
) { .
.
. }
/
/ T
h
r
i
f
t g
e
n
e
r
a
t
e
d c
o
d
e d
o
e
s
n
'
t a
d
d a t
y
p
e p
a
r
a
m
e
t
e
r t
o A
s
y
n
c
M
e
t
h
o
d
C
a
l
l
b
a
c
k
.
.
.
@
S
u
p
p
r
e
s
s
W
a
r
n
i
n
g
s
(
"
r
a
w
t
y
p
e
s
"
)
@
O
v
e
r
r
i
d
e
p
u
b
l
i
c v
o
i
d s
e
a
r
c
h
(
S
e
a
r
c
h
R
e
q
u
e
s
t r
e
q
u
e
s
t
,
A
s
y
n
c
M
e
t
h
o
d
C
a
l
l
b
a
c
k r
e
s
u
l
t
H
a
n
d
l
e
r
) t
h
r
o
w
s T
E
x
c
e
p
t
i
o
n {
.
.
.
}
}
29
Slide 30
Slide 30 text
Shop server
(
or ShopInternalService
)
•
Facebook Ni y
→
Armeria
•
Thri binary protocol
→
HTTP
-
need to use di
fferent port
•
Added additional port with services configured
, and switched clients to
Armeria client with new port
:
•
Shop proxy
•
Talk server
30
Slide 31
Slide 31 text
Just one connection
31
Slide 32
Slide 32 text
Reduced listen drops
32
Slide 33
Slide 33 text
Reduced
'
Can
'
t get a client
'
errors
33
Slide 34
Slide 34 text
Shop proxy
•
Tomcat HTTP Thri
→
Armeria
•
Same protocol
, no client changes
!
•
Started Armeria binary on same machine and
redirected tra
ffic gradually by reconfiguring nginx
34
Slide 35
Slide 35 text
Shop proxy
+
IMON
•
No code needed
(
removed tons of monitoring code
!)
35
Slide 36
Slide 36 text
Client
-
side load balancing
•
Armeria uses a single persistent HTTP
/2 connection
•
Very e
fficient
•
Incompatible with L
4 load balancing
• .. which only balances connection open
•
Implemented simple round
-
robin client
-
side load balancer
•
Will be added to Armeria soon
36
Slide 37
Slide 37 text
Client
-
side load balancing in action
37
Slide 38
Slide 38 text
Shop proxy asynchronification
•
Blocking RPCs in a thread pool
→
fully asynchronous using futures
•
Can use all machine resources as needed
, no more tuning
38
Slide 39
Slide 39 text
Look
, less threads
!
•
Dramatic reduction considering this is O
/
S thread count
.
39
Slide 40
Slide 40 text
Shop server asynchronification
•
In progress
, many APIs ported
•
Dagger producers
•
Asynchronous dependency injection
•
Define methods that return
L
i
s
t
e
n
a
b
l
e
F
u
t
u
r
e
<
T
>,
define methods that take raw
T, and
everything is hooked up automatically by the compiler
.
•
See the Devoxx
2014 slides
.
40
Slide 41
Slide 41 text
Reduced latency
• 3 consecutive blocking calls
→ 3 simultaneous async calls
41
Slide 42
Slide 42 text
Reduced latency
(
cont
'
d
)
•
Multiple nested synchronous MongoDB calls on cache miss
→
Now running asynchronously in parallel
42
Slide 43
Slide 43 text
Reduced CPU usage
43
Slide 44
Slide 44 text
44
Learning more
Slide 45
Slide 45 text
Learning more
•
O
fficial web site
: line
.github
.io
/
armeria
•
Feel free to ask questions and to discuss ideas via issue tracker
.
•
We need you to make it even better
:
•
API usability issues
•
Documentation issues
•
Performance issues
•
Feature requests
45