Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Armeria: LINE's next generation RPC layer

Trustin Lee
February 16, 2016

Armeria: LINE's next generation RPC layer

The slides for my talk at LINE Developer Meetup in Fukuoka on February 16, 2016

Newer slides with up-to-date API and features:
https://speakerdeck.com/trustin/building-asynchronous-microservices-with-armeria

Trustin Lee

February 16, 2016
Tweet

More Decks by Trustin Lee

Other Decks in Programming

Transcript

  1. Armeria
    :
    LINE
    '
    s next generation RPC layer
    line
    .github
    .io
    /
    armeria
    이희승
    (@
    trustin
    /イ
    ·ヒスン
    ) ·
    kojilin
    1

    View Slide

  2. Agenda

    What and why

    Case study
    : Sticker
    ·
    theme shop by kojilin

    Learning more
    2

    View Slide

  3. What is Armeria
    ?
    • (
    A
    )
    synchronous RPC library

    Java
    8

    HTTP
    /1 · 2

    Thri

    with smooth migration in mind
    • ... all these becomes
    '
    why
    ?'
    as well
    .
    3

    View Slide

  4. Why asynchrony
    ?

    Synchronous calls do not scale up
    .

    maxConcurrentReq

    maxThreads

    Ine
    fficient CPU usage due to contention and context switches

    Fast
    ·
    Priority requests blocked by others

    Cascading fluctuation of thread count

    We need real asynchrony to solve this fundamentally
    .
    4

    View Slide

  5. Is Armeria really asynchronous
    ?

    Yes
    !

    maxConcurrentReq
    ⋙ maxThreads

    Can serve
    ·
    send multiple reqs simultaneously with only a few threads
    .

    Slow requests do not block others
    .

    Bonus
    :

    Asynchronous DNS lookups

    Asynchronous connection pool
    5

    View Slide

  6. Why HTTP
    /2?

    Solves known performance issues with HTTP
    /1:

    Serving multiple requests simultaneously in fewer connections

    Reduced tra
    ffic overhead thanks to binary header compression

    Retains flexibility and extensibility of HTTP
    /1

    Smooth migration path from HTTP
    /1

    High quality tools
    -
    cURL
    , web browsers
    , Wireshark
    6

    View Slide

  7. Is Armeria
    '
    s HTTP
    /1 · 2 prod
    . ready
    ?

    Built on Netty
    4.1 and its HTTP
    /1 · 2 implementation

    HTTP
    /1

    TechEmpower web framework benchmark all
    -
    time high
    -
    ranker

    Apple
    , Twitter
    , ...

    HTTP
    /2

    Google
    -
    gRPC

    LINE
    -
    authentication
    , sticker
    ·
    theme shop
    , ...
    7

    View Slide

  8. What is Thri
    ?
    1. Write a
    .
    t
    h
    r
    i
    f
    t file that contains service definitions
    .
    2. Generate your favorite language code using Thri compiler
    .
    3. Fill the stub
    . i
    .e
    . business logic
    n
    a
    m
    e
    s
    p
    a
    c
    e j
    a
    v
    a c
    o
    m
    .
    l
    i
    n
    e
    c
    o
    r
    p
    .
    e
    x
    a
    m
    p
    l
    e
    .
    t
    h
    r
    i
    f
    t
    s
    e
    r
    v
    i
    c
    e H
    e
    l
    l
    o
    S
    e
    r
    v
    i
    c
    e {
    s
    t
    r
    i
    n
    g h
    e
    l
    l
    o
    (
    1
    :
    s
    t
    r
    i
    n
    g n
    a
    m
    e
    )
    }
    0
    1
    .
    0
    2
    .
    0
    3
    .
    0
    4
    .
    8

    View Slide

  9. What is Thri
    ? (
    Cont
    '
    d
    )
    c
    l
    a
    s
    s M
    y
    H
    e
    l
    l
    o
    S
    e
    r
    v
    i
    c
    e i
    m
    p
    l
    e
    m
    e
    n
    t
    s H
    e
    l
    l
    o
    S
    e
    r
    v
    i
    c
    e
    .
    A
    s
    y
    n
    c
    I
    f
    a
    c
    e
    @
    O
    v
    e
    r
    r
    i
    d
    e
    p
    u
    b
    l
    i
    c v
    o
    i
    d h
    e
    l
    l
    o
    (
    S
    t
    r
    i
    n
    g n
    a
    m
    e
    , A
    s
    y
    n
    c
    M
    e
    t
    h
    o
    d
    C
    a
    l
    l
    b
    a
    c
    k c
    b
    ) {
    c
    b
    .
    o
    n
    C
    o
    m
    p
    l
    e
    t
    e
    (
    "
    H
    e
    l
    l
    o
    , " + n
    a
    m
    e + '
    !
    '
    )
    ;
    }
    }
    c
    l
    i
    e
    n
    t
    .
    h
    e
    l
    l
    o
    (
    "
    w
    o
    r
    l
    d
    "
    , n
    e
    w A
    s
    y
    n
    c
    M
    e
    t
    h
    o
    d
    C
    a
    l
    l
    b
    a
    c
    k
    <
    S
    t
    r
    i
    n
    g
    >
    (
    ) {
    p
    u
    b
    l
    i
    c v
    o
    i
    d o
    n
    C
    o
    m
    p
    l
    e
    t
    e
    (
    S
    t
    r
    i
    n
    g n
    a
    m
    e
    ) { .
    .
    . }
    .
    .
    .
    }
    )
    ;
    0
    1
    .
    0
    2
    .
    0
    3
    .
    0
    4
    .
    0
    5
    .
    0
    6
    .
    0
    7
    .
    0
    8
    .
    0
    9
    .
    1
    0
    .
    9

    View Slide

  10. Wiring up
    S
    e
    r
    v
    e
    r
    B
    u
    i
    l
    d
    e
    r b
    u
    i
    l
    d
    e
    r = n
    e
    w S
    e
    r
    v
    e
    r
    B
    u
    i
    l
    d
    e
    r
    (
    )
    ;
    b
    u
    i
    l
    d
    e
    r
    .
    p
    o
    r
    t
    (
    8
    0
    8
    0
    , S
    e
    s
    s
    i
    o
    n
    P
    r
    o
    t
    o
    c
    o
    l
    .
    H
    T
    T
    P
    )
    .
    p
    o
    r
    t
    (
    8
    4
    3
    3
    , S
    e
    s
    s
    i
    o
    n
    P
    r
    o
    t
    o
    c
    o
    l
    .
    H
    T
    T
    P
    S
    )
    .
    s
    s
    l
    C
    o
    n
    t
    e
    x
    t
    (
    .
    .
    .
    )
    ;
    .
    s
    e
    r
    v
    i
    c
    e
    A
    t
    (
    "
    /
    t
    h
    r
    i
    f
    t
    /
    h
    e
    l
    l
    o
    "
    ,
    T
    h
    r
    i
    f
    t
    S
    e
    r
    v
    i
    c
    e
    .
    o
    f
    (
    n
    e
    w M
    y
    H
    e
    l
    l
    o
    S
    e
    r
    v
    i
    c
    e
    (
    )
    )
    )
    ;
    S
    e
    r
    v
    e
    r s
    e
    r
    v
    e
    r = b
    u
    i
    l
    d
    e
    r
    .
    b
    u
    i
    l
    d
    (
    )
    ;
    s
    e
    r
    v
    e
    r
    .
    s
    t
    a
    r
    t
    (
    )
    ;
    0
    1
    .
    0
    2
    .
    0
    3
    .
    0
    4
    .
    0
    5
    .
    0
    6
    .
    0
    7
    .
    0
    8
    .
    0
    9
    .
    10

    View Slide

  11. Wiring up
    (
    cont
    '
    d
    )
    H
    e
    l
    l
    o
    S
    e
    r
    v
    i
    c
    e
    .
    I
    f
    a
    c
    e c
    l
    i
    e
    n
    t = C
    l
    i
    e
    n
    t
    s
    .
    n
    e
    w
    C
    l
    i
    e
    n
    t
    (
    "
    t
    b
    i
    n
    a
    r
    y
    +
    h
    t
    t
    p
    :
    /
    /
    1
    2
    7
    .
    0
    .
    0
    .
    1
    :
    8
    0
    8
    0
    /
    t
    h
    r
    i
    f
    t
    /
    h
    e
    l
    l
    o
    "
    ,
    H
    e
    l
    l
    o
    S
    e
    r
    v
    i
    c
    e
    .
    I
    f
    a
    c
    e
    .
    c
    l
    a
    s
    s
    )
    ;
    a
    s
    s
    e
    r
    t
    E
    q
    u
    a
    l
    s
    (
    "
    H
    e
    l
    l
    o
    , A
    r
    m
    e
    r
    i
    a
    !
    "
    ,
    c
    l
    i
    e
    n
    t
    .
    h
    e
    l
    l
    o
    (
    "
    A
    r
    m
    e
    r
    i
    a
    "
    )
    )
    ;
    0
    1
    .
    0
    2
    .
    0
    3
    .
    0
    4
    .
    0
    5
    .
    0
    6
    .
    11

    View Slide

  12. Wait
    , but why Thri
    ?

    Suitable for complex
    & evolving distributed services
    :

    Well
    -
    defined service definitions

    Well known schema evolution strategy

    Interoperability with many languages

    Faster
    ·
    more compact than text
    -
    based schemes

    Many ways to encode
    : TBinary
    , TCompact
    , T
    JSON
    , TText
    , ...

    We
    '
    re using it already successfully at LINE
    .
    12

    View Slide

  13. Thri myths

    Where
    '
    s my Swagger that produces beautiful docs
    ?

    Where
    '
    s my cURL
    ·
    telnet that sends a test request
    ?

    I want to serve static files and webapps on the same port
    ·
    JVM
    !
    13

    View Slide

  14. Automated documentation generator
    14

    View Slide

  15. Testing from web browser

    Share the URL to reproduce a Thri call
    . 15

    View Slide

  16. Wiring up
    b
    u
    i
    l
    d
    e
    r
    .
    s
    e
    r
    v
    i
    c
    e
    U
    n
    d
    e
    r
    (
    "
    /
    d
    o
    c
    s
    "
    , n
    e
    w D
    o
    c
    S
    e
    r
    v
    i
    c
    e
    (
    )
    )
    ;

    Add a single service and you
    '
    re done
    !

    Automatically discovers all available Thri services
    16

    View Slide

  17. But
    , how do we migrate legacy apps
    ?

    Servlet
    /
    JSP
    -
    based REST
    /
    RPC services

    Serving static resources

    Talking to other non
    -
    Thri services
    17

    View Slide

  18. Armeria
    Ὁ Servlets
    !

    Powered by a real Servlet engine
    (
    Tomcat
    )

    Takes only the good part from Tomcat
    :

    Tomcat opens
    '
    zero
    '
    sockets
    .

    Tomcat gets HTTP
    /2 support for free
    !

    Let Armeria serve your RPC calls at its best performance
    while serving your rich
    ·
    legacy webapps on the same port
    ·
    JVM
    !
    18

    View Slide

  19. Armeria is protocol
    -
    agnostic
    !
    • H
    t
    t
    p
    F
    i
    l
    e
    S
    e
    r
    v
    i
    c
    e

    Async HTTP
    /1 · 2 client

    HTTP
    /2 even on a cleartext connection

    No context switches when used in an Armeria service

    more to come on your demand
    !

    More services

    More protocols
    19

    View Slide

  20. Wiring up
    b
    u
    i
    l
    d
    e
    r
    .
    s
    e
    r
    v
    i
    c
    e
    U
    n
    d
    e
    r
    (
    "
    /
    f
    i
    l
    e
    s
    "
    ,
    H
    t
    t
    p
    F
    i
    l
    e
    S
    e
    r
    v
    i
    c
    e
    .
    f
    o
    r
    C
    l
    a
    s
    s
    P
    a
    t
    h
    (
    "
    /
    s
    t
    a
    t
    i
    c
    "
    )
    )
    .
    s
    e
    r
    v
    i
    c
    e
    U
    n
    d
    e
    r
    (
    "
    /
    w
    e
    b
    a
    p
    p
    "
    ,
    T
    o
    m
    c
    a
    t
    S
    e
    r
    v
    i
    c
    e
    .
    f
    o
    r
    F
    i
    l
    e
    S
    y
    s
    t
    e
    m
    (
    "
    m
    y
    .
    w
    a
    r
    "
    )
    )
    ;
    0
    1
    .
    0
    2
    .
    0
    3
    .
    0
    4
    .
    20

    View Slide

  21. Wiring up
    (
    cont
    '
    d
    )
    S
    i
    m
    p
    l
    e
    H
    t
    t
    p
    C
    l
    i
    e
    n
    t c
    l
    i
    e
    n
    t = C
    l
    i
    e
    n
    t
    s
    .
    n
    e
    w
    C
    l
    i
    e
    n
    t
    (
    "
    n
    o
    n
    e
    +
    h
    t
    t
    p
    :
    /
    /
    e
    x
    a
    m
    p
    l
    e
    .
    c
    o
    m
    /
    "
    , S
    i
    m
    p
    l
    e
    H
    t
    t
    p
    C
    l
    i
    e
    n
    t
    .
    c
    l
    a
    s
    s
    )
    ;
    S
    i
    m
    p
    l
    e
    H
    t
    t
    p
    R
    e
    q
    u
    e
    s
    t r
    e
    q = S
    i
    m
    p
    l
    e
    H
    t
    t
    p
    R
    e
    q
    u
    e
    s
    t
    B
    u
    i
    l
    d
    e
    r
    .
    f
    o
    r
    G
    e
    t
    (
    "
    /
    i
    n
    d
    e
    x
    .
    h
    t
    m
    l
    "
    )
    .
    h
    e
    a
    d
    e
    r
    (
    "
    A
    u
    t
    h
    o
    r
    i
    z
    a
    t
    i
    o
    n
    "
    , "
    .
    .
    .
    "
    )
    .
    b
    u
    i
    l
    d
    (
    )
    ;
    F
    u
    t
    u
    r
    e
    <
    S
    i
    m
    p
    l
    e
    H
    t
    t
    p
    R
    e
    s
    p
    o
    n
    s
    e
    > f = c
    l
    i
    e
    n
    t
    .
    e
    x
    e
    c
    u
    t
    e
    (
    r
    e
    q
    )
    ;
    S
    i
    m
    p
    l
    e
    H
    t
    t
    p
    R
    e
    s
    p
    o
    n
    s
    e r
    e
    s = f
    .
    g
    e
    t
    (
    )
    ;
    0
    1
    .
    0
    2
    .
    0
    3
    .
    0
    4
    .
    0
    5
    .
    0
    6
    .
    0
    7
    .
    0
    8
    .
    0
    9
    .
    21

    View Slide

  22. Things we did not achieve
    (
    yet
    )

    Thri usability

    Incorporating Thri compiler into build process easily

    Code generated by Thri compiler isn
    '
    t beautiful
    .

    Integrated metrics

    Incorporating metrics into documentation service

    We have IMON support though
    .

    Redis
    , HBase
    , Protobuf
    , RESTful API
    ...
    22

    View Slide

  23. 23
    Case study
    :
    Shop
    ❤ Armeria

    View Slide

  24. Overview
    24

    View Slide

  25. What we will visit

    SearchFe
    (
    Search front
    -
    end
    )

    ShopInternalService
    (
    known as
    '
    Shop server
    ')

    Shop proxy
    • ... and asynchronification
    25

    View Slide

  26. Why a search API
    ?

    Exposing Elasticsearch API directly gives too much control to clients

    Can

    t control ranking

    Exposes potentially dangerous operations without complicated
    authentication schemes

    JDK compatibility

    Elasticsearch clients should use same JVM as cluster

    Decouples index concepts
    (
    localized string fields
    )
    with query concepts
    26

    View Slide

  27. Why Armeria
    ?

    Need an RPC layer
    , may as well use LINE standard

    Thri is good for performance
    , compatibility
    , and clearness of API design

    Can add features to Armeria and apply to all servers that use it
    :

    Documentation service with request debugging

    Graceful shutdown

    Health check
    27

    View Slide

  28. s
    t
    r
    u
    c
    t S
    e
    a
    r
    c
    h
    R
    e
    q
    u
    e
    s
    t {
    /
    / T
    h
    e u
    s
    e
    r
    '
    s i
    n
    p
    u
    t t
    e
    x
    t q
    u
    e
    r
    y
    . I
    f e
    m
    p
    t
    y
    , w
    i
    l
    l p
    e
    r
    f
    o
    r
    m n
    o t
    e
    x
    t m
    a
    t
    c
    h
    i
    n
    g -
    /
    / t
    h
    i
    s c
    a
    n b
    e u
    s
    e
    d f
    o
    r e
    x
    a
    m
    p
    l
    e t
    o r
    e
    t
    u
    r
    n s
    h
    o
    w
    c
    a
    s
    e
    s
    .
    1
    : o
    p
    t
    i
    o
    n
    a
    l s
    t
    r
    i
    n
    g q
    u
    e
    r
    y = "
    "
    ,
    /
    / T
    h
    e u
    s
    e
    r
    '
    s l
    o
    c
    a
    l
    e
    .
    2
    : S
    h
    o
    p
    .
    L
    o
    c
    a
    l
    e l
    o
    c
    a
    l
    e = {
    "
    l
    a
    n
    g
    u
    a
    g
    e
    "
    : "
    e
    n
    "
    , "
    c
    o
    u
    n
    t
    r
    y
    "
    : "
    U
    S
    "
    }
    ,
    .
    .
    .
    }
    s
    t
    r
    u
    c
    t S
    e
    a
    r
    c
    h
    R
    e
    s
    p
    o
    n
    s
    e {
    /
    / T
    h
    e r
    e
    s
    u
    l
    t
    i
    n
    g p
    r
    o
    d
    u
    c
    t
    s f
    o
    r t
    h
    e q
    u
    e
    r
    y
    .
    1
    : S
    h
    o
    p
    .
    P
    r
    o
    d
    u
    c
    t
    L
    i
    s
    t r
    e
    s
    u
    l
    t
    s
    ,
    .
    .
    .
    }
    s
    e
    r
    v
    i
    c
    e S
    e
    a
    r
    c
    h
    F
    e {
    S
    e
    a
    r
    c
    h
    R
    e
    s
    p
    o
    n
    s
    e s
    e
    a
    r
    c
    h
    (
    1
    : S
    e
    a
    r
    c
    h
    R
    e
    q
    u
    e
    s
    t r
    e
    q
    u
    e
    s
    t
    )
    } 28

    View Slide

  29. @
    C
    o
    m
    p
    o
    n
    e
    n
    t
    p
    u
    b
    l
    i
    c c
    l
    a
    s
    s S
    e
    a
    r
    c
    h
    F
    e
    H
    a
    n
    d
    l
    e
    r i
    m
    p
    l
    e
    m
    e
    n
    t
    s S
    e
    a
    r
    c
    h
    F
    e
    .
    A
    s
    y
    n
    c
    I
    f
    a
    c
    e {
    .
    .
    .
    p
    r
    i
    v
    a
    t
    e f
    i
    n
    a
    l C
    l
    i
    e
    n
    t s
    e
    a
    r
    c
    h
    C
    l
    i
    e
    n
    t
    ;
    .
    .
    .
    @
    I
    n
    j
    e
    c
    t
    S
    e
    a
    r
    c
    h
    F
    e
    H
    a
    n
    d
    l
    e
    r
    (
    C
    l
    i
    e
    n
    t s
    e
    a
    r
    c
    h
    C
    l
    i
    e
    n
    t
    , .
    .
    .
    ) { .
    .
    . }
    /
    / T
    h
    r
    i
    f
    t g
    e
    n
    e
    r
    a
    t
    e
    d c
    o
    d
    e d
    o
    e
    s
    n
    '
    t a
    d
    d a t
    y
    p
    e p
    a
    r
    a
    m
    e
    t
    e
    r t
    o A
    s
    y
    n
    c
    M
    e
    t
    h
    o
    d
    C
    a
    l
    l
    b
    a
    c
    k
    .
    .
    .
    @
    S
    u
    p
    p
    r
    e
    s
    s
    W
    a
    r
    n
    i
    n
    g
    s
    (
    "
    r
    a
    w
    t
    y
    p
    e
    s
    "
    )
    @
    O
    v
    e
    r
    r
    i
    d
    e
    p
    u
    b
    l
    i
    c v
    o
    i
    d s
    e
    a
    r
    c
    h
    (
    S
    e
    a
    r
    c
    h
    R
    e
    q
    u
    e
    s
    t r
    e
    q
    u
    e
    s
    t
    ,
    A
    s
    y
    n
    c
    M
    e
    t
    h
    o
    d
    C
    a
    l
    l
    b
    a
    c
    k r
    e
    s
    u
    l
    t
    H
    a
    n
    d
    l
    e
    r
    ) t
    h
    r
    o
    w
    s T
    E
    x
    c
    e
    p
    t
    i
    o
    n {
    .
    .
    .
    }
    }
    29

    View Slide

  30. Shop server
    (
    or ShopInternalService
    )

    Facebook Ni y

    Armeria

    Thri binary protocol

    HTTP
    -
    need to use di
    fferent port

    Added additional port with services configured
    , and switched clients to
    Armeria client with new port
    :

    Shop proxy

    Talk server
    30

    View Slide

  31. Just one connection
    31

    View Slide

  32. Reduced listen drops
    32

    View Slide

  33. Reduced
    '
    Can
    '
    t get a client
    '
    errors
    33

    View Slide

  34. Shop proxy

    Tomcat HTTP Thri

    Armeria

    Same protocol
    , no client changes
    !

    Started Armeria binary on same machine and
    redirected tra
    ffic gradually by reconfiguring nginx
    34

    View Slide

  35. Shop proxy
    +
    IMON

    No code needed
    (
    removed tons of monitoring code
    !)
    35

    View Slide

  36. Client
    -
    side load balancing

    Armeria uses a single persistent HTTP
    /2 connection

    Very e
    fficient

    Incompatible with L
    4 load balancing
    • .. which only balances connection open

    Implemented simple round
    -
    robin client
    -
    side load balancer

    Will be added to Armeria soon
    36

    View Slide

  37. Client
    -
    side load balancing in action
    37

    View Slide

  38. Shop proxy asynchronification

    Blocking RPCs in a thread pool

    fully asynchronous using futures

    Can use all machine resources as needed
    , no more tuning
    38

    View Slide

  39. Look
    , less threads
    !

    Dramatic reduction considering this is O
    /
    S thread count
    .
    39

    View Slide

  40. Shop server asynchronification

    In progress
    , many APIs ported

    Dagger producers

    Asynchronous dependency injection

    Define methods that return
    L
    i
    s
    t
    e
    n
    a
    b
    l
    e
    F
    u
    t
    u
    r
    e
    <
    T
    >,
    define methods that take raw
    T, and
    everything is hooked up automatically by the compiler
    .

    See the Devoxx
    2014 slides
    .
    40

    View Slide

  41. Reduced latency
    • 3 consecutive blocking calls
    → 3 simultaneous async calls
    41

    View Slide

  42. Reduced latency
    (
    cont
    '
    d
    )

    Multiple nested synchronous MongoDB calls on cache miss

    Now running asynchronously in parallel
    42

    View Slide

  43. Reduced CPU usage
    43

    View Slide

  44. 44
    Learning more

    View Slide

  45. Learning more

    O
    fficial web site
    : line
    .github
    .io
    /
    armeria

    Feel free to ask questions and to discuss ideas via issue tracker
    .

    We need you to make it even better
    :

    API usability issues

    Documentation issues

    Performance issues

    Feature requests
    45

    View Slide