Armeria: LINE's next generation RPC layer

9b123f408258511b201ca1230f260340?s=47 Trustin Lee
February 16, 2016

Armeria: LINE's next generation RPC layer

The slides for my talk at LINE Developer Meetup in Fukuoka on February 16, 2016

Newer slides with up-to-date API and features:
https://speakerdeck.com/trustin/building-asynchronous-microservices-with-armeria

9b123f408258511b201ca1230f260340?s=128

Trustin Lee

February 16, 2016
Tweet

Transcript

  1. 1.

    Armeria : LINE ' s next generation RPC layer line

    .github .io / armeria 이희승 (@ trustin /イ ·ヒスン ) · kojilin 1
  2. 2.

    Agenda • What and why • Case study : Sticker

    · theme shop by kojilin • Learning more 2
  3. 3.

    What is Armeria ? • ( A ) synchronous RPC

    library • Java 8 • HTTP /1 · 2 • Thri • with smooth migration in mind • ... all these becomes ' why ?' as well . 3
  4. 4.

    Why asynchrony ? • Synchronous calls do not scale up

    . • maxConcurrentReq ≤ maxThreads • Ine fficient CPU usage due to contention and context switches • Fast · Priority requests blocked by others • Cascading fluctuation of thread count • We need real asynchrony to solve this fundamentally . 4
  5. 5.

    Is Armeria really asynchronous ? • Yes ! • maxConcurrentReq

    ⋙ maxThreads • Can serve · send multiple reqs simultaneously with only a few threads . • Slow requests do not block others . • Bonus : • Asynchronous DNS lookups • Asynchronous connection pool 5
  6. 6.

    Why HTTP /2? • Solves known performance issues with HTTP

    /1: • Serving multiple requests simultaneously in fewer connections • Reduced tra ffic overhead thanks to binary header compression • Retains flexibility and extensibility of HTTP /1 • Smooth migration path from HTTP /1 • High quality tools - cURL , web browsers , Wireshark 6
  7. 7.

    Is Armeria ' s HTTP /1 · 2 prod .

    ready ? • Built on Netty 4.1 and its HTTP /1 · 2 implementation • HTTP /1 • TechEmpower web framework benchmark all - time high - ranker • Apple , Twitter , ... • HTTP /2 • Google - gRPC • LINE - authentication , sticker · theme shop , ... 7
  8. 8.

    What is Thri ? 1. Write a . t h

    r i f t file that contains service definitions . 2. Generate your favorite language code using Thri compiler . 3. Fill the stub . i .e . business logic n a m e s p a c e j a v a c o m . l i n e c o r p . e x a m p l e . t h r i f t s e r v i c e H e l l o S e r v i c e { s t r i n g h e l l o ( 1 : s t r i n g n a m e ) } 0 1 . 0 2 . 0 3 . 0 4 . 8
  9. 9.

    What is Thri ? ( Cont ' d ) c

    l a s s M y H e l l o S e r v i c e i m p l e m e n t s H e l l o S e r v i c e . A s y n c I f a c e @ O v e r r i d e p u b l i c v o i d h e l l o ( S t r i n g n a m e , A s y n c M e t h o d C a l l b a c k c b ) { c b . o n C o m p l e t e ( " H e l l o , " + n a m e + ' ! ' ) ; } } c l i e n t . h e l l o ( " w o r l d " , n e w A s y n c M e t h o d C a l l b a c k < S t r i n g > ( ) { p u b l i c v o i d o n C o m p l e t e ( S t r i n g n a m e ) { . . . } . . . } ) ; 0 1 . 0 2 . 0 3 . 0 4 . 0 5 . 0 6 . 0 7 . 0 8 . 0 9 . 1 0 . 9
  10. 10.

    Wiring up S e r v e r B u

    i l d e r b u i l d e r = n e w S e r v e r B u i l d e r ( ) ; b u i l d e r . p o r t ( 8 0 8 0 , S e s s i o n P r o t o c o l . H T T P ) . p o r t ( 8 4 3 3 , S e s s i o n P r o t o c o l . H T T P S ) . s s l C o n t e x t ( . . . ) ; . s e r v i c e A t ( " / t h r i f t / h e l l o " , T h r i f t S e r v i c e . o f ( n e w M y H e l l o S e r v i c e ( ) ) ) ; S e r v e r s e r v e r = b u i l d e r . b u i l d ( ) ; s e r v e r . s t a r t ( ) ; 0 1 . 0 2 . 0 3 . 0 4 . 0 5 . 0 6 . 0 7 . 0 8 . 0 9 . 10
  11. 11.

    Wiring up ( cont ' d ) H e l

    l o S e r v i c e . I f a c e c l i e n t = C l i e n t s . n e w C l i e n t ( " t b i n a r y + h t t p : / / 1 2 7 . 0 . 0 . 1 : 8 0 8 0 / t h r i f t / h e l l o " , H e l l o S e r v i c e . I f a c e . c l a s s ) ; a s s e r t E q u a l s ( " H e l l o , A r m e r i a ! " , c l i e n t . h e l l o ( " A r m e r i a " ) ) ; 0 1 . 0 2 . 0 3 . 0 4 . 0 5 . 0 6 . 11
  12. 12.

    Wait , but why Thri ? • Suitable for complex

    & evolving distributed services : • Well - defined service definitions • Well known schema evolution strategy • Interoperability with many languages • Faster · more compact than text - based schemes • Many ways to encode : TBinary , TCompact , T JSON , TText , ... • We ' re using it already successfully at LINE . 12
  13. 13.

    Thri myths • Where ' s my Swagger that produces

    beautiful docs ? • Where ' s my cURL · telnet that sends a test request ? • I want to serve static files and webapps on the same port · JVM ! 13
  14. 16.

    Wiring up b u i l d e r .

    s e r v i c e U n d e r ( " / d o c s " , n e w D o c S e r v i c e ( ) ) ; • Add a single service and you ' re done ! • Automatically discovers all available Thri services 16
  15. 17.

    But , how do we migrate legacy apps ? •

    Servlet / JSP - based REST / RPC services • Serving static resources • Talking to other non - Thri services 17
  16. 18.

    Armeria Ὁ Servlets ! • Powered by a real Servlet

    engine ( Tomcat ) • Takes only the good part from Tomcat : • Tomcat opens ' zero ' sockets . • Tomcat gets HTTP /2 support for free ! • Let Armeria serve your RPC calls at its best performance while serving your rich · legacy webapps on the same port · JVM ! 18
  17. 19.

    Armeria is protocol - agnostic ! • H t t

    p F i l e S e r v i c e • Async HTTP /1 · 2 client • HTTP /2 even on a cleartext connection • No context switches when used in an Armeria service • more to come on your demand ! • More services • More protocols 19
  18. 20.

    Wiring up b u i l d e r .

    s e r v i c e U n d e r ( " / f i l e s " , H t t p F i l e S e r v i c e . f o r C l a s s P a t h ( " / s t a t i c " ) ) . s e r v i c e U n d e r ( " / w e b a p p " , T o m c a t S e r v i c e . f o r F i l e S y s t e m ( " m y . w a r " ) ) ; 0 1 . 0 2 . 0 3 . 0 4 . 20
  19. 21.

    Wiring up ( cont ' d ) S i m

    p l e H t t p C l i e n t c l i e n t = C l i e n t s . n e w C l i e n t ( " n o n e + h t t p : / / e x a m p l e . c o m / " , S i m p l e H t t p C l i e n t . c l a s s ) ; S i m p l e H t t p R e q u e s t r e q = S i m p l e H t t p R e q u e s t B u i l d e r . f o r G e t ( " / i n d e x . h t m l " ) . h e a d e r ( " A u t h o r i z a t i o n " , " . . . " ) . b u i l d ( ) ; F u t u r e < S i m p l e H t t p R e s p o n s e > f = c l i e n t . e x e c u t e ( r e q ) ; S i m p l e H t t p R e s p o n s e r e s = f . g e t ( ) ; 0 1 . 0 2 . 0 3 . 0 4 . 0 5 . 0 6 . 0 7 . 0 8 . 0 9 . 21
  20. 22.

    Things we did not achieve ( yet ) • Thri

    usability • Incorporating Thri compiler into build process easily • Code generated by Thri compiler isn ' t beautiful . • Integrated metrics • Incorporating metrics into documentation service • We have IMON support though . • Redis , HBase , Protobuf , RESTful API ... 22
  21. 25.

    What we will visit • SearchFe ( Search front -

    end ) • ShopInternalService ( known as ' Shop server ') • Shop proxy • ... and asynchronification 25
  22. 26.

    Why a search API ? • Exposing Elasticsearch API directly

    gives too much control to clients • Can ’ t control ranking • Exposes potentially dangerous operations without complicated authentication schemes • JDK compatibility – Elasticsearch clients should use same JVM as cluster • Decouples index concepts ( localized string fields ) with query concepts 26
  23. 27.

    Why Armeria ? • Need an RPC layer , may

    as well use LINE standard • Thri is good for performance , compatibility , and clearness of API design • Can add features to Armeria and apply to all servers that use it : • Documentation service with request debugging • Graceful shutdown • Health check 27
  24. 28.

    s t r u c t S e a r

    c h R e q u e s t { / / T h e u s e r ' s i n p u t t e x t q u e r y . I f e m p t y , w i l l p e r f o r m n o t e x t m a t c h i n g - / / t h i s c a n b e u s e d f o r e x a m p l e t o r e t u r n s h o w c a s e s . 1 : o p t i o n a l s t r i n g q u e r y = " " , / / T h e u s e r ' s l o c a l e . 2 : S h o p . L o c a l e l o c a l e = { " l a n g u a g e " : " e n " , " c o u n t r y " : " U S " } , . . . } s t r u c t S e a r c h R e s p o n s e { / / T h e r e s u l t i n g p r o d u c t s f o r t h e q u e r y . 1 : S h o p . P r o d u c t L i s t r e s u l t s , . . . } s e r v i c e S e a r c h F e { S e a r c h R e s p o n s e s e a r c h ( 1 : S e a r c h R e q u e s t r e q u e s t ) } 28
  25. 29.

    @ C o m p o n e n t

    p u b l i c c l a s s S e a r c h F e H a n d l e r i m p l e m e n t s S e a r c h F e . A s y n c I f a c e { . . . p r i v a t e f i n a l C l i e n t s e a r c h C l i e n t ; . . . @ I n j e c t S e a r c h F e H a n d l e r ( C l i e n t s e a r c h C l i e n t , . . . ) { . . . } / / T h r i f t g e n e r a t e d c o d e d o e s n ' t a d d a t y p e p a r a m e t e r t o A s y n c M e t h o d C a l l b a c k . . . @ S u p p r e s s W a r n i n g s ( " r a w t y p e s " ) @ O v e r r i d e p u b l i c v o i d s e a r c h ( S e a r c h R e q u e s t r e q u e s t , A s y n c M e t h o d C a l l b a c k r e s u l t H a n d l e r ) t h r o w s T E x c e p t i o n { . . . } } 29
  26. 30.

    Shop server ( or ShopInternalService ) • Facebook Ni y

    → Armeria • Thri binary protocol → HTTP - need to use di fferent port • Added additional port with services configured , and switched clients to Armeria client with new port : • Shop proxy • Talk server 30
  27. 34.

    Shop proxy • Tomcat HTTP Thri → Armeria • Same

    protocol , no client changes ! • Started Armeria binary on same machine and redirected tra ffic gradually by reconfiguring nginx 34
  28. 35.

    Shop proxy + IMON • No code needed ( removed

    tons of monitoring code !) 35
  29. 36.

    Client - side load balancing • Armeria uses a single

    persistent HTTP /2 connection • Very e fficient • Incompatible with L 4 load balancing • .. which only balances connection open • Implemented simple round - robin client - side load balancer • Will be added to Armeria soon 36
  30. 38.

    Shop proxy asynchronification • Blocking RPCs in a thread pool

    → fully asynchronous using futures • Can use all machine resources as needed , no more tuning 38
  31. 40.

    Shop server asynchronification • In progress , many APIs ported

    • Dagger producers • Asynchronous dependency injection • Define methods that return L i s t e n a b l e F u t u r e < T >, define methods that take raw T, and everything is hooked up automatically by the compiler . • See the Devoxx 2014 slides . 40
  32. 42.

    Reduced latency ( cont ' d ) • Multiple nested

    synchronous MongoDB calls on cache miss → Now running asynchronously in parallel 42
  33. 45.

    Learning more • O fficial web site : line .github

    .io / armeria • Feel free to ask questions and to discuss ideas via issue tracker . • We need you to make it even better : • API usability issues • Documentation issues • Performance issues • Feature requests 45