Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building Asynchronous Microservices with Armeria

Building Asynchronous Microservices with Armeria

Let us introduce Armeria, LINE's open-source reactive asynchronous microservice framework based on Java, Netty, HTTP/2, gRPC and Thrift.
We will have a quick tour of Armeria's easy-to-learn yet powerful features such as distributed tracing and service discovery, as well as learning why asynchronous and reactive microservice framework is important for services at any scale.

Previously presented at:

- TWJUG Meetup in Taipei on June 30, 2018
- LINE Developer Meetup in Tokyo on February 27, 2018

Previous slides that include case study:

Trustin Lee

June 30, 2018

More Decks by Trustin Lee

Other Decks in Programming


  1. LINE Corporation
    June 2018
    Building Asynchronous
    Microservices with Armeria

    View Slide

  2. Why build another
    microservice framework?

    View Slide

  3. Asynchrony

    Most web frameworks are blocking and synchronous.
    – 1000 concurrent requests require 1000 threads.
    – In modern systems, a service sends requests to another backend.
    – Threads are busy doing nothing but just waiting for the backend.
    – Request queue is not consumed fast enough.
    – Result:

    Fragile system that falls apart even on a tiny backend failure

    Inefficient system that takes more memory and CPU than what it actually does

    View Slide

  4. Reactive Streams

    Should we really buffer everything until a response is sent?
    – Retrieving 100,000 rows from a table

    Why should a client send many requests to reduce the server memory
    – Retrieving 1,000 rows 100 times

    What if we want to stream infinitely?
    – Stock quotes

    Our protocol is not a problem here.
    – Our ‘blocking’ programming model is blocking ‘us’.

    View Slide

  5. Better RPC support

    RPC has been hardly a 1st-class citizen in web
    – Distinction from RPC-level and HTTP-level failure
    – Access logs with RPC-level details
    – Debugging console

    What if …
    – We build a microservice framework with RPC at its core?

    View Slide

  6. More reasons back in 2015

    Leverage Java 8 syntax
    – Lambda expressions
    – Default interface methods
    – Static interface methods

    Provide ‘working’ HTTP/2 implementation
    – ‘H2C’ support (HTTP/2 over ‘C’leartext)
    – OpenSSL-based faster TLS connection

    View Slide

  7. Better UX for devs

    This is the last yet most important part.

    Nobody wants a framework that’s hard to use.

    Can we design …
    – a user-friendly asynchronous programming API
    – … with modest learning curve?

    View Slide

  8. Our answer – Armeria

    – Both ciphertext and cleartext, a.k.a. h2 and h2c

    – Implements Reactive Streams API

    Ability to mix different types of services in a single server
    – gRPC, Thrift, REST services
    – Legacy webapps via embedded Tomcat or Jetty

    – Netty with /dev/epoll transport and BoringSSL SSLEngine

    Easy to learn and fun to use (hopefully)

    View Slide

  9. Armeria by examples

    View Slide

  10. Hello, world!
    Server server = new ServerBuilder()
    .service("/", (ctx, req) -> HttpResponse.of("Hello, world!"))

    View Slide

  11. Hello, world – Parameterized
    Server server = new ServerBuilder()
    (ctx, req) -> HttpResponse.of("Hello, %s!",

    View Slide

  12. Hello, world – Annotated
    Server server = new ServerBuilder()
    .annotatedService(new Object() {
    public String hello(@Param String name) {
    return String.format("Hello, %s!", name);

    View Slide

  13. With CompletableFuture
    Supplier> random = ...; // Takes long!
    Object randomService = new Object() {
    public CompletableFuture nextInt() {
    // Request a random number.
    CompletableFuture f1 = random.get();
    // Convert the random number into a response.
    CompletableFuture f2 =
    f1.thenApply(num -> HttpResponse.of("%d\n", num))
    .exceptionally(t -> HttpResponse.of(500));
    return f2;
    new ServerBuilder()

    View Slide

  14. Simplification with method chaining
    // Annotated version:
    public CompletableFuture nextIntSimple() {
    return random.get()
    .thenApply(num -> HttpResponse.of("%d\n", num))
    .exceptionally(t -> HttpResponse.of(500));
    // Non-annotated version:
    HttpService randomService = (ctx, req) -> HttpResponse.from(
    .thenApply(num -> HttpResponse.of("%d\n", num))
    .exceptionally(t -> HttpResponse.of(500)));

    View Slide

  15. Streaming a million integers
    Server server = new ServerBuilder()
    (ctx, req) -> {
    HttpResponseWriter res = HttpResponse.streaming();
    streamNumbers(res, /* start */ 0);
    return res;

    View Slide

  16. Streaming a million integers
    void streamNumbers(HttpResponseWriter res, int start) {
    int end = Math.min(start + 1_000, 1_000_000);
    int i;
    for (i = start; i < end; i++) {
    res.write(HttpData.ofUtf8("%d\n", i));
    if (i == 1_000_000) {
    int nextStart = i;
    res.onDemand(() -> streamNumbers(res, nextStart));

    View Slide

  17. There are more!

    Query parameters and cookies

    Request and response converters

    Exception handlers

    Content type negotiation

    Regular-expression path mapping

    Aims to be on-par or better than

    View Slide

  18. Armeria by examples
    Thrift and gRPC

    View Slide

  19. gRPC & Thrift basics
    a. Write an IDL file – .proto or .thrift
    b. A compiler compiles it into .java files.
    c. Implement interfaces generated at ‘b’.
    namespace java com.example
    service HelloService {
    string hello(1:string name)
    syntax = "proto3";
    package grpc.hello;
    option java_package = "com.example";
    service HelloService {
    rpc Hello (HelloRequest)
    returns (HelloReply) {}
    message HelloRequest {
    string name = 1;
    message HelloReply {
    string message = 1;

    View Slide

  20. gRPC
    Server server = new ServerBuilder()
    .service(new GrpcServiceBuilder().addService(new MyHelloService())
    class MyHelloService extends HelloServiceGrpc.HelloServiceImplBase {
    public void hello(HelloRequest req, StreamObserver cb) {
    HelloReply reply = HelloReply.newBuilder()
    .setMessage("Hello, " + req.getName() + '!')

    View Slide

  21. Thrift
    Server server = new ServerBuilder()
    .service("/hello", THttpService.of(new MyHelloService()))
    class MyHelloService implements HelloService.AsyncIface {
    public void hello(String name, AsyncMethodCallback cb) {
    cb.onComplete("Hello, " + name + '!');

    View Slide

  22. What’s better than the official impls?

    – Works on HTTP/1·2, TLS on·off
    – gRPC-Web support, i.e. can call gRPC services from JavaScript frontends

    – HTTP/2
    – TTEXT a.k.a human-readable JSON

    Can mix gRPC, Thrift, REST, Tomcat, Jetty, …

    Can leverage the common Armeria features
    – Decorators
    – Web-based dashboard

    View Slide

  23. Decorating services
    Separation of concerns

    View Slide

  24. Intercepting an HTTP request
    THttpService.of(new MyHelloService())
    .decorate((delegate, ctx, req) -> {
    logger.info("{} Received a request", ctx);
    HttpResponse res = delegate.serve(ctx, req);
    res.completionFuture().thenAccept(unused -> {
    logger.info("{} Sent a response", ctx);
    return res;

    Can’t access RPC information yet.

    Need to decompose THttpService into:
    – ThriftCallService – RPC layer
    – THttpService – HTTP layer

    View Slide

  25. Intercepting an RPC request

    Complete access to the RPC call information

    Override the parameters and methods
    ThriftCallService.of(new MyHelloService())
    .decorate((delegate, ctx, req) -> {
    logger.info("{} Method: {}, Params: {}",
    ctx, req.method(), req.params());
    RpcResponse res = delegate.serve(ctx, req);
    res.thenAccept(value -> {
    logger.info("{} Result: {}", ctx, value);
    return res;

    View Slide

  26. Decorator as core extension mechanism

    Request throttling

    Metric collection

    Distributed tracing (Zipkin)

    HTTP content encoding

    HTTP authn/z


    Circuit breakers

    Automatic retries

    View Slide

  27. Distributed tracing with Brave·Zipkin
    import brave.Tracing;
    import brave.propagation.CurrentTraceContext;
    import brave.sampler.Sampler;
    Server server = new ServerBuilder()
    .sampler(Sampler.create(/* 5% */ 0.05f))

    View Slide

  28. Playing better with RPC
    Armeria documentation service

    View Slide

  29. Armeria documentation service

    Enabled by adding DocService to ServerBuilder

    Browse and invoke RPC services in an Armeria server

    Supports both gRPC and Thrift

    We have a plan to add:
    – Support for plain HTTP, like Swagger
    – Metric monitoring console
    – Runtime configuration editor, e.g. logger level

    View Slide

  30. View Slide

  31. View Slide

  32. Share the URL to reproduce a call.

    View Slide

  33. All the pieces fit together
    The ultimate all-in-one micro(?) service

    View Slide

  34. ServerBuilder sb = new ServerBuilder();
    // Thrift services
    sb.service("/thrift/foo", THttpService.of(myThriftFooService));
    sb.service("/thrift/bar", THttpService.of(myThriftBarService));
    // gRPC services
    sb.service(new GrpcServiceBuilder().addService(myGrpcFooService)
    // Static files
    sb.service("prefix:/files", HttpFileService.forFileSystem("htdocs"));
    // Legacy webapps
    sb.service("prefix:/webapp/foo", TomcatService.forFileSystem("foo.war"));
    sb.service("prefix:/webapp/bar", TomcatService.forFileSystem("bar.war"));
    // Documentation service
    sb.service("prefix:/internal/docs", new DocService());
    // L7 health check service
    sb.service("/internal/healthcheck", new HttpHealthCheckService());
    // Prometheus exposition
    sb.service("/internal/prometheus", new PrometheusExpositionService(...));
    // Enable logging for all services.
    // Enable distributed tracing for all services.
    // Send all access logs to Kafka.

    View Slide

  35. What about client-side?

    View Slide

  36. Armeria client API

    Similar experience to the server-side API
    – e.g. Common type, Decorators, RPC-friendliness

    Client-specific stuff
    – Circuit breakers
    – Automatic retries
    – Client-side load balancing
    – Retrofit integration

    View Slide

  37. // Send DNS queries periodically to discover service hosts.
    DnsServiceEndpointGroup group =
    // Register the group into the registry.
    EndpointGroupRegistry.register("myService", group,
    // Create a new HTTP client that connects to the group, with retries and logging.
    HttpClient client = new HttpClientBuilder("http://group:myService")
    // Send a request and print the response content.
    .thenAccept(res -> System.out.println(res.content().toStringUtf8()))
    .exceptionally(cause -> { /* Handle an error. */ });

    View Slide

  38. Got interested?

    View Slide

  39. Let’s build Armeria together!

    Use it.

    Ask questions.

    Request new features.

    Tell us what rocks and sucks.

    Consider joining the effort.

    View Slide

  40. Meet us at GitHub and Slack



    View Slide