Building Asynchronous Microservices with Armeria

Let us introduce Armeria, LINE's open-source reactive asynchronous microservice framework based on Java, Netty, HTTP/2, gRPC and Thrift.
We will have a quick tour of Armeria's easy-to-learn yet powerful features such as distributed tracing and service discovery, as well as learning why asynchronous and reactive microservice framework is important for services at any scale.

Previously presented at:

- TWJUG Meetup in Taipei on June 30, 2018
- LINE Developer Meetup in Tokyo on February 27, 2018

Previous slides that include case study:
https://speakerdeck.com/trustin/armeria-lines-next-generation-rpc-layer

Transcript

1. LINE Corporation
   June 2018
   Building Asynchronous
   Microservices with Armeria
   

   View Slide

2. Why build another
   microservice framework?
   

   View Slide

3. Asynchrony
   ●
   Most web frameworks are blocking and synchronous.
   – 1000 concurrent requests require 1000 threads.
   – In modern systems, a service sends requests to another backend.
   – Threads are busy doing nothing but just waiting for the backend.
   – Request queue is not consumed fast enough.
   – Result:
   ●
   Fragile system that falls apart even on a tiny backend failure
   ●
   Inefficient system that takes more memory and CPU than what it actually does
   

   View Slide

4. Reactive Streams
   ●
   Should we really buffer everything until a response is sent?
   – Retrieving 100,000 rows from a table
   ●
   Why should a client send many requests to reduce the server memory
   pressure?
   – Retrieving 1,000 rows 100 times
   ●
   What if we want to stream infinitely?
   – Stock quotes
   ●
   Our protocol is not a problem here.
   – Our ‘blocking’ programming model is blocking ‘us’.
   

   View Slide

5. Better RPC support
   ●
   RPC has been hardly a 1st-class citizen in web
   frameworks.
   – Distinction from RPC-level and HTTP-level failure
   – Access logs with RPC-level details
   – Debugging console
   ●
   What if …
   – We build a microservice framework with RPC at its core?
   

   View Slide

6. More reasons back in 2015
   ●
   Leverage Java 8 syntax
   – Lambda expressions
   – Default interface methods
   – Static interface methods
   ●
   Provide ‘working’ HTTP/2 implementation
   – ‘H2C’ support (HTTP/2 over ‘C’leartext)
   – OpenSSL-based faster TLS connection
   

   View Slide

7. Better UX for devs
   ●
   This is the last yet most important part.
   ●
   Nobody wants a framework that’s hard to use.
   ●
   Can we design …
   – a user-friendly asynchronous programming API
   – … with modest learning curve?
   

   View Slide

8. Our answer – Armeria
   ●
   HTTP/2
   – Both ciphertext and cleartext, a.k.a. h2 and h2c
   ●
   Reactive
   – Implements Reactive Streams API
   ●
   Ability to mix different types of services in a single server
   – gRPC, Thrift, REST services
   – Legacy webapps via embedded Tomcat or Jetty
   ●
   High-performance
   – Netty with /dev/epoll transport and BoringSSL SSLEngine
   ●
   Easy to learn and fun to use (hopefully)
   

   View Slide

9. Armeria by examples
   HTTP / REST
   

   View Slide

10. Hello, world!
    Server server = new ServerBuilder()
    .http(8080)
    .https(8443)
    .tlsSelfSigned()
    .service("/", (ctx, req) -> HttpResponse.of("Hello, world!"))
    .build();
    server.start();
    

    View Slide

11. Hello, world – Parameterized
    Server server = new ServerBuilder()
    .http(8080)
    .service("/hello/:name",
    (ctx, req) -> HttpResponse.of("Hello, %s!",
    ctx.pathParam("name")))
    .build();
    server.start();
    

    View Slide

12. Hello, world – Annotated
    Server server = new ServerBuilder()
    .http(8080)
    .annotatedService(new Object() {
    @Get("/hello/:name")
    public String hello(@Param String name) {
    return String.format("Hello, %s!", name);
    }
    })
    .build();
    server.start();
    

    View Slide

13. With CompletableFuture
    Supplier> random = ...; // Takes long!
    Object randomService = new Object() {
    @Get("/random")
    public CompletableFuture nextInt() {
    // Request a random number.
    CompletableFuture f1 = random.get();
    // Convert the random number into a response.
    CompletableFuture f2 =
    f1.thenApply(num -> HttpResponse.of("%d\n", num))
    .exceptionally(t -> HttpResponse.of(500));
    return f2;
    }
    };
    new ServerBuilder()
    .http(8080).annotatedService(randomService).build().start();
    

    View Slide

14. Simplification with method chaining
    // Annotated version:
    @Get("/random")
    public CompletableFuture nextIntSimple() {
    return random.get()
    .thenApply(num -> HttpResponse.of("%d\n", num))
    .exceptionally(t -> HttpResponse.of(500));
    }
    // Non-annotated version:
    HttpService randomService = (ctx, req) -> HttpResponse.from(
    random.get()
    .thenApply(num -> HttpResponse.of("%d\n", num))
    .exceptionally(t -> HttpResponse.of(500)));
    

    View Slide

15. Streaming a million integers
    Server server = new ServerBuilder()
    .http(8080)
    .service("/count",
    (ctx, req) -> {
    HttpResponseWriter res = HttpResponse.streaming();
    res.write(HttpHeaders.of(200));
    streamNumbers(res, /* start */ 0);
    return res;
    })
    .build();
    

    View Slide

16. Streaming a million integers
    (cont’d)
    void streamNumbers(HttpResponseWriter res, int start) {
    int end = Math.min(start + 1_000, 1_000_000);
    int i;
    for (i = start; i < end; i++) {
    res.write(HttpData.ofUtf8("%d\n", i));
    }
    if (i == 1_000_000) {
    res.close();
    return;
    }
    int nextStart = i;
    res.onDemand(() -> streamNumbers(res, nextStart));
    }
    

    View Slide

17. There are more!
    ●
    Query parameters and cookies
    ●
    Request and response converters
    ●
    Exception handlers
    ●
    Content type negotiation
    ●
    Regular-expression path mapping
    ●
    Aims to be on-par or better than
    

    View Slide

18. Armeria by examples
    Thrift and gRPC
    

    View Slide

19. gRPC & Thrift basics
    a. Write an IDL file – .proto or .thrift
    b. A compiler compiles it into .java files.
    c. Implement interfaces generated at ‘b’.
    namespace java com.example
    service HelloService {
    string hello(1:string name)
    }
    syntax = "proto3";
    package grpc.hello;
    option java_package = "com.example";
    service HelloService {
    rpc Hello (HelloRequest)
    returns (HelloReply) {}
    }
    message HelloRequest {
    string name = 1;
    }
    message HelloReply {
    string message = 1;
    }
    

    View Slide

20. gRPC
    Server server = new ServerBuilder()
    .http(8080)
    .service(new GrpcServiceBuilder().addService(new MyHelloService())
    .build())
    .build();
    class MyHelloService extends HelloServiceGrpc.HelloServiceImplBase {
    @Override
    public void hello(HelloRequest req, StreamObserver cb) {
    HelloReply reply = HelloReply.newBuilder()
    .setMessage("Hello, " + req.getName() + '!')
    .build();
    cb.onNext(reply);
    cb.onCompleted();
    }
    }
    

    View Slide

21. Thrift
    Server server = new ServerBuilder()
    .http(8080)
    .service("/hello", THttpService.of(new MyHelloService()))
    .build();
    class MyHelloService implements HelloService.AsyncIface {
    @Override
    public void hello(String name, AsyncMethodCallback cb) {
    cb.onComplete("Hello, " + name + '!');
    }
    }
    

    View Slide

22. What’s better than the official impls?
    ●
    gRPC
    – Works on HTTP/1·2, TLS on·off
    – gRPC-Web support, i.e. can call gRPC services from JavaScript frontends
    ●
    Thrift
    – HTTP/2
    – TTEXT a.k.a human-readable JSON
    ●
    Can mix gRPC, Thrift, REST, Tomcat, Jetty, …
    ●
    Can leverage the common Armeria features
    – Decorators
    – Web-based dashboard
    

    View Slide

23. Decorating services
    Separation of concerns
    

    View Slide

24. Intercepting an HTTP request
    THttpService.of(new MyHelloService())
    .decorate((delegate, ctx, req) -> {
    logger.info("{} Received a request", ctx);
    HttpResponse res = delegate.serve(ctx, req);
    res.completionFuture().thenAccept(unused -> {
    logger.info("{} Sent a response", ctx);
    });
    return res;
    })
    ●
    Can’t access RPC information yet.
    ●
    Need to decompose THttpService into:
    – ThriftCallService – RPC layer
    – THttpService – HTTP layer
    

    View Slide

25. Intercepting an RPC request
    ●
    Complete access to the RPC call information
    ●
    Override the parameters and methods
    ThriftCallService.of(new MyHelloService())
    .decorate((delegate, ctx, req) -> {
    logger.info("{} Method: {}, Params: {}",
    ctx, req.method(), req.params());
    RpcResponse res = delegate.serve(ctx, req);
    res.thenAccept(value -> {
    logger.info("{} Result: {}", ctx, value);
    });
    return res;
    })
    .decorate(THttpService.newDecorator())
    

    View Slide

26. Decorator as core extension mechanism
    ●
    Request throttling
    ●
    Metric collection
    ●
    Distributed tracing (Zipkin)
    ●
    HTTP content encoding
    ●
    HTTP authn/z
    ●
    CORS
    ●
    Circuit breakers
    ●
    Automatic retries
    ●
    …
    

    View Slide

27. Distributed tracing with Brave·Zipkin
    import brave.Tracing;
    import brave.propagation.CurrentTraceContext;
    import brave.sampler.Sampler;
    Server server = new ServerBuilder()
    .http(8080)
    .service("/my_service",
    myService.decorate(
    HttpTracingService.newDecorator(
    Tracing.newBuilder()
    .currentTraceContext(CurrentTraceContext.Default.create())
    .localServiceName("myService")
    .spanReporter(spanReporter)
    .sampler(Sampler.create(/* 5% */ 0.05f))
    .build())))
    .build();
    

    View Slide

28. Playing better with RPC
    Armeria documentation service
    

    View Slide

29. Armeria documentation service
    ●
    Enabled by adding DocService to ServerBuilder
    ●
    Browse and invoke RPC services in an Armeria server
    ●
    Supports both gRPC and Thrift
    ●
    We have a plan to add:
    – Support for plain HTTP, like Swagger
    – Metric monitoring console
    – Runtime configuration editor, e.g. logger level
    

    View Slide

30. View Slide

31. View Slide

32. ●
    Share the URL to reproduce a call.
    

    View Slide

33. All the pieces fit together
    The ultimate all-in-one micro(?) service
    

    View Slide

34. ServerBuilder sb = new ServerBuilder();
    // Thrift services
    sb.service("/thrift/foo", THttpService.of(myThriftFooService));
    sb.service("/thrift/bar", THttpService.of(myThriftBarService));
    // gRPC services
    sb.service(new GrpcServiceBuilder().addService(myGrpcFooService)
    .addService(myGrpcBarService)
    .build());
    // Static files
    sb.service("prefix:/files", HttpFileService.forFileSystem("htdocs"));
    // Legacy webapps
    sb.service("prefix:/webapp/foo", TomcatService.forFileSystem("foo.war"));
    sb.service("prefix:/webapp/bar", TomcatService.forFileSystem("bar.war"));
    // Documentation service
    sb.service("prefix:/internal/docs", new DocService());
    // L7 health check service
    sb.service("/internal/healthcheck", new HttpHealthCheckService());
    // Prometheus exposition
    sb.service("/internal/prometheus", new PrometheusExpositionService(...));
    // Enable logging for all services.
    sb.decorator(LoggingService.newDecorator());
    // Enable distributed tracing for all services.
    sb.decorator(HttpTracingService.newDecorator(...));
    // Send all access logs to Kafka.
    sb.decorator(KafkaStructuredLoggingService.newDecorator(...));
    

    View Slide

35. What about client-side?
    

    View Slide

36. Armeria client API
    ●
    Similar experience to the server-side API
    – e.g. Common type, Decorators, RPC-friendliness
    ●
    Client-specific stuff
    – Circuit breakers
    – Automatic retries
    – Client-side load balancing
    – Retrofit integration
    

    View Slide

37. // Send DNS queries periodically to discover service hosts.
    DnsServiceEndpointGroup group =
    DnsServiceEndpointGroup.of("my-service.cluster.local");
    // Register the group into the registry.
    EndpointGroupRegistry.register("myService", group,
    EndpointSelectionStrategy.WEIGHTED_ROUND_ROBIN);
    // Create a new HTTP client that connects to the group, with retries and logging.
    HttpClient client = new HttpClientBuilder("http://group:myService")
    .decorator(RetryingHttpClient.newDecorator(
    RetryStrategy.onServerErrorStatus()))
    .decorator(LoggingClient.newDecorator())
    .build();
    // Send a request and print the response content.
    client.get("/greet").aggregate()
    .thenAccept(res -> System.out.println(res.content().toStringUtf8()))
    .exceptionally(cause -> { /* Handle an error. */ });
    

    View Slide

38. Got interested?
    

    View Slide

39. Let’s build Armeria together!
    ●
    Use it.
    ●
    Ask questions.
    ●
    Request new features.
    ●
    Tell us what rocks and sucks.
    ●
    Consider joining the effort.
    

    View Slide

40. Meet us at GitHub and Slack
    ●
    https://github.com/line/armeria
    ●
    https://line-slacknow.herokuapp.com/line-armeria/
    

    View Slide