Building Asynchronous Microservices with Armeria

Let us introduce Armeria, LINE's open-source reactive asynchronous microservice framework based on Java, Netty, HTTP/2, gRPC and Thrift.
We will have a quick tour of Armeria's easy-to-learn yet powerful features such as distributed tracing and service discovery, as well as learning why asynchronous and reactive microservice framework is important for services at any scale.

Previously presented at:

- TWJUG Meetup in Taipei on June 30, 2018
- LINE Developer Meetup in Tokyo on February 27, 2018

Previous slides that include case study:
https://speakerdeck.com/trustin/armeria-lines-next-generation-rpc-layer

Transcript

1. LINE Corporation June 2018 Building Asynchronous Microservices with Armeria
2. Why build another microservice framework?
3. Asynchrony • Most web frameworks are blocking and synchronous. – 1000 concurrent requests require 1000 threads. – In modern systems, a service sends requests to another backend. – Threads are busy doing nothing but just waiting for the backend. – Request queue is not consumed fast enough. – Result: • Fragile system that falls apart even on a tiny backend failure • Inefficient system that takes more memory and CPU than what it actually does
4. Reactive Streams • Should we really buffer everything until a response is sent? – Retrieving 100,000 rows from a table • Why should a client send many requests to reduce the server memory pressure? – Retrieving 1,000 rows 100 times • What if we want to stream infinitely? – Stock quotes • Our protocol is not a problem here. – Our ‘blocking’ programming model is blocking ‘us’.
5. Better RPC support • RPC has been hardly a 1st-class citizen in web frameworks. – Distinction from RPC-level and HTTP-level failure – Access logs with RPC-level details – Debugging console • What if … – We build a microservice framework with RPC at its core?
6. More reasons back in 2015 • Leverage Java 8 syntax – Lambda expressions – Default interface methods – Static interface methods • Provide ‘working’ HTTP/2 implementation – ‘H2C’ support (HTTP/2 over ‘C’leartext) – OpenSSL-based faster TLS connection
7. Better UX for devs • This is the last yet most important part. • Nobody wants a framework that’s hard to use. • Can we design … – a user-friendly asynchronous programming API – … with modest learning curve?
8. Our answer – Armeria • HTTP/2 – Both ciphertext and cleartext, a.k.a. h2 and h2c • Reactive – Implements Reactive Streams API • Ability to mix different types of services in a single server – gRPC, Thrift, REST services – Legacy webapps via embedded Tomcat or Jetty • High-performance – Netty with /dev/epoll transport and BoringSSL SSLEngine • Easy to learn and fun to use (hopefully)
9. Armeria by examples HTTP / REST
10. Hello, world! Server server = new ServerBuilder() .http(8080) .https(8443) .tlsSelfSigned() .service("/", (ctx, req) -> HttpResponse.of("Hello, world!")) .build(); server.start();
11. Hello, world – Parameterized Server server = new ServerBuilder() .http(8080) .service("/hello/:name", (ctx, req) -> HttpResponse.of("Hello, %s!", ctx.pathParam("name"))) .build(); server.start();
12. Hello, world – Annotated Server server = new ServerBuilder() .http(8080) .annotatedService(new Object() { @Get("/hello/:name") public String hello(@Param String name) { return String.format("Hello, %s!", name); } }) .build(); server.start();
13. With CompletableFuture Supplier<CompletableFuture<Integer>> random = ...; // Takes long! Object randomService = new Object() { @Get("/random") public CompletableFuture<HttpResponse> nextInt() { // Request a random number. CompletableFuture<Integer> f1 = random.get(); // Convert the random number into a response. CompletableFuture<HttpResponse> f2 = f1.thenApply(num -> HttpResponse.of("%d\n", num)) .exceptionally(t -> HttpResponse.of(500)); return f2; } }; new ServerBuilder() .http(8080).annotatedService(randomService).build().start();
14. Simplification with method chaining // Annotated version: @Get("/random") public CompletableFuture<HttpResponse> nextIntSimple() { return random.get() .thenApply(num -> HttpResponse.of("%d\n", num)) .exceptionally(t -> HttpResponse.of(500)); } // Non-annotated version: HttpService randomService = (ctx, req) -> HttpResponse.from( random.get() .thenApply(num -> HttpResponse.of("%d\n", num)) .exceptionally(t -> HttpResponse.of(500)));
15. Streaming a million integers Server server = new ServerBuilder() .http(8080) .service("/count", (ctx, req) -> { HttpResponseWriter res = HttpResponse.streaming(); res.write(HttpHeaders.of(200)); streamNumbers(res, /* start */ 0); return res; }) .build();
16. Streaming a million integers (cont’d) void streamNumbers(HttpResponseWriter res, int start) { int end = Math.min(start + 1_000, 1_000_000); int i; for (i = start; i < end; i++) { res.write(HttpData.ofUtf8("%d\n", i)); } if (i == 1_000_000) { res.close(); return; } int nextStart = i; res.onDemand(() -> streamNumbers(res, nextStart)); }
17. There are more! • Query parameters and cookies • Request and response converters • Exception handlers • Content type negotiation • Regular-expression path mapping • Aims to be on-par or better than <your favorite web framework>
18. Armeria by examples Thrift and gRPC
19. gRPC & Thrift basics a. Write an IDL file – .proto or .thrift b. A compiler compiles it into .java files. c. Implement interfaces generated at ‘b’. namespace java com.example service HelloService { string hello(1:string name) } syntax = "proto3"; package grpc.hello; option java_package = "com.example"; service HelloService { rpc Hello (HelloRequest) returns (HelloReply) {} } message HelloRequest { string name = 1; } message HelloReply { string message = 1; }
20. gRPC Server server = new ServerBuilder() .http(8080) .service(new GrpcServiceBuilder().addService(new MyHelloService()) .build()) .build(); class MyHelloService extends HelloServiceGrpc.HelloServiceImplBase { @Override public void hello(HelloRequest req, StreamObserver<HelloReply> cb) { HelloReply reply = HelloReply.newBuilder() .setMessage("Hello, " + req.getName() + '!') .build(); cb.onNext(reply); cb.onCompleted(); } }
21. Thrift Server server = new ServerBuilder() .http(8080) .service("/hello", THttpService.of(new MyHelloService())) .build(); class MyHelloService implements HelloService.AsyncIface { @Override public void hello(String name, AsyncMethodCallback<String> cb) { cb.onComplete("Hello, " + name + '!'); } }
22. What’s better than the official impls? • gRPC – Works on HTTP/1·2, TLS on·off – gRPC-Web support, i.e. can call gRPC services from JavaScript frontends • Thrift – HTTP/2 – TTEXT a.k.a human-readable JSON • Can mix gRPC, Thrift, REST, Tomcat, Jetty, … • Can leverage the common Armeria features – Decorators – Web-based dashboard
23. Decorating services Separation of concerns
24. Intercepting an HTTP request THttpService.of(new MyHelloService()) .decorate((delegate, ctx, req) -> { logger.info("{} Received a request", ctx); HttpResponse res = delegate.serve(ctx, req); res.completionFuture().thenAccept(unused -> { logger.info("{} Sent a response", ctx); }); return res; }) • Can’t access RPC information yet. • Need to decompose THttpService into: – ThriftCallService – RPC layer – THttpService – HTTP layer
25. Intercepting an RPC request • Complete access to the RPC call information • Override the parameters and methods ThriftCallService.of(new MyHelloService()) .decorate((delegate, ctx, req) -> { logger.info("{} Method: {}, Params: {}", ctx, req.method(), req.params()); RpcResponse res = delegate.serve(ctx, req); res.thenAccept(value -> { logger.info("{} Result: {}", ctx, value); }); return res; }) .decorate(THttpService.newDecorator())
26. Decorator as core extension mechanism • Request throttling • Metric collection • Distributed tracing (Zipkin) • HTTP content encoding • HTTP authn/z • CORS • Circuit breakers • Automatic retries • …
27. Distributed tracing with Brave·Zipkin import brave.Tracing; import brave.propagation.CurrentTraceContext; import brave.sampler.Sampler; Server server = new ServerBuilder() .http(8080) .service("/my_service", myService.decorate( HttpTracingService.newDecorator( Tracing.newBuilder() .currentTraceContext(CurrentTraceContext.Default.create()) .localServiceName("myService") .spanReporter(spanReporter) .sampler(Sampler.create(/* 5% */ 0.05f)) .build()))) .build();
28. Playing better with RPC Armeria documentation service
29. Armeria documentation service • Enabled by adding DocService to ServerBuilder • Browse and invoke RPC services in an Armeria server • Supports both gRPC and Thrift • We have a plan to add: – Support for plain HTTP, like Swagger – Metric monitoring console – Runtime configuration editor, e.g. logger level
32. • Share the URL to reproduce a call.
33. All the pieces fit together The ultimate all-in-one micro(?) service
34. ServerBuilder sb = new ServerBuilder(); // Thrift services sb.service("/thrift/foo", THttpService.of(myThriftFooService)); sb.service("/thrift/bar", THttpService.of(myThriftBarService)); // gRPC services sb.service(new GrpcServiceBuilder().addService(myGrpcFooService) .addService(myGrpcBarService) .build()); // Static files sb.service("prefix:/files", HttpFileService.forFileSystem("htdocs")); // Legacy webapps sb.service("prefix:/webapp/foo", TomcatService.forFileSystem("foo.war")); sb.service("prefix:/webapp/bar", TomcatService.forFileSystem("bar.war")); // Documentation service sb.service("prefix:/internal/docs", new DocService()); // L7 health check service sb.service("/internal/healthcheck", new HttpHealthCheckService()); // Prometheus exposition sb.service("/internal/prometheus", new PrometheusExpositionService(...)); // Enable logging for all services. sb.decorator(LoggingService.newDecorator()); // Enable distributed tracing for all services. sb.decorator(HttpTracingService.newDecorator(...)); // Send all access logs to Kafka. sb.decorator(KafkaStructuredLoggingService.newDecorator(...));
35. What about client-side?
36. Armeria client API • Similar experience to the server-side API – e.g. Common type, Decorators, RPC-friendliness • Client-specific stuff – Circuit breakers – Automatic retries – Client-side load balancing – Retrofit integration
37. // Send DNS queries periodically to discover service hosts. DnsServiceEndpointGroup group = DnsServiceEndpointGroup.of("my-service.cluster.local"); // Register the group into the registry. EndpointGroupRegistry.register("myService", group, EndpointSelectionStrategy.WEIGHTED_ROUND_ROBIN); // Create a new HTTP client that connects to the group, with retries and logging. HttpClient client = new HttpClientBuilder("http://group:myService") .decorator(RetryingHttpClient.newDecorator( RetryStrategy.onServerErrorStatus())) .decorator(LoggingClient.newDecorator()) .build(); // Send a request and print the response content. client.get("/greet").aggregate() .thenAccept(res -> System.out.println(res.content().toStringUtf8())) .exceptionally(cause -> { /* Handle an error. */ });
38. Got interested?
39. Let’s build Armeria together! • Use it. • Ask questions. • Request new features. • Tell us what rocks and sucks. • Consider joining the effort.
40. Meet us at GitHub and Slack • https://github.com/line/armeria • https://line-slacknow.herokuapp.com/line-armeria/