Slide 1

Slide 1 text

@armeria_project Trustin Lee, LINE Aug 2019 Armeria The Only Thrift · gRPC · REST Microservice Framework You’ll Need

Slide 2

Slide 2 text

@armeria_project What is Armeria? A user-friendly Java microservice framework based on asynchronous and reactive model with HTTP/2 and modern RPC protocols in mind

Slide 3

Slide 3 text

@armeria_project What is Armeria? A user-friendly Java microservice framework based on asynchronous and reactive model with HTTP/2 and modern RPC protocols in mind

Slide 4

Slide 4 text

@armeria_project One fine day of synchronous microservice Shard 1 Shard 2 Shard 3 A bunch of clients Thread 1 Thread 2 Thread 3 Thread 4 Pending requests (Queue) Workers Read S1 Read S2 Read S3 Read S1 Read S2 Read S3 Read S1 Read S2 Read S3 Read S1 Read S2 Read S3 Read S2 Read S3 Read S1 Read S1 Read S2 Read S3 Read S1

Slide 5

Slide 5 text

@armeria_project … until Shard 2 ruins it. Shard 1 Shard 2 Shard 3 A bunch of clients Thread 1 Thread 2 Thread 3 Thread 4 Pending requests (Queue) Workers Read S1 Read S2 Read S3 Read S1 Read S2 Read S3 Read S1 Read S2 Read S3 Read S1 Read S2 Read S3 Read S2 Read S3 Read S1 Read S1 Read S2 Read S3 Read S1 Timeout!

Slide 6

Slide 6 text

@armeria_project Shard 1 & 3: Why are no requests coming? Workers: We’re busy waiting for Shard 2. Shard 1 Shard 2 Shard 3 A bunch of clients Thread 1 Thread 2 Thread 3 Thread 4 Pending requests (Queue) Workers Read S1 Read S2 Read S3 Read S1 Read S2 Read S3 Read S1 Read S2 Read S3 Read S1 Read S2 Read S3 Read S2 Read S3 Read S1 Read S2 Read S2 Read S2 Read S2 Timeouts! Timeouts! Timeouts! Timeouts!

Slide 7

Slide 7 text

@armeria_project How can we solve this? ● Add more CPUs? – They are very idle. ● Add more threads? – They will all get stuck with Shard 2 in no time. – Waste of CPU cycles – context switches are not free. – Waste of memory – each thread needs its own stack. ● Result: – Fragile system that falls apart even on a tiny backend failure – Inefficient system that takes more memory and CPU than what it actually needs ● Maybe you can work around this, but you’ll have to keep tuning and adding hacks.

Slide 8

Slide 8 text

@armeria_project What is Armeria? A user-friendly Java microservice framework based on asynchronous and reactive model with HTTP/2 and modern RPC protocols in mind

Slide 9

Slide 9 text

@armeria_project Problems with large payloads ● We solved blocking problem with asynchronous programming, but can we send 10MB personalized response to 100K clients? – Can’t hold that much in RAM – 10MB × 100K = 1TB ● What if we/they send too fast? – Different bandwidth & processing power ● We need ‘just enough buffering.’ – Expect OutOfMemoryError otherwise.

Slide 10

Slide 10 text

@armeria_project What is Armeria? A user-friendly Java microservice framework based on asynchronous and reactive model with HTTP/2 and modern RPC protocols in mind

Slide 11

Slide 11 text

@armeria_project RPC vs. HTTP impedance mismatch ● RPC has been hardly a 1st-class citizen in web frameworks. – Method name – Parameter names and values – Return value or exception POST /some_service HTTP/1.1 Host: example.com Content-Length: 96 HTTP/1.1 200 OK Host: example.com Content-Length: 192 Failed RPC call

Slide 12

Slide 12 text

@armeria_project Access logs re-imagined ● What if we provide access logs with … – Universal API for gRPC · Thrift · REST – RPC-level information – Precise send · receive timings ● Imagine streaming to data processing pipeline, e.g. Kafka. 192.167.1.2 - - [10/Oct/2000:13:55:36 -0700] "POST /some_service HTTP/1.1" 200 2326

Slide 13

Slide 13 text

@armeria_project Making a debug call ● Sending an ad-hoc query in RPC is hard. – Find a proper service definition, e.g. .thrift or .proto files – Set up code generator, build, IDE, etc. – Write some code that makes an RPC call. ● HTTP in contrast: – cURL, telnet command, web-based tools and more. ● There’s some effort to make things easier. – gRPC reflection service + grpcurl ● What if we build something more convenient and collaborative?

Slide 14

Slide 14 text

@armeria_project More than one RPC protocol ● Supports both gRPC & Thrift … – on a single HTTP port – in a single JVM – without any proxies? ● Can serve non-RPC requests together. – REST API – Static files – Exposing metrics – Health-check requests from load balancers – Traditional JEE webapps ● Share common logic between different endpoints!

Slide 15

Slide 15 text

@armeria_project What is Armeria? A user-friendly Java microservice framework based on asynchronous and reactive model with HTTP/2 and modern RPC protocols in mind

Slide 16

Slide 16 text

@armeria_project Better UX for devs ● The last yet most important ● Nobody wants a framework that’s hard to use. ● Can we design … – a user-friendly asynchronous API – … with modest learning curve?

Slide 17

Slide 17 text

@armeria_project Our answer – Armeria ● Asynchronous + High-performance – Netty with JNI-based /dev/epoll transport and BoringSSL ● Seamless integration with Reactive Streams – RxJava2, Project Reactor, Spring WebFlux… ● Ability to mix different types of services in a single server – gRPC, Thrift, REST, Tomcat, Jetty… ● Not just HTTP/2 – gRPC on HTTP/1 – HAProxy ● Easy to learn and fun to use (hopefully)

Slide 18

Slide 18 text

@armeria_project Armeria by examples HTTP · REST

Slide 19

Slide 19 text

@armeria_project Hello, world! Server server = new ServerBuilder() .http(8080) .https(8443) .tlsSelfSigned() .haproxy(8080) .service("/", (ctx, req) -> HttpResponse.of("Hello, world!")) .build(); server.start(); Protocol auto-detection at 8080

Slide 20

Slide 20 text

@armeria_project Hello, world – Parameterized Server server = new ServerBuilder() .http(8080) .service("/hello/:name", (ctx, req) -> HttpResponse.of("Hello, %s!", ctx.pathParam("name"))) .build(); server.start();

Slide 21

Slide 21 text

@armeria_project Hello, world – Annotated Server server = new ServerBuilder() .http(8080) .annotatedService(new Object() { @Get("/hello/:name") public String hello(@Param String name) { return String.format("Hello, %s!", name); } }) .build(); server.start();

Slide 22

Slide 22 text

@armeria_project There are more! ● CompletableFuture and reactive responses ● Query parameters and cookies ● Request and response converters ● Exception handlers ● Content type negotiation ● Regular-expression path mapping ● Aims to be on-par or better than

Slide 23

Slide 23 text

@armeria_project Armeria by examples Thrift · gRPC

Slide 24

Slide 24 text

@armeria_project gRPC & Thrift basics a. Write an IDL file – .proto or .thrift b. A compiler compiles it into .java files. c. Implement interfaces generated at ‘b’. namespace java com.example service HelloService { string hello(1:string name) } syntax = "proto3"; package grpc.hello; option java_package = "com.example"; service HelloService { rpc Hello (HelloRequest) returns (HelloReply) {} } message HelloRequest { string name = 1; } message HelloReply { string message = 1; }

Slide 25

Slide 25 text

@armeria_project gRPC Server server = new ServerBuilder() .http(8080) .service(new GrpcServiceBuilder().addService(new MyHelloService()) .build()) .build(); class MyHelloService extends HelloServiceGrpc.HelloServiceImplBase { @Override public void hello(HelloRequest req, StreamObserver cb) { HelloReply reply = HelloReply.newBuilder() .setMessage("Hello, " + req.getName() + '!') .build(); cb.onNext(reply); cb.onCompleted(); } }

Slide 26

Slide 26 text

@armeria_project Thrift Server server = new ServerBuilder() .http(8080) .service("/hello", THttpService.of(new MyHelloService())) .build(); class MyHelloService implements HelloService.AsyncIface { @Override public void hello(String name, AsyncMethodCallback cb) { cb.onComplete("Hello, " + name + '!'); } }

Slide 27

Slide 27 text

@armeria_project What’s better than the official impls? ● gRPC – Works on both HTTP/1 and 2 – gRPC-Web support, i.e. can call gRPC services from JavaScript frontends ● Thrift – HTTP/2 – TTEXT (human-readable REST-ish JSON) ● Can mix gRPC, Thrift, REST, Tomcat, Jetty, … ● Can leverage the common Armeria features – Decorators – Web-based dashboard

Slide 28

Slide 28 text

@armeria_project Decorating services Separation of concerns a.k.a. filter, decorator, interceptor, middleware

Slide 29

Slide 29 text

@armeria_project Intercepting an HTTP request THttpService.of(new MyHelloService()) .decorate((delegate, ctx, req) -> { logger.info("{} Received a request", ctx); HttpResponse res = delegate.serve(ctx, req); res.completionFuture().thenAccept(unused -> { logger.info("{} Sent a response", ctx); }); return res; }) ● Can’t access RPC information yet. ● Need to decompose THttpService into: – ThriftCallService – RPC layer – THttpService – HTTP layer

Slide 30

Slide 30 text

@armeria_project Intercepting an RPC request ● Complete access to the RPC call information ● Override the parameters and methods ThriftCallService.of(new MyHelloService()) .decorate((delegate, ctx, req) -> { logger.info("{} Method: {}, Params: {}", ctx, req.method(), req.params()); RpcResponse res = delegate.serve(ctx, req); res.thenAccept(value -> { logger.info("{} Result: {}", ctx, value); }); return res; }) .decorate(THttpService.newDecorator())

Slide 31

Slide 31 text

@armeria_project Decorator as core extension mechanism ● Request throttling ● Metric collection ● Distributed tracing (Zipkin) ● HTTP content encoding ● HTTP authn/z ● CORS policies ● Circuit breakers ● Automatic retries ● …

Slide 32

Slide 32 text

@armeria_project Distributed tracing with Zipkin import brave.Tracing; import brave.sampler.Sampler; Server server = new ServerBuilder() .http(8080) .service("/my_service", myService.decorate( BraveService.newDecorator( Tracing.newBuilder() .currentTraceContext(…) .localServiceName("myService") .spanReporter(spanReporter) .sampler(Sampler.create(/* 5% */ 0.05f)) .build()))) .build();

Slide 33

Slide 33 text

@armeria_project Playing better with RPC Armeria documentation service

Slide 34

Slide 34 text

@armeria_project Armeria documentation service ● Enabled by adding DocService to ServerBuilder ● Browse and invoke RPC services in an Armeria server – No fiddling with binary payloads – Send a request without writing code ● Supports gRPC, Thrift and annotated services ● We have a plan to add: – Metric monitoring console – Runtime configuration editor, e.g. logger level

Slide 35

Slide 35 text

@armeria_project

Slide 36

Slide 36 text

@armeria_project ● Share the URL to reproduce a call.

Slide 37

Slide 37 text

@armeria_project All pieces together The ultimate all-in-one micro service

Slide 38

Slide 38 text

@armeria_project ServerBuilder sb = new ServerBuilder(); // Thrift services sb.service("/thrift/foo", THttpService.of(myThriftFooService)); sb.service("/thrift/bar", THttpService.of(myThriftBarService)); // gRPC services sb.service(new GrpcServiceBuilder().addService(myGrpcFooService) .addService(myGrpcBarService) .build()); // Static files sb.service("prefix:/files", HttpFileService.forFileSystem("htdocs")); // Legacy webapps sb.service("prefix:/webapp/foo", TomcatService.forFileSystem("foo.war")); sb.service("prefix:/webapp/bar", TomcatService.forFileSystem("bar.war")); // Documentation service sb.service("prefix:/internal/docs", new DocService()); // L7 health check service sb.service("/internal/healthcheck", HealthCheckService.of()); // Prometheus exposition sb.service("/internal/prometheus", new PrometheusExpositionService(...)); // Enable logging for all services. sb.decorator(LoggingService.newDecorator()); // Enable distributed tracing for all services. sb.decorator(BraveService.newDecorator(...)); // Send all access logs to Kafka. sb.accessLogWriter(new KafkaAccessLogWriter(...), true);

Slide 39

Slide 39 text

@armeria_project What about client-side?

Slide 40

Slide 40 text

@armeria_project Armeria client API ● Similar experience to the server-side API – e.g. Common types, Decorators, RPC-friendliness ● Client-specific stuff – Circuit breakers – Automatic retries – Client-side load balancing – Retrofit integration – https://square.github.io/retrofit/

Slide 41

Slide 41 text

@armeria_project // Send DNS queries periodically to discover service hosts (k8s style). // Send health check requests to /internal/healthcheck and remove dead hosts. EndpointGroup group = HealthCheckedEndpointGroup.of( DnsServiceEndpointGroup.of("my-service.cluster.local"), "/internal/healthcheck"); // Register the group into the registry. EndpointGroupRegistry.register("myService", group, EndpointSelectionStrategy.WEIGHTED_ROUND_ROBIN); // Create a new HTTP client that connects to the group // with retries, circuit breaker and logging. HttpClient client = new HttpClientBuilder("http://group:myService") .decorator(RetryingHttpClient.newDecorator(onServerErrorStatus())) .decorator(CircuitBreakerHttpClient.newDecorator(...)) .decorator(LoggingClient.newDecorator()) .build(); // Send a request and print the response content. client.get("/greet").aggregate() .thenAccept(res -> System.out.println(res.contentUtf8())) .exceptionally(cause -> { /* Handle an error. */ });

Slide 42

Slide 42 text

@armeria_project Got interested?

Slide 43

Slide 43 text

@armeria_project Let’s build Armeria together! ● Give it a try. ● Ask questions. ● Request new features. ● Tell us what rocks and sucks. ● Consider joining the effort.

Slide 44

Slide 44 text

@armeria_project Meet us at GitHub github.com/line/armeria line.github.io/armeria