Armeria: The Only Thrift, gRPC, REST Microservice Framework You'll Need

@armeria_project
Trustin Lee, LINE
Aug 2019
Armeria
The Only Thrift · gRPC · REST
Microservice Framework You’ll Need

View Slide

@armeria_project
What is Armeria?
A user-friendly Java microservice framework
based on asynchronous and reactive model
with HTTP/2 and modern RPC protocols in mind

View Slide

@armeria_project
One fine day of
synchronous microservice
Shard 1
Shard 2
Shard 3
A bunch of
clients
Thread 1
Thread 2
Thread 3
Thread 4
Pending requests (Queue) Workers
Read
S1
Read
S2
Read
S3 Read
S1
Read
S2
Read
S3
Read
S1
Read
S2
Read
S3
Read
S1
Read
S2
Read
S3
Read
S2
Read
S3
Read
S1
Read
S1
Read
S2
Read
S3
Read
S1

View Slide

@armeria_project
… until Shard 2 ruins it.
Shard 1
Shard 2
Shard 3
A bunch of
clients
Thread 1
Thread 2
Thread 3
Thread 4
Read
S1
Read
S2
Read
S3 Read
S1
Read
S2
Read
S3
Read
S1
Read
S2
Read
S3
Read
S1
Read
S2
Read
S3
Read
S2
Read
S3
Read
S1
Read
S1
Read
S2
Read
S3
Read
S1
Timeout!

View Slide

@armeria_project
Shard 1 & 3: Why are no requests coming?
Workers: We’re busy waiting for Shard 2.
Shard 1
Shard 2
Shard 3
A bunch of
clients
Thread 1
Thread 2
Thread 3
Thread 4
Read
S1
Read
S2
Read
S3 Read
S1
Read
S2
Read
S3
Read
S1
Read
S2
Read
S3
Read
S1
Read
S2
Read
S3
Read
S2
Read
S3
Read
S1
Read
S2
Read
S2
Read
S2
Read
S2
Timeouts!
Timeouts!
Timeouts!
Timeouts!

View Slide

@armeria_project
How can we solve this?
● Add more CPUs?
– They are very idle.
● Add more threads?
– They will all get stuck with Shard 2 in no time.
– Waste of CPU cycles – context switches are not free.
– Waste of memory – each thread needs its own stack.
● Result:
– Fragile system that falls apart even on a tiny backend failure
– Inefficient system that takes more memory and CPU than what it actually needs
● Maybe you can work around this, but you’ll have to keep tuning and adding hacks.

View Slide

@armeria_project
What is Armeria?

View Slide

@armeria_project
Problems with large payloads
● We solved blocking problem with asynchronous programming,
but can we send 10MB personalized response to 100K clients?
– Can’t hold that much in RAM – 10MB × 100K = 1TB
● What if we/they send too fast?
– Different bandwidth & processing power
● We need ‘just enough buffering.’
– Expect OutOfMemoryError otherwise.

View Slide

@armeria_project
What is Armeria?

View Slide

@armeria_project
RPC vs. HTTP impedance mismatch
● RPC has been hardly a 1st-class citizen in web frameworks.
– Method name
– Parameter names and values
– Return value or exception
POST /some_service HTTP/1.1
Host: example.com
Content-Length: 96

HTTP/1.1 200 OK
Host: example.com
Content-Length: 192

Failed RPC call

View Slide

@armeria_project
Access logs re-imagined
● What if we provide access logs with …
– Universal API for gRPC · Thrift · REST
– RPC-level information
– Precise send · receive timings
● Imagine streaming to data processing pipeline, e.g. Kafka.
192.167.1.2 - - [10/Oct/2000:13:55:36 -0700]
"POST /some_service HTTP/1.1" 200 2326

View Slide

@armeria_project
Making a debug call
● Sending an ad-hoc query in RPC is hard.
– Find a proper service definition, e.g. .thrift or .proto files
– Set up code generator, build, IDE, etc.
– Write some code that makes an RPC call.
● HTTP in contrast:
– cURL, telnet command, web-based tools and more.
● There’s some effort to make things easier.
– gRPC reflection service + grpcurl
● What if we build something more convenient and collaborative?

View Slide

@armeria_project
More than one RPC protocol
● Supports both gRPC & Thrift …
– on a single HTTP port
– in a single JVM
– without any proxies?
● Can serve non-RPC requests together.
– REST API – Static files
– Exposing metrics – Health-check requests from load balancers
– Traditional JEE webapps
● Share common logic between different endpoints!

View Slide

@armeria_project
What is Armeria?

View Slide

@armeria_project
Better UX for devs
● The last yet most important
● Nobody wants a framework that’s hard to use.
● Can we design …
– a user-friendly asynchronous API
– … with modest learning curve?

View Slide

@armeria_project
Our answer – Armeria
● Asynchronous + High-performance
– Netty with JNI-based /dev/epoll transport and BoringSSL
● Seamless integration with Reactive Streams
– RxJava2, Project Reactor, Spring WebFlux…
● Ability to mix different types of services in a single server
– gRPC, Thrift, REST, Tomcat, Jetty…
● Not just HTTP/2
– gRPC on HTTP/1
– HAProxy
● Easy to learn and fun to use (hopefully)

View Slide

@armeria_project
Armeria by examples
HTTP · REST

View Slide

@armeria_project
Hello, world!
Server server = new ServerBuilder()
.http(8080)
.https(8443)
.tlsSelfSigned()
.haproxy(8080)
.service("/", (ctx, req) -> HttpResponse.of("Hello, world!"))
.build();
server.start();
Protocol auto-detection at 8080

View Slide

@armeria_project
Hello, world – Parameterized
.http(8080)
.service("/hello/:name",
(ctx, req) -> HttpResponse.of("Hello, %s!",
ctx.pathParam("name")))
.build();
server.start();

View Slide

@armeria_project
Hello, world – Annotated
.http(8080)
.annotatedService(new Object() {
@Get("/hello/:name")
public String hello(@Param String name) {
return String.format("Hello, %s!", name);
}
})
.build();
server.start();

View Slide

@armeria_project
There are more!
●
CompletableFuture and reactive responses
● Query parameters and cookies
● Request and response converters
● Exception handlers
● Content type negotiation
● Regular-expression path mapping
● Aims to be on-par or better than

View Slide

@armeria_project
Armeria by examples
Thrift · gRPC

View Slide

@armeria_project
gRPC & Thrift basics
a. Write an IDL file – .proto or .thrift
b. A compiler compiles it into .java files.
c. Implement interfaces generated at ‘b’.
namespace java com.example
service HelloService {
string hello(1:string name)
}
syntax = "proto3";
package grpc.hello;
option java_package = "com.example";
service HelloService {
rpc Hello (HelloRequest)
returns (HelloReply) {}
}
message HelloRequest {
string name = 1;
}
message HelloReply {
string message = 1;
}

View Slide

@armeria_project
gRPC
.http(8080)
.service(new GrpcServiceBuilder().addService(new MyHelloService())
.build())
.build();
class MyHelloService extends HelloServiceGrpc.HelloServiceImplBase {
@Override
public void hello(HelloRequest req, StreamObserver cb) {
HelloReply reply = HelloReply.newBuilder()
.setMessage("Hello, " + req.getName() + '!')
.build();
cb.onNext(reply);
cb.onCompleted();
}
}

View Slide

@armeria_project
Thrift
.http(8080)
.service("/hello", THttpService.of(new MyHelloService()))
.build();
class MyHelloService implements HelloService.AsyncIface {
@Override
public void hello(String name, AsyncMethodCallback cb) {
cb.onComplete("Hello, " + name + '!');
}
}

View Slide

@armeria_project
What’s better than the official impls?
● gRPC
– Works on both HTTP/1 and 2
– gRPC-Web support, i.e. can call gRPC services from JavaScript frontends
● Thrift
– HTTP/2
– TTEXT (human-readable REST-ish JSON)
● Can mix gRPC, Thrift, REST, Tomcat, Jetty, …
● Can leverage the common Armeria features
– Decorators
– Web-based dashboard

View Slide

@armeria_project
Decorating services
Separation of concerns
a.k.a. filter, decorator, interceptor, middleware

View Slide

@armeria_project
Intercepting an HTTP request
THttpService.of(new MyHelloService())
.decorate((delegate, ctx, req) -> {
logger.info("{} Received a request", ctx);
HttpResponse res = delegate.serve(ctx, req);
res.completionFuture().thenAccept(unused -> {
logger.info("{} Sent a response", ctx);
});
return res;
})
● Can’t access RPC information yet.
● Need to decompose THttpService into:
– ThriftCallService – RPC layer
– THttpService – HTTP layer

View Slide

@armeria_project
Intercepting an RPC request
● Complete access to the RPC call information
● Override the parameters and methods
ThriftCallService.of(new MyHelloService())
.decorate((delegate, ctx, req) -> {
logger.info("{} Method: {}, Params: {}",
ctx, req.method(), req.params());
RpcResponse res = delegate.serve(ctx, req);
res.thenAccept(value -> {
logger.info("{} Result: {}", ctx, value);
});
return res;
})
.decorate(THttpService.newDecorator())

View Slide

@armeria_project
Decorator as core extension mechanism
● Request throttling
● Metric collection
● Distributed tracing (Zipkin)
● HTTP content encoding
● HTTP authn/z
● CORS policies
● Circuit breakers
● Automatic retries
● …

View Slide

@armeria_project
Distributed tracing with Zipkin
import brave.Tracing;
import brave.sampler.Sampler;
.http(8080)
.service("/my_service",
myService.decorate(
BraveService.newDecorator(
Tracing.newBuilder()
.currentTraceContext(…)
.localServiceName("myService")
.spanReporter(spanReporter)
.sampler(Sampler.create(/* 5% */ 0.05f))
.build())))
.build();

View Slide

@armeria_project
Playing better with RPC
Armeria documentation service

View Slide

@armeria_project
Armeria documentation service
● Enabled by adding DocService to ServerBuilder
● Browse and invoke RPC services in an Armeria server
– No fiddling with binary payloads
– Send a request without writing code
● Supports gRPC, Thrift and annotated services
● We have a plan to add:
– Metric monitoring console
– Runtime configuration editor, e.g. logger level

View Slide

@armeria_project

View Slide

@armeria_project
● Share the URL to reproduce a call.

View Slide

@armeria_project
All pieces together
The ultimate all-in-one micro service

View Slide

@armeria_project
ServerBuilder sb = new ServerBuilder();
// Thrift services
sb.service("/thrift/foo", THttpService.of(myThriftFooService));
sb.service("/thrift/bar", THttpService.of(myThriftBarService));
// gRPC services
sb.service(new GrpcServiceBuilder().addService(myGrpcFooService)
.addService(myGrpcBarService)
.build());
// Static files
sb.service("prefix:/files", HttpFileService.forFileSystem("htdocs"));
// Legacy webapps
sb.service("prefix:/webapp/foo", TomcatService.forFileSystem("foo.war"));
sb.service("prefix:/webapp/bar", TomcatService.forFileSystem("bar.war"));
// Documentation service
sb.service("prefix:/internal/docs", new DocService());
// L7 health check service
sb.service("/internal/healthcheck", HealthCheckService.of());
// Prometheus exposition
sb.service("/internal/prometheus", new PrometheusExpositionService(...));
// Enable logging for all services.
sb.decorator(LoggingService.newDecorator());
// Enable distributed tracing for all services.
sb.decorator(BraveService.newDecorator(...));
// Send all access logs to Kafka.
sb.accessLogWriter(new KafkaAccessLogWriter(...), true);

View Slide

@armeria_project
What about client-side?

View Slide

@armeria_project
Armeria client API
● Similar experience to the server-side API
– e.g. Common types, Decorators, RPC-friendliness
● Client-specific stuff
– Circuit breakers
– Automatic retries
– Client-side load balancing
– Retrofit integration – https://square.github.io/retrofit/

View Slide

@armeria_project
// Send DNS queries periodically to discover service hosts (k8s style).
// Send health check requests to /internal/healthcheck and remove dead hosts.
EndpointGroup group = HealthCheckedEndpointGroup.of(
DnsServiceEndpointGroup.of("my-service.cluster.local"),
"/internal/healthcheck");
// Register the group into the registry.
EndpointGroupRegistry.register("myService", group,
EndpointSelectionStrategy.WEIGHTED_ROUND_ROBIN);
// Create a new HTTP client that connects to the group
// with retries, circuit breaker and logging.
HttpClient client = new HttpClientBuilder("http://group:myService")
.decorator(RetryingHttpClient.newDecorator(onServerErrorStatus()))
.decorator(CircuitBreakerHttpClient.newDecorator(...))
.decorator(LoggingClient.newDecorator())
.build();
// Send a request and print the response content.
client.get("/greet").aggregate()
.thenAccept(res -> System.out.println(res.contentUtf8()))
.exceptionally(cause -> { /* Handle an error. */ });

View Slide

@armeria_project
Got interested?

View Slide

@armeria_project
Let’s build Armeria together!
● Give it a try.
● Ask questions.
● Request new features.
● Tell us what rocks and sucks.
● Consider joining the effort.

View Slide

@armeria_project
Meet us at GitHub
github.com/line/armeria
line.github.io/armeria

View Slide

Armeria: The Only Thrift, gRPC, REST Microservice Framework You'll Need

Armeria: The Only Thrift, gRPC, REST Microservice Framework You'll Need

Trustin Lee

More Decks by Trustin Lee

Other Decks in Programming

Featured

Transcript