初めてのgRPC / Starting gRPC

, LINE
gRPC
JJUG CCC 2019 Spring #ccc_a1 1

View Slide

• LINE

• (Maeda Kentaro)
• Twitter: kencharos

View Slide

Agenda • gRPC
• Protocol Buffers
• gRPC Java
• gRPC

View Slide

&"
l
!% gRPC '
#($ )
4
JJUG CCC 2019 Spring #ccc_a1

View Slide

Agenda • gRPC
• Protocol Buffres
• gRPC Java
• gRPC

View Slide

0 / I ) / / ) / /
F P
l oil
l epC ) rc
l 2 / R P N F
nm
l ( F 1 0 / C
l 0 /t u g bj da F
6
0 ) CG
https://grpc.io/

View Slide

l e > p
• - I >c A
B T
• > tPT P c DH
o B >gp I
• D >c DL I BD C
B C
l u
• + ( 2 s
• L / )r
l f l > RAL g I
DH T
7
4/ )

View Slide

l ) 1 / 2
• ( m UT RU C C
l S t C
• a
• UTe a
• L P
l n H i
• s C L
P c P C i
8

View Slide

) PR o i hli
fi
l ) I R
nH SR
l o rLR+( e g e
l +( H tuwJ a yG ld i
• ,1B1# ) # # # D# # # D # 0C ###
l ptuwJ i i fi
• s c b LR v
9

View Slide

l I L P
C D
l I
R B
10
proto file
message
( )
service
(PRC )
protoc
Protocol Buffers
gRPC plugin

Message

Service

Protocol Buffers

gRPC


View Slide

l e / UWT N
l . gab UWT p tP
• P ( . / P
l / A H NPL
T A J
l L . / P . UWT w
E wEL . / . r LC
• A . R O SI
• . . / )/ + L o T
11
. / ( / P n

View Slide

Agenda • gRPC
• gRPC Java
• gRPC

View Slide

l proto"&A5!<;3CLI &
• https://github.com/protocolbuffers/protobuf
l #%'8F$,B
• gRPCOpen API12/B?+ E
l gRPC Java :4protoc gradle/maven [email protected]
6
)7 9
• (*:4>=6
• gRPC-web, gRPC-gateway, doc-gen Java(-#%'
6:4
• Go, C++ C12!<; :4
13
protoc

View Slide

l gRPC version 3
l
message, RPC
service
!
• packageOptionimport "

!

14
proto
syntax = "proto3";
package my.rpc;
option java_package = "my.rpc";
message SomeMessage{///}
service SomeService{///}

View Slide

l Java 1!'

message 0
8.5%*

l Java1!3249*
15
message
message Person {
int32 id = 1;
string name = 2;
Gender gender = 3;
enum Gender {
man = 0;
woman = 1;
}
}

-/ int32, int64, uint32, uint64,double,
, bool
6(: string
: bytes
:"$ enum
#7 repeated $.
)+ map
message 1!&message

View Slide

l
Protocol Buffers ('.I7+/&H8?K
= L9
• +/&,#*0 .-!/$ .-!
36

• +/&H8:NM

l EA Person id150DG
,
08 96 01 (%P
16

0000 0100
212)#%[email protected]@K
<+/&H8 = 1
1001 0110 0000 0001
;C1)#%$"QF>

21(%J4B
000 0001 ++ 001 0110
-> 10010110
-> 128 + 16 + 4 + 2 -> 150

View Slide

l proto
#%C?,#%!A2 D8

• " $&'
proto
<;!:3
0G

•
message
/=B.#%!A2>)14F
E1
• #%!A26E-79

reserved

17

//version1
message Something {
int32 id = 1;
string name = 2;
}
//version2
*@+
message Something {
int32 id = 1;
string name = 2[deprecated=true];
string fullName = 3;
}
//version3
(15
message Something {
int32 id = 1;
reserved 2;
string fullName = 3;
}

View Slide

l
Java service &
RPC
' / ,$"message-%message
(
l *"' / stream "-% ).

RPC 40 +0
l !#
18
service
service GreetingService {
rpc Hello(Person) returns (Greeting)
rpc HelloStream(stream Person) returns (stream Greeting)
}

View Slide

Agenda • gRPC
• gRPC Java
• gRPC

View Slide

20
proto
syntax = "proto3";
// The fizbuz service definition.
service FizBuzService {
rpc FizBuzOne (InputNumber) returns (FizBuzAnswer) {}
rpc FizBuzRange (FromTo) returns (stream FizBuzAnswer) {}
rpc FizBuzBatch (stream InputNumber) returns (FizBuzList) {}
rpc FizBuzMany (stream InputNumber) returns (stream FizBuzAnswer) {}
}
message InputNumber {
int32 num = 1;
}
message FizBuzAnswer {
string answer = 1;
}
//

View Slide

21
Protocol Buffers
l maven/gradle Protocol Buffers
gRPC plugin
$
• src/main/proto proto"
apply plugin: 'com.google.protobuf'
apply plugin: "idea"
protobuf {
protoc {
artifact = "com.google.protobuf:protoc: 3.6.1 "
}
plugins {
grpc { // grpc plugin #
artifact = "io.grpc:protoc-gen-grpc-java:1.18.1"
}
}
generateProtoTasks { //plugin
"
all()*.plugins {
grpc {} // grpc plugin !$
}
}
ProtocolBuffrers
gRPC plugin#
gRPC plugin!$

View Slide

22

l
build/generated

l
gradle

• https://github.com/google/pr
otobuf-gradle-plugin

View Slide

23
gRPC Java

Stub RPCproto'5
.-
gRPC Server Stub /12
gRPC Sevice gRPC Server
& RPCProto
'5.-%!3 Server 4:
Channel gRPC Server /1
7,RPC$)
#8

message PRC*96
Stub-gRPC Service "
0(+
Protocol Buferes message

View Slide

24
gRPC Java

Stub
Stub
gRPC Server
gRPC
Service
gRPC
Service
Channel
RPC
RPC
message
message
l
Proto

View Slide

25
:gRPC Server
l ServerBuilder addService GrpcService&+
'
• !$ Netty TLS #%

NettryServiceBuilder
()
• RPC"* Netty NIO !
public static void main(String[] args)throws Exception{
var builder = (NettyServerBuilder)ServerBuilder
.forPort(6565)
.addService(new FizBuzBasicGrpcService());
// builder.sslContext(...);
var server = builder.build();
server.start();
server.awaitTermination();
}

View Slide

26
:Channel
l ManagedChannel * 9
l -
5)8< RPC#3
.'
2!1 * ;
l gRPC TLS 4%, TLS +/
& usePlaintext :"0$76
.(
ManagedChannel createChannel() {
return ManagedChannelBuilder.forAddress("localhost", 6565)
.usePlaintext()
.build();
}

View Slide

27
:Stub
l 1;65 ServiceGrpc (33C $65'
!" Channel : 65

l 0B-=RPC%*(&[email protected],
• 942/+ BlockingStub FutureStub0
var blockStub = FizBuzServiceGrpc.newBlockingStub(channel);
var futureStub = FizBuzServiceGrpc.newFutureStub(channel);
var stub = FizBuzServiceGrpc.newStub(channel);
Stub RPC
BlockingStub unary,
server side stream
<.('!A8)
FutureStub unary ><. (Future A8)
Stub 7 StreamObsever)#!

View Slide

28
RPC
l
RPC6# .;3 stream RPC(=
$
l !RPC4:.87"
0),

• 5+ Unary& 1
• 87"*<9-'/:%
RPC RPC
Unary RPC rpc Hello(In) returns (Out) 121
Server Streaming RPC rpc Hello(In) returns (stream Out) 121
Client Streaming RPC rpc Hello(stream In) returns (Out) 121
Bidirectional Streaming RPC rpc Hello(stream In) returns (stream Out) 121

View Slide

29
RPC

l %)-,/!1TRPCZI6(&NCH
:
• (&NCH(&NHD (=RPC
Da)L[[email protected]^
• 4TWJRH>`3 XU8E_
".*' Y^
l
gRPC Java StreamObserver +$ )]
[email protected];
• Stub, #*b[RPCQ5SG

• RPC%&+Bc gRPC E_2d
AM7O
• <=
A;V\0

View Slide

30
StreamObserver
l
2StreamObserver
Stub gRPC
Service
RPC(Unary)
Stream
Observer
Stream
Observer
message
message
onNext
onCompleted
onNext
onCompleted

View Slide

31
StreamObserver
l onNext : .+0/(+!/
• $2 *#
-& .+
• -& *#
$2 (+
• )3
1,%"
l onCompleted : .+ 4 0/!/
• %"
RPC '5
l onError : !/

View Slide

32
Unary RPC
l &*"! +;
#
! +:
• 84'")%*/0 1
.2
• &*"7 StreamObserver gRPC!$("36
9- 36,8
Stub gRPC
Service
RPC5<
Stream
Observer
Stream
Observer
message
message
onNext
onCompleted
onNext
onCompleted

View Slide

33
Unary RPC
// Client
InputNumber input = InputNumber.newBuilder().setNum(1).build();
FizBuzAnswer a1 = blockStub.fizBuzOne(input);
System.out.println(a1.getAnswer());
// Server Unary
@Override
public void fizBuzOne(InputNumber request,
StreamObserver responseObserver) {
String answer = calc.calc(request.getNum());
// send response to client
responseObserver.onNext(FizBuzAnswer.newBuilder().setAnswer(answer).build());
// send RPC finish to client
responseObserver.onCompleted();
}

View Slide

34
()message POJO
l Protocol Buffers message 67(immutable)
getter setter ,

l message (4 builder +9
l message AbstractMessage %&-"!
/"!%/8*. ,
var num = InputNumber.newBuilder() // builder )1
.setNum(10) // #25
.build(); // message (4
var num2 = InputNumber.newBuilder(num) // $3 builder )1
.setNum(100) // '0
.build(); // (4

View Slide

35
Server Streaming RPC
l
* (&
)
• Unary onNext(&!#
• BlockingStub Iterator " $'
Stub gRPC
Service
RPC%+
Stream
Observer
Stream
Observer
message
message
onNext
onCompleted
onNext
onCompleted
message
onNext

View Slide

Iterator a2 =
blockStub.fizBuzRange(FromTo.newBuilder().setFrom(2).setTo(20).build());
while (a2.hasNext()) {
System.out.println(a2.next().getAnswer());
}
// server side stream
@Override
public void fizBuzRange(FromTo request, StreamObserver observer) {
var anserList = IntStream.range(request.getFrom(), request.getTo())
.boxed().map(i -> calc.calc(i)).collect(Collectors.toList());
// send stream data
for (String answer : anserList) {
observer.onNext(FizBuzAnswer.newBuilder().setAnswer(answer).build());
}
observer.onCompleted();
}
36
Server Streaming RPC

View Slide

37
Client Streaming RPC
l ($&&"

)
• !%'
Stub gRPC
Service
RPC#*
Stream
Observer
Stream
Observer
message
onNext
onCompleted
onCompleted
message
onNext
message
onNext

View Slide

38
Client Streaming RPC(Stub)
// 32"(5#- Latch
CountDownLatch waitFinish = new CountDownLatch(1);
// RPC %&1 StreamObserver
+/
var requestSender = stub.fizBuzBatch(new StreamObserver() {
@Override
public void onNext(FizBuzList value) {
// 1Client Streaming
0
System.out.println(“received answers” + value);
}
@Override
public void onCompleted() {
waitFinish.countDown(); // !6
'6
}
@Override
public void onError(Throwable t) {waitFinish.countDown();}
});
// ,)4*$
requestSender.onNext(InputNumber.newBuilder().setNum(11).build());
// ,)'6.
requestSender.onCompleted();
// RPC'6
waitFinish.await();
,)

1

View Slide

39
Client Streaming RPC(Server)
@Override
public StreamObserver fizBuzBatch(StreamObserver
responseObserver) {
// !$

var inputNumbers = new ArrayList();
// "
return new StreamObserver() {
@Override
public void onNext(InputNumber value) {
// $

inputNumbers.add(value);
}
@Override
// StubonComplete#
var answers = inputNumbers.stream().map(i -> i.getNum())
.map(i -> calc.calc(i)).collect(toList());
responseObserver.onNext(FizBuzList.newBuilder().addAllAnswer(answers).build());
}
@Override
public void onError(Throwable t) {t.printStackTrace();}
};
}

%
!

View Slide

40
Bidirectional Streaming RPC
l "# !:5604

• Ping-Pong $8$%9)*2/
6+
• onCompleted (RPC'7
3.&
-,;
Stub gRPC
Service
RPC1<
Stream
Observer
Stream
Observer
message
onNext
onCompleted
onCompleted
message onNext
message
onNext
message onNext

View Slide

@Override
public StreamObserver
fizBuzMany(StreamObserver responseObserver) {
return new StreamObserver() {
@Override
public void onNext(InputNumber value) {
// one by one data send.
var answer = calc.calc(value.getNum());
responseObserver.onNext(
FizBuzAnswer.newBuilder().setAnswer(answer).build());
}
@Override
}
@Override
public void onError(Throwable t) {
t.printStackTrace();
}
};
}
42
Bidirectional Streaming RPC(Server)

View Slide

43
gRPC
l API ,<9Request-Response
Unary ,<
• 04 *7
l
Streaming API=-
.:

• #6? ;)13
• %'!6? ;)53?
• #2>
/+2>$"&'?
l
Streaming 2> deadline 8(

View Slide

44
deadline
l
l Stub -+1deadline
6 2*
• RPC'5,." 0&!'5$(% 0
+1
l Server - deadline)#/

// deadline 100 4+1
var blockStub = FizBuzServiceGrpc.newBlockingStub(channel)
.withDeadlineAfter(100, TimeUnit.MICROSECONDS);
// Server - deadline 31
if(Context.current().getDeadline().isExpired()) {
responseObserver.onError(new RuntimeException());
}

View Slide

45
(
)Stream
l Reactive-gRPC(https://github.com/salesforce/reactive-grpc)
• Salesforce gRPC4HRxJava/Reactor ; Stub,
Service $)BA
• StreamObserver [email protected]$
)BA'%( D ."
' + -!+-.78
• 1K."' +N3 gRPC :>N2O
J9M #%*!#%
l gRPC-kotlin (https://github.com/rouzwawi/grpc-kotlin)
• $/&0;M$)BA
• Stream >NE=/,G5$)?

View Slide

Agenda • gRPC
• gRPC Java
• gRPC

View Slide

l
API Gateway API
#!
l ! REST API Gateway REST-gRPC
"
l ! gRPC
l proto
%$
50

Service
Service
Service
API
Gateway
Browser
REST
gRPC
proto
proto
proto

View Slide

l gRPC &*"$
• %+ (,
!)
•
• Protocol Bufferes POJO gRPC #

'
51

Domain
Logic
gRPC
Service
Web
Controller
service
message
queue
gRPC
Stub
JDBC
Other service
Database
Kafka
REST
Other service
Web
Client
External
WebAPI

View Slide

2 A . I
. I u b
l p r t
l 2 s . I dl A
i go s
• 1 1
• 2 AP em
52
. I

View Slide

l - S CmsC PR B R C h gn
S P CmsC
• B iptb cotBr t P
l . :: - S R D
• : : :: /
• : 2: / :: /
• S : 2 P G vI L
l S R
• - . @ . @ N . / P
B B td cot
• . :: Nu - aC C y
• - . @ t Cel CR
53
. :: -

View Slide

54
// GrpcService Spring Bean
@GRpcService(interceptors = ValidationInterceptor.class)
public class FizBuzReactorGrpcService extends
ReactorFizBuzServiceGrpc.FizBuzServiceImplBase {
// Spring Bean

@Autowired
private FizBuzCalc cal;
@Override
public Mono fizBuzOne(Mono request) {
return request.map(InputNumber::getNum)
.map(cal::calc)
.map(a -> FizBuzAnswer.newBuilder().setAnswer(a).build());
}

View Slide

l E /
G C
55
syntax = "proto3";
import "google/api/annotations.proto";
// The fizbuz service definition.
service FizBuzService {
// calculate Fizbuz Answer
rpc FizBuzOne (InputNumber) returns (FizBuzAnswer) {
option (google.api.http) = {
get: "/fizbuz/{num}"
};
}
rpc FizBuzRange (FromTo) returns (stream FizBuzAnswer) {}
}
http

Google api

proto

RPC

http

REST

View Slide

l - 2 B https://github.com/grpc-ecosystem/grpc-
gateway S 1 2 Rc ePurR
w H fgR a tsPJ
2
R ). - h a
• ( j N R aSiT RPC
ySp
• ) R aSp v Slo
• . R a tsPJ HnGO
56
- 2 B

View Slide

l CI2 GO t Ta
tS
• R RG t S
l Gg tS u 2
aGO t
• Ch R 2 PlL C
l a t CI2 a
n L
• a C t PlL 2
S Ae lL C
59
t

View Slide

l VP K
• R l
• R TP
l V u C- a g
VP K -
l V l e H
60
V

View Slide

l B B o B
• Pe a BPe v
• np g
l S P e i
• S P e C u
• C r
• ,, C ce i
l Bv t B b B
R b
61
Pe a

View Slide

l

62

var stub = FizBuzServiceGrpc.newBlockingStub(ch).withInterceptors(
new ClientInterceptor() {
public ClientCall interceptCall(
MethodDescriptor method,
CallOptions callOptions, Channel next) {
var call = next.newCall(method, callOptions);
return new ForwardingClientCall.SimpleForwardingClientCall<>(call){
@Override
public void start(Listener responseListener, Metadata headers) {
//
headers.put(Metadata.Key.of("cred",
Metadata.ASCII_STRING_MARSHALLER), "password" );
super.start(responseListener, headers);
} }; }});

View Slide

l

63

public class AuthenticateInterceptor implements ServerInterceptor {
@Override
public Listener interceptCall(ServerCall call, Metadata headers,
ServerCallHandler next) {
var cred = headers.keys().stream().filter(k -> k.equals(“cred”)).findFirst().get(); // Metadata &
boolean ok = cred.equals(“password”); // '(,
return new ForwardingServerCallListener
.SimpleForwardingServerCallListener(next.startCall(call, headers)) {
@Override // %$ , )
public void onMessage(ReqT message) {
if (ok) { // '(OK,#
super.onMessage(message);
} else { // '(NG "+/
Status status = Status.UNAUTHENTICATED; // '(NG
var metadata = new Metadata();
metadata.put( // 0. -

Metadata.Key.of(“error_detail”, Metadata.ASCII_STRING_MARSHALLER), “cred invalid”);
call.close(status, metadata); // close RPC+/*! Status, Metadata "
}
}

View Slide

l R
• P R
• C
64
public Listener interceptCall(
ServerCall call, Metadata headers, ServerCallHandler next) {
var cred = headers.keys().stream().filter(k -> k.equals("cred")).findFirst().get();
//
var username = getUsrename(cred);
// # Context username % !
var newContext = Context.current().withValue(Context.key("username"), username);
// Context " $RPC
Context.get("username').get()
return Contexts.interceptCall(newContext, call, headers, next);
}

View Slide

l ) 1 . e PD )( ( J
. 1 u J
l O d P
e )( ( lD . 1 t
• 2 ai BPD p
) 1 . o D rD d
ai P
• BP Vfe DG m s P
• n ) https://github.com/envoyproxy/protoc-gen-validate/
c i JP D J
65
ai

View Slide

l R
l
l
66
// validator.proto
syntax = “proto3”;
package validator;
import “google/protobuf/descriptor.proto”;
//
range
extend google.protobuf.FieldOptions {
Range range = 50000;
}
// Range
message Range {
int32 min = 1;
int32 max = 2;
}
package my.rpc;
// proto import
import "validator/validator.proto";
message InputNumber {
//
Range
int32 num = 1 [(validator.range) = {min:0,
max:100}];
}

View Slide

l C
67
// AbstractMessage
%
public void checkSample(AbstractMessage message) {
// "$ !
Map fields = message.getAllFields();
fields.forEach((desc, val) -> {
// Range #' !
var rangeOpt = desc.getOptions().getAllFields().values().stream()
.filter(v -> v instanceof Range).map(r -> (Range)r).findFirst();
// Range &
rangeOpt.ifPresent(range -> {
if (range.getMin() <= (Integer)val && (Integer)val <= range.getMax()) {
System.out.println("OK");
}
});
});
}

View Slide

l e CH rH p vL H
I x
69
p v

i 4 nH C vo
U v
i
/2
n C
- L / G v o
X TR T
p v
X TR C
t sNP
/2
7
A H G T
k
y o

View Slide

l R TH G a SP u
l I H G I
• e 4 / 2P n G 2P C
R G C G
• l HhyL 2P w
l I L G AC
• 4 7 L U S b t nG
70
T a S

gRPC Server A
gRPC Server B

View Slide

l NR s n
Ev g u y D
l R s , ,
h Ek E
l llo Ee r SR n i
c CD PD
71
SR S a
gRPC Server A
gRPC Server B
DNS
Consul
Eureka,,
Channel

View Slide

l )
l v
)
• v )
no) (
• e
e
72
gRPC Server A
gRPC Server B

envoy
envoy
envoy
xDS

View Slide

l !!

API
Gateway
Browser
REST
gRPC
gRPC
gateway
proto
proto
proto
Spring boot
gRPC
envoy
envoy
Spring boot
gRPC
envoy
Spring boot
gRPC
envoy
Envoy

consul prometheus
zipkin elasticsearch
73
73

View Slide

l Google7<.=-4 gPRC >;6
8*9,0
l proto %( API % #+:&)
$ %!#
l "'

2/?5
31
74


View Slide

$/17. 2(*'
• https://grpc.io/
• https://github.com/grpc-ecosystem
• https://github.com/grpc/grpc-web
• https://developers.google.com/protocol-buffers/
• https://github.com/pseudomuto/protoc-gen-doc
• https://github.com/salesforce/reactive-grpc
• https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2.md
• https://github.com/envoyproxy/protoc-gen-validate/
0#49-)3!
• https://github.com/kencharos/grpc-sample
• gRPC%8gRPC-gateway, gRPC-Web, L4L7!5"6+
• https://github.com/kencharos/envoy-service-discovery-spring
• envoy6+ !kotlinconsul;&

75
*',:

View Slide

THANK YOU

View Slide

初めてのgRPC / Starting gRPC

初めてのgRPC / Starting gRPC

LINE Developers
PRO

More Decks by LINE Developers

Other Decks in Technology

Featured

Transcript

初めてのgRPC / Starting gRPC

初めてのgRPC / Starting gRPC

LINE Developers PRO

More Decks by LINE Developers

Other Decks in Technology

Featured

Transcript

LINE Developers
PRO