Microservices / gRPC

Transcript

1. Microservices / gRPC Oct 2017 @EVO Vladimir Magamedov social: vmagamedov
2. REST or RPC • REST for Internet-scale services • RPC for organization-scale services • GraphQL?
3. HTTP or not HTTP • REST: HTTP • ZeroRPC: MessagePack format • JSON-RPC: JSON format • Finagle: Finagle-Thrift, Mux, HTTP • gRPC: HTTP/2-only
4. Good parts HTTP is extensible: • content-type • cache • cookies • authentication • proxies
5. Good parts HTTP is widely used: • servers: apache, nginx • proxies: haproxy, varnish, squid • clients: browsers, curl • security: tls, oauth2 • libraries: for any language
6. Brief history 1991 HTTP birth … 8 years … June 1999, RFC 2616, HTTP/1.1, Internet Standard … 16 years … May 2015, RFC 7540, HTTP/2, Performance Improvement … 26 years in 2017, older than some people here :)
7. Performance changes • HTTP/1.1 • Keep-Alive • Pipelining • HTTP/2 • Headers compression • Multiplexing • Prioritization • Flow control
8. Keep-Alive • Fixes reconnection overhead • Requires connection pools
9. Pipelining
10. Multiplexing
11. Framing, streams • DATA • HEADERS • PRIORITY • RST_STREAM • SETTINGS • PUSH_PROMISE • PING • GOAWAY • WINDOW_UPDATE • CONTINUATION
12. HPACK • 61 entries in the static table • dynamic table uses first-in, first-out order
13. Flow control • To not affect other streams in the same connection • Only DATA frames are affected • Fixes problems with slow clients and slow servers • Can prevent some congestion events
14. Cancellation • Send RST_STREAM frame instead of disconnect
15. Prioritization • Stream can have dependencies and weight • It’s a tree structure • Optional feature, disabled/ignored by default
16. Deprecates • Domain sharding • CSS sprites?
17. Vulnerabilities • Stream Reuse crashes IIS • Slow Read using small window • Dependency Cycle in priority tree • HPACK Bomb by abusing dynamic headers table See: https://www.imperva.com/docs/Imperva_HII_HTTP2.pdf
18. Example: 1/2 def handle(sock): conn = h2.connection.H2Connection(client_side=False) conn.initiate_connection() sock.sendall(conn.data_to_send()) while True: data = sock.recv(65535) if not data: break events = conn.receive_data(data) for event in events: if isinstance(event, h2.events.RequestReceived): send_response(conn, event) data_to_send = conn.data_to_send() if data_to_send: sock.sendall(data_to_send) sock = socket.socket() sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) sock.bind(('0.0.0.0', 8080)) sock.listen(5) while True: handle(sock.accept()[0])
19. Example: 2/2 def send_response(conn, event): stream_id = event.stream_id response_data = json.dumps(dict(event.headers)).encode('utf-8') conn.send_headers( stream_id=stream_id, headers=[ (':status', '200'), ('server', 'basic-h2-server/1.0'), ('content-length', str(len(response_data))), ('content-type', 'application/json'), ], ) conn.send_data( stream_id=stream_id, data=response_data, end_stream=True ) More info: https://python-hyper.org/h2/en/stable/basic-usage.html
20. Now you ~know HTTP/2 nothing to worry about
21. Make it even better? • Fix TCP protocol
22. QUIC Quick UDP Internet Connections • Multiplexed connections over UDP • 0-RTT connectivity overhead for secure connections • Avoids all connection streams stalling when single packet was lost • Congestion avoidance algorithms in application space, instead of kernel space
23. Questions?
24. gRPC • RPC protocol over HTTP/2 protocol • Uses IDL to declare services • Strict interface == less human errors • Code generation for 10 languages (officially supported)
25. gRPC Request Response Example UNARY UNARY Simple call UNARY STREAM Download STREAM UNARY Upload STREAM STREAM Chat
26. gRPC Frames Request HEADERS Metadata DATA Message(s) … DATA Frames Response HEADERS Initial metadata DATA Message(s) … DATA HEADERS Trailing metadata
27. Wire protocol [HEADERS] :method POST :scheme http :path /helloworld.Greeter/SayHello te trailers content-type application/grpc+proto {custom metadata} [DATA] {message} EOS [HEADERS] :status 200 content-type application/grpc+proto {custom initial metadata} [DATA] {message} [HEADERS] grpc-status 0 {custom trailing metadata} EOS
28. New helloworld service: 1/5 Describe service in the helloworld.proto file • Language Guide: https://developers.google.com/protocol-buffers/docs/proto3 • Style Guide: https://developers.google.com/protocol-buffers/docs/style syntax = "proto3"; package helloworld; service Greeter { rpc SayHello (HelloRequest) returns (HelloReply) {} } message HelloRequest { string name = 1; } message HelloReply { string message = 1; }
29. New helloworld service: 2/5 Generate target language stub files using protoc compiler •helloworld_pb2.py will contain message classes •helloworld_grpc.py will contain server-side and client-side classes $ tree . . └── helloworld ├── __init__.py └── helloworld.proto $ python3 -m grpc_tools.protoc -I. --python_out=. \ --python_grpc_out=. helloworld/helloworld.proto $ tree . . └── helloworld ├── __init__.py ├── helloworld.proto ├── helloworld_pb2.py └── helloworld_grpc.py
30. New helloworld service: 3/5 Implement service methods by subclassing GreeterBase class import asyncio from grpclib.server import Server from .helloworld_pb2 import HelloReply from .helloworld_grpc import GreeterBase class Greeter(GreeterBase): async def SayHello(self, stream): request = await stream.recv_message() message = 'Hello, {}!'.format(request.name) await stream.send_message(HelloReply(message=message))
31. New helloworld service: 4/5 Package compiled files into helloworld-proto Python package and upload it to the local PyPI server $ cat setup.py from setuptools import setup, find_packages setup( name='helloworld-proto', version='0.1.0', description='HelloWorld service stub files', packages=find_packages(), ... ) $ python setup.py sdist $ twine upload -r pypi.uaprom dist/helloworld-proto-0.1.0.tar.gz
32. New helloworld service: 5/5 Call helloworld service using GreeterStub class import asyncio from grpclib.client import Channel from .helloworld_pb2 import HelloRequest from .helloworld_grpc import GreeterStub async def make_request(): channel = Channel(loop=asyncio.get_event_loop()) stub = GreeterStub(channel) return await stub.SayHello(HelloRequest(name='World'))
33. Package? syntax = "proto3"; package helloworld; ^^^^^^^^^^ service Greeter { rpc SayHello ... -> /helloworld.Greeter/SayHello ^^^^^^^^^^ } message HelloRequest ... -> helloworld.HelloRequest ^^^^^^^^^^ message HelloReply ... -> helloworld.HelloReply ^^^^^^^^^^
34. Options? --python_out • to generate *_pb2.py files --grpc_python_out • to generate *_pb2_grpc.py files and using grpcio library --python_grpc_out • to generate *_grpc.py files and using grpclib library
35. Files structure? • Proto-files location is significant for imports and generated files location
36. Implementations • grpc/grpc-java • grpc/grpc-go • grpc/grpc: C++, Node.js, Python, Ruby, Obj-C, PHP, C# • bindings to the C-based core • vmagamedov/grpclib: pure-Python, asyncio-friendly
37. Protocol Buffers Working with generated code in Python: https://developers.google.com/ protocol-buffers/docs/reference/python-generated Google examples: https://github.com/googleapis/googleapis/tree/master/google Search well-known types before implementing yours Proto3 supports a canonical encoding in JSON
38. Metadata • Sent as headers and trailers • Used for authentication • Used for deadline propagation • Used for distributed tracing
39. Deadlines • Deadline - fixed point in time • Actually sent as timeout, no need for clocks synchronization A B C timeout: 100ms timeout: 100ms -80ms -50ms A B C timeout: 100ms timeout: 20ms -80ms -20ms
40. Distributed Tracing Opentracing: http://opentracing.io, Zipkin: http://zipkin.io
41. Circuit Braker • Prevents cascading failures • Fail fast and rapid recovery • Saves resources
42. Routing dev A B C trunk C local dev cluster dev machine Browser
43. Routing prod A B C stable C staging production cluster 90% 10% Retry on error
44. Load Balancing Service Discovery • DNS • unreliable for dynamic services • can’t manage ports • Specialized proxy (Linkerd: https://linkerd.io) • load balancing with sophisticated strategies • tracing • retries • routing, versions, A/B testing • circuit braker • first-class HTTP/2 and gRPC support • nice service discovery only by name: • hello.dev-cluster.uaprom:50051 • https://gitlab.uaprom/service/mesh/blob/master/example/hello.proto • Built into client LB and SD • brings complexities from infrastructure into services
45. http://dev-cluster.uaprom:9990
46. http://dev-cluster.uaprom:9411 http://dev-cluster.uaprom:8500
47. Hiku + gRPC: 1/4 gRPC • Binary format • Simple protocol • Initially supports streaming • Can handle any content, incl. large files upload/download Hiku • reads/writes query encoded with Protocol Buffers • serializes result into Protocol Buffers message
48. Hiku + gRPC: 2/4 syntax = 'proto3'; package uaprom.product; message Product { sint64 id = 1; string description = 2; } message Root { Product product_by_id = 1; }
49. Hiku + gRPC: 3/4 GRAPH = Graph([ Node('Product', [ Field('id', Integer, product_sg), Field('description', String, product_sg.c( if_some([S.x, S.this.ext], S.x.description, '') )), ]), Root([ Link('product_by_id', Optional[TypeRef[‘Product']], product_by_id, requires=None, options=[Option('id', Integer)]), ]), ])
50. Hiku + gRPC: 4/4 from hiku.readers.protobuf import transform from hiku.writers.protobuf import populate # receive pb_query query = transform(pb_query) result = hiku_engine.execute(GRAPH, query) pb_result = Root() populate(pb_result, GRAPH, result, query) # reply pb_result More info: http://hiku.readthedocs.io/en/latest/protobuf.html
51. Questions?