Boosting Python web apps with Protocol buffers abd GRPC

Slides of my recent talk at PyCon India 2017

Transcript

1. Boosting Python Web Apps with Protocol Buffers & Naren Yellavula @Citrix R&D @Narenarya3
2. $ whoami • Software Engineer 2 @ Citrix R&D India • Blogger @Impythonist & Medium • Pythonista, Gopher and Open Source Lover A programmer with 4 eyes since 4th class
3. Why JSON is good? JSON is good because • It is simple • It looks like JavaScript object/Python dict • People cannot take XML any more • REST popularity than SOAP(XML) I am a JSON Fan Boy! Who not?
4. The world of Monolith
5. The world of Micro services
6. Almighty JSON? JSON is not useful where: • Readability doesn’t makes any sense and Performance matters. • A standard API contract between machines • No type safety It is not a silver bullet
7. What are protocol buffers? (Part 1) Just a format which looks exactly like Go structs Protobuf Go struct Python Class syntax ‘proto3’; message Person { string name = 1; int32 age = 2; } type Person struct { Name string Age int32 } class Person(object): __init__(self): self.name = “” self.age = “”
8. What are protocol buffers? (Part 2) • Serialization & Deserialization of data • Simpler, Smaller & Faster • Has data types like Enum and Services • Better maintainability
9. 30-40 % lesser size means lesser latency Can you prove it?
10. Installing Protobuf tool & Python library • Install protoc compiler on any machine. Do configure and make install from executable downloaded here. https://github.com/google/protobuf/releases/latest • Now install python library through pip $ pip install protobuf $ pip install google //For protobuf to JSON
11. protoc -I=$SRC_DIR --python_out=$DST_DIR $SRC_DIR/addressbook.proto Compile a proto file
12. Important methods • SerializeToString(): serializes the message and returns it as a string. Note that the bytes are binary, not text; we only use the str type as a convenient container. • ParseFromString(data): parses a message from the given string.
13. Enough Talk! Show me the code?
14. We love Python | Everyone loves Python People say JavaScript is a simplified C where you strip types and use everything as usual. But what is Python? It is almost close to English Python is given a first citizen support for protobufs and grpc
15. gRPC? gRPC is a modern RPC implementation that connects distributed systems and allows them to execute remote functions. Built on: 1) HTTP 2.0 2) IDL 3) Protobufs
16. Client Server gRPC
17. Why I should use it? To build: • Low latency, highly scalable distributed systems • Mobile clients • Communication between different microservices (diverse engineering teams)
18. HTTP 2.0? • Binary Protocol - No more text parsing • Native stream - No need of web sockets • Stream multiplexing - Single connection • Header compression - Bandwidth saving
19. Install grpc tools for Python Install necessary package $ pip install grpcio-tools Create proto file with messages and services python -m grpc_tools.protoc -I=$SRC_DIR --python_out=$DST_DIR $SRC_DIR/detailservice.proto
21. Show me gRPC code...
22. O(1010) RPC Per Second. If GRPC saves one byte transfer per request, it saves 1010 bytes bandwidth/second Courtesy - Ray Tsang, Google For example, Google executes
23. I wrote this for Python developers
24. Q&A Thank You Delhi! narenarya@live.com