From PHP and REST to gRPC and Go Full

Journey from PHP and REST to gRPC and Go.

Transcript

1. From PHP and REST To gRPC and Go Winnipeg, January 29 2020 Yuriy Tuz Bold Commerce @yuriytuz
2. #GoTurns10
3. whoami
4. Agenda • Personal experience • Past life • Microservices intro • Why gRPC • gRPC in GO ◦ Simple implementation
5. In the beginning • Laravel 4.x / 5.x • PHP 7.x ◦ Introduced types
6. Back in the day
7. REST (HTTP + JSON) • Not a bad solution • Service to service communications • (De)Serialization performance • JSON is not type safe in PHP • Text protocol
8. Thank you Questions?
9. REST in Laravel • api.php • Magic • response()->json([“customers”=>$c, ...]) • Docs accuracy
10. Problem • Technology is driving demand • Languages and platforms ◦ Different dev experience • Software architecture is evolving ◦ Microservices and containers ◦ CI/CD
11. An Idea
12. An Idea • Bloated service ◦ 1000+ lines • Hack Day • How?
13. No Idea
14. How/why? • Why split? • Why Go? • OG mayhem to the rescue • Why not REST (HTTP+JSON)? ◦ No formal contract ◦ Hard to implement correctly
15. Possible Solution
16. What do we need? • PHP RPC client • RPC server
17. Implementation/Findings • PHP client not fun ◦ Working POC • First experience with Go • Refactoring • Benchmarks ◦ ~100x faster
18. time.Now()
19. Microservices https://microservices.io • Highly maintainable and testable • Loosely coupled • Independently deployable • Organized around business capabilities • Owned by a small team
20. Organizations which design systems … are constrained to produce designs which are copies of the communication structures of these organizations” Conway, Melvin E. (1968)
21. https://martinfowler.com/articles/microservices.html Cross-functional teams lead to capability oriented architecture
22. until code is in production, no value is actually being generated
23. Mayhem (simplified)
24. Stack • Protocol buffers + gRPC ◦ Thanks Google • Tim Burks ◦ API’s at Google ◦ “Stop writing interface code for your APIs by hand”
25. Protocol buffers are a language-neutral, platform-neutral extensible mechanism for serializing structured data. Protocol Buffers
26. What does this mean? • Serialization mechanism • Interface description language
27. Serialization Mechanism • 0a 05 68 65 6c 6c 6f • 68 65 6c 6c 6f -> hello • 05 -> length of “hello” • 0a is “0000 1010”, so • field_number = 1 • wire_type = 2 (length delimited)
28. Interface Description Language message Person { int64 id = 1; string shop_identifier = 2; reserved 3; string platform_type = 4; string first_name = 5; string last_name = 6; string email = 7; repeated Address addresses = 13; repeated Label labels = 14; } Interface for your API
29. gRPC • Open source messaging system by Google that can run anywhere • Low latency • Highly scalable • Great for distributed systems • Accurate, efficient, language independent
30. Adopted by many (and more)
31. Let’s do it Implementation
32. echo.proto
33. Let’s generate • protoc --go_out=:. ./echo.proto ◦ Default protobuf generation • protoc --go_out=plugins=grpc:. ./echo.proto ◦ Client and Server interface ◦ gRPC
34. Success!
35. Server
36. Generated Interface
37. Server implementation
38. Server Spin up server
39. Client
40. Client interface
41. Client
42. So far • Server • Client
43. New Requirement Logging
44. gRPC middleware • Server or Client • Few exist in gRPC ecosystem ◦ Prometheus ◦ Open tracing • https://github.com/grpc-ecosystem/go-grpc-middleware
45. How?
46. gRPC middleware
47. >1 interceptor?
48. Directory structure • Client & server implementation • echo.pb.go ◦ gRPC generated • echo.proto ◦ API definition
49. Helpful tools
50. grpcurl • “curl” for gRPC • Requires server reflection
52. Examples
53. grpcui • Postman for RPC • Also requires reflection
56. Remember this?
57. gRPC gateway • Have .proto file? ◦ Get free REST API (in few simple steps) ◦ Make sure you go get plugin ◦ protoc --go_out=plugins=grpc:. --grpc-gateway_out=logtostderr=true,grpc_api_configuration=./router.yaml :. ./echo.proto
58. gRPC Gateway
59. router.yaml
60. Almost there
61. New directory structure • *.pb.gw.go ◦ Reverse proxy
62. Final thoughts The good • Code generation ◦ Less error prone • API spec ◦ OpenAPI • gRPC ◦ HTTP2 + language support • Type safety • Plugins • Dev ramp up speed The bad • Code generation • Model conversions ◦ Nested messages • Hard/scary to get started ◦ Documentation • API versioning • gRPC Gateway silent failure
63. TL;DR • Use gRPC + Go ◦ Microservices • Generate code • Implement stubs • Profit
64. Credits/Useful links • Gopher art by Ashley McNamara ◦ https://github.com/ashleymcnamara/gophers • gRPC ◦ https://grpc.io/ • Google API Design Guide ◦ https://cloud.google.com/apis/design/ • Protobuf Style Guide ◦ https://developers.google.com/protocol-buffers/docs/style • rpcurl and grpcui ◦ https://github.com/fullstorydev • gRPC gateway ◦ https://github.com/grpc-ecosystem/grpc-gateway
65. Questions? @yuriytuz