From Monolith to Microservice: Building APIs with gRPC & golang

045d790f605ea8d482380c5075d802d5?s=47 Cecy C.
June 09, 2018

From Monolith to Microservice: Building APIs with gRPC & golang

Case study & tutorial on how to break up a legacy monolith codebase into microservices using gRPC + Protobuf + golang + the language of your choice!

045d790f605ea8d482380c5075d802d5?s=128

Cecy C.

June 09, 2018
Tweet

Transcript

  1. FROM MONOLITH TO MICROSERVICE Modernizing legacy codebases with grpc +

    go
  2. (or gRPC for dummies)

  3. Hi, I’m Cecy Correa @cecycorrea Software Engineer, Context.IO

  4. None
  5. gRPC to break up your monolith!

  6. what is gRPC?

  7. gRPC Remote Procedure Calls

  8. oh ok

  9. what is RPC?

  10. Data exchange between 2 processes

  11. None
  12. so an API?

  13. RPC or REST?

  14. None
  15. The “RPC” part stands for “remote procedure call,” and it’s

    essentially the same as calling a function in JavaScript, PHP, Python and so on, taking a method name and arguments. Source: Smashing Magazine
  16. RPC == good for actions

  17. REST == CRUD & modeling your domain

  18. [REST] GET users/:id/photos/:photo_id

  19. [RPC] getUserPhotos(args...)

  20. Why gRPC?

  21. Why gRPC? • Use HTTP/2

  22. Why gRPC? • Use HTTP/2 • Language agnostic

  23. Why gRPC? • Use HTTP/2 • Language agnostic • Highly

    efficient / scalable
  24. Why gRPC? • Use HTTP/2 • Language agnostic • Highly

    efficient / scalable • Handle large amounts of data
  25. HTTP2!

  26. HTTP2!

  27. HTTP2! • Binary: moar data! Moar efficient!

  28. HTTP2! • Binary: moar data! Moar efficient! • Multiplexed: multiple

    files at a time
  29. HTTP2! • Binary: moar data! Moar efficient! • Multiplexed: multiple

    files at a time • Same connection!
  30. Source: CSS Tricks

  31. Who is using it? • Google • Netflix • Docker

    • Square • CoreOS • Twitch (Twirp)
  32. None
  33. Let’s get started!

  34. gRPC + Protobuf

  35. GRPC PROTOBUF The protocol Define the service Messages that are

    exchanged back and forth
  36. What we will do • Use protobuf to define your

    service • Use protoc to generate client and server stubs based on your service definition • Create a gRPC server to handle requests
  37. Install all the things go to grpc.io for options

  38. Step 1: Create a proto file

  39. Sample proto file

  40. None
  41. None
  42. Sample proto file

  43. Step 2: Create your server / client stubs

  44. protoc -I proto --php_out=lib --grpc_out=lib --plugin=protoc-gen-grpc=`which grpc_php_plugin` service.proto

  45. protoc -I DIR --LANG_out=DIR --grpc_out=DIR --plugin=protoc-gen-grpc=`which grpc_LANG_plugin` WHERE_YOUR_PROTO_IS.proto

  46. project-dir ├── client | └── client.go|js|rb|py ... ├── proto |

    └── service.proto ├── server | └── server.go|js|rb|py ... └── libs └── ruby └── python └── ...
  47. Step 3. Create a gRPC server

  48. None
  49. Sample proto file

  50. None
  51. Step 4: Write some clients!

  52. None
  53. *Note: you will need to install some stuff to get

    started in PHP • composer install grpc as a dependency • pecl install grpc / protobuf • Full instructions on getting started: https://grpc.io/docs/quickstart/php.html
  54. None
  55. None
  56. Let’s see it in action!

  57. Use case: Context.IO

  58. None
  59. Pain points • Legacy codebase (~10 years old) • Monolith

    codebase • Heavily coupled == hard to unit test • Scaling problems
  60. How gRPC helped • Replace small pieces of functionality with

    microservices • Easy to do by “resource”
  61. From REST to microservice GET /discovery Discovery service GET /user/:id/contacts

    Contacts service GET | POST /user/:id/webhooks Webhooks service GET | POST /webhooks GET /user/:id/messages Messages services GET /user/:id/folders Folder service
  62. Our PHP API became the frontend for a microservice ecosystem

  63. The brave new future: Completely generated client libraries & docs

  64. This solves... • Feature parity between API and client libraries

    • Feature parity between API and docs • Lack of knowledge in the team around certain languages • Easy for other teams to consume your service
  65. ...and gRPC can also generate REST stubs!

  66. So you can gRPC -> REST -> gRPC

  67. grpc-gateway

  68. Generate stubs + gateway path/to/your_service.pb.gw.go

  69. None
  70. What does our API look like now?

  71. Monolith -> Monorepo

  72. None
  73. None
  74. None
  75. [CODE] github.com/cecyc/dad-joke-service

  76. Questions? Happy to talk in the hall! or ping @cecycorrea

    on Twitter