From Monolith to Microservice: Building APIs with gRPC & golang

Case study & tutorial on how to break up a legacy monolith codebase into microservices using gRPC + Protobuf + golang + the language of your choice!

Transcript

1. FROM MONOLITH
   TO
   MICROSERVICE
   Modernizing legacy codebases with grpc + go
   

   View Slide

2. (or gRPC for dummies)
   

   View Slide

3. Hi, I’m
   Cecy Correa
   @cecycorrea
   Software Engineer,
   Context.IO
   

   View Slide

4. View Slide

5. gRPC to break up your monolith!
   

   View Slide

6. what is gRPC?
   

   View Slide

7. gRPC Remote Procedure Calls
   

   View Slide

8. oh ok
   

   View Slide

9. what is RPC?
   

   View Slide

10. Data exchange between 2
    processes
    

    View Slide

11. View Slide

12. so an API?
    

    View Slide

13. RPC or REST?
    

    View Slide

14. View Slide

15. The “RPC” part stands for “remote
    procedure call,” and it’s essentially
    the same as calling a function in
    JavaScript, PHP, Python and so on,
    taking a method name and
    arguments.
    Source: Smashing Magazine
    

    View Slide

16. RPC == good for actions
    

    View Slide

17. REST == CRUD
    &
    modeling your domain
    

    View Slide

18. [REST]
    GET users/:id/photos/:photo_id
    

    View Slide

19. [RPC]
    getUserPhotos(args...)
    

    View Slide

20. Why gRPC?
    

    View Slide

21. Why gRPC?
    ● Use HTTP/2
    

    View Slide

22. Why gRPC?
    ● Use HTTP/2
    ● Language agnostic
    

    View Slide

23. Why gRPC?
    ● Use HTTP/2
    ● Language agnostic
    ● Highly efficient / scalable
    

    View Slide

24. Why gRPC?
    ● Use HTTP/2
    ● Language agnostic
    ● Highly efficient / scalable
    ● Handle large amounts of data
    

    View Slide

25. HTTP2!
    

    View Slide

26. HTTP2!
    

    View Slide

27. HTTP2!
    ● Binary: moar data! Moar efficient!
    

    View Slide

28. HTTP2!
    ● Binary: moar data! Moar efficient!
    ● Multiplexed: multiple files at a time
    

    View Slide

29. HTTP2!
    ● Binary: moar data! Moar efficient!
    ● Multiplexed: multiple files at a time
    ● Same connection!
    

    View Slide

30. Source: CSS Tricks
    

    View Slide

31. Who is using it?
    ● Google
    ● Netflix
    ● Docker
    ● Square
    ● CoreOS
    ● Twitch (Twirp)
    

    View Slide

32. View Slide

33. Let’s get started!
    

    View Slide

34. gRPC + Protobuf
    

    View Slide

35. GRPC PROTOBUF
    The protocol Define the service
    Messages that are
    exchanged back and forth
    

    View Slide

36. What we will do
    ● Use protobuf to define your service
    ● Use protoc to generate client and server stubs based on
    your service definition
    ● Create a gRPC server to handle requests
    

    View Slide

37. Install all the things
    go to grpc.io for options
    

    View Slide

38. Step 1: Create a proto file
    

    View Slide

39. Sample proto file
    

    View Slide

40. View Slide

41. View Slide

42. Sample proto file
    

    View Slide

43. Step 2: Create your server / client stubs
    

    View Slide

44. protoc -I proto --php_out=lib --grpc_out=lib
    --plugin=protoc-gen-grpc=`which
    grpc_php_plugin` service.proto
    

    View Slide

45. protoc -I DIR --LANG_out=DIR --grpc_out=DIR
    --plugin=protoc-gen-grpc=`which
    grpc_LANG_plugin` WHERE_YOUR_PROTO_IS.proto
    

    View Slide

46. project-dir
    ├── client
    | └── client.go|js|rb|py ...
    ├── proto
    | └── service.proto
    ├── server
    | └── server.go|js|rb|py ...
    └── libs
    └── ruby
    └── python
    └── ...
    

    View Slide

47. Step 3. Create a gRPC server
    

    View Slide

48. View Slide

49. Sample proto file
    

    View Slide

50. View Slide

51. Step 4: Write some clients!
    

    View Slide

52. View Slide

53. *Note: you will need to install some stuff to get started
    in PHP
    ● composer install grpc as a dependency
    ● pecl install grpc / protobuf
    ● Full instructions on getting started:
    https://grpc.io/docs/quickstart/php.html
    

    View Slide

54. View Slide

55. View Slide

56. Let’s see it in action!
    

    View Slide

57. Use case: Context.IO
    

    View Slide

58. View Slide

59. Pain points
    ● Legacy codebase (~10 years old)
    ● Monolith codebase
    ● Heavily coupled == hard to unit test
    ● Scaling problems
    

    View Slide

60. How gRPC helped
    ● Replace small pieces of functionality with microservices
    ● Easy to do by “resource”
    

    View Slide

61. From REST to microservice
    GET /discovery Discovery service
    GET /user/:id/contacts Contacts service
    GET | POST /user/:id/webhooks Webhooks service
    GET | POST /webhooks
    GET /user/:id/messages Messages services
    GET /user/:id/folders Folder service
    

    View Slide

62. Our PHP API became the frontend
    for a microservice ecosystem
    

    View Slide

63. The brave new future:
    Completely generated client libraries & docs
    

    View Slide

64. This solves...
    ● Feature parity between API and client libraries
    ● Feature parity between API and docs
    ● Lack of knowledge in the team around certain languages
    ● Easy for other teams to consume your service
    

    View Slide

65. ...and
    gRPC can also generate REST stubs!
    

    View Slide

66. So you can gRPC -> REST -> gRPC
    

    View Slide

67. grpc-gateway
    

    View Slide

68. Generate stubs + gateway
    path/to/your_service.pb.gw.go
    

    View Slide

69. View Slide

70. What does our API look like now?
    

    View Slide

71. Monolith -> Monorepo
    

    View Slide

72. View Slide

73. View Slide

74. View Slide

75. [CODE]
    github.com/cecyc/dad-joke-service
    

    View Slide

76. Questions?
    Happy to talk in the hall!
    or ping @cecycorrea on
    Twitter
    

    View Slide