Real World Migration from HTTP to gRPC in Ruby #grpcconf

Real World Migration from HTTP to gRPC in Ruby #grpcconf

This is a slide I presented at gRPC Conf 2020 titled "Real World Migration from HTTP to gRPC in Ruby".

In this talk, I presented the details of our migration approach from HTTP/1.1 to gRPC, and the resulting performance improvements. Average latency got decreased by 50%, and 90%tile latency was decreased to a seventh.

cf. https://sched.co/cRfr

2b4f9f20b554d4c77c46fe6a9930d6fe?s=128

Nao Minami

July 27, 2020
Tweet

Transcript

  1. ©2020 Wantedly, Inc. Real World Migration from HTTP to gRPC

    in Ruby 27.Jul.2020 - Nao Minami (@south37) Wantedly, Inc. gRPC Conf 2020
  2. ©2020 Wantedly, Inc. /BP.JOBNJ *OGSBTUSVDUVSF5FBN 8BOUFEMZ *OD  43& 4JUF3FMJBCJMJUZ&OHJOFFSJOH

      %FWFMPQNFOU*OGSBTUSVDUVSF  .JDSPTFSWJDFT H31$ FUD
  3. ©2020 Wantedly, Inc. 1. Why gRPC 2. How we migrated

    from HTTP to gRPC 3. What we achieved by using gRPC In this talk…
  4. ©2020 Wantedly, Inc. 1. Why gRPC

  5. ©2020 Wantedly, Inc. Background: our products Our products w 8BOUFEMZ7JTJUKPCTBOEDBOEJEBUFTNBUDIJOH

    w 8BOUFEMZ1FPQMFQSPGFTTJPOBMDPOUBDUTNBOBHFNFOU w FUD These products share the same components such as “user’s profile data”
  6. ©2020 Wantedly, Inc. Background: our microservices architecture Our products are

    built with 100+ microservices w  NJDSPTFSWJDFTXSJUUFOJO3VCZ (P 1ZUIPO /PEFKT FUDSVOPOBLTDMVTUFS w 8FIBECFFOVTJOH)551GPSUIFNBKPSJUZPGPVSNJDSPTFSWJDFT w lVTFS`TQSPpMFEBUBzJTNBOBHFECZl6TFS4FSWJDFz XIJDIJTXSJUUFOJO3VCZ LTDMVTUFS $PSF4FSWJDFT 6TFS4FSWJDF 3VCZ
  7. ©2020 Wantedly, Inc. What is “UserService”? Get/Update user’s profile data

    Low latency and high throughput are required w 6TFEGSPNNBOZNJDSPTFSWJDFTIJHIUISPVHIQVUJTSFRVJSFE w -BUFODZPGUFOEJSFDUMZB⒎FDUTUIFVTFSFYQFSJFODFMPXMBUFODZJTSFRVJSFE 6TFS4FSWJDF (FUVQEBUFVTFS`TQSPpMFEBUB
  8. ©2020 Wantedly, Inc. What was the problem of UserService? Latency

    caused by using HTTP/1.1 w 8JUIPVULFFQBMJWF )551EPFTOPUQFSGPSNXFMMGPSTFWFSBMSFBTPOT w FH%/4MPPLVQ 5$1XBZIBOETIBLF 5$1TMPXTUBSU FUD 6TFS4FSWJDF -BUFODZDBVTFECZVTJOH)551
  9. ©2020 Wantedly, Inc.

  10. ©2020 Wantedly, Inc. Why gRPC? High-Performance w -POHMJWFE SFBMUJNFDPNNVOJDBUJPOTUSFBNTCZVTJOH)551 w

    -PXMBUFODZ IJHIUISPVHIQVU Protocol Buffers w l4JOHMFTPVSDFPGUSVUIzPGBTFSWJDFEFpOJUJPO w "EEJUJPOBMHPPEQPJOUXFXFSFBMSFBEZVTJOH1SPUPDPM#V⒎FSTPWFS)551 Multiple Languages Support w 8FIBWFNJDSPTFSWJDFTXSJUUFOJOTFWFSBMMBOHVBHFT BOEH31$TVQQPSUTBMMPGUIFN
  11. ©2020 Wantedly, Inc. For these reasons, we chose gRPC

  12. ©2020 Wantedly, Inc. 2. How we migrated from HTTP to

    gRPC
  13. ©2020 Wantedly, Inc. 1. Preparation 2. Migration How we migrated

    from HTTP to gRPC
  14. ©2020 Wantedly, Inc. 1. Preparation 2. Migration How we migrated

    from HTTP to gRPC
  15. ©2020 Wantedly, Inc. 1. Schema Management 2. Monitoring 3. Productive

    Development Environment 4. Load Balancing Preparations for using gRPC
  16. ©2020 Wantedly, Inc. 1. Schema Management 2. Monitoring 3. Productive

    Development Environment 4. Load Balancing Preparations for using gRPC
  17. ©2020 Wantedly, Inc. Without a monorepo, we have to share

    the same proto files across the multiple repositories Schema Management 4BNFQSPUPpMFT 3FQPTJUPSZ" 3FQPTJUPSZ# 3FQPTJUPSZ$
  18. ©2020 Wantedly, Inc. How should we share proto files? w

    $PQZQBTUFNBOVBMMZ  7FSZIBSEUPVQEBUFNVMUJQMFSFQPTJUPSJFTDPOUJOVPVTMZ w %PXOMPBEQSPUPpMFTBVUPNBUJDBMMZ  /FFEUPHFOFSBUFUIFDPEFNBOVBMMZ 4V⒎FSGSPNUIFEJ⒎FSFODFTJOFBDIEFWFMPQFS`TFOWJSPONFOU FHWFSTJPOPG QSPUPD QSFTFODFPGQMVHJOT FUD  We decided to make a central repository Schema Management
  19. ©2020 Wantedly, Inc. apis: a central repository for schema management

    w lBQJTzTUPSFTBMMQSPUPpMFTGPSPVSDPNQBOZ Schema Management
  20. ©2020 Wantedly, Inc. How to use “apis” repository? Code is

    generated in the CI pipelines of “apis” w 3VCZ (P BOE/PEFKTDPEFJTHFOFSBUFEGPSFBDIQSPUPpMF w (FOFSBUFEDPEFJTEJTUSJCVUFEWJBQBDLBHFNBOBHFST #VOEMFS (P.PEVMFT OQN  w %FWFMPQFSTEPOUOFFEUPHFOFSBUFDPEFCZUIFNTFMWFT
  21. ©2020 Wantedly, Inc. # Gemfile gem 'apis', git: 'git@github.com:wantedly/apis-ruby.git', tag:

    'xxx.yyy.zzz' 3VCZXJUI#VOEMFS
  22. ©2020 Wantedly, Inc. How we develop gRPC servers and clients

    in Ruby?
  23. ©2020 Wantedly, Inc. gRPC servers: Implement a class which inherits

    a gRPC service class fetched from “apis” w #ZJNQMFNFOUJOHJOTUBODFNFUIPETGPSFBDI31$NFUIPE XFDBOVTFH31$ TFSWFSTJO3VCZ # NOTE: Load UsersPb::UserService from apis require "wantedly/users/users_services_pb" class UserServer < UsersPb::UserService::Service # @param [UsersPb::ListUsersRequest] req # @param [GRPC::ActiveCall::SingleReqView] call # @return [UsersPb::ListUsersResponse] def list_users(req, call) svc = UsersReadService.new(req) UsersPb::ListUsersResponse.new(users: svc.users) end end How we develop gRPC servers?
  24. ©2020 Wantedly, Inc. gRPC clients: Use a stub class fetched

    from “apis” w #ZVTJOHBTUVCPCKFDUBTBH31$DMJFOU XFDBODPNNVOJDBUFXJUIH31$TFSWFST # NOTE: Load UsersPb::UserService from apis require "wantedly/users/users_services_pb" client = UsersPb::UserService::Stub.new( url, :this_channel_is_insecure ) How we develop gRPC clients?
  25. ©2020 Wantedly, Inc. 1. Schema Management 2. Monitoring 3. Productive

    Development Environment 4. Load Balancing Preparations for using gRPC
  26. ©2020 Wantedly, Inc. Application monitoring is required in production w

    8FXFSFVTJOH%BUBEPH /FX3FMJD )POFZCBEHFS FUDGPSNPOJUPSJOHPVS )551TFSWFST 8BOUFEUPDPOUJOVFUPVTFUIFNGPSH31$TFSWFST Monitoring
  27. ©2020 Wantedly, Inc. Monitoring How to monitor gRPC servers? =>

    gRPC interceptors w /FFEUPTFOENFUSJDTPGFBDIH31$SFRVFTU w 8JUIH31$JOUFSDFQUPST XFDBOJNQMFNFOUTVDIGFBUVSFT $MJFOUH31$JOUFSDFQUPST $MJFOU 4FSWFSH31$JOUFSDFQUPST 4FSWFS 3FRVFTU 3FTQPOTF
  28. ©2020 Wantedly, Inc. Monitoring We implemented some gRPC interceptors w

    H31$JOUFSDFQUPSTGPSFBDI4BB4QSPEVDUBSFOPUP⒏DJBMMZQSPWJEFEBUUIJTUJNF w 8FJNQMFNFOUFEUIFNJO3VCZ BOENBEFUIFNBWBJMBCMFUPUIF3VCZDPNNVOJUZ BTPQFOTPVSDFTPGUXBSF 044
  29. ©2020 Wantedly, Inc. 1. Schema Management 2. Monitoring 3. Productive

    Development Environment 4. Load Balancing Required preparations for using gRPC
  30. ©2020 Wantedly, Inc. Need to create a productive development environment

    for gRPC w 'PS)551TFSWFST UIFFDPTZTUFNJTNBUVSFBOENBOZPQFOTPVSDFMJCSBSJFTFYJTU w 5IJTJTOPUUIFDBTFXJUIH31$TFSWFST FTQFDJBMMZJO3VCZ8FOFFEUPDSFBUFB QSPEVDUJWJUZEFWFMPQNFOUFOWJSPONFOUCZPVSTFMWFT Productive Development Environment
  31. ©2020 Wantedly, Inc. How to create a productive development environment?

  32. ©2020 Wantedly, Inc. Use a common utility library named “servicex”

    w 8FXBOUTPNFVTFGVMGFBUVSFT FHIPUSFMPBEJOH VUJMJUZGPSQSPUPDPMCV⒎FST FUD  w 8FXBOUUPBWPJESFQFBUJOHUIFTBNFDPOpHVSBUJPOJONVMUJQMFNJDSPTFSWJDFT JNQMFNFOUFElTFSWJDFYz How to create a productive development environment? .JDSPTFSWJDF" .JDSPTFSWJDF# .JDSPTFSWJDF$ lTFSWJDFYz
  33. ©2020 Wantedly, Inc. gRPC servers: grpc-server command w H31$TFSWJDFDMBTTFTBSFBVUPNBUJDBMMZMPBEFE w

    4PNFH31$JOUFSDFQUPSTBSFBVUPNBUJDBMMZTFU w )PUSFMPBEJOHGFBUVSF w %FWFMPQFSTDBOKVTUGPDVTPOUIFJNQMFNFOUBUJPOPGH31$TFSWJDFDMBTTFT The features of our internal library “servicex” $ bundle exec grpc-server handling /grpc.health.v1.Health/Check with #<Method: Grpc::Health::Checker#check> handling /grpc.health.v1.Health/Watch with #<Method: Grpc::Health::Checker(Grpc::Health::V1::Health::Service)#watch> handling /wantedly.users.UserService/ListUsers with #<Method: UsersGrpcService#list_users> . . . gRPC server starting... * Listening on tcp://0.0.0.0:6046 * Environment: development
  34. ©2020 Wantedly, Inc. gRPC clients: Servicex::Grpc.stub_for w 4PNFH31$JOUFSDFQUPSTBSFBVUPNBUJDBMMZTFU w 6TFS"HFOUJTBVUPNBUJDBMMZDPOpHVSFE

    w %FWFMPQFSTDBOKVTUGPDVTPOJNQMFNFOUJOHUIFCVTJOFTTMPHJD The features of our internal library “servicex” # NOTE: Load UsersPb::UserService from apis require "wantedly/users/users_services_pb" grpc_server_url = “xxx” client = Servicex::Grpc.stub_for( UsersPb::UserService, grpc_server_url, ) client.get_user(UsersPb::GetUserRequest.new(…))
  35. ©2020 Wantedly, Inc. Pb: utility for using Protocol Buffers w

    8JUIl1Cz XFDBODSFBUFQSPUPCVGPCKFDUTNPSFFBTJMZ w FH1CUP@UJNFTUBNQDBODSFBUFBUJNFTUBNQPCKFDUGSPNB4USJOHPCKFDU The features of our internal library “servicex” [1] pry(main)> Pb.to_timestamp("2019-05-15T00:00:00+09:00") => <Google::Protobuf::Timestamp: seconds: 1557846000, nanos: 0> 8JUIl1Cz [1] pry(main)> Google::Protobuf::Timestamp.new( [1] pry(main)* seconds: Time.parse("2019-05-15T00:00:00+09:00").to_i [1] pry(main)* ) => <Google::Protobuf::Timestamp: seconds: 1557846000, nanos: 0> 8JUIPVUl1Cz
  36. ©2020 Wantedly, Inc. With Pb.to_proto, we can create user-defined protobuf

    objects easily The features of “Pb” [1] pry(main)> Pb.to_proto(Account, { created_at: "2019-05-15T00:00:00+09:00" }) => <Account: id: 0, created_at: <Google::Protobuf::Timestamp: seconds: 1557846000, nanos: 0>> 8JUIl1Cz [1] pry(main)> Account.new( [1] pry(main)* created_at: Google::Protobuf::Timestamp.new( [1] pry(main)* seconds: Time.parse("2019-05-15T00:00:00+09:00").to_i [1] pry(main)* ) [1] pry(main)* ) => <Account: id: 0, created_at: <Google::Protobuf::Timestamp: seconds: 1557846000, nanos: 0>> 8JUIPVUl1Cz
  37. ©2020 Wantedly, Inc. Pb is published as open source software

    (OSS) w 1CJTJODMVEFEJOTFSWJDFY CVUXFNBEFJUBWBJMBCMFUPUIF3VCZDPNNVOJUZBT PQFOTPVSDFTPGUXBSF 044  w IUUQTSVCZHFNTPSHHFNTUIF@QC w IUUQTSVCZHFNTPSHHFNTQCTFSJBMJ[FS The features of “Pb”
  38. ©2020 Wantedly, Inc. Use gRPC combined with parts of Ruby

    on Rails w 8FNJHSBUFEGSPN3VCZPO3BJMT BOEXBOUFEUPLFFQVTJOHQBSUTPGGFBUVSFTJOJU FH"DUJWF3FDPSE "DUJWF4VQQPSU "VUPMPBEJOH SBJMTDPOTPMF FUD  %FDJEFEUPVTFHSQDHFNXJUISBJMTHFN w 8FGPVOEUIBUJU`TWFSZVTFGVMUPCVJMEBH31$TFSWFSXJUIQBSUTPG3VCZPO3BJMT 8FSFDPNNFOEUIJTDPOpHVSBUJPOJGZPV`SFJOUIFTBNFTJUVBUJPO # Gemfile ruby '2.7.1' gem 'grpc' gem 'rails', '~> 6.0.3' How to create a productive development environment?
  39. ©2020 Wantedly, Inc. 1. Schema Management 2. Monitoring 3. Productive

    Development Environment 4. Load Balancing Required preparations for using gRPC
  40. ©2020 Wantedly, Inc. For gRPC, L4 Load-Balancers aren’t very useful

    w H31$JTCVJMUPO)551 XIJDIJTEFTJHOFEUPIBWFBTJOHMFMPOHMJWFE5$1 DPOOFDUJPO w *O--PBE#BMBODFST PODFUIF5$1DPOOFDUJPOJTFTUBCMJTIFE BMMSFRVFTUTXJMMHFU QJOOFEUPBTJOHMFEFTUJOBUJPO Load Balancing 6TFS4FSWJDF 4PNF.JDSPTFSWJDF
  41. ©2020 Wantedly, Inc. We use Istio for L7 load-balancing in

    k8s clusters w &OWPZDPOUBJOFSTDPOpHVSFECZ*TUJPBSFJOKFDUFEBTTJEFDBSQSPYJFT BOEVTFEGPS - "QQMJDBUJPO MPBECBMBODJOH Load Balancing 6TFS4FSWJDF 4PNF.JDSPTFSWJDF &OWPZ
  42. ©2020 Wantedly, Inc. 1. Preparation 2. Migration How we migrated

    from HTTP to gRPC
  43. ©2020 Wantedly, Inc. Basic strategy was “Try Small” w "UpSTU

    XFNBEFBQSPUPUZQFBTBQSPPGPGDPODFQU 1P$  w /FYU XFVTFEH31$GPSBQBSUPGUIFUSB⒏DJOQSPEVDUJPO BOEHSBEVBMMZJODSFBTFE UIFQFSDFOUBHF How we migrated from HTTP to gRPC?
  44. ©2020 Wantedly, Inc. PoC - schematic diagram QSPEVDUJPOLTDMVTUFS 6TFS4FSWJDF )551

    Tried PoC in development k8s cluster w #ZVTJOHlEFWFMPQNFOULTDMVTUFSz XFDPVMEUSZH31$XJUIPVUXPSSZJOHBCPVU B⒎FDUJOHQSPEVDUJPOFOWJSPONFOU EFWFMPQNFOULTDMVTUFS 6TFS4FSWJDF 5SZH31$
  45. ©2020 Wantedly, Inc. The result of PoC Latency decreased dramatically

    w #ZTFFJOHUIJTSFTVMU XFEFDJEFEUPUSZH31$JOBQSPEVDUJPOLTDMVTUFS %ile Latency [ms]
  46. ©2020 Wantedly, Inc. How we tried gRPC in production?

  47. ©2020 Wantedly, Inc. How we tried gRPC in production? At

    first, we used gRPC for a small part of the traffic w #ZUSZJOHTNBMM XFDPVMENJOJNJ[FUIFQSFQBSBUJPODPTUTUPNJOJNJ[FUIFSJTL Then, gradually increased the percentage w 'JOBMMZ XFDPVMEBDIJFWFUIFDPNQMFUFNJHSBUJPO )551  6TFS H31$  *ODSFBTFUIFQFSDFOUBHF )551  6TFS H31$ 
  48. ©2020 Wantedly, Inc. With these steps, we migrated to gRPC

  49. ©2020 Wantedly, Inc. 3. What we achieved by using gRPC

  50. ©2020 Wantedly, Inc. Results achieved by using gRPC Average Latency

    decreased w "WFSBHFMBUFODZEFDSFBTFECZBCPVUNTQFSSFRVFTU w 5IFJNQBDUXBTNPSFSFNBSLBCMFXIFONBOZSFRVFTUTUP6TFS4FSWJDFXFSF JOWPMWFE'PSFYBNQMF JOTPNFNJDSPTFSWJDFT BWFSBHFMBUFODZEFDSFBTFECZBIBMG HTTP/1.1 gRPC Average latency decreased from 130ms to 65ms in this service
  51. ©2020 Wantedly, Inc. We achieved huge performance improvement

  52. ©2020 Wantedly, Inc. What we have been doing since the

    migration?
  53. ©2020 Wantedly, Inc. The use of gRPC is increasing throughout

    the organization w "MNPTUBMMDMJFOUTPG6TFS4FSWJDFBSFVTJOHH31$ w /FXNJDSPTFSWJDFTBSFCFJOHEFWFMPQFEVTJOHH31$ w H31$TUBSUFEUPCFVTFEJOPUIFSMBOHVBHFT (P /PEFKT FUD What we have been doing since the migration? 3VCZ 3VCZ /PEF 3VCZ (P H31$ /FX
  54. ©2020 Wantedly, Inc. Summary

  55. ©2020 Wantedly, Inc. Summary 1. Why gRPC w )JHI1FSGPSNBODF w

    1SPUPDPM#V⒎FST w .VMUJQMF-BOHVBHFT4VQQPSU 2. How we migrated from HTTP to gRPC w $SFBUFEBXPSLqPXGPS1SPUPDPM#V⒎FSTVTJOHPVSlBQJTzSFQPTJUPSZ w %FWFMPQFEMJCSBSJFTGPSQSPEVDUJWJUZ BOEQVCMJTIFETPNFPGUIFNBT044 w #BTJDTUSBUFHZGPSNJHSBUJPOXBTl5SZ4NBMMz 3. What we achieved by using gRPC w "DIJFWFEUIFQFSGPSNBODFJNQSPWFNFOU w 5IFVTFPGH31$JTJODSFBTJOHUISPVHIPVUUIFPSHBOJ[BUJPO
  56. ©2020 Wantedly, Inc. • Alina Grubnyak https://unsplash.com/photos/hdpDLIXZg7Y Photo Credit