Real World Migration from HTTP to gRPC in Ruby #grpcconf

This is a slide I presented at gRPC Conf 2020 titled "Real World Migration from HTTP to gRPC in Ruby".

In this talk, I presented the details of our migration approach from HTTP/1.1 to gRPC, and the resulting performance improvements. Average latency got decreased by 50%, and 90%tile latency was decreased to a seventh.

cf. https://sched.co/cRfr

Transcript

1. ©2020 Wantedly, Inc.
   Real World Migration
   from HTTP to gRPC in Ruby
   27.Jul.2020 - Nao Minami (@south37) Wantedly, Inc.
   gRPC Conf 2020
   

   View Slide

2. ©2020 Wantedly, Inc.
   /BP.JOBNJ
   *OGSBTUSVDUVSF5FBN 8BOUFEMZ *OD
   43& 4JUF3FMJBCJMJUZ&OHJOFFSJOH
   
   %FWFMPQNFOU*OGSBTUSVDUVSF
   .JDSPTFSWJDFT H31$ FUD
   

   View Slide

3. ©2020 Wantedly, Inc.
   1. Why gRPC
   2. How we migrated from HTTP to gRPC
   3. What we achieved by using gRPC
   In this talk…
   

   View Slide

4. ©2020 Wantedly, Inc.
   1. Why gRPC
   

   View Slide

5. ©2020 Wantedly, Inc.
   Background: our products
   Our products
   w 8BOUFEMZ7JTJUKPCTBOEDBOEJEBUFTNBUDIJOH
   w 8BOUFEMZ1FPQMFQSPGFTTJPOBMDPOUBDUTNBOBHFNFOU
   w FUD
   These products share the same components
   such as “user’s profile data”
   

   View Slide

6. ©2020 Wantedly, Inc.
   Background: our microservices architecture
   Our products are built with 100+ microservices
   w NJDSPTFSWJDFTXSJUUFOJO3VCZ (P 1ZUIPO /PEFKT FUDSVOPOBLTDMVTUFS
   w 8FIBECFFOVTJOH)551GPSUIFNBKPSJUZPGPVSNJDSPTFSWJDFT
   w lVTFS`TQSPpMFEBUBzJTNBOBHFECZl6TFS4FSWJDFz XIJDIJTXSJUUFOJO3VCZ
   LTDMVTUFS
   $PSF4FSWJDFT
   6TFS4FSWJDF 3VCZ
   
   

   View Slide

7. ©2020 Wantedly, Inc.
   What is “UserService”?
   Get/Update user’s profile data
   Low latency and high throughput are required
   w 6TFEGSPNNBOZNJDSPTFSWJDFTIJHIUISPVHIQVUJTSFRVJSFE
   w -BUFODZPGUFOEJSFDUMZB⒎FDUTUIFVTFSFYQFSJFODFMPXMBUFODZJTSFRVJSFE
   6TFS4FSWJDF
   (FUVQEBUFVTFS`TQSPpMFEBUB
   

   View Slide

8. ©2020 Wantedly, Inc.
   What was the problem of UserService?
   Latency caused by using HTTP/1.1
   w 8JUIPVULFFQBMJWF )551EPFTOPUQFSGPSNXFMMGPSTFWFSBMSFBTPOT
   w FH%/4MPPLVQ 5$1XBZIBOETIBLF 5$1TMPXTUBSU FUD
   6TFS4FSWJDF
   -BUFODZDBVTFECZVTJOH)551
   

   View Slide

9. ©2020 Wantedly, Inc.
   

   View Slide

10. ©2020 Wantedly, Inc.
    Why gRPC?
    High-Performance
    w -POHMJWFE SFBMUJNFDPNNVOJDBUJPOTUSFBNTCZVTJOH)551
    w -PXMBUFODZ IJHIUISPVHIQVU
    Protocol Buffers
    w l4JOHMFTPVSDFPGUSVUIzPGBTFSWJDFEFpOJUJPO
    w "EEJUJPOBMHPPEQPJOUXFXFSFBMSFBEZVTJOH1SPUPDPM#V⒎FSTPWFS)551
    Multiple Languages Support
    w 8FIBWFNJDSPTFSWJDFTXSJUUFOJOTFWFSBMMBOHVBHFT BOEH31$TVQQPSUTBMMPGUIFN
    

    View Slide

11. ©2020 Wantedly, Inc.
    For these reasons,
    we chose gRPC
    

    View Slide

12. ©2020 Wantedly, Inc.
    2. How we migrated from HTTP to gRPC
    

    View Slide

13. ©2020 Wantedly, Inc.
    1. Preparation
    2. Migration
    How we migrated from HTTP to gRPC
    

    View Slide

14. ©2020 Wantedly, Inc.
    1. Preparation
    2. Migration
    How we migrated from HTTP to gRPC
    

    View Slide

15. ©2020 Wantedly, Inc.
    1. Schema Management
    2. Monitoring
    3. Productive Development Environment
    4. Load Balancing
    Preparations for using gRPC
    

    View Slide

16. ©2020 Wantedly, Inc.
    1. Schema Management
    2. Monitoring
    3. Productive Development Environment
    4. Load Balancing
    Preparations for using gRPC
    

    View Slide

17. ©2020 Wantedly, Inc.
    Without a monorepo, we have to share the
    same proto files across the multiple repositories
    Schema Management
    4BNFQSPUPpMFT
    3FQPTJUPSZ" 3FQPTJUPSZ# 3FQPTJUPSZ$
    

    View Slide

18. ©2020 Wantedly, Inc.
    How should we share proto files?
    w $PQZQBTUFNBOVBMMZ
    7FSZIBSEUPVQEBUFNVMUJQMFSFQPTJUPSJFTDPOUJOVPVTMZ
    w %PXOMPBEQSPUPpMFTBVUPNBUJDBMMZ
    /FFEUPHFOFSBUFUIFDPEFNBOVBMMZ
    4V⒎FSGSPNUIFEJ⒎FSFODFTJOFBDIEFWFMPQFS`TFOWJSPONFOU FHWFSTJPOPG
    QSPUPD QSFTFODFPGQMVHJOT FUD
    
    We decided to make a central repository
    Schema Management
    

    View Slide

19. ©2020 Wantedly, Inc.
    apis: a central repository for schema management
    w lBQJTzTUPSFTBMMQSPUPpMFTGPSPVSDPNQBOZ
    Schema Management
    

    View Slide

20. ©2020 Wantedly, Inc.
    How to use “apis” repository?
    Code is generated in the CI pipelines of “apis”
    w 3VCZ (P BOE/PEFKTDPEFJTHFOFSBUFEGPSFBDIQSPUPpMF
    w (FOFSBUFEDPEFJTEJTUSJCVUFEWJBQBDLBHFNBOBHFST #VOEMFS (P.PEVMFT OQN
    
    w %FWFMPQFSTEPOUOFFEUPHFOFSBUFDPEFCZUIFNTFMWFT
    

    View Slide

21. ©2020 Wantedly, Inc.
    # Gemfile
    gem 'apis', git: 'git@github.com:wantedly/apis-ruby.git', tag: 'xxx.yyy.zzz'
    3VCZXJUI#VOEMFS
    

    View Slide

22. ©2020 Wantedly, Inc.
    How we develop gRPC servers and clients
    in Ruby?
    

    View Slide

23. ©2020 Wantedly, Inc.
    gRPC servers: Implement a class which inherits
    a gRPC service class fetched from “apis”
    w #ZJNQMFNFOUJOHJOTUBODFNFUIPETGPSFBDI31$NFUIPE XFDBOVTFH31$
    TFSWFSTJO3VCZ
    # NOTE: Load UsersPb::UserService from apis
    require "wantedly/users/users_services_pb"
    class UserServer < UsersPb::UserService::Service
    # @param [UsersPb::ListUsersRequest] req
    # @param [GRPC::ActiveCall::SingleReqView] call
    # @return [UsersPb::ListUsersResponse]
    def list_users(req, call)
    svc = UsersReadService.new(req)
    UsersPb::ListUsersResponse.new(users: svc.users)
    end
    end
    How we develop gRPC servers?
    

    View Slide

24. ©2020 Wantedly, Inc.
    gRPC clients: Use a stub class fetched from “apis”
    w #ZVTJOHBTUVCPCKFDUBTBH31$DMJFOU XFDBODPNNVOJDBUFXJUIH31$TFSWFST
    # NOTE: Load UsersPb::UserService from apis
    require "wantedly/users/users_services_pb"
    client = UsersPb::UserService::Stub.new(
    url,
    :this_channel_is_insecure
    )
    How we develop gRPC clients?
    

    View Slide

25. ©2020 Wantedly, Inc.
    1. Schema Management
    2. Monitoring
    3. Productive Development Environment
    4. Load Balancing
    Preparations for using gRPC
    

    View Slide

26. ©2020 Wantedly, Inc.
    Application monitoring is required in production
    w 8FXFSFVTJOH%BUBEPH /FX3FMJD )POFZCBEHFS FUDGPSNPOJUPSJOHPVS
    )551TFSWFST
    8BOUFEUPDPOUJOVFUPVTFUIFNGPSH31$TFSWFST
    Monitoring
    

    View Slide

27. ©2020 Wantedly, Inc.
    Monitoring
    How to monitor gRPC servers?
    => gRPC interceptors
    w /FFEUPTFOENFUSJDTPGFBDIH31$SFRVFTU
    w 8JUIH31$JOUFSDFQUPST XFDBOJNQMFNFOUTVDIGFBUVSFT
    $MJFOUH31$JOUFSDFQUPST
    $MJFOU
    4FSWFSH31$JOUFSDFQUPST
    4FSWFS
    3FRVFTU
    3FTQPOTF
    

    View Slide

28. ©2020 Wantedly, Inc.
    Monitoring
    We implemented some gRPC interceptors
    w H31$JOUFSDFQUPSTGPSFBDI4BB4QSPEVDUBSFOPUP⒏DJBMMZQSPWJEFEBUUIJTUJNF
    w 8FJNQMFNFOUFEUIFNJO3VCZ BOENBEFUIFNBWBJMBCMFUPUIF3VCZDPNNVOJUZ
    BTPQFOTPVSDFTPGUXBSF 044
    
    

    View Slide

29. ©2020 Wantedly, Inc.
    1. Schema Management
    2. Monitoring
    3. Productive Development Environment
    4. Load Balancing
    Required preparations for using gRPC
    

    View Slide

30. ©2020 Wantedly, Inc.
    Need to create a productive development
    environment for gRPC
    w 'PS)551TFSWFST UIFFDPTZTUFNJTNBUVSFBOENBOZPQFOTPVSDFMJCSBSJFTFYJTU
    w 5IJTJTOPUUIFDBTFXJUIH31$TFSWFST FTQFDJBMMZJO3VCZ8FOFFEUPDSFBUFB
    QSPEVDUJWJUZEFWFMPQNFOUFOWJSPONFOUCZPVSTFMWFT
    Productive Development Environment
    

    View Slide

31. ©2020 Wantedly, Inc.
    How to create a productive development
    environment?
    

    View Slide

32. ©2020 Wantedly, Inc.
    Use a common utility library named “servicex”
    w 8FXBOUTPNFVTFGVMGFBUVSFT FHIPUSFMPBEJOH VUJMJUZGPSQSPUPDPMCV⒎FST FUD
    
    w 8FXBOUUPBWPJESFQFBUJOHUIFTBNFDPOpHVSBUJPOJONVMUJQMFNJDSPTFSWJDFT
    JNQMFNFOUFElTFSWJDFYz
    How to create a productive development environment?
    .JDSPTFSWJDF" .JDSPTFSWJDF# .JDSPTFSWJDF$
    lTFSWJDFYz
    

    View Slide

33. ©2020 Wantedly, Inc.
    gRPC servers: grpc-server command
    w H31$TFSWJDFDMBTTFTBSFBVUPNBUJDBMMZMPBEFE
    w 4PNFH31$JOUFSDFQUPSTBSFBVUPNBUJDBMMZTFU
    w )PUSFMPBEJOHGFBUVSF
    w %FWFMPQFSTDBOKVTUGPDVTPOUIFJNQMFNFOUBUJPOPGH31$TFSWJDFDMBTTFT
    The features of our internal library “servicex”
    $ bundle exec grpc-server
    handling /grpc.health.v1.Health/Check with #
    handling /grpc.health.v1.Health/Watch with #Grpc::Health::Checker(Grpc::Health::V1::Health::Service)#watch>
    handling /wantedly.users.UserService/ListUsers with #
    .
    .
    .
    gRPC server starting...
    * Listening on tcp://0.0.0.0:6046
    * Environment: development
    

    View Slide

34. ©2020 Wantedly, Inc.
    gRPC clients: Servicex::Grpc.stub_for
    w 4PNFH31$JOUFSDFQUPSTBSFBVUPNBUJDBMMZTFU
    w 6TFS"HFOUJTBVUPNBUJDBMMZDPOpHVSFE
    w %FWFMPQFSTDBOKVTUGPDVTPOJNQMFNFOUJOHUIFCVTJOFTTMPHJD
    The features of our internal library “servicex”
    # NOTE: Load UsersPb::UserService from apis
    require "wantedly/users/users_services_pb"
    grpc_server_url = “xxx”
    client = Servicex::Grpc.stub_for(
    UsersPb::UserService,
    grpc_server_url,
    )
    client.get_user(UsersPb::GetUserRequest.new(…))
    

    View Slide

35. ©2020 Wantedly, Inc.
    Pb: utility for using Protocol Buffers
    w 8JUIl1Cz XFDBODSFBUFQSPUPCVGPCKFDUTNPSFFBTJMZ
    w FH1CUP@UJNFTUBNQDBODSFBUFBUJNFTUBNQPCKFDUGSPNB4USJOHPCKFDU
    The features of our internal library “servicex”
    [1] pry(main)> Pb.to_timestamp("2019-05-15T00:00:00+09:00")
    =>
    8JUIl1Cz
    [1] pry(main)> Google::Protobuf::Timestamp.new(
    [1] pry(main)* seconds: Time.parse("2019-05-15T00:00:00+09:00").to_i
    [1] pry(main)* )
    =>
    8JUIPVUl1Cz
    

    View Slide

36. ©2020 Wantedly, Inc.
    With Pb.to_proto, we can create user-defined
    protobuf objects easily
    The features of “Pb”
    [1] pry(main)> Pb.to_proto(Account, { created_at: "2019-05-15T00:00:00+09:00" })
    => created_at: >
    8JUIl1Cz
    [1] pry(main)> Account.new(
    [1] pry(main)* created_at: Google::Protobuf::Timestamp.new(
    [1] pry(main)* seconds: Time.parse("2019-05-15T00:00:00+09:00").to_i
    [1] pry(main)* )
    [1] pry(main)* )
    => created_at: >
    8JUIPVUl1Cz
    

    View Slide

37. ©2020 Wantedly, Inc.
    Pb is published as open source software (OSS)
    w 1CJTJODMVEFEJOTFSWJDFY CVUXFNBEFJUBWBJMBCMFUPUIF3VCZDPNNVOJUZBT
    PQFOTPVSDFTPGUXBSF 044
    
    w IUUQTSVCZHFNTPSHHFNTUIF@QC
    w IUUQTSVCZHFNTPSHHFNTQCTFSJBMJ[FS
    The features of “Pb”
    

    View Slide

38. ©2020 Wantedly, Inc.
    Use gRPC combined with parts of Ruby on Rails
    w 8FNJHSBUFEGSPN3VCZPO3BJMT BOEXBOUFEUPLFFQVTJOHQBSUTPGGFBUVSFTJOJU
    FH"DUJWF3FDPSE "DUJWF4VQQPSU "VUPMPBEJOH SBJMTDPOTPMF FUD
    
    %FDJEFEUPVTFHSQDHFNXJUISBJMTHFN
    w 8FGPVOEUIBUJU`TWFSZVTFGVMUPCVJMEBH31$TFSWFSXJUIQBSUTPG3VCZPO3BJMT
    8FSFDPNNFOEUIJTDPOpHVSBUJPOJGZPV`SFJOUIFTBNFTJUVBUJPO
    # Gemfile
    ruby '2.7.1'
    gem 'grpc'
    gem 'rails', '~> 6.0.3'
    How to create a productive development environment?
    

    View Slide

39. ©2020 Wantedly, Inc.
    1. Schema Management
    2. Monitoring
    3. Productive Development Environment
    4. Load Balancing
    Required preparations for using gRPC
    

    View Slide

40. ©2020 Wantedly, Inc.
    For gRPC, L4 Load-Balancers aren’t very useful
    w H31$JTCVJMUPO)551 XIJDIJTEFTJHOFEUPIBWFBTJOHMFMPOHMJWFE5$1
    DPOOFDUJPO
    w *O--PBE#BMBODFST PODFUIF5$1DPOOFDUJPOJTFTUBCMJTIFE BMMSFRVFTUTXJMMHFU
    QJOOFEUPBTJOHMFEFTUJOBUJPO
    Load Balancing
    6TFS4FSWJDF
    4PNF.JDSPTFSWJDF
    

    View Slide

41. ©2020 Wantedly, Inc.
    We use Istio for L7 load-balancing in k8s clusters
    w &OWPZDPOUBJOFSTDPOpHVSFECZ*TUJPBSFJOKFDUFEBTTJEFDBSQSPYJFT BOEVTFEGPS
    - "QQMJDBUJPO
    MPBECBMBODJOH
    Load Balancing
    6TFS4FSWJDF
    4PNF.JDSPTFSWJDF
    &OWPZ
    

    View Slide

42. ©2020 Wantedly, Inc.
    1. Preparation
    2. Migration
    How we migrated from HTTP to gRPC
    

    View Slide

43. ©2020 Wantedly, Inc.
    Basic strategy was “Try Small”
    w "UpSTU XFNBEFBQSPUPUZQFBTBQSPPGPGDPODFQU 1P$
    
    w /FYU XFVTFEH31$GPSBQBSUPGUIFUSB⒏DJOQSPEVDUJPO BOEHSBEVBMMZJODSFBTFE
    UIFQFSDFOUBHF
    How we migrated from HTTP to gRPC?
    

    View Slide

44. ©2020 Wantedly, Inc.
    PoC - schematic diagram
    QSPEVDUJPOLTDMVTUFS
    6TFS4FSWJDF
    )551
    Tried PoC in development k8s cluster
    w #ZVTJOHlEFWFMPQNFOULTDMVTUFSz XFDPVMEUSZH31$XJUIPVUXPSSZJOHBCPVU
    B⒎FDUJOHQSPEVDUJPOFOWJSPONFOU
    EFWFMPQNFOULTDMVTUFS
    6TFS4FSWJDF
    5SZH31$
    

    View Slide

45. ©2020 Wantedly, Inc.
    The result of PoC
    Latency decreased dramatically
    w #ZTFFJOHUIJTSFTVMU XFEFDJEFEUPUSZH31$JOBQSPEVDUJPOLTDMVTUFS
    %ile
    Latency [ms]
    

    View Slide

46. ©2020 Wantedly, Inc.
    How we tried gRPC in production?
    

    View Slide

47. ©2020 Wantedly, Inc.
    How we tried gRPC in production?
    At first, we used gRPC for a small part of the traffic
    w #ZUSZJOHTNBMM XFDPVMENJOJNJ[FUIFQSFQBSBUJPODPTUTUPNJOJNJ[FUIFSJTL
    Then, gradually increased the percentage
    w 'JOBMMZ XFDPVMEBDIJFWFUIFDPNQMFUFNJHSBUJPO
    )551
    
    6TFS
    H31$
    
    *ODSFBTFUIFQFSDFOUBHF
    )551
    
    6TFS
    H31$
    
    

    View Slide

48. ©2020 Wantedly, Inc.
    With these steps,
    we migrated to gRPC
    

    View Slide

49. ©2020 Wantedly, Inc.
    3. What we achieved by using gRPC
    

    View Slide

50. ©2020 Wantedly, Inc.
    Results achieved by using gRPC
    Average Latency decreased
    w "WFSBHFMBUFODZEFDSFBTFECZBCPVUNTQFSSFRVFTU
    w 5IFJNQBDUXBTNPSFSFNBSLBCMFXIFONBOZSFRVFTUTUP6TFS4FSWJDFXFSF
    JOWPMWFE'PSFYBNQMF JOTPNFNJDSPTFSWJDFT BWFSBHFMBUFODZEFDSFBTFECZBIBMG
    HTTP/1.1 gRPC
    Average latency decreased
    from 130ms to 65ms in this service
    

    View Slide

51. ©2020 Wantedly, Inc.
    We achieved huge
    performance improvement
    

    View Slide

52. ©2020 Wantedly, Inc.
    What we have been doing since the
    migration?
    

    View Slide

53. ©2020 Wantedly, Inc.
    The use of gRPC is increasing throughout the
    organization
    w "MNPTUBMMDMJFOUTPG6TFS4FSWJDFBSFVTJOHH31$
    w /FXNJDSPTFSWJDFTBSFCFJOHEFWFMPQFEVTJOHH31$
    w H31$TUBSUFEUPCFVTFEJOPUIFSMBOHVBHFT (P /PEFKT FUD
    
    What we have been doing since the migration?
    3VCZ
    3VCZ
    /PEF
    3VCZ (P
    H31$
    /FX
    

    View Slide

54. ©2020 Wantedly, Inc.
    Summary
    

    View Slide

55. ©2020 Wantedly, Inc.
    Summary
    1. Why gRPC
    w )JHI1FSGPSNBODF
    w 1SPUPDPM#V⒎FST
    w .VMUJQMF-BOHVBHFT4VQQPSU
    2. How we migrated from HTTP to gRPC
    w $SFBUFEBXPSLqPXGPS1SPUPDPM#V⒎FSTVTJOHPVSlBQJTzSFQPTJUPSZ
    w %FWFMPQFEMJCSBSJFTGPSQSPEVDUJWJUZ BOEQVCMJTIFETPNFPGUIFNBT044
    w #BTJDTUSBUFHZGPSNJHSBUJPOXBTl5SZ4NBMMz
    3. What we achieved by using gRPC
    w "DIJFWFEUIFQFSGPSNBODFJNQSPWFNFOU
    w 5IFVTFPGH31$JTJODSFBTJOHUISPVHIPVUUIFPSHBOJ[BUJPO
    

    View Slide

56. ©2020 Wantedly, Inc.
    • Alina Grubnyak https://unsplash.com/photos/hdpDLIXZg7Y
    Photo Credit
    

    View Slide