[RubyConf 2018] High-Speed Cables for Ruby

RubyConf: https://rubyconf.org/2018/program.html#session-712
Video: https://www.youtube.com/watch?v=8XRcOZXOzV4
AnyCable: https://anycable.io

Transcript

1. HIGH-SPEED CABLES
   FOR RUBY
   Vladimir Dementyev
   @evilmartians
   

   View Slide

2. View Slide

3. CABLES
   Tools for building
   real-time applications
   

   View Slide

4. palkan_tula
   palkan RubyConf 2018
   The real-time web is a web … that enable users to receive
   information as soon as it is published by its authors,
   rather than … check a source periodically for updates.
   4
   * https://en.wikipedia.org/wiki/Real-time_web
   BORING DEFINITION
   

   View Slide

5. palkan_tula
   palkan RubyConf 2018
   5
   Request-Response
   REAL-TIME
   

   View Slide

6. palkan_tula
   palkan RubyConf 2018
   5
   Request-Response
   REAL-TIME
   

   View Slide

7. palkan_tula
   palkan RubyConf 2018
   6
   Bi-Directional
   Persistent
   REAL-TIME
   

   View Slide

8. palkan_tula
   palkan RubyConf 2018
   6
   Bi-Directional
   Persistent
   REAL-TIME
   

   View Slide

9. palkan_tula
   palkan RubyConf 2018
   7
   The opening of the first McDonalds in Russia,
   Moscow, 1991
   REAL-TIME
   Limited number of
   concurrent requests
   (request queue)
   

   View Slide

10. palkan_tula
    palkan RubyConf 2018
    8
    Large number of
    concurrent clients
    REAL-TIME
    Opening gala,
    Millennium Biltmore, 1923
    

    View Slide

11. palkan_tula
    palkan RubyConf 2018
    EXAMPLES
    9
    Messaging (i.e. chats)
    Notifications
    Live updates
    Online games
    Other
    

    View Slide

12. palkan_tula
    palkan RubyConf 2018
    WEBSOCKETS
    10
    

    View Slide

13. palkan_tula
    palkan RubyConf 2018
    CONCURRENCY
    11
    

    View Slide

14. © https://twitter.com/erneestoc/status/974485805770072064
    

    View Slide

15. palkan_tula
    palkan RubyConf 2018
    THE RED RUBY PILL?
    13
    

    View Slide

16. palkan_tula
    palkan RubyConf 2018
    14
    Vladimir
    Dementyev
    

    View Slide

17. palkan_tula
    palkan RubyConf 2018
    15
    @palkan
    @palkan_tula
    Vladimir Dementyev
    789
    645
    330
    415
    

    View Slide

18. palkan_tula
    palkan RubyConf 2018
    ACTION CABLE TESTING
    https://github.com/rails/rails/pull/33659
    16
    

    View Slide

19. palkan_tula
    palkan RubyConf 2018
    17
    TEAM
    LID
    Team
    Me
    

    View Slide

20. palkan_tula
    palkan RubyConf 2018
    18
    

    View Slide

21. palkan_tula
    palkan RubyConf 2018
    19
    

    View Slide

22. palkan_tula
    palkan RubyConf 2018
    evilmartians.com
    20
    

    View Slide

23. View Slide

24. palkan_tula
    palkan RubyConf 2018
    evilmartians.com/blog
    22
    

    View Slide

25. palkan_tula
    palkan RubyConf 2018
    23
    evilmartians.com
    Brooklyn, NY
    

    View Slide

26. THIS TALK
    

    View Slide

27. palkan_tula
    palkan RubyConf 2018
    THE TALK
    25
    What is “cable”? ✅
    The tale of Action Cable, Rack & Ruby 3x3
    From Action to Any: another point of view
    

    View Slide

28. RUBY
    &
    CABLES
    RailsConf 2015
    

    View Slide

29. palkan_tula
    palkan RubyConf 2018
    IN A NUTSHELL
    27
    Server
    Client
    WebSocket
    Client
    WebSocket
    

    View Slide

30. palkan_tula
    palkan RubyConf 2018
    IN A NUTSHELL
    27
    Server
    Client
    WebSocket
    Client
    WebSocket
    stream C
    Broadcaster
    stream B
    stream A
    stream B
    

    View Slide

31. palkan_tula
    palkan RubyConf 2018
    IN A NUTSHELL
    27
    Server
    Client
    WebSocket
    Client
    WebSocket
    stream C
    Broadcaster
    stream B
    stream A
    stream B
    channel X
    channel Y
    channel Z
    

    View Slide

32. palkan_tula
    palkan RubyConf 2018
    CHANNELS
    28
    class AnswersChannel < ApplicationCable ::Channel
    def subscribed
    reject_subscription unless current_user.admin?
    end
    def follow(params)
    stream_from "questions/ #{params['id']}"
    end
    end
    

    View Slide

33. palkan_tula
    palkan RubyConf 2018
    CHAT IN 5MIN
    29
    

    View Slide

34. palkan_tula
    palkan RubyConf 2018
    CHAT IN 5MIN
    29
    

    View Slide

35. palkan_tula
    palkan RubyConf 2018
    BENCHMARKS
    30
    TODAY’S SPECIALS
    HashRocket RTT $9.99
    RAM (all-you-can-eat) $19.59
    CPU Steak 8’’ $20.18
    

    View Slide

36. palkan_tula
    palkan RubyConf 2018
    SHOOTOUT
    31
    Client
    Server
    broadcast
    to all
    message
    send message back
    https://hashrocket.com/blog/posts/websocket-shootout
    

    View Slide

37. palkan_tula
    palkan RubyConf 2018
    SHOOTOUT
    32
    Broadcast RTT
    0,0s
    1,2s
    2,4s
    3,6s
    4,8s
    6,0s
    7,2s
    8,4s
    9,6s
    10,8s
    12,0s
    Number of connections
    1000 2000 3000 4000 5000 6000 7000 8000 9000 10000
    Go Erlang Action Cable (8x)
    https://github.com/anycable/anycable/tree/master/benchmarks
    

    View Slide

38. View Slide

39. CABLE
    THEOREM
    Action
    Cable
    Low
    Latency
    Crowded
    Streams
    Low
    Latency
    Action
    Cable
    

    View Slide

40. High Resources Usage
    CABLE
    THEOREM
    Action
    Cable
    Low
    Latency
    Crowded
    Streams
    Low
    Latency
    Action
    Cable
    

    View Slide

41. palkan_tula
    palkan RubyConf 2018
    CPU
    36
    * running WebSocket shootout
    

    View Slide

42. palkan_tula
    palkan RubyConf 2018
    CPU
    36
    * running WebSocket shootout
    

    View Slide

43. palkan_tula
    palkan RubyConf 2018
    MEMORY
    37
    RSS
    0MB
    160MB
    320MB
    480MB
    640MB
    800MB
    960MB
    1120MB
    1280MB
    1440MB
    1600MB
    Number of connections
    1000 2000 3000 4000 5000 6000 7000 8000 9000 10000
    Go Erlang Action Cable (8x)
    

    View Slide

44. WHAT ABOUT
    REAL LIFE?
    …or don’t trust benchmarks
    I WANT TO
    BELIEVE IN
    BENCHMARKS
    

    View Slide

45. palkan_tula
    palkan RubyConf 2018
    39
    “With Action Cable, we could
    easily have 20+ 1GB dynos
    running during the weekends,
    with every growing memory”
    –Jon Stenqvist,
    CEO, Equipe
    equipe.com
    NOTE: having ~5-10k conns at peak time
    

    View Slide

46. WHYYYY???!!
    

    View Slide

47. palkan_tula
    palkan RubyConf 2018
    ACTION CABLE
    41
    rack.hijack
    

    View Slide

48. palkan_tula
    palkan RubyConf 2018
    rack.hijack
    42
    Server
    Application
    

    View Slide

49. palkan_tula
    palkan RubyConf 2018
    rack.hijack
    42
    Server
    Application
    IO
    

    View Slide

50. palkan_tula
    palkan RubyConf 2018
    rack.hijack
    42
    Server
    Application
    

    View Slide

51. palkan_tula
    palkan RubyConf 2018
    rack.hijack
    42
    Server
    Application
    IO
    

    View Slide

52. palkan_tula
    palkan RubyConf 2018
    THE hijack PRICE
    43
    https://bowild.wordpress.com/2018/05/01/rubys-rack-push-decoupling-the-real-time-web-application-from-the-web/
    –Bo
    “Due to code duplication and extra work,
    the memory consumption for hijack
    based solutions is higher and their
    performance is slower (more system calls,
    more context switches, etc’)..”
    https://bowild.wordpress.com/2018/05/01/rubys-rack-push-decoupling-the-real-time-web-application-from-the-web/
    

    View Slide

53. palkan_tula
    palkan RubyConf 2018
    THE hijack PRICE
    44
    Separate IO loop (server)
    WebSockets protocol implementation overhead
    

    View Slide

54. palkan_tula
    palkan RubyConf 2018
    plezi.io
    github.com/boazsegev/iodine
    WHO IS BO?
    45
    Boaz Segev (Bo)
    musician (now),
    developer (past)
    

    View Slide

55. palkan_tula
    palkan RubyConf 2018
    Broadcast RTT
    0,0s
    1,2s
    2,4s
    3,6s
    4,8s
    6,0s
    7,2s
    8,4s
    9,6s
    10,8s
    12,0s
    Number of connections
    1000 3000 5000 7000 9000
    Go Iodine Action Cable (8x)
    IODINE
    46
    https://github.com/anycable/anycable/tree/master/benchmarks
    RSS
    0MB
    160MB
    320MB
    480MB
    640MB
    800MB
    960MB
    1120MB
    1280MB
    1440MB
    1600MB
    Number of connections
    1000 3000 5000 7000 9000
    Go Iodine Action Cable (8x)
    

    View Slide

56. palkan_tula
    palkan RubyConf 2018
    RACK API: FUTURE
    47
    https://github.com/rack/rack/pull/1272
    

    View Slide

57. palkan_tula
    palkan RubyConf 2018
    RACK API: FUTURE
    48
    rack.hijack
    Direct access to IO (socket)
    rack.upgrade
    Public interface to “abstracted” IO managed by
    web server
    

    View Slide

58. palkan_tula
    palkan RubyConf 2018
    RACK API IN RAILS
    49
    https://github.com/rails/rails/pull/33295
    

    View Slide

59. palkan_tula
    palkan RubyConf 2018
    RACK API IN RAILS
    49
    https://github.com/rails/rails/pull/33295
    NO PERFORMANCE
    IMPROVEMENTS
    

    View Slide

60. palkan_tula
    palkan RubyConf 2018
    ACTION CABLE
    50
    rack.hijack
    Long-lived objects
    

    View Slide

61. palkan_tula
    palkan RubyConf 2018
    ALLOCATIONS
    51
    Retained objects for one connection:
    websocket ~40
    actioncable ~640
    other ~100
    ~60kB
    

    View Slide

62. palkan_tula
    palkan RubyConf 2018
    ACTION CABLE
    52
    rack.hijack
    Long-lived objects
    Heap fragmentation
    

    View Slide

63. palkan_tula
    palkan RubyConf 2018
    HEAPY HEAPY SHAKE
    53
    

    View Slide

64. palkan_tula
    palkan RubyConf 2018
    HEAPY HEAPY SHAKE
    53
    Heap
    fragmentation?
    Are you sure?
    

    View Slide

65. RUBY 3X3
    The future we’re
    waiting for
    

    View Slide

66. View Slide

67. palkan_tula
    palkan RubyConf 2018
    56
    GENERATIONAL
    HYPOTHESIS
    

    View Slide

68. palkan_tula
    palkan RubyConf 2018
    RUBY 3X3
    57
    Compacting GC
    

    View Slide

69. palkan_tula
    palkan RubyConf 2018
    RUBY 3X3
    58
    Compacting GC
    Guilds
    

    View Slide

70. palkan_tula
    palkan RubyConf 2018
    RUBY DOLL v2
    59
    Thread Fiber
    

    View Slide

71. palkan_tula
    palkan RubyConf 2018
    WHAT IS FIBER?
    https://twitter.com/avdi/status/1057256814721556482
    60
    

    View Slide

72. palkan_tula
    palkan RubyConf 2018
    RUBY DOLL v2
    61
    Thread Fiber
    1MB
    128KB
    

    View Slide

73. palkan_tula
    palkan RubyConf 2018
    RUBY DOLL v3
    62
    http://www.atdot.net/~ko1/activities/2018_rubyconf2018.pdf
    Thread Fiber
    Guild
    1MB
    128KB
    

    View Slide

74. palkan_tula
    palkan RubyConf 2018
    –Julian Coolesh (@raventid)
    “Guilds are for parallelism,
    not for concurrency.”
    63
    

    View Slide

75. palkan_tula
    palkan RubyConf 2018
    RUBY 3X3
    64
    Compacting GC
    Guilds
    Dynamic thread stack size
    https://rubykaigi.org/2018/presentations/sugiyama-k.html
    

    View Slide

76. palkan_tula
    palkan RubyConf 2018
    RUBY DOLL v3
    65
    Thread Threadlet
    Guild Fiber
    https://bugs.ruby-lang.org/issues/13618
    

    View Slide

77. palkan_tula
    palkan RubyConf 2018
    RUBY 3X3
    66
    Compacting GC
    Guilds
    Dynamic thread stack size
    Threadlets
    

    View Slide

78. palkan_tula
    palkan RubyConf 2018
    RUBY 3X3
    66
    Compacting GC
    Guilds
    Dynamic thread stack size
    Threadlets
    

    View Slide

79. palkan_tula
    palkan RubyConf 2018
    binding.pry
    67
    

    View Slide

80. palkan_tula
    palkan RubyConf 2018
    CABLE RACE
    68
    

    View Slide

81. palkan_tula
    palkan RubyConf 2018
    CABLE RACE
    69
    

    View Slide

82. AnyCable
    One cable to rule them all
    

    View Slide

83. View Slide

84. palkan_tula
    palkan RubyConf 2018
    Client
    Channels
    Broadcaster
    Server
    ACTION CABLE
    72
    

    View Slide

85. palkan_tula
    palkan RubyConf 2018
    Action Cable
    73
    Client
    Channels
    Broadcaster
    Server
    AnyCable
    

    View Slide

86. palkan_tula
    palkan RubyConf 2018
    AnyCable
    74
    ?
    WS Server
    

    View Slide

87. palkan_tula
    palkan RubyConf 2018
    75
    https://grpc.io
    gRPC
    

    View Slide

88. palkan_tula
    palkan RubyConf 2018
    76
    https://grpc.io
    gRPC =
    universal
    RPC
    framework
    

    View Slide

89. palkan_tula
    palkan RubyConf 2018
    77
    https://grpc.io
    gRPC =
    HTTP/2 +
    protobuf
    

    View Slide

90. palkan_tula
    palkan RubyConf 2018
    AnyCable
    78
    syntax = "proto3";
    package anycable;
    service RPC {
    rpc Connect (ConnectionRequest) returns (ConnectionResponse) {}
    rpc Command (CommandMessage) returns (CommandResponse) {}
    rpc Disconnect (DisconnectRequest) returns (DisconnectResponse) {}
    }
    https://docs.anycable.io/#/rpc_proto
    

    View Slide

91. palkan_tula
    palkan RubyConf 2018
    AnyCable
    79
    Go WS
    

    View Slide

92. View Slide

93. palkan_tula
    palkan RubyConf 2018
    AnyCable
    81
    anycable-go
    erlycable
    WebSocket Servers
    

    View Slide

94. palkan_tula
    palkan RubyConf 2018
    Broadcast RTT
    0,0s
    1,2s
    2,4s
    3,6s
    4,8s
    6,0s
    7,2s
    8,4s
    9,6s
    10,8s
    12,0s
    Number of connections
    1000 3000 5000 7000 9000
    AnyCable-Go Iodine
    Action Cable (8x)
    AnyCable
    82
    https://github.com/anycable/anycable/tree/master/benchmarks
    RSS
    0MB
    160MB
    320MB
    480MB
    640MB
    800MB
    960MB
    1120MB
    1280MB
    1440MB
    1600MB
    Number of connections
    1000 3000 5000 7000 9000
    AnyCable-Go Iodine
    Action Cable (8x)
    

    View Slide

95. palkan_tula
    palkan RubyConf 2018
    CPU
    83
    anycable-go erlycable actioncable
    

    View Slide

96. palkan_tula
    palkan RubyConf 2018
    CPU
    83
    anycable-go erlycable actioncable
    

    View Slide

97. palkan_tula
    palkan RubyConf 2018
    equipe.com
    84
    ActionCable: 20+ 2X (1GB) dynos AnyCable: 4 X (0.5GB) dynos
    $1000 $100
    

    View Slide

98. palkan_tula
    palkan RubyConf 2018
    SHORT-LIVED OBJECTS
    85
    WS RPC
    connect
    identifiers (JSON + GlobalID)
    command (subscribe/perform)
    with identifiers
    transmissions, streams to subscribe
    

    View Slide

99. palkan_tula
    palkan RubyConf 2018
    AnyCable
    86
    Go WS
    Bottleneck?
    

    View Slide

100. palkan_tula
     palkan RubyConf 2018
     gRPC RPS
     87
     RPC type \ Concurrency 1 10 50
     AnyCable RPC 1600 2200 2400
     Noop RPC with connection object building* 1500 1900 2300
     Noop RPC 3000 4600 5500
     * Only build Action Cable connection object without performing an action
     https://github.com/anycable/anycable/blob/master/benchmarks/2018-05-27-rpc-bench.md
     

     View Slide

101. palkan_tula
     palkan RubyConf 2018
     MRUBY CACHE
     https://rubykaigi.org/2018/presentations/palkan_tula.html
     88
     

     View Slide

102. palkan_tula
     palkan RubyConf 2018
     PLUG-N-PLAY
     89
     # Gemfile
     gem 'anycable-rails'
     # config/cable.yml
     production:
     type: :any_cable
     # config/environments/production.rb
     config.action_cable.url = 'ws: //example.com:3334'
     # install WebSocket server
     $ brew install anycable/anycable/anycable-go
     $ bundle exec anycable # => Run gRPC server
     $ anycable-go --port=3334 # => Run WebSocket server
     

     View Slide

103. palkan_tula
     palkan RubyConf 2018
     MORE FEATURES
     90
     Zero-disconnect deployment
     

     View Slide

104. palkan_tula
     palkan RubyConf 2018
     DISCONNECT
     91
     WebSocket Server
     App
     App
     gRPC
     App
     App
     App
     App
     Envoy Proxy
     * https://www.envoyproxy.io
     

     View Slide

105. palkan_tula
     palkan RubyConf 2018
     MORE FEATURES
     92
     Zero-disconnect deployment
     Metrics & Stats
     

     View Slide

106. palkan_tula
     palkan RubyConf 2018
     METRICS
     93
     

     View Slide

107. palkan_tula
     palkan RubyConf 2018
     MORE FEATURES
     94
     Zero-disconnect deployment
     Metrics & Stats
     Rails-free
     

     View Slide

108. LiteCable
     github.com/palkan/litecable
     Rails No More
     

     View Slide

109. palkan_tula
     palkan RubyConf 2018
     MORE FEATURES
     96
     Zero-disconnect deployment
     Metrics & Stats
     Rails-free
     NEW Supports GraphQL subscriptions
     

     View Slide

110. palkan_tula
     palkan RubyConf 2018
     GRAPHQL ANYCABLE
     97
     github.com/Envek/graphql-anycable
     
     

     View Slide

111. palkan_tula
     palkan RubyConf 2018
     github.com/anycable
     https://anycable.io
     AnyCable
     98
     

     View Slide

112. palkan_tula
     palkan RubyConf 2018
     docs.anycable.io
     99
     

     View Slide

113. palkan_tula
     palkan RubyConf 2018
     anycable.io/pro
     100
     

     View Slide

114. RUBY
     OR NOT
     © RailsClub Moscow
     

     View Slide

115. palkan_tula
     palkan RubyConf 2018
     RUBY OR NOT
     102
     Real-time in Ruby is possible…
     …but more efficient with the help of
     other languages
     …and could become much better
     with Ruby 3
     

     View Slide

116. THANK YOU!
     Vladimir Dementyev
     anycable.io
     evilmartians.com
     @palkan
     @palkan_tula
     @evilmartians
     

     View Slide