[RubyKaigi 2018] AnyCable: One cable to rule them all

Video: https://www.youtube.com/watch?v=jXCPuNICT8s

RubyKaigi: http://rubykaigi.org/2018/presentations/palkan_tula.html

AnyCable: https://anycable.io

Transcript

1. palkan_tula
   palkan
   ANYCABLE
   Vladimir Dementyev
   One cable to rule
   them all
   

   View Slide

2. palkan_tula
   palkan
   2
   Vladimir
   Dementyev
   

   View Slide

3. palkan_tula
   palkan
   ! Moscow
   ✈
   # Tokyo
   
   # Sendai
   

   View Slide

4. palkan_tula
   palkan
   4
   @palkan
   @palkan_tula
   

   View Slide

5. palkan_tula
   palkan
   5
   

   View Slide

6. palkan_tula
   palkan
   https://evilmartians.com
   6
   

   View Slide

7. palkan_tula
   palkan
   https://evilmartians.com
   7
   

   View Slide

8. palkan_tula
   palkan
   https://evilmartians.com
   8
   

   View Slide

9. palkan_tula
   palkan
   9
   https://techracho.bpsinc.jp/hachi8833/2017_10_10/46290
   https://evilmartians.com
   

   View Slide

10. palkan_tula
    palkan
    THE TALK
    10
    Cables of Ruby and the rest of
    the world
    The tale of Action Cable
    AnyCable, the half-blood prince
    

    View Slide

11. palkan_tula
    palkan
    Part 1
    CABLES
    Tools for building
    real-time applications
    

    View Slide

12. palkan_tula
    palkan
    REAL-TIME
    12
    Messaging (i.e. chats)
    Notifications
    Live updates
    Online games
    Other
    

    View Slide

13. palkan_tula
    palkan
    RUBY CABLES
    13
    *based on https://www.ruby-toolbox.com/categories/HTTP_Pub_Sub
    

    View Slide

14. palkan_tula
    palkan
    RUBY CABLES
    14
    

    View Slide

15. © https://twitter.com/erneestoc/status/974485805770072064
    

    View Slide

16. palkan_tula
    palkan
    THE RED RUBY PILL?
    16
    

    View Slide

17. palkan_tula
    palkan
    Part 2
    ACTION
    CABLE
    

    View Slide

18. palkan_tula
    palkan
    IN A NUTSHELL
    18
    Server
    Client
    WebSocket
    Client
    WebSocket
    

    View Slide

19. palkan_tula
    palkan
    IN A NUTSHELL
    18
    Server
    Client
    WebSocket
    Client
    WebSocket
    stream C
    Broadcaster
    stream B
    stream A
    stream B
    

    View Slide

20. palkan_tula
    palkan
    IN A NUTSHELL
    18
    Server
    Client
    WebSocket
    Client
    WebSocket
    stream C
    Broadcaster
    stream B
    stream A
    stream B
    channel X
    channel Y
    channel Z
    

    View Slide

21. palkan_tula
    palkan
    CHANNELS
    19
    class AnswersChannel < ApplicationCable ::Channel
    def subscribed
    reject_subscription unless current_user.admin?
    end
    def follow(params)
    stream_from "questions/ #{params['id']}"
    end
    end
    

    View Slide

22. palkan_tula
    palkan
    CHAT IN 5MIN
    20
    

    View Slide

23. palkan_tula
    palkan
    CHAT IN 5MIN
    20
    

    View Slide

24. palkan_tula
    palkan
    WHAT’S WRONG?
    21
    

    View Slide

25. palkan_tula
    palkan
    SHOOTOUT
    22
    https://hashrocket.com/blog/posts/websocket-shootout
    

    View Slide

26. palkan_tula
    palkan
    SHOOTOUT
    23
    Client
    Server
    broadcast
    to all
    message
    send message back
    

    View Slide

27. palkan_tula
    palkan
    SHOOTOUT
    24
    Broadcast RTT
    0,0s
    0,8s
    1,6s
    2,4s
    3,2s
    4,0s
    4,8s
    5,6s
    6,4s
    7,2s
    8,0s
    Number of connections
    1000 2000 3000 4000 5000 6000 7000 8000 9000 10000
    Go Erlang Action Cable (8x) Action Cable (2x)
    https://github.com/anycable/anycable/tree/master/benchmarks
    

    View Slide

28. View Slide

29. palkan_tula
    palkan
    CABLE
    THEOREM
    Action
    Cable
    Low
    Latency
    Crowded
    Streams
    Low
    Latency
    Action
    Cable
    

    View Slide

30. palkan_tula
    palkan
    CABLE
    THEOREM
    Action
    Cable
    Low
    Latency
    Crowded
    Streams
    Low
    Latency
    Action
    Cable
    High Resources Usage
    

    View Slide

31. palkan_tula
    palkan
    CPU
    28
    * running WebSocket shootout
    

    View Slide

32. palkan_tula
    palkan
    CPU
    28
    * running WebSocket shootout
    

    View Slide

33. palkan_tula
    palkan
    MEMORY
    29
    20k idle connections
    MB
    0
    200
    400
    600
    800
    1 000
    1 200
    1 400
    1 600
    Go Erlang Action Cable (8x)
    

    View Slide

34. palkan_tula
    palkan
    WHAT
    ABOUT
    REAL
    LIFE?
    I WANT TO
    BELIEVE IN
    BENCHMARKS
    

    View Slide

35. palkan_tula
    palkan
    http://equipe.com
    “With Action Cable, we could
    easily have 20+ 1GB dynos
    running during the weekends,
    with every growing memory”
    –Jon Stenqvist, CEO, Equipe
    

    View Slide

36. palkan_tula
    palkan
    WHY?
    32
    

    View Slide

37. palkan_tula
    palkan
    –Bo
    “Due to code duplication and extra work,
    the memory consumption for hijack
    based solutions is higher and their
    performance is slower (more system
    calls, more context switches, etc’)..”
    33
    https://bowild.wordpress.com/2018/05/01/rubys-rack-push-decoupling-the-real-time-web-application-from-the-web/
    THE hijack PRICE
    

    View Slide

38. palkan_tula
    palkan
    34
    Separate IO loop (server)
    WebSockets protocol
    implementation overhead
    THE hijack PRICE
    

    View Slide

39. palkan_tula
    palkan
    RACK API PROPOSAL
    35
    https://github.com/rack/rack/pull/1272
    

    View Slide

40. palkan_tula
    palkan
    ActionCable
    36
    Long-lived objects
    

    View Slide

41. palkan_tula
    palkan
    ALLOCATIONS
    37
    Retained objects for one client:
    websocket ~40
    actioncable ~640
    other ~100
    ~60kB
    

    View Slide

42. palkan_tula
    palkan
    ActionCable
    38
    Long-lived objects
    Inefficient pub/sub (with JSON
    roundtrip for each client)
    

    View Slide

43. palkan_tula
    palkan
    THE PATCH
    39
    https://github.com/rails/rails/pull/27044
    

    View Slide

44. View Slide

45. View Slide

46. palkan_tula
    palkan
    ActionCable
    41
    Long-lived objects
    Inefficient pub/sub (JSON
    roundtrip for each client)
    => Heap fragmentation?
    

    View Slide

47. palkan_tula
    palkan
    HEAPY HEAPY SHAKE
    42
    

    View Slide

48. palkan_tula
    palkan
    IS
    THERE
    A WAY
    OUT?
    

    View Slide

49. palkan_tula
    palkan
    Part 3
    AnyCable
    The half-blood prince
    

    View Slide

50. palkan_tula
    palkan
    AnyCable
    45
    

    View Slide

51. palkan_tula
    palkan
    Client (protocol)
    Channels
    Streams
    Server
    ActionCable
    46
    

    View Slide

52. palkan_tula
    palkan
    Client (protocol)
    Channels
    Streams
    Server
    ActionCable
    47
    

    View Slide

53. palkan_tula
    palkan
    ActionCable
    48
    Client (protocol)
    Channels
    Streams
    Server
    ?
    

    View Slide

54. palkan_tula
    palkan
    ActionCable
    49
    Client (protocol)
    Channels
    Streams
    Server
    AnyCable
    

    View Slide

55. palkan_tula
    palkan
    AnyCable
    50
    ?
    WS Server
    

    View Slide

56. palkan_tula
    palkan
    51
    https://grpc.io
    gRPC
    

    View Slide

57. palkan_tula
    palkan
    52
    https://grpc.io
    gRPC =
    Google RPC
    

    View Slide

58. palkan_tula
    palkan
    53
    https://grpc.io
    gRPC =
    universal
    RPC
    framework
    

    View Slide

59. palkan_tula
    palkan
    54
    https://grpc.io
    gRPC =
    HTTP/2 +
    protobuf
    

    View Slide

60. palkan_tula
    palkan
    AnyCable
    55
    syntax = "proto3";
    package anycable;
    service RPC {
    rpc Connect (ConnectionRequest) returns (ConnectionResponse) {}
    rpc Command (CommandMessage) returns (CommandResponse) {}
    rpc Disconnect (DisconnectRequest) returns (DisconnectResponse) {}
    }
    

    View Slide

61. palkan_tula
    palkan
    AnyCable
    56
    Go WS
    

    View Slide

62. palkan_tula
    palkan
    AnyCable
    57
    Go WS
    Bottleneck?
    

    View Slide

63. palkan_tula
    palkan
    gRPC RPS
    58
    RPC type \ Concurrency 1 10 50 100
    AnyCable RPC (single
    connection)
    1600 2200 2400 2800
    AnyCable RPC (connection
    pool)
    1500 1900 2300 2900
    Noop RPC (single connection) 3000 4600 5500 6000
    * Only build Action Cable connection object without performing an action
    https://github.com/anycable/anycable/blob/master/benchmarks/2018-05-27-rpc-bench.md
    

    View Slide

64. View Slide

65. palkan_tula
    palkan
    AnyCable
    60
    anycable-go
    erlycable
    WebSocket Servers
    

    View Slide

66. palkan_tula
    palkan
    CPU
    61
    anycable-go erlycable action_cable
    

    View Slide

67. palkan_tula
    palkan
    CPU
    61
    anycable-go erlycable action_cable
    

    View Slide

68. palkan_tula
    palkan
    SHOOTOUT
    62
    Broadcast RTT
    0,0s
    0,8s
    1,6s
    2,4s
    3,2s
    4,0s
    4,8s
    5,6s
    6,4s
    7,2s
    8,0s
    Number of connections
    1000 2000 3000 4000 5000 6000 7000 8000 9000 10000
    anycable-go erlycable Action Cable (8x)
    https://github.com/anycable/anycable/tree/master/benchmarks
    

    View Slide

69. palkan_tula
    palkan
    63
    ActionCable: 20+ 2X (1GB) dynos AnyCable: 4 X (0.5GB) dynos
    $1000 $100
    http://equipe.com
    

    View Slide

70. palkan_tula
    palkan
    AnyCable
    64
    Short-lived objects instead of
    long-lived
    

    View Slide

71. palkan_tula
    palkan
    AnyCable
    65
    WS RPC
    connect
    identifiers (JSON + GlobalID)
    command (subscribe/perform)
    with identifiers
    initialize “connection”,
    authenticate
    initialize “connection”
    and channel,
    lazily restore identifiers,
    perform command
    transmissions, streams to subscribe
    * “connection” is a temp object quacking like ActionCable ::Connection
    

    View Slide

72. palkan_tula
    palkan
    AnyCable
    66
    class RPCHandler < Anycable ::RPC ::Service
    def connect(request, _unused_call)
    # WebSocket mock; rack env is built from request headers
    socket = build_socket(env: rack_env(request))
    # Pluggable connection factory
    connection = connection_factory.call(socket)
    connection.handle_open
    if socket.closed?
    Anycable ::ConnectionResponse.new(status: Anycable ::Status ::FAILURE)
    else
    Anycable ::ConnectionResponse.new(
    status: Anycable ::Status ::SUCCESS,
    # push identifiers JSON back to WebSocket server to re-use in
    # subsequent calls
    identifiers: connection.identifiers_json,
    transmissions: socket.transmissions
    )
    end
    end
    end
    

    View Slide

73. palkan_tula
    palkan
    AnyCable
    67
    class RPCHandler < Anycable ::RPC ::Service
    def command(message, _unused_call)
    socket = build_socket
    connection = factory.call(
    socket,
    # identifiers JSON from WebSocket server
    identifiers: message.connection_identifiers
    )
    result = connection.handle_channel_command(
    message.identifier, message.command, message.data
    )
    Anycable ::CommandResponse.new(
    status: result ? Anycable ::Status ::SUCCESS : Anycable ::Status ::FAILURE,
    disconnect: socket.closed?,
    stop_streams: socket.stop_streams?,
    streams: socket.streams,
    transmissions: socket.transmissions
    )
    end
    

    View Slide

74. palkan_tula
    palkan
    Possible Improvements
    68
    Re-use socket mock objects
    Re-use connection objects
    Object pool?
    

    View Slide

75. palkan_tula
    palkan
    AnyCable
    69
    Short-lived objects instead of
    long-lived
    Efficient pub/sub (lives within
    WebSocket server)
    … and more
    

    View Slide

76. palkan_tula
    palkan
    MORE FEATURES
    70
    Zero-disconnect deployment
    

    View Slide

77. palkan_tula
    palkan
    DISCONNECT
    71
    Client Re-connect & Re-subscribe
    Client Disconnected ActionCable
    ActionCable
    

    View Slide

78. palkan_tula
    palkan
    DISCONNECT
    72
    WebSocket Server
    Client Connected
    Client Connected App
    App
    RPC
    RPC
    

    View Slide

79. palkan_tula
    palkan
    DISCONNECT
    73
    WebSocket Server
    App
    App
    gRPC
    App
    App
    App
    App
    Envoy Proxy
    * https://www.envoyproxy.io
    

    View Slide

80. palkan_tula
    palkan
    MORE FEATURES
    74
    Zero-disconnect deployment
    Metrics & Stats
    

    View Slide

81. palkan_tula
    palkan
    METRICS
    75
    

    View Slide

82. palkan_tula
    palkan
    PLUG-N-PLAY
    76
    gem 'anycable-rails', group: :production
    rails generate anycable
    config.action_cable.url = ‘ws: //example.com:3334'
    ./bin/anycable # => Run RPC server
    brew install anycable/anycable/anycable-go
    anycable-go # => Run WebSocket server
    

    View Slide

83. palkan_tula
    palkan
    COMPATIBILITY
    77
    Feature Status
    Connection Identifiers +
    Connection Request (cookies, params) +
    Disconnect Handling +
    Subscribe to channels +
    Parameterized subscriptions +
    Unsubscribe from channels +
    Subscription Instance Variables -
    Performing Channel Actions +
    Streaming +
    Remote Disconnect wip (planned for 0.6.0)
    Custom stream callbacks -
    Broadcasting +
    

    View Slide

84. palkan_tula
    palkan
    MORE FEATURES
    78
    Zero-disconnect deployment
    Metrics & Stats
    Rails-free
    

    View Slide

85. palkan_tula
    palkan
    LiteCable
    Rails No More
    © Призрачная Колыма
    

    View Slide

86. palkan_tula
    palkan
    LiteCable
    80
    Rails-free Action Cable
    No deps (even ActiveSupport)
    Compatible with AnyCable
    Compatible with Action Cable
    clients
    

    View Slide

87. palkan_tula
    palkan
    module Chat
    class Channel < LiteCable ::Channel ::Base
    identifier :chat
    def subscribed
    stream_from "chat_ #{chat_id}"
    end
    end
    end
    81
    LiteCable
    

    View Slide

88. palkan_tula
    palkan
    run Rack ::Builder.new do
    map '/cable' do
    use LiteCable ::Server ::Middleware,
    connection_class: Chat ::Connection
    run proc { |_| [200, {}, ['']] }
    end
    end
    82
    LiteCable
    

    View Slide

89. palkan_tula
    palkan
    HANAMI CABLE
    83
    http://gabrielmalakias.com.br/ruby/hanami/iot/2017/05/26/websockets-connecting-litecable-to-hanami.html
    

    View Slide

90. palkan_tula
    palkan
    NEW RACK API
    84
    https://github.com/palkan/litecable/pull/10
    

    View Slide

91. palkan_tula
    palkan
    github.com/palkan/litecable
    LiteCable
    85
    

    View Slide

92. palkan_tula
    palkan
    Part 4
    mAnyCable
    From Ruby to Go
    and back again
    

    View Slide

93. palkan_tula
    palkan
    CHANNEL
    87
    Many channels are as simple as
    that:
    What about not calling RPC in this
    case?
    class AnswersChannel < ApplicationCable ::Channel
    def follow(params)
    stream_from "questions/ #{params['id']}"
    end
    end
    

    View Slide

94. View Slide

95. palkan_tula
    palkan
    MAYBE?
    *https://goby-lang.org
    89
    

    View Slide

96. palkan_tula
    palkan
    GOBY
    90
    

    View Slide

97. palkan_tula
    palkan
    91
    Ruby
    

    View Slide

98. palkan_tula
    palkan
    ACLI
    92
    Action Cable CLI
    github.com/palkan/acli
    

    View Slide

99. palkan_tula
    palkan
    ACLI
    92
    Action Cable CLI
    github.com/palkan/acli
    

    View Slide

100. palkan_tula
     palkan
     MRUBY
     93
     

     View Slide

101. palkan_tula
     palkan
     MRUBY + GO
     94
     github.com/mitchellh/go-mruby
     

     View Slide

102. palkan_tula
     palkan
     MRUBY + GO
     95
     

     View Slide

103. palkan_tula
     palkan
     NOTE
     96
     All the Golang code in this
     slides is simplified for readability
     I just removed all the error
     handling stuff)
     

     View Slide

104. palkan_tula
     palkan
     MRUBY-GO
     97
     package mrb
     import (
     "io/ioutil"
     "sync"
     "github.com/mitchellh/go-mruby"
     )
     // Engine represents one running mruby VM
     type Engine struct {
     VM *mruby.Mrb
     }
     // NewEngine builds new mruby VM and return new engine
     func NewEngine() *Engine {
     return &Engine{VM: mruby.NewMrb()}
     }
     

     View Slide

105. palkan_tula
     palkan
     MRUBY-GO
     98
     // LoadString loads, parses and eval Ruby code within a vm
     func (engine *Engine) LoadString(contents string) {
     ctx := mruby.NewCompileContext(engine.VM)
     defer ctx.Close()
     filename, _ := nanoid.Nanoid()
     ctx.SetFilename(fmt.Sprintf("%s.rb", filename))
     parser := mruby.NewParser(engine.VM)
     defer parser.Close()
     parser.Parse(contents, ctx)
     parsed := parser.GenerateCode()
     engine.VM.Run(parsed, nil)
     }
     

     View Slide

106. palkan_tula
     palkan
     MRUBY-GO
     99
     // LoadFile loads, parses and eval Ruby file within a vm
     func (engine *Engine) LoadFile(path string) error {
     contents, err := ioutil.ReadFile(path)
     if err != nil {
     return err
     }
     return engine.LoadString(string(contents))
     }
     // Eval runs arbitrary code within a vm
     func (engine *Engine) Eval(code string) (*mruby.MrbValue, error) {
     return engine.VM.LoadString(code)
     }
     

     View Slide

107. palkan_tula
     palkan
     MRUBY-GO
     100
     func TestLoadString(t *testing.T) {
     engine := NewEngine()
     engine.LoadString(
     `
     module Example
     def self.add(a, b)
     a + b
     end
     end
     `,
     )
     result, err := engine.Eval("Example.add(20, 22)")
     assert.Nil(t, err)
     assert.Equal(t, 42, result.Fixnum())
     }
     

     View Slide

108. palkan_tula
     palkan
     anycable-go
     101
     

     View Slide

109. palkan_tula
     palkan
     MRUBY CHANNELS
     102
     RPC server responds with
     cacheable methods source
     Load these methods into mRuby
     VM
     Invoke mRuby methods instead
     of calling RPC
     

     View Slide

110. palkan_tula
     palkan
     MRUBY CHANNELS
     103
     RPC server responds with
     cacheable methods source [WIP]
     Load these methods into mRuby
     VM
     Invoke mRuby methods instead
     of calling RPC
     

     View Slide

111. palkan_tula
     palkan
     MRUBY CHANNELS
     104
     RPC server responds with
     cacheable methods source [WIP]
     Load these methods into mRuby
     VM
     Invoke mRuby methods instead
     of calling RPC
     

     View Slide

112. palkan_tula
     palkan
     MRUBY CHANNELS
     105
     // NewMCache builds a new cache struct for mruby engine
     func NewMCache() *MCache {
     return &MCache{
     store: make(map[string]map[string]*MAction),
     }
     }
     func(c *MCache) NewEngine() (engine *mrb.Engine){
     baseChannelSource := box.String("mrb/files/channel.rb")
     engine := mrb.NewEngine()
     // Build base channel class
     engine.LoadString(baseChannelSource)
     }
     

     View Slide

113. palkan_tula
     palkan
     MRUBY CHANNELS
     106
     // Put compiles method and put it in the cache
     func (c *MCache) Put(channel string, action string, source string) (err
     error) {
     var maction *MAction
     maction, _ = NewMAction(c, channel, source)
     if _, ok := c.store[channel]; !ok {
     c.store[channel] = make(map[string]*MAction)
     }
     c.store[channel][action] = maction
     return
     }
     

     View Slide

114. palkan_tula
     palkan
     MRUBY CHANNELS
     107
     // NewMAction compiles a channel class within VM
     func NewMAction(cache *MCache, ch string, src string) *MAction {
     var buf strings.Builder
     channelClass := "CachedChannel_" + ch
     buf.WriteString(
     "class " + channelClass + " < AnyCable ::Channel\n",
     )
     buf.WriteString("identify \"" + ch + "\"\n")
     buf.WriteString(src + ”\nend\n")
     engine := cache.NewEngine()
     engine.LoadString(buf.String())
     mchannel := engine.VM.Class(channelClass, nil)
     mchannelValue := mchannel.MrbValue(engine.VM)
     return &MAction{engine: engine, compiled: mchannelValue}
     }
     

     View Slide

115. palkan_tula
     palkan
     MRUBY CHANNELS
     108
     // Perform executes action within mruby VM
     func (m *MAction) Perform(data string) *node.CommandResult {
     m.mu.Lock()
     defer m.mu.Unlock()
     result, _ := m.compiled.Call("perform", mruby.String(data))
     decoded := MCallResult{}
     mruby.Decode(&decoded, result)
     res := &node.CommandResult{
     Transmissions: decoded.Transmissions,
     StopAllStreams: decoded.StopAllStreams,
     Streams: decoded.Streams,
     }
     return res
     }
     

     View Slide

116. palkan_tula
     palkan
     mRuby vs. RPC
     109
     mRuby #perform call – 0.05ms
     RPC #perform call* – 0.7ms
     (In theory) we can achieve 20k APS**
     *local connection
     **action per second
     

     View Slide

117. palkan_tula
     palkan
     Outro
     RUBY
     OR NOT
     © RailsClub Moscow
     

     View Slide

118. palkan_tula
     palkan
     WHAT HAVE WE DONE
     111
     Moved low-level stuff from
     Ruby to Go and glued them
     together with gRPC
     Moved some logic from
     Ruby to Go to be executed
     with mRuby
     

     View Slide

119. palkan_tula
     palkan
     RUBY OR NOT
     112
     WebSockets in Ruby are possible…
     …but more efficient with the help of
     other languages
     …including mRuby
     

     View Slide

120. palkan_tula
     palkan
     RUBY OR NOT
     113
     New Rack API could help us…
     …but won’t save us from ourselves
     …memory problems are likely to not
     go anywhere
     …maybe, compacting GC?
     

     View Slide

121. palkan_tula
     palkan
     THANKS?
     QUESTIONS!
     anycable.io
     evilmartians.com
     @palkan
     @palkan_tula
     

     View Slide