[RubyKaigi 2018] AnyCable: One cable to rule them all

[RubyKaigi 2018] AnyCable: One cable to rule them all

52cc8a838bf44a589d2572833b2dd1b9?s=128

Vlad Dem

June 01, 2018
Tweet

Transcript

  1. palkan_tula palkan ANYCABLE Vladimir Dementyev One cable to rule them

    all
  2. palkan_tula palkan 2 Vladimir Dementyev

  3. palkan_tula palkan ! Moscow ✈ # Tokyo # Sendai

  4. palkan_tula palkan 4 @palkan @palkan_tula

  5. palkan_tula palkan 5

  6. palkan_tula palkan https://evilmartians.com 6

  7. palkan_tula palkan https://evilmartians.com 7

  8. palkan_tula palkan https://evilmartians.com 8

  9. palkan_tula palkan 9 https://techracho.bpsinc.jp/hachi8833/2017_10_10/46290 https://evilmartians.com

  10. palkan_tula palkan THE TALK 10 Cables of Ruby and the

    rest of the world The tale of Action Cable AnyCable, the half-blood prince
  11. palkan_tula palkan Part 1 CABLES Tools for building real-time applications

  12. palkan_tula palkan REAL-TIME 12 Messaging (i.e. chats) Notifications Live updates

    Online games Other
  13. palkan_tula palkan RUBY CABLES 13 *based on https://www.ruby-toolbox.com/categories/HTTP_Pub_Sub

  14. palkan_tula palkan RUBY CABLES 14

  15. © https://twitter.com/erneestoc/status/974485805770072064

  16. palkan_tula palkan THE RED RUBY PILL? 16

  17. palkan_tula palkan Part 2 ACTION CABLE

  18. palkan_tula palkan IN A NUTSHELL 18 Server Client WebSocket Client

    WebSocket
  19. palkan_tula palkan IN A NUTSHELL 18 Server Client WebSocket Client

    WebSocket stream C Broadcaster stream B stream A stream B
  20. palkan_tula palkan IN A NUTSHELL 18 Server Client WebSocket Client

    WebSocket stream C Broadcaster stream B stream A stream B channel X channel Y channel Z
  21. palkan_tula palkan CHANNELS 19 class AnswersChannel < ApplicationCable ::Channel def

    subscribed reject_subscription unless current_user.admin? end def follow(params) stream_from "questions/ #{params['id']}" end end
  22. palkan_tula palkan CHAT IN 5MIN 20

  23. palkan_tula palkan CHAT IN 5MIN 20

  24. palkan_tula palkan WHAT’S WRONG? 21

  25. palkan_tula palkan SHOOTOUT 22 https://hashrocket.com/blog/posts/websocket-shootout

  26. palkan_tula palkan SHOOTOUT 23 Client Server broadcast to all message

    send message back
  27. palkan_tula palkan SHOOTOUT 24 Broadcast RTT 0,0s 0,8s 1,6s 2,4s

    3,2s 4,0s 4,8s 5,6s 6,4s 7,2s 8,0s Number of connections 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 Go Erlang Action Cable (8x) Action Cable (2x) https://github.com/anycable/anycable/tree/master/benchmarks
  28. None
  29. palkan_tula palkan CABLE THEOREM Action Cable Low Latency Crowded Streams

    Low Latency Action Cable
  30. palkan_tula palkan CABLE THEOREM Action Cable Low Latency Crowded Streams

    Low Latency Action Cable High Resources Usage
  31. palkan_tula palkan CPU 28 * running WebSocket shootout

  32. palkan_tula palkan CPU 28 * running WebSocket shootout

  33. palkan_tula palkan MEMORY 29 20k idle connections MB 0 200

    400 600 800 1 000 1 200 1 400 1 600 Go Erlang Action Cable (8x)
  34. palkan_tula palkan WHAT ABOUT REAL LIFE? I WANT TO BELIEVE

    IN BENCHMARKS
  35. palkan_tula palkan http://equipe.com “With Action Cable, we could easily have

    20+ 1GB dynos running during the weekends, with every growing memory” –Jon Stenqvist, CEO, Equipe
  36. palkan_tula palkan WHY? 32

  37. palkan_tula palkan –Bo “Due to code duplication and extra work,

    the memory consumption for hijack based solutions is higher and their performance is slower (more system calls, more context switches, etc’)..” 33 https://bowild.wordpress.com/2018/05/01/rubys-rack-push-decoupling-the-real-time-web-application-from-the-web/ THE hijack PRICE
  38. palkan_tula palkan 34 Separate IO loop (server) WebSockets protocol implementation

    overhead THE hijack PRICE
  39. palkan_tula palkan RACK API PROPOSAL 35 https://github.com/rack/rack/pull/1272

  40. palkan_tula palkan ActionCable 36 Long-lived objects

  41. palkan_tula palkan ALLOCATIONS 37 Retained objects for one client: websocket

    ~40 actioncable ~640 other ~100 ~60kB
  42. palkan_tula palkan ActionCable 38 Long-lived objects Inefficient pub/sub (with JSON

    roundtrip for each client)
  43. palkan_tula palkan THE PATCH 39 https://github.com/rails/rails/pull/27044

  44. None
  45. None
  46. palkan_tula palkan ActionCable 41 Long-lived objects Inefficient pub/sub (JSON roundtrip

    for each client) => Heap fragmentation?
  47. palkan_tula palkan HEAPY HEAPY SHAKE 42

  48. palkan_tula palkan IS THERE A WAY OUT?

  49. palkan_tula palkan Part 3 AnyCable The half-blood prince

  50. palkan_tula palkan AnyCable 45

  51. palkan_tula palkan Client (protocol) Channels Streams Server ActionCable 46

  52. palkan_tula palkan Client (protocol) Channels Streams Server ActionCable 47

  53. palkan_tula palkan ActionCable 48 Client (protocol) Channels Streams Server ?

  54. palkan_tula palkan ActionCable 49 Client (protocol) Channels Streams Server AnyCable

  55. palkan_tula palkan AnyCable 50 ? WS Server

  56. palkan_tula palkan 51 https://grpc.io gRPC

  57. palkan_tula palkan 52 https://grpc.io gRPC = Google RPC

  58. palkan_tula palkan 53 https://grpc.io gRPC = universal RPC framework

  59. palkan_tula palkan 54 https://grpc.io gRPC = HTTP/2 + protobuf

  60. palkan_tula palkan AnyCable 55 syntax = "proto3"; package anycable; service

    RPC { rpc Connect (ConnectionRequest) returns (ConnectionResponse) {} rpc Command (CommandMessage) returns (CommandResponse) {} rpc Disconnect (DisconnectRequest) returns (DisconnectResponse) {} }
  61. palkan_tula palkan AnyCable 56 Go WS

  62. palkan_tula palkan AnyCable 57 Go WS Bottleneck?

  63. palkan_tula palkan gRPC RPS 58 RPC type \ Concurrency 1

    10 50 100 AnyCable RPC (single connection) 1600 2200 2400 2800 AnyCable RPC (connection pool) 1500 1900 2300 2900 Noop RPC (single connection) 3000 4600 5500 6000 * Only build Action Cable connection object without performing an action https://github.com/anycable/anycable/blob/master/benchmarks/2018-05-27-rpc-bench.md
  64. None
  65. palkan_tula palkan AnyCable 60 anycable-go erlycable WebSocket Servers

  66. palkan_tula palkan CPU 61 anycable-go erlycable action_cable

  67. palkan_tula palkan CPU 61 anycable-go erlycable action_cable

  68. palkan_tula palkan SHOOTOUT 62 Broadcast RTT 0,0s 0,8s 1,6s 2,4s

    3,2s 4,0s 4,8s 5,6s 6,4s 7,2s 8,0s Number of connections 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 anycable-go erlycable Action Cable (8x) https://github.com/anycable/anycable/tree/master/benchmarks
  69. palkan_tula palkan 63 ActionCable: 20+ 2X (1GB) dynos AnyCable: 4

    X (0.5GB) dynos $1000 $100 http://equipe.com
  70. palkan_tula palkan AnyCable 64 Short-lived objects instead of long-lived

  71. palkan_tula palkan AnyCable 65 WS RPC connect identifiers (JSON +

    GlobalID) command (subscribe/perform) with identifiers initialize “connection”, authenticate initialize “connection” and channel, lazily restore identifiers, perform command transmissions, streams to subscribe * “connection” is a temp object quacking like ActionCable ::Connection
  72. palkan_tula palkan AnyCable 66 class RPCHandler < Anycable ::RPC ::Service

    def connect(request, _unused_call) # WebSocket mock; rack env is built from request headers socket = build_socket(env: rack_env(request)) # Pluggable connection factory connection = connection_factory.call(socket) connection.handle_open if socket.closed? Anycable ::ConnectionResponse.new(status: Anycable ::Status ::FAILURE) else Anycable ::ConnectionResponse.new( status: Anycable ::Status ::SUCCESS, # push identifiers JSON back to WebSocket server to re-use in # subsequent calls identifiers: connection.identifiers_json, transmissions: socket.transmissions ) end end end
  73. palkan_tula palkan AnyCable 67 class RPCHandler < Anycable ::RPC ::Service

    def command(message, _unused_call) socket = build_socket connection = factory.call( socket, # identifiers JSON from WebSocket server identifiers: message.connection_identifiers ) result = connection.handle_channel_command( message.identifier, message.command, message.data ) Anycable ::CommandResponse.new( status: result ? Anycable ::Status ::SUCCESS : Anycable ::Status ::FAILURE, disconnect: socket.closed?, stop_streams: socket.stop_streams?, streams: socket.streams, transmissions: socket.transmissions ) end
  74. palkan_tula palkan Possible Improvements 68 Re-use socket mock objects Re-use

    connection objects Object pool?
  75. palkan_tula palkan AnyCable 69 Short-lived objects instead of long-lived Efficient

    pub/sub (lives within WebSocket server) … and more
  76. palkan_tula palkan MORE FEATURES 70 Zero-disconnect deployment

  77. palkan_tula palkan DISCONNECT 71 Client Re-connect & Re-subscribe Client Disconnected

    ActionCable ActionCable
  78. palkan_tula palkan DISCONNECT 72 WebSocket Server Client Connected Client Connected

    App App RPC RPC
  79. palkan_tula palkan DISCONNECT 73 WebSocket Server App App gRPC App

    App App App Envoy Proxy * https://www.envoyproxy.io
  80. palkan_tula palkan MORE FEATURES 74 Zero-disconnect deployment Metrics & Stats

  81. palkan_tula palkan METRICS 75

  82. palkan_tula palkan PLUG-N-PLAY 76 gem 'anycable-rails', group: :production rails generate

    anycable config.action_cable.url = ‘ws: //example.com:3334' ./bin/anycable # => Run RPC server brew install anycable/anycable/anycable-go anycable-go # => Run WebSocket server
  83. palkan_tula palkan COMPATIBILITY 77 Feature Status Connection Identifiers + Connection

    Request (cookies, params) + Disconnect Handling + Subscribe to channels + Parameterized subscriptions + Unsubscribe from channels + Subscription Instance Variables - Performing Channel Actions + Streaming + Remote Disconnect wip (planned for 0.6.0) Custom stream callbacks - Broadcasting +
  84. palkan_tula palkan MORE FEATURES 78 Zero-disconnect deployment Metrics & Stats

    Rails-free
  85. palkan_tula palkan LiteCable Rails No More © Призрачная Колыма

  86. palkan_tula palkan LiteCable 80 Rails-free Action Cable No deps (even

    ActiveSupport) Compatible with AnyCable Compatible with Action Cable clients
  87. palkan_tula palkan module Chat class Channel < LiteCable ::Channel ::Base

    identifier :chat def subscribed stream_from "chat_ #{chat_id}" end end end 81 LiteCable
  88. palkan_tula palkan run Rack ::Builder.new do map '/cable' do use

    LiteCable ::Server ::Middleware, connection_class: Chat ::Connection run proc { |_| [200, {}, ['']] } end end 82 LiteCable
  89. palkan_tula palkan HANAMI CABLE 83 http://gabrielmalakias.com.br/ruby/hanami/iot/2017/05/26/websockets-connecting-litecable-to-hanami.html

  90. palkan_tula palkan NEW RACK API 84 https://github.com/palkan/litecable/pull/10

  91. palkan_tula palkan github.com/palkan/litecable LiteCable 85

  92. palkan_tula palkan Part 4 mAnyCable From Ruby to Go and

    back again
  93. palkan_tula palkan CHANNEL 87 Many channels are as simple as

    that: What about not calling RPC in this case? class AnswersChannel < ApplicationCable ::Channel def follow(params) stream_from "questions/ #{params['id']}" end end
  94. None
  95. palkan_tula palkan MAYBE? *https://goby-lang.org 89

  96. palkan_tula palkan GOBY 90

  97. palkan_tula palkan 91 Ruby

  98. palkan_tula palkan ACLI 92 Action Cable CLI github.com/palkan/acli

  99. palkan_tula palkan ACLI 92 Action Cable CLI github.com/palkan/acli

  100. palkan_tula palkan MRUBY 93

  101. palkan_tula palkan MRUBY + GO 94 github.com/mitchellh/go-mruby

  102. palkan_tula palkan MRUBY + GO 95

  103. palkan_tula palkan NOTE 96 All the Golang code in this

    slides is simplified for readability I just removed all the error handling stuff)
  104. palkan_tula palkan MRUBY-GO 97 package mrb import ( "io/ioutil" "sync"

    "github.com/mitchellh/go-mruby" ) // Engine represents one running mruby VM type Engine struct { VM *mruby.Mrb } // NewEngine builds new mruby VM and return new engine func NewEngine() *Engine { return &Engine{VM: mruby.NewMrb()} }
  105. palkan_tula palkan MRUBY-GO 98 // LoadString loads, parses and eval

    Ruby code within a vm func (engine *Engine) LoadString(contents string) { ctx := mruby.NewCompileContext(engine.VM) defer ctx.Close() filename, _ := nanoid.Nanoid() ctx.SetFilename(fmt.Sprintf("%s.rb", filename)) parser := mruby.NewParser(engine.VM) defer parser.Close() parser.Parse(contents, ctx) parsed := parser.GenerateCode() engine.VM.Run(parsed, nil) }
  106. palkan_tula palkan MRUBY-GO 99 // LoadFile loads, parses and eval

    Ruby file within a vm func (engine *Engine) LoadFile(path string) error { contents, err := ioutil.ReadFile(path) if err != nil { return err } return engine.LoadString(string(contents)) } // Eval runs arbitrary code within a vm func (engine *Engine) Eval(code string) (*mruby.MrbValue, error) { return engine.VM.LoadString(code) }
  107. palkan_tula palkan MRUBY-GO 100 func TestLoadString(t *testing.T) { engine :=

    NewEngine() engine.LoadString( ` module Example def self.add(a, b) a + b end end `, ) result, err := engine.Eval("Example.add(20, 22)") assert.Nil(t, err) assert.Equal(t, 42, result.Fixnum()) }
  108. palkan_tula palkan anycable-go 101

  109. palkan_tula palkan MRUBY CHANNELS 102 RPC server responds with cacheable

    methods source Load these methods into mRuby VM Invoke mRuby methods instead of calling RPC
  110. palkan_tula palkan MRUBY CHANNELS 103 RPC server responds with cacheable

    methods source [WIP] Load these methods into mRuby VM Invoke mRuby methods instead of calling RPC
  111. palkan_tula palkan MRUBY CHANNELS 104 RPC server responds with cacheable

    methods source [WIP] Load these methods into mRuby VM Invoke mRuby methods instead of calling RPC
  112. palkan_tula palkan MRUBY CHANNELS 105 // NewMCache builds a new

    cache struct for mruby engine func NewMCache() *MCache { return &MCache{ store: make(map[string]map[string]*MAction), } } func(c *MCache) NewEngine() (engine *mrb.Engine){ baseChannelSource := box.String("mrb/files/channel.rb") engine := mrb.NewEngine() // Build base channel class engine.LoadString(baseChannelSource) }
  113. palkan_tula palkan MRUBY CHANNELS 106 // Put compiles method and

    put it in the cache func (c *MCache) Put(channel string, action string, source string) (err error) { var maction *MAction maction, _ = NewMAction(c, channel, source) if _, ok := c.store[channel]; !ok { c.store[channel] = make(map[string]*MAction) } c.store[channel][action] = maction return }
  114. palkan_tula palkan MRUBY CHANNELS 107 // NewMAction compiles a channel

    class within VM func NewMAction(cache *MCache, ch string, src string) *MAction { var buf strings.Builder channelClass := "CachedChannel_" + ch buf.WriteString( "class " + channelClass + " < AnyCable ::Channel\n", ) buf.WriteString("identify \"" + ch + "\"\n") buf.WriteString(src + ”\nend\n") engine := cache.NewEngine() engine.LoadString(buf.String()) mchannel := engine.VM.Class(channelClass, nil) mchannelValue := mchannel.MrbValue(engine.VM) return &MAction{engine: engine, compiled: mchannelValue} }
  115. palkan_tula palkan MRUBY CHANNELS 108 // Perform executes action within

    mruby VM func (m *MAction) Perform(data string) *node.CommandResult { m.mu.Lock() defer m.mu.Unlock() result, _ := m.compiled.Call("perform", mruby.String(data)) decoded := MCallResult{} mruby.Decode(&decoded, result) res := &node.CommandResult{ Transmissions: decoded.Transmissions, StopAllStreams: decoded.StopAllStreams, Streams: decoded.Streams, } return res }
  116. palkan_tula palkan mRuby vs. RPC 109 mRuby #perform call –

    0.05ms RPC #perform call* – 0.7ms (In theory) we can achieve 20k APS** *local connection **action per second
  117. palkan_tula palkan Outro RUBY OR NOT © RailsClub Moscow

  118. palkan_tula palkan WHAT HAVE WE DONE 111 Moved low-level stuff

    from Ruby to Go and glued them together with gRPC Moved some logic from Ruby to Go to be executed with mRuby
  119. palkan_tula palkan RUBY OR NOT 112 WebSockets in Ruby are

    possible… …but more efficient with the help of other languages …including mRuby
  120. palkan_tula palkan RUBY OR NOT 113 New Rack API could

    help us… …but won’t save us from ourselves …memory problems are likely to not go anywhere …maybe, compacting GC?
  121. palkan_tula palkan THANKS? QUESTIONS! anycable.io evilmartians.com @palkan @palkan_tula