[RubyKaigi 2018] AnyCable: One cable to rule them all

Video: https://www.youtube.com/watch?v=jXCPuNICT8s

RubyKaigi: http://rubykaigi.org/2018/presentations/palkan_tula.html

AnyCable: https://anycable.io

Transcript

1. palkan_tula palkan ANYCABLE Vladimir Dementyev One cable to rule them all
2. palkan_tula palkan 2 Vladimir Dementyev
3. palkan_tula palkan ! Moscow ✈ # Tokyo # Sendai
4. palkan_tula palkan 4 @palkan @palkan_tula
5. palkan_tula palkan 5
6. palkan_tula palkan https://evilmartians.com 6
7. palkan_tula palkan https://evilmartians.com 7
8. palkan_tula palkan https://evilmartians.com 8
9. palkan_tula palkan 9 https://techracho.bpsinc.jp/hachi8833/2017_10_10/46290 https://evilmartians.com
10. palkan_tula palkan THE TALK 10 Cables of Ruby and the rest of the world The tale of Action Cable AnyCable, the half-blood prince
11. palkan_tula palkan Part 1 CABLES Tools for building real-time applications
12. palkan_tula palkan REAL-TIME 12 Messaging (i.e. chats) Notifications Live updates Online games Other
13. palkan_tula palkan RUBY CABLES 13 *based on https://www.ruby-toolbox.com/categories/HTTP_Pub_Sub
14. palkan_tula palkan RUBY CABLES 14
15. © https://twitter.com/erneestoc/status/974485805770072064
16. palkan_tula palkan THE RED RUBY PILL? 16
17. palkan_tula palkan Part 2 ACTION CABLE
18. palkan_tula palkan IN A NUTSHELL 18 Server Client WebSocket Client WebSocket
19. palkan_tula palkan IN A NUTSHELL 18 Server Client WebSocket Client WebSocket stream C Broadcaster stream B stream A stream B
20. palkan_tula palkan IN A NUTSHELL 18 Server Client WebSocket Client WebSocket stream C Broadcaster stream B stream A stream B channel X channel Y channel Z
21. palkan_tula palkan CHANNELS 19 class AnswersChannel < ApplicationCable ::Channel def subscribed reject_subscription unless current_user.admin? end def follow(params) stream_from "questions/ #{params['id']}" end end
22. palkan_tula palkan CHAT IN 5MIN 20
23. palkan_tula palkan CHAT IN 5MIN 20
24. palkan_tula palkan WHAT’S WRONG? 21
25. palkan_tula palkan SHOOTOUT 22 https://hashrocket.com/blog/posts/websocket-shootout
26. palkan_tula palkan SHOOTOUT 23 Client Server broadcast to all message send message back
27. palkan_tula palkan SHOOTOUT 24 Broadcast RTT 0,0s 0,8s 1,6s 2,4s 3,2s 4,0s 4,8s 5,6s 6,4s 7,2s 8,0s Number of connections 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 Go Erlang Action Cable (8x) Action Cable (2x) https://github.com/anycable/anycable/tree/master/benchmarks
29. palkan_tula palkan CABLE THEOREM Action Cable Low Latency Crowded Streams Low Latency Action Cable
30. palkan_tula palkan CABLE THEOREM Action Cable Low Latency Crowded Streams Low Latency Action Cable High Resources Usage
31. palkan_tula palkan CPU 28 * running WebSocket shootout
32. palkan_tula palkan CPU 28 * running WebSocket shootout
33. palkan_tula palkan MEMORY 29 20k idle connections MB 0 200 400 600 800 1 000 1 200 1 400 1 600 Go Erlang Action Cable (8x)
34. palkan_tula palkan WHAT ABOUT REAL LIFE? I WANT TO BELIEVE IN BENCHMARKS
35. palkan_tula palkan http://equipe.com “With Action Cable, we could easily have 20+ 1GB dynos running during the weekends, with every growing memory” –Jon Stenqvist, CEO, Equipe
36. palkan_tula palkan WHY? 32
37. palkan_tula palkan –Bo “Due to code duplication and extra work, the memory consumption for hijack based solutions is higher and their performance is slower (more system calls, more context switches, etc’)..” 33 https://bowild.wordpress.com/2018/05/01/rubys-rack-push-decoupling-the-real-time-web-application-from-the-web/ THE hijack PRICE
38. palkan_tula palkan 34 Separate IO loop (server) WebSockets protocol implementation overhead THE hijack PRICE
39. palkan_tula palkan RACK API PROPOSAL 35 https://github.com/rack/rack/pull/1272
40. palkan_tula palkan ActionCable 36 Long-lived objects
41. palkan_tula palkan ALLOCATIONS 37 Retained objects for one client: websocket ~40 actioncable ~640 other ~100 ~60kB
42. palkan_tula palkan ActionCable 38 Long-lived objects Inefficient pub/sub (with JSON roundtrip for each client)
43. palkan_tula palkan THE PATCH 39 https://github.com/rails/rails/pull/27044
46. palkan_tula palkan ActionCable 41 Long-lived objects Inefficient pub/sub (JSON roundtrip for each client) => Heap fragmentation?
47. palkan_tula palkan HEAPY HEAPY SHAKE 42
48. palkan_tula palkan IS THERE A WAY OUT?
49. palkan_tula palkan Part 3 AnyCable The half-blood prince
50. palkan_tula palkan AnyCable 45
51. palkan_tula palkan Client (protocol) Channels Streams Server ActionCable 46
52. palkan_tula palkan Client (protocol) Channels Streams Server ActionCable 47
53. palkan_tula palkan ActionCable 48 Client (protocol) Channels Streams Server ?
54. palkan_tula palkan ActionCable 49 Client (protocol) Channels Streams Server AnyCable
55. palkan_tula palkan AnyCable 50 ? WS Server
56. palkan_tula palkan 51 https://grpc.io gRPC
57. palkan_tula palkan 52 https://grpc.io gRPC = Google RPC
58. palkan_tula palkan 53 https://grpc.io gRPC = universal RPC framework
59. palkan_tula palkan 54 https://grpc.io gRPC = HTTP/2 + protobuf
60. palkan_tula palkan AnyCable 55 syntax = "proto3"; package anycable; service RPC { rpc Connect (ConnectionRequest) returns (ConnectionResponse) {} rpc Command (CommandMessage) returns (CommandResponse) {} rpc Disconnect (DisconnectRequest) returns (DisconnectResponse) {} }
61. palkan_tula palkan AnyCable 56 Go WS
62. palkan_tula palkan AnyCable 57 Go WS Bottleneck?
63. palkan_tula palkan gRPC RPS 58 RPC type \ Concurrency 1 10 50 100 AnyCable RPC (single connection) 1600 2200 2400 2800 AnyCable RPC (connection pool) 1500 1900 2300 2900 Noop RPC (single connection) 3000 4600 5500 6000 * Only build Action Cable connection object without performing an action https://github.com/anycable/anycable/blob/master/benchmarks/2018-05-27-rpc-bench.md
65. palkan_tula palkan AnyCable 60 anycable-go erlycable WebSocket Servers
66. palkan_tula palkan CPU 61 anycable-go erlycable action_cable
67. palkan_tula palkan CPU 61 anycable-go erlycable action_cable
68. palkan_tula palkan SHOOTOUT 62 Broadcast RTT 0,0s 0,8s 1,6s 2,4s 3,2s 4,0s 4,8s 5,6s 6,4s 7,2s 8,0s Number of connections 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 anycable-go erlycable Action Cable (8x) https://github.com/anycable/anycable/tree/master/benchmarks
69. palkan_tula palkan 63 ActionCable: 20+ 2X (1GB) dynos AnyCable: 4 X (0.5GB) dynos $1000 $100 http://equipe.com
70. palkan_tula palkan AnyCable 64 Short-lived objects instead of long-lived
71. palkan_tula palkan AnyCable 65 WS RPC connect identifiers (JSON + GlobalID) command (subscribe/perform) with identifiers initialize “connection”, authenticate initialize “connection” and channel, lazily restore identifiers, perform command transmissions, streams to subscribe * “connection” is a temp object quacking like ActionCable ::Connection
72. palkan_tula palkan AnyCable 66 class RPCHandler < Anycable ::RPC ::Service def connect(request, _unused_call) # WebSocket mock; rack env is built from request headers socket = build_socket(env: rack_env(request)) # Pluggable connection factory connection = connection_factory.call(socket) connection.handle_open if socket.closed? Anycable ::ConnectionResponse.new(status: Anycable ::Status ::FAILURE) else Anycable ::ConnectionResponse.new( status: Anycable ::Status ::SUCCESS, # push identifiers JSON back to WebSocket server to re-use in # subsequent calls identifiers: connection.identifiers_json, transmissions: socket.transmissions ) end end end
73. palkan_tula palkan AnyCable 67 class RPCHandler < Anycable ::RPC ::Service def command(message, _unused_call) socket = build_socket connection = factory.call( socket, # identifiers JSON from WebSocket server identifiers: message.connection_identifiers ) result = connection.handle_channel_command( message.identifier, message.command, message.data ) Anycable ::CommandResponse.new( status: result ? Anycable ::Status ::SUCCESS : Anycable ::Status ::FAILURE, disconnect: socket.closed?, stop_streams: socket.stop_streams?, streams: socket.streams, transmissions: socket.transmissions ) end
74. palkan_tula palkan Possible Improvements 68 Re-use socket mock objects Re-use connection objects Object pool?
75. palkan_tula palkan AnyCable 69 Short-lived objects instead of long-lived Efficient pub/sub (lives within WebSocket server) … and more
76. palkan_tula palkan MORE FEATURES 70 Zero-disconnect deployment
77. palkan_tula palkan DISCONNECT 71 Client Re-connect & Re-subscribe Client Disconnected ActionCable ActionCable
78. palkan_tula palkan DISCONNECT 72 WebSocket Server Client Connected Client Connected App App RPC RPC
79. palkan_tula palkan DISCONNECT 73 WebSocket Server App App gRPC App App App App Envoy Proxy * https://www.envoyproxy.io
80. palkan_tula palkan MORE FEATURES 74 Zero-disconnect deployment Metrics & Stats
81. palkan_tula palkan METRICS 75
82. palkan_tula palkan PLUG-N-PLAY 76 gem 'anycable-rails', group: :production rails generate anycable config.action_cable.url = ‘ws: //example.com:3334' ./bin/anycable # => Run RPC server brew install anycable/anycable/anycable-go anycable-go # => Run WebSocket server
83. palkan_tula palkan COMPATIBILITY 77 Feature Status Connection Identifiers + Connection Request (cookies, params) + Disconnect Handling + Subscribe to channels + Parameterized subscriptions + Unsubscribe from channels + Subscription Instance Variables - Performing Channel Actions + Streaming + Remote Disconnect wip (planned for 0.6.0) Custom stream callbacks - Broadcasting +
84. palkan_tula palkan MORE FEATURES 78 Zero-disconnect deployment Metrics & Stats Rails-free
85. palkan_tula palkan LiteCable Rails No More © Призрачная Колыма
86. palkan_tula palkan LiteCable 80 Rails-free Action Cable No deps (even ActiveSupport) Compatible with AnyCable Compatible with Action Cable clients
87. palkan_tula palkan module Chat class Channel < LiteCable ::Channel ::Base identifier :chat def subscribed stream_from "chat_ #{chat_id}" end end end 81 LiteCable
88. palkan_tula palkan run Rack ::Builder.new do map '/cable' do use LiteCable ::Server ::Middleware, connection_class: Chat ::Connection run proc { |_| [200, {}, ['']] } end end 82 LiteCable
89. palkan_tula palkan HANAMI CABLE 83 http://gabrielmalakias.com.br/ruby/hanami/iot/2017/05/26/websockets-connecting-litecable-to-hanami.html
90. palkan_tula palkan NEW RACK API 84 https://github.com/palkan/litecable/pull/10
91. palkan_tula palkan github.com/palkan/litecable LiteCable 85
92. palkan_tula palkan Part 4 mAnyCable From Ruby to Go and back again
93. palkan_tula palkan CHANNEL 87 Many channels are as simple as that: What about not calling RPC in this case? class AnswersChannel < ApplicationCable ::Channel def follow(params) stream_from "questions/ #{params['id']}" end end
95. palkan_tula palkan MAYBE? *https://goby-lang.org 89
96. palkan_tula palkan GOBY 90
97. palkan_tula palkan 91 Ruby
98. palkan_tula palkan ACLI 92 Action Cable CLI github.com/palkan/acli
99. palkan_tula palkan ACLI 92 Action Cable CLI github.com/palkan/acli
100. palkan_tula palkan MRUBY 93
101. palkan_tula palkan MRUBY + GO 94 github.com/mitchellh/go-mruby
102. palkan_tula palkan MRUBY + GO 95
103. palkan_tula palkan NOTE 96 All the Golang code in this slides is simplified for readability I just removed all the error handling stuff)
104. palkan_tula palkan MRUBY-GO 97 package mrb import ( "io/ioutil" "sync" "github.com/mitchellh/go-mruby" ) // Engine represents one running mruby VM type Engine struct { VM *mruby.Mrb } // NewEngine builds new mruby VM and return new engine func NewEngine() *Engine { return &Engine{VM: mruby.NewMrb()} }
105. palkan_tula palkan MRUBY-GO 98 // LoadString loads, parses and eval Ruby code within a vm func (engine *Engine) LoadString(contents string) { ctx := mruby.NewCompileContext(engine.VM) defer ctx.Close() filename, _ := nanoid.Nanoid() ctx.SetFilename(fmt.Sprintf("%s.rb", filename)) parser := mruby.NewParser(engine.VM) defer parser.Close() parser.Parse(contents, ctx) parsed := parser.GenerateCode() engine.VM.Run(parsed, nil) }
106. palkan_tula palkan MRUBY-GO 99 // LoadFile loads, parses and eval Ruby file within a vm func (engine *Engine) LoadFile(path string) error { contents, err := ioutil.ReadFile(path) if err != nil { return err } return engine.LoadString(string(contents)) } // Eval runs arbitrary code within a vm func (engine *Engine) Eval(code string) (*mruby.MrbValue, error) { return engine.VM.LoadString(code) }
107. palkan_tula palkan MRUBY-GO 100 func TestLoadString(t *testing.T) { engine := NewEngine() engine.LoadString( ` module Example def self.add(a, b) a + b end end `, ) result, err := engine.Eval("Example.add(20, 22)") assert.Nil(t, err) assert.Equal(t, 42, result.Fixnum()) }
108. palkan_tula palkan anycable-go 101
109. palkan_tula palkan MRUBY CHANNELS 102 RPC server responds with cacheable methods source Load these methods into mRuby VM Invoke mRuby methods instead of calling RPC
110. palkan_tula palkan MRUBY CHANNELS 103 RPC server responds with cacheable methods source [WIP] Load these methods into mRuby VM Invoke mRuby methods instead of calling RPC
111. palkan_tula palkan MRUBY CHANNELS 104 RPC server responds with cacheable methods source [WIP] Load these methods into mRuby VM Invoke mRuby methods instead of calling RPC
112. palkan_tula palkan MRUBY CHANNELS 105 // NewMCache builds a new cache struct for mruby engine func NewMCache() *MCache { return &MCache{ store: make(map[string]map[string]*MAction), } } func(c *MCache) NewEngine() (engine *mrb.Engine){ baseChannelSource := box.String("mrb/files/channel.rb") engine := mrb.NewEngine() // Build base channel class engine.LoadString(baseChannelSource) }
113. palkan_tula palkan MRUBY CHANNELS 106 // Put compiles method and put it in the cache func (c *MCache) Put(channel string, action string, source string) (err error) { var maction *MAction maction, _ = NewMAction(c, channel, source) if _, ok := c.store[channel]; !ok { c.store[channel] = make(map[string]*MAction) } c.store[channel][action] = maction return }
114. palkan_tula palkan MRUBY CHANNELS 107 // NewMAction compiles a channel class within VM func NewMAction(cache *MCache, ch string, src string) *MAction { var buf strings.Builder channelClass := "CachedChannel_" + ch buf.WriteString( "class " + channelClass + " < AnyCable ::Channel\n", ) buf.WriteString("identify \"" + ch + "\"\n") buf.WriteString(src + ”\nend\n") engine := cache.NewEngine() engine.LoadString(buf.String()) mchannel := engine.VM.Class(channelClass, nil) mchannelValue := mchannel.MrbValue(engine.VM) return &MAction{engine: engine, compiled: mchannelValue} }
115. palkan_tula palkan MRUBY CHANNELS 108 // Perform executes action within mruby VM func (m *MAction) Perform(data string) *node.CommandResult { m.mu.Lock() defer m.mu.Unlock() result, _ := m.compiled.Call("perform", mruby.String(data)) decoded := MCallResult{} mruby.Decode(&decoded, result) res := &node.CommandResult{ Transmissions: decoded.Transmissions, StopAllStreams: decoded.StopAllStreams, Streams: decoded.Streams, } return res }
116. palkan_tula palkan mRuby vs. RPC 109 mRuby #perform call – 0.05ms RPC #perform call* – 0.7ms (In theory) we can achieve 20k APS** *local connection **action per second
117. palkan_tula palkan Outro RUBY OR NOT © RailsClub Moscow
118. palkan_tula palkan WHAT HAVE WE DONE 111 Moved low-level stuff from Ruby to Go and glued them together with gRPC Moved some logic from Ruby to Go to be executed with mRuby
119. palkan_tula palkan RUBY OR NOT 112 WebSockets in Ruby are possible… …but more efficient with the help of other languages …including mRuby
120. palkan_tula palkan RUBY OR NOT 113 New Rack API could help us… …but won’t save us from ourselves …memory problems are likely to not go anywhere …maybe, compacting GC?
121. palkan_tula palkan THANKS? QUESTIONS! anycable.io evilmartians.com @palkan @palkan_tula