[RubyKaigi 2018] AnyCable: One cable to rule them all

by Vladimir Dementyev

Slide 1

Slide 1 text

palkan_tula palkan ANYCABLE Vladimir Dementyev One cable to rule them all

Slide 2

Slide 2 text

palkan_tula palkan 2 Vladimir Dementyev

Slide 3

Slide 3 text

palkan_tula palkan ! Moscow ✈ # Tokyo # Sendai

Slide 4

Slide 4 text

palkan_tula palkan 4 @palkan @palkan_tula

Slide 5

Slide 5 text

palkan_tula palkan 5

Slide 6

Slide 6 text

palkan_tula palkan https://evilmartians.com 6

Slide 7

Slide 7 text

palkan_tula palkan https://evilmartians.com 7

Slide 8

Slide 8 text

palkan_tula palkan https://evilmartians.com 8

Slide 9

Slide 9 text

palkan_tula palkan 9 https://techracho.bpsinc.jp/hachi8833/2017_10_10/46290 https://evilmartians.com

Slide 10

Slide 10 text

palkan_tula palkan THE TALK 10 Cables of Ruby and the rest of the world The tale of Action Cable AnyCable, the half-blood prince

Slide 11

Slide 11 text

palkan_tula palkan Part 1 CABLES Tools for building real-time applications

Slide 12

Slide 12 text

palkan_tula palkan REAL-TIME 12 Messaging (i.e. chats) Notiﬁcations Live updates Online games Other

Slide 13

Slide 13 text

palkan_tula palkan RUBY CABLES 13 *based on https://www.ruby-toolbox.com/categories/HTTP_Pub_Sub

Slide 14

Slide 14 text

palkan_tula palkan RUBY CABLES 14

Slide 15

Slide 15 text

Slide 16

Slide 16 text

palkan_tula palkan THE RED RUBY PILL? 16

Slide 17

Slide 17 text

palkan_tula palkan Part 2 ACTION CABLE

Slide 18

Slide 18 text

palkan_tula palkan IN A NUTSHELL 18 Server Client WebSocket Client WebSocket

Slide 19

Slide 19 text

palkan_tula palkan IN A NUTSHELL 18 Server Client WebSocket Client WebSocket stream C Broadcaster stream B stream A stream B

Slide 20

Slide 20 text

palkan_tula palkan IN A NUTSHELL 18 Server Client WebSocket Client WebSocket stream C Broadcaster stream B stream A stream B channel X channel Y channel Z

Slide 21

Slide 21 text

palkan_tula palkan CHANNELS 19 class AnswersChannel < ApplicationCable ::Channel def subscribed reject_subscription unless current_user.admin? end def follow(params) stream_from "questions/ #{params['id']}" end end

Slide 22

Slide 22 text

palkan_tula palkan CHAT IN 5MIN 20

Slide 23

Slide 23 text

palkan_tula palkan CHAT IN 5MIN 20

Slide 24

Slide 24 text

palkan_tula palkan WHAT’S WRONG? 21

Slide 25

Slide 25 text

palkan_tula palkan SHOOTOUT 22 https://hashrocket.com/blog/posts/websocket-shootout

Slide 26

Slide 26 text

palkan_tula palkan SHOOTOUT 23 Client Server broadcast to all message send message back

Slide 27

Slide 27 text

palkan_tula palkan SHOOTOUT 24 Broadcast RTT 0,0s 0,8s 1,6s 2,4s 3,2s 4,0s 4,8s 5,6s 6,4s 7,2s 8,0s Number of connections 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 Go Erlang Action Cable (8x) Action Cable (2x) https://github.com/anycable/anycable/tree/master/benchmarks

Slide 28

Slide 28 text

No content

Slide 29

Slide 29 text

palkan_tula palkan CABLE THEOREM Action Cable Low Latency Crowded Streams Low Latency Action Cable

Slide 30

Slide 30 text

palkan_tula palkan CABLE THEOREM Action Cable Low Latency Crowded Streams Low Latency Action Cable High Resources Usage

Slide 31

Slide 31 text

palkan_tula palkan CPU 28 * running WebSocket shootout

Slide 32

Slide 32 text

palkan_tula palkan CPU 28 * running WebSocket shootout

Slide 33

Slide 33 text

palkan_tula palkan MEMORY 29 20k idle connections MB 0 200 400 600 800 1 000 1 200 1 400 1 600 Go Erlang Action Cable (8x)

Slide 34

Slide 34 text

palkan_tula palkan WHAT ABOUT REAL LIFE? I WANT TO BELIEVE IN BENCHMARKS

Slide 35

Slide 35 text

palkan_tula palkan http://equipe.com “With Action Cable, we could easily have 20+ 1GB dynos running during the weekends, with every growing memory” –Jon Stenqvist, CEO, Equipe

Slide 36

Slide 36 text

palkan_tula palkan WHY? 32

Slide 37

Slide 37 text

palkan_tula palkan –Bo “Due to code duplication and extra work, the memory consumption for hijack based solutions is higher and their performance is slower (more system calls, more context switches, etc’)..” 33 https://bowild.wordpress.com/2018/05/01/rubys-rack-push-decoupling-the-real-time-web-application-from-the-web/ THE hijack PRICE

Slide 38

Slide 38 text

palkan_tula palkan 34 Separate IO loop (server) WebSockets protocol implementation overhead THE hijack PRICE

Slide 39

Slide 39 text

palkan_tula palkan RACK API PROPOSAL 35 https://github.com/rack/rack/pull/1272

Slide 40

Slide 40 text

palkan_tula palkan ActionCable 36 Long-lived objects

Slide 41

Slide 41 text

palkan_tula palkan ALLOCATIONS 37 Retained objects for one client: websocket ~40 actioncable ~640 other ~100 ~60kB

Slide 42

Slide 42 text

palkan_tula palkan ActionCable 38 Long-lived objects Inefﬁcient pub/sub (with JSON roundtrip for each client)

Slide 43

Slide 43 text

palkan_tula palkan THE PATCH 39 https://github.com/rails/rails/pull/27044

Slide 44

Slide 44 text

No content

Slide 45

Slide 45 text

No content

Slide 46

Slide 46 text

palkan_tula palkan ActionCable 41 Long-lived objects Inefﬁcient pub/sub (JSON roundtrip for each client) => Heap fragmentation?

Slide 47

Slide 47 text

palkan_tula palkan HEAPY HEAPY SHAKE 42

Slide 48

Slide 48 text

palkan_tula palkan IS THERE A WAY OUT?

Slide 49

Slide 49 text

palkan_tula palkan Part 3 AnyCable The half-blood prince

Slide 50

Slide 50 text

palkan_tula palkan AnyCable 45

Slide 51

Slide 51 text

palkan_tula palkan Client (protocol) Channels Streams Server ActionCable 46

Slide 52

Slide 52 text

palkan_tula palkan Client (protocol) Channels Streams Server ActionCable 47

Slide 53

Slide 53 text

palkan_tula palkan ActionCable 48 Client (protocol) Channels Streams Server ?

Slide 54

Slide 54 text

palkan_tula palkan ActionCable 49 Client (protocol) Channels Streams Server AnyCable

Slide 55

Slide 55 text

palkan_tula palkan AnyCable 50 ? WS Server

Slide 56

Slide 56 text

palkan_tula palkan 51 https://grpc.io gRPC

Slide 57

Slide 57 text

palkan_tula palkan 52 https://grpc.io gRPC = Google RPC

Slide 58

Slide 58 text

palkan_tula palkan 53 https://grpc.io gRPC = universal RPC framework

Slide 59

Slide 59 text

palkan_tula palkan 54 https://grpc.io gRPC = HTTP/2 + protobuf

Slide 60

Slide 60 text

palkan_tula palkan AnyCable 55 syntax = "proto3"; package anycable; service RPC { rpc Connect (ConnectionRequest) returns (ConnectionResponse) {} rpc Command (CommandMessage) returns (CommandResponse) {} rpc Disconnect (DisconnectRequest) returns (DisconnectResponse) {} }

Slide 61

Slide 61 text

palkan_tula palkan AnyCable 56 Go WS

Slide 62

Slide 62 text

palkan_tula palkan AnyCable 57 Go WS Bottleneck?

Slide 63

Slide 63 text

palkan_tula palkan gRPC RPS 58 RPC type \ Concurrency 1 10 50 100 AnyCable RPC (single connection) 1600 2200 2400 2800 AnyCable RPC (connection pool) 1500 1900 2300 2900 Noop RPC (single connection) 3000 4600 5500 6000 * Only build Action Cable connection object without performing an action https://github.com/anycable/anycable/blob/master/benchmarks/2018-05-27-rpc-bench.md

Slide 64

Slide 64 text

No content

Slide 65

Slide 65 text

palkan_tula palkan AnyCable 60 anycable-go erlycable WebSocket Servers

Slide 66

Slide 66 text

palkan_tula palkan CPU 61 anycable-go erlycable action_cable

Slide 67

Slide 67 text

palkan_tula palkan CPU 61 anycable-go erlycable action_cable

Slide 68

Slide 68 text

palkan_tula palkan SHOOTOUT 62 Broadcast RTT 0,0s 0,8s 1,6s 2,4s 3,2s 4,0s 4,8s 5,6s 6,4s 7,2s 8,0s Number of connections 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 anycable-go erlycable Action Cable (8x) https://github.com/anycable/anycable/tree/master/benchmarks

Slide 69

Slide 69 text

palkan_tula palkan 63 ActionCable: 20+ 2X (1GB) dynos AnyCable: 4 X (0.5GB) dynos $1000 $100 http://equipe.com

Slide 70

Slide 70 text

palkan_tula palkan AnyCable 64 Short-lived objects instead of long-lived

Slide 71

Slide 71 text

palkan_tula palkan AnyCable 65 WS RPC connect identifiers (JSON + GlobalID) command (subscribe/perform) with identifiers initialize “connection”, authenticate initialize “connection” and channel, lazily restore identifiers, perform command transmissions, streams to subscribe * “connection” is a temp object quacking like ActionCable ::Connection

Slide 72

Slide 72 text

palkan_tula palkan AnyCable 66 class RPCHandler < Anycable ::RPC ::Service def connect(request, _unused_call) # WebSocket mock; rack env is built from request headers socket = build_socket(env: rack_env(request)) # Pluggable connection factory connection = connection_factory.call(socket) connection.handle_open if socket.closed? Anycable ::ConnectionResponse.new(status: Anycable ::Status ::FAILURE) else Anycable ::ConnectionResponse.new( status: Anycable ::Status ::SUCCESS, # push identifiers JSON back to WebSocket server to re-use in # subsequent calls identifiers: connection.identifiers_json, transmissions: socket.transmissions ) end end end

Slide 73

Slide 73 text

palkan_tula palkan AnyCable 67 class RPCHandler < Anycable ::RPC ::Service def command(message, _unused_call) socket = build_socket connection = factory.call( socket, # identifiers JSON from WebSocket server identifiers: message.connection_identifiers ) result = connection.handle_channel_command( message.identifier, message.command, message.data ) Anycable ::CommandResponse.new( status: result ? Anycable ::Status ::SUCCESS : Anycable ::Status ::FAILURE, disconnect: socket.closed?, stop_streams: socket.stop_streams?, streams: socket.streams, transmissions: socket.transmissions ) end

Slide 74

Slide 74 text

palkan_tula palkan Possible Improvements 68 Re-use socket mock objects Re-use connection objects Object pool?

Slide 75

Slide 75 text

palkan_tula palkan AnyCable 69 Short-lived objects instead of long-lived Efﬁcient pub/sub (lives within WebSocket server) … and more

Slide 76

Slide 76 text

palkan_tula palkan MORE FEATURES 70 Zero-disconnect deployment

Slide 77

Slide 77 text

palkan_tula palkan DISCONNECT 71 Client Re-connect & Re-subscribe Client Disconnected ActionCable ActionCable

Slide 78

Slide 78 text

palkan_tula palkan DISCONNECT 72 WebSocket Server Client Connected Client Connected App App RPC RPC

Slide 79

Slide 79 text

palkan_tula palkan DISCONNECT 73 WebSocket Server App App gRPC App App App App Envoy Proxy * https://www.envoyproxy.io

Slide 80

Slide 80 text

palkan_tula palkan MORE FEATURES 74 Zero-disconnect deployment Metrics & Stats

Slide 81

Slide 81 text

palkan_tula palkan METRICS 75

Slide 82

Slide 82 text

palkan_tula palkan PLUG-N-PLAY 76 gem 'anycable-rails', group: :production rails generate anycable config.action_cable.url = ‘ws: //example.com:3334' ./bin/anycable # => Run RPC server brew install anycable/anycable/anycable-go anycable-go # => Run WebSocket server

Slide 83

Slide 83 text

palkan_tula palkan COMPATIBILITY 77 Feature Status Connection Identiﬁers + Connection Request (cookies, params) + Disconnect Handling + Subscribe to channels + Parameterized subscriptions + Unsubscribe from channels + Subscription Instance Variables - Performing Channel Actions + Streaming + Remote Disconnect wip (planned for 0.6.0) Custom stream callbacks - Broadcasting +

Slide 84

Slide 84 text

palkan_tula palkan MORE FEATURES 78 Zero-disconnect deployment Metrics & Stats Rails-free

Slide 85

Slide 85 text

Slide 86

Slide 86 text

palkan_tula palkan LiteCable 80 Rails-free Action Cable No deps (even ActiveSupport) Compatible with AnyCable Compatible with Action Cable clients

Slide 87

Slide 87 text

palkan_tula palkan module Chat class Channel < LiteCable ::Channel ::Base identifier :chat def subscribed stream_from "chat_ #{chat_id}" end end end 81 LiteCable

Slide 88

Slide 88 text

palkan_tula palkan run Rack ::Builder.new do map '/cable' do use LiteCable ::Server ::Middleware, connection_class: Chat ::Connection run proc { |_| [200, {}, ['']] } end end 82 LiteCable

Slide 89

Slide 89 text

palkan_tula palkan HANAMI CABLE 83 http://gabrielmalakias.com.br/ruby/hanami/iot/2017/05/26/websockets-connecting-litecable-to-hanami.html

Slide 90

Slide 90 text

palkan_tula palkan NEW RACK API 84 https://github.com/palkan/litecable/pull/10

Slide 91

Slide 91 text

palkan_tula palkan github.com/palkan/litecable LiteCable 85

Slide 92

Slide 92 text

palkan_tula palkan Part 4 mAnyCable From Ruby to Go and back again

Slide 93

Slide 93 text

palkan_tula palkan CHANNEL 87 Many channels are as simple as that: What about not calling RPC in this case? class AnswersChannel < ApplicationCable ::Channel def follow(params) stream_from "questions/ #{params['id']}" end end

Slide 94

Slide 94 text

No content

Slide 95

Slide 95 text

palkan_tula palkan MAYBE? *https://goby-lang.org 89

Slide 96

Slide 96 text

palkan_tula palkan GOBY 90

Slide 97

Slide 97 text

palkan_tula palkan 91 Ruby

Slide 98

Slide 98 text

palkan_tula palkan ACLI 92 Action Cable CLI github.com/palkan/acli

Slide 99

Slide 99 text

palkan_tula palkan ACLI 92 Action Cable CLI github.com/palkan/acli

Slide 100

Slide 100 text

palkan_tula palkan MRUBY 93

Slide 101

Slide 101 text

palkan_tula palkan MRUBY + GO 94 github.com/mitchellh/go-mruby

Slide 102

Slide 102 text

palkan_tula palkan MRUBY + GO 95

Slide 103

Slide 103 text

palkan_tula palkan NOTE 96 All the Golang code in this slides is simpliﬁed for readability I just removed all the error handling stuff)

Slide 104

Slide 104 text

palkan_tula palkan MRUBY-GO 97 package mrb import ( "io/ioutil" "sync" "github.com/mitchellh/go-mruby" ) // Engine represents one running mruby VM type Engine struct { VM *mruby.Mrb } // NewEngine builds new mruby VM and return new engine func NewEngine() *Engine { return &Engine{VM: mruby.NewMrb()} }

Slide 105

Slide 105 text

palkan_tula palkan MRUBY-GO 98 // LoadString loads, parses and eval Ruby code within a vm func (engine *Engine) LoadString(contents string) { ctx := mruby.NewCompileContext(engine.VM) defer ctx.Close() filename, _ := nanoid.Nanoid() ctx.SetFilename(fmt.Sprintf("%s.rb", filename)) parser := mruby.NewParser(engine.VM) defer parser.Close() parser.Parse(contents, ctx) parsed := parser.GenerateCode() engine.VM.Run(parsed, nil) }

Slide 106

Slide 106 text

palkan_tula palkan MRUBY-GO 99 // LoadFile loads, parses and eval Ruby file within a vm func (engine *Engine) LoadFile(path string) error { contents, err := ioutil.ReadFile(path) if err != nil { return err } return engine.LoadString(string(contents)) } // Eval runs arbitrary code within a vm func (engine *Engine) Eval(code string) (*mruby.MrbValue, error) { return engine.VM.LoadString(code) }

Slide 107

Slide 107 text

palkan_tula palkan MRUBY-GO 100 func TestLoadString(t *testing.T) { engine := NewEngine() engine.LoadString( ` module Example def self.add(a, b) a + b end end `, ) result, err := engine.Eval("Example.add(20, 22)") assert.Nil(t, err) assert.Equal(t, 42, result.Fixnum()) }

Slide 108

Slide 108 text

palkan_tula palkan anycable-go 101

Slide 109

Slide 109 text

palkan_tula palkan MRUBY CHANNELS 102 RPC server responds with cacheable methods source Load these methods into mRuby VM Invoke mRuby methods instead of calling RPC

Slide 110

Slide 110 text

palkan_tula palkan MRUBY CHANNELS 103 RPC server responds with cacheable methods source [WIP] Load these methods into mRuby VM Invoke mRuby methods instead of calling RPC

Slide 111

Slide 111 text

palkan_tula palkan MRUBY CHANNELS 104 RPC server responds with cacheable methods source [WIP] Load these methods into mRuby VM Invoke mRuby methods instead of calling RPC

Slide 112

Slide 112 text

palkan_tula palkan MRUBY CHANNELS 105 // NewMCache builds a new cache struct for mruby engine func NewMCache() *MCache { return &MCache{ store: make(map[string]map[string]*MAction), } } func(c *MCache) NewEngine() (engine *mrb.Engine){ baseChannelSource := box.String("mrb/files/channel.rb") engine := mrb.NewEngine() // Build base channel class engine.LoadString(baseChannelSource) }

Slide 113

Slide 113 text

palkan_tula palkan MRUBY CHANNELS 106 // Put compiles method and put it in the cache func (c *MCache) Put(channel string, action string, source string) (err error) { var maction *MAction maction, _ = NewMAction(c, channel, source) if _, ok := c.store[channel]; !ok { c.store[channel] = make(map[string]*MAction) } c.store[channel][action] = maction return }

Slide 114

Slide 114 text

palkan_tula palkan MRUBY CHANNELS 107 // NewMAction compiles a channel class within VM func NewMAction(cache *MCache, ch string, src string) *MAction { var buf strings.Builder channelClass := "CachedChannel_" + ch buf.WriteString( "class " + channelClass + " < AnyCable ::Channel\n", ) buf.WriteString("identify \"" + ch + "\"\n") buf.WriteString(src + ”\nend\n") engine := cache.NewEngine() engine.LoadString(buf.String()) mchannel := engine.VM.Class(channelClass, nil) mchannelValue := mchannel.MrbValue(engine.VM) return &MAction{engine: engine, compiled: mchannelValue} }

Slide 115

Slide 115 text

palkan_tula palkan MRUBY CHANNELS 108 // Perform executes action within mruby VM func (m *MAction) Perform(data string) *node.CommandResult { m.mu.Lock() defer m.mu.Unlock() result, _ := m.compiled.Call("perform", mruby.String(data)) decoded := MCallResult{} mruby.Decode(&decoded, result) res := &node.CommandResult{ Transmissions: decoded.Transmissions, StopAllStreams: decoded.StopAllStreams, Streams: decoded.Streams, } return res }

Slide 116

Slide 116 text

palkan_tula palkan mRuby vs. RPC 109 mRuby #perform call – 0.05ms RPC #perform call* – 0.7ms (In theory) we can achieve 20k APS** *local connection **action per second

Slide 117

Slide 117 text

Slide 118

Slide 118 text

palkan_tula palkan WHAT HAVE WE DONE 111 Moved low-level stuff from Ruby to Go and glued them together with gRPC Moved some logic from Ruby to Go to be executed with mRuby

Slide 119

Slide 119 text

palkan_tula palkan RUBY OR NOT 112 WebSockets in Ruby are possible… …but more efﬁcient with the help of other languages …including mRuby

Slide 120

Slide 120 text

palkan_tula palkan RUBY OR NOT 113 New Rack API could help us… …but won’t save us from ourselves …memory problems are likely to not go anywhere …maybe, compacting GC?

Slide 121

Slide 121 text

palkan_tula palkan THANKS? QUESTIONS! anycable.io evilmartians.com @palkan @palkan_tula