[RubyConf 2018] High-Speed Cables for Ruby

by Vladimir Dementyev

Slide 1

Slide 1 text

HIGH-SPEED CABLES FOR RUBY Vladimir Dementyev @evilmartians

Slide 2

Slide 2 text

No content

Slide 3

Slide 3 text

CABLES Tools for building real-time applications

Slide 4

Slide 4 text

palkan_tula palkan RubyConf 2018 The real-time web is a web … that enable users to receive information as soon as it is published by its authors, rather than … check a source periodically for updates. 4 * https://en.wikipedia.org/wiki/Real-time_web BORING DEFINITION

Slide 5

Slide 5 text

palkan_tula palkan RubyConf 2018 5 Request-Response REAL-TIME

Slide 6

Slide 6 text

palkan_tula palkan RubyConf 2018 5 Request-Response REAL-TIME

Slide 7

Slide 7 text

palkan_tula palkan RubyConf 2018 6 Bi-Directional Persistent REAL-TIME

Slide 8

Slide 8 text

palkan_tula palkan RubyConf 2018 6 Bi-Directional Persistent REAL-TIME

Slide 9

Slide 9 text

palkan_tula palkan RubyConf 2018 7 The opening of the ﬁrst McDonalds in Russia, Moscow, 1991 REAL-TIME Limited number of concurrent requests (request queue)

Slide 10

Slide 10 text

palkan_tula palkan RubyConf 2018 8 Large number of concurrent clients REAL-TIME Opening gala, Millennium Biltmore, 1923

Slide 11

Slide 11 text

palkan_tula palkan RubyConf 2018 EXAMPLES 9 Messaging (i.e. chats) Notiﬁcations Live updates Online games Other

Slide 12

Slide 12 text

palkan_tula palkan RubyConf 2018 WEBSOCKETS 10

Slide 13

Slide 13 text

palkan_tula palkan RubyConf 2018 CONCURRENCY 11

Slide 14

Slide 14 text

Slide 15

Slide 15 text

palkan_tula palkan RubyConf 2018 THE RED RUBY PILL? 13

Slide 16

Slide 16 text

palkan_tula palkan RubyConf 2018 14 Vladimir Dementyev

Slide 17

Slide 17 text

palkan_tula palkan RubyConf 2018 15 @palkan @palkan_tula Vladimir Dementyev 789 645 330 415

Slide 18

Slide 18 text

palkan_tula palkan RubyConf 2018 ACTION CABLE TESTING https://github.com/rails/rails/pull/33659 16

Slide 19

Slide 19 text

palkan_tula palkan RubyConf 2018 17 TEAM LID Team Me

Slide 20

Slide 20 text

palkan_tula palkan RubyConf 2018 18

Slide 21

Slide 21 text

palkan_tula palkan RubyConf 2018 19

Slide 22

Slide 22 text

palkan_tula palkan RubyConf 2018 evilmartians.com 20

Slide 23

Slide 23 text

No content

Slide 24

Slide 24 text

palkan_tula palkan RubyConf 2018 evilmartians.com/blog 22

Slide 25

Slide 25 text

palkan_tula palkan RubyConf 2018 23 evilmartians.com Brooklyn, NY

Slide 26

Slide 26 text

THIS TALK

Slide 27

Slide 27 text

palkan_tula palkan RubyConf 2018 THE TALK 25 What is “cable”? ✅ The tale of Action Cable, Rack & Ruby 3x3 From Action to Any: another point of view

Slide 28

Slide 28 text

RUBY & CABLES RailsConf 2015

Slide 29

Slide 29 text

palkan_tula palkan RubyConf 2018 IN A NUTSHELL 27 Server Client WebSocket Client WebSocket

Slide 30

Slide 30 text

palkan_tula palkan RubyConf 2018 IN A NUTSHELL 27 Server Client WebSocket Client WebSocket stream C Broadcaster stream B stream A stream B

Slide 31

Slide 31 text

palkan_tula palkan RubyConf 2018 IN A NUTSHELL 27 Server Client WebSocket Client WebSocket stream C Broadcaster stream B stream A stream B channel X channel Y channel Z

Slide 32

Slide 32 text

palkan_tula palkan RubyConf 2018 CHANNELS 28 class AnswersChannel < ApplicationCable ::Channel def subscribed reject_subscription unless current_user.admin? end def follow(params) stream_from "questions/ #{params['id']}" end end

Slide 33

Slide 33 text

palkan_tula palkan RubyConf 2018 CHAT IN 5MIN 29

Slide 34

Slide 34 text

palkan_tula palkan RubyConf 2018 CHAT IN 5MIN 29

Slide 35

Slide 35 text

palkan_tula palkan RubyConf 2018 BENCHMARKS 30 TODAY’S SPECIALS HashRocket RTT $9.99 RAM (all-you-can-eat) $19.59 CPU Steak 8’’ $20.18

Slide 36

Slide 36 text

palkan_tula palkan RubyConf 2018 SHOOTOUT 31 Client Server broadcast to all message send message back https://hashrocket.com/blog/posts/websocket-shootout

Slide 37

Slide 37 text

palkan_tula palkan RubyConf 2018 SHOOTOUT 32 Broadcast RTT 0,0s 1,2s 2,4s 3,6s 4,8s 6,0s 7,2s 8,4s 9,6s 10,8s 12,0s Number of connections 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 Go Erlang Action Cable (8x) https://github.com/anycable/anycable/tree/master/benchmarks

Slide 38

Slide 38 text

No content

Slide 39

Slide 39 text

CABLE THEOREM Action Cable Low Latency Crowded Streams Low Latency Action Cable

Slide 40

Slide 40 text

High Resources Usage CABLE THEOREM Action Cable Low Latency Crowded Streams Low Latency Action Cable

Slide 41

Slide 41 text

palkan_tula palkan RubyConf 2018 CPU 36 * running WebSocket shootout

Slide 42

Slide 42 text

palkan_tula palkan RubyConf 2018 CPU 36 * running WebSocket shootout

Slide 43

Slide 43 text

palkan_tula palkan RubyConf 2018 MEMORY 37 RSS 0MB 160MB 320MB 480MB 640MB 800MB 960MB 1120MB 1280MB 1440MB 1600MB Number of connections 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 Go Erlang Action Cable (8x)

Slide 44

Slide 44 text

WHAT ABOUT REAL LIFE? …or don’t trust benchmarks I WANT TO BELIEVE IN BENCHMARKS

Slide 45

Slide 45 text

palkan_tula palkan RubyConf 2018 39 “With Action Cable, we could easily have 20+ 1GB dynos running during the weekends, with every growing memory” –Jon Stenqvist, CEO, Equipe equipe.com NOTE: having ~5-10k conns at peak time

Slide 46

Slide 46 text

WHYYYY???!!

Slide 47

Slide 47 text

palkan_tula palkan RubyConf 2018 ACTION CABLE 41 rack.hijack

Slide 48

Slide 48 text

palkan_tula palkan RubyConf 2018 rack.hijack 42 Server Application

Slide 49

Slide 49 text

palkan_tula palkan RubyConf 2018 rack.hijack 42 Server Application IO

Slide 50

Slide 50 text

palkan_tula palkan RubyConf 2018 rack.hijack 42 Server Application

Slide 51

Slide 51 text

palkan_tula palkan RubyConf 2018 rack.hijack 42 Server Application IO

Slide 52

Slide 52 text

palkan_tula palkan RubyConf 2018 THE hijack PRICE 43 https://bowild.wordpress.com/2018/05/01/rubys-rack-push-decoupling-the-real-time-web-application-from-the-web/ –Bo “Due to code duplication and extra work, the memory consumption for hijack based solutions is higher and their performance is slower (more system calls, more context switches, etc’)..” https://bowild.wordpress.com/2018/05/01/rubys-rack-push-decoupling-the-real-time-web-application-from-the-web/

Slide 53

Slide 53 text

palkan_tula palkan RubyConf 2018 THE hijack PRICE 44 Separate IO loop (server) WebSockets protocol implementation overhead

Slide 54

Slide 54 text

palkan_tula palkan RubyConf 2018 plezi.io github.com/boazsegev/iodine WHO IS BO? 45 Boaz Segev (Bo) musician (now), developer (past)

Slide 55

Slide 55 text

palkan_tula palkan RubyConf 2018 Broadcast RTT 0,0s 1,2s 2,4s 3,6s 4,8s 6,0s 7,2s 8,4s 9,6s 10,8s 12,0s Number of connections 1000 3000 5000 7000 9000 Go Iodine Action Cable (8x) IODINE 46 https://github.com/anycable/anycable/tree/master/benchmarks RSS 0MB 160MB 320MB 480MB 640MB 800MB 960MB 1120MB 1280MB 1440MB 1600MB Number of connections 1000 3000 5000 7000 9000 Go Iodine Action Cable (8x)

Slide 56

Slide 56 text

palkan_tula palkan RubyConf 2018 RACK API: FUTURE 47 https://github.com/rack/rack/pull/1272

Slide 57

Slide 57 text

palkan_tula palkan RubyConf 2018 RACK API: FUTURE 48 rack.hijack Direct access to IO (socket) rack.upgrade Public interface to “abstracted” IO managed by web server

Slide 58

Slide 58 text

palkan_tula palkan RubyConf 2018 RACK API IN RAILS 49 https://github.com/rails/rails/pull/33295

Slide 59

Slide 59 text

palkan_tula palkan RubyConf 2018 RACK API IN RAILS 49 https://github.com/rails/rails/pull/33295 NO PERFORMANCE IMPROVEMENTS

Slide 60

Slide 60 text

palkan_tula palkan RubyConf 2018 ACTION CABLE 50 rack.hijack Long-lived objects

Slide 61

Slide 61 text

palkan_tula palkan RubyConf 2018 ALLOCATIONS 51 Retained objects for one connection: websocket ~40 actioncable ~640 other ~100 ~60kB

Slide 62

Slide 62 text

palkan_tula palkan RubyConf 2018 ACTION CABLE 52 rack.hijack Long-lived objects Heap fragmentation

Slide 63

Slide 63 text

palkan_tula palkan RubyConf 2018 HEAPY HEAPY SHAKE 53

Slide 64

Slide 64 text

palkan_tula palkan RubyConf 2018 HEAPY HEAPY SHAKE 53 Heap fragmentation? Are you sure?

Slide 65

Slide 65 text

RUBY 3X3 The future we’re waiting for

Slide 66

Slide 66 text

No content

Slide 67

Slide 67 text

palkan_tula palkan RubyConf 2018 56 GENERATIONAL HYPOTHESIS

Slide 68

Slide 68 text

palkan_tula palkan RubyConf 2018 RUBY 3X3 57 Compacting GC

Slide 69

Slide 69 text

palkan_tula palkan RubyConf 2018 RUBY 3X3 58 Compacting GC Guilds

Slide 70

Slide 70 text

palkan_tula palkan RubyConf 2018 RUBY DOLL v2 59 Thread Fiber

Slide 71

Slide 71 text

palkan_tula palkan RubyConf 2018 WHAT IS FIBER? https://twitter.com/avdi/status/1057256814721556482 60

Slide 72

Slide 72 text

palkan_tula palkan RubyConf 2018 RUBY DOLL v2 61 Thread Fiber 1MB 128KB

Slide 73

Slide 73 text

palkan_tula palkan RubyConf 2018 RUBY DOLL v3 62 http://www.atdot.net/~ko1/activities/2018_rubyconf2018.pdf Thread Fiber Guild 1MB 128KB

Slide 74

Slide 74 text

palkan_tula palkan RubyConf 2018 –Julian Coolesh (@raventid) “Guilds are for parallelism, not for concurrency.” 63

Slide 75

Slide 75 text

palkan_tula palkan RubyConf 2018 RUBY 3X3 64 Compacting GC Guilds Dynamic thread stack size https://rubykaigi.org/2018/presentations/sugiyama-k.html

Slide 76

Slide 76 text

palkan_tula palkan RubyConf 2018 RUBY DOLL v3 65 Thread Threadlet Guild Fiber https://bugs.ruby-lang.org/issues/13618

Slide 77

Slide 77 text

palkan_tula palkan RubyConf 2018 RUBY 3X3 66 Compacting GC Guilds Dynamic thread stack size Threadlets

Slide 78

Slide 78 text

palkan_tula palkan RubyConf 2018 RUBY 3X3 66 Compacting GC Guilds Dynamic thread stack size Threadlets

Slide 79

Slide 79 text

palkan_tula palkan RubyConf 2018 binding.pry 67

Slide 80

Slide 80 text

palkan_tula palkan RubyConf 2018 CABLE RACE 68

Slide 81

Slide 81 text

palkan_tula palkan RubyConf 2018 CABLE RACE 69

Slide 82

Slide 82 text

AnyCable One cable to rule them all

Slide 83

Slide 83 text

No content

Slide 84

Slide 84 text

palkan_tula palkan RubyConf 2018 Client Channels Broadcaster Server ACTION CABLE 72

Slide 85

Slide 85 text

palkan_tula palkan RubyConf 2018 Action Cable 73 Client Channels Broadcaster Server AnyCable

Slide 86

Slide 86 text

palkan_tula palkan RubyConf 2018 AnyCable 74 ? WS Server

Slide 87

Slide 87 text

palkan_tula palkan RubyConf 2018 75 https://grpc.io gRPC

Slide 88

Slide 88 text

palkan_tula palkan RubyConf 2018 76 https://grpc.io gRPC = universal RPC framework

Slide 89

Slide 89 text

palkan_tula palkan RubyConf 2018 77 https://grpc.io gRPC = HTTP/2 + protobuf

Slide 90

Slide 90 text

palkan_tula palkan RubyConf 2018 AnyCable 78 syntax = "proto3"; package anycable; service RPC { rpc Connect (ConnectionRequest) returns (ConnectionResponse) {} rpc Command (CommandMessage) returns (CommandResponse) {} rpc Disconnect (DisconnectRequest) returns (DisconnectResponse) {} } https://docs.anycable.io/#/rpc_proto

Slide 91

Slide 91 text

palkan_tula palkan RubyConf 2018 AnyCable 79 Go WS

Slide 92

Slide 92 text

No content

Slide 93

Slide 93 text

palkan_tula palkan RubyConf 2018 AnyCable 81 anycable-go erlycable WebSocket Servers

Slide 94

Slide 94 text

palkan_tula palkan RubyConf 2018 Broadcast RTT 0,0s 1,2s 2,4s 3,6s 4,8s 6,0s 7,2s 8,4s 9,6s 10,8s 12,0s Number of connections 1000 3000 5000 7000 9000 AnyCable-Go Iodine Action Cable (8x) AnyCable 82 https://github.com/anycable/anycable/tree/master/benchmarks RSS 0MB 160MB 320MB 480MB 640MB 800MB 960MB 1120MB 1280MB 1440MB 1600MB Number of connections 1000 3000 5000 7000 9000 AnyCable-Go Iodine Action Cable (8x)

Slide 95

Slide 95 text

palkan_tula palkan RubyConf 2018 CPU 83 anycable-go erlycable actioncable

Slide 96

Slide 96 text

palkan_tula palkan RubyConf 2018 CPU 83 anycable-go erlycable actioncable

Slide 97

Slide 97 text

palkan_tula palkan RubyConf 2018 equipe.com 84 ActionCable: 20+ 2X (1GB) dynos AnyCable: 4 X (0.5GB) dynos $1000 $100

Slide 98

Slide 98 text

palkan_tula palkan RubyConf 2018 SHORT-LIVED OBJECTS 85 WS RPC connect identiﬁers (JSON + GlobalID) command (subscribe/perform) with identiﬁers transmissions, streams to subscribe

Slide 99

Slide 99 text

palkan_tula palkan RubyConf 2018 AnyCable 86 Go WS Bottleneck?

Slide 100

Slide 100 text

palkan_tula palkan RubyConf 2018 gRPC RPS 87 RPC type \ Concurrency 1 10 50 AnyCable RPC 1600 2200 2400 Noop RPC with connection object building* 1500 1900 2300 Noop RPC 3000 4600 5500 * Only build Action Cable connection object without performing an action https://github.com/anycable/anycable/blob/master/benchmarks/2018-05-27-rpc-bench.md

Slide 101

Slide 101 text

palkan_tula palkan RubyConf 2018 MRUBY CACHE https://rubykaigi.org/2018/presentations/palkan_tula.html 88

Slide 102

Slide 102 text

palkan_tula palkan RubyConf 2018 PLUG-N-PLAY 89 # Gemfile gem 'anycable-rails' # config/cable.yml production: type: :any_cable # config/environments/production.rb config.action_cable.url = 'ws: //example.com:3334' # install WebSocket server $ brew install anycable/anycable/anycable-go $ bundle exec anycable # => Run gRPC server $ anycable-go --port=3334 # => Run WebSocket server

Slide 103

Slide 103 text

palkan_tula palkan RubyConf 2018 MORE FEATURES 90 Zero-disconnect deployment

Slide 104

Slide 104 text

palkan_tula palkan RubyConf 2018 DISCONNECT 91 WebSocket Server App App gRPC App App App App Envoy Proxy * https://www.envoyproxy.io

Slide 105

Slide 105 text

palkan_tula palkan RubyConf 2018 MORE FEATURES 92 Zero-disconnect deployment Metrics & Stats

Slide 106

Slide 106 text

palkan_tula palkan RubyConf 2018 METRICS 93

Slide 107

Slide 107 text

palkan_tula palkan RubyConf 2018 MORE FEATURES 94 Zero-disconnect deployment Metrics & Stats Rails-free

Slide 108

Slide 108 text

LiteCable github.com/palkan/litecable Rails No More

Slide 109

Slide 109 text

palkan_tula palkan RubyConf 2018 MORE FEATURES 96 Zero-disconnect deployment Metrics & Stats Rails-free NEW Supports GraphQL subscriptions

Slide 110

Slide 110 text

palkan_tula palkan RubyConf 2018 GRAPHQL ANYCABLE 97 github.com/Envek/graphql-anycable

Slide 111

Slide 111 text

palkan_tula palkan RubyConf 2018 github.com/anycable https://anycable.io AnyCable 98

Slide 112

Slide 112 text

palkan_tula palkan RubyConf 2018 docs.anycable.io 99

Slide 113

Slide 113 text

palkan_tula palkan RubyConf 2018 anycable.io/pro 100

Slide 114

Slide 114 text

Slide 115

Slide 115 text

palkan_tula palkan RubyConf 2018 RUBY OR NOT 102 Real-time in Ruby is possible… …but more efﬁcient with the help of other languages …and could become much better with Ruby 3

Slide 116

Slide 116 text

THANK YOU! Vladimir Dementyev anycable.io evilmartians.com @palkan @palkan_tula @evilmartians