Upgrade to Pro — share decks privately, control downloads, hide ads and more …

[RubyConf 2018] High-Speed Cables for Ruby

[RubyConf 2018] High-Speed Cables for Ruby

Vladimir Dementyev

November 16, 2018
Tweet

More Decks by Vladimir Dementyev

Other Decks in Programming

Transcript

  1. HIGH-SPEED CABLES
    FOR RUBY
    Vladimir Dementyev
    @evilmartians

    View Slide

  2. View Slide

  3. CABLES
    Tools for building
    real-time applications

    View Slide

  4. palkan_tula
    palkan RubyConf 2018
    The real-time web is a web … that enable users to receive
    information as soon as it is published by its authors,
    rather than … check a source periodically for updates.
    4
    * https://en.wikipedia.org/wiki/Real-time_web
    BORING DEFINITION

    View Slide

  5. palkan_tula
    palkan RubyConf 2018
    5
    Request-Response
    REAL-TIME

    View Slide

  6. palkan_tula
    palkan RubyConf 2018
    5
    Request-Response
    REAL-TIME

    View Slide

  7. palkan_tula
    palkan RubyConf 2018
    6
    Bi-Directional
    Persistent
    REAL-TIME

    View Slide

  8. palkan_tula
    palkan RubyConf 2018
    6
    Bi-Directional
    Persistent
    REAL-TIME

    View Slide

  9. palkan_tula
    palkan RubyConf 2018
    7
    The opening of the first McDonalds in Russia,
    Moscow, 1991
    REAL-TIME
    Limited number of
    concurrent requests
    (request queue)

    View Slide

  10. palkan_tula
    palkan RubyConf 2018
    8
    Large number of
    concurrent clients
    REAL-TIME
    Opening gala,
    Millennium Biltmore, 1923

    View Slide

  11. palkan_tula
    palkan RubyConf 2018
    EXAMPLES
    9
    Messaging (i.e. chats)
    Notifications
    Live updates
    Online games
    Other

    View Slide

  12. palkan_tula
    palkan RubyConf 2018
    WEBSOCKETS
    10

    View Slide

  13. palkan_tula
    palkan RubyConf 2018
    CONCURRENCY
    11

    View Slide

  14. © https://twitter.com/erneestoc/status/974485805770072064

    View Slide

  15. palkan_tula
    palkan RubyConf 2018
    THE RED RUBY PILL?
    13

    View Slide

  16. palkan_tula
    palkan RubyConf 2018
    14
    Vladimir
    Dementyev

    View Slide

  17. palkan_tula
    palkan RubyConf 2018
    15
    @palkan
    @palkan_tula
    Vladimir Dementyev
    789
    645
    330
    415

    View Slide

  18. palkan_tula
    palkan RubyConf 2018
    ACTION CABLE TESTING
    https://github.com/rails/rails/pull/33659
    16

    View Slide

  19. palkan_tula
    palkan RubyConf 2018
    17
    TEAM
    LID
    Team
    Me

    View Slide

  20. palkan_tula
    palkan RubyConf 2018
    18

    View Slide

  21. palkan_tula
    palkan RubyConf 2018
    19

    View Slide

  22. palkan_tula
    palkan RubyConf 2018
    evilmartians.com
    20

    View Slide

  23. View Slide

  24. palkan_tula
    palkan RubyConf 2018
    evilmartians.com/blog
    22

    View Slide

  25. palkan_tula
    palkan RubyConf 2018
    23
    evilmartians.com
    Brooklyn, NY

    View Slide

  26. THIS TALK

    View Slide

  27. palkan_tula
    palkan RubyConf 2018
    THE TALK
    25
    What is “cable”? ✅
    The tale of Action Cable, Rack & Ruby 3x3
    From Action to Any: another point of view

    View Slide

  28. RUBY
    &
    CABLES
    RailsConf 2015

    View Slide

  29. palkan_tula
    palkan RubyConf 2018
    IN A NUTSHELL
    27
    Server
    Client
    WebSocket
    Client
    WebSocket

    View Slide

  30. palkan_tula
    palkan RubyConf 2018
    IN A NUTSHELL
    27
    Server
    Client
    WebSocket
    Client
    WebSocket
    stream C
    Broadcaster
    stream B
    stream A
    stream B

    View Slide

  31. palkan_tula
    palkan RubyConf 2018
    IN A NUTSHELL
    27
    Server
    Client
    WebSocket
    Client
    WebSocket
    stream C
    Broadcaster
    stream B
    stream A
    stream B
    channel X
    channel Y
    channel Z

    View Slide

  32. palkan_tula
    palkan RubyConf 2018
    CHANNELS
    28
    class AnswersChannel < ApplicationCable ::Channel
    def subscribed
    reject_subscription unless current_user.admin?
    end
    def follow(params)
    stream_from "questions/ #{params['id']}"
    end
    end

    View Slide

  33. palkan_tula
    palkan RubyConf 2018
    CHAT IN 5MIN
    29

    View Slide

  34. palkan_tula
    palkan RubyConf 2018
    CHAT IN 5MIN
    29

    View Slide

  35. palkan_tula
    palkan RubyConf 2018
    BENCHMARKS
    30
    TODAY’S SPECIALS
    HashRocket RTT $9.99
    RAM (all-you-can-eat) $19.59
    CPU Steak 8’’ $20.18

    View Slide

  36. palkan_tula
    palkan RubyConf 2018
    SHOOTOUT
    31
    Client
    Server
    broadcast
    to all
    message
    send message back
    https://hashrocket.com/blog/posts/websocket-shootout

    View Slide

  37. palkan_tula
    palkan RubyConf 2018
    SHOOTOUT
    32
    Broadcast RTT
    0,0s
    1,2s
    2,4s
    3,6s
    4,8s
    6,0s
    7,2s
    8,4s
    9,6s
    10,8s
    12,0s
    Number of connections
    1000 2000 3000 4000 5000 6000 7000 8000 9000 10000
    Go Erlang Action Cable (8x)
    https://github.com/anycable/anycable/tree/master/benchmarks

    View Slide

  38. View Slide

  39. CABLE
    THEOREM
    Action
    Cable
    Low
    Latency
    Crowded
    Streams
    Low
    Latency
    Action
    Cable

    View Slide

  40. High Resources Usage
    CABLE
    THEOREM
    Action
    Cable
    Low
    Latency
    Crowded
    Streams
    Low
    Latency
    Action
    Cable

    View Slide

  41. palkan_tula
    palkan RubyConf 2018
    CPU
    36
    * running WebSocket shootout

    View Slide

  42. palkan_tula
    palkan RubyConf 2018
    CPU
    36
    * running WebSocket shootout

    View Slide

  43. palkan_tula
    palkan RubyConf 2018
    MEMORY
    37
    RSS
    0MB
    160MB
    320MB
    480MB
    640MB
    800MB
    960MB
    1120MB
    1280MB
    1440MB
    1600MB
    Number of connections
    1000 2000 3000 4000 5000 6000 7000 8000 9000 10000
    Go Erlang Action Cable (8x)

    View Slide

  44. WHAT ABOUT
    REAL LIFE?
    …or don’t trust benchmarks
    I WANT TO
    BELIEVE IN
    BENCHMARKS

    View Slide

  45. palkan_tula
    palkan RubyConf 2018
    39
    “With Action Cable, we could
    easily have 20+ 1GB dynos
    running during the weekends,
    with every growing memory”
    –Jon Stenqvist,
    CEO, Equipe
    equipe.com
    NOTE: having ~5-10k conns at peak time

    View Slide

  46. WHYYYY???!!

    View Slide

  47. palkan_tula
    palkan RubyConf 2018
    ACTION CABLE
    41
    rack.hijack

    View Slide

  48. palkan_tula
    palkan RubyConf 2018
    rack.hijack
    42
    Server
    Application

    View Slide

  49. palkan_tula
    palkan RubyConf 2018
    rack.hijack
    42
    Server
    Application
    IO

    View Slide

  50. palkan_tula
    palkan RubyConf 2018
    rack.hijack
    42
    Server
    Application

    View Slide

  51. palkan_tula
    palkan RubyConf 2018
    rack.hijack
    42
    Server
    Application
    IO

    View Slide

  52. palkan_tula
    palkan RubyConf 2018
    THE hijack PRICE
    43
    https://bowild.wordpress.com/2018/05/01/rubys-rack-push-decoupling-the-real-time-web-application-from-the-web/
    –Bo
    “Due to code duplication and extra work,
    the memory consumption for hijack
    based solutions is higher and their
    performance is slower (more system calls,
    more context switches, etc’)..”
    https://bowild.wordpress.com/2018/05/01/rubys-rack-push-decoupling-the-real-time-web-application-from-the-web/

    View Slide

  53. palkan_tula
    palkan RubyConf 2018
    THE hijack PRICE
    44
    Separate IO loop (server)
    WebSockets protocol implementation overhead

    View Slide

  54. palkan_tula
    palkan RubyConf 2018
    plezi.io
    github.com/boazsegev/iodine
    WHO IS BO?
    45
    Boaz Segev (Bo)
    musician (now),
    developer (past)

    View Slide

  55. palkan_tula
    palkan RubyConf 2018
    Broadcast RTT
    0,0s
    1,2s
    2,4s
    3,6s
    4,8s
    6,0s
    7,2s
    8,4s
    9,6s
    10,8s
    12,0s
    Number of connections
    1000 3000 5000 7000 9000
    Go Iodine Action Cable (8x)
    IODINE
    46
    https://github.com/anycable/anycable/tree/master/benchmarks
    RSS
    0MB
    160MB
    320MB
    480MB
    640MB
    800MB
    960MB
    1120MB
    1280MB
    1440MB
    1600MB
    Number of connections
    1000 3000 5000 7000 9000
    Go Iodine Action Cable (8x)

    View Slide

  56. palkan_tula
    palkan RubyConf 2018
    RACK API: FUTURE
    47
    https://github.com/rack/rack/pull/1272

    View Slide

  57. palkan_tula
    palkan RubyConf 2018
    RACK API: FUTURE
    48
    rack.hijack
    Direct access to IO (socket)
    rack.upgrade
    Public interface to “abstracted” IO managed by
    web server

    View Slide

  58. palkan_tula
    palkan RubyConf 2018
    RACK API IN RAILS
    49
    https://github.com/rails/rails/pull/33295

    View Slide

  59. palkan_tula
    palkan RubyConf 2018
    RACK API IN RAILS
    49
    https://github.com/rails/rails/pull/33295
    NO PERFORMANCE
    IMPROVEMENTS

    View Slide

  60. palkan_tula
    palkan RubyConf 2018
    ACTION CABLE
    50
    rack.hijack
    Long-lived objects

    View Slide

  61. palkan_tula
    palkan RubyConf 2018
    ALLOCATIONS
    51
    Retained objects for one connection:
    websocket ~40
    actioncable ~640
    other ~100
    ~60kB

    View Slide

  62. palkan_tula
    palkan RubyConf 2018
    ACTION CABLE
    52
    rack.hijack
    Long-lived objects
    Heap fragmentation

    View Slide

  63. palkan_tula
    palkan RubyConf 2018
    HEAPY HEAPY SHAKE
    53

    View Slide

  64. palkan_tula
    palkan RubyConf 2018
    HEAPY HEAPY SHAKE
    53
    Heap
    fragmentation?
    Are you sure?

    View Slide

  65. RUBY 3X3
    The future we’re
    waiting for

    View Slide

  66. View Slide

  67. palkan_tula
    palkan RubyConf 2018
    56
    GENERATIONAL
    HYPOTHESIS

    View Slide

  68. palkan_tula
    palkan RubyConf 2018
    RUBY 3X3
    57
    Compacting GC

    View Slide

  69. palkan_tula
    palkan RubyConf 2018
    RUBY 3X3
    58
    Compacting GC
    Guilds

    View Slide

  70. palkan_tula
    palkan RubyConf 2018
    RUBY DOLL v2
    59
    Thread Fiber

    View Slide

  71. palkan_tula
    palkan RubyConf 2018
    WHAT IS FIBER?
    https://twitter.com/avdi/status/1057256814721556482
    60

    View Slide

  72. palkan_tula
    palkan RubyConf 2018
    RUBY DOLL v2
    61
    Thread Fiber
    1MB
    128KB

    View Slide

  73. palkan_tula
    palkan RubyConf 2018
    RUBY DOLL v3
    62
    http://www.atdot.net/~ko1/activities/2018_rubyconf2018.pdf
    Thread Fiber
    Guild
    1MB
    128KB

    View Slide

  74. palkan_tula
    palkan RubyConf 2018
    –Julian Coolesh (@raventid)
    “Guilds are for parallelism,
    not for concurrency.”
    63

    View Slide

  75. palkan_tula
    palkan RubyConf 2018
    RUBY 3X3
    64
    Compacting GC
    Guilds
    Dynamic thread stack size
    https://rubykaigi.org/2018/presentations/sugiyama-k.html

    View Slide

  76. palkan_tula
    palkan RubyConf 2018
    RUBY DOLL v3
    65
    Thread Threadlet
    Guild Fiber
    https://bugs.ruby-lang.org/issues/13618

    View Slide

  77. palkan_tula
    palkan RubyConf 2018
    RUBY 3X3
    66
    Compacting GC
    Guilds
    Dynamic thread stack size
    Threadlets

    View Slide

  78. palkan_tula
    palkan RubyConf 2018
    RUBY 3X3
    66
    Compacting GC
    Guilds
    Dynamic thread stack size
    Threadlets

    View Slide

  79. palkan_tula
    palkan RubyConf 2018
    binding.pry
    67

    View Slide

  80. palkan_tula
    palkan RubyConf 2018
    CABLE RACE
    68

    View Slide

  81. palkan_tula
    palkan RubyConf 2018
    CABLE RACE
    69

    View Slide

  82. AnyCable
    One cable to rule them all

    View Slide

  83. View Slide

  84. palkan_tula
    palkan RubyConf 2018
    Client
    Channels
    Broadcaster
    Server
    ACTION CABLE
    72

    View Slide

  85. palkan_tula
    palkan RubyConf 2018
    Action Cable
    73
    Client
    Channels
    Broadcaster
    Server
    AnyCable

    View Slide

  86. palkan_tula
    palkan RubyConf 2018
    AnyCable
    74
    ?
    WS Server

    View Slide

  87. palkan_tula
    palkan RubyConf 2018
    75
    https://grpc.io
    gRPC

    View Slide

  88. palkan_tula
    palkan RubyConf 2018
    76
    https://grpc.io
    gRPC =
    universal
    RPC
    framework

    View Slide

  89. palkan_tula
    palkan RubyConf 2018
    77
    https://grpc.io
    gRPC =
    HTTP/2 +
    protobuf

    View Slide

  90. palkan_tula
    palkan RubyConf 2018
    AnyCable
    78
    syntax = "proto3";
    package anycable;
    service RPC {
    rpc Connect (ConnectionRequest) returns (ConnectionResponse) {}
    rpc Command (CommandMessage) returns (CommandResponse) {}
    rpc Disconnect (DisconnectRequest) returns (DisconnectResponse) {}
    }
    https://docs.anycable.io/#/rpc_proto

    View Slide

  91. palkan_tula
    palkan RubyConf 2018
    AnyCable
    79
    Go WS

    View Slide

  92. View Slide

  93. palkan_tula
    palkan RubyConf 2018
    AnyCable
    81
    anycable-go
    erlycable
    WebSocket Servers

    View Slide

  94. palkan_tula
    palkan RubyConf 2018
    Broadcast RTT
    0,0s
    1,2s
    2,4s
    3,6s
    4,8s
    6,0s
    7,2s
    8,4s
    9,6s
    10,8s
    12,0s
    Number of connections
    1000 3000 5000 7000 9000
    AnyCable-Go Iodine
    Action Cable (8x)
    AnyCable
    82
    https://github.com/anycable/anycable/tree/master/benchmarks
    RSS
    0MB
    160MB
    320MB
    480MB
    640MB
    800MB
    960MB
    1120MB
    1280MB
    1440MB
    1600MB
    Number of connections
    1000 3000 5000 7000 9000
    AnyCable-Go Iodine
    Action Cable (8x)

    View Slide

  95. palkan_tula
    palkan RubyConf 2018
    CPU
    83
    anycable-go erlycable actioncable

    View Slide

  96. palkan_tula
    palkan RubyConf 2018
    CPU
    83
    anycable-go erlycable actioncable

    View Slide

  97. palkan_tula
    palkan RubyConf 2018
    equipe.com
    84
    ActionCable: 20+ 2X (1GB) dynos AnyCable: 4 X (0.5GB) dynos
    $1000 $100

    View Slide

  98. palkan_tula
    palkan RubyConf 2018
    SHORT-LIVED OBJECTS
    85
    WS RPC
    connect
    identifiers (JSON + GlobalID)
    command (subscribe/perform)
    with identifiers
    transmissions, streams to subscribe

    View Slide

  99. palkan_tula
    palkan RubyConf 2018
    AnyCable
    86
    Go WS
    Bottleneck?

    View Slide

  100. palkan_tula
    palkan RubyConf 2018
    gRPC RPS
    87
    RPC type \ Concurrency 1 10 50
    AnyCable RPC 1600 2200 2400
    Noop RPC with connection object building* 1500 1900 2300
    Noop RPC 3000 4600 5500
    * Only build Action Cable connection object without performing an action
    https://github.com/anycable/anycable/blob/master/benchmarks/2018-05-27-rpc-bench.md

    View Slide

  101. palkan_tula
    palkan RubyConf 2018
    MRUBY CACHE
    https://rubykaigi.org/2018/presentations/palkan_tula.html
    88

    View Slide

  102. palkan_tula
    palkan RubyConf 2018
    PLUG-N-PLAY
    89
    # Gemfile
    gem 'anycable-rails'
    # config/cable.yml
    production:
    type: :any_cable
    # config/environments/production.rb
    config.action_cable.url = 'ws: //example.com:3334'
    # install WebSocket server
    $ brew install anycable/anycable/anycable-go
    $ bundle exec anycable # => Run gRPC server
    $ anycable-go --port=3334 # => Run WebSocket server

    View Slide

  103. palkan_tula
    palkan RubyConf 2018
    MORE FEATURES
    90
    Zero-disconnect deployment

    View Slide

  104. palkan_tula
    palkan RubyConf 2018
    DISCONNECT
    91
    WebSocket Server
    App
    App
    gRPC
    App
    App
    App
    App
    Envoy Proxy
    * https://www.envoyproxy.io

    View Slide

  105. palkan_tula
    palkan RubyConf 2018
    MORE FEATURES
    92
    Zero-disconnect deployment
    Metrics & Stats

    View Slide

  106. palkan_tula
    palkan RubyConf 2018
    METRICS
    93

    View Slide

  107. palkan_tula
    palkan RubyConf 2018
    MORE FEATURES
    94
    Zero-disconnect deployment
    Metrics & Stats
    Rails-free

    View Slide

  108. LiteCable
    github.com/palkan/litecable
    Rails No More

    View Slide

  109. palkan_tula
    palkan RubyConf 2018
    MORE FEATURES
    96
    Zero-disconnect deployment
    Metrics & Stats
    Rails-free
    NEW Supports GraphQL subscriptions

    View Slide

  110. palkan_tula
    palkan RubyConf 2018
    GRAPHQL ANYCABLE
    97
    github.com/Envek/graphql-anycable

    View Slide

  111. palkan_tula
    palkan RubyConf 2018
    github.com/anycable
    https://anycable.io
    AnyCable
    98

    View Slide

  112. palkan_tula
    palkan RubyConf 2018
    docs.anycable.io
    99

    View Slide

  113. palkan_tula
    palkan RubyConf 2018
    anycable.io/pro
    100

    View Slide

  114. RUBY
    OR NOT
    © RailsClub Moscow

    View Slide

  115. palkan_tula
    palkan RubyConf 2018
    RUBY OR NOT
    102
    Real-time in Ruby is possible…
    …but more efficient with the help of
    other languages
    …and could become much better
    with Ruby 3

    View Slide

  116. THANK YOU!
    Vladimir Dementyev
    anycable.io
    evilmartians.com
    @palkan
    @palkan_tula
    @evilmartians

    View Slide