Upgrade to Pro — share decks privately, control downloads, hide ads and more …

TCP Socket programming in Ruby

TCP Socket programming in Ruby

As software developers, a lot of the time we're building applications that rely on some sort of network connection. Due to Ruby's great abstractions we take most of the network related stuff for granted. We think we know how all of that works, but do we? Let's go over the fundamentals together, learn about how Ruby models TCP Sockets and how we can make a good use of it.

Even if you're not doing network programming, being able to dive multiple levels to understand what's going on will give you a great advantage. The sort of concepts we'll go over this talk don't apply to Ruby only. All modern languages support the Berkeley Sockets API so this knowledge is portable and it will serve you for many years to come.

During the talk we will go through the fundamentals of programming with sockets. This includes creating sockets, client and server life-cycle, reading and writing data, non-blocking IO and connection multiplexing.

Presented at: RubyConf Uruguay 2014

92d08794b535e41a4082c57ea547546e?s=128

Sebastian Sogamoso

May 23, 2014
Tweet

Transcript

  1. TCP Socket Programming in Ruby

  2. @sebasoga Sebastián Sogamoso

  3. None
  4. What is a Socket?

  5. A way to speak to other programs using standard Unix

    file descriptors
  6. A way to communicate within a process, between processes on

    the same machine, or between processes on different continents
  7. “Everything in Unix is a file”

  8. Yes, a socket acts like a file

  9. socket(2)

  10. Socket Domain

  11. AF_LOCAL (UNIX) Socket Domain

  12. AF_INET AF_LOCAL (UNIX) Socket Domain

  13. AF_INET AF_LOCAL (UNIX) AF_INET6 Socket Domain

  14. AF_INET AF_LOCAL (UNIX) AF_INET6 … Socket Domain

  15. AF_INET AF_LOCAL (UNIX) AF_INET6 … Socket Domain Internet

  16. AF_INET AF_LOCAL (UNIX) AF_INET6 … Socket Domain IPV4

  17. AF_INET AF_LOCAL (UNIX) AF_INET6 … Socket Domain IPV6 IPV4

  18. Socket Type

  19. SOCK_RAW Socket Type

  20. SOCK_DRAM SOCK_RAW Socket Type

  21. SOCK_DRAM SOCK_RAW SOCK_STREAM Socket Type

  22. SOCK_DRAM SOCK_RAW SOCK_STREAM Socket Type …

  23. Socket Type SOCK_DRAM SOCK_RAW SOCK_STREAM … TCP

  24. Socket Protocol

  25. IPPROTO_TCP IPPROTO_SCTP IPPROTO_UDP … Socket Protocol

  26. Socket Type: SOCK_STREAM Socket Domain: AF_INET / AF_INET6 TCP Socket

  27. Address (IP) TCP Socket

  28. Port Address (IP) TCP Socket

  29. Port Address (IP) TCP Socket Unique

  30. TCP Socket Theory

  31. TCP Socket Programming in Ruby

  32. require ‘socket’

  33. require ‘socket’ From Ruby standard library

  34. require ‘socket’ ! socket = Socket.new(:INET, :STREAM, 0)

  35. require ‘socket’ ! socket = Socket.new(:INET, :STREAM)

  36. require ‘socket’ ! socket = Socket.new(:INET, :STREAM) Domain

  37. require ‘socket’ ! socket = Socket.new(:INET, :STREAM) Socket::AF_INET

  38. require ‘socket’ ! socket = Socket.new(:INET, :STREAM) Type

  39. require ‘socket’ ! socket = Socket.new(:INET, :STREAM) Socket::SOCK_STREAM

  40. “Everything in Unix is a file”

  41. Ruby Standard Library Socket

  42. Ruby Standard Library Socket BasicSocket

  43. Ruby Standard Library Socket BasicSocket IO

  44. Yes, a socket acts like a file

  45. Ruby Standard Library .open Socket

  46. Socket Ruby Standard Library .open #close

  47. Socket Ruby Standard Library .open #close #read

  48. Socket Ruby Standard Library .open #close #read #write

  49. Sockets API

  50. Sockets (Berkley Socket API) API

  51. Sockets (POSIX Socket) API

  52. socket(2). Creates an endpoint for communication

  53. socket(2). Creates an endpoint for communication bind(2). Assigns a socket

    to an address
  54. socket(2). Creates an endpoint for communication listen(2). Prepares a socket

    for incoming connections bind(2). Assigns a socket to an address
  55. accept(2). Manages new incoming connections on a socket

  56. connect(2). Connects to a socket in the given address accept(2).

    Manages new incoming connections on a socket
  57. connect(2). Connects to a socket in the given address getpeername(2).

    Gets information about of the peer socket accept(2). Manages new incoming connections on a socket
  58. Server

  59. require ‘socket’ ! server = Socket.new(:INET, :STREAM)

  60. require ‘socket’ ! server = Socket.new(:INET, :STREAM) ! address =

    Socket.pack_sockaddr_in( 5000,‘0.0.0.0’)
  61. require ‘socket’ ! server = Socket.new(:INET, :STREAM) ! address =

    Socket.pack_sockaddr_in( 5000,‘0.0.0.0’) Struct containing an adress
  62. require ‘socket’ ! server = Socket.new(:INET, :STREAM) ! address =

    Socket.pack_sockaddr_in( 5000,‘0.0.0.0’) Port
  63. require ‘socket’ ! server = Socket.new(:INET, :STREAM) ! address =

    Socket.pack_sockaddr_in( 5000,‘0.0.0.0’) Host
  64. require ‘socket’ ! server = Socket.new(:INET, :STREAM) ! address =

    Socket.pack_sockaddr_in( 5000,‘0.0.0.0’) ! server.bind(address)
  65. require ‘socket’ ! server = Socket.new(:INET, :STREAM) ! address =

    Socket.pack_sockaddr_in( 5000,‘0.0.0.0’) ! server.bind(address) server.accept(128)
  66. require ‘socket’ ! server = Socket.new(:INET, :STREAM) ! address =

    Socket.pack_sockaddr_in( 5000,‘0.0.0.0’) ! server.bind(address) server.accept(128) Size of listen queue
  67. require ‘socket’ ! server = Socket.new(:INET, :STREAM) ! address =

    Socket.pack_sockaddr_in( 5000,‘0.0.0.0’) ! server.bind(address) server.accept(128) socket.close
  68. Client

  69. require ‘socket’ ! client = Socket.new(:INET, :STREAM) ! address =

    Socket.pack_sockaddr_in( 80,‘rubyconfuruguay.org’)
  70. require ‘socket’ ! client = Socket.new(:INET, :STREAM) ! address =

    Socket.pack_sockaddr_in( 80,‘rubyconfuruguay.org’) ! client.connect(address)
  71. connect(2) is blocking

  72. require ‘socket’ ! client = Socket.new(:INET, :STREAM) ! address =

    Socket.pack_sockaddr_in( 80,‘rubyconfuruguay.org’) ! client.connect(address) ! client.close
  73. Wrappers TCP Sockets

  74. TCP Sockets Wrappers (Syntactic sugar)

  75. Server

  76. require ‘socket’ ! server = Socket.new(:INET, :STREAM) ! address =

    Socket.pack_sockaddr_in( 5000,‘0.0.0.0’) ! server.bind(address) server.accept(128) socket.close
  77. require ‘socket’ ! Socket.tcp_server_loop(5000) do |c| . . . c.close

    end
  78. Client

  79. require ‘socket’ ! client = Socket.new(:INET, :STREAM) ! address =

    Socket.pack_sockaddr_in( 80,‘rubysur.org’) ! client.connect(address) ! client.close
  80. require ‘socket’ ! Socket.tcp(‘rubysur.org’, 80) do |c| . . .

    c.close end
  81. require ‘socket’ ! Socket.tcp(‘rubysur.org’, 80) do |c| . . .

    c.close end ! #OR ! TCPSocket.new(‘rubysur.org’, 80)
  82. Socket IO

  83. Reading

  84. require ‘socket’ ! Socket.tcp_server(5000) do |conn| ! end

  85. require ‘socket’ ! Socket.tcp_server(5000) do |conn| data = conn.read end

  86. require ‘socket’ ! Socket.tcp_server(5000) do |conn| data = conn.read end

    As with any IO object
  87. require ‘socket’ ! Socket.tcp_server(5000) do |conn| data = conn.read process(data)

    conn.close end
  88. read(2) is blocking

  89. Won’t return while stream of data is open, even if

    no data is being received
  90. require ‘socket’ ! chunk_size = 1024 # 1kb

  91. require ‘socket’ ! chunk_size = 1024 # 1kb ! Socket.tcp_server(5000)

    do |c| ! end
  92. require ‘socket’ ! chunk_size = 1024 # 1kb ! Socket.tcp_server(5000)

    do |c| while data = c.read(chunk_size) do process(data) end end
  93. require ‘socket’ ! chunk_size = 1024 # 1 kb !

    Socket.tcp_server(5000) do |c| while data = c.read(chunk_size) do process(data) end end Returns each time ‘chunk_size’ of data is received
  94. require ‘socket’ ! chunk_size = 1024 # 1kb ! Socket.tcp_server(5000)

    do |c| while data = c.read(chunk_size) do process(data) end c.close end
  95. It won’t return until it receives data this size of

    the ‘chunk_size’
  96. require ‘socket’ ! chunk_size = 1024 # 1kb ! Socket.tcp_server(5000)

    do |c| ! end
  97. require ‘socket’ ! chunk_size = 1024 # 1kb ! Socket.tcp_server(5000)

    do |c| while data = c.readpartial(chunk_size) do process(data) end end
  98. require ‘socket’ ! chunk_size = 1024 # 1kb ! Socket.tcp_server(5000)

    do |c| while data = c.readpartial(chunk_size) do process(data) end end Returns chunks of data upto ‘chunk_size’ when available
  99. require ‘socket’ ! chunk_size = 1024 # 1kb ! Socket.tcp_server(5000)

    do |c| begin while data = c.readpartial(chunk_size) do process(data) end recue EOFError end end
  100. require ‘socket’ ! chunk_size = 1024 # 1kb ! Socket.tcp_server(5000)

    do |c| begin while data = c.readpartial(chunk_size) do process(data) end recue EOFError end end Raises EOFError when stream is closed
  101. require ‘socket’ ! chunk_size = 1024 # 1kb ! Socket.tcp_server(5000)

    do |c| begin while data = c.readpartial(chunk_size) do process(data) end recue EOFError end c.close end
  102. readpartial is eager returning data as it is available. It

    will still block until it receives EOF
  103. Writing

  104. write(2) is blocking

  105. require ‘socket’ ! Socket.tcp_server(5000) do |c| ! end

  106. require ‘socket’ ! Socket.tcp_server(5000) do |c| c.write(‘Hello Montevideo!’) c.close end

  107. Server/Client Lifecycle

  108. Server Client

  109. Server socket Client

  110. Server bind socket Client

  111. Server listen bind socket Client

  112. Server accept listen bind socket Client

  113. Client Server socket accept listen bind socket

  114. Client Server socket connect accept listen bind socket

  115. Client Server socket connect accept listen bind socket

  116. Client Server socket connect accept listen bind socket write

  117. Client Server socket connect accept listen bind socket read write

  118. Client Server socket connect read accept listen bind socket write

    write
  119. Client Server socket connect read accept listen bind socket write

    write Processes request
  120. Client Server socket connect write write read accept listen bind

    socket
  121. Client Server socket connect write write read accept listen bind

    socket read
  122. Client Server socket connect write write read accept listen bind

    socket read
  123. Client Server socket connect write write read accept listen bind

    socket read
  124. Client Server socket connect write read close write read accept

    listen bind socket
  125. Client Server socket connect write read close write read accept

    listen bind socket
  126. Client Server socket connect write read close write read accept

    listen bind socket close
  127. Client Server socket connect write read close close write read

    accept listen bind socket
  128. Non-blocking IO & Connections

  129. Reading

  130. require ‘socket’ ! Socket.tcp_server(5000) do |connection| loop do ! end

    end
  131. require ‘socket’ ! Socket.tcp_server(5000) do |connection| loop do data =

    connection.read_nonblock(1024) process(data) end end
  132. require ‘socket’ ! Socket.tcp_server(5000) do |connection| loop do begin data

    = connection.read_nonblock(1024) process(data) rescue Error::EAGAIN IO.select([connection]) ; retry end end end
  133. require ‘socket’ ! Socket.tcp_server(5000) do |connection| loop do begin data

    = connection.read_nonblock(1024) process(data) rescue Error::EAGAIN IO.select([connection]) ; retry end end end When there is no data to read
  134. require ‘socket’ ! Socket.tcp_server(5000) do |connection| loop do begin data

    = connection.read_nonblock(1024) process(data) rescue Error::EAGAIN IO.select([connection]) ; retry end end end Wait until it is ready to be read
  135. require ‘socket’ ! Socket.tcp_server(5000) do |connection| loop do begin data

    = connection.read_nonblock(1024) process(data) rescue Error::EAGAIN IO.select([connection]) ; retry rescue EOFError ; break end end connection.close end
  136. require ‘socket’ ! Socket.tcp_server(5000) do |connection| loop do begin data

    = connection.read_nonblock(1024) process(data) rescue Error::EAGAIN IO.select([connection]) ; retry rescue EOFError ; break end end connection.close end When the connection is closed (EOF)
  137. Writing

  138. require ‘socket’ ! TCPSocket.new(‘0.0.0.0’, 5000)

  139. require ‘socket’ ! TCPSocket.new(‘0.0.0.0’, 5000) message = ‘RubyConf Uruguay 2014’

    * 10_000
  140. require ‘socket’ ! TCPSocket.new(‘0.0.0.0’, 5000) message = ‘RubyConf Uruguay 2014’

    * 10_000 ! bytes = connection.write_nonblock(message)
  141. require ‘socket’ ! TCPSocket.new(‘0.0.0.0’, 5000) message = ‘RubyConf Uruguay 2014’

    * 10_000 ! loop do bytes = connection.write_nonblock(message) break if bytes >= message.size message.slice!(0, bytes) IO.select([], [connection]) end
  142. require ‘socket’ ! TCPSocket.new(‘0.0.0.0’, 5000) message = ‘RubyConf Uruguay 2014’

    * 10_000 ! begin loop do bytes = connection.write_nonblock(message) break if bytes >= message.size message.slice!(0, bytes) IO.select([], [connection]) end rescue Error::EAGAIN IO.select([], [connection]) ; retry end
  143. Connection

  144. require ‘socket’ ! client = Socket.new(:INET, :STREAM) address = Socket.pack_sockaddr_in(

    80,‘rubysur.org’)
  145. require ‘socket’ ! client = Socket.new(:INET, :STREAM) address = Socket.pack_sockaddr_in(

    80,‘rubysur.org’) ! client.connect_nonblock(address)
  146. require ‘socket’ ! client = Socket.new(:INET, :STREAM) address = Socket.pack_sockaddr_in(

    80,‘rubysur.org’) ! client.connect_nonblock(address) rescue Errno::EINPROGRESS
  147. require ‘socket’ ! client = Socket.new(:INET, :STREAM) address = Socket.pack_sockaddr_in(

    80,‘rubysur.org’) ! client.connect_nonblock(address) rescue Errno::EINPROGRESS Notifies and connects on background
  148. require ‘socket’ ! client = Socket.new(:INET, :STREAM) address = Socket.pack_sockaddr_in(

    80,‘rubysur.org’) ! client.connect_nonblock(address) rescue Errno::EINPROGRESS rescue Errno::EALREADY Non-blocking connection already in progress
  149. Connection Multiplexing

  150. Connection multiplexing refers to working with multiple active sockets

  151. We can use select(2) to get the sockets that are

    ready for reading, writing, or have an exceptional condition pending
  152. select(2) is blocking

  153. sockets = [<TCPSocket>, <TCPSocket>, …] ! IO.select(sockets, sockets, sockets, 5)

  154. sockets = [<TCPSocket>, <TCPSocket>, …] ! IO.select(sockets, sockets, sockets, 5)

    Sockets to wait until readable
  155. sockets = [<TCPSocket>, <TCPSocket>, …] ! IO.select(sockets, sockets, sockets, 5)

    Sockets to wait until writable
  156. sockets = [<TCPSocket>, <TCPSocket>, …] ! IO.select(sockets, sockets, sockets, 5)

    Sockets to wait until an exception occurs
  157. sockets = [<TCPSocket>, <TCPSocket>, …] ! IO.select(sockets, sockets, sockets, 5)

    Timeout in seconds
  158. sockets = [<TCPSocket>, <TCPSocket>, …] ! IO.select(sockets, sockets, sockets, 5)

    => [[<TCPSocket>, <TCPSocket>], [<TCPSocket>], [] ]
  159. sockets = [<TCPSocket>, <TCPSocket>, …] ! IO.select(sockets, sockets, sockets, 5)

    => [[<TCPSocket>, <TCPSocket>], [<TCPSocket>], [] ] ! IO.select(sockets, sockets, sockets, 1) => nil
  160. #isTCPDead

  161. None
  162. Why should I care about this stuff?

  163. You will be able to do magic…

  164. None
  165. This concepts go beyond Ruby

  166. Recommended Reading

  167. None
  168. None
  169. Thank you @sebasoga