TCP Socket programming in Ruby

As software developers, a lot of the time we're building applications that rely on some sort of network connection. Due to Ruby's great abstractions we take most of the network related stuff for granted. We think we know how all of that works, but do we? Let's go over the fundamentals together, learn about how Ruby models TCP Sockets and how we can make a good use of it.

Even if you're not doing network programming, being able to dive multiple levels to understand what's going on will give you a great advantage. The sort of concepts we'll go over this talk don't apply to Ruby only. All modern languages support the Berkeley Sockets API so this knowledge is portable and it will serve you for many years to come.

During the talk we will go through the fundamentals of programming with sockets. This includes creating sockets, client and server life-cycle, reading and writing data, non-blocking IO and connection multiplexing.

Presented at: RubyConf Uruguay 2014

Transcript

1. TCP Socket Programming in Ruby
2. @sebasoga Sebastián Sogamoso
4. What is a Socket?
5. A way to speak to other programs using standard Unix file descriptors
6. A way to communicate within a process, between processes on the same machine, or between processes on different continents
7. “Everything in Unix is a file”
8. Yes, a socket acts like a file
9. socket(2)
10. Socket Domain
11. AF_LOCAL (UNIX) Socket Domain
12. AF_INET AF_LOCAL (UNIX) Socket Domain
13. AF_INET AF_LOCAL (UNIX) AF_INET6 Socket Domain
14. AF_INET AF_LOCAL (UNIX) AF_INET6 … Socket Domain
15. AF_INET AF_LOCAL (UNIX) AF_INET6 … Socket Domain Internet
16. AF_INET AF_LOCAL (UNIX) AF_INET6 … Socket Domain IPV4
17. AF_INET AF_LOCAL (UNIX) AF_INET6 … Socket Domain IPV6 IPV4
18. Socket Type
19. SOCK_RAW Socket Type
20. SOCK_DRAM SOCK_RAW Socket Type
21. SOCK_DRAM SOCK_RAW SOCK_STREAM Socket Type
22. SOCK_DRAM SOCK_RAW SOCK_STREAM Socket Type …
23. Socket Type SOCK_DRAM SOCK_RAW SOCK_STREAM … TCP
24. Socket Protocol
25. IPPROTO_TCP IPPROTO_SCTP IPPROTO_UDP … Socket Protocol
26. Socket Type: SOCK_STREAM Socket Domain: AF_INET / AF_INET6 TCP Socket
27. Address (IP) TCP Socket
28. Port Address (IP) TCP Socket
29. Port Address (IP) TCP Socket Unique
30. TCP Socket Theory
31. TCP Socket Programming in Ruby
32. require ‘socket’
33. require ‘socket’ From Ruby standard library
34. require ‘socket’ ! socket = Socket.new(:INET, :STREAM, 0)
35. require ‘socket’ ! socket = Socket.new(:INET, :STREAM)
36. require ‘socket’ ! socket = Socket.new(:INET, :STREAM) Domain
37. require ‘socket’ ! socket = Socket.new(:INET, :STREAM) Socket::AF_INET
38. require ‘socket’ ! socket = Socket.new(:INET, :STREAM) Type
39. require ‘socket’ ! socket = Socket.new(:INET, :STREAM) Socket::SOCK_STREAM
40. “Everything in Unix is a file”
41. Ruby Standard Library Socket
42. Ruby Standard Library Socket BasicSocket
43. Ruby Standard Library Socket BasicSocket IO
44. Yes, a socket acts like a file
45. Ruby Standard Library .open Socket
46. Socket Ruby Standard Library .open #close
47. Socket Ruby Standard Library .open #close #read
48. Socket Ruby Standard Library .open #close #read #write
49. Sockets API
50. Sockets (Berkley Socket API) API
51. Sockets (POSIX Socket) API
52. socket(2). Creates an endpoint for communication
53. socket(2). Creates an endpoint for communication bind(2). Assigns a socket to an address
54. socket(2). Creates an endpoint for communication listen(2). Prepares a socket for incoming connections bind(2). Assigns a socket to an address
55. accept(2). Manages new incoming connections on a socket
56. connect(2). Connects to a socket in the given address accept(2). Manages new incoming connections on a socket
57. connect(2). Connects to a socket in the given address getpeername(2). Gets information about of the peer socket accept(2). Manages new incoming connections on a socket
58. Server
59. require ‘socket’ ! server = Socket.new(:INET, :STREAM)
60. require ‘socket’ ! server = Socket.new(:INET, :STREAM) ! address = Socket.pack_sockaddr_in( 5000,‘0.0.0.0’)
61. require ‘socket’ ! server = Socket.new(:INET, :STREAM) ! address = Socket.pack_sockaddr_in( 5000,‘0.0.0.0’) Struct containing an adress
62. require ‘socket’ ! server = Socket.new(:INET, :STREAM) ! address = Socket.pack_sockaddr_in( 5000,‘0.0.0.0’) Port
63. require ‘socket’ ! server = Socket.new(:INET, :STREAM) ! address = Socket.pack_sockaddr_in( 5000,‘0.0.0.0’) Host
64. require ‘socket’ ! server = Socket.new(:INET, :STREAM) ! address = Socket.pack_sockaddr_in( 5000,‘0.0.0.0’) ! server.bind(address)
65. require ‘socket’ ! server = Socket.new(:INET, :STREAM) ! address = Socket.pack_sockaddr_in( 5000,‘0.0.0.0’) ! server.bind(address) server.accept(128)
66. require ‘socket’ ! server = Socket.new(:INET, :STREAM) ! address = Socket.pack_sockaddr_in( 5000,‘0.0.0.0’) ! server.bind(address) server.accept(128) Size of listen queue
67. require ‘socket’ ! server = Socket.new(:INET, :STREAM) ! address = Socket.pack_sockaddr_in( 5000,‘0.0.0.0’) ! server.bind(address) server.accept(128) socket.close
68. Client
69. require ‘socket’ ! client = Socket.new(:INET, :STREAM) ! address = Socket.pack_sockaddr_in( 80,‘rubyconfuruguay.org’)
70. require ‘socket’ ! client = Socket.new(:INET, :STREAM) ! address = Socket.pack_sockaddr_in( 80,‘rubyconfuruguay.org’) ! client.connect(address)
71. connect(2) is blocking
72. require ‘socket’ ! client = Socket.new(:INET, :STREAM) ! address = Socket.pack_sockaddr_in( 80,‘rubyconfuruguay.org’) ! client.connect(address) ! client.close
73. Wrappers TCP Sockets
74. TCP Sockets Wrappers (Syntactic sugar)
75. Server
76. require ‘socket’ ! server = Socket.new(:INET, :STREAM) ! address = Socket.pack_sockaddr_in( 5000,‘0.0.0.0’) ! server.bind(address) server.accept(128) socket.close
77. require ‘socket’ ! Socket.tcp_server_loop(5000) do |c| . . . c.close end
78. Client
79. require ‘socket’ ! client = Socket.new(:INET, :STREAM) ! address = Socket.pack_sockaddr_in( 80,‘rubysur.org’) ! client.connect(address) ! client.close
80. require ‘socket’ ! Socket.tcp(‘rubysur.org’, 80) do |c| . . . c.close end
81. require ‘socket’ ! Socket.tcp(‘rubysur.org’, 80) do |c| . . . c.close end ! #OR ! TCPSocket.new(‘rubysur.org’, 80)
82. Socket IO
83. Reading
84. require ‘socket’ ! Socket.tcp_server(5000) do |conn| ! end
85. require ‘socket’ ! Socket.tcp_server(5000) do |conn| data = conn.read end
86. require ‘socket’ ! Socket.tcp_server(5000) do |conn| data = conn.read end As with any IO object
87. require ‘socket’ ! Socket.tcp_server(5000) do |conn| data = conn.read process(data) conn.close end
88. read(2) is blocking
89. Won’t return while stream of data is open, even if no data is being received
90. require ‘socket’ ! chunk_size = 1024 # 1kb
91. require ‘socket’ ! chunk_size = 1024 # 1kb ! Socket.tcp_server(5000) do |c| ! end
92. require ‘socket’ ! chunk_size = 1024 # 1kb ! Socket.tcp_server(5000) do |c| while data = c.read(chunk_size) do process(data) end end
93. require ‘socket’ ! chunk_size = 1024 # 1 kb ! Socket.tcp_server(5000) do |c| while data = c.read(chunk_size) do process(data) end end Returns each time ‘chunk_size’ of data is received
94. require ‘socket’ ! chunk_size = 1024 # 1kb ! Socket.tcp_server(5000) do |c| while data = c.read(chunk_size) do process(data) end c.close end
95. It won’t return until it receives data this size of the ‘chunk_size’
96. require ‘socket’ ! chunk_size = 1024 # 1kb ! Socket.tcp_server(5000) do |c| ! end
97. require ‘socket’ ! chunk_size = 1024 # 1kb ! Socket.tcp_server(5000) do |c| while data = c.readpartial(chunk_size) do process(data) end end
98. require ‘socket’ ! chunk_size = 1024 # 1kb ! Socket.tcp_server(5000) do |c| while data = c.readpartial(chunk_size) do process(data) end end Returns chunks of data upto ‘chunk_size’ when available
99. require ‘socket’ ! chunk_size = 1024 # 1kb ! Socket.tcp_server(5000) do |c| begin while data = c.readpartial(chunk_size) do process(data) end recue EOFError end end
100. require ‘socket’ ! chunk_size = 1024 # 1kb ! Socket.tcp_server(5000) do |c| begin while data = c.readpartial(chunk_size) do process(data) end recue EOFError end end Raises EOFError when stream is closed
101. require ‘socket’ ! chunk_size = 1024 # 1kb ! Socket.tcp_server(5000) do |c| begin while data = c.readpartial(chunk_size) do process(data) end recue EOFError end c.close end
102. readpartial is eager returning data as it is available. It will still block until it receives EOF
103. Writing
104. write(2) is blocking
105. require ‘socket’ ! Socket.tcp_server(5000) do |c| ! end
106. require ‘socket’ ! Socket.tcp_server(5000) do |c| c.write(‘Hello Montevideo!’) c.close end
107. Server/Client Lifecycle
108. Server Client
109. Server socket Client
110. Server bind socket Client
111. Server listen bind socket Client
112. Server accept listen bind socket Client
113. Client Server socket accept listen bind socket
114. Client Server socket connect accept listen bind socket
115. Client Server socket connect accept listen bind socket
116. Client Server socket connect accept listen bind socket write
117. Client Server socket connect accept listen bind socket read write
118. Client Server socket connect read accept listen bind socket write write
119. Client Server socket connect read accept listen bind socket write write Processes request
120. Client Server socket connect write write read accept listen bind socket
121. Client Server socket connect write write read accept listen bind socket read
122. Client Server socket connect write write read accept listen bind socket read
123. Client Server socket connect write write read accept listen bind socket read
124. Client Server socket connect write read close write read accept listen bind socket
125. Client Server socket connect write read close write read accept listen bind socket
126. Client Server socket connect write read close write read accept listen bind socket close
127. Client Server socket connect write read close close write read accept listen bind socket
128. Non-blocking IO & Connections
129. Reading
130. require ‘socket’ ! Socket.tcp_server(5000) do |connection| loop do ! end end
131. require ‘socket’ ! Socket.tcp_server(5000) do |connection| loop do data = connection.read_nonblock(1024) process(data) end end
132. require ‘socket’ ! Socket.tcp_server(5000) do |connection| loop do begin data = connection.read_nonblock(1024) process(data) rescue Error::EAGAIN IO.select([connection]) ; retry end end end
133. require ‘socket’ ! Socket.tcp_server(5000) do |connection| loop do begin data = connection.read_nonblock(1024) process(data) rescue Error::EAGAIN IO.select([connection]) ; retry end end end When there is no data to read
134. require ‘socket’ ! Socket.tcp_server(5000) do |connection| loop do begin data = connection.read_nonblock(1024) process(data) rescue Error::EAGAIN IO.select([connection]) ; retry end end end Wait until it is ready to be read
135. require ‘socket’ ! Socket.tcp_server(5000) do |connection| loop do begin data = connection.read_nonblock(1024) process(data) rescue Error::EAGAIN IO.select([connection]) ; retry rescue EOFError ; break end end connection.close end
136. require ‘socket’ ! Socket.tcp_server(5000) do |connection| loop do begin data = connection.read_nonblock(1024) process(data) rescue Error::EAGAIN IO.select([connection]) ; retry rescue EOFError ; break end end connection.close end When the connection is closed (EOF)
137. Writing
138. require ‘socket’ ! TCPSocket.new(‘0.0.0.0’, 5000)
139. require ‘socket’ ! TCPSocket.new(‘0.0.0.0’, 5000) message = ‘RubyConf Uruguay 2014’ * 10_000
140. require ‘socket’ ! TCPSocket.new(‘0.0.0.0’, 5000) message = ‘RubyConf Uruguay 2014’ * 10_000 ! bytes = connection.write_nonblock(message)
141. require ‘socket’ ! TCPSocket.new(‘0.0.0.0’, 5000) message = ‘RubyConf Uruguay 2014’ * 10_000 ! loop do bytes = connection.write_nonblock(message) break if bytes >= message.size message.slice!(0, bytes) IO.select([], [connection]) end
142. require ‘socket’ ! TCPSocket.new(‘0.0.0.0’, 5000) message = ‘RubyConf Uruguay 2014’ * 10_000 ! begin loop do bytes = connection.write_nonblock(message) break if bytes >= message.size message.slice!(0, bytes) IO.select([], [connection]) end rescue Error::EAGAIN IO.select([], [connection]) ; retry end
143. Connection
144. require ‘socket’ ! client = Socket.new(:INET, :STREAM) address = Socket.pack_sockaddr_in( 80,‘rubysur.org’)
145. require ‘socket’ ! client = Socket.new(:INET, :STREAM) address = Socket.pack_sockaddr_in( 80,‘rubysur.org’) ! client.connect_nonblock(address)
146. require ‘socket’ ! client = Socket.new(:INET, :STREAM) address = Socket.pack_sockaddr_in( 80,‘rubysur.org’) ! client.connect_nonblock(address) rescue Errno::EINPROGRESS
147. require ‘socket’ ! client = Socket.new(:INET, :STREAM) address = Socket.pack_sockaddr_in( 80,‘rubysur.org’) ! client.connect_nonblock(address) rescue Errno::EINPROGRESS Notifies and connects on background
148. require ‘socket’ ! client = Socket.new(:INET, :STREAM) address = Socket.pack_sockaddr_in( 80,‘rubysur.org’) ! client.connect_nonblock(address) rescue Errno::EINPROGRESS rescue Errno::EALREADY Non-blocking connection already in progress
149. Connection Multiplexing
150. Connection multiplexing refers to working with multiple active sockets
151. We can use select(2) to get the sockets that are ready for reading, writing, or have an exceptional condition pending
152. select(2) is blocking
153. sockets = [<TCPSocket>, <TCPSocket>, …] ! IO.select(sockets, sockets, sockets, 5)
154. sockets = [<TCPSocket>, <TCPSocket>, …] ! IO.select(sockets, sockets, sockets, 5) Sockets to wait until readable
155. sockets = [<TCPSocket>, <TCPSocket>, …] ! IO.select(sockets, sockets, sockets, 5) Sockets to wait until writable
156. sockets = [<TCPSocket>, <TCPSocket>, …] ! IO.select(sockets, sockets, sockets, 5) Sockets to wait until an exception occurs
157. sockets = [<TCPSocket>, <TCPSocket>, …] ! IO.select(sockets, sockets, sockets, 5) Timeout in seconds
158. sockets = [<TCPSocket>, <TCPSocket>, …] ! IO.select(sockets, sockets, sockets, 5) => [[<TCPSocket>, <TCPSocket>], [<TCPSocket>], [] ]
159. sockets = [<TCPSocket>, <TCPSocket>, …] ! IO.select(sockets, sockets, sockets, 5) => [[<TCPSocket>, <TCPSocket>], [<TCPSocket>], [] ] ! IO.select(sockets, sockets, sockets, 1) => nil
160. #isTCPDead
162. Why should I care about this stuff?
163. You will be able to do magic…
165. This concepts go beyond Ruby
166. Recommended Reading
169. Thank you @sebasoga