Tweet
Tweet

Slide 1

Slide 1 text

TCP Socket Network Programming in Ruby

Slide 2

Slide 2 text

@qhwa * husband * software engineer at Alibaba * creator of Gira

Slide 3

Slide 3 text

TCP? Why should I care about?

Slide 4

Slide 4 text

raw network programming is fun

Slide 5

Slide 5 text

unix system functions are powerful

Slide 6

Slide 6 text

for example

Slide 7

Slide 7 text

3VCZPO3BJMT 3VCZ &WFOU.BDIJOF

Slide 8

Slide 8 text

3VCZPO3BJMT 3VCZ &WFOU.BDIJOF

Slide 9

Slide 9 text

3VCZPO3BJMT 3VCZ &WFOU.BDIJOF

Slide 10

Slide 10 text

3VCZPO3BJMT 3VCZ &WFOU.BDIJOF

Slide 11

Slide 11 text

3VCZPO3BJMT 3VCZ &WFOU.BDIJOF

Slide 12

Slide 12 text

3VCZPO3BJMT 3VCZ &WFOU.BDIJOF

Slide 13

Slide 13 text

some crystal b.t.w.

Slide 14

Slide 14 text

Let’s play with TCP Socket!

Slide 15

Slide 15 text

what will be covered today?

Slide 16

Slide 16 text

* classical style * Interactive * Multiplexing * Event based * Make a Socks5 Proxy * Stateful Socks5 Proxy

Slide 17

Slide 17 text

some background knowledges before going on (incase you didn’t know)

Slide 18

Slide 18 text

Everything is file in UNIX read/write IO = read/ write through File Describer

Slide 19

Slide 19 text

No content

Slide 20

Slide 20 text

5$14PDLFU

Slide 21

Slide 21 text

5$14PDLFU #BTJD4PDLFU

Slide 22

Slide 22 text

5$14PDLFU #BTJD4PDLFU

Slide 23

Slide 23 text

5$14PDLFU #BTJD4PDLFU *0

Slide 24

Slide 24 text

5$14PDLFU #BTJD4PDLFU *0

Slide 25

Slide 25 text

Socket Domain

Slide 26

Slide 26 text

Socket Domain "'@-0$"-

Slide 27

Slide 27 text

Socket Domain "'@-0$"- "'@*/&5

Slide 28

Slide 28 text

Socket Domain "'@-0$"- "'@*/&5 "'@*/&5

Slide 29

Slide 29 text

Socket Domain "'@-0$"- "'@*/&5 "'@*/&5 ʜ

Slide 30

Slide 30 text

Socket Domain "'@-0$"- "'@*/&5 "'@*/&5 ʜ

Slide 31

Slide 31 text

Socket Domain "'@-0$"- "'@*/&5 "'@*/&5 6/*9 ʜ

Slide 32

Slide 32 text

Socket Domain "'@-0$"- "'@*/&5 "'@*/&5 6/*9 ʜ

Slide 33

Slide 33 text

Socket Domain "'@-0$"- "'@*/&5 "'@*/&5 6/*9 *1W ʜ

Slide 34

Slide 34 text

Socket Domain "'@-0$"- "'@*/&5 "'@*/&5 6/*9 *1W ʜ

Slide 35

Slide 35 text

Socket Domain "'@-0$"- "'@*/&5 "'@*/&5 6/*9 *1W *1W ʜ

Slide 36

Slide 36 text

Socket Type

Slide 37

Slide 37 text

Socket Type 40$,@3"8

Slide 38

Slide 38 text

Socket Type 40$,@3"8 40$,@%3".

Slide 39

Slide 39 text

Socket Type 40$,@3"8 40$,@%3". 40$,@453&".

Slide 40

Slide 40 text

Socket Type 40$,@3"8 40$,@%3". 40$,@453&". ʜ

Slide 41

Slide 41 text

Socket Type 40$,@3"8 40$,@%3". 40$,@453&". ʜ

Slide 42

Slide 42 text

Socket Type 40$,@3"8 40$,@%3". 40$,@453&". 5$1 ʜ

Slide 43

Slide 43 text

Socket Protocol

Slide 44

Slide 44 text

Socket Protocol *1@4$51

Slide 45

Slide 45 text

Socket Protocol *1@4$51 *1@6%1

Slide 46

Slide 46 text

Socket Protocol *1@4$51 *1@6%1 *1@5$1

Slide 47

Slide 47 text

Socket Protocol *1@4$51 *1@6%1 *1@5$1 ʜ

Slide 48

Slide 48 text

Socket Protocol *1@4$51 *1@6%1 *1@5$1 ʜ

Slide 49

Slide 49 text

Socket Protocol *1@4$51 *1@6%1 *1@5$1 5$1 ʜ

Slide 50

Slide 50 text

TCP Socket 40$,@453&". "'@*/&5"'@*/&5

Slide 51

Slide 51 text

Let’s code

Slide 52

Slide 52 text

1. classical style

Slide 53

Slide 53 text

No content

Slide 54

Slide 54 text

server

Slide 55

Slide 55 text

bind server

Slide 56

Slide 56 text

bind server

Slide 57

Slide 57 text

bind listen server

Slide 58

Slide 58 text

bind listen server

Slide 59

Slide 59 text

bind listen accept server

Slide 60

Slide 60 text

bind listen accept server client

Slide 61

Slide 61 text

bind listen accept connect server client

Slide 62

Slide 62 text

bind listen accept connect server client

Slide 63

Slide 63 text

bind listen accept read connect server client

Slide 64

Slide 64 text

bind listen accept read connect write server client

Slide 65

Slide 65 text

bind listen accept read connect write server client

Slide 66

Slide 66 text

bind listen accept read connect write server client

Slide 67

Slide 67 text

bind listen accept read write connect write server client

Slide 68

Slide 68 text

bind listen accept read write connect write server client

Slide 69

Slide 69 text

bind listen accept read write connect write read server client

Slide 70

Slide 70 text

bind listen accept read write connect write read close server client

Slide 71

Slide 71 text

bind listen accept read write connect write read close server client

Slide 72

Slide 72 text

bind listen accept read write close connect write read close server client

Slide 73

Slide 73 text

No content

Slide 74

Slide 74 text

require 'socket'

Slide 75

Slide 75 text

require 'socket'

Slide 76

Slide 76 text

require 'socket' class Server

Slide 77

Slide 77 text

require 'socket' class Server

Slide 78

Slide 78 text

require 'socket' class Server def start(bind: '0.0.0.0', port: nil)

Slide 79

Slide 79 text

require 'socket' class Server def start(bind: '0.0.0.0', port: nil) sock = Socket.new(:INET, :STREAM)

Slide 80

Slide 80 text

require 'socket' class Server def start(bind: '0.0.0.0', port: nil) sock = Socket.new(:INET, :STREAM)

Slide 81

Slide 81 text

require 'socket' class Server def start(bind: '0.0.0.0', port: nil) sock = Socket.new(:INET, :STREAM) sock.bind Addrinfo.tcp bind, port

Slide 82

Slide 82 text

require 'socket' class Server def start(bind: '0.0.0.0', port: nil) sock = Socket.new(:INET, :STREAM) sock.bind Addrinfo.tcp bind, port sock.listen 0

Slide 83

Slide 83 text

require 'socket' class Server def start(bind: '0.0.0.0', port: nil) sock = Socket.new(:INET, :STREAM) sock.bind Addrinfo.tcp bind, port sock.listen 0

Slide 84

Slide 84 text

require 'socket' class Server def start(bind: '0.0.0.0', port: nil) sock = Socket.new(:INET, :STREAM) sock.bind Addrinfo.tcp bind, port sock.listen 0 client, client_addr = sock.accept

Slide 85

Slide 85 text

require 'socket' class Server def start(bind: '0.0.0.0', port: nil) sock = Socket.new(:INET, :STREAM) sock.bind Addrinfo.tcp bind, port sock.listen 0 client, client_addr = sock.accept client.puts "Hello there! %s" % Time.now

Slide 86

Slide 86 text

require 'socket' class Server def start(bind: '0.0.0.0', port: nil) sock = Socket.new(:INET, :STREAM) sock.bind Addrinfo.tcp bind, port sock.listen 0 client, client_addr = sock.accept client.puts "Hello there! %s" % Time.now client.close

Slide 87

Slide 87 text

require 'socket' class Server def start(bind: '0.0.0.0', port: nil) sock = Socket.new(:INET, :STREAM) sock.bind Addrinfo.tcp bind, port sock.listen 0 client, client_addr = sock.accept client.puts "Hello there! %s" % Time.now client.close sock.close

Slide 88

Slide 88 text

require 'socket' class Server def start(bind: '0.0.0.0', port: nil) sock = Socket.new(:INET, :STREAM) sock.bind Addrinfo.tcp bind, port sock.listen 0 client, client_addr = sock.accept client.puts "Hello there! %s" % Time.now client.close sock.close end

Slide 89

Slide 89 text

require 'socket' class Server def start(bind: '0.0.0.0', port: nil) sock = Socket.new(:INET, :STREAM) sock.bind Addrinfo.tcp bind, port sock.listen 0 client, client_addr = sock.accept client.puts "Hello there! %s" % Time.now client.close sock.close end

Slide 90

Slide 90 text

require 'socket' class Server def start(bind: '0.0.0.0', port: nil) sock = Socket.new(:INET, :STREAM) sock.bind Addrinfo.tcp bind, port sock.listen 0 client, client_addr = sock.accept client.puts "Hello there! %s" % Time.now client.close sock.close end end

Slide 91

Slide 91 text

require 'socket' class Server def start(bind: '0.0.0.0', port: nil) sock = Socket.new(:INET, :STREAM) sock.bind Addrinfo.tcp bind, port sock.listen 0 client, client_addr = sock.accept client.puts "Hello there! %s" % Time.now client.close sock.close end end

Slide 92

Slide 92 text

require 'socket' class Server def start(bind: '0.0.0.0', port: nil) sock = Socket.new(:INET, :STREAM) sock.bind Addrinfo.tcp bind, port sock.listen 0 client, client_addr = sock.accept client.puts "Hello there! %s" % Time.now client.close sock.close end end Server.new.start(port: 23333, bind: '127.0.0.1')

Slide 93

Slide 93 text

require 'socket' class Server def start(bind: '0.0.0.0', port: nil) sock = Socket.new(:INET, :STREAM) sock.bind Addrinfo.tcp bind, port sock.listen 0 client, client_addr = sock.accept client.puts "Hello there! %s" % Time.now client.close sock.close end end Server.new.start(port: 23333, bind: '127.0.0.1')

Slide 94

Slide 94 text

ruby’s sugar

Slide 95

Slide 95 text

No content

Slide 96

Slide 96 text

require 'socket'

Slide 97

Slide 97 text

require 'socket' class Server

Slide 98

Slide 98 text

require 'socket' class Server

Slide 99

Slide 99 text

require 'socket' class Server def start(bind: '0.0.0.0', port: nil)

Slide 100

Slide 100 text

require 'socket' class Server def start(bind: '0.0.0.0', port: nil) Socket.tcp_server_loop(bind, port) do |sock, remote|

Slide 101

Slide 101 text

require 'socket' class Server def start(bind: '0.0.0.0', port: nil) Socket.tcp_server_loop(bind, port) do |sock, remote| sock.puts "Hello there! %s" % Time.now

Slide 102

Slide 102 text

require 'socket' class Server def start(bind: '0.0.0.0', port: nil) Socket.tcp_server_loop(bind, port) do |sock, remote| sock.puts "Hello there! %s" % Time.now sock.close

Slide 103

Slide 103 text

require 'socket' class Server def start(bind: '0.0.0.0', port: nil) Socket.tcp_server_loop(bind, port) do |sock, remote| sock.puts "Hello there! %s" % Time.now sock.close end

Slide 104

Slide 104 text

require 'socket' class Server def start(bind: '0.0.0.0', port: nil) Socket.tcp_server_loop(bind, port) do |sock, remote| sock.puts "Hello there! %s" % Time.now sock.close end end

Slide 105

Slide 105 text

require 'socket' class Server def start(bind: '0.0.0.0', port: nil) Socket.tcp_server_loop(bind, port) do |sock, remote| sock.puts "Hello there! %s" % Time.now sock.close end end

Slide 106

Slide 106 text

require 'socket' class Server def start(bind: '0.0.0.0', port: nil) Socket.tcp_server_loop(bind, port) do |sock, remote| sock.puts "Hello there! %s" % Time.now sock.close end end end

Slide 107

Slide 107 text

require 'socket' class Server def start(bind: '0.0.0.0', port: nil) Socket.tcp_server_loop(bind, port) do |sock, remote| sock.puts "Hello there! %s" % Time.now sock.close end end end

Slide 108

Slide 108 text

require 'socket' class Server def start(bind: '0.0.0.0', port: nil) Socket.tcp_server_loop(bind, port) do |sock, remote| sock.puts "Hello there! %s" % Time.now sock.close end end end Server.new.start(port: 23333, bind: '127.0.0.1')

Slide 109

Slide 109 text

require 'socket' class Server def start(bind: '0.0.0.0', port: nil) Socket.tcp_server_loop(bind, port) do |sock, remote| sock.puts "Hello there! %s" % Time.now sock.close end end end Server.new.start(port: 23333, bind: '127.0.0.1')

Slide 110

Slide 110 text

2. interactive

Slide 111

Slide 111 text

... def start(bind: '0.0.0.0', port: nil)

Slide 112

Slide 112 text

... def start(bind: '0.0.0.0', port: nil) Socket.tcp_server_loop(bind, port) do |sock, client_addr|

Slide 113

Slide 113 text

... def start(bind: '0.0.0.0', port: nil) Socket.tcp_server_loop(bind, port) do |sock, client_addr| log { "client connected: %s:%d" % [client_addr.ip_address, client_addr.ip_port] }

Slide 114

Slide 114 text

... def start(bind: '0.0.0.0', port: nil) Socket.tcp_server_loop(bind, port) do |sock, client_addr| log { "client connected: %s:%d" % [client_addr.ip_address, client_addr.ip_port] }

Slide 115

Slide 115 text

... def start(bind: '0.0.0.0', port: nil) Socket.tcp_server_loop(bind, port) do |sock, client_addr| log { "client connected: %s:%d" % [client_addr.ip_address, client_addr.ip_port] } sock.puts "Welcome buddy!"

Slide 116

Slide 116 text

... def start(bind: '0.0.0.0', port: nil) Socket.tcp_server_loop(bind, port) do |sock, client_addr| log { "client connected: %s:%d" % [client_addr.ip_address, client_addr.ip_port] } sock.puts "Welcome buddy!"

Slide 117

Slide 117 text

... def start(bind: '0.0.0.0', port: nil) Socket.tcp_server_loop(bind, port) do |sock, client_addr| log { "client connected: %s:%d" % [client_addr.ip_address, client_addr.ip_port] } sock.puts "Welcome buddy!" loop do

Slide 118

Slide 118 text

... def start(bind: '0.0.0.0', port: nil) Socket.tcp_server_loop(bind, port) do |sock, client_addr| log { "client connected: %s:%d" % [client_addr.ip_address, client_addr.ip_port] } sock.puts "Welcome buddy!" loop do input = sock.gets.chomp

Slide 119

Slide 119 text

... def start(bind: '0.0.0.0', port: nil) Socket.tcp_server_loop(bind, port) do |sock, client_addr| log { "client connected: %s:%d" % [client_addr.ip_address, client_addr.ip_port] } sock.puts "Welcome buddy!" loop do input = sock.gets.chomp if input == "q"

Slide 120

Slide 120 text

... def start(bind: '0.0.0.0', port: nil) Socket.tcp_server_loop(bind, port) do |sock, client_addr| log { "client connected: %s:%d" % [client_addr.ip_address, client_addr.ip_port] } sock.puts "Welcome buddy!" loop do input = sock.gets.chomp if input == "q" sock.close

Slide 121

Slide 121 text

... def start(bind: '0.0.0.0', port: nil) Socket.tcp_server_loop(bind, port) do |sock, client_addr| log { "client connected: %s:%d" % [client_addr.ip_address, client_addr.ip_port] } sock.puts "Welcome buddy!" loop do input = sock.gets.chomp if input == "q" sock.close break

Slide 122

Slide 122 text

... def start(bind: '0.0.0.0', port: nil) Socket.tcp_server_loop(bind, port) do |sock, client_addr| log { "client connected: %s:%d" % [client_addr.ip_address, client_addr.ip_port] } sock.puts "Welcome buddy!" loop do input = sock.gets.chomp if input == "q" sock.close break end

Slide 123

Slide 123 text

... def start(bind: '0.0.0.0', port: nil) Socket.tcp_server_loop(bind, port) do |sock, client_addr| log { "client connected: %s:%d" % [client_addr.ip_address, client_addr.ip_port] } sock.puts "Welcome buddy!" loop do input = sock.gets.chomp if input == "q" sock.close break end sock.puts input.reverse

Slide 124

Slide 124 text

3. multiplexing

Slide 125

Slide 125 text

def start(bind: '0.0.0.0', port: nil) Socket.tcp_server_loop(bind, port) do |sock, client_addr| sock.puts "Welcome buddy!" loop do input = sock.gets.chomp if input == "q" sock.close else sock.puts input.reverse end end end end

Slide 126

Slide 126 text

def start(bind: '0.0.0.0', port: nil) Socket.tcp_server_loop(bind, port) do |sock, client_addr| sock.puts "Welcome buddy!" loop do input = sock.gets.chomp if input == "q" sock.close else sock.puts input.reverse end end end end Thread.new do end

Slide 127

Slide 127 text

def start(bind: '0.0.0.0', port: nil) Socket.tcp_server_loop(bind, port) do |sock, client_addr| sock.puts "Welcome buddy!" loop do input = sock.gets.chomp if input == "q" sock.close else sock.puts input.reverse end end end end Thread.new do end

Slide 128

Slide 128 text

def start(bind: '0.0.0.0', port: nil) Socket.tcp_server_loop(bind, port) do |sock, client_addr| log { "client connected: %s:%d" % [client_addr.ip_address, client_addr.ip_port] } sock.puts "Welcome buddy!" loop do input = sock.gets.chomp if input == 'q' sock.close break else sock.puts input.reverse end end end end

Slide 129

Slide 129 text

def start(bind: '0.0.0.0', port: nil) Socket.tcp_server_loop(bind, port) do |sock, client_addr| log { "client connected: %s:%d" % [client_addr.ip_address, client_addr.ip_port] } sock.puts "Welcome buddy!" loop do input = sock.gets.chomp if input == 'q' sock.close break else sock.puts input.reverse end end end end if fork else sock.close end

Slide 130

Slide 130 text

def start(bind: '0.0.0.0', port: nil) Socket.tcp_server_loop(bind, port) do |sock, client_addr| log { "client connected: %s:%d" % [client_addr.ip_address, client_addr.ip_port] } sock.puts "Welcome buddy!" loop do input = sock.gets.chomp if input == 'q' sock.close break else sock.puts input.reverse end end end end if fork else sock.close end

Slide 131

Slide 131 text

def start(bind: '0.0.0.0', port: nil) Socket.tcp_server_loop(bind, port) do |sock, client_addr| log { "client connected: %s:%d" % [client_addr.ip_address, client_addr.ip_port] } sock.puts "Welcome buddy!" loop do input = sock.gets.chomp if input == 'q' sock.close break else sock.puts input.reverse end end end end if fork else sock.close end

Slide 132

Slide 132 text

No content

Slide 133

Slide 133 text

socket

Slide 134

Slide 134 text

socket client

Slide 135

Slide 135 text

socket client

Slide 136

Slide 136 text

socket client

Slide 137

Slide 137 text

socket socket client

Slide 138

Slide 138 text

socket socket client

Slide 139

Slide 139 text

socket socket client

Slide 140

Slide 140 text

3. EventMachine

Slide 141

Slide 141 text

No content

Slide 142

Slide 142 text

require 'eventmachine'

Slide 143

Slide 143 text

require 'eventmachine'

Slide 144

Slide 144 text

require 'eventmachine' class Server

Slide 145

Slide 145 text

require 'eventmachine' class Server def start(bind: '0.0.0.0', port: nil)

Slide 146

Slide 146 text

require 'eventmachine' class Server def start(bind: '0.0.0.0', port: nil) EM.run { EM.start_server bind, port, Handler }

Slide 147

Slide 147 text

No content

Slide 148

Slide 148 text

module Handler

Slide 149

Slide 149 text

module Handler

Slide 150

Slide 150 text

module Handler def receive_data data

Slide 151

Slide 151 text

module Handler def receive_data data data.chomp!

Slide 152

Slide 152 text

module Handler def receive_data data data.chomp! if data == "q"

Slide 153

Slide 153 text

module Handler def receive_data data data.chomp! if data == "q" close_connection

Slide 154

Slide 154 text

module Handler def receive_data data data.chomp! if data == "q" close_connection else

Slide 155

Slide 155 text

module Handler def receive_data data data.chomp! if data == "q" close_connection else send_data data.reverse

Slide 156

Slide 156 text

module Handler def receive_data data data.chomp! if data == "q" close_connection else send_data data.reverse send_data "\n"

Slide 157

Slide 157 text

4. EM SOCKS5 proxy server IUUQTHJUIVCDPNRIXBSVCZOFUXPSL QSPHSBNNJOHCMPCNBTUFSTFSWFS @FWFOUNBDIJOF@TPDLT@TFSWFSSC

Slide 158

Slide 158 text

5. EM SOCKS5 version 2 IUUQTHJUIVCDPNRIXBSVCZOFUXPSL QSPHSBNNJOHCMPCNBTUFSTFSWFS @TUSPOHFS@FWFOUNBDIJOF@TPDLT@TFSW FSSC

Slide 159

Slide 159 text

6. stateful SOCK5 proxy server

Slide 160

Slide 160 text

module StatefulHandler include Socks5Handler def connect ip, _ = client_addr # to allow: redis-cli set allowed:127.0.0.1 1 # to disallow: redis-cli del allowed:127.0.0.1 EM.defer -> { Redis.new.get "allowed:#{ip}" }, -> (allowed) { allowed ? super : close_connection } end end

Slide 161

Slide 161 text

TCP Programming models

Slide 162

Slide 162 text

block IO start end non-block IO start end check check check check check signal driven IO notify end async IO start notify XBJUGPS EBUB DPQZEBUB GSPN LFSOFMUP VTFSTQBDF

Slide 163

Slide 163 text

conclusions

Slide 164

Slide 164 text

TCP Socket is easy, right?

Slide 165

Slide 165 text

Thank you!