Systems Programming - SoCoded 2013

Transcript

1. Systems Programming
2. operate and control the computer hardware and to provide a platform for running application software http://en.wikipedia.org/wiki/System_software “
3. application software “
4. web software “
5. Building Blocks The Kernel System Calls Sockets File Descriptors HTTP
7. The Kernel
8. Your Code The Kernel Hardware
9. Your Code
10. Your Code User Mode
11. Your Code Math User Mode
12. Your Code Math Access Memory User Mode
13. The Kernel
14. The Kernel Anything
15. The Kernel Anything Everything
16. System Calls
17. User Mode Kernel Mode
18. User Mode Kernel Mode Your Program System Calls
19. $ strace -p <server pid>
21. 120,602 Calls One Request
22. linux$ wc -l syscall_table.S 326
23. mov! eax, 5 mov! ebx, path mov! ecx, flags int! 80h open(path, flags)
24. linux$ cat syscall_table.S | grep -c sys_ni_syscal 65 Not Implemented
25. File Descriptors
26. Everything is a File
27. Things that are Files (Non Exhaustive)
28. Things that are Files (Non Exhaustive) Files
29. Things that are Files (Non Exhaustive) Links DVDs Hard Drives USB Keys Sockets* Files Directories Sound Cards Graphics Cards Printers Pipes Terminals UNIX Sockets Shared Memory
30. read() write() close() lseek()* File API
31. 0 -> /dev/null 1 -> unicorn.log 2 -> unicorn.log 3 -> pipe:[3753224456] 4 -> pipe:[3753224456] 5 -> socket:[19730] 6 -> pipe:[3753224457] 7 -> socket:[19731] 8 -> socket:[19732] 9 -> pipe:[3753224457] 10 -> production.log 11 -> socket:[3766066290] 12 -> socket:[3753228795] 13 -> socket:[3753229266] 14 -> socket:[3753229276] 15 -> socket:[3753229285] 16 -> socket:[3753229307] 17 -> socket:[3753235718] 18 -> socket:[3753235910] 19 -> socket:[3753242775] 20 -> socket:[3753242776] 21 -> socket:[3753242778] 22 -> socket:[3753239097] 23 -> socket:[3753242942] 24 -> socket:[3753242944] 25 -> socket:[3753239202] 26 -> socket:[3753239203] 27 -> socket:[3753239204] 28 -> socket:[3753242981] 29 -> socket:[3753242982] 30 -> socket:[3753243929] 31 -> socket:[3753240046] 32 -> socket:[3753245881] 33 -> socket:[3753248970] 34 -> socket:[3753253476] 35 -> pygments-ruby.log 36 -> /dev/null 37 -> pipe:[3753260503] 38 -> pipe:[3753260504] 39 -> socket:[3753344667] 40 -> pipe:[3753260505] 41 -> socket:[3765810413] 42 -> socket:[3765810440] 43 -> socket:[3765848228] $ ls -l /proc/32660/fd
32. BSD Sockets
33. 1980 1970 1960 1990 Multics Unix (Unics) BSD BSD 4.2 TCP/IP
34. socket() bind() listen() accept() read() and write() close()
35. HTTP
36. HTTP Request GET /index.html HTTP/1.1 User-Agent: curl/7.24.0 Host: www.google.com Accept: */*
37. HTTP Response HTTP/1.1 200 OK Date: Tue, 05 Mar 2013 23:03:52 GMT Cache-Control: private, max-age=0 Content-Type: text/html Server: gws Transfer-Encoding: chunked <data>
38. There Will be Code
39. require 'socket' NOT_FOUND = "HTTP/1.1 404 Not Found\nContent-Length: 9\n\nNot Found" OK = "HTTP/1.1 200 OK\n" socket = Socket.new(:INET, :STREAM) sockaddr = Socket.sockaddr_in(11080, '127.0.0.1') socket.bind(sockaddr) socket.listen(5) while client_socket = socket.accept[0] path = client_socket.readline.split[1] filename = File.expand_path("../#{path}", __FILE__) if File.file?(filename) contents = File.read(filename) client_socket.write(OK) client_socket.write("Content-Length: #{contents.size}\n\n") client_socket.write("#{contents}") else client_socket.write(NOT_FOUND) end client_socket.close end
40. require 'socket' NOT_FOUND = "HTTP/1.1 404 Not Found\nContent-Length: 9\n\nNot Found" OK = "HTTP/1.1 200 OK\n" socket = Socket.new(:INET, :STREAM) sockaddr = Socket.sockaddr_in(11080, '127.0.0.1') socket.bind(sockaddr) socket.listen(5) while client_socket = socket.accept[0] path = client_socket.readline.split[1] filename = File.expand_path("../#{path}", __FILE__) if File.file?(filename) contents = File.read(filename) client_socket.write(OK) client_socket.write("Content-Length: #{contents.size}\n\n") client_socket.write("#{contents}") else client_socket.write(NOT_FOUND) end client_socket.close end Require the socket lib
41. require 'socket' NOT_FOUND = "HTTP/1.1 404 Not Found\nContent-Length: 9\n\nNot Found" OK = "HTTP/1.1 200 OK\n" socket = Socket.new(:INET, :STREAM) sockaddr = Socket.sockaddr_in(11080, '127.0.0.1') socket.bind(sockaddr) socket.listen(5) while client_socket = socket.accept[0] path = client_socket.readline.split[1] filename = File.expand_path("../#{path}", __FILE__) if File.file?(filename) contents = File.read(filename) client_socket.write(OK) client_socket.write("Content-Length: #{contents.size}\n\n") client_socket.write("#{contents}") else client_socket.write(NOT_FOUND) end client_socket.close end Some constants for later
42. require 'socket' NOT_FOUND = "HTTP/1.1 404 Not Found\nContent-Length: 9\n\nNot Found" OK = "HTTP/1.1 200 OK\n" socket = Socket.new(:INET, :STREAM) sockaddr = Socket.sockaddr_in(11080, '127.0.0.1') socket.bind(sockaddr) socket.listen(5) while client_socket = socket.accept[0] path = client_socket.readline.split[1] filename = File.expand_path("../#{path}", __FILE__) if File.file?(filename) contents = File.read(filename) client_socket.write(OK) client_socket.write("Content-Length: #{contents.size}\n\n") client_socket.write("#{contents}") else client_socket.write(NOT_FOUND) end client_socket.close end socket = Socket.new(:INET, :STREAM)
43. require 'socket' NOT_FOUND = "HTTP/1.1 404 Not Found\nContent-Length: 9\n\nNot Found" OK = "HTTP/1.1 200 OK\n" socket = Socket.new(:INET, :STREAM) sockaddr = Socket.sockaddr_in(11080, '127.0.0.1') socket.bind(sockaddr) socket.listen(5) while client_socket = socket.accept[0] path = client_socket.readline.split[1] filename = File.expand_path("../#{path}", __FILE__) if File.file?(filename) contents = File.read(filename) client_socket.write(OK) client_socket.write("Content-Length: #{contents.size}\n\n") client_socket.write("#{contents}") else client_socket.write(NOT_FOUND) end client_socket.close end sockaddr = Socket.sockaddr_in(11080, '127.0.0.1') socket.bind(sockaddr)
44. require 'socket' NOT_FOUND = "HTTP/1.1 404 Not Found\nContent-Length: 9\n\nNot Found" OK = "HTTP/1.1 200 OK\n" socket = Socket.new(:INET, :STREAM) sockaddr = Socket.sockaddr_in(11080, '127.0.0.1') socket.bind(sockaddr) socket.listen(5) while client_socket = socket.accept[0] path = client_socket.readline.split[1] filename = File.expand_path("../#{path}", __FILE__) if File.file?(filename) contents = File.read(filename) client_socket.write(OK) client_socket.write("Content-Length: #{contents.size}\n\n") client_socket.write("#{contents}") else client_socket.write(NOT_FOUND) end client_socket.close end socket.listen(5)
45. require 'socket' NOT_FOUND = "HTTP/1.1 404 Not Found\nContent-Length: 9\n\nNot Found" OK = "HTTP/1.1 200 OK\n" socket = Socket.new(:INET, :STREAM) sockaddr = Socket.sockaddr_in(11080, '127.0.0.1') socket.bind(sockaddr) socket.listen(5) while client_socket = socket.accept[0] path = client_socket.readline.split[1] filename = File.expand_path("../#{path}", __FILE__) if File.file?(filename) contents = File.read(filename) client_socket.write(OK) client_socket.write("Content-Length: #{contents.size}\n\n") client_socket.write("#{contents}") else client_socket.write(NOT_FOUND) end client_socket.close end while client_socket = socket.accept[0]
46. require 'socket' NOT_FOUND = "HTTP/1.1 404 Not Found\nContent-Length: 9\n\nNot Found" OK = "HTTP/1.1 200 OK\n" socket = Socket.new(:INET, :STREAM) sockaddr = Socket.sockaddr_in(11080, '127.0.0.1') socket.bind(sockaddr) socket.listen(5) while client_socket = socket.accept[0] path = client_socket.readline.split[1] filename = File.expand_path("../#{path}", __FILE__) if File.file?(filename) contents = File.read(filename) client_socket.write(OK) client_socket.write("Content-Length: #{contents.size}\n\n") client_socket.write("#{contents}") else client_socket.write(NOT_FOUND) end client_socket.close end path = client_socket.readline.split[1]
47. require 'socket' NOT_FOUND = "HTTP/1.1 404 Not Found\nContent-Length: 9\n\nNot Found" OK = "HTTP/1.1 200 OK\n" socket = Socket.new(:INET, :STREAM) sockaddr = Socket.sockaddr_in(11080, '127.0.0.1') socket.bind(sockaddr) socket.listen(5) while client_socket = socket.accept[0] path = client_socket.readline.split[1] filename = File.expand_path("../#{path}", __FILE__) if File.file?(filename) contents = File.read(filename) client_socket.write(OK) client_socket.write("Content-Length: #{contents.size}\n\n") client_socket.write("#{contents}") else client_socket.write(NOT_FOUND) end client_socket.close end path = client_socket.readline.split[1] GET /index.html HTTP/1.1
48. require 'socket' NOT_FOUND = "HTTP/1.1 404 Not Found\nContent-Length: 9\n\nNot Found" OK = "HTTP/1.1 200 OK\n" socket = Socket.new(:INET, :STREAM) sockaddr = Socket.sockaddr_in(11080, '127.0.0.1') socket.bind(sockaddr) socket.listen(5) while client_socket = socket.accept[0] path = client_socket.readline.split[1] filename = File.expand_path("../#{path}", __FILE__) if File.file?(filename) contents = File.read(filename) client_socket.write(OK) client_socket.write("Content-Length: #{contents.size}\n\n") client_socket.write("#{contents}") else client_socket.write(NOT_FOUND) end client_socket.close end filename = File.expand_path("../#{path}", __FILE__)
49. require 'socket' NOT_FOUND = "HTTP/1.1 404 Not Found\nContent-Length: 9\n\nNot Found" OK = "HTTP/1.1 200 OK\n" socket = Socket.new(:INET, :STREAM) sockaddr = Socket.sockaddr_in(11080, '127.0.0.1') socket.bind(sockaddr) socket.listen(5) while client_socket = socket.accept[0] path = client_socket.readline.split[1] filename = File.expand_path("../#{path}", __FILE__) if File.file?(filename) contents = File.read(filename) client_socket.write(OK) client_socket.write("Content-Length: #{contents.size}\n\n") client_socket.write("#{contents}") else client_socket.write(NOT_FOUND) end client_socket.close end if File.file?(filename)
50. require 'socket' NOT_FOUND = "HTTP/1.1 404 Not Found\nContent-Length: 9\n\nNot Found" OK = "HTTP/1.1 200 OK\n" socket = Socket.new(:INET, :STREAM) sockaddr = Socket.sockaddr_in(11080, '127.0.0.1') socket.bind(sockaddr) socket.listen(5) while client_socket = socket.accept[0] path = client_socket.readline.split[1] filename = File.expand_path("../#{path}", __FILE__) if File.file?(filename) contents = File.read(filename) client_socket.write(OK) client_socket.write("Content-Length: #{contents.size}\n\n") client_socket.write("#{contents}") else client_socket.write(NOT_FOUND) end client_socket.close end contents = File.read(filename) client_socket.write(OK) client_socket.write("Content-Length: #{contents.size}\n\n") client_socket.write("#{contents}")
51. require 'socket' NOT_FOUND = "HTTP/1.1 404 Not Found\nContent-Length: 9\n\nNot Found" OK = "HTTP/1.1 200 OK\n" socket = Socket.new(:INET, :STREAM) sockaddr = Socket.sockaddr_in(11080, '127.0.0.1') socket.bind(sockaddr) socket.listen(5) while client_socket = socket.accept[0] path = client_socket.readline.split[1] filename = File.expand_path("../#{path}", __FILE__) if File.file?(filename) contents = File.read(filename) client_socket.write(OK) client_socket.write("Content-Length: #{contents.size}\n\n") client_socket.write("#{contents}") else client_socket.write(NOT_FOUND) end client_socket.close end contents = File.read(filename) client_socket.write(OK) client_socket.write("Content-Length: #{contents.size}\n\n") client_socket.write("#{contents}") OK = "HTTP/1.1 200 OK\n"
52. require 'socket' NOT_FOUND = "HTTP/1.1 404 Not Found\nContent-Length: 9\n\nNot Found" OK = "HTTP/1.1 200 OK\n" socket = Socket.new(:INET, :STREAM) sockaddr = Socket.sockaddr_in(11080, '127.0.0.1') socket.bind(sockaddr) socket.listen(5) while client_socket = socket.accept[0] path = client_socket.readline.split[1] filename = File.expand_path("../#{path}", __FILE__) if File.file?(filename) contents = File.read(filename) client_socket.write(OK) client_socket.write("Content-Length: #{contents.size}\n\n") client_socket.write("#{contents}") else client_socket.write(NOT_FOUND) end client_socket.close end client_socket.write(NOT_FOUND)
53. require 'socket' NOT_FOUND = "HTTP/1.1 404 Not Found\nContent-Length: 9\n\nNot Found" OK = "HTTP/1.1 200 OK\n" socket = Socket.new(:INET, :STREAM) sockaddr = Socket.sockaddr_in(11080, '127.0.0.1') socket.bind(sockaddr) socket.listen(5) while client_socket = socket.accept[0] path = client_socket.readline.split[1] filename = File.expand_path("../#{path}", __FILE__) if File.file?(filename) contents = File.read(filename) client_socket.write(OK) client_socket.write("Content-Length: #{contents.size}\n\n") client_socket.write("#{contents}") else client_socket.write(NOT_FOUND) end client_socket.close end client_socket.write(NOT_FOUND) NOT_FOUND = "HTTP/1.1 404 Not Found\nContent-Length: 9\n\nNot Found"
54. require 'socket' NOT_FOUND = "HTTP/1.1 404 Not Found\nContent-Length: 9\n\nNot Found" OK = "HTTP/1.1 200 OK\n" socket = Socket.new(:INET, :STREAM) sockaddr = Socket.sockaddr_in(11080, '127.0.0.1') socket.bind(sockaddr) socket.listen(5) while client_socket = socket.accept[0] path = client_socket.readline.split[1] filename = File.expand_path("../#{path}", __FILE__) if File.file?(filename) contents = File.read(filename) client_socket.write(OK) client_socket.write("Content-Length: #{contents.size}\n\n") client_socket.write("#{contents}") else client_socket.write(NOT_FOUND) end client_socket.close end client_socket.close
55. require 'socket' NOT_FOUND = "HTTP/1.1 404 Not Found\nContent-Length: 9\n\nNot Found" OK = "HTTP/1.1 200 OK\n" socket = Socket.new(:INET, :STREAM) sockaddr = Socket.sockaddr_in(11080, '127.0.0.1') socket.bind(sockaddr) socket.listen(5) while client_socket = socket.accept[0] path = client_socket.readline.split[1] filename = File.expand_path("../#{path}", __FILE__) if File.file?(filename) contents = File.read(filename) client_socket.write(OK) client_socket.write("Content-Length: #{contents.size}\n\n") client_socket.write("#{contents}") else client_socket.write(NOT_FOUND) end client_socket.close end
56. def new_ipv6_server(addr, port, opt) opt.key?(:ipv6only) or return Kgio::TCPServer.new(addr, port) defined?(IPV6_V6ONLY) or abort "Socket::IPV6_V6ONLY not defined, upgrade Ruby and/or your OS" sock = Socket.new(AF_INET6, SOCK_STREAM, 0) sock.setsockopt(IPPROTO_IPV6, IPV6_V6ONLY, opt[:ipv6only] ? 1 : 0) sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1) sock.bind(Socket.pack_sockaddr_in(port, addr)) IO_PURGATORY << sock Kgio::TCPServer.for_fd(sock.fileno) end ret = IO.select(l, nil, SELF_PIPE, @timeout) and ready = ret[0] Unicorn
57. def new_ipv6_server(addr, port, opt) opt.key?(:ipv6only) or return Kgio::TCPServer.new(addr, port) defined?(IPV6_V6ONLY) or abort "Socket::IPV6_V6ONLY not defined, upgrade Ruby and/or your OS" sock = Socket.new(AF_INET6, SOCK_STREAM, 0) sock.setsockopt(IPPROTO_IPV6, IPV6_V6ONLY, opt[:ipv6only] ? 1 : 0) sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1) sock.bind(Socket.pack_sockaddr_in(port, addr)) IO_PURGATORY << sock Kgio::TCPServer.for_fd(sock.fileno) end ret = IO.select(l, nil, SELF_PIPE, @timeout) and ready = ret[0] Unicorn sock = Socket.new(AF_INET6, SOCK_STREAM, 0)
58. def new_ipv6_server(addr, port, opt) opt.key?(:ipv6only) or return Kgio::TCPServer.new(addr, port) defined?(IPV6_V6ONLY) or abort "Socket::IPV6_V6ONLY not defined, upgrade Ruby and/or your OS" sock = Socket.new(AF_INET6, SOCK_STREAM, 0) sock.setsockopt(IPPROTO_IPV6, IPV6_V6ONLY, opt[:ipv6only] ? 1 : 0) sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1) sock.bind(Socket.pack_sockaddr_in(port, addr)) IO_PURGATORY << sock Kgio::TCPServer.for_fd(sock.fileno) end ret = IO.select(l, nil, SELF_PIPE, @timeout) and ready = ret[0] Unicorn sock.bind( Socket.pack_sockaddr_in(port, addr))
59. def new_ipv6_server(addr, port, opt) opt.key?(:ipv6only) or return Kgio::TCPServer.new(addr, port) defined?(IPV6_V6ONLY) or abort "Socket::IPV6_V6ONLY not defined, upgrade Ruby and/or your OS" sock = Socket.new(AF_INET6, SOCK_STREAM, 0) sock.setsockopt(IPPROTO_IPV6, IPV6_V6ONLY, opt[:ipv6only] ? 1 : 0) sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1) sock.bind(Socket.pack_sockaddr_in(port, addr)) IO_PURGATORY << sock Kgio::TCPServer.for_fd(sock.fileno) end ret = IO.select(l, nil, SELF_PIPE, @timeout) and ready = ret[0] Unicorn IO.select(l, nil, SELF_PIPE, @timeout)
60. The Kernel System Calls Sockets File Descriptors HTTP
61. Further Reading Linux System Programming Robert Love
62. Further Reading Unix Network Programming Richard Stevens
63. Further Reading Advanced Programming in the UNIX Environment Richard Stevens
64. Andy Delcambre adelcambre @
66. Thanks! adelcambre @