Python for web architectures

Transcript

1. PythonͰWebΞʔΩςΫνϟΛֶͿ ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 1

2. ࣗݾ঺հ — খ୔पฏ — @oza_shu — MSPۀքͰWEBαʔϏεͷαʔόӡ༻ ٢঵ࣉ.pm13 2018/02/09 -

   Ozawa Shuhei ( @oza_shu ) 2

3. ٢঵ࣉ.pm13ͷςʔϚ — ʮ৽͍͠௅ઓɺ৽͍͠ࢹ఺ʯ — ৽೥౓ʹ޲͚ܾͨҙత ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei

   ( @oza_shu ) 3

4. 2018೥ͷ๊ෛ ͪΌΜͱௐ΂ͯͪΌΜͱཧղ͢Δ ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu )

   4

5. ͪΌΜͱௐ΂ͯɺͪΌΜͱཧղ͢Δ αʔόͷӡ༻Ͱো֐ରԠ͢Δ࣌ɺͳΜͱͳ͘ͰରԠ͢Δ͜ͱ΋ — ͜ͷϩάग़ྗͰݕࡧͯ͠ɺQiitaΈͨΓͱ͔... — ͦͷQiitaͱ͔ಡΜͰ΋;ΜΘΓཧղͩͬͨΓͱ͔... ٢঵ࣉ.pm13 2018/02/09 - Ozawa

   Shuhei ( @oza_shu ) 5

6. ࿦ཧతղܾΛ໨ࢦ͢ — ͪΌΜͱௐ΂ͯͪΌΜͱཧղ͢Δ — ՝୊Λղܾ͢Δ ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei

   ( @oza_shu ) 6

7. ྫ͑͹ ͜ΕΒWebΞʔΩςΫνϟΛPythonͰ͓͞Β͍ - TCPίωΫγϣϯ͸ͲΜͳ௨৴Λ͍ͯ͠Δͷ͔ - WEBαʔόͷಈ͖ɺߏ଄ཧղ - Apache͕٧·ΔͬͯͲ͏͍͏͜ͱ - C10K໰୊ͱ͸

   ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 7

8. TCP/IP௨৴ — IPΞυϨεͱϙʔτΛࢦఆͯ͠ίωΫγϣϯ઀ଓΛ͢Δ — TCP joke΍SYN→SYN/ACK→ACK — socket(),bind(),listen(),accept(),connect(),write(),read(),close() ٢঵ࣉ.pm13 2018/02/09

   - Ozawa Shuhei ( @oza_shu ) 8

9. γϦΞϧϞσϧ — socket(),bind(),listen()Ͱιέοτͷ࡞੒ def create_listen_socket(host, port): """ αʔόʔ͕઀ଓཁٻΛड͚औΔιέοτΛઃఆ͢Δ """ #

   ΞυϨεϑΝϛϦʔɺιέοτλΠϓɺϓϩτίϧ൪߸Λࢦఆͯ͠৽͍͠ιέοτΛ࡞੒ sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) sock.bind((host, port)) sock.listen(100) return sock ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 9

10. — ϝοηʔδΛsocketʹ઀ଓͯ͠ૹ৴ if __name__ == '__main__': while True: try: sock

    = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.connect((HOST, PORT)) print('\nConnected to {}:{}'.format(HOST, PORT)) print("Type message, enter to send, 'q' to quit") msg = input() if msg == 'q': break chatmodule.send_msg(sock, msg) # ૹ৴͢Δ·ͰBlock print('Sent message: {}'.format(msg)) msg = chatmodule.recv_msg(sock) # messageΛ׬શʹड৴͢Δ·ͰBlock print('Received echo: ' + msg) except ConnectionError: print('Socket error') break finally: sock.close() print('Closed connection to server\n') ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 10

11. — ϝοηʔδΛsocket͔Βड৴ def handle_client(sock, addr): """ sockΛ௨ͯ͡client͔ΒσʔλΛड͚ͱΓ,echoΛฦ͢ """ try: msg

    = chatmodule.recv_msg(sock) # messageΛ׬શʹड৴͢Δ·Ͱblock print('{}: {}'.format(addr, msg)) chatmodule.send_msg(sock, msg) # ૹ৴͢Δ·Ͱblock except (ConnectionError, BrokenPipeError): print('Socket error') finally: print('Closed connection to {}'.format(addr)) sock.close() if __name__ == '__main__': listen_sock = chatmodule.create_listen_socket(HOST, PORT) # ιέοτࣗ਎ͷΞυϨεΛฦ͢ # ͜ͷؔ਺͸ɺIPv4/v6ιέοτͷϙʔτ൪߸Λௐ΂Δ৔߹ͳͲʹ࢖༻ɻ addr = listen_sock.getsockname() print('Listening on {}'.format(addr)) while True: client_sock, addr = listen_sock.accept() print('Connection from {}'.format(addr)) handle_client(client_sock, addr) ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 11

12. ΫϥΠΞϯτͱαʔόͷιέοτ௨৴ͷಈ͖Λ·ͱΊΔͱ 1. αʔό͸socketɺbindɺlistenͰΫϥΠΞϯτͷ઀ଓΛ଴ͪड͚Δ 2. ઀ଓ͖ͨΒacceptʹΑΓ࣮ࡍͷσʔλͷಡΈग़͠·Ͱ଴ͭ 3. σʔλ͕͖ͨΒɺϦΫΤετΛॲཧͯ͠ɺΫϥΠΞϯτʹϨεϙϯεΛฦ͢ 4. ΫϥΠΞϯτͱͷ઀ଓΛcloseͰดͯ͡ɺ·ͨaccept଴ͪঢ়ଶʹͳΓϥΠ Ξϯτ͔Βͷ઀ଓΛLISTEN

    ΫϥΠΞϯτ͔Βͷ઀ଓΛaccept()ͨ͋͠ͱɺ ϧʔϓΛൈ͚Δ·Ͱ͸৽نͷΫϥΠΞϯτ઀ଓΛϒϩοΫ͍ͯ͠Δ ͜ΕͰ͸1:1ͷ௨৴͔͠Ͱ͖ͳ͍ ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 12

13. ϚϧνϓϩηεϞσϧ — fork֤ͤͯ͞ϓϩηε͕֤ΫϥΠΞϯτͱ௨৴Ͱ͖Δ — workerϞσϧ — accept()ͨ͋͠ͱͷॲཧΛϓϩηεʹॲཧͤ͞Δ — preforkϞσϧ —

    accept()͔Βclose()·ͰͷॲཧΛϓϩηεʹॲཧͤ͞Δ ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 13

14. ίʔυྫ workerϞσϧ while True: client_sock,addr = listen_sock.accept() proc = Process(target=handle_client,

    args=[client_sock, addr]) proc.start() print('Connection from {}'.format(addr)) proc.join(1) ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 14

15. σϝϦοτ:Apache٧·Δ MaxClientsʹୡ͢Δͱɺ accept()͕͞Εͳ͍ͷͰɺ ઀ଓ͸ະॲཧͱͳΓ٧·Δ ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei (

    @oza_shu ) 15

16. ϚϧνεϨουϞσϧ جຊతʹ͸Ϛϧνϓϩηε/εϨου͸ಉ͡ - 1ίωΫγϣϯ1εϨουͷϞσϧ - ϦΫΤετ͝ͱʹεϨουΛੜ੒ - εϨουϓʔϧ - ࣄલʹεϨουΛPool͓ͯ͘͠Ϟσϧ

    εϨου͸ϝϞϦڞ༗͍ͯ͠ΔͷͰɺεϨουηʔϑͳ࣮૷͕ ඞཁʹͳΔ (Ωϡʔ΍ϩοΫͷॲཧ͸ׂѪ) ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 16

17. ίʔυྫ 1ίωΫγϣϯ1εϨουͷϞσϧ if __name__ == '__main__': listen_sock = chatmodule.create_listen_socket(HOST, PORT)

    addr = listen_sock.getsockname() print('Listening on {}'.format(addr)) while True: client_sock,addr = listen_sock.accept() # Thread ͸ࣗಈతʹhandle_client()ؔ਺Λ࣮ߦ͠ɺಉ࣌ʹ͜ͷwhile loopΛ࣮ߦ thread = threading.Thread(target=handle_client, args=[client_sock, addr], daemon=True) thread.start() print('Connection from {}'.format(addr)) ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 17

18. C10K໰୊ Πϯλʔωοτ͕ൃలͯ͠Webαʔόʔ͕ಉ࣌ʹ1ສͷΫϥΠΞϯτΛॲཧ͢Δ࣌୅Ͱ͸ Ϛϧνϓϩηε/εϨουͰ͸ॲཧ͕Ͱ͖ͳ͍ͷͰɺ ղܾࡦͱͯ͠ɺΠϕϯτۦಈΞʔΩςΫνϟͰ͋ΔNginx͕஀ੜ Nginx͕໨ࢦ͢΋ͷ͸ΠϕϯτϧʔϓͰ1ͭͷεϨουͰ਺ສͷಉ࣌઀ଓΛॲཧ͢Δ͜ͱɻ ΋ͪΖΜϚϧνεϨουɾΞϓϩʔνͰղܾͰ͖ͨΓɺ ଞʹ͸Scalingͷ໰୊Ͱ΋͋Δɻ — ΞʔόϯɾΤΞʔγοϓࣾ —

    C500k໰୊ʹ௚໘ͨ͠໛༷ — C10Mͷ໰୊ — ࠓ͸ɺ1000ສͷಉ࣌઀ଓ΁ͷ௅ઓ ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 18

19. ΠϕϯτۦಈϞσϧ ৽͍͠Πϕϯτ͕ΩϡʔʹೖΕΒΕɺ εϨου͸ΠϕϯτϧʔϓΛ࣮ߦ ΠϕϯτϧʔϓͰͷεϨουΛෳ਺ͷ઀ଓʹϚοϐϯά ઀ଓɺϦΫΤετ͔Βൃੜͨ͠ΠϕϯτΛॲཧͤ͞Δ ΠϕϯτۦಈܕϓϩάϥϛϯάΛॻ͔ͳ͍ͱཧղͰ͖ͳ ͍ɻɻɻ ٢঵ࣉ.pm13 2018/02/09 -

    Ozawa Shuhei ( @oza_shu ) 19

20. ΠϕϯτۦಈϞσϧ ࠓޙ͸ҎԼ΋ௐ΂͓͖͍ͯͨ - Πϕϯτۦಈܕϓϩάϥϛϯά - ͦΕ͸ԿͰ͋ΓɺͲͷΑ͏ʹػೳ͢ΔͷͰ͔͢ʁ - asyncioϞδϡʔϧ - asyncioϕʔεͷϓϩάϥϛϯά

    - Twisted - Gevent ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 20

21. ֶΜͩ͜ͱ — TCP઀ଓͷͳ͕Ε — Apacheͷͭ·Γ͸ͳΜͳͷ͔ — ss,netstat,lsofίϚϯυ΁ͷཧղ ٢঵ࣉ.pm13 2018/02/09 -

    Ozawa Shuhei ( @oza_shu ) 21

22. WEBͷ࣍͸DB Ͳ͏͢Ε͹DBͷؾ࣋ͪʹͳΕΔͷ͔ ྑ͍ϝιου͋Ε͹ڭ͍͑ͯͩ͘͞ ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu

    ) 22

23. ·ͱΊ ࠓ೥͸ͪΌΜͱௐ΂ͯɺͪΌΜͱཧղ͢Δ!! ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu )

    23