$30 off During Our Annual Pro Sale. View Details »

Python for web architectures

Shuhei Ozawa
February 09, 2018
810

Python for web architectures

Python Programming for web architectures

Shuhei Ozawa

February 09, 2018
Tweet

Transcript

  1. PythonͰWebΞʔΩςΫνϟΛֶͿ
    ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 1

    View Slide

  2. ࣗݾ঺հ
    — খ୔पฏ
    — @oza_shu
    — MSPۀքͰWEBαʔϏεͷαʔόӡ༻
    ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 2

    View Slide

  3. ٢঵ࣉ.pm13ͷςʔϚ
    — ʮ৽͍͠௅ઓɺ৽͍͠ࢹ఺ʯ
    — ৽೥౓ʹ޲͚ܾͨҙత
    ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 3

    View Slide

  4. 2018೥ͷ๊ෛ
    ͪΌΜͱௐ΂ͯͪΌΜͱཧղ͢Δ
    ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 4

    View Slide

  5. ͪΌΜͱௐ΂ͯɺͪΌΜͱཧղ͢Δ
    αʔόͷӡ༻Ͱো֐ରԠ͢Δ࣌ɺͳΜͱͳ͘ͰରԠ͢Δ͜ͱ΋
    — ͜ͷϩάग़ྗͰݕࡧͯ͠ɺQiitaΈͨΓͱ͔...
    — ͦͷQiitaͱ͔ಡΜͰ΋;ΜΘΓཧղͩͬͨΓͱ͔...
    ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 5

    View Slide

  6. ࿦ཧతղܾΛ໨ࢦ͢
    — ͪΌΜͱௐ΂ͯͪΌΜͱཧղ͢Δ
    — ՝୊Λղܾ͢Δ
    ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 6

    View Slide

  7. ྫ͑͹
    ͜ΕΒWebΞʔΩςΫνϟΛPythonͰ͓͞Β͍
    - TCPίωΫγϣϯ͸ͲΜͳ௨৴Λ͍ͯ͠Δͷ͔
    - WEBαʔόͷಈ͖ɺߏ଄ཧղ
    - Apache͕٧·ΔͬͯͲ͏͍͏͜ͱ
    - C10K໰୊ͱ͸
    ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 7

    View Slide

  8. TCP/IP௨৴
    — IPΞυϨεͱϙʔτΛࢦఆͯ͠ίωΫγϣϯ઀ଓΛ͢Δ
    — TCP joke΍SYN→SYN/ACK→ACK
    — socket(),bind(),listen(),accept(),connect(),write(),read(),close()
    ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 8

    View Slide

  9. γϦΞϧϞσϧ
    — socket(),bind(),listen()Ͱιέοτͷ࡞੒
    def create_listen_socket(host, port):
    """ αʔόʔ͕઀ଓཁٻΛड͚औΔιέοτΛઃఆ͢Δ """
    # ΞυϨεϑΝϛϦʔɺιέοτλΠϓɺϓϩτίϧ൪߸Λࢦఆͯ͠৽͍͠ιέοτΛ࡞੒
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    sock.bind((host, port))
    sock.listen(100)
    return sock
    ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 9

    View Slide

  10. — ϝοηʔδΛsocketʹ઀ଓͯ͠ૹ৴
    if __name__ == '__main__':
    while True:
    try:
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    sock.connect((HOST, PORT))
    print('\nConnected to {}:{}'.format(HOST, PORT))
    print("Type message, enter to send, 'q' to quit")
    msg = input()
    if msg == 'q': break
    chatmodule.send_msg(sock, msg) # ૹ৴͢Δ·ͰBlock
    print('Sent message: {}'.format(msg))
    msg = chatmodule.recv_msg(sock) # messageΛ׬શʹड৴͢Δ·ͰBlock
    print('Received echo: ' + msg)
    except ConnectionError:
    print('Socket error')
    break
    finally:
    sock.close()
    print('Closed connection to server\n')
    ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 10

    View Slide

  11. — ϝοηʔδΛsocket͔Βड৴
    def handle_client(sock, addr):
    """ sockΛ௨ͯ͡client͔ΒσʔλΛड͚ͱΓ,echoΛฦ͢ """
    try:
    msg = chatmodule.recv_msg(sock) # messageΛ׬શʹड৴͢Δ·Ͱblock
    print('{}: {}'.format(addr, msg))
    chatmodule.send_msg(sock, msg) # ૹ৴͢Δ·Ͱblock
    except (ConnectionError, BrokenPipeError):
    print('Socket error')
    finally:
    print('Closed connection to {}'.format(addr))
    sock.close()
    if __name__ == '__main__':
    listen_sock = chatmodule.create_listen_socket(HOST, PORT)
    # ιέοτࣗ਎ͷΞυϨεΛฦ͢
    # ͜ͷؔ਺͸ɺIPv4/v6ιέοτͷϙʔτ൪߸Λௐ΂Δ৔߹ͳͲʹ࢖༻ɻ
    addr = listen_sock.getsockname()
    print('Listening on {}'.format(addr))
    while True:
    client_sock, addr = listen_sock.accept()
    print('Connection from {}'.format(addr))
    handle_client(client_sock, addr)
    ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 11

    View Slide

  12. ΫϥΠΞϯτͱαʔόͷιέοτ௨৴ͷಈ͖Λ·ͱΊΔͱ
    1. αʔό͸socketɺbindɺlistenͰΫϥΠΞϯτͷ઀ଓΛ଴ͪड͚Δ
    2. ઀ଓ͖ͨΒacceptʹΑΓ࣮ࡍͷσʔλͷಡΈग़͠·Ͱ଴ͭ
    3. σʔλ͕͖ͨΒɺϦΫΤετΛॲཧͯ͠ɺΫϥΠΞϯτʹϨεϙϯεΛฦ͢
    4. ΫϥΠΞϯτͱͷ઀ଓΛcloseͰดͯ͡ɺ·ͨaccept଴ͪঢ়ଶʹͳΓϥΠ
    Ξϯτ͔Βͷ઀ଓΛLISTEN
    ΫϥΠΞϯτ͔Βͷ઀ଓΛaccept()ͨ͋͠ͱɺ
    ϧʔϓΛൈ͚Δ·Ͱ͸৽نͷΫϥΠΞϯτ઀ଓΛϒϩοΫ͍ͯ͠Δ
    ͜ΕͰ͸1:1ͷ௨৴͔͠Ͱ͖ͳ͍
    ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 12

    View Slide

  13. ϚϧνϓϩηεϞσϧ
    — fork֤ͤͯ͞ϓϩηε͕֤ΫϥΠΞϯτͱ௨৴Ͱ͖Δ
    — workerϞσϧ
    — accept()ͨ͋͠ͱͷॲཧΛϓϩηεʹॲཧͤ͞Δ
    — preforkϞσϧ
    — accept()͔Βclose()·ͰͷॲཧΛϓϩηεʹॲཧͤ͞Δ
    ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 13

    View Slide

  14. ίʔυྫ workerϞσϧ
    while True:
    client_sock,addr = listen_sock.accept()
    proc = Process(target=handle_client,
    args=[client_sock, addr])
    proc.start()
    print('Connection from {}'.format(addr))
    proc.join(1)
    ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 14

    View Slide

  15. σϝϦοτ:Apache٧·Δ
    MaxClientsʹୡ͢Δͱɺ
    accept()͕͞Εͳ͍ͷͰɺ
    ઀ଓ͸ະॲཧͱͳΓ٧·Δ
    ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 15

    View Slide

  16. ϚϧνεϨουϞσϧ
    جຊతʹ͸Ϛϧνϓϩηε/εϨου͸ಉ͡
    - 1ίωΫγϣϯ1εϨουͷϞσϧ
    - ϦΫΤετ͝ͱʹεϨουΛੜ੒
    - εϨουϓʔϧ
    - ࣄલʹεϨουΛPool͓ͯ͘͠Ϟσϧ
    εϨου͸ϝϞϦڞ༗͍ͯ͠ΔͷͰɺεϨουηʔϑͳ࣮૷͕
    ඞཁʹͳΔ
    (Ωϡʔ΍ϩοΫͷॲཧ͸ׂѪ)
    ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 16

    View Slide

  17. ίʔυྫ 1ίωΫγϣϯ1εϨουͷϞσϧ
    if __name__ == '__main__':
    listen_sock = chatmodule.create_listen_socket(HOST, PORT)
    addr = listen_sock.getsockname()
    print('Listening on {}'.format(addr))
    while True:
    client_sock,addr = listen_sock.accept()
    # Thread ͸ࣗಈతʹhandle_client()ؔ਺Λ࣮ߦ͠ɺಉ࣌ʹ͜ͷwhile loopΛ࣮ߦ
    thread = threading.Thread(target=handle_client,
    args=[client_sock, addr],
    daemon=True)
    thread.start()
    print('Connection from {}'.format(addr))
    ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 17

    View Slide

  18. C10K໰୊
    Πϯλʔωοτ͕ൃలͯ͠Webαʔόʔ͕ಉ࣌ʹ1ສͷΫϥΠΞϯτΛॲཧ͢Δ࣌୅Ͱ͸
    Ϛϧνϓϩηε/εϨουͰ͸ॲཧ͕Ͱ͖ͳ͍ͷͰɺ
    ղܾࡦͱͯ͠ɺΠϕϯτۦಈΞʔΩςΫνϟͰ͋ΔNginx͕஀ੜ
    Nginx͕໨ࢦ͢΋ͷ͸ΠϕϯτϧʔϓͰ1ͭͷεϨουͰ਺ສͷಉ࣌઀ଓΛॲཧ͢Δ͜ͱɻ
    ΋ͪΖΜϚϧνεϨουɾΞϓϩʔνͰղܾͰ͖ͨΓɺ
    ଞʹ͸Scalingͷ໰୊Ͱ΋͋Δɻ
    — ΞʔόϯɾΤΞʔγοϓࣾ
    — C500k໰୊ʹ௚໘ͨ͠໛༷
    — C10Mͷ໰୊
    — ࠓ͸ɺ1000ສͷಉ࣌઀ଓ΁ͷ௅ઓ
    ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 18

    View Slide

  19. ΠϕϯτۦಈϞσϧ
    ৽͍͠Πϕϯτ͕ΩϡʔʹೖΕΒΕɺ
    εϨου͸ΠϕϯτϧʔϓΛ࣮ߦ
    ΠϕϯτϧʔϓͰͷεϨουΛෳ਺ͷ઀ଓʹϚοϐϯά
    ઀ଓɺϦΫΤετ͔Βൃੜͨ͠ΠϕϯτΛॲཧͤ͞Δ
    ΠϕϯτۦಈܕϓϩάϥϛϯάΛॻ͔ͳ͍ͱཧղͰ͖ͳ
    ͍ɻɻɻ
    ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 19

    View Slide

  20. ΠϕϯτۦಈϞσϧ
    ࠓޙ͸ҎԼ΋ௐ΂͓͖͍ͯͨ
    - Πϕϯτۦಈܕϓϩάϥϛϯά
    - ͦΕ͸ԿͰ͋ΓɺͲͷΑ͏ʹػೳ͢ΔͷͰ͔͢ʁ
    - asyncioϞδϡʔϧ
    - asyncioϕʔεͷϓϩάϥϛϯά
    - Twisted
    - Gevent
    ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 20

    View Slide

  21. ֶΜͩ͜ͱ
    — TCP઀ଓͷͳ͕Ε
    — Apacheͷͭ·Γ͸ͳΜͳͷ͔
    — ss,netstat,lsofίϚϯυ΁ͷཧղ
    ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 21

    View Slide

  22. WEBͷ࣍͸DB
    Ͳ͏͢Ε͹DBͷؾ࣋ͪʹͳΕΔͷ͔
    ྑ͍ϝιου͋Ε͹ڭ͍͑ͯͩ͘͞
    ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 22

    View Slide

  23. ·ͱΊ
    ࠓ೥͸ͪΌΜͱௐ΂ͯɺͪΌΜͱཧղ͢Δ!!
    ٢঵ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 23

    View Slide