Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Python for web architectures
Search
Shuhei Ozawa
February 09, 2018
0
830
Python for web architectures
Python Programming for web architectures
Shuhei Ozawa
February 09, 2018
Tweet
Share
More Decks by Shuhei Ozawa
See All by Shuhei Ozawa
Amebaアフィリエイト基盤の GKEアーキテクチャと マイクロサービス
ozashu
0
110
production_ready_envoy
ozashu
2
990
ログ・係数集約と可視化・分析
ozashu
0
100
PyQではじめるPython
ozashu
0
370
インフラエンジニアのWEBアプリ入門
ozashu
1
7.8k
Featured
See All Featured
Pencils Down: Stop Designing & Start Developing
hursman
115
11k
Practical Orchestrator
shlominoach
180
9.7k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
1
1.2k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
18
1.7k
Happy Clients
brianwarren
91
6.3k
Documentation Writing (for coders)
carmenintech
59
3.8k
Git: the NoSQL Database
bkeepers
PRO
421
63k
Adopting Sorbet at Scale
ufuk
66
8.5k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
24
2.2k
Building a Scalable Design System with Sketch
lauravandoore
455
32k
The Power of CSS Pseudo Elements
geoffreycrofte
58
4.9k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
319
20k
Transcript
PythonͰWebΞʔΩςΫνϟΛֶͿ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 1
ࣗݾհ — খपฏ — @oza_shu — MSPۀքͰWEBαʔϏεͷαʔόӡ༻ ٢ࣉ.pm13 2018/02/09 -
Ozawa Shuhei ( @oza_shu ) 2
٢ࣉ.pm13ͷςʔϚ — ʮ৽͍͠ઓɺ৽͍͠ࢹʯ — ৽ʹ͚ܾͨҙత ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei
( @oza_shu ) 3
2018ͷ๊ෛ ͪΌΜͱௐͯͪΌΜͱཧղ͢Δ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu )
4
ͪΌΜͱௐͯɺͪΌΜͱཧղ͢Δ αʔόͷӡ༻ͰোରԠ͢Δ࣌ɺͳΜͱͳ͘ͰରԠ͢Δ͜ͱ — ͜ͷϩάग़ྗͰݕࡧͯ͠ɺQiitaΈͨΓͱ͔... — ͦͷQiitaͱ͔ಡΜͰ;ΜΘΓཧղͩͬͨΓͱ͔... ٢ࣉ.pm13 2018/02/09 - Ozawa
Shuhei ( @oza_shu ) 5
ཧతղܾΛࢦ͢ — ͪΌΜͱௐͯͪΌΜͱཧղ͢Δ — ՝Λղܾ͢Δ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei
( @oza_shu ) 6
ྫ͑ ͜ΕΒWebΞʔΩςΫνϟΛPythonͰ͓͞Β͍ - TCPίωΫγϣϯͲΜͳ௨৴Λ͍ͯ͠Δͷ͔ - WEBαʔόͷಈ͖ɺߏཧղ - Apache͕٧·ΔͬͯͲ͏͍͏͜ͱ - C10Kͱ
٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 7
TCP/IP௨৴ — IPΞυϨεͱϙʔτΛࢦఆͯ͠ίωΫγϣϯଓΛ͢Δ — TCP jokeSYN→SYN/ACK→ACK — socket(),bind(),listen(),accept(),connect(),write(),read(),close() ٢ࣉ.pm13 2018/02/09
- Ozawa Shuhei ( @oza_shu ) 8
γϦΞϧϞσϧ — socket(),bind(),listen()Ͱιέοτͷ࡞ def create_listen_socket(host, port): """ αʔόʔ͕ଓཁٻΛड͚औΔιέοτΛઃఆ͢Δ """ #
ΞυϨεϑΝϛϦʔɺιέοτλΠϓɺϓϩτίϧ൪߸Λࢦఆͯ͠৽͍͠ιέοτΛ࡞ sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) sock.bind((host, port)) sock.listen(100) return sock ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 9
— ϝοηʔδΛsocketʹଓͯ͠ૹ৴ if __name__ == '__main__': while True: try: sock
= socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.connect((HOST, PORT)) print('\nConnected to {}:{}'.format(HOST, PORT)) print("Type message, enter to send, 'q' to quit") msg = input() if msg == 'q': break chatmodule.send_msg(sock, msg) # ૹ৴͢Δ·ͰBlock print('Sent message: {}'.format(msg)) msg = chatmodule.recv_msg(sock) # messageΛશʹड৴͢Δ·ͰBlock print('Received echo: ' + msg) except ConnectionError: print('Socket error') break finally: sock.close() print('Closed connection to server\n') ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 10
— ϝοηʔδΛsocket͔Βड৴ def handle_client(sock, addr): """ sockΛ௨ͯ͡client͔ΒσʔλΛड͚ͱΓ,echoΛฦ͢ """ try: msg
= chatmodule.recv_msg(sock) # messageΛશʹड৴͢Δ·Ͱblock print('{}: {}'.format(addr, msg)) chatmodule.send_msg(sock, msg) # ૹ৴͢Δ·Ͱblock except (ConnectionError, BrokenPipeError): print('Socket error') finally: print('Closed connection to {}'.format(addr)) sock.close() if __name__ == '__main__': listen_sock = chatmodule.create_listen_socket(HOST, PORT) # ιέοτࣗͷΞυϨεΛฦ͢ # ͜ͷؔɺIPv4/v6ιέοτͷϙʔτ൪߸ΛௐΔ߹ͳͲʹ༻ɻ addr = listen_sock.getsockname() print('Listening on {}'.format(addr)) while True: client_sock, addr = listen_sock.accept() print('Connection from {}'.format(addr)) handle_client(client_sock, addr) ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 11
ΫϥΠΞϯτͱαʔόͷιέοτ௨৴ͷಈ͖Λ·ͱΊΔͱ 1. αʔόsocketɺbindɺlistenͰΫϥΠΞϯτͷଓΛͪड͚Δ 2. ଓ͖ͨΒacceptʹΑΓ࣮ࡍͷσʔλͷಡΈग़͠·Ͱͭ 3. σʔλ͕͖ͨΒɺϦΫΤετΛॲཧͯ͠ɺΫϥΠΞϯτʹϨεϙϯεΛฦ͢ 4. ΫϥΠΞϯτͱͷଓΛcloseͰดͯ͡ɺ·ͨacceptͪঢ়ଶʹͳΓϥΠ Ξϯτ͔ΒͷଓΛLISTEN
ΫϥΠΞϯτ͔ΒͷଓΛaccept()ͨ͋͠ͱɺ ϧʔϓΛൈ͚Δ·Ͱ৽نͷΫϥΠΞϯτଓΛϒϩοΫ͍ͯ͠Δ ͜ΕͰ1:1ͷ௨৴͔͠Ͱ͖ͳ͍ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 12
ϚϧνϓϩηεϞσϧ — fork֤ͤͯ͞ϓϩηε͕֤ΫϥΠΞϯτͱ௨৴Ͱ͖Δ — workerϞσϧ — accept()ͨ͋͠ͱͷॲཧΛϓϩηεʹॲཧͤ͞Δ — preforkϞσϧ —
accept()͔Βclose()·ͰͷॲཧΛϓϩηεʹॲཧͤ͞Δ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 13
ίʔυྫ workerϞσϧ while True: client_sock,addr = listen_sock.accept() proc = Process(target=handle_client,
args=[client_sock, addr]) proc.start() print('Connection from {}'.format(addr)) proc.join(1) ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 14
σϝϦοτ:Apache٧·Δ MaxClientsʹୡ͢Δͱɺ accept()͕͞Εͳ͍ͷͰɺ ଓະॲཧͱͳΓ٧·Δ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei (
@oza_shu ) 15
ϚϧνεϨουϞσϧ جຊతʹϚϧνϓϩηε/εϨουಉ͡ - 1ίωΫγϣϯ1εϨουͷϞσϧ - ϦΫΤετ͝ͱʹεϨουΛੜ - εϨουϓʔϧ - ࣄલʹεϨουΛPool͓ͯ͘͠Ϟσϧ
εϨουϝϞϦڞ༗͍ͯ͠ΔͷͰɺεϨουηʔϑͳ࣮͕ ඞཁʹͳΔ (ΩϡʔϩοΫͷॲཧׂѪ) ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 16
ίʔυྫ 1ίωΫγϣϯ1εϨουͷϞσϧ if __name__ == '__main__': listen_sock = chatmodule.create_listen_socket(HOST, PORT)
addr = listen_sock.getsockname() print('Listening on {}'.format(addr)) while True: client_sock,addr = listen_sock.accept() # Thread ࣗಈతʹhandle_client()ؔΛ࣮ߦ͠ɺಉ࣌ʹ͜ͷwhile loopΛ࣮ߦ thread = threading.Thread(target=handle_client, args=[client_sock, addr], daemon=True) thread.start() print('Connection from {}'.format(addr)) ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 17
C10K Πϯλʔωοτ͕ൃలͯ͠Webαʔόʔ͕ಉ࣌ʹ1ສͷΫϥΠΞϯτΛॲཧ͢Δ࣌Ͱ Ϛϧνϓϩηε/εϨουͰॲཧ͕Ͱ͖ͳ͍ͷͰɺ ղܾࡦͱͯ͠ɺΠϕϯτۦಈΞʔΩςΫνϟͰ͋ΔNginx͕ੜ Nginx͕ࢦ͢ͷΠϕϯτϧʔϓͰ1ͭͷεϨουͰສͷಉ࣌ଓΛॲཧ͢Δ͜ͱɻ ͪΖΜϚϧνεϨουɾΞϓϩʔνͰղܾͰ͖ͨΓɺ ଞʹScalingͷͰ͋Δɻ — ΞʔόϯɾΤΞʔγοϓࣾ —
C500kʹ໘༷ͨ͠ — C10Mͷ — ࠓɺ1000ສͷಉ࣌ଓͷઓ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 18
ΠϕϯτۦಈϞσϧ ৽͍͠Πϕϯτ͕ΩϡʔʹೖΕΒΕɺ εϨουΠϕϯτϧʔϓΛ࣮ߦ ΠϕϯτϧʔϓͰͷεϨουΛෳͷଓʹϚοϐϯά ଓɺϦΫΤετ͔Βൃੜͨ͠ΠϕϯτΛॲཧͤ͞Δ ΠϕϯτۦಈܕϓϩάϥϛϯάΛॻ͔ͳ͍ͱཧղͰ͖ͳ ͍ɻɻɻ ٢ࣉ.pm13 2018/02/09 -
Ozawa Shuhei ( @oza_shu ) 19
ΠϕϯτۦಈϞσϧ ࠓޙҎԼௐ͓͖͍ͯͨ - Πϕϯτۦಈܕϓϩάϥϛϯά - ͦΕԿͰ͋ΓɺͲͷΑ͏ʹػೳ͢ΔͷͰ͔͢ʁ - asyncioϞδϡʔϧ - asyncioϕʔεͷϓϩάϥϛϯά
- Twisted - Gevent ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 20
ֶΜͩ͜ͱ — TCPଓͷͳ͕Ε — Apacheͷͭ·ΓͳΜͳͷ͔ — ss,netstat,lsofίϚϯυͷཧղ ٢ࣉ.pm13 2018/02/09 -
Ozawa Shuhei ( @oza_shu ) 21
WEBͷ࣍DB Ͳ͏͢ΕDBͷؾ࣋ͪʹͳΕΔͷ͔ ྑ͍ϝιου͋Εڭ͍͑ͯͩ͘͞ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu
) 22
·ͱΊ ࠓͪΌΜͱௐͯɺͪΌΜͱཧղ͢Δ!! ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu )
23