Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Python for web architectures
Search
Shuhei Ozawa
February 09, 2018
0
910
Python for web architectures
Python Programming for web architectures
Shuhei Ozawa
February 09, 2018
Tweet
Share
More Decks by Shuhei Ozawa
See All by Shuhei Ozawa
Amebaアフィリエイト基盤の GKEアーキテクチャと マイクロサービス
ozashu
0
190
production_ready_envoy
ozashu
2
1.1k
ログ・係数集約と可視化・分析
ozashu
0
140
PyQではじめるPython
ozashu
0
420
インフラエンジニアのWEBアプリ入門
ozashu
1
8k
Featured
See All Featured
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
35
2.4k
Unsuck your backbone
ammeep
671
58k
A designer walks into a library…
pauljervisheath
207
24k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
331
22k
The Web Performance Landscape in 2024 [PerfNow 2024]
tammyeverts
8
700
Intergalactic Javascript Robots from Outer Space
tanoku
271
27k
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
10
970
Building a Scalable Design System with Sketch
lauravandoore
462
33k
Producing Creativity
orderedlist
PRO
346
40k
The MySQL Ecosystem @ GitHub 2015
samlambert
251
13k
Building Applications with DynamoDB
mza
95
6.5k
It's Worth the Effort
3n
185
28k
Transcript
PythonͰWebΞʔΩςΫνϟΛֶͿ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 1
ࣗݾհ — খपฏ — @oza_shu — MSPۀքͰWEBαʔϏεͷαʔόӡ༻ ٢ࣉ.pm13 2018/02/09 -
Ozawa Shuhei ( @oza_shu ) 2
٢ࣉ.pm13ͷςʔϚ — ʮ৽͍͠ઓɺ৽͍͠ࢹʯ — ৽ʹ͚ܾͨҙత ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei
( @oza_shu ) 3
2018ͷ๊ෛ ͪΌΜͱௐͯͪΌΜͱཧղ͢Δ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu )
4
ͪΌΜͱௐͯɺͪΌΜͱཧղ͢Δ αʔόͷӡ༻ͰোରԠ͢Δ࣌ɺͳΜͱͳ͘ͰରԠ͢Δ͜ͱ — ͜ͷϩάग़ྗͰݕࡧͯ͠ɺQiitaΈͨΓͱ͔... — ͦͷQiitaͱ͔ಡΜͰ;ΜΘΓཧղͩͬͨΓͱ͔... ٢ࣉ.pm13 2018/02/09 - Ozawa
Shuhei ( @oza_shu ) 5
ཧతղܾΛࢦ͢ — ͪΌΜͱௐͯͪΌΜͱཧղ͢Δ — ՝Λղܾ͢Δ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei
( @oza_shu ) 6
ྫ͑ ͜ΕΒWebΞʔΩςΫνϟΛPythonͰ͓͞Β͍ - TCPίωΫγϣϯͲΜͳ௨৴Λ͍ͯ͠Δͷ͔ - WEBαʔόͷಈ͖ɺߏཧղ - Apache͕٧·ΔͬͯͲ͏͍͏͜ͱ - C10Kͱ
٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 7
TCP/IP௨৴ — IPΞυϨεͱϙʔτΛࢦఆͯ͠ίωΫγϣϯଓΛ͢Δ — TCP jokeSYN→SYN/ACK→ACK — socket(),bind(),listen(),accept(),connect(),write(),read(),close() ٢ࣉ.pm13 2018/02/09
- Ozawa Shuhei ( @oza_shu ) 8
γϦΞϧϞσϧ — socket(),bind(),listen()Ͱιέοτͷ࡞ def create_listen_socket(host, port): """ αʔόʔ͕ଓཁٻΛड͚औΔιέοτΛઃఆ͢Δ """ #
ΞυϨεϑΝϛϦʔɺιέοτλΠϓɺϓϩτίϧ൪߸Λࢦఆͯ͠৽͍͠ιέοτΛ࡞ sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) sock.bind((host, port)) sock.listen(100) return sock ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 9
— ϝοηʔδΛsocketʹଓͯ͠ૹ৴ if __name__ == '__main__': while True: try: sock
= socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.connect((HOST, PORT)) print('\nConnected to {}:{}'.format(HOST, PORT)) print("Type message, enter to send, 'q' to quit") msg = input() if msg == 'q': break chatmodule.send_msg(sock, msg) # ૹ৴͢Δ·ͰBlock print('Sent message: {}'.format(msg)) msg = chatmodule.recv_msg(sock) # messageΛશʹड৴͢Δ·ͰBlock print('Received echo: ' + msg) except ConnectionError: print('Socket error') break finally: sock.close() print('Closed connection to server\n') ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 10
— ϝοηʔδΛsocket͔Βड৴ def handle_client(sock, addr): """ sockΛ௨ͯ͡client͔ΒσʔλΛड͚ͱΓ,echoΛฦ͢ """ try: msg
= chatmodule.recv_msg(sock) # messageΛશʹड৴͢Δ·Ͱblock print('{}: {}'.format(addr, msg)) chatmodule.send_msg(sock, msg) # ૹ৴͢Δ·Ͱblock except (ConnectionError, BrokenPipeError): print('Socket error') finally: print('Closed connection to {}'.format(addr)) sock.close() if __name__ == '__main__': listen_sock = chatmodule.create_listen_socket(HOST, PORT) # ιέοτࣗͷΞυϨεΛฦ͢ # ͜ͷؔɺIPv4/v6ιέοτͷϙʔτ൪߸ΛௐΔ߹ͳͲʹ༻ɻ addr = listen_sock.getsockname() print('Listening on {}'.format(addr)) while True: client_sock, addr = listen_sock.accept() print('Connection from {}'.format(addr)) handle_client(client_sock, addr) ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 11
ΫϥΠΞϯτͱαʔόͷιέοτ௨৴ͷಈ͖Λ·ͱΊΔͱ 1. αʔόsocketɺbindɺlistenͰΫϥΠΞϯτͷଓΛͪड͚Δ 2. ଓ͖ͨΒacceptʹΑΓ࣮ࡍͷσʔλͷಡΈग़͠·Ͱͭ 3. σʔλ͕͖ͨΒɺϦΫΤετΛॲཧͯ͠ɺΫϥΠΞϯτʹϨεϙϯεΛฦ͢ 4. ΫϥΠΞϯτͱͷଓΛcloseͰดͯ͡ɺ·ͨacceptͪঢ়ଶʹͳΓϥΠ Ξϯτ͔ΒͷଓΛLISTEN
ΫϥΠΞϯτ͔ΒͷଓΛaccept()ͨ͋͠ͱɺ ϧʔϓΛൈ͚Δ·Ͱ৽نͷΫϥΠΞϯτଓΛϒϩοΫ͍ͯ͠Δ ͜ΕͰ1:1ͷ௨৴͔͠Ͱ͖ͳ͍ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 12
ϚϧνϓϩηεϞσϧ — fork֤ͤͯ͞ϓϩηε͕֤ΫϥΠΞϯτͱ௨৴Ͱ͖Δ — workerϞσϧ — accept()ͨ͋͠ͱͷॲཧΛϓϩηεʹॲཧͤ͞Δ — preforkϞσϧ —
accept()͔Βclose()·ͰͷॲཧΛϓϩηεʹॲཧͤ͞Δ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 13
ίʔυྫ workerϞσϧ while True: client_sock,addr = listen_sock.accept() proc = Process(target=handle_client,
args=[client_sock, addr]) proc.start() print('Connection from {}'.format(addr)) proc.join(1) ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 14
σϝϦοτ:Apache٧·Δ MaxClientsʹୡ͢Δͱɺ accept()͕͞Εͳ͍ͷͰɺ ଓະॲཧͱͳΓ٧·Δ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei (
@oza_shu ) 15
ϚϧνεϨουϞσϧ جຊతʹϚϧνϓϩηε/εϨουಉ͡ - 1ίωΫγϣϯ1εϨουͷϞσϧ - ϦΫΤετ͝ͱʹεϨουΛੜ - εϨουϓʔϧ - ࣄલʹεϨουΛPool͓ͯ͘͠Ϟσϧ
εϨουϝϞϦڞ༗͍ͯ͠ΔͷͰɺεϨουηʔϑͳ࣮͕ ඞཁʹͳΔ (ΩϡʔϩοΫͷॲཧׂѪ) ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 16
ίʔυྫ 1ίωΫγϣϯ1εϨουͷϞσϧ if __name__ == '__main__': listen_sock = chatmodule.create_listen_socket(HOST, PORT)
addr = listen_sock.getsockname() print('Listening on {}'.format(addr)) while True: client_sock,addr = listen_sock.accept() # Thread ࣗಈతʹhandle_client()ؔΛ࣮ߦ͠ɺಉ࣌ʹ͜ͷwhile loopΛ࣮ߦ thread = threading.Thread(target=handle_client, args=[client_sock, addr], daemon=True) thread.start() print('Connection from {}'.format(addr)) ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 17
C10K Πϯλʔωοτ͕ൃలͯ͠Webαʔόʔ͕ಉ࣌ʹ1ສͷΫϥΠΞϯτΛॲཧ͢Δ࣌Ͱ Ϛϧνϓϩηε/εϨουͰॲཧ͕Ͱ͖ͳ͍ͷͰɺ ղܾࡦͱͯ͠ɺΠϕϯτۦಈΞʔΩςΫνϟͰ͋ΔNginx͕ੜ Nginx͕ࢦ͢ͷΠϕϯτϧʔϓͰ1ͭͷεϨουͰສͷಉ࣌ଓΛॲཧ͢Δ͜ͱɻ ͪΖΜϚϧνεϨουɾΞϓϩʔνͰղܾͰ͖ͨΓɺ ଞʹScalingͷͰ͋Δɻ — ΞʔόϯɾΤΞʔγοϓࣾ —
C500kʹ໘༷ͨ͠ — C10Mͷ — ࠓɺ1000ສͷಉ࣌ଓͷઓ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 18
ΠϕϯτۦಈϞσϧ ৽͍͠Πϕϯτ͕ΩϡʔʹೖΕΒΕɺ εϨουΠϕϯτϧʔϓΛ࣮ߦ ΠϕϯτϧʔϓͰͷεϨουΛෳͷଓʹϚοϐϯά ଓɺϦΫΤετ͔Βൃੜͨ͠ΠϕϯτΛॲཧͤ͞Δ ΠϕϯτۦಈܕϓϩάϥϛϯάΛॻ͔ͳ͍ͱཧղͰ͖ͳ ͍ɻɻɻ ٢ࣉ.pm13 2018/02/09 -
Ozawa Shuhei ( @oza_shu ) 19
ΠϕϯτۦಈϞσϧ ࠓޙҎԼௐ͓͖͍ͯͨ - Πϕϯτۦಈܕϓϩάϥϛϯά - ͦΕԿͰ͋ΓɺͲͷΑ͏ʹػೳ͢ΔͷͰ͔͢ʁ - asyncioϞδϡʔϧ - asyncioϕʔεͷϓϩάϥϛϯά
- Twisted - Gevent ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 20
ֶΜͩ͜ͱ — TCPଓͷͳ͕Ε — Apacheͷͭ·ΓͳΜͳͷ͔ — ss,netstat,lsofίϚϯυͷཧղ ٢ࣉ.pm13 2018/02/09 -
Ozawa Shuhei ( @oza_shu ) 21
WEBͷ࣍DB Ͳ͏͢ΕDBͷؾ࣋ͪʹͳΕΔͷ͔ ྑ͍ϝιου͋Εڭ͍͑ͯͩ͘͞ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu
) 22
·ͱΊ ࠓͪΌΜͱௐͯɺͪΌΜͱཧղ͢Δ!! ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu )
23