Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Python for web architectures
Search
Shuhei Ozawa
February 09, 2018
0
920
Python for web architectures
Python Programming for web architectures
Shuhei Ozawa
February 09, 2018
Tweet
Share
More Decks by Shuhei Ozawa
See All by Shuhei Ozawa
Amebaアフィリエイト基盤の GKEアーキテクチャと マイクロサービス
ozashu
0
200
production_ready_envoy
ozashu
2
1.2k
ログ・係数集約と可視化・分析
ozashu
0
140
PyQではじめるPython
ozashu
0
430
インフラエンジニアのWEBアプリ入門
ozashu
1
8k
Featured
See All Featured
It's Worth the Effort
3n
187
28k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
46
7.6k
Navigating Team Friction
lara
189
15k
A Tale of Four Properties
chriscoyier
160
23k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
36
2.5k
No one is an island. Learnings from fostering a developers community.
thoeni
21
3.4k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
358
30k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
33
2.4k
The Invisible Side of Design
smashingmag
301
51k
Documentation Writing (for coders)
carmenintech
74
5k
Thoughts on Productivity
jonyablonski
70
4.8k
Automating Front-end Workflow
addyosmani
1370
200k
Transcript
PythonͰWebΞʔΩςΫνϟΛֶͿ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 1
ࣗݾհ — খपฏ — @oza_shu — MSPۀքͰWEBαʔϏεͷαʔόӡ༻ ٢ࣉ.pm13 2018/02/09 -
Ozawa Shuhei ( @oza_shu ) 2
٢ࣉ.pm13ͷςʔϚ — ʮ৽͍͠ઓɺ৽͍͠ࢹʯ — ৽ʹ͚ܾͨҙత ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei
( @oza_shu ) 3
2018ͷ๊ෛ ͪΌΜͱௐͯͪΌΜͱཧղ͢Δ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu )
4
ͪΌΜͱௐͯɺͪΌΜͱཧղ͢Δ αʔόͷӡ༻ͰোରԠ͢Δ࣌ɺͳΜͱͳ͘ͰରԠ͢Δ͜ͱ — ͜ͷϩάग़ྗͰݕࡧͯ͠ɺQiitaΈͨΓͱ͔... — ͦͷQiitaͱ͔ಡΜͰ;ΜΘΓཧղͩͬͨΓͱ͔... ٢ࣉ.pm13 2018/02/09 - Ozawa
Shuhei ( @oza_shu ) 5
ཧతղܾΛࢦ͢ — ͪΌΜͱௐͯͪΌΜͱཧղ͢Δ — ՝Λղܾ͢Δ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei
( @oza_shu ) 6
ྫ͑ ͜ΕΒWebΞʔΩςΫνϟΛPythonͰ͓͞Β͍ - TCPίωΫγϣϯͲΜͳ௨৴Λ͍ͯ͠Δͷ͔ - WEBαʔόͷಈ͖ɺߏཧղ - Apache͕٧·ΔͬͯͲ͏͍͏͜ͱ - C10Kͱ
٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 7
TCP/IP௨৴ — IPΞυϨεͱϙʔτΛࢦఆͯ͠ίωΫγϣϯଓΛ͢Δ — TCP jokeSYN→SYN/ACK→ACK — socket(),bind(),listen(),accept(),connect(),write(),read(),close() ٢ࣉ.pm13 2018/02/09
- Ozawa Shuhei ( @oza_shu ) 8
γϦΞϧϞσϧ — socket(),bind(),listen()Ͱιέοτͷ࡞ def create_listen_socket(host, port): """ αʔόʔ͕ଓཁٻΛड͚औΔιέοτΛઃఆ͢Δ """ #
ΞυϨεϑΝϛϦʔɺιέοτλΠϓɺϓϩτίϧ൪߸Λࢦఆͯ͠৽͍͠ιέοτΛ࡞ sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) sock.bind((host, port)) sock.listen(100) return sock ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 9
— ϝοηʔδΛsocketʹଓͯ͠ૹ৴ if __name__ == '__main__': while True: try: sock
= socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.connect((HOST, PORT)) print('\nConnected to {}:{}'.format(HOST, PORT)) print("Type message, enter to send, 'q' to quit") msg = input() if msg == 'q': break chatmodule.send_msg(sock, msg) # ૹ৴͢Δ·ͰBlock print('Sent message: {}'.format(msg)) msg = chatmodule.recv_msg(sock) # messageΛશʹड৴͢Δ·ͰBlock print('Received echo: ' + msg) except ConnectionError: print('Socket error') break finally: sock.close() print('Closed connection to server\n') ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 10
— ϝοηʔδΛsocket͔Βड৴ def handle_client(sock, addr): """ sockΛ௨ͯ͡client͔ΒσʔλΛड͚ͱΓ,echoΛฦ͢ """ try: msg
= chatmodule.recv_msg(sock) # messageΛશʹड৴͢Δ·Ͱblock print('{}: {}'.format(addr, msg)) chatmodule.send_msg(sock, msg) # ૹ৴͢Δ·Ͱblock except (ConnectionError, BrokenPipeError): print('Socket error') finally: print('Closed connection to {}'.format(addr)) sock.close() if __name__ == '__main__': listen_sock = chatmodule.create_listen_socket(HOST, PORT) # ιέοτࣗͷΞυϨεΛฦ͢ # ͜ͷؔɺIPv4/v6ιέοτͷϙʔτ൪߸ΛௐΔ߹ͳͲʹ༻ɻ addr = listen_sock.getsockname() print('Listening on {}'.format(addr)) while True: client_sock, addr = listen_sock.accept() print('Connection from {}'.format(addr)) handle_client(client_sock, addr) ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 11
ΫϥΠΞϯτͱαʔόͷιέοτ௨৴ͷಈ͖Λ·ͱΊΔͱ 1. αʔόsocketɺbindɺlistenͰΫϥΠΞϯτͷଓΛͪड͚Δ 2. ଓ͖ͨΒacceptʹΑΓ࣮ࡍͷσʔλͷಡΈग़͠·Ͱͭ 3. σʔλ͕͖ͨΒɺϦΫΤετΛॲཧͯ͠ɺΫϥΠΞϯτʹϨεϙϯεΛฦ͢ 4. ΫϥΠΞϯτͱͷଓΛcloseͰดͯ͡ɺ·ͨacceptͪঢ়ଶʹͳΓϥΠ Ξϯτ͔ΒͷଓΛLISTEN
ΫϥΠΞϯτ͔ΒͷଓΛaccept()ͨ͋͠ͱɺ ϧʔϓΛൈ͚Δ·Ͱ৽نͷΫϥΠΞϯτଓΛϒϩοΫ͍ͯ͠Δ ͜ΕͰ1:1ͷ௨৴͔͠Ͱ͖ͳ͍ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 12
ϚϧνϓϩηεϞσϧ — fork֤ͤͯ͞ϓϩηε͕֤ΫϥΠΞϯτͱ௨৴Ͱ͖Δ — workerϞσϧ — accept()ͨ͋͠ͱͷॲཧΛϓϩηεʹॲཧͤ͞Δ — preforkϞσϧ —
accept()͔Βclose()·ͰͷॲཧΛϓϩηεʹॲཧͤ͞Δ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 13
ίʔυྫ workerϞσϧ while True: client_sock,addr = listen_sock.accept() proc = Process(target=handle_client,
args=[client_sock, addr]) proc.start() print('Connection from {}'.format(addr)) proc.join(1) ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 14
σϝϦοτ:Apache٧·Δ MaxClientsʹୡ͢Δͱɺ accept()͕͞Εͳ͍ͷͰɺ ଓະॲཧͱͳΓ٧·Δ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei (
@oza_shu ) 15
ϚϧνεϨουϞσϧ جຊతʹϚϧνϓϩηε/εϨουಉ͡ - 1ίωΫγϣϯ1εϨουͷϞσϧ - ϦΫΤετ͝ͱʹεϨουΛੜ - εϨουϓʔϧ - ࣄલʹεϨουΛPool͓ͯ͘͠Ϟσϧ
εϨουϝϞϦڞ༗͍ͯ͠ΔͷͰɺεϨουηʔϑͳ࣮͕ ඞཁʹͳΔ (ΩϡʔϩοΫͷॲཧׂѪ) ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 16
ίʔυྫ 1ίωΫγϣϯ1εϨουͷϞσϧ if __name__ == '__main__': listen_sock = chatmodule.create_listen_socket(HOST, PORT)
addr = listen_sock.getsockname() print('Listening on {}'.format(addr)) while True: client_sock,addr = listen_sock.accept() # Thread ࣗಈతʹhandle_client()ؔΛ࣮ߦ͠ɺಉ࣌ʹ͜ͷwhile loopΛ࣮ߦ thread = threading.Thread(target=handle_client, args=[client_sock, addr], daemon=True) thread.start() print('Connection from {}'.format(addr)) ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 17
C10K Πϯλʔωοτ͕ൃలͯ͠Webαʔόʔ͕ಉ࣌ʹ1ສͷΫϥΠΞϯτΛॲཧ͢Δ࣌Ͱ Ϛϧνϓϩηε/εϨουͰॲཧ͕Ͱ͖ͳ͍ͷͰɺ ղܾࡦͱͯ͠ɺΠϕϯτۦಈΞʔΩςΫνϟͰ͋ΔNginx͕ੜ Nginx͕ࢦ͢ͷΠϕϯτϧʔϓͰ1ͭͷεϨουͰສͷಉ࣌ଓΛॲཧ͢Δ͜ͱɻ ͪΖΜϚϧνεϨουɾΞϓϩʔνͰղܾͰ͖ͨΓɺ ଞʹScalingͷͰ͋Δɻ — ΞʔόϯɾΤΞʔγοϓࣾ —
C500kʹ໘༷ͨ͠ — C10Mͷ — ࠓɺ1000ສͷಉ࣌ଓͷઓ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 18
ΠϕϯτۦಈϞσϧ ৽͍͠Πϕϯτ͕ΩϡʔʹೖΕΒΕɺ εϨουΠϕϯτϧʔϓΛ࣮ߦ ΠϕϯτϧʔϓͰͷεϨουΛෳͷଓʹϚοϐϯά ଓɺϦΫΤετ͔Βൃੜͨ͠ΠϕϯτΛॲཧͤ͞Δ ΠϕϯτۦಈܕϓϩάϥϛϯάΛॻ͔ͳ͍ͱཧղͰ͖ͳ ͍ɻɻɻ ٢ࣉ.pm13 2018/02/09 -
Ozawa Shuhei ( @oza_shu ) 19
ΠϕϯτۦಈϞσϧ ࠓޙҎԼௐ͓͖͍ͯͨ - Πϕϯτۦಈܕϓϩάϥϛϯά - ͦΕԿͰ͋ΓɺͲͷΑ͏ʹػೳ͢ΔͷͰ͔͢ʁ - asyncioϞδϡʔϧ - asyncioϕʔεͷϓϩάϥϛϯά
- Twisted - Gevent ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 20
ֶΜͩ͜ͱ — TCPଓͷͳ͕Ε — Apacheͷͭ·ΓͳΜͳͷ͔ — ss,netstat,lsofίϚϯυͷཧղ ٢ࣉ.pm13 2018/02/09 -
Ozawa Shuhei ( @oza_shu ) 21
WEBͷ࣍DB Ͳ͏͢ΕDBͷؾ࣋ͪʹͳΕΔͷ͔ ྑ͍ϝιου͋Εڭ͍͑ͯͩ͘͞ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu
) 22
·ͱΊ ࠓͪΌΜͱௐͯɺͪΌΜͱཧղ͢Δ!! ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu )
23