Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Python for web architectures
Search
Shuhei Ozawa
February 09, 2018
0
920
Python for web architectures
Python Programming for web architectures
Shuhei Ozawa
February 09, 2018
Tweet
Share
More Decks by Shuhei Ozawa
See All by Shuhei Ozawa
Amebaアフィリエイト基盤の GKEアーキテクチャと マイクロサービス
ozashu
0
210
production_ready_envoy
ozashu
2
1.2k
ログ・係数集約と可視化・分析
ozashu
0
140
PyQではじめるPython
ozashu
0
430
インフラエンジニアのWEBアプリ入門
ozashu
1
8.1k
Featured
See All Featured
Put a Button on it: Removing Barriers to Going Fast.
kastner
60
4k
Optimising Largest Contentful Paint
csswizardry
37
3.5k
[RailsConf 2023] Rails as a piece of cake
palkan
57
5.9k
Documentation Writing (for coders)
carmenintech
75
5.1k
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
12
1.2k
Git: the NoSQL Database
bkeepers
PRO
431
66k
Building a Modern Day E-commerce SEO Strategy
aleyda
44
7.8k
Embracing the Ebb and Flow
colly
88
4.9k
The Illustrated Children's Guide to Kubernetes
chrisshort
49
51k
How To Stay Up To Date on Web Technology
chriscoyier
791
250k
GraphQLの誤解/rethinking-graphql
sonatard
73
11k
The World Runs on Bad Software
bkeepers
PRO
72
11k
Transcript
PythonͰWebΞʔΩςΫνϟΛֶͿ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 1
ࣗݾհ — খपฏ — @oza_shu — MSPۀքͰWEBαʔϏεͷαʔόӡ༻ ٢ࣉ.pm13 2018/02/09 -
Ozawa Shuhei ( @oza_shu ) 2
٢ࣉ.pm13ͷςʔϚ — ʮ৽͍͠ઓɺ৽͍͠ࢹʯ — ৽ʹ͚ܾͨҙత ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei
( @oza_shu ) 3
2018ͷ๊ෛ ͪΌΜͱௐͯͪΌΜͱཧղ͢Δ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu )
4
ͪΌΜͱௐͯɺͪΌΜͱཧղ͢Δ αʔόͷӡ༻ͰোରԠ͢Δ࣌ɺͳΜͱͳ͘ͰରԠ͢Δ͜ͱ — ͜ͷϩάग़ྗͰݕࡧͯ͠ɺQiitaΈͨΓͱ͔... — ͦͷQiitaͱ͔ಡΜͰ;ΜΘΓཧղͩͬͨΓͱ͔... ٢ࣉ.pm13 2018/02/09 - Ozawa
Shuhei ( @oza_shu ) 5
ཧతղܾΛࢦ͢ — ͪΌΜͱௐͯͪΌΜͱཧղ͢Δ — ՝Λղܾ͢Δ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei
( @oza_shu ) 6
ྫ͑ ͜ΕΒWebΞʔΩςΫνϟΛPythonͰ͓͞Β͍ - TCPίωΫγϣϯͲΜͳ௨৴Λ͍ͯ͠Δͷ͔ - WEBαʔόͷಈ͖ɺߏཧղ - Apache͕٧·ΔͬͯͲ͏͍͏͜ͱ - C10Kͱ
٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 7
TCP/IP௨৴ — IPΞυϨεͱϙʔτΛࢦఆͯ͠ίωΫγϣϯଓΛ͢Δ — TCP jokeSYN→SYN/ACK→ACK — socket(),bind(),listen(),accept(),connect(),write(),read(),close() ٢ࣉ.pm13 2018/02/09
- Ozawa Shuhei ( @oza_shu ) 8
γϦΞϧϞσϧ — socket(),bind(),listen()Ͱιέοτͷ࡞ def create_listen_socket(host, port): """ αʔόʔ͕ଓཁٻΛड͚औΔιέοτΛઃఆ͢Δ """ #
ΞυϨεϑΝϛϦʔɺιέοτλΠϓɺϓϩτίϧ൪߸Λࢦఆͯ͠৽͍͠ιέοτΛ࡞ sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) sock.bind((host, port)) sock.listen(100) return sock ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 9
— ϝοηʔδΛsocketʹଓͯ͠ૹ৴ if __name__ == '__main__': while True: try: sock
= socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.connect((HOST, PORT)) print('\nConnected to {}:{}'.format(HOST, PORT)) print("Type message, enter to send, 'q' to quit") msg = input() if msg == 'q': break chatmodule.send_msg(sock, msg) # ૹ৴͢Δ·ͰBlock print('Sent message: {}'.format(msg)) msg = chatmodule.recv_msg(sock) # messageΛશʹड৴͢Δ·ͰBlock print('Received echo: ' + msg) except ConnectionError: print('Socket error') break finally: sock.close() print('Closed connection to server\n') ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 10
— ϝοηʔδΛsocket͔Βड৴ def handle_client(sock, addr): """ sockΛ௨ͯ͡client͔ΒσʔλΛड͚ͱΓ,echoΛฦ͢ """ try: msg
= chatmodule.recv_msg(sock) # messageΛશʹड৴͢Δ·Ͱblock print('{}: {}'.format(addr, msg)) chatmodule.send_msg(sock, msg) # ૹ৴͢Δ·Ͱblock except (ConnectionError, BrokenPipeError): print('Socket error') finally: print('Closed connection to {}'.format(addr)) sock.close() if __name__ == '__main__': listen_sock = chatmodule.create_listen_socket(HOST, PORT) # ιέοτࣗͷΞυϨεΛฦ͢ # ͜ͷؔɺIPv4/v6ιέοτͷϙʔτ൪߸ΛௐΔ߹ͳͲʹ༻ɻ addr = listen_sock.getsockname() print('Listening on {}'.format(addr)) while True: client_sock, addr = listen_sock.accept() print('Connection from {}'.format(addr)) handle_client(client_sock, addr) ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 11
ΫϥΠΞϯτͱαʔόͷιέοτ௨৴ͷಈ͖Λ·ͱΊΔͱ 1. αʔόsocketɺbindɺlistenͰΫϥΠΞϯτͷଓΛͪड͚Δ 2. ଓ͖ͨΒacceptʹΑΓ࣮ࡍͷσʔλͷಡΈग़͠·Ͱͭ 3. σʔλ͕͖ͨΒɺϦΫΤετΛॲཧͯ͠ɺΫϥΠΞϯτʹϨεϙϯεΛฦ͢ 4. ΫϥΠΞϯτͱͷଓΛcloseͰดͯ͡ɺ·ͨacceptͪঢ়ଶʹͳΓϥΠ Ξϯτ͔ΒͷଓΛLISTEN
ΫϥΠΞϯτ͔ΒͷଓΛaccept()ͨ͋͠ͱɺ ϧʔϓΛൈ͚Δ·Ͱ৽نͷΫϥΠΞϯτଓΛϒϩοΫ͍ͯ͠Δ ͜ΕͰ1:1ͷ௨৴͔͠Ͱ͖ͳ͍ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 12
ϚϧνϓϩηεϞσϧ — fork֤ͤͯ͞ϓϩηε͕֤ΫϥΠΞϯτͱ௨৴Ͱ͖Δ — workerϞσϧ — accept()ͨ͋͠ͱͷॲཧΛϓϩηεʹॲཧͤ͞Δ — preforkϞσϧ —
accept()͔Βclose()·ͰͷॲཧΛϓϩηεʹॲཧͤ͞Δ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 13
ίʔυྫ workerϞσϧ while True: client_sock,addr = listen_sock.accept() proc = Process(target=handle_client,
args=[client_sock, addr]) proc.start() print('Connection from {}'.format(addr)) proc.join(1) ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 14
σϝϦοτ:Apache٧·Δ MaxClientsʹୡ͢Δͱɺ accept()͕͞Εͳ͍ͷͰɺ ଓະॲཧͱͳΓ٧·Δ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei (
@oza_shu ) 15
ϚϧνεϨουϞσϧ جຊతʹϚϧνϓϩηε/εϨουಉ͡ - 1ίωΫγϣϯ1εϨουͷϞσϧ - ϦΫΤετ͝ͱʹεϨουΛੜ - εϨουϓʔϧ - ࣄલʹεϨουΛPool͓ͯ͘͠Ϟσϧ
εϨουϝϞϦڞ༗͍ͯ͠ΔͷͰɺεϨουηʔϑͳ࣮͕ ඞཁʹͳΔ (ΩϡʔϩοΫͷॲཧׂѪ) ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 16
ίʔυྫ 1ίωΫγϣϯ1εϨουͷϞσϧ if __name__ == '__main__': listen_sock = chatmodule.create_listen_socket(HOST, PORT)
addr = listen_sock.getsockname() print('Listening on {}'.format(addr)) while True: client_sock,addr = listen_sock.accept() # Thread ࣗಈతʹhandle_client()ؔΛ࣮ߦ͠ɺಉ࣌ʹ͜ͷwhile loopΛ࣮ߦ thread = threading.Thread(target=handle_client, args=[client_sock, addr], daemon=True) thread.start() print('Connection from {}'.format(addr)) ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 17
C10K Πϯλʔωοτ͕ൃలͯ͠Webαʔόʔ͕ಉ࣌ʹ1ສͷΫϥΠΞϯτΛॲཧ͢Δ࣌Ͱ Ϛϧνϓϩηε/εϨουͰॲཧ͕Ͱ͖ͳ͍ͷͰɺ ղܾࡦͱͯ͠ɺΠϕϯτۦಈΞʔΩςΫνϟͰ͋ΔNginx͕ੜ Nginx͕ࢦ͢ͷΠϕϯτϧʔϓͰ1ͭͷεϨουͰສͷಉ࣌ଓΛॲཧ͢Δ͜ͱɻ ͪΖΜϚϧνεϨουɾΞϓϩʔνͰղܾͰ͖ͨΓɺ ଞʹScalingͷͰ͋Δɻ — ΞʔόϯɾΤΞʔγοϓࣾ —
C500kʹ໘༷ͨ͠ — C10Mͷ — ࠓɺ1000ສͷಉ࣌ଓͷઓ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 18
ΠϕϯτۦಈϞσϧ ৽͍͠Πϕϯτ͕ΩϡʔʹೖΕΒΕɺ εϨουΠϕϯτϧʔϓΛ࣮ߦ ΠϕϯτϧʔϓͰͷεϨουΛෳͷଓʹϚοϐϯά ଓɺϦΫΤετ͔Βൃੜͨ͠ΠϕϯτΛॲཧͤ͞Δ ΠϕϯτۦಈܕϓϩάϥϛϯάΛॻ͔ͳ͍ͱཧղͰ͖ͳ ͍ɻɻɻ ٢ࣉ.pm13 2018/02/09 -
Ozawa Shuhei ( @oza_shu ) 19
ΠϕϯτۦಈϞσϧ ࠓޙҎԼௐ͓͖͍ͯͨ - Πϕϯτۦಈܕϓϩάϥϛϯά - ͦΕԿͰ͋ΓɺͲͷΑ͏ʹػೳ͢ΔͷͰ͔͢ʁ - asyncioϞδϡʔϧ - asyncioϕʔεͷϓϩάϥϛϯά
- Twisted - Gevent ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 20
ֶΜͩ͜ͱ — TCPଓͷͳ͕Ε — Apacheͷͭ·ΓͳΜͳͷ͔ — ss,netstat,lsofίϚϯυͷཧղ ٢ࣉ.pm13 2018/02/09 -
Ozawa Shuhei ( @oza_shu ) 21
WEBͷ࣍DB Ͳ͏͢ΕDBͷؾ࣋ͪʹͳΕΔͷ͔ ྑ͍ϝιου͋Εڭ͍͑ͯͩ͘͞ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu
) 22
·ͱΊ ࠓͪΌΜͱௐͯɺͪΌΜͱཧղ͢Δ!! ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu )
23