Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Python for web architectures
Search
Shuhei Ozawa
February 09, 2018
0
910
Python for web architectures
Python Programming for web architectures
Shuhei Ozawa
February 09, 2018
Tweet
Share
More Decks by Shuhei Ozawa
See All by Shuhei Ozawa
Amebaアフィリエイト基盤の GKEアーキテクチャと マイクロサービス
ozashu
0
190
production_ready_envoy
ozashu
2
1.1k
ログ・係数集約と可視化・分析
ozashu
0
140
PyQではじめるPython
ozashu
0
420
インフラエンジニアのWEBアプリ入門
ozashu
1
8k
Featured
See All Featured
Understanding Cognitive Biases in Performance Measurement
bluesmoon
29
1.8k
Connecting the Dots Between Site Speed, User Experience & Your Business [WebExpo 2025]
tammyeverts
5
260
KATA
mclloyd
30
14k
Bootstrapping a Software Product
garrettdimon
PRO
307
110k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
53
2.8k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
231
53k
Why Our Code Smells
bkeepers
PRO
337
57k
Documentation Writing (for coders)
carmenintech
72
4.9k
Being A Developer After 40
akosma
90
590k
Making Projects Easy
brettharned
116
6.3k
Into the Great Unknown - MozCon
thekraken
39
1.9k
Build your cross-platform service in a week with App Engine
jlugia
231
18k
Transcript
PythonͰWebΞʔΩςΫνϟΛֶͿ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 1
ࣗݾհ — খपฏ — @oza_shu — MSPۀքͰWEBαʔϏεͷαʔόӡ༻ ٢ࣉ.pm13 2018/02/09 -
Ozawa Shuhei ( @oza_shu ) 2
٢ࣉ.pm13ͷςʔϚ — ʮ৽͍͠ઓɺ৽͍͠ࢹʯ — ৽ʹ͚ܾͨҙత ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei
( @oza_shu ) 3
2018ͷ๊ෛ ͪΌΜͱௐͯͪΌΜͱཧղ͢Δ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu )
4
ͪΌΜͱௐͯɺͪΌΜͱཧղ͢Δ αʔόͷӡ༻ͰোରԠ͢Δ࣌ɺͳΜͱͳ͘ͰରԠ͢Δ͜ͱ — ͜ͷϩάग़ྗͰݕࡧͯ͠ɺQiitaΈͨΓͱ͔... — ͦͷQiitaͱ͔ಡΜͰ;ΜΘΓཧղͩͬͨΓͱ͔... ٢ࣉ.pm13 2018/02/09 - Ozawa
Shuhei ( @oza_shu ) 5
ཧతղܾΛࢦ͢ — ͪΌΜͱௐͯͪΌΜͱཧղ͢Δ — ՝Λղܾ͢Δ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei
( @oza_shu ) 6
ྫ͑ ͜ΕΒWebΞʔΩςΫνϟΛPythonͰ͓͞Β͍ - TCPίωΫγϣϯͲΜͳ௨৴Λ͍ͯ͠Δͷ͔ - WEBαʔόͷಈ͖ɺߏཧղ - Apache͕٧·ΔͬͯͲ͏͍͏͜ͱ - C10Kͱ
٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 7
TCP/IP௨৴ — IPΞυϨεͱϙʔτΛࢦఆͯ͠ίωΫγϣϯଓΛ͢Δ — TCP jokeSYN→SYN/ACK→ACK — socket(),bind(),listen(),accept(),connect(),write(),read(),close() ٢ࣉ.pm13 2018/02/09
- Ozawa Shuhei ( @oza_shu ) 8
γϦΞϧϞσϧ — socket(),bind(),listen()Ͱιέοτͷ࡞ def create_listen_socket(host, port): """ αʔόʔ͕ଓཁٻΛड͚औΔιέοτΛઃఆ͢Δ """ #
ΞυϨεϑΝϛϦʔɺιέοτλΠϓɺϓϩτίϧ൪߸Λࢦఆͯ͠৽͍͠ιέοτΛ࡞ sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) sock.bind((host, port)) sock.listen(100) return sock ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 9
— ϝοηʔδΛsocketʹଓͯ͠ૹ৴ if __name__ == '__main__': while True: try: sock
= socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.connect((HOST, PORT)) print('\nConnected to {}:{}'.format(HOST, PORT)) print("Type message, enter to send, 'q' to quit") msg = input() if msg == 'q': break chatmodule.send_msg(sock, msg) # ૹ৴͢Δ·ͰBlock print('Sent message: {}'.format(msg)) msg = chatmodule.recv_msg(sock) # messageΛશʹड৴͢Δ·ͰBlock print('Received echo: ' + msg) except ConnectionError: print('Socket error') break finally: sock.close() print('Closed connection to server\n') ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 10
— ϝοηʔδΛsocket͔Βड৴ def handle_client(sock, addr): """ sockΛ௨ͯ͡client͔ΒσʔλΛड͚ͱΓ,echoΛฦ͢ """ try: msg
= chatmodule.recv_msg(sock) # messageΛશʹड৴͢Δ·Ͱblock print('{}: {}'.format(addr, msg)) chatmodule.send_msg(sock, msg) # ૹ৴͢Δ·Ͱblock except (ConnectionError, BrokenPipeError): print('Socket error') finally: print('Closed connection to {}'.format(addr)) sock.close() if __name__ == '__main__': listen_sock = chatmodule.create_listen_socket(HOST, PORT) # ιέοτࣗͷΞυϨεΛฦ͢ # ͜ͷؔɺIPv4/v6ιέοτͷϙʔτ൪߸ΛௐΔ߹ͳͲʹ༻ɻ addr = listen_sock.getsockname() print('Listening on {}'.format(addr)) while True: client_sock, addr = listen_sock.accept() print('Connection from {}'.format(addr)) handle_client(client_sock, addr) ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 11
ΫϥΠΞϯτͱαʔόͷιέοτ௨৴ͷಈ͖Λ·ͱΊΔͱ 1. αʔόsocketɺbindɺlistenͰΫϥΠΞϯτͷଓΛͪड͚Δ 2. ଓ͖ͨΒacceptʹΑΓ࣮ࡍͷσʔλͷಡΈग़͠·Ͱͭ 3. σʔλ͕͖ͨΒɺϦΫΤετΛॲཧͯ͠ɺΫϥΠΞϯτʹϨεϙϯεΛฦ͢ 4. ΫϥΠΞϯτͱͷଓΛcloseͰดͯ͡ɺ·ͨacceptͪঢ়ଶʹͳΓϥΠ Ξϯτ͔ΒͷଓΛLISTEN
ΫϥΠΞϯτ͔ΒͷଓΛaccept()ͨ͋͠ͱɺ ϧʔϓΛൈ͚Δ·Ͱ৽نͷΫϥΠΞϯτଓΛϒϩοΫ͍ͯ͠Δ ͜ΕͰ1:1ͷ௨৴͔͠Ͱ͖ͳ͍ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 12
ϚϧνϓϩηεϞσϧ — fork֤ͤͯ͞ϓϩηε͕֤ΫϥΠΞϯτͱ௨৴Ͱ͖Δ — workerϞσϧ — accept()ͨ͋͠ͱͷॲཧΛϓϩηεʹॲཧͤ͞Δ — preforkϞσϧ —
accept()͔Βclose()·ͰͷॲཧΛϓϩηεʹॲཧͤ͞Δ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 13
ίʔυྫ workerϞσϧ while True: client_sock,addr = listen_sock.accept() proc = Process(target=handle_client,
args=[client_sock, addr]) proc.start() print('Connection from {}'.format(addr)) proc.join(1) ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 14
σϝϦοτ:Apache٧·Δ MaxClientsʹୡ͢Δͱɺ accept()͕͞Εͳ͍ͷͰɺ ଓະॲཧͱͳΓ٧·Δ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei (
@oza_shu ) 15
ϚϧνεϨουϞσϧ جຊతʹϚϧνϓϩηε/εϨουಉ͡ - 1ίωΫγϣϯ1εϨουͷϞσϧ - ϦΫΤετ͝ͱʹεϨουΛੜ - εϨουϓʔϧ - ࣄલʹεϨουΛPool͓ͯ͘͠Ϟσϧ
εϨουϝϞϦڞ༗͍ͯ͠ΔͷͰɺεϨουηʔϑͳ࣮͕ ඞཁʹͳΔ (ΩϡʔϩοΫͷॲཧׂѪ) ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 16
ίʔυྫ 1ίωΫγϣϯ1εϨουͷϞσϧ if __name__ == '__main__': listen_sock = chatmodule.create_listen_socket(HOST, PORT)
addr = listen_sock.getsockname() print('Listening on {}'.format(addr)) while True: client_sock,addr = listen_sock.accept() # Thread ࣗಈతʹhandle_client()ؔΛ࣮ߦ͠ɺಉ࣌ʹ͜ͷwhile loopΛ࣮ߦ thread = threading.Thread(target=handle_client, args=[client_sock, addr], daemon=True) thread.start() print('Connection from {}'.format(addr)) ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 17
C10K Πϯλʔωοτ͕ൃలͯ͠Webαʔόʔ͕ಉ࣌ʹ1ສͷΫϥΠΞϯτΛॲཧ͢Δ࣌Ͱ Ϛϧνϓϩηε/εϨουͰॲཧ͕Ͱ͖ͳ͍ͷͰɺ ղܾࡦͱͯ͠ɺΠϕϯτۦಈΞʔΩςΫνϟͰ͋ΔNginx͕ੜ Nginx͕ࢦ͢ͷΠϕϯτϧʔϓͰ1ͭͷεϨουͰສͷಉ࣌ଓΛॲཧ͢Δ͜ͱɻ ͪΖΜϚϧνεϨουɾΞϓϩʔνͰղܾͰ͖ͨΓɺ ଞʹScalingͷͰ͋Δɻ — ΞʔόϯɾΤΞʔγοϓࣾ —
C500kʹ໘༷ͨ͠ — C10Mͷ — ࠓɺ1000ສͷಉ࣌ଓͷઓ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 18
ΠϕϯτۦಈϞσϧ ৽͍͠Πϕϯτ͕ΩϡʔʹೖΕΒΕɺ εϨουΠϕϯτϧʔϓΛ࣮ߦ ΠϕϯτϧʔϓͰͷεϨουΛෳͷଓʹϚοϐϯά ଓɺϦΫΤετ͔Βൃੜͨ͠ΠϕϯτΛॲཧͤ͞Δ ΠϕϯτۦಈܕϓϩάϥϛϯάΛॻ͔ͳ͍ͱཧղͰ͖ͳ ͍ɻɻɻ ٢ࣉ.pm13 2018/02/09 -
Ozawa Shuhei ( @oza_shu ) 19
ΠϕϯτۦಈϞσϧ ࠓޙҎԼௐ͓͖͍ͯͨ - Πϕϯτۦಈܕϓϩάϥϛϯά - ͦΕԿͰ͋ΓɺͲͷΑ͏ʹػೳ͢ΔͷͰ͔͢ʁ - asyncioϞδϡʔϧ - asyncioϕʔεͷϓϩάϥϛϯά
- Twisted - Gevent ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu ) 20
ֶΜͩ͜ͱ — TCPଓͷͳ͕Ε — Apacheͷͭ·ΓͳΜͳͷ͔ — ss,netstat,lsofίϚϯυͷཧղ ٢ࣉ.pm13 2018/02/09 -
Ozawa Shuhei ( @oza_shu ) 21
WEBͷ࣍DB Ͳ͏͢ΕDBͷؾ࣋ͪʹͳΕΔͷ͔ ྑ͍ϝιου͋Εڭ͍͑ͯͩ͘͞ ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu
) 22
·ͱΊ ࠓͪΌΜͱௐͯɺͪΌΜͱཧղ͢Δ!! ٢ࣉ.pm13 2018/02/09 - Ozawa Shuhei ( @oza_shu )
23