Save 37% off PRO during our
Black Friday Sale! »
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
High Performance FastAPI
Ikuo Suyama
October 15, 2021
Programming
14
4.9k
High Performance FastAPI
PyCon JP 2021 発表資料です。
Ikuo Suyama
October 15, 2021
Tweet
Share
More Decks by Ikuo Suyama
See All by Ikuo Suyama
martin_lover
0
100
martin_lover
15
5.8k
martin_lover
0
920
martin_lover
19
26k
martin_lover
1
1.9k
martin_lover
0
160
martin_lover
2
3.7k
martin_lover
1
730
martin_lover
0
320
Other Decks in Programming
See All in Programming
yukukotani
2
290
sorami
3
880
aszx87410
2
340
pauloxnet
1
130
cockscomb
0
400
techtekt
0
300
murabayashi
0
380
texmeijin
0
170
tetsunosuke
0
130
appleboy
2
960
whitegreen
0
170
akiroom
0
130
Featured
See All Featured
maltzj
504
37k
danielanewman
6
790
malarkey
193
9k
aarron
261
37k
jmmastey
21
5.1k
robhawkes
56
3.1k
matthewcrist
73
7.7k
mza
82
4.4k
eitanlees
120
11k
keavy
111
15k
cromwellryan
107
6.6k
lara
27
3.5k
Transcript
)JHI1FSGPSNBODF 'BTU"1* *LVP4VZBNB
ಃࢁҭஉ*LVP4VZBNB !NBSUJO@MPWFS@TF 4NBSU/FXT *OD ✤"ET#BDLFOE&OHJOFFS ✤Πϯλʔωοτࠂ͕ઐ ✤zࡾͷ൧ΑΓࠂ͕͖z ✤+7.͍ 1ZUIPOྺ͘Β͍ ✤͓खॊΒ͔ʹʜ🙇
͜ͷηογϣϯ͔ΒಘΒΕΔ͜ͱ ✤ ϓϩϑΝΠϦϯάख๏ͱϘτϧωοΫͷಛఆ ✤զʑͷڥͰൃੜͨ͠۩ମతͳ՝ͱͦͷରࡦ 1ZUIPO8FC"QQMJDBUJPO ύϑΥʔϚϯενϡʔχϯάͷ۩ମతͳख๏
λΠτϧ͔Β͔ΔΑ͏ʹ ͜ͷຊʹ େม͓ੈʹͳΓ·ͨ͠🙇
*OUSPEVDUJPO ෛՙςετͱϓϩϑΝΠϦϯά ໘ͨ͠՝ͱରࡦ ·ͱΊ "HFOEB
*OUSPEVDUJPO ෛՙςετͱϓϩϑΝΠϦϯά ໘ͨ͠՝ͱରࡦ ·ͱΊ "HFOEB
"CPVU4NBSU/FXT ݄ؒ 2,000ສਓ ɹɹɹར༻ऀ 116.7ར༻ ※2 1ਓ͋ͨΓ ʢ20198݄ ถ߹ࢉʣ
No.1 χϡʔεΞϓϦ Ϣʔβʔ※1 ੜ׆ऀͷʮຖͷश׳ʯ ຊ࠷େͷχϡʔεΞϓϦ ※1. Nielsen Mobile NetView 20211݄࣌ ʢεϚʔτχϡʔεͷར༻ऀΛΞϓϦέʔγϣϯϨϕϧͰूܭʣ ※2. ࣗࣾ20211݄ฏۉ
1ZUIPOJO4NBSU/FXT$PVQPO$IBOOFM OFXTTFSWJDF DPVQPOTFSWJDF BQJHBUFXBZ DPVQPOBENJO 3FEJT 3%# 8FCαʔόʔʹ'BTU"1*Λ࠾༻
• 'MBTLMJLFͳΞϊςʔγϣϯϕʔεͷϧʔςΟϯά"1* • 0QFO"1*ରԠ • ؆қతͳ%*ػೳ • ֮͑Δ͜ͱ͕গͳ͘ɺૉૣ͘։ൃΛ্ཱͪ͛ΒΕΔ l'BTU"1*ɺ1ZUIPOͷඪ४Ͱ͋Δܕώϯτʹج͍ͮͯ 1ZUIPOҎ߱Ͱ"1*Λߏங͢ΔͨΊͷɺ
ϞμϯͰɺߴ ߴύϑΥʔϚϯε ͳɺ8FCϑϨʔϜϫʔΫͰ͢ɻz 'BTU"1* αʔϏεͷਝͳ্ཱͪ͛Λ࣮ݱʂ
'BTU"1*ඦฉҰߦʹ͔ͣ
1IBTFE3PMMPVUʜ OFXTTFSWJDF DPVQPOTFSWJDF BQJHBUFXBZ DPVQPOBENJO 3FEJT 3%#
'VMMZ3PMMPVU OFXTTFSWJDF DPVQPOTFSWJDF BQJHBUFXBZ DPVQPOBENJO 3FEJT 3%# εϚχϡʔͷશྗʂʂ
'VMMZ3PMMPVU OFXTTFSWJDF DPVQPOTFSWJDF BQJHBUFXBZ DPVQPOBENJO 3FEJT 3%# εϚχϡʔͷશྗʂʂ 🔥 🔥
🔥 🔥
'VMMZ3PMMPVU OFXTTFSWJDF DPVQPOTFSWJDF BQJHBUFXBZ DPVQPOBENJO 3FEJT 3%# εϚχϡʔͷશྗʂʂ 🔥 🔥
🔥 🔥 ύϑΥʔϚϯενϡʔχϯάͷཱྀʜ
*OUSPEVDUJPO ෛՙςετͱϓϩϑΝΠϦϯά ໘ͨ͠՝ͱରࡦ ·ͱΊ "HFOEB
ύϑΥʔϚϯενϡʔχϯάͷ৺ಘ ̍ɽੑೳඪΛܾΊΔ ̎ɽܭଌɾϘτϧωοΫΛಛఆ͢Δ ̏ɽϘτϧωοΫΛղফ͢Δ ੑೳඪΛୡͨ͠ʁ &/%🎉 :&4 /0
8FC"QQMJDBUJPOʹ͓͚Δʮੑೳʯ ඪ͕ͳ͍ͱແݶʹΓଓ͚ͯ͠·͏ʂ εϧʔϓοτ ୯Ґ࣌ؒʹͲΕ͘Β͍ͨ͘͞Μ ϦΫΤετΛ͚͞Δ͔ʁ ? ָ͗͢͠ΔͷͰ ϨΠςϯγ ̍ͭͷϦΫΤετʹ ͲΕ͘Β͍͕͔͔࣌ؒΔ͔ʁ
ɽੑೳඪΛܾΊΔ
-ʜฏۉͪϓϩηε Еʜ ୯Ґ࣌ؒ͋ͨΓ ౸ண 8ʜฏۉࡏ࣌ؒ 8FC"QQMJDBUJPOੑೳͷʹஔ͖͑Δͱʜ WIP(Parallerism) = Throughput ×
Latency L = λW Ϧτϧͷ๏ଇ ɽੑೳඪΛܾΊΔ
ྫɿ SQTͷγεςϜΛDPSFͰࡹ͖͍ͨ ɹDPSFͷΠϯελϯε ͋Δ͍1PET ΛฒΔͱͯ͠ ̍͋ͨΓ SQTࡹ͚ྑ͍ͷͰ ͭ·Γ̍ΠϯελϯεͷੑೳඪΛ εϧʔϓοτSQT Ҏ্
ɺϨΠςϯγNT ҎԼ ɺͱઃఆ͢Δ 5(WIP)/100rps = 0.05s = 50ms ɿ࣮ࡍʹ$16Λ͍͖ΕΔΘ͚Ͱͳ͍͠ɺ*0$PODVSSFODZ͋Δ͠ɺ ,Tͷ1PEͰར༻Ͱ͖Δ$16ίΞͱMJNJU͕Ұக͠ͳ͍͠ɺ ʜͳͲͳͲͳͲͬͱෳࡶͰ͕͢ɺେମͷ҆ͱͯ͠ ੑೳඪͷܾΊํɿྫ ɽੑೳඪΛܾΊΔ
ϧʔϧϓϩάϥϜ͕Ͳ͜Ͱ࣌ؒΛফඅ͢Δ͜ͱʹͳΔ͔Δ͜ͱͰ͖ͳ͍ɻ ϘτϧωοΫڻ͖͘ՕॴͰى͜ΔͷͰ͋Δɻ ͕ͨͬͯ͠ɺͲ͕͜ϘτϧωοΫͳͷ͔Λ͖ͬΓͤ͞Δ·Ͱɺ ਪଌΛߦͬͨΓɺεϐʔυϋοΫΛͯ͠ͳΒͳ͍ɻ ϧʔϧܭଌ͢͠ɻܭଌ͢Δ·ͰͷͨΊͷௐΛͯ͠ͳΒͳ͍ɻ ίʔυͷҰ෦͕ΓΛѹ͠ͳ͍ͷͰ͋Εɺͳ͓͞ΒͰ͋Δɻ 3PC1JLF/PUFTPO1SPHSBNNJOHJO$ <8JLJQFEJB>ύΠΫ$ϓϩάϥϛϯάʹؔ͢Δ֮͑ॻ͖ ɽܭଌɾϘτϧωοΫΛಛఆ͢Δ ϘτϧωοΫҎ֎νϡʔχϯάͯ͠ແବແବ
ਪଌ͢ΔͳܭଌͤΑ ܭଌͤͣϘτϧωοΫΛΔ͜ͱͰ͖ͳ͍͠ɺ
̍ɽੑೳඪΛܾΊΔ ̎ɽܭଌɾϘτϧωοΫΛಛఆ͢Δ ̏ɽϘτϧωοΫΛղফ͢Δ ੑೳඪΛୡͨ͠ʁ &/%🎉 :&4 /0 ෛՙΛ͔͚ͯ 4ZTUFNશମͷෛՙঢ়گΛ֬ೝ "QQMJDBUJPOͷෛՙঢ়گΛ֬ೝ
-PDBMͰͰ͖ΔݶΓಉ͡ঢ়گΛ࠶ݱ͠ɺ ෛՙΛ͔͚ͭͭܭଌ ϘτϧωοΫͷ ࠷ٙΘ͍͠ՕॴΛ͢ ϘτϧωοΫ͕ղফ͞Ε͔ͨʁ ੑೳ্͕͔ͨ͠ʁ ෛՙࢼݧڥʹ σϓϩΠͯ͠ܭଌ ϘτϧωοΫ "QQMJDBUJPO ෛՙࢼݧʹར༻Ͱ͖Δ ڥΛ४උ͢Δ ɽܭଌɾϘτϧωοΫΛಛఆ͢Δ "QQMJDBUJPO͕ ϘτϧωοΫʹͳΔ·Ͱमਖ਼ :&4 /0 :&4 /0 ܭଌͷखॱͱπʔϧͨͪ Datadog APM • fastapi_profiler • py-spy ϑΟʔυόοΫϧʔϓΛ ૣ͍ͨ͘͠
• 1ZUIPOͷ͍ΘΏΔϕϯνϚʔΫɾෛՙࢼݧπʔϧ • 1ZUIPO"1*ͰγφϦΦΛهड़ • γϯϓϧ͔ͭඞཁेͳઃఆ • ΫϥελԽͯ͠େ͖ͳෛՙʹରԠ -PDVTUJTBOFBTZUPVTF TDSJQUBCMFBOE
TDBMBCMFQFSGPSNBODFUFTUJOHUPPM -0$645 ɽܭଌɾϘτϧωοΫΛಛఆ͢Δ ෛՙΛ͔͚Δɿ-0$645
ෛՙΛ͔͚Δɿ-0$645 ಉ࣌ଓΛࢦఆ rps,median/95% tile latencyΛάϥϑԽ ಉ࣌ଓΛ૿͠ͳ͕ΒϨΠςϯγ͕ѱԽ͢Δݶք·ͰෛՙΛ͔͚ɺ ࠷େεϧʔϓοτͱͦͷ࣌ͷϨΠςϯγΛ֬ೝ ɽܭଌɾϘτϧωοΫΛಛఆ͢Δ
• ෛՙΛ͔͚ͨঢ়ଶͰͷγεςϜશମͷϦιʔεঢ়گΛ֬ೝ • 2ΞϓϦέʔγϣϯ͕ϘτϧωοΫʹͳ͍ͬͯΔ͔ʁ • ϘτϧωοΫ͕ΞϓϦέʔγϣϯʹҠΔ·ͰɺΞϓϦέʔγϣϯͷνϡʔχϯά Λͯ͠ແବແବ • ͍͍ͩͨ%BUBCBTFͳͲͷϛυϧΣΞ͕ઌʹϘτϧωοΫʹͳΔ •
6QTUSFBNͷαʔϏεଓ͢ΔϛυϧΣΞͷϦιʔεʹ༨༟͕͋Δ͕ɺΞϓ Ϧέʔγϣϯͷεϧʔϓοτ্͕͕Βͳ͍ঢ়ଶ·Ͱνϡʔχϯά • ΞϓϦέʔγϣϯҎ֎ͷνϡʔχϯάࠓճׂѪ🙇 γεςϜશମͷෛՙঢ়گΛ֬ೝɿ%BUBEPH ҎޙͷΞϓϦέʔγϣϯ͕ϘτϧωοΫͰ͋Δલఏ ɽܭଌɾϘτϧωοΫΛಛఆ͢Δ
αʔϏεຖͷϦΫΤετ CPU/MemoryͳͲͷར༻ঢ়گΛ֬ೝ ϛυϧΣΞͷCPU Usage ίωΫγϣϯͷมԽΛ֬ೝ ࡉ͔͍ঢ়گίϯςφʹBUUBDIͯ֬͠ೝͨ͠Γ͢Δ͕ɺ શମͷΛ௫Ήͷʹେมศར ɽܭଌɾϘτϧωοΫΛಛఆ͢Δ
• ηοτΞοϓ؆୯ɺαΠυΧʔͱߦͷىಈίʔυ • ࣮ࡍʹՔಇ͍ͯ͠ΔڥͰͷσʔλ͕खʹೖΔ • ඞཁͳใେମἧ͓ͬͯΓɺϘτϧωοΫͷൃݟʹཱͭ ΞϓϦέʔγϣϯܭଌᶃɿ%BUBEPH"1. %BUBEPH"1.ͱ$POUJOVPVT1SPGJMFSɺ8FCαʔϏεɺΩϡʔɺσʔλϕʔε͕ ϦΫΤετɺΤϥʔɺϨΠςϯγʔΛࢹ͢ΔͨΊͷඪ४ͷύϑΥʔϚϯεμογϡ ϘʔυΛ༻ͯ͠ɺΞϓϦέʔγϣϯΛৄࡉʹՄࢹԽ͠·͢ɻ
%BUBEPH"1. ɽܭଌɾϘτϧωοΫΛಛఆ͢Δ
ɽܭଌɾϘτϧωοΫΛಛఆ͢Δ ෛՙڥΞϓϦέʔγϣϯܭଌɿ%BUBEPH"1. Latency histgram CPU time by scripts CPU time
by function ͍͕ͭ͜Ұ൪࣌ؒΛ͓ͬͯΓɺ ϘτϧωοΫީิ ͕ͩߴ͍💸ɺಛఆͷΠϯελϯεͷΈʹదԠ͢ΔͳͲͷΛʜ
-PDBMΞϓϦέʔγϣϯܭଌᶃɿGBTUBQJ@QSPGJMFS ɽܭଌɾϘτϧωοΫΛಛఆ͢Δ "'BTU"1*.JEEMFXBSFPGQZJOTUSVNFOUUP DIFDLZPVSTFSWJDFDPEFQFSGPSNBODF GBTUBQJ@QSPGJMFS • 'BTU"1*ϛυϧΣΞͱͯ͠QZJOTUSVNFOUΛ౷߹ • ϩʔΧϧͰίʔυΛमਖ਼ͭͭ͠ɺ$16࣌ؒΛܭଌ͢Δͱ͖ʹศར •
ύϑΥʔϚϯεӨڹ͕େ͖͍ͨΊɺຊ൪ෛՙঢ়گԼͰ͑ͳ͍
-PDBMΞϓϦέʔγϣϯܭଌᶃɿGBTUBQJ@QSPGJMFS ɽܭଌɾϘτϧωοΫΛಛఆ͢Δ add_middleware ͢Δ͚ͩͰOK sortϝιου͕ ͍ʹϘτϧωοΫީิ
• 3VTUͷϓϩϑΝΠϥɺΦʔόʔϔου • ϩʔΧϧڥෛՙڥͰͬ͞ͱ$16Λܭଌ͍ͨ͠ͱ͖ʹศར • (*-ϚϧνεϨουॲཧͷϘτϧωοΫ͕ٙΘ͍͠ͱ͖ʹศར • GBTUBQJ@QSPGJMFSGBTUBQJཧԼʹͳ͍εϨουΛܭଌͰ͖ͳ͍ʜ -PDBMΞϓϦέʔγϣϯܭଌᶄɿQZTQZ ɽܭଌɾϘτϧωοΫΛಛఆ͢Δ
*UMFUTZPVWJTVBMJ[FXIBUZPVS1ZUIPOQSPHSBN JTTQFOEJOHUJNFPOXJUIPVUSFTUBSUJOHUIF QSPHSBNPSNPEJGZJOHUIFDPEFJOBOZXBZ QZTQZ
͜ͷϝιου͕ΤϯυϙΠϯτ ॲཧͷຆͲͷ࣌ؒΛΊ͍ͯΔ ɽܭଌɾϘτϧωοΫΛಛఆ͢Δ -PDBMΞϓϦέʔγϣϯܭଌᶄɿQZTQZ
*OUSPEVDUJPO ෛՙςετͱϓϩϑΝΠϦϯά ໘ͨ͠՝ͱରࡦ ·ͱΊ "HFOEB
'BTU"1*ͷσϓϩΠ࣌ߏͱෛՙಛੑ (VOJDPSOTVQQPSUTXPSLJOHBTBQSPDFTTNBOBHFSBOEBMMPXJOHVTFSTUPUFMMJUXIJDITQFDJGJDXPSLFS QSPDFTTDMBTTUPVTF5IFO(VOJDPSOXPVMETUBSUPOFPSNPSFXPSLFSQSPDFTTFTVTJOHUIBUDMBTT "OE6WJDPSOIBTB(VOJDPSODPNQBUJCMFXPSLFSDMBTT 4FSWFS8PSLFST(VOJDPSOXJUI6WJDPSO <Process Manager> <ASGI Server> <ASGI
Framework> <Web Framework> ϓϩηεͷىಈͱGPSL TPDLFUΛXPSLFSʹ͢ ϦΫΤετϨεϙϯε ඇಉظॲཧ Request Response Func call callback Extends <Application> Worker ΞϓϦέʔγϣϯ ϏδωεϩδοΫ
'BTU"1*ͷσϓϩΠ࣌ߏͱෛՙಛੑ (VOJDPSOTVQQPSUTXPSLJOHBTBQSPDFTTNBOBHFSBOEBMMPXJOHVTFSTUPUFMMJUXIJDITQFDJGJDXPSLFS QSPDFTTDMBTTUPVTF5IFO(VOJDPSOXPVMETUBSUPOFPSNPSFXPSLFSQSPDFTTFTVTJOHUIBUDMBTT "OE6WJDPSOIBTB(VOJDPSODPNQBUJCMFXPSLFSDMBTT 4FSWFS8PSLFST(VOJDPSOXJUI6WJDPSO <Process Manager> <ASGI Server> <ASGI
Framework> <Web Framework> Request Response Func call callback Extends <Application> Worker ϓϩηεཧͷΈͳͷͰ ύϑΥʔϚϯεӨڹͳ͠ 4UBSMFUUFΑΓૣ ͘ͳΒͳ͍͕ 1ZUIPO"4(*ք ࠷ 4UBSMFUUFΑΓԼ͕ϘτϧωοΫʹͳΔ͜ͱࠓճͳ͔ͬͨ
*0CPVOEͳॲཧ͕͍ $16CPVOEͳॲཧ͕͍ .VMUJ5ISFBE͕͍ ॲཧܥͷมߋʹΑΔશମతͳੑೳ্ ̏ɽ໘ͨ͠՝
*0CPVOEͳॲཧ͕͍ ✤ ໘ͨ͠՝ 3FEJTͷSFBEΞΫηε͕͍ ✤ ঢ় • $16Λ͍ͬͯͳ͍ͷʹεϧʔϓοτ্͕͕Βͳ͍ •
UPQ WNTUBUͰDQVVTΛ֬ೝ ✤ ݪҼ • %#ΞΫηεͳͲɺωοτϫʔΫ*0Λཁ͢Δॲཧ • ϑΝΠϧॻ͖ࠐΈ
• ωοτϫʔΫ͕བྷΉͱ͜ΖBTZODBXBJUΛ͏ • ରԠϥΠϒϥϦଟ͍ • FHBJPSFEJTIUUQYʜ *0CPVOE3FEJTΞΫηε͕͍ ରࡦᶃɿBDZODJPΛ͏ BTZODJPBTZODBXBJUߏจΛ͍ฒߦॲཧͷ ίʔυΛॻͨ͘ΊͷϥΠϒϥϦͰ͢ɻ
BTZODJP
ରࡦᶃɿBDZODJPΛ͏ fastAPIͰϧʔςΟϯά ϝιουΛasyncʹ͢Δ͚ͩ async method ͷ෦Ͱawait *0CPVOE3FEJTΞΫηε͕͍
• ͦΕͰωοτϫʔΫ͕བྷΉͱ͜ΖجຊతʹΊͪΌ͍ • ϝϞϦࢀরͱൺΔͱdഒ͕͔͔ۙ࣌ؒ͘Δ • ݁Ռ߹ੑͷڐ༰͕ߴ͍߹ɺΞϓϦέʔγϣϯϝϞϦʹऔ ಘͨ݁͠ՌΛΩϟογϡ͢Δ ରࡦᶄɿDBDIF͢Δ Latency Comparison
Numbers (~2012) ---------------------------------- : Main memory reference 100 ns : Round trip within same datacenter 500,000 ns 500 us — Latency Numbers Every Programmer Should Know *0CPVOE3FEJTΞΫηε͕͍
3FEJTΞΫηε͕͍ʜରࡦᶄɿDBDIF͢Δ OrderedDictΛར༻ͨ͠ ؆қͳLRUCache Cache͕ͳ͔ͬͨ߹ͷΈ RedisʹΞΫηε method͝ͱΩϟγϡͯ͘͠ΕΔ lru_cacheΞϊςʔγϣϯศར *0CPVOE3FEJTΞΫηε͕͍
✤ ໘ͨ͠՝ 63-Τϯίʔυॲཧ͕͍ ҐஔใͷHFPIBTIͷܭࢉ͕͍ Ґஔใ͔Βͷڑͷܭࢉ͕͍ ✤ ঢ়
• UPQ WNTUBUͰVTFSUJNF͕ுΓ͘ɺ-"͕ߴ͍ • LTڥͰ͋Ε5ISPUUMF͞ΕΔ ✤ ओͳݪҼ • ුಈখܭࢉͳͲγϯϓϧʹॏ͍ܭࢉ • ඇޮͳΞϧΰϦζϜ $16CPVOEͳॲཧ͕͍
ରࡦɿࣄલܭࢉDBDIF͢Δ τϥϯβΫγϣϯ͝ͱʹ݁Ռ͕มΘΒͳ͍ͷ ΦϑϥΠϯͰࣄલʹܭࢉ͓͔ͯ͘͠ɺ ܭࢉ݁ՌΛDBDIF͢Δ͜ͱͰॏ͍ܭࢉࣗମΛόΠύε͢Δ urllib/parse.py Ͱଟ͘ͷ࣌ؒΛ͍ͬͯΔ Urlͷܭࢉ݁ՌΛ Cache͢Δ $16CPVOE63-Τϯίʔυॲཧ
• ϦΫΤετ͝ͱʹΦϯϥΠϯͰܭࢉ͠ͳ͚ΕͳΒͣɺ͔ͭܭࢉෛՙͷ ߴ͍ͷ • (FPIBTIʜҐஔใ MBUMPO ΛಛఆͷྖҬ͝ͱʹIBTIԽ͢Δ • $ݴޠͳͲͷਫ४ݴޠʹ༧ΊίϯύΠϧͯ͠͠·͏ •
ؔ͝ͱͳͲɺ෦తʹదԠ͢Δ͜ͱ͕Մೳ ରࡦɿ$ZUIPO $ZUIPOJTBOPQUJNJTJOHTUBUJDDPNQJMFSGPSCPUIUIF1ZUIPO QSPHSBNNJOHMBOHVBHFBOEUIFFYUFOEFE$ZUIPO QSPHSBNNJOHMBOHVBHF $ZUIPO $16CPVOEHFPIBTIͷܭࢉ͕͍
ࢦఆ͞ΕͨҐஔใ͔Β geohash(maptile)Λܭࢉ͢Δؔ .pyxʹॻ͍ͯίϯύΠϧ͢Δ ݺͼग़͠ଆ௨ৗͷPythonͱಉ͡ ͜Μͳײ͡ͷ setup.pyΛ४උͯ͠ docker build͢Δͱ͖ʹ ίϯύΠϧ͍ͯ͠Δ ରࡦɿ$ZUIPO
$16CPVOEHFPIBTIͷܭࢉ͕͍
• ϦΫΤετͷҐஔใͱɺۙ͘ͷΫʔϙϯͱͷڑͷܭࢉ • ۙ͘ʹ͋ΔΫʔϙϯͷ͚ͩϧʔϓͯ͠ܭࢉ͍ͯͨ͠ • ϧʔϓॲཧΛɺOVNQZͷϕΫτϧܭࢉΛ༻͍ͯ$16DBDIFͷར༻Λ࠷దԽ͢Δ • $16$BDIF͔Βͷϩʔυ3".ΑΓང͔ʹߴ ରࡦɿOVNQZ $16CPVOEڑͷܭࢉ͕͍
Latency Comparison Numbers (~2012) ---------------------------------- L1 cache reference 0.5 ns L2 cache reference 7 ns 14x L1 cache : Main memory reference 100 ns 20x L2 cache, 200x L1 cache — Latency Numbers Every Programmer Should Know
ϕΫτϧԽͯ͠ɺ ҎޙͷܭࢉϕΫτϧܭࢉ ϦΫΤετͷҐஔใͱͦΕͧΕͷ ΫʔϙϯͷڑΛ ΦϯϥΠϯͰܭࢉ͍ͨ͠ ରࡦɿOVNQZ $16CPVOEڑͷܭࢉ͕͍
.VMUJ5ISFBE͕͍ ✤ ໘ͨ͠՝ -PHHJOH͕͍ ✤ ঢ় • 'BTU"1*Ҏ֎ͷ5ISFBEͰ$16Λ͍ͬͯΔ •
%BUBEPH"1.Ͱ֬ೝ • QZTQZUPQͰ(*-ͷ͕͍ ✤ ݪҼ • ࠓճMPHSVͱ͍͏ϥΠϒϥϦ͕.VMUJ5ISFBEͰϩάΛॻ͍͍ͯͯɺͦ ͜ʹ͕͔͔͍࣌ؒͬͯͨ • 1ZUIPO5ISFBEΛѻ͑Δ͕ɺ(*-͕͋ΔͨΊޮ͕ѱ͍
.VMUJ5ISFBE͕͍ *O$1ZUIPO UIFHMPCBMJOUFSQSFUFSMPDL PS(*- JTBNVUFY UIBUQSPUFDUTBDDFTTUP1ZUIPOPCKFDUT QSFWFOUJOHNVMUJQMF UISFBETGSPNFYFDVUJOH1ZUIPOCZUFDPEFTBUPODF (MPCBM*OUFSQSFUFS-PDL (*-(MPCBM*OUFSQSFUFS-PDL
• 5ISFBEΛཱͯͯಉ࣌ʹ࣮ߦ͞ΕΔͷϓϩηεͰ̍ͭ • 1ZUIPOͰฒྻॲཧ.VJUM1SPDFTTJOH ͜Εʹฒߦॲཧ BTZODJPΛΈ߹Θͤͯରॲ͢Δ
.VMUJ5ISFBE͕͍ LogใͷγϦΞϥΠζͱ emmitʹ͕͔͔͍࣌ؒͬͯΔ Queue͔Βऔಘͯ͠σγ ϦΞϥΠζ͍ͯ͠Δ༷ _enqueue͕true(default) ͩͱɺSimpleQueueͱlogग़ྗ༻ ͷThreadΛͨͯΔ multiprocessing.SimpleQueue ͷ࣮ମProcessؒ௨৴ͷPipe
8SJUFS5ISFBEͱͷσʔλड͚͕ͦ͠͏
.VMUJ5ISFBE-PHHJOH͕͍ ରࡦɿϩΪϯάͷ੍ͱϥΠϒϥϦͷมߋ w GBTUMPHHJOHʹΓ͑ͨ w ͦͦ͜͜༰ྔͷେ͖͍MPHΛు͖͍͕ͨɺະղܾ
• ීஈ͍ͬͯΔ$1ZUIPOҎ֎ʹ༷ʑͳॲཧܥ͕͋Δ • ڃݴޠʹίϯύΠϧ͢Δͷɺ+*5ίϯύΠϥΛ࣋ͭͷɺFUDʜ • ࠓճͭΛࢼͯ͠ɺ1Z1ZΛ࠾༻ͨ͠ DJOEFS 1Z1Z
• $16͕ϘτϧωοΫͰ͋Εɺશମతͳੑೳ্͕ظͰ͖Δ ॲཧܥͷมߋʹΑΔશମతͳੑೳ্
ॲཧܥͷมߋDJOEFS $JOEFSJT*OTUBHSBNTJOUFSOBMQFSGPSNBODFPSJFOUFEQSPEVDUJPO WFSTJPOPG$1ZUIPO *UDPOUBJOTBOVNCFSPGQFSGPSNBODFPQUJNJ[BUJPOT JODMVEJOH CZUFDPEFJOMJOFDBDIJOH FBHFSFWBMVBUJPOPGDPSPVUJOFT ʜ DJOEFS •
'BDFCPPLۘͷύϑΥʔϚϯεվળ͞Εͨ$1ZUIPO • զʑͷڥͰɺ ఔͷੑೳ্͕ݟΒΕͨ • $1ZUIPOͱͷޓੑ͕ߴ͘ɺϥΠϒϥϦ΄ͱΜͲ͑Δ • ͕ɺࣗલͰϏϧυ͢Δඞཁ͕͋ͬͨΓ(JU)VC׆ൃͰͳ͔ͬͨΓυΩϡϝϯτ͕օແͩͬͨ Γͱɺӡ༻໘ͰͭΒ͘அ೦
ॲཧܥͷมߋ1Z1Z "GBTU DPNQMJBOUBMUFSOBUJWFJNQMFNFOUBUJPOPG1ZUIPO 0OBWFSBHF 1Z1ZJTUJNFTGBTUFS UIBO$1ZUIPO 1Z1Z • +*5ίϯύΠϥ*ODNJOJNBSL($
• $1ZUIPOͱͷޓੑ͔ͳΓߴ͍ • ͪΖΜσϝϦοτͦΕͳΓʹ͋Δ • ͕ɺੑೳ໘ͰͷϝϦοτ͕େ͖͔ͬͨͨΊɺݱࡏຊ൪Քಇத • ͍ۙੑೳ্Λ֬ೝ
ۜͷؙͰͳ͍ ✤ ࡌͤସ͑ɾӡ༻ʹ͋ͨͬͯ໘ͨ͠՝ ࠷৽൛͕1ZUIPO Ұ෦ͷϥΠϒϥϦ͕͑ͳ͍ ($0QUJPOࢦఆ࿙ΕͰ00.ࢮ
'BTU"1*ͱͷΈ߹ΘͤͰϝϞϦϦʔΫʁ ॲཧܥͷมߋ1Z1Z
՝ᶃɿ࠷৽൛͕1ZUIPO ✤ ՝ɿݱࡏͷ࠷৽൛͕1ZUIPO • CFUB DPNNJOHTPPO ✤ ରԠ • ϕʔεͰ։ൃ͍ͯͨ͠ͷͰɺҰ෦ͷՕॴΛμϯάϨʔυ
• ηΠνԋࢉࢠ:= • Ґஔઐ༻Ҿdef huga(hoge, /, …) • ࣮࣭͋Μ·Γࠔ͍ͬͯͳ͍ ॲཧܥͷมߋ1Z1Z
՝ᶄɿҰ෦ͷϥΠϒϥϦ͕͑ͳ͍ ✤ ՝ɿ1&1֎෦ιʔεͷϏϧυΛ͏ϥΠϒϥϦ΄΅/( • FH • PSKTPOʜSVTUߴ+40/γϦΞϥΠβ😭 • EEUSBDFQZʜ%BUB%PH"1.ܭଌ༻😭 •
GBTUBQJQSPGJMFSʜ͜͜·Ͱେ׆༂ͷϓϩϑΝΠϥ😭 • CMBDL DBVTFECZUZQFEBTU ʜ-JOUFS'PSNBUUFS QZTFOܦ༝Ͱར༻😭 ✤ ରԠ • ผͷϥΠϒϥϦΛ͏ • ։ൃڥ༻ͱσϓϩΠ༻ͰผʑͷॲཧܥΛར༻͢Δ ॲཧܥͷมߋ1Z1Z
՝ᶄɿҰ෦ͷϥΠϒϥϦ͕͑ͳ͍ $1ZUIPOͰಈ͕͘1Z1ZͰಈ͔ͳ͍ͱ͍ͬͨෆ۩߹Λճආ͢ΔͨΊɺ $%ͷࡍʹຊ൪༻ίϯςφΛ༻͍ͯFFςετΛ࣮ࢪ͍ͯ͠Δ ։ൃ༻ / CPython3.7 CI্ͰͷUTͪ͜ΒͰ࣮ߦ ຊ൪༻ / PyPy3.7
ॲཧܥͷมߋ1Z1Z
՝ᶅɿ($0QUJPOࢦఆ࿙ΕͰ00.ࢮ ✤ ՝ɿϝϞϦ͕ϞϦϞϦ͠ɺ00.,JMMFSʹΑΓࢮ • 1Z1Zͷ($JODNJOJNBSL • ($0QUJPOͷઃఆ͕ඞཁ ✤ ରԠ •
1:1:@($@%( Λࢦఆͯ͠($-PHΛग़ྗ͢Δ • 1:1:@($@."9Λࢦఆͯ͠ώʔϓ্ݶΛ੍ݶ͢Δ ॲཧܥͷมߋ1Z1Z
՝ᶅɿ($0QUJPOࢦఆ࿙ΕͰ00.ࢮ MaxWorker x PYPY_GC_MAX Λ֬อ ࠷ݶ PYPY_GC_MAXࢦఆ ॲཧܥͷมߋ1Z1Z
՝ᶆɿ'BTU"1*ͱͷΈ߹ΘͤͰϝϞϦϦʔΫʁ ✤ ՝ɿϝϞϦ͕ϞϦϞϦ͠ɺ00.,JMMFSͰࢮʢ̎ʣ • &DIPαʔόʔͰɺෛՙΛ͔͚ΔͱϝϞϦ͕୯ௐ૿Ճ͢Δ • Ϧόϓϩ͍ͯͨ͠OHJOYͰ,FFQ"MJWF͍ͯ͠ͳ͔ͬͨʜ ✤ ରԠ •
)551ଓʢ%FGBVMU,FFQ"MJWFʣʹ͢Δͱղফͨ͠ • VWJDPSOͷUJNFPVULFFQBMJWFઃఆ͓ͯ͘͠ • ͋·Γਂ͍͍ͯ͠ͳ͍͕ɺDPOOFDUJPOपΓͷΆ͍ͷͰɺ 'BTU"1*ͱ͍͏ΑΓVWJDPSOͷ͔ ॲཧܥͷมߋ1Z1Z
̏ɽ໘ͨ͠՝ <5ISPVHIQVU> SQT <UJMF-BUFODZ> NT <5ISPVHIQVU> SQT ‐ <UJMF-BUFODZ>
NT ‐ ͯ͢ରԠͨ͠ޙɺΠϯελϯεͰͷੑೳվળ
*OUSPEVDUJPO ෛՙςετͱϓϩϑΝΠϦϯά ໘ͨ͠՝ͱରࡦ ·ͱΊ "HFOEB
·ͱΊ ✤ ෛՙςετͱϓϩϑΝΠϦϯά • ඪΛܾΊΔˠܭଌ͢ΔˠϘτϧωοΫʹରԠ͢Δ • ཱͭπʔϧͨͪɿMPDVTU EBUBEPH GBTUBQJQSPGJMFS QZTQZ
✤ ՝ͱରԠ • *0CPVOEBTZODJP DBDIF • $16CPVOEDBDIF $ZUIPO OVNQZ • ॲཧܥͷมߋ1Z1Z
)JHI1FSGPSNBODFͳ 'BTU"1*-JGFΛ
Thank You for Your Kind Attention! UXJUUFS!NBSUJO@MPWFS@TF
martin_lover
yukukotani
sorami
aszx87410
pauloxnet
cockscomb
techtekt
murabayashi
texmeijin
tetsunosuke
appleboy
whitegreen
akiroom
maltzj
danielanewman
malarkey
aarron
jmmastey
robhawkes
matthewcrist
mza
eitanlees
keavy
cromwellryan
lara
