API Gateway / Lambda / Kinesis を使ったストリーミングなバッチ実行基盤の実装

"1*(BUFXBZ-BNCEB,JOFTJT
Λ࢖ͬͨετϦʔϛϯάͳ
όον࣮ߦج൫ͷ࣮૷
ࣰా݈
גࣜձࣾϦϒηϯε

View Slide

ࣗݾ঺հ
w ࣰా݈ ͠ͷ͚ͩͨ͠

w !UBLFTIJOPEB
w 3VCZJTU
w "TBLVTBSC
w גࣜձࣾϦϒηϯε
w EPPS௞ିϓϩμΫτάϧʔϓάϧʔϓϦʔμ

View Slide

ΑΖ͘͠
͓Ͷ͕͍͠·͢

View Slide

௞ିෆಈ࢈ϝσΟΞ

View Slide

఻౷తͳόονδϣϒߏ੒
ը૾഑৴
αʔό
֤छ%#
ॲཧ" ॲཧ# ॲཧ$
֤෺݅৘ใ
ఏڙݩ

View Slide

࠷ॳͷ͏ͪ͸ѱ͘ͳ͍
w ։ൃ͸؆୯
w αʔϏεͷ֦େ܏޲΋ಡΊͳ͍
w ֶशίετ΋ͦ͏ߴ͘ͳ͍
w Φʔόʔϔου͸δϣϒؒͷड͚౉͘͠Β͍ͳͷͰɺ
࣮ߦ଎౓΋ͦͦ͜͜ૣ͍
w ॳظͷ࣮૷ͱͯ͠͸ී௨

View Slide

෺݅਺֦େͱڞʹ๚ΕΔਏΈ
֤෺݅৘ใ
ఏڙݩ
ը૾഑৴
αʔό
֤छ%#
ॲཧ" ॲཧ# ॲཧ$
֤ࣾҧ͏ϑΥʔϚοτ
ऴΘΒͳ͍લஈδϣϒ
αʔόͷ͓कΓ
ϩʔυ࣌ؒͷ֦େ
લஈδϣϒʹґଘͨ͠
ॲཧ಺༰

View Slide

༷ʑͳ՝୊ͷൃੜ
w ֤ࣾϑΥʔϚοτΛѻ͏͜ͱʹΑΔʰҧ͍ͷٵऩ૚ʱͷෳࡶԽ
w Φʔόʔϔουߴͯ͘΋ྑ͍ͷͰ෼ࢄॲཧ͍ͤͨ͞
w δϣϒ͕ஈ֊Λ͍࣋ͬͯΔͷͰεέʔϧͤ͞ʹ͍͘
w ଟॏ࣮ߦ࣌ͷ଴ͪ߹Θͤͱͦͷ࢓૊ΈͷՄൖੑ
w ͚ͨ͜ͱ͖ͷϦϥϯͷෳࡶԽ
w લஈδϣϒʹґଘͨ͠ॲཧػߏͷͨΊɺαʔϏεվળ͕େม
w ΤϯςΟςΟͰ਺ඦສ෺݅ͷ֤छσʔλετΞϛυϧ΢ΣΞͷ
σʔλͷೖΕସ͑
w αʔόϝϯςφϯεͷෳࡶԽ
w όονʹΑΔαʔϏε಺༰ͷྼԽ

View Slide

-BNCEBͰͷόον
εέʔϥϏϦςΟͷ֬อ
αʔόͷϓϩϏδϣχϯάΛग़དྷΔ͚ͩ͠ͳ͍
֤δϣϒ͸εςʔτϨεʹ͢Δ
wଞδϣϒͷ݁Ռʹࠨӈ͞Εͳ͍
wೖྗͱग़ྗΛಉ͡ʹ͢Δ
wΤϯςΟςΟ୯ҐͰೖग़ྗ
ϦΞϧλΠϜॲཧ΁ͷҠߦ
ઃܭͷࢦ਑

View Slide

εέʔϥϏϦςΟͷ֬อ
w ෺݅਺ͷ֦େʹ଱͑Δ
w ӡ༻ෛՙͷܰݮ
w ෺݅਺͕૿͑Δຖʹ߄ͯͳ͍
w ෺݅਺͸قઅมಈ͢Δ
w ෺݅਺͸৘ใఏڙݩͷࣄۀઓུͰมΘΔ

View Slide

w ϓϩϏδϣχϯάίʔυͷϝϯςφϯεෛՙͷ௿ݮ
w εέʔϧͰ͖Δج൫ͷ࢓૊ΈΛͦͷ··࢖͍͍ͨ
w αʔόߏ੒ʹ͍ͭͯݟΔ࣌ؒΛݮΒ͠ɺϏδωεʹ஫ྗ͍ͨ͠
w ֤ࣾϑΥʔϚοτͷରԠͰαʔόͷࠩҟΛੜΈͨ͘ͳ͍
w $*$%πʔϧઃఆෛՙͷ௿ݮ
w ֤δϣϒͷಠཱੑΛߴΊΔ ޙड़
͜ͱͰɺঃʑʹαʔόϩʔϧ
͕૿͑ͯ͠·͏͜ͱͷ཈੍
αʔόͷϓϩϏδϣχϯά
ɹΛग़དྷΔ͚ͩ͠ͳ͍

View Slide

wଞδϣϒͷ݁Ռʹࠨӈ͞Εͳ͍
wೖྗͱग़ྗΛಉ͡ʹ͢Δ
wΤϯςΟςΟ୯ҐͰೖग़ྗ
֤δϣϒ͸εςʔτϨεʹ͢Δ
w ଞδϣϒͷ࣮ߦ݁ՌΛߟྀͤͣδϣϒΛ͘ΊΔ
w ଟॏॲཧΛ΍Γ΍͘͢ͳΔ
w ΤϯςΟςΟ୯ҐͰೖग़ྗॲཧ͢Δ͜ͱͰɺঢ়ଶΛͲ
͔͜ʹ഑ஔͨ͠Γࢀরͨ͠Γ͠ͳ͍Α͏ʹ͢Δ

View Slide

ϦΞϧλΠϜॲཧ΁ͷҠߦ
w όον࿈ܞͷଞʹ"1*ͷΑ͏ͳܗͰϦΞϧλΠϜͰγε
ςϜʹ൓ө͢ΔޱΛ༻ҙ͓͖͍ͯͨ͠
w ೥Ͱ͕͢ɺόον࿈ܞ͕ओྲྀͳͷͰ͢
w ϦΞϧλΠϜJTԿ ࿦૪͕5XJUUFͰ͋Γ·͕ͨ͠ɺόο
ν͡Όແ͍͘Β͍ʹͱΒ͑ͯͩ͘͞Εʜ

View Slide

શମߏ੒ઃܭ

View Slide

ઃܭҊओʹ&MBTUJD#FBOTUBMLͷ8PSLFSΛ࢖͏
4ʹதؒॲཧ݁ՌΛ֨ೲ͠ɺ෼ࢄͷ୯Ґ͸4ʹ֨ೲ͞Εͨ
ΦϒδΣΫτͷ਺Ͱ੍ޚ͢Δ ޙड़
ɻ
ॲཧྔͷͿΕ͸424ͰٵऩͰ͖Δɻ
֤෺݅৘ใ
ఏڙݩ
࿈ܞαʔό
ݩσʔλ
ม׵δϣϒ#
ม׵݁Ռ
&MBTUJD#FBOTUBML
8PSLFS

δϣϒ$
ม׵݁Ռ
ɾɾɾ
3%4
ը૾഑৴4
$MPVE4FBSDI
ม׵δϣϒ"
165 165
δϣϒ%
ม׵݁Ռ
ॻ͖ࠐΈ
δϣϒ

View Slide

ઃܭҊQSPTDPOT
QSPT
w Λ࢖͏ͷͰར༻ݴޠͳͲ͕ൺֱతࣗ༝
w Λ࢖͏ͷͰϓϩϏδϣχϯά͸࠷௿ݶͰ͢Ή
w 4ʹॲཧ݁Ռ͕֨ೲ͞Ε͍ͯΔͷͰͦͷؾʹͳΕ͹Ϧϥϯͷ࢓૊Έ
Λ࡞Γ΍͍͢
DPOT
w ॲཧ෼ࢄͷ୯Ґ͕4ͷΦϒδΣΫτ୯Ґʹͳͬͯ͠·͍ɺ4ΦϒδΣ
Ϋτͦͷ΋ͷʹ੍ޚͷҙຯΛՃ͑ͯ͠·͏͜ͱͰɺγεςϜશମͷ
ཧղ΍੍ޚʹෳࡶੑΛ΋ͨΒͦ͠͏
w σϑΥϧτͰͳΜͱͳΔࣄ͸গͳ͘ɺ݁ہFCFYUFOTJPOTͷԼ͕
໘౗ͳ͜ͱʹͳΔ͜ͱ͕༧૝͞ΕΔ
w &$͕୔ࢁཱ͓͕͔͔ͬͯۚΔ

View Slide

DPOTͷํ͕ڧ͍ײ͡
͕ͯ͠೰·͍͠

View Slide

ઃܭҊओʹ-BNCEBΛ࢖͏
தؒॲཧ݁Ռ͸࢒ͣ͞ɺ-BNEBͰͭͳ͍Ͱ͍͘ɻ
֤෺݅৘ใ
ఏڙݩ
࿈ܞαʔό
ݩσʔλ
-BNCEB
ॲཧ$
3%4
ը૾഑৴4
$MPVE4FBSDI
ม׵ॲཧ"
165
ॲཧ%
ม׵ॲཧ#
ॻ͖ࠐΈ
δϣϒ

View Slide

ઃܭҊQSPTDPOT
QSPT
w αʔόͷϓϩϏδϣχϯά͸ෆཁ
w 424͕ແͯ͘΋ॲཧྔ͸͋Δఔ౓༨༟ݟͯϦϛοτղআ͓͚ͯ͠͹
͍͍
w ҰͭͷॲཧΛ؆ܿʹ؅ཧग़དྷΔ
w ॲཧͷଟॏ౓͸࣍ஈͷ-BNCEBͷݺͼํͰ੍ޚͰ͖Δ ·ͱΊͯΤ
ϯςΟςΟ͚ͩ౉͔͢ΤϯςΟςΟʹ͢Δ͔

DPOT
w ॲཧ͕ଟ͘ͷ-BNEBΛଟஈͰར༻͢ΔͷͰςετ͕େม
w ϩΪϯάͷ࢓૊ΈΛ੔͓͔͑ͯͳ͍ͱΤϥʔ࣌ͷ௥੻͕େม
w -BNCEBຖʹϦϙδτϦΛ෼͚ͳ͍ͱ$*$%͕େม
w ॻ͖ࠐΈετϨʔδ͕૿͑ΔͳͲͰɺ࣍ஈͷ-BNEB͕૿͑Δͱݩ
-BNCEBʹखΛՃ͑ͳ͍ͱ͍͚ͳ͍

View Slide

-BNCEBͷݺͼग़͠ॱ
ͷґଘ͕ڧ͍ؾ͕͢Δ

View Slide

ઃܭҊ"1*(8ͱ,JOFTJTͰϨΠϠʔΛ໌֬ʹ͢Δ
ΞϓϦέʔγϣϯͷϨΠϠʔΛҙࣝͤ͞Δɻ
֤෺݅৘ใ
ఏڙݩ
࿈ܞαʔό
ݩσʔλ
-BNCEB
ॲཧ$
3%4
ը૾഑৴4
$MPVE4FBSDI
ม׵ॲཧ"
165
ॲཧ%
ม׵ॲཧ#
ॻ͖ࠐΈ
δϣϒ
"1*(BUFXBZ

View Slide

֤෺݅৘ใ
ఏڙݩ
࿈ܞαʔό
ݩσʔλ
-BNCEB
ॲཧ$
3%4
ը૾഑৴4
$MPVE4FBSDI
ม׵ॲཧ"
165
ॲཧ%
ม׵ॲཧ#
ॻ͖ࠐΈ
ॲཧ
"1*(BUFXBZ
ೖྗ૚
ΞϓϦέʔγϣϯͷϨΠϠʔΛҙࣝͤ͞Δ
αʔϏε૚ ετΞ૚
,JOFTJT

View Slide

ઃܭҊQSPTDPOT
QSPT
w ઃܭҊͷར఺ΛڗडͰ͖Δ
w ϨΠϠʔΛҙࣝͤ͞Δ͜ͱ͕Ͱ͖ɺΞϓϦέʔγϣϯͷෳࡶԽΛ཈੍Ͱ
͖Δ
w ॻ͖ࠐΈ෦෼͸ετϨʔδͷੑೳʹ߹ΘͤͯLJOFTJTͰྲྀྔΛௐ੔Ͱ͖Δ
w "1*(8Λ௥Ճ͢Δ͜ͱͰɺόονͰ͸ແ͍࿈ܞػೳΛ௥Ճ͢Δඪ४Խ
͞ΕͨΠϯλʔϑΣʔεΛఆٛͰ͖Δ
DPOT

View Slide

εέʔϥϏϦςΟ ෳࡶ͞ ॊೈੑ
Ҋ ˚ ʓ º
Ҋ ʓ ʓ ˚
Ҋ ʓ ʓ ʓ

View Slide

શମతͳQSPTDPOT
QSPT
w όονશମͷॲཧ࣌ؒ͸Φʔόϔου෼૿Ճ͢Δ͕ɺ࿈ܞ࣌ࠁΛ߹
ΘͤΔඞཁ͕ͳ͘ͳΓɺஞ࣍ॲཧग़དྷΔΑ͏ʹͳΔͨΊʹɺݟͨ໨
ͷ࿈ܞ଎౓্͕͕Δɻ
w ݸผͷ࿈ܞॲཧɾڞ௨ͷॲཧɾετΞॲཧɺͦΕͧΕʹ͍ͭͯ؆୯
ʹεέʔϧΛద੾ʹͰ͖Δɻ
w ϦΞϧλΠϜॲཧ΁ͷҠߦ͕؆୯ɻ
DPOT

View Slide

Ωϡʔ͕ؒʹແ͍ͷ͸
ෆ҆
-BNCEB

View Slide

੍ݶ͑͞ղআ͢Ε͹໰୊ͳ͍
wδϣϒؒͷॲཧೳྗʹ͕ࠩ͋Δͱ͖ʹશମ͕٧·Δͷ
Ͱɺ-BNCEBͷ৔߹ͩͱॲཧຖʹεέʔϧ͢ΔͷͰج
ຊ͸ແͯ͘΋ͭͳ͛ΒΕΔɻ
w"84ʹద੾ͳ༨༟Λ੍࣋ͬͯݶղআਃ੥͍ͯ͠ͳ͍ͱ
ΤϥʔʹͳΔɻ

View Slide

εέʔϥϏϦςΟ

View Slide

σʔλͷ෼ׂͱॲཧͷଟॏԽ
֤෺݅৘ใ
ఏڙݩ
࿈ܞαʔό
-BNCEB
෼ׂ ݸผॲཧ
શೖྗσʔλ
$47

ඇಉظݺͼग़͠
w શೖྗσʔλΛద౰ͳαΠζʹTQMJU
ͯ͠ɺඇಉظͰ࣍ஈͷ-BNCEBΛݺ
ͼग़͢ɻ
w ଟॏ౓ͳͲ͸TQMJUαΠζͰௐ੔͢
Δɻ

View Slide

࣌ؒ
෼ׂॲཧ
ݸผॲཧ
ඇಉظJOWPLF
ॲཧΛඇಉظͰݺͼग़͢͜ͱͰεέʔϧ͢Δ
શೖྗσʔλ

View Slide

෼ׂॲཧ
ݸผॲཧ
͜ͷਤ͸ҰྫͳͷͰɺ
࣮ࡍͷσʔλ਺͸ௐ੔͢Δ

View Slide

֤෺݅৘ใ
ఏڙݩ
࿈ܞαʔό
-BNCEB ॲཧ
3%4
෼ׂ
"1*(BUFXBZ
ೖྗ૚ αʔϏε૚ ετΞ૚
,JOFTJT
ݸผॲཧ
ॲཧ
શೖྗσʔλ
ద౰ͳσʔλ਺
ʹ෼͚ͯඇಉظݺͼग़͠
όοναΠζͰ
όϧΫϩʔυ
࣍ஈͷݺͼग़ཻ͠౓Ͱ-BNCEBͷଟॏ౓Λ੍ޚ͢Δ

View Slide

-BNCEB͕ෆ޲͖ͳॲཧ෦෼
w શೖྗσʔλΛద౰ͳαΠζʹ੾ͬͯ
ݸผॲཧʹ౉͢෦෼͸ɺσʔλΛͳΊ
ΔͷͰ୯ൃͷॲཧΛ૝ఆͨ͠-BNCEB
ʹ͸޲͍ͯͳ͍ɻ
࣍ͷΑ͏ͳղܾࡦʁ
ઌͷ3F*OWFOUͰλΠϜΞ΢τ ෼
͕৳
ͼͨͷͰɺؒʹ߹͏ɻ
෼ׂ਺ͳͲͰௐ੔͢Δɻ
͚ͩ͜͜&$ͳͲʹ͢Δɻ
֤෺݅৘ใ
ఏڙݩ
࿈ܞαʔό
-BNCEB
෼ׂ ݸผॲཧ
શೖྗσʔλ
$47

ඇಉظݺͼग़͠

View Slide

εςʔτϨε

View Slide

ͦ΋ͦ΋'"2ʹॻ͍ͯ͋Δʼʻ
2ͳͥ"84-BNCEBؔ਺͸εςʔτϨεͰͳ͚Ε͹ͳΒ
ͳ͍ͷͰ͔͢
ؔ਺ΛεςʔτϨεʹ͢Δ͜ͱʹΑΓɺ"84-BNCEB͸ඞ
ཁͳ͚ͩؔ਺Λਝ଎ʹ࣮ߦ͠ɺΠϕϯτड৴ͷճ਺ʹεέʔ
ϧ͢Δ͜ͱ͕Ͱ͖·͢ɻ"84-BNCEBͷϓϩάϥϜϞσϧ
͸εςʔτϨεͰ͕͢ɺίʔυ͸"NB[PO4·ͨ͸
"NB[PO%ZOBNP%#ͳͲଞͷ΢ΣϒαʔϏεΛݺͼग़͢͜
ͱͰεςʔτϑϧͳσʔλʹΞΫηεͰ͖·͢ɻ
IUUQTBXTBNB[PODPNKQMBNCEBGBRT

View Slide

Ͱ͢ΑͶɻ

View Slide

εςʔτϨεͳߏ੒Λҡ֦࣋ͯ͠ுੑΛ࢒͢

-BNCEB ॲཧ 3%4
"1*(BUFXBZ ,JOTTJT
w෭࡞༻͸ετΞ૚Ҏ
֎Ͱ͸ى͜͞ͳ͍
wॱং΍࠶࣮ߦͰΤϯ
ςΟςΟͷJE͕มΘ
Βͳ͍Α͏ʹ͢Δ
wຊ݅Ͱ͸ɺஈ໨ͰΤϯςΟςΟΛ
֬ఆͰ͖ΔͨΊʹɺஈ໨ͷ
-BNCEBͰೖྗσʔλͷҰ෦Λ࢖͍ɺ
66*% 4)"ϋογϡ
Λ࢖ͬͯ*%
Λੜ੒ͨ͠ɻ
ஈ໨

View Slide

-BNCEB ॲཧ 3%4
"1*(BUFXBZ ,JOTTJT
wεςʔτϨεʹ͢ΔͨΊʹɺ࣋ͪճ͢σʔλ͸ΤϯςΟ
ςΟ୯ҐͰ࣋ͪճ͢ɻ
wςʔϒϧ୯ҐͳͲͰཻ౓Λࡉ͔࣋ͪ͘ճ͢ͱɺͲ͔͜
ͷετΞ૚Λࢀর͠ͳ͚Ε͹ͳΒͣɺॻ͖ࠐ·Εͨࣄ
ͷ֬ೝ΍ঢ়گͷ௥੻͕ඞཁʹͳΓɺશମͷෳࡶੑ͕ߴ
·Δɻ

View Slide

-BNCEB ॲཧ" 3%4
"1*(BUFXBZ ,JOFTJT

ॲཧ#
ॲཧ$
wؒʹLJOFTJTΛೖΕΔ͜ͱͰɺॲཧͷ֦ுΛ-BNCEBͷ
௥Ճ͚ͩͰՄೳʹ͢Δɻ
wଞͷ-BNCEBͷঢ়ଶΛؾʹͤͣॲཧΛ௥Ճɻ
wͨͩ͠࠷ऴతͳετΞઌʹ͍ͭͯ͸ߟྀ͕ཁΔɻ
,JOFTJT

View Slide

σʔλετΞ

View Slide

σʔλετΞ΁ͷྲྀ͠ࠐΈΛू໿
3%4
ը૾഑৴4
$MPVE4FBSDI
,JOFTJT
w ֤छετΞ૚ʹӬଓԽ͢Δͱ͜ΖͰ͸ɺ
ΤϯςΟςΟ͝ͱʹ,JOFTJTܦ༝Ͱ
-BNCEBΛ࢖͍ॻ͖ࠐΉɻ
w όοναΠζΛఆٛͰ͖ΔͷͰɺॻ͖ࠐ
Έॲཧ଎౓ʹ߹Θͤͯॻ͖ࠐΉɻ
w ӬଓԽϛυϧ΢ΣΞ͕૿͑ͨͱͯ͠΋ɺ
,JOFTJTʹ-BNCEBΛ͚ͭΔ͚ͩͰ௥Ճग़
དྷΔɻ

View Slide

ॻ͖ࠐΈॱͷ໰୊
3%4
ը૾഑৴4
$MPVE4FBSDI
,JOFTJT
w ෳ਺ͷॻ͖ࠐΈઌΛΞτϛοΫʹͰ͖ͳ͍ͨ
Ίʹɺ3%4΍$MPVE4FBSDIΛࢀর͢Δ8FC
ΞϓϦέʔγϣϯ͸ɺͦΕͧΕෆ੔߹͕͋Δ
͜ͱΛલఏͱͨ͠࡞Γͱͨ͠ɻ
w ҰॠΛ੾ΓऔΔͱɺ3%4ʹ͸Ϩίʔυ͕͋Δ
͕ɺ$MPVE4FBSDIʹ͸ର৅ͷϨίʔυ͕ͳ͍
ͱ͍͏Α͏ͳঢ়ଶ΋͋ΓಘΔɻ

View Slide

։ൃ

View Slide

ͦͦ͜͜ී௨ʹ։ൃͰ͖Δ

View Slide

͍͔ͭ͘ͷ໰୊

View Slide

-BNCEBͷ֎͔Β஋Λ౉ͤͳ͍
w)FSPLV΍"84#FBOTUBMLͷΑ͏ʹ؀ڥม਺ͷΑ͏ͳ
ܗͰɺ֎෦͔Β؀ڥ༻ͷઃఆ஋Λ౉ͤͳ͍ɻ
w4UBHJOH 1SPEVDUJPOͷΑ͏ʹ؀ڥผͷ-BNCEB͸ɺ
σϓϩΠ࣌ʹɺ؀ڥ஋ΛຒΊࠐΉܗͰ[JQʹ·ͱΊɺσ
ϓϩΠ͢ΔΑ͏ʹͨ͠ɻ

View Slide

{
lambda_params: {
default: {
handler: 'index.handler',
description: 'sample',
timeout: 60, // seconds
memorySize: 128, // MB
role: 'arn:aws:iam::.....' // arn
},
staging: {
region: 'ap-northeast-1',
functionName: 'lambda-test-staging',
},
production: {
region: 'ap-northeast-1',
functionName: 'lambda-test-production',
},
},
app: {
default: {
lambdaName: ‘Hogehoge',
# Circle CI の環境変数
databaseUrl: (() => process.env.DATABASE_URL)()
},
staging: { secret: 'aaa' },
production: { secret: 'bbb' }
}
}
͜͏͍͏ͷ༻ҙͯ͠

View Slide

gulp.task('configure', () => {
# AP_ENV => Circle CI の環境変数
let env = process.env.AP_ENV || 'test'
let src = configure.build(env)
return file('config.js',
`module.exports = ${JSON.stringify(src)}\n`,
{ src: true })
.pipe(gulp.dest('dist'))
})
͜͏͍͏λεΫΛ࡞ͬͯʜ

View Slide

{
"lambda_params": {
"handler": "index.handler",
"description": "sample",
"timeout": 60,
"memorySize": 128,
"role": "arn:aws:iam::.....",
"region": "ap-northeast-1",
"functionName": "lambda-test-staging"
},
"app": {
"databaseUrl": “postgresql://...../”
"lambdaName": "Hogehoge",
"secret": "aaa"
}
}
σϓϩΠ࣌ʹಉࠝ͢Δ

View Slide

αϯϓϧஔ͍͓͖ͯ·ͨ͠
IUUQTHJUIVCDPNUBLFTIJOPEBCBCFMMBNCEBUFNQMBUF

View Slide

/PEFKTͷόʔδϣϯ͕ݹ͍
wW
w͠ΐ͏͕ແ͍͚Ͳ͠ΐ͏͕͋Δ

View Slide

݁߹ςετ͕େม
w͝૝૾ͷ௨Γɻ࣍ʑͱ-BNCEBΛ఻೻͢ΔͷͰɺؒͰ
Կ͔͕͋ΔͱΘ͔Γʹ͍͘ɻ
w/PEFKTΛࣗલͰཱͯͯͲ͏ʹ͔͢Ε͹ɺςετ؀ڥ
࡞ΕΔͱ͸ࢥ͏͕ɺԿ͔͕స౗ͯ͠Δؾ͕͢ΔͷͰɺ
ϩΪϯάͷ࢓૊ΈͳͲͰͳΜͱ͔͍ͨ͠ɻ

View Slide

ڞ௨ͷॲཧతͳ
w͝૝૾ͷ௨Γɻ
wOQNͷ1BDLBHFΛ࡞ͬͯɺSFRVJSF͢Δͱ͔ɻ
wϥΠϒϥϦԽͰ͖ΔΑ͏ɺॲཧͷॻ͖ํΛ௚ަԽͰ͖
ΔΑ͏ҙࣝ͢ΔΠϯηϯςΟϒʹ͸ͳΔɻ
wϙδςΟϒʜɻ

View Slide

ϦϙδτϦ͕େྔʹग़དྷΔ
wҰͭͰ΋Ͱ͖Δ͕ɺ$JSDMF$*Ͱ޻෉ͤͣʹ࡞ΔͱɺҰ
ͭͷσϓϩΠͰҰؾʹ-BNCEBͷϦϦʔε͕࢝·ΔΑ
͏ͳܗʹͳΔɻ
w$JSDMF$*ͷઃఆΛ؆ܿʹ͢ΔͱɺϦϙδτϦ͕େྔʹ
ग़དྷͯ͠·͏ɻ

View Slide

SFEVDFతͳॲཧɾूܭͱ͔
w࣮૷ͯ͠ͳ͍Ͱ͕͢ɺଟ෼%ZOBNPͱ͔ʹॻ͖ࠐ
ΜͰ͓͘ɻ
w,JOFTJTΛ͏·͔ͭ͘͏ʁ

View Slide

݁ہͲ͏ͳͷ͔

View Slide

޲͍͍ͯΔόονͷಛੑ
wॲཧதʹॻ͖ࠐΈ͕ෆཁ΋͘͠͸΄ͱΜͲͳ͍
wॲཧ݁Ռ͕΄ͱΜͲࢀরܥ
wΦʔόʔϔου͕ΫϦςΟΧϧʹޮ͔ͳ͍σʔλྔ
wͦΕͬͯͲΜ͚ͩʁ෼͔Βͳ͍ʜɻ

View Slide

wεςʔτϨεͷ੍໿΍؆қʹฒྻԽͰ͖Δͱ͜Ζ͔Βɺ
όονʹىҼ͢ΔαʔϏεվળ͕΍Γ΍͍͢ɻ
wىಈॱ΍ϦΧόϦͷखॱ͕੟ҬԽ͠΍͍͢όοναʔ
όͷৼΔ෣͍ΛίʔυʹઃఆͰ͖ΔΑ͏ʹͳͬͨɻ
wҰํͰɺιʔείʔυ؅ཧ΍$*$%पΓͷෳࡶੑ͸૿
͕ͨ͠ɺͦΕ͸πʔϧͷ໰୊ͱߟ͑Δɻ
݁ہͲ͏ͳͷ͔

View Slide

ͭ͠΋Μʁ

View Slide

͋Γ͕ͱ͏
͍͟͝·ͨ͠

View Slide

API Gateway / Lambda / Kinesis を使ったストリーミングなバッチ実行基盤の実装

API Gateway / Lambda / Kinesis を使ったストリーミングなバッチ実行基盤の実装

Takeshi Shinoda

More Decks by Takeshi Shinoda

Other Decks in Programming

Featured

Transcript