Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
サーバーレスでデータ連携する際にハマったところ / ninno-techfest-202110...
Search
kasacchiful
October 22, 2021
Programming
0
610
サーバーレスでデータ連携する際にハマったところ / ninno-techfest-20211022-lt
2021/10/22 (金) NINNO Tech Fest #2 の LT で発表した資料
kasacchiful
October 22, 2021
Tweet
Share
More Decks by kasacchiful
See All by kasacchiful
WorkersでDiscord botを試してみた / 20250822workers-tech-talk-niigata
kasacchiful
0
35
地域コミュニティへの「感謝」と「恩返し」 / 20250726jawsug-tochigi
kasacchiful
0
150
Amazon Q Developer for CLI を使って PHP Conference 新潟 2025 参加者向けにグルメサイトを構築した話 / 20250620niigata-5min-tech
kasacchiful
1
94
ワイがおすすめする新潟の食 / 20250530phpconf-niigata-eve
kasacchiful
0
400
生成AIでメタデータを生成してみた / 20250525generate-metadata-using-generative-ai
kasacchiful
0
73
Strands Agents SDK で AIエージェント作成 を試してみた / 20250525strands-agents
kasacchiful
0
280
いろんな世界を見てみよう / 20250508ninno_tech_fest
kasacchiful
0
41
Amazon Q Developer for CLIのある生活 / 20250427ai_craft_hacks_niigata1
kasacchiful
1
97
AWSのコンテナサービス / jawsug-akita-aws-container-services
kasacchiful
0
90
Other Decks in Programming
See All in Programming
DockerからECSへ 〜 AWSの海に出る前に知っておきたいこと 〜
ota1022
5
1.9k
KessokuでDIでもgoroutineを活用する / Go Connect #6
mazrean
0
130
Langfuseと歩む生成AI活用推進
licux
3
320
Claude Codeで挑むOSSコントリビュート
eycjur
0
190
go test -json そして testing.T.Attr / Kyoto.go #63
utgwkk
1
200
More Approvers for Greater OSS and Japan Community
tkikuc
1
110
時間軸から考えるTerraformを使う理由と留意点
fufuhu
8
3.5k
開発チーム・開発組織の設計改善スキルの向上
masuda220
PRO
18
9.6k
AIエージェント開発、DevOps and LLMOps
ymd65536
1
370
HTMLの品質ってなんだっけ? “HTMLクライテリア”の設計と実践
unachang113
1
590
個人軟體時代
ethanhuang13
0
280
testingを眺める
matumoto
1
130
Featured
See All Featured
Making Projects Easy
brettharned
117
6.4k
Six Lessons from altMBA
skipperchong
28
4k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
161
15k
How to train your dragon (web standard)
notwaldorf
96
6.2k
Fashionably flexible responsive web design (full day workshop)
malarkey
407
66k
The MySQL Ecosystem @ GitHub 2015
samlambert
251
13k
Music & Morning Musume
bryan
46
6.8k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
53
2.9k
Building an army of robots
kneath
306
46k
jQuery: Nuts, Bolts and Bling
dougneiner
64
7.9k
Building a Modern Day E-commerce SEO Strategy
aleyda
43
7.5k
The Pragmatic Product Professional
lauravandoore
36
6.8k
Transcript
αʔόʔϨεͰσʔλ࿈ܞ͢Δࡍʹ ϋϚͬͨͱ͜Ζ ּݪ /*//05FDI'FTU
ຊ͓͢͠Δ͜ͱ wαʔόʔϨεͰσʔλ࿈ܞج൫Λߏஙͨ͠ࡍɺϋϚͬͨͱ͜Ζ͕৭ʑ ͋ͬͨͷͰɺ؆୯ʹ͓͠͠·͢ɻ w"84Ͱߏங͍ͯ͠·͢ɻ"84͋·ΓΘ͔Βͳ͍ํɺ༰ࠐΈೖͬͯ ΔͷͰ͍͔͠͠Ε·ͤΜɻ w-5ͳͷʹϖʔδ࡞ͬͨͷͰɺۦ͚ͰਐΈ·͢ɻ
ࣗݾհ w ּݪ w ΫϥεϝιουגࣜձࣾσʔλΞφϦςΟΫεࣄۀຊ෦ w ৽ׁݝ৽ׁࢢࡏॅ w ͖ͳ"84αʔϏε4ɺ-BNCEBɺ4BHF.BLFS
w +"846(৽ׁࢧ෦ʗ1ZUIPOػցֶशษڧձJO৽ׁʗ +B445৽ׁʗ48"/**ʗFUD @kasacchiful @kasacchiful
։࢝ w IUUQTQZNMOJJHBUBDPOOQBTTDPNFWFOU 1ZUIPOػցֶशษڧձJO৽ׁ
w IUUQTKBXTVHOJJHBUBDPOOQBTTDPN +"846(/JJHBUB
+"841"/,3"5*0/ ʙ ͷ࣌ؒΠϕϯτ w IUUQTKBXTQBOLSBUJPOKBXTVHKQ
ຊ αʔόʔϨεͷσʔλج൫
ྫ͑͜Μͳͭ σʔλੳʹ͓͚Δ֤छ"84αʔϏε
ྫ͑͜Μͳͭ 4ʹσʔλೖΕͯɺ-BNCEBͰՃɺ݁ՌΛ4ʹ͓͘ɻ
ྫ͑͜Μͳͭ 4ʹՃࡁσʔλ͓͍ͨΒɺ͋ͱੳʹͲ͏ͧɻ
ࠓճ͜Μͳײ͡ͷͭ 4UFQ'VODUJPOTͷεςʔτϚγϯͰ-BNCEBͷϫʔΫϑϩʔΛ੍ޚͯ͠ɺσʔλ ΛՃ w ճεέδϡʔϧ࣮ߦ
4UFQ'VODUJPOTͷεςʔτϚγϯͰ-BNCEBͷϫʔΫϑϩʔΛ੍ޚͯ͠ɺσʔλ ΛՃ w ճεέδϡʔϧ࣮ߦ ࠓճ͜Μͳײ͡ͷͭ σʔλൃੜݩ͔ΒɺσʔλΛऔ ಘͯ͠4ʹอଘ ֤ϑΝΠϧຖʹɺ࠷ݶͷσʔ λՃΛͯ͠ɺ4ʹอଘ
2VJDL4JHIU #* ༻ʹ ෳϑΝΠϧͷσʔλΛ·ͱΊ ͯదʹܗ͢Δ
͍Ζ͍ΖϋϚͬͨ ϋϚͬͨΛ4ͭհ
ϋϚͬͨͦͷ ಛఆͷσʔλϑΝΠϧଟ͗͢
Έ͋ΔಛఆͷσʔλϑΝΠϧ͚ͩҟৗʹଟ͍ w ؒͷσʔλ͕ϑΝΠϧʹ͋Δ w தϛϦඵ୯ҐͷϨίʔυ w ಛఆͷॲཧ͚͕͔͔ͩ࣌ؒΔ ಛఆͷσʔλϑΝΠϧଟ͗͢
ಛఆͷσʔλϑΝΠϧଟ͗͢ ରॲ๏͋ΔಛఆͷσʔλϑΝΠϧ͚ͩɺຖ࣌ॲཧʹมߋ w ݅ଟ͍σʔλɺ#*ʹग़ྗ͠ͳ͍߲ͩͬͨ w ࣍ॲཧ͔ΒΓͯ͠ɺຖ࣌ॲཧʹมߋ w ࣍ॲཧͷϘτϧωοΫΛআ͍ͨ
ϋϚͬͨͦͷ Athena ͷΫΥʔλ
Έ"UIFOBͷΫΤϦಉ࣮࣌ߦͷΫΥʔλʹҾ͔͔ͬΔ w σʔλҠߦ࣌ʹɺ࣍ॲཧͷ࠷ޙͷ-BNCEBͰΤϥʔʹͳΔ w લஈͰॲཧͨ͠ෳσʔλΛ"UIFOBͬͯ42-ΫΤϦͰऔಘ͢Δͱ͜ΖͰ্ݶʹҾ͔͔ͬΔ w -BNCEBؔͭʹ͖ͭɺɹstart-query-executionɹ"1*Λճίʔϧ w Ұ࣌తʹόʔετͰ্ݶ·Ͱ૿͑Δ͚ͲɺσʔλҠߦ࣌ʹͰ಄ଧͪ w
্ݶ؇ਃ͢Ε্ݶ͋͛ΒΕΔ "UIFOBͷΫΥʔλ
"UIFOBͷΫΥʔλ IUUQTEPDTBXTBNB[PODPNKB@KQTUFQGVODUJPOTMBUFTUEHMJNJUTPWFSWJFXIUNM
ରॲ๏4UFQ'VODUJPOTͷ.BQεςʔτͷ࠷େಉ࣮࣌ߦΛઃఆ w .BQεςʔτ ྻ͢ͱɺಉ࣮࣌ߦͰྻཁૉΛॲཧ͢ΔΠϝʔδ ͷ࠷େಉ࣮࣌ߦΛઃ ఆ͠ɺ"UIFOBͷTUBSURVFSZFYFDVUJPO"1*ίʔϧΛ࠷େ·Ͱʹ͓͑͞Δ "UIFOBͷΫΥʔλ
.BQεςʔτʹ͍ͭͯɺҎԼͷهࣄΛࢀߟʹ "UIFOBͷΫΥʔλ IUUQTEFWDMBTTNFUIPEKQBSUJDMFTTUFQGVODUJPOTVQEBUFNBQTUBUF IUUQTEPDTBXTBNB[PODPNKB@KQTUFQGVODUJPOTMBUFTUEHBNB[POTUBUFTMBOHVBHFNBQTUBUFIUNM
ϋϚͬͨͦͷ Step Functions ͷΫΥʔλ
Έ4UFQ'VODUJPOTͷΠϕϯτཤྺ͕ΫΥʔλʹҾ͔͔ͬΔ w ͋Δಛఆͷ͚ͩɺຖ࣌ॲཧͷϑΝΠϧ͕ҟৗʹଟ͍ w ࣌ؒܦͬͯҟৗऴྃɻ4UFQ'VODUJPOTͷΠϕϯτཤྺͷ্ݶ౸ୡ Πϕϯτ w ্ݶ؇ෆՄͷ߲
4UFQ'VODUJPOTͷΫΥʔλ { "error": "States.Runtime", "cause": "The execution reached the maximum number of history events (25000)." }
4UFQ'VODUJPOTͷΫΥʔλ IUUQTEPDTBXTBNB[PODPNKB@KQTUFQGVODUJPOTMBUFTUEHMJNJUTPWFSWJFXIUNM
ରॲ๏4UFQ'VODUJPOTͷεςʔτϚγϯΛೖΕࢠʹ w εςʔτϚγϯΛೖΕࢠʹ͢Δ͜ͱͰɺΠϕϯτཤྺ্ݶʹҾ͔͔ͬΒͳ͍Α͏ʹͨ͠ w -BNCEBͷಉ࣮࣌ߦ͕͔ͳΓ૿͑ΔͷͰɺҎԼͷରԠΛՃ 㾎 -BNCEBͷಉ࣮࣌ߦͷ্ݶ؇ਃ 㾎 4UFQ'VODUJPOTͷ.BQεςʔτͷ࠷େಉ࣮࣌ߦΛઃఆ 4UFQ'VODUJPOTͷΫΥʔλ
4UFQ'VODUJPOTͷΫΥʔλ มߋલ มߋޙ
4UFQ'VODUJPOTͷΫΥʔλ มߋલ มߋޙ
ϋϚͬͨͦͷ Lambdaͷεέʔϧ͕͍͔ͭͳ͍
Έճ͚ͩ-BNCEBͷ3BUF-JNJUΤϥʔʹૺ۰ w ಉ࣮࣌ߦͷΤϥʔͷΑ͏͚ͩͲʜ w ͢Ͱʹಉ࣮࣌ߦͷ্ݶΛҾ্͖͍͛ͯΔͷͷɺ֤ؔͷϞχλϦϯάݟΔݶΓɺಉ࣮࣌ ߦʹ౸ୡ͍ͯ͠ͳ͍ -BNCEBͷεέʔϧ͕͍͔ͭͳ͍ { "error":
"Lambda.TooManyRequestsException", "cause": "Rate Exceeded. (Service: Lambda, Status Code: 429, Request ID: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx, Extended Request ID: null)" }
-BNCEBͷεέʔϧ͕͍͔ͭͳ͍ IUUQTEPDTBXTBNB[PODPNKB@KQMBNCEBMBUFTUEHJOWPDBUJPOTDBMJOHIUNM
ରॲ๏-BNCEBؔͷ3FUSZઃఆΛݟ͠ w 4UFQ'VODUJPOTͷ.BQεςʔτͷ࠷େಉ࣮࣌ߦΛݟ͠ w 4UFQ'VODUJPOTͰఆٛ͢Δ-BNCEBͷ3FUSZઃఆΛݟ͠ -BNCEBͷεέʔϧ͕͍͔ͭͳ͍
ରॲ๏-BNCEBؔͷ3FUSZઃఆΛݟ͠ w 3FUSZͷִؒʹ͍ͭͯҎԼͷهࣄ͕ৄ͍͠ -BNCEBͷεέʔϧ͕͍͔ͭͳ͍ $ node -e '((i,m,b)=>{for(let w=i,c=0;c<m;c++){console.log(w+=(c==0?0:b**c))}})(2,7,1.85)'
2 3.85 7.272500000000001 13.604125000000002 25.317631250000005 46.987617812500005 87.07709295312502 IUUQTEFWDMBTTNFUIPEKQBSUJDMFTXBJU@UJNF@BOE@QBSBNT@JO@TUFQ@GVODUJPO@SFUSZ
ରॲ๏-BNCEBؔͷ3FUSZ݅Λݟ͠ w -BNCEBͷ1SPWJTJPOFE$PODVSSFODZઃఆࠓճ࣮ࢪͯ͠ͳ͍ -BNCEBͷεέʔϧ͕͍͔ͭͳ͍ IUUQTEFWDMBTTNFUIPEKQBSUJDMFTMBNCEBQSPWJTJPOFEDPODVSSFODZDPMETUBSU
ิ σʔλͷՃʹ AWS Glueͱ͍͏αʔϏε͋ΔΑʁ
(MVFΘͣʹɺΘ͟Θ͟4UFQ'VODUJPOT -BNCEBͰΉඞཁ͋Δͷ͔ʁ w 4UFQ'VODUJPOT -BNCEBͷ߹ɺΑ͘ΘΕΔ։ൃϑϨʔϜϫʔΫ͕͑ΔͷͰɺෳਓ Ͱͷ։ൃ͕͍͢͠ɻ 㾎 ࠓճ4FSWFSMFTT'SBNFXPSLͬͨɻ w σʔλϑΝΠϧ͕ଟͯ͘ɺσʔλ݅͋ͨΓͷ༰ྔ͕ͦ͜·Ͱେ͖͘ͳ͚Εɺ࣍ୈ
Ͱ-BNCEBͰॲཧ͕Ͱ͖Δɻ w -BNCEBͰΓΕͳ͍σʔλ༰ྔΛѻ͏߹ɺ(MVFͬͨํ͕͍͍ɻ 㾎 ࠷େϝϞϦׂ.#ɺ࠷େ࣮ߦ࣌ؒɺUNQσΟϨΫτϦαΠζ.# σʔλͷՃͳΒ(MVF͕͋Δ
·ͱΊ • αʔόʔϨεαʔϏεΛۦͯ͠ɺσʔλੳج൫Λ ߏஙՄೳ • αʔόʔϨεͷΑ͋͘ΔΞʔΩςΫνϟύλʔϯΛ͏ ·͍͘͜ͳ͠·͠ΐ͏
None