Upgrade to Pro — share decks privately, control downloads, hide ads and more …

AWS Glue 触ってみた

AWS Glue 触ってみた

2022/5/26(木)に行われた【サーバーレス LT vol.2 #r_serverlesslt】にて発表した
「AWS Glue 触ってみた」
の資料です。

https://rakus.connpass.com/event/244702/

67683b0a8d24009c731787b49881a9de?s=128

tomuro

May 26, 2022
Tweet

More Decks by tomuro

Other Decks in Technology

Transcript

  1. "84(MVF৮ͬͯΈͨ גࣜձࣾϞϦαϫ খࣨو࢙

  2. w ࣗݾ঺հ w "84(MVFࢼ͍ͨ͠ w $47σʔλͷ४උ w 4ͷ४උ w "84(MVF%BUB$BUBMPHͷઃఆ

    w σʔλϕʔεͷ४උ w ςʔϒϧͷ४උ w Ϋϩʔϥͷ४උͱ࣮ߦ w "84(MVF%BUB$BUBMPHʹΞΫηε w ·ͱΊ ΞδΣϯμ
  3. ໊લɿখࣨو࢙ʢBLB͓ΉΖΜʣ ࣗݾ঺հ ॴଐɿגࣜձࣾϞϦαϫ ɹɹɹγεςϜ։ൃ෦ɹ ࢓ࣄɿςοΫϚωʔδϟʔ ɹɹɹ"84ΠϯϑϥΤϯδχΞ !PNVSPO ޷͖ͳαʔϏεɿ$MPVE'PSNBUJPO

  4. ϛογϣϯ จࣈΛ௨ͯ͡ੈքதͷਓʑͷੜ׆ͷதʹଘࡏ͠ଓ͚Δ גࣜձࣾϞϦαϫ w ࣸਅ২ࣈػͷൃ໌͔Βελʔτͯ͠ɺࠓ͸ϑΥϯτൢച͕ϝΠϯ

  5. Ϗδϣϯ Ϟϊͮ͘Γͱٕज़Λָ͠ΊΔ ࣋ଓՄೳͳΤΩεύʔτूஂ΁ਐԽ

  6. ͭͷόϦϡʔ w ڞ૑ͷਫ਼ਆ w ૬ޓཧղͱϦεϖΫτΛ΋ͬͯڠۀ͠ɺΫϩεϘʔμʔͰ ڥքͷڱؒΛ౿Έൈ͖ڞʹ૑Δ w Ϣʔβʔ΍ӡ༻ͳͲݟ͑ʹ͍͘ར༻ऀΛೝࣝ͠ 6TFS'JSTU΍%FW0QTΛେࣄʹ͢Δ w

    σʔλυϦϒϯ w σʔλͱ߹ཧతͳߦಈ൑அʹՁ஋ج४Λ͓͘ w Ҽ਺෼ղͱσʔλ෼ੳΛݩʹͨ͠ߴ଎1%$"αΠΫϧΛ࣮ફ͢Δ w ָ׆ w ͜Ε͸଄ޠͰɺָͪΜY҆Β͔Yָ͍͠Λ௥ٻ͢Δ։ൃ׆ಈʹ Ձ஋ج४Λ͓͘ w ߹ཧԽΛ͸͔ΓࣗಈԽ΍লྗԽ΍ੑೳվળ΍ϑϩʔվળΛਐΊΔ w ୀ۶ͳλεΫ΍ෆ҆ͳλεΫ͔Βղ์Λ໨ࢦ͢
  7. ࠓ೥΍Γ͍ͨ͜ͱ IUUQTTQFBLFSEFDLDPNUPNVSPYJOIVSPUBLVUPLBJGBUFZBUVUBLPUPUPZBSJDBOTJUBLPUP TMJEF IUUQTKBXTVHCHOSDPOOQBTTDPNFWFOU

  8. "84(MVFࢼ͍ͨ͠

  9. "84αʔϏεͷϓϨϑΟοΫε๏ଇ w "NB[POɿ୯ମͱͯ͠੒ΓཱͭαʔϏε͕ଟ͍ w &$ 4 3%4 $MPVE'SPOU 3PVUF w

    ͦͷαʔϏε͚ͩཧղ͢Ε͹ར༻Մೳ w "84ɿଞͷ"84αʔϏεͱ࿈ܞ͢Δ΋ͷ͕ଟ͍ w *". (MVF 8"' $MPVE'PSNBUJPO w ୯ମͰ࢖͑ͣ"84΁ͷཧղ͕ඞཁͰ೉౓ߴΊ "NB[POͱ"84 ✴ ஫ҙʣݸਓతͳײ૝Ͱ͋ΓඇެࣜͰ͢ "84-BNCEBͳͲݱࡏ͸ζϨ͍ͯΔ΋ͷ΋͋Γ
  10. "84(MVFίϯιʔϧͰදࣔ τοϓը໘͸ӳޠ

  11. "84(MVFίϯιʔϧͰදࣔ "84(MVF%BUB$BUBMPH ࠓճ͸σʔλΧλϩάͷઃఆΛͯ͠Έ·͢ w σʔλΧλϩάͱ͸ʁ w "84(MVF%BUB$BUBMPH͸ɺσʔλͷ৔ॴɺε ΩʔϚɺ͓ΑͼϥϯλΠϜϝτϦΫε΁ͷΠϯσοΫ εͰ͢ɻσʔλΧλϩά಺ͷ৘ใ͸ɺ&5-δϣϒͷ ࡞੒ͱ؂ࢹʹ࢖༻͠·͢ɻ%BUB$BUBMPHͷ৘ใ͸

    ϝλσʔλςʔϒϧͱͯ͠อଘ͞Εɺ֤ςʔϒϧ͕ ͭͷσʔλετΞΛࢦఆ͠·͢ɻ IUUQTEPDTBXTBNB[PODPNKB@KQHMVFMBUFTUEHQPQVMBUFEBUBDBUBMPHIUNM
  12. "84(MVFίϯιʔϧͰදࣔ "84(MVF%BUB$BUBMPH ࠓճ͸σʔλΧλϩάͷઃఆΛͯ͠Έ·͢ w σʔλΧλϩάͱ͸ʁ w "84(MVF%BUB$BUBMPH͸ɺσʔλͷ৔ॴɺε ΩʔϚɺ͓ΑͼϥϯλΠϜϝτϦΫε΁ͷΠϯσοΫ εͰ͢ɻσʔλΧλϩά಺ͷ৘ใ͸ɺ&5-δϣϒͷ ࡞੒ͱ؂ࢹʹ࢖༻͠·͢ɻ%BUB$BUBMPHͷ৘ใ͸

    ϝλσʔλςʔϒϧͱͯ͠อଘ͞Εɺ֤ςʔϒϧ͕ ͭͷσʔλετΞΛࢦఆ͠·͢ɻ IUUQTEPDTBXTBNB[PODPNKB@KQHMVFMBUFTUEHQPQVMBUFEBUBDBUBMPHIUNM 🤔
  13. Θ͔ΒΜ͔Β৮ͬͯΈΔ

  14. σʔλͷ४උ $47Λ४උ 'VMM/BNF $PVOUSZ $SFBUFE"U *E &NBJM $PPQFS%FDLPX 4FOFHBM 5IV%FD(.5

     ೔ຊඪ४࣌  .VSSBZ!DSBJHDB %PNFOJDP3FJDIFSU -BPT 5IV'FC(.5  ೔ຊඪ४࣌  .FSM![PFZDB "OUXBO3PNBHVFSB /FVUSBM;POF 5IV"VH(.5  ೔ຊඪ४࣌  /PSNB@+BLVCPXTLJ!DBUIZOFU $IBTF%PPMFZ .POHPMJB 5IV%FD(.5  ೔ຊඪ४࣌  $BSMPT@)FJEFOSFJDI!HBWJOJP -POEPO%JDLFOT #VSVOEJ 8FE0DU(.5  ೔ຊඪ४࣌  $MBVEJOF@)BSSJT!DIBOEMFSDPVL 5PSSBODF4UBNN 4XFEFO 'SJ+VM(.5  ೔ຊඪ४࣌  &NJMJB@,FSU[NBOO!TFMJOBDPN %S$PEZ+FXFTT 4VSJOBNF 'SJ/PW(.5  ೔ຊඪ४࣌  &UIZM.PSBS!OJDPMFUUFOBNF "TIB1SPTBDDP "SVCB 4VO.BZ(.5  ೔ຊඪ४࣌  0MBG@,VMBT!BMFDOBNF .BWJT,JOH (SFFOMBOE 5IV+VO(.5  ೔ຊඪ४࣌  *NNBOVFM!OZBIDPVL .BUJMEB,JIO &DVBEPS .PO0DU(.5  ೔ຊඪ४࣌  5PZ!UZSFFLJOGP &EB$SPOB *SBR 8FE+BO(.5  ೔ຊඪ४࣌  SZMFF!OPCMFUW  w μϛʔ$47Λੜ੒ w ໊લɺࠃɺ࡞੒೔ɺ*%ɺ&.BJMΛ࣋ͬͨσʔλ w IUUQXXXEBUBCBTFUFTUEBUBDPNΛར༻ͯ͠ੜ੒
  15. 4Λ४උ σʔλͷ४උ w ద౰ͳ໊લͷόέοτ࡞੒ UFTUHMVF w ೥݄೔ΛΩʔόϦϡʔͰύεΛ੾Δʢ೚ҙʣ w "QBDIF)JWFͷྲّྀ ZZZZNNEE

    w $47ΛΞοϓϩʔυ
  16. "84(MVF%BUB$BUBMPHͷ४උ σʔλϕʔεͷ४උ w σʔλϕʔεͷ௥Ճ w UFTUEC࡞੒ ˞τοϓͱҧ͍σʔλϕʔεͷϚωδϝϯτίϯιʔϧ͸೔ຊޠදࣔՄೳ

  17. "84(MVF%BUB$BUBMPHͷ४උ ςʔϒϧͷ४උ w ΫϩʔϥΛ࢖༻ͯ͠ ςʔϒϧ͕௥ՃͰ͖Δ

  18. "84(MVF%BUB$BUBMPHͷ४උ Ϋϩʔϥͷ४උ w ద౰ͳ໊લΛઃఆ

  19. "84(MVF%BUB$BUBMPHͷ४උ Ϋϩʔϥͷ४උ w ࡞ͬͨ4όέοτΛࢦఆ

  20. "84(MVF%BUB$BUBMPHͷ४උ Ϋϩʔϥͷ४උ w *".ϩʔϧΛ࡞੒

  21. "84(MVF%BUB$BUBMPHͷ४උ Ϋϩʔϥͷ४උ w ࡞ͬͨσʔλϕʔεΛࢦఆ

  22. "84(MVF%BUB$BUBMPHͷ४උ Ϋϩʔϥͷ࣮ߦ w ࡞ͬͨΫϩʔϥΛ࣮ߦ

  23. "84(MVF%BUB$BUBMPHͷ४උ ςʔϒϧ͕࡞੒͞ΕΔ

  24. "84(MVF%BUB$BUBMPHͷ४උ ςʔϒϧͷεΩʔϚΛݟΔ w $47ͷϔομʔ w ΧϥϜࣗಈੜ੒ w 4ͷύε w ύʔςΟγϣϯͱͯ͠ೝࣝ

  25. "84(MVF%BUB$BUBMPHʹΞΫηε "UIFOBͰࢀরՄೳ w ࡞ͬͨ%#ͱςʔϒϧ͕ݟ͑Δ w ΫΤϦͰ$47ΛղੳͰ͖Δ

  26. "84(MVF%BUB$BUBMPHʹΞΫηε "UIFOBͰࢀরՄೳ w 4ͷύε͕ύʔςΟγϣϯʹͳΔ w ޮ཰Α͘ΞΫηεՄೳ w ύεΛ੾ͬͯσʔλΛ૿΍ͤΔ w Ϋϩʔϥͷ࠶࣮ߦ͕ඞཁ

  27. (MVFΛ࢖͏͜ͱ͕Ͱ͖ͨ🎉

  28. ·ͱΊ w ղੳ༻σʔλϕʔε͕؆୯ʹ࡞੒Ͱ͖Δ w (MVF%BUB$BUBMPHͱ͍͏ػೳΛར༻ w ΋ͱ΋ͱ"UIFOBͷػೳ͕ͩͬͨ(MVFʹ෼཭͞Εͨ w (MVF͸&5-͔Β࢝·ͬͯ%BUB$BUBMPH͕௥Ճ͞Εͨ w

    $47΍+40/ϑΝΠϧͳͲ͸(MVF$SBXMFSͰղੳͰ͖Δ w αʔόʔϨεͳͷͰઃఆ͚ͩͰ࢖͑Δ w σʔλΧλϩάͷ%#ʹςʔϒϧ͕ࣗಈੜ੒͞ΕΔ w "UIFOB΍2VJDL4JHIUͰղੳͰ͖Δ w 4ͷύε͸ύʔςΟγϣϯͱͯ͠ೝࣝ w ޮ཰Α͘σʔλղੳ͕Ͱ͖Δ (MVFΛ࢖͏ͱαʔόϨεͰ෼ੳՄೳ
  29. ࢀߟʹͨ͠ॻ੶ "84Ͱ͸͡ΊΔσʔλϨΠΫ

  30. ͓·͚୯ମͷϑΝΠϧΛղੳ͍ͨ͠ͳΒ 44FMFDU͕؆୯

  31. ஥ؒืूதͰ͢ʂ w ϓϩδΣΫτϚωʔδϟʔ w 1.0#31%9ܦݧऀ w όοΫΤϯυΤϯδχΞ w "84 (P1ZUIPO

    %PDLFSܦݧऀ 1IPUP1",65"40 'POU.PSJTBXB