AWS Glue 触ってみた

2022/5/26(木)に行われた【サーバーレス LT vol.2 #r_serverlesslt】にて発表した
「AWS Glue 触ってみた」
の資料です。

https://rakus.connpass.com/event/244702/

Transcript

1. "84(MVF৮ͬͯΈͨ
   גࣜձࣾϞϦαϫ
   খࣨو࢙
   

   View Slide

2. w ࣗݾ঺հ
   w "84(MVFࢼ͍ͨ͠
   w $47σʔλͷ४උ
   w 4ͷ४උ
   w "84(MVF%BUB$BUBMPHͷઃఆ
   w σʔλϕʔεͷ४උ
   w ςʔϒϧͷ४උ
   w Ϋϩʔϥͷ४උͱ࣮ߦ
   w "84(MVF%BUB$BUBMPHʹΞΫηε
   w ·ͱΊ
   ΞδΣϯμ
   

   View Slide

3. ໊લɿখࣨو࢙ʢBLB͓ΉΖΜʣ
   ࣗݾ঺հ
   ॴଐɿגࣜձࣾϞϦαϫ
   ɹɹɹγεςϜ։ൃ෦ɹ
   ࢓ࣄɿςοΫϚωʔδϟʔ
   ɹɹɹ"84ΠϯϑϥΤϯδχΞ
   !PNVSPO
   ޷͖ͳαʔϏεɿ$MPVE'PSNBUJPO
   

   View Slide

4. ϛογϣϯ
   จࣈΛ௨ͯ͡ੈքதͷਓʑͷੜ׆ͷதʹଘࡏ͠ଓ͚Δ
   גࣜձࣾϞϦαϫ
   w ࣸਅ২ࣈػͷൃ໌͔Βελʔτͯ͠ɺࠓ͸ϑΥϯτൢച͕ϝΠϯ
   

   View Slide

5. Ϗδϣϯ
   Ϟϊͮ͘Γͱٕज़Λָ͠ΊΔ
   ࣋ଓՄೳͳΤΩεύʔτूஂ΁ਐԽ
   

   View Slide

6. ͭͷόϦϡʔ
   w ڞ૑ͷਫ਼ਆ
   w ૬ޓཧղͱϦεϖΫτΛ΋ͬͯڠۀ͠ɺΫϩεϘʔμʔͰ
   ڥքͷڱؒΛ౿Έൈ͖ڞʹ૑Δ
   w Ϣʔβʔ΍ӡ༻ͳͲݟ͑ʹ͍͘ར༻ऀΛೝࣝ͠
   6TFS'JSTU΍%FW0QTΛେࣄʹ͢Δ
   w σʔλυϦϒϯ
   w σʔλͱ߹ཧతͳߦಈ൑அʹՁ஋ج४Λ͓͘
   w Ҽ਺෼ղͱσʔλ෼ੳΛݩʹͨ͠ߴ଎1%$"αΠΫϧΛ࣮ફ͢Δ
   w ָ׆
   w ͜Ε͸଄ޠͰɺָͪΜY҆Β͔Yָ͍͠Λ௥ٻ͢Δ։ൃ׆ಈʹ
   Ձ஋ج४Λ͓͘
   w ߹ཧԽΛ͸͔ΓࣗಈԽ΍লྗԽ΍ੑೳվળ΍ϑϩʔվળΛਐΊΔ
   w ୀ۶ͳλεΫ΍ෆ҆ͳλεΫ͔Βղ์Λ໨ࢦ͢
   

   View Slide

7. ࠓ೥΍Γ͍ͨ͜ͱ
   IUUQTTQFBLFSEFDLDPNUPNVSPYJOIVSPUBLVUPLBJGBUFZBUVUBLPUPUPZBSJDBOTJUBLPUP TMJEF
   IUUQTKBXTVHCHOSDPOOQBTTDPNFWFOU
   

   View Slide

8. "84(MVFࢼ͍ͨ͠
   

   View Slide

9. "84αʔϏεͷϓϨϑΟοΫε๏ଇ
   w "NB[POɿ୯ମͱͯ͠੒ΓཱͭαʔϏε͕ଟ͍
   w &$ 4 3%4 $MPVE'SPOU 3PVUF
   w ͦͷαʔϏε͚ͩཧղ͢Ε͹ར༻Մೳ
   w "84ɿଞͷ"84αʔϏεͱ࿈ܞ͢Δ΋ͷ͕ଟ͍
   w *". (MVF 8"' $MPVE'PSNBUJPO
   w ୯ମͰ࢖͑ͣ"84΁ͷཧղ͕ඞཁͰ೉౓ߴΊ
   "NB[POͱ"84
   ✴ ஫ҙʣݸਓతͳײ૝Ͱ͋ΓඇެࣜͰ͢
   "84-BNCEBͳͲݱࡏ͸ζϨ͍ͯΔ΋ͷ΋͋Γ
   

   View Slide

10. "84(MVFίϯιʔϧͰදࣔ
    τοϓը໘͸ӳޠ
    

    View Slide

11. "84(MVFίϯιʔϧͰදࣔ
    "84(MVF%BUB$BUBMPH
    ࠓճ͸σʔλΧλϩάͷઃఆΛͯ͠Έ·͢
    w σʔλΧλϩάͱ͸ʁ
    w "84(MVF%BUB$BUBMPH͸ɺσʔλͷ৔ॴɺε
    ΩʔϚɺ͓ΑͼϥϯλΠϜϝτϦΫε΁ͷΠϯσοΫ
    εͰ͢ɻσʔλΧλϩά಺ͷ৘ใ͸ɺ&5-δϣϒͷ
    ࡞੒ͱ؂ࢹʹ࢖༻͠·͢ɻ%BUB$BUBMPHͷ৘ใ͸
    ϝλσʔλςʔϒϧͱͯ͠อଘ͞Εɺ֤ςʔϒϧ͕
    ͭͷσʔλετΞΛࢦఆ͠·͢ɻ
    IUUQTEPDTBXTBNB[PODPNKB@KQHMVFMBUFTUEHQPQVMBUFEBUBDBUBMPHIUNM
    

    View Slide

12. "84(MVFίϯιʔϧͰදࣔ
    "84(MVF%BUB$BUBMPH
    ࠓճ͸σʔλΧλϩάͷઃఆΛͯ͠Έ·͢
    w σʔλΧλϩάͱ͸ʁ
    w "84(MVF%BUB$BUBMPH͸ɺσʔλͷ৔ॴɺε
    ΩʔϚɺ͓ΑͼϥϯλΠϜϝτϦΫε΁ͷΠϯσοΫ
    εͰ͢ɻσʔλΧλϩά಺ͷ৘ใ͸ɺ&5-δϣϒͷ
    ࡞੒ͱ؂ࢹʹ࢖༻͠·͢ɻ%BUB$BUBMPHͷ৘ใ͸
    ϝλσʔλςʔϒϧͱͯ͠อଘ͞Εɺ֤ςʔϒϧ͕
    ͭͷσʔλετΞΛࢦఆ͠·͢ɻ
    IUUQTEPDTBXTBNB[PODPNKB@KQHMVFMBUFTUEHQPQVMBUFEBUBDBUBMPHIUNM
    🤔
    

    View Slide

13. Θ͔ΒΜ͔Β৮ͬͯΈΔ
    

    View Slide

14. σʔλͷ४උ
    $47Λ४උ
    'VMM/BNF $PVOUSZ $SFBUFE"U *E &NBJM
    $PPQFS%FDLPX 4FOFHBM 5IV%FD(.5 ೔ຊඪ४࣌
    .VSSBZ!DSBJHDB
    %PNFOJDP3FJDIFSU -BPT 5IV'FC(.5 ೔ຊඪ४࣌
    .FSM![PFZDB
    "OUXBO3PNBHVFSB /FVUSBM;POF 5IV"VH(.5 ೔ຊඪ४࣌
    /PSNB@+BLVCPXTLJ!DBUIZOFU
    $IBTF%PPMFZ .POHPMJB 5IV%FD(.5 ೔ຊඪ४࣌
    $BSMPT@)FJEFOSFJDI!HBWJOJP
    -POEPO%JDLFOT #VSVOEJ 8FE0DU(.5 ೔ຊඪ४࣌
    $MBVEJOF@)BSSJT!DIBOEMFSDPVL
    5PSSBODF4UBNN 4XFEFO 'SJ+VM(.5 ೔ຊඪ४࣌
    &NJMJB@,FSU[NBOO!TFMJOBDPN
    %S$PEZ+FXFTT 4VSJOBNF 'SJ/PW(.5 ೔ຊඪ४࣌
    &UIZM.PSBS!OJDPMFUUFOBNF
    "TIB1SPTBDDP "SVCB 4VO.BZ(.5 ೔ຊඪ४࣌
    0MBG@,VMBT!BMFDOBNF
    .BWJT,JOH (SFFOMBOE 5IV+VO(.5 ೔ຊඪ४࣌
    *NNBOVFM!OZBIDPVL
    .BUJMEB,JIO &DVBEPS .PO0DU(.5 ೔ຊඪ४࣌
    5PZ!UZSFFLJOGP
    &EB$SPOB *SBR 8FE+BO(.5 ೔ຊඪ४࣌
    SZMFF!OPCMFUW
    
    w μϛʔ$47Λੜ੒
    w ໊લɺࠃɺ࡞੒೔ɺ*%ɺ&.BJMΛ࣋ͬͨσʔλ
    w IUUQXXXEBUBCBTFUFTUEBUBDPNΛར༻ͯ͠ੜ੒
    

    View Slide

15. 4Λ४උ
    σʔλͷ४උ
    w ద౰ͳ໊લͷόέοτ࡞੒
    UFTUHMVF
    w ೥݄೔ΛΩʔόϦϡʔͰύεΛ੾Δʢ೚ҙʣ
    w "QBDIF)JWFͷྲّྀ
    ZZZZNNEE
    w $47ΛΞοϓϩʔυ
    

    View Slide

16. "84(MVF%BUB$BUBMPHͷ४උ
    σʔλϕʔεͷ४උ
    w σʔλϕʔεͷ௥Ճ
    w UFTUEC࡞੒
    ˞τοϓͱҧ͍σʔλϕʔεͷϚωδϝϯτίϯιʔϧ͸೔ຊޠදࣔՄೳ
    

    View Slide

17. "84(MVF%BUB$BUBMPHͷ४උ
    ςʔϒϧͷ४උ
    w ΫϩʔϥΛ࢖༻ͯ͠
    ςʔϒϧ͕௥ՃͰ͖Δ
    

    View Slide

18. "84(MVF%BUB$BUBMPHͷ४උ
    Ϋϩʔϥͷ४උ
    w ద౰ͳ໊લΛઃఆ
    

    View Slide

19. "84(MVF%BUB$BUBMPHͷ४උ
    Ϋϩʔϥͷ४උ
    w ࡞ͬͨ4όέοτΛࢦఆ
    

    View Slide

20. "84(MVF%BUB$BUBMPHͷ४උ
    Ϋϩʔϥͷ४උ
    w *".ϩʔϧΛ࡞੒
    

    View Slide

21. "84(MVF%BUB$BUBMPHͷ४උ
    Ϋϩʔϥͷ४උ
    w ࡞ͬͨσʔλϕʔεΛࢦఆ
    

    View Slide

22. "84(MVF%BUB$BUBMPHͷ४උ
    Ϋϩʔϥͷ࣮ߦ
    w ࡞ͬͨΫϩʔϥΛ࣮ߦ
    

    View Slide

23. "84(MVF%BUB$BUBMPHͷ४උ
    ςʔϒϧ͕࡞੒͞ΕΔ
    

    View Slide

24. "84(MVF%BUB$BUBMPHͷ४උ
    ςʔϒϧͷεΩʔϚΛݟΔ
    w $47ͷϔομʔ
    w ΧϥϜࣗಈੜ੒
    w 4ͷύε
    w ύʔςΟγϣϯͱͯ͠ೝࣝ
    

    View Slide

25. "84(MVF%BUB$BUBMPHʹΞΫηε
    "UIFOBͰࢀরՄೳ
    w ࡞ͬͨ%#ͱςʔϒϧ͕ݟ͑Δ
    w ΫΤϦͰ$47ΛղੳͰ͖Δ
    

    View Slide

26. "84(MVF%BUB$BUBMPHʹΞΫηε
    "UIFOBͰࢀরՄೳ
    w 4ͷύε͕ύʔςΟγϣϯʹͳΔ
    w ޮ཰Α͘ΞΫηεՄೳ
    w ύεΛ੾ͬͯσʔλΛ૿΍ͤΔ
    w Ϋϩʔϥͷ࠶࣮ߦ͕ඞཁ
    

    View Slide

27. (MVFΛ࢖͏͜ͱ͕Ͱ͖ͨ🎉
    

    View Slide

28. ·ͱΊ
    w ղੳ༻σʔλϕʔε͕؆୯ʹ࡞੒Ͱ͖Δ
    w (MVF%BUB$BUBMPHͱ͍͏ػೳΛར༻
    w ΋ͱ΋ͱ"UIFOBͷػೳ͕ͩͬͨ(MVFʹ෼཭͞Εͨ
    w (MVF͸&5-͔Β࢝·ͬͯ%BUB$BUBMPH͕௥Ճ͞Εͨ
    w $47΍+40/ϑΝΠϧͳͲ͸(MVF$SBXMFSͰղੳͰ͖Δ
    w αʔόʔϨεͳͷͰઃఆ͚ͩͰ࢖͑Δ
    w σʔλΧλϩάͷ%#ʹςʔϒϧ͕ࣗಈੜ੒͞ΕΔ
    w "UIFOB΍2VJDL4JHIUͰղੳͰ͖Δ
    w 4ͷύε͸ύʔςΟγϣϯͱͯ͠ೝࣝ
    w ޮ཰Α͘σʔλղੳ͕Ͱ͖Δ
    (MVFΛ࢖͏ͱαʔόϨεͰ෼ੳՄೳ
    

    View Slide

29. ࢀߟʹͨ͠ॻ੶
    "84Ͱ͸͡ΊΔσʔλϨΠΫ
    

    View Slide

30. ͓·͚୯ମͷϑΝΠϧΛղੳ͍ͨ͠ͳΒ
    44FMFDU͕؆୯
    

    View Slide

31. ஥ؒืूதͰ͢ʂ
    w ϓϩδΣΫτϚωʔδϟʔ
    w 1.0#31%9ܦݧऀ
    w όοΫΤϯυΤϯδχΞ
    w "84 (P1ZUIPO %PDLFSܦݧऀ
    1IPUP1",65"40
    'POU.PSJTBXB
    

    View Slide