݄ιϑτόϯΫגࣜձࣾதཬߒ೭ݸҎ্ͷ)BEPPQΫϥελʔΛ͍ࣺͯΔํ๏)BEPPQ4QBSL$POGFSFODF+BQBO
View Slide
ࣗݾհதཬߒ೭ φΧβτώϩϢΩॴଐιϑτόϯΫגࣜձࣾגࣜձࣾ"HPPQϙδγϣϯόοΫΤϯυνʔϜϦʔμʔ"845FDIOJDBM.FJTUFS
͜ΜͳهࣄΛॻ͍͍ͯ·͢IUUQTRJJUBDPNOBLB[BYJUFNTFEEGBB
Ґஔใऩू ղੳ ਓͷྲྀΕ
.VTUͷϩάΛؒόον։࢝·Ͱʹ%8)ʹೖ8BOU%8)ͷೖૣ͚Εૣ͍΄Ͳϕλʔػೳ໘ඇػೳ໘ϩάͷϘϦϡʔϜԯ݅5#Φʔμʔࠓޙϩάྔ͕૿͑ଓ͚ΔͨΊεέʔϥϏϦςΟ͕ඞཁίετޮٴͼ҆ఆੑ͕ߴ͍͜ͱ&5-ॲཧͷཁ݅
ϩάऩूγεςϜ4 3FETIJGU,JOFTJT+FOLJOTPO&$%FWFMPQFST%BUB"OBMZTUT&-# &$ऩूɾੵ &5- ղੳ&$&$':ΞʔΩςΫνϟʔ
ϩάऩूγεςϜ4 3FETIJGU,JOFTJT+FOLJOTPO&$%FWFMPQFST%BUB"OBMZTUT&-# &$ऩूɾੵ &5- ղੳ&$&$':ΞʔΩςΫνϟʔεέʔϥϏϦςΟͳͲʹ՝
࣌ؒͷϩάΛೖྗɺ1ZUIPONVMUJQSPDFTTJOHͰ$16Λͬͯ%8)ʹϩʔυՄೳͳܗࣜʹม͋Δͷରͷϩά͕एׯҟͳΔͨΊ૿͑ଓ͚Δϩάʹରͯ͠&$ͷεέʔϧΞοϓͰରԠ࣌ͷ&5-ॲཧͷ࣮
DYMBSHFϦϦʔεW$16(J#.FNֹ݄ສԁDYMBSHFԼ०ϦϦʔεW$16(J#.FNֹ݄ສԁεέʔϧΞοϓ಄ଧͪϦεΫίϯϐϡʔςΟϯά࠷దԽΠϯελϯελΠϓ
࣌&5-ॲཧ ࣌&5-ॲཧ͋Δ࣌ؒͷॲཧ͕Ҿ͘ͱޙଓ͕٧·Δ&$ 4࣌&5-ॲཧ&5-ॲཧΠϝʔδ
ϩάऩूγεςϜ4 3FETIJGU,JOFTJT+FOLJOTPO&$%FWFMPQFST%BUB"OBMZTUT&-# &$ऩूɾੵ &5- ղੳ&$&$':ΞʔΩςΫνϟʔ՝ͷղܾʹணख
৽ΞʔΩςΫνϟʔߏεέʔϧΞτϞσϧ4QBSLPO&.3Λ࠾༻ॲཧͷಠཱੑ࣌ؒͷ&5-ॲཧΛΫϥελʔ͕୲͍ࣺͯՄೳͳ&.3όονॲཧج൫
ϩάऩूγεςϜ43FETIJGU+FOLJOTPO&$%FWFMPQFST%BUB"OBMZTUT&5- ղੳऩूɾੵʜʜ4QBSLPO&.3)JWFPO&.3ʜ4-BNCEB,JOFTJT&-# &$৽ΞʔΩςΫνϟʔ(MVF
৽ΞʔΩςΫνϟʔͷϙΠϯτ &.3ͷεςοϓػೳͷ׆༻ -BNCEBʹΑΔ&.3Ϋϥελʔͷ੍ޚ ϑϧϚωʔδυ)JWFϝλετΞͷ׆༻ ΞυϗοΫੳ༻&.3Ϋϥελʔͷఏڙ
&.3ͷεςοϓػೳͷ׆༻ 1 2 3 4εςοϓҙͷॲཧΛॱ൪ʹ࣮ߦ͢ΔΈશεςοϓྃޙʹΫϥελʔΛऴྃ͢Δͷ੍ޚ͕؆୯ʹߦ͑Δ݅ذ܁Γฦ͕͠ͳ͍γϯϓϧͳόονॲཧϑϩʔͰ͋ΕεςοϓػೳͰेΧόʔՄೳ
&5-ॲཧͷεςοϓ&.3ຊ൪ࢿࡐ+BS4IFMMςετࢿࡐ4IFMM+BSੜϩά1BSRVFU &5-ॲཧ༻ͷࢿࡐΛ4͔Βμϯϩʔυ TEJTUDQͰॲཧରͷϩάΛू TQBSLTVCNJUͰσʔλՃͱ1BSRVFUมΛ࣮ࢪ1 23
-BNCEBʹΑΔΫϥελʔ੍ޚ 1 2 3 4&.3Ϋϥελʔىಈ༻ͷ-BNCEBؔΛ࣮1ZUIPOͰ3VO+PC'MPXΛίʔϧ3VO+PC'MPXύϥϝʔλʔ͕ඇৗʹଟ͍ઃఆϑΝΠϧϥΠϒϥϦ)0$0/Ͱཧ͠࠶ར༻ੑΛ্1MBZ'SBNFXPSLͳͲͰ༗໊ͳ-JHIUCFOE͕ࣾ։ൃϩάऩूγεςϜͳͲͰ1MBZ'SBNFXPSLΛ׆༻͓ͯ͠Γ)0$0/ʹ׳Ε͍ͯͨͷ͕࠾༻ཧ༝
&5-༻Ϋϥελʔىಈϑϩʔ 1 2 3 4Name = hourly-etlReleaseLabel = emr-5.20.0Steps = [{spark-submithourly-etl.jar${YYYYMMDD}/${HH}}...IPVSMZDPOGIPVSMZDPOG)0$0/ύʔε3VO+PC'MPX࣮ߦ
&5-ॲཧͷϦΧόϦʔޭ ޭ ࣦഊ͕ҟৗऴྃ1 2 3 4ϦΧόϦ༻ىಈ༻Ϋϥελʔͷঢ়ଶΛఆظతʹνΣοΫىಈ༻-BNCEBΛݺͼग़ࣦ͠ഊͨ࣌ؒ͠ͷΫϥελʔΛىಈ
ϑϧϚωʔδυ)JWFϝλετΞ 1 2 3 4"84(MVF%BUB$BUBMPH<&.3ରԠ>Ҏલ.Z42-PO3%4Λ)JWFϝλετΞͱͯ͠ར༻3%4͕410'ʹͳΒͳ͍Α͏ʹؾΛݣ͏ඞཁ͕͋ͬͨΫϥελʔΛେྔʹىಈ͢ΔͱଓͰ͖ͳ͍FUD(MVF%BUB$BUBMPHϑϧϚωʔδυ͔ͭಉ࣌ଓ੍ݶͳ͠
ΞυϗοΫੳ༻ΫϥελʔΤϯδχΞΞφϦετ͕؆୯ʹ&.3ΫϥελʔΛىಈՄೳʹ1 2 3 4
Ϋϥελʔͷམͱ͠Εࢭ 1 2 3 4I I Iࢹ༻ΫϥελʔͷՔಇ࣌ؒΛνΣοΫ4MBDL௨BEIPDDMVTUFS
ίετͷຖ࣌ͷ&5-ॲཧҎલΑΓ҆ՁʹεϙοτΠϯελϯεΛੵۃతʹ׆༻͍ͯ͠ΔͨΊ4શମͷίετ૿ՃΞυϗοΫੳ͕ؾܰʹͰ͖ΔͨΊੳ݁Ռͷσʔλ͕4ʹੵ͍͍ͯͬͯ͠ΔෆཁͳσʔλΛఆظతʹআ͢ΔΑ͏ͳΈ͕ඞཁ
ϩάऩूγεςϜ43FETIJGU+FOLJOTPO&$%FWFMPQFST%BUB"OBMZTUT&5- ղੳऩूɾੵʜʜ4QBSLPO&.3)JWFPO&.3ʜ4-BNCEB,JOFTJT&-# &$࠶ܝ৽ΞʔΩςΫνϟʔ(MVF
THANK YOU!ANY QUESTIONS?