Slide 1

Slide 1 text

4ʹ͋ΔσʔλΛ"UIFOBͷΫΤϦͰऔಘ ͯ͠-BNCEB 1BOEBT ͰՃ޻ͯ͠Έͨ Ϋϥεϝιουגࣜձࣾɹּݪ޺ ",*#""840OMJOF

Slide 2

Slide 2 text

ຊ೔͓࿩͢͠Δ͜ͱ wҰൠతʹɺ"UIFOB͸෼ੳ༻్ʹ࢖͍·͕͢ɺࠓճ͸4όέοτʹ͋Δ σʔλΛऔಘ͢Δ༻్Ͱ࢖ͬͨࡍͷ͓࿩Λ͠·͢ɻ w"UIFOBΫΤϦΛ-BNCEB͔Β࣮ߦ͠ɺΫΤϦ݁ՌΛ-BNCEBͰऔಘͨ͠ ޙɺ1BOEBTͰσʔλՃ޻Λߦ͍·ͨ͠ɻ w"UIFOBΫΤϦ࣮ߦɾ݁Ռऔಘʹ͸ɺ"84%BUB8SBOHMFS͕ศརͰ͢ɻ

Slide 3

Slide 3 text

ࣗݾ঺հ w ּݪ޺ w ೥݄δϣΠϯ w σʔλΞφϦςΟΫεࣄۀຊ෦ϏδωειϦϡʔγϣϯ෦Ϗοά σʔλνʔϜ w ޷͖ͳ"84αʔϏε4ɺ-BNCEBɺ4BHF.BLFS w ৽ׁݝ৽ׁࢢࡏॅ w +"846(৽ׁࢧ෦ʗ1ZUIPOػցֶशษڧձJO৽ׁʗ+B445৽ׁʗ 48"/**ʗFUD @kasacchiful @kasacchiful

Slide 4

Slide 4 text

+"846(/JJHBUB ʮϓνΩϟονΞοϓձʯΛɺຖि໦༵ΑΓ։࠵ w IUUQTKBXTVHOJJHBUBDPOOQBTTDPN

Slide 5

Slide 5 text

೔ ։࢝ w IUUQTQZNMOJJHBUBDPOOQBTTDPNFWFOU 1ZUIPOػցֶशษڧձJO৽ׁ

Slide 6

Slide 6 text

ຊ୊ AthenaΫΤϦͰσʔλऔಘ

Slide 7

Slide 7 text

"NB[PO"UIFOB 4಺ͷσʔλΛඪ४42-Ͱ෼ੳՄೳͳαʔϏε w ඞཁͳ͜ͱ w 4όέοτʹ͋Δ෼ੳର৅ͷσʔλϑΝΠϧΛࢦఆ w εΩʔϚΛఆٛ (MVF%BUB$BUBMPHΛ࢖༻ w 42-ΫΤϦΛॻ͍࣮ͯߦ

Slide 8

Slide 8 text

͜Μͳ༻్Ͱ"UIFOBΛ࢖ͬͨ ෼ੳͰ͸ͳ͘ɺ&5-ͷz&zͷ༻్Ͱ࢖༻ w &YUSBDU நग़ Ͱ"UIFOBΛ࢖༻ w 4όέοτʹ͋ΔϑΝΠϧͷ਺͕ඇৗʹଟ͍ w ύʔςΟγϣϯԽ͞ΕͯϑΝΠϧ͕֨ೲ͞Ε͍ͯΔ w ෳ਺ϑΝΠϧʹ෼ࢄ͞Ε͍ͯΔσʔλͷҰ෦෼͚ͩ࢖͍͍ͨ w ෳ਺ϑΝΠϧͷσʔλΛ࿈݁ͯ͠࢖͍͍ͨ w 5SBOTGPSN-PBE͸-BNCEBͰ

Slide 9

Slide 9 text

ࢼͯ͠ΈΔ 4಺ʹ͋Δ1BSRVFUσʔλΛ"UIFOBΫΤϦͰऔಘ w Πϯϓοτ4όέοτʹ͋Δ1BSRVFUϑΝΠϧ w ύʔςΟγϣϯԽ͞Εͯ֨ೲ w "UIFOBͰΫΤϦ࣮ߦˠ݁ՌΛ-BNCEBͰऔಘ w ΫΤϦ݁Ռ͕4όέοτʹอଘ͞ΕΔͷͰɺ-BNCEBͰऔಘ ͢Δ w ݁Ռ಺༰Λ֬ೝ w ࠓճ͸"1*ୟ͍ͯ-BNCEBىಈͤ͞ɺΫΤϦ݁ՌΛฦ͚ͩ͢ʹ ͢Δ

Slide 10

Slide 10 text

·ͣ͸࣮ߦ݁Ռ͔Β -BNCEB͔Β"UIFOBΫΤϦ࣮ߦ͠ɺ݁ՌΛ1BOEBTʹ֨ೲͯ͠+40/ग़ྗ

Slide 11

Slide 11 text

Πϯϓοτσʔλ 4όέοτ಺ʹ͋Δ1BSRVFUϑΝΠϧͷσʔλ w l4EBUBMBLFCVDLFUMPHVTFS@JEEBUFTBNQMFQBSRVFUz w VTFS@JEͱEBUF ZZZZNNEE ͰύʔςΟγϣϯԽ log1/1/2021/11/01/sample_data_64_0.parque t log1/1/2021/11/02/sample_data_59_0.parque t log1/1/2021/11/03/sample_data_56_0.parque t log1/1/2021/11/04/sample_data_57_0.parque t log1/1/2021/11/05/sample_data_58_0.parque t log1/1/2021/11/06/sample_data_65_0.parque t log1/1/2021/11/07/sample_data_61_0.parque t log1/1/2021/11/08/sample_data_63_0.parque t log1/1/2021/11/09/sample_data_62_0.parque t log1/1/2021/11/10/sample_data_60_0.parquet

Slide 12

Slide 12 text

"UIFOBͷςʔϒϧఆٛ (MVF%BUB$BUBMPH࢖༻ w ࠓճ͸4FSWFSMFTT'SBNFXPSLͷSFTPVSDFTͰఆٛ $MPVE'PSNBUJPO w (MVF%BUBCBTF(MVF5BCMF"UIFOB8PSL(SPVQ

Slide 13

Slide 13 text

"UIFOBͷςʔϒϧఆٛ (MVF%BUB$BUBMPH࢖༻ w 5BCMF͸εΩʔϚઃఆ΍ύʔςΟγϣϯઃఆͳͲఆٛ

Slide 14

Slide 14 text

-BNCEB࣮૷ -BNCEB͔Β"UIFOBʹΫΤϦ࣮ߦ͠ɺ݁ՌΛऔಘ͢Δ w -BNCEBʹ͸4ɺ"UIFOBɺ(MVF΁ͷΞΫηε͕Ͱ͖Δݖݶ͕ඞཁ w ݁ՌΛ-BNCEB্ͷ1BOEBTσʔλϑϨʔϜʹ֨ೲͰ͖Ε͹ɺ͋ͱ͸1BOEBTΛૢ࡞ͯ͠σʔ λՃ޻Ͱ͖Δ

Slide 15

Slide 15 text

ΫΤϦ࣮ߦ TUBSU@RVFSZ@FYFDVUJPO

Slide 16

Slide 16 text

ΫΤϦ࣮ߦ RVFSZ@FYFDVUJPO@JE͔ΒɺΫΤϦͷ࣮ߦঢ়ଶΛ֬ೝ

Slide 17

Slide 17 text

ΫΤϦ࣮ߦ ࣮ߦ݁ՌΛऔಘ

Slide 18

Slide 18 text

"UIFOBΫΤϦ࣮ߦͷࢀߟهࣄ IUUQTEFWDMBTTNFUIPEKQBSUJDMFTSVOBNB[POBUIFOBTRVFSZXJUIBXTMBNCEBKB

Slide 19

Slide 19 text

࣮ߦ݁Ռͷදࣔ औಘ࣮ͨ͠ߦ݁Ռ͸ɺ1BOEBTσʔλϑϨʔϜΛ+40/Խͨ͠಺༰Λฦ͢

Slide 20

Slide 20 text

࣮ߦ݁Ռ ࠶ܝ -BNCEB͔Β"UIFOBΫΤϦ࣮ߦ͠ɺ݁ՌΛ1BOEBTʹ֨ೲͯ͠+40/ग़ྗ

Slide 21

Slide 21 text

๊͍ͨײ৘ AthenaΫΤϦ࣮ߦ ໘౗ͩͳ

Slide 22

Slide 22 text

"84%BUB8SBOHMFS͕ศར IUUQTHJUIVCDPNBXTMBCTBXTEBUBXSBOHMFS

Slide 23

Slide 23 text

"84%BUB8SBOHMFSͷಛ௃ 1BOEBTͷػೳΛ"84ʹ֦ு͢ΔɺΦʔϓϯιʔεͷ1ZUIPOϥΠϒϥϦ w 1BOEBTσʔλϑϨʔϜͱ"84ͷσʔλؔ࿈ͷαʔϏεͱΛ͏·͘઀ଓͯ͘͠ΕΔ w 3FETIJGU(MVF"UIFOB&.3ͳͲ w ௨ৗͷ&5-λεΫʹඞཁͳؔ਺͕ἧ͍ͬͯΔ

Slide 24

Slide 24 text

ΫΤϦ࣮ߦ BUIFOBSFBE@TRM@RVFSZͰαΫοͱσʔλϑϨʔϜऔಘ

Slide 25

Slide 25 text

࣮ߦ݁Ռ "84%BUB8SBOHMFS࢖༻ -BNCEB͔Β"UIFOBΫΤϦ࣮ߦ͠ɺ݁ՌΛ1BOEBTʹ֨ೲͯ͠+40/ग़ྗ

Slide 26

Slide 26 text

஫ҙ఺ ϑΝΠϧαΠζ͕େ͖ͯ͘ɺͦͷ··ͩͱ-BNCEBʹ৐Βͳ͍ w -BNCEBͷσϓϩΠύοέʔδ͸ඇѹॖ࣌ʹ.#ҎԼʹ͢Δඞཁ͕͋Δ w "84%BUB8SBOHMFSΛී௨ʹQJQΠϯετʔϧ͢Δͱɺ.#௒͑Δ w (JU)VCͷ3FMFBTFϖʔδʹ͋Δɺ-BNCEB-BZFS༻ͷ[JQϑΝΠϧΛར༻͠Α͏

Slide 27

Slide 27 text

"84%BUB8SBOHMFSΛ-BNCEBͰ࢖͏ࢀߟهࣄ IUUQTEFWDMBTTNFUIPEKQBSUJDMFTVTFBXTXSBOHMFSJOBXTMBNCEB

Slide 28

Slide 28 text

4UFQ'VODUJPOTͷεςʔτϚγϯ্Ͱͷ࣮ߦ 4಺ʹ͋Δ1BSRVFUσʔλΛ"UIFOBΫΤϦͰऔಘ w Ұ൪࠷ޙͷ.BQλεΫʹ࣮ͯߦ w લஈͷ1BSBMMFMλεΫʹͯॲཧ͞Εͨෳ਺ϑΝΠϧ͔Βɺඞ ཁͳσʔλ͚ͩநग़͢Δࡍʹ"UIFOB࢖༻ w ฒྻ࣮ߦͷࡍ͸ɺ"UIFOBͷΫΥʔλʹ஫ҙ

Slide 29

Slide 29 text

"UIFOBͷΫΥʔλ "UIFOBͷ4UBSU2VFSZ&YFDVUJPOͷΫΥʔλ w .BQλεΫͰ"UIFOBͷΫΤϦΛ࣮ߦ͢ΔͱɺΫΥʔλʹҾ͔͔ͬΓ΍͍͢ w ্ݶ؇࿨ਃ੥͢Δ͔ɺεςʔτϚγϯͷ.BY$PODVSSFODZͰಉ࣮࣌ߦ਺Λ཈͑Δ͔ IUUQTEPDTBXTBNB[PODPNKB@KQBUIFOBMBUFTUVHTFSWJDFMJNJUTIUNM

Slide 30

Slide 30 text

·ͱΊ • ҰൠతʹɺAthena͸෼ੳ༻్ʹ࢖͍·͕͢ɺࠓճ͸ S3όέοτʹ͋ΔσʔλΛऔಘ͢Δ༻్Ͱ࢖͍·͠ ͨɻ • AthenaΫΤϦΛLambda͔Β࣮ߦ͠ɺΫΤϦ݁ՌΛऔ ಘͯ͠PandasͰσʔλՃ޻Ͱ͖·͢ɻ • AthenaΫΤϦ࣮ߦɾ݁Ռऔಘʹ͸ɺAWS Data Wrangler͕ศརͰ͢ɻ

Slide 31

Slide 31 text

No content