Link
Embed
Share
Beginning
This slide
Copy link URL
Copy link URL
Copy iframe embed code
Copy iframe embed code
Copy javascript embed code
Copy javascript embed code
Share
Tweet
Share
Tweet
Slide 1
Slide 1 text
ႈੑΛߟྀͨ͠ σʔλ࿈ܞδϣϒͷઃܭ Hirohito Sasakawa data engineer at M3, inc.
Slide 2
Slide 2 text
ΞδΣϯμ • ΤϜεϦʔͷσʔλج൫ͷߏ • σʔλ࿈ܞδϣϒͷߏ • σʔλ࿈ܞͱႈੑ • ચ͍ସ͑ • ࠩߋ৽ • ·ͱΊ
Slide 3
Slide 3 text
M3ͷσʔλج൫ͷߏ 2VFSZ 2VFSZ 2VFSZ 1SPEVDUJPO … 3FQPSU … 4ZOD 4ZOD 2VFSZ #JH2VFSZ
Slide 4
Slide 4 text
M3ͷσʔλج൫ͷߏ 2VFSZ 2VFSZ 2VFSZ 1SPEVDUJPO … 3FQPSU … 4ZOD 2VFSZ 4ZOD #JH2VFSZ
Slide 5
Slide 5 text
σʔλ࿈ܞδϣϒͷߏ "QQMJDBUJPO %#T #JH2VFSZ %JHEBH &NCVML %JHEBHͱ&NCVMLͷར༻ ར༻࣮ଟ ϫʔΫϑϩʔɺσʔλநग़ͷ࣮͕؆୯ σʔλιʔεʹԠͨ͡ϓϥάΠϯػߏ .64"ग़ுه.64"Ͱσʔλج൫ڥඋͱ,1*ϞχλϦϯ άπʔϧͷಋೖΛ࣮ࢪ͠·ͨ͠ IUUQTXXXNUFDICMPHFOUSZNVTBEFWFMPQNFOUEBUBQMBUGPSN
Slide 6
Slide 6 text
Embulkͷ࣮ ೖྗσʔλʹ͍ͭͯͷઃఆ %#ଓใҎ֎ ΄΅42- ग़ྗσʔλʹ͍ͭͯͷઃఆ
Slide 7
Slide 7 text
BigQuery্ͷσʔλ "QQMJDBUJPO %#T 4PVSDF ݩςʔϒϧͷ ίϐʔ .BSU සग़͢ΔՃΛ ༧Ίద༻ͨ͠ σʔλ
Slide 8
Slide 8 text
ػցֶशϓϩμΫτ͔Βͷσʔλऔಘ "QQMJDBUJPO %#T 4PVSDF .BSU NEPXOMPBEFS .-1SPEVDU #2ʹΞΫηεͯ͠ EBUBGSBNFܗࣜͰ ΠΠײ͡ʹσʔλΛ औಘ͢Δ 1ZUIPOQBDLBHF
Slide 9
Slide 9 text
σʔλ࿈ܞδϣϒͷߏ (࠶ܝ) "QQMJDBUJPO %#T 4PVSDF ݩςʔϒϧͷ ίϐʔ .BSU සग़͢ΔՃΛ ༧Ίద༻ͨ͠ σʔλ
Slide 10
Slide 10 text
σʔλ࿈ܞδϣϒͷߏ (࠶ܝ) "QQMJDBUJPO %#T 4PVSDF ݩςʔϒϧͷ ίϐʔ .BSU සग़͢ΔՃΛ ༧Ίద༻ͨ͠ σʔλ ͜͜ͱ ͜͜Λ ָʹ͍ͨ͠
Slide 11
Slide 11 text
δϣϒͷࣦഊͱϦΧόϦ δϣϒ͍ΖΜͳཧ༝Ͱࣦഊ͢Δ σʔλߋ৽ͷԆ σʔλ૿Ճ 4BB4ଆͷෆ۩߹ͳͲͳͲ ϦΧόϦδϣϒͷ࠶࣮ߦ͚ͩͰࡁ·͍ͤͨ ΦϖϨʔγϣϯϛεͷࢭ ్த·ͰՃ͞ΕͨσʔλΛআͯ͠ʜ☠ ႈͳσʔλ࿈ܞδϣϒΛ࣮͍ͨ͠
Slide 12
Slide 12 text
σʔλ࿈ܞύλʔϯ1: ચ͍ସ͑ "QQMJDBUJPO %#T ຖճσʔλΛ্ॻ͖͢ΔPSผςʔϒϧʹ͢Δ ΄ͱΜͲͷ߹͜ΕͰ͍͍ ႈͳͷͰϦΧόϦ୯७ʹ࠶࣮ߦͰ0, ศརʂ
Slide 13
Slide 13 text
σʔλ࿈ܞύλʔϯ2: ࠩߋ৽ "QQMJDBUJPO %#T લճ࿈ܞͱͷࠩͷΈΛ࿈ܞ σʔλαΠζ͕େ͖͍ςʔϒϧͰඞཁʹͳΔ ୯७ʹ࣮͢ΔͱႈʹͳΒͳ͍ δϣϒࣦഊ࣌ͷϦΧόϦେม ೖͬͯ͠·ͬͨσʔλΛಛఆআͯ͠Γ͢ͳͲ
Slide 14
Slide 14 text
σʔλ࿈ܞύλʔϯ2: ࠩߋ৽ "QQMJDBUJPO %#T લճ࿈ܞͱͷࠩͷΈΛ࿈ܞ σʔλαΠζ͕େ͖͍ςʔϒϧͰඞཁʹͳΔ ୯७ʹ࣮͢ΔͱႈʹͳΒͳ͍ δϣϒࣦഊ࣌ͷϦΧόϦେม ೖͬͯ͠·ͬͨσʔλΛಛఆআͯ͠Γ͢ͳͲ
Slide 15
Slide 15 text
ႈͳࠩߋ৽ (1/2) "QQMJDBUJPO %#T લճͷςʔϒϧ
Slide 16
Slide 16 text
ႈͳࠩߋ৽ (1/2) "QQMJDBUJPO %#T લճͷςʔϒϧ લճͷςʔϒϧ Λίϐʔ 4UFQ
Slide 17
Slide 17 text
ႈͳࠩߋ৽ (1/2) "QQMJDBUJPO %#T લճͷςʔϒϧ લճͷςʔϒϧ Λίϐʔ 4UFQ લճ͔Βͷ ࠩσʔλΛநग़ 4UFQ
Slide 18
Slide 18 text
ႈͳࠩߋ৽ (2/2) "QQMJDBUJPO %#T લճͷςʔϒϧ ྆ऀΛ݁߹ͨ͠ ςʔϒϧΛ࡞ 4UFQ
Slide 19
Slide 19 text
ႈͳࠩߋ৽ (2/2) "QQMJDBUJPO %#T લճͷςʔϒϧ ྆ऀΛ݁߹ͨ͠ ςʔϒϧΛ࡞ 4UFQ 4UFQ͔Β࠶࣮ߦͰ0, ඞཁ͕͋ΕɺͲͷϑΣʔζ͔Β ͰΓͤΔ ϦΧόϦ
Slide 20
Slide 20 text
ࣦഊ࣌ͷϦΧόϦ ϦΧόϦ࠶࣮ߦͰ0, ඞཁʹԠͯ͡ҙͷ4UFQ͔Βͷ࠶࣮ߦ͕Մೳ 4UFQ·Ͱਖ਼ৗऴ͍ྃͯ͠ΔͷͰɺ4UFQ͔Β࠶࣮ߦͳͲ ͖ͬ͞ௐͯͨΒҎԼͷࢿྉͰղઆ͞Ε͍ͯ·ͨ͠ʜ ;0;0508/ͷ%8)Λ3FETIJGU͔Β#JH2VFSZʹ ͓Ҿӽͨ͠͠ IUUQTTQFBLFSEFDLDPNTIJP[BLJNPWJOH[P[PUPXOEXIGSPNSFETIJGUUPCJHRVFSZ
Slide 21
Slide 21 text
ҙ ͜ͷख๏ͷҙࣄ߲ ͦͦσʔλ͕ଟ͗ͯ͢ɺ͜ͷख๏ΛͱΔͷͰաڈͷ ςʔϒϧΛอ͓࣋ͯ͘͠ͷݱ࣮తͰͳ͍ աڈͷ͋Δ࣌ʹΔʹσʔλͷ࣋ͪํΛ͢Δ ඞཁ͕͋Δ FHCJUFNQPSBMEBUBNPEFMJOH ૢ࡞ཤྺ࣌ࢦఆΞΫηεͷ࣮ݱ#J5FNQPSBM%BUB.PEFMͷ࣮ફ IUUQTTQFBLFSEFDLDPNGJNQMFNFOUJOHDPNNBOEIJTUPSZBOEUFNQPSBMBDDFTT
Slide 22
Slide 22 text
·ͱΊ ΤϜεϦʔͷσʔλج൫ͱ࿈ܞδϣϒʹ͍ͭͯফԽ #JH2VFSZXJUI %JHEBH&NCVML Ғେ ႈͳσʔλ࿈ܞδϣϒͷઃܭํ๏Λհ ચ͍ସ͑ ࠩߋ৽