(PPHMF#JH2VFSZͷ/BPZB*UP,"*;&/QMBUGPSN*OD:"1$"TJB5PLZP˞(PPHMFࣾͷճ͠ऀͰ͋Γ·ͤΜ
View Slide
ΞδΣϯμ• #JH2VFSZ֓؍• #JH2VFSZͷ෦• ,"*;&/QMBUGPSN*ODͰͷ͍Ͳ͜Ζ
#JH2VFSZ֓؍
(PPHMF#JH2VFSZ
#JH2VFSZͱ• ڊେͳσʔλͷ42- ͳͲΛඵͰ࣮ߦ͢ΔΫϥυαʔϏε– ԯϨίʔυΛඵ ˞– 8FCΠϯλʔϑΣʔε͓Αͼ3&45"1*• (PPHMFࣾͰΘΕ͖ͯͨ%SFNFMΛαʔϏεԽ– ݄$MPTFEϦϦʔε– ݄Ұൠެ։– ܧଓతʹόʔδϣϯΞοϓ– ݄#JH2VFSZ4USFBNJOH˞(PPHMFͷދͷࢠʮ#JH2VFSZʯΛ'MVFOUEϢʔβʔ͕Θͳ͍ཧ༝͕ͳ͘ͳͬͨཧ༝IUUQRJJUBDPNLB[VOPSJJUFNTBDBDCCBBBG
ͲΜͳ͜ͱʹΘΕΔ͔• Ϣʔεέʔε– ϩάղੳ– %BUBXBSF)PVTF– • ͍ͯͳ͍༻్– ۀ%#͍3%#.4Ͱͳ͍Αɺͱ͍͏͜ͱ
#JH2VFSZͳ͍͔ͥ• جຊɺϑϧεΩϟϯͰ͕ΜΔ– 3%#.4ͷ#5SFFΠϯσοΫεͱ͔ͳ͍• 42-Λࢄॲཧ– .11 .BTTJWFMZ1BSBMMFM1SPDFTTJOH2VFSZ&OHJOF%SFNFM• ઍͷσΟεΫͱߴωοτϫʔΫͰεέʔϧΞτ– 5#ͷσʔλΛඵͰϦʔυ͢Δ*0
ͨͩ͠• ͍3%#.4Ͱͳ͍• େਓͰҰʹ͏ͷͰͳ͍– ओʹόονॲཧʹ͏• εΩʔϚϨεͰͳ͍5#نσʔλͰઢܗҎԼͰεέʔϧ͢Δ͕ɺٯʹখ͞ͳσʔλͰඵͷΦʔόʔϔου͕͋ΔͷͰ
BigQuery読書会、@harukasan 資料より引用
ଞͷྨࣅ࣮ͱͷϙδγϣχϯά• -BSHF#BUDI– ҆ఆͯ͠ڊେͳόονΛ࣮ߦͰ͖Δ– ΫΤϦ࣮ߦ࣌ͷΦʔόʔϔου͕େ͖͍ ेඵʙे– .BQ3FEVDFɺ)BEPPQ)JWF• 4IPSU#BUDI– ΫΤϦ࣮ߦ࣌ͷΦʔόʔϔου͕ඵ– ΞυϗοΫΫΤϦʹ͍͍ͯΔ– .112VFSZ&OHJOF1SFTUPɺ*NQBMBɺ#JH2VFSZ %SFNFM• 4USFBN1SPDFTTJOH– όον࣮ߦͰ͖ͳ͍͕ετϦʔϜʹରͯ͠ϦΞϧλΠϜॲཧͰ͖Δ– /PSJLSBɺ"QBDIF,BGLBɺ5XJUUFS4UPSNFUD"NB[PO3FETIJGU4IPSU#BUDI ৄ͘͠ͳ͍ͷͰলུcf. Batch processing and Stream processing by SQL h;p://www.slideshare.net/tagomoris/hcj2014-‐sql
Ձ֨• ྉۚ– σʔλอ(#݄– ΫΤϦ5# εΩϟϯͨ͠σʔλͷαΠζ"NB[PO4ΑΓ࣮͍҆($1ؔͷษڧձ͍͘ͱ ͷແྉνέοτΒ͑ͨΓ͠·͢
#JH2VFSZͷ෦ ͚ͩ͢͜͠
(PPHMF#JH%BUB4UBDL• ʰ(PPHMFΛࢧ͑Δٕज़ʱ– #JH%BUB4UBDL– ('4ɺ#JH5BCMFɺ.BQ3FEVDFFUD• #JH%BUB4UBDL– #JH%BUB4UBDLͷ্ʹߏங͞Εͨɺͷ՝Λղফ͢Δ࣮܈– $PMPTTVT .FHBTUPSF 4QBOOFS 'MVNF+BWB %SFNFMطʹ(PPHMFࣾ#JH%BUB4UBDLͩͱ͔͍͏ͪΒ΄Β
#JH2VFSZͷٕज़ελοΫ(PPHMF'JMF4ZTUFN ('4$PMPTTVT'JMF4ZTUFN $'4$PMVNO*0%SFNFMࢄ'4('4ͷվྑܕ'4ৄࡉඇެ։#JH2VFSZͷͨΊͷྻࢦϑΝΠϧϑΥʔϚοτฒྻ42-࣮ߦΤϯδϯσʔληϯλʔΛ·͍ͨͰࢄ͞ΕͯΔσʔλΛฒྻ͔ͭߴʹऔಘͰ͖ΔΒ͍͠
$PMVNO*0Dremel: InteracIve Analysis of Web-‐Scale Datasets h;p://research.google.com/pubs/archive/36632.pdf ߦͰͳ͘ྻ୯ҐͰɻಛఆྻΛγʔέϯγϟϧʹಡΊΔͭ$PMPTTVTͰฒྻಡΈࠐΈ
%SFNFMDremel: InteracIve Analysis of Web-‐Scale Datasets h;p://research.google.com/pubs/archive/36632.pdf
Root Mixer Mixer 1 Shard 0-‐8 Mixer 1 Shard 9-‐16 Mixer 1 Shard 17-‐24 Shard 0 Shard 10 Shard 12 Shard 20 Shard 24 Distributed Storage (e.g., CFS) Dremel serving tree Google BigQuery AnalyIcs P.284 Chapter 9 Understanding Query ExecuIon ࢄ
Root Mixer Mixer 1 Shard 0-‐8 Mixer 1 Shard 9-‐16 Mixer 1 Shard 17-‐24 Shard 0 Shard 10 Shard 12 Shard 20 Shard 24 Distributed Storage (e.g., CFS) Dremel serving tree Google BigQuery AnalyIcs P.284 Chapter 9 Understanding Query ExecuIon $'4$PMVNO*0Ͱಛఆྻͷσʔλ͕Ұ෦ฦͬͯ͘Δࢄ ू
Root Mixer Mixer 1 Shard 0-‐8 Mixer 1 Shard 9-‐16 Mixer 1 Shard 17-‐24 Shard 0 Shard 10 Shard 12 Shard 20 Shard 24 Distributed Storage (e.g., CFS) Dremel serving tree Google BigQuery AnalyIcs P.284 Chapter 9 Understanding Query ExecuIon $'4$PMVNO*0Ͱಛఆྻͷσʔλ͕Ұ෦ฦͬͯ͘ΔྻΛॱ൪ʹಡΈߦΛऔಘɻ8)&3&۟ͳͲΛݟͯඞཁͳߦͷΈʹߜΓϝϞϦͰอ࣋ࢄ ू
Root Mixer Mixer 1 Shard 0-‐8 Mixer 1 Shard 9-‐16 Mixer 1 Shard 17-‐24 Shard 0 Shard 10 Shard 12 Shard 20 Shard 24 Distributed Storage (e.g., CFS) Dremel serving tree Google BigQuery AnalyIcs P.284 Chapter 9 Understanding Query ExecuIon $'4$PMVNO*0Ͱಛఆྻͷσʔλ͕Ұ෦ฦͬͯ͘ΔྻΛॱ൪ʹಡΈߦΛऔಘɻ8)&3&۟ͳͲΛݟͯඞཁͳߦͷΈʹߜΓϝϞϦͰอ֤࣋TIBSE͔ΒσʔλΛूɻྫ͑ιʔτ-*.*5ͷߜΓࠐΈͳͲ͢Δࢄ ू
Root Mixer Mixer 1 Shard 0-‐8 Mixer 1 Shard 9-‐16 Mixer 1 Shard 17-‐24 Shard 0 Shard 10 Shard 12 Shard 20 Shard 24 Distributed Storage (e.g., CFS) Dremel serving tree Google BigQuery AnalyIcs P.284 Chapter 9 Understanding Query ExecuIon $'4$PMVNO*0Ͱಛఆྻͷσʔλ͕Ұ෦ฦͬͯ͘ΔྻΛॱ൪ʹಡΈߦΛऔಘɻ8)&3&۟ͳͲΛݟͯඞཁͳߦͷΈʹߜΓϝϞϦͰอ֤࣋TIBSE͔ΒσʔλΛूɻྫ͑ιʔτ-*.*5ͷߜΓࠐΈͳͲ͢Δूͨ݁͠ՌΛDBMMFSʹฦ͢ࢄ ू
#JH2VFSZͷ͍͢͝ॴ• ΧϥϜܕ*0ɺ42-ͷׂ౷࣏– Ͱ͜Εɺ.11తʹ͘͠ͳ͍• ͡Ό͋ɺ#JH2VFSZͷԿ͕͍͔͢͝– (PPHMFͷͰ͔͍Πϯϑϥ
ׂͱ֖ͳ͍ŋŋŋ
͜ΜͳΫιΫΤϦͰඵɺ̐ඵͩ
,"*;&/QMBUGPSN*ODͰͷ͍Ͳ͜Ζ
Ϣʔεέʔε• ΞΫηεϩάͷอଘௐࠪ• ΞϓϦέʔγϣϯϩάͷղੳ %BUBXBSF)PVTF• "#ςετͷ༗ҙࠩఆ
ΞΫηεϩά
ΞΫηεϩά#JH2VFSZ• /HJOYͷϩάΛqVFOUQMVHJOCJHRVFSZͰૹΓଓ͚Δ– &&Ͱ҉߸Խ͞ΕͯΔΑ• Կ͔༻͕͋ͬͨΒ42-Ͱղੳ– %BJMZ8FFLMZ.POUIMZ17– ϓϩμΫγϣϯͷσόοά
qVFOUQMVHJOCJHRVFSZ• CZUBHPNPSJT͞ΜɺZVHVJ͞Μଞ• ઌ͔Β,"*;&/QMBUGPSN*OD͕ϝϯςφʹ– ࣮࣭ɺԶQBUDIFTXFMDPNFͰ͢
ΞϓϦέʔγϣϯͷϩάղੳ
ϩάΛඈ͢• 3BJMT͔ΒUEMPHHFSSVCZͰqVFOUE• qVFOUEQMVHJOCJHRVFSZͰ#2ʹඈ͢
ϩάΛඈ͢ܖػ• ϦΫΤετຖ– "QQMJDBUJPO$POUSPMMFS– ϩάΠϯϢʔβͷଐੑΛඈ͢ˠ%"6."6ͷࢉग़ʹ• Ϟσϧͷঢ়ଶมߋ࣌– "DUJWF3FDPSE0CTFSWFS– ϞσϧຖʹదͳଐੑΛݟસͬͯඈ͢– #JH2VFSZෳࡶͳ42-Ͱී௨ʹԠ͢Δ㱺ϓϩμΫτϚωʔδϟ͕ؾܰʹ42-ॻ͍ͯΔ
ਖ਼نԽ͋·Γ͠ͳ͍• ελʔεΩʔϚ– %8)ͷఆ൪ͷϞσϦϯά• ϑΝΫτςʔϒϧŋŋŋϩά• ࣍ݩςʔϒϧŋŋŋϚελʔσʔλ ސ٬໊ͱ͔– ਖ਼نԽ͠ͳ͍ͷ͕ηΦϦʔ
"#ςετ༗ҙࠩఆ• "#ςετͷαʔϏεͳͷͰ͆• ৄࡉൿີ• SFRTFDͱ͔qVUFOEͰૹͬͯΔ͚ͲͬͪΌΒ͞– ˞SFRTFDͷ)551SFRVFTUqVFOUE͕όοϑΝϦϯά͢ΔͷͰ#JH2VFSZͷ"1*ίʔϧͣͬͱগͳ͍
֎෦πʔϧͱͷଓ• ΤΫηϧ– #JH2VFSZ$POOFDUPSGPS&YDFMCZ(PPHMF– ϐϘοτੳʹ• %0.0 #*– FYQFSJNFOUBMͳ#JH2VFSZΠϯλϑΣʔε͋ͬͨ– 5BCMFBVϝδϟʔͲ͜ΖରԠ࢝͠ΊͯΔ
໘ͳͱ͜Ζ• qVFOUEQMVHJOCJHRVFSZ͕εΩʔϚϑΝΠϧΛཁٻ͢Δ– ͕͔ͩ͠͠IBLPCFSB͞Μ͕QBUDIΛॻ͍ͯ͘Εͨ– W͔ΒGFUDI@TDIFNBػೳ͕͑ΔΑ• ࣍ݩςʔϒϧͷߋ৽– 61%"5&Ͱ͖ͳ͍ͷͰ– ؒͱ͔ʹҰճফͯ͠࡞ΔɺΈ͍ͨͳ– 1SFTUPΈ͍ͨʹҧ͏σʔλιʔεΛ+0*/Ͱ͖ͨΓ͢Δͱخ͍͠ͷ͕ͩŋŋŋ
࢛ํࢁ• (PPHMF"OBMZUJDT#JH2VFSZศརͦ͏– ("ͷੜϩάΛ#JH2VFSZͰղੳͰ͖ΔΦϓγϣϯ– ͨͩ͠("ͷ༗ྉαʔϏε• Ͱ͔͍σʔλͷΠϯϙʔτ– (PPHMF%BUB4UPSFʹஔ͍͔ͯΒΠϯϙʔτ͢Δͱߴ• 5BCMF%FDPSBUPST– σʔλͷ࣌ؒൣғΛࢦఆͯ͠ΫΤϦɻεΩϟϯରͷσʔλ͕খ͘͞ͳΔͷͰΫΤϦඅ༻ΛઅͰ͖Δ• +0*/੍ݶ.#ੲͷ– +0*/&"$)Λ͏ͱ.BQ3FEVDFͷTIV⒐FΈ͍ͨͳॲཧͰڊେͳ+0*/ ԯYԯͱ͔ŋŋŋͯ͘͠ΕΔΑ
·ͱΊ• #JH2VFSZϑϧεΩϟϯͰͰ͔͍σʔλͷ42-͕ඵͳαʔϏε• ΫιΫΤϦྗۀͰॲཧͪ͠Ό͏ΧοίΠΠ• ׂ౷࣏(PPHMFͷ%$نͰ֖ͳ͍ฒྻॲཧܥ• όονɺϩάղੳͳΜ͔ʹ͑·͢• ࢲ(PPHMFࣾͷճ͠ऀͰ͍͟͝·ͤΜ
5IBOLTֆCZ͋ΘΏ͖
naoya
cybozuinsideout
tkhresk
sajikix
codenote
i35_267
is_ryo
ufried
nagix
oldbigbuddha
tomoaki25
kgsi
bengo4com
maggiecrowley
tanoku
marcduiker
lynnandtonic
smashingmag
erikaheidi
csswizardry
aleyda
bluesmoon
reverentgeek
marcelosomers
jonrohan