͋Δ͍ Hadoop ͱ ͍͏ݴ༿Λฉ͍ͨ͜ͱ͕ͳ͍ํ͍ͳ͍Ͱ͠ΐ͏ɻ MapReduce 2004 ʹ Google ʹΑͬͯఏএ͞ΕͨɼେنσʔλΛฒྻࢄॲཧ͢ΔͨΊ ͷϑϨʔϜϫʔΫͰ͢ɻGoogle ࣮ࡍʹࣾͰ MapReduce Λ͍ͬͯͨΒ͍͠Ͱ͕͢ɼιϑτ ΣΞࣗମެ։͞Ε͓ͯΒͣɼ࣮ࡍʹ MapReduce Λ୭͕͑Δͱ͍͏ঢ়گʹ͋Γ·ͤΜͰ ͨ͠ɻ ͦͷঢ়گΛม͑ͨͷ͕ Hadoop ͷొͰ͢ɻతʹݴ͏ͱɼHadoop ͱ Java ʹΑΔ MapRe- duce ͷ࣮Ͱ͢ɻHadoop Apache Nutch ͱ͍͏શจݕࡧΤϯδϯج൫ͷͨΊʹ࣮͞Ε࢝ ΊͨͷͰ͕͢ɼ2006 ʹ Apache Hadoop ϓϩδΣΫτͱͯ͠ಠཱ͠·͢ɻͦͯ͠ͱͱ MapReduce Λग़ൃͱͯ͠։ൃ͕࢝·ͬͨ Hadoop Ͱ͕͢ɼMapReduce Ҏ֎ͷϑϨʔϜϫʔΫ ͰσʔλॲཧΛߦ͏͜ͱՄೳͱͳΓɼݱࡏʹ͓͍͔ͯͳΒͣ͠ʮ୯ͳΔ MapReduce ͷ࣮ ʯͰ͋Δͱ͍͑·ͤΜɻΉ͠ΖɼMapReduce Λ֩ͱͯ͠ൃలͨ͠ฒྻࢄσʔλॲཧͷͨΊ ͷιϑτΣΞΤίγεςϜͱ͍͏ͷ͕ɼݱࡏʹ͓͚Δ Hadoop ͷ࢟Ͱ͋Δͱ͍͑ΔͰ͠ΐ͏ɻ § 2 σʔλϕʔεۀք͔Βͷ MapReduce ൷ 2009 ɼWeb ্ʹ͋Δهࣄ͕ެ։͞Εͯେ͖ͳΛݺͼ·ͨ͠ɻ“MapReduce: A major step backwards” ͱ͞Εͨͦͷهࣄɼ࣌ͷΫϥυίϯϐϡʔςΟϯάϒʔϜ૬·ͬͯେ ྲྀߦ͍ͯͨ͠ MapReduce(͋Δ͍ Hadoop) Λ௧ʹ൷͢ΔͷͰͨ͠ɻຊޠʹ͢ΔͳΒ ʮMapReduce: େ͖ͳٕज़తޙୀʯͱͳΔͰ͠ΐ͏͔ɻλΠτϧ͔Βͯ͠ઓతͳͦͷهࣄɼख ݫ͘͠ MapReduce ͷ͜ͱΛ൷͍ͯ͠·͢ɻ هࣄͷதʹཁͷϦετ͕͋ΔͷͰɼ༁ͱͱʹҾ༻͠·͢ɿ 1. A giant step backward in the programming paradigm for large-scale data intensive applications େنσʔλΠϯςϯγϒΞϓϦέʔγϣϯʹ͓͚ΔϓϩάϥϛϯάύϥμΠϜͷେ͍ ͳΔޙୀ 2. A sub-optimal implementation, in that it uses brute force instead of indexing ࡧҾΛΘͳ͍૯ͨΓํࣜͷɼ͓Αͦ࠷ળͱݴ͍͕͍࣮ͨ 3. Not novel at all – it represents a specific implementation of well known techniques developed nearly 25 years ago શ͘৽نੑʹ͚ܽΔ – 25 ۙ͘લʹ։ൃ͞Εͨख๏ͷݶఆతͳ࣮Ͱ͔͠ͳ͍ 4. Missing most of the features that are routinely included in current DBMS ݱͷ DBMS ͕ͨ͘͝Γલʹ͍࣋ͬͯΔػೳͷຆͲ͕͚͍ܽͯΔ 5. Incompatible with all of the tools DBMS users have come to depend on DBMS Ϣʔβ͕͍ͬͯΔπʔϧͷ΄ͱΜͲશͯͱޓੑ͕ͳ͍
ؒͷؔʹେ্͖͚ͯ͘Լؔͷτϥϯδοτ (transit) ͱରͳؔͷϐΞ (peer) ͷ͕̎ͭଘࡏ͢Δɻ͜ͷ AS ͷڧऑؔଓ͍ͯ͠Δ AS ͷ͍ͬͯΔܦ࿏ͷɺͭ·Γ౸ୡੑͷ͞Ͱܾ·ͬͯ͘Δɻ τϥϯδοτ͓ۚΛࢧͬͯଞͷ AS ͔ΒଓੑΛങ͏ؔͰ͋Δɻ ʮτϥϯδοτΛങ͏ʯ ͱݴ͏ɻଓੑΛചΔଆΛ AS τϥϯδολɺങ͏ଆΛࢠ AS ΧελϚʔͳͲͱݺͿɻτ ϥϯδοτͰྲྀΕΔτϥϑΟοΫྔʹԠͯ͡ Mbps ୯ՁͰֹ݄ྉ͕ۚ՝ۚ͞ΕΔ͜ͱ͕ଟ͍ɻ ΄ͱΜͲͷ AS τϥϯδοτΛ্Ґͷ AS ͔Βߪೖ͍ͯ͠ΔɻͲ͔͜ΒτϥϯδοτΛങΘ ͳ͍࠷্Ґͷ AS Tier 1 ͱݺΕΔɻݱࡏ 14 ͷ AS ͕ Tier 1 ͱ͞Ε͓ͯΓɺຊͷࣄۀऀͰ NTT Communications ͷ AS2914 ͕།Ұͷ Tier 1 Ͱ͋Δɻ ҰํͰϐΞڧऑؔͷ͍ۙ AS ಉ͕࢜ແঈͰܦ࿏ަΛߦ͏ؔͰ͋ΔɻҰൠʹʮϐΞΛு Δʯͱදݱ͞ΕΔ͕ɺBGP ʹ͓͚ΔྡؔΛϐΞͱݺͿͨΊɺτϥϯδοτͷؔͰٕज़త ʹʮϐΞΛுΔʯͷͰจ຺ʹҙ͢Δඞཁ͕͋ΔɻͱͱτϥϯδοτΛ௨ͯ͠ߦΘΕ͍ͯͨ τϥϑΟοΫަΛϐΞʹΑͬͯߦ͏͜ͱʹΑΓɺैྔ՝ۚͰ͋Δτϥϯδοτͷ༻ྉΛԼ ͛Δ͜ͱ͕Ͱ͖Δɻ § 1.4 ܦ࿏ࠂͱτϥϑΟοΫίϯτϩʔϧ ͯ͞ɺܦ࿏ࠂΛߦ͏ͱɺવͦͷܦ࿏Ѽͷύέοτ͕ྲྀΕ͜ΜͰ͘Δɻਖ਼֬ʹର AS ͷ ϧʔλʹ͓͍ͯϧʔςΟϯάͷϓϩηεͰϕετύεʹબΕΕɺͰ͋Δɻܦ࿏ࠂΛͯ͠Β ͏ͱɺͦͷܦ࿏Ѽͷύέοτ͕ର AS ͷϧʔλʹٵ͍ࠐ·ΕΔɻࣗ AS ͷϧʔλͰϕετύε ʹબΕͨରϧʔλʹٵ͍ࠐ·ΕΔ͕ɺࣗͷ AS ͷϧʔλͳͷͰ੍ޚ؆୯Ͱ͋Δɻ ඞཁͳτϥϑΟοΫࣗͷ AS Λ௨ͯ͠ྲྀͯ͠ྑ͍͕ɺࣗʹશؔ͘ͷͳ͍τϥϑΟοΫ Ͱ͋Εɺͨͱ͓͑ۚͷ͔͔Βͳ͍ϐΞͰ͋ͬͯྲྀͨ͘͠ͳ͍͠ɺ·͓ͯۚ͠ͷ͔͔Δτϥϯ δοτʹࢮΜͰྲྀͨ͘͠ͳ͍ɻฏͨ͘ݴ͑λμΓېࢭͰ͋ΔɻτϥϑΟοΫ͕૿͑Δ΄Ͳ ઃඋ͕ඞཁʹͳΔ͔Βͩɻ ͦ͏͍ͬͨܦࡁతͳಈػʹΑͬͯɺτϥϑΟοΫΛίϯτϩʔϧ͠ͳ͚ΕͳΒͳ͍ͷ͕ͩɺج ຊݪଇଘࡏ͢ΔɻτϥϯδοτͰɺ AS ࢠ AS ͷࠂ͢Δܦ࿏Λશੈքʹରͯ͠ࠂ͢Δ ͕͋Γɺࢠ AS AS ͔Βͷܦ࿏Λͯ͢ड͚ೖΕΔɻϐΞͰɺࣗͷࢠ AS ͱࣗ AS ͷ ܦ࿏Λࠂ͠ɺϐΞઌͷ AS ͱͦͷࢠ AS ͷࠂ͢Δܦ࿏Λड͚ೖΕΔɻ͜ͷΜͷৄ͍͠ඇ ৗʹ໘ന͍ͱ͜Ζ͚ͩΕͲɺࠓճͷຊ͔ΒҳΕΔͷͰ·ͨͷػձʹɻ τϥϑΟοΫΛίϯτϩʔϧ͢Δํ๏͍͔ͭ͘ଘࡏ͢Δ͕ɺAS ؒʹ͓͍ͯ·ͣܦ࿏Λࠂ ͢Δ͔͠ͳ͍͔ͷೋͰ͋ΔɻϝτϦοΫͳͲͷࡉ͔͍ύϥϝʔλܦ࿏Λࠂ͢ΔͱܾΊ͔ͯΒ ͷͰ͋Δɻ § 1.5 ܦ࿏ϑΟϧλ Ͳͷ૬खʹͲͷܦ࿏Λࠂ͢Δ͔ɺͲͷ૬ख͔ΒͷͲͷܦ࿏ࠂΛड͚ೖΕΔ͔ɺͦΕΛ੍ޚ͢ ΔͨΊʹϧʔλʹܦ࿏ϑΟϧλΛઃఆ͢ΔɻͦͷͨΊɺࣗ AS ͕৽͘͠ܦ࿏ࠂΛߦ͏ͱ͖ྡ AS ʹͦΕΛ͑ͯϑΟϧλΛ։͚ͯΒΘͳ͚Ε͍͚͍ͳ͍ɻ͜Εʹϝʔϧ͕ΘΕ͍ͯΔ ͜ͱ͕ଟ͍͕ɺྡ AS ͕ࣗ AS ͷܦ࿏Λࠂ͍ͯ͠Δ߹ɺ͞Βʹͦͷྡ AS ʹϑΟϧλ Λۭ͚ͯΒ͏ඞཁ͕͋Δɻ͜ͷΑ͏ͳέʔετϥϯδοτͰى͜Γ͏Δͷ͕ͩɺ࠷্Ґͷ Tier
AS Λ๊͑ͯ 156,530 ͷ IPv4 ͷܦ࿏Λࠂ͍ͯ͠Δɻਓखͷ͔͔ΔϝʔϧͳΜ͔ͰͬͯΒΕ ͳ͍ͷͰɺܦ࿏ใΛσʔλϕʔεԽͯࣗ͠ಈతʹϑΟϧλΛੜͯ͠ө͞ΕΔΈ͕ඞཁʹ ͳͬͯ͘Δɻ § 2 ࢿݯཧͱܦ࿏ϋΠδϟοΫ § 2.1 AS ൪߸ͱ IP ΞυϨεͷؔ ΠϯλʔωοτࣗతͳγεςϜͰ͋Δ͕ɺAS ൪߸ IP ΞυϨεͳͲͷࢿݯॏෳར༻ ͕ੜ͡ͳ͍Α͏ʹ୭͔͕౷ҰతʹཧΛߦΘͳ͚ΕͳΒͳ͍ɻ͜͏͍ͬͨࢿݯ֊తʹ৫͞ Εͨ Internet Registry ʹΑͬͯཧ͞Ε͍ͯͯɺຊͷ߹Ͱɺ࠷্Ґͷ IANAɺΞδΞଠฏ ༸Ҭͷ APNICɺຊࠃͷ JPNIC ͷΑ͏ʹͯ͠֊తʹࢿݯׂ͕ΓͯΒΕ͍ͯΔɻͪͳΈʹ ԤभҬ RIPEɺถҬ ARIN Ͱɺຊʹ͓͚Δ APNIC ʹ૬͢ΔɻຊࠃͰ AS ൪߸ IP ΞυϨεΛऔಘ͢Δ߹ JPNIC Λ૭ޱͱׂͯ͠ΓৼΓΛड͚Δɻ AS ൪߸ IP ΞυϨεͦΕͧΕ͕औಘΛਃͨ͠৫ݸਓʹׂΓৼΒΕΔͨΊɺAS ൪߸ͱ IP ΞυϨεࢿݯཧ্ͷؔੑΛ͍࣋ͬͯͳ͍ɻ·ͨɺҰͭͷ৫͕ෳͷ AS Λӡ༻ ͢Δ͜ͱଟ͍͠ɺଞͷ৫ͷࢠ AS ʹΞυϨεϒϩοΫΛି༩͢Δ͜ͱ͋ΔͨΊɺ࣮ࡍʹͲͷ AS ͔ΒͲͷܦ࿏͕ࠂ͞ΕΔ͔ɺࠂ͞ΕͯΈͳ͍ͱΘ͔Βͳ͍ § 2.2 ܦ࿏ϋΠδϟοΫ Internet Registry ͔Β·ׂͩΓৼΓΛड͚͍ͯͳ͍ΞυϨεϒϩοΫɺଞͷ৫͕औಘ͍ͯ͠ ΔΞυϨεϒϩοΫΛউखʹࠂ͢Δ͜ͱٕज़తʹՄೳͰ͋Δɻ୯७ʹϧʔλʹͦ͏͍͏ઃఆ ΛೖΕΕ͍͍͚ͩͰɺ໔ڐͷΑ͏ͳͷ͍Βͳ͍ɻ͜͏͍͏ࣄྫΛܦ࿏ϋΠδϟοΫͱݺͿɻ લষͰܦ࿏Λྲྀ͢ͱτϥϑΟοΫ͕ྲྀΕΔ͜ͱΛઆ໌ͨ͠௨Γɺܦ࿏ϋΠδϟοΫΛड͚Δͱɺ ຊདྷࣗͷ AS ʹྲྀΕΔͣͩͬͨτϥϑΟοΫ͕ଞͷ AS ʹྲྀΕͯ͠·͏ɻύέοτ͕౸ୡෆೳ Ͱഁغ͞ΕΔՄೳੑɺ౪ௌɾվ᜵ͷڪΕ͋Δɻ ܦ࿏ϋΠδϟοΫසൟʹى͖͍ͯͯɺͦͷ΄ͱΜͲΞυϨεͷଧͪؒҧ͍ AS ֎ʹग़͞ ͳ͍ܦ࿏Λ࿙Βͯ͠͠·͏ͳͲͷઃఆϛεͰ͋Δɻେنͳࣄ݅ͱͯ͠ύΩελϯςϨίϜ͕ YouTube ͷܦ࿏ΛϋΠδϟοΫͯ͠͠·͍ɺYouTube ʹΞΫηεෆೳʹͳͬͨࣄ݅ͳͲ͕ڍ͛Β ΕΔɻ͜ͷࣄ݅ͰύΩελϯࠃͰ YouTube ͷΞΫηεΛःஅ͢ΔͨΊʹِͷܦ࿏ใΛྲྀ ͯ͠τϥϑΟοΫΛٵ͍ࠐ͏ͱͨ͠ͱ͜ΖઃఆϛεͰِͷܦ࿏ใΛ AS ֎෦ʹ࿙Βͯ͠͠·ͬ ͨͷ͕ݪҼͩͱݴΘΕ͍ͯΔɻ ͜͏ͨ͠ܦ࿏ϋΠδϟοΫ͕ߦΘΕͨࡍʹɺ͙͢ʹؾͮ͘͜ͱ͕ग़དྷΔΈ͕ඞཁʹͳΔɻܦ ࿏ϋΠδϟοΫ͔Ͳ͏͔ΛௐΔͨΊʹɺͦͷܦ࿏ͷ Origin AS ͕ଥͰ͋Δ͔ΛݟΕ͍͍ ͷ͕ͩɺલड़ͷͱ͓Γ AS ͱ IP ΞυϨεͷରԠදͳΜͯͷͳ͍ͨΊɺௐΔ͜ͱͰ͖ͳ͍ɻ ࠜຊతͳղܾͱͯ͠ॺ໊͖ͷܦ࿏Λྲྀ͢ BGPsec ͳͲ͕ఏҊ͞Ε͍ͯΔ͕ɺͻͱ·ͣͷӡ༻త ͳରॲͱͯ͠ɺOrigin AS ͱܦ࿏ͷؔੑͷใΛ࣋ͬͨσʔλϕʔεΛ࡞ͯ͠ɺͦΕͱ BGP Ͱ࣮ࡍʹྲྀΕ͍ͯΔܦ࿏Λൺֱ͢Δ͜ͱͰɺϋΠδϟοΫΛݕ͢ΔऔΓΈ͕ߦΘΕ͍ͯΔɻͦ ͷͨΊʹવɺਖ਼֬ͳ Origin AS ͱܦ࿏ͷؔੑͷใΛ࣋ͬͨσʔλϕʔε͕ඞཁʹͳΔɻ *1 http://as-rank.caida.org/