Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
Hadoop初心者が脱初心者したかった話
Search
Sponsored
·
Your Podcast. Everywhere. Effortlessly.
Share. Educate. Inspire. Entertain. You do you. We'll handle the rest.
→
onomotoharu
December 22, 2016
400
0
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
Hadoop初心者が脱初心者したかった話
#CyberZ #Hadoop #SkillWednesday
onomotoharu
December 22, 2016
Transcript
32 slides
Hadoop ॳ৺ऀ͕ॳ৺ऀ͔ͨͬͨ͠ SkillWednesday#20161221 খݩय़
୭ খݩय़ BLBೋ *%நग़ج൫νʔϜ ΠεΩʔɹ Ṍ͚ࣄ ϚΠϯΫϥϑτ
ͳͥ)BEPPQ
طଘΫϥελ͕Ϡό͍ Ҡߦ͠ͳ͖Ό
ٕज़ඪʹྑͦ͞͏ શવ͠Βͳ͍ྖҬͩ͠
ࠓ)BEPPQͷ͠·ͤΜ
ͬͨ͜ͱ ࣌ظॱ
ωοτهࣄΛಡΉ ຊΛಡΉ ͱΓ͋͑ͣ࡞ͬͯΈΔ ಈ͍͍ͯΔͷΛ৮Δ ϕςϥϯʹฉ͘
ωοτهࣄΛಡΉ ˒ˑˑ ຊΛಡΉ ˒ˑˑ ͱΓ͋͑ͣ࡞ͬͯΈΔ ˑˑˑ ಈ͍͍ͯΔͷΛ৮Δ ˒˒ˑ ϕςϥϯʹฉ͘ ˒˒˒
Φεεϝ
ωοτهࣄΛಡΉ ˒ˑˑ
ಛʹࢀߟʹͳͬͨهࣄ • YARNͷհ • https://www.ibm.com/developerworks/jp/ analytics/library/bd-yarn-intro/ • HadoopͲͷΑ͏ʹಈ͘ͷ͔ • http://gihyo.jp/admin/serial/01/
how_hadoop_works
Ͳ͏ͩͬͨͷ • ·ͣ“Hadoop ೖ”ͳͲͰάάͬͯग़ͯདྷΔ SlideshareΛಡΜͰΈΔ • ຊʹ;ΜΘΓͱ͔͠Θ͔Βͳ͍ • ͻͱ·ͱ·Γͷࢿྉ͍͖ͳΓಡΉͷϋʔυϧ ͕ߴ͍
• ܲͱཛঢ়ଶ • ඇৗʹ͕͔͔࣌ؒΔ • ࠓͷཧղʹదͨ͠ࢿྉ͔ࣗͰஅෆՄ
ຊΛಡΉ ˒ˑˑ
ಡΜͩຊ • HadoopΦϖϨʔγϣϯ -γεςϜӡ༻ཧΨΠυ
Ͳ͏ͩͬͨͷ • Ͳ͔͜ΒಡΊ͍͍ͷ… • ۀͳͲͰ৮Εͨ͜ͱͱΕ͍͗ͯ͢Δ • “Ͳ͏خ͍͠”ͷ͔Πϝʔδ͕͔ͳ͍ • ਖ਼͋·Γ༰͍֮͑ͯͳ͍Ͱ͢ …ͷͰ͜ͷ͔̍͠ಡΊͳ͔ͬͨ
ͱΓ͋͑ͣͬͯΈΔ ˑˑˑ
ͬͨ͜ͱ • QuickStarts for CDH 5.8 • shྲྀ͢ͱΫϥελ͕Ͱ͖Δ • ͍͖ͳΓCMυϯͬͯͳΔ
• http://www.cloudera.com/ downloads/ quickstart_vms/5-8.html
Ͳ͏ͩͬͨͷ • ΫϥελཱͬͨʂదͳCSVΛೖΕͯHive Ͱୟ͍ͯΈͨʂ • …͔͜͜ΒԿΛ͢Ε͍͍ͷ͔ʁ • ͦͦଞσΟετϦϏϡʔγϣϯͳͲͱൺ ֱ͢ΔͨΊͷͬΆ͍ •
ࣗͰࢼߦࡨޡ͢Δ෦͕ͳ͍
ಈ͍͍ͯΔͷΛ৮Δ ˒˒ˑ
ͬͨ͜ͱ • CosmoͷEMRͷઃఆΛ(ͨͿ Μ)શ෦ಡΜͩ • ࣗͳΓʹཧղ্ͨ͠Ͱಉ͡ ઃఆͰΫϥελ࡞ • …ͷલʹAWSͷࢿྉಡΈ ͭͭϓϨʔϯͳͷΛ࡞
Ͳ͏ͩͬͨͷ • AWSͷࢿྉΛಡΈͳ͕ΒཱͯͯΈΔ • EMRͷػೳࢥ͕;ΜΘΓ͔Δ • طଘͷઃఆΛಡΈͳ͕ΒཱͯΔ • ͳͥ͜ͷઃఆ͕͍Δͷ͔ɺάάΔɺҙਤ͕ ͔ΔˠεοΩϦ&Εͳ͍
• ࣗͷPJͰͷԠ༻·ͩԕͦ͏
ϕςϥϯʹฉ͘ ˒˒˒
ͬͨ͜ͱ • ༰ • ࣮ࡍʹ࡞ۀ͍ͯ͠Δͱ͖ʹෆ໌ • ೲಘ͍͔ͳ͍ࣄ • ۩ମతͳϢʔεέʔε͕ු͔ͳ͍ύλʔϯ •
ํ๏ • slackͰ෦ཱͯ(#hadoop) • ͦͷͰ͏
Ͳ͏ͩͬͨͷ • “શମײ͔ͭΊͨͭΓ…͚ͩͲ͕݀͋Δ”ͷ ݀Λ࠹͛Δ • ࣗͷཧղ/PJͷཁ͕݅͋Δఔڞ༗ࡁΈ ཧղʹ߹Θͤͨձ(ΠϯλϥΫςΟϒ!)
ͱݴ͍ͭͭ ·ͱΊͱײ
ωοτهࣄΛಡΉ ຊΛಡΉ ͱΓ͋͑ͣ࡞ͬͯΈΔ ಈ͍͍ͯΔͷΛ৮Δ ϕςϥϯʹฉ͘
ͲͷεςοϓແବͰͳ͔ͬͨ
࠷ॳҙຯෆ໌Ͱ ͱΓ͋͑ͣಡΈ·͘Δ͔͠ͳ͍
͋ʙɹ͍͍ ͳΔ΄ͲͶ શʹཧղͨ͠ l z
ͷස͕ͩΜͩΜ্͕Δͣ ͩΜͩΜࣝϝογϡ͕େ͖͔ͭ͘ࡉ͔͘ͳͬͯ৽͍ࣝ͠Λٵऩ͘͢͠ͳΔΑ͏ͳΠϝʔδ
Θ͔ͬͨ͜ͱ Hadoopʹ৮ͬͯΈͨײ
Զୡͷઓ͍ɺ ࠓ࢝·͔ͬͨΓͩ…! l
More Decks by onomotoharu
See All by onomotoharu
もし僕らのことばがAPIであったなら
onomotoharu
1
330
Featured
See All Featured
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
25
2.2k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
35
3.9k
Highjacked: Video Game Concept Design
rkendrick25
PRO
2
590
How to Get Subject Matter Experts Bought In and Actively Contributing to SEO & PR Initiatives.
livdayseo
1
250
Applied NLP in the Age of Generative AI
inesmontani
PRO
4
2.6k
ラッコキーワード サービス紹介資料
rakko
5
6M
Data-driven link building: lessons from a $708K investment (BrightonSEO talk)
szymonslowik
1
1.5k
Abbi's Birthday
coloredviolet
5
11k
Optimising Largest Contentful Paint
csswizardry
37
4k
Building an army of robots
kneath
307
47k
Pawsitive SEO: Lessons from My Dog (and Many Mistakes) on Thriving as a Consultant in the Age of AI
davidcarrasco
1
320
Building Adaptive Systems
keathley
44
3.3k