Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
The Transporter 4 MySQL
Search
Sagar Nikam
July 11, 2013
Programming
0
58
The Transporter 4 MySQL
import & export to HDFS, Hive, HBase, csv, sql, text, tsv, xml
Sagar Nikam
July 11, 2013
Tweet
Share
More Decks by Sagar Nikam
See All by Sagar Nikam
Who Changed my Password ?
sagarnikam123
1
60
Small Wonders of BioAccumulators
sagarnikam123
2
83
BIM_2010_20-Project Presentation
sagarnikam123
2
45
Toxicology of Modern Medicine
sagarnikam123
1
54
Other Decks in Programming
See All in Programming
受け取る人から提供する人になるということ
little_rubyist
0
250
RubyLSPのマルチバイト文字対応
notfounds
0
120
3 Effective Rules for Using Signals in Angular
manfredsteyer
PRO
0
100
OSSで起業してもうすぐ10年 / Open Source Conference 2024 Shimane
furukawayasuto
0
110
Nurturing OpenJDK distribution: Eclipse Temurin Success History and plan
ivargrimstad
0
960
『ドメイン駆動設計をはじめよう』のモデリングアプローチ
masuda220
PRO
8
540
Click-free releases & the making of a CLI app
oheyadam
2
120
Macとオーディオ再生 2024/11/02
yusukeito
0
370
役立つログに取り組もう
irof
28
9.6k
距離関数を極める! / SESSIONS 2024
gam0022
0
290
C++でシェーダを書く
fadis
6
4.1k
Compose 1.7のTextFieldはPOBox Plusで日本語変換できない
tomoya0x00
0
190
Featured
See All Featured
Designing for humans not robots
tammielis
250
25k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
191
16k
Keith and Marios Guide to Fast Websites
keithpitt
409
22k
Adopting Sorbet at Scale
ufuk
73
9.1k
Into the Great Unknown - MozCon
thekraken
32
1.5k
Testing 201, or: Great Expectations
jmmastey
38
7.1k
For a Future-Friendly Web
brad_frost
175
9.4k
Automating Front-end Workflow
addyosmani
1366
200k
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
kevinliebholz
27
4.3k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
226
22k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
48k
Designing for Performance
lara
604
68k
Transcript
The Transporter 4 MySQL A visual Guide 4 Importing &
Exporting csv, xml, tsv, text, sql, MySql, HDFS, Hive, HBase
contents • Export 2 csv • Import csv • Exporting
2 sql • Importing sql • Exporting 2 xml • Importing xml • Exporting 2 tsv • Exporting 2 text • Exporting 2 HDFS • Installing Apache Sqoop • exporting 2 HDFS • Importing 2 MySQL • Exporting 2 Hive • Exporting 2 HBase
open MySQL
exporting to CSV
exporting CSV – Problem ?
check Appamor status
AA Status
AA
go to apparmor.d
edit usr.sbin.mysqld
usr.sbin.mysqld
after editing usr.sbin.mysqld
make /tmp/data directory
make /data directory
mounting
reload apparmor
restarting apparmor . . .
exporting .csv
Importing csv
importing .csv – problem ?
importing .csv
edit my.cnf
my.cnf file
after editing my.cnf
importing .csv
exporting sql
view tables
create country table
inserting data into country table
exporting .sql
exported .sql file
Importing sql
importing .sql
exporting xml
exporting .xml
importing xml
importing xml
exporting tsv
exporting .tsv
exported .tsv file
exporting txt
exporting .txt
exported .txt file
exporting to HDFS
installing Apache Sqoop
Apache Sqoop Home Page
Sqoop folder structure
add mysqlJ connector in SqoopHome/lib
export & check version
sqoop help
list databases
make dir in HDFS
check mysql tables
grant permissions
exporting 2 HDFS
exporting completes
Intermediate .java file (mapReduce)
HDFS table
exporting to HDFS
exporting to HDFS
importing to MySQL (4m HDFS)
HDFS table
make MySQL table structure
importing to MySQL
importing completes
imported table
exporting to Hive
check Hive for redundant table
exporting 2 Hive
libthrift version error
export fake HBase_Home
check Hive tables
export to HBase
mysql table
employee table
start HBase daemons
exporting to HBase
exporting completes
all Sqoop queries
finish