Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Transporter 4 MySQL

Sponsored · Ship Features Fearlessly Turn features on and off without deploys. Used by thousands of Ruby developers. →
Avatar for Sagar Nikam Sagar Nikam
July 11, 2013
Programming
0
61

The Transporter 4 MySQL

import & export to HDFS, Hive, HBase, csv, sql, text, tsv, xml

Avatar for Sagar Nikam

Sagar Nikam

July 11, 2013
Tweet

More Decks by Sagar Nikam

See All by Sagar Nikam
Who Changed my Password ?
Avatar for Sagar Nikam sagarnikam123
1
68
Small Wonders of BioAccumulators
Avatar for Sagar Nikam sagarnikam123
2
90
BIM_2010_20-Project Presentation
Avatar for Sagar Nikam sagarnikam123
2
50
Toxicology of Modern Medicine
Avatar for Sagar Nikam sagarnikam123
1
64

Other Decks in Programming

See All in Programming
Fluid Templating in TYPO3 14
Avatar for Simon Praetorius s2b
0
130
【卒業研究】会話ログ分析によるユーザーごとの関心に応じた話題提案手法
Avatar for MomokoShirakawa momok47
0
200
AIで開発はどれくらい加速したのか？AIエージェントによるコード生成を、現場の評価と研究開発の評価の両面からdeep diveしてみる
Avatar for どすこい daisuketakeda
1
2.5k
Grafana：建立系統全知視角的捷徑
Avatar for Blueswen blueswen
0
330
Smart Handoff/Pickup ガイド - Claude Code セッション管理
Avatar for rasshii yukiigarashi
0
140
AIによる高速開発をどう制御するか？ ガードレール設置で開発速度と品質を両立させたチームの事例
Avatar for tonkotsuboy_com tonkotsuboy_com
7
2.4k
CSC307 Lecture 04
Avatar for Javier Gonzalez-Sanchez javiergs
PRO
0
660
AI Agent の開発と運用を支える Durable Execution #AgentsInProd
Avatar for izumin5210 izumin5210
7
2.3k
なるべく楽してバックエンドに型をつけたい！(楽とは言ってない)
Avatar for HIBIKI CUBE hibiki_cube
0
140
AI Schema Enrichment for your Oracle AI Database
Avatar for thatjeffsmith thatjeffsmith
0
320
AI巻き込み型コードレビューのススメ
Avatar for Nealle nealle
2
880
ノイジーネイバー問題を解決する 公平なキューイング
Avatar for Shoichi Ochi occhi
0
110

Featured

See All Featured
Color Theory Basics | Prateek | Gurzu
Avatar for Gurzu gurzu
0
200
SEO for Brand Visibility & Recognition
Avatar for Aleyda Solis aleyda
0
4.2k
Mind Mapping
Avatar for Hélio Medeiros helmedeiros
PRO
0
89
How People are Using Generative and Agentic AI to Supercharge Their Products, Projects, Services and Value Streams Today
Avatar for Helen Beal helenjbeal
1
130
Primal Persuasion: How to Engage the Brain for Learning That Lasts
Avatar for Mike Taylor tmiket
0
260
How to audit for AI Accessibility on your Front & Back End
Avatar for davetheseo davetheseo
0
180
Building AI with AI
Avatar for Ines Montani inesmontani
PRO
1
700
4 Signs Your Business is Dying
Avatar for Josh Pigford shpigford
187
22k
Everyday Curiosity
Avatar for Cassini Nazir cassininazir
0
130
Winning Ecommerce Organic Search in an AI Era - #searchnstuff2025
Avatar for Aleyda Solis aleyda
1
1.9k
The Pragmatic Product Professional
Avatar for Laura Van Doore lauravandoore
37
7.1k
Building the Perfect Custom Keyboard
Avatar for Naoto Takai takai
2
690

Transcript

  1. The Transporter 4 MySQL A visual Guide 4 Importing &

    Exporting csv, xml, tsv, text, sql, MySql, HDFS, Hive, HBase

  2. contents • Export 2 csv • Import csv • Exporting

    2 sql • Importing sql • Exporting 2 xml • Importing xml • Exporting 2 tsv • Exporting 2 text • Exporting 2 HDFS • Installing Apache Sqoop • exporting 2 HDFS • Importing 2 MySQL • Exporting 2 Hive • Exporting 2 HBase

  3. open MySQL

  4. exporting to CSV

  5. exporting CSV – Problem ?

  6. check Appamor status

  7. AA Status

  8. AA

  9. go to apparmor.d

  10. edit usr.sbin.mysqld

  11. usr.sbin.mysqld

  12. after editing usr.sbin.mysqld

  13. make /tmp/data directory

  14. make /data directory

  15. mounting

  16. reload apparmor

  17. restarting apparmor . . .

  18. exporting .csv

  19. Importing csv

  20. importing .csv – problem ?

  21. importing .csv

  22. edit my.cnf

  23. my.cnf file

  24. after editing my.cnf

  25. importing .csv

  26. exporting sql

  27. view tables

  28. create country table

  29. inserting data into country table

  30. exporting .sql

  31. exported .sql file

  32. Importing sql

  33. importing .sql

  34. exporting xml

  35. exporting .xml

  36. importing xml

  37. importing xml

  38. exporting tsv

  39. exporting .tsv

  40. exported .tsv file

  41. exporting txt

  42. exporting .txt

  43. exported .txt file

  44. exporting to HDFS

  45. installing Apache Sqoop

  46. Apache Sqoop Home Page

  47. Sqoop folder structure

  48. add mysqlJ connector in SqoopHome/lib

  49. export & check version

  50. sqoop help

  51. list databases

  52. make dir in HDFS

  53. check mysql tables

  54. grant permissions

  55. exporting 2 HDFS

  56. exporting completes

  57. Intermediate .java file (mapReduce)

  58. HDFS table

  59. exporting to HDFS

  60. exporting to HDFS

  61. importing to MySQL (4m HDFS)

  62. HDFS table

  63. make MySQL table structure

  64. importing to MySQL

  65. importing completes

  66. imported table

  67. exporting to Hive

  68. check Hive for redundant table

  69. exporting 2 Hive

  70. libthrift version error

  71. export fake HBase_Home

  72. check Hive tables

  73. export to HBase

  74. mysql table

  75. employee table

  76. start HBase daemons

  77. exporting to HBase

  78. exporting completes

  79. all Sqoop queries

  80. finish