Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Paper discuss : Hadoop & HBase

Paper discuss : Hadoop & HBase

bleu.tw@gmail.com

1. Bridging Apache HBase and Relational Database Operating Mechanism Design and Implementation Based on PHP and MySQL Application
2. SeparaTags A Sensor Data Processing Platform based on Android and Hadoop for Building Intelligent Cities
3. Constructing gazetteers from volunteered Big Geo-Data based on Hadoop
4. Visualizing social network concepts

Bleu (Jia-Huei Ren)

December 01, 2014
Tweet

More Decks by Bleu (Jia-Huei Ren)

Other Decks in Education

Transcript

  1. Cloud-based applications : Hadoop & HBase 49941131 Jia-Huei Ren bleu.tw@gmail.com

  2. Paper 1. Bridging Apache HBase and Relational Database Operating Mechanism

    Design and Implementation Based on PHP and MySQL Application 2. SeparaTags A Sensor Data Processing Platform based on Android and Hadoop for Building Intelligent Cities 3. Constructing gazetteers from volunteered Big Geo-Data based on Hadoop 4. Visualizing social network concepts
  3. It seems like programming design…

  4. But we do not talk about the code today

  5. Bridging Apache HBase and Relational Database Operating Mechanism Design and

    Implementation Based on PHP and MySQL Application A Thesis Submitted to Institute of Computer Science and Information Engineering Shu-Te University In Partial Fulfillment of the Requirements For the Degree of Master In Computer Science and Information Engineering Kuen-Fu Tzeng
  6. Introduction PHPMyAdmin MySQL HBaseMyAdmin Hbase Hadoop

  7. Thrift

  8. MySQL : PHPMyAdmin

  9. What’s for HBase?

  10. HBaseMyAdmin

  11. View Attributes

  12. Use SQL to HBase

  13. Create Table

  14. Import data from MySQL into HBase

  15. Conclusions • SQL transform • Nice UI/UX • Future work

    – Support PDO – Support another NoSQL Database – Written in C (PHP Extension) to improve performance – ORM Customized design
  16. SeparaTags : A Sensor Data Processing Platform based on Android

    and Hadoop for Building Intelligent Cities Master’s Program in Networking and Communication, Department of Computer Science and Information Engineering Tamkang University Wei-Che Liao
  17. Procedure

  18. System Architecture

  19. Hadoop Image Processing Interface

  20. Exchangeable image file format

  21. Screenshot

  22. Image Upload

  23. Check Upload

  24. By Photo ID

  25. By User ID

  26. Conclusions • Building Intelligent Cities Easily • Future work –

    Integration – Compatibility
  27. Constructing gazetteers from volunteered Big Geo-Data based on Hadoop Computers,

    Environment and Urban Systems(2014) Song Gao , Linna Li , Wenwen Li , Krzysztof Janowicz , Yue Zhang
  28. Volunteered Geographic Information(VGI) ?

  29. Flickr www.agrigeoplaza.com

  30. System architecture

  31. (A) parks; (B) schools; (C) museums; (D) coffee shops; (E)

    streets; (F) rivers
  32. The results of spatial join workflow based on Hadoop for

    parks (A) by US states; (B) by US counties; (C) by US ZIP codes; (D) by US census tracts
  33. The geographic footprints for Harvard University: fuzzy membership scores

  34. connecting all points following the longitude

  35. the California SR1 map from Wikipedia

  36. Conclusions • Higher efficiency • Popular tags analysis • Scalable

  37. If we had more time…

  38. Visualizing social network concepts Decision Support Systems 49 (2010) 151–

    161 Bin Zhua, Stephanie Watts , Hsinchun Chen
  39. You may think social network is…

  40. But actually…

  41. That’s not social network

  42. Social network is…

  43. Visualizing?

  44. Objective • To propose an approach called concept visualization to

    facilitate the understanding of social network concepts.
  45. Methods • Web crawler • Text Mining – Natural Language

    Processing • Social Network Clustering – Circular SOM
  46. Web Crawler

  47. Text Mining

  48. Text Input

  49. Convert into a structured format • 2014/11/24 • Nov 24,2014

    • 2014.11.24 • 2014-11-24 20141124 • Wow, Cathy is so beautiful! • Wow, Cathy is so beautiful beauty!
  50. Classifying, Clustering

  51. Analyze

  52. Natural Language Processing

  53. Social Network Clustering and Visualizing

  54. What is SOM? • Unsupervised Learning Network • N-dimension ->

    2-dimension
  55. The Network visualization using the conventional approach

  56. The interface of the NetVizer System

  57. Design/Operating principle Set Keywords Crawlering Text Mining Analysis Visualizing Stored

    in Hadoop
  58. Conclusions  The proposed concept visualization approach that explicitly presents

    the network concepts of degree centrality, betweenness centrality, subgroup identification, gatekeepers, and structural equivalence.  Further works – Other domains – Effectiveness – Automatic
  59. References • Osama Abu Abbas(2008), “Comparisons Between Data Clustering Algorithms”

    • Da-Yu Yuan, Muh-Chyun Tang(2000) , “Exploring Intellectual Network Structure of an Interdisciplinary Research Community: A Case Study of Taiwan’s STS Community” • Bala Deshpande(2012), “3 ways to use text mining with RapidMiner to juice up your job search” • 陳信宇(2010), “Reinforcement Learning of Robot: Integrating Genetic Programming and Neural Network” • Bin Wang(2014), “Visualizing Multivariate Network on The Surface of A Sphere”
  60. Thanks