Paper discuss : Hadoop & HBase

bleu.tw@gmail.com

1. Bridging Apache HBase and Relational Database Operating Mechanism Design and Implementation Based on PHP and MySQL Application
2. SeparaTags A Sensor Data Processing Platform based on Android and Hadoop for Building Intelligent Cities
3. Constructing gazetteers from volunteered Big Geo-Data based on Hadoop
4. Visualizing social network concepts

Transcript

1. Cloud-based applications : Hadoop & HBase 49941131 Jia-Huei Ren bleu.tw@gmail.com
2. Paper 1. Bridging Apache HBase and Relational Database Operating Mechanism Design and Implementation Based on PHP and MySQL Application 2. SeparaTags A Sensor Data Processing Platform based on Android and Hadoop for Building Intelligent Cities 3. Constructing gazetteers from volunteered Big Geo-Data based on Hadoop 4. Visualizing social network concepts
3. It seems like programming design…
4. But we do not talk about the code today
5. Bridging Apache HBase and Relational Database Operating Mechanism Design and Implementation Based on PHP and MySQL Application A Thesis Submitted to Institute of Computer Science and Information Engineering Shu-Te University In Partial Fulfillment of the Requirements For the Degree of Master In Computer Science and Information Engineering Kuen-Fu Tzeng
6. Introduction PHPMyAdmin MySQL HBaseMyAdmin Hbase Hadoop
7. Thrift
8. MySQL : PHPMyAdmin
9. What’s for HBase?
10. HBaseMyAdmin
11. View Attributes
12. Use SQL to HBase
13. Create Table
14. Import data from MySQL into HBase
15. Conclusions • SQL transform • Nice UI/UX • Future work – Support PDO – Support another NoSQL Database – Written in C (PHP Extension) to improve performance – ORM Customized design
16. SeparaTags : A Sensor Data Processing Platform based on Android and Hadoop for Building Intelligent Cities Master’s Program in Networking and Communication, Department of Computer Science and Information Engineering Tamkang University Wei-Che Liao
17. Procedure
18. System Architecture
19. Hadoop Image Processing Interface
20. Exchangeable image file format
21. Screenshot
22. Image Upload
23. Check Upload
24. By Photo ID
25. By User ID
26. Conclusions • Building Intelligent Cities Easily • Future work – Integration – Compatibility
27. Constructing gazetteers from volunteered Big Geo-Data based on Hadoop Computers, Environment and Urban Systems(2014) Song Gao , Linna Li , Wenwen Li , Krzysztof Janowicz , Yue Zhang
28. Volunteered Geographic Information(VGI) ?
29. Flickr www.agrigeoplaza.com
30. System architecture
31. (A) parks; (B) schools; (C) museums; (D) coffee shops; (E) streets; (F) rivers
32. The results of spatial join workflow based on Hadoop for parks (A) by US states; (B) by US counties; (C) by US ZIP codes; (D) by US census tracts
33. The geographic footprints for Harvard University: fuzzy membership scores
34. connecting all points following the longitude
35. the California SR1 map from Wikipedia
36. Conclusions • Higher efficiency • Popular tags analysis • Scalable
37. If we had more time…
38. Visualizing social network concepts Decision Support Systems 49 (2010) 151– 161 Bin Zhua, Stephanie Watts , Hsinchun Chen
39. You may think social network is…
40. But actually…
41. That’s not social network
42. Social network is…
43. Visualizing?
44. Objective • To propose an approach called concept visualization to facilitate the understanding of social network concepts.
45. Methods • Web crawler • Text Mining – Natural Language Processing • Social Network Clustering – Circular SOM
46. Web Crawler
47. Text Mining
48. Text Input
49. Convert into a structured format • 2014/11/24 • Nov 24,2014 • 2014.11.24 • 2014-11-24 20141124 • Wow, Cathy is so beautiful! • Wow, Cathy is so beautiful beauty!
50. Classifying, Clustering
51. Analyze
52. Natural Language Processing
53. Social Network Clustering and Visualizing
54. What is SOM? • Unsupervised Learning Network • N-dimension -> 2-dimension
55. The Network visualization using the conventional approach
56. The interface of the NetVizer System
57. Design/Operating principle Set Keywords Crawlering Text Mining Analysis Visualizing Stored in Hadoop
58. Conclusions  The proposed concept visualization approach that explicitly presents the network concepts of degree centrality, betweenness centrality, subgroup identification, gatekeepers, and structural equivalence.  Further works – Other domains – Effectiveness – Automatic
59. References • Osama Abu Abbas(2008), “Comparisons Between Data Clustering Algorithms” • Da-Yu Yuan, Muh-Chyun Tang(2000) , “Exploring Intellectual Network Structure of an Interdisciplinary Research Community: A Case Study of Taiwan’s STS Community” • Bala Deshpande(2012), “3 ways to use text mining with RapidMiner to juice up your job search” • 陳信宇(2010), “Reinforcement Learning of Robot: Integrating Genetic Programming and Neural Network” • Bin Wang(2014), “Visualizing Multivariate Network on The Surface of A Sphere”
60. Thanks