Having a hadoop cluster ready in 10 min. with one click

In this talk, I shared how Ansible works with a case study of getting Hadoop Cluster ready with one click. Recorded video: https://www.youtube.com/watch?v=vbHCB5oQT18

Transcript

1. Having Hadoop Cluster Ready in 10 min. with One Click Büsra Köken Software Engineer
2. I am Büsra Köken Software Engineer You can find me on twitter by @busrakoken Hello!
3. Outline ◉ Automation tools ◉ Ansible, how it works ◉ Case study explanation ◉ Summary and Q&A
4. Automation
5. IT Automation Tools Puppet 2005 Ansible 2012 CFEngine 1993 Saltstack 2011
6. Why Ansible? Written in Python Create your own module in any language YAML FILES EASY to get started and learn Batteries included Big Community
7. ◉ Over ssh​ ◉ Inventory​ ◉ Playbooks ◉ ​Tasks​ ◉ Modules​ ◉ Directory structure​ How Ansible Works​
8. Directory Structure
9. Case Study Explanation Create Servers Run a big data job Install Hadoop Destroy the Environment
10. How my project looks directory structure
11. Host.ini file
12. deploy_cluster.yml map hosts to roles
13. Create Droplet with digital_ocean module
14. Hadoop Configuration with template module
15. If you need to say “yes” at some point... you can do it with expect* module!
16. Destroy created servers now!
17. DEMO
18. Let’s Review What We Have Seen Easy to get started just like Python​ Clean architecture Makes complex tasks easy Straightforward configuration Extendable module structure
19. Any questions ? You can contact me via; ◉ busrakoken@gmail.com ◉ linkedin.com/in/busrakoken ◉ Project Github link: github/busrakoken/cloudy-hadoop