Keedio Stack: BDaaS deployment for dummies by Alessio Comisso at Big Data Spain 2015

The Big data ecosystem is thriving. Driven by the productivity of the open source approach and by the markets, the community is constantly developing new tools, plugins and functionalities to improve and simplify all the aspects of data science. Unfortunately the life of system administrators is increasingly becoming more complicated. From the selection of the right architecture to the installation of the tools, their configuration and testing, a very large amount of time needs to be spent setting up the system. The burden is even more demanding when the infrastructure has high availability and strong security requirements. For the enterprise this means committing a considerable amount of resources to the maintenance of the systems, which require dedicated Unix sysadmin support.

At Keedio we like simplicity, and in this workshop we shall demonstrate how you can, within minutes, easily deploy a full big data stack which is highly available and secure.

Transcript

2. KEEDIO STACK: BDaaS DEPLOYMENT FOR DUMMIES Alessio Comisso
3. 2 16/10/15 KEEDIO is a joint business initiative of Santander Group with the Alfonso X el Sabio University, specialized in Big Data & Cloud Computing technologies. WHO IS KEEDIO? “Solve real-world problems related to data processing, based on talent, and the use of disruptive and innovative technologies”
4. KEEDIO actively participate in the community contributing with patches to existing third party open-source projects in the Big Data and Cloud ecosystem. We have developed several open source projects in order to facilitate the building. Those projects are, among others: 3 KEEDIO STACK: BDaaS DEPLOYMENT FOR DUMMIES KEEDIO OPEN SOURCE You are welcome to contribute to them J http://github.com/keedio BUILDOOP Hadoop Ecosystem Builder. KEEDIO STACK Big Data Ecosystem tools integrated together with Apache Ambari. Apache Flume plugins Several sources, interceptors and sinks to ingest any kind of data to your data lake. Kafka-HUE & Storm-Hue HUE apps for Apache Kafka and Apache Storm. KEEDIO OpenStack Sahara plugin Big Data as a Service. @keedio #BDS15
5. At KEEDIO we have defined our own technology STACK, based on the most widely used tools of the Big Data ecosystem, along with our custom-developed plugins and add-ons. We use technologies like VAGRANT and APACHE AMBARI to streamline provisioning, deployment, and process monitoring. In our software stack, these tools rely on the availability of packaged architectures produced by BUILDOOP. This way, defining and deploying Big Data software stacks is easier than ever! Thanks to the integration effort between APACHE AMBARI and the different Big Data tools, our users can enjoy their own Big Data customized distributions, specifically tailored to their needs. Junio 15 KEEDIO STACK 4 KEEDIO STACK: BDaaS DEPLOYMENT FOR DUMMIES @keedio #BDS15
6. KEEDIO STACK 5 KEEDIO STACK: BDaaS DEPLOYMENT FOR DUMMIES DEVELOPER §  Near zero time configuration §  Fully reproducible §  Simplify deployment §  Quick configuration §  Pre-Tested integration SYSADMIN DATA SCIENTIST §  Turn-key solution §  Point and click §  Simplified tuning @keedio #BDS15
7. “Vagrant provides easy to configure, reproducible, and portable work environments built on top of industry-standard technology and controlled by a single consistent workflow…” KEEDIO-VAGRANT 6 KEEDIO STACK: BDaaS DEPLOYMENT FOR DUMMIES @keedio #BDS15
8. With Puppet, you define the state of your IT infrastructure, and Puppet automatically enforces the desired state. Puppet automates every step of the software delivery process, from provisioning of physical and virtual machines to orchestration and reporting PUPPET PROVISIONER 7 KEEDIO STACK: BDaaS DEPLOYMENT FOR DUMMIES @keedio #BDS15
9. We will deploy our stack on a Openstack based private cloud (Juno). We will use the vagrant-openstack-provider project to leverage vagrant on this infrastructure. OPENSTACK PRIVATE CLOUD 8 KEEDIO STACK: BDaaS DEPLOYMENT FOR DUMMIES @keedio #BDS15
10. 9 KEEDIO STACK: BDaaS DEPLOYMENT FOR DUMMIES ARCHITECTURES @keedio #BDS15
11. 10 DEMO TIME We are going to deploy a cluster with Security enabled in minutes Let’s get started!
12. 11 KEEDIO STACK: BDaaS DEPLOYMENT FOR DUMMIES DEMO @keedio #BDS15
13. 12 KEEDIO STACK: BDaaS DEPLOYMENT FOR DUMMIES DEMO @keedio #BDS15
14. 13 KEEDIO STACK: BDaaS DEPLOYMENT FOR DUMMIES @keedio #BDS15 You are invited to our next workshop in the following weeks. Sign up to our newsletter to stay tuned: http://www.keedio.com Download and try yourself the KEEDIO-Vagrant demo at: http://www.keedio.org/demo
15. @keedio Thanks!