An introduction to Apache Bigtop

A short introduction to Apache Bigtop, what is it and what can it be used for ?
How can it be used to test and deploy the Hadoop stack ?

Transcript

1. Apache Bigtop • What is it ? • What is it for ? • Automated Testing • Platforms • Dependencies www.semtech-solutions.co.nz info@semtech-solutions.co.nz
2. Apache Bigtop – What is it ? • For testing Hadoop and related tools • For test development • Uses Jenkins continuous integration • Based on Cloudera's CDH • BigTop for Hadoop is equivalent to a Linux release – i.e. Ubuntu, Fedora, CentOS • Use it for deploying the Hadoop stack • Guarantees version inter-operability within the stack www.semtech-solutions.co.nz info@semtech-solutions.co.nz
3. Apache Bigtop – What is it for ? It is for testing Hadoop at a number of levels – Packaging – Platform – Runtime – Upgrades – Integration testing – Validation ! www.semtech-solutions.co.nz info@semtech-solutions.co.nz
4. Automation Methodology – An Example www.semtech-solutions.co.nz info@semtech-solutions.co.nz
5. Apache Bigtop – Platforms Platforms supported – Ubuntu 10.10 – CentOS 5/6 – Fedora 18 – Mageia 1 – openSUSE 12.2 www.semtech-solutions.co.nz info@semtech-solutions.co.nz
6. Apache Bigtop – Dependencies The following tools are required – Java JDK 1.6 Apache Ant – Apache Maven wget – tar git – subversion gcc – gcc-c++ make – fuse protobuf-compiler – autoconf automake – libtool sharutils – asciidoc xmlto www.semtech-solutions.co.nz info@semtech-solutions.co.nz
7. Apache Bigtop – The Players Who has bought into the use of Bigtop ? – Canonical – Cloudera – Ebay – EMC – Hortonworks – Illumos – Intel – Trendmicro – WANdisco www.semtech-solutions.co.nz info@semtech-solutions.co.nz
8. Light Humour A bit of humour ........ www.semtech-solutions.co.nz info@semtech-solutions.co.nz
9. Contact Us • Feel free to contact us at – www.semtech-solutions.co.nz – info@semtech-solutions.co.nz • We offer IT project consultancy • We are happy to hear about your problems • You can just pay for those hours that you need • To solve your problems