An introduction to Apache Falcon

A short introduction to Apache Falcon, what is it and what is it used for ?
How can it help with Hadoop based data life cycle management ? What is it's
architecture and what are the benefits of using it ?

Transcript

1. Apache Falcon • What is it ? • Benefits • Architecture • Example www.semtech-solutions.co.nz info@semtech-solutions.co.nz
2. Apache Falcon – What is it ? • A data life cycle management framework • Created for Hadoop • Logic based in Falcon rather than apps • Simplifies data management • Developed by InMobi and HortonWorks • Falcon can manage – Work flows – Replication – Provides data abstraction www.semtech-solutions.co.nz info@semtech-solutions.co.nz
3. Apache Falcon – What is it ? • Falcon provides services – Data import / replication – Scheduling / coordination – Lifecycle policies – Cluster management – SLA Management • An enterprise solution for data lifecycle management • Currently an Apache incubator project www.semtech-solutions.co.nz info@semtech-solutions.co.nz
4. Apache Falcon – Benefits • Reduce workflow / ETL development time • Reduce costs • No need to re implement functionality – Already in Falcon – Already tested • Use a single Falcon configuration file to – Define replication points – Define data processing pipeline www.semtech-solutions.co.nz info@semtech-solutions.co.nz
5. Apache Falcon – Architecture www.semtech-solutions.co.nz info@semtech-solutions.co.nz
6. Apache Falcon – BI Example • Falcon used to manage work flow • Falcon used to manage Cluster data replication • BI example – Staged and presented data replicated – Presented data visible for • Reporting • Analytics • See next slide ..... www.semtech-solutions.co.nz info@semtech-solutions.co.nz
7. Apache Falcon – BI Example www.semtech-solutions.co.nz info@semtech-solutions.co.nz
8. Contact Us • Feel free to contact us at – www.semtech-solutions.co.nz – info@semtech-solutions.co.nz • We offer IT project consultancy • We are happy to hear about your problems • You can just pay for those hours that you need • To solve your problems