An introduction to Apache S4

A short introduction to Apache S4, what is it and what is it used for ? What is its
architecture ? Why was there a need to develop it ?

Transcript

1. Apache S4 • What is it ? • Building Blocks • Characteristics • Architecture • Why was it needed ? www.semtech-solutions.co.nz info@semtech-solutions.co.nz
2. Apache S4 – What is it ? • S4 = SSSS • Simple Scalable Streaming System • For data processing • Aims to hide processing complexity • A platform for data processing development • Distributed • Scalable • Fault Tolerant • High performance www.semtech-solutions.co.nz info@semtech-solutions.co.nz
3. Apache S4 – What is it ? • It is modular • Written in Java • Currently in the Apache incubator • Offers plug-ins • For complex data stream processing • Production deployment at Yahoo www.semtech-solutions.co.nz info@semtech-solutions.co.nz
4. Apache S4 – Building Blocks What are the S4 building blocks ? – Message queues – Processors – Serializer – Checkpointing backend Plus – A simple API www.semtech-solutions.co.nz info@semtech-solutions.co.nz
5. Apache S4 – Characteristics • Cluster management uses Zoo Keeper • Symmetric nodes – No centralization – No single point of failure • No limit on nodes – Throughput increases linearly • Fault tolerant failover to standby server www.semtech-solutions.co.nz info@semtech-solutions.co.nz
6. Apache S4 – Architecture www.semtech-solutions.co.nz info@semtech-solutions.co.nz
7. Apache S4 – Why was in needed ? www.semtech-solutions.co.nz info@semtech-solutions.co.nz
8. Contact Us • Feel free to contact us at – www.semtech-solutions.co.nz – info@semtech-solutions.co.nz • We offer IT project consultancy • We are happy to hear about your problems • You can just pay for those hours that you need • To solve your problems