Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Kafka - A Little Introduction

Kafka - A Little Introduction

This is a presentation I gave on Kafka (http://incubator.apache.org/kafka/)- a persistent, distributed messaging system developed at LinkedIn.

I gave it during one of our monthly tech presentation evenings in our office.

Paul Ingles

May 01, 2012
Tweet

More Decks by Paul Ingles

Other Decks in Technology

Transcript

  1. Disk/Memory Performance Source: http://queue.acm.org/detail.cfm?id=1563874 Disk SSD Memory 1 10 100

    1,000 10,000 100,000 1M 10M 100M 1000M Random access Sequential Access Read values/second
  2. Disk/Memory Performance Source: http://queue.acm.org/detail.cfm?id=1563874 Disk SSD Memory 1 10 100

    1,000 10,000 100,000 1M 10M 100M 1000M Random access Sequential Access Read values/second
  3. Disk/Memory Performance Source: http://queue.acm.org/detail.cfm?id=1563874 Disk SSD Memory 1 10 100

    1,000 10,000 100,000 1M 10M 100M 1000M Random access Sequential Access Read values/second
  4. Disk/Memory Performance Source: http://queue.acm.org/detail.cfm?id=1563874 Disk SSD Memory 1 10 100

    1,000 10,000 100,000 1M 10M 100M 1000M Random access Sequential Access Read values/second Sequential disk read faster than random memory read
  5. Token Offset: 0 Broker: kafka.local Topic: Testing Input MR Job

    Output Offset: 130098 Broker: kafka.local Topic: Testing Output Sequence File
  6. Token Offset: 0 Broker: kafka.local Topic: Testing Input MR Job

    Output Offset: 130098 Broker: kafka.local Topic: Testing Output Sequence File