Tweet
Tweet

Slide 1

Slide 1 text

Kafka A little introduction

Slide 2

Slide 2 text

No content

Slide 3

Slide 3 text

Pub-Sub Messaging System

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

Distributed

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

Performance

Slide 11

Slide 11 text

Disk/Memory Performance Source: http://queue.acm.org/detail.cfm?id=1563874 Disk SSD Memory 1 10 100 1,000 10,000 100,000 1M 10M 100M 1000M Random access Sequential Access Read values/second

Slide 12

Slide 12 text

Disk/Memory Performance Source: http://queue.acm.org/detail.cfm?id=1563874 Disk SSD Memory 1 10 100 1,000 10,000 100,000 1M 10M 100M 1000M Random access Sequential Access Read values/second

Slide 13

Slide 13 text

Disk/Memory Performance Source: http://queue.acm.org/detail.cfm?id=1563874 Disk SSD Memory 1 10 100 1,000 10,000 100,000 1M 10M 100M 1000M Random access Sequential Access Read values/second

Slide 14

Slide 14 text

Disk/Memory Performance Source: http://queue.acm.org/detail.cfm?id=1563874 Disk SSD Memory 1 10 100 1,000 10,000 100,000 1M 10M 100M 1000M Random access Sequential Access Read values/second Sequential disk read faster than random memory read

Slide 15

Slide 15 text

Persistent

Slide 16

Slide 16 text

No content

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

No content

Slide 19

Slide 19 text

Length Magic Value Checksum Payload 4 bytes 1 byte 4 bytes n bytes

Slide 20

Slide 20 text

No content

Slide 21

Slide 21 text

No content

Slide 22

Slide 22 text

No content

Slide 23

Slide 23 text

Token Offset: 0 Broker: kafka.local Topic: Testing Input MR Job Output Offset: 130098 Broker: kafka.local Topic: Testing Output Sequence File

Slide 24

Slide 24 text

Token Offset: 0 Broker: kafka.local Topic: Testing Input MR Job Output Offset: 130098 Broker: kafka.local Topic: Testing Output Sequence File

Slide 25

Slide 25 text

No content

Slide 26

Slide 26 text

Useful Things • http://incubator.apache.org/kafka/ • https://github.com/pingles/clj-kafka