Mongodb as kombu transport

Gave this talk at the April's 24th 2012 London MUG meetup. It's about how kombu uses mongodb as transport for producing / consuming messages.

Transcript

1. KOMBU / CELERY @flaper87
2. WHY?
3. BECAUSE IT’S MONGODB WHY?
4. WOULDN’T BE GIVING THIS TALK WHY? BECAUSE IT’S MONGODB
5. WOULDN’T BE GIVING THIS TALK REALLY, IT’S BECAUSE OF THAT
6. WOULDN’T BE GIVING THIS TALK REALLY, IT’S BECAUSE OF THAT IT’S PERSISTENT
7. WOULDN’T BE GIVING THIS TALK REALLY, IT’S BECAUSE OF THAT I’M ALREADY USING IT
8. WOULDN’T BE GIVING THIS TALK REALLY, IT’S BECAUSE OF THAT I CAN SHARD IT
9. TELL ME MORE
10. TELL ME MORE 1 COLLECTION TO RULE THEM ALL
11. 1 COLLECTION TO RULE THEM ALL UNLESS WE’RE FANNING OUT
12. 1 COLLECTION TO RULE THEM ALL UNLESS WE’RE FANNING OUT AND NOT MESSING AROUND
13. READS FASTER / WRITES SLOWER INDEX ON QUEUES
14. +1, WE NEED TO POLL READS FASTER / WRITES SLOWER INDEX ON QUEUES
15. PRODUCE AND CONSUME
16. PRODUCE AND CONSUME INSERTS FOR PUTTING
17. A B-TREE! INSERTS FOR PUTTING findAndModify FOR CONSUMING PRODUCE AND CONSUME
18. A B-TREE! INSERTS FOR PUTTING findAndModify FOR CONSUMING BOTH LOCKING OPS PRODUCE AND CONSUME
19. A B-TREE! USE JOURNAL FOR DURABILITY
20. A B-TREE! USE JOURNAL FOR DURABILITY THE PENALTY ISN’T THAT HIGH
21. A B-TREE! USE JOURNAL FOR DURABILITY THE PENALTY ISN’T THAT HIGH CRASH-SAFE... HOPEFULLY
22. A B-TREE! WHAT IF I DON’T WANT KOMBU?
23. A B-TREE! WHAT IF I DON’T WANT KOMBU? EVERYTHING’S POSSIBLE
24. A B-TREE! WHAT IF I DON’T WANT KOMBU? EVERYTHING’S POSSIBLE CREATE A POLLER
25. A B-TREE! WHAT IF I DON’T WANT KOMBU? EVERYTHING’S POSSIBLE LOOP FOREVER
26. A B-TREE! WHAT IF I DON’T WANT KOMBU? EVERYTHING’S POSSIBLE CUSTOMIZE YOUR QUEUES
27. SUGGESTIONS
28. SUGGESTIONS KEEP IT IN A SEPARATE CLUSTER
29. SUGGESTIONS SHARD OR RS IF POSSIBLE
30. SUGGESTIONS REMEMBER, IT’S NOT AMQP
31. SUGGESTIONS USE MAP /REDUCE FOR COMPLEX QUEUE
33. anks, 10gen! Reach me on twitter: @flaper87