Marconi from MongoDB's POV

@flaper87 Marconi from MongoDB's POV

@flaper87 Steps out of hell: Marconi, 2 mins intro Stamped
with MongoDB Welcome to indexes hell Fight race conditions Other ideas Takeaways

Who's this dude? @flaper87 (a.k.a, Flavio Percoco): Very proud Red
Hatter, who's also part of the MongoDB Master team and has contributed to Open Source since... forever. One of those who thinks about programming when he's not programming. member of

@flaper87 Marconi, 2 mins intro Queues as a Service

@flaper87 Marconi, 2 mins intro Notifications as a Service Queues
as a Service

@flaper87 Marconi, 2 mins intro All that, using lego bricks

@flaper87 M ongoDB

@flaper87 Name Field ------------------- scope -> p_q expires -> e
ttl -> t uuid -> u claim -> c marker -> k Message Schema

@flaper87 Name Field ------------------- id -> id expires -> e
Claim Schema

@flaper87 Compound Indexes

@flaper87 {Eq, Sort, Range} Index order matters

@flaper87 [start, End) Pure evilness →

@flaper87 #Hint: Use hints ;)

@flaper87 Avoid counts!

@flaper87 Aggregate

@flaper87 Race Fighters No `limit` for updates...

@flaper87 Race Fighters FIFO … when ObjectID is not enough

@flaper87 Race Fighters Use TTL Collections tweak expiration field

@flaper87 Race Fighters Handling connection errors...

@flaper87 Race Fighters Parsing Errors

@flaper87 scale Multiple databases – q+p hash distribution

@flaper87 scale Marconi Partitions – Multiple MongoDB nodes

@flaper87 hacks # NOTE(flaper87): Dirty hack! # This sets the
expiration time to # `expires` on messages that would # expire before claim. Transaction-less

@flaper87 Ideas 1 collection per project • Smaller indexes •
Fewer lookups

@flaper87 Takeaways Compound indexes are good Range queries are evil
Transaction-less trade-off Database lock can be avoided OIDs are awesome but not fully reliable

@flaper87 Q&A Thanks we're hiring http:/ /openstack.redhat.com/ http:/ /www.openstack.org/

Marconi from MongoDB's POV

Marconi from MongoDB's POV

flaper87

More Decks by flaper87

Other Decks in Programming

Featured

Transcript

@flaper87 Marconi from MongoDB's POV

@flaper87 Steps out of hell: Marconi, 2 mins intro Stamped

Who's this dude? @flaper87 (a.k.a, Flavio Percoco): Very proud Red

@flaper87 Marconi, 2 mins intro Queues as a Service

@flaper87 Marconi, 2 mins intro Notifications as a Service Queues

@flaper87 Marconi, 2 mins intro All that, using lego bricks

@flaper87 M ongoDB

@flaper87 Name Field ------------------- scope -> p_q expires -> e

@flaper87 Name Field ------------------- id -> id expires -> e

@flaper87 Compound Indexes

@flaper87 {Eq, Sort, Range} Index order matters

@flaper87 [start, End) Pure evilness →

@flaper87 #Hint: Use hints ;)

@flaper87 Avoid counts!

@flaper87 Aggregate

@flaper87 Race Fighters No `limit` for updates...

@flaper87 Race Fighters FIFO … when ObjectID is not enough

@flaper87 Race Fighters Use TTL Collections tweak expiration field

@flaper87 Race Fighters Handling connection errors...

@flaper87 Race Fighters Parsing Errors

@flaper87 scale Multiple databases – q+p hash distribution

@flaper87 scale Marconi Partitions – Multiple MongoDB nodes

@flaper87 hacks # NOTE(flaper87): Dirty hack! # This sets the

@flaper87 Ideas 1 collection per project • Smaller indexes •

@flaper87 Takeaways Compound indexes are good Range queries are evil

@flaper87 Q&A Thanks we're hiring http:/ /openstack.redhat.com/ http:/ /www.openstack.org/