Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Replicating MongoDB

Sponsored · Ship Features Fearlessly Turn features on and off without deploys. Used by thousands of Ruby developers. →

Replicating MongoDB

...what could go wrong?
Presentation for the Javantura conference in Zagreb.

Philipp Krenn

November 15, 2014

Transcript

63 slides
  1. Single instance $ mkdir 1 $ mongod --dbpath 1 --port

    27001 --logpath log1 $ mongo --port 27001 > db.test.insert({ name: "Philipp", city: "Wien" }) > db.test.find() Stop instance
  2. Add replication $ mkdir 2 $ mkdir 3 $ mongod

    --replSet javantura --dbpath 1 --port 27001 --logpath log1 --oplogSize 20 $ mongod --replSet javantura --dbpath 2 --port 27002 --logpath log2 --oplogSize 20 $ mongod --replSet javantura --dbpath 3 --port 27003 --logpath log3 --oplogSize 20
  3. Configure replication Start on the old instance, otherwise data lost

    rs.initiate() rs.status() rs.add("PK-MBP:27002") rs.add("PK-MBP:27003") rs.status() db.isMaster() db.test.find() db.test.insert({ name: "Peter", city: "Steyr" }) db.test.find()
  4. Read from secondaries $ mongo --port 27002 > db.test.find() >

    rs.slaveOk() > db.test.find() > db.test.insert({ name: "Dieter", city: "Graz" }) slaveOk only valid for the current connection
  5. Failover Kill primary with [Ctrl]+[C] Write to new primary >

    rs.status() > db.test.insert({ name: "Dieter", city: "Graz" }) > db.test.find()
  6. Restart old primary $ mongod --replSet name --dbpath 1 --port

    27001 --logpath log1 --oplogSize 20 $ mongo --port 27001 > rs.status() > rs.slaveOk() > db.test.find()
  7. CAP Select Availability or Consistency Partition-tolerance is a prerequisite for

    distributed systems "The network is reliable": http://aphyr.com/posts/288-the-network-is-reliable
  8. Multiple primaries Unlikely but possible Bugs: https://jira.mongodb.org/browse/SERVER-9765 Test script with

    no replies: https://groups.google.com/ forum/#!topic/mongodb-dev/-mH6BOYyzeI
  9. Naiv approach: Transmit original query Statement Based Replication (SBR) DELETE

    FROM test.table WHERE quantity > 20 LIMIT 1 db.collection.remove({ quantity: { $gt: 20 }}, true) //justOne: true
  10. Oplog size 32bit: 48MB 64bit OS X: 183MB 64bit *nix,

    Windows: 1GB to 50GB (5% free disk)
  11. Capped collection in oplog.rs of the local database > use

    local > show collections me 0.000MB / 0.008MB oplog.rs 0.000MB / 20.000MB replset.minvalid 0.000MB / 0.008MB slaves 0.000MB / 0.008MB startup_log 0.003MB / 10.000MB system.indexes 0.001MB / 0.008MB system.replset 0.000MB / 0.008MB
  12. > db.oplog.rs.find() { "h": NumberLong("-265486071808715859"), "ns": "test.test", "o": { "_id":

    ObjectId("541a8ed285ea5f8ae059d530"), "name": "Dieter" "city": "Graz" }, "op": "i", "ts": Timestamp(1411026642, 1), "v": 2 } ...

More Decks by Philipp Krenn

Other Decks in Programming