Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Replicating MongoDB

Replicating MongoDB

...what could go wrong?
Presentation for the Javantura conference in Zagreb.

Philipp Krenn

November 15, 2014
Tweet

More Decks by Philipp Krenn

Other Decks in Programming

Transcript

  1. Single instance $ mkdir 1 $ mongod --dbpath 1 --port

    27001 --logpath log1 $ mongo --port 27001 > db.test.insert({ name: "Philipp", city: "Wien" }) > db.test.find() Stop instance
  2. Add replication $ mkdir 2 $ mkdir 3 $ mongod

    --replSet javantura --dbpath 1 --port 27001 --logpath log1 --oplogSize 20 $ mongod --replSet javantura --dbpath 2 --port 27002 --logpath log2 --oplogSize 20 $ mongod --replSet javantura --dbpath 3 --port 27003 --logpath log3 --oplogSize 20
  3. Configure replication Start on the old instance, otherwise data lost

    rs.initiate() rs.status() rs.add("PK-MBP:27002") rs.add("PK-MBP:27003") rs.status() db.isMaster() db.test.find() db.test.insert({ name: "Peter", city: "Steyr" }) db.test.find()
  4. Read from secondaries $ mongo --port 27002 > db.test.find() >

    rs.slaveOk() > db.test.find() > db.test.insert({ name: "Dieter", city: "Graz" }) slaveOk only valid for the current connection
  5. Failover Kill primary with [Ctrl]+[C] Write to new primary >

    rs.status() > db.test.insert({ name: "Dieter", city: "Graz" }) > db.test.find()
  6. Restart old primary $ mongod --replSet name --dbpath 1 --port

    27001 --logpath log1 --oplogSize 20 $ mongo --port 27001 > rs.status() > rs.slaveOk() > db.test.find()
  7. CAP Select Availability or Consistency Partition-tolerance is a prerequisite for

    distributed systems "The network is reliable": http://aphyr.com/posts/288-the-network-is-reliable
  8. Multiple primaries Unlikely but possible Bugs: https://jira.mongodb.org/browse/SERVER-9765 Test script with

    no replies: https://groups.google.com/ forum/#!topic/mongodb-dev/-mH6BOYyzeI
  9. Naiv approach: Transmit original query Statement Based Replication (SBR) DELETE

    FROM test.table WHERE quantity > 20 LIMIT 1 db.collection.remove({ quantity: { $gt: 20 }}, true) //justOne: true
  10. Oplog size 32bit: 48MB 64bit OS X: 183MB 64bit *nix,

    Windows: 1GB to 50GB (5% free disk)
  11. Capped collection in oplog.rs of the local database > use

    local > show collections me 0.000MB / 0.008MB oplog.rs 0.000MB / 20.000MB replset.minvalid 0.000MB / 0.008MB slaves 0.000MB / 0.008MB startup_log 0.003MB / 10.000MB system.indexes 0.001MB / 0.008MB system.replset 0.000MB / 0.008MB
  12. > db.oplog.rs.find() { "h": NumberLong("-265486071808715859"), "ns": "test.test", "o": { "_id":

    ObjectId("541a8ed285ea5f8ae059d530"), "name": "Dieter" "city": "Graz" }, "op": "i", "ts": Timestamp(1411026642, 1), "v": 2 } ...