Upgrade to Pro — share decks privately, control downloads, hide ads and more …

MongoDB Transaction Pattern

MongoDB Transaction Pattern

17bca47a4961a24f82ef972aa9e1c986?s=128

Ian Yang

May 20, 2013
Tweet

Transcript

  1. MongoDB Transaction Pattern Who? Ian Yang From? Intridea Inc. When?

    2013-05-15 Wed
  2. Outline 1 Overview 2 Write Operation Properties 3 Two Phrase

    Commit 4 Recovering 5 Rollback 6 Multiple Applications 7 References
  3. Transaction (ACID) Atomicity all or nothing. Consistency does not violate

    any integrity constraints Isolation allow concurrent execution of transactions Durability Transaction is persistent after commit
  4. MongoDB Features Atomicity Only on the level of a single

    document. Write Concern Return write operation result
  5. MongoDB Transaction Atomicity done or rollback Consistency rollback may leave

    system in inconsistent state. Isolation other transactions can see intermediate result. Durability write concern, replica set, journal.
  6. Requirements of Write Operations in Transaction Revertible for recover and

    rollback Idempotent for resume, recover and rollback
  7. Not Revertible db.users.save(user); db.users.update ({ username: "ian"}, user); db.users.update ({

    username: "ian"}, {$set: {balance: 120}});
  8. Revertible db.users.update ({ username: "ian"}, {$inc: {balance: 20}}); db.users.update ({

    username: "ian"}, {$inc: {balance: -20}}); db.users.update ({ username: "ian"}, {$push: {tx: 20}}); db.users.update ({ username: "ian"}, {$pull: {tx: 20}});
  9. Not Idempotent db.users.update ({ username: "ian"}, {$inc: {balance: 20}}); db.users.update

    ({ username: "ian"}, {$push: {tx: 20}}); db.users.update( {username: "ian", balance: 100}, {$inc: {balance: 20}} );
  10. Idempotent db.users.update( {username: "ian", tx: {$ne: tx._id}}, {$inc: {balance: 20},

    $push: {tx: tx._id}} ); db.users.update( {username: "ian", tx: tx._id}, {$inc: {balance: -20}, $pull: {tx: tx._id}} );
  11. Sample Transfer 20 dollars from Ian to Daniel

  12. Create Transaction Transaction provides a unique identifier, enough information to

    recover and rollback, a document for atomic write operation.
  13. var tx = { _id: ObjectId (), source: ian._id ,

    dest: daniel._id , amount: 20 }; db.transactions.create(tx);
  14. Transaction State Machine State represents transaction stages Different actions should

    be taken in different stages to recover or rollback
  15. Step 1: Create Transaction var tx = { _id: ObjectId

    (), source: ian._id , dest: daniel._id , amount: 20, state: 'initial ' }; db.transactions.create(tx);
  16. Step 2: initial → pending db.transactions.update( {_id: tx._id , state:

    'initial '}, {$set: {state: 'pending '}} );
  17. Step 3: Apply Transaction db.users.update( {_id: tx.source , tx: {$ne:

    tx._id}}, {$push: {tx: tx._id}, $inc: { balance: - tx.amount }} ); db.users.update( {_id: tx.dest , tx: {$ne: tx._id}}, {$push: {tx: tx._id}, $inc: { balance: tx.amount }} );
  18. Step 4: pending → committed db.transactions.update( {_id: tx._id , state:

    'pending '}, {$set: {state: 'committed '}} );
  19. Step 5: Remove Pending Transaction db.users.update( {_id: tx.source , tx:

    tx._id}, {$pull: {tx: tx._id}} ); db.users.update( {_id: tx.dest , tx: tx._id}, {$pull: {tx: tx._id}} );
  20. Step 6: committed → done db.transactions.update( {_id: tx._id , state:

    'committed '}, {$set: {state: 'done'}} );
  21. Recover Initial Transactions Recover from Step 2: initial → pending

  22. Recover Pending Transactions Recover from Step 3: Apply Transactions

  23. Recover Committed Transactions Recover from Step 5: Remove Pending Transaction

  24. General Rule Recover from the Step just following the step

    transaction just entered current state. State transition creates a check point
  25. Rollback Operations Better to create a new transaction in reverse

    direction for Committed Transactions and Done Transactions Use following steps to rollback initial and pending transactions
  26. Rollback 1: state → canceling db.transactions.update( {_id: t._id , state:

    {$in: ['initial ', ' pending ']}}, {$set: {state: 'canceling '}} );
  27. Rollback 2: Undo the Transaction db.users.update( {_id: tx.dest , tx:

    {$ne: tx._id}}, {$pull: {tx: tx._id}, $inc: { balance: - tx.amount }} ); db.users.update( {_id: tx.source , tx: tx._id}, {$pull: {tx: tx._id}, $inc: { balance: tx.amount }} );
  28. Rollback 3: canceling → canceled db.transactions.update( {_id: t._id , state:

    {$in: ['canceling ']}}, {$set: {state: 'canceled '}} );
  29. Recover Rollback Restart from Rollback 2 for canceling transactions

  30. Conclusion Create a document for any transaction Store transaction info

    in the document Apply only Revertible and Idempotent operations
  31. Conclusion (Cont.) Transit state to save current work Transit state

    if several operation branches exists
  32. Enough? Only 1 application can handle a give transaction at

    any point in time.
  33. Background A picks a pending transaction to recover B picks

    also a pending transaction to recover
  34. Scenario 1 A is so slow, when it starts apply

    transaction, B already has done the transaction. A must rollback. If A exists abnormally, the transaction is in inconsistent state.
  35. Scenario 2 B just finished Step 5: Removing Pending Transaction

    A applied transactions and exited abnormally B change state to done without error
  36. Solution findAndModify lock db.transactions.findAndModify ({ query: {state: "initial", application: {$exists:

    0}}, update: {$set: {state: "pending", application: "A1"}}, new: true });
  37. Lock Release db.transactions.findAndModify ({ query: {application: "A1"}, update: {$unset: {application:

    1}}, new: true });
  38. Lock Forever? Application exists abnormally before release the lock Solution:

    timeout
  39. Timeout Lock now = new Date(); timeout = new Date(now.valueOf

    () + 60 * 10 * 1000); db.transactions.findAndModify ({ query: {state: "initial", lockUntil: {$not: {$gte: now}}}, update: {$set: {state: "pending", lockUntil: timeout}}, new: true });
  40. Client Requirements Client timeout to ensure the operation is finished

    in expected time. Check time after return from block IO Defer transactions cleanup
  41. References http://docs.mongodb.org/ manual/tutorial/ perform-two-phase-commits/ http://docs.mongodb.org/ manual/tutorial/ isolate-sequence-of-operations/

  42. Q & A