Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
MongoDB Transaction Pattern
Search
Ian Yang
May 20, 2013
Technology
0
400
MongoDB Transaction Pattern
Ian Yang
May 20, 2013
Tweet
Share
More Decks by Ian Yang
See All by Ian Yang
Migrate to JRuby
doitian
4
770
Tmux Cheatsheet
doitian
2
310
Javascript Test Frameworks
doitian
1
140
Backbone Intro
doitian
0
230
A Tour of Go
doitian
4
900
Other Decks in Technology
See All in Technology
業務自動化プラットフォーム Google Agentspace に入門してみる #devio2025
maroon1st
0
190
Where will it converge?
ibknadedeji
0
180
Oracle Base Database Service 技術詳細
oracle4engineer
PRO
11
77k
生成AIとM5Stack / M5 Japan Tour 2025 Autumn 東京
you
PRO
0
210
10年の共創が示す、これからの開発者と企業の関係 ~ Crossroad
soracom
PRO
1
190
研究開発部メンバーの働き⽅ / Sansan R&D Profile
sansan33
PRO
3
20k
KMP の Swift export
kokihirokawa
0
330
SREとソフトウェア開発者の合同チームはどのようにS3のコストを削減したか?
muziyoshiz
1
100
Oracle Cloud Infrastructure:2025年9月度サービス・アップデート
oracle4engineer
PRO
0
410
DataOpsNight#8_Terragruntを用いたスケーラブルなSnowflakeインフラ管理
roki18d
1
340
空間を設計する力を考える / 20251004 Naoki Takahashi
shift_evolve
PRO
3
330
OCI Network Firewall 概要
oracle4engineer
PRO
1
7.8k
Featured
See All Featured
The Invisible Side of Design
smashingmag
301
51k
Measuring & Analyzing Core Web Vitals
bluesmoon
9
610
Being A Developer After 40
akosma
91
590k
The Straight Up "How To Draw Better" Workshop
denniskardys
237
140k
YesSQL, Process and Tooling at Scale
rocio
173
14k
We Have a Design System, Now What?
morganepeng
53
7.8k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
285
14k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
45
2.5k
Documentation Writing (for coders)
carmenintech
75
5k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
252
21k
Six Lessons from altMBA
skipperchong
28
4k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
248
1.3M
Transcript
MongoDB Transaction Pattern Who? Ian Yang From? Intridea Inc. When?
2013-05-15 Wed
Outline 1 Overview 2 Write Operation Properties 3 Two Phrase
Commit 4 Recovering 5 Rollback 6 Multiple Applications 7 References
Transaction (ACID) Atomicity all or nothing. Consistency does not violate
any integrity constraints Isolation allow concurrent execution of transactions Durability Transaction is persistent after commit
MongoDB Features Atomicity Only on the level of a single
document. Write Concern Return write operation result
MongoDB Transaction Atomicity done or rollback Consistency rollback may leave
system in inconsistent state. Isolation other transactions can see intermediate result. Durability write concern, replica set, journal.
Requirements of Write Operations in Transaction Revertible for recover and
rollback Idempotent for resume, recover and rollback
Not Revertible db.users.save(user); db.users.update ({ username: "ian"}, user); db.users.update ({
username: "ian"}, {$set: {balance: 120}});
Revertible db.users.update ({ username: "ian"}, {$inc: {balance: 20}}); db.users.update ({
username: "ian"}, {$inc: {balance: -20}}); db.users.update ({ username: "ian"}, {$push: {tx: 20}}); db.users.update ({ username: "ian"}, {$pull: {tx: 20}});
Not Idempotent db.users.update ({ username: "ian"}, {$inc: {balance: 20}}); db.users.update
({ username: "ian"}, {$push: {tx: 20}}); db.users.update( {username: "ian", balance: 100}, {$inc: {balance: 20}} );
Idempotent db.users.update( {username: "ian", tx: {$ne: tx._id}}, {$inc: {balance: 20},
$push: {tx: tx._id}} ); db.users.update( {username: "ian", tx: tx._id}, {$inc: {balance: -20}, $pull: {tx: tx._id}} );
Sample Transfer 20 dollars from Ian to Daniel
Create Transaction Transaction provides a unique identifier, enough information to
recover and rollback, a document for atomic write operation.
var tx = { _id: ObjectId (), source: ian._id ,
dest: daniel._id , amount: 20 }; db.transactions.create(tx);
Transaction State Machine State represents transaction stages Different actions should
be taken in different stages to recover or rollback
Step 1: Create Transaction var tx = { _id: ObjectId
(), source: ian._id , dest: daniel._id , amount: 20, state: 'initial ' }; db.transactions.create(tx);
Step 2: initial → pending db.transactions.update( {_id: tx._id , state:
'initial '}, {$set: {state: 'pending '}} );
Step 3: Apply Transaction db.users.update( {_id: tx.source , tx: {$ne:
tx._id}}, {$push: {tx: tx._id}, $inc: { balance: - tx.amount }} ); db.users.update( {_id: tx.dest , tx: {$ne: tx._id}}, {$push: {tx: tx._id}, $inc: { balance: tx.amount }} );
Step 4: pending → committed db.transactions.update( {_id: tx._id , state:
'pending '}, {$set: {state: 'committed '}} );
Step 5: Remove Pending Transaction db.users.update( {_id: tx.source , tx:
tx._id}, {$pull: {tx: tx._id}} ); db.users.update( {_id: tx.dest , tx: tx._id}, {$pull: {tx: tx._id}} );
Step 6: committed → done db.transactions.update( {_id: tx._id , state:
'committed '}, {$set: {state: 'done'}} );
Recover Initial Transactions Recover from Step 2: initial → pending
Recover Pending Transactions Recover from Step 3: Apply Transactions
Recover Committed Transactions Recover from Step 5: Remove Pending Transaction
General Rule Recover from the Step just following the step
transaction just entered current state. State transition creates a check point
Rollback Operations Better to create a new transaction in reverse
direction for Committed Transactions and Done Transactions Use following steps to rollback initial and pending transactions
Rollback 1: state → canceling db.transactions.update( {_id: t._id , state:
{$in: ['initial ', ' pending ']}}, {$set: {state: 'canceling '}} );
Rollback 2: Undo the Transaction db.users.update( {_id: tx.dest , tx:
{$ne: tx._id}}, {$pull: {tx: tx._id}, $inc: { balance: - tx.amount }} ); db.users.update( {_id: tx.source , tx: tx._id}, {$pull: {tx: tx._id}, $inc: { balance: tx.amount }} );
Rollback 3: canceling → canceled db.transactions.update( {_id: t._id , state:
{$in: ['canceling ']}}, {$set: {state: 'canceled '}} );
Recover Rollback Restart from Rollback 2 for canceling transactions
Conclusion Create a document for any transaction Store transaction info
in the document Apply only Revertible and Idempotent operations
Conclusion (Cont.) Transit state to save current work Transit state
if several operation branches exists
Enough? Only 1 application can handle a give transaction at
any point in time.
Background A picks a pending transaction to recover B picks
also a pending transaction to recover
Scenario 1 A is so slow, when it starts apply
transaction, B already has done the transaction. A must rollback. If A exists abnormally, the transaction is in inconsistent state.
Scenario 2 B just finished Step 5: Removing Pending Transaction
A applied transactions and exited abnormally B change state to done without error
Solution findAndModify lock db.transactions.findAndModify ({ query: {state: "initial", application: {$exists:
0}}, update: {$set: {state: "pending", application: "A1"}}, new: true });
Lock Release db.transactions.findAndModify ({ query: {application: "A1"}, update: {$unset: {application:
1}}, new: true });
Lock Forever? Application exists abnormally before release the lock Solution:
timeout
Timeout Lock now = new Date(); timeout = new Date(now.valueOf
() + 60 * 10 * 1000); db.transactions.findAndModify ({ query: {state: "initial", lockUntil: {$not: {$gte: now}}}, update: {$set: {state: "pending", lockUntil: timeout}}, new: true });
Client Requirements Client timeout to ensure the operation is finished
in expected time. Check time after return from block IO Defer transactions cleanup
References http://docs.mongodb.org/ manual/tutorial/ perform-two-phase-commits/ http://docs.mongodb.org/ manual/tutorial/ isolate-sequence-of-operations/
Q & A