Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
MongoDB Transaction Pattern
Search
Ian Yang
May 20, 2013
Technology
0
330
MongoDB Transaction Pattern
Ian Yang
May 20, 2013
Tweet
Share
More Decks by Ian Yang
See All by Ian Yang
Migrate to JRuby
doitian
4
740
Tmux Cheatsheet
doitian
2
230
Javascript Test Frameworks
doitian
1
110
Backbone Intro
doitian
1
210
A Tour of Go
doitian
4
630
Other Decks in Technology
See All in Technology
日本が誇るイタリアのダンスミュージック!? ユーロビートって何??
minorun365
PRO
2
230
知識と実践を紡ぐGenAI / Connecting Knowledge and experience with GenAI
aki_moon
2
190
エンジニアゼロの組織から内製開発の DX をどう実現したのか / How did we achieve DX in in-house development in an organization with zero engineers?
genkiogasawara
7
3.2k
Exadata Database Service on Dedicated Infrastructure(ExaDB-D) UI スクリーン・キャプチャ集
oracle4engineer
PRO
2
1.7k
開発スピードの維持向上を支える、テスト設計の 漸進的進化への取り組み / Continuous Test Design Development for Speed of Product Development
ropqa
0
190
TDD - Test Driven Drupal
opdavies
0
3k
QA経験のないエンジニアリング マネージャーがQAのカジュアル面談に出て 苦労していること・気づいたこと / scrum fest niigata 2024
yoshikiiida
2
680
iThome2024 Wailing Wall of Enterprise Security
notsurprised
0
300
TypeScript の抽象構文木を用いた、数百を超える API の大規模リファクタリング戦略
yanaemon
6
1.3k
Password cracking: past, present, future
openwall
0
300
株式会社EventHub・エンジニア採用資料
eventhub
0
2.1k
CloudflareとHonoを使って飲食店のレビューができるLINEアプリを作った
shinaps
1
610
Featured
See All Featured
Building Flexible Design Systems
yeseniaperezcruz
320
37k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
26
2.3k
Large-scale JavaScript Application Architecture
addyosmani
504
110k
Clear Off the Table
cherdarchuk
86
310k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
23
1.7k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
22
1.4k
RailsConf 2023
tenderlove
9
590
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
117
18k
KATA
mclloyd
16
12k
Creatively Recalculating Your Daily Design Routine
revolveconf
211
11k
Atom: Resistance is Futile
akmur
260
25k
The Art of Programming - Codeland 2020
erikaheidi
43
12k
Transcript
MongoDB Transaction Pattern Who? Ian Yang From? Intridea Inc. When?
2013-05-15 Wed
Outline 1 Overview 2 Write Operation Properties 3 Two Phrase
Commit 4 Recovering 5 Rollback 6 Multiple Applications 7 References
Transaction (ACID) Atomicity all or nothing. Consistency does not violate
any integrity constraints Isolation allow concurrent execution of transactions Durability Transaction is persistent after commit
MongoDB Features Atomicity Only on the level of a single
document. Write Concern Return write operation result
MongoDB Transaction Atomicity done or rollback Consistency rollback may leave
system in inconsistent state. Isolation other transactions can see intermediate result. Durability write concern, replica set, journal.
Requirements of Write Operations in Transaction Revertible for recover and
rollback Idempotent for resume, recover and rollback
Not Revertible db.users.save(user); db.users.update ({ username: "ian"}, user); db.users.update ({
username: "ian"}, {$set: {balance: 120}});
Revertible db.users.update ({ username: "ian"}, {$inc: {balance: 20}}); db.users.update ({
username: "ian"}, {$inc: {balance: -20}}); db.users.update ({ username: "ian"}, {$push: {tx: 20}}); db.users.update ({ username: "ian"}, {$pull: {tx: 20}});
Not Idempotent db.users.update ({ username: "ian"}, {$inc: {balance: 20}}); db.users.update
({ username: "ian"}, {$push: {tx: 20}}); db.users.update( {username: "ian", balance: 100}, {$inc: {balance: 20}} );
Idempotent db.users.update( {username: "ian", tx: {$ne: tx._id}}, {$inc: {balance: 20},
$push: {tx: tx._id}} ); db.users.update( {username: "ian", tx: tx._id}, {$inc: {balance: -20}, $pull: {tx: tx._id}} );
Sample Transfer 20 dollars from Ian to Daniel
Create Transaction Transaction provides a unique identifier, enough information to
recover and rollback, a document for atomic write operation.
var tx = { _id: ObjectId (), source: ian._id ,
dest: daniel._id , amount: 20 }; db.transactions.create(tx);
Transaction State Machine State represents transaction stages Different actions should
be taken in different stages to recover or rollback
Step 1: Create Transaction var tx = { _id: ObjectId
(), source: ian._id , dest: daniel._id , amount: 20, state: 'initial ' }; db.transactions.create(tx);
Step 2: initial → pending db.transactions.update( {_id: tx._id , state:
'initial '}, {$set: {state: 'pending '}} );
Step 3: Apply Transaction db.users.update( {_id: tx.source , tx: {$ne:
tx._id}}, {$push: {tx: tx._id}, $inc: { balance: - tx.amount }} ); db.users.update( {_id: tx.dest , tx: {$ne: tx._id}}, {$push: {tx: tx._id}, $inc: { balance: tx.amount }} );
Step 4: pending → committed db.transactions.update( {_id: tx._id , state:
'pending '}, {$set: {state: 'committed '}} );
Step 5: Remove Pending Transaction db.users.update( {_id: tx.source , tx:
tx._id}, {$pull: {tx: tx._id}} ); db.users.update( {_id: tx.dest , tx: tx._id}, {$pull: {tx: tx._id}} );
Step 6: committed → done db.transactions.update( {_id: tx._id , state:
'committed '}, {$set: {state: 'done'}} );
Recover Initial Transactions Recover from Step 2: initial → pending
Recover Pending Transactions Recover from Step 3: Apply Transactions
Recover Committed Transactions Recover from Step 5: Remove Pending Transaction
General Rule Recover from the Step just following the step
transaction just entered current state. State transition creates a check point
Rollback Operations Better to create a new transaction in reverse
direction for Committed Transactions and Done Transactions Use following steps to rollback initial and pending transactions
Rollback 1: state → canceling db.transactions.update( {_id: t._id , state:
{$in: ['initial ', ' pending ']}}, {$set: {state: 'canceling '}} );
Rollback 2: Undo the Transaction db.users.update( {_id: tx.dest , tx:
{$ne: tx._id}}, {$pull: {tx: tx._id}, $inc: { balance: - tx.amount }} ); db.users.update( {_id: tx.source , tx: tx._id}, {$pull: {tx: tx._id}, $inc: { balance: tx.amount }} );
Rollback 3: canceling → canceled db.transactions.update( {_id: t._id , state:
{$in: ['canceling ']}}, {$set: {state: 'canceled '}} );
Recover Rollback Restart from Rollback 2 for canceling transactions
Conclusion Create a document for any transaction Store transaction info
in the document Apply only Revertible and Idempotent operations
Conclusion (Cont.) Transit state to save current work Transit state
if several operation branches exists
Enough? Only 1 application can handle a give transaction at
any point in time.
Background A picks a pending transaction to recover B picks
also a pending transaction to recover
Scenario 1 A is so slow, when it starts apply
transaction, B already has done the transaction. A must rollback. If A exists abnormally, the transaction is in inconsistent state.
Scenario 2 B just finished Step 5: Removing Pending Transaction
A applied transactions and exited abnormally B change state to done without error
Solution findAndModify lock db.transactions.findAndModify ({ query: {state: "initial", application: {$exists:
0}}, update: {$set: {state: "pending", application: "A1"}}, new: true });
Lock Release db.transactions.findAndModify ({ query: {application: "A1"}, update: {$unset: {application:
1}}, new: true });
Lock Forever? Application exists abnormally before release the lock Solution:
timeout
Timeout Lock now = new Date(); timeout = new Date(now.valueOf
() + 60 * 10 * 1000); db.transactions.findAndModify ({ query: {state: "initial", lockUntil: {$not: {$gte: now}}}, update: {$set: {state: "pending", lockUntil: timeout}}, new: true });
Client Requirements Client timeout to ensure the operation is finished
in expected time. Check time after return from block IO Defer transactions cleanup
References http://docs.mongodb.org/ manual/tutorial/ perform-two-phase-commits/ http://docs.mongodb.org/ manual/tutorial/ isolate-sequence-of-operations/
Q & A