Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
MongoDB Transaction Pattern
Search
Ian Yang
May 20, 2013
Technology
0
370
MongoDB Transaction Pattern
Ian Yang
May 20, 2013
Tweet
Share
More Decks by Ian Yang
See All by Ian Yang
Migrate to JRuby
doitian
4
750
Tmux Cheatsheet
doitian
2
270
Javascript Test Frameworks
doitian
1
120
Backbone Intro
doitian
1
220
A Tour of Go
doitian
4
780
Other Decks in Technology
See All in Technology
ウォンテッドリーのデータパイプラインを支える ETL のための analytics, rds-exporter / analytics, rds-exporter for ETL to support Wantedly's data pipeline
unblee
0
130
急成長する企業で作った、エンジニアが輝ける制度/ 20250227 Rinto Ikenoue
shift_evolve
0
130
php-conference-nagoya-2025
fuwasegu
0
150
DevinでAI AWSエンジニア製造計画 序章 〜CDKを添えて〜/devin-load-to-aws-engineer
tomoki10
0
130
Goで作って学ぶWebSocket
ryuichi1208
3
2.8k
いまからでも遅くない!コンテナでWebアプリを動かしてみよう!コンテナハンズオン編
nomu
0
150
30→150人のエンジニア組織拡大に伴うアジャイル文化を醸成する役割と取り組みの変化
nagata03
0
180
AWS Well-Architected Frameworkで学ぶAmazon ECSのセキュリティ対策
umekou
2
140
依存パッケージの更新はコツコツが勝つコツ! / phpcon_nagoya2025
blue_goheimochi
3
210
AWSを活用したIoTにおけるセキュリティ対策のご紹介
kwskyk
0
350
OCI Success Journey OCIの何が評価されてる?疑問に答える事例セミナー(2025年2月実施)
oracle4engineer
PRO
2
150
Apache Iceberg Case Study in LY Corporation
lycorptech_jp
PRO
0
320
Featured
See All Featured
The Invisible Side of Design
smashingmag
299
50k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
40
2k
A designer walks into a library…
pauljervisheath
205
24k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
251
21k
Building Flexible Design Systems
yeseniaperezcruz
328
38k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
193
16k
Building Adaptive Systems
keathley
40
2.4k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
27
1.6k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
irinanazarova
6
570
VelocityConf: Rendering Performance Case Studies
addyosmani
328
24k
Building a Modern Day E-commerce SEO Strategy
aleyda
38
7.1k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
366
25k
Transcript
MongoDB Transaction Pattern Who? Ian Yang From? Intridea Inc. When?
2013-05-15 Wed
Outline 1 Overview 2 Write Operation Properties 3 Two Phrase
Commit 4 Recovering 5 Rollback 6 Multiple Applications 7 References
Transaction (ACID) Atomicity all or nothing. Consistency does not violate
any integrity constraints Isolation allow concurrent execution of transactions Durability Transaction is persistent after commit
MongoDB Features Atomicity Only on the level of a single
document. Write Concern Return write operation result
MongoDB Transaction Atomicity done or rollback Consistency rollback may leave
system in inconsistent state. Isolation other transactions can see intermediate result. Durability write concern, replica set, journal.
Requirements of Write Operations in Transaction Revertible for recover and
rollback Idempotent for resume, recover and rollback
Not Revertible db.users.save(user); db.users.update ({ username: "ian"}, user); db.users.update ({
username: "ian"}, {$set: {balance: 120}});
Revertible db.users.update ({ username: "ian"}, {$inc: {balance: 20}}); db.users.update ({
username: "ian"}, {$inc: {balance: -20}}); db.users.update ({ username: "ian"}, {$push: {tx: 20}}); db.users.update ({ username: "ian"}, {$pull: {tx: 20}});
Not Idempotent db.users.update ({ username: "ian"}, {$inc: {balance: 20}}); db.users.update
({ username: "ian"}, {$push: {tx: 20}}); db.users.update( {username: "ian", balance: 100}, {$inc: {balance: 20}} );
Idempotent db.users.update( {username: "ian", tx: {$ne: tx._id}}, {$inc: {balance: 20},
$push: {tx: tx._id}} ); db.users.update( {username: "ian", tx: tx._id}, {$inc: {balance: -20}, $pull: {tx: tx._id}} );
Sample Transfer 20 dollars from Ian to Daniel
Create Transaction Transaction provides a unique identifier, enough information to
recover and rollback, a document for atomic write operation.
var tx = { _id: ObjectId (), source: ian._id ,
dest: daniel._id , amount: 20 }; db.transactions.create(tx);
Transaction State Machine State represents transaction stages Different actions should
be taken in different stages to recover or rollback
Step 1: Create Transaction var tx = { _id: ObjectId
(), source: ian._id , dest: daniel._id , amount: 20, state: 'initial ' }; db.transactions.create(tx);
Step 2: initial → pending db.transactions.update( {_id: tx._id , state:
'initial '}, {$set: {state: 'pending '}} );
Step 3: Apply Transaction db.users.update( {_id: tx.source , tx: {$ne:
tx._id}}, {$push: {tx: tx._id}, $inc: { balance: - tx.amount }} ); db.users.update( {_id: tx.dest , tx: {$ne: tx._id}}, {$push: {tx: tx._id}, $inc: { balance: tx.amount }} );
Step 4: pending → committed db.transactions.update( {_id: tx._id , state:
'pending '}, {$set: {state: 'committed '}} );
Step 5: Remove Pending Transaction db.users.update( {_id: tx.source , tx:
tx._id}, {$pull: {tx: tx._id}} ); db.users.update( {_id: tx.dest , tx: tx._id}, {$pull: {tx: tx._id}} );
Step 6: committed → done db.transactions.update( {_id: tx._id , state:
'committed '}, {$set: {state: 'done'}} );
Recover Initial Transactions Recover from Step 2: initial → pending
Recover Pending Transactions Recover from Step 3: Apply Transactions
Recover Committed Transactions Recover from Step 5: Remove Pending Transaction
General Rule Recover from the Step just following the step
transaction just entered current state. State transition creates a check point
Rollback Operations Better to create a new transaction in reverse
direction for Committed Transactions and Done Transactions Use following steps to rollback initial and pending transactions
Rollback 1: state → canceling db.transactions.update( {_id: t._id , state:
{$in: ['initial ', ' pending ']}}, {$set: {state: 'canceling '}} );
Rollback 2: Undo the Transaction db.users.update( {_id: tx.dest , tx:
{$ne: tx._id}}, {$pull: {tx: tx._id}, $inc: { balance: - tx.amount }} ); db.users.update( {_id: tx.source , tx: tx._id}, {$pull: {tx: tx._id}, $inc: { balance: tx.amount }} );
Rollback 3: canceling → canceled db.transactions.update( {_id: t._id , state:
{$in: ['canceling ']}}, {$set: {state: 'canceled '}} );
Recover Rollback Restart from Rollback 2 for canceling transactions
Conclusion Create a document for any transaction Store transaction info
in the document Apply only Revertible and Idempotent operations
Conclusion (Cont.) Transit state to save current work Transit state
if several operation branches exists
Enough? Only 1 application can handle a give transaction at
any point in time.
Background A picks a pending transaction to recover B picks
also a pending transaction to recover
Scenario 1 A is so slow, when it starts apply
transaction, B already has done the transaction. A must rollback. If A exists abnormally, the transaction is in inconsistent state.
Scenario 2 B just finished Step 5: Removing Pending Transaction
A applied transactions and exited abnormally B change state to done without error
Solution findAndModify lock db.transactions.findAndModify ({ query: {state: "initial", application: {$exists:
0}}, update: {$set: {state: "pending", application: "A1"}}, new: true });
Lock Release db.transactions.findAndModify ({ query: {application: "A1"}, update: {$unset: {application:
1}}, new: true });
Lock Forever? Application exists abnormally before release the lock Solution:
timeout
Timeout Lock now = new Date(); timeout = new Date(now.valueOf
() + 60 * 10 * 1000); db.transactions.findAndModify ({ query: {state: "initial", lockUntil: {$not: {$gte: now}}}, update: {$set: {state: "pending", lockUntil: timeout}}, new: true });
Client Requirements Client timeout to ensure the operation is finished
in expected time. Check time after return from block IO Defer transactions cleanup
References http://docs.mongodb.org/ manual/tutorial/ perform-two-phase-commits/ http://docs.mongodb.org/ manual/tutorial/ isolate-sequence-of-operations/
Q & A