Detection of Mutual Inconsistency in
Distributed Systems
Papers We Love PDX - June 2016
Slide 2
Slide 2 text
Caitie McCaffrey
@caitie
Distributed Systems Engineer
caitiem.com
Slide 3
Slide 3 text
1983
Slide 4
Slide 4 text
No content
Slide 5
Slide 5 text
No content
Slide 6
Slide 6 text
No content
Slide 7
Slide 7 text
No content
Slide 8
Slide 8 text
Replication for Availability
Slide 9
Slide 9 text
“In some environments it is
desirable or necessary to permit
users to continue modifying
resources such as files when the
network is partitioned”
Slide 10
Slide 10 text
“Files are not stored redundantly
but are kept on removable storage
media which can be carried around
during prolonged partitions. Thus
availability & consistency are
simultaneously achieved”
Disk Toting
Slide 11
Slide 11 text
The Partitioning Problem
Slide 12
Slide 12 text
The Partitioning Problem
Slide 13
Slide 13 text
The Partitioning Problem
Slide 14
Slide 14 text
The Partitioning Problem
Slide 15
Slide 15 text
How Do We Detect Inconsistencies?
The Partitioning Problem
Slide 16
Slide 16 text
Paper Goals
Do Not Restrict Availability of
Systems during Partition
Effectively Detect Mutual
Inconsistencies
Slide 17
Slide 17 text
Origin
Points
Version
Vectors
&
Slide 18
Slide 18 text
Origin Points
• System wide unique identifier generated when
the file is created
• Immutable attribute of the file
• Suggested: Creation Time & Creation Site Pair
• Detect Name Conflicts
Slide 19
Slide 19 text
Name Conflicts
B
A
Slide 20
Slide 20 text
Name Conflicts
B
A
Slide 21
Slide 21 text
Name Conflicts
B
A
Conflict!
Slide 22
Slide 22 text
Version Conflicts
C
C
Slide 23
Slide 23 text
Version Conflicts
C
C
Slide 24
Slide 24 text
Version Conflicts
C
C
Slide 25
Slide 25 text
Version Conflicts
C
C
Origin Points are
Not Enough
Slide 26
Slide 26 text
What about TimeStamps?
Slide 27
Slide 27 text
Version Conflicts
T0
T0
Slide 28
Slide 28 text
Version Conflicts
T0
T0
Slide 29
Slide 29 text
Version Conflicts
T0
T0
No Conflict!
Slide 30
Slide 30 text
Version Conflicts
T0
T0
Slide 31
Slide 31 text
Version Conflicts
T0
T1
Slide 32
Slide 32 text
Version Conflicts
T0
T1
Slide 33
Slide 33 text
Version Conflicts
T1
T1
No Conflict!
Slide 34
Slide 34 text
“Each partition can break into sub
partitions and/or merge with other
partitions many times before the
entire network finally becomes
connected”
The Partition Problem
Slide 35
Slide 35 text
ABCD
Partition Graph
T0
Slide 36
Slide 36 text
ABCD
AB CD
Partition Graph
T0
T0
T0
Slide 37
Slide 37 text
ABCD
AB CD
Partition Graph
T0
T0
T1
T2
Slide 38
Slide 38 text
ABCD
AB CD
D
BC
A
Partition Graph
T0
T0
T0
T1
T1
Slide 39
Slide 39 text
ABCD
AB CD
D
BC
A
Partition Graph
T0
T0
T0
T1
T1
No Conflict!
T1
Slide 40
Slide 40 text
ABCD
AB CD
D
BC
A
Partition Graph
T0
T0
T0
T1
T1
T2
Slide 41
Slide 41 text
ABCD
AB CD
D
BC
A
BCD
Partition Graph
T0
T0
T0
T1
T1
T2
Slide 42
Slide 42 text
ABCD
AB CD
D
BC
A
BCD
Partition Graph
T0
T0
T0
T1
T1
T2
Conflict!
Slide 43
Slide 43 text
ABCD
AB CD
D
BC
A
BCD
Partition Graph
T0
T0
T0
T2
T1
T3
T2
Slide 44
Slide 44 text
ABCD
AB CD
D
BC
A
BCD
ABCD
Partition Graph
T0
T0
T0
T2
T1
T3
T2
T2
Slide 45
Slide 45 text
ABCD
AB CD
D
BC
A
BCD
ABCD
Partition Graph
T0
T0
T0
T2
T1
T3
T2
T2
Conflict!
Slide 46
Slide 46 text
“What is more difficult is
to find a mechanism which
detects version conflicts
only when they are real”
Slide 47
Slide 47 text
Version Vectors
“A Version Vector for a file f is a sequence of n
pairs, where n is the number of sites at which f
is stored … the ith vector entry counts the
number Vi of updates to f made at site Si”
Slide 48
Slide 48 text
Compatible Version Vectors
Slide 49
Slide 49 text
Incompatible Version Vectors
Slide 50
Slide 50 text
Version Vectors
• For each update of f on site Si we increment
the Sith component of the version vector
• When conflicts are reconciled the Max Sith
entry is set in the version vector
• Version vectors encode the partial order
defined by the partition graph
Slide 51
Slide 51 text
ABCD
Partition Graph
Slide 52
Slide 52 text
ABCD
AB CD
Partition Graph
Slide 53
Slide 53 text
ABCD
AB CD
Partition Graph
Slide 54
Slide 54 text
ABCD
AB CD
D
BC
A
Partition Graph
T2
Slide 55
Slide 55 text
ABCD
AB CD
D
BC
A
Partition Graph
T2
No Conflict!
Slide 56
Slide 56 text
ABCD
AB CD
D
BC
A
Partition Graph
T2
Slide 57
Slide 57 text
ABCD
AB CD
D
BC
A
Partition Graph
T2
Slide 58
Slide 58 text
ABCD
AB CD
D
BC
A
BCD
Partition Graph
T2
Slide 59
Slide 59 text
ABCD
AB CD
D
BC
A
BCD
Partition Graph
T2
No Conflict!
Slide 60
Slide 60 text
ABCD
AB CD
D
BC
A
BCD
Partition Graph
T2
Slide 61
Slide 61 text
ABCD
AB CD
D
BC
A
BCD
Partition Graph
T2
ABCD
Slide 62
Slide 62 text
ABCD
AB CD
D
BC
A
BCD
ABCD
Partition Graph
T2
Conflict!
Slide 63
Slide 63 text
Conflict Resolution
“A conflict detection mechanism, while valuable,
has increased effect if there is also a method for
reconciling conflicts automatically”
Slide 64
Slide 64 text
Conflict Resolution
“Clearly, conflict reconciliation must take into
account the semantics of the operations which
were done to the data object in conflict”