Handbook of Knowledge Representation - Chapter 2: Satisfiability Solvers

Handbook of
Knowledge Representation
Chapter 2: Satisﬁability Solvers
Junya Yamaguchi
May 2, 2018
Tokyo Institute of Technology, Inoue Lab
[email protected]

View Slide

SAT Problem Example
• An assignment is a function ϕ : V → {0, 1}
F = ( x
literal
∨ y ∨ z ) ∧ (¬x ∨ ¬y)
clause
∧ (¬y ∨ ¬z) ∧ (¬z ∨ ¬x)
CNF formula
assignment ϕ(x, y, z) = (0, 0, 1) derives F = 1.
• A formula is satisﬁable (SAT) iff there exists an assignment that
evaluates it TRUE, which is called model.
• Otherwise, we say a formula is unsatisﬁable (UNSAT).
1

View Slide

SAT ιϧόͷ׆༂
• ࠷ѱܭࢉྔ͕ࢦ਺ؔ਺Ͱ͋Δ͜ͱͰ༗໊͕ͩɺSAT ιϧό͸ଟ͘ͷྖҬͰޭ
੷Λ࢒͍ͯ͠Δ
• ιϑτ΢ΣΞ/ϋʔυ΢ΣΞݕূ
• ࣗಈςετύλʔϯੜ੒
• ϓϥϯχϯά
• εέδϡʔϦϯά
• ೥ʹҰճ SAT ͷίϯϖ͕͋Δ (SAT Competition)
• ͦ͜Ͱ਺ଟ͘ͷૉ੖Β͍͠ SAT ιϧό͕஀ੜͨ͠
• Ϟμϯͳ SAT ιϧόʹͳΔͱɺ਺ສม਺ɾ਺ઍ੍໿͔Β੒Δඇৗʹ೉͍͠໰
୊Ͱ΋ղ͚Δ
2

View Slide

SAT ͱ஌ֶࣝशͷؔ܎
• ࠓͰ͸ͨ͘͞ΜͷԠ༻͕͋Δ͕ɺͦͷىݯ͸ ஌ֶࣝश
• ஌ࣝͷදݱྗͱܭࢉྔͷτϨʔυΦϑʹؔ͢Δݚڀ (ୈ 3 ষͰѻ͏) ͕ϝΠϯ
• Ծఆ: ࠷ѱܭࢉྔ͕ଟ߲ࣜ࣌ؒͰ͋Γͳ͕Β΋ɺΊͪΌͪ͘ΌΤϨΨϯτͰදݱ
ྗͷߴ͍දݱݴޠΛΈ͚͍ͭͨ
• 90 ೥୅ͷ͸͡Ίʹɺ2 ͭͷ࿦จ͕͜ͷԾఆʹ௅Μͩ
• ͋Δಛघͳ໰୊Λআ͚͹ɺ΄ͱΜͲͷϥϯμϜ SAT ໰୊͸ͱͯ΋؆୯ʹͱ͚Δ
͜ͱΛࣔͨ͠࿦จ
• ͦͷಛघͳ೉͍͠໰୊Ͱ͑͞΋ɺϩʔΧϧαʔνͷςΫχοΫΛ࢖͑͹؆୯ʹղ
͘͜ͱ͕Ͱ͖Δ͜ͱΛࣔͨ͠࿦จ
3

View Slide

ࢲͨͪ͸࠷ѱܭࢉྔʹͱΒΘΕͯͳ͍͔ʁ
• ਺ඦສ΋ͷม਺Λ΋ͭݱ࣮ͷ SAT ໰୊Ͱͷ੒ޭ
• యܕతͳ SAT ໰୊΍ݱ࣮తͳ NP ׬શͳ໰୊ (ͷܭࢉྔ) ʹରͯ͠͸ɺޮ཰Α͘
ղ͘͜ͱ͕Ͱ͖ΔҰൠղ๏͕͋Δ
• ࠷ѱܭࢉྔʹϏϏΓ͗ͯ͢͸͍͚ͳ͍
• ͜ͷ·· SAT ιϧό͕੒௕͢Ε͹ɺ΋ͬͱෳࡶͳ஌ࣝදݱͷݴޠΛѻ͑ΔΑ
͏ʹͳΔ͸ͣ
• NO: ࠷ѱ࣌ܭࢉྔ͕ଟ߲ࣜͳΞϧΰϦζϜͰͷදݱ
• YES: SAT ιϧόͰղ͚Δൣғͷදݱ
4

View Slide

ষͷߏ੒
1. ࣍ͷ 2 छྨͷιϧόʹ͓͍ͯ࢖ΘΕ͍ͯΔओͳςΫχοΫ
• ׬શ (complete) SAT ιϧό
• ෆ׬શ (incomplete) SAT ιϧό
2. ࣮ફతͳ SAT ූ߸Խʹର͢Δ্هςΫχοΫͷ༗ޮੑ
3. SAT ιϧόͷকདྷ΁ͷల๬
5

View Slide

ఆٛ
• ໋୊࿦ཧࣜ (propositional or Boolean formula) ͸ɺม਺ͷू߹্ʹఆٛ
͞ΕΔ࿦ཧࣜ
• ֤ม਺͸ {FALSE, TRUE} ͷͲͪΒ͔ͷ஋ΛऔΔ
• ศ্ٓ {0, 1} Ͱද͢ࣄ͕ଟ͍
• ม਺ͷू߹ V ʹର͢Δ ׬શׂ౰ͯ (truth assignment)1͸ɺࣸ૾
σ : V → {0, 1} ͷ͜ͱɻ
• ಛʹɺ໋୊࿦ཧࣜΛ 1 ʹධՁ͢ΔΑ͏ͳ׬શׂ౰ͯΛ satisfying assignment
΍ Ϟσϧ (model) ͱݺͿɻ
• SAT ιϧόͰѻ͏ SAT ໰୊͸ CNF (conjunctive normal form) ͱݺ͹Ε
Δಛผͳܗࣜͷ໋୊࿦ཧ͚ࣜͩʹ੍ݶ
1
׬શׂ౰ͯ͸୯ʹʮׂ౰ͯʯͱݺ͹ΕΔ͜ͱ΋͋Γ·͕͢ɺຊεϥΠυͰ͸෦෼ׂ౰ͯͷ͜ͱΛׂ౰ͯͱݺͼɺ׬શׂ౰ͯͱ۠ผ͠·͢ɻ
6

View Slide

ॆ଍Մೳੑ൑ఆ໰୊
Boolean satisﬁability testing (SAT) ໰୊
ೖྗ CNF ܗࣜͷ໋୊࿦ཧࣜ (CNF ࣜ) F
࣭໰ F ʹ͸Ϟσϧ͕͋Δ͔ʁ
CNF ࣜ અͷબݴ
F = C1
∧ C2
· · · ∧ Cm
·ͨ͸ F = {Ci
}m
i=1
અ Ϧςϥϧͷ࿈ݴ
C = l1
∨ l2
· · · ∨ ln
·ͨ͸ C = {li
}n
i=1
Ϧςϥϧ ม਺ x ͔ͦͷ൱ఆ ¬x.
ม਺ x ∈ {0, 1}
7

View Slide

ఆٛ
• અʹؚ·ΕΔϦςϥϧ਺ΛɺઅͷαΠζͱݺͿɻ
• e.g. (x ∨ ¬y ∨ z) ͳΒαΠζ͸ 3
• αΠζ͕ 0 ͷઅΛۭઅ (empty clause)ɺαΠζ͕ 1 ͷઅΛ୯Ґઅ (unit
clause)ɺͦͯ͠αΠζ͕ 2 ͷઅΛόΠφϦઅ (binary clause) ͱͦΕͧΕ
ݺͿɻ
• ͢΂ͯͷઅͷαΠζ͕ͪΐ͏Ͳ k Ͱ͋Δ CNF ࣜΛ k-SAT ͱݺͿɻ
• 2-SAT ͸ଟ߲ࣜ࣌ؒͰٻղՄೳ
• 3-SAT Ҏ্ʹͳΔͱ NP ׬શ
(x1
∨ x2
∨ x3
) ∧ (x4
∨ x5
∨ x6
) ∧ (x7
∨ x8
∨ x9
)
8

View Slide

ఆٛ
• ෦෼ׂ౰ͯ (partial assignment) ͸ɺม਺ू߹ͷ෦෼ू߹ʹର͢Δ׬શׂ
౰ͯͷ͜ͱɻ
• CNF ࣜ F ΁ͷ෦෼ׂ౰ͯ ρ ʹରͯ͠ɺρ Λ୅ೖͯ͠ಘΒΕͨࣜͷ͜ͱΛ
simpliﬁed ͞ΕͨࣜͱݺͼɺF|ρ
Ͱද͢ɻ
• 1 ͭҎ্ͷϦςϥϧ͕ 1 ʹධՁ͞Εͨ͢΂ͯͷઅΛ࡟আ
• 0 ʹධՁ͞Εͨ͢΂ͯͷϦςϥϧΛ࡟আ
ࠓޙɺೖྗͱͯ͠༩͑Δ໋୊࿦ཧࣜʹ͸ CNF ࣜΛ҉໧ͷ͏ͪʹԾఆ͢Δ͕ɺ
ଟ͘ͷ৔߹͜ͷԾఆ͸໰୊ʹͳΒͳ͍ɻ
9

View Slide

2.2 SAT Solver Technology
Complete Methods

View Slide

Complete Methods
• A complete SAT solver, given the input formula F, either produces
a satisfying assignment for F or proves that F is unsatisﬁable.
• Recent complete methods remain variants of a process introduced
several decades ago, DPLL
• DPLL procedure:
• Was introduced in the early 1960`s.
• Can prune of the search space based on falsiﬁed clauses.
• Performs a backtrack search in the space of partial truth assignments.
• Main improvements to DPLL:
• Smart branch selection heuristics
• Extensions like clause learning and randomized restarts
• Well-crafted data structures such as lazy implementations and watched
literals
10

View Slide

2.2.1 The DPLL Procedure

View Slide

DPLL Procedure
11

View Slide

DPLL Procedure
• Repeatedly select an unassigned literal l.
• The step to choose l is called branching step.
• Setting l to TRUE or FALSE is called a decision
• decision level is used to reffer the recursion depth at that stage
• Recursively search for a satisfying assignment for F|l
and F|¬l
.
12

View Slide

DPLL Procedure
13

View Slide

DPLL Procedure – Unit Propagation
• Initial state:
F = (x) ∧ (¬x ∨ y) ∧ (¬x ∨ ¬y ∨ z)
ρ = ∅
• There exists an unit clause (x). Execute unit propagation.
F = (x)∧(¬x∨y) ∧ (¬x∨¬y ∨ z)
ρ = {x}
• Now, another unit clause (y) found, let’s more simplify the formula.
F = (x) ∧ (¬x ∨ y)∧(¬x ∨ ¬y∨z)
ρ = {x, y}
• Execute unit propagation in the same way.
F = (x) ∧ (¬x ∨ y) ∧ (¬x ∨ ¬y ∨ z)
ρ = {x, y, z}
• UnitPropagate() ends since there are no unit clauses. 14

View Slide

Key Features of Modern DPLL-Based SAT Solvers
• Variable selection heuristic
• Clause learning
• The watched literals
scheme
• Conflict-directed
Backjump
• Fast backjump
• Assignment stack
shrinking
• Cnflict clause minimization
• Randomized restrts
• a.k.a. decision strategy
• MOMS, BOHM maximize a moderately
complex function of the dcurrent var.
and cls. state.
• DLIS selects and fix the literal occuring
most frequently in the yet unsatisfied
clauses
• VSIDS chooses a literal based on its
weight which preiodically decays but is
boosted if a clause in which it appears
is used in deriving a conflict.
15

View Slide

• Clause learning
scheme
Backjump
• Fast backjump
shrinking
• Critical role in the success of modern
complete SAT solvers.
• The idea here is to
• cache lcauses of conﬂictz as
learned clauses.
• utilize this information to prune
the search in a different part of the
search space encountered later.
16

View Slide

• Clause learning
scheme
Backjump
• Fast backjump
shrinking
• is a implementaion technique to
accelerate unit propagation, introduced
in zChaff.
• which key idea is to maintain and
lwatchztwo special literals for each not
yet satisﬁed clause.
• {U, U} : Not yet statisﬁed
• {0, U} : Unit clause
• {0, 0} : Empty clause
• has high compatibility with clause
learning.
17

View Slide

• Clause learning
scheme
Backjump
• Fast backjump
shrinking
• Backtrack just returns to previous
branching point.
• Backjump repeats backtrack safely as
long as possible.
18

View Slide

• Clause learning
scheme
Backjump
• Fast backjump
shrinking
• It lets a solver jump directly to a lower
decision level d where;
• even one branch leads to a conflict
involving variables at levels d or
lower only.
• for completeness, the level d is not
marked as unsatisfiable.
• While conflict-directed backjumping is
always beneficial, fast backjumping
may not be so.
19

View Slide

• Clause learning
scheme
Backjump
• Fast backjump
shrinking
• It’s a technique to learn smaller and
more pertinent learnt clauses.
• When a conﬂict occurs because of a
clause C′, and the size of learnt clause
C exceeds a certain threshold length;
• the solver backtracks to almost the
highest decision level of the
literals in C,
• it then starts assigning to FALSE
the unassigned literals of C′ until a
new conﬂict is encountered.
20

View Slide

• Clause learning
scheme
Backjump
• Fast backjump
shrinking
• The idea is to try to reduce the size of a
learned conﬂict clause C by repeatedly
identifying and removing any literals of
C that are implied to be FALSE when
the rest of the literals in C are set to
FALSE.
a ∨ b, ¬a ∨ c
b ∨ c 21

View Slide

• Clause learning
scheme
Backjump
• Fast backjump
shrinking
• It allows a SAT solver to arbitrarily stop
the search and restart their branching
process from decision level zero.
• Most of the current SAT solvers, employ
aggressive restart strategies,
sometimes restarting after as few as 20
to 50 backtracks.
22

View Slide

Clause Learning and Iterative DPLL
23

View Slide

Clause Learning and Iterative DPLL
24

View Slide

Cleause Learning
• The idea of clause learning came from Artiﬁcial Intelligence;
• seeking to improve backtrack search algorithms by generating
explanations for failure points,
• For general constraint satisfaction problems the explanations are
called lconﬂictszor lno-goodsz.
• In early works, clause learning could only obtain unusable clauses,
but various studies have led to success.
25

View Slide

Implication graph
Implication graph
The implication graph G at a given stage of DPLL is a directed
acyclic graph with edges labeled with sets of clauses.
26

View Slide

How to build implication graph
1. Create a node for each decision literal, labeled with that literal.
2. If there is an unit clause C when regarding nodes as the partial
assingment;
C =
k
∨
i=1
li
∨ l, where {li
} are assigned to FALSE
• Add a node l (if not in G)
• Add edges (li
, l) for all i = 1, . . . , k (if not in G)
• These edges are labeld with C to specify the cause of implication.
3. Repeat step 2 while no such clause found.
4. Add a special ’conﬂict’ node ¯
Λ. For any variable x s.t. x, ¬x are both
in G, add edges (x, ¯
Λ) and (¬x, ¯
Λ).
27

View Slide

Conflict on Implication Graph
• An implication graph may not contain a conflict at all, or it may
contain many conflict variables and several ways of deriving any
single literal.
28

View Slide

Conflict Graph
A conflict graph H = (VH
, EH
) is a subgraph of the implication graph
with the following propeties:
• H contains ¯
Λ and exactly one conflict variable.
• ∀l ∈ VH
have a path to ¯
Λ
• ∀l ∈ VH
\
{
¯
Λ
}
satisfies either of the following:
1. l is a decision literal
2. ∃¬l1
, . . . , ¬lk
∈ VH
s.t. ∀¬li
holds (¬li
, l) ∈ EH
and there exists a
known clause (l1
∨ · · · ∨ lk
∨ l)
29

View Slide

Reason/Conﬂict Side
30

View Slide

Learning Scheme - Relsat
• Relsat uses the cut whose conﬂict side consists of all implied
variables at the current decision level.
• The learnt clause (¬a ∨ ¬b) has exactly one variable from the current
decision level (i.e. b).
• After learning it and backtraking until unassigning b (to decision
level 2), ﬁlpping the value b to ¬b.
31

View Slide

Learning Scheme - Relsat
• Relsat uses the cut whose conﬂict side consists of all implied
variables at the current decision level.
• The learnt clause (¬a ∨ ¬b) has exactly one variable from the current
decision level (i.e. b).
• After learning it and backtraking until unassigning b (to decision
level 2), ﬁlpping the value b to ¬b.
32

View Slide

Unique Implication Points (UIPs)
• An UIP (unique implication point) is a node at the latest decision
level s.t.
• every path from the lastest decision variable to conﬂict literal must go
through the node.
• Intuitively, an UIP is single reason at the current decisioin level.
• There can be more than one UIP , but not 0.
• The decision variabe is the obvious UIP.
33

View Slide

Application of UIPs
• Learning scheme with UIP:
• relsat uses the decision variable as UIP
• Grasp and zChaff use FirstUIP, which is the UIP closest to the conﬂict
variable.
• Grasp also uses all UIPs to learn multiple clauses.
• kUIP – Generalized UIP to k previous decision levels.
• However, the study showed 1UIP is quite robust and outperforms all
other schemes they consider on most of the benchmarks.
34

View Slide

A Proof Complexity Perspective

View Slide

Propositional proof complexity
• Propositional proof complexity is the study of the proofs of validity
of mathematical statements expressed in a Boolean form.
• A propositional proof system is an algorithm A
Input A propositional statement S and a purported proof π
Returns Rejection / Acceptance
• The crucial property of A is that:
• For all invalid S, A rejects the pair (S, π) for all π
• For all valid S, A accepts the pair (S, π) for some π
35

View Slide

Resolution
• Resolution is a very simple proof system with only one rule.
a ∨ b, ¬a ∨ c
b ∨ c
• Repeated application of this rule can derive an empty disjunction iff
the initial formula is unsatisﬁable;
• Such a derivation serves as a proof of unsatisﬁability of the formula.
• There are several resolution types due to the implementaion
restriction.
36

View Slide

General Resolution
1 Paul Beame University of Washington Satisﬁability and Unsatisﬁability: Proof Complexity and Algorithms.
http://slideplayer.com/slide/4978670/
37

View Slide

Tree-like Resolution
1 Paul Beame University of Washington Satisﬁability and Unsatisﬁability: Proof Complexity and Algorithms.
38

View Slide

Variation of Resolution
• Tree-like resolution uses non-empty derived clauses exactly once in
the proof and is equivalent to an optimal DPLL procedure.
• Regular resolution allows any variable to be resolved upon at most
once along any “path” in the proof from an initial clause to Λ,
allowing (restricted) reuse of derived clauses.
• While these and other reﬁnements are sound and complete as proof
systems, they differ vastly in efﬁciency.
39

View Slide

SAT Solver as Proof System
• Most of today`s complete SAT solvers implement a subset of the
resolution proof system.
• Till recently, it was not clear:
• Where do they ﬁt in the proof system hierarchy?
• How do we compare them to reﬁnements of resolution, e.g. regular
resolution?
• Clause learning and random restarts was considered to be
important.
• Despite overwhelming empirical evidence, for many years not much
was known of the ultimate strengths and weaknesses of the two.
40

View Slide

FIrstNewCut
• Beame, Kautz, and Sabharwal answered several of these questions
in a formal proof complexity framework.
• They characterized clause learning as a proof system called CL, and
proposed a new learning scheme called FirstNewCut
• FirstNewCut can provide exponentially shorter proofs than
reﬁnement of general resolution satisfying a natural
self-reduction property.
• These include regular and ordered resolution.
• These are already known to be much stronger than the ordinary DPLL
procedure.
• They also showed that a slight variant of clause learning with
unlimited restarts is as powerful as general resolution itself.
41

View Slide

Ralation Between SAT/UNSAT Proof Complexity
• INTEREST is a familiy of unsatisfiable formulas, BECAUSE only
proofs of unsatisfiability can be large.
• The minimum proofs of satisfiability are O(n).
• However, in practice, many formulas are satisfiable.
• Achlioptas, Beame, and Molloy have shown
• Negative proof complexity results for unsatisfiable formulas can be
used;
• for the run-time lower bounds of specific inference algorithms,
especially DPLL, running on satisfiable formulas as well.
• The key observation in their work is that before hitting a satisfying
assignment, an algorithm is very likely to explore a large
unsatisfiable part of the search space due to uninformative initial
assignments.
42

View Slide

Branching Heuristic for SAT/UNSAT proofs.
• Proof complexity does not capture everything we intuitively mean by
the power of a reasoning system because it says nothing about how
difﬁcult it is to ﬁnd shortest proofs.
• However, it is a good notion with which to begin our analysis
because the size of proofs provides a lower bound on the running
time of any implementation of the system.
Negative Positive
Proofs Size Must be large Small proofs exist
Perfect Branching Hopeless Possible
43

View Slide

DPLL and Tree-like Resolution
For a CNF formula F, the size of the smallest DPLL refutation of F is equal
to the size of the smallest tree-like resolution refutation of F.
• The best performances are the same.
• Considering a propositional proof system CL is exponentially
stronger than tree-like resolution, clause learning is very effective.
44

View Slide

Natural Reﬁnement / Proper
• CS
(F) : the length of a shortest refutation of a formula F under a
proof system S
• tree-like, regular, linear, positive, negative, semantic, and ordered
resolution are natural reﬁnements.
• tree-like, reglar, and ordered resolution are exponentially-proper.
45

View Slide

The Power of CL
• (pros) There are some formulas that CL is more effective than some
exponentially-propered resolutions.
• (cons) CL may not be able to simulate all regular resolution proofs!
46

View Slide

CL– with non-redundant vs. RES
• CL– – : CL allowed to branch on a literal whose value is already
assigned.
• Such a branch can:
• lead to an immediate conflict;
• allow one to learn a key conflict clause that would otherwise have
not been learned.
• A clause learning scheme will be called non-redundant if on a
conflict, it always learns a clause not already known.
• Most of the practical clause learning schemes are non-redundant.
47

View Slide

Symmetry Breaking

View Slide

Symmetry
• Symmetry in real-world problems:
• in FPGA routing, all available wires or channels used for connecting two
switch boxes are equivalent;
• in circuit modeling, all gates of the same ltypezare interchangeable;
• in planning, all identical boxes that need to be moved from city A to
city B are equivalent;
• in multi-processor scheduling, all available processors are equivalent;
• in cache coherency protocols in distributed computing, all available
identical caches are equivalent.
• A key property of such objects is that when selecting k of them, we
can choose any k, without loss of generality .
• We would like to treat without-loss-of-generality reasoning in an
automatic fashion.
48

View Slide

Semantic Meaning
• A CNF formula consists of constraints over different kinds of
variables that typically represent tuples of these high level objects.
• During the problem modeling phase, we could have a Boolean
variable zw,c that is TRUE iff the ﬁrst end of wire w is attached to
connector c.
• When the formula is converted into DIMACS format, the semantic
meaning of the variables is discarded.
49

View Slide

Symmetry Breaking Predicates (SBPs)
• Symmetry Breaking Predicates (SBPs) is a method to deal with this
problem:
• Symmetries can be broken by adding one lexicographic ordering
constraint, called llex-constraintsz.
• This constraint removes all redundant solutions but the lexically-ﬁrst
solutions.
• The idea is to identify the set of permutations of variables {πi
},
where πi
keeps the formula unchanged.
• Let σ be a lexically-ﬁrst solution, πi
(σ) is also a satisfying assignment.
50

View Slide

Symmetry in SAT – Graph Isomorphism
• Shatter by Aloul et al. improves the number of lex-constraints from
O(n2) to O(n), where n is the number of variables.
• They use techniques of graph isomorphism to generate SBPs.
• Since there is no polynomial-time algorithms to praph isomorphism,
the number os SBPs to break all symetries can be exponential.
• Shatter handled this by discarding “large” symmetries.
51

View Slide

Symmetry in SAT – Non-CNF Formulation
• Utilizing non-CNF formulations , known as pseudo-Boolean
inequalities, is another approach for symmetry problem.
• PBS by Aloul et al.
• pbChaff by Dixon et al.
• Galena by Chai and Kuehlmann
• Instead of resolution system, they use Cutting Planes proof system.
• It is difﬁcult to implement, so pseudo-Boolean solvers often implement
only a subset of it.
52

View Slide

2.3 SAT Solver Technology -
Incomplete Methods

View Slide

Incomplete SAT Solver
• What about an incomplete method?
• It can provide the model of satisfiable formula, but it is not always
possible to solve;
• It can’t say anything about unsatisfiability.
• Incomplete methods significantly outperform DPLL-base methods
for some problems.
• They are generally based on stochastic local search, while typical
complete methods are based on exhaustive branching and
backtracking search.
53

View Slide

Local Search on Satisfiability
• Original impetus of local search on satisfiability problems was the
successful application for finding solutions to N-queens problems.
• Researchers regarded N-queens was an easy problem.
• They felt that it would fail in practice for SAT.
54

View Slide

GSAT
• GSAT is based on randomized local search technique.
• Initialize truth assignment randomlly.
• Then, greedily ﬂip the assignment of the variable that leads to the
greatest decrease in the total number of UNSAT clauses.
• Finish the search if a model is found, otherwise repeat ﬂiping.
55

View Slide

GSAT
• GSAT outperformed DPLL-based solvers on various SAT problem.
• From randomly generated problems to SAT encodings of graph
coloring problems.
• Most time is spent searching plateaus
• Plateaus are possible sideways movements that don’t increase/
decrease the # of UNSAT clauses too much.
56

View Slide

Walksat
57

View Slide

Walksat
• Walksat is a refinement of GSAT, employing random walk moves
of a standard Metropolis search.
• For the purpose to increase randomness.
• Always select the variable to flip from unsatisfied clauses.
• If there is a variable that can be flipped without affecting satisfied
clauses, pick it up.
• Otherwise, perform a p-greedy strategy.
• This idea is inpired by the O(n2) randomized algorithm for 2-SAT.
• It guarantees that one can reach a model by selecting only from
variables on unsatified clauses.
58

View Slide

Walksat vs. GSAT
http://ai.cs.unibas.ch/_files/teaching/fs16/ai/slides/ai32.pdf
59

View Slide

Phase Transition Phenomenon

View Slide

Clause-to-Variable Ratio
• Random k-SAT with n variables and m clauses.
• For each clause, select k unique variables uniformaly.
• For each variable, negate it with probablity 0.5
• clause-to-variable ratio
α =
m
n
• The difﬁculity of random k-SAT problems is characterized by α.
• For random 3-SAT, the run-time of problems with α > 4.26 tends to be
much larger than the opposite one.
60

View Slide

Phase Transition
• Left: The run-time
• Right: The ratio of unsatisﬁable problem
• As n grows, the phenomenon becomes shaper and shaper.
61

View Slide

2.3.2 Survey Propagation

View Slide

Survey Propagation
• Survey Propagation (SP) is a new algorithm for solving hard
combinational problems.
• It showed good results on random 3-SAT instances with one million
variables.
• SP has a connection to Blief Propagation (BP):
• BF is a method to calculate marginal probability on graphical models
(factor-graph).
• Both using an iterative process of local “message” update.
• Unfortunately, the success of SP is limited to random SAT instances.
62

View Slide

Survey Propagation
63

View Slide

2.4 Runtime Variance and
Problem Structure

View Slide

Exceptionally Hard Problem
• The performance of backtrack-style modhods can vary:
• The way to select the next variable to branch on.
• In what order the possible values are assigned to the variable.
• There are exceptionally hard problems.
• They ignore the “easy-hard-easy” phenomenon.
• They occur in the under-constrained area.
• However, subsequent research showed that such instances are not
inherently difﬁcult.
• Simply renaming the variables or by considering a different search
heuristic such can easily solved such problems.
• The “hardness” is in the combination of the instance with the search
method, rather than in the instances per se2.
• This is why the median is used to measure the “hardness” of problems.
2Adverb: by or in itself or themselves; intrinsically.
64

View Slide

Fat and Heavy Tailed Behavior
• Runtime distributions of search methos provides a better
characterization.
• Quite often complete backtrack search methods exhibit fat and
heavy-tailed behavior.
• Looks like lognormal, Wibull dist.
• Such distributions can be observed:
• when running a deterministic backtracking procedure on a distribution
of random instances;
• by repeated runs of a randomized backtracking procedure on a single
instance.
65

View Slide

Runtime of CSP
Summarizing CSP Hardness with Continuous Probability Distributions. Proceedings of the National Conference on Artiﬁcial Intelligence. 327-333.
66

View Slide

Heavy Tail
• fat-tailedness –
• kurtosis = µ4
µ2
2
, where µ2
, µ4
are 2nd and 4th moment resp.
• A distribution with igh central peak and long tails has generally large
kurtosis.
• A heavy-taild distribution is “heavier” than fat-tailed one.
• It have some inﬁnite moments, e.g. mean, variance, etc.
• DPLL style complete backtrack search methods have been shown to
exhibit heavy-tailed behavior.
• They can solve some problems so quickly, meanwhile do some
problems super slowly.
67

View Slide

2.4.2 Backdoors

View Slide

Backdoors
• The idea of heavy-tailed behavior comes from backdoor variables .
• Variables which gives us a polynomial subproblem when set.
• This is for explaining how a backtrack search method can get “lucky”
on certain runs.
• When backdoor variables are identiﬁed early on in the search and set
the right way, we say it’s lucky.
• The deﬁnition of a backdoor depends on a particular algorithm,
reffered to as sub-solver.
• It solves a tractable sub-case of the general constraint satisfaction
problem.
68

View Slide

Sub-Solver and Backdoor
• Intuitively, the backdoor are variables such that when set correctly,
the sub-solver can solve the remaining problem.
• A stronger notion of backdoors, strong backdoor, considers both
satisﬁable and unsatisﬁable problems.
69

View Slide

Backdoor Detection Complexity
• Nishimura et al. provided that detecting backdoor sets where the
sub-solver solves Horn or 2-CNF formulas, both of which are linear
time problems.
• They proved that the detection of such a strong backdoor set is
ﬁxed-parameter tractable;
• while the detection of a weak backdoor set is not.
• Dilkina et al. proved that the complexity of backdoor detection
jumps from NP to NP-hard/coNP-hard when adding certain
obvious inconsistency checks to the underlying class.
70

View Slide

Backdoor Size
• The backdoor size of random formulas is large.
• For random 3-SAT near the phase transition, about 30% of variables are
backdoors.
• That’s why DPLL-based search algorithm can’t solve these problems
• Structured (real-world) problems have very few backdoor variables.
71

View Slide

2.4.3 Restarts

View Slide

Restart Strategy
• Another way to exploit heavy-tailed behavior is to add restarts to
backtracking.
• Gomes et al. proposed randomized rapid restarts (RRR) to take
advantage of heavy-tailed behavior and boost the efficiency of
complete backtrack search.
• They also showed that restart strategy with a fixed cutoff
eliminates heavy-tail behavior and has finite moments.
72

View Slide

Luby’s Restarts
• Luby et al. showed that:
• When the distriution is known, the optimal restart policy is the ﬁxed
cutoff.
• When there is no a priori to the distribution, a universal strategy
minimizes the expected cost.
1, 1, 2, 1, 1, 2, 4, 1, 1, 2, 4, 8, . . .
• Theoretically the universal strategy is within a log factor of the optimal
one, but the convergence is too slowly in practice.
• State-of-the-art SAT solver’s restart:
• Default cutoff value, which is increased linearly
73

View Slide

Bayesian Framework
• Horvitz et al. introduced a Bayesian framework for learning
predictive models of randomized backtrack solvers based on
“knowledgeless” situation.
• Kautz et al. extended it, enabling restart policies to use information
of real-time observations about a solver.
• They also showed the optimal policy for dynamic restarts.
• It can be implemented using dynamic programming technique.
74

View Slide

Handbook of Knowledge Representation - Chapter 2: Satisfiability Solvers

Handbook of Knowledge Representation - Chapter 2: Satisfiability Solvers

OpenJNY

More Decks by OpenJNY

Featured

Transcript