Tweet
Tweet

Slide 1

Slide 1 text

Handbook of Knowledge Representation Chapter 2: Satisfiability Solvers Junya Yamaguchi May 2, 2018 Tokyo Institute of Technology, Inoue Lab yamaguchi.j.ab@m.titehc.ac.jp

Slide 2

Slide 2 text

SAT Problem Example • An assignment is a function ϕ : V → {0, 1} F = ( x literal ∨ y ∨ z ) ∧ (¬x ∨ ¬y) clause ∧ (¬y ∨ ¬z) ∧ (¬z ∨ ¬x) CNF formula assignment ϕ(x, y, z) = (0, 0, 1) derives F = 1. • A formula is satisfiable (SAT) iff there exists an assignment that evaluates it TRUE, which is called model. • Otherwise, we say a formula is unsatisfiable (UNSAT). 1

Slide 3

Slide 3 text

SAT ιϧόͷ׆༂ • ࠷ѱܭࢉྔ͕ࢦ਺ؔ਺Ͱ͋Δ͜ͱͰ༗໊͕ͩɺSAT ιϧό͸ଟ͘ͷྖҬͰޭ ੷Λ࢒͍ͯ͠Δ • ιϑτ΢ΣΞ/ϋʔυ΢ΣΞݕূ • ࣗಈςετύλʔϯੜ੒ • ϓϥϯχϯά • εέδϡʔϦϯά • ೥ʹҰճ SAT ͷίϯϖ͕͋Δ (SAT Competition) • ͦ͜Ͱ਺ଟ͘ͷૉ੖Β͍͠ SAT ιϧό͕஀ੜͨ͠ • Ϟμϯͳ SAT ιϧόʹͳΔͱɺ਺ສม਺ɾ਺ઍ੍໿͔Β੒Δඇৗʹ೉͍͠໰ ୊Ͱ΋ղ͚Δ 2

Slide 4

Slide 4 text

SAT ͱ஌ֶࣝशͷؔ܎ • ࠓͰ͸ͨ͘͞ΜͷԠ༻͕͋Δ͕ɺͦͷىݯ͸ ஌ֶࣝश • ஌ࣝͷදݱྗͱܭࢉྔͷτϨʔυΦϑʹؔ͢Δݚڀ (ୈ 3 ষͰѻ͏) ͕ϝΠϯ • Ծఆ: ࠷ѱܭࢉྔ͕ଟ߲ࣜ࣌ؒͰ͋Γͳ͕Β΋ɺΊͪΌͪ͘ΌΤϨΨϯτͰදݱ ྗͷߴ͍දݱݴޠΛΈ͚͍ͭͨ • 90 ೥୅ͷ͸͡Ίʹɺ2 ͭͷ࿦จ͕͜ͷԾఆʹ௅Μͩ • ͋Δಛघͳ໰୊Λআ͚͹ɺ΄ͱΜͲͷϥϯμϜ SAT ໰୊͸ͱͯ΋؆୯ʹͱ͚Δ ͜ͱΛࣔͨ͠࿦จ • ͦͷಛघͳ೉͍͠໰୊Ͱ͑͞΋ɺϩʔΧϧαʔνͷςΫχοΫΛ࢖͑͹؆୯ʹղ ͘͜ͱ͕Ͱ͖Δ͜ͱΛࣔͨ͠࿦จ 3

Slide 5

Slide 5 text

ࢲͨͪ͸࠷ѱܭࢉྔʹͱΒΘΕͯͳ͍͔ʁ • ਺ඦສ΋ͷม਺Λ΋ͭݱ࣮ͷ SAT ໰୊Ͱͷ੒ޭ • యܕతͳ SAT ໰୊΍ݱ࣮తͳ NP ׬શͳ໰୊ (ͷܭࢉྔ) ʹରͯ͠͸ɺޮ཰Α͘ ղ͘͜ͱ͕Ͱ͖ΔҰൠղ๏͕͋Δ • ࠷ѱܭࢉྔʹϏϏΓ͗ͯ͢͸͍͚ͳ͍ • ͜ͷ·· SAT ιϧό͕੒௕͢Ε͹ɺ΋ͬͱෳࡶͳ஌ࣝදݱͷݴޠΛѻ͑ΔΑ ͏ʹͳΔ͸ͣ • NO: ࠷ѱ࣌ܭࢉྔ͕ଟ߲ࣜͳΞϧΰϦζϜͰͷදݱ • YES: SAT ιϧόͰղ͚Δൣғͷදݱ 4

Slide 6

Slide 6 text

ষͷߏ੒ 1. ࣍ͷ 2 छྨͷιϧόʹ͓͍ͯ࢖ΘΕ͍ͯΔओͳςΫχοΫ • ׬શ (complete) SAT ιϧό • ෆ׬શ (incomplete) SAT ιϧό 2. ࣮ફతͳ SAT ූ߸Խʹର͢Δ্هςΫχοΫͷ༗ޮੑ 3. SAT ιϧόͷকདྷ΁ͷల๬ 5

Slide 7

Slide 7 text

ఆٛ • ໋୊࿦ཧࣜ (propositional or Boolean formula) ͸ɺม਺ͷू߹্ʹఆٛ ͞ΕΔ࿦ཧࣜ • ֤ม਺͸ {FALSE, TRUE} ͷͲͪΒ͔ͷ஋ΛऔΔ • ศ্ٓ {0, 1} Ͱද͢ࣄ͕ଟ͍ • ม਺ͷू߹ V ʹର͢Δ ׬શׂ౰ͯ (truth assignment)1͸ɺࣸ૾ σ : V → {0, 1} ͷ͜ͱɻ • ಛʹɺ໋୊࿦ཧࣜΛ 1 ʹධՁ͢ΔΑ͏ͳ׬શׂ౰ͯΛ satisfying assignment ΍ Ϟσϧ (model) ͱݺͿɻ • SAT ιϧόͰѻ͏ SAT ໰୊͸ CNF (conjunctive normal form) ͱݺ͹Ε Δಛผͳܗࣜͷ໋୊࿦ཧ͚ࣜͩʹ੍ݶ 1 ׬શׂ౰ͯ͸୯ʹʮׂ౰ͯʯͱݺ͹ΕΔ͜ͱ΋͋Γ·͕͢ɺຊεϥΠυͰ͸෦෼ׂ౰ͯͷ͜ͱΛׂ౰ͯͱݺͼɺ׬શׂ౰ͯͱ۠ผ͠·͢ɻ 6

Slide 8

Slide 8 text

ॆ଍Մೳੑ൑ఆ໰୊ Boolean satisfiability testing (SAT) ໰୊ ೖྗ CNF ܗࣜͷ໋୊࿦ཧࣜ (CNF ࣜ) F ࣭໰ F ʹ͸Ϟσϧ͕͋Δ͔ʁ CNF ࣜ અͷબݴ F = C1 ∧ C2 · · · ∧ Cm ·ͨ͸ F = {Ci }m i=1 અ Ϧςϥϧͷ࿈ݴ C = l1 ∨ l2 · · · ∨ ln ·ͨ͸ C = {li }n i=1 Ϧςϥϧ ม਺ x ͔ͦͷ൱ఆ ¬x. ม਺ x ∈ {0, 1} 7

Slide 9

Slide 9 text

ఆٛ • અʹؚ·ΕΔϦςϥϧ਺ΛɺઅͷαΠζͱݺͿɻ • e.g. (x ∨ ¬y ∨ z) ͳΒαΠζ͸ 3 • αΠζ͕ 0 ͷઅΛۭઅ (empty clause)ɺαΠζ͕ 1 ͷઅΛ୯Ґઅ (unit clause)ɺͦͯ͠αΠζ͕ 2 ͷઅΛόΠφϦઅ (binary clause) ͱͦΕͧΕ ݺͿɻ • ͢΂ͯͷઅͷαΠζ͕ͪΐ͏Ͳ k Ͱ͋Δ CNF ࣜΛ k-SAT ͱݺͿɻ • 2-SAT ͸ଟ߲ࣜ࣌ؒͰٻղՄೳ • 3-SAT Ҏ্ʹͳΔͱ NP ׬શ (x1 ∨ x2 ∨ x3 ) ∧ (x4 ∨ x5 ∨ x6 ) ∧ (x7 ∨ x8 ∨ x9 ) 8

Slide 10

Slide 10 text

ఆٛ • ෦෼ׂ౰ͯ (partial assignment) ͸ɺม਺ू߹ͷ෦෼ू߹ʹର͢Δ׬શׂ ౰ͯͷ͜ͱɻ • CNF ࣜ F ΁ͷ෦෼ׂ౰ͯ ρ ʹରͯ͠ɺρ Λ୅ೖͯ͠ಘΒΕͨࣜͷ͜ͱΛ simplified ͞ΕͨࣜͱݺͼɺF|ρ Ͱද͢ɻ • 1 ͭҎ্ͷϦςϥϧ͕ 1 ʹධՁ͞Εͨ͢΂ͯͷઅΛ࡟আ • 0 ʹධՁ͞Εͨ͢΂ͯͷϦςϥϧΛ࡟আ ࠓޙɺೖྗͱͯ͠༩͑Δ໋୊࿦ཧࣜʹ͸ CNF ࣜΛ҉໧ͷ͏ͪʹԾఆ͢Δ͕ɺ ଟ͘ͷ৔߹͜ͷԾఆ͸໰୊ʹͳΒͳ͍ɻ 9

Slide 11

Slide 11 text

2.2 SAT Solver TechnologyŠ Complete Methods

Slide 12

Slide 12 text

Complete Methods • A complete SAT solver, given the input formula F, either produces a satisfying assignment for F or proves that F is unsatisfiable. • Recent complete methods remain variants of a process introduced several decades ago, DPLL • DPLL procedure: • Was introduced in the early 1960`s. • Can prune of the search space based on falsified clauses. • Performs a backtrack search in the space of partial truth assignments. • Main improvements to DPLL: • Smart branch selection heuristics • Extensions like clause learning and randomized restarts • Well-crafted data structures such as lazy implementations and watched literals 10

Slide 13

Slide 13 text

2.2.1 The DPLL Procedure

Slide 14

Slide 14 text

DPLL Procedure 11

Slide 15

Slide 15 text

DPLL Procedure • Repeatedly select an unassigned literal l. • The step to choose l is called branching step. • Setting l to TRUE or FALSE is called a decision • decision level is used to reffer the recursion depth at that stage • Recursively search for a satisfying assignment for F|l and F|¬l . 12

Slide 16

Slide 16 text

DPLL Procedure • Repeatedly select an unassigned literal l. • The step to choose l is called branching step. • Setting l to TRUE or FALSE is called a decision • decision level is used to reffer the recursion depth at that stage • Recursively search for a satisfying assignment for F|l and F|¬l . 12

Slide 17

Slide 17 text

DPLL Procedure • Repeatedly select an unassigned literal l. • The step to choose l is called branching step. • Setting l to TRUE or FALSE is called a decision • decision level is used to reffer the recursion depth at that stage • Recursively search for a satisfying assignment for F|l and F|¬l . 12

Slide 18

Slide 18 text

DPLL Procedure • Repeatedly select an unassigned literal l. • The step to choose l is called branching step. • Setting l to TRUE or FALSE is called a decision • decision level is used to reffer the recursion depth at that stage • Recursively search for a satisfying assignment for F|l and F|¬l . 12

Slide 19

Slide 19 text

DPLL Procedure 13

Slide 20

Slide 20 text

DPLL Procedure – Unit Propagation • Initial state: F = (x) ∧ (¬x ∨ y) ∧ (¬x ∨ ¬y ∨ z) ρ = ∅ • There exists an unit clause (x). Execute unit propagation. F = (x)∧(¬x∨y) ∧ (¬x∨¬y ∨ z) ρ = {x} • Now, another unit clause (y) found, let’s more simplify the formula. F = (x) ∧ (¬x ∨ y)∧(¬x ∨ ¬y∨z) ρ = {x, y} • Execute unit propagation in the same way. F = (x) ∧ (¬x ∨ y) ∧ (¬x ∨ ¬y ∨ z) ρ = {x, y, z} • UnitPropagate() ends since there are no unit clauses. 14

Slide 21

Slide 21 text

Key Features of Modern DPLL-Based SAT Solvers • Variable selection heuristic • Clause learning • The watched literals scheme • Conflict-directed Backjump • Fast backjump • Assignment stack shrinking • Cnflict clause minimization • Randomized restrts • a.k.a. decision strategy • MOMS, BOHM maximize a moderately complex function of the dcurrent var. and cls. state. • DLIS selects and fix the literal occuring most frequently in the yet unsatisfied clauses • VSIDS chooses a literal based on its weight which preiodically decays but is boosted if a clause in which it appears is used in deriving a conflict. 15

Slide 22

Slide 22 text

Key Features of Modern DPLL-Based SAT Solvers • Variable selection heuristic • Clause learning • The watched literals scheme • Conflict-directed Backjump • Fast backjump • Assignment stack shrinking • Cnflict clause minimization • Randomized restrts • Critical role in the success of modern complete SAT solvers. • The idea here is to • cache lcauses of conflictz as learned clauses. • utilize this information to prune the search in a different part of the search space encountered later. 16

Slide 23

Slide 23 text

Key Features of Modern DPLL-Based SAT Solvers • Variable selection heuristic • Clause learning • The watched literals scheme • Conflict-directed Backjump • Fast backjump • Assignment stack shrinking • Cnflict clause minimization • Randomized restrts • is a implementaion technique to accelerate unit propagation, introduced in zChaff. • which key idea is to maintain and lwatchztwo special literals for each not yet satisfied clause. • {U, U} : Not yet statisfied • {0, U} : Unit clause • {0, 0} : Empty clause • has high compatibility with clause learning. 17

Slide 24

Slide 24 text

Key Features of Modern DPLL-Based SAT Solvers • Variable selection heuristic • Clause learning • The watched literals scheme • Conflict-directed Backjump • Fast backjump • Assignment stack shrinking • Cnflict clause minimization • Randomized restrts • Backtrack just returns to previous branching point. • Backjump repeats backtrack safely as long as possible. 18

Slide 25

Slide 25 text

Key Features of Modern DPLL-Based SAT Solvers • Variable selection heuristic • Clause learning • The watched literals scheme • Conflict-directed Backjump • Fast backjump • Assignment stack shrinking • Cnflict clause minimization • Randomized restrts • It lets a solver jump directly to a lower decision level d where; • even one branch leads to a conflict involving variables at levels d or lower only. • for completeness, the level d is not marked as unsatisfiable. • While conflict-directed backjumping is always beneficial, fast backjumping may not be so. 19

Slide 26

Slide 26 text

Key Features of Modern DPLL-Based SAT Solvers • Variable selection heuristic • Clause learning • The watched literals scheme • Conflict-directed Backjump • Fast backjump • Assignment stack shrinking • Cnflict clause minimization • Randomized restrts • It’s a technique to learn smaller and more pertinent learnt clauses. • When a conflict occurs because of a clause C′, and the size of learnt clause C exceeds a certain threshold length; • the solver backtracks to almost the highest decision level of the literals in C, • it then starts assigning to FALSE the unassigned literals of C′ until a new conflict is encountered. 20

Slide 27

Slide 27 text

Key Features of Modern DPLL-Based SAT Solvers • Variable selection heuristic • Clause learning • The watched literals scheme • Conflict-directed Backjump • Fast backjump • Assignment stack shrinking • Cnflict clause minimization • Randomized restrts • The idea is to try to reduce the size of a learned conflict clause C by repeatedly identifying and removing any literals of C that are implied to be FALSE when the rest of the literals in C are set to FALSE. a ∨ b, ¬a ∨ c b ∨ c 21

Slide 28

Slide 28 text

Key Features of Modern DPLL-Based SAT Solvers • Variable selection heuristic • Clause learning • The watched literals scheme • Conflict-directed Backjump • Fast backjump • Assignment stack shrinking • Cnflict clause minimization • Randomized restrts • It allows a SAT solver to arbitrarily stop the search and restart their branching process from decision level zero. • Most of the current SAT solvers, employ aggressive restart strategies, sometimes restarting after as few as 20 to 50 backtracks. 22

Slide 29

Slide 29 text

Clause Learning and Iterative DPLL 23

Slide 30

Slide 30 text

Clause Learning and Iterative DPLL 23

Slide 31

Slide 31 text

Clause Learning and Iterative DPLL 23

Slide 32

Slide 32 text

Clause Learning and Iterative DPLL 23

Slide 33

Slide 33 text

Clause Learning and Iterative DPLL 24

Slide 34

Slide 34 text

Clause Learning and Iterative DPLL 24

Slide 35

Slide 35 text

Clause Learning and Iterative DPLL 24

Slide 36

Slide 36 text

Clause Learning and Iterative DPLL 24

Slide 37

Slide 37 text

Clause Learning and Iterative DPLL 24

Slide 38

Slide 38 text

Cleause Learning • The idea of clause learning came from Artificial Intelligence; • seeking to improve backtrack search algorithms by generating explanations for failure points, • For general constraint satisfaction problems the explanations are called lconflictszor lno-goodsz. • In early works, clause learning could only obtain unusable clauses, but various studies have led to success. 25

Slide 39

Slide 39 text

Implication graph Implication graph The implication graph G at a given stage of DPLL is a directed acyclic graph with edges labeled with sets of clauses. 26

Slide 40

Slide 40 text

How to build implication graph 1. Create a node for each decision literal, labeled with that literal. 2. If there is an unit clause C when regarding nodes as the partial assingment; C = k ∨ i=1 li ∨ l, where {li } are assigned to FALSE • Add a node l (if not in G) • Add edges (li , l) for all i = 1, . . . , k (if not in G) • These edges are labeld with C to specify the cause of implication. 3. Repeat step 2 while no such clause found. 4. Add a special ’conflict’ node ¯ Λ. For any variable x s.t. x, ¬x are both in G, add edges (x, ¯ Λ) and (¬x, ¯ Λ). 27

Slide 41

Slide 41 text

Conflict on Implication Graph • An implication graph may not contain a conflict at all, or it may contain many conflict variables and several ways of deriving any single literal. 28

Slide 42

Slide 42 text

Conflict Graph A conflict graph H = (VH , EH ) is a subgraph of the implication graph with the following propeties: • H contains ¯ Λ and exactly one conflict variable. • ∀l ∈ VH have a path to ¯ Λ • ∀l ∈ VH \ { ¯ Λ } satisfies either of the following: 1. l is a decision literal 2. ∃¬l1 , . . . , ¬lk ∈ VH s.t. ∀¬li holds (¬li , l) ∈ EH and there exists a known clause (l1 ∨ · · · ∨ lk ∨ l) 29

Slide 43

Slide 43 text

Conflict Graph A conflict graph H = (VH , EH ) is a subgraph of the implication graph with the following propeties: • H contains ¯ Λ and exactly one conflict variable. • ∀l ∈ VH have a path to ¯ Λ • ∀l ∈ VH \ { ¯ Λ } satisfies either of the following: 1. l is a decision literal 2. ∃¬l1 , . . . , ¬lk ∈ VH s.t. ∀¬li holds (¬li , l) ∈ EH and there exists a known clause (l1 ∨ · · · ∨ lk ∨ l) 29

Slide 44

Slide 44 text

Reason/Conflict Side 30

Slide 45

Slide 45 text

Learning Scheme - Relsat • Relsat uses the cut whose conflict side consists of all implied variables at the current decision level. • The learnt clause (¬a ∨ ¬b) has exactly one variable from the current decision level (i.e. b). • After learning it and backtraking until unassigning b (to decision level 2), filpping the value b to ¬b. 31

Slide 46

Slide 46 text

Learning Scheme - Relsat • Relsat uses the cut whose conflict side consists of all implied variables at the current decision level. • The learnt clause (¬a ∨ ¬b) has exactly one variable from the current decision level (i.e. b). • After learning it and backtraking until unassigning b (to decision level 2), filpping the value b to ¬b. 32

Slide 47

Slide 47 text

Unique Implication Points (UIPs) • An UIP (unique implication point) is a node at the latest decision level s.t. • every path from the lastest decision variable to conflict literal must go through the node. • Intuitively, an UIP is single reason at the current decisioin level. • There can be more than one UIP , but not 0. • The decision variabe is the obvious UIP. 33

Slide 48

Slide 48 text

Application of UIPs • Learning scheme with UIP: • relsat uses the decision variable as UIP • Grasp and zChaff use FirstUIP, which is the UIP closest to the conflict variable. • Grasp also uses all UIPs to learn multiple clauses. • kUIP – Generalized UIP to k previous decision levels. • However, the study showed 1UIP is quite robust and outperforms all other schemes they consider on most of the benchmarks. 34

Slide 49

Slide 49 text

A Proof Complexity Perspective

Slide 50

Slide 50 text

Propositional proof complexity • Propositional proof complexity is the study of the proofs of validity of mathematical statements expressed in a Boolean form. • A propositional proof system is an algorithm A Input A propositional statement S and a purported proof π Returns Rejection / Acceptance • The crucial property of A is that: • For all invalid S, A rejects the pair (S, π) for all π • For all valid S, A accepts the pair (S, π) for some π 35

Slide 51

Slide 51 text

Resolution • Resolution is a very simple proof system with only one rule. a ∨ b, ¬a ∨ c b ∨ c • Repeated application of this rule can derive an empty disjunction iff the initial formula is unsatisfiable; • Such a derivation serves as a proof of unsatisfiability of the formula. • There are several resolution types due to the implementaion restriction. 36

Slide 52

Slide 52 text

General Resolution 1 Paul Beame University of Washington Satisfiability and Unsatisfiability: Proof Complexity and Algorithms. http://slideplayer.com/slide/4978670/ 37

Slide 53

Slide 53 text

Tree-like Resolution 1 Paul Beame University of Washington Satisfiability and Unsatisfiability: Proof Complexity and Algorithms. http://slideplayer.com/slide/4978670/ 38

Slide 54

Slide 54 text

Variation of Resolution • Tree-like resolution uses non-empty derived clauses exactly once in the proof and is equivalent to an optimal DPLL procedure. • Regular resolution allows any variable to be resolved upon at most once along any “path” in the proof from an initial clause to Λ, allowing (restricted) reuse of derived clauses. • While these and other refinements are sound and complete as proof systems, they differ vastly in efficiency. 39

Slide 55

Slide 55 text

SAT Solver as Proof System • Most of today`s complete SAT solvers implement a subset of the resolution proof system. • Till recently, it was not clear: • Where do they fit in the proof system hierarchy? • How do we compare them to refinements of resolution, e.g. regular resolution? • Clause learning and random restarts was considered to be important. • Despite overwhelming empirical evidence, for many years not much was known of the ultimate strengths and weaknesses of the two. 40

Slide 56

Slide 56 text

FIrstNewCut • Beame, Kautz, and Sabharwal answered several of these questions in a formal proof complexity framework. • They characterized clause learning as a proof system called CL, and proposed a new learning scheme called FirstNewCut • FirstNewCut can provide exponentially shorter proofs than refinement of general resolution satisfying a natural self-reduction property. • These include regular and ordered resolution. • These are already known to be much stronger than the ordinary DPLL procedure. • They also showed that a slight variant of clause learning with unlimited restarts is as powerful as general resolution itself. 41

Slide 57

Slide 57 text

Ralation Between SAT/UNSAT Proof Complexity • INTEREST is a familiy of unsatisfiable formulas, BECAUSE only proofs of unsatisfiability can be large. • The minimum proofs of satisfiability are O(n). • However, in practice, many formulas are satisfiable. • Achlioptas, Beame, and Molloy have shown • Negative proof complexity results for unsatisfiable formulas can be used; • for the run-time lower bounds of specific inference algorithms, especially DPLL, running on satisfiable formulas as well. • The key observation in their work is that before hitting a satisfying assignment, an algorithm is very likely to explore a large unsatisfiable part of the search space due to uninformative initial assignments. 42

Slide 58

Slide 58 text

Branching Heuristic for SAT/UNSAT proofs. • Proof complexity does not capture everything we intuitively mean by the power of a reasoning system because it says nothing about how difficult it is to find shortest proofs. • However, it is a good notion with which to begin our analysis because the size of proofs provides a lower bound on the running time of any implementation of the system. Negative Positive Proofs Size Must be large Small proofs exist Perfect Branching Hopeless Possible 43

Slide 59

Slide 59 text

DPLL and Tree-like Resolution For a CNF formula F, the size of the smallest DPLL refutation of F is equal to the size of the smallest tree-like resolution refutation of F. • The best performances are the same. • Considering a propositional proof system CL is exponentially stronger than tree-like resolution, clause learning is very effective. 44

Slide 60

Slide 60 text

Natural Refinement / Proper • CS (F) : the length of a shortest refutation of a formula F under a proof system S • tree-like, regular, linear, positive, negative, semantic, and ordered resolution are natural refinements. • tree-like, reglar, and ordered resolution are exponentially-proper. 45

Slide 61

Slide 61 text

The Power of CL • (pros) There are some formulas that CL is more effective than some exponentially-propered resolutions. • (cons) CL may not be able to simulate all regular resolution proofs! 46

Slide 62

Slide 62 text

CL– with non-redundant vs. RES • CL– – : CL allowed to branch on a literal whose value is already assigned. • Such a branch can: • lead to an immediate conflict; • allow one to learn a key conflict clause that would otherwise have not been learned. • A clause learning scheme will be called non-redundant if on a conflict, it always learns a clause not already known. • Most of the practical clause learning schemes are non-redundant. 47

Slide 63

Slide 63 text

Symmetry Breaking

Slide 64

Slide 64 text

Symmetry • Symmetry in real-world problems: • in FPGA routing, all available wires or channels used for connecting two switch boxes are equivalent; • in circuit modeling, all gates of the same ltypezare interchangeable; • in planning, all identical boxes that need to be moved from city A to city B are equivalent; • in multi-processor scheduling, all available processors are equivalent; • in cache coherency protocols in distributed computing, all available identical caches are equivalent. • A key property of such objects is that when selecting k of them, we can choose any k, without loss of generality . • We would like to treat without-loss-of-generality reasoning in an automatic fashion. 48

Slide 65

Slide 65 text

Semantic Meaning • A CNF formula consists of constraints over different kinds of variables that typically represent tuples of these high level objects. • During the problem modeling phase, we could have a Boolean variable zw,c that is TRUE iff the first end of wire w is attached to connector c. • When the formula is converted into DIMACS format, the semantic meaning of the variables is discarded. 49

Slide 66

Slide 66 text

Symmetry Breaking Predicates (SBPs) • Symmetry Breaking Predicates (SBPs) is a method to deal with this problem: • Symmetries can be broken by adding one lexicographic ordering constraint, called llex-constraintsz. • This constraint removes all redundant solutions but the lexically-first solutions. • The idea is to identify the set of permutations of variables {πi }, where πi keeps the formula unchanged. • Let σ be a lexically-first solution, πi (σ) is also a satisfying assignment. 50

Slide 67

Slide 67 text

Symmetry in SAT – Graph Isomorphism • Shatter by Aloul et al. improves the number of lex-constraints from O(n2) to O(n), where n is the number of variables. • They use techniques of graph isomorphism to generate SBPs. • Since there is no polynomial-time algorithms to praph isomorphism, the number os SBPs to break all symetries can be exponential. • Shatter handled this by discarding “large” symmetries. 51

Slide 68

Slide 68 text

Symmetry in SAT – Non-CNF Formulation • Utilizing non-CNF formulations , known as pseudo-Boolean inequalities, is another approach for symmetry problem. • PBS by Aloul et al. • pbChaff by Dixon et al. • Galena by Chai and Kuehlmann • Instead of resolution system, they use Cutting Planes proof system. • It is difficult to implement, so pseudo-Boolean solvers often implement only a subset of it. 52

Slide 69

Slide 69 text

2.3 SAT Solver Technology - Incomplete Methods

Slide 70

Slide 70 text

Incomplete SAT Solver • What about an incomplete method? • It can provide the model of satisfiable formula, but it is not always possible to solve; • It can’t say anything about unsatisfiability. • Incomplete methods significantly outperform DPLL-base methods for some problems. • They are generally based on stochastic local search, while typical complete methods are based on exhaustive branching and backtracking search. 53

Slide 71

Slide 71 text

Local Search on Satisfiability • Original impetus of local search on satisfiability problems was the successful application for finding solutions to N-queens problems. • Researchers regarded N-queens was an easy problem. • They felt that it would fail in practice for SAT. 54

Slide 72

Slide 72 text

GSAT • GSAT is based on randomized local search technique. • Initialize truth assignment randomlly. • Then, greedily flip the assignment of the variable that leads to the greatest decrease in the total number of UNSAT clauses. • Finish the search if a model is found, otherwise repeat fliping. 55

Slide 73

Slide 73 text

GSAT • GSAT outperformed DPLL-based solvers on various SAT problem. • From randomly generated problems to SAT encodings of graph coloring problems. • Most time is spent searching plateaus • Plateaus are possible sideways movements that don’t increase/ decrease the # of UNSAT clauses too much. 56

Slide 74

Slide 74 text

Walksat 57

Slide 75

Slide 75 text

Walksat • Walksat is a refinement of GSAT, employing random walk moves of a standard Metropolis search. • For the purpose to increase randomness. • Always select the variable to flip from unsatisfied clauses. • If there is a variable that can be flipped without affecting satisfied clauses, pick it up. • Otherwise, perform a p-greedy strategy. • This idea is inpired by the O(n2) randomized algorithm for 2-SAT. • It guarantees that one can reach a model by selecting only from variables on unsatified clauses. 58

Slide 76

Slide 76 text

Walksat vs. GSAT http://ai.cs.unibas.ch/_files/teaching/fs16/ai/slides/ai32.pdf 59

Slide 77

Slide 77 text

Phase Transition Phenomenon

Slide 78

Slide 78 text

Clause-to-Variable Ratio • Random k-SAT with n variables and m clauses. • For each clause, select k unique variables uniformaly. • For each variable, negate it with probablity 0.5 • clause-to-variable ratio α = m n • The difficulity of random k-SAT problems is characterized by α. • For random 3-SAT, the run-time of problems with α > 4.26 tends to be much larger than the opposite one. 60

Slide 79

Slide 79 text

Phase Transition • Left: The run-time • Right: The ratio of unsatisfiable problem • As n grows, the phenomenon becomes shaper and shaper. 61

Slide 80

Slide 80 text

2.3.2 Survey Propagation

Slide 81

Slide 81 text

Survey Propagation • Survey Propagation (SP) is a new algorithm for solving hard combinational problems. • It showed good results on random 3-SAT instances with one million variables. • SP has a connection to Blief Propagation (BP): • BF is a method to calculate marginal probability on graphical models (factor-graph). • Both using an iterative process of local “message” update. • Unfortunately, the success of SP is limited to random SAT instances. 62

Slide 82

Slide 82 text

Survey Propagation http://slideplayer.com/slide/4559400/ 63

Slide 83

Slide 83 text

2.4 Runtime Variance and Problem Structure

Slide 84

Slide 84 text

Exceptionally Hard Problem • The performance of backtrack-style modhods can vary: • The way to select the next variable to branch on. • In what order the possible values are assigned to the variable. • There are exceptionally hard problems. • They ignore the “easy-hard-easy” phenomenon. • They occur in the under-constrained area. • However, subsequent research showed that such instances are not inherently difficult. • Simply renaming the variables or by considering a different search heuristic such can easily solved such problems. • The “hardness” is in the combination of the instance with the search method, rather than in the instances per se2. • This is why the median is used to measure the “hardness” of problems. 2Adverb: by or in itself or themselves; intrinsically. 64

Slide 85

Slide 85 text

Fat and Heavy Tailed Behavior • Runtime distributions of search methos provides a better characterization. • Quite often complete backtrack search methods exhibit fat and heavy-tailed behavior. • Looks like lognormal, Wibull dist. • Such distributions can be observed: • when running a deterministic backtracking procedure on a distribution of random instances; • by repeated runs of a randomized backtracking procedure on a single instance. 65

Slide 86

Slide 86 text

Runtime of CSP Summarizing CSP Hardness with Continuous Probability Distributions. Proceedings of the National Conference on Artificial Intelligence. 327-333. 66

Slide 87

Slide 87 text

Heavy Tail • fat-tailedness – • kurtosis = µ4 µ2 2 , where µ2 , µ4 are 2nd and 4th moment resp. • A distribution with igh central peak and long tails has generally large kurtosis. • A heavy-taild distribution is “heavier” than fat-tailed one. • It have some infinite moments, e.g. mean, variance, etc. • DPLL style complete backtrack search methods have been shown to exhibit heavy-tailed behavior. • They can solve some problems so quickly, meanwhile do some problems super slowly. 67

Slide 88

Slide 88 text

2.4.2 Backdoors

Slide 89

Slide 89 text

Backdoors • The idea of heavy-tailed behavior comes from backdoor variables . • Variables which gives us a polynomial subproblem when set. • This is for explaining how a backtrack search method can get “lucky” on certain runs. • When backdoor variables are identified early on in the search and set the right way, we say it’s lucky. • The definition of a backdoor depends on a particular algorithm, reffered to as sub-solver. • It solves a tractable sub-case of the general constraint satisfaction problem. 68

Slide 90

Slide 90 text

Sub-Solver and Backdoor • Intuitively, the backdoor are variables such that when set correctly, the sub-solver can solve the remaining problem. • A stronger notion of backdoors, strong backdoor, considers both satisfiable and unsatisfiable problems. 69

Slide 91

Slide 91 text

Backdoor Detection Complexity • Nishimura et al. provided that detecting backdoor sets where the sub-solver solves Horn or 2-CNF formulas, both of which are linear time problems. • They proved that the detection of such a strong backdoor set is fixed-parameter tractable; • while the detection of a weak backdoor set is not. • Dilkina et al. proved that the complexity of backdoor detection jumps from NP to NP-hard/coNP-hard when adding certain obvious inconsistency checks to the underlying class. 70

Slide 92

Slide 92 text

Backdoor Size • The backdoor size of random formulas is large. • For random 3-SAT near the phase transition, about 30% of variables are backdoors. • That’s why DPLL-based search algorithm can’t solve these problems • Structured (real-world) problems have very few backdoor variables. 71

Slide 93

Slide 93 text

2.4.3 Restarts

Slide 94

Slide 94 text

Restart Strategy • Another way to exploit heavy-tailed behavior is to add restarts to backtracking. • Gomes et al. proposed randomized rapid restarts (RRR) to take advantage of heavy-tailed behavior and boost the efficiency of complete backtrack search. • They also showed that restart strategy with a fixed cutoff eliminates heavy-tail behavior and has finite moments. 72

Slide 95

Slide 95 text

Luby’s Restarts • Luby et al. showed that: • When the distriution is known, the optimal restart policy is the fixed cutoff. • When there is no a priori to the distribution, a universal strategy minimizes the expected cost. 1, 1, 2, 1, 1, 2, 4, 1, 1, 2, 4, 8, . . . • Theoretically the universal strategy is within a log factor of the optimal one, but the convergence is too slowly in practice. • State-of-the-art SAT solver’s restart: • Default cutoff value, which is increased linearly 73

Slide 96

Slide 96 text

Bayesian Framework • Horvitz et al. introduced a Bayesian framework for learning predictive models of randomized backtrack solvers based on “knowledgeless” situation. • Kautz et al. extended it, enabling restart policies to use information of real-time observations about a solver. • They also showed the optimal policy for dynamic restarts. • It can be implemented using dynamic programming technique. 74