Robust Sparse Analysis Regularization

Slide 1

Slide 1 text

Robust Sparse Analysis Regularization Samuel Vaiter Tuesday, November 8, 11

Slide 2

Slide 2 text

Inverse Problems Tuesday, November 8, 11

Slide 3

Slide 3 text

Inverse Problems Several problems Inpaiting Super-resolution Tuesday, November 8, 11

Slide 4

Slide 4 text

Inverse Problems ill-posed Linear hypothesis One model y = x0 + w Observations Operator Unknown signal Noise Several problems Inpaiting Super-resolution Tuesday, November 8, 11

Slide 5

Slide 5 text

Inverse Problems ill-posed Linear hypothesis One model y = x0 + w Observations Operator Unknown signal Noise Several problems Inpaiting Super-resolution Regularization x? 2 argmin x 2RN 1 2 || y x ||2 2 + J ( x ) Tuesday, November 8, 11

Slide 6

Slide 6 text

Inverse Problems ill-posed Linear hypothesis One model y = x0 + w Observations Operator Unknown signal Noise x? 2 argmin x = y J ( x ) Noiseless 0 Several problems Inpaiting Super-resolution Regularization x? 2 argmin x 2RN 1 2 || y x ||2 2 + J ( x ) Tuesday, November 8, 11

Slide 7

Slide 7 text

space domain Synthesis Sparsity of Natural Images Tuesday, November 8, 11

Slide 8

Slide 8 text

space domain Synthesis Sparsity of Natural Images frequency-space domain Orthogonal Wavelets Tuesday, November 8, 11

Slide 9

Slide 9 text

space domain Synthesis Sparsity of Natural Images frequency-space domain Orthogonal Wavelets Tuesday, November 8, 11

Slide 10

Slide 10 text

Many almost null coe cients space domain Synthesis Sparsity of Natural Images frequency-space domain Orthogonal Wavelets Tuesday, November 8, 11

Slide 11

Slide 11 text

Many almost null coe cients space domain Good approximation Sparsity of natural images in frequency-space domain J(x) = min ↵ || ↵ ||1 subject to x = D↵ Synthesis Sparsity of Natural Images frequency-space domain Orthogonal Wavelets Tuesday, November 8, 11

Slide 12

Slide 12 text

An Other Sparsity Measure space domain Tuesday, November 8, 11

Slide 13

Slide 13 text

An Other Sparsity Measure space domain analysis r Tuesday, November 8, 11

Slide 14

Slide 14 text

An Other Sparsity Measure space domain analysis r Tuesday, November 8, 11

Slide 15

Slide 15 text

non-null coe cients : discontinuities An Other Sparsity Measure space domain analysis r Tuesday, November 8, 11

Slide 16

Slide 16 text

non-null coe cients : discontinuities Sparsity of cartoon images after gradient operator application J ( x ) = || D ⇤ x ||1 Here TV : D⇤ = r An Other Sparsity Measure space domain analysis r Tuesday, November 8, 11

Slide 17

Slide 17 text

Sparse Regularizations Synthesis argmin ↵2RQ 1 2 ||y ↵||2 2 + ||↵||1 = D x = D↵ Tuesday, November 8, 11

Slide 18

Slide 18 text

Sparse Regularizations Synthesis argmin ↵2RQ 1 2 ||y ↵||2 2 + ||↵||1 = D x = D↵ Analysis argmin x 2RN 1 2 || y x ||2 2 + || D ⇤ x ||1 Tuesday, November 8, 11

Slide 19

Slide 19 text

Sparse Regularizations = 6= 0 D x ↵ Synthesis argmin ↵2RQ 1 2 ||y ↵||2 2 + ||↵||1 = D x = D↵ Analysis argmin x 2RN 1 2 || y x ||2 2 + || D ⇤ x ||1 Tuesday, November 8, 11

Slide 20

Slide 20 text

Sparse Regularizations = 6= 0 D x ↵ = D⇤ x ↵ Synthesis argmin ↵2RQ 1 2 ||y ↵||2 2 + ||↵||1 = D x = D↵ Analysis argmin x 2RN 1 2 || y x ||2 2 + || D ⇤ x ||1 Tuesday, November 8, 11

Slide 21

Slide 21 text

Sparse approx. of x ? in D Sparse Regularizations = 6= 0 D x ↵ = D⇤ x ↵ Synthesis argmin ↵2RQ 1 2 ||y ↵||2 2 + ||↵||1 = D x = D↵ Analysis argmin x 2RN 1 2 || y x ||2 2 + || D ⇤ x ||1 Tuesday, November 8, 11

Slide 22

Slide 22 text

Correlation of x ? and D sparse Sparse approx. of x ? in D Sparse Regularizations = 6= 0 D x ↵ = D⇤ x ↵ Synthesis argmin ↵2RQ 1 2 ||y ↵||2 2 + ||↵||1 = D x = D↵ Analysis argmin x 2RN 1 2 || y x ||2 2 + || D ⇤ x ||1 Tuesday, November 8, 11

Slide 23

Slide 23 text

Signal Model of Synthesis Sparsity Tuesday, November 8, 11

Slide 24

Slide 24 text

Signal Model of Synthesis Sparsity d1 d2 Tuesday, November 8, 11

Slide 25

Slide 25 text

Signal Model of Synthesis Sparsity d1 d2 Tuesday, November 8, 11

Slide 26

Slide 26 text

||↵||1 = 1 Signal Model of Synthesis Sparsity d1 d2 Tuesday, November 8, 11

Slide 27

Slide 27 text

||↵||1 = 1 Signal Model of Synthesis Sparsity d1 d2 y = ↵ Tuesday, November 8, 11

Slide 28

Slide 28 text

||↵||1 = 1 Signal Model of Synthesis Sparsity d1 d2 y = ↵ Tuesday, November 8, 11

Slide 29

Slide 29 text

||↵||1 = 1 Signal Model of Synthesis Sparsity d1 d2 y = ↵ ↵? Tuesday, November 8, 11

Slide 30

Slide 30 text

||↵||1 = 1 sparsest solution Signal Model of Synthesis Sparsity d1 d2 y = ↵ ↵? Tuesday, November 8, 11

Slide 31

Slide 31 text

Analysis Counterpart Tuesday, November 8, 11

Slide 32

Slide 32 text

Analysis Counterpart d1 d2 d3 Tuesday, November 8, 11

Slide 33

Slide 33 text

Analysis Counterpart d1 d2 d3 Tuesday, November 8, 11

Slide 34

Slide 34 text

Analysis Counterpart d1 d2 d3 || D ⇤ x ||1 = 1 Tuesday, November 8, 11

Slide 35

Slide 35 text

Analysis Counterpart d1 d2 d3 || D ⇤ x ||1 = 1 y = x Tuesday, November 8, 11

Slide 36

Slide 36 text

Analysis Counterpart d1 d2 d3 || D ⇤ x ||1 = 1 y = x Tuesday, November 8, 11

Slide 37

Slide 37 text

Analysis Counterpart d1 d2 d3 || D ⇤ x ||1 = 1 y = x x ? Tuesday, November 8, 11

Slide 38

Slide 38 text

Behaviour of Solutions x?( , , D ) = argmin x 2RN 1 2 || y x ||2 2 + || D ⇤ x ||1 Tuesday, November 8, 11

Slide 39

Slide 39 text

piecewise a ne y 7! x ? Observations Behaviour of Solutions x?( , , D ) = argmin x 2RN 1 2 || y x ||2 2 + || D ⇤ x ||1 Tuesday, November 8, 11

Slide 40

Slide 40 text

piecewise a ne y 7! x ? Observations Behaviour of Solutions x?( , , D ) = argmin x 2RN 1 2 || y x ||2 2 + || D ⇤ x ||1 P depends on sign(D ⇤ x ? (y)) and AP is a linear operator If [y, ¯ y] ⇢ P x ?(¯ y ) = AP x ?( y ) u y ¯ y P Tuesday, November 8, 11

Slide 41

Slide 41 text

piecewise a ne y 7! x ? Observations 7! x ? piecewise a ne Scaling Behaviour of Solutions x?( , , D ) = argmin x 2RN 1 2 || y x ||2 2 + || D ⇤ x ||1 P depends on sign(D ⇤ x ? (y)) and AP is a linear operator If [y, ¯ y] ⇢ P x ?(¯ y ) = AP x ?( y ) u y ¯ y P Tuesday, November 8, 11

Slide 42

Slide 42 text

piecewise a ne y 7! x ? Observations 7! x ? piecewise a ne Scaling D 7! x ? open question Dictionary Behaviour of Solutions x?( , , D ) = argmin x 2RN 1 2 || y x ||2 2 + || D ⇤ x ||1 P depends on sign(D ⇤ x ? (y)) and AP is a linear operator If [y, ¯ y] ⇢ P x ?(¯ y ) = AP x ?( y ) u y ¯ y P Tuesday, November 8, 11

Slide 43

Slide 43 text

there exists a unique solution x ? such that ||x? x0 ||2 = O(||w||2) supp( D ⇤ x ?) ✓ supp( D ⇤ x0) then, for every x0 such that supp(D ⇤ x0) = I, If RC(I) < 1 and > ||w||2CI, RC ( I ) : function of the support Robustness of Estimators D ⇤ x0 D ⇤ x ? x?( , , D ) = argmin x 2RN 1 2 || y x ||2 2 + || D ⇤ x ||1 y = x0 + w Tuesday, November 8, 11

Slide 44

Slide 44 text

Joint work with — Gabriel Peyr´ e (CEREMADE, Dauphine) — Charles Dossal (IMB, Bordeaux I) — Jalal Fadili (GREYC, ENSICAEN) Any questions ? Robust Sparse Analysis Regularization arXiv:1109.6222 Thanks Tuesday, November 8, 11