Regularizations of Inverse Problems

GRETSI'13, Quartz, Brest, September 2013.

1. Régularisation de problèmes inverses Analyse unifiée de la robustesse Samuel VAITER CNRS, CEREMADE, Université Paris-Dauphine, France Travaux en collaboration avec M. GOLBABAEE, G. PEYRÉ et J. FADILI
2. Linear Inverse Problems inpainting denoising super-resolution Forward model y = x0 + w observations noise input operator
3. The Variational Approach x argmin x RN 1 2 ||y x||2 2 + J(x) Data fidelity Regularity
4. The Variational Approach x argmin x RN 1 2 ||y x||2 2 + J(x) Data fidelity Regularity J(x) = || x||2 J(x) = || x||1
5. The Variational Approach x argmin x RN 1 2 ||y x||2 2 + J(x) Data fidelity Regularity J(x) = || x||2 J(x) = || x||1 sparsity analysis-sparsity group-sparsity nuclear norm Tikhonov Total Variation Anti-sparse Polyhedral L1 + TV ... atomic norm decomposable norm Candidate J
6. Objectives Model selection performance x0 x w Prior model J
7. Objectives Model selection performance x0 x w Prior model J
8. Objectives Model selection performance x0 x w Prior model J How close ? in term of SNR in term of features
9. Union of Linear Models Union of models: T T linear spaces
10. Union of Linear Models Union of models: T T linear spaces T sparsity
11. Union of Linear Models Union of models: T T linear spaces block sparsity T sparsity
12. Union of Linear Models Union of models: T T linear spaces block sparsity T sparsity analysis sparsity
13. Union of Linear Models Union of models: T T linear spaces block sparsity low rank T sparsity analysis sparsity
14. Union of Linear Models Union of models: T T linear spaces block sparsity low rank Objective Encode T in a function T sparsity analysis sparsity
15. Gauges 1 J(x) J : RN R+ convex J( x) = J(x), 0
16. Gauges 1 J(x) J : RN R+ convex C C = {x : J(x) 1} J( x) = J(x), 0
17. Gauges 1 J(x) J : RN R+ convex C C = {x : J(x) 1} Geometry of C Union of Models (T )T T x T x x 0 T0 x x 0 T0 x 0 x T T0 ||x||1 |x1|+||x2,3|| ||x|| ||x|| J( x) = J(x), 0
18. Subdifferential |x| 0
19. Subdifferential |x| 0
20. J(x) = RN : x , J(x ) J(x)+ , x x Subdifferential |x| 0
21. J(x) = RN : x , J(x ) J(x)+ , x x Subdifferential |x| 0 |·|(0) = [ 1,1] x = 0, |·|(x) = {sign(x)}
22. From the Subdifferential to the Model J(x) x 0 J(x) x 0
23. From the Subdifferential to the Model J(x) x 0 J(x) x 0 Tx= VectHull( J(x)) Tx Tx Tx = : supp( ) supp(x)
24. From the Subdifferential to the Model J(x) x 0 J(x) x 0 ex = ProjTx ( J(x)) ex ex ex = sign(x) Tx= VectHull( J(x)) Tx Tx Tx = : supp( ) supp(x)
25. Regularizations and their Models J(x) = ||x||1 ex = sign(x) Tx = : supp( ) supp(x) x x J(x) = b ||xb|| ex = (N (xb))b B Tx = : supp( ) supp(x) x N (xb) = xb/||xb|| J(x) = ||x||∗ ex =UV Tx = : U V = 0 x x =UΛV ∗ J(x) = ||x||∞ ex = |I| 1 sign(x) Tx = : I sign(xI ) x x I = {i : |xi | = ||x||∞}
26. Dual Certificates and Model Selection x argmin x RN 1 2 ||y x||2 2 + J(x) Hypothesis: Ker Tx0 = {0} J regular enough
27. Dual Certificates and Model Selection x argmin x RN 1 2 ||y x||2 2 + J(x) Hypothesis: Ker Tx0 = {0} J regular enough ¯ D = Im ri( J(x0)) Tight dual certificates: x = x0 J(x) x
28. Dual Certificates and Model Selection x argmin x RN 1 2 ||y x||2 2 + J(x) Hypothesis: 0 = ( + Tx0 ) ex0 Minimal norm pre-certificate: Tx = Tx0 and ||x x0|| = O(||w||) If 0 ¯ D,||w|| small enough and ||w||, then x is the unique solution. Moreover, [V. et al. 2013] 1: [Fuchs 2004] 1 2: [Bach 2008] Ker Tx0 = {0} J regular enough ¯ D = Im ri( J(x0)) Tight dual certificates: x = x0 J(x) x
29. Example: Sparse Deconvolution x = i xi (· i) J(x) = ||x||1 Increasing : reduces correlation. reduces resolution. x0 x0
30. Example: Sparse Deconvolution x = i xi (· i) J(x) = ||x||1 Increasing : reduces correlation. reduces resolution. x0 x0 I = j : x0[j] = 0 || 0,Ic || < 1 0 ¯ D support recovery || 0,Ic || 1 2 20
31. Example: 1D TV Denoising J(x) = || x||1 = Id I = {i : ( x0)i = 0} x0
32. Example: 1D TV Denoising J(x) = || x||1 = Id I = {i : ( x0)i = 0} x0 +1 1 0 = div( 0) where j I,( 0)j = 0 x0 I J || 0,Ic || < 1 Support stability
33. Example: 1D TV Denoising J(x) = || x||1 = Id I = {i : ( x0)i = 0} x0 +1 1 0 = div( 0) where j I,( 0)j = 0 x0 I J || 0,Ic || < 1 Support stability x0 || 0,Ic || = 1 2-stability only
34. Conclusion Gauges: encode linear models as singular points
35. Conclusion Gauges: encode linear models as singular points Certificates: guarantees of model selection / 2 robustness (see poster 208 for a pure robustness result)
36. Conclusion Merci de votre attention ! Gauges: encode linear models as singular points Certificates: guarantees of model selection / 2 robustness (see poster 208 for a pure robustness result)