Low Complexity Models: Robustness

GDR ISIS, Télécom ParisTech, Paris, February 2014.

1. Low Complexity Models: Robustness and Sensivity Samuel Vaiter CEREMADE, Univ. Paris-Dauphine 4 février 2014 GDR Isis : Apprentissage de représentations et traitement du signal J. Fadili G. Peyré C. Dossal M. Golbabaee C. Deledalle IMB GREYC CEREMADE
2. Outline Introduction General Framework Performance Guarantees
3. Linear Inverse Problem y = Φ x0 + w Observations in Rq Unkown vector in Rn Degradation operator Noise denoising inpainting deblurring
4. Variational Regularization x ∈ argmin x∈RN 1 2 ||Φx − y||2 + λ J(x) (Pλ(y)) Trade-off between data fidelity and prior regularization Issue considered 1. Performance guarantees: 2 error + model selection
5. Outline Introduction General Framework Performance Guarantees
6. Gauge J(x) 0 J(|λ|x) = |λ|J(x) J convex x → J(x) 1 C C a convex set (0 ∈ C) C = {x : J(x) 1} homogeneous env.
7. Signal Models and Gauges (group) sparsity || · ||1, (|| · ||1,2) antisparsity || · ||∞ low-rank || · ||∗ sparse gradient ||∇ · ||1,2
8. Canonical Model Space 0 x ∂J(x) Tx ex Model space Tx = VectHull(∂J(x))⊥ Generalized sign vector ex = PTx (∂J(x)) Sparsity || · ||1 Tx = {η : supp(η) ⊆ supp(x)} ex = sign(x) Trace Norm || · ||∗ Tx = {η : U∗ ⊥ ηV⊥ = 0} ex = UV ∗ SVD: x = UΛV ∗
9. Outline Introduction General Framework Performance Guarantees
10. Certificate / Lagrange Multiplier x ∈ argmin Φx=Φx0 J(x) (P0(Φx0)) ∂J(x) x Φx = Φx0 α Dual certificates: Dx0 = Im Φ∗ ∩ ∂J(x0) Proposition ∃α ∈ Dx0 ⇔ x0 solution de (P0(Φx0))
11. Performance Guarantees with 2 norm Tight dual certificates ¯ Dx = Im Φ∗ ∩ ri ∂J(x) Restricted Injectivity Ker Φ ∩ Tx = {0} (RICx ) Theorem If ∃α ∈ ¯ Dx and (RICx ) satisfied and a solution x of (Pλ(y)), then λ ∼ ||w|| ⇒ ||x − x || Cα||w|| PW: [Grasmair et al. 2011] J(x − x ) = O(||w||)
12. Performance Guarantees with Model Selection α ∈ Dx =⇒ α = Φ∗η and αT = ex Minimal-norm precertificates α0 ∈ argmin α=Φ∗η αTx =ex ||η|| Proposition If (RICx ), then α0=(Φ+ Tx Φ)∗ex Theorem If α0 ∈ ¯ Dx0 , for λ ∼ ||w|| small enough, the unique solution x of (Pλ(y)) satifies Tx = Tx0 and ||x0 − x || = O(||w||) PW: [Fuchs 2004] ( 1), [Bach 2008] ( 1 − 2, nuclear)
13. 1D TV Denoising Φ = Id J(x) = ||∇x||1 α0 ∈ ¯ Dx ⇐⇒ α0 = div q and ||qIc ,0||∞ < 1 i xi i xi k q0,k k +1 −1 Support stability No support stability Both are 2-stable
14. Future Work • Extension to the infinite dimensional setting Grid-free setting Total Variation case • Efficient SURE computation Model SURE vs Algorithm SURE • Better understanding of the geometry Optimization over ¯ Dx Behavior of α ∈ Dx \ ¯ Dx • Performance in CS settings
15. Thanks for your attention ! V., J. Fadili, G. Peyré and C. Dossal, Robust sparse analysis regularization, Information Theory, 2013 V., C. Deledalle, J. Fadili, G. Peyré and C. Dossal, Local Behavior of Sparse Analysis Regularization: Applications to Risk Estimation, ACHA, 2012 V., M. Golbabaee, M. J. Fadili et G. Peyré, Model Selection with Piecewise Regular Gauges, Tech. report, http://arxiv.org/abs/1307.2342, 2013 J. Fadili, V. and G. Peyré, Linear Convergence Rates for Gauge Regularization, ongoing work V., C. Deledalle, J. Fadili, G. Peyré and C. Dossal, The Degrees of Freedom of Block Analysis Regularizers, ongoing work