Low Complexity Models: Robustness and Sensivity Samuel Vaiter CEREMADE, Univ. Paris-Dauphine 4 février 2014 GDR Isis : Apprentissage de représentations et traitement du signal J. Fadili G. Peyré C. Dossal M. Golbabaee C. Deledalle IMB GREYC CEREMADE 

Outline Introduction General Framework Performance Guarantees 

Linear Inverse Problem y = Φ x0 + w Observations in Rq Unkown vector in Rn Degradation operator Noise denoising inpainting deblurring 

Variational Regularization x ∈ argmin x∈RN 1 2 ||Φx − y||2 + λ J(x) (Pλ(y)) Trade-off between data fidelity and prior regularization Issue considered 1. Performance guarantees: 2 error + model selection 

Outline Introduction General Framework Performance Guarantees 

Gauge J(x) 0 J(|λ|x) = |λ|J(x) J convex x → J(x) 1 C C a convex set (0 ∈ C) C = {x : J(x) 1} homogeneous env. 

Signal Models and Gauges (group) sparsity || · ||1, (|| · ||1,2) antisparsity || · ||∞ low-rank || · ||∗ sparse gradient ||∇ · ||1,2 

Canonical Model Space 0 x ∂J(x) Tx ex Model space Tx = VectHull(∂J(x))⊥ Generalized sign vector ex = PTx (∂J(x)) Sparsity || · ||1 Tx = {η : supp(η) ⊆ supp(x)} ex = sign(x) Trace Norm || · ||∗ Tx = {η : U∗ ⊥ ηV⊥ = 0} ex = UV ∗ SVD: x = UΛV ∗ 

Outline Introduction General Framework Performance Guarantees 

Certificate / Lagrange Multiplier x ∈ argmin Φx=Φx0 J(x) (P0(Φx0)) ∂J(x) x Φx = Φx0 α Dual certificates: Dx0 = Im Φ∗ ∩ ∂J(x0) Proposition ∃α ∈ Dx0 ⇔ x0 solution de (P0(Φx0)) 

Performance Guarantees with 2 norm Tight dual certificates ¯ Dx = Im Φ∗ ∩ ri ∂J(x) Restricted Injectivity Ker Φ ∩ Tx = {0} (RICx ) Theorem If ∃α ∈ ¯ Dx and (RICx ) satisfied and a solution x of (Pλ(y)), then λ ∼ ||w|| ⇒ ||x − x || Cα||w|| PW: [Grasmair et al. 2011] J(x − x ) = O(||w||) 

Performance Guarantees with Model Selection α ∈ Dx =⇒ α = Φ∗η and αT = ex Minimal-norm precertificates α0 ∈ argmin α=Φ∗η αTx =ex ||η|| Proposition If (RICx ), then α0=(Φ+ Tx Φ)∗ex Theorem If α0 ∈ ¯ Dx0 , for λ ∼ ||w|| small enough, the unique solution x of (Pλ(y)) satifies Tx = Tx0 and ||x0 − x || = O(||w||) PW: [Fuchs 2004] ( 1), [Bach 2008] ( 1 − 2, nuclear) 

1D TV Denoising Φ = Id J(x) = ||∇x||1 α0 ∈ ¯ Dx ⇐⇒ α0 = div q and ||qIc ,0||∞ < 1 i xi i xi k q0,k k +1 −1 Support stability No support stability Both are 2-stable 

Future Work • Extension to the infinite dimensional setting Grid-free setting Total Variation case • Efficient SURE computation Model SURE vs Algorithm SURE • Better understanding of the geometry Optimization over ¯ Dx Behavior of α ∈ Dx \ ¯ Dx • Performance in CS settings 

Thanks for your attention ! V., J. Fadili, G. Peyré and C. Dossal, Robust sparse analysis regularization, Information Theory, 2013 V., C. Deledalle, J. Fadili, G. Peyré and C. Dossal, Local Behavior of Sparse Analysis Regularization: Applications to Risk Estimation, ACHA, 2012 V., M. Golbabaee, M. J. Fadili et G. Peyré, Model Selection with Piecewise Regular Gauges, Tech. report, http://arxiv.org/abs/1307.2342, 2013 J. Fadili, V. and G. Peyré, Linear Convergence Rates for Gauge Regularization, ongoing work V., C. Deledalle, J. Fadili, G. Peyré and C. Dossal, The Degrees of Freedom of Block Analysis Regularizers, ongoing work