Low Complexity Models: Robustness and Sensivity Samuel Vaiter CEREMADE, Univ. Paris-Dauphine February 3th, 2014 Présoutenance de thèse J. Fadili G. Peyré C. Dossal M. Golbabaee C. Deledalle IMB GREYC CEREMADE 

Outline Introduction General Framework Performance Guarantees Parameter Selection 

Linear Inverse Problem y = Φ x0 + w Observations in Rq Unkown vector in Rn Degradation operator Noise denoising inpainting deblurring 

Variational Regularization x ∈ argmin x∈RN 1 2 ||Φx − y||2 + λ J(x) (Pλ(y)) Trade-off between data fidelity and prior regularization 2 issues considered in this thesis 1. Performance guarantees: 2 error + model selection 2. Parameter selection: sensivity analysis + risk estimation 

Outline Introduction General Framework Performance Guarantees Parameter Selection 

Gauge J(x) 0 J(|λ|x) = |λ|J(x) J convex x → J(x) 1 C C a convex set (0 ∈ C) C = {x : J(x) 1} homogeneous env. 

Signal Models and Gauges (group) sparsity || · ||1, (|| · ||1,2) antisparsity || · ||∞ low-rank || · ||∗ sparse gradient ||∇ · ||1,2 

Canonical Model Space 0 x ∂J(x) Tx ex Model space Tx = VectHull(∂J(x))⊥ Generalized sign vector ex = PTx (∂J(x)) Sparsity || · ||1 Tx = {η : supp(η) ⊆ supp(x)} ex = sign(x) Trace Norm || · ||∗ Tx = {η : U∗ ⊥ ηV⊥ = 0} ex = UV ∗ SVD: x = UΛV ∗ 

Outline Introduction General Framework Performance Guarantees Parameter Selection 

Certificate / Lagrange Multiplier x ∈ argmin Φx=Φx0 J(x) (P0(Φx0)) ∂J(x) x Φx = Φx0 α Dual certificates: Dx0 = Im Φ∗ ∩ ∂J(x0) Proposition ∃α ∈ Dx0 ⇔ x0 solution de (P0(Φx0)) 

Performance Guarantees with 2 norm Tight dual certificates ¯ Dx = Im Φ∗ ∩ ri ∂J(x) Restricted Injectivity Ker Φ ∩ Tx = {0} (RICx ) Theorem If ∃α ∈ ¯ Dx and (RICx ) satisfied and a solution x of (Pλ(y)), then λ ∼ ||w|| ⇒ ||x − x || Cα||w|| PW: [Grasmair et al. 2011] J(x − x ) = O(||w||) 

Performance Guarantees with Model Selection α ∈ Dx =⇒ α = Φ∗η and αT = ex Minimal-norm precertificates α0 ∈ argmin α=Φ∗η αTx =ex ||η|| Proposition If (RICx ), then α0=(Φ+ Tx Φ)∗ex Theorem If α0 ∈ ¯ Dx0 , for λ ∼ ||w|| small enough, the unique solution x of (Pλ(y)) satifies Tx = Tx0 and ||x0 − x || = O(||w||) PW: [Fuchs 2004] ( 1), [Bach 2008] ( 1 − 2, nuclear) 

1D TV Denoising Φ = Id J(x) = ||∇x||1 α0 ∈ ¯ Dx ⇐⇒ α0 = div q and ||qIc ,0||∞ < 1 i xi i xi k q0,k k +1 −1 Support stability No support stability Both are 2-stable 

Outline Introduction General Framework Performance Guarantees Parameter Selection 

Parameter Selection Y = Φx0 + W ∼ N(Φx0, σ2) Prediction: µ(y) = Φx∗(y) Prediction risk: R(λ) = EW [||Φx0 − µ(Y )||2] In practice, projected risk or estimation risk 

Stein Unbiased Risk Estimation First order approximation µ(y + δ) = µ(y) + Dµ(y) · δ + O(||δ||2) Stein Unbiased Risk Estimation SURE(y) = ||y − µ(y)||2 − σ2Q + 2σ2df (y) df (y) = tr[Dµ(y)] Proposition (Stein 1981) If µ weakly differentiable, then EW [SURE(Y )] = EW [||Φx0 − µ(Y )||2] 

Local Behavior x (y) ∈ argmin x∈Rn 1 2 ||y − Φx||2 + λJ(x) (Pλ(y)) We assume that T = {Tx : x ∈ Rn} is finite. Theorem Assuming that J is definable in an O-minimal structure O, y → µ(y) = Φx (y) is differentiable except on a zero measure set H and div(µ)(y) = −ΦT (Φ∗ T ΦT + De(x∗))−1Φ∗ T where T = Tx∗ . H is definable in O and can be explicitely stated. PW: [Dossal 2012] ( 1, df = ||x (y)||0 ) 

Risk Estimation in Practice Φ subsampled Radon transform (16 measures), J(x) = ||∇x||1,2 x0 Φ+y x∗ λopt (y) 

Future Work • Extension to the infinite dimensional setting Grid-free setting Total Variation case • Efficient SURE computation Model SURE vs Algorithm SURE • Better understanding of the geometry Optimization over ¯ Dx Behavior of α ∈ Dx \ ¯ Dx • Performance in CS settings 

Thanks for your attention ! V., J. Fadili, G. Peyré and C. Dossal, Robust sparse analysis regularization, Information Theory, 2013 V., C. Deledalle, J. Fadili, G. Peyré and C. Dossal, Local Behavior of Sparse Analysis Regularization: Applications to Risk Estimation, ACHA, 2012 V., M. Golbabaee, M. J. Fadili et G. Peyré, Model Selection with Piecewise Regular Gauges, Tech. report, http://arxiv.org/abs/1307.2342, 2013 J. Fadili, V. and G. Peyré, Linear Convergence Rates for Gauge Regularization, ongoing work V., C. Deledalle, J. Fadili, G. Peyré and C. Dossal, The Degrees of Freedom of Block Analysis Regularizers, ongoing work