Low Complexity Models: Robustness

4807c637e2e5e8a5c5e68b287e8492a9?s=47 Samuel Vaiter
February 04, 2014

Low Complexity Models: Robustness

GDR ISIS, Télécom ParisTech, Paris, February 2014.

4807c637e2e5e8a5c5e68b287e8492a9?s=128

Samuel Vaiter

February 04, 2014
Tweet

Transcript

  1. Low Complexity Models: Robustness and Sensivity Samuel Vaiter CEREMADE, Univ.

    Paris-Dauphine 4 février 2014 GDR Isis : Apprentissage de représentations et traitement du signal J. Fadili G. Peyré C. Dossal M. Golbabaee C. Deledalle IMB GREYC CEREMADE
  2. Outline Introduction General Framework Performance Guarantees

  3. Linear Inverse Problem y = Φ x0 + w Observations

    in Rq Unkown vector in Rn Degradation operator Noise denoising inpainting deblurring
  4. Variational Regularization x ∈ argmin x∈RN 1 2 ||Φx −

    y||2 + λ J(x) (Pλ(y)) Trade-off between data fidelity and prior regularization Issue considered 1. Performance guarantees: 2 error + model selection
  5. Outline Introduction General Framework Performance Guarantees

  6. Gauge J(x) 0 J(|λ|x) = |λ|J(x) J convex x →

    J(x) 1 C C a convex set (0 ∈ C) C = {x : J(x) 1} homogeneous env.
  7. Signal Models and Gauges (group) sparsity || · ||1, (||

    · ||1,2) antisparsity || · ||∞ low-rank || · ||∗ sparse gradient ||∇ · ||1,2
  8. Canonical Model Space 0 x ∂J(x) Tx ex Model space

    Tx = VectHull(∂J(x))⊥ Generalized sign vector ex = PTx (∂J(x)) Sparsity || · ||1 Tx = {η : supp(η) ⊆ supp(x)} ex = sign(x) Trace Norm || · ||∗ Tx = {η : U∗ ⊥ ηV⊥ = 0} ex = UV ∗ SVD: x = UΛV ∗
  9. Outline Introduction General Framework Performance Guarantees

  10. Certificate / Lagrange Multiplier x ∈ argmin Φx=Φx0 J(x) (P0(Φx0))

    ∂J(x) x Φx = Φx0 α Dual certificates: Dx0 = Im Φ∗ ∩ ∂J(x0) Proposition ∃α ∈ Dx0 ⇔ x0 solution de (P0(Φx0))
  11. Performance Guarantees with 2 norm Tight dual certificates ¯ Dx

    = Im Φ∗ ∩ ri ∂J(x) Restricted Injectivity Ker Φ ∩ Tx = {0} (RICx ) Theorem If ∃α ∈ ¯ Dx and (RICx ) satisfied and a solution x of (Pλ(y)), then λ ∼ ||w|| ⇒ ||x − x || Cα||w|| PW: [Grasmair et al. 2011] J(x − x ) = O(||w||)
  12. Performance Guarantees with Model Selection α ∈ Dx =⇒ α

    = Φ∗η and αT = ex Minimal-norm precertificates α0 ∈ argmin α=Φ∗η αTx =ex ||η|| Proposition If (RICx ), then α0=(Φ+ Tx Φ)∗ex Theorem If α0 ∈ ¯ Dx0 , for λ ∼ ||w|| small enough, the unique solution x of (Pλ(y)) satifies Tx = Tx0 and ||x0 − x || = O(||w||) PW: [Fuchs 2004] ( 1), [Bach 2008] ( 1 − 2, nuclear)
  13. 1D TV Denoising Φ = Id J(x) = ||∇x||1 α0

    ∈ ¯ Dx ⇐⇒ α0 = div q and ||qIc ,0||∞ < 1 i xi i xi k q0,k k +1 −1 Support stability No support stability Both are 2-stable
  14. Future Work • Extension to the infinite dimensional setting Grid-free

    setting Total Variation case • Efficient SURE computation Model SURE vs Algorithm SURE • Better understanding of the geometry Optimization over ¯ Dx Behavior of α ∈ Dx \ ¯ Dx • Performance in CS settings
  15. Thanks for your attention ! V., J. Fadili, G. Peyré

    and C. Dossal, Robust sparse analysis regularization, Information Theory, 2013 V., C. Deledalle, J. Fadili, G. Peyré and C. Dossal, Local Behavior of Sparse Analysis Regularization: Applications to Risk Estimation, ACHA, 2012 V., M. Golbabaee, M. J. Fadili et G. Peyré, Model Selection with Piecewise Regular Gauges, Tech. report, http://arxiv.org/abs/1307.2342, 2013 J. Fadili, V. and G. Peyré, Linear Convergence Rates for Gauge Regularization, ongoing work V., C. Deledalle, J. Fadili, G. Peyré and C. Dossal, The Degrees of Freedom of Block Analysis Regularizers, ongoing work