Pro Yearly is on sale from $80 to $50! »

Low Complexity Models: Robustness and Sensivity

4807c637e2e5e8a5c5e68b287e8492a9?s=47 Samuel Vaiter
February 03, 2014

Low Complexity Models: Robustness and Sensivity

GT Statistique et Imagerie, Paris-Dauphine, Paris, February 2014 (pre-defense).

4807c637e2e5e8a5c5e68b287e8492a9?s=128

Samuel Vaiter

February 03, 2014
Tweet

Transcript

  1. Low Complexity Models: Robustness and Sensivity Samuel Vaiter CEREMADE, Univ.

    Paris-Dauphine February 3th, 2014 Présoutenance de thèse J. Fadili G. Peyré C. Dossal M. Golbabaee C. Deledalle IMB GREYC CEREMADE
  2. Outline Introduction General Framework Performance Guarantees Parameter Selection

  3. Linear Inverse Problem y = Φ x0 + w Observations

    in Rq Unkown vector in Rn Degradation operator Noise denoising inpainting deblurring
  4. Variational Regularization x ∈ argmin x∈RN 1 2 ||Φx −

    y||2 + λ J(x) (Pλ(y)) Trade-off between data fidelity and prior regularization 2 issues considered in this thesis 1. Performance guarantees: 2 error + model selection 2. Parameter selection: sensivity analysis + risk estimation
  5. Outline Introduction General Framework Performance Guarantees Parameter Selection

  6. Gauge J(x) 0 J(|λ|x) = |λ|J(x) J convex x →

    J(x) 1 C C a convex set (0 ∈ C) C = {x : J(x) 1} homogeneous env.
  7. Signal Models and Gauges (group) sparsity || · ||1, (||

    · ||1,2) antisparsity || · ||∞ low-rank || · ||∗ sparse gradient ||∇ · ||1,2
  8. Canonical Model Space 0 x ∂J(x) Tx ex Model space

    Tx = VectHull(∂J(x))⊥ Generalized sign vector ex = PTx (∂J(x)) Sparsity || · ||1 Tx = {η : supp(η) ⊆ supp(x)} ex = sign(x) Trace Norm || · ||∗ Tx = {η : U∗ ⊥ ηV⊥ = 0} ex = UV ∗ SVD: x = UΛV ∗
  9. Outline Introduction General Framework Performance Guarantees Parameter Selection

  10. Certificate / Lagrange Multiplier x ∈ argmin Φx=Φx0 J(x) (P0(Φx0))

    ∂J(x) x Φx = Φx0 α Dual certificates: Dx0 = Im Φ∗ ∩ ∂J(x0) Proposition ∃α ∈ Dx0 ⇔ x0 solution de (P0(Φx0))
  11. Performance Guarantees with 2 norm Tight dual certificates ¯ Dx

    = Im Φ∗ ∩ ri ∂J(x) Restricted Injectivity Ker Φ ∩ Tx = {0} (RICx ) Theorem If ∃α ∈ ¯ Dx and (RICx ) satisfied and a solution x of (Pλ(y)), then λ ∼ ||w|| ⇒ ||x − x || Cα||w|| PW: [Grasmair et al. 2011] J(x − x ) = O(||w||)
  12. Performance Guarantees with Model Selection α ∈ Dx =⇒ α

    = Φ∗η and αT = ex Minimal-norm precertificates α0 ∈ argmin α=Φ∗η αTx =ex ||η|| Proposition If (RICx ), then α0=(Φ+ Tx Φ)∗ex Theorem If α0 ∈ ¯ Dx0 , for λ ∼ ||w|| small enough, the unique solution x of (Pλ(y)) satifies Tx = Tx0 and ||x0 − x || = O(||w||) PW: [Fuchs 2004] ( 1), [Bach 2008] ( 1 − 2, nuclear)
  13. 1D TV Denoising Φ = Id J(x) = ||∇x||1 α0

    ∈ ¯ Dx ⇐⇒ α0 = div q and ||qIc ,0||∞ < 1 i xi i xi k q0,k k +1 −1 Support stability No support stability Both are 2-stable
  14. Outline Introduction General Framework Performance Guarantees Parameter Selection

  15. Parameter Selection Y = Φx0 + W ∼ N(Φx0, σ2)

    Prediction: µ(y) = Φx∗(y) Prediction risk: R(λ) = EW [||Φx0 − µ(Y )||2] In practice, projected risk or estimation risk
  16. Stein Unbiased Risk Estimation First order approximation µ(y + δ)

    = µ(y) + Dµ(y) · δ + O(||δ||2) Stein Unbiased Risk Estimation SURE(y) = ||y − µ(y)||2 − σ2Q + 2σ2df (y) df (y) = tr[Dµ(y)] Proposition (Stein 1981) If µ weakly differentiable, then EW [SURE(Y )] = EW [||Φx0 − µ(Y )||2]
  17. Local Behavior x (y) ∈ argmin x∈Rn 1 2 ||y

    − Φx||2 + λJ(x) (Pλ(y)) We assume that T = {Tx : x ∈ Rn} is finite. Theorem Assuming that J is definable in an O-minimal structure O, y → µ(y) = Φx (y) is differentiable except on a zero measure set H and div(µ)(y) = −ΦT (Φ∗ T ΦT + De(x∗))−1Φ∗ T where T = Tx∗ . H is definable in O and can be explicitely stated. PW: [Dossal 2012] ( 1, df = ||x (y)||0 )
  18. Risk Estimation in Practice Φ subsampled Radon transform (16 measures),

    J(x) = ||∇x||1,2 x0 Φ+y x∗ λopt (y)
  19. Future Work • Extension to the infinite dimensional setting Grid-free

    setting Total Variation case • Efficient SURE computation Model SURE vs Algorithm SURE • Better understanding of the geometry Optimization over ¯ Dx Behavior of α ∈ Dx \ ¯ Dx • Performance in CS settings
  20. Thanks for your attention ! V., J. Fadili, G. Peyré

    and C. Dossal, Robust sparse analysis regularization, Information Theory, 2013 V., C. Deledalle, J. Fadili, G. Peyré and C. Dossal, Local Behavior of Sparse Analysis Regularization: Applications to Risk Estimation, ACHA, 2012 V., M. Golbabaee, M. J. Fadili et G. Peyré, Model Selection with Piecewise Regular Gauges, Tech. report, http://arxiv.org/abs/1307.2342, 2013 J. Fadili, V. and G. Peyré, Linear Convergence Rates for Gauge Regularization, ongoing work V., C. Deledalle, J. Fadili, G. Peyré and C. Dossal, The Degrees of Freedom of Block Analysis Regularizers, ongoing work