Finite identification and local linear convergence of proximal

by J. Fadili

Transcript

1. Finite Identification and Local Linear Convergence of Proximal Splitting Algorithms Jalal Fadili Normandie Université-ENSICAEN, GREYC CNRS UMR 6072 Joint work with Jingwei Liang, Gabriel Peyré and Russell Luke Journées du GDR MOA 2015
2. MOA’15- Class of problems: motivations 2 x y H n × 1 L × 1 m × 1 m × n n × L Dictionary Sensing } A " Measurement/degradation x y H n × 1 m × 1 m × n n × L Dictionary Sensing } x y H n × 1 m × 1 m × n n × L Dictionary Sensing } Inverse problem Prior knowledge (regularization, constraints) y 2 Rm x0 2 Rn Forward model
3. MOA’15- Class of problems: motivations 2 x y H n × 1 L × 1 m × 1 m × n n × L Dictionary Sensing } A " Measurement/degradation x y H n × 1 m × 1 m × n n × L Dictionary Sensing } x y H n × 1 m × 1 m × n n × L Dictionary Sensing } Inverse problem Prior knowledge (regularization, constraints) x0 typically lives in a low-dimensional manifold y 2 Rm x0 2 Rn Forward model
4. MOA’15- Class of problems: motivations 3 x y H n × 1 L × 1 m × 1 m × n n × L Dictionary Sensing } A " Measurement/degradation x y H n × 1 m × 1 m × n n × L Dictionary Sensing } x y H n × 1 m × 1 m × n n × L Dictionary Sensing } Inverse problem Forward model Prior knowledge (regularization, constraints) Many applications in data sciences: signal/image processing, machine learning, statistics, etc.. x0 typically lives in a low-dimensional manifold y 2 Rm x0 2 Rn Solve an inverse problem through regularization : min x 2Rn F ( x ) | {z } Data fidelity + G ( x ) | {z } Regularization, constraints G promotes objects living in the same manifold as x0. F and G 2 0 (Rn)
5. MOA’15- Low-complexity regularization min x 2Rn F ( x ) + G ( x ) Low-complexity () Low-dimensional manifold F and G 2 0 (Rn)
6. MOA’15- Low-complexity regularization min x 2Rn F ( x ) + G ( x ) Low-complexity () Low-dimensional manifold Sparse vectors R2 R3 F and G 2 0 (Rn)
7. MOA’15- Low-complexity regularization min x 2Rn F ( x ) + G ( x ) Low-complexity () Low-dimensional manifold Sparse vectors G ( x ) = k x k 1 (tightest convex relaxation of `0) R2 R3 F and G 2 0 (Rn)
8. MOA’15- Low-complexity regularization min x 2Rn F ( x ) + G ( x ) Low-complexity () Low-dimensional manifold Sparse vectors G ( x ) = k x k 1 (tightest convex relaxation of `0) Low-rank matrices R2 R3 Sym 2 (R) F and G 2 0 (Rn)
9. MOA’15- Low-complexity regularization min x 2Rn F ( x ) + G ( x ) Low-complexity () Low-dimensional manifold Sparse vectors G ( x ) = k x k 1 (tightest convex relaxation of `0) Low-rank matrices G ( x ) = k x k ⇤ (tightest convex relaxation of rank) R2 R3 Sym 2 (R) F and G 2 0 (Rn)
10. MOA’15- Proximal splitting and local linear convergence 5 45 40 35 30
    
    25 20 15 10 -8 -6 -4 -2 0
     2 4 6 10 12 14 16 18 8 6 4 2 0 8 
      5 10 15 20 25 30 35
    
    
    
    
    10-14 10-12 10-10 10-8 10-6 10-4 10-2 100 102 F ( x ) + G ( x ) (Inertial) Forward-Backward LASSO min x 2Rn 1 2 k y A x k2 2 + k x k 1