Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Regularizations of Inverse Problems
Search
Samuel Vaiter
September 06, 2013
Science
0
43
Regularizations of Inverse Problems
GRETSI'13, Quartz, Brest, September 2013.
Samuel Vaiter
September 06, 2013
Tweet
Share
More Decks by Samuel Vaiter
See All by Samuel Vaiter
Low Complexity Regularizations: A ''Localization'' Result
svaiter
0
65
A First Look at Proximal Methods
svaiter
0
210
Fast Distributed Total Variation
svaiter
0
110
Low Complexity Regularizations: A Localization Result
svaiter
0
67
Low Complexity Regularizations: a Localization Result
svaiter
0
120
Model Selection with Partly Smooth Functions
svaiter
0
41
Low Complexity Models: Robustness
svaiter
0
64
Low Complexity Models: Robustness and Sensivity
svaiter
0
36
Recovery Guarantees for Low Complexity Models
svaiter
1
76
Other Decks in Science
See All in Science
MCMCのR-hatは分散分析である
moricup
0
460
統計的因果探索: 背景知識とデータにより因果仮説を探索する
sshimizu2006
4
1k
データベース11: 正規化(1/2) - 望ましくない関係スキーマ
trycycle
PRO
0
950
AIに仕事を奪われる 最初の医師たちへ
ikora128
0
990
知能とはなにかーヒトとAIのあいだー
tagtag
0
140
Transport information Geometry: Current and Future II
lwc2017
0
210
動的トリートメント・レジームを推定するDynTxRegimeパッケージ
saltcooky12
0
200
風の力で振れ幅が大きくなる振り子!? 〜タコマナローズ橋はなぜ落ちたのか〜
syotasasaki593876
1
100
データベース08: 実体関連モデルとは?
trycycle
PRO
0
950
凸最適化からDC最適化まで
santana_hammer
1
310
My Little Monster
juzishuu
0
130
データベース12: 正規化(2/2) - データ従属性に基づく正規化
trycycle
PRO
0
980
Featured
See All Featured
Leading Effective Engineering Teams in the AI Era
addyosmani
6
440
Visualization
eitanlees
149
16k
A designer walks into a library…
pauljervisheath
209
24k
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
30
2.9k
Docker and Python
trallard
46
3.6k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
140
34k
Speed Design
sergeychernyshev
32
1.2k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
667
120k
Learning to Love Humans: Emotional Interface Design
aarron
274
41k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
367
27k
GitHub's CSS Performance
jonrohan
1032
470k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
31
9.7k
Transcript
Régularisation de problèmes inverses Analyse unifiée de la robustesse Samuel
VAITER CNRS, CEREMADE, Université Paris-Dauphine, France Travaux en collaboration avec M. GOLBABAEE, G. PEYRÉ et J. FADILI
Linear Inverse Problems inpainting denoising super-resolution Forward model y =
x0 + w observations noise input operator
The Variational Approach x argmin x RN 1 2 ||y
x||2 2 + J(x) Data fidelity Regularity
The Variational Approach x argmin x RN 1 2 ||y
x||2 2 + J(x) Data fidelity Regularity J(x) = || x||2 J(x) = || x||1
The Variational Approach x argmin x RN 1 2 ||y
x||2 2 + J(x) Data fidelity Regularity J(x) = || x||2 J(x) = || x||1 sparsity analysis-sparsity group-sparsity nuclear norm Tikhonov Total Variation Anti-sparse Polyhedral L1 + TV ... atomic norm decomposable norm Candidate J
Objectives Model selection performance x0 x w Prior model J
Objectives Model selection performance x0 x w Prior model J
Objectives Model selection performance x0 x w Prior model J
How close ? in term of SNR in term of features
Union of Linear Models Union of models: T T linear
spaces
Union of Linear Models Union of models: T T linear
spaces T sparsity
Union of Linear Models Union of models: T T linear
spaces block sparsity T sparsity
Union of Linear Models Union of models: T T linear
spaces block sparsity T sparsity analysis sparsity
Union of Linear Models Union of models: T T linear
spaces block sparsity low rank T sparsity analysis sparsity
Union of Linear Models Union of models: T T linear
spaces block sparsity low rank Objective Encode T in a function T sparsity analysis sparsity
Gauges 1 J(x) J : RN R+ convex J( x)
= J(x), 0
Gauges 1 J(x) J : RN R+ convex C C
= {x : J(x) 1} J( x) = J(x), 0
Gauges 1 J(x) J : RN R+ convex C C
= {x : J(x) 1} Geometry of C Union of Models (T )T T x T x x 0 T0 x x 0 T0 x 0 x T T0 ||x||1 |x1|+||x2,3|| ||x|| ||x|| J( x) = J(x), 0
Subdifferential |x| 0
Subdifferential |x| 0
J(x) = RN : x , J(x ) J(x)+ ,
x x Subdifferential |x| 0
J(x) = RN : x , J(x ) J(x)+ ,
x x Subdifferential |x| 0 |·|(0) = [ 1,1] x = 0, |·|(x) = {sign(x)}
From the Subdifferential to the Model J(x) x 0 J(x)
x 0
From the Subdifferential to the Model J(x) x 0 J(x)
x 0 Tx= VectHull( J(x)) Tx Tx Tx = : supp( ) supp(x)
From the Subdifferential to the Model J(x) x 0 J(x)
x 0 ex = ProjTx ( J(x)) ex ex ex = sign(x) Tx= VectHull( J(x)) Tx Tx Tx = : supp( ) supp(x)
Regularizations and their Models J(x) = ||x||1 ex = sign(x)
Tx = : supp( ) supp(x) x x J(x) = b ||xb|| ex = (N (xb))b B Tx = : supp( ) supp(x) x N (xb) = xb/||xb|| J(x) = ||x||∗ ex =UV Tx = : U V = 0 x x =UΛV ∗ J(x) = ||x||∞ ex = |I| 1 sign(x) Tx = : I sign(xI ) x x I = {i : |xi | = ||x||∞}
Dual Certificates and Model Selection x argmin x RN 1
2 ||y x||2 2 + J(x) Hypothesis: Ker Tx0 = {0} J regular enough
Dual Certificates and Model Selection x argmin x RN 1
2 ||y x||2 2 + J(x) Hypothesis: Ker Tx0 = {0} J regular enough ¯ D = Im ri( J(x0)) Tight dual certificates: x = x0 J(x) x
Dual Certificates and Model Selection x argmin x RN 1
2 ||y x||2 2 + J(x) Hypothesis: 0 = ( + Tx0 ) ex0 Minimal norm pre-certificate: Tx = Tx0 and ||x x0|| = O(||w||) If 0 ¯ D,||w|| small enough and ||w||, then x is the unique solution. Moreover, [V. et al. 2013] 1: [Fuchs 2004] 1 2: [Bach 2008] Ker Tx0 = {0} J regular enough ¯ D = Im ri( J(x0)) Tight dual certificates: x = x0 J(x) x
Example: Sparse Deconvolution x = i xi (· i) J(x)
= ||x||1 Increasing : reduces correlation. reduces resolution. x0 x0
Example: Sparse Deconvolution x = i xi (· i) J(x)
= ||x||1 Increasing : reduces correlation. reduces resolution. x0 x0 I = j : x0[j] = 0 || 0,Ic || < 1 0 ¯ D support recovery || 0,Ic || 1 2 20
Example: 1D TV Denoising J(x) = || x||1 = Id
I = {i : ( x0)i = 0} x0
Example: 1D TV Denoising J(x) = || x||1 = Id
I = {i : ( x0)i = 0} x0 +1 1 0 = div( 0) where j I,( 0)j = 0 x0 I J || 0,Ic || < 1 Support stability
Example: 1D TV Denoising J(x) = || x||1 = Id
I = {i : ( x0)i = 0} x0 +1 1 0 = div( 0) where j I,( 0)j = 0 x0 I J || 0,Ic || < 1 Support stability x0 || 0,Ic || = 1 2-stability only
Conclusion Gauges: encode linear models as singular points
Conclusion Gauges: encode linear models as singular points Certificates: guarantees
of model selection / 2 robustness (see poster 208 for a pure robustness result)
Conclusion Merci de votre attention ! Gauges: encode linear models
as singular points Certificates: guarantees of model selection / 2 robustness (see poster 208 for a pure robustness result)