Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Regularizations of Inverse Problems
Search
Samuel Vaiter
September 06, 2013
Science
0
42
Regularizations of Inverse Problems
GRETSI'13, Quartz, Brest, September 2013.
Samuel Vaiter
September 06, 2013
Tweet
Share
More Decks by Samuel Vaiter
See All by Samuel Vaiter
Low Complexity Regularizations: A ''Localization'' Result
svaiter
0
59
A First Look at Proximal Methods
svaiter
0
200
Fast Distributed Total Variation
svaiter
0
110
Low Complexity Regularizations: A Localization Result
svaiter
0
66
Low Complexity Regularizations: a Localization Result
svaiter
0
110
Model Selection with Partly Smooth Functions
svaiter
0
35
Low Complexity Models: Robustness
svaiter
0
58
Low Complexity Models: Robustness and Sensivity
svaiter
0
35
Recovery Guarantees for Low Complexity Models
svaiter
1
75
Other Decks in Science
See All in Science
データベース11: 正規化(1/2) - 望ましくない関係スキーマ
trycycle
PRO
0
640
高校生就活へのDA導入の提案
shunyanoda
0
260
局所保存性・相似変換対称性を満たす機械学習モデルによる数値流体力学
yellowshippo
1
280
アナログ計算機『計算尺』を愛でる Midosuji Tech #4/Analog Computing Device Slide Rule now and then
quiver
1
180
baseballrによるMLBデータの抽出と階層ベイズモデルによる打率の推定 / TokyoR118
dropout009
1
440
Ignite の1年間の軌跡
ktombow
0
130
Symfony Console Facelift
chalasr
2
450
01_篠原弘道_SIPガバニングボード座長_ポスコロSIPへの期待.pdf
sip3ristex
0
530
02_西村訓弘_プログラムディレクター_人口減少を機にひらく未来社会.pdf
sip3ristex
0
480
KH Coderチュートリアル(スライド版)
koichih
1
40k
Collective Predictive Coding Hypothesis and Beyond (@Japanese Association for Philosophy of Science, 26th October 2024)
tanichu
0
140
科学で迫る勝敗の法則(名城大学公開講座.2024年10月) / The principle of victory discovered by science (Open lecture in Meijo Univ. 2024)
konakalab
0
350
Featured
See All Featured
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
29
9.5k
Building an army of robots
kneath
306
45k
Optimising Largest Contentful Paint
csswizardry
37
3.3k
Facilitating Awesome Meetings
lara
54
6.4k
Adopting Sorbet at Scale
ufuk
77
9.4k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
161
15k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
15
1.5k
Practical Orchestrator
shlominoach
188
11k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
20
1.3k
The Invisible Side of Design
smashingmag
299
51k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
PRO
181
53k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
45
7.4k
Transcript
Régularisation de problèmes inverses Analyse unifiée de la robustesse Samuel
VAITER CNRS, CEREMADE, Université Paris-Dauphine, France Travaux en collaboration avec M. GOLBABAEE, G. PEYRÉ et J. FADILI
Linear Inverse Problems inpainting denoising super-resolution Forward model y =
x0 + w observations noise input operator
The Variational Approach x argmin x RN 1 2 ||y
x||2 2 + J(x) Data fidelity Regularity
The Variational Approach x argmin x RN 1 2 ||y
x||2 2 + J(x) Data fidelity Regularity J(x) = || x||2 J(x) = || x||1
The Variational Approach x argmin x RN 1 2 ||y
x||2 2 + J(x) Data fidelity Regularity J(x) = || x||2 J(x) = || x||1 sparsity analysis-sparsity group-sparsity nuclear norm Tikhonov Total Variation Anti-sparse Polyhedral L1 + TV ... atomic norm decomposable norm Candidate J
Objectives Model selection performance x0 x w Prior model J
Objectives Model selection performance x0 x w Prior model J
Objectives Model selection performance x0 x w Prior model J
How close ? in term of SNR in term of features
Union of Linear Models Union of models: T T linear
spaces
Union of Linear Models Union of models: T T linear
spaces T sparsity
Union of Linear Models Union of models: T T linear
spaces block sparsity T sparsity
Union of Linear Models Union of models: T T linear
spaces block sparsity T sparsity analysis sparsity
Union of Linear Models Union of models: T T linear
spaces block sparsity low rank T sparsity analysis sparsity
Union of Linear Models Union of models: T T linear
spaces block sparsity low rank Objective Encode T in a function T sparsity analysis sparsity
Gauges 1 J(x) J : RN R+ convex J( x)
= J(x), 0
Gauges 1 J(x) J : RN R+ convex C C
= {x : J(x) 1} J( x) = J(x), 0
Gauges 1 J(x) J : RN R+ convex C C
= {x : J(x) 1} Geometry of C Union of Models (T )T T x T x x 0 T0 x x 0 T0 x 0 x T T0 ||x||1 |x1|+||x2,3|| ||x|| ||x|| J( x) = J(x), 0
Subdifferential |x| 0
Subdifferential |x| 0
J(x) = RN : x , J(x ) J(x)+ ,
x x Subdifferential |x| 0
J(x) = RN : x , J(x ) J(x)+ ,
x x Subdifferential |x| 0 |·|(0) = [ 1,1] x = 0, |·|(x) = {sign(x)}
From the Subdifferential to the Model J(x) x 0 J(x)
x 0
From the Subdifferential to the Model J(x) x 0 J(x)
x 0 Tx= VectHull( J(x)) Tx Tx Tx = : supp( ) supp(x)
From the Subdifferential to the Model J(x) x 0 J(x)
x 0 ex = ProjTx ( J(x)) ex ex ex = sign(x) Tx= VectHull( J(x)) Tx Tx Tx = : supp( ) supp(x)
Regularizations and their Models J(x) = ||x||1 ex = sign(x)
Tx = : supp( ) supp(x) x x J(x) = b ||xb|| ex = (N (xb))b B Tx = : supp( ) supp(x) x N (xb) = xb/||xb|| J(x) = ||x||∗ ex =UV Tx = : U V = 0 x x =UΛV ∗ J(x) = ||x||∞ ex = |I| 1 sign(x) Tx = : I sign(xI ) x x I = {i : |xi | = ||x||∞}
Dual Certificates and Model Selection x argmin x RN 1
2 ||y x||2 2 + J(x) Hypothesis: Ker Tx0 = {0} J regular enough
Dual Certificates and Model Selection x argmin x RN 1
2 ||y x||2 2 + J(x) Hypothesis: Ker Tx0 = {0} J regular enough ¯ D = Im ri( J(x0)) Tight dual certificates: x = x0 J(x) x
Dual Certificates and Model Selection x argmin x RN 1
2 ||y x||2 2 + J(x) Hypothesis: 0 = ( + Tx0 ) ex0 Minimal norm pre-certificate: Tx = Tx0 and ||x x0|| = O(||w||) If 0 ¯ D,||w|| small enough and ||w||, then x is the unique solution. Moreover, [V. et al. 2013] 1: [Fuchs 2004] 1 2: [Bach 2008] Ker Tx0 = {0} J regular enough ¯ D = Im ri( J(x0)) Tight dual certificates: x = x0 J(x) x
Example: Sparse Deconvolution x = i xi (· i) J(x)
= ||x||1 Increasing : reduces correlation. reduces resolution. x0 x0
Example: Sparse Deconvolution x = i xi (· i) J(x)
= ||x||1 Increasing : reduces correlation. reduces resolution. x0 x0 I = j : x0[j] = 0 || 0,Ic || < 1 0 ¯ D support recovery || 0,Ic || 1 2 20
Example: 1D TV Denoising J(x) = || x||1 = Id
I = {i : ( x0)i = 0} x0
Example: 1D TV Denoising J(x) = || x||1 = Id
I = {i : ( x0)i = 0} x0 +1 1 0 = div( 0) where j I,( 0)j = 0 x0 I J || 0,Ic || < 1 Support stability
Example: 1D TV Denoising J(x) = || x||1 = Id
I = {i : ( x0)i = 0} x0 +1 1 0 = div( 0) where j I,( 0)j = 0 x0 I J || 0,Ic || < 1 Support stability x0 || 0,Ic || = 1 2-stability only
Conclusion Gauges: encode linear models as singular points
Conclusion Gauges: encode linear models as singular points Certificates: guarantees
of model selection / 2 robustness (see poster 208 for a pure robustness result)
Conclusion Merci de votre attention ! Gauges: encode linear models
as singular points Certificates: guarantees of model selection / 2 robustness (see poster 208 for a pure robustness result)