with contributions from J.-F. Aujol, A. I. Aviles-Rivero, A. Chambolle, T. Feld, G. Gilboa and C.-B. Schönlieb ICIAM 2019 Nonlinear Spectral Decompositions with Applications in Imaging and Data Science July 18th 2019 N. Papadakis Eigenfunctions and calibrable sets of absolutely one-homogeneous functionals 0 / 18
Variation on Ω TV(u) = sup z∈C∞ c ; ||z||∞≤1 Ω udiv(z) Discrete setting in imaging TV(u) = i ||(∇u)i ||2 Denoising with TV: piecewise constant prior • Intensively used for image regularization [Rudin, Osher, Fatemi ‘92] N. Papadakis Eigenfunctions and calibrable sets of absolutely one-homogeneous functionals 1 / 18
problem w.r.t subgradient of TV λu ∈ ∂TV(u) • Atoms of TV regularization in inverse problems. Ex 1D: Local minima of TV(u) ||u||2 are eigenfunctions of TV [Andreu et al 2001] N. Papadakis Eigenfunctions and calibrable sets of absolutely one-homogeneous functionals 2 / 18
problem w.r.t subgradient of TV λu ∈ ∂TV(u) • Atoms of TV regularization in inverse problems. Ex 1D: Local minima of TV(u) ||u||2 are eigenfunctions of TV [Andreu et al 2001] Calibrable sets • C ⊂ Ω such that χC is an eigenfunction of TV • A calibrable set is convex with bounded curvature [Belettini et al. 2002]: ess sup q∈∂C κ(q) ≤ Per(C) |C| [Gilboa et al. 2016] N. Papadakis Eigenfunctions and calibrable sets of absolutely one-homogeneous functionals 2 / 18
problem w.r.t subgradient of TV λu ∈ ∂TV(u) • Atoms of TV regularization in inverse problems. Ex 1D: Local minima of TV(u) ||u||2 are eigenfunctions of TV [Andreu et al 2001] Calibrable sets • C ⊂ Ω such that χC is an eigenfunction of TV • A calibrable set is convex with bounded curvature [Belettini et al. 2002]: ess sup q∈∂C κ(q) ≤ Per(C) |C| [Gilboa et al. 2016] Cheeger set of Ω : min C⊂Ω TV(χC ) |C| Cheeger cut of Ω : min C⊂Ω TV(χC ) max(|C| |Ω\C|) N. Papadakis Eigenfunctions and calibrable sets of absolutely one-homogeneous functionals 2 / 18
problem w.r.t subgradient of TV λu ∈ ∂TV(u) • Atoms of TV regularization in inverse problems. Ex 1D: Local minima of TV(u) ||u||2 are eigenfunctions of TV [Andreu et al 2001] Calibrable sets • C ⊂ Ω such that χC is an eigenfunction of TV • A calibrable set is convex with bounded curvature [Belettini et al. 2002]: ess sup q∈∂C κ(q) ≤ Per(C) |C| [Gilboa et al. 2016] Cheeger set of Ω : min C⊂Ω TV(χC ) |C| Cheeger cut of Ω : min C⊂Ω TV(χC ) max(|C| |Ω\C|) N. Papadakis Eigenfunctions and calibrable sets of absolutely one-homogeneous functionals 2 / 18
• H(u) = ||u||2 : eigenfunctions of TV • H(u) = ||u||1 : calibrable sets of TV N. Papadakis Eigenfunctions and calibrable sets of absolutely one-homogeneous functionals 3 / 18
• H(u) = ||u||2 : eigenfunctions of TV • H(u) = ||u||1 : calibrable sets of TV Numerical approaches • Maximum Cheeger set with a projection algorithm [Carlier et al. 2008] N. Papadakis Eigenfunctions and calibrable sets of absolutely one-homogeneous functionals 3 / 18
• H(u) = ||u||2 : eigenfunctions of TV • H(u) = ||u||1 : calibrable sets of TV Numerical approaches • Maximum Cheeger set with a projection algorithm [Carlier et al. 2008] • PDE associated to the TV flow u(0) = u0 ∂t u = −p p ∈ ∂TV(u) - u0 is an eigenfunction: u(t) = (1 − tλ)+u0 N. Papadakis Eigenfunctions and calibrable sets of absolutely one-homogeneous functionals 3 / 18
• H(u) = ||u||2 : eigenfunctions of TV • H(u) = ||u||1 : calibrable sets of TV Numerical approaches • Maximum Cheeger set with a projection algorithm [Carlier et al. 2008] • PDE associated to the TV flow u(0) = u0 ∂t u = −p p ∈ ∂TV(u) - u0 is an eigenfunction: u(t) = (1 − tλ)+u0 - At instinction time, u(t) is an eigenfunction [Bungert et al 2019] N. Papadakis Eigenfunctions and calibrable sets of absolutely one-homogeneous functionals 3 / 18
• H(u) = ||u||2 : eigenfunctions of TV • H(u) = ||u||1 : calibrable sets of TV Numerical approaches • Maximum Cheeger set with a projection algorithm [Carlier et al. 2008] • PDE associated to the TV flow u(0) = u0 ∂t u = −p p ∈ ∂TV(u) - u0 is an eigenfunction: u(t) = (1 − tλ)+u0 - At instinction time, u(t) is an eigenfunction [Bungert et al 2019] - Implicit scheme for discrete TV flow: ut+1 − ut ∆t = −pt+1 ⇔ ut+1 = argmin u 1 2∆t ||u − ut ||2 + TV(u) N. Papadakis Eigenfunctions and calibrable sets of absolutely one-homogeneous functionals 3 / 18
J • Minimization of generalized non linear Rayleigh quotient min u R(u) := J(u) H(u) N. Papadakis Eigenfunctions and calibrable sets of absolutely one-homogeneous functionals 4 / 18
J • Minimization of generalized non linear Rayleigh quotient min u R(u) := J(u) H(u) • Finite dimension setting X ⊂ Rn • J : X → R of full domain, proper, convex, lower semi-continuous and absolutely one-homogeneous: J(αu) = |α|J(u) N. Papadakis Eigenfunctions and calibrable sets of absolutely one-homogeneous functionals 4 / 18
J • Minimization of generalized non linear Rayleigh quotient min u R(u) := J(u) H(u) • Finite dimension setting X ⊂ Rn • J : X → R of full domain, proper, convex, lower semi-continuous and absolutely one-homogeneous: J(αu) = |α|J(u) Discrete Non-Local Total Variation [Gilboa and Osher 2008] J(u) = ij wij |ui − uj | wij≥0 ⇒ Regularization in imaging / Clustering with graph laplacian N. Papadakis Eigenfunctions and calibrable sets of absolutely one-homogeneous functionals 4 / 18
Computation of eigenfunctions and calibrable sets 2 Discretization of the flow 3 Illustration and application to graph clustering N. Papadakis Eigenfunctions and calibrable sets of absolutely one-homogeneous functionals 5 / 18
J(u) H(u) • “Derivative” of R, for p ∈ ∂J(u) and q ∈ ∂H(u): H(u)p − J(u)q H2(u) = 1 H(u) (p − R(u)q) • New PDE [Nossek and Gilboa ‘18, Aujol et al. ‘18, Feld et al. ‘19] ∂t u = R(u)q − p • When R(u)q is locally Lipschitz: results from [Brezis ‘73]: ∂t u = A(u) − p N. Papadakis Eigenfunctions and calibrable sets of absolutely one-homogeneous functionals 6 / 18
H(u) q − p • J(u) is the (non local) total variation • H(u) = ||u||p , p ≥ 1 is the p norm N. Papadakis Eigenfunctions and calibrable sets of absolutely one-homogeneous functionals 7 / 18
H(u) q − p • J(u) is the (non local) total variation • H(u) = ||u||p , p ≥ 1 is the p norm Bounded subgradients in finite dimension [Burger et al. ‘16]: ∃CH < ∞ such that q ≤ CH, ∀q ∈ ∂H N. Papadakis Eigenfunctions and calibrable sets of absolutely one-homogeneous functionals 7 / 18
H(u) q − p • J(u) is the (non local) total variation • H(u) = ||u||p , p ≥ 1 is the p norm Bounded subgradients in finite dimension [Burger et al. ‘16]: ∃CH < ∞ such that q ≤ CH, ∀q ∈ ∂H Full time discrete analysis • Existence and uniqueness in time continuous setting [Brezis ‘73] for H(u) = ||u||p with p ≥ 2 • Moreau-Yosida regularization to have a locally-Lipschitz mapping when p ∈ [1; 2) N. Papadakis Eigenfunctions and calibrable sets of absolutely one-homogeneous functionals 7 / 18
q − p (1) J, H: Absolutely one homogeneous functional p ∈ ∂J(u) ⇒ J(u) = p, u N. Papadakis Eigenfunctions and calibrable sets of absolutely one-homogeneous functionals 8 / 18
q − p (1) J, H: Absolutely one homogeneous functional p ∈ ∂J(u) ⇒ J(u) = p, u • Norm conservation d dt 1 2 ||u||2 = u, ut = J(u) H(u) u, q − u, p = 0 • u does not vanish N. Papadakis Eigenfunctions and calibrable sets of absolutely one-homogeneous functionals 8 / 18
q − p (1) J, H: Absolutely one homogeneous functional p ∈ ∂J(u) ⇒ J(u) = p, u • Norm conservation d dt 1 2 ||u||2 = u, ut = J(u) H(u) u, q − u, p = 0 • u does not vanish H: 2 norm q = u ||u||2 ∈ ∂||u||2 ⇒ ∂t u = J(u) ||u||2 2 u − p N. Papadakis Eigenfunctions and calibrable sets of absolutely one-homogeneous functionals 8 / 18
q − p (1) J, H: Absolutely one homogeneous functional p ∈ ∂J(u) ⇒ J(u) = p, u • Norm conservation d dt 1 2 ||u||2 = u, ut = J(u) H(u) u, q − u, p = 0 • u does not vanish H: 2 norm q = u ||u||2 ∈ ∂||u||2 ⇒ ∂t u = J(u) ||u||2 2 u − p • Convergence of the PDE ⇔ eigenfunction of J ∂t u = 0 ⇔ p = J(u) ||u||2 2 u ∈ ∂J(u) N. Papadakis Eigenfunctions and calibrable sets of absolutely one-homogeneous functionals 8 / 18
q − p (1) J: (non-local) Total Variation J(u) = i ||(∇u)i ||2 J(u) = ij wij |ui − uj |, wij ≥ 0 Any p ∈ ∂J(u) checks 1 - Zero mean ¯ p = p, 1 = 0 N. Papadakis Eigenfunctions and calibrable sets of absolutely one-homogeneous functionals 8 / 18
(q − ¯ q1) − p (1) J: (non-local) Total Variation J(u) = i ||(∇u)i ||2 J(u) = ij wij |ui − uj |, wij ≥ 0 Any p ∈ ∂J(u) checks 1 - Zero mean ¯ p = p, 1 = 0 • Adaptation of the flow: ¯ u0 = 0 ⇒ u(t) = 0 N. Papadakis Eigenfunctions and calibrable sets of absolutely one-homogeneous functionals 8 / 18
(q − ¯ q1) − p (1) J: (non-local) Total Variation J(u) = i ||(∇u)i ||2 J(u) = ij wij |ui − uj |, wij ≥ 0 Any p ∈ ∂J(u) checks 1 - Zero mean ¯ p = p, 1 = 0 • Adaptation of the flow: ¯ u0 = 0 ⇒ u(t) = 0 • Constant norm + zero mean ⇒ steady points u∗ are non trivial N. Papadakis Eigenfunctions and calibrable sets of absolutely one-homogeneous functionals 8 / 18
(q − ¯ q1) − p (1) J: (non-local) Total Variation J(u) = i ||(∇u)i ||2 J(u) = ij wij |ui − uj |, wij ≥ 0 Any p ∈ ∂J(u) checks 1 - Zero mean ¯ p = p, 1 = 0 • Adaptation of the flow: ¯ u0 = 0 ⇒ u(t) = 0 • Constant norm + zero mean ⇒ steady points u∗ are non trivial 2 - Co-area p ∈ ∂J(1u> ), ∀ ∈ [umin, umax ] N. Papadakis Eigenfunctions and calibrable sets of absolutely one-homogeneous functionals 8 / 18
(q − ¯ q1) − p (1) J: (non-local) Total Variation J(u) = i ||(∇u)i ||2 J(u) = ij wij |ui − uj |, wij ≥ 0 Any p ∈ ∂J(u) checks 1 - Zero mean ¯ p = p, 1 = 0 • Adaptation of the flow: ¯ u0 = 0 ⇒ u(t) = 0 • Constant norm + zero mean ⇒ steady points u∗ are non trivial 2 - Co-area p ∈ ∂J(1u> ), ∀ ∈ [umin, umax ] • For H(u) = ||u||1 , take q = (1u>0 − 1u≤0) ∈ ∂||u||1 • If u∗ a steady point of (1) then ⇒ v = 1u∗>0 − |1u∗>0| is a calibrable set N. Papadakis Eigenfunctions and calibrable sets of absolutely one-homogeneous functionals 8 / 18
(q − ¯ q1) − p (1) J: (non-local) Total Variation J(u) = i ||(∇u)i ||2 J(u) = ij wij |ui − uj |, wij ≥ 0 Any p ∈ ∂J(u) checks 1 - Zero mean ¯ p = p, 1 = 0 • Adaptation of the flow: ¯ u0 = 0 ⇒ u(t) = 0 • Constant norm + zero mean ⇒ steady points u∗ are non trivial 2 - Co-area p ∈ ∂J(1u> ), ∀ ∈ [umin, umax ] • For H(u) = ||u||1 , take q = (1u>0 − 1u≤0) ∈ ∂||u||1 • If u∗ a steady point of (1) then ⇒ v = 1u∗>0 − |1u∗>0| is a calibrable set N. Papadakis Eigenfunctions and calibrable sets of absolutely one-homogeneous functionals 8 / 18
(ui+1,j − ui,j )2 + (ui,j+1 − ui,j )2 • H(u) = ij u2 i,j u p N. Papadakis Eigenfunctions and calibrable sets of absolutely one-homogeneous functionals 9 / 18
(ui+1,j − ui,j )2 + (ui,j+1 − ui,j )2 • H(u) = ij u2 i,j u p • H(u) = ij |ui,j | u p v = 1u>0 N. Papadakis Eigenfunctions and calibrable sets of absolutely one-homogeneous functionals 9 / 18
J(u) = ij wij |ui − uj | H(u) = ij |ui,j | with wij = exp(−||fi − fj ||2) u p v = 1u>0 N. Papadakis Eigenfunctions and calibrable sets of absolutely one-homogeneous functionals 10 / 18
)2 + (ui,j+1 − ui,j )2 H(u) = ij u2 i,j 100 iterations 10000 iterations u p Pointwise ratio p/u J(uk ) ||uk+1 − uk ||2 N. Papadakis Eigenfunctions and calibrable sets of absolutely one-homogeneous functionals 13 / 18
• Linear coupling L l=1 ul (x) = 0, for all node x: - One-homogeneous constraint - Compatible with zero mean property of eigenfunction - Fast projection w.r.t simplex [Bresson et al. ‘13, Hein et al. ‘14] N. Papadakis Eigenfunctions and calibrable sets of absolutely one-homogeneous functionals 16 / 18
• Linear coupling L l=1 ul (x) = 0, for all node x: - One-homogeneous constraint - Compatible with zero mean property of eigenfunction - Fast projection w.r.t simplex [Bresson et al. ‘13, Hein et al. ‘14] Transductive learning • Label a few nodes N. Papadakis Eigenfunctions and calibrable sets of absolutely one-homogeneous functionals 16 / 18
• Linear coupling L l=1 ul (x) = 0, for all node x: - One-homogeneous constraint - Compatible with zero mean property of eigenfunction - Fast projection w.r.t simplex [Bresson et al. ‘13, Hein et al. ‘14] Transductive learning • Label a few nodes and diffuse the labels N. Papadakis Eigenfunctions and calibrable sets of absolutely one-homogeneous functionals 16 / 18
Numerical estimation of eigenfunctions • Data clustering Perspectives • Existence in the infinite dimension setting [Bungert et al. 2019] • Convergence of the discrete scheme for multi-class clustering N. Papadakis Eigenfunctions and calibrable sets of absolutely one-homogeneous functionals 18 / 18