Upgrade to Pro — share decks privately, control downloads, hide ads and more …

ICIP 2011

ICIP 2011

1509d0ae6901a6cf2c2fd7cc97f02fc0?s=128

Olivier Lézoray

September 21, 2011
Tweet

Transcript

  1. M. Ghoniem, A. Elmoataz, O. Lézoray Université de Caen Basse-Normandie,

    GREYC, CNRS UMR 6072 Olivier.lezoray@unicaen.fr Discrete Infinity Harmonic Functions: Towards a Unified Interpolation Framework on Graphs
  2. Outline •  Introduction/Motivations •  Partial difference operators on graphs • 

    Non Local Infinity Laplacian on Graphs •  Infinity harmonic interpolation on Graphs •  Experimentations •  Conclusion
  3. Introduction/Motivations •  Large data sets are now collected in the

    form of: •  Graphs or networks •  Data in High dimensional Spaces
  4. •  Analysis of these types of data is a major

    challenge for image and machine learning communities •  Need to transfer tools for signal and image processing to Graphs and networks •  Examples of such effort includes: Diffusion wavelets ( Coifman-Magionni2006 ) Spectral graph wavelet ( Hammond-vandergheynst 2009) Partial difference Equations on Graphs (Elmoataz-Bougleux-Lézoray 2008, Ta-Elmoataz-Lezoray 2011) •  Our goal is to provide methods that mimic on graphs well-known PDE variational formulations under a functional analysis point of view. •  To do this we use Partial difference Equations (PdE) over graphs. •  PdEs mimic PDEs in domains having a graph Exploit this framework for interpolation in image and data processing Introduction/Motivations
  5. Introduction/Motivations •  Many problems on image and Data Processing can

    be formulated as interpolation problems •  Inpainting •  Colorization •  Semi-supervised image and High dimensional data clustering
  6. inpainting as representative example of interpolation •  Inpainting = reconstruction

    of a missing/damaged/undesirable Domain in a 2D/3D image or a sequence •  Usual assumption: the inpainting domain is known •  Many methods were developed for interpolation Geometry : local PDEs and variational methods Texture: exemplar based methods Both geometry and texture: non local PDEs and variational methods
  7. The associated formal Euler-Lagrange equation : p-Laplacian equation 0 D

    D min subject to . p f f dx f f ∂ ∫ ∇ = f 0 :! " R 2 # R p → ∞ 2 0 ( ) 0 p p f f on D f f f on D − ⎡ ⎤ ∇ ⎢ ⎥ Δ = ∇ ⋅ = ⎢ ⎥ ∇ ⎣ ⎦ = ∂ 0 \ | given D f Ω missing on D Ω p=2 harmonic inpainting Laplace Equation p=1 total variation inpainting 1-laplacian equation p-harmonic inpainting Infinity Laplacian equation Function f is said to be p-harmonic
  8. If f minimizes D min p f f dx ∫

    ∇ 2 , i ij i i j f f f f x x x ∂ ∂ = = ∂ ∂ ∂ p → ∞ , ( ) 0 ij i j i j f f f f on D ∞ ∑ Δ = = Infinity harmonic Inpainting Discretization : finite difference method, Oberman discretization Then as f satifies The Infinity laplacian equation has several applications Image processing, computer vision, optimal transportation Related to tug-of-War game ( Aronson, Grandall, Casselles, Peres, etc) 0 \ | given D f Ω missing on D Ω
  9. Some limitations of local infinity harmonic inpainting: •  Discontinuities are

    not well recovered •  Texture and repetitive structures are not recovered •  Difficulties to discretize the infinity Laplacian equation on graphs with arbitrary topology Infinity harmonic Inpainting
  10. Our motivations and objectives We would like to extend the

    PDE infinity Laplacian equation (infinity harmonic functions) on weighted graphs with the following objectives: •  Unify local and nonlocal interpolation problems on images •  Interpolation on graphs with an arbitrary topology How ? With the framework of Partial difference Equations (PdEs) on Graphs
  11. ( , , ) G V E w = {

    } 1 2 , ,.., , n n i V u u u u R = ∈ E = uv !V "V / u #v and u,v $V { } ( ) deg ( ) V uv v N u u w ∈ ∑ = : , 0 uv vu uv w E R w w w + → = >= Partial difference operators on graphs
  12. f :V ! R, or f :V ! R m

    ( ) , ( ) ( ) H V u V f g f u g u ∈ ∑ = ( ( ))T V u V f f u R ∈ = ∈ ( ) u V G f f u ∈ ∑ ∫ = ( ) H V Images and data are functions on graphs ( ) , ( ) ( ) H E uv E F G F uv G uv ∈ ∑ = ( ) H E Hilbert space on vertices Hilbert space on edges Partial difference operators on graphs The analog of the integral of a function is considered as
  13. * : ( ) ( ) w d f H

    E H V → ( ) ( ( ) ( )) def w uv d f uv w f v f u = − : ( ) ( ) w d f H v H E → Difference operator : Adjoint and Divergence operators on graphs : Each function F of H(E) has the following property * ( )( ) ( )( ) ( ( ) ( ) w w uv v V d H u div H u w H vu H uv ∈ ∑ = − = − ( )( ) 0 w u V div F u ∈ ∑ = df ,G H (E ) = def f ,d *G H (V ) Elmoataz-Lezoray-Bougleux, Friedman,Bensoussan, etc Partial difference operators on graphs
  14. ( ) max ( ) ( ) ( ) w

    uv v V f u w f v f u ∞ ∈ ∇ = − ( ) max(0, ( )) max(0, ( ) ( )) v v uv f u f u w f v f u + ∂ = ∂ = − !w ± f (u) = "v ± f (u):v ! u ( ) Discrete gradients and their norms ( ) min(0, ( )) min(0, ( ) ( )) v v uv f u f u w f v f u − ∂ = − ∂ = − − !w f (u) = "v f (u):v ! u ( ) !w f (u) p = w(u,v) p/2 f (v)" f (u) p v!u # $ % & ' ( ) 1/ p Upwind derivatives -  Formulations independent of topology of the graph -  Unifies Local and non local -  Provides adaptation Partial difference operators on graphs Partial derivatives ( ) ( ) ( ( ) ( )) v w uv f u d f uv w f v f u ∂ = = − !w ± f (u) ! = max v!u w uv M ± 0, f (v)! f (u) ( ) ( ) !± w f (u) p = w(u,v) p/2 M ± 0, f (v)" f (u) ( ) p v!u # $ % & ' ( ) 1/ p M + = max and M ! = min
  15. Anisotropic p-Laplacian 2 * , ( ) def p a

    w p w w w f d d f d f − Δ = !w, p a f (u) = w(u,v)1/2 f (u)" f (v) v!u # p"2 ( f (u)" f (v)), , 1 ( ) ( ) 2 w p p a w u V p J f f u p ∈ ∑ = ∇ , : ( ) ( ) a w p H V H V Δ → , , ( ) ( ), , ( ) ( ) a w p a w p J f f u u V f H V f u ∂ = Δ ∀ ∈ ∀ ∈ ∂ Property : Difference operators on graphs
  16. is known on some subset of the graph 0 0

    : f V V R ⊂ → : f V R → Inpainting = Interpolation 0 \ D V V = 0 \ | given V D f D on missing V We consider a general domain a Graph ( , , ) G V E w = 0 f p w u p min ( ) f D f u ∈ ∑ ∇ 0 0 f=f on V Infinity harmonic interpolation on graphs As on the Euclidean domain , we study minimizers of the discrete functional as p goes to infinity
  17. p w u p min ( ) f D f

    u ∈ ∑ ∇ !w, p a f (u) = "w ' # f (u) p#1 p#1 # "w ' + f (u) p#1 p#1 with w '(u,v) =w(u,v) p p#1 p → ∞ From the p-Laplacian and the oriented gradient norms definitions, we can show that: We study minimizers of the discrete functional As !w, p a f (u) = w(u,v)1/2 f (u)" f (v) v!u ! p"2 ( f (u)" f (v)) = 0 The minimization leads !w," f (u) = #w $ f (u) " $ #w + f (u) " Infinity harmonic interpolation on graphs
  18. •  Infinity Laplacian on Graphs = local and non local

    infinity Laplacian , ( ) ( ) ( ) w w w f u f u f u − + ∞ ∞ ∞ Δ = ∇ − ∇ Infinity harmonic interpolation on graphs !w " f (u) # = max v!u w uv max( f (u)" f (v),0) ( ) !w + f (u) " = max( f (v)) v!u # f (u) =!( f )(u)# f (u) !w " f (u) # = f (u)" min( f (v)) v!u = f (u)"!( f )(u) !w + f (u) " = max v!u w uv max( f (v)# f (u),0) ( ) •  Unweighted graph w(u,v)=1 Morphological Gradients on graphs , ( ) 2 ( ) ( )( ) ( )( ) w f u f u f u f u δ ε ∞ Δ = − − Morphological Laplacian on graphs
  19. •  Non local Infinity Laplacian on Euclidean domains , (

    , , ) n R G E w Ω ⊂ = Ω Infinity harmonic interpolation on graphs { } ( , ) , ( , ) 0 E x y w x y = ∈Ω×Ω > : w R+ Ω×Ω → [ ] 0,1 1 ( , ) ( , ) 0 w x y if y x x y w x y if y x α α ∈ = ≠ − = = , , , ( ) ( ) ( ) ( ) ( ) max min w y y x y y x f y f x f y f x f x x y x y α α ∞ ∈Ω ≠ ∈Ω ≠ ⎛ ⎞ ⎛ ⎞ − − ⎜ ⎟ ⎜ ⎟ −Δ = + ⎜ ⎟ ⎜ ⎟ − − ⎝ ⎠ ⎝ ⎠ , ( ) ( ) ( ) w w w f x f x f x − + ∞ ∞ ∞ Δ = ∇ − ∇ ( ) ( ) , ( ) max ( , )max( ( ) ( ),0) min ( , )min( ( ) ( ),0) w f x w x y f y f x w x y f y f x ∞ Ω Ω −Δ = − + − •  Non Local Hölder infinity Laplacian (Chambolle-Lindgren-Monneau 2011)
  20. is known on some subset of the graph 0 0

    : f V V R ⊂ → : f V R → Inpainting = solution of the following dirichlet problem 0 \ D V V = !" w,# f (u) = 0 on D =V \V0 f (u) = f 0(u) on $D We consider a general domain a Graph ( , , ) G V E w = 0 f Find f Infinity harmonic interpolation on graphs This Dirichlet problem has an unique solution 0 \ | given V D f D on missing V
  21. Infinity harmonic interpolation on graphs [ ] 1 ( )

    ( )( ) ( )( ) ( )( ) 2 f u NLA f u NLD f u NLE f u = = + The equation can be rewritten as (a non local infinity-harmonious function) : ( )( ) ( ) ( ) ( )( ) ( ) ( ) w w NLD f u f u f u NLE f u f u f u + ∞ − ∞ = + ∇ = − ∇ Non local morphological Dilation Non local morphological Erosion !f (u) !t = "w + f (u) # !f (u) !t = $ "w $ f (u) # Non Local PdEs morphology on Graphs Ta-Elmoataz-Lézoray (2008,2011) !"w,# f (u) = 0 We can show that the sequence Admits a unique solution and converges to the f=NLA(f) f n+1(u) = NLA( f n )(u) Simple digital algorithm= repeated average of non local dilatation and erosion f (u) = 1 2 2 f (u)+ !w + f (u) " # !w # f (u) " $ % & ' ( ) or
  22. f :! " Z 2 # R, becomes f :V

    # R p i = (x i , y i ) $ !%u $V 2 2 2 ( ) ( ) ( , ) xp( ) f f F u F v w u v e σ − − = Non local inpainting
  23. Non local inpainting

  24. None
  25. Non local inpainting

  26. Non local upscaling

  27. Non local upscaling

  28. Semi-supervised segmentation

  29. •  We  proposed   –  A  connec,on  between  the  anisotropic

     p-­‐Laplacian   and  discrete  norms  of  morphological  gradients  on   graphs   –  Derived  an  explicit  expression  of  the  infinity  Laplacian   on  graphs   –   An  algorithm  for  solving  the  Dirichlet  problem  with   infinity-­‐harmonic  func,ons  on  graphs   –  Unifies  local  and  nonlocal  interpola,on  on  images   •  We  used  this  formalism  to  perform  local  and   nolocal  interpola,on  on  graphs   Conclusion
  30. Thanks,   Any  ques,ons  ?