ICIP 2014

Paper ranked in the "top 10% papers of ICIP'14", Award finalist for the ICIP Best Paper Award (9 papers among 1219 accepted papers)

Transcript

1. Graph Signal Decomposition for Multi-scale Detail Manipulation Moncef Hidane 1,3,4, Olivier L´ ezoray2,5, Abderrahim Elmoataz2,5 1Univ. Bordeaux, Talence, France 2Normandie Univ., Caen, France 3CNRS, IMS, UMR 5218, Talence, France 4CNRS, IMB, UMR 5251, Talence, France 5CNRS, GREYC, UMR 6072, Caen, France IEEE International Conference on Image Processing Paris, October 29, 2014 M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 1 / 30
2. Outline 1 Introduction 2 Signals on Weighted Graphs as a Model for Data 3 Multi-scale Decomposition 4 Detail Manipulation 5 Conclusion M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 2 / 30
3. Outline 1 Introduction 2 Signals on Weighted Graphs as a Model for Data 3 Multi-scale Decomposition 4 Detail Manipulation 5 Conclusion M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 3 / 30
4. Detail Manipulation: Images Original image Coarse scale Fine scale Fig. 1. Sum of successive layers from an image (top row) and a 3D mesh (bottom row) h Original Coarse scale details Medium scale details Fig. 1. Sum of successive layers from an image (top row) and a 3D Original Coarse scale details Med rs from an image (top row) and a 3D mesh (bottom row) hierarchical decomposition. Coarse scale details Medium scale details Fine scale details M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 4 / 30
5. Detail Manipulation: Meshes Original Coarse scale details Medium scale details from an image (top row) and a 3D mesh (bottom row) hierarchical decomposition. Coarse scale details Medium scale details Fine scale details Original mesh Coarse scale Fine scale M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 5 / 30
6. Detail Manipulation: Point Clouds Original point cloud Coarse scale Fine scale Fig. 2. From top to bottom rows: detail manipulation for an image, a 3D mesh and a 3D color provides a scale of detail manipulation. Fig. 2. From top to bottom rows: detail manipulation for an image, a 3D mesh a provides a scale of detail manipulation. M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 6 / 30
7. Multi-scale Image Decomposition = + + + + + + + + + … M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 7 / 30
8. This Talk M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 8 / 30
9. This Talk We introduce a common framework for image, mesh and point clouds multi-scale decomposition. M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 8 / 30
10. This Talk We introduce a common framework for image, mesh and point clouds multi-scale decomposition. We model images, meshes and point clouds as signals defined on graphs. M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 8 / 30
11. This Talk We introduce a common framework for image, mesh and point clouds multi-scale decomposition. We model images, meshes and point clouds as signals defined on graphs. We represent a given graph signal as a sum of successive layers, each capturing a given scale of detail. M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 8 / 30
12. This Talk We introduce a common framework for image, mesh and point clouds multi-scale decomposition. We model images, meshes and point clouds as signals defined on graphs. We represent a given graph signal as a sum of successive layers, each capturing a given scale of detail. We perform detail manipulation by separate processing of the layers. M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 8 / 30
13. Outline 1 Introduction 2 Signals on Weighted Graphs as a Model for Data 3 Multi-scale Decomposition 4 Detail Manipulation 5 Conclusion M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 9 / 30
14. Images as Signals on Weighted Graphs M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 10 / 30
15. Images as Signals on Weighted Graphs We identify the set of pixels with the set of vertices. M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 10 / 30
16. Images as Signals on Weighted Graphs We identify the set of pixels with the set of vertices. Two pixels are connected if their features are similar: gray level; color; texture descriptors; patches. M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 10 / 30
17. Images as Signals on Weighted Graphs We identify the set of pixels with the set of vertices. Two pixels are connected if their features are similar: gray level; color; texture descriptors; patches. An image is a signal (gray level or color) on the set of vertices. M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 10 / 30
18. Images as Signals on Weighted Graphs We identify the set of pixels with the set of vertices. Two pixels are connected if their features are similar: gray level; color; texture descriptors; patches. An image is a signal (gray level or color) on the set of vertices.
                    
    
                      
    
                      
    
                    
    
                    
    
    
    
     
     
     
     
     
     
     
    
    
             
             
             
             
             
             
             
                    
                    
    
    
     
     
     
     
     
     
     
    
    
             
             
             
             
             
             
             
                    
                    
    
    
     
     
     
     
     
     
     
    
    
             
             
             
             
             
             
             
                    
                    
    
    
     
     
     
     
     
     
     
    
    
             
             
             
             
             
             
             
                    
                    
    
    
     
     
     
     
     
     
     
    
    
             
             
             
             
             
             
             
                    
                    
    
    
     
     
     
     
     
     
     
    
    
             
             
             
             
             
             
             
                    
                    
    
    
     
     
     
     
     
     
     
    
    
             
             
             
             
             
             
             
                    
                   
    
    
    
    
    
    
    
    
    
    
    
             
             
             
             
             
             
            
          
          
    
    
    
    
    
    
    
    
    
    
    
             
             
             
             
             
             
            
          
           
    
    
    
    
     
    
     
    
     
    
     
    
     
    
     
    
     
    
    
    
    
    
    
     
     
     
     
     
     
     
     
     
    
    
     
     
     
     
     
     
     
     
     
    
    
     
     
     
     
     
     
     
     
     
    
    
     
     
     
     
     
     
     
     
     
    
    
     
     
     
     
     
     
     
     
     
    
    
     
     
     
     
     
     
     
     
     
    
    
     
     
     
     
     
     
     
     
     
    
    
     
     
     
     
     
     
     
    
    
    
    
     
     
     
     
     
     
     
    
    
    
    
     
     
     
     
     
     
     
    
    
             
             
             
             
             
             
             
                    
                    
    
    
     
     
     
     
     
     
     
    
    
             
             
             
             
             
             
             
                    
                    
    
    
     
     
     
     
     
     
     
    
    
             
             
             
             
             
             
             
                    
                    
    
    
     
     
     
     
     
     
     
    
    
             
             
             
             
             
             
             
                    
                    
    
    
     
     
     
     
     
     
     
    
    
             
             
             
             
             
             
             
                    
                    
    
    
     
     
     
     
     
     
     
    
    
             
             
             
             
             
             
             
                    
                    
    
    
     
     
     
     
     
     
     
    
    
             
             
             
             
             
             
             
                    
                    
    
    
     
     
     
     
     
     
     
    
     
                       
                       
                       
                       
                       
                       
                      
                    
                    
    
    
     
     
     
     
     
     
     
    
     
                       
                       
                       
                       
                       
                       
                      
                    
                    
    
    
    
    
     
    
     
    
     
    
     
    
     
    
     
    
     
    
    
    
    
    
    
     
     
     
     
     
     
     
     
     
    
    
     
     
     
     
     
     
     
     
     
    
    
     
     
     
     
     
     
     
     
     
    
    
     
     
     
     
     
     
     
     
     
    
    
     
     
     
     
     
     
     
     
     
    
    
     
     
     
     
     
     
     
     
     
    
    
     
     
     
     
     
     
     
     
     
    
    
     
     
     
     
     
     
     
    
    
    
    
     
     
     
     
     
     
     
    
    
    
    
     
     
     
     
     
     
     
    
    
             
             
             
             
             
             
             
                    
                    
    
    
     
     
     
     
     
     
     
    
    
             
             
             
             
             
             
             
                    
                    
    
    
     
     
     
     
     
     
     
    
    
             
             
             
             
             
             
             
                    
                    
    
    
     
     
     
     
     
     
     
    
    
             
             
             
             
             
             
             
                    
                    
    
    
     
     
     
     
     
     
     
    
    
             
             
             
             
             
             
             
                    
                    
    
    Original image 4-grid graph Patch-based! nearset neighbors! graph M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 10 / 30
19. Meshes and Point Clouds as Signals on Weighted Graphs M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 11 / 30
20. Meshes and Point Clouds as Signals on Weighted Graphs We identify the set of 3D points with the set of vertices. M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 11 / 30
21. Meshes and Point Clouds as Signals on Weighted Graphs We identify the set of 3D points with the set of vertices. For meshes the connectivity of the graph is given the triangulation; the signal is the x, y, z spatial components. M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 11 / 30
22. Meshes and Point Clouds as Signals on Weighted Graphs We identify the set of 3D points with the set of vertices. For meshes the connectivity of the graph is given the triangulation; the signal is the x, y, z spatial components. For point clouds the connectivity of the graph is constructed through RGB similarity; the signal is the RGB color components. M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 11 / 30
23. Meshes and Point Clouds as Signals on Weighted Graphs We identify the set of 3D points with the set of vertices. For meshes the connectivity of the graph is given the triangulation; the signal is the x, y, z spatial components. For point clouds the connectivity of the graph is constructed through RGB similarity; the signal is the RGB color components. 3D mesh 3D point cloud M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 11 / 30
24. Some Notations A weighted graph G = (V , E, w) V is a set of N vertices E is a set of edges (⊂ V × V ) w : E →]0, +∞[ is a similarity weight functions M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 12 / 30
25. Some Notations A weighted graph G = (V , E, w) V is a set of N vertices E is a set of edges (⊂ V × V ) w : E →]0, +∞[ is a similarity weight functions We denote by X = RN the set of scalar signals defined on a fixed graph with N vertices. M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 12 / 30
26. Outline 1 Introduction 2 Signals on Weighted Graphs as a Model for Data 3 Multi-scale Decomposition 4 Detail Manipulation 5 Conclusion M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 13 / 30
27. Scale Through Energy Minimization The scale of features present in a graph signal f ∈ X is revealed through the following minimization minimize u∈X E(u; f , λ) = λJ(u) + 1 2 u − f 2. M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 14 / 30
28. Scale Through Energy Minimization The scale of features present in a graph signal f ∈ X is revealed through the following minimization minimize u∈X E(u; f , λ) = λJ(u) + 1 2 u − f 2. The functional J : X → R+ should measure the smoothness of u. M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 14 / 30
29. Scale Through Energy Minimization The scale of features present in a graph signal f ∈ X is revealed through the following minimization minimize u∈X E(u; f , λ) = λJ(u) + 1 2 u − f 2. The functional J : X → R+ should measure the smoothness of u. For denoising, λ is related to the noise level. M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 14 / 30
30. Scale Through Energy Minimization The scale of features present in a graph signal f ∈ X is revealed through the following minimization minimize u∈X E(u; f , λ) = λJ(u) + 1 2 u − f 2. The functional J : X → R+ should measure the smoothness of u. For denoising, λ is related to the noise level. For decomposition, λ plays the role of a scale parameter. M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 14 / 30
31. Scale Through Energy Minimization The scale of features present in a graph signal f ∈ X is revealed through the following minimization minimize u∈X E(u; f , λ) = λJ(u) + 1 2 u − f 2. The functional J : X → R+ should measure the smoothness of u. For denoising, λ is related to the noise level. For decomposition, λ plays the role of a scale parameter. Denoting ˆ u = argmin u∈X E(u; f , λ), we have a one-scale decomposition of the form f = ˆ u + ˆ v. M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 14 / 30
32. Total Variation of Graph Signals Jw (u) = N i=1   N j=1 wi,j (uj − ui )2   1/2 M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 15 / 30
33. Total Variation of Graph Signals Jw (u) = N i=1   N j=1 wi,j (uj − ui )2   1/2 M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 15 / 30
34. Total Variation of Graph Signals Jw (u) = N i=1   N j=1 wi,j (uj − ui )2   1/2 M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 15 / 30
35. Total Variation of Graph Signals Jw (u) = N i=1   N j=1 wi,j (uj − ui )2   1/2 M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 15 / 30
36. Total Variation of Graph Signals Jw (u) = N i=1   N j=1 wi,j (uj − ui )2   1/2 M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 15 / 30
37. Total Variation of Graph Signals Jw (u) = N i=1   N j=1 wi,j (uj − ui )2   1/2 M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 15 / 30
38. Total Variation Regularization over Graphs We are lead to minimize u∈X E(u; f , λ) = λJw (u) + 1 2 u − f 2. 1Moncef Hidane, Olivier L´ ezoray, and Abderrahim Elmoataz. “Nonlinear Multilayered Representation of Graph-signals”. Journal of Mathematical Imaging and Vision 45.2 (2013), pp. 114–137. M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 16 / 30
39. Total Variation Regularization over Graphs We are lead to minimize u∈X E(u; f , λ) = λJw (u) + 1 2 u − f 2. The energy is not differentiable! Further details about the model and the optimization can be found in1. 1Moncef Hidane, Olivier L´ ezoray, and Abderrahim Elmoataz. “Nonlinear Multilayered Representation of Graph-signals”. Journal of Mathematical Imaging and Vision 45.2 (2013), pp. 114–137. M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 16 / 30
40. From Decomposition to Multilayered Representation M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 17 / 30
41. From Decomposition to Multilayered Representation The minimization of the TV- 2 energy on graphs yields a one-scale decomposition. M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 17 / 30
42. From Decomposition to Multilayered Representation The minimization of the TV- 2 energy on graphs yields a one-scale decomposition. In order to be able to manipulate graph signals at multiple scales, it is important to turn those decompositions into multi-scale ones. M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 17 / 30
43. From Decomposition to Multilayered Representation The minimization of the TV- 2 energy on graphs yields a one-scale decomposition. In order to be able to manipulate graph signals at multiple scales, it is important to turn those decompositions into multi-scale ones. One way to obtain multi-scale decompositions is to iterate the decompositions on the successive residuals:        v−1 = f , ui = argmin u∈X E(u; vi−1 ; λi ), i ≥ 0, vi = vi−1 − ui , i ≥ 0, M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 17 / 30
44. Multilayered Representation: Remarks f = v-1 u0 v0 u1 v1 u2 v2 … … un vn 1 0 2 n M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 18 / 30
45. Multilayered Representation: Remarks f = v-1 u0 v0 u1 v1 u2 v2 … … un vn 1 0 2 n f = u0 + u1 + . . . + un + vn . M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 18 / 30
46. Multilayered Representation: Remarks f = v-1 u0 v0 u1 v1 u2 v2 … … un vn 1 0 2 n f = u0 + u1 + . . . + un + vn . The layers ui are parametrized by three variables: graph topology and weights; the energy function E; the sequence λ0, . . . , λn involved in the successive minimizations. M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 18 / 30
47. Multilayered Representation: Remarks f = v-1 u0 v0 u1 v1 u2 v2 … … un vn 1 0 2 n f = u0 + u1 + . . . + un + vn . The layers ui are parametrized by three variables: graph topology and weights; the energy function E; the sequence λ0, . . . , λn involved in the successive minimizations. In order to extract the successive layers in a coherent manner, the sequence of scales (λi )i≥0 should be monotone. M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 18 / 30
48. Multilayered Representation: Remarks f = v-1 u0 v0 u1 v1 u2 v2 … … un vn 1 0 2 n f = u0 + u1 + . . . + un + vn . The layers ui are parametrized by three variables: graph topology and weights; the energy function E; the sequence λ0, . . . , λn involved in the successive minimizations. In order to extract the successive layers in a coherent manner, the sequence of scales (λi )i≥0 should be monotone. M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 18 / 30
49. Examples f u0 P2 i=0 ui P3 i=0 ui P4 i=0 ui f u0 P3 i=0 ui P6 i=0 ui P10 i=0 ui Fig. 1. Sum of successive layers from an image (top row) and a 3D mesh (bottom row) hierarchical decomposition. Original Coarse scale details Medium scale details Fine scale details Figure : Sum of successive layers extracted from an image (top row) and a 3D mesh (bottom row) by recursive TV- 2 minimization over a graph. M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 19 / 30
50. Outline 1 Introduction 2 Signals on Weighted Graphs as a Model for Data 3 Multi-scale Decomposition 4 Detail Manipulation 5 Conclusion M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 20 / 30
51. Principle M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 21 / 30
52. Principle To manipulate details of a graph signals f ∈ X, we first decompose it into n + 1 layers ui , i ∈ {0, ..., n}. M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 21 / 30
53. Principle To manipulate details of a graph signals f ∈ X, we first decompose it into n + 1 layers ui , i ∈ {0, ..., n}. Then, we edit f by weighting each layer and adding the layers back together. M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 21 / 30
54. Principle To manipulate details of a graph signals f ∈ X, we first decompose it into n + 1 layers ui , i ∈ {0, ..., n}. Then, we edit f by weighting each layer and adding the layers back together. We consider three levels of detail manipulation: coarse (g1 ); intermediate (g2 ); fine (g3 ). M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 21 / 30
55. Choosing the Coefficients M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 22 / 30
56. Choosing the Coefficients Coarse scale version g1 = l1 i=0 (1 + iδ1 )ui . M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 22 / 30
57. Choosing the Coefficients Coarse scale version g1 = l1 i=0 (1 + iδ1 )ui . Medium scale version g2 = g1 + l2 i=l1+1 (δ2 + (i − l1 − 1)δ1 δ2 )ui . M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 22 / 30
58. Choosing the Coefficients Coarse scale version g1 = l1 i=0 (1 + iδ1 )ui . Medium scale version g2 = g1 + l2 i=l1+1 (δ2 + (i − l1 − 1)δ1 δ2 )ui . Fine scale version g3 = g2 + n i=l2+1 (δ2 2 + (i − l2 − 1)δ1 δ2 )ui . M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 22 / 30
59. Choosing the Coefficients Coarse scale version g1 = l1 i=0 (1 + iδ1 )ui . Medium scale version g2 = g1 + l2 i=l1+1 (δ2 + (i − l1 − 1)δ1 δ2 )ui . Fine scale version g3 = g2 + n i=l2+1 (δ2 2 + (i − l2 − 1)δ1 δ2 )ui . For images: δ1 = 2.5, δ2 = 0.25 M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 22 / 30
60. Choosing the Coefficients Coarse scale version g1 = l1 i=0 (1 + iδ1 )ui . Medium scale version g2 = g1 + l2 i=l1+1 (δ2 + (i − l1 − 1)δ1 δ2 )ui . Fine scale version g3 = g2 + n i=l2+1 (δ2 2 + (i − l2 − 1)δ1 δ2 )ui . For images: δ1 = 2.5, δ2 = 0.25 For meshes and point clouds: δ1 = 0.15, δ2 = 1 M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 22 / 30
61. Choosing the Coefficients Coarse scale version g1 = l1 i=0 (1 + iδ1 )ui . Medium scale version g2 = g1 + l2 i=l1+1 (δ2 + (i − l1 − 1)δ1 δ2 )ui . Fine scale version g3 = g2 + n i=l2+1 (δ2 2 + (i − l2 − 1)δ1 δ2 )ui . For images: δ1 = 2.5, δ2 = 0.25 For meshes and point clouds: δ1 = 0.15, δ2 = 1 l1 and l2 are set by the user. M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 22 / 30
62. Examples: An Image Fig. 1. Sum of successive layers from an image (top row) and a 3D mesh (bottom row) hierarchical decomposition. Original Coarse scale details Medium scale details Fine scale details Original Coarse scale Medium scale Fine scale M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 23 / 30
63. Examples: A Mesh Fig. 1. Sum of successive layers from an image (top row) and a 3D mesh (bottom row) hierarchical decomposition. Original Coarse scale details Medium scale details Fine scale details Original Coarse scale Medium scale Fine scale M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 24 / 30
64. Examples: A Point Cloud . From top to bottom rows: detail manipulation for an image, a 3D mesh and a 3D colored point cloud. Each colum des a scale of detail manipulation. Original Coarse scale Medium scale Fine scale M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 25 / 30
65. A Comparison Original Farbmann et al. 2008 Ours Chen et al. 2007 M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 26 / 30
66. An Application : 3D Model Enhancement Décompositions multi-échelles hiérarchiques de signaux sur graphes Maillage coloré original Maillage coloré grossier Maillage coloré original Maillage coloré grossier Maillage coloré intermédiaire Maillage coloré fin Input low-quality 3D model Enhanced 3D model M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 27 / 30
67. Outline 1 Introduction 2 Signals on Weighted Graphs as a Model for Data 3 Multi-scale Decomposition 4 Detail Manipulation 5 Conclusion M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 28 / 30
68. Conclusion A graph-based common framework for image, mesh and point clouds detail manipulation. M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 29 / 30
69. Conclusion A graph-based common framework for image, mesh and point clouds detail manipulation. Multilayered representation through iterative energy minimization. M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 29 / 30
70. Conclusion A graph-based common framework for image, mesh and point clouds detail manipulation. Multilayered representation through iterative energy minimization. Detail manipulation by individual processing of the layers. M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 29 / 30
71. Conclusion A graph-based common framework for image, mesh and point clouds detail manipulation. Multilayered representation through iterative energy minimization. Detail manipulation by individual processing of the layers. One perspective is to replace TV regularization with Laplacian regularization can benefit from fast linear algebra solvers; the decomposition is linear if the graph is fixed, but the overall decomposition is nonlinear if the graph is built from the data. M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 29 / 30
72. Conclusion A graph-based common framework for image, mesh and point clouds detail manipulation. Multilayered representation through iterative energy minimization. Detail manipulation by individual processing of the layers. One perspective is to replace TV regularization with Laplacian regularization can benefit from fast linear algebra solvers; the decomposition is linear if the graph is fixed, but the overall decomposition is nonlinear if the graph is built from the data. Another perspective is to work on complete automatic parameters selection, especially for meshes and point clouds. M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 29 / 30
73. Thank you for your attention! M. Hidane, O. L´ ezoray, A. Elmoataz Graph Signal Detail Manipulation October 29, 2014 30 / 30