## Slide 1

### Slide 1 text

1/24 Introduction Background: Spectral Geometry Approach: Isospectralization Applications/Results Summary Isospectralization, or how to hear shape, style, and correspondence Sricharan Chiruvolu Zorah Lähner June 24, 2020 Sricharan Chiruvolu Isospectralization

## Slide 2

### Slide 2 text

2/24 Introduction Background: Spectral Geometry Approach: Isospectralization Applications/Results Summary Introduction Published in CVPR 2019. Explores the practical possibility of using the spectrum of an object for shape reconstruction. Proposes an inverse mapping between geometric domain and its Laplacian via simple regularizers. Isospectralization: A numerical optimization procedure to deform a mesh in order to align its (ﬁnite) Laplacian spectrum with a given one. Sricharan Chiruvolu Isospectralization

## Slide 3

### Slide 3 text

3/24 Introduction Background: Spectral Geometry Approach: Isospectralization Applications/Results Summary Isospectralization Figure 1: Isospectralization Sricharan Chiruvolu Isospectralization

## Slide 4

### Slide 4 text

4/24 Introduction Background: Spectral Geometry Approach: Isospectralization Applications/Results Summary Riemannian Manifold and Geodesics Isometry and Isomorphism Laplace-Beltrami Operator Direct and Inverse Problems Spectral Geometry Relationship between geometric structures of manifolds and spectra of canonically deﬁned diﬀerential operators. Eigenvalue problems involving the Laplace operator on manifolds have proven to be a consistently fertile area of geometric analysis with deep connections to number theory, physics, and applied mathematics. Sricharan Chiruvolu Isospectralization

## Slide 5

### Slide 5 text

5/24 Introduction Background: Spectral Geometry Approach: Isospectralization Applications/Results Summary Riemannian Manifold and Geodesics Isometry and Isomorphism Laplace-Beltrami Operator Direct and Inverse Problems Manifolds (As discussed in previous talks) In computer vision, the notion of a manifold is used quite frequently. in rotation averaging (SO3), structure and motion (“Essential” manifold) to capture the shape of an object (“Shape” manifolds) to model a set of images (“Grassman” manifolds) to simply represent a sphere (Sn). ... Sricharan Chiruvolu Isospectralization

## Slide 6

### Slide 6 text

6/24 Introduction Background: Spectral Geometry Approach: Isospectralization Applications/Results Summary Riemannian Manifold and Geodesics Isometry and Isomorphism Laplace-Beltrami Operator Direct and Inverse Problems Riemannian Manifold and Geodesics (As discussed in previous talks) A Riemannian manifold is a manifold with an inner product deﬁned in the tangent space at each point. Geodesics are locally shortest curves. They preserve a direction on a surface and have many interesting properties. Sricharan Chiruvolu Isospectralization

## Slide 7

### Slide 7 text

7/24 Introduction Background: Spectral Geometry Approach: Isospectralization Applications/Results Summary Riemannian Manifold and Geodesics Isometry and Isomorphism Laplace-Beltrami Operator Direct and Inverse Problems Riemannian Manifold and Geodesics Figure 2: Geodesic and tangent space in a Riemannian manifold. 1 1Medinria: DT-MRI processing and visualization software; Xavier Pennec. Sricharan Chiruvolu Isospectralization

## Slide 8

### Slide 8 text

8/24 Introduction Background: Spectral Geometry Approach: Isospectralization Applications/Results Summary Riemannian Manifold and Geodesics Isometry and Isomorphism Laplace-Beltrami Operator Direct and Inverse Problems Isometry and Isomorphism Figure 3: Isometry and Isomorphism.2 2Smooth Shells: Multi-Scale Shape Registration with Functional Maps; Eisenberger, Lahner and Cremers. Sricharan Chiruvolu Isospectralization

## Slide 9

### Slide 9 text

9/24 Introduction Background: Spectral Geometry Approach: Isospectralization Applications/Results Summary Riemannian Manifold and Geodesics Isometry and Isomorphism Laplace-Beltrami Operator Direct and Inverse Problems Laplace-Beltrami Operator Figure 4: Laplace-Beltrami operator 3 Laplacian ( f ) = div(gradient(f )). The Laplacian is generalized to the Riemannian manifold (M; g) by the Laplace-Beltrami operator ( g ). 3Khan Academy; Grant Sanderson. Sricharan Chiruvolu Isospectralization

## Slide 10

### Slide 10 text

10/24 Introduction Background: Spectral Geometry Approach: Isospectralization Applications/Results Summary Riemannian Manifold and Geodesics Isometry and Isomorphism Laplace-Beltrami Operator Direct and Inverse Problems Isospectrals Given a compact Riemannian manifold, we can associate to it the Laplace-Beltrami operator. Two closed Riemannian manifolds are said to be isospectral if the eigenvalues of their Laplace–Beltrami operator (Laplacians) coincide. Thus, spectral geometry is the connection between the spectrum Spec(M; g), i.e. the eigenvalues, and the geometry of the manifold (M; g). This fundamentally deals with two kinds of problems: Direct problems Inverse problems Sricharan Chiruvolu Isospectralization

## Slide 11

### Slide 11 text

11/24 Introduction Background: Spectral Geometry Approach: Isospectralization Applications/Results Summary Riemannian Manifold and Geodesics Isometry and Isomorphism Laplace-Beltrami Operator Direct and Inverse Problems Direct and Inverse Problems Direct Problems Direct problems attempt to infer the behavior of the eigenvalues of a Riemannian manifold from knowledge of the geometry. Compute (exactly or not) the spectrum Spec(M, g)? And (or) ﬁnd properties on the spectrum Spec(M, g)? Inverse Problems Inverse problem - does the data of the spectrum Spec(M, g) determine the "shape" of the manifold (M, g) ? Which geometric information of the manifold can we determine from the spectrum? Sricharan Chiruvolu Isospectralization

## Slide 12

### Slide 12 text

12/24 Introduction Background: Spectral Geometry Approach: Isospectralization Applications/Results Summary Hearing the Shape of a Drum Isospectralization Regularizers for ﬂat shapes Regularizers for surfaces shapes Hearing the Shape of a Drum Figure 5: Membranes of these two diﬀerent shapes (but otherwise identical) would "sound" the same. "Can one hear the shape of a drum?" Mark Kac’s 1966 question, "Can one hear the shape of a drum?" set oﬀ research into spectral theory, with the idea of understanding the extent to which the spectrum allows one to read back the geometry. Sricharan Chiruvolu Isospectralization

## Slide 13

### Slide 13 text

13/24 Introduction Background: Spectral Geometry Approach: Isospectralization Applications/Results Summary Hearing the Shape of a Drum Isospectralization Regularizers for ﬂat shapes Regularizers for surfaces shapes Hearing the Shape of a Drum Theoretically Negative. Why? - There exists isospectral manifolds that are nonisometric. With additional priors, practically feasible. We can recover the structure of an object from its spectrum. Sricharan Chiruvolu Isospectralization

## Slide 14

### Slide 14 text

14/24 Introduction Background: Spectral Geometry Approach: Isospectralization Applications/Results Summary Hearing the Shape of a Drum Isospectralization Regularizers for ﬂat shapes Regularizers for surfaces shapes Notations Figure 6: Notations Discretization. Sricharan Chiruvolu Isospectralization

## Slide 15

### Slide 15 text

15/24 Introduction Background: Spectral Geometry Approach: Isospectralization Applications/Results Summary Hearing the Shape of a Drum Isospectralization Regularizers for ﬂat shapes Regularizers for surfaces shapes Isospectralization Find the optimal embedding that aligns the spectrum to another: min V∈Rn×d λ (∆X (V)) − µ ω + ρX (V) V is the (unknown) embedding of the mesh vertices in R. ∆X (V) is the associated discrete Laplacian. Sricharan Chiruvolu Isospectralization

## Slide 16

### Slide 16 text

16/24 Introduction Background: Spectral Geometry Approach: Isospectralization Applications/Results Summary Hearing the Shape of a Drum Isospectralization Regularizers for ﬂat shapes Regularizers for surfaces shapes Flat Shape Regularizers Encourage Smaller Boundary Edge Lengths ρX (V) = eij ∈Eb lij (V) Penalize Triangle Flips ρX (V ) = (min (0, rX (V )))2 rX (V ) = ijk∈F Rπ/22 vj − vi T vk − vi Sricharan Chiruvolu Isospectralization

## Slide 17

### Slide 17 text

17/24 Introduction Background: Spectral Geometry Approach: Isospectralization Applications/Results Summary Hearing the Shape of a Drum Isospectralization Regularizers for ﬂat shapes Regularizers for surfaces shapes Flat Shape Regularizers Encourage Small Interior Edge Lengths ρX (V) = eij ∈Ei l2 ij (V) Note: Optimization is eﬀected by the mesh resolution and spectral bandwidth. Sricharan Chiruvolu Isospectralization

## Slide 18

### Slide 18 text

18/24 Introduction Background: Spectral Geometry Approach: Isospectralization Applications/Results Summary Hearing the Shape of a Drum Isospectralization Regularizers for ﬂat shapes Regularizers for surfaces shapes Shape reconstruction on Flat Shapes Figure 7: Shape recovery optimization for ﬂat shapes. Sricharan Chiruvolu Isospectralization

## Slide 19

### Slide 19 text

19/24 Introduction Background: Spectral Geometry Approach: Isospectralization Applications/Results Summary Hearing the Shape of a Drum Isospectralization Regularizers for ﬂat shapes Regularizers for surfaces shapes Surface Regularizers Encourage Vertices to Lie Close to Barycenter of neighbors, L is the initial graph Laplacian. ρX (V) = Lg 0 V 2 2 Encourage small total displacement from initial embedding - volume to grow/shrink to disambiguate isometric shapes. ρX (V) = n i=1 ai vi − vi 0 2 Sricharan Chiruvolu Isospectralization

## Slide 20

### Slide 20 text

20/24 Introduction Background: Spectral Geometry Approach: Isospectralization Applications/Results Summary Hearing the Shape of a Drum Isospectralization Regularizers for ﬂat shapes Regularizers for surfaces shapes Shape reconstruction on Surfaces Figure 8: Two isometric shapes ("Surfaces") with diﬀerent volume; their (identical) spectra. This is the reason we need the second Regularizer. Sricharan Chiruvolu Isospectralization

## Slide 21

### Slide 21 text

21/24 Introduction Background: Spectral Geometry Approach: Isospectralization Applications/Results Summary Non-isometric shape matching Style Transfer Non-isometric shape matching Figure 9: Isospectralization as pre-processing for dense correspondence matching. Sricharan Chiruvolu Isospectralization

## Slide 22

### Slide 22 text

22/24 Introduction Background: Spectral Geometry Approach: Isospectralization Applications/Results Summary Non-isometric shape matching Style Transfer Style Transfer Figure 10: Isospectralization for Style Transfer Sricharan Chiruvolu Isospectralization

## Slide 23

### Slide 23 text

23/24 Introduction Background: Spectral Geometry Approach: Isospectralization Applications/Results Summary Summary Improvement for modern shape matching problems via addressing a decades-old problem. Applications in various domains of geometry processing and computer vision. No mathematical backing on optimization yet. No experimental proof on shape interpolation yet. Sricharan Chiruvolu Isospectralization

## Slide 24

### Slide 24 text

24/24 Introduction Background: Spectral Geometry Approach: Isospectralization Applications/Results Summary End Quick Demo. Questions. Thanks for listening! Sricharan Chiruvolu Isospectralization