1/24
Introduction
Background: Spectral Geometry
Approach: Isospectralization
Applications/Results
Summary
Isospectralization, or how to hear shape,
style, and correspondence
Sricharan Chiruvolu
Zorah Lähner
June 24, 2020
Sricharan Chiruvolu Isospectralization

2/24
Introduction
Background: Spectral Geometry
Approach: Isospectralization
Applications/Results
Summary
Introduction
Published in CVPR 2019.
Explores the practical possibility of using the spectrum of an
object for shape reconstruction.
Proposes an inverse mapping between geometric domain and
its Laplacian via simple regularizers.
Isospectralization: A numerical optimization procedure to
deform a mesh in order to align its (ﬁnite) Laplacian spectrum
with a given one.
Sricharan Chiruvolu Isospectralization

3/24
Introduction
Background: Spectral Geometry
Approach: Isospectralization
Applications/Results
Summary
Isospectralization
Figure 1: Isospectralization
Sricharan Chiruvolu Isospectralization

4/24
Introduction
Background: Spectral Geometry
Approach: Isospectralization
Applications/Results
Summary
Riemannian Manifold and Geodesics
Isometry and Isomorphism
Laplace-Beltrami Operator
Direct and Inverse Problems
Spectral Geometry
Relationship between geometric structures of manifolds and
spectra of canonically deﬁned diﬀerential operators.
Eigenvalue problems involving the Laplace operator on
manifolds have proven to be a consistently fertile area of
geometric analysis with deep connections to number theory,
physics, and applied mathematics.
Sricharan Chiruvolu Isospectralization

5/24
Introduction
Background: Spectral Geometry
Approach: Isospectralization
Applications/Results
Summary
Riemannian Manifold and Geodesics
Isometry and Isomorphism
Laplace-Beltrami Operator
Direct and Inverse Problems
Manifolds
(As discussed in previous talks) In computer vision, the notion of a
manifold is used quite frequently.
in rotation averaging (SO3),
structure and motion (“Essential” manifold)
to capture the shape of an object (“Shape” manifolds)
to model a set of images (“Grassman” manifolds)
to simply represent a sphere (Sn).
...
Sricharan Chiruvolu Isospectralization

6/24
Introduction
Background: Spectral Geometry
Approach: Isospectralization
Applications/Results
Summary
Riemannian Manifold and Geodesics
Isometry and Isomorphism
Laplace-Beltrami Operator
Direct and Inverse Problems
Riemannian Manifold and Geodesics
(As discussed in previous talks)
A Riemannian manifold is a manifold with an inner product
deﬁned in the tangent space at each point.
Geodesics are locally shortest curves. They preserve a
direction on a surface and have many interesting properties.
Sricharan Chiruvolu Isospectralization

7/24
Introduction
Background: Spectral Geometry
Approach: Isospectralization
Applications/Results
Summary
Riemannian Manifold and Geodesics
Isometry and Isomorphism
Laplace-Beltrami Operator
Direct and Inverse Problems
Riemannian Manifold and Geodesics
Figure 2: Geodesic and tangent space in a Riemannian manifold. 1
1Medinria: DT-MRI processing and visualization software; Xavier Pennec.
Sricharan Chiruvolu Isospectralization

8/24
Introduction
Background: Spectral Geometry
Approach: Isospectralization
Applications/Results
Summary
Riemannian Manifold and Geodesics
Isometry and Isomorphism
Laplace-Beltrami Operator
Direct and Inverse Problems
Isometry and Isomorphism
Figure 3: Isometry and Isomorphism.2
2Smooth Shells: Multi-Scale Shape Registration with Functional Maps;
Eisenberger, Lahner and Cremers.
Sricharan Chiruvolu Isospectralization

9/24
Introduction
Background: Spectral Geometry
Approach: Isospectralization
Applications/Results
Summary
Riemannian Manifold and Geodesics
Isometry and Isomorphism
Laplace-Beltrami Operator
Direct and Inverse Problems
Laplace-Beltrami Operator
Figure 4: Laplace-Beltrami operator 3
Laplacian ( f ) = div(gradient(f )).
The Laplacian is generalized to the Riemannian manifold
(M; g) by the Laplace-Beltrami operator ( g ).
3Khan Academy; Grant Sanderson.
Sricharan Chiruvolu Isospectralization

10/24
Introduction
Background: Spectral Geometry
Approach: Isospectralization
Applications/Results
Summary
Riemannian Manifold and Geodesics
Isometry and Isomorphism
Laplace-Beltrami Operator
Direct and Inverse Problems
Isospectrals
Given a compact Riemannian manifold, we can associate to it
the Laplace-Beltrami operator.
Two closed Riemannian manifolds are said to be isospectral
if the eigenvalues of their Laplace–Beltrami operator
(Laplacians) coincide.
Thus, spectral geometry is the connection between the
spectrum Spec(M; g), i.e. the eigenvalues, and the geometry
of the manifold (M; g).
This fundamentally deals with two kinds of problems:
Direct problems
Inverse problems
Sricharan Chiruvolu Isospectralization

11/24
Introduction
Background: Spectral Geometry
Approach: Isospectralization
Applications/Results
Summary
Riemannian Manifold and Geodesics
Isometry and Isomorphism
Laplace-Beltrami Operator
Direct and Inverse Problems
Direct and Inverse Problems
Direct Problems
Direct problems attempt to infer the behavior of the eigenvalues of
a Riemannian manifold from knowledge of the geometry.
Compute (exactly or not) the spectrum Spec(M, g)? And (or) ﬁnd
properties on the spectrum Spec(M, g)?
Inverse Problems
Inverse problem - does the data of the spectrum Spec(M, g)
determine the "shape" of the manifold (M, g) ?
Which geometric information of the manifold can we determine
from the spectrum?
Sricharan Chiruvolu Isospectralization

12/24
Introduction
Background: Spectral Geometry
Approach: Isospectralization
Applications/Results
Summary
Hearing the Shape of a Drum
Isospectralization
Regularizers for ﬂat shapes
Regularizers for surfaces shapes
Hearing the Shape of a Drum
Figure 5: Membranes of these two diﬀerent shapes (but otherwise
identical) would "sound" the same.
"Can one hear the shape of a drum?"
Mark Kac’s 1966 question, "Can one hear the shape of a drum?"
set oﬀ research into spectral theory, with the idea of understanding
the extent to which the spectrum allows one to read back the
geometry.
Sricharan Chiruvolu Isospectralization

13/24
Introduction
Background: Spectral Geometry
Approach: Isospectralization
Applications/Results
Summary
Hearing the Shape of a Drum
Isospectralization
Regularizers for ﬂat shapes
Regularizers for surfaces shapes
Hearing the Shape of a Drum
Theoretically Negative. Why? - There exists isospectral
manifolds that are nonisometric.
With additional priors, practically feasible. We can recover the
structure of an object from its spectrum.
Sricharan Chiruvolu Isospectralization

14/24
Introduction
Background: Spectral Geometry
Approach: Isospectralization
Applications/Results
Summary
Hearing the Shape of a Drum
Isospectralization
Regularizers for ﬂat shapes
Regularizers for surfaces shapes
Notations
Figure 6: Notations
Discretization.
Sricharan Chiruvolu Isospectralization

15/24
Introduction
Background: Spectral Geometry
Approach: Isospectralization
Applications/Results
Summary
Hearing the Shape of a Drum
Isospectralization
Regularizers for ﬂat shapes
Regularizers for surfaces shapes
Isospectralization
Find the optimal embedding that aligns the spectrum to
another:
min
V∈Rn×d
λ (∆X (V)) − µ ω
+ ρX (V)
V is the (unknown) embedding of the mesh vertices in R.
∆X (V) is the associated discrete Laplacian.
Sricharan Chiruvolu Isospectralization

16/24
Introduction
Background: Spectral Geometry
Approach: Isospectralization
Applications/Results
Summary
Hearing the Shape of a Drum
Isospectralization
Regularizers for ﬂat shapes
Regularizers for surfaces shapes
Flat Shape Regularizers
Encourage Smaller Boundary Edge Lengths
ρX (V) =
eij ∈Eb
lij (V)
Penalize Triangle Flips
ρX (V ) = (min (0, rX (V )))2
rX (V ) =
ijk∈F
Rπ/22 vj − vi T
vk − vi
Sricharan Chiruvolu Isospectralization

17/24
Introduction
Background: Spectral Geometry
Approach: Isospectralization
Applications/Results
Summary
Hearing the Shape of a Drum
Isospectralization
Regularizers for ﬂat shapes
Regularizers for surfaces shapes
Flat Shape Regularizers
Encourage Small Interior Edge Lengths
ρX (V) =
eij ∈Ei
l2
ij
(V)
Note: Optimization is eﬀected by the mesh resolution and
spectral bandwidth.
Sricharan Chiruvolu Isospectralization

18/24
Introduction
Background: Spectral Geometry
Approach: Isospectralization
Applications/Results
Summary
Hearing the Shape of a Drum
Isospectralization
Regularizers for ﬂat shapes
Regularizers for surfaces shapes
Shape reconstruction on Flat Shapes
Figure 7: Shape recovery optimization for ﬂat shapes.
Sricharan Chiruvolu Isospectralization

19/24
Introduction
Background: Spectral Geometry
Approach: Isospectralization
Applications/Results
Summary
Hearing the Shape of a Drum
Isospectralization
Regularizers for ﬂat shapes
Regularizers for surfaces shapes
Surface Regularizers
Encourage Vertices to Lie Close to Barycenter of neighbors, L
is the initial graph Laplacian.
ρX (V) = Lg
0
V 2
2
Encourage small total displacement from initial embedding -
volume to grow/shrink to disambiguate isometric shapes.
ρX (V) =
n
i=1
ai vi − vi
0
2
Sricharan Chiruvolu Isospectralization

20/24
Introduction
Background: Spectral Geometry
Approach: Isospectralization
Applications/Results
Summary
Hearing the Shape of a Drum
Isospectralization
Regularizers for ﬂat shapes
Regularizers for surfaces shapes
Shape reconstruction on Surfaces
Figure 8: Two isometric shapes ("Surfaces") with diﬀerent volume; their
(identical) spectra. This is the reason we need the second Regularizer.
Sricharan Chiruvolu Isospectralization

21/24
Introduction
Background: Spectral Geometry
Approach: Isospectralization
Applications/Results
Summary
Non-isometric shape matching
Style Transfer
Non-isometric shape matching
Figure 9: Isospectralization as pre-processing for dense correspondence
matching.
Sricharan Chiruvolu Isospectralization

22/24
Introduction
Background: Spectral Geometry
Approach: Isospectralization
Applications/Results
Summary
Non-isometric shape matching
Style Transfer
Style Transfer
Figure 10: Isospectralization for Style Transfer
Sricharan Chiruvolu Isospectralization

23/24
Introduction
Background: Spectral Geometry
Approach: Isospectralization
Applications/Results
Summary
Summary
Improvement for modern shape matching problems via
addressing a decades-old problem.
Applications in various domains of geometry processing and
computer vision.
No mathematical backing on optimization yet.
No experimental proof on shape interpolation yet.
Sricharan Chiruvolu Isospectralization

24/24
Introduction
Background: Spectral Geometry
Approach: Isospectralization
Applications/Results
Summary
End
Quick Demo.
Questions.
Thanks for listening!
Sricharan Chiruvolu Isospectralization