Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Isospectralization

Sricharan
June 24, 2020

 Isospectralization

Isospectralization, or how to hear shape, style, and correspondence - A Review.

Sricharan

June 24, 2020
Tweet

More Decks by Sricharan

Other Decks in Research

Transcript

  1. 1/24
    Introduction
    Background: Spectral Geometry
    Approach: Isospectralization
    Applications/Results
    Summary
    Isospectralization, or how to hear shape,
    style, and correspondence
    Sricharan Chiruvolu
    Zorah Lähner
    June 24, 2020
    Sricharan Chiruvolu Isospectralization

    View Slide

  2. 2/24
    Introduction
    Background: Spectral Geometry
    Approach: Isospectralization
    Applications/Results
    Summary
    Introduction
    Published in CVPR 2019.
    Explores the practical possibility of using the spectrum of an
    object for shape reconstruction.
    Proposes an inverse mapping between geometric domain and
    its Laplacian via simple regularizers.
    Isospectralization: A numerical optimization procedure to
    deform a mesh in order to align its (finite) Laplacian spectrum
    with a given one.
    Sricharan Chiruvolu Isospectralization

    View Slide

  3. 3/24
    Introduction
    Background: Spectral Geometry
    Approach: Isospectralization
    Applications/Results
    Summary
    Isospectralization
    Figure 1: Isospectralization
    Sricharan Chiruvolu Isospectralization

    View Slide

  4. 4/24
    Introduction
    Background: Spectral Geometry
    Approach: Isospectralization
    Applications/Results
    Summary
    Riemannian Manifold and Geodesics
    Isometry and Isomorphism
    Laplace-Beltrami Operator
    Direct and Inverse Problems
    Spectral Geometry
    Relationship between geometric structures of manifolds and
    spectra of canonically defined differential operators.
    Eigenvalue problems involving the Laplace operator on
    manifolds have proven to be a consistently fertile area of
    geometric analysis with deep connections to number theory,
    physics, and applied mathematics.
    Sricharan Chiruvolu Isospectralization

    View Slide

  5. 5/24
    Introduction
    Background: Spectral Geometry
    Approach: Isospectralization
    Applications/Results
    Summary
    Riemannian Manifold and Geodesics
    Isometry and Isomorphism
    Laplace-Beltrami Operator
    Direct and Inverse Problems
    Manifolds
    (As discussed in previous talks) In computer vision, the notion of a
    manifold is used quite frequently.
    in rotation averaging (SO3),
    structure and motion (“Essential” manifold)
    to capture the shape of an object (“Shape” manifolds)
    to model a set of images (“Grassman” manifolds)
    to simply represent a sphere (Sn).
    ...
    Sricharan Chiruvolu Isospectralization

    View Slide

  6. 6/24
    Introduction
    Background: Spectral Geometry
    Approach: Isospectralization
    Applications/Results
    Summary
    Riemannian Manifold and Geodesics
    Isometry and Isomorphism
    Laplace-Beltrami Operator
    Direct and Inverse Problems
    Riemannian Manifold and Geodesics
    (As discussed in previous talks)
    A Riemannian manifold is a manifold with an inner product
    defined in the tangent space at each point.
    Geodesics are locally shortest curves. They preserve a
    direction on a surface and have many interesting properties.
    Sricharan Chiruvolu Isospectralization

    View Slide

  7. 7/24
    Introduction
    Background: Spectral Geometry
    Approach: Isospectralization
    Applications/Results
    Summary
    Riemannian Manifold and Geodesics
    Isometry and Isomorphism
    Laplace-Beltrami Operator
    Direct and Inverse Problems
    Riemannian Manifold and Geodesics
    Figure 2: Geodesic and tangent space in a Riemannian manifold. 1
    1Medinria: DT-MRI processing and visualization software; Xavier Pennec.
    Sricharan Chiruvolu Isospectralization

    View Slide

  8. 8/24
    Introduction
    Background: Spectral Geometry
    Approach: Isospectralization
    Applications/Results
    Summary
    Riemannian Manifold and Geodesics
    Isometry and Isomorphism
    Laplace-Beltrami Operator
    Direct and Inverse Problems
    Isometry and Isomorphism
    Figure 3: Isometry and Isomorphism.2
    2Smooth Shells: Multi-Scale Shape Registration with Functional Maps;
    Eisenberger, Lahner and Cremers.
    Sricharan Chiruvolu Isospectralization

    View Slide

  9. 9/24
    Introduction
    Background: Spectral Geometry
    Approach: Isospectralization
    Applications/Results
    Summary
    Riemannian Manifold and Geodesics
    Isometry and Isomorphism
    Laplace-Beltrami Operator
    Direct and Inverse Problems
    Laplace-Beltrami Operator
    Figure 4: Laplace-Beltrami operator 3
    Laplacian ( f ) = div(gradient(f )).
    The Laplacian is generalized to the Riemannian manifold
    (M; g) by the Laplace-Beltrami operator ( g ).
    3Khan Academy; Grant Sanderson.
    Sricharan Chiruvolu Isospectralization

    View Slide

  10. 10/24
    Introduction
    Background: Spectral Geometry
    Approach: Isospectralization
    Applications/Results
    Summary
    Riemannian Manifold and Geodesics
    Isometry and Isomorphism
    Laplace-Beltrami Operator
    Direct and Inverse Problems
    Isospectrals
    Given a compact Riemannian manifold, we can associate to it
    the Laplace-Beltrami operator.
    Two closed Riemannian manifolds are said to be isospectral
    if the eigenvalues of their Laplace–Beltrami operator
    (Laplacians) coincide.
    Thus, spectral geometry is the connection between the
    spectrum Spec(M; g), i.e. the eigenvalues, and the geometry
    of the manifold (M; g).
    This fundamentally deals with two kinds of problems:
    Direct problems
    Inverse problems
    Sricharan Chiruvolu Isospectralization

    View Slide

  11. 11/24
    Introduction
    Background: Spectral Geometry
    Approach: Isospectralization
    Applications/Results
    Summary
    Riemannian Manifold and Geodesics
    Isometry and Isomorphism
    Laplace-Beltrami Operator
    Direct and Inverse Problems
    Direct and Inverse Problems
    Direct Problems
    Direct problems attempt to infer the behavior of the eigenvalues of
    a Riemannian manifold from knowledge of the geometry.
    Compute (exactly or not) the spectrum Spec(M, g)? And (or) find
    properties on the spectrum Spec(M, g)?
    Inverse Problems
    Inverse problem - does the data of the spectrum Spec(M, g)
    determine the "shape" of the manifold (M, g) ?
    Which geometric information of the manifold can we determine
    from the spectrum?
    Sricharan Chiruvolu Isospectralization

    View Slide

  12. 12/24
    Introduction
    Background: Spectral Geometry
    Approach: Isospectralization
    Applications/Results
    Summary
    Hearing the Shape of a Drum
    Isospectralization
    Regularizers for flat shapes
    Regularizers for surfaces shapes
    Hearing the Shape of a Drum
    Figure 5: Membranes of these two different shapes (but otherwise
    identical) would "sound" the same.
    "Can one hear the shape of a drum?"
    Mark Kac’s 1966 question, "Can one hear the shape of a drum?"
    set off research into spectral theory, with the idea of understanding
    the extent to which the spectrum allows one to read back the
    geometry.
    Sricharan Chiruvolu Isospectralization

    View Slide

  13. 13/24
    Introduction
    Background: Spectral Geometry
    Approach: Isospectralization
    Applications/Results
    Summary
    Hearing the Shape of a Drum
    Isospectralization
    Regularizers for flat shapes
    Regularizers for surfaces shapes
    Hearing the Shape of a Drum
    Theoretically Negative. Why? - There exists isospectral
    manifolds that are nonisometric.
    With additional priors, practically feasible. We can recover the
    structure of an object from its spectrum.
    Sricharan Chiruvolu Isospectralization

    View Slide

  14. 14/24
    Introduction
    Background: Spectral Geometry
    Approach: Isospectralization
    Applications/Results
    Summary
    Hearing the Shape of a Drum
    Isospectralization
    Regularizers for flat shapes
    Regularizers for surfaces shapes
    Notations
    Figure 6: Notations
    Discretization.
    Sricharan Chiruvolu Isospectralization

    View Slide

  15. 15/24
    Introduction
    Background: Spectral Geometry
    Approach: Isospectralization
    Applications/Results
    Summary
    Hearing the Shape of a Drum
    Isospectralization
    Regularizers for flat shapes
    Regularizers for surfaces shapes
    Isospectralization
    Find the optimal embedding that aligns the spectrum to
    another:
    min
    V∈Rn×d
    λ (∆X (V)) − µ ω
    + ρX (V)
    V is the (unknown) embedding of the mesh vertices in R.
    ∆X (V) is the associated discrete Laplacian.
    Sricharan Chiruvolu Isospectralization

    View Slide

  16. 16/24
    Introduction
    Background: Spectral Geometry
    Approach: Isospectralization
    Applications/Results
    Summary
    Hearing the Shape of a Drum
    Isospectralization
    Regularizers for flat shapes
    Regularizers for surfaces shapes
    Flat Shape Regularizers
    Encourage Smaller Boundary Edge Lengths
    ρX (V) =
    eij ∈Eb
    lij (V)
    Penalize Triangle Flips
    ρX (V ) = (min (0, rX (V )))2
    rX (V ) =
    ijk∈F
    Rπ/22 vj − vi T
    vk − vi
    Sricharan Chiruvolu Isospectralization

    View Slide

  17. 17/24
    Introduction
    Background: Spectral Geometry
    Approach: Isospectralization
    Applications/Results
    Summary
    Hearing the Shape of a Drum
    Isospectralization
    Regularizers for flat shapes
    Regularizers for surfaces shapes
    Flat Shape Regularizers
    Encourage Small Interior Edge Lengths
    ρX (V) =
    eij ∈Ei
    l2
    ij
    (V)
    Note: Optimization is effected by the mesh resolution and
    spectral bandwidth.
    Sricharan Chiruvolu Isospectralization

    View Slide

  18. 18/24
    Introduction
    Background: Spectral Geometry
    Approach: Isospectralization
    Applications/Results
    Summary
    Hearing the Shape of a Drum
    Isospectralization
    Regularizers for flat shapes
    Regularizers for surfaces shapes
    Shape reconstruction on Flat Shapes
    Figure 7: Shape recovery optimization for flat shapes.
    Sricharan Chiruvolu Isospectralization

    View Slide

  19. 19/24
    Introduction
    Background: Spectral Geometry
    Approach: Isospectralization
    Applications/Results
    Summary
    Hearing the Shape of a Drum
    Isospectralization
    Regularizers for flat shapes
    Regularizers for surfaces shapes
    Surface Regularizers
    Encourage Vertices to Lie Close to Barycenter of neighbors, L
    is the initial graph Laplacian.
    ρX (V) = Lg
    0
    V 2
    2
    Encourage small total displacement from initial embedding -
    volume to grow/shrink to disambiguate isometric shapes.
    ρX (V) =
    n
    i=1
    ai vi − vi
    0
    2
    Sricharan Chiruvolu Isospectralization

    View Slide

  20. 20/24
    Introduction
    Background: Spectral Geometry
    Approach: Isospectralization
    Applications/Results
    Summary
    Hearing the Shape of a Drum
    Isospectralization
    Regularizers for flat shapes
    Regularizers for surfaces shapes
    Shape reconstruction on Surfaces
    Figure 8: Two isometric shapes ("Surfaces") with different volume; their
    (identical) spectra. This is the reason we need the second Regularizer.
    Sricharan Chiruvolu Isospectralization

    View Slide

  21. 21/24
    Introduction
    Background: Spectral Geometry
    Approach: Isospectralization
    Applications/Results
    Summary
    Non-isometric shape matching
    Style Transfer
    Non-isometric shape matching
    Figure 9: Isospectralization as pre-processing for dense correspondence
    matching.
    Sricharan Chiruvolu Isospectralization

    View Slide

  22. 22/24
    Introduction
    Background: Spectral Geometry
    Approach: Isospectralization
    Applications/Results
    Summary
    Non-isometric shape matching
    Style Transfer
    Style Transfer
    Figure 10: Isospectralization for Style Transfer
    Sricharan Chiruvolu Isospectralization

    View Slide

  23. 23/24
    Introduction
    Background: Spectral Geometry
    Approach: Isospectralization
    Applications/Results
    Summary
    Summary
    Improvement for modern shape matching problems via
    addressing a decades-old problem.
    Applications in various domains of geometry processing and
    computer vision.
    No mathematical backing on optimization yet.
    No experimental proof on shape interpolation yet.
    Sricharan Chiruvolu Isospectralization

    View Slide

  24. 24/24
    Introduction
    Background: Spectral Geometry
    Approach: Isospectralization
    Applications/Results
    Summary
    End
    Quick Demo.
    Questions.
    Thanks for listening!
    Sricharan Chiruvolu Isospectralization

    View Slide