Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Do Deep Neural Networks Learn Full Waveform LiDAR Data?

teddy
November 18, 2019

Do Deep Neural Networks Learn Full Waveform LiDAR Data?

I have talked about raw full waveform lidar data analysis using deep learning method called pointnet at Joint Student Seminar on Remote Sensing and Geoinformatics.

teddy

November 18, 2019
Tweet

More Decks by teddy

Other Decks in Research

Transcript

  1. Do Deep Neural Networks Learn
    Full Waveform LiDAR Data?
    Takayuki SHINOHARA
    Matsuoka Laboratory, Tokyo Institute of Technology
    2019/11/18 Joint Student Seminar on Remote Sensing and Geoinformatics

    View full-size slide

  2. Background and Objective
    2

    View full-size slide

  3. Full Waveform LiDAR
    nAdvantages
    • Recording the entire reflected
    signal “waveform” discretely.
    • Providing not only 3D point
    clouds, but also additional
    information
    about the target properties.
    nData Analysis
    • Costly and time consuming at
    manual processing.
    Automatic analysis method for "raw” waveform data is needed.
    3
    Cited from “Urban land cover classification using airborne LiDAR data: A review”

    View full-size slide

  4. Raw Waveform analysis
    nDeep learning (S. Zorzi et al., 2019)
    • 2 stages supervised classification
    ‣ 1st stage: Waveform analysis by 1D CNN
    ⁃ Highly miss classification
    ‣ 2nd stage: Spatial analysis by 2D CNN
    ⁃ Raw classification results are converted
    to grid data
    ⁃ Prediction the class each grid.
    • Problems
    ‣ Spatial raw waveforms are not used
    in 1st stage.
    Þ Spatial learning method for
    raw waveform data are needed
    to improve performance. 4
    Waveform
    Spatial

    View full-size slide

  5. Solve problem
    nPoint Net(Qi et.al, 2017)
    • One of the deep learning method for spatially irregular data.
    • Input order invariant network.
    nAuto Encoder as representation learning
    • Representation learning is data driven feature extraction method.
    • Auto Encoder is one of the method.
    • Auto Encoder can extract low dimensional latent vector from high
    dimensional data such as Image or some spatial data.
    Objective in this study:
    Using a deep learning method,
    a new representation learning method
    for spatially distributed raw full-waveform data.
    5

    View full-size slide

  6. Proposed Method
    6

    View full-size slide

  7. Proposed network (FWNetAE)
    nAuto Encoder for full waveform data
    • Encoder: PointNet based
    • Decoder: simple Multi Layer Perceptron(MLP)
    7
    62
    2,048

    1D Conv
    features
    features
    features
    features
    features
    features

    Input
    full waveform
    LiDAR
    Data
    Output
    full-waveform
    LiDAR
    Data
    Max Pool MLP 62
    2,048
    Latent vector
    PointNet based Encoder Decoder
    T-nets

    View full-size slide

  8. Proposed network (FWNetAE)
    nEncoder: Point Net based Architecture
    • 1D CNN: Extract local features
    • MaxPool: Extract grobal features
    • T-nets: Extract rotation invariant features
    8
    62
    2,048

    1D Conv
    features
    features
    features
    features
    features
    features

    Input
    full waveform
    LiDAR
    Data
    Output
    full-waveform
    LiDAR
    Data
    Max Pool MLP 62
    2,048
    Latent vector
    PointNet based Encoder Decoder
    T-nets

    View full-size slide

  9. Proposed network (FWNetAE)
    nDecoder: MLP based architecture
    • Fully connected layers to produce reconstructed data !
    of 2,048 × 62
    dimensions, i.e., the same as those of input data
    9
    62
    2,048

    1D Conv
    features
    features
    features
    features
    features
    features

    Input
    full waveform
    LiDAR
    Data
    Output
    full-waveform
    LiDAR
    Data
    Max Pool MLP 62
    2,048
    Latent vector
    PointNet based Encoder Decoder
    T-nets

    View full-size slide

  10. Loss function
    nLoss function for reconstruction
    • FWNetAE aims at reconstructing target reconstructed data !
    .
    • Reconstruction loss functions.
    ‣ Spatial matching loss
    ‣ Waveform reconstruction
    10
    %&'()
    =
    1
    2N
    .
    /01
    2
    3 /
    − 6
    /
    7
    + /
    − 6
    /
    7), 1
    <'=)>?@A
    =
    1
    2N
    .
    /01
    2
    .
    B01
    C
    /,B
    − ̂
    /,B
    7
    . 2
    Optimization process
    Minimize these function

    View full-size slide

  11. Experimental Results
    11

    View full-size slide

  12. Dataset and Implementation
    nDataset
    • Dublin City Dataset
    ‣ Sample Size:
    ⁃ Train, Val, Test: 300,000, 100,000 100,000
    nImplementation and training results
    • Implementation
    ‣ Framework: PyTorch
    ‣ Hardware: 7th generation 3.8 GHz i7 CPU, 32 GB of RAM,
    and a Quadro P6000 GPU
    • Result
    ‣ Training time: approximately 9 hours
    ‣ Reconstruction error(test data): spatial=0.051, waveform=0.29
    12
    Used data

    View full-size slide

  13. Reconstruction Results 1/2
    nSpatial reconstruction
    13
    A matching shape was observed.
    27
    Failure case

    View full-size slide

  14. Reconstruction Results 2/2
    nWaveform reconstruction
    14
    28
    Failure case
    A matching shape was observed.

    View full-size slide

  15. Latent Space Visualization
    nComparison of some method
    15
    Nonspatial AE
    PCA Proposed method
    (FWNetAE)
    Learnable algorithms
    Not Learnable

    View full-size slide

  16. Conclusion and Future Study
    16

    View full-size slide

  17. Conclusions and Future Study
    nConclusions
    • This paper presents a novel representation learning method for
    spatially distributed full-waveform data observed from an ALS using an
    AE-based architecture called FWNetAE.
    • The results demonstrate a generalization error for invisible test data.
    • Moreover, the FWNetAE encoded a meaningful latent vector and the
    decoders reconstructed the spatial geometry and its waveform value
    from the encoded latent vector.
    • However, the PointNet-based encoders could not extract features at
    various resolutions.
    nFuture Study
    • Modern Hieratical learning: PointNet++, Dynamic Graph CNN
    • Application for Supervised Learning
    17

    View full-size slide

  18. Do Deep Neural Networks Learn
    Full Waveform LiDAR Data?
    Yes, they do.
    18

    View full-size slide