as “waveform” discretely. • Definition of FW data ⁃ x, y, z, and waveform • 3D point clouds and additional information regarding the target properties. The shape of waveform and power of the backscatter are related to characteristics of the targets. 3 Cited from “Urban land cover classification using airborne LiDAR data: A review”
(2019)] • Point Clouds and features from waveform • Waveform information improves the semantic segmentation task (Land Use/Land Cover classification) n2D image[Zorzi et al. (2019)] • Waveform and height information converted to 2D. • Deep Learning(CNN) based methods can easily deal with 2D images. • Spatial feature extraction improves semantic segmentation. 4 Direct and spatial feature extraction method without converting to feature or grid is needed
• Local Group ⁃ Waveform improves the segmentation performance. ‣ e.g. Vegetations have many return peaks. • Global Group ⁃ An Large area of spatial/geometric context improves the segmentation performance. ‣ e.g. Roads have a uniform distribution of similar waveforms. OBJECTIVE: We propose a deep learning-based method combining local and global feature extractions for FW data sem. seg. 5 Images are cited from Awange J., Kiema J. (2019) Light Detection And Ranging (LiDAR). In: Environmental Geoinformatics. Environmental Science and Engineering. Springer, Cham. https://doi.org/10.1007/978-3-030-03017-9_21
nDefine neighbors • Convolution needs neighbor points. 10 Providing waveform relationships in large area. nUpsample • Recovering the number of points from subsampled points. • Skip connection provides vanished high res. info..
for grouped waveform or features. • Waveform is defined as sequential data like audio signals. • 1D CNN are widely used in audio recognition. • MaxPool operation after convolution are used the same as PointNet++. 11 Calculating waveform features for grouped data.
!"# = − $ $ % $ & ' & ∗ $,& log $,& ) = 1/ ) log + ∑)*+ , ) nTraining detail • Optimizer: ADAM with learning rate 0.005 • Convolution ⁃ Radius ball=3, 5, and 15 ⁃ #neighbors = 16, 32, and 64 ⁃ #features = 256, 512, and 1024 • Hardware: P100 on TSUBAME3.0 12 Loss for Local module and global module at same time each loss is weighted cross entropy class ratio aware weight
⁃ Test: red area • Class ⁃ Ground, Build., Veg., Power Line, Trans. Tower, and Street Path nDivided into 2 group • Global group ⁃ Geometry is effective. ⁃ Ground, Build., Street Path • Local group ⁃ Local features are effective. ⁃ Veg., PowerLine, Trans.Tower 15 600 m 70 m 24 m test [Zorzi et al., 2019]
PointNet++ w/ waveform vs. PointNet++ w/o waveform • Hierarchical feature extraction ⁃ PointNet++ w/ Hierarchical vs. PointNet++ w/o Hierarchical • Local module ⁃ PointNet++ w/ local vs. PointNet++ w/o local 18 We showed the effetiveness of PointNet++ based model, waveform information, and local module
• Prediction step ⁃ Rule-based predictions using the output of local module and global module. ⁃ Local Module predicts local group (Vegetation, Power Line, Transmission Tower), which we made the assumption that the local waveform information is effective. 19 Our local module was able to predict local group
deep learning-based semantic segmentation model for full-waveform LiDAR data. • Our model consists of the local module which predicts class from each waveform, and the global module which predicts class from geometry. • Experimental results showed that our model predicts higher accuracy than previous methods. nUnsolved Problems and Future Works • Explicit Geometric information ⁃ Combine geometric and waveform feature extractions • Heuristic combination method ⁃ Changing heuristic rule to learnable functions 21