ABEJA Innovation Meetup NIPS PointNet++

Transcript

1. PointNet++: 
 Deep Hierarchical Feature Learning on Point Sets 
 in a Metric Space NIPSಡΈձˏABEJA 1
2. PointNet++ͷ֓ཁ ▸ ஶऀ: Charles R. Qi, Li Yi, Hao Su, Leonidas J. Guibas
 ɹɹ ˏελϯϑΥʔυ ▸ ֓ཁ ▸ ఺܈Λͦͷ··ೖྗ͠ɺͦͷΫϥε෼ྨɺ
 SegmentationΛߦ͏PointNetͷվྑใࠂ ▸ PointNetͷऑ఺Ͱ͋ͬͨ఺܈ີ౓ґଘΛࠀ෰ɺ
 ͓Αͼ֊૚తͳֶशΛͰ͖ΔΑ͏ʹ
 ʮSampling Layerʯͱ ʮGrouped LayerʯΛఏҊ 2
3. എܠ ▸ ̏࣍ݩͷधཁ 3 ࣗಈӡస AR ઃܭ
4. ࿩ͷྲྀΕ ▸ എܠ ▸ PointNetʹ͍ͭͯ ▸ ख๏ ▸ ࣮ݧ ▸ ·ͱΊ 4
5. എܠ ▸ ̏࣍ݩͷσʔλ 5 ɹɹ఺܈ɹɹ ɹɹϝογϡɹɹ Voxel ౤Өɹ RGB-D
6. എܠ ▸ طଘͷख๏ ▸ ఺܈Λผͷදݱʹม׵͍ͯͨ͠ 6 Unstructured, Unordered ͳ఺܈Λͦͷ··ೖྗ
 Ͱ΋͏·͍͘͘Α͏ͳख๏ ==> PointNetΛఏҊ@CVPR2017
7. PointNetͷ͓͞Β͍ ▸ ղ͘λεΫ 7 Classification Segmentation Scene Parsing ೖྗ
8. PointNetͷ͓͞Β͍ ▸ ૚ઃܭ 8
9. PointNetͷ͓͞Β͍ ▸ ՝୊ 9 PointNet͸֤఺ʹ͓͍ͯɺlocalͷ৘ใ͕ফ͑Δ ֊૚తಛ௃ֶशͰ͖ͳ͍ ෳ਺֊૚ͷந৅ԽͰ͖ͳ͍ Globalͷಛ௃ֶशͷΈ ͋Δ఺΋͘͠͸શͯͷ఺
10. PointNetͷ͓͞Β͍ ▸ localͷ৘ใ͕ফ͑Δͱ 10 globalͷ৘ใ͸ɺઈର࠲ඪʹґଘͯ͠͠·͏ͷͰɺ segmentationͰ͸ະ஌ͷ΋ͷʹରԠͰ͖ͳ͍
11. PointNet++Ͱ͸ ▸ ֊૚తֶश ▸ localͳ৘ใΛ࢒͢ 11 ▸ ఺܈ີ౓ʹϩόετʹ
12. ΞʔΩςΫνϟ 12
13. ֊૚తͳֶश 13
14. ֊૚తͳֶश ▸ Sampling layer ▸ Farthest Point Sampling (FPS) 14 https://www.groundai.com/project/parametric-manifold-learning-via-sparse-multidimensional-scaling/
15. ▸ Grouping layer ▸ radius based ball query ֊૚తͳֶश 15 PointNet layer Convolution layer Input ΍Δԋࢉ ԋࢉͰݟΔ ൣғ Radius ball query ɹ఺܈ɹ PointNetʢॱ൪ීวʣ ߦྻʢݻఆͷϐΫηϧʣ ৞ΈࠐΈʢॱ൪ґଘʣ ɹີͳߦྻɹ
16. ֊૚తͳֶश ▸ PointNet layer 16 N1ݸͷ఺ʹର͠ C1ݸͷಛ௃࡞੒ ॏΈshare
17. ֊૚తͳֶश ▸ PointNet layer 17 x1,y1,z1,ΫΤϦ1,ಛ௃1 x2,y2,z2,ΫΤϦ2,ಛ௃2 x3,y3,z3,ΫΤϦ3,ಛ௃3 xN1,yN1,zN1,ΫΤϦN1ಛ௃N1 MLP MLP MLP MLP x1,y1,z1,ಛ௃1 x2,y2,z2,ಛ௃2 x3,y3,z3,ಛ௃3 xN1,yN1,zN1,ಛ௃N1 ॏΈShare
18. ີ౓ґଘࠀ෰ख๏ ▸ ̏࣍ݩͷଌఆͰ͸఺܈ີ౓͕Ұൠతͳ՝୊ 18 ==> ఺܈ີ౓ʹϩόετʹ͍ͨ͠
19. ີ౓ґଘࠀ෰ख๏ ▸ SamplingͱGroupingΛෳ਺༻ҙ 19 MRGͷํ͕௿࣍Ͱपลͱͷಛ௃Λर͑Δ
20. Classification ࣮ݧ 20
21. ▸ ModelNet40ʹରͯ͠ Classification ࣮ݧ 21 PointNetʹൺ΂ɺPointNet++͸෼ྨਫ਼౓޲্ CNNϕʔεͷख๏ʹউར
22. ີ౓ґଘ࣮ݧ 22 ಛʹ఺܈਺͕গͳ͍ͱɺMRG͕༗ޮ
23. Segmentation ࣮ݧ 23 ಺ૠɿɹIDW (ٯڑ཭Ճॏ) Unitpointnet: ֤఺ͰMLP
24. Segmentation ࣮ݧ ▸ ݁Ռ 24 MSGΛೖΕΔ͜ͱͰɺෆۉҰͳ఺܈Ͱ΋͏·͍͘͘
25. Segmentation ࣮ݧ ▸ ݁Ռ 25 PointNetΑΓՈ۩ͷsegmentation্͕ख͍͘͘
26. ඇϢʔΫϦου ڑ཭ۭؒͰͷ࣮ݧ 26 WKS , HKS, multi-scale Gaussian curvature
27. Feature Visualization ▸ ࠷ॳͷ૚ͷॏΈΛՄࢹԽ 27 ฏ໘ɺίʔφʔͱ͔Λֶश
28. ·ͱΊ ▸ PointNetΛ֦ுͨ͠ख๏PointNet++Λൃද ▸ CVPR2017=>NIPS2017൒೥ʹ̍ຊ௨͍ͯ͠Δɻɻɻ ▸ Sampling layerɺGrouped layerΛऔΓೖΕ֊૚తͳֶश ▸ MRGɺMSGΛఏҊ͠ɺ఺܈ີ౓ʹґଘ͠ͳֶ͍श ▸ ̏࣍ݩ఺܈ͷσʔληοτʹରͯ͠ɺSoTAୡ੒ ▸ ݱ஍ʹߦͬͨײ૝ ▸ ஶऀͱ࿩͢͜ͱͰࡉ͔ͳ৘ใΛर͑Δ 28