Slide 1

Slide 1 text

PointNet++: 
 Deep Hierarchical Feature Learning on Point Sets 
 in a Metric Space NIPSಡΈձˏABEJA 1

Slide 2

Slide 2 text

PointNet++ͷ֓ཁ ▸ ஶऀ: Charles R. Qi, Li Yi, Hao Su, Leonidas J. Guibas
 ɹɹ ˏελϯϑΥʔυ ▸ ֓ཁ ▸ ఺܈Λͦͷ··ೖྗ͠ɺͦͷΫϥε෼ྨɺ
 SegmentationΛߦ͏PointNetͷվྑใࠂ ▸ PointNetͷऑ఺Ͱ͋ͬͨ఺܈ີ౓ґଘΛࠀ෰ɺ
 ͓Αͼ֊૚తͳֶशΛͰ͖ΔΑ͏ʹ
 ʮSampling Layerʯͱ ʮGrouped LayerʯΛఏҊ 2

Slide 3

Slide 3 text

എܠ ▸ ̏࣍ݩͷधཁ 3 ࣗಈӡస AR ઃܭ

Slide 4

Slide 4 text

࿩ͷྲྀΕ ▸ എܠ ▸ PointNetʹ͍ͭͯ ▸ ख๏ ▸ ࣮ݧ ▸ ·ͱΊ 4

Slide 5

Slide 5 text

എܠ ▸ ̏࣍ݩͷσʔλ 5 ɹɹ఺܈ɹɹ ɹɹϝογϡɹɹ Voxel ౤Өɹ RGB-D

Slide 6

Slide 6 text

എܠ ▸ طଘͷख๏ ▸ ఺܈Λผͷදݱʹม׵͍ͯͨ͠ 6 Unstructured, Unordered ͳ఺܈Λͦͷ··ೖྗ
 Ͱ΋͏·͍͘͘Α͏ͳख๏ ==> PointNetΛఏҊ@CVPR2017

Slide 7

Slide 7 text

PointNetͷ͓͞Β͍ ▸ ղ͘λεΫ 7 Classification Segmentation Scene Parsing ೖྗ

Slide 8

Slide 8 text

PointNetͷ͓͞Β͍ ▸ ૚ઃܭ 8

Slide 9

Slide 9 text

PointNetͷ͓͞Β͍ ▸ ՝୊ 9 PointNet͸֤఺ʹ͓͍ͯɺlocalͷ৘ใ͕ফ͑Δ ֊૚తಛ௃ֶशͰ͖ͳ͍ ෳ਺֊૚ͷந৅ԽͰ͖ͳ͍ Globalͷಛ௃ֶशͷΈ ͋Δ఺΋͘͠͸શͯͷ఺

Slide 10

Slide 10 text

PointNetͷ͓͞Β͍ ▸ localͷ৘ใ͕ফ͑Δͱ 10 globalͷ৘ใ͸ɺઈର࠲ඪʹґଘͯ͠͠·͏ͷͰɺ segmentationͰ͸ະ஌ͷ΋ͷʹରԠͰ͖ͳ͍

Slide 11

Slide 11 text

PointNet++Ͱ͸ ▸ ֊૚తֶश ▸ localͳ৘ใΛ࢒͢ 11 ▸ ఺܈ີ౓ʹϩόετʹ

Slide 12

Slide 12 text

ΞʔΩςΫνϟ 12

Slide 13

Slide 13 text

֊૚తͳֶश 13

Slide 14

Slide 14 text

֊૚తͳֶश ▸ Sampling layer ▸ Farthest Point Sampling (FPS) 14 https://www.groundai.com/project/parametric-manifold-learning-via-sparse-multidimensional-scaling/

Slide 15

Slide 15 text

▸ Grouping layer ▸ radius based ball query ֊૚తͳֶश 15 PointNet layer Convolution layer Input ΍Δԋࢉ ԋࢉͰݟΔ ൣғ Radius ball query ɹ఺܈ɹ PointNetʢॱ൪ීวʣ ߦྻʢݻఆͷϐΫηϧʣ ৞ΈࠐΈʢॱ൪ґଘʣ ɹີͳߦྻɹ

Slide 16

Slide 16 text

֊૚తͳֶश ▸ PointNet layer 16 N1ݸͷ఺ʹର͠ C1ݸͷಛ௃࡞੒ ॏΈshare

Slide 17

Slide 17 text

֊૚తͳֶश ▸ PointNet layer 17 x1,y1,z1,ΫΤϦ1,ಛ௃1 x2,y2,z2,ΫΤϦ2,ಛ௃2 x3,y3,z3,ΫΤϦ3,ಛ௃3 xN1,yN1,zN1,ΫΤϦN1ಛ௃N1 MLP MLP MLP MLP x1,y1,z1,ಛ௃1 x2,y2,z2,ಛ௃2 x3,y3,z3,ಛ௃3 xN1,yN1,zN1,ಛ௃N1 ॏΈShare

Slide 18

Slide 18 text

ີ౓ґଘࠀ෰ख๏ ▸ ̏࣍ݩͷଌఆͰ͸఺܈ີ౓͕Ұൠతͳ՝୊ 18 ==> ఺܈ີ౓ʹϩόετʹ͍ͨ͠

Slide 19

Slide 19 text

ີ౓ґଘࠀ෰ख๏ ▸ SamplingͱGroupingΛෳ਺༻ҙ 19 MRGͷํ͕௿࣍Ͱपลͱͷಛ௃Λर͑Δ

Slide 20

Slide 20 text

Classification ࣮ݧ 20

Slide 21

Slide 21 text

▸ ModelNet40ʹରͯ͠ Classification ࣮ݧ 21 PointNetʹൺ΂ɺPointNet++͸෼ྨਫ਼౓޲্ CNNϕʔεͷख๏ʹউར

Slide 22

Slide 22 text

ີ౓ґଘ࣮ݧ 22 ಛʹ఺܈਺͕গͳ͍ͱɺMRG͕༗ޮ

Slide 23

Slide 23 text

Segmentation ࣮ݧ 23 ಺ૠɿɹIDW (ٯڑ཭Ճॏ) Unitpointnet: ֤఺ͰMLP

Slide 24

Slide 24 text

Segmentation ࣮ݧ ▸ ݁Ռ 24 MSGΛೖΕΔ͜ͱͰɺෆۉҰͳ఺܈Ͱ΋͏·͍͘͘

Slide 25

Slide 25 text

Segmentation ࣮ݧ ▸ ݁Ռ 25 PointNetΑΓՈ۩ͷsegmentation্͕ख͍͘͘

Slide 26

Slide 26 text

ඇϢʔΫϦου ڑ཭ۭؒͰͷ࣮ݧ 26 WKS , HKS, multi-scale Gaussian curvature

Slide 27

Slide 27 text

Feature Visualization ▸ ࠷ॳͷ૚ͷॏΈΛՄࢹԽ 27 ฏ໘ɺίʔφʔͱ͔Λֶश

Slide 28

Slide 28 text

·ͱΊ ▸ PointNetΛ֦ுͨ͠ख๏PointNet++Λൃද ▸ CVPR2017=>NIPS2017൒೥ʹ̍ຊ௨͍ͯ͠Δɻɻɻ ▸ Sampling layerɺGrouped layerΛऔΓೖΕ֊૚తͳֶश ▸ MRGɺMSGΛఏҊ͠ɺ఺܈ີ౓ʹґଘ͠ͳֶ͍श ▸ ̏࣍ݩ఺܈ͷσʔληοτʹରͯ͠ɺSoTAୡ੒ ▸ ݱ஍ʹߦͬͨײ૝ ▸ ஶऀͱ࿩͢͜ͱͰࡉ͔ͳ৘ใΛर͑Δ 28