Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields

第58回 コンピュータビジョン勉強会＠関東
深層学習＋3D論文読み会
Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields

Transcript

1. Mip-NeRF 360: Unbounded Anti-
   Aliased Neural Radiance Fields
   Authors: Jonathan T. Barron, Ben Mildenhall, Dor Verbin
   Presenter: @peisuke
   

   View Slide

2. ࣗݾ঺հ
   ໊લɿ౻ຊ ܟհ
   ॴଐɿ"#&+"ʢ-BCTϥϘ௕ʣ
   ෭ۀɿ.-Λத৺ʹ૬ஊ͔Β։ൃ·Ͱ΍ͬͯ·͢
   ੋඇ͓੠͕͚Լ͍͞
   ׆ಈɿ5XJUUFS!QFJTVLF
   (JUIVC IUUQTHJUIVCDPNQFJTVLF
   2JJUB IUUQTRJJUBDPNQFJTVLF
   4MJEF4IBSF IUUQTXXXTMJEFTIBSFOFU'VKJNPUP,FJTVLF
   4QFBLFS%FDL IUUQTTQFBLFSEFDLDPNQFJTVLF
   গ͚ͩ͠ॻ͍ͯ·͢
   

   View Slide

3. ֓ཁ
   • ͲΜͳ࿦จʁ
   • /F3'Ͱ͸ϘϦϡʔϜϨϯμϦϯάʹ͓͍֤ͯ஍఺ͷ৭ɾີ౓Λ//Ͱ
   ग़ྗ͢Δ͜ͱͰը૾Λੜ੒
   • /F3'ʹ͓͍ͯɺத৺͔Βԕ͍޿େͳۭؒΛ//Ͱ࠶ݱ͢Δ͜ͱ͕೉͠
   ͘ɺத৺Ҏ֎ͷ෮ݩ͕೉͔ͬͨ͠໰୊Λղܾ
   • ޻෉఺͸ʁ
   • ۭؒΛม׵ͯ͠ԕ͍ྖҬΛখ͍͞ྖҬʹຒΊࠐΉ͜ͱͰԕ͞ͷ໰୊Λ
   ղܾ
   • ͦͷଞʹɺৠཹʹΑΔߴ଎Խɾਖ਼نԽʹΑΔϊΠζ࡟ݮͳͲΛ͍ͯ͠
   Δ
   

   View Slide

4. ख๏ͷσϞ
   

   View Slide

5. /F3'ͷ֓ཁʢʣ
   • ۭؒ಺ͷ఺ʢҐஔ޲͖ͷ࣍ݩʣʹؔ͢Δ৭ɾີ౓Λ༧ଌ͢Δ
   ωοτϫʔΫ
   • ΧϝϥͷҐஔΛܾΊΔͱɺωοτϫʔΫͷग़ྗ͢Δ஋ΛݩʹϘ
   ϦϡʔϜϨϯμϦϯά͢Δ͜ͱͰը૾Λੜ੒
   1PTJUJPOBM&ODPEJOH͞Εͨ఺ͷ
   ৘ใ ࣍ϖʔδͰઆ໌
   
   ີ౓͔Β
   XΛܭࢉ
   

   View Slide

6. /F3'ͷ֓ཁʢʣ
   • ۭؒ಺ͷ఺Λߴ࣍ݩͷಛ௃ྔʹม׵ʢQPTJUJPOBMFODPEJOHɺ
   1&ʣ͢Δ͜ͱͰߴਫ਼౓Խ
   𝛾 𝒙 = sin 2!𝒙𝜋 , cos 2!𝒙𝜋 , … , sin 2"𝒙𝜋 , cos 2"𝒙𝜋
   

   View Slide

7. NJQ/F3'ͷ֓ཁʢʣ
   • 𝛾 𝒙 Λ఺ͱͯ͠ѻ͍ɺͦͷҐஔͷ৭ΛϨϯμϦϯά͢Δͱɺ
   पғͷ৘ใΛಘΒΕͣδϟΪʔ͕ൃੜ͢Δ
   • ࢹઢΛίʔϯͰଊ͑ɺίʔϯ಺ͷੵ෼Λ༻͍ΔࣄͰߴਫ਼౓Խ
   • ίʔϯΛΨ΢ε෼෍Ͱۙࣅɺ1&ͷظ଴஋Λ࢖͏͜ͱͰɺੵ෼Λ
   ؆қతʹܭࢉ
   ఺Ͱ͸ͳ͘ɺ֤ྖҬͷ
   1&ͷظ଴஋Λ࢖͏
   

   View Slide

8. NJQ/F3'ͷ֓ཁʢʣ
   • 1&ͨ͠ಛ௃ྔͷظ଴஋ΛऔΔͱɺࢹ఺͔Βԕ͍ྖҬͰ͸ߴप೾ྖ
   Ҭ͕খ͘͞ͳΔ
   • ڑ཭ʹԠͨ͡ૈ͞ΛՃຯͯ͠ωοτϫʔΫ͕৭Λग़ྗͯ͘͠ΕΔ
   ࢹ఺ۙ͘Ͱ͸ɺ෼෍͕
   ڱ͍ͨΊߴप೾ྖҬͷ
   ৘ใ͕࢒Δ
   ࢹ఺͔Β཭ΕΔͱɺ෼
   ෍͕޿͘ͳΔͨΊ௿प
   ೾੒෼ͷΈʹͳΔ
   

   View Slide

9. NJQ/F3'ͷ՝୊
   • Χϝϥ͔Βԕ͍ྖҬ͸ίʔϯ
   ͕େ͖͘ͳΔ
   • த৺ʹ͋Δ෺ମҎ֎ͷपғ؀
   ڥͷ෮ݩ͕ࠔ೉
   த৺ྖҬ΄Ͳࡉ͔
   ͍৘ใ͕ඞཁ
   पғͷ৘ใ͸ૈͯ͘ྑ͍
   ֎ଆͷྖҬͷσʔλ͕ૄʹͳΔͨΊɺ
   ্ख͘෮ݩͰ͖ͳ͍
   

   View Slide

10. 4DFOFBOE3BZ1BSBNFUFSJ[BUJPO
    • த৺͔Β൒ܘͷྖҬ·Ͱ͸มΘΒͣ
    • ൒ܘҎ্ͷྖҬ͸ɺԕ͘ͳΔ΄Ͳڑ཭ΛॖΊɺ൒ܘͷྖҬʹ
    ແݶԕํ͕ऩ·ΔΑ͏ʹม׵
    • ࣍ۙࣅͯ͠ϠίϏߦྻΛ࢖͍Ψ΢ε෼෍Λม׵
    ਤͷପԁ͸ҎԼͰม׵
    

    View Slide

11. ৠཹΛ༻͍ͨߴ଎Խʢʣ
    • NJQ/F3'Ͱ͸ɺ·ͣ͸શҬΛૈཻ͍౓ͰϨΠτϨʔγϯά͠ɺ෺ମ͕Կ
    ॲʹ͋Δͷ͔Λେ·͔ʹ೺Ѳ
    • ࣍ʹɺ෺ମ͕ଘࡏ͢ΔҐஔʹ੍ݶ͠ɺ࠶౓ϨΠτϨʔγϯά
    • ͜ΕΛ܁Γฦ͢͜ͱͰৄࡉͳܗঢ়Λ෮ݩՄೳɺܭࢉ͕஗͍ͱ͍͏՝୊͕͋
    Δ
    

    View Slide

12. ৠཹΛ༻͍ͨߴ଎Խʢ̎ʣ
    • ૈ͍ྖҬ୳ࡧϑΣʔζͰ͸ɺ৭ͷ৘ใΛ࣋ͨͳ͍ɺΑΓখ͞ͳϞσϧ
    ʢ1SPQ.-1ʣͰ෺ମͷҐஔΛ୳ࡧ
    • ࠷ޙͷϨϯμϦϯάͷΈɺ৭ͷ৘ใΛ࣋ͭϞσϧʢ/F3'.-1ʣͰ୳ࡧ
    • ֤ϞσϧʹҰ؏ੑΛ࣋ͨͤΔͨΊɺωοτϫʔΫͷৠཹΛ༻͍Δ
    

    View Slide

13. ωοτϫʔΫͷৠཹʹ͍ͭͯ
    • /F3'.-1͔ΒಘΒΕΔ෺ମͷҐஔͷ৘ใΛ༻͍ͯɺ1SPQ.-1Λֶश
    ͢Δ͜ͱͰҰ؏ੑΛอͭ
    • ௚઀྆ํͷωοτϫʔΫؒͰඞཁͱͳΔCJOͷີ౓͸ҟͳΔͷͰ௚઀਺஋
    Λൺֱͨ͠ৠཹ͕ग़དྷͳ͍
    • /F3'.-1͔ΒಘΒΕΔॏΈͷώετάϥϜΑΓ΋ɺ1SPQ.-1ͷώε
    τάϥϜ͕େ͖͘ͱ͍͏੍໿Λ࢖͏
    • ͜Ε͸ͪΐͬͱෳࡶͳͷͰ࣍ͷϖʔδͰઆ໌
    ࠷ऴతʹɺ෺ମྖҬʹ͓
    ͍ͯX͕େ͖͘ͳΔɻ·
    ͨɺૈ͍CJOͷ1SPQ.-1
    ͕ಘΒΕΔ
    

    View Slide

14. • ϨΠ্ʹ͋Δ෺ମͷॏΈXͷώετάϥϜΛߟ͑Δ
    ώετάϥϜͷൺֱ
    ϨΠ্ͷ෺ମͷॏΈ
    w
    ෺ମͷॏΈͷώετάϥϜ
    

    View Slide

15. ώετάϥϜͷൺֱ
    /F3'.-1ͷ
    ώετάϥϜ
    1SPQ.-1ͷ
    ώετάϥϜ /F3'.-1ͷͱ͋ΔྖҬ5JͷॏΈXJ
    ΑΓ΋ɺ5JΛؚΉྖҬ5Kͷώετά
    ϥϜXKͷ໘ੵͷํ͕େ͖͘ͳΔ
    

    View Slide

16. ਖ਼ଇԽ
    • /F3'Λֶश͢Δͱɺಛʹԕ͍ΤϦΞʹ͓͍ͯ͸ܗঢ়෮ݩͷͨ
    Ίͷ৘ใ͕଍Γͣෆྑઃఆ໰୊͕ൃੜ͠ɺϊΠζ͕ൃੜ
    • ਖ਼ଇԽʹΑͬͯϊΠζͷ࡟ݮΛߦ͏
    

    View Slide

17. ਖ਼ଇԽ
    • ॏΈ͆ͷ෼෍͕ͳΔ΂͘σϧλؔ਺ʹۙ͘ͳΔΑ͏ͳ੍໿Λͭ
    ͚Δ
    σϧλؔ਺ʹͳΔͱɺ
    X V
    ɺX W
    ͕ڞʹͷ࣌
    ʹɺVW͕ʹ͖ۙͮɺͦ
    ΕҎ֎͸ɺΑ্ͬͯࣜ͸
    θϩʹۙͮ͘
    𝑢 ≠ 𝑣ͷ࣌ʹɺw 𝑢 , 𝑤 𝑣 > 0
    Ͱ͋ΔͱϖφϧςΟͱͳΔ
    

    View Slide

18. ਖ਼ଇԽ
    • ઌఔͷੵ෼͸௚઀ܭࢉͰ͖ͳ͍͕ɺX͸֤ΠϯλʔόϧͰҰఆ
    ஋ʹͳΔͷͰɺҎԼͷΑ͏ʹॻ͖දͤΔ
    • ޯ഑Λܭࢉ͢ΔͱɺਤͷΑ͏ͳํ޲ʹॏΈ͕มԽ͢Δྗ͕ಇ͘
    

    View Slide

19. ࠷దԽ
    • ը૾ͷ෮ݩϩεɺਖ਼ଇԽʹΑͬͯ/F3'.-1Λɺ࠷ޙͷ߲Ͱ
    1SPQ.-1Λֶश
    • Ϟσϧ
    • 1SPQ.-1͸ύϥϝʔλͷӅΕ૚Λ࣋ͭɺϨΠϠʔͷ.-1
    • /F3'.-1͸ύϥϝʔλͷӅΕ૚Λ࣋ͭɺϨΠϠʔͷ.-1
    ը૾ͷ෮ݩϩε ਖ਼ଇԽ߲ 1SPQ.-1ͷֶश
    

    View Slide

20. ࣮ݧ݁Ռ
    • σʔλ
    • ৽͍͠σʔληοτɺͭͷ԰֎ɺͭͷ԰಺
    • ৚݅
    • /F3' NJQ/F3' /F3'ͱൺֱ
    • $0-."1ͰࣗݾҐஔਪఆ
    • ܭࢉ؀ڥ
    • 516WXJUIDPSFT
    • ࢦඪ
    • 14/3 44*. -1*14
    

    View Slide

21. ఆྔධՁ
    

    View Slide

22. ఆੑධՁ
    

    View Slide

23. ·ͱΊ
    • NJQ/F3'ͷ՝୊Ͱ͋ͬͨԕํͷྖҬΛਫ਼౓ྑ͘෮ݩ͢Δख๏
    ΛఏҊ
    • 4DFOFBOE3BZ1BSBNFUFSJ[BUJPOʹΑΔԕํ෮ݩߴਫ਼౓Խɺ
    ৠཹʹΑΔߴ଎Խɺਖ਼ଇԽʹΑΔϊΠζআڈΛఏҊ
    

    View Slide