ライトたくさんReSTIRを実装しよう

2020年に発表された論文「Spatiotemporal reservoir resampling for real-time ray tracing with dynamic direct lighting」を実装してみます
これは2024年8月10日に行われた Kernel/VM探検隊@東京 No17 での発表資料です
発表動画:
ソースコード: https://github.com/Fadis/gct/tree/kernelvm_20240810

Transcript

1. ϥΠτͨ͘͞Μ ReSTIR Λ࣮૷͠Α͏ NAOMASA MATSUBAYASHI
2. ίϯϐϡʔλάϥϑΟΫε ͜Μͳܗͷ෺ମ͕ ͜ͷลʹ͋ͬͯ
3. ͦΕΛ͜ͷล͔Βݟͨ࣌ εΫϦʔϯͷ֤ϐΫηϧ͸ Կ৭ʹͳΓ·͔͢? ίϯϐϡʔλάϥϑΟΫε
4. ίϯϐϡʔλάϥϑΟΫε p x ࢹ఺ͱ͋ΔϐΫηϧ Λ௨Γ ෺ମͷද໘ͱͿ͔ͭΔҐஔ p x
5. ίϯϐϡʔλάϥϑΟΫε p x ͔Βࢹ఺ʹ޲͔ͬͯඈΜͰ͘Δ ޫͷΤωϧΪʔͰ ͷ৭͕ܾ·Δ x p
6. ίϯϐϡʔλάϥϑΟΫε ޫݯ͔ΒඈΜͰ͖ͨޫ͕ ෺ମͷද໘ʹͿ͔ͭΓ ͞·͟·ͳํ޲ʹ޲͖Λม͑Δ
7. ίϯϐϡʔλάϥϑΟΫε p x ޫݯ͔Β ʹඈΜͰ͖ͨޫͷ͏ͪ ෺ମͷද໘Ͱ޲͖Λม͑ͨ݁Ռ ͷํ޲ʹඈΜͰ͖ͨ෼Ͱ ͷ৭͕ܾ·Δ x p p
8. ϨϯμϦϯάํఔࣜ p ωi ωo x n ํ޲ʹඈΜͰ͘Δޫ ωo Lo (x, ωo) = Le (x, ωo) + ∫ S2 fs (x, ωi , ωo) Li (x, ωi)|ωi ⋅ n|dωi ෺ମ͔Β ํ޲ʹ ൃͤΒΕΔޫ ωo ͔Βೖ͖ͬͯͨޫͷ͏ͪ ํ޲ʹ޲͖Λม͑Δޫͷׂ߹ ωi ωo ͔Βೖͬͯ͘Δޫ ωi Λ޲͍ͨ໘͕ड͚ΒΕΔ ޫͷׂ߹ n
9. p Lo (x, ωo) = Le (x, ωo) + ∫ S2 fs (x, ωi , ωo) Li (x, ωi)|ωi ⋅ n|dωi ωi ωo x n Λத৺ͱ͢Δશͯͷํ޲ʹ͍ͭͯͷੵ෼ x ϦΞϧλΠϜϨϯμϦϯάΛ͢Δʹ͸ ͜ͷ਺஋ੵ෼Λճආ͍ͨ͠ ϨϯμϦϯάํఔࣜ
10. p Lo (x, ωo) = Le (x, ωo) + ∫ S2 fs (x, ωi , ωo) Li (x, ωi)|ωi ⋅ n|dωi x l Lo (x, ωo) = Le (x, ωo) + fs x, l − x l − x , ωo L l − x l − x ⋅ n །Ұͷޫݯ͕఺ͩͱ͢Δͱੵ෼ΛճආͰ͖Δ ͔Β ʹಧ͘ޫͷΤωϧΪʔ l x ఺ޫݯ
11. p x l0 Lo (x, ωo) = Le (x, ωo) + fs x, l0 − x l0 − x , ωo L0 l0 − x l0 − x ⋅ n + fs x, l1 − x l1 − x , ωo L1 l1 − x l1 − x ⋅ n ఺ޫݯ͕ෳ਺͋ͬͨΒ l1 ͦΕͧΕͷ఺ޫݯͷӨڹΛܭࢉͯ͠଍͢ ܭࢉίετ͕఺ޫݯͷ਺ʹൺྫͯ͠૿͑Δ
12. x x ன ໷ ౎ࢢͷ໷ܠͷΑ͏ͳγʔϯ͸ େྔͷޫݯʹরΒ͞Ε͍ͯΔ
13. ΤωϧΪʔ: ڑ཭: ද໘ੵ: ୯Ґ໘ੵ͋ͨΓͷΤωϧΪʔ: L 1 4π L 4π ΤωϧΪʔ: ڑ཭: ද໘ੵ: ୯Ґ໘ੵ͋ͨΓͷΤωϧΪʔ: L 2 16π L 16π ΤωϧΪʔ: ڑ཭: ද໘ੵ: ୯Ґ໘ੵ͋ͨΓͷΤωϧΪʔ: L 3 36π L 36π ఺ޫݯ͔ΒͷޫͷΤωϧΪʔ͸ ڑ཭ͷ2৐ʹ൓ൺྫ͢Δ ٯೋ৐ͷ๏ଇ
14. x ͜ͷลͷޫݯ͕ਖ਼͘͠ܭࢉͰ͖͍ͯΕ͹ େମ͍͋ͬͯΔޫͷΤωϧΪʔ͕ٻ·Δ
15. x ۙ͘ʹ͋ΔޫݯͰ͋ͬͯ΋ ޫݯͱ ͷؒʹःṭ෺͕͋Δ৔߹ ޫݯ͔Βͷ௚઀ޫ͸ಧ͔ͳ͍ x
16. ϥΠτΧϦϯά γʔϯͷ֤෦෼ʹ ಛʹେ͖ͳӨڹΛ༩͑Δޫݯ਺ݸΛ ࣄલʹٻΊ͓ͯ͘
17. γʔϯͷ֤෦෼ʹ ಛʹେ͖ͳӨڹΛ༩͑Δޫݯ਺ݸΛ ࣄલʹٻΊ͓ͯ͘ ःṭ෺ͷӨڹΛௐ΂Δҝߴίετ ःṭ෺͕มܗͨ͠Γޫݯ͕Ҡಈͨ͠Γ͢Δͱ ௥ैͰ͖ͳ͍ ϥΠτΧϦϯά
18. ୔ࢁͷޫݯʹরΒ͞ΕΔγʔϯΛඳ͖͍ͨ ޫݯ΍γʔϯ͸ಈ͘ ϦΞϧλΠϜϨϯμϦϯάʹ଱͑Δ଎͞Ͱ
19. Benedikt Bitterli, Chris Wyman, Matt Pharr, Peter Shirley, Aaron Lefohn, and Wojciech Jarosz. 2020. Spatiotemporal reservoir resampling for real-time ray tracing with dynamic direct lighting. ACM Trans. Graph. 39, 4, Article 148 (August 2020), 17 pages. https://doi.org/ 10.1145/3386569.3392481 ReSTIR Reservoir-based Spatio-Temporal Importance Resampling
20. Lo (x, ωo) = Le (x, ωo) + ∫ S2 fs (x, ωi , ωo) Li (x, ωi)|ωi ⋅ n|dωi ϞϯςΧϧϩϨΠτϨʔγϯά ͜ͷੵ෼ΛϞϯςΧϧϩ๏Ͱܭࢉ͢Δ x ϥϯμϜͳํ޲ ͔ΒඈΜͰ͘Δޫ Λௐ΂Δ ωi Li (x, ωi) े෼ͳճ਺܁Γฦ͢ͱ શͯͷํ޲ʹ͍ͭͯͷੵ෼Λ ۙࣅͰ͖Δ
21. ॏ఺αϯϓϦϯά x େ͖ͳΤωϧΪʔͷޫ͕ඈΜͰ͘Δํ޲ͷ༧૝͕ͭ࣌͘ ͦͷํ޲ʹภͬͨ ཚ਺Ͱ αϯϓϦϯά͢Δ গͳ͍ճ਺ͷαϯϓϦϯάͰ શͯͷํ޲ʹ͍ͭͯͷੵ෼ʹ͍ۙ஋͕ಘΒΕΔ
22. x ୔ࢁͷ఺ޫݯͷத͔Β
23. x ʹಧ͘ΤωϧΪʔ͕େ͖͍ޫݯ΄Ͳߴ͍֬཰Ͱग़ΔநબശͰ x ޫݯΛ1͚ͭͩબͼɺࢹ఺ʹಧ͘ΤωϧΪʔΛܭࢉ͢Δ
24. x0 ྡ઀͢ΔϐΫηϧͰͷܭࢉ݁ՌΛूΊΔͱ Өڹͷେ͖͍ޫݯ਺ݸͷܭࢉΛ͢Δͷͱ΄΅ಉ͡ x1 x1
25. ໰୊1 ʹಧ͘ޫͷΤωϧΪʔͷେ͖͞ΛͲ͏ٻΊΔ͔ x
26. x n l − x ʹಧ͘ޫͷΤωϧΪʔʹؔΘΔཁૉ x ޫݯ͔Β์ͨΕΔΤωϧΪʔͷେ͖͞ ޫݯ͔Β ·Ͱͷڑ཭ʹΑΔݮਰ x l ໘ͷ޲͖ Ͱड͚ΒΕΔޫͷΤωϧΪʔͷׂ߹ n ఆ਺ 1 l − x 2 max (l − x) l − x ⋅ n,0
27. x n ޫݯ͔Β ·Ͱͷڑ཭ʹΑΔݮਰ x l ໘ͷ޲͖ Ͱड͚ΒΕΔޫͷΤωϧΪʔͷׂ߹ n ఆ਺ 1 l − x 2 max (l − x) l − x ⋅ n,0 ःṭ෺ͷ༗ແ { 1 0 ( ͱ Λ݁Ϳઢ෼ͱަࠩ͢Δ໘͕ແ͍) l x ( ͱ Λ݁Ϳઢ෼ͱަࠩ͢Δ໘͕͋Δ) l x l − x
28. RTX ON
29. RTX ON
30. ௖఺഑ྻ BVH raygenγΣʔμ ͜ͷઢ෼ͱ ަࠩ͢Δ ࡾ֯ܗΛ͍ͩ͘͞ ަࠩ൑ఆ hitγΣʔμ ͜ͷࡾ֯ܗͱ ަࠩ͠·͢ ม׵
31. ௖఺഑ྻ BVH raygenγΣʔμ ͜ͷઢ෼ͱ ަࠩ͢Δ ࡾ֯ܗΛ͍ͩ͘͞ ަࠩ൑ఆ hitγΣʔμ ͜ͷࡾ֯ܗͱ ަࠩ͠·͢ ม׵ ଎͍ ஗͍
32. ୔ࢁͷޫݯʹরΒ͞ΕΔγʔϯΛඳ͖͍ͨ ޫݯ΍γʔϯ͸ಈ͘ ϦΞϧλΠϜϨϯμϦϯάʹ଱͑Δ଎͞Ͱ
33. ௖఺഑ྻ BVH raygenγΣʔμ ͜ͷઢ෼ͱ ަࠩ͢Δ ࡾ֯ܗΛ͍ͩ͘͞ ަࠩ൑ఆ hitγΣʔμ ͜ͷࡾ֯ܗͱ ަࠩ͠·͢ ม׵ γʔϯ͕มܗ͢Δ = ͜ͷม׵͕සൟʹඞཁ ஗͍
34. RTSDF Real-Time Signed Distance Field Tan Y.W., Chua N., Koh C., Bhojan A., RTSDF: Real-time signed distance fields for soft shadow approximation in games, 2022, arXiv preprint https://arxiv.org/abs/ 2210.06160.
35. ௖఺഑ྻ ίϯϐϡʔτγΣʔμ ͜ͷઢ෼ͱ ަࠩ͢Δ໘ͷ ࠲ඪΛ͍ͩ͘͞ ͜ͷ͋ͨΓͰ ໘ͱަࠩ͠·͢ ม׵ SDF ަࠩ൑ఆ ࡶ ଎͍ ଎͍
36. ϋʔυ΢ΣΞ
37. ໰୊2 શͯͷޫݯ͔ΒͷΤωϧΪʔ͕Θ͔Βͳ͍ঢ়ଶͰ Ͳ͏΍ͬͯૂͬͨ֬཰෼෍ͰޫݯͷநબΛߦ͏͔
38. x ൪໨ͷޫݯ͔Β ʹಧ͘ޫͷΤωϧΪʔΛ ͱ͢Δ n x Ln L0 L1 L2 L3 L4 L5 L6 L7 ൪໨ͷޫݯ͕બ͹ΕΔ֬཰ 6 = L6 ∑7 i=0 Ln શͯͷ ΛٻΊΔඞཁ͕͋Δ Ln ߴՁ
39. Benedikt Bitterli, Chris Wyman, Matt Pharr, Peter Shirley, Aaron Lefohn, and Wojciech Jarosz. 2020. Spatiotemporal reservoir resampling for real-time ray tracing with dynamic direct lighting. ACM Trans. Graph. 39, 4, Article 148 (August 2020), 17 pages. https://doi.org/ 10.1145/3386569.3392481 ReSTIR Reservoir-based Spatio-Temporal Importance Resampling
40. Min-Te Chao. 1982. A General Purpose Unequal Probability Sampling Plan. Biometrika 69, 3 (Dec. 1982), 653–656. https://doi.org/10/fd87zs WRS Weighted Reservoir Sampling
41. Reservoir struct reservoir { unsigned int y; float w_y; float w_sum; unsigned int m; }; ࠓબΜͰ͍Δཁૉ ࠓબΜͰ͍ΔཁૉͷॏΈ ࠓ·ͰʹબΜͩཁૉͷ ॏΈͷ૯࿨ ࠓ·ͰʹબΜͩཁૉͷ਺ (optional) Ε͟἖͊ʔ
42. const unsigned int candidate = random() * max_candidate_index; const float weight = calculate_weight( candidate ); res.w_sum += weight; res.m += 1u; if( random() < weight / res.w_sum ) { res.y = candidate; res.w_y = weight; } 0.0Ҏ্1.0ҎԼͷ Ұ༷ͳཚ਺ ۉ౳ͳ֬཰Ͱ৽͍͠ཁૉͷީิΛ1ͭબͼ ͦͷཁૉͷॏΈΛٻΊΔ
43. const float weight = calculate_weight( candidate ); res.w_sum += weight; res.m += 1u; if( random() < weight / res.w_sum ) { res.y = candidate; res.w_y = weight; } ࠓ·Ͱʹࢼͨ͠ީิͷॏΈͷ૯࿨ͱ ࠓ·Ͱʹࢼͨ͠ީิͷݸ਺Λߋ৽
44. res.w_sum += weight; res.m += 1u; if( random() < weight / res.w_sum ) { res.y = candidate; res.w_y = weight; } ৽͍͠ީิͷॏΈ ࠓ·ͰͷશͯͷީิͷॏΈͷ૯࿨ ͕ 0.0Ҏ্1.0ҎԼͷཚ਺Ҏ্ͳΒ ࠓબΜͰ͍ΔཁૉΛߋ৽͢Δ
45. } ৽͍͠ީิͷॏΈ ࠓ·ͰͷશͯͷީิͷॏΈͷ૯࿨ ͕ 0.0Ҏ্1.0ҎԼͷཚ਺Ҏ্ͳΒ ࠓબΜͰ͍ΔཁૉΛߋ৽͢Δ ϙΠϯτ ॏΈ͸૯࿨͕1.0ʹͳΔΑ͏ʹਖ਼نԽ͞Ε͍ͯͳͯ͘΋ྑ͍ શମͷॏΈͷ૯࿨͕Θ͔Βͳͯ͘΋ ୔ࢁͷreservoirͰ্ͷૢ࡞Λ܁Γฦͤ͹ ͦͷ͏ͪਖ਼͍֬͠཰෼෍ʹͳΔ
46. ຊ౰ʹ?
47. #!/usr/bin/env python3 # -*- coding: utf-8 -*- import random class Reservoir: def __init__(self): self.y = None self.w = 0 self.w_sum = 0 self.m = 0 def update( self, sample_id, weight ): self.w_sum += weight self.m += 1 b_accept = random.random() < weight / self.w_sum; if b_accept: self.y = sample_id self.w_y = weight return b_accept weight = [ random.random() for i in range( 0, 10 ) ] hist = [ 0 for i in range( 0, 10 ) ] res = [ Reservoir() for i in range( 0, 500 ) ] pdf = 1.0/len( weight ) for i in range( 0, 100 ): for j in range( 0, len( res ) ): sample_id = random.randint( 0, len( weight ) - 1 ) p_hat = weight[ sample_id ] w = p_hat / pdf res[ j ].update( sample_id, w ) hist[ res[ j ].y ] += 1 w_sum = sum( weight ) h_sum = sum( hist ) for i in range( 0, len(hist) -1 ): print( ( hist[ i ]/h_sum ) / ( weight[ i ]/w_sum ) ) ࣮ࡍʹ΍ͬͯΈΑ͏ 10ݸͷཁૉΛ ͜ͷॏΈͰநબͯ͠΄͍͠ Reservoir 500ݸ 100αΠΫϧ܁Γฦͨ࣌͠఺Ͱͷ ॏΈ͔ΒٻΊͨ֬཰ͱ ࣮ࡍʹཁૉ͕બ͹Εׂͨ߹ͷ ൺΛදࣔ
48. 1.078738107561038 1.0089287723978344 1.0175850866004104 1.2341449375734417 0.9609675746812903 0.9064680817904596 1.0359533238321195 0.9793422612692094 0.922534390847772 ਖ਼ղͱͷൺͳͷͰ 1.0ʹ͍ۙఔྑ͍ ਖ਼͍֬͠཰ʹ ߹ΘͤΑ͏ͱ͍͏ؾ͸͋Δ ఔ౓ͷ֬཰Ͱग़ͯ͘Δ
49. ෺ମͷද໘ʹେྔReservoirΛฒ΂ 1ϑϨʔϜຖʹ1ճreservoirͷߋ৽ΛࢼΈΔ
50. ໰୊3 ReservoirΛͲ͜ʹه࿥͢Δ͔
51. ϋογϡςʔϒϧʹ ه࿥͢Δ Reservoir Reservoir Reservoir Reservoir Reservoir Reservoir Reservoir Reservoir Reservoir hash ೋେ೿ൊ͕͋Δ εΫϦʔϯͷϐΫηϧຖʹ ه࿥͢Δ εΫϦʔϯ
52. ϋογϡςʔϒϧʹ ه࿥͢Δ Reservoir Reservoir Reservoir Reservoir hash εΫϦʔϯͷϐΫηϧຖʹ ه࿥͢Δ ར఺ ݟ͍͑ͯΔ෺Λඳ͘ͷʹ ඞཁͳ࠷খݶͷReservoir͚͕ͩ࢒Δ ܽ఺ ҰॠͰ΋ࢹք͔Β֎Εͨ Reservoir͸Ϧηοτ͞ΕΔ ར఺ ࠓݟ͍͑ͯͳ͍෦෼ͷReservoir΋ ࢒͓ͯ͘͜͠ͱ͕Ͱ͖Δ ܽ఺ Reservoir͕ͲΜͲΜ૿͑ΔͷͰ ཁΒͳ͍෺ΛࣺͯΔॲཧ͕ඞཁ
53. ϋογϡςʔϒϧʹ ه࿥͢Δ Reservoir Reservoir Reservoir Reservoir Reservoir Reservoir Reservoir Reservoir Reservoir hash εΫϦʔϯͷϐΫηϧຖʹ ه࿥͢Δ ࠓճ͸ͬͪ͜ͷख๏Ͱ͍͘
54. t − 2 t − 1 t p p p Reservoir͸෺ମͷද໘ͷ֤఺ʹ͍ͭͯͷ৘ใ ͋ΔϐΫηϧ ʹө͍ͬͯΔ෺ମͷද໘͸ ࣌ࠁ ͱڞʹมԽ͢Δ p t
55. p Optical Flow ͷϩʔΧϧ࠲ඪʹ 1ϑϨʔϜલͷߦྻΛֻ͚ͯ ʹө͍ͬͯΔ෺ମͷද໘͕ 1ϑϨʔϜલ͸Ͳ͜ʹ͔͋ͬͨΛ ٻΊΔ p p 1ϑϨʔϜલͷҐஔʹ͋Δ ReservoirΛ ͷҐஔͷաڈͷReservoirͱ͢Δ p
56. ࣮૷
57. // distance field { distance_field.clear( rec ); { auto render_pass_token = rec.begin_render_pass( distance_field.get_distance_field_image().get_render_pass_begin_info(), vk::SubpassContents::eInline ); rec->setViewport( 0, { distance_field.get_distance_field_image().get_viewport() } ); rec->setScissor( 0, { distance_field.get_distance_field_image().get_scissor() } ); rec->setCullMode( vk::CullModeFlagBits::eNone ); rec->setDepthCompareOp( vk::CompareOp::eAlways ); rec.bind_descriptor_set( vk::PipelineBindPoint::eGraphics, 1u, sg->get_resource()->pipeline_layout, global_descriptor_set ); (*il)( rec, *voxel_csg, false ); } rec.graphics_to_compute_barrier( {}, { distance_field.get_working_image().get_image()->get_factory() } ); distance_field( rec ); } tone.set( rec, 0 ); { if( !update_optflow ) { update_optflow = true; } else { SDFΛੜ੒ gct.cpp
58. GόοϑΝΛੜ੒ else { update_optflow = false; } rec.copy( global_data, global_uniform ); rec.transfer_to_graphics_barrier( { global_uniform->get_buffer() }, {} ); { auto render_pass_token = rec.begin_render_pass( gbuffer.get_render_pass_begin_info( 0 ), vk::SubpassContents::eInline ); rec->setViewport( 0, 1, &gbuffer.get_viewport() ); rec->setScissor( 0, 1, &gbuffer.get_scissor() ); rec->setCullMode( vk::CullModeFlagBits::eBack ); rec->setDepthCompareOp( vk::CompareOp::eLessOrEqual ); rec.bind_descriptor_set( vk::PipelineBindPoint::eGraphics, 1u, sg->get_resource()->pipeline_layout, global_descriptor_set ); (*il)( rec, *geometry_csg, true ); } if( walk.get_current_camera() == 0 ) { sg->rotate_visibility( rec ); } rec.convert_image( gbuffer.get_image( 0 ), vk::ImageLayout::eGeneral ); } rec.barrier( {}, gct.cpp
59. reservoir->get_factory() } ); hgauss( rec, 0, res.width, res.height, 1u ); rec.barrier( {}, { temporary_diffuse->get_factory(), temporary_specular->get_factory() } ); vgauss( rec, 0, res.width, res.height, 1u ); rec.barrier( {}, { previous_diffuse->get_factory(), previous_specular->get_factory() } ); update_reservoir( rec, 0, res.width, res.height, 1u ); rec.copy( reservoir->get_factory(), previous_reservoir->get_factory() ); rec.fill( direct->get_factory(), gct::color::web::black ); rec.barrier( {}, { direct->get_factory() } ); direct_light( rec, 0, 1000u, 1u, 1u ); lighting( rec, 0, res.width, res.height, 1u ); rtao( rec ); rec.barrier( {}, { direct->get_factory(), diffuse->get_factory(), WRS ReservoirͰબ͹Ε͍ͯΔ ఺ޫݯʹΑΔর໌Λܭࢉ gct.cpp
60. void main() { const ivec2 screen_pos = ivec2( gl_GlobalInvocationID.xy ); vec3 normal = imageLoad( gbuffer, ivec3( screen_pos, 2 ) ).xyz; const uint instance = uint( imageLoad( gbuffer, ivec3( screen_pos, 4 ) ).z ); const vec3 pos = imageLoad( gbuffer, ivec3( screen_pos, 0 ) ).xyz; const uint light_count = global_uniforms.light_count; const ivec3 image_size = imageSize( gbuffer ); const vec3 optical_flow = imageLoad( gbuffer, ivec3( screen_pos, 5 ) ).xyz; vec4 r = vec4( 32767, 0.0, 0.0, 0.0 ); const ivec2 previous_screen_pos = screen_pos - ivec2( ( optical_flow.xy * vec2( 0.5, 0.5 ) ) * ( vec2( image_size.xy ) ) ); { ivec2 n = previous_screen_pos; const bool valid_pos = ( n.x >= 0 && n.x < image_size.x && n.y >= 0 && n.y < image_size.y ); const vec4 hist = imageLoad( previous_reservoir, ivec3( n, 1 ) ); const vec3 prev_normal = hist.xyz; const uint prev_instance = uint( hist.w ); const bool history_valid = valid_pos && ( instance == prev_instance ) && ( dot( normal, prev_normal ) >= 0.99 ); r = history_valid ? imageLoad( previous_reservoir, ivec3( n, 0 ) ) : r; } vec2 rand_seed = vec2( screen_pos.xy ) + vec2( global_uniforms.frame_counter.x * 48.0, global_uniforms.frame_counter.x * 37.0 ); const uint y = uint( r.x ); const float w_y = r.y; wrs.comp Optical FlowΛ࢖ͬͯલͷϑϨʔϜͷReservoirΛҾ͖ܧ͙
61. vec2 rand_seed = vec2( screen_pos.xy ) + vec2( global_uniforms.frame_counter.x * 48.0, global_uniforms.frame_counter.x * 37.0 ); const uint y = uint( r.x ); const float w_y = r.y; const float speed = min( length( optical_flow.xy ), 1.0 ); const float w_sum = r.z * mix( 1.0, 0.01, speed ); const float m = r.w; uint light_index = 0; float weight = 0.0; for( uint retry = 0; retry != 16; retry++ ) { rand_seed = rand1_vec2( rand_seed ); light_index = active_light[ uint( rand_seed.x * light_count ) ]; float recv_ratio = dot( normalize( light[ light_index ].world_position.xyz - pos.xyz ), normal ); const float d = distance( pos.xyz, light[ light_index ].world_position.xyz ); weight = recv_ratio * max( max( light[ light_index ].energy.r, light[ light_index ].energy.g ), light[ light_index ].energy.b ) / ( d * d ) / light_count; if( recv_ratio > 0.0 ) { break; } } rand_seed = rand1_vec2( rand_seed ); const float rand01 = rand_seed.x; bool update = ( rand01 < weight / w_sum + weight ); const vec3 ray_begin = ( matrix_pool[ global_uniforms.voxel ] * vec4( pos, 1.0 ) ).xyz; const vec3 ray_end = wrs.comp ۉ౳ͳ֬཰Ͱ఺ޫݯΛ1ͭબͼ ःṭ෺͕ແ͔ͬͨ৔߹෺ମͷද໘ʹಧ͘ޫͷΤωϧΪʔΛܭࢉ͢Δ
62. ), light[ light_index ].energy.b ) / ( d * d ) / light_count; if( recv_ratio > 0.0 ) { break; } } rand_seed = rand1_vec2( rand_seed ); const float rand01 = rand_seed.x; bool update = ( rand01 < weight / w_sum + weight ); const vec3 ray_begin = ( matrix_pool[ global_uniforms.voxel ] * vec4( pos, 1.0 ) ).xyz; const vec3 ray_end = ( matrix_pool[ global_uniforms.voxel ] * vec4( light[ light_index ].world_position.xyz, 1.0 ) ).xyz; const vec3 ray_dir = normalize( ray_end - ray_begin ); const float ray_distance = distance( ray_begin, ray_end ); vec3 ray_cur = ray_begin + ray_dir * 4.0/512.0; float ray_traveled = 4.0/512.0; float prev_r = 0.0; float grad = 4.0/512.0; for( int j = 0; j < 64; j++ ) { float r = texture( distance_field, ray_cur ).r; prev_r = r; grad = r - prev_r; ray_cur += ray_dir * r; ray_traveled += r; } imageStore( reservoir, ivec3( screen_pos, 0 ), ( update && ( ray_traveled >= ray_distance || grad > 0.0 ) ) ? vec4( light_index, weight, w_sum + weight, m + 1 ) : wrs.comp SDFΛ࢖ͬͯ෺ମͷද໘͔Β఺ޫݯ·Ͱͷؒʹ ःṭ෺͕ͳ͍ࣄΛ֬ೝ͢Δ
63. bool update = ( rand01 < weight / w_sum + weight ); const vec3 ray_begin = ( matrix_pool[ global_uniforms.voxel ] * vec4( pos, 1.0 ) ).xyz; const vec3 ray_end = ( matrix_pool[ global_uniforms.voxel ] * vec4( light[ light_index ].world_position.xyz, 1.0 ) ).xyz; const vec3 ray_dir = normalize( ray_end - ray_begin ); const float ray_distance = distance( ray_begin, ray_end ); vec3 ray_cur = ray_begin + ray_dir * 4.0/512.0; float ray_traveled = 4.0/512.0; float prev_r = 0.0; float grad = 4.0/512.0; for( int j = 0; j < 64; j++ ) { float r = texture( distance_field, ray_cur ).r; prev_r = r; grad = r - prev_r; ray_cur += ray_dir * r; ray_traveled += r; } imageStore( reservoir, ivec3( screen_pos, 0 ), ( update && ( ray_traveled >= ray_distance || grad > 0.0 ) ) ? vec4( light_index, weight, w_sum + weight, m + 1 ) : vec4( y, w_y, y == 32767 ? 0.0 : w_sum + weight, y == 32767 ? 0 : m + 1 ) ); imageStore( reservoir, ivec3( screen_pos, 1 ), vec4( normal, instance ) ); } wrs.comp ཚ਺ͰબΜͩޫݯ͕ःṭ͞Ε͓ͯΒͣɺWRSͰߋ৽͢΂͖ͱ൑அ͞Εͨ৔߹ Reservoir͕ࢦ͍ͯ͠Δ఺ޫݯΛߋ৽͢Δ
64. Ұ༷ཚ਺
65. ReSTIR
66. ); } rec.barrier( {}, { reservoir->get_factory() } ); hgauss( rec, 0, res.width, res.height, 1u ); rec.barrier( {}, { temporary_diffuse->get_factory(), temporary_specular->get_factory() } ); vgauss( rec, 0, res.width, res.height, 1u ); rec.barrier( {}, { previous_diffuse->get_factory(), previous_specular->get_factory() } ); update_reservoir( rec, 0, res.width, res.height, 1u ); rec.copy( reservoir->get_factory(), previous_reservoir->get_factory() ); rec.fill( direct->get_factory(), gct::color::web::black ); rec.barrier( {}, { direct->get_factory() } ); direct_light( rec, 0, 1000u, 1u, 1u ); lighting( rec, 0, res.width, res.height, 1u ); rtao( rec ); gct.cpp લͷϑϨʔϜͰͷ֦ࢄͱ൓ࣹʹAdaptive BlurΛ͔͚͓ͯ͘
67. uint light_index = uint( imageLoad( reservoir, ivec3( screen_pos, 0 ) ).x ); vec3 diffuse = vec3( 0.0, 0.0, 0.0 ); vec3 specular = vec3( 0.0, 0.0, 0.0 ); const bool light_valid = ( light_index != 32767 ); light_index = light_valid ? light_index : 0; const float shadow_level = 1.0; const vec3 N = normal; const vec3 V = normalize(global_uniforms.eye_pos.xyz-pos); const vec3 L = normalize( light[ light_index ].world_position.xyz-pos); const float d = distance( pos.xyz, light[ light_index ].world_position.xyz ); const vec3 energy = light[ light_index ].energy.rgb / ( d * d ); diffuse = light_valid ? diffuse_with_mask( L, V, N, albedo.rgb, roughness, metallicness, emissive, energy, shadow_level ) : diffuse; specular = light_valid ? specular_with_mask( L, V, N, albedo.rgb, roughness, metallicness, energy, shadow_level ) : specular; const ivec3 image_size = imageSize( gbuffer ); const vec3 optical_flow = imageLoad( gbuffer, ivec3( screen_pos, 5 ) ).xyz; const uint instance = uint( imageLoad( gbuffer, ivec3( screen_pos, 4 ) ).z ); const ivec2 previous_screen_pos = screen_pos - ivec2( ( optical_flow.xy * vec2( 0.5, 0.5 ) ) * ( vec2( image_size.xy ) ) ); lighting.comp Reservoir͕ࢦ͍ͯ͠Δ఺ޫݯʹΑΔ൓ࣹͱ֦ࢄΛܭࢉ͢Δ
68. specular = light_valid ? specular_with_mask( L, V, N, albedo.rgb, roughness, metallicness, energy, shadow_level ) : specular; const ivec3 image_size = imageSize( gbuffer ); const vec3 optical_flow = imageLoad( gbuffer, ivec3( screen_pos, 5 ) ).xyz; const uint instance = uint( imageLoad( gbuffer, ivec3( screen_pos, 4 ) ).z ); const ivec2 previous_screen_pos = screen_pos - ivec2( ( optical_flow.xy * vec2( 0.5, 0.5 ) ) * ( vec2( image_size.xy ) ) ); vec4 existing_diffuse = vec4( 0.0, 0.0, 0.0, 1.0 ); vec4 existing_specular = vec4( 0.0, 0.0, 0.0, 1.0 ); uvec2 hist = uvec2( 0, instance ); const bool history_in_range = ( previous_screen_pos.x >= 0 && previous_screen_pos.x < image_size.x && previous_screen_pos.y >= 0 && previous_screen_pos.y < image_size.y ); const uvec2 history_value = imageLoad( previous_history, previous_screen_pos ).xy; const vec4 diffuse_value = imageLoad( previous_diffuse_image, previous_screen_pos ); const vec4 specular_value = imageLoad( previous_specular_image, previous_screen_pos ); const bool history_valid = history_in_range && ( history_value.y == instance ); existing_diffuse = history_valid ? diffuse_value : existing_diffuse; existing_specular = history_valid ? specular_value : existing_specular; hist = history_valid ? history_value : hist; const float speed = length( optical_flow.xy ); hist.x = min( hist.x, uint( mix( 300.0, 1.0, speed ) ) ); imageStore( diffuse_image, screen_pos, ( vec4( diffuse, 1.0 ) + existing_diffuse * hist.x )/( hist.x + 1 ) ); imageStore( specular_image, screen_pos, ( vec4( specular, 1.0 ) + existing_specular * hist.x )/( hist.x + 1 ) ); lighting.comp Optical FlowΛ࢖ͬͯաڈͷϑϨʔϜͰͷ֦ࢄͱ൓ࣹΛҾ͖ܧ͙
69. const ivec3 image_size = imageSize( gbuffer ); const vec3 optical_flow = imageLoad( gbuffer, ivec3( screen_pos, 5 ) ).xyz; const uint instance = uint( imageLoad( gbuffer, ivec3( screen_pos, 4 ) ).z ); const ivec2 previous_screen_pos = screen_pos - ivec2( ( optical_flow.xy * vec2( 0.5, 0.5 ) ) * ( vec2( image_size.xy ) ) ); vec4 existing_diffuse = vec4( 0.0, 0.0, 0.0, 1.0 ); vec4 existing_specular = vec4( 0.0, 0.0, 0.0, 1.0 ); uvec2 hist = uvec2( 0, instance ); const bool history_in_range = ( previous_screen_pos.x >= 0 && previous_screen_pos.x < image_size.x && previous_screen_pos.y >= 0 && previous_screen_pos.y < image_size.y ); const uvec2 history_value = imageLoad( previous_history, previous_screen_pos ).xy; const vec4 diffuse_value = imageLoad( previous_diffuse_image, previous_screen_pos ); const vec4 specular_value = imageLoad( previous_specular_image, previous_screen_pos ); const bool history_valid = history_in_range && ( history_value.y == instance ); existing_diffuse = history_valid ? diffuse_value : existing_diffuse; existing_specular = history_valid ? specular_value : existing_specular; hist = history_valid ? history_value : hist; const float speed = length( optical_flow.xy ); hist.x = min( hist.x, uint( mix( 300.0, 1.0, speed ) ) ); imageStore( diffuse_image, screen_pos, ( vec4( diffuse, 1.0 ) + existing_diffuse * hist.x )/( hist.x + 1 ) ); imageStore( specular_image, screen_pos, ( vec4( specular, 1.0 ) + existing_specular * hist.x )/( hist.x + 1 ) ); imageStore( history, screen_pos, uvec4( hist.x + 1, hist.y, 0, 0 ) ); } lighting.comp ࠓܭࢉͨ͠1αϯϓϧͱաڈͷϑϨʔϜ͔ΒҾ͖ܧ͍ͩαϯϓϧΛࠞͥΔ
70. ReSTIR+ Spatio-Temporal Denoiser
71. 4ݸ
72. 8ݸ
73. 16ݸ
74. 32ݸ
75. 64ݸ
76. 128ݸ
77. 256ݸ
78. 512ݸ
79. ·ͱΊ ReSTIR ϦΞϧλΠϜϨϯμϦϯάͰ ୔ࢁͷޫݯʹরΒ͞ΕΔγʔϯΛ ඳ͘ख๏ WRSΛ࢖ͬͯॏ఺αϯϓϦϯά
80. https://fadis.booth.pm/ ٕज़ॻయͰຊग़ͯ͠·͢ /&8