Application of MaterialX and Standard Surface to Video Game Graphics
This is the English translation of CEDEC2022 "Application of MaterialX and Standard Surface to Video Game Graphics".
This document explains how to apply the Standard Surface described by MaterialX to Video Game Graphics.
Self introduction Past Projects ・Ace Combat 6 ・Ace Combat: Assault Horizon ・Tales Of Card Evolve ・THE IDOLM@STER MILLION LIVE! ・Pokkén Tournament ・New Pokémon Snap Past talks ・CEDEC2011 Continuous integration of ACE COMBAT ASSAULT HORIZON ・CEDEC2011 In-game camera production example in “ACE COMBAT ASSAULT HORIZON” in-game camera production example ・CEDEC2015 Game editor design and implementation in Pokkén Tournament ・CEDEC2019 Basic design of game editor and stable and fast asset management in console video game development ・CEDEC2019 Implementation of Light Baker using DirectX Raytracing ・CEDEC2021 New Pokémon Snap Lighting Overview Wataru Tada I have worked as a character modeler, motion, background, lighting artist, shader artist, and now as a TA, I research graphics and support work for various projects. Past talks CEDEC2008 Linear space and physically correct lighting CEDEC2019 "Super Smash Bros. Ultimate" - To draw the borrowed IP as beautifully as possible CEDEC2020 HDRI production technique with high reliability CEDEC2021 Video Game Graphics for the Raytracing Generation, Get Grounded and Prepared - Quickly and Easily Validate Next-Generation Quality Real-Time Physically-Based Lighting with NVIDIA Falcor CEDEC2021 New Pokémon Snap Lighting Overview Masayuki Suzuki 2
Contents • About MaterialX • Implementation detail • Comparison with DCC tools • Performance measurement • Other features • Conclusion 3 This slide has a gray background to prevent optical illusion.
About MaterialX • MaterialX is a standard format and related library for describing materials, developed by ILM, and published on GitHub(Apache 2.0 license). (https://github.com/AcademySoftwareFoundation/MaterialX) • MaterialX describes materials in XML (.mtlx file), and MaterialX can output shaders such as GLSL, MDL, and OSL. • Various DCC tools, such as Maya and Houdini support MaterialX, and MaterialX provides Python bindings. • MaterialX is mainly used for offline rendering, but this talk describes real-time rendering. • This talk uses MaterialX version 1.38.0 4
About Standard Surface • Standard Surface is a standardized MaterialX file (standard_surface.mtlx) that is used to represent physical materials and published on GitHub(Apache 2.0 license). https://github.com/Autodesk/standard-surface • Standard Surface has a horizontal and vertical layered structure and can represent most physical materials such as Diffuse, Subsurface scattering, Sheen, etc. 6
Parameters of Standard Surface • Standard Surface defines material parameters for each category, such as Coating and Sheen. • In addition, Standard Surface standardizes parameters so that various DCC tools can handle the same material parameters. Maya+Arnold Houdini 7
Implementation in in-house LookDev environment • The in-house LookDev environment has implemented MaterialX to use Standard Surface. • As a result, artists can handle the same material parameters as DCC tools, making it easier to control rendering. In-house LookDev environment Maya+Arnold Houdini 8
How to implement the in-house LookDev environment • MaterialX is from the film industry and cannot be used for video game graphics as is. • MaterialX specifications are insufficient for symbolic representation of video game graphics, such as dither. • MaterialX's GLSL output is mainly fragment shaders, and processing cannot be described in vertex shaders. • So, We decided to disassemble the various MaterialX nodes and combine them with Video game nodes to implement them in the in- house shader editor. Video game graphics node MaterialX node In-house shader editor MaterialX node MaterialX 10
Rendering flow of in-house LookDev environment • The rendering flow is to draw the luminance value to the render target and apply the exposure. [CEDEC2021][SIGGRAPH ASIA 2021] New Pokémon Snap Lighting Overview Draw to 16bit float render target with luminance 1000cd/㎡ as 1 in GPU Apply exposure to render target 13
Application of exposure • MaterialX View does not support exposure, so we modified the program to apply exposure. Added Exposure to Advanced Settings MaterialX View uniforms Exposure Apply exposure to output MaterialX View fragment shader output 15
In-house HDRI Support • We applied the HDRI and sun directional light sources used in the in- house LookDev environment to the MaterialX View (The in-house LookDev environment uses directional light and HDRI IBL for lighting)
MaterialX file describing the directional light source of the sun HDRI Applies to MaterialX View Exposure Split Direct Light 16
In-house HDRI Support • MaterialX View can now use the same exposures and HDRIs as the in-house LookDev environment so that rendering results can be compared. HDRI for debugging MaterialX View In-house LookDev environment MaterialX View In-house LookDev environment MaterialX View In-house LookDev environment 17
HLSL porting of MaterialX nodes • We ported the GLSL in the libraries/lights/genglsl, pbrlib, stdlib folders of the MaterialX repository to HLSL. • Add include guard • Type change (vec3 to float3, etc.) • Change the multiplication order of mul • Change $envMatrix to ConstantBuffer • In-house shader editor outputs shaders using ported HLSL functions. mx_oren_nayar_diffuse_bsdf.hlsl mx_dielectric_bsdf.hlsl ... mx_oren_nayar_diffuse_bsdf.glsl mx_dielectric_bsdf.glsl ... 18
Implementation in in-house shader editor • The in-house shader editor has a node for each function ported to HLSL. • For consistency of terminology, the node names are the same as the MaterialX node names. Compute node example of in-house shader editor 19
BSDF node implementation • BSDF functions are separate nodes for Reflection, Indirect, and Transmission. BSDF node example of in-house shader editor Reflection node (used for Direct Lighting) Indirect node (used for Indirect Lighting) Transmission node (used for Specular transmission) 20
Creating Standard Surface materials • We created the Standard Surface material using the Standard Surface documentation(https://autodesk.github.io/standard- surface/) and the Standard Surface GLSL code as a reference. • Since MaterialX View's shader file output features do not output specular transmission, we used the shader code output by Nsight. Nsight shader code display 21
Standard Surface node structure • The Standard Surface material consists of Direct Lighting, Indirect Lighting, and Transmission. • When adding a light source, it is necessary to add a node. Direct Lighting Indirect Lighting Standard Surface Pixel shader output (RGB) Pixel shader input Add Transmission Pixel shader output (alpha) 23
Direct Lighting node structure • Indirect Lighting has a similar node structure • Each element such as Diffuse and Subsurface has a weight. Diffuse Subsurface Translucent Sheen Transmission Dielectric Metal Coat Emission 出力 Lerp Multiply Add 24 Add Coat and Emission to get the same results as in MaterialX View. Input Emission output to Coat, same as Arnold.
Rendering comparison • We compared Arnold with MaterialX View and the in-house Lookdev environment to confirm the following. • How correct is the Standard Surface material implementation in the in- house Lookdev environment? • How similar is real-time rendering to offline rendering? In-house Lookdev environment MaterialX View Maya + Arnold 26
Standard surface material comparison • We compared renderings by writing parameters such as base_color in Excel and outputting a MaterialX file from Excel; in the Arnold and in-house Lookdev environments, the scripts directly referenced the Excel file to set material parameters. Excel MaterialX View In-house Lookdev environment MaterialX file (.mtlx) Arnold 27
Camera Parameter setting • The Excel file for material comparison can also describe camera parameters, screenshots can be taken with the same camera in Arnold, MaterialX View, and in-house Lookdev environment. • MaterialX View can take screenshots with startup arguments. • The background HDRI in the in-house Lookdev environment is in LatLong format to match MaterialX View. Arnold MaterialX View In-house Lookdev environment 28
Arnold rendering • We applied the parameters of the MaterialX file to aiStandardSurface to render Arnold. • aiMaterialXShader can also be used to render the MaterialX file. aiStandardSurface aiMaterialXShader 29
Comparison of MaterialX View and aiStandardSurface We compared rendering in MaterialX View and Arnold. Arnold for Maya Materials • Standard Surface • aiStandard Surface ← used for comparison • aiMaterialXShader 30
Comparison of MaterialX View and aiStandardSurface Comparison result of MaterialX View and Arnold • There are many small differences. • Some differences can be resolved with parameter mapping. • The big difference is imperfect transmission. • Subsurface may also have problems. • There are many layers that are not compatible with indirect light. 31
Comparison of MaterialX View and aiStandardSurfacet • Scene settings • 1m radius sphere • Taken with a 400mm lens and 35mm film at a distance of 38m from the origin (depending on the situation) • Lighting composed of IBL and directional light sources • Arnold settings • RayDepth is basically 1 (Transmission and Transparency are 2 at DoubleSide) 32
Comparison - Dielectric (nonmetal)- Specular - Direct Light Overlay images for comparison MaterialX View is a little brighter at higher roughness, but the difference is not noticeable at IOR 1.5. Next, check with indirect light 41 MaterialX View From front From above specular_roughness 0 specular_roughness 0.1 specular_roughness 0.3 specular_roughness 0.5 specular_roughness 0.7 specular_roughness 1 IOR 1 specular_roughness 1 IOR 3
MaterialX View The difference is significant, especially when roughness is high. Next, check with only a directional light source. Comparison - Diffuse Daytime Night Overlay images for comparison diffuse_roughness 0 diffuse_roughness 0.33 diffuse_roughness 0.66 diffuse_roughness 1 50
Arnold From front From above Comparison - Diffuse - Direct Light Overlay images for comparison diffuse_roughness 0 diffuse_roughness 0.33 diffuse_roughness 0.66 diffuse_roughness 1 51
MaterialX View From front From above Overlay images for comparison Comparison - Diffuse - Direct Light diffuse_roughness 0 diffuse_roughness 0.33 diffuse_roughness 0.66 diffuse_roughness 1 52
MaterialX View From front From above From Arnold's value, we found that dividing by Π√2 gives a match. Next, compare the diffuse_roughness of MaterialX View to 1/4.44 Overlay images for comparison Comparison - Diffuse - Direct Light diffuse_roughness 0 diffuse_roughness 0.33 diffuse_roughness 0.66 diffuse_roughness 1 53
Arnold From front From above Overlay images for comparison Comparison - Diffuse - Direct Light diffuse_roughness 0 diffuse_roughness 0.33 diffuse_roughness 0.66 diffuse_roughness 1 54
From front From above Comparison - Diffuse - Direct Light Overlay images for comparison MaterialX View diffuse_roughness 0 diffuse_roughness 0.075 diffuse_roughness 0.15 diffuse_roughness 0.225 55
Match. We found that there were changes since Arnold 6.0.1.0 that made a difference. Next, check with indirect light. From front From above Comparison - Diffuse - Direct Light Overlay images for comparison MaterialX View diffuse_roughness 0 diffuse_roughness 0.075 diffuse_roughness 0.15 diffuse_roughness 0.225 56
MaterialX View In Arnold, diffuse_roughness shows that the coarser it gets, the darker it gets, but MaterialX View shows no difference in diffuse_roughness. Next, check the indirect light using HDRI White indirect light Overlay images for comparison Comparison - Diffuse - Indirect Light diffuse_roughness 0 diffuse_roughness 0.33 diffuse_roughness 0.66 diffuse_roughness 1 59
• In MaterialX View, the IrradianceMap corresponding to HDRI is used, but if it is unavailable, the HDRI is converted to SH 2nd order in MaterialX View and used as IrradianceMap. • In this case, we did not prepare an IrradianceMap, which is different from the SH 2nd order. Therefore, check the difference. Comparison - Diffuse - IBL Indirect Light 60 • base(1.0) • base_color(1, 1, 1) • diffuse_roughness(0)
HDRI Result MaterialX View • This can be a problem because SH second order cannot reproduce rapid changes. • This method is not good at extreme IBLs like the second from the right. Comparison - Diffuse - IBL Indirect Light 63
Regarding Diffuse roughness, Arnold and MaterialX View do not consider the effect of inter-reflection between microfacets. On the other hand, the Roughness of metal reflection corresponds to the effect of inter-reflection. Inter-reflection between microfacet irregularities 64
Comparison - Metal - Direct Light specular_roughness 0 specular_roughness 0.1 specular_roughness 0.3 specular_roughness 0.5 specular_roughness 0.7 specular_roughness 1 From front From above Overlay images for comparison Arnold 71
Comparison - Metal - Direct Light MaterialX View specular_roughness 0 specular_roughness 0.1 specular_roughness 0.3 specular_roughness 0.5 specular_roughness 0.7 specular_roughness 1 From front From above Overlay images for comparison 72
Concentric pattern problem in MaterialX View It can be improved by changing the sampler for reflection from Anisotropy to Linear. Comparison - Metal - Indirect Light 77
Comparison - Metal - Color difference by microfacet Check the effect by adding color to the base color 78 • base(1) • base_color (0.9, 0.62, 0.19) • metalness (1) • specular(0) • Change specular_roughness
Arnold specular_roughness 0 specular_roughness 0.33 specular_roughness 0.66 specular_roughness 1 Daytime Night Daytime Night MaterialX View The rougher the specular_roughness, the darker the color. Comparison - Metal - Color difference by microfacet 79
specular_roughness 0 specular_roughness 0.33 specular_roughness 0.66 specular_roughness 1 Arnold Daytime Night Overlay images for comparison Comparison - Metal - Color difference by microfacet 80
MaterialX View specular_roughness 0 specular_roughness 0.33 specular_roughness 0.66 specular_roughness 1 Daytime Night Overlay images for comparison Comparison - Metal - Color difference by microfacet 81
MaterialX View Since the result is difficult to understand, check with white indirect light specular_roughness 0 specular_roughness 0.33 specular_roughness 0.66 specular_roughness 1 Daytime Night Overlay images for comparison Comparison - Metal - Color difference by microfacet 82
Comparison - Metal - Color difference by microfacet specular_roughness 0 specular_roughness 0.33 specular_roughness 0.66 specular_roughness 1 Overlay images for comparison White indirect light Arnold 83
MaterialX View The effect of darkening colors is weaker in MaterialX View. Comparison - Metal - Color difference by microfacet specular_roughness 0 specular_roughness 0.33 specular_roughness 0.66 specular_roughness 1 White indirect light Overlay images for comparison 85 The red component is bright even if coarse, so the attenuation is small. The blue component is highly attenuated, so the color is darker.
not filled (thin_walled enabled) filled (thin_walled disabled) We are enabling and disabling thin_walled and the difference between DoubleSide and SingleSide. Comparison - Transmission - Thin walled and DoubleSide specular_roughness 0 specular_roughness 0.33 specular_roughness 0 specular_roughness 0.33 DoubleSide SingleSide DoubleSide SingleSide 93
specular_roughness 0 specular_roughness 0.1 specular_roughness 0.3 specular_roughness 0.5 specular_roughness 0.7 specular_roughness 1 Arnold 昼 の シ ー ン The background is also blurred because it is refracted by the roughness of the surface (thin frosted glass). Furthermore, it is emphasized because it is taken from a distance with a telephoto lens. 400mm lens Distance 38m 21mm lens Distance 3m Both specular_roughness is 0.1 in the following figure, but the impression changes with the lens. 95 MaterialX View does not support refraction of transmitted light due to specular_roughness and its variation. Therefore, Arnold also renders with less variation for comparison. Comparison - Transmission - Transmission by specular_roughness
Comparison - Transmission - extra roughness Variation of refracted light depends on surface roughness (specular_roughness) transmission_extra_roughness can give additional Roughness of the materials It is possible to make the surface smooth and the materials coarse without using coat. It is also possible to make the surface rough but the materials smooth again. specular_roughness 0 specular_roughness 0.5 transmission_extra_roughness 0.5 transmission extra_roughness -0.5 specular_roughness 0 specular_roughness 0.5 96 (Is this difficult in reality?)
Comparison - Transmission Since refraction is not available in MaterialX View, Arnold uses Extra roughness to compare under the same conditions. • specular(1) • specular_color(1, 1, 1) • Change specular_roughness • specular_IOR(1.5) • transmission(1) • transmission_color(1, 1, 1) • transmission_extra_roughness(roughnessCounter) • thin_walled(true) Extra roughness is grayed out in the UI when thin walled is enabled but can be used. 97
specular_roughness 0 specular_roughness 0.1 specular_roughness 0.3 specular_roughness 0.5 specular_roughness 0.7 specular_roughness 1 Arnold Daytime Comparison - Transmission The value of specular_roughness was countered with transmission_extra_roughness to keep the transmitted light from being scattered. Overlay images for comparison 98
specular_roughness 0 specular_roughness 0.1 specular_roughness 0.3 specular_roughness 0.5 specular_roughness 0.7 specular_roughness 1 Daytime Comparison - Transmission MaterialX View MaterialX View does not maintain high brightness. The higher the transparency, the more problematic it is. Blending is a problem. At least blending with multiplied alpha would improve it. Overlay images for comparison 100
specular_roughness 0 specular_roughness 0.1 specular_roughness 0.3 specular_roughness 0.5 specular_roughness 0.7 specular_roughness 1 Daytime Comparison - Transmission Only specular_roughness 0 is very different. In Arnold, only when thin walled is enabled during transmission and specular_roughness is 0, there is a phenomenon where reflections become stronger. We reported this to Autodesk and received a response that there was a problem. In-house Lookdev environment Overlay images for comparison 104
MaterialX View Color is not supported in MaterialX View and is converted to gray internally. Alpha blend does not support color. In the in-house LookDev environment, blend using the pixel shader Src1Color. specular_roughness 0 specular_roughness 0.1 specular_roughness 0.3 specular_roughness 0.5 specular_roughness 0.7 specular_roughness 1 Daytime Overlay images for comparison Comparison - Transmission - Transmission color 108
Colored the same way as Arnold's StandardSuraface. Compare with Arnold again In-house Lookdev environment specular_roughness 0 specular_roughness 0.1 specular_roughness 0.3 specular_roughness 0.5 specular_roughness 0.7 specular_roughness 1 Daytime Comparison - Transmission - Transmission color Overlay images for comparison 109
Except for the problem when specular_roughness is 0 Same result as Arnold's aiStandardSuraface In-house Lookdev environment specular_roughness 0 specular_roughness 0.1 specular_roughness 0.3 specular_roughness 0.5 specular_roughness 0.7 specular_roughness 1 Daytime Comparison - Transmission - Transmission color Overlay images for comparison 112
Comparison - Thin Film 113 https://docs.arnoldrenderer.com/ https://en.wikipedia.org/wiki/Thin-film_interference Interference occurs when the thickness becomes as thin as the wavelength of light. No thin film With thin film
Comparison - Thin Film Enable only Specular and change the thickness of the film (thickness unit is nm) • specular(1) • specular_color(1, 1, 1) • specular_roughness(0) • specular_IOR(1) • transmission(0 and 1) • transmission_color(1,1,1) • thin_film_IOR(1.4) • Change thin_film_thickness 114
thin_film_thickness 350nm thin_film_thickness 500nm thin_film_thickness 650nm Comparison - Thin Film transmission 0 thin_film_thickness 350nm thin_film_thickness 500nm thin_film_thickness 650nm transmission 1 MaterialX View Overlay images for comparison Daytime White IBL (Black background color) 116 In transmission1, it is thin due to the blending problem mentioned earlier. Check with in-house LookDev that solves it.
thin_film_thickness 350nm thin_film_thickness 500nm thin_film_thickness 650nm Comparison - Thin Film transmission 0 thin_film_thickness 350nm thin_film_thickness 500nm thin_film_thickness 650nm transmission 1 In-house Lookdev environment Overlay images for comparison Daytime White IBL (Black background color) Compare with Arnold again in this state 117
opacity 1,1,1 opacity 1, 0.95, 0.95 opacity 1, 0.5, 0 opacity 1, 0, 0 Daytime MaterialX View In MaterialX View, opacity colors are converted to grayscale and processed. Blending using Src1Color as before Overlay images for comparison Comparison - Opacity – Opacity color 124
What does it mean that Opacity is colored? Is there such a thing in the real world? Transmission: Transmits a specific color, and the rest are absorbed. Opacity: Transmits a specific color and reflects the rest. Comparison - Opacity – Opacity color 129 R G B RGB Opacity Transmission
Reflection is real. R G B RGB 130 For example, a dichroic mirror transmits specific wavelengths and reflects the rest. Comparison - Opacity – Opacity color RGB Blue (Transmission) Red (Reflection)
Comparison - Sheen 131 With Sheen No Sheen https://docs.arnoldrenderer.com/ sheen is the density of raised hairs sheen_roughness is the variation in the orientation of the raised hairs
Comparison - Sheen – Direct Light sheen_roughness 0 sheen_roughness 0.05 sheen_roughness 0.2 sheen_roughness 0.33 sheen_roughness 0.66 sheen_roughness 1 From front From above Arnold Overlay images for comparison 136
Comparison - Sheen – Direct Light MaterialX View sheen_roughness 0 sheen_roughness 0.05 sheen_roughness 0.2 sheen_roughness 0.33 sheen_roughness 0.66 sheen_roughness 1 From front From above Overlay images for comparison 138 Also check with indirect light.
MaterialX View uses curvature to represent subsurface, so the result differs from Arnold. MaterialX View does not support subsurface_anisotropy and does not support indirect light. Comparison - Subsurface 142
Arnold's subsurface type is random walk by default. Change subsurface_scale (the coefficient on radius) Only use Specular and Subsurface • specular(1) • specular_color(1, 1, 1) • specular_IOR(1.5) • specular_roughness(0) • subsurface(1) • subsurface_color(0.84, 0.55, 0.45) • subsurface_radius(1, 0.1, 0.03) • Change subsurface_scale Comparison - Subsurface 143
Comparison - Subsurface Daytime MaterialX View subsurface_scale 0.01 subsurface_scale 0.03 subsurface_scale 0.1 subsurface_scale 0.3 subsurface_scale 1.0 The colors are very different from Arnold’s. Arnold seems to be darker than the color specified by subsurface_color (MaterialX View has more natural colors). Check SubsurfaceColor only with directional light source and white directional light. Overlay images for comparison 146
subsurface_color 0.64, 0.64, 0.64 subsurface_color 0.09, 0.09, 0.09 subsurface_color 0.64, 0.25, 0.04 subsurface_color 0.16, 0.25, 0.64 subsurface_color 0.64, 0.81, 0 Directional light White IBL Comparison - Subsurface – Subsurface color Overlay images for comparison MaterialX View 154 The subsurface_radius on the left is 0.01 (1cm), but there is a problem that the width of the subsurface cannot be reduced.
Comparison about the problem that Subsurface Radius cannot be reduced • subsurface(1) • subsurface_color(1, 1, 1) • subsurface_radius(1, 0.1, 0.03) • Change subsurface_scale Check with direct light only Comparison - Subsurface – Subsurface color 155
Arnold Red diffuse transmitted light is gradually penetrating. subsurface_scale 0.01 subsurface_scale 0.05 subsurface_scale 0.2 subsurface_scale 0.5 From the side Overlay images for comparison Comparison - Subsurface – Subsurface scale 156
If SubsurfaceRadius is smaller than a specific value, there is a problem that it cannot be made smaller. If the SubsurfaceRadius is smaller than a specific value, the color will be gray. The problem is that it cannot blur only a few centimeters. (The sphere in the figure has a radius of 1 meter.) There is a limit on the radius in the code, so remove the limit. MaterialX View subsurface_scale 0.01 subsurface_scale 0.05 subsurface_scale 0.2 subsurface_scale 0.5 From the side Overlay images for comparison Comparison - Subsurface – Subsurface scale 157
SubsurfaceScale0.01 could be expressed. We have posted a GitHub issue about this. There was a problem that the degree of hanging changed depending on the camera zoom (the model was scaled); this has also been reported. Next, compare with indirect light. subsurface_scale 0.01 subsurface_scale 0.05 subsurface_scale 0.2 subsurface_scale 0.5 In-house Lookdev environment From the side Overlay images for comparison Comparison - Subsurface – Subsurface scale 158
The problem that subsurface_radius cannot be made slight and subsurface_color color does not match has been improved and compared. Comparison - Subsurface 161
In-house Lookdev environment Closer to Arnold Arnold has a feature that when subsurface_radius is relatively large, the complementary color appears on the opposite side, but MaterialX View does not have this feature. Also, subsurface_anisotropy is not supported. Daytime subsurface_scale 0.01 subsurface_scale 0.03 subsurface_scale 0.1 subsurface_scale 0.3 subsurface_scale 1.0 Overlay images for comparison Comparison - Subsurface 164
MaterialX View uses Subsurface with Curvature, so objects with a slight curvature or flat surfaces will not be the same as Arnold. Comparison – Subsurface - Shapes that cannot use curvature 165
Thin Walled during Subsurface This is used to express the diffuse transmission of a thin plate-like paper or leaves. The value of Subsurface determines how much light is diffused to the back side. • base(1) • base_color(1, 1, 1) • Change subsurface • subsurface_color(0, 1, 0) • subsurface_scale(1) Comparison - Subsurface –Thin Walled 169
weight 0 weight 0.1 weight 0.3 weight 1 MaterialX View Daytime Light source from the left IBL Comparison - Subsurface –Thin Walled Overlay images for comparison 172 No difference between Arnold and MaterialX View
There is a problem for emission. Colored coat for emission only • emission(1) • emission_color(1, 1, 1) • coat(0 and 1) • coat_color(White(1, 1, 1) Red(1, 0.05, 0.02) Green(0.02, 1, 0.02)) 177 Comparison - Coat - Relation to emission
Overlay images for comparison coat 0 coat_color (1,1,1) coat_color (1, 0.05, 0.02) coat_color (0.02 ,1, 0.02) 180 Comparison - Coat MaterialX View Daytime Daytime(Enlarge) The reflection at the edges is uncomfortable. The coat should be on top of the emission layer, but it doesn't look like it.
Overlay images for comparison coat 0 coat_color (1,1,1) coat_color (1, 0.05, 0.02) coat_color (0.02 ,1, 0.02) 184 Comparison - Coat MaterialX View Black IBL Black IBL(Enlarge) We have posted about this issue in the MaterialX View GitHub issue In addition, Arnold does not support the coat_affect parameter (aiMaterialX supports it), but there may be a problem with coat_affect_color processing, so it is under investigation (already reported)
The significant difference is the unsupported transmission. Other than that, it is relatively close with some modifications. However, the performance is not practical in many parts, so it is necessary to customize the contents for each layer. MaterialX is a good reference. The important thing is to follow the layer structure of Standard Surface. If you follow it, even if the processing in each layer is slightly different, there will not be a big difference overall. Is MaterialX practical for Video Game Graphics? 185
Performance measurement • Since video game graphics require performance, we measured performance using our in-house Lookdev environment. OS Windows11 21H2 CPU Core i9-10900X GPU Geforce 2080 Ti Resolution 3840 x 2160 Number of sphere model vertices 6559 Measurement software PIX 2203.30 Measurement Environment Screenshot 187
Importance sampling • MaterialX uses importance sampling with a Radiance environment map and GGX Directional Albedo. • MaterialX provides alternative functions for performance. (There are some side effects when using Pre-filter, for example, Specular Anisotropic cannot be used.) Radiance Environment Map GGX Directional Albedo Drawcall duration(msec) Screenshot Filtered importance sampling (256 sampling) Importance sampling (64 sampling) 52.82 Pre filter Importance sampling (64 sampling) 23.35 Pre filter Approximate function (Curve fit) 1.07 Standard Surface with all elements 188
Deletion of Standard Surface elements • The rendering duration can be reduced by outputting a shader with the minimum necessary elements for each material. For example, remove transmission and metal calculations. Diffuse Subsurface Translucent Sheen Transmission Dielectric Metal Coat Emission Output Diffuse Translucent Dielectric Output Standard Surface with all elements Node structure with some elements reduced (same as traditional PBR) Emission Metal 189
Performance measurement example Standard surface element Drawcall duration(msec) Screenshot Diffuse, Subsurface, Translucent, Sheen, Transmission, Dielectric, Metal, Coat, Emission (All elements) 1.07 Diffuse, Translucent, Dielectric, Metal, Emission (Traditional PBR) 0.48 Diffuse, Dielectric, Emission 0.29 Diffuse 0.13 • Pre Filter for Radiance environment map and approximate function for GGX Directional Albedo. • The reduction of material elements not related to the representation can reduce rendering duration. 190
MaterialX file export • In-house Lookdev environment exports material parameters to MaterialX files so MaterialX View can display the same materials. MaterialX View In-house Lookdev environment MaterialX file (.mtlx) 192
MaterialX file import • The in-house Lookdev environment can import simple materials from AMD's MaterialX Library and other sources, which helps render comparisons. (https://matlib.gpuopen.com/main/materials/all) Import result to in-house Lookdev environment AMD’s MaterialX Library 193
Conclusion • Standard Surface can now be applied in video game graphics. • Artists can now control video game materials with the same material parameter names as the DCC tool. • It is now possible to import MaterialX format materials from publicly available material libraries and compare drawing results with offline renderers such as Arnold. • Since MaterialX has become a standard language, it has become possible to discuss openly with other companies. • Information is communicated to the Arnold development team, and verification questions are posted to the MaterialX GitHub issue. 194
Standard Surface guide for Artists • The in-house LookDev environment has a Standard Surface hierarchy diagram for artists, with a guide that shows which layer corresponds to the material parameter currently being edited. • Clicking on a material element in the diagram selects the corresponding parameter. Using the guide 199
BaseColor alpha value • The node configuration allows switching using the BaseColor alpha value as Opacity or Transmission. • Using the alpha value of the BaseColor texture as Opacity is similar to traditional game graphics. Uses for BaseColor alpha values Opacity output Transmission output same as Standard Surface Opacity value Transmission value Opacity BaseColor alpha Opacity value Transmission Opacity value BaseColor alpha Opacity 0.1 (10% translucency in traditional game graphics) Transmission 0.9 (Specular is not missing) 201
Radiance environment map mip offset • MaterialX uses a fixed value (-1.5) as the mipmap offset when referencing the radiance environment map in filtered importance sampling. • However, this value depends on the HDRI resolution (2048x1024) included with the MaterialX View, so the in-house Lookdev environment uses a variable value. Offset -1.5 Offset 0.5 MaterialX View In-house Lookdev environment Offset -1.5 202
Irradiance Environment Map • MaterialX View can choose to use a pre-created Irradiance environment map or to bake an Irradiance environment map based on SH values. • Still, we decided to use SH values to have the same values as the in- house LookDev environment. The SH values inside MaterialX View are output as text and referenced in the in-house LookDev environment. Irradiance environment map baked based on SH values SH HDRI MaterialX View In-house Lookdev environment 203
Dichroic filter • Since Opacity has RGB information, the material can now represent a dichroic filter using the pixel shader Src1Color (Dual source blending). Blend Settings Direct Lighting Indirect Lighting Standard Surface Pixel shader output(Src0) Pixel shader input Transmission Pixel shader output(Src1) Rendering result Dichroic filter example(Photo) 204
Difference between DiffuseRoughness and SpecularRoughness • Specular is greatly affected by fine irregularities on the surface, but Diffuse is not. • Snow and skin are also diffuse, but they are the same, with more light penetrating the interior than ordinary objects. What is DiffuseRoughness? 205
Refraction and Reflection of Light What is DiffuseRoughness? 206 Light entering and exiting a plate of glass at IOR 1.5 Intensity is distributed by reflection angle, refraction angle, and critical angle. Reflection and refraction several times
Cross section of nonmetallic object What is DiffuseRoughness? 207 Incident light Specular reflection (one reflection refraction) An object composed of IOR 1.5 and air crystals Outgoing light is indicated by the yellow line. Diffuse reflection (multiple reflection refraction)
Cross section of nonmetallic object What is DiffuseRoughness? 208 Incident light Specular reflection (one reflection refraction) When the surface is polished smooth, the specular becomes sharp. Diffuse changes tiny and has little effect on surface roughness. Diffuse reflection (multiple reflection refraction)
Cross section of nonmetallic object What is DiffuseRoughness? 209 What does it mean to have a rougher diffuse surface? Differences appear in more significant irregularities than in specular roughness. Incident light Diffuse reflection (multiple reflection refraction) Specular reflection (one reflection refraction)
Cross section of nonmetallic object What is DiffuseRoughness? 210 Specular reflection (one reflection refraction) An example of coarse diffuse but smooth specular About the same as the coat Diffuse reflection (multiple reflection refraction) Viewed from a distance
Effects of diffuse_roughness on microfacets between uneven shape Arnold and MaterialX View are both Oren-Nayer, but they do not consider the bounce between the bumps. • The coarser the color, the darker the color • However, the brighter the albedo, the more the bounce mitigated the effect. • These are not taken into account. Inter-reflection between dielectric microfacet irregularities 211
Rendering with a nonmetallic model of fine irregularities such as the following Inter-reflection between dielectric microfacet irregularities 212 Rendered with more bounce times 500 convexities 100,000 convexities
Rendering with variation in uneven height difference Inter-reflection between dielectric microfacet irregularities 213 base_color is coarse in white but not darkened. 500 convexities 100,000 convexities
Indirect light set to 0.5 gray Inter-reflection between dielectric microfacet irregularities 214 If base_color is white, it is coarse but not darkened If bace_color is white, there is no energy loss, so it will look the same as the background even if it is coarse 500 convexities 100,000 convexities
Oren-Nayar Reference Inter-reflection between dielectric microfacet irregularities 215 https://mimosa-pudica.net/improved-oren-nayar.html Parts that are brighter as the base_color are brighter ρ: bese_color Coefficient multiplied by base_color. The rougher the sigma (something like roughness), the larger Losses are avoided in the following areas
Oren-Nayar Reference Inter-reflection between dielectric microfacet irregularities 216 https://mimosa-pudica.net/improved-oren-nayar.html Comparison If the loss is compensated
Oren-Nayar Reference Inter-reflection between dielectric microfacet irregularities 217 https://mimosa-pudica.net/improved-oren-nayar.html Comparison If the loss is compensated
Oren-Nayar Reference Inter-reflection between dielectric microfacet irregularities 218 https://mimosa-pudica.net/improved-oren-nayar.html Comparison If the loss is compensated As diffuse_roughness is increased, the image becomes brighter and the colors darker. For example, on desert ground, the difference in appearance is significant.
Is Arnold's aiMaterialXShader the same as aiStandardSurface? We compared some parameters. Are aiStandardSurface and aiMaterialXShader the same? 221 Basically, the same result. Same for Maya standardSurface
bandainamcostudios
gaelvaroquaux
th0x0472
miyake
takatsunobuhiro
line_developers_tw
nktamago
ikezoemakoto
ramimac
motsuhiro131
line_developers
prashantspol
_kensh
thoeni
erikaheidi
bryan
chrislema
62gerente
robhawkes
philhawksworth
jcasabona
addyosmani
bkeepers
hatefulcrawdad