Wide gamut texture workflow using DCC tools

This is the English translation of CEDEC2023 "Wide gamut texture workflow using DCC tools".
This document describes how artists can create wide color gamut textures such as Rec2020 using DCC tools and OpenColorIO, and the limitations of object colors to consider when working with wide color gamut textures.

Transcript

1. Wide gamut texture workflow using DCC tools Bandai Namco Studios Inc. Wataru Tada Masayuki Suzuki Shohei Yamaguchi
2. Self introduction Past Projects ・Ace Combat 6 ・Ace Combat: Assault Horizon ・Tales Of Card Evolve ・THE IDOLM@STER MILLION LIVE! ・Pokkén Tournament ・New Pokémon Snap Wataru Tada Past talks ・CEDEC2011 Continuous integration of ACE COMBAT ASSAULT HORIZON ・CEDEC2011 In-game camera production example in “ACE COMBAT ASSAULT HORIZON” in-game camera production example ・CEDEC2015 Game editor design and implementation in Pokkén Tournament ・CEDEC2019 Basic design of game editor and stable and fast asset management in console video game development ・CEDEC2019 Implementation of Light Baker using DirectX Raytracing ・CEDEC2021 New Pokémon Snap Lighting Overview ・CEDEC2022 Application of MaterialX and Standard Surface to Video Game Graphics Past Projects ・New Pokémon Snap Shohei Yamaguchi Past talks ・CEDEC2020 HDRI production technique with high reliability ・CEDEC2021 New Pokémon Snap Lighting Overview Masayuki Suzuki Past talks ・CEDEC2008 Linear space and physically correct lighting ・CEDEC2019 "Super Smash Bros. Ultimate" - To draw the borrowed IP as beautifully as possible ・CEDEC2020 HDRI production technique with high reliability ・CEDEC2021 Video Game Graphics for the Raytracing Generation, Get Grounded and Prepared - Quickly and Easily Validate Next-Generation Quality Real-Time Physically-Based Lighting with NVIDIA Falcor ・CEDEC2021 New Pokémon Snap Lighting Overview ・CEDEC2022 Application of MaterialX and Standard Surface to Video Game Graphics I have worked as a character modeler, motion, background, lighting artist, shader artist, and now as a TA, I research graphics and support work for various projects.
3. Contents • Fundamentals • Implementation • DCC tools and OCIO • Color gamut and monitors • Data debugging
4. Fundamentals
5. Color gamut • Color gamut is the range of color saturation. It becomes a triangle in the RGB three primaries standard of monitors and projectors
6. sRGB usual color gamut • The color gamut most familiar to us is sRGB
7. Wide color gamut • A wide gamut is a color gamut expressing higher saturation than sRGB. The triangles on the chromaticity diagram of that gamut are larger than those of sRGB • For example, ACES, AdobeRGB, DCI-P3, Rec.2020, etc • The workflow of this document uses Rec.2020
8. What are the advantages of a wide color gamut? • Wide gamut extends the expression of high color saturation And improve the adjustment range of grading Difference in saturation between sRGB and Rec.2020 Rec.2020 sRGB Differences in grading results Rec.2020 sRGB
9. How the wide gamut image looks like • When displaying images in the sRGB gamut and Rec.2020 gamut, the sRGB gamut image appears to be more saturated and the Rec.2020 gamut image appears to be less saturated • This is the case when displayed as it is without any settings • Correct color management makes the colors look the same sRGB Rec.2020 sRGB Rec.2020
10. sRGB monitor Rec.2020 monitor(SDR) ⬆ sRGB Image *The Rec.2020 image has moved
11. Pixel value and color • Which is more saturated, (1.0, 0.0, 0.0) or (1.0, 0.0, 0.0)? → I don't know the correct answer. May be the same, may be different • Why？ • → Because there is no specification of color space • (1.0, 0.0, 0.0) of Rec.2020 and (1.0, 0.0, 0.0) of sRGB, Rec.2020 has higher saturation • In other words, the color is determined only when the pixel value is set with the color space • If you've only ever worked in sRGB, you might think, "I never thought about that!"
12. Pixel value and color • Considering pixel values along with color space, (1.0, 0.0, 0.0) in sRGB and (0.627, 0.069, 0.016) in Rec.2020 are the same color
13. Color space
14. sRGB Rec.2020 RGB_to_XYZ ・Cube is RGB space ・Point cloud is the color of the object ・ Cube and point cloud transform to XYZ and fit in 0-1 ・XYZ space represents absolute coordinates, and RGB space represents relative coordinates ・ XYZ is not intuitive to display directly because it is used for storage or intermediate conversion ・ The feature is that Y represents the brightness *Rec.2020 has less outliers from the object color and less deformation to XYZ
15. sRGB Rec.2020 XYZ_to_xyz ・The figure shows the ratio of each of XYZ as xyz, where x, y, and z add up to 1 ・ The xy chromaticity diagram shows this from above
16. sRGB Rec.2020 xyz_to_xyY ・Use brightness Y in the height direction for the xy chromaticity diagram *Since the xy chromaticity diagram differs from the human senses, the width of the color gamut is not intuitive. You don't feel the difference unless you make a significant change in the green coordinates, but you feel the difference in the blue coordinates with a bit of change Reference, u'v' chromaticity diagram and LAB in the supplementary materials
17. What materials exist outside the sRGB color gamut?
18. Measurement outside the sRGB color gamut • The method is to measure with a colorimeter to check if it is outside the sRGB gamut • The camera shoots RAW to determine if it is outside the sRGB gamut based on the camera's native color gamut (color reproduction is not accurate)
19. Measurement outside the sRGB color gamut Konica Minolta Luminance Color Meter CS-150 α7R III Develop settings • White balance is set up for D65 to be white • Develop neutrally in the camera's native color gamut • Applied the camera matrix when determining the color gamut
20. Measurement outside the sRGB color gamut sRGB
21. Measurement outside the sRGB color gamut Rec.2020
22. Measurement outside the sRGB color gamut sRGB
23. Confirmed out of sRGB color gamut from shooting
24. Confirmed out of sRGB color gamut from shooting
25. Confirmed out of sRGB color gamut from shooting
26. Confirmed out of sRGB color gamut from shooting
27. Confirmed out of sRGB color gamut from shooting
28. Confirmed out of sRGB color gamut from shooting
29. What materials exist outside the sRGB color gamut? Summary • Outside, the sRGB gamut can be checked with a measuring instrument • Also, cameras can be used to check to some extent • These results also take lighting into account • Some plant flowers, leaves, signs, and light-emitting signs are outside the sRGB gamut • Those outside the sRGB gamut are within the gamut in Rec. 2020
30. Differences between rendering and photography in a narrow color gamut
31. Differences between rendering and photography in a narrow color gamut • Differences in color gamut affect post-processing, etc • Difference in color of high brightness and blooms • Difference in White Balance • Reasons • With a narrow gamut, high-saturation pixels have zero or small channels, leaving little room for them, while the camera processes with a wide gamut and then converts to the display space
32. High brightness color difference • Exposure adjustment in the sRGB color gamut causes discomfort in saturated bright areas •As a result, the periphery and the center are also primarily green •Also, this result is correct because the center has a more significant value •There is no discomfort when viewed on an HDR monitor, but SDR makes it look uncomfortable
33. High brightness color difference • Exposure adjustment in the sRGB color gamut causes discomfort in saturated bright areas • Images taken with a camera (linear development) •As a result, the periphery and the center are also primarily green •Also, this result is correct because the center has a more significant value •There is no discomfort when viewed on an HDR monitor, but SDR makes it look uncomfortable •As a result, the center color was white •Since the camera has a wide color gamut, all channels are subject to change, and bright pixels become white •After development, even if you change the color gamut to sRGB, white parts are still white
34. Verification using HDR photos Bracket shooting and convert to HDR while keeping the camera color gamut Luminous color used High brightness color difference
35. The result of converting camera color gamut HDR image to sRGB Photo (linear development) High brightness color difference
36. The result of clamping an HDR image with SDR in the camera color gamut state and then converting it to sRGB Photo (linear development) High brightness color difference
37. High brightness color difference
38. camera space R G B 0.99 3.53 1.20 sRGB space R G B 0.95 1.02 1 camera space R G B 0.99 1 1 sRGB space R G B 0 5.06 0 High brightness color difference • The left node is converted directly to sRGB, and the right node is clamped to the camera gamut and then converted • It is better to soft clip depending on the target brightness of the monitor
39. Difference in white balance x y 0.685481, 0.314148 0.437236, 0.518884 0.185182, 0.723421 0.159008, 0.361109 0.13555, 0.0514863 0.30236, 0.13068 Experiments in white balancing with high-saturation light
40. Difference in white balance • Camera color gamut allows white balance without problems, even with high saturation Develop with camera color gamut
41. Difference in white balance • Camera color gamut allows white balance without problems, even with high saturation Develop with camera color gamut Converted to sRGB color gamut with white balance adjusted in-camera color gamut Color checker color differences are caused by color rendering of high saturation lights, etc., but rendering should not cause problems
42. Difference in white balance • The sRGB gamut is narrow, so high saturation creates a 0 channel Develop with sRGB color gamut
43. Difference in white balance • The sRGB gamut is narrow, so high saturation creates a 0 channel Develop with sRGB color gamut White balance cannot be matched because the value of 0 cannot be restored
44. Differences between rendering and photography in a narrow color gamut Summary • Photographs have a wide color gamut internally and are often processed as such • Also, because photos have a wide color gamut, high luminance tends to be white (except for HDR photography) • If post-processing is done in a narrow gamut, even if it is a narrow gamut output, it will not look the same as in the camera *May be improved by reducing the saturation and then processing and restoring the saturation regardless of the gamut
45. About object color
46. About object color Diffuse Albedo (base color) of object colors can also be out of the sRGB gamut for highly saturated objects, and object colors with specular removal tend to be more saturated
47. Color checker with specular removal Using a polarizing filter and a colorimeter, we tested the color checker specular removal When removing specularity, we attached a polarizing film to the light and a polarizing filter for the camera to the measuring device The light is placed at a 45-degree angle, and measurements are taken from the front When no polarization is used, no22 is adjusted so that the reflectance is 0.19 (8bit value 121) → (Details are [CEDEC2020] "HDRI production technique with high-reliability" p153) When using polarized light, no22 was adjusted so that the reflectance was 0.138 (8bit value 104) *The reflectance of 0.138 (8-bit value 104) is estimated by photographing the transmission of unpolarized light diffusion by shining light on the paper from the reverse side, examining the attenuation of the polarization filter, and comparing it with regular photography *It is more accurate to use a colorimeter that can remove specular using polarized light (measurement illumination condition M3)
48. Color checker 1 out of sRGB color gamut Color checker with specular removal 6 out of sRGB color gamut 116 83 69 194 147 127 92 119 155 92 107 65 128 126 173 97 186 169 220 124 47 69 92 171 194 86 97 89 57 102 163 189 65 222 156 39 35 64 147 70 146 70 169 60 57 233 196 19 184 86 148 0 132 163 241 239 234 197 198 198 159 160 161 121 121 121 83 84 85 51 50 51 102 60 39 185 133 109 71 103 142 72 91 38 117 114 163 79 179 161 216 112 0 45 81 163 187 67 81 75 35 88 155 180 41 222 150 0 0 37 137 44 137 43 161 0 15 233 190 0 179 64 135 0 120 155 241 238 232 193 194 192 151 151 151 104 104 104 59 62 62 9 16 15 Color checker with specular removal
49. Color checker Color checker with specular removal Color Checker sRGB out-of-gamut display (cyan) Color Checker sRGB out-of-gamut display (cyan) Color checker with specular removal
50. Texture with specular removal Rendering with arnold Rendering with specular disabled(experiment) Rendering Color checker Rendering result Texture with specular removal is dark, but it is added to the rendering result to look natural *This result uses textures in the sRGB gamut, so it lacks saturation.Rec.2020 textures should be used
51. Texture without specular removal Rendering with arnold Rendering Color checker Rendering result With this texture, the specular is double ridden, so the rendered result is brighter
52. Example of specular removal texture Before removal
53. After removal Example of specular removal texture
54. After removal Example of specular removal texture Example of out of sRGB gamut due to specular removal
55. Before removal Example of specular removal texture
56. After removal Example of specular removal texture
57. After removal Example of specular removal texture Example of out of sRGB gamut due to specular removal
58. Before removal Example of specular removal texture
59. After removal Example of specular removal texture
60. After removal Example of specular removal texture Example of out of sRGB gamut due to specular removal
61. Diffuse transmission also has no specular component, so the texture is darker in color Example of diffuse transmission texture Diffuse reflection (specular removal) Diffuse transmission (specular does not enter)
62. Example of diffuse transmission texture Example of color outside the sRGB gamut due to diffuse transmission
63. MacAdam limit MacAdam limit of object color boundary and Munsell color system
64. • The figure shows the MacAdam limit on xyY chromaticity, representing the theoretical color range of diffuse reflection • If it reflects all wavelengths of white light, it is white • The height axis represents Y (luminance) MacAdam limit MacAdam limit is the theoretical range that can be reproduced as an object color
65. Calculation method • The outermost color is the reflection cut off from only a portion of the visible light wavelength • The width of the change is plotted as xyY as the wavelength of the reflection In this movie, only the reflection characteristics are plotted for clarity, but in reality, the wavelength distribution of D65 is further multiplied and then plotted in xyY MacAdam limit For each reflected wavelength on the left side of the figure, multiply by the XYZ isochromatic function to obtain the XYZ value, which is then converted to xyY and plotted
66. MacAdam limit and color gamut The figure shows MacAdam limit overlaid with sRGB and DiffuseAlbedo that can be expressed in Rec.2020 sRGB : Many areas are not covered Rec.2020 : The coverage area is large, but there are some areas that exceed the MacAdam limit
67. Munsell color system Munsell color system and sRGB color gamut • The Munsell color system divides hue, lightness, and saturation into equally spaced parts humans can perceive • It is defined as the edge of an object's color, but only as far as it can be reproduced as a color sample
68. Munsell color system and sRGB color gamut • The figure shows the Munsell color system plotted in xyY Use real.dat below https://www.rit.edu/science/munsell-color-science-lab-educational-resources Since real.dat is Illuminant C, it is converted to D65 using the Bradford method and plotted
69. Munsell color system and sRGB color gamut • The figure shows the Munsell color system overlaid with the sRGB area
70. Munsell color system and sRGB color gamut • The figure shows the Munsell color system overlaid with the Rec.2020 area
71. Wide gamut notes Wide gamut can easily exceed object colors • For example, RGB(0.8,0,0) in Rec.2020 is out of object color • On the other hand, RGB(0.8,0.8,0.8) has no problem • The above problem does not occur in the sRGB gamut sRGB primary colors Rec.2020 primary colors Rec.2020 primary color diffuse exceeds object color Figure shows a photo rendered in Rec. 2020 and displayed on a Rec. 2020 HDR monitor and developed in the Rec. 2020 gamut *When checking on Rec.2020's HDR monitor, Rec.2020's primary color materials appear to be luminescent objects
72. About object color Summary • Diffuse Albedo needs to remove specular. • Diffuse Albedo with specular removed tends to have high saturation • The diffuse color range is basically within the MacAdam limit • Drawing a texture directly with a wide color gamut may exceed the object's color
73. About fluorescence
74. About fluorescence • Fluorescence absorbs a specific wavelength and emits a longer wavelength • It also absorbs blue and radiates green, invisible ultraviolet light, and blue • It can also exceed object colors due to its high saturation and brightness • The fluorescent component emits light with an intensity proportional to the incident light, so it can be considered a Diffuse Albedo?
75. About fluorescence Photo of fluorescent material on a white desk R G B In the red channel, some are much brighter than the white desk
76. About colorimeter Nix Spectro 2 https://www.nixsensor.com/nix-spectro/ Measuring the color of fluorescent materials with a colorimeter • Spectrophotometer (10nm step) • When measuring fluorescent colors, use measurement illumination condition M1 (including ultraviolet light)
77. yellow orange pink green blue R G B R G B R G B R G B R G B XYZ 0.775 1.088 0.120 XYZ 1.005 0.667 0.047 XYZ 0.821 0.441 0.557 XYZ 0.221 0.538 0.104 XYZ 0.213 0.315 0.749 sRGB 0.780 1.294 -0.052 sRGB 2.208 0.279 -0.031 sRGB 1.704 0.055 0.545 sRGB -0.165 0.800 0.013 sRGB -0.167 0.416 0.739 Rec.2020 0.913 1.243 0.080 Rec.2020 1.476 0.409 0.033 Rec.2020 1.111 0.175 0.521 Rec.2020 0.161 0.725 0.079 Rec.2020 0.064 0.379 0.695 Fluorescent pen measurement Rec. 2020, the color will be within the gamut if the reflectance is 100% or more In sRGB, areas with negative values are not saturated enough, even if the reflectance is 100% or more Measurements on white paper with a fluorescent pen
78. Fluorescent pen measurement sRGB Rec.2020 物体色 In the diagram, the difference cannot be seen on the plane • Relationship between fluorescence and color gamut Three points were outside the object color None are included in sRGB
79. Fluorescence looks different depending on the light source • The actual visibility of fluorescent materials is greatly affected by differences in the color rendering properties of the light and the amount of UV light • If a fluorescent material is used as a Diffuse Albedo, it will look the same if it is defined as a light with good color rendering and the same UV intensity as a D light source • The images below are photos of the sun and shade; the yellow and orange fluorescence is much more noticeable in the shade • Blue sky light occupies many blue and ultraviolet rays (even if the sun is blocked, blue and ultraviolet rays tend to remain) In the sun Shade (blue sky)
80. R G B In the sun Shade (blue sky) Fluorescence looks different depending on the light source • In blue skylight, orange and pink react strongly even though the light is weak when viewed with the R channel • Yellow reacts strongly in the G channel
81. Diffuse Albedo IBL Directional light Rendering space and other settings Traditional workflow Wide color gamut workflow Rendering fluorescence Differences from normal sRGB color gamut flow • Diffuse Albedo requires a value of 1 or more, so EXR was used. • The light values are Rec.2020, and the rendering color gamut is also Rec.2020 • The display used sRGB (if viewing on a Rec.2020 monitor, the display should be Rec.2020) • The figure shows the result of rendering with the wide gamut workflow *Actually, the blue color of this white paper print exceeds 8bit255 White paper often contains optical brighteners
82. Rendering fluorescence Rendering in the sun Photo in the sun Rendering in the shade Photo in the shade • Since the colorimeter value is a D light source, including UV light, the sun view is compatible and appears the same • In the shade, the amount of blue and ultraviolet light is higher, so the rendering results do not show fluorescence as prominently as in the photo
83. About fluorescence Summary • Fluorescence becomes highly saturated and may exceed the object color • Fluorescence may also be considered as Diffuse Albedo (only the first reflection) • However, not all fluorescence can be reproduced under all lighting conditions
84. Implementation
85. Application example of wide color gamut texture • HDR applications (games using HDR) are examples of applications for wide color gamut textures • The color gamut of HDR applications is Rec.2020, so to take advantage of the color gamut, render in Rec.2020 and then convert the color gamut to match the monitor • Wide color gamut textures are effective in rendering with Rec.2020 Rendering (Lighting, Post-processing...) HDR monitor SDR monitor Color gamut conversion Rec.2020 sRGB
86. Texture color gamut Rendering(Rec.2020) • When rendering in Rec.2020 color gamut, if textures were created in sRGB color gamut, gamut conversion was performed offline or in the shader Color gamut conversion Shader code example Texture(sRGB) Texture(Rec.2020)
87. Limitations due to color gamut conversion • The sRGB color gamut is narrower than the Rec.2020 color gamut, so converting an sRGB texture to a Rec.2020 texture will limit the texture values Color gamut conversion example sRGB (1,0,0) → Rec.2020 (0.6274,0.0691,0.0164) • In PBR, BaseColor textures are used as reflectance for diffuse reflections; when BaseColor textures are converted from sRGB to Rec.2020 gamut, reflectance is limited → In this case, highly reflective materials cannot be rendered Materials that cannot be expressed with sRGB textures
88. Materials that cannot be expressed with sRGB textures • We cut a green paper cup and pasted it on the monitor to see if we could reproduce the color of the paper in Photoshop • When the monitor had the sRGB color gamut, the green color of the paper could not be reproduced, but when the monitor color gamut was set to Rec.2020, the green color of the paper could be reproduced →This green paper is a material that cannot be expressed with sRGB • Figure shows a camera shot with the monitor set to the sRGB color gamut • The paper and monitor colors do not match • Figure shows a camera shot with the monitor set to Rec. 2020 color gamut • The paper and monitor colors match. Green paper cup *In this figure, the lighting is not D65, but indoor fluorescent light, so the color is not correct as albedo *For reference, the color of the colorimeter
89. Create Rec.2020 texture Rendering(Rec.2020) • To avoid gamut conversion limitations, Rec. 2020 textures must be created without gamut conversion from sRGB to Rec. 2020 Texture(Rec.2020) Wide color gamut image DCC Tools
90. How to create Rec.2020 textures • When creating textures from images taken by a camera, such as photogrammetry or HDRI, the camera's color gamut is wider than Rec. 2020, so Rec. 2020 textures can be created by gamut conversion Image captured by the camera Rec.2020 Texture Color gamut conversion
91. How to create Rec.2020 textures • When creating a Rec.2020 texture using the DCC tool, use the DCC tool's color management to output the Rec.2020 texture Rec.2020 Texture Texture output DCC Tools
92. Texture format Rec.2020 8bit png sRGB 8bit png • Rec.2020 color gamut textures can also be used in 8-bit format • Wide color gamut textures do not require rich image formats
93. sRGB flag • Human eyes are sensitive to dark areas • To increase the amount of information in dark areas in the 8-bit texture, the texture has an sRGB flag • The GPU converts the sRGB flag to a linear value so it can be processed at high speed (DXGI_FORMAT_R8G8B8A8_UNORM_SRGB in DirectX) • 8-bit Rec. 2020 textures are also used with the sRGB flag turned on to increase the amount of information in dark areas linear value Values stored in textures with the sRGB flag enabled linear value GPU 0 0.25 0.5 0.75 1.0 0 0.25 0.5 0.75 1.0 0 1.0 0.5371 0.7354 0.8808
94. About gradation • Although the 8-bit Rec.2020 texture has a wider color gamut than sRGB, the amount of 8-bit information has not changed so the gradation may be insufficient → We tested with 8-bit png in the Rec.2020 gamut with gradients and found that the lack of gradation was not a problem (sRGB flag of textures is enabled) Rec.2020 8bit png sRGB 8bit png
95. sRGB Rec.2020 Comparison using sRGB 8-bit gradation (256-tone gradation with dither disabled, width 2048 pixels) Rec.2020 items are based on sRGB gradients → sRGB degamma → Rec.2020 color gamut conversion → sRGB gamma encoding → Save as 8bit Rec.2020 texture → sRGB degamma → sRGB color gamut conversion → sRGB gamma encoding About gradation sRGB Rec.2020 sRGB Rec.2020 sRGB Rec.2020 sRGB Rec.2020 sRGB Rec.2020 sRGB Rec.2020 • There is no problem when viewed typically, but when emphasized, the gradation degradation in the highly saturated areas is apparent • If dither is added to this result, it becomes less noticeable
96. Rec.2020 texture creation environment for DCC tools • When creating Rec. 2020 textures in DCC tools, artists want to display the preview screen gamut as Rec. 2020 on the monitor • Windows HDR applications have a Rec.2020 color gamut, so if a DCC tool is built as an HDR application, the preview screen gamut can be displayed on an HDR monitor as Rec.2020 →We checked if the DCC tool is an HDR application HDR monitor DCC tool
97. Determine HDR/SDR for the application • In preparation, connect an HDR monitor to the PC and enable HDR in the Windows settings • Applications whose window brightness does not change when the slider bar of "SDR Content Brightness" in the HDR settings is moved are HDR applications. The application that changes the brightness of the window is an SDR application When the slider bar for "SDR Content Brightness" is set to the leftmost position, the monitor will output 80 nits Chrome displaying HDR video (HDR application) Notepad (SDR application)
98. HDR support status of DCC tools • The "SDR Content Brightness" slider controls show that DCC tools such as Substance Painter and Maya are created as SDR applications →Since the color gamut of the preview screen of the DCC tool is not Rec.2020, it is supported by workflows such as monitors and external tools • Color management is required to create Rec.2020 textures in DCC tools for SDR applications There are also DCC tools, such as Krita, that are written as HDR applications with an OpenGL to DirectX conversion layer https://www.intel.com/content/dam/develop/external/us/en/documents/painting-with-light-krita.pdf Substance Painter Maya
99. DCC tools and OCIO
100. How to create a wide color gamut texturet • Even if the DCC tool is an SDR application, the DCC tool can create Rec. 2020 textures if it supports color management, such as OpenColorIO (OCIO) Rec.2020 Texture Texture output Substance Painter
101. OpenColorIO(OCIO) • OCIO is a set of libraries and settings for color management • It is available on github • Version 2.0.0 was released in January 2021 (OpenColorIO v2) • OCIO can define gamut conversion by writing a Config file • From an artist's point of view, it appears as a color management system on DCC tools https://github.com/AcademySoftwareFoundation/OCIO
102. Pre-installed OCIO Config • Substance Painter has pre-installed OCIO Config, selectable from project settings • C:¥Program Files¥Adobe¥Adobe Substance 3D Painter¥resources¥ocio • We can read the configuration file to understand the process Folder config.ocio substance Workflow for linearizing sRGB aces_1.0.3 ACES AP1 (ACEScg) is the working color space aces_1.2
103. Loading custom OCIO Config • Open the project settings and select "OpenColorIO" for color management and "Custom" for OpenColorIO configuration to create your own settings
104. Traditional Color Management • Project settings with color management set to Legacy • The workflow was based on sRGB sRGB texture loading Working color space (sRGB) Export sRGB texture Substance Painter internals and monitor color gamut
105. Color management using OCIO • Separate control of working color space and display color space by OCIO Config • Separate control can be made for reading and exporting Bitmap import color space Export color space (sRGB/Rec.2020) Working color space (Linear/Rec.2020) Display color space (sRGB/sRGB) Monitor color gamut Color gamut inside Substance Painter
106. Create a custom OCIO Config
107. What the pre-installed Config does not solvet 1. Change to Rec. 2020 base 2. Maintain viewport look and feel 3. Export in Rec.2020 gamut Folder config.ocio substance Workflow for linearizing sRGB aces_1.0.3 ACES AP1 (ACEScg) is the working color space aces_1.2
108. Creating OpenColorIO Config • Instead of writing OpenColorIO Config from scratch, download OpenColorIO Config for ACES1.2 from github, modify some items, and use it • https://github.com/colour-science/OpenColorIO-Configs
109. OCIO Config • config.ocio is written in text format (YAML)
110. roles • In OCIO, a " roles " field sets the color space used in each steps of the color pipeline • Used as a color management system by various DCC tools, but may have its own rules
111. Working color space
112. Working color space • "Working color space" is the basic color space in which internal calculations are performed • Generally, ACES AP0 or ACES AP1 is often used as the working color space • ACES AP0 = ACES2065-1 • ACES AP1 = ACEScg • The working color space was set to Rec.2020 • Because we are building a workflow for HDR gaming • It is important to set it to the workflow you intended
113. roles: scene_linear • In Substance Painter, the scene_linear role corresponds to the working color space • By changing scene_linear from ACEScg to Linear - Rec.2020, you can work with Rec.2020 inside Substance Painter OCIO Config Substance Painter
114. Viewport
115. Viewport color management • When using OCIO for CMS (color management system), viewport color management is enabled • However, ACES 1.2 includes RRTs in the viewport display • ※RRT(Reference Rendering Transform) A look that reflects the characteristics of the film Reference: https://area.autodesk.jp/column/trend_tech/scene_linear_workflow/04_fujifilm/ • It looks different from software not managed by OCIO • Therefore, edit the OCIO configuration file to display the software as per the standard without RRT
116. Viewport and RRT • Looking at the config.ocio in the preinstalled aces_1.2 folder, Output - sRGB is available for ColorSpace • However, 3DLUT (Log2_48_nits_Shaper.RRT.sRGB.spi3d) that includes RRT is used
117. Viewport color management sRGB sRGB+RRT
118. ViewTransform colorspaces • Added an item to colorspaces to specify sRGB as the standard display color space • Create ColorSpace for Output with reference to Utility - sRGB – Texture transform
119. ViewTransform colorspaces • We would also like to check on a Rec.2020 monitor, but as with sRGB, Output - Rec.2020 for Output includes RRT • We created our own ColorSpace
120. displays • Added the colorspace created earlier as a View to displays
121. active_views • Just adding colorspace to displays does not display it in the viewport menu • The colorspace is displayed in the viewport by setting it to active_views • By setting the View we just created as active_views, we can now select colorspace from the viewport in Substance Painter Substance Painter OCIO Config
122. DCC Tool OCIO Config item Maya displays Houdini (Karma) active_views Substance Painter active_views DCC tools and OCIO Config • OCIO Config specification is fixed, but each DCC tool has a different Config display name DCC tool View settings
123. Texture output
124. Color space when exporting • We want to export with Rec.2020 color gamut, sRGB gamma • Since it is a special color space, we also create this • Name it Utility - sRGB - Rec.2020 - Texture
125. Color space when exporting • Substance Painter's import/export colorspace is defined in config.ocio with its own role • Roles are interpreted differently depending on the software • Reference: https://OCIO.readthedocs.io/en/main/guides/authoring/overview.html#roles
126. Color space when exporting • Rec.2020 can now be set as the color space for export by setting the OCIO Config • The texture export dialog shows that the output color space has been changed
127. Texture Examples • BaseColor textures from JadeToad sample exported as 8bit png in sRGB gamut and Rec.2020 gamut Rec.2020 sRGB JadeToad sample
128. Color gamut and monitors
129. Wide color gamut display in viewport • Substance Painter is an SDR application, so the viewport can only display sRGB color gamut →To preview Rec.2020 textures, support workflows such as monitors and external tools • There are three types of workflow depending on the monitor • A monitor that can display a wide color gamut regardless of Windows settings • General HDR Monitor • SDR monitor
130. A monitor that can display a wide color gamut regardless of Windows settings • By setting the monitor gamut to Rec. 2020, the artist can work with a wide gamut of colors in the Substance Painter preview window • OCIO Config uses a value that takes into account the gamma of the monitor • This method is a wide gamut display, so it is not possible to display Substance Painter as an HDR application (GUI other than the preview window is also affected by the gamut change) ASUS ProArt PA32UCX-P ColorEdge PROMINENCE CG3146
131. General HDR Monitor • Substance Painter's preview window cannot preview Rec.2020 textures, so use a tool made as an HDR application to preview Rec.2020 textures Substance Painter Rec.2020 Texture HDR support tools
132. SDR monitor • On SDR monitors, colors outside the sRGB gamut are clipped, so work with textures carefully
133. Data debugging
134. Data debugging tools • It is difficult to detect errors in the color gamut of textures visually, so create a data debugging tool such as an image viewer to deal with it Debug tool example Feature Purpose Texture gamut conversion Investigate color gamut errors Display pixels outside the sRGB gamut Check if the material cannot be expressed without using Rec.2020 textures Error display for pixels outside the object color Check to see if textures are being created in colors that are not intended Example of debug features
135. Object color determination • The tool reads the plot value for object color determination as a 3D texture and uses the 3D texture value inside the shader Pixels that exceed the object color are displayed in purple error color The original image 3D texture size for object color determination is 256x256x256 Houdini Plot value for object color Shader code example
136. Saturation change due to color gamut difference • The saturation of the texture changes depending on the color gamut of the monitor and texture • If the texture looks low or high saturation, check the color gamut and monitor settings Texture type sRGB monitor sRGB monitor (Color managed to software) Rec.2020 monitor Rec.2020 monitor (Color managed to software) sRGB Texture correct saturation correct saturation high saturation correct saturation Rec.2020 Texture low saturation correct saturation correct saturation correct saturation sRGB texture converted to Rec.2020 low saturation correct saturation correct saturation correct saturation
137. HDR Screenshot • Working from home, it is not possible to see other artists' monitors directly • Screen-sharing software such as Windows remote desktop and messenger applications do not support HDR → To share information about HDR, use Win + Alt + Print Screen keys to take HDR screenshots (.jxr files) or convert render targets to AVIF files AVIF file Image formats that support HDR released in 2019
138. Information sharing through AVIF files • Windows pre-installed Photo and Paint cannot display HDR image files • There is an HDR viewer on the Microsoft Store, but getting all project members to install it is difficult • Therefore, we convert HDR screenshots to AVIF files and share the information using Chrome’s HDR feature →We also share information by uploading AVIF files to our internal web server Displaying AVIF files in Chrome
139. Conclusion • By widening the color gamut, it is possible to express colors that cannot be expressed with sRGB in the real world. • A wide color gamut expands the range of editing. Wide color gamut textures can be created by color management with DCC tools • In that case, it is necessary to consider the monitor's specs and settings and the data's color gamut • It is also possible to debug wide color gamut textures and share information
140. References • https://OCIO.org/ • https://github.com/AcademySoftwareFoundation/OCIO • https://OCIO.slack.com/. • Color management with OCIO | Substance 3D Painter • https://acescentral.com/
141. Questions?
142. Supplemental Materials
143. HDR on Windows • Windows displays HDR and SDR applications on the same monitor by managing the frame buffer with Float16 format scRGB • To create an HDR application, use DirectX and set DXGI to Rec.2020, or use UWP to create an application →DCC tools not using DirectX or UWP will likely be created as SDR applications https://learn.microsoft.com/en-us/windows/win32/direct3darticles/high-dynamic-range SDR application sRGB frame buffer HDR application Rec.2020 frame buffer scRGB frame buffer Monitor
144. Substance Painter Color Picker • The color picker displays the values of the working color space →When the working color space is set to Rec. 2020, RGB values in the Rec. 2020 gamut can be easily checked
145. AVIF file conversion tool example DirectXTex::LoadFromEXRFile etc Image file Pixel value Rec709 Linear OETF RGBA 16bit float or 32bit float RGB for libavif Y410(YUV444 10bit) avifImageRGBToYUV avifEncoderAddImage、avifEncoderFinish AVIF file PQ encoded RGB uint16_t format Convert to Rec.2020 PQ with CPU
146. HDR image viewer example • Since the publicly available HDR image viewers lacked functionality, we are using our own HDR image viewer • The window part is created with .NET (C#), and the image display part is created with DirectX12 as a separate process for HDR display (Inter-process communication uses TCP/IP) • This viewer has functions such as changing the color space of textures, displaying outside the sRGB color gamut, determining object colors, and displaying picked colors in an xy chromaticity diagram
147. sRGB out-of-gamut determination example • The tool converts RGB to xy values inside the shader and determines internally of the triangle whether it falls within sRGB of the xy chromaticity diagram The figure shows pixels outside the sRGB gamut in a purple error color The original image
148. sRGB out-of-gamut determination example The video shows the color of the cursor position being checked on an xy chromaticity diagram
149. Object color judgment example The figure shows whether the texture color at the cursor position is outside the object color • Example using highlighter texture: A cross-section of MacAdam limit is displayed depending on the brightness • Even white paper has a color that exceeds the object's color due to optical brighteners
150. • The video shows an AVIF file being dragged, dropped into Chrome and displayed. • The dynamic range differs from the Photoshop display in the upper right (sky, etc.) • It also has a brighter display than the white SDR Notepad
151. • The video shows how information is shared by uploading AVIF files to the company's Confluence.
152. Chromaticity diagram closer to human senses than xy chromaticity diagram sRGB sRGB LAB u’v’ chromaticity diagram
153. Chromaticity diagram closer to human senses than xy chromaticity diagram u’v’ chromaticity diagram LAB Rec.2020 Rec.2020 Even though there does not seem to be much difference between sRGB and Rec.2020 red in the xy chromaticity diagram, the coordinates in this chromaticity diagram show that there is quite a difference
154. Questions or Comments • Please contact us if you have any questions or comments • Wataru Tada: w-tada@bns-g.com • Masayuki Suzuki: m6-suzuki@bns-g.com • Shohei Yamaguchi: s6-yamaguchi@bns-g.com