Virtual texturing proposal

A overview of the virtual texturing project I am currently undertaking for my honours degree. It is a fairly high level overview of virtual texturing and the methodology I will be undertaking in developing and utilizing a virtual texturing system.

Transcript

1. Increasing Visual Fidelity Using Virtual Texturing Honours Proposal Stuart Milne 5th December 2012
2. Contents 1. Abstract.....................................................................................................................................3 2. Introduction...............................................................................................................................3 3. Research Question .....................................................................................................................4 4. Literature Review .......................................................................................................................5 4.1 Background ..........................................................................................................................5 4.2 Paging ..................................................................................................................................5 4.3 Page translation ...................................................................................................................7 4.3 Page faults ...........................................................................................................................7 4.4 Streaming ............................................................................................................................8 4.5 Filtering ...............................................................................................................................8 5. Methodology .............................................................................................................................9 5.1 Methods overview ................................................................................................................9 5.2 Project plan ........................................................................................................................ 10 5.3 Risk assessment ..................................................................................................................... 11 6. Summary ................................................................................................................................. 11 References .................................................................................................................................. 12 Bibliography ................................................................................................................................ 14
3. 1. Abstract Traditional texturing approaches suffer from limitations imposed by an inefficient use of graphics memory. These limitations have a significant impact on the visual fidelity of real- time rendered applications. Virtual texturing is a technology which utilises graphics memory in an efficient manner by intelligently streaming necessary texture data on demand at runtime. This addresses the limitations imposed by traditional texturing approaches and allows for an increase in texture detail in real-time applications. The proposed research will investigate the methods utilised in the implementation of an efficient virtual texturing system. The research will include possible optimisations and workarounds for limitations unique to virtual texturing solutions. The project will produce an implementation of a virtual texturing system for use in a real-time application. The developed system will be utilised in an application designed to demonstrate the capabilities of the approach. The artefact produced will be used as a baseline to compare against a comparable traditionally textured approach in terms of both visual fidelity and performance criteria. 2. Introduction When rendering realistic real-time scenes in games and graphical applications much of the perceived realism is achieved through the use of texture mapping. The resolution and amount of textures used in current generation games continues to increase, however, the amount of graphics memory available in hardware is not increasing at the same pace. This is evident on current generation console systems on which it is not uncommon to find texture sizes 2-4 times smaller than their PC counterparts. The PC edition of Max Payne 3 (Rockstar Vancouver, 2012), for example, shipped with texture assets four times the size of the console editions (Burnes, 2012). This is primarily due to the disparity in available graphics memory between current generation consoles and modern enthusiast video cards found in PC hardware. There is an increasing demand for higher quality textures as consumer graphics become more sophisticated and screen resolution’s increase. However conventional texture mapping techniques are currently limited by the memory available on consumer hardware. Traditional texturing approaches utilise the graphics hardware inefficiently in terms of memory utilisation. This places limitations on the amount of texture detail in a rendered view of a real-time scene and correlates directly to the perceived visual fidelity of the application. As stated by Mittring (2008a, p. 24) “it is no longer possible to keep all textures in graphic card memory and not even in main memory”. Virtual texturing is a technology which utilises graphics memory more efficiently than traditional texturing approaches. Utilising techniques conceptually similar to virtual memory
4. (Denning, 1970a) in modern operating systems, it is possible to utilise much larger, and therefore more detailed, or varied, textures sets when rendering a scene in real-time. The aim of this project is to develop a real-time virtual texturing solution that demonstrates improved visual fidelity over traditional texture mapping. This development will include the identification of appropriate virtual texturing techniques, the development of a design for realising these techniques, and the subsequent implementation of this design. The artefact produced will be evaluated by comparing performance measurements against non-virtual approaches and a comparison of the complexity of textures which can be rendered. The expected quantitative increase in texture complexity leads to increased subjective visual fidelity. A rigorous approach to the latter subjective evaluation is outside the scope of this project but may be demonstrated with a qualitative comparison of the final artefact. The remainder of this proposal identifies the research question associated with this aim, provides a preliminary review of literature supporting the process, outlines the methodology to be adopted in achieving the aim, and presents a project plan. 3. Research Question How can virtual texturing techniques be utilised to increase visual fidelity in real-time rendering applications?  What are the existing methods utilised in virtual texturing solutions?  What optimisations can be performed on the data representation and algorithms to maximise the potential gains in memory usage and performance?  What elements of a real-time rendered scene can benefit most from virtual texturing techniques?  How does virtual texturing compare against a traditional texturing approach in terms of performance and detail?  What are the limitations and engine design implications when utilising a virtual texturing system for rendering?  What improvements and future work on virtual texturing techniques can be made based on the results of these investigations?
5. 4. Literature Review 4.1 Background Although real time virtual texturing solutions have only recently been demonstrated in real- time consumer applications (Splash Damage, 2011; id Software 2011), similar methods have been studied and implemented in offline rendering for at least two decades. Peachy, then working for Pixar Animation Studios outlined an offline rendering technique dubbed ‘Texture On Demand’ (Peachy, 1990a) which closely resembles the tiling and streaming techniques used in virtual texturing. Texture tiling techniques have previously been described and implemented to save graphics memory in terrain rendering (Cosman, 1994) and image rendering (nVidia, 2004). Tanner, Midgal and Jones (1998) described a limited form of virtual texturing dubbed clipmapping for massive terrain rendering that utilised mipmap pyramids similar to the quad tree representation used in virtual texturing. In 2007 id Software realeased Quake Wars: Enemy Territory which utilised a system similar to clipmapping called “MegaTexture” and later announced an improved version of this technology (Carmack, 2005, Cited in Anonym, 2005) which removed the geometry limitations inherent in the clipmap based system. This revised megatexture technology is considered to be the first implementation of virtual texturing as it is used today. 4.2 Paging The underlying principle of virtual texturing is based upon the virtual memory technique utilised by modern operating systems. Virtual memory is a technique in which all forms of a machine’s data storage (main memory, hard disk) are virtualised into one addressable memory pool. This gives the illusion of an increased memory store to the application using it (Denning, 1970b p. 156). Barrett (2008a) describes a technique which mimics this approach by dividing relatively large textures, typically residing on disk or optical storage, into tiles, or pages. The individual pages are then cached and loaded onto the GPU on demand for rendering a given frame. To the renderer it appears that a high resolution texture is in texture memory when in reality only the individual pages necessary to draw the current frame are present. Figure 1: Texture paging; only the tile the pixel belongs to is loaded into graphics memory (Peachy, 1990b)
6. With this approach, as new pages are required they are asynchronously requested from the slower memory they are resident on and stored in a page cache in main or graphics memory. In order to make these requests it is necessary to identify which pages are required to render the view. Barrett (2008b) describes a technique commonly adopted (Chajdas et al. 2010a, pp. 186-187; Mayer, 2010a, pp. 39-41; Mittring, 2008b, p.26) for determining the visible pages and identifying the ones not present in the cache. The approach outlined utilises a custom fragment shader to output the tile coordinates for each texel in the view into a texture buffer which is made available to the CPU. The CPU analyses the visible tile texture and requests any pages which are not currently present in the page cache. Mayer (2010b) identifies other determination methods including;  The world space tile determination strategy used for clipmap based solutions (Tanner, Migdal and Jones, 1998b)  Pre-calculated tile determination using potential visible sets (Airy, 1990)  Texture space conservative tile determination (Lefebvre, Darbon and Neyret, 2004). A detailed investigation of these alternative techniques is outside the scope of this proposal. Page size plays an important role in effective cache utilisation. Increased page size results in more wasted pixels in the page cache (only some of the texels in the page are required for rendering the view). However smaller page sizes require a larger lookup table, increased management overhead and impacts streaming performance. Mayer (2010c, pp. 26-28) investigates the performance implications of varying page sizes and determines a maximum page size of 256 x 256 pixels is advisable for optimal performance on current hardware.
7. 4.3 Page translation Once present in the page cache, the location of a page bears no relation to those in the virtual texture. It is necessary to translate the texture coordinates of the virtual texture into valid coordinates of the texels in graphics memory. Continuing with the approach outlined above this is usually achieved using a page lookup table, also called an indirection texture (Mittring, 2008c, p. 26). The indirection texture is updated by the CPU when pages are added to the page cache. For each page in the virtual texture, it points to the corresponding page in the cache. A drawback of this approach is the size of the indirection table, which corresponds to the amount of tiles present in the virtual texture. Sufficiently large virtual textures can contain so many tiles that the physical size of the indirection texture incurs a large memory overhead. . Figure 2: Page translation using indirection texture (Hollenmeersch et al., 2010) Hollenmeersch et al. (2012) describe an alternative technique using a sorted list of virtual tile addresses present in the page cache to ascertain if the page is cached and a second list which maps each element in the sorted list to its position in the page cache. This technique addresses limitations in the use of an indirection texture as a lookup table is not required. 4.3 Page faults Unlike virtual memory, it is not feasible to stall real-time applications when a required page is not present in the cache. Barrett (2008c) proposes a strategy to ensure valid, ‘good enough’ texture data is available for every texel in a scene. This method is now ubiquitous in virtual texturing solutions (Obert, Waveren, and Sellers, 2012; Chajdas et al. 2010b; Mayer, 2010d; Mittring, 2008d).
8. This technique utilises a quad tree tile representation, each level of the quad tree corresponds to a higher mipmap (Valve software, 2008) level. At the root of the tree is the lowest mipmap level, equal in size to the page size. This mipmap contains the entire virtual texture in a single page, albeit at a low resolution. This approach, dubbed ‘sparse virtual textures’ (Barrett, 2008d) populates the page cache using a breadth-first tree traversal of the mipmap quad tree, ensuring any texture data required for rendering is present in the page cache at at-least one mipmap level. Figure 3: Sparse virtual texture (Barret, 2008) Figure 4: Quad tree of mipmaps. (Hollenmeersch eta al., 2010) 4.4 Streaming Tile streaming is typically performed asynchronously in separate threads while the render thread performs texture updates. It is important that the transfer of texture data is as quick as possible in order to minimise page cache faults which have a detrimental impact on visual quality as lower resolution mipmap fall-backs are used. The overall streaming latency is determined by bandwidth usage (more data will take longer to be streamed) and the time taken to compress/decompress this data. Mayer (2010e) identifies DXT based compression techniques as being well suited to the bandwidth demands of virtual texturing techniques and additionally provides a comparison of GPU and CPU based decompression methods. Waveren (2006) and Intel (2008) have performed investigations into current compression techniques in the real-time context and both provide parallel texture streaming solutions well suited to real-time applications which utilise large texture sets. 4.5 Filtering Texture filtering (Microsoft, 2012) is a technique applied in order to address the potential disparity between texels (elements of a texture) and pixels on the screen. Due to the texture indirection inherent in virtual texturing techniques special consideration must be taken when applying a filtering process on the texture data. (Lefebvre, S. and Neyret, F. pp. 210-211). Hollenmeersch et al. (2010, pp. 229-231) outline considerations to be taken regarding bilinear, trilinear and anisotropic filtering approaches and present a technique for manually selecting mipmaps for filtering purposes. A key concern is that native hardware filtering cannot be relied upon and manual mipmap selection must be performed. In addition the difference in texture locality in the page cache relative to the layout in the virtual texture can produce filtering artefacts if not addressed. A common solution to the latter consideration (Chajdas et al. 2010c. pp. 189-190; Mittring,
9. 2008e. p. 32; Waveren, 2009) is to apply borders, typically 4 pixels, around the pages. Theses borders contain the texel data present in the surrounding pages on the virtual texture, thus allowing artefact-free filtering. 5. Methodology 5.1 Methods overview In answering the research question a prototype application will be designed and developed that renders a textured city scene. The application will utilise the DirextX 11 graphics api. The application will be capable of rendering the scene with virtual texturing enabled, or disabled. When disabled, smaller texture assets, representative of the typical sizes used in real time applications, will be used in place of the large virtual textures. This artefact will be evaluated in terms of performance and visual fidelity across both approaches. In order to best demonstrate the potentials gains in visual fidelity virtual texturing offers the scene is large in scale, to allow the application to demonstrate texture detail at both a macro and micro scale. The scene is comprised of modelled city block utilised in an application previously developed by the researcher. The modelled city block has already been textured using relatively large texture assets (on average 4096 x 4096 pixels) which could be utilised by the virtual texturing system; however it is proposed that higher resolution textures will be created in order to fully utilise the potential of a virtual texturing approach. The higher resolution textures utilised by the virtual texturing system will be compared against the scene in its vanilla, traditionally textured form. In the eventuality that higher resolution assets cannot be created the current texture set will be used both with and without virtual texturing. This will allow relative performance characteristics to be compared and potential gains in terms of increased visual fidelity to be postulated based on the performance, particularly in memory utilisation, of each approach. Additionally the existing assets can be downscaled to a smaller resolution for use in the traditional texturing implementation in order to simulate the disparity in texture size typically found in comparison to virtual texturing. As the existing assets are larger than typically used for conventionally textured real-time applications this would not be an overly artificial approach in terms of evaluating the relative visual fidelity of each technique. The quantitative performance evaluation will be performed using metrics gathered by the application at runtime and through the use of profilers that will monitor memory consumption, bandwidth use and the frame rate for both texturing approaches. Memory utilisation in particular will be a prime indicator of the potential for increased visual fidelity due to memory savings. Nvidia’s PerfHUD and NSight, Microsoft’s GPUView and the Visual Studio Graphics Debugger will be used to perform this monitoring.
10. A qualitative comparison between the two texturing approaches will be performed and documented using comparison screenshots. The quantitative and qualitative evaluations will be analysed in order to identify correlations between performance data and marked improvements in visual fidelity. 5.2 Project plan The project will be broken down into the following phases:  Base application framework  Building a DirectX 11 application framework  Porting existing mesh rendering code to framework  Implementing performance measurement tools  Rendering and paging systems  Tile cache and address translation implementation  Page fault tolerance and LOD utilising mipmaps  Tile determination using render pre-pass  Virtual texture rendering  Filtering implementation  Streaming implementation  Compression/decompression implementation  Non virtual texture implementation  Decoupling of virtual texture specific code from rendering pipeline  Conventional texture mapping implementation  Optimisation (Optional)  Investigation of alternative techniques. ie. Sorted list tile determination  Utilisation of GPGPU techniques  Evaluation  Data collection  Critically evaluate results Dec Jan. Feb. March. April Base framework Paging systems Streaming strategies Rendering Trad. texturing Optimisation Evaluation Draft dissertation Final dissertation May
11. 5.3 Risk assessment The project proposed consists entirely of undertaking software development and research using standard development tools. The creation of texture assets is the only external dependency the project proposes. An alternative approach using existing assets has been identified in the methods overview which ensures that this dependency is not critical to the success of the project. Therefore there is no significant external risk to the success of the project. 6. Summary This project proposes to evaluate the potential for increased visual fidelity using virtual texturing techniques in real time applications. The existing literature indicates that the proposed techniques can provide a significant performance advantage in terms of graphics memory utilisation over conventional texture mapping techniques. Performance improvements in the outlined area should address current limitations imposed on texture assets by current hardware. This may allow for an increase in amount of texture detail that can be applied to real time graphics applications. If realised, these improvements will have a direct positive impact on the visual fidelity of an application utilising the described methods.
12. References Airey, J. M. 1990. Increasing update rates in the building walkthrough system with automatic model-space subdivision and potentially visible set calculations. [PhD Thesis]. The University of North Carolina. Barrett, S. 2008. Sparse Virtual Textures. [online]. Available from: http://silverspaceship.com/src/svt/ [Accessed 4 December 2012] Burnes, A. 2012. Max Payne 3 benchmarked in a New York minute. [online]. Gefoce.com. Available from: http://www.geforce.com/optimize/guides/max-payne-3-benchmarked-in-a- new-york-minute#1 [Accessed 4 December 2012] Carmack, J. 2005. Quakecon 2005 keynote. Cited in: Anonym. 2005. John Carmack’s quakecon 2005 keynote – complete transcript. [online]. GNT.com. Available from: http://us.generation-nt.com/john-carmack-quakecon-2005-keynote-complete-transcript-help- 29744482.html [Accessed 6 December 2012] Chajdas, M. G. et al. 2010. Virtual texture mapping 101. In: W. Engel, ed. GPU pro. MA: A K Peters Ltd. 2010. Pp.185-195 Cosman, M. 1994. Global terrain texture: lowering the cost. In E. G. Monroe, ed, Proceedings of 1994 IMAGE VII Conference, Tucson 12-17 June 1994. The IMAGE Society pp. 53-64 Denning, J. P. 1970. Virtual memory. ACM Computing Surveys. 2(3). pp. 153-189 Hollemeersch C-F. et al. 2010. Accelerating virtual texturing using CUDA. In: W. Engel, ed. GPU pro. MA: A K Peters Ltd. 2010. Pp.623-642 Hollenmeersch, C. F. et al. Texture-size independent address translation for virtual texturing. In: ACM SIGGRAPH 2012 Posters. 2012. New York: ACM. No. 33 id Software. 2011. Rage. [Disk]. PC, Microsoft Xbox 360 and Sony Playstation 3. Bethesda Softworks. Intel Corporation. 2008. Geospatial Texture Streaming From Slow Storage Devices.[online]. Available from: http://software.intel.com/en-us/articles/geospatial-texture-streaming-from- slow-storage-devices. [Accessed 5 December 2012] Lefebvre, S. Neyret, F. 2003. Pattern based procedural textures. In: M. Zyda, M. Capps, eds. I3D ’03 Proceedings of the 2003 symposium on Interactive 3d Graphics. San Diego 27-31 July 2003. New York: ACM. Pp. 203-212 Lefebvre, S. Darbon, J. Neyret, F. 2004. Unified texture management for arbitrary meshes.[online]. Available from: http://hal.inria.fr/inria-00070783/ [Accessed 5 December 2012]. Mayer, A. J. 2010. Virtual texturing. [M. Thesis]. Technischen Universität Wien.
13. Microsoft, 2012. Texture filtering. [online]. Available from: http://msdn.microsoft.com/en- gb/library/windows/desktop/bb206250(v=vs.85).aspx [Accessed 3 December 2012] Mittring, M. and Crytek GmbH. Advanced virtual texture topics. In: SIGRAPH ’09: ACM SIGRAPH 2008 Games. Los Angeles, 5-11 August 2008. New York: ACM. pp. 23-51 NVIDIA Corporation. 2004. Texture atlas whitepaper – improve batching using texture atlases. [online]. Available from: http://http.download.nvidia.com/developer/NVTextureSuite/Atlas_Tools/Texture_Atlas_Whit epaper.pdf [Accessed 4 December 2012] Obert, J., van Waveren, J.M.P. and Sellers, G. 2012. Virtual texturing in software and hardware. In: SIGRAPH 2012: ACM SIGRAPH 2012 Courses. Los Angeles 5-9 August 2012. New York: ACM. Peachy, D. 1990. Texture on demand. Pixar Animation Studios Technical Memo #217. [online]. Available from http://graphics.pixar.com/library/indexAuthorPeachey.html [Accessed 1 December 2012] Rockstar Vancouver. 2012. Max Payne 3. [Disk]. PC, Microsoft Xbox 360 and Sony Playstation 3. Rockstar Games. Splash Damage. 2011. Brink. [Disk]. PC, Microsoft Xbox 360 and Playstation 3. Bethesda Softworks. Tanner, C., Migdal, J. and Jones, M. 1998. The Clipmap: a virtual mipmap. In: S. Cunningham, ed. Proceedings of the 25th annual conference on Computer graphics and interactive techniques, Orlando, 19-24 July, 1998. New York: ACM. pp. 151-158 Valve Software, 2008. MIP Mapping. [online]. Available from: https://developer.valvesoftware.com/wiki/MIP_Mapping [Accessed 7 December 2012] Van Waveren, J. M. P. 2006. Real time texture streaming & decompression.[online]. Available from: http://software.intel.com/file/17248/ [Accessed 5 December 2012] Van Waveren, J. M. P. 2009. Id tech 5 challenges; from texture virtualisation to massive parallelisation. [online]. Available from: http://s09.idav.ucdavis.edu/talks/05- JP_id_Tech_5_Challenges.pdf [Accessed 7 December 2012]
14. Bibliography Wei, L-Y. 2004. Tile-based texture mapping on graphics hardware. In: M Pharr ed. GPU Gems 2. Boston: Addison-Wesley Professional. Cornel. D Texture virtualisation for terrain rendering.[online]. Available from: http://www.cg.tuwien.ac.at/courses/.../DCornel_TerrainRendering.pdf [Accessed 1 December 2012] Haines, E. Akenine-Moller, T. Hoffman, N. 2008. Realtime rendering. 3rd ed. MA. A K Peters Ltd. Neu, A. 2010. Virtual texturing. [Bachelor thesis]. Rheinisch Westfalische Technische Hochschule Aachen University. Olick, J. 2008. Current generation parallelism in games. In: SIGGRAPH ’08. ACM Sigraph 2008 classes. New York: ACM. Article No. 20. Weinhaus, F. M. Devarajan, V. 1997. Texture mapping 3D models of real-world scenes. R. Munt zed. ACM Computing Surveys 29(4) New York: ACM. pp. 325-365