Texture filtering

Last updated

In computer graphics, texture filtering or texture smoothing is the method used to determine the texture color for a texture mapped pixel, using the colors of nearby texels (ie. pixels of the texture).

Contents

Filtering describes how a texture is applied at many different shapes, size, angles and scales. Depending on the chosen filter algorithm, the result will show varying degrees of blurriness, detail, spatial aliasing, temporal aliasing and blocking. Depending on the circumstances, filtering can be performed in software (such as a software rendering package) or in hardware, eg. with either real time or GPU accelerated rendering circuits, or in a mixture of both. For most common interactive graphical applications, modern texture filtering is performed by dedicated hardware which optimizes memory access through memory cacheing and pre-fetch, and implements a selection of algorithms available to the user and developer.

There are two main categories of texture filtering: magnification filtering and minification filtering. [1] Depending on the situation, texture filtering is either a type of reconstruction filter where sparse data is interpolated to fill gaps (magnification), or a type of anti-aliasing (AA) where texture samples exist at a higher frequency than required for the sample frequency needed for texture fill (minification).

There are many methods of texture filtering, which make different trade-offs between computational complexity, memory bandwidth and image quality.

The need for filtering

During the texture mapping process for any arbitrary 3D surface, a texture lookup takes place to find out where on the texture each pixel center falls. For texture-mapped polygonal surfaces composed of triangles typical of most surfaces in 3D games and movies, every pixel (or subordinate pixel sample) of that surface will be associated with some triangle(s) and a set of barycentric coordinates, which are used to provide a position within a texture. Such a position may not lie perfectly on the "pixel grid," necessitating some function to account for these cases. In other words, since the textured surface may be at an arbitrary distance and orientation relative to the viewer, one pixel does not usually correspond directly to one texel. Some form of filtering has to be applied to determine the best color for the pixel. Insufficient or incorrect filtering will show up in the image as artifacts (errors in the image), such as 'blockiness', jaggies, or shimmering.

There can be different types of correspondence between a pixel and the texel/texels it represents on the screen. These depend on the position of the textured surface relative to the viewer, and different forms of filtering are needed in each case. Given a square texture mapped on to a square surface in the world, at some viewing distance the size of one screen pixel is exactly the same as one texel. Closer than that, the texels are larger than screen pixels, and need to be scaled up appropriately — a process known as texture magnification. Farther away, each texel is smaller than a pixel, and so one pixel covers multiple texels. In this case an appropriate color has to be picked based on the covered texels, via texture minification. Graphics APIs such as OpenGL allow the programmer to set different choices for minification and magnification filters. [1]

Note that even in the case where the pixels and texels are exactly the same size, one pixel will not necessarily match up exactly to one texel. It may be misaligned or rotated, and cover parts of up to four neighboring texels. Hence some form of filtering is still required.

Mipmapping

Mipmapping is a standard technique used to save some of the filtering work needed during texture minification. [2] It is also highly beneficial for cache coherency - without it the memory access pattern during sampling from distant textures will exhibit extremely poor locality, adversely affecting performance even if no filtering is performed.

During texture magnification, the number of texels that need to be looked up for any pixel is always four or fewer; during minification, however, as the textured polygon moves farther away potentially the entire texture might fall into a single pixel. This would necessitate reading all of its texels and combining their values to correctly determine the pixel color, a prohibitively expensive operation. Mipmapping avoids this by prefiltering the texture and storing it in smaller sizes down to a single pixel. As the textured surface moves farther away, the texture being applied switches to the prefiltered smaller size. Different sizes of the mipmap are referred to as 'levels', with Level 0 being the largest size (used closest to the viewer), and increasing levels used at increasing distances.

Filtering methods

This section lists the most common texture filtering methods, in increasing order of computational cost and image quality.

Nearest-neighbor interpolation

Nearest-neighbor interpolation is the simplest and crudest filtering method — it simply uses the color of the texel closest to the pixel center for the pixel color. While simple, this results in a large number of artifacts - texture 'blockiness' during magnification, [3] and aliasing and shimmering during minification. [4] This method is fast during magnification but during minification the stride through memory becomes arbitrarily large and it can often be less efficient than MIP-mapping due to the lack of spatially coherent texture access and cache-line reuse. [5]

Nearest-neighbor with mipmapping

This method still uses nearest neighbor interpolation, but adds mipmapping — first the nearest mipmap level is chosen according to distance, then the nearest texel center is sampled to get the pixel color. This reduces the aliasing and shimmering significantly during minification but does not eliminate it entirely. In doing so it improves texture memory access and cache-line reuse through avoiding arbitrarily large access strides through texture memory during rasterization. This does not help with blockiness during magnification as each magnified texel will still appear as a large rectangle.

Linear mipmap filtering

Less commonly used, OpenGL and other APIs support nearest-neighbor sampling from individual mipmaps whilst linearly interpolating the two nearest mipmaps relevant to the sample.

Bilinear filtering

In Bilinear filtering, the four nearest texels to the pixel center are sampled (at the closest mipmap level), and their colors are combined by weighted average according to distance. [6] This removes the 'blockiness' seen during magnification, as there is now a smooth gradient of color change from one texel to the next, instead of an abrupt jump as the pixel center crosses the texel boundary. [7] Bilinear filtering for magnification filtering is common. When used for minification it is often used with mipmapping; though it can be used without, it would suffer the same aliasing and shimmering problems as nearest-neighbor filtering when minified too much. For modest minification ratios, however, it can be used as an inexpensive hardware accelerated weighted texture supersample.

The Nintendo 64 used an unusual version of bilinear filtering where only three pixels are used, instead of four due to hardware optimization concerns. This introduces a noticeable "triangulation bias" in some textures. [8]

Trilinear filtering

Trilinear filtering is a remedy to a common artifact seen in mipmapped bilinearly filtered images: an abrupt and very noticeable change in quality at boundaries where the renderer switches from one mipmap level to the next. Trilinear filtering solves this by doing a texture lookup and bilinear filtering on the two closest mipmap levels (one higher and one lower quality), and then linearly interpolating the results. [9] This results in a smooth degradation of texture quality as distance from the viewer increases, rather than a series of sudden drops. Of course, closer than Level 0 there is only one mipmap level available, and the algorithm reverts to bilinear filtering.

Anisotropic filtering

Anisotropic filtering is the highest quality filtering available in current consumer 3D graphics cards. Simpler, "isotropic" techniques use only square mipmaps which are then interpolated using bi– or trilinear filtering. (Isotropic means same in all directions, and hence is used to describe a system in which all the maps are squares rather than rectangles or other quadrilaterals.)

When a surface is at a high angle relative to the camera, the fill area for a texture will not be approximately square. Consider the common case of a floor in a game: the fill area is far wider than it is tall. In this case, none of the square maps are a good fit. The result is blurriness and/or shimmering, depending on how the fit is chosen. Anisotropic filtering corrects this by sampling the texture as a non-square shape. The goal is to sample a texture to match the pixel footprint as projected into texture space, and such a footprint is not always axis aligned to the texture. Further, when dealing with sample theory a pixel is not a little square [10] therefore its footprint would not be a projected square. Footprint assembly in texture space samples some approximation of the computed function of a projected pixel in texture space but the details are often approximate, [11] highly proprietary and steeped in opinions about sample theory. Conceptually though the goal is to sample a more correct anisotropic sample of appropriate orientation to avoid the conflict between aliasing on one axis vs. blurring on the other when projected size differs.

In anisotropic implementations, the filtering may incorporate the same filtering algorithms used to filter the square maps of traditional mipmapping during the construction of the intermediate or final result.

Percentage Closer filtering

Depth-based shadow mapping can use an interesting Percentage Closer Filter (PCF) with depth-mapped textures that broadens one's perception of the kinds of texture filters that might be applied. In PCF a depth map of the scene is rendered from the light source. During the subsequent rendering of the scene this depth map is then projected back into the scene from the position of the light and a comparison is performed between the projective depth coordinate and the fetched texture sample depth. The projective coordinate will be the scene pixels depth from the light but the fetched depth from the depth map will represent the depth of the scene along that projected direction. In this way a determination of visibility to the light and therefore illumination by the light can be made for the rendered pixel. So this texturing operation is a boolean test of whether the pixel is lit, however multiple samples can be tested for a given pixel and the boolean results summed and averaged. In this way in combination with varying parameters like sampled texel location and even jittered depth map projection location a post-depth-comparison average or percentage of samples closer and therefore illuminated can be computed for a pixel. Critically, the summation of boolean results and generation of a percentage value must be performed after the depth comparison of projective depth and sample fetch, so this depth comparison becomes an integral part of the texture filter. This percentage can then be used to weight an illumination calculation and provide not just a boolean illumination or shadow value but a soft shadow penumbra result. [12] [13] A version of this is supported in modern hardware where a comparison is performed and a post boolean comparison bilinear filter by distance is applied [14]

See also

Related Research Articles

In digital signal processing, spatial anti-aliasing is a technique for minimizing the distortion artifacts (aliasing) when representing a high-resolution image at a lower resolution. Anti-aliasing is used in digital photography, computer graphics, digital audio, and many other applications.

<span class="mw-page-title-main">Texture mapping</span> Method of defining surface detail on a computer-generated graphic or 3D model

Texture mapping is a method for mapping a texture on a computer-generated graphic. Texture here can be high frequency detail, surface texture, or color.

<span class="mw-page-title-main">GeForce 3 series</span> Series of GPUs by Nvidia

The GeForce 3 series (NV20) is the third generation of Nvidia's GeForce line of graphics processing units (GPUs). Introduced in February 2001, it advanced the GeForce architecture by adding programmable pixel and vertex shaders, multisample anti-aliasing and improved the overall efficiency of the rendering process.

In computer graphics, mipmaps or pyramids are pre-calculated, optimized sequences of images, each of which is a progressively lower resolution representation of the previous. The height and width of each image, or level, in the mipmap is a factor of two smaller than the previous level. Mipmaps do not have to be square. They are intended to increase rendering speed and reduce aliasing artifacts. A high-resolution mipmap image is used for high-density samples, such as for objects close to the camera; lower-resolution images are used as the object appears farther away. This is a more efficient way of downfiltering (minifying) a texture than sampling all texels in the original texture that would contribute to a screen pixel; it is faster to take a constant number of samples from the appropriately downfiltered textures. Mipmaps are widely used in 3D computer games, flight simulators, other 3D imaging systems for texture filtering, and 2D and 3D GIS software. Their use is known as mipmapping. The letters MIP in the name are an acronym of the Latin phrase multum in parvo, meaning "much in little".

<span class="mw-page-title-main">RIVA 128</span> Graphics Chip by Nvidia

The RIVA 128, or "NV3", was a consumer graphics processing unit created in 1997 by Nvidia. It was the first to integrate 3D acceleration in addition to traditional 2D and video acceleration. Its name is an acronym for Real-time Interactive Video and Animation accelerator.

<span class="mw-page-title-main">Anisotropic filtering</span> Method of enhancing the image quality of textures on surfaces of computer graphics

In 3D computer graphics, anisotropic filtering is a method of enhancing the image quality of textures on surfaces of computer graphics that are at oblique viewing angles with respect to the camera where the projection of the texture appears to be non-orthogonal.

Trilinear filtering is an extension of the bilinear texture filtering method, which also performs linear interpolation between mipmaps.

Cone tracing and beam tracing are a derivative of the ray tracing algorithm that replaces rays, which have no thickness, with thick rays.

<span class="mw-page-title-main">Subsurface scattering</span>

Subsurface scattering (SSS), also known as subsurface light transport (SSLT), is a mechanism of light transport in which light that penetrates the surface of a translucent object is scattered by interacting with the material and exits the surface potentially at a different point. Light generally penetrates the surface and gets scattered a number of times at irregular angles inside the material before passing back out of the material at a different angle than it would have had if it had been reflected directly off the surface.

<span class="mw-page-title-main">Reflection mapping</span>

In computer graphics, reflection mapping or environment mapping is an efficient image-based lighting technique for approximating the appearance of a reflective surface by means of a precomputed texture. The texture is used to store the image of the distant environment surrounding the rendered object.

In computer graphics, per-pixel lighting refers to any technique for lighting an image or scene that calculates illumination for each pixel on a rendered image. This is in contrast to other popular methods of lighting such as vertex lighting, which calculates illumination at each vertex of a 3D model and then interpolates the resulting values over the model's faces to calculate the final per-pixel color values.

<span class="mw-page-title-main">Xenos (graphics chip)</span> GPU used in the Xbox 360

The Xenos is a custom graphics processing unit (GPU) designed by ATI, used in the Xbox 360 video game console developed and produced for Microsoft. Developed under the codename "C1", it is in many ways related to the R520 architecture and therefore very similar to an ATI Radeon X1800 XT series of PC graphics cards as far as features and performance are concerned. However, the Xenos introduced new design ideas that were later adopted in the TeraScale microarchitecture, such as the unified shader architecture. The package contains two separate dies, the GPU and an eDRAM, featuring a total of 337 million transistors.

<span class="mw-page-title-main">Image scaling</span> Changing the resolution of a digital image

In computer graphics and digital imaging, imagescaling refers to the resizing of a digital image. In video technology, the magnification of digital material is known as upscaling or resolution enhancement.

In computer graphics, a texture mapping unit (TMU) is a component in modern graphics processing units (GPUs). They are able to rotate, resize, and distort a bitmap image to be placed onto an arbitrary plane of a given 3D model as a texture, in a process called texture mapping. In modern graphics cards it is implemented as a discrete stage in a graphics pipeline, whereas when first introduced it was implemented as a separate processor, e.g. as seen on the Voodoo2 graphics card.

Glaze3D was a family of graphics cards announced by BitBoys Oy on August 2, 1999, that would have produced substantially better performance than other consumer products available at the time. The family, which would have come in the Glaze3D 1200, Glaze3D 2400 and Glaze3D 4800 models, was supposed to offer full support for DirectX 7, OpenGL 1.2, AGP 4×, 4× anisotropic filtering, full-screen anti-aliasing and a host of other technologies not commonly seen at the time. The 1.5 million gate GPU would have been fabricated by Infineon on a 0.2 μm eDRAM process, later to be reduced to 0.17 μm with a minimum of 9 MB of embedded DRAM and 128 to 512 MB of external SDRAM. The maximum supported video resolution was 2048×1536 pixels.

The term post-processing is used in the video and film industry for quality-improvement image processing methods used in video playback devices, such as stand-alone DVD-Video players; video playing software; and transcoding software. It is also commonly used in real-time 3D rendering to add additional effects.

PICA200 is a graphics processing unit (GPU) designed by Digital Media Professionals Inc. (DMP), a Japanese GPU design startup company, for use in embedded devices such as vehicle systems, mobile phones, cameras, and game consoles. The PICA200 is an IP Core which can be licensed to other companies to incorporate into their SOCs. It was most notably licensed for use in the Nintendo 3DS.

<span class="mw-page-title-main">Voodoo2</span> Series of Graphics Cards

The Voodoo2 is a set of three specialized 3D graphics chips on a single chipset setup, made by 3dfx. It was released in February 1998 as a replacement for the original Voodoo Graphics chipset. The card runs at a chipset clock rate of 90 MHz and uses 100 MHz EDO DRAM, and is available for the PCI interface. The Voodoo2 comes in two models, one with 8 MB RAM and one with 12 MB RAM. The 8 MB card has 2 MB of memory per texture mapping unit (TMU) vs. 4 MB on the 12 MB model. The 4 MB framebuffer on both cards support a maximum screen resolution of 800 × 600, while the increased texture memory on the 12 MB card allows more detailed textures. Some boards with 8 MB can be upgraded to 12 MB with an additional daughter board.

This is a glossary of terms relating to computer graphics.

References

  1. 1 2 "Chapter 9 - OpenGL Programming Guide". Glprogramming.com. 2009-02-13. Filtering. Retrieved 2018-01-14.
  2. ↑ Williams, Lance (1983). "Pyramidal parametrics" (PDF). ACM SIGGRAPH Computer Graphics. 17 (3): 1–11. doi:10.1145/964967.801126. ISSN   0097-8930. Archived from the original (PDF) on 2016-06-06. Retrieved 2017-02-14.
  3. ↑ "Game Engine Design: Texture Mapping" (PDF). uncc.edu. Texture Magnification. Archived from the original (PDF) on 2021-04-17. Retrieved 2017-02-14.
  4. ↑ "Game Engine Design: Texture Mapping" (PDF). uncc.edu. Texture Minification. Archived from the original (PDF) on 2021-04-17. Retrieved 2017-02-14.
  5. ↑ Hendrik Lensch (2007-11-29). "Computer Graphics: Texture Filtering & Sampling Theory" (PDF). Max Planck Society. MipMaps. Retrieved 2018-01-14.
  6. ↑ Markus Hadwiger (2015-03-09). "GPU and GPGPU Programming Lecture 12: GPU Texturing 2" (PDF). KAUST. Texture Reconstruction: Magnification. Archived from the original (PDF) on 2021-01-24. Retrieved 2017-02-14.
  7. ↑ Markus Hadwiger (2015-03-09). "GPU and GPGPU Programming Lecture 12: GPU Texturing 2" (PDF). KAUST. Texture Anti-Aliasing: MIP Mapping. Archived from the original (PDF) on 2021-01-24. Retrieved 2017-02-14.
  8. ↑ "Video game system with coprocessor providing high speed efficient 3D graphics and digital audio signal processing" . Retrieved 14 November 2022.
  9. ↑ Hendrik Lensch (2007-11-29). "Computer Graphics: Texture Filtering & Sampling Theory" (PDF). Max Planck Society. MipMapping II. Retrieved 2018-01-14.
  10. ↑ Alvy Ray Smith (1995-07-17). "A Pixel Is Not A Little Square! (And a Voxel is Not a Little Cube) - Technical Memo 6" (PDF). cs.princeton.edu. Retrieved 2018-01-14.
  11. ↑ Hendrik Lensch (2007-11-29). "Computer Graphics: Texture Filtering & Sampling Theory" (PDF). Max Planck Society. Anisotropic Filtering. Retrieved 2018-01-14.
  12. ↑ Reeves, William T.; Salesin, David H.; Cook, Robert L. (1987-08-01). "Rendering antialiased shadows with depth maps" (PDF). ACM SIGGRAPH Computer Graphics. 21 (4). Association for Computing Machinery (ACM): 283–291. doi:10.1145/37402.37435. ISSN   0097-8930. S2CID   18320563. Archived from the original (PDF) on 2018-01-15.
  13. ↑ Randima Fernando (2008-07-02). "Percentage-Closer Soft Shadows" (PDF). NVIDIA Corporation . Retrieved 2018-01-14.
  14. ↑ "WebGL WEBGL_depth_texture Khronos Ratified Extension Specification". Khronos.org. 2014-07-15. Retrieved 2018-01-14.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.