Deep learning anti-aliasing

Last updated

Deep learning anti-aliasing (DLAA) is a form of spatial anti-aliasing created by Nvidia. [1] DLAA depends on and requires Tensor Cores available in Nvidia RTX cards. [1]

Contents

DLAA is similar to deep learning super sampling (DLSS) in its anti-aliasing method, [2] with one important differentiation being that the goal of DLSS is to increase performance at the cost of image quality, [3] whereas the main priority of DLAA is improving image quality at the cost of performance (irrelevant of resolution upscaling or downscaling). [4] DLAA is similar to temporal anti-aliasing (TAA) in that they are both spatial anti-aliasing solutions relying on past frame data. [3] [5] Compared to TAA, DLAA is substantially better when it comes to shimmering, flickering, and handling small meshes like wires. [6]

DLAA collects game rendering data such as raw low-resolution input, motion vectors, depth buffers, and exposure information. This information is then used by DLAA to improve its anti-aliasing, with the aim of reducing temporal instability.

Differences between TAA and DLAA

TAA is used in many modern video games and game engines; [7] however, all previous implementations have used some form of manually written heuristics to prevent temporal artifacts such as ghosting and flickering. One example of this is neighborhood clamping which forcefully prevents samples collected in previous frames from deviating too much compared to nearby pixels in newer frames. This helps to identify and fix many temporal artifacts, but deliberately removing fine details in this way is analogous to applying a blur filter, and thus the final image can appear blurry when using this method. [8]

DLAA uses an auto-encoder convolutional neural network [9] trained to identify and fix temporal artifacts, instead of manually programmed heuristics as mentioned above. Because of this, DLAA can generally resolve detail better than other TAA and TAAU implementations, while also removing most temporal artifacts.

Differences between DLSS and DLAA

While DLSS handles upscaling with a focus on performance, DLAA handles anti-aliasing with a focus on visual quality. DLAA runs at the given screen resolution with no upscaling or downscaling functionality. [10]

DLSS and DLAA share the same AI-driven anti-aliasing method. [11] As such, DLAA functions like DLSS without the upscaling part. Both are made by Nvidia and require Tensor Cores.

See also

Related Research Articles

<span class="mw-page-title-main">ClearType</span> Font-rendering technology by Microsoft

ClearType is Microsoft's implementation of subpixel rendering technology in rendering text in a font system. ClearType attempts to improve the appearance of text on certain types of computer display screens by sacrificing color fidelity for additional intensity variation. This trade-off is asserted to work well on LCD flat panel monitors.

In digital signal processing, spatial anti-aliasing is a technique for minimizing the distortion artifacts (aliasing) when representing a high-resolution image at a lower resolution. Anti-aliasing is used in digital photography, computer graphics, digital audio, and many other applications.

In computer graphics, mipmaps or pyramids are pre-calculated, optimized sequences of images, each of which is a progressively lower resolution representation of the previous. The height and width of each image, or level, in the mipmap is a factor of two smaller than the previous level. Mipmaps do not have to be square. They are intended to increase rendering speed and reduce aliasing artifacts. A high-resolution mipmap image is used for high-density samples, such as for objects close to the camera; lower-resolution images are used as the object appears farther away. This is a more efficient way of downscaling a texture than sampling all texels in the original texture that would contribute to a screen pixel; it is faster to take a constant number of samples from the appropriately downfiltered textures. Mipmaps are widely used in 3D computer games, flight simulators, other 3D imaging systems for texture filtering, and 2D and 3D GIS software. Their use is known as mipmapping. The letters MIP in the name are an acronym of the Latin phrase multum in parvo, meaning "much in little".

<span class="mw-page-title-main">RIVA 128</span> Graphics processing unit developed by Nvidia

The RIVA 128, or "NV3", was a consumer graphics processing unit created in 1997 by Nvidia. It was the first to integrate 3D acceleration in addition to traditional 2D and video acceleration. Its name is an acronym for Real-time Interactive Video and Animation accelerator.

<span class="mw-page-title-main">Anisotropic filtering</span> Method of enhancing the image quality of textures on surfaces of computer graphics

In 3D computer graphics, anisotropic filtering is a method of enhancing the image quality of textures. It only applies on surfaces at oblique viewing angles to the camera and where the projection of the texture appears to be non-orthogonal. As per its etymology, anisotropic filtering does not filter the same in every direction.

In computer graphics, texture filtering or texture smoothing is the method used to determine the texture color for a texture mapped pixel, using the colors of nearby texels.

<span class="mw-page-title-main">Pixel-art scaling algorithms</span> Upscaling filters for pixel art graphics

Pixel art scaling algorithms are graphical filters that attempt to enhance the appearance of hand-drawn 2D pixel art graphics. These algorithms are a form of automatic image enhancement. Pixel art scaling algorithms employ methods significantly different than the common methods of image rescaling, which have the goal of preserving the appearance of images.

<span class="mw-page-title-main">Video scaler</span> System which converts video signals from one display resolution to another

A video scaler is a system that converts video signals from one display resolution to another; typically, scalers are used to convert a signal from a lower resolution to a higher resolution, a process known as "upconversion" or "upscaling".

<span class="mw-page-title-main">Image scaling</span> Changing the resolution of a digital image

In computer graphics and digital imaging, imagescaling refers to the resizing of a digital image. In video technology, the magnification of digital material is known as upscaling or resolution enhancement.

<span class="mw-page-title-main">Supersampling</span> Spatial anti-aliasing method

Supersampling or supersampling anti-aliasing (SSAA) is a spatial anti-aliasing method, i.e. a method used to remove aliasing from images rendered in computer games or other computer programs that generate imagery. Aliasing occurs because unlike real-world objects, which have continuous smooth curves and lines, a computer screen shows the viewer a large number of small squares. These pixels all have the same size, and each one has a single color. A line can only be shown as a collection of pixels, and therefore appears jagged unless it is perfectly horizontal or vertical. The aim of supersampling is to reduce this effect. Color samples are taken at several instances inside the pixel, and an average color value is calculated. This is achieved by rendering the image at a much higher resolution than the one being displayed, then shrinking it to the desired size, using the extra pixels for calculation. The result is a downsampled image with smoother transitions from one line of pixels to another along the edges of objects. The number of samples determines the quality of the output.

Multisample anti-aliasing (MSAA) is a type of spatial anti-aliasing, a technique used in computer graphics to remove jaggies.

Temporal anti-aliasing (TAA) is a spatial anti-aliasing technique for computer-generated video that combines information from past frames and the current frame to remove jaggies in the current frame. In TAA, each pixel is sampled once per frame but in each frame the sample is at a different location within the pixel. Pixels sampled in past frames are blended with pixels sampled in the current frame to produce an anti-aliased image. Although this method makes TAA achieve a result comparable to supersampling, the technique inevitably causes ghosting and blurriness to the image.

<span class="mw-page-title-main">Deferred shading</span> Screen-space shading technique

In the field of 3D computer graphics, deferred shading is a screen-space shading technique that is performed on a second rendering pass, after the vertex and pixel shaders are rendered. It was first suggested by Michael Deering in 1988.

Anti-aliasing may refer to any of a number of techniques to combat the problems of aliasing in a sampled signal such as a digital image or digital audio recording

Fast approximate anti-aliasing (FXAA) is a screen-space anti-aliasing algorithm created by Timothy Lottes at Nvidia.

<span class="mw-page-title-main">GPUOpen</span> Middleware software suite

GPUOpen is a middleware software suite originally developed by AMD's Radeon Technologies Group that offers advanced visual effects for computer games. It was released in 2016. GPUOpen serves as an alternative to, and a direct competitor of Nvidia GameWorks. GPUOpen is similar to GameWorks in that it encompasses several different graphics technologies as its main components that were previously independent and separate from one another. However, GPUOpen is partially open source software, unlike GameWorks which is proprietary and closed.

Foveated rendering is a rendering technique which uses an eye tracker integrated with a virtual reality headset to reduce the rendering workload by greatly reducing the image quality in the peripheral vision.

<span class="mw-page-title-main">GeForce 20 series</span> Series of GPUs by Nvidia

The GeForce 20 series is a family of graphics processing units developed by Nvidia. Serving as the successor to the GeForce 10 series, the line started shipping on September 20, 2018, and after several editions, on July 2, 2019, the GeForce RTX Super line of cards was announced.

Deep learning super sampling (DLSS) is a family of real-time deep learning image enhancement and upscaling technologies developed by Nvidia that are available in a number of video games. The goal of these technologies is to allow the majority of the graphics pipeline to run at a lower resolution for increased performance, and then infer a higher resolution image from this that approximates the same level of detail as if the image had been rendered at this higher resolution. This allows for higher graphical settings and/or frame rates for a given output resolution, depending on user preference.

<span class="mw-page-title-main">Intel Arc</span> Graphics processing unit brand

Intel Arc is a brand of graphics processing units designed by Intel. These are discrete GPUs mostly marketed for the high-margin gaming PC market. The brand also covers Intel's consumer graphics software and services.

References

  1. 1 2 Kostovic, Aleksandar (2021-09-20). "Nvidia Readies Deep Learning Anti-Aliasing Debut with The Elder Scrolls Online Update". Tom's Hardware. Retrieved 2022-02-20.
  2. ↑ Hruska, Joel (2021-09-21). "Nvidia's DLAA Could Be a Huge Step Forward for Anti-Aliasing". ExtremeTech.
  3. 1 2 Liu, Edward (2020-03-23). "DLSS 2.0 – Image Reconstruction for Real-Time Rendering With Deep Learning" (PDF). Behind the Pixels.
  4. ↑ "Nvidia's DLAA makes PC games look better with little performance hit". PCWorld. Retrieved 2024-04-20.
  5. ↑ Yang, Lei; Liu, Shiqiu; Salvi, Marco. "A Survey of Temporal Antialiasing Techniques" (PDF). Computer Graphics Forum. 39 (2): 607–621. doi:10.1111/cgf.14018 – via Behind the Pixels.
  6. ↑ De Meo, Francesco (2021-09-23). "The Elder Scrolls Online DLAA vs DLSS vs TAA Comparison Video Highlights DLAA Superior Image Quality". Wccftech. Retrieved 2022-02-20.
  7. ↑ Karis, Brian. "High Quality Temporal Supersamplin" (PDF).
  8. ↑ "GTC 2020: DLSS 2.0 - Image Reconstruction for Real-time Rendering with Deep Learning". NVIDIA Developer. 2020-06-09. Retrieved 2022-06-26.
  9. ↑ "NVIDIA DLSS 2.0: A Big Leap In AI Rendering". www.nvidia.com. Retrieved 2022-06-26.
  10. ↑ maxus24; on; Studios, in Game Testing Manufacturer: Zenimax Online (2021-09-22). "NVIDIA DLAA Anti-Aliasing Review - DLSS at Native Resolution". TechPowerUp. Retrieved 2024-03-22.{{cite web}}: CS1 maint: numeric names: authors list (link)
  11. ↑ Archer, James (2022-06-27). "Nvidia DLAA: How it works, supported games and performance vs DLSS". Rock, Paper, Shotgun. Retrieved 2022-07-09.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.