Deep Learning Anti-Aliasing

Last updated

Deep Learning Anti-Aliasing (DLAA) is a form of spatial anti-aliasing created by Nvidia. [1] DLAA depends on and requires Tensor Cores available in Nvidia RTX cards. [1]

Contents

DLAA is similar to Deep Learning Super Sampling (DLSS) in its anti-aliasing method, [2] with one important differentiation being that the goal of DLSS is to increase performance at the cost of image quality, [3] whereas the main priority of DLAA is improving image quality at the cost of performance (irrelevant of resolution upscaling or downscaling). [4] DLAA is similar to temporal anti-aliasing (TAA) in that they are both spatial anti-aliasing solutions relying on past frame data. [3] [5] Compared to TAA, DLAA is substantially better when it comes to shimmering, flickering, and handling small meshes like wires. [6]

Technical overview

DLAA collects game rendering data including raw low-resolution input, motion vectors, depth buffers, and exposure information. This information feeds into a convolutional neural network that processes the image to reduce aliasing while preserving fine detail. [3]

The neural network architecture employs an auto-encoder design trained on high-quality reference images. The training dataset includes diverse scenarios focusing on challenging cases like sub-pixel details, high-contrast edges, and transparent surfaces. The network then processes frames in real-time. [3]

Unlike traditional anti-aliasing solutions that rely on manually written heuristics, such as TAA, DLAA uses its neural network to preserve fine details while eliminating unwanted visual artifacts. [3]

Differences between TAA and DLAA

TAA is used in many modern video games and game engines; [7] however, all previous implementations have used some form of manually written heuristics to prevent temporal artifacts such as ghosting and flickering. One example of this is neighborhood clamping which forcefully prevents samples collected in previous frames from deviating too much compared to nearby pixels in newer frames. This helps to identify and fix many temporal artifacts, but deliberately removing fine details in this way is analogous to applying a blur filter, and thus the final image can appear blurry when using this method. [8]

DLAA uses an auto-encoder convolutional neural network [9] trained to identify and fix temporal artifacts, instead of manually programmed heuristics as mentioned above. Because of this, DLAA can generally resolve detail better than other TAA and TAAU implementations, while also removing most temporal artifacts.

Differences between DLSS and DLAA

While DLSS handles upscaling with a focus on performance, DLAA handles anti-aliasing with a focus on visual quality. DLAA runs at the given screen resolution with no upscaling or downscaling functionality provided by DLAA. [10]

DLSS and DLAA share the same AI-driven anti-aliasing method. [11] As such, DLAA functions like DLSS without the upscaling part. Both are made by Nvidia and require Tensor Cores. However, DLSS and DLAA cannot be enabled at the same time, only one can be selected depending on whether performance or image quality is prioritized.

Reception

TechPowerUp found that "[c]ompared to TAA and DLSS, DLAA is clearly producing the best image quality, especially at lower resolutions", arguing that, while "DLSS was already doing a better job than TAA at reconstructing small objects", "DLAA does an even better job". [12]

Related Research Articles

<span class="mw-page-title-main">Jaggies</span> Jagged effect produced by naive upscaling of raster shapes with few pixels defining a curve

Jaggies are artifacts in raster images, most frequently from aliasing, which in turn is often caused by non-linear mixing effects producing high-frequency components, or missing or poor anti-aliasing filtering prior to sampling.

In digital signal processing, spatial anti-aliasing is a technique for minimizing the distortion artifacts (aliasing) when representing a high-resolution image at a lower resolution. Anti-aliasing is used in digital photography, computer graphics, digital audio, and many other applications.

<span class="mw-page-title-main">GeForce 3 series</span> Series of GPUs by Nvidia

The GeForce 3 series (NV20) is the third generation of Nvidia's GeForce line of graphics processing units (GPUs). Introduced in February 2001, it advanced the GeForce architecture by adding programmable pixel and vertex shaders, multisample anti-aliasing and improved the overall efficiency of the rendering process.

In computer graphics, mipmaps or pyramids are pre-calculated, optimized sequences of images, each of which is a progressively lower resolution representation of the previous. The height and width of each image, or level, in the mipmap is a factor of two smaller than the previous level. Mipmaps do not have to be square. They are intended to increase rendering speed and reduce aliasing artifacts. A high-resolution mipmap image is used for high-density samples, such as for objects close to the camera; lower-resolution images are used as the object appears farther away. This is a more efficient way of downscaling a texture than sampling all texels in the original texture that would contribute to a screen pixel; it is faster to take a constant number of samples from the appropriately downfiltered textures. Mipmaps are widely used in 3D computer games, flight simulators, other 3D imaging systems for texture filtering, and 2D and 3D GIS software. Their use is known as mipmapping. The letters MIP in the name are an acronym of the Latin phrase multum in parvo, meaning "much in little".

<span class="mw-page-title-main">Anisotropic filtering</span> Method of enhancing the image quality of textures on surfaces of computer graphics

In 3D computer graphics, anisotropic filtering(AF) is a technique that improves the appearance of textures, especially on surfaces viewed at sharp angles. It helps make textures look sharper and more detailed by reducing blur and aliasing that can occur when surfaces are angled away from the viewer. Anisotropic filtering works by applying different amounts of filtering in different directions, unlike simpler methods like bilinear and trilinear filtering which filter equally in all directions.

<span class="mw-page-title-main">Scalable Link Interface</span> Brand name; multi-GPU technology by Nvidia

Scalable Link Interface (SLI) is the brand name for a now discontinued multi-GPU technology developed by Nvidia for linking two or more video cards together to produce a single output. SLI is a parallel processing algorithm for computer graphics, meant to increase the available processing power.

1080i is a term used in high-definition television (HDTV) and video display technology. It means a video mode with 1080 lines of vertical resolution. The "i" stands for interlaced scanning method. This format was once a standard in HDTV. It was particularly used for broadcast television. This is because it can deliver high-resolution images without needing excessive bandwidth. This format is used in the SMPTE 292M standard.

<span class="mw-page-title-main">Pixel-art scaling algorithms</span> Upscaling filters for pixel art graphics

Pixel art scaling algorithms are graphical filters that attempt to enhance the appearance of hand-drawn 2D pixel art graphics. These algorithms are a form of automatic image enhancement. Pixel art scaling algorithms employ methods significantly different than the common methods of image rescaling, which have the goal of preserving the appearance of images.

<span class="mw-page-title-main">Video scaler</span> System which converts video signals from one display resolution to another

A video scaler is a system that converts video signals from one display resolution to another; typically, scalers are used to convert a signal from a lower resolution to a higher resolution, a process known as "upconversion" or "upscaling".

<span class="mw-page-title-main">Image scaling</span> Changing the resolution of a digital image

In computer graphics and digital imaging, imagescaling refers to the resizing of a digital image. In video technology, the magnification of digital material is known as upscaling or resolution enhancement.

Multisample anti-aliasing (MSAA) is a type of spatial anti-aliasing, a technique used in computer graphics to remove jaggies.

Temporal anti-aliasing (TAA) is a spatial anti-aliasing technique for computer-generated video that combines information from past frames and the current frame to remove jaggies in the current frame. In TAA, each pixel is sampled once per frame but in each frame the sample is at a different location within the frame. Pixels sampled in past frames are blended with pixels sampled in the current frame to produce an anti-aliased image. Although this method makes TAA achieve a result comparable to supersampling, the technique inevitably causes ghosting and blurriness to the image.

Anti-aliasing may refer to any of a number of techniques to combat the problems of aliasing in a sampled signal such as a digital image or digital audio recording.

Fast approximate anti-aliasing (FXAA) is a screen-space anti-aliasing algorithm created by Timothy Lottes at Nvidia.

<span class="mw-page-title-main">GPUOpen</span> Middleware software suite

GPUOpen is a middleware software suite originally developed by AMD's Radeon Technologies Group that offers advanced visual effects for computer games. It was released in 2016. GPUOpen serves as an alternative to, and a direct competitor of Nvidia GameWorks. GPUOpen is similar to GameWorks in that it encompasses several different graphics technologies as its main components that were previously independent and separate from one another. However, GPUOpen is partially open source software, unlike GameWorks which is proprietary and closed.

<span class="mw-page-title-main">StyleGAN</span> Novel generative adversarial network

The Style Generative Adversarial Network, or StyleGAN for short, is an extension to the GAN architecture introduced by Nvidia researchers in December 2018, and made source available in February 2019.

Deep Learning Super Sampling (DLSS) is a suite of real-time deep learning image enhancement and upscaling technologies developed by Nvidia that are available in a number of video games. The goal of these technologies is to allow the majority of the graphics pipeline to run at a lower resolution for increased performance, and then infer a higher resolution image from this that approximates the same level of detail as if the image had been rendered at this higher resolution. This allows for higher graphical settings and/or frame rates for a given output resolution, depending on user preference.

<span class="mw-page-title-main">Video super-resolution</span> Generating high-resolution video frames from given low-resolution ones

Video super-resolution (VSR) is the process of generating high-resolution video frames from the given low-resolution video frames. Unlike single-image super-resolution (SISR), the main goal is not only to restore more fine details while saving coarse ones, but also to preserve motion consistency.

<span class="mw-page-title-main">Intel Arc</span> Graphics processing unit brand

Intel Arc is a brand of graphics processing units designed by Intel. These are discrete GPUs mostly marketed for the high-margin gaming PC market. The brand also covers Intel's consumer graphics software and services.

<span class="mw-page-title-main">GeForce 50 series</span> Series of GPUs by Nvidia

The GeForce 50 series is a series of consumer graphics processing units (GPUs) developed by Nvidia as part of its GeForce line of graphics cards, succeeding the GeForce 40 series. Announced at CES 2025, it debuted with the release of the RTX 5080 and RTX 5090 on January 30, 2025. It is based on Nvidia's Blackwell architecture featuring Nvidia RTX's fourth-generation RT cores for hardware-accelerated real-time ray tracing, and fifth-generation deep-learning-focused Tensor Cores. The GPUs are manufactured by TSMC on an improved custom 4NP process node.

References

  1. 1 2 Kostovic, Aleksandar (2021-09-20). "Nvidia Readies Deep Learning Anti-Aliasing Debut with The Elder Scrolls Online Update". Tom's Hardware. Retrieved 2022-02-20.
  2. ↑ Hruska, Joel (2021-09-21). "Nvidia's DLAA Could Be a Huge Step Forward for Anti-Aliasing". ExtremeTech.
  3. 1 2 3 4 5 Liu, Edward (2020-03-23). "DLSS 2.0 – Image Reconstruction for Real-Time Rendering With Deep Learning" (PDF). Behind the Pixels.
  4. ↑ "Nvidia's DLAA makes PC games look better with little performance hit". PCWorld. Retrieved 2024-04-20.
  5. ↑ Yang, Lei; Liu, Shiqiu; Salvi, Marco. "A Survey of Temporal Antialiasing Techniques" (PDF). Computer Graphics Forum. 39 (2): 607–621. doi:10.1111/cgf.14018 – via Behind the Pixels.
  6. ↑ De Meo, Francesco (2021-09-23). "The Elder Scrolls Online DLAA vs DLSS vs TAA Comparison Video Highlights DLAA Superior Image Quality". Wccftech. Retrieved 2022-02-20.
  7. ↑ Karis, Brian. "High Quality Temporal Supersamplin" (PDF).
  8. ↑ "GTC 2020: DLSS 2.0 - Image Reconstruction for Real-time Rendering with Deep Learning". NVIDIA Developer. 2020-06-09. Retrieved 2022-06-26.
  9. ↑ "NVIDIA DLSS 2.0: A Big Leap In AI Rendering". www.nvidia.com. Retrieved 2022-06-26.
  10. ↑ maxus24; on; Studios, in Game Testing Manufacturer: Zenimax Online (2021-09-22). "NVIDIA DLAA Anti-Aliasing Review - DLSS at Native Resolution". TechPowerUp. Retrieved 2024-03-22.{{cite web}}: CS1 maint: numeric names: authors list (link)
  11. ↑ Archer, James (2022-06-27). "Nvidia DLAA: How it works, supported games and performance vs DLSS". Rock, Paper, Shotgun. Retrieved 2022-07-09.
  12. ↑ "NVIDIA DLAA Anti-Aliasing Review - DLSS at Native Resolution". TechPowerUp. Archived from the original on 2025-01-30. Retrieved 2025-01-31.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.