Video Acceleration API

Last updated
libva
Original author(s) Intel
Initial release2008
Stable release
2.20.0 [1] / 14 September 2023;25 days ago (14 September 2023)
Repository
Written in C
Operating system Linux, Android, BSD, Windows 10, Windows 11
Type API
License MIT license
Website www.freedesktop.org/wiki/Software/vaapi/   OOjs UI icon edit-ltr-progressive.svg

Video Acceleration API (VA-API) is an open source application programming interface that allows applications such as VLC media player or GStreamer to use hardware video acceleration capabilities, usually provided by the graphics processing unit (GPU). It is implemented by the free and open-source library libva, combined with a hardware-specific driver, usually provided together with the GPU driver.

Contents

VA-API video decode/encode interface is platform and window system independent but is primarily targeted at Direct Rendering Infrastructure (DRI) in X Window System on Unix-like operating systems (including Linux, FreeBSD, Solaris), and Android, however it can potentially also be used with direct framebuffer and graphics sub-systems for video output. Accelerated processing includes support for video decoding, video encoding, subpicture blending, and rendering. [2]

The VA-API specification was originally designed by Intel for its GMA (Graphics Media Accelerator) series of GPU hardware with the specific purpose of eventually replacing the XvMC standard as the default Unix multi-platform equivalent of Microsoft Windows DirectX Video Acceleration (DxVA) API, but today the API is no longer limited to Intel-specific hardware or GPUs. [3] Other hardware and manufacturers can freely use this open standard API for hardware accelerated video processing with their own hardware without paying a royalty fee. [4]

Overview

An example of vainfo output, showing supported video codecs for VA-API acceleration Vainfo screenshot.png
An example of vainfo output, showing supported video codecs for VA-API acceleration

The main motivation for VA-API is to enable hardware-accelerated video decode at various entry-points (VLD, IDCT, motion compensation, deblocking [5] ) for the prevailing coding standards today (MPEG-2, MPEG-4 ASP/H.263, MPEG-4 AVC/H.264, H.265/HEVC, and VC-1/WMV3). Extending XvMC was considered, but due to its original design for MPEG-2 MotionComp only, it made more sense to design an interface from scratch that can fully expose the video decode capabilities in today's GPUs. [6]

Supported hardware and drivers

As of 2022, VA-API is natively supported by: [7]

Supported video codecs

VA-API currently supports these video codecs in the official mainline version, but note that exactly which video codecs are supported depends on the hardware and the driver's capabilities.

Processes that can be accelerated with VA-API

Video decoding and post-processing processes that can be offloaded and accelerated if both the device drivers and GPU hardware supports them:

Software architecture

Video Acceleration API architecture Vaapi illustration.svg
Video Acceleration API architecture

The current interface is window system independent, so that it can potentially be used with graphics sub-systems other than the DRI (Direct Rendering Infrastructure) in X Window System, such as direct with framebuffer, and it can work with third-party DRM (Direct Rendering Manager) libraries. In a nutshell, it is a scheme to pass various types of data buffers from the application to the GPU for decoding or encoding a compressed bit-stream.

Software supporting VA-API

A log of Firefox showing VA-API being used for video decoding acceleration VA-API log of video playback on Firefox.png
A log of Firefox showing VA-API being used for video decoding acceleration

See also

Related Research Articles

<span class="mw-page-title-main">MPlayer</span> Free and open source media player

MPlayer is a free and open-source media player software application. It is available for Linux, OS X and Microsoft Windows. Versions for OS/2, Syllable, AmigaOS, MorphOS and AROS Research Operating System are also available. A port for DOS using DJGPP is also available. Versions for the Wii Homebrew Channel and Amazon Kindle have also been developed.

<span class="mw-page-title-main">FFmpeg</span> Multimedia framework

FFmpeg is a free and open-source software project consisting of a suite of libraries and programs for handling video, audio, and other multimedia files and streams. At its core is the command-line ffmpeg tool itself, designed for processing of video and audio files. It is widely used for format transcoding, basic editing, video scaling, video post-production effects and standards compliance.

SMPTE 421, informally known as VC-1, is a video coding format. Most of it was initially developed as Microsoft's proprietary video format Windows Media Video 9 in 2003. With some enhancements including the development of a new Advanced Profile, it was officially approved as a SMPTE standard on April 3, 2006. It was primarily marketed as a lower-complexity competitor to the H.264/MPEG-4 AVC standard. After its development, several companies other than Microsoft asserted that they held patents that applied to the technology, including Panasonic, LG Electronics and Samsung Electronics.

DirectX Video Acceleration (DXVA) is a Microsoft API specification for the Microsoft Windows and Xbox 360 platforms that allows video decoding to be hardware-accelerated. The pipeline allows certain CPU-intensive operations such as iDCT, motion compensation and deinterlacing to be offloaded to the GPU. DXVA 2.0 allows more operations, including video capturing and processing operations, to be hardware-accelerated as well.

X-Video Motion Compensation (XvMC), is an extension of the X video extension (Xv) for the X Window System. The XvMC API allows video programs to offload portions of the video decoding process to the GPU video-hardware. In theory this process should also reduce bus bandwidth requirements. Currently, the supported portions to be offloaded by XvMC onto the GPU are motion compensation and inverse discrete cosine transform (iDCT) for MPEG-2 video. XvMC also supports offloading decoding of mo comp, iDCT, and VLD for not only MPEG-2 but also MPEG-4 ASP video on VIA Unichrome hardware.

ATI Avivo is a set of hardware and low level software features present on the ATI Radeon R520 family of GPUs and all later ATI Radeon products. ATI Avivo was designed to offload video decoding, encoding, and post-processing from a computer's CPU to a compatible GPU. ATI Avivo compatible GPUs have lower CPU usage when a player and decoder software that support ATI Avivo is used. ATI Avivo has been long superseded by Unified Video Decoder (UVD) and Video Coding Engine (VCE).

CoreAVC was a proprietary codec for decoding the H.264/MPEG-4 AVC video format.

Unified Video Decoder is the name given to AMD's dedicated video decoding ASIC. There are multiple versions implementing a multitude of video codecs, such as H.264 and VC-1.

The following is a list of H.264/MPEG-4 AVC products and implementations.

PureVideo is Nvidia's hardware SIP core that performs video decoding. PureVideo is integrated into some of the Nvidia GPUs, and it supports hardware decoding of multiple video codec standards: MPEG-2, VC-1, H.264, HEVC, and AV1. PureVideo occupies a considerable amount of a GPU's die area and should not be confused with Nvidia NVENC. In addition to video decoding on chip, PureVideo offers features such as edge enhancement, noise reduction, deinterlacing, dynamic contrast enhancement and color enhancement.

X-Video Bitstream Acceleration (XvBA), designed by AMD Graphics for its Radeon GPU and APU, is an arbitrary extension of the X video extension (Xv) for the X Window System on Linux operating-systems. XvBA API allows video programs to offload portions of the video decoding process to the GPU video-hardware. Currently, the portions designed to be offloaded by XvBA onto the GPU are currently motion compensation (MC) and inverse discrete cosine transform (IDCT), and variable-length decoding (VLD) for MPEG-2, MPEG-4 ASP, MPEG-4 AVC (H.264), WMV3, and VC-1 encoded video.

Video Decode and Presentation API for Unix (VDPAU) is a royalty-free application programming interface (API) as well as its implementation as free and open-source library distributed under the MIT License. VDPAU is also supported by Nvidia.

Intel Quick Sync Video is Intel's brand for its dedicated video encoding and decoding hardware core. Quick Sync was introduced with the Sandy Bridge CPU microarchitecture on 9 January 2011 and has been found on the die of Intel CPUs ever since.

<span class="mw-page-title-main">Distributed Codec Engine</span>

Distributed Codec Engine (DCE) is an API and its implementation as software library ("libdce") by Texas Instruments. The library was released under the Revised BSD License and some additional terms.

Video Code Engine is AMD's video encoding application-specific integrated circuit implementing the video codec H.264/MPEG-4 AVC. Since 2012 it was integrated into all of their GPUs and APUs except Oland.

Crystal HD is Broadcom's hardware semiconductor intellectual property (SIP) core that performs video decoding.

Nvidia NVENC is a feature in Nvidia graphics cards that performs video encoding, offloading this compute-intensive task from the CPU to a dedicated part of the GPU. It was introduced with the Kepler-based GeForce 600 series in March 2012.

AOMedia Video 1 (AV1) is an open, royalty-free video coding format initially designed for video transmissions over the Internet. It was developed as a successor to VP9 by the Alliance for Open Media (AOMedia), a consortium founded in 2015 that includes semiconductor firms, video on demand providers, video content producers, software development companies and web browser vendors. The AV1 bitstream specification includes a reference video codec. In 2018, Facebook conducted testing that approximated real-world conditions, and the AV1 reference encoder achieved 34%, 46.2% and 50.3% higher data compression than libvpx-vp9, x264 High profile, and x264 Main profile respectively.

Nvidia NVDEC is a feature in its graphics cards that performs video decoding, offloading this compute-intensive task from the CPU.

Video Core Next is AMD's brand for its dedicated video encoding and decoding hardware core. It is a family of hardware accelerator designs for encoding and decoding video, and is built into AMD's GPUs and APUs since AMD Raven Ridge, released January 2018.

References

  1. "update NEWS for 2.20.0".
  2. "VA-API Video Acceleration On Intel Medfield - Phoronix". Phoronix.com. Retrieved 23 February 2019.
  3. "Video4Linux2: Path to a Standardized Video Codec API" (PDF). Events.linuxfoundation.org. Retrieved 23 February 2019.
  4. Nathan Willis (2009-07-01). "VA API slowly, but surely, making progress". Lwn.net.
  5. "Mplayer, FFmpeg Gain VA-API Support - Phoronix". Phoronix.com. Retrieved 23 February 2019.
  6. "vaapi". Freedesktop.org. Retrieved 23 February 2019.
  7. 1 2 "Hardware video acceleration". wiki.ArchLinux.org.
  8. Vilerino, Sil (2023-02-15). "Video acceleration API (VA-API) now available on Windows!". DirectX Developer Blog. Retrieved 2023-03-08.
  9. 1 2 "Hardware/vaapi". wiki.libav.org. Archived from the original on 2017-02-01. Retrieved 2017-01-20.
  10. "Emby Server 3.0.6400 Released". Emby.media. 29 August 2016. Retrieved 23 February 2019.
  11. "Mailing list entry that describes uses of VA-API". Lists.moblin.org. Retrieved 23 February 2019.
  12. "RealPlayer for MID & Intel/Linux FAQ". HelixCommunity.org. Archived from the original on 2012-03-24. Retrieved 2011-05-12.
  13. "Archived copy". Archived from the original on 2016-09-16. Retrieved 2016-08-31.{{cite web}}: CS1 maint: archived copy as title (link)
  14. "Fluendo's New Codecs Support VDPAU, VA-API - Phoronix". Phoronix.com. Retrieved 23 February 2019.
  15. "H.264 VA-API GPU Video Acceleration For Flash - Phoronix". Phoronix.com. Retrieved 23 February 2019.
  16. "Hardware-accelerated video decoding, encoding and processing on Intel graphics through VA-API". Cgit.freedesktop.org. Retrieved 23 February 2019.
  17. 1 2 "XBMC Gets Working Intel VA-API Support - Phoronix". Phoronix.com. Retrieved 23 February 2019.
  18. "Mplayer in vaapi - Gitorious". Archived from the original on 2013-08-03. Retrieved 2014-02-10.
  19. "Release Notes - 0.25 - MythTV Official Wiki". Mythtv.org. Retrieved 23 February 2019.
  20. "VLC 1.1.0 release - VideoLAN". Videolan.org. Retrieved 23 February 2019.
  21. "[Client] 2.2.6 Released – Significant CPU usage reductions included". Bluecherrydvr.com. 2 March 2017. Retrieved 23 February 2019.
  22. "the xine project - News Feed". Xine-project.org. Retrieved 23 February 2019.
  23. "OBS Studio Now Supports VA-API For Video Encoding - Phoronix". www.phoronix.com. Retrieved 2020-08-06.
  24. "Firefox on Fedora finally gets VA-API on Wayland". mastransky.wordpress.com. 3 June 2020. Retrieved 2020-08-19.
  25. "Firefox 80 Available With VA-API On X11, WebGL Parallel Shader Compile Support". www.phoronix.com. Retrieved 2020-08-25.