X-Video Motion Compensation

Last updated

X-Video Motion Compensation (XvMC), is an extension of the X video extension (Xv) for the X Window System. The XvMC API allows video programs to offload portions of the video decoding process to the GPU video-hardware. In theory this process should also reduce bus bandwidth requirements. Currently, the supported portions to be offloaded by XvMC onto the GPU are motion compensation (mo comp) and inverse discrete cosine transform (iDCT) for MPEG-2 video. XvMC also supports offloading decoding of mo comp, iDCT, and VLD ("Variable-Length Decoding", more commonly known as "slice level acceleration") for not only MPEG-2 but also MPEG-4 ASP video on VIA Unichrome (S3 Graphics Chrome Series) hardware.

Contents

XvMC was the first UNIX equivalent of the Microsoft Windows DirectX Video Acceleration (DxVA) API. Popular software applications known to take advantage of XvMC include MPlayer, MythTV, and xine.

Device drivers

Each hardware video GPU capable of XvMC video acceleration requires a X11 software device driver to enable these features.

Hardware manufacturers

Nvidia

There are currently three X11 Nvidia drivers available: a 2D-only open source but obfuscated driver maintained by Nvidia called nv, a proprietary binary driver by Nvidia, and an open source driver based on reverse engineering of the binary driver developed by the Linux community called Nouveau. Nouveau is not pursuing XvMC support, [1] the 2D nv driver does not support XvMC, and the official proprietary binary driver by Nvidia only supports MPEG-2 offloading (mo comp and iDCT) on hardware up to and including the GeForce 7000 series.

VIA

VIA provides open source device drivers for some of its VIA Unichrome (S3 Graphics Chrome Series) hardware, supporting offloading of MPEG-2 and MPEG-4 ASP video. Thanks to VLD level of decoding VIA offloads much more decoding tasks from CPU than GPUs supporting iDCT or mo comp levels only. Keep in mind that not all devices are supported and there are some other caveats. [2]

Intel

Intel provides official open source device drivers which supports MPEG-2 offloading (mo comp and iDCT) on Intel's 8xx/9xx range of integrated graphics chips. [3]

ATI/AMD

Although ATI was the first manufacturer to provide MPEG-2 acceleration in their graphic boards with the Rage 128 GPU, it has never provided documentation on how to use it. So no XvMC is available, and will probably never be. XvMC is supported on Radeon-4000 cards (which have UVD) by Catalyst driver from 8.10 and higher at an experimental level (meaning that it doesn't work "out of the box".) [4] (See also X-Video Bitstream Acceleration). Motion compensation support in other ATI/AMD hardware is planned for the future. (Now all hardware has support for MPEG-2 acceleration on iDCT and mo comp levels).

Matrox

There are no device drivers which support XvMC on Matrox hardware, (although Matrox Parhelia hardware has support for MPEG-2 acceleration on mo comp level).

S3

Binary device driver by S3 only supports MPEG-2 offloading in initial 2.0.16 driver on Chrome 20 GPUs. [ citation needed ]

XvMC API technical overview

This is from X-Vxvideoideo Motion Compensation - API specification v. 1.0 [5]

XvPort

XvMC extends the X video extension (Xv) and makes use of the familiar concept of the XvPort. Ports have attributes that can be set and queried through Xv. In XvMC ports can also have hardware motion compensation contexts created for use with them. Ports which support XvImages (i.e. they have an "XV_IMAGE" port encoding as described in the Xv version 2.2 API addendum) can be queried for the list of XvMCSurface types they support. If they support any XvMCSurface types an XvMCContext can be created for that port.

XvMCContext

XvMCContext describes the state of the motion compensation pipeline. An individual XvMCContext can be created for use with a single port, surface type, motion compensation type, width and height combination. For example, a context might be created for a particular port that does MPEG-2 motion compensation on 720 x 480 4:2:0 surfaces. Once the context is created, referencing it implies the port, surface type, size and the motion compensation type. Contexts may be "direct" or "indirect". For indirect contexts the X display server renders all video using the data passed to it by the client. For direct contexts the client libraries render the video with little or no interaction with the X display server.

XvMCSurfaces

XvMCSurfaces are buffers into which the motion compensation hardware can render. The data in the buffers themselves are not client accessible and may be stored in a hardware-specific format. Any number of buffers can be created for use with a particular context (resources permitting).

Video Pipeline

XvMC provides video acceleration starting at one of two places in the video pipeline. Acceleration starting at the first point, which we shall call the "Motion Compensation" level, begins after the inverse quantization and IDCT at the place where motion compensation is to be applied. The second point, which we shall call the "IDCT" level, begins before the IDCT just after the inverse quantization.

Rendering

Rendering is done by presenting the library with a target XvMCSurface and up to two reference XvMCSurfaces for the motion compensation, a buffer of 8x8 blocks and a command buffer which describes how to use the 8x8 blocks along with motion compensation vectors to construct the data in the target XvMCSurface. When the pipeline starts at the iDCT level, Xv will perform the IDCT on the blocks before performing the motion compensation. A function is provided to copy/overlay a portion of the XvMCSurface to a drawable with arbitrary scaling.

XvMCSubpictures

XvMCSubpictures are separate surfaces that may be blended with the target surface. Any number of XvMCSubpictures may be created for use with a context (resources permitting). Both "backend" and "frontend" subpicture behavior are supported.

Software support

XvMC acceleration is supported in:

XvMC have been removed in Mesa 22.3.

Processes that could be accelerated

Even though XvMC currently only supports hardware acceleration of motion compensation (mo comp) and inverse discrete cosine transform (iDCT), (and Variable-Length Decoding for VIA Unichrome GPU), additional video decoding processes could be passed on to modern GPUs which could be accelerated via GPU fragment programs. XvMC could be extended in the future to support the same processes as the newer competing hardware video acceleration APIs like VDPAU, XvBA, and VAAPI:

Current limitations

Besides not matching all of the features and function of DxVA (which is the Microsoft equivalent API for Windows), and lacking support for other video formats than MPEG-2 in Linux device drivers from Intel and Nvidia, the XvMC API specification version 1.0 [7] currently also has these other limitations:

VA API is a more modern video acceleration API which support the video acceleration features of modern GPUs. [8]

See also

Related Research Articles

<span class="mw-page-title-main">GeForce 256</span> GPU by Nvidia

The GeForce 256 is the original release in Nvidia's "GeForce" product line. Announced on August 31, 1999 and released on October 11, 1999, the GeForce 256 improves on its predecessor by increasing the number of fixed pixel pipelines, offloading host geometry calculations to a hardware transform and lighting (T&L) engine, and adding hardware motion compensation for MPEG-2 video. It offered a notably large leap in 3D PC gaming performance and was the first fully Direct3D 7-compliant 3D accelerator.

<span class="mw-page-title-main">Graphics processing unit</span> Specialized electronic circuit; graphics accelerator

A graphics processing unit (GPU) is a specialized electronic circuit initially designed for digital image processing and to accelerate computer graphics, being present either as a discrete video card or embedded on motherboards, mobile phones, personal computers, workstations, and game consoles. After their initial design, GPUs were found to be useful for non-graphic calculations involving embarrassingly parallel problems due to their parallel structure. Other non-graphical uses include the training of neural networks and cryptocurrency mining.

OpenMAX, often shortened as "OMX", is a non-proprietary and royalty-free cross-platform set of C-language programming interfaces. It provides abstractions for routines that are especially useful for processing of audio, video, and still images. It is intended for low power and embedded system devices that need to efficiently process large amounts of multimedia data in predictable ways, such as video codecs, graphics libraries, and other functions for video, image, audio, voice and speech.

DirectX Video Acceleration (DXVA) is a Microsoft API specification for the Microsoft Windows and Xbox 360 platforms that allows video decoding to be hardware-accelerated. The pipeline allows certain CPU-intensive operations such as iDCT, motion compensation and deinterlacing to be offloaded to the GPU. DXVA 2.0 allows more operations, including video capturing and processing operations, to be hardware-accelerated as well.

<span class="mw-page-title-main">ATI Rage</span> Series of video cards

The ATI Rage is a series of graphics chipsets developed by ATI Technologies offering graphical user interface (GUI) 2D acceleration, video acceleration, and 3D acceleration developed by ATI Technologies. It is the successor to the ATI Mach series of 2D accelerators.

ATI Avivo is a set of hardware and low level software features present on the ATI Radeon R520 family of GPUs and all later ATI Radeon products. ATI Avivo was designed to offload video decoding, encoding, and post-processing from a computer's CPU to a compatible GPU. ATI Avivo compatible GPUs have lower CPU usage when a player and decoder software that support ATI Avivo is used. ATI Avivo has been long superseded by Unified Video Decoder (UVD) and Video Coding Engine (VCE).

Unified Video Decoder is the name given to AMD's dedicated video decoding ASIC. There are multiple versions implementing a multitude of video codecs, such as H.264 and VC-1.

PureVideo is Nvidia's hardware SIP core that performs video decoding. PureVideo is integrated into some of the Nvidia GPUs, and it supports hardware decoding of multiple video codec standards: MPEG-2, VC-1, H.264, HEVC, and AV1. PureVideo occupies a considerable amount of a GPU's die area and should not be confused with Nvidia NVENC. In addition to video decoding on chip, PureVideo offers features such as edge enhancement, noise reduction, deinterlacing, dynamic contrast enhancement and color enhancement.

Video Acceleration API (VA-API) is an open source application programming interface that allows applications such as VLC media player or GStreamer to use hardware video acceleration capabilities, usually provided by the graphics processing unit (GPU). It is implemented by the free and open-source library libva, combined with a hardware-specific driver, usually provided together with the GPU driver.

X-Video Bitstream Acceleration (XvBA), designed by AMD Graphics for its Radeon GPU and APU, is an arbitrary extension of the X video extension (Xv) for the X Window System on Linux operating-systems. XvBA API allows video programs to offload portions of the video decoding process to the GPU video-hardware. Currently, the portions designed to be offloaded by XvBA onto the GPU are currently motion compensation (MC) and inverse discrete cosine transform (IDCT), and variable-length decoding (VLD) for MPEG-2, MPEG-4 ASP, MPEG-4 AVC (H.264), WMV3, and VC-1 encoded video.

Video Decode and Presentation API for Unix (VDPAU) is a royalty-free application programming interface (API) as well as its implementation as free and open-source library distributed under the MIT License. VDPAU is also supported by Nvidia.

Intel Quick Sync Video is Intel's brand for its dedicated video encoding and decoding hardware core. Quick Sync was introduced with the Sandy Bridge CPU microarchitecture on 9 January 2011 and has been found on the die of Intel CPUs ever since.

<span class="mw-page-title-main">Distributed Codec Engine</span>

Distributed Codec Engine (DCE) is an API and its implementation as software library ("libdce") by Texas Instruments. The library was released under the Revised BSD License and some additional terms.

nVidia introduced the Video Processing Engine or VPE with the GeForce 4 MX. It is a feature of nVidia's GeForce graphics processor line that offers dedicated hardware to offload parts of the MPEG2 decoding and encoding. The GeForce Go FX 5700 rolled out the VPE 3.0. The VPE later developed into nVidia's PureVideo.

Nvidia's High-Definition Digital Processing (HDVP) is an HDTV accelerator on the GeForce 2 GTS. It has a downscaler that supports 1080i and 720p to SDTV resolution. In combination with a tuner chip it creates an accelerated HDTV viewing system that supports time-shifted recording. The Geforce 2 GTS also includes second generation motion compensation, improved from the motion compensation on the GeForce 256. It does not seem to include IDCT acceleration. The HDVP also includes de-interlace acceleration including bob, weave, temporal filter, and advanced de-interlacing. Finally, HDVP supports subpicture compositing, and color enhancements including brightness, hue, contrast, and saturation. nVidia's HDVP would endure through the GeForce 4 series in the Geforce 4 Ti NV25.

Codec acceleration describes computer hardware that offloads the computationally intensive compression or decompression. This allows, for instance, a mobile phone to decode what would generally be a very difficult, and expensive video to decode it with no stuttering, and using less battery life than un-accelerated decoding would have taken. Similar acceleration is used on a broad variety of other appliances and computers for similar reasons. What could take a general purpose processor 100 Watts to decode on a general purpose processor, could take 10W on a graphics processing unit, and even less on a dedicated hardware codec.

Video Code Engine is AMD's video encoding application-specific integrated circuit implementing the video codec H.264/MPEG-4 AVC. Since 2012 it was integrated into all of their GPUs and APUs except Oland.

Crystal HD is Broadcom's hardware semiconductor intellectual property (SIP) core that performs video decoding.

Nvidia NVDEC is a feature in its graphics cards that performs video decoding, offloading this compute-intensive task from the CPU. NVDEC is a successor of PureVideo and is available in Kepler and later NVIDIA GPUs.

Video Core Next is AMD's brand for its dedicated video encoding and decoding hardware core. It is a family of hardware accelerator designs for encoding and decoding video, and is built into AMD's GPUs and APUs since AMD Raven Ridge, released January 2018.

References

  1. ↑ nouveau Wiki - Feature Matrix
  2. ↑ "Archived copy". Archived from the original on September 26, 2015. Retrieved February 26, 2009.{{cite web}}: CS1 maint: archived copy as title (link)
  3. ↑ Linux Graphics Drivers from Intel - i810 - Intel 8xx integrated graphics chipsets
  4. ↑ AMD's X-Video Bitstream Acceleration // phoronix, October 28, 2008
  5. ↑ X-Video Motion Compensation - API specification v. 1.0 Archived May 15, 2007, at the Wayback Machine
  6. ↑ MythTV 0.25 release notes
  7. ↑ XvMC API specification version 1.0 Archived May 15, 2007, at the Wayback Machine
  8. ↑ freedesktop.org - Software/vaapi
  9. ↑ https://web.archive.org/web/20150305200949/http://www.tuaw.com/2011/01/20/xbmc-for-ios-and-atv2-now-available/ XBMC for iOS and Apple TV now available
  10. ↑ "VideoToolbox - Apple Developer Documentation". developer.apple.com. Retrieved 2018-08-05.
  11. ↑ "Archived copy" (PDF). Archived from the original (PDF) on April 9, 2011. Retrieved January 16, 2012.{{cite web}}: CS1 maint: archived copy as title (link) OpenVideo Decode (OVD) API
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.