GStreamer

Last updated
GStreamer
Developer(s) GStreamer Team
Initial release11 January 2001;22 years ago (2001-01-11) [1]
Stable release
1.22.7 [2]   OOjs UI icon edit-ltr-progressive.svg / 13 November 2023
Repository
Written in C [3]
Operating system BSDs, OpenSolaris, Android, macOS, iOS, Windows, OS/400
Type Multimedia framework
License LGPL-2.1-or-later [4]
Website gstreamer.freedesktop.org

GStreamer is a pipeline-based multimedia framework that links together a wide variety of media processing systems to complete complex workflows. For instance, GStreamer can be used to build a system that reads files in one format, processes them, and exports them in another. The formats and processes can be changed in a plug and play fashion.

Contents

GStreamer supports a wide variety of media-handling components, including simple audio playback, audio and video playback, recording, streaming and editing. The pipeline design serves as a base to create many types of multimedia applications such as video editors, transcoders, streaming media broadcasters and media players.

It is designed to work on a variety of operating systems, e.g. the BSDs, OpenSolaris, Android, macOS, iOS, Windows, OS/400.

GStreamer is free and open-source software subject to the terms of the LGPL-2.1-or-later [4] and is being hosted at freedesktop.org.

Distribution and adoption

The GNOME desktop environment, a heavy user of GStreamer, has included GStreamer since GNOME version 2.2 and encourages GNOME and GTK applications to use it. Other projects also use or support it, such as the Phonon media framework and the Songbird media player. It is also used in the WebKit browser engine. [5]

GStreamer also operates in embedded devices like the Jolla Phone, the Palm Pre, [6] Tizen and the Nokia 770, N800, N810, N900 and N9 Internet Tablets running the Maemo operating system.

In addition to source code releases, the GStreamer project provides binary builds for Android, iOS, OSX and Windows. [7]

The LIGO Laboratory make use of GStreamer to simulate and analyze gravitational wave data. The GStreamer interface is called GstLAL. [8]

Software architecture

Overview GStreamer overview.svg
Overview
GStreamer core with three different types of plugins GStreamer and plug-in types.svg
GStreamer core with three different types of plugins
GStreamer is a Pipeline. GStreamer Technical Overview.svg
GStreamer is a Pipeline.

GStreamer is written in the C programming language with the type system based on GObject and the GLib 2.0 object model.

Language bindings

A library written in one programming language may be used in another language if bindings are written; GStreamer has a range of bindings for various languages such as Go, Python, Rust, Vala, C++, Perl, GNU Guile, C# and Ruby.

Overview

GStreamer processes media by connecting a number of processing elements into a pipeline. Each element is provided by a plug-in. Elements can be grouped into bins, which can be further aggregated, thus forming a hierarchical graph. This is an example of a filter graph.

Elements communicate by means of pads. A source pad on one element can be connected to a sink pad on another. When the pipeline is in the playing state, data buffers flow from the source pad to the sink pad. Pads negotiate the kind of data that will be sent using capabilities.

The diagram to the right could exemplify playing an MP3 file using GStreamer. The file source reads an MP3 file from a computer's hard-drive and sends it to the MP3 decoder. The decoder decodes the file data and converts it into PCM samples which then pass to the sound-driver. The sound-driver sends the PCM sound samples to the computer's speakers.

Plug-ins

GStreamer uses a plug-in architecture which makes the most of GStreamer's functionality implemented as shared libraries. [9] GStreamer's base functionality contains functions for registering and loading plug-ins and for providing the fundamentals of all classes in the form of base classes. Plug-in libraries get dynamically loaded to support a wide spectrum of codecs, container formats, input/output drivers and effects.

Plug-ins can be installed semi-automatically when they are first needed. For that purpose distributions can register a backend that resolves feature-descriptions to package-names.

Since version 0.9, the plug-ins come grouped into three sets (named after the film The Good, the Bad and the Ugly ). [10]

Plug-in set nameDescription
GoodThis package contains the GStreamer plug-ins from the "good" set, a set of high quality plug-ins under the LGPL license. [11]
BadGStreamer Bad Plug-ins comprises a set of plug-ins not up-to-par compared to the rest. They might closely approach good-quality plug-ins, but they lack something: perhaps a good code review, some documentation, a set of tests, a real live maintainer, or some actual wide use. [12]
UglyThis package contains plug-ins from the "ugly" set, a set of good-quality plug-ins that might pose distribution problems. [13]

Individual distributions may further sub-classify these plug-ins: for example Ubuntu groups the "bad" and "ugly" sets into the "Universe" or the "Multiverse" components.

In addition, there is a GStreamer FFmpeg plug-in (called gst-libav for historic reasons [14] ) that extends the number of supported media formats.

Video acceleration

Through special plugins provided by Texas Instruments, GStreamer makes use of hardware acceleration provided by e.g. Texas Instruments DaVinci. GStreamer and TI DMAI.svg
Through special plugins provided by Texas Instruments, GStreamer makes use of hardware acceleration provided by e.g. Texas Instruments DaVinci.

There are various SIP blocks that can do the computations to decode certain video codecs, such as PureVideo, UVD, QuickSync Video, TI Ducati and more. Such needs to be supported by the device driver, which in turn provides one or multiple interfaces, like VDPAU, VAAPI, Distributed Codec Engine or DXVA to end-user software like MPlayer to access this hardware and offload computation to it.

Media formats

The Good, Bad and Ugly GStreamer plugins mentioned earlier provide, alongside processing elements/filters of all kinds, support for a wide variety of file formats, protocols and multimedia codecs. In addition to those, support for more than a hundred compression formats (including MPEG-1, MPEG-2, MPEG-4, H.261, H.263, H.264, RealVideo, MP3, WMV, etc. [20] ) is transparently provided through the gst-libav plug-in.

History and development

Early days

Erik Walthinsen founded the GStreamer project in 1999. Many of its core design ideas came from a research project at the Oregon Graduate Institute. [21] Wim Taymans joined the project soon thereafter and greatly expanded on many aspects of the system. Many other software developers have contributed since then.

The first major release was 0.1.0 which was announced on 11 January 2001. [1] Not long after, GStreamer picked up its first commercial backer. Towards the end of January 2001, they hired Erik Walthinsen to develop methods for embedding GStreamer in smaller (cell phone-class) devices. Another RidgeRun employee, Brock A. Frazier, designed the GStreamer logo. RidgeRun later struggled financially and had to lay off its staff, including Erik Walthinsen. GStreamer progress was mostly unaffected.

The project released a series of major releases with 0.2.0 coming out in July 2001, 0.4.0 in September 2002, and 0.8.0 in March 2004. During that period the project also changed its versioning strategy and while the first releases were simply new versions, later on the middle number started signifying release series. This meant the project did release a string of 0.6.x and 0.8.x releases which was meant to stay binary compatible within those release series. Erik Walthinsen more or less left GStreamer development behind during this time, focusing on other ventures.

All release series, the project face difficulties. Every series is not very popular in the Linux community mostly because of stability issues and a serious lack of features compared to competing projects like Xine, MPlayer, and VLC. The project also suffers a lack of leadership as Wim Taymans, the project lead since Erik Walthinsen had left, had largely stopped participating.

The 0.10 series

In 2004, a new company was founded, Fluendo, which wanted to use GStreamer to write a streaming server Flumotion and also provide multimedia solutions for GStreamer. During this time, Fluendo hired most of the core developers including Wim Taymans and attracted the support of companies such as Nokia and Intel to bring GStreamer to a professional level and drive community adoption.

With Wim Taymans back at the helm, the core of GStreamer was redesigned and became what is the current 0.10.x series, which had its first release (0.10.0) in December 2005. [22] It has maintained API and ABI compatibility since.

With a new stable core in place, GStreamer gained in popularity in 2006, being used by media players including Totem, Rhythmbox and Banshee with many more to follow. It was also adopted by corporations such as Nokia, Motorola, Texas Instruments, Freescale, Tandberg, and Intel.

In 2007, most of the core GStreamer developers left Fluendo, including GStreamer maintainer Wim Taymans who went on to co-found Collabora Multimedia together with other GStreamer veterans, while others joined Sun Microsystems, Oblong Industries, and Songbird.

Between June 2012 and August 2014, GStreamer 0.10 was also distributed by Collabora and Fluendo as a multiplatform SDK, [23] on the third-party gstreamer.com website (rather than gstreamer.freedesktop.org for the upstream community project). The goal was to provide application developers with a SDK that would be functionally identical on Windows, Mac OS X, iOS, and Android. The SDK initiative aimed to facilitate the commercial adoption of the GStreamer project, as it provided a standardized entry point to developing multimedia applications with GStreamer, without needing to build the entire platform by oneself. Users of the SDK also benefited from documentation Archived 2012-06-16 at the Wayback Machine , tutorials and instructions specific to that SDK.

The 1.x series

GStreamer 1.0 was released on September 24, 2012. [24] The 1.x series is parallel installable to GStreamer 0.10 to ease the transition, and provides many architectural advantages over the 0.10 series. [25] Generally speaking, GStreamer 1.0 brought significant improvements for:

Beyond the technical improvements, the 1.x series is also defined by a new release versioning scheme. As the GStreamer roadmap explains, [26] all 1.x.y versions carry a -1.0 API version suffix and have a stable API/ABI. The API/ABI can only be broken by a new major release series (i.e.: 2.x); however, there are currently no plans for a 2.0 release series. Until then, the new version numbering scheme can be used to predict the intended use of each release. The roadmap cites some examples:

In March 2013, the GStreamer project maintainers issued a statement [27] to clarify that the 0.10 series is no longer maintained. The statement reasserted the GStreamer project's willingness to help application and plugin developers migrate to the new technology, and hinted that those for whom switching to the 1.x series was still considered impossible could seek assistance from various consulting companies.

1.2 added support for DASH adaptive streaming, JPEG 2000 images, VP9 and Daala video, and decoding-only support for WebP.

Version 1.14 was released on March 19, 2018, [28] adding support for WebRTC, AV1, Nvidia NVDEC, and Secure Reliable Transport, among other changes.

Version 1.22 was released on January 23, 2023, [29] adding improved support for AV1, in addition to support for HLS, DASH and Microsoft Smooth Streaming for adaptive bitrate streaming.

See also

Related Research Articles

<span class="mw-page-title-main">Winamp</span> Media player for Microsoft Windows

Winamp is a media player for Microsoft Windows originally developed by Justin Frankel and Dmitry Boldyrev by their company Nullsoft, which they later sold to AOL in 1999 for $80 million. It was then acquired by Radionomy in 2014, now known as the Llama Group. Since version 2 it has been sold as freemium and supports extensibility with plug-ins and skins, and features music visualization, playlist and a media library, supported by a large online community.

<span class="mw-page-title-main">MPlayer</span> Free and open source media player

MPlayer is a free and open-source media player software application. It is available for Linux, OS X and Microsoft Windows. Versions for OS/2, Syllable, AmigaOS, MorphOS and AROS Research Operating System are also available. A port for DOS using DJGPP is also available. Versions for the Wii Homebrew Channel and Amazon Kindle have also been developed.

Theora is a free lossy video compression format. It was developed by the Xiph.Org Foundation and distributed without licensing fees alongside their other free and open media projects, including the Vorbis audio format and the Ogg container.

<span class="mw-page-title-main">FFmpeg</span> Multimedia framework

FFmpeg is a free and open-source software project consisting of a suite of libraries and programs for handling video, audio, and other multimedia files and streams. At its core is the command-line ffmpeg tool itself, designed for processing of video and audio files. It is widely used for format transcoding, basic editing, video scaling, video post-production effects and standards compliance.

<span class="mw-page-title-main">OpenMPT</span> Open source module tracker

OpenMPT is an open-source audio module tracker for Windows. It was previously called ModPlug Tracker, and was first released by Olivier Lapicque in September 1997.

The following comparison of video players compares general and technical information for notable software media player programs.

FAAC or Freeware Advanced Audio Coder is a software project which includes the AAC encoder FAAC and decoder FAAD2. It supports MPEG-2 AAC as well as MPEG-4 AAC. It supports several MPEG-4 Audio object types, file formats, multichannel and gapless encoding/decoding and MP4 metadata tags. The encoder and decoder is compatible with standard-compliant audio applications using one or more of these object types and facilities. It also supports Digital Radio Mondiale.

<span class="mw-page-title-main">Indeo</span> Audio and video formats by Intel

Indeo Video is a family of audio and video formats and codecs first released in 1992, and designed for real-time video playback on desktop CPUs. While its original version was related to Intel's DVI video stream format, a hardware-only codec for the compression of television-quality video onto compact discs, Indeo was distinguished by being one of the first codecs allowing full-speed video playback without using hardware acceleration. Also unlike Cinepak and TrueMotion S, the compression used the same Y'CbCr 4:2:0 colorspace as the ITU's H.261 and ISO's MPEG-1. Indeo use was free of charge to allow for broadest usage.

libavcodec is a free and open-source library of codecs for encoding and decoding video and audio data.

<span class="mw-page-title-main">Banshee (media player)</span> Open source media player

Banshee was a cross-platform open-source media player, called Sonance until 2005. Built upon Mono and Gtk#, it used the GStreamer multimedia platform for encoding, and decoding various media formats, including Ogg Vorbis, MP3 and FLAC. Banshee can play and import audio CDs and supports many portable media players, including Apple's iPod, Android devices and Creative's ZEN players. Other features include Last.fm integration, album artwork fetching, smart playlists and podcast support. Banshee is released under the terms of the MIT License. Stable versions are available for many Linux distributions, as well as a beta preview for OS X and an alpha preview for Windows.

<span class="mw-page-title-main">Java Media Framework</span>

The Java Media Framework (JMF) is a Java library that enables audio, video and other time-based media to be added to Java applications and applets. This optional package, which can capture, play, stream, and transcode multiple media formats, extends the Java Platform, Standard Edition and allows development of cross-platform multimedia applications.

<span class="mw-page-title-main">Moonlight (runtime)</span> Implementation of Microsoft Silverlight for some Unix-based operating systems

Moonlight is a discontinued free and open source implementation for Linux and other Unix-based operating systems of the Microsoft Silverlight application framework, developed and then abandoned by the Mono Project. Like Silverlight, Moonlight was a web application framework which provided capabilities similar to those of Adobe Flash, integrating multimedia, graphics, animations and interactivity into a single runtime environment.

Video Acceleration API (VA-API) is an open source application programming interface that allows applications such as VLC media player or GStreamer to use hardware video acceleration capabilities, usually provided by the graphics processing unit (GPU). It is implemented by the free and open-source library libva, combined with a hardware-specific driver, usually provided together with the GPU driver.

Video Decode and Presentation API for Unix (VDPAU) is a royalty-free application programming interface (API) as well as its implementation as free and open-source library distributed under the MIT License. VDPAU is also supported by Nvidia.

The following comparison of audio players compares general and technical information for a number of software media player programs. For the purpose of this comparison, "audio players" are defined as any media player explicitly designed to play audio files, with limited or no support for video playback. Multi-media players designed for video playback, which can also play music, are included under comparison of video player software.

The HTML5 specification introduced the video element for the purpose of playing videos, partially replacing the object element. HTML5 video is intended by its creators to become the new standard way to show video on the web, instead of the previous de facto standard of using the proprietary Adobe Flash plugin, though early adoption was hampered by lack of agreement as to which video coding formats and audio coding formats should be supported in web browsers. As of 2020, HTML5 video is the only widely supported video playback technology in modern browsers, with the Flash plugin being phased out.

<span class="mw-page-title-main">GNOME Videos</span> Media player software of the desktop environment GNOME

GNOME Videos, formerly known as Totem, is a media player for the GNOME computer desktop environment. GNOME Videos uses the Clutter and GTK+ toolkits. It is officially included in GNOME starting from version 2.10, but de facto it was already included in most GNOME environments. Totem utilizes the GStreamer framework for playback, though until version 2.27.1, it could alternatively be configured to use the Xine libraries instead of GStreamer.

<span class="mw-page-title-main">Libav</span> Multimedia framework

Libav is an abandoned free software project, forked from FFmpeg in 2011, that contains libraries and programs for handling multimedia data.

<span class="mw-page-title-main">Opus (audio format)</span> Lossy audio coding format

Opus is a lossy audio coding format developed by the Xiph.Org Foundation and standardized by the Internet Engineering Task Force, designed to efficiently code speech and general audio in a single format, while remaining low-latency enough for real-time interactive communication and low-complexity enough for low-end embedded processors. Opus replaces both Vorbis and Speex for new applications, and several blind listening tests have ranked it higher-quality than any other standard audio format at any given bitrate until transparency is reached, including MP3, AAC, and HE-AAC.

Video Code Engine is AMD's video encoding application-specific integrated circuit implementing the video codec H.264/MPEG-4 AVC. Since 2012 it was integrated into all of their GPUs and APUs except Oland.

References

  1. 1 2 "GStreamer "Slipstream" 0.1.0 released". 11 January 2001. Archived from the original on 11 November 2012. Retrieved 3 November 2010.
  2. ↑ "GStreamer 1.22.7 stable bug-fix release".
  3. ↑ "GStreamer", Ohloh Analysis Summary, Ohloh, archived from the original on 2014-06-26, retrieved 2016-11-06
  4. 1 2 "What are the exact licensing terms for GStreamer and its plugins?". freedesktop.org . Archived from the original on 2021-06-07. Retrieved 2021-06-07.
  5. ↑ "Igalia Multimedia". Archived from the original on 2021-09-01. Retrieved 2021-09-01.
  6. ↑ "webOS and GStreamer". webOShelp. Archived from the original on 22 March 2009. Retrieved 25 July 2009.
  7. ↑ "GStreamer: Download". gstreamer.freedesktop.org. Archived from the original on 2015-05-10. Retrieved 15 May 2015.
  8. ↑ "GstLAL Project Page". Wiki.ligo.org. Archived from the original on 23 April 2020. Retrieved 6 May 2019.
  9. ↑ Overview of the modules, Gstreamer.freedesktop.org, archived from the original on 25 January 2012, retrieved 8 February 2012
  10. ↑ GStreamer 0.9 development series - Hung by a Thread, Gstreamer.freedesktop.org, archived from the original on 27 January 2013, retrieved 24 February 2013
  11. ↑ "GStreamer Good Plug-ins". Archived from the original on 2019-02-07. Retrieved 2019-02-05.
  12. ↑ "GStreamer Bad Plug-ins". Archived from the original on 2019-02-07. Retrieved 2019-02-05.
  13. ↑ "GStreamer Ugly Plug-ins". Archived from the original on 2019-02-07. Retrieved 2019-02-05.
  14. ↑ "subprojects/gst-libav/README.md · 47ac79d7b8cc078f4890d0ce21f47e1c1af2c736 · GStreamer / gstreamer · GitLab". GitLab. Retrieved 2022-07-12.
  15. ↑ "GStreamer OpenMAX IL wrapper plugin". gstreamer.freedesktop.org. Archived from the original on 2017-07-10. Retrieved 2017-07-21.
  16. ↑ "Gstreamer 1.0 for raspbian". GRaspberrypi.org. Archived from the original on 2017-07-10. Retrieved 2017-07-21.
  17. ↑ "GStreamer Plug-ins for TI hardware". Processors.wiki.ti.com. Archived from the original on 2017-06-28. Retrieved 2017-07-21.
  18. ↑ "Fluendo Codec Pack Release 11 bring VDPAU and VAAPI support". 2010-03-25. Archived from the original on 2014-06-24.
  19. ↑ Debian Webmaster. "Debian - Details of package gstreamer0.10-crystalhd in wheezy". Packages.debian.org. Archived from the original on 2017-06-29. Retrieved 2017-07-21.
  20. ↑ "subprojects/gst-libav/ext/libav/gstavcodecmap.c · 47ac79d7b8cc078f4890d0ce21f47e1c1af2c736 · GStreamer / gstreamer · GitLab". GitLab. Retrieved 2022-07-12.
  21. ↑ Edge, Jake (26 October 2010). "GStreamer: Past, present, and future". LWN.net . Retrieved 15 May 2022.
  22. ↑ "GStreamer 0.10.0 stable release - Announcement of the first release in 0.10 stable series". gstreamer.freedesktop.org. Archived from the original on 2017-07-07. Retrieved 2017-07-21.
  23. ↑ "GStreamer documentation". Docs.gstreamer.com. Archived from the original on 2016-10-28. Retrieved 2017-07-21.
  24. ↑ "GStreamer 1.0 released". gstreamer.freedesktop.org. Archived from the original on 2017-07-07. Retrieved 2017-07-21.
  25. ↑ "GStreamer 1.0 and 0.10". Lwn.net. Archived from the original on 2017-06-13. Retrieved 2017-07-21.
  26. ↑ "ReleasePlanning2013 - gstreamer Wiki". Archived from the original on 2013-08-15. Retrieved 2013-09-16.
  27. ↑ "GStreamer 0.10 no longer maintained". Lists.freedesktop.org. 11 March 2013. Archived from the original on 2017-07-10. Retrieved 2017-07-21.
  28. ↑ "GStreamer 1.14 release notes". Archived from the original on 2018-03-20. Retrieved 2018-09-08.
  29. ↑ "GStreamer 1.22 release notes". gstreamer.freedesktop.org. Retrieved 2023-05-18.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.