BBC Redux

Last updated

BBC Redux is a BBC Research & Development system that digitally records [1] television and radio output in the United Kingdom produced by the British Broadcasting Corporation. [2] :2 It has been operating since 2007 [1] and contains several petabytes of recordings [1] and subtitle data. [1] It is notable for being the proof of concept for the Flash video streaming version of the BBC iPlayer. [2] :15

Contents

It is an internal research project developed for testing [3] [4] [5] which acts as a giant video on demand or personal video recorder (PVR). [3] It contains a complete digital archive, recording both television and radio twenty-four hours a day, of all of the BBC's national and also some regional broadcast output since mid-2007, [6] [7] and is automatically compiled without human input. [2] :5 [5] The BBC stated that BBC Redux is one of its major contributions to the field of digital archiving and preservation. [8]

Some accounts for accessing the system on a temporary basis were made available at Mashed 08 [9] and again at Culture Hack Day 2011, [10] providing streaming-only access to BBC content broadcast during the weekend of the event. [11] As well as streaming, the system enables high-quality downloads of television and radio content, [12] and has had the option to download subtitles from programmes since 2008. [13]

BBC Redux had originally been developed at the BBC's Kingswood Warren campus, [14] in only two months, [15] and with the investment required being significantly less than the iPlayer. [5] The saved content can be used for broadcast compliance checking [2] :16 and by BBC programme researchers. [16] As of 2012, BBC Redux is only available to employees, because existing legal contracts with content producers limit how material can be broadcast, distributed and made available to general consumers. [17] [18] The Readme file for associated API frameworks hosted on GitHub states:

BBC Snippets and BBC Redux are tools designed to allow BBC staff to develop new ways to view and navigate content. As such, they're not open to the public. [19]

Technical details

The system records over 100 megabits per second, continuously. [2] :3As of August 2011, BBC Redux contained 300,000 hours of recorded audio and video. [20]

A series of standard Digital Video Broadcasting terrestrial antennas and satellite dishes, coupled to DVB-T and DVB-S TV tuner cards are used to capture the incoming DVB multiplexes [18] transmitted for over-the-air Freesat and Freeview terrestrial television. [2] :5 These raw MPEG transport streams are split into single-programme MPEG transport streams, encapsulated in RTP, and sent using UDP IP multicast within the IPv4 multicast address range 233.122.227.0/24 from AS31459. [2] :5 From the multicast streams individual television programmes can be extracted and saved, without requiring any transcoding or conversion of the contained MPEG-2 video data. [2] :5

As of May 2009, racks of Sun Fire T1000 and T2000 machines were used acquiring and storing the incoming programmes respectively; while commodity x86-64 computers were used for database operations and playback transcoding. [2] :4 The T2000 storage nodes are connected by 10 Gigabit Ethernet on the network side, [2] :7 and then by serial attached SCSI to RAID boxes containing high-capacity commodity Serial ATA hard disk drives. [2] :7 The "fsck-free" ZFS file system is used after experiments with Unix File System (UFS) proved it to be too slow. [2] :8 Sun Microsystems had to manually repair the filesystems on two occasions using Unix dd. [2] :11–13

The software is based on open source technologies, [5] and uses a combination of mod_perl and C running on OpenSolaris. [2] :9,14 A series of "lolcat" images are used for the system's HTTP 404 and error pages. [21]

Projects

As part of the European Union (EU) "NoTube" project running between 2009–12, a recommendation research system using Lonclass categorisation and Tanimoto coefficient matching was tested by the BBC R&D Audience Experience team and integrated with 23,000 recordings delivered from Redux. [22] The matching dataset was gathered over a period of five months. [23]

For a BBC Digital Media Initiative (DMI) demonstration entitled "Million Minutes", files from the BBC's D-3 video tape archive were imported into the Redux system during 2009–2010. [16] This also used commercial software from Artesia Digital Media Group and involved creating a representational state transfer (REST) interface onto the content stored within Redux. [24]

During 2010, Safari [25] and Google Chrome [26] browser extensions were developed to integrate Redux content with the main www.bbc.co.uk/programmes directory. [27]

In January 2012, the BBC's Multimedia Classification team announced they were hoping to test and add "mood-based navigation" to the existing BBC Redux interface, [28] [29] [30] along with audience measurement and other rich metadata comprising work part-funded by the Technology Strategy Board. [6] During 2010–2011 BBC Research and Development integrated content archived in BBC Redux with the BBC's existing internal BBC InFax system, allowing finding of metadata and archive content within the same browser window, [31] covering news and subtitles from over the previous five years. [32]

During March 2012, the Atlas index changed the method of equivalence matching used for indexing against BBC Redux. [33]

Related Research Articles

Digital Video Broadcasting open standard for digital television broadcasting

Digital Video Broadcasting (DVB) is a set of international open standards for digital television. DVB standards are maintained by the DVB Project, an international industry consortium, and are published by a Joint Technical Committee (JTC) of the European Telecommunications Standards Institute (ETSI), European Committee for Electrotechnical Standardization (CENELEC) and European Broadcasting Union (EBU).

DVB-T, short for Digital Video Broadcasting — Terrestrial, is the DVB European-based consortium standard for the broadcast transmission of digital terrestrial television that was first published in 1997 and first broadcast in Singapore in February, 1998. This system transmits compressed digital audio, digital video and other data in an MPEG transport stream, using coded orthogonal frequency-division multiplexing modulation. It is also the format widely used worldwide for Electronic News Gathering for transmission of video and audio from a mobile newsgathering vehicle to a central receive point. It is also used in the US by Amateur television operators.

Advanced Television Systems Committee (ATSC) standards are an American set of standards for digital television transmission over terrestrial, cable and satellite networks. It is largely a replacement for the analog NTSC standard and, like that standard, is used mostly in the United States, Mexico, Canada, and South Korea. Several former NTSC users, in particular Japan, have not used ATSC during their digital television transition, because they adopted their own system called ISDB.

Broadcast television systems are the encoding or formatting standards for the transmission and reception of terrestrial television signals. There were three main analog television systems in use around the world until the late 2010s: NTSC, PAL, and SECAM. Now in digital terrestrial television (DTT), there are four main systems in use around the world: ATSC, DVB, ISDB and DTMB.

VLC media player Free and open-source media player and streaming media server

VLC media player is a free and open-source, portable, cross-platform media player software and streaming media server developed by the VideoLAN project. VLC is available for desktop operating systems and mobile platforms, such as Android, iOS, iPadOS, Windows 10 Mobile, and Windows Phone. VLC is also available on digital distribution platforms such as Apple's App Store, Google Play, and Microsoft Store.

Digital terrestrial television is a technology for terrestrial television in which land-based (terrestrial) television stations broadcast television content by radio waves to televisions in consumers' residences in a digital format. DTTV is a major technological advance over the previous analog television, and has largely replaced analog which had been in common use since the middle of the 20th century. Test broadcasts began in 1998 with the changeover to DTTV beginning in 2006 and is now complete in many countries. The advantages of digital terrestrial television are similar to those obtained by digitising platforms such as cable TV, satellite, and telecommunications: more efficient use of limited radio spectrum bandwidth, provision of more television channels than analog, better quality images, and potentially lower operating costs for broadcasters.

DVB-S2 Digital satellite television standard

Digital Video Broadcasting - Satellite - Second Generation (DVB-S2) is a digital television broadcast standard that has been designed as a successor for the popular DVB-S system. It was developed in 2003 by the Digital Video Broadcasting Project, an international industry consortium, and ratified by ETSI in March 2005. The standard is based on, and improves upon DVB-S and the electronic news-gathering system, used by mobile units for sending sounds and images from remote locations worldwide back to their home television stations.

Internet Protocol television Television transmitted over a computer network

Internet Protocol television (IPTV) is the delivery of television content over Internet Protocol (IP) networks. This is in contrast to delivery through traditional terrestrial, satellite, and cable television formats. Unlike downloaded media, IPTV offers the ability to stream the source media continuously. As a result, a client media player can begin playing the content almost immediately. This is known as streaming media.

Datacasting is the broadcasting of data over a wide area via radio waves. It most often refers to supplemental information sent by television stations along with digital terrestrial television, but may also be applied to digital signals on analog TV or radio. It generally does not apply to data which is inherent to the medium, such as PSIP data which defines virtual channels for DTT or direct broadcast satellite systems; or to things like cable modem or satellite modem, which use a completely separate channel for data.

MPEG transport stream or simply transport stream (TS) is a standard digital container format for transmission and storage of audio, video, and Program and System Information Protocol (PSIP) data. It is used in broadcast systems such as DVB, ATSC and IPTV.

1080p Video mode

1080p is a set of HDTV high-definition video modes characterized by 1,920 pixels displayed across the screen horizontally and 1,080 pixels down the screen vertically; the p stands for progressive scan, i.e. non-interlaced. The term usually assumes a widescreen aspect ratio of 16:9, implying a resolution of 2.1 megapixels. It is often marketed as Full HD or FHD, to contrast 1080p with 720p resolution screens. Although 1080p is sometimes informally referred to as 2K, these terms reflect two distinct technical standards, with differences including resolution and aspect ratio.

Common Interface

In Digital Video Broadcasting, the Common Interface is a technology which allows decryption of pay TV channels. Pay TV stations want to choose which encryption method to use. The Common Interface allows TV manufacturers to support many different pay TV stations, by allowing to plug in exchangeable conditional-access modules (CAM) for various encryption schemes.

BBC Research & Development

BBC Research & Development is the technical research department of the BBC.

IP over DVB implies that Internet Protocol datagrams are distributed using some digital television system, for example DVB-H, DVB-T, DVB-S, DVB-C or their successors like DVB-S2. This may take the form of IP over MPEG, where the datagrams are transferred over the MPEG transport stream, or the datagrams may be carried in the DVB baseband frames directly, as in GSE.

DVB-T2 is an abbreviation for "Digital Video Broadcasting — Second Generation Terrestrial"; it is the extension of the television standard DVB-T, issued by the consortium DVB, devised for the broadcast transmission of digital terrestrial television. DVB has been standardized by ETSI.

High-definition television (HD) describes a television system providing an image resolution of substantially higher resolution than the previous generation of technologies. The term has been used since 1936, but in modern times refers to the generation following standard-definition television (SDTV), often abbreviated to HDTV or HD-TV. It is the current de facto standard video format used in most broadcasts: terrestrial broadcast television, cable television, satellite television and Blu-ray Discs.

GPAC Project on Advanced Content

GPAC Project on Advanced Content is an implementation of the MPEG-4 Systems standard written in ANSI C. GPAC provides tools for media playback, vector graphics and 3D rendering, MPEG-4 authoring and distribution.

Hybrid Broadcast Broadband TV Industry standard for hybrid digital television

Hybrid Broadcast Broadband TV (HbbTV) is both an industry standard and promotional initiative for hybrid digital TV to harmonise the broadcast, IPTV, and broadband delivery of entertainment to the end consumer through connected TVs and set-top boxes. The HbbTV Association, comprising digital broadcasting and Internet industry companies, has established a standard for the delivery of broadcast TV and broadband TV to the home, through a single user interface, creating an open platform as an alternative to proprietary technologies. Products and services using the HbbTV standard can operate over different broadcasting technologies, such as satellite, cable, or terrestrial networks.

Digital multimedia broadcasting The South Korean digital TV standard

Digital multimedia broadcasting (DMB) is a digital radio transmission technology developed in South Korea as part of the national IT project for sending multimedia such as TV, radio and datacasting to mobile devices such as mobile phones, laptops and GPS navigation systems. This technology, sometimes known as mobile TV, should not be confused with Digital Audio Broadcasting which was developed as a research project for the European Union. DMB was developed in South Korea as the next generation digital technology to replace FM radio, but the technological foundations were laid by Prof. Dr. Gert Siegle and Dr. Hamed Amor at Robert Bosch GmbH in Germany. The world's first official mobile TV service started in South Korea in May 2005, although trials were available much earlier. It can operate via satellite (S-DMB) or terrestrial (T-DMB) transmission. DMB has also some similarities with the main competing mobile TV standard, DVB-H.

References

  1. 1 2 3 4 Solon, Olivia (16 June 2011). "The people and projects inside the BBC's vast R&D lab". Wired. Archived from the original on 21 July 2012. Retrieved 12 May 2012.
  2. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Bird, Tom (28 May 2009). "BBC Redux" (PDF). Sheffield: UKNOF 13: 2–16.Cite journal requires |journal= (help)
  3. 1 2 Butterworth, Brandon (27 October 2008). "History of the 'BBC Redux' project". BBC Internet Blog. Retrieved 12 May 2012.
  4. Saran, Cliff (20 April 2012). "BBC wants to build a cultural API". ComputerWeekly.com . Retrieved 13 May 2012. an internal version of BBC iPlayer called Redux, which was originally developed for testing
  5. 1 2 3 4 Rose, Anthony (2008). Kozamernik, F. (ed.). "Evolution of the BBC iPlayer" (PDF). EBU Technical Review. European Broadcasting Union (Q4): 2. Retrieved 13 May 2012.;
    Rose, Anthony (24 February 2009). Kozamernik, F. (ed.). "Evolution of the BBC iPlayer" (PDF). EBU Technical Review. European Broadcasting Union (The Best of 2008): 5. Retrieved 13 May 2012.
  6. 1 2 Tinley, Dominic (8 November 2011). "ABC-IP and work on audio archiving research". BBC Research & Development Blog.
  7. Tweed, Jonathan (7 November 2011). "Introducing Tellytopic: Navigating programme archives by topic". MetaBroadcast. Retrieved 13 May 2012. BBC Redux, a playable archive of almost everything broadcast since mid-2007
  8. Wright, Richard (16 June 2010). "The Value of Everything". BBC Research & Development Blog.
  9. Johnson, Bobbie (26 June 2008). "Hackers fan the flames of imagination at Mashed". The Guardian. Retrieved 13 May 2012.
  10. Williams, Dan W. (14 January 2011). "BBC Archive − A number of Redux accounts giving…". Tumblr.
  11. "Datasets". Culture Hack Day. 14 January 2011. Archived from the original on 2 December 2012. Retrieved 12 May 2012.
  12. "Programme" (PDF). Geneva: European Broadcasting Union. 24 September 2008. Retrieved 20 May 2012. BBC Research developed a Redux site enabling a high quality download service of some BBC TV products.Unknown parameter |conference= ignored (help)
  13. "BBC Redux". Mashed 08. Backnetwork. Madgex. 2008. Archived from the original on 15 September 2011. Retrieved 12 May 2012.
  14. Forrester, Ian (14 March 2011). "Goodbye Kingswood warren". Cubic Garden.
  15. Singleton, Alex (May 2008) [13 March 2008]. "Linux at the BBC" (PDF). Linux Format (105): 50. Archived from the original (Platform-neutral content) on 1 September 2012. Retrieved 19 May 2012. Brandon Butterworth explains: 'People thought it would be really difficult to run on all platforms, that it would take years to develop. But I led a team to do a Flash version that would work on Linux and we did it in two months'
  16. 1 2 Highlights of the Year (PDF). BBC R&D Annual Review 2009–2010 (Report). 10 March 2011. p. 4. Retrieved 13 May 2012.
  17. Rivera, Ralph (11 May 2012). "BBC Online Briefing: Spring 2012 Final Q&A" (television question and answer session, offset 23:28–28:30). BBC Internet Blog.
  18. 1 2 Tinworth, Adam (20 April 2012). "The UK's cultural heritage as digital public space". One Man and His Blog. Retrieved 13 May 2012. BBC Redux is iPlayer on steroids – with an API. It records the whole digital multiplex, … But the rights are complicated- because the BBC doesn't own many of the programmes it broadcasts.
  19. Haynes, Matt (25 January 2012). "README.md". redux-client-ruby. GitHub. Retrieved 13 May 2012.
  20. Davies, Sam; Bland, Denise (1 August 2011). "BBC R&D – Archives Research" (PDF). p. 4. Retrieved 13 May 2012.
  21. Eden, Terence (30 September 2011). "I do like the BBC Redux 404 page". @edent. Twitter.
  22. "NoTube Archive Browser Experiment 1". NoTube Archive Browser. 19 December 2011. Archived from the original on 6 March 2011. Retrieved 13 May 2012.
  23. Miller, Libby (14 October 2011). "Algorithms for recommendations in various N-Screen implementations". NoTube. Retrieved 15 May 2012.
  24. James, Bruce (5 November 2009). "ArcHak". CustardCat. Archived from the original on 4 March 2016. Retrieved 13 May 2012.
  25. Starling, Sam (4 September 2011). "README.md". Safari-Redux-Switcher. GitHub. Retrieved 15 May 2012.
  26. Robertson, Duncan (26 August 2010). "README.md". Redux Switcher. GitHub. Retrieved 15 May 2012.
  27. "Prototyping Weeknotes #29". BBC Research & Development Blog. 27 August 2010. Retrieved 13 May 2012.
  28. DTG Staff (18 January 2012). "BBC R&D develops 'mood based navigation'". News. Digital TV Group. Archived from the original on 29 March 2012. Retrieved 12 May 2012.
  29. Campbell, Rosie (16 January 2012). "An Affective Interface for Mood-Based Navigation". BBC Research & Development Blog. Retrieved 13 May 2012.
  30. BBC Research and Development. "Multimedia Classification". Projects. Retrieved 13 May 2012. User Evaluation on BBC Redux (by BBC staff) …
  31. Highlights of the Year (PDF). BBC R&D Annual Report 2010–2011 (Report). 15 November 2011. p. 5.
  32. McAthy, Rachel (20 April 2012). "#ODCC – Open data and the 'new digital fields of exchange'". Journalism.co.uk. Mousetrap Media Ltd.
  33. Tweed, Jonathan (21 March 2012). "Atlas release notes - week to 20th March". MetaBroadcast. Archived from the original on 29 July 2017. Retrieved 15 May 2012. Changes to the behaviour of BBC Redux equivalence