Automatic content recognition

Last updated

Automatic content recognition [1] (ACR) is a technology used to identify content played on a media device or presented within a media file. Devices with ACR can allow for the collection of content consumption information automatically at the screen level itself, without any user-based input or search efforts. This information may be collected for purposes such as personalized advertising, content recommendations, sale to customer data aggregators and other applications. [2] [1]

Contents

How it works

To start the process, a short media clip (audio, video, or both) is selected from within a media file or captured as displayed on a device such as a smart TV. Using techniques such as fingerprinting and watermarking, the selected content is compared by the ACR software with a database of known recorded works. [1] If the fingerprint of the media clip finds a match, the ACR software returns the corresponding metadata regarding the media as well as other associated or recommended content back to the client application for display to the user, or for collection by the device manufacturer or a data aggregator. [2]

Fingerprints and watermarking

Two leading methodologies for audio-based ACR are acoustic fingerprinting and watermarking. Similarly, video fingerprinting is used to facilitate ACR for visual media.

Acoustic fingerprinting generates unique fingerprints from the audio content itself. Fingerprinting techniques are agnostic to content format, codec, bit rate and compression techniques. [3] This makes employment of acoustic fingerprinting possible across various networks and channels[ clarification needed ] and is widely used for interactive TV, second screen application, and content monitoring sectors. [4] [5] Popular apps like Shazam, YouTube, Facebook, [6] TheTake, WeChat and Weibo reportedly use audio fingerprinting methodology to recognize content played from a TV to trigger additional features like votes, lotteries, topics or purchases.[ citation needed ]

In contrast to fingerprinting, digital watermarking require the inclusion of digital "tags"[ further explanation needed ] embedded within the digital content stream prior to distribution. For example, a broadcast encoder might insert a watermark every few seconds that could be used to identify the broadcast channel, program ID, and time stamp. This watermark is normally inaudible or invisible to the users, but is detectable by display devices like phones or tablets which can read the watermarks to identify the content it is playing. [7] Watermarking technology is also utilized in the media protection field to help identify where illegal copies originate. [8]

History

In 2011, ACR technology was applied to TV content by the Shazam service, which captured the attention of the television industry. Shazam was previously a music recognition service which recognized music from sound recordings. By utilizing its own fingerprint technology to identify live channels and videos, Shazam extended their business to television programming. Also in 2011, Samba TV (at the time known as Flingo [9] ) introduced its patented video ACR technology, which uses video fingerprinting to identify on-screen content and power cross-screen interactive TV apps on Smart TVs. [10] In 2012, satellite communications provider DIRECTV partnered with TV loyalty vendor Viggle to provide an interactive viewing experience on the second screen. In 2013, LG partnered with Cognitive Networks (later purchased by Vizio and renamed Inscape), an ACR vendor, to provide ACR driven interaction. [11] In 2015, ACR technology spread to even more applications and smart TVs. Social applications and TV manufacturers like Facebook, Twitter, Google, WeChat, Weibo, LG, Samsung, and Vizio TV have used ACR technology either developed by themselves or integrated from third party ACR providers.[ citation needed ] In 2016, additional applications and mobile OS embedded with automatic content recognition services were available including Peach, Omusic and Mi OS. [12] [13] [14]

Applications

Advertising and customer data collection

Data collected on the media consumption habits of customers is very valuable to device manufacturers, advertisers, and data aggregation companies. ACR technology helps these companies survey the interests of customers and collect data so that they can be more precisely targeted with personalized marketing and advertising campaigns. It was reported in Nov 2021 that smart television manufacturer Vizio is more profitable from the sale of their customers' data than from the televisions they sold. [15]

Audience measurement

Real-time audience measurement metrics are now achievable by applying ACR technology into smart TVs, set top boxes and mobile devices such as smart phones and tablets. This measurement data is essential to quantify audience consumption to set advertising pricing policies.

Content identification

ACR technology helps audiences retrieve information about the content they watched or listened to. [16] The identified video and music content can be linked to internet content providers for on-demand viewing, third parties for additional background information, or complementary media.

Content enhancement

Because devices can be "aware" of content being watched or listened to, second screen devices can feed users complementary content beyond what is presented on the primary viewing screen. ACR technology can not only identify the content, but also it can identify the precise location within the content and present additional information to users. ACR can also enable a variety of interactive features such as polls, coupons, lottery or purchase of goods based on timestamp. [17]

Privacy concerns

Organizations ranging from consumer rights advocates Electronic Frontier Foundation to tech web sites such as PCMag have expressed serious objections to the collection of user viewing consumption habits by their devices on privacy grounds. [18] [19]

Technology providers

ACR service providers include ACRCloud, Beatgrid, Digimarc, Gracenote, Inscape Data Services, Kantar Media, Pex, Red Bee Media, Samba TV, Shazam and Zapr Media Labs.

See also

Related Research Articles

<span class="mw-page-title-main">Interactive television</span>

Interactive television is a form of media convergence, adding data services to traditional television technology. It has included on-demand delivery of content, online shopping, and viewer polls. Interactive TV is an example of how new information technology can be integrated vertically into established technologies and commercial structures.

<span class="mw-page-title-main">Digital on-screen graphic</span> Watermark-like TV station logo

A digital on-screen graphic, digitally originated graphic is a watermark-like station logo that most television broadcasters overlay over a portion of the screen area of their programs to identify the channel. They are thus a form of permanent visual station identification, increasing brand recognition and asserting ownership of the video signal.

<span class="mw-page-title-main">Digital signage</span> Sub-segment of electronic signage

Digital signage is a segment of electronic signage. Digital displays use technologies such as LCD, LED, projection and e-paper to display digital images, video, web pages, weather data, restaurant menus, or text. They can be found in public spaces, transportation systems, museums, stadiums, retail stores, hotels, restaurants and corporate buildings etc., to provide wayfinding, exhibitions, marketing and outdoor advertising. They are used as a network of electronic displays that are centrally managed and individually addressable for the display of text, animated or video messages for advertising, information, entertainment and merchandising to targeted audiences.

<span class="mw-page-title-main">Ashwin Navin</span> American businessman

Ashwin Navin is an American entrepreneur, who is the CEO and co-founder of Samba TV, a data and analytics service that measures television viewership using opt-in data from Internet-connected devices and set-top boxes. The company has been compared to more traditional TV measurement firms like Nielsen which rely on the people meter to gather viewership data.

Vizio Holding Corp. is an American publicly traded company that designs and sells televisions, sound bars, viewer data, and advertising. The company was founded in 2002 and is based in Irvine, California. In February 2024, it entered into an agreement to be acquired by Walmart, so Walmart can expand advertising sales in video content that streams for free on Vizio devices.

<span class="mw-page-title-main">Gracenote</span> American data company

Gracenote, Inc. is a company and service that provides music, video, and sports metadata and automatic content recognition (ACR) technologies to entertainment services and companies worldwide. Formerly CDDB, Gracenote maintains and licenses an Internet-accessible database containing information about the contents of audio compact discs and vinyl records. From 2008 to 2014, it was owned by Sony, later sold to Tribune Media, and has been owned since 2017 by Nielsen Holdings. In 2019, Nielsen Holdings announced plans to split into two separate publicly traded companies, Nielsen Global Connect and Nielsen Global Media. In October 2022, Nielsen Holdings completed the sale of Global Media, including the Gracenote subsidiary, to a private equity consortium.

Video fingerprinting or video hashing are a class of dimension reduction techniques in which a system identifies, extracts, and then summarizes characteristic

Xumo, LLC is an American internet television and consumer electronics company. It is a joint venture of Charter Communications and Comcast that operates the free ad-supported streaming television (FAST) and advertising video on demand (AVOD) service Xumo Play, and distributes Xumo Stream Box digital media players and Xumo TV smart TVs. The Xumo Play platform's service operations are based in the Greater Los Angeles suburb of Irvine, California. As of October 2020, Xumo Play has 24 million monthly active users.

<span class="mw-page-title-main">Shazam (music app)</span> Music identification application

Shazam is an application that can identify music based on a short sample played using the microphone on the device. It was created by the British company Shazam Entertainment, based in London, and has been owned by Apple since 2018. The software is available for Android, macOS, iOS, Wear OS, watchOS and as a Google Chrome extension.

Civolution is a provider of technology and services for identifying, managing, and monetizing audio and video media content. The company offers a portfolio of proprietary and patented digital watermarking and digital audio and video fingerprinting technology for media protection: forensic tracking of media assets in pre-release, digital cinema, pay TV and online; media intelligence: audience measurement, broadcast monitoring, internet and radio tracking; media interaction: automatic content recognition and triggering for second screen and connected television.

<span class="mw-page-title-main">Hybrid Broadcast Broadband TV</span> Industry standard for hybrid digital television

Hybrid Broadcast Broadband TV (HbbTV) is both an industry standard and promotional initiative for hybrid digital TV to harmonise the broadcast, Internet Protocol Television (IPTV), and broadband delivery of entertainment to the end consumer through connected TVs and set-top boxes. The HbbTV Association, comprising digital broadcasting and Internet industry companies, has established a standard for the delivery of broadcast TV and broadband TV to the home, through a single user interface, creating an open platform as an alternative to proprietary technologies. Products and services using the HbbTV standard can operate over different broadcasting technologies, such as satellite, cable, or terrestrial networks.

<span class="mw-page-title-main">Smart TV</span> TV set with integrated Internet features

A smart TV, also known as a connected TV (CTV), is a traditional television set with integrated Internet and interactive Web 2.0 features that allow users to stream music and videos, browse the internet, and view photos. Smart TVs are a technological convergence of computers, televisions, and digital media players. Besides the traditional functions of television sets provided through traditional broadcasting media, these devices can provide access to over-the-top media services such as streaming television and internet radio, along with home networking access.

Samba TV is a television technology company that offers real-time insights and audience analytics. It was founded in 2008 by early employees of BitTorrent, including Samba TV's current chief executive officer, Ashwin Navin. The company develops software for televisions, set-top boxes, smart phones and tablets to enable interactive television through personalization. Through its portfolio of applications and TV platform technologies, Samba TV is built directly into the TV or set-top box and will recognize onscreen content—live or time-shifted—and make relevant information available to users at their request.

<span class="mw-page-title-main">Toon Goggles</span> American on-demand entertainment service

Toon Goggles is an American on-demand entertainment service for children that provides animated cartoons, live-action shows, games and music worldwide via the web and mobile applications on smartphones, OTT devices, smart TVs and tablets, led by CEO and co-founder Stephen Hodge.

A second screen involves the use of a computing device to provide a different viewing experience for content on another device.

<span class="mw-page-title-main">Yahoo! Smart TV</span>

Yahoo! Smart TV was a Smart TV platform developed by Yahoo! based upon the Yahoo! Desktop Widgets (Konfabulator) platform. Yahoo! Connected TV announced on August 20, 2008, at the Intel Developer Forum in San Francisco as the Widget Channel, it integrated the Yahoo! Widgets Engine with a new television oriented user interface to enable Internet connected applications to run and display on a 10-foot user interface. The platform was slowly being abandoned by its manufacturers, and was eventually deprecated. New apps that were based on Konfabulator stopped being added effective March 30, 2018, but existing apps can still be updated and installed, and HTML5 based apps are not affected by this.

Search by sound is the retrieval of information based on audio input. There are a handful of applications, specifically for mobile devices that utilize search by sound. Shazam, Soundhound, Axwave, ACRCloud and others have seen considerable success by using a simple algorithm to match an acoustic fingerprint to a song in a library. These applications take a sample clip of a song, or a user-generated melody and check a music library/music database to see where the clip matches with the song. From there, song information will be queried and displayed to the user.

<span class="mw-page-title-main">Axwave</span>

Axwave, Inc. was an international software development company that developed a proprietary fingerprinting-based automatic content recognition (ACR) technology. Axwave was founded by Damián Scavo, former algorithmic trader and Loris D'Acunto, nuclear physicist. Axwave was headquartered in Menlo Park CA, and had offices in New York, Italy and Poland.

Inscape is a provider of ACR services to Smart TV OEMs. The company was founded in 2009 as TV Interactive Systems, later renamed Cognitive Media Networks Inc. On August 10, 2015, Vizio acquired Cognitive Media Networks and renamed it Inscape. In July 2016 Vizio announced Inscape will spin off and operate as a separate, privately owned company.

The Apple TV app is a line of media player software programs developed by Apple Inc. for viewing television shows and films delivered by Apple to consumer electronic devices. It can stream content from the iTunes Store, the Apple TV Channels a la carte video on demand service, and the Apple TV+ original content subscription service. On iPhone, iPad, iPod Touch, Vision Pro, and Apple TV devices it can also index and access content from linked apps of other video on demand services.

References

  1. 1 2 3 "ACR(Automatic Content Recognition)". Archived from the original on 28 February 2017. Retrieved 27 February 2017.
  2. 1 2 "Automated content recognition creating content aware ecosystems" (PDF). Civolution. Archived from the original (PDF) on 23 September 2015. Retrieved 24 June 2015.
  3. ↑ "Panako: a scalable acoustic fingerprinting system handling time-scale and pitch modification". Universiteit Gent. Retrieved 27 February 2017.
  4. ↑ Main, Sami. "Nielsen Is Bringing Real-Time Interactive Ads to Smart TVs to Keep Streaming Audiences Engaged". Adweek. Retrieved 2018-01-11.
  5. ↑ Brink, Kyle. "A Primer on Automated Content". Viggle. Archived from the original on 2015-06-24. Retrieved 22 June 2015.
  6. ↑ "Facebook Automatic Content Recognition". Starcom MediaVest Group. SMG. Archived from the original on 6 July 2015. Retrieved 6 July 2015.
  7. ↑ Brink, Kyle (14 April 2014). "SVP of Product Development". A Primer on Automated Content Recognition. Viggle. Retrieved 22 June 2015.
  8. ↑ Solana, Anna. "How these hidden video watermarks can help spot piracy, doctored images | ZDNet". ZDNet. Retrieved 2018-01-11.
  9. ↑ Baumgartner, Jeff (2013-09-24). "Flingo Rebrands as Samba TV". Multichannel News. Retrieved 2021-10-05.
  10. ↑ Swedlow, Tracy (July 7, 2011). "Interactive TV News Round-Up (II): Flingo, Hulu, ITU". Archived from the original on 2011-07-09.
  11. ↑ "LG partners with Cognitive Networks to make Smart TVs smarter and more interactive". engadget. Retrieved 23 August 2016.
  12. ↑ "ACRCloud Powers Song Recognition For Hottest New Social Network, Peach". Music Industry News Network. Archived from the original on 8 March 2016. Retrieved 3 March 2016.
  13. ↑ Victoria, Ho (16 February 2016). "Xiaomi will help you name that song you can't stop humming". Mashable. Retrieved 3 March 2016.
  14. ↑ "ACRCloud Powers The Launch Of Taiwan's First Music/Humming Recognition Service For Omusic". Music Industry News Network. Archived from the original on 8 March 2016. Retrieved 3 March 2016.
  15. ↑ Dunn, Thom (2021-11-18). "TV manufacturer Vizio makes more money selling data than TVs". Boing Boing. Retrieved 2021-11-22.
  16. ↑ Weiss, Tom (January 23, 2018). "Tom Weiss: Breaking the barriers to addressable advertising in Europe". Broadband TV News. Retrieved 30 August 2018.
  17. ↑ Wolf, Michael. "Three Ways Automatic Content Recognition Will Change TV". Forbes. Retrieved 20 June 2015.
  18. ↑ "Samsung, LG, and Vizio smart TVs are recording—and sharing data about—everything you watch Consumer Reports investigates the information brokers who want to turn your viewing habits into cash". Consumer Reports. Retrieved 27 February 2017.
  19. ↑ "How to Stop Smart TVs From Snooping on You". PCMAG. Retrieved 2021-11-22.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.