Content storage management

Last updated

Content storage management (CSM) is a technique for the evolution of traditional media archive technology used by media companies and content owners to store and protect valuable file-based media assets. CSM solutions focus on active management of content and media assets regardless of format, type and source, interfaces between proprietary content source/destination devices and any format and type of commodity IT centric storage technology. These digital media files (or assets) most often contain video but in rarer cases may be still pictures or sound. A CSM system may be directed manually but is more often directed by upper-level systems, which may include media asset management (MAM), automation, or traffic.

Contents

Typically, CSM systems are server-based software applications that reside between the media network, which connects the various broadcast or manipulation devices, and the storage network, which connects the nearline and archive storage tiers. The most basic function of CSM is the automated retrieval of high-resolution digital content (or essence) either from a data tape library (with the aid of a robot), or from a data server, and the delivery of that content either to a workstation, playout or editing device. CSM also performs this process in reverse – moving content back to storage. In a given media operation, CSM may be used to facilitate content manipulation and repurposing; systems interoperability through high and low bit rate content transcoding; and/or site-to-site content replication for disaster recovery.

CSM solutions comply with the well known Reference Model for an Open Archival Information System (OAIS), which is fundamental to long-term archive and content preservation in file-based environments and are characterized by a set of application-specific functionality, which can include:

History of CSM

The concept of CSM was first introduced in November 2006 by Brian Campanotti and Rino Petricola from Front Porch Digital Inc, one of the providers of CSM solutions. CSM solutions have their origins in archive management systems, but their functionality is considerably broader. Likewise, CSM systems resemble hierarchical storage management (HSM) systems in that both migrate among storage tiers, but they are distinct. For example, CSM software is media-specific, with internal decision-making models based on media-centric parameters, and with programmatic interfaces specific to control and management by broadcast business systems such as traffic, inventory, and automation. CSM groups related assets (typically video file and related multilingual audio files) into a single managed object for storage and retrieval. CSM does not use stub files, which are pointers to files, but instead handles actual media files directly. CSM migrates files by copying them between systems rather than transferring them from system to system. And finally, by definition, CSM does not stand alone but functions as an abstraction layer between disparate systems including various storage devices, broadcast editing and playout devices and even end consumer devices such as set top boxes and Web streaming sites.

The most significant differentiator between CSM and HSM solutions is the difference between migration between tiered storage fundamental to HSM and the active storage management fundamental to CSM solutions. Conceptually, HSM solutions are used to age content to less expensive tiers of storage based on static policies examining parameters such as least recently accessed, file size, specific directories/paths, etc. and treat each file as a separate and unique entity. Although CSM solutions also manage tiered storage (although not specifically limited to any type, amount or storage technology) but do not rely on static policies but rather can be living policies that can be assigned dynamically to content entering the system and govern replication policies, storage persistence, age-based migration as well as more advanced content aware processing steps such as transcoding, rewrapping, reformatting and subjective quality analysis. In some cases, the content stored within the CSM system is intentionally different than the content that entered the system initially. As an example, content entering a CSM solution in a legacy media format such as Pinnacle MPEG2 may be intentionally transcoded to MPEG2 IMX50 and wrapped in an industry standard format such as MXF to ensure longer term compatibility with other systems connected to the CSM solution. These media-centric workflows are not inherently supported by HSM solutions.

Based on the content centric focus of CSM solutions, they do not have legacy reliance on key HSM traits such as the use of stub files, which must remain on the original file system from where the file was migrated to cheaper tiers of storage. Although beneficial in some environments where HSM is being used simply as a "disk extender" for economic motivation, this feature is a significant limitation in active file-based content workflows typical in media, entertainment and preservationist applications where CSM solutions are key. Because of the inability for broadcast devices to maintain these HSM specific stub files on their internal (online) disk, the use of HSM systems in these advanced applications mandates a copy of the media content be made first from the online storage to some other tier of spinning disk prior to the HSM system taking ownership and migrating the content to other tiers of disk. As the HSM systems do not have the ability to directly interface to these various broadcast and media devices such as encoders, video servers, editing systems, etc. there is usually another software application responsible for this additional step such as a media asset management (MAM) system or other application specific utility.

CSM solutions extend the realm of content control and management directly from the internals of the content source and destination devices through any number and type of mass storage devices (disk, tape, optical, offline, etc.) all managed by configurable intelligent policies similar in concept to information lifecycle policies (ILM) but enhanced to become "content aware".

CSM solutions are not meant to age content into less expensive storage over time as its perceived value diminishes but rather actively play a part in a fairly symmetrical content lifecycle where content stored within the CSM system yesterday or five years ago is just as likely to be requested to be restored. All content stored within a CSM storage infrastructure is equally viable for restore operations at any point of time and often no reliable predictability is possible.

Augmenting the overall content storage capacities by allowing the addition of less expensive storage technologies is an advantage of CSM solutions but preservation is also a key driver. In terms of preservation, high-value content (historical, cultural, monetary, etc.) can be assigned differing ILM policies that can govern the number of copies or instances of the content maintained by the system and also geographical distribution to other CSM solutions via WAN connectivity providing distribution as well as disaster recovery functionality.

Example of CSM in Broadcast An example of the role of CSM within a broadcast setting is as follows: A movie is ingested into the content owner's workflow leveraging an encoding device and a control system such as automation or MAM. The control system instructs the CSM system to simultaneously analyze the content to ensure acceptable subjective quality, copy this original material into the archive library, make two copies or instances of it for protection and generate a low resolution Windows Media proxy version for Web access. An editor uses the MAM system to view the proxy generated by the CSM solution, and from his/her desktop workstation selects shots for use in a promo for the movie. The editor then sends his/her edit decision list (EDL) to the CSM system, which restores to the editing system only the desired broadcast-quality segments based on mark in and mark out timecode values defined within it. The CSM system then transcodes or rewraps the segments as necessary, so they can be used by the editor in the creation of the promo. Once the editor completes the promo, the editing system sends it via the CSM system (which can also repeat the replication and proxy generation process) to the on-air video servers for playout to air.

See also

Related Research Articles

The Advanced Authoring Format (AAF) is a file format for professional cross-platform data interchange, designed for the video post-production and authoring environment. It was created by the Advanced Media Workflow Association (AMWA), and is now being standardized through the Society of Motion Picture and Television Engineers (SMPTE).

Quantum Corporation is a data storage, management, and protection company that provides technology to store, manage, archive, and protect video and unstructured data throughout the data lifecycle. Their products are used by enterprises, media and entertainment companies, government agencies, big data companies, and life science organizations. Quantum is headquartered in San Jose, California and has offices around the world, supporting customers globally in addition to working with a network of distributors, VARs, DMRs, OEMs and other suppliers.

Transcoding is the direct digital-to-digital conversion of one encoding to another, such as for video data files, audio files, or character encoding. This is usually done in cases where a target device does not support the format or has limited storage capacity that mandates a reduced file size, or to convert incompatible or obsolete data to a better-supported or modern format.

Digital asset management (DAM) and the implementation of its use as a computer application is required in the collection of digital assets to ensure that the owner, and possibly their delegates, can perform operations on the data files.

<span class="mw-page-title-main">Video server</span> Device that is dedicated to delivering video

A video server is a computer-based device that is dedicated to delivering video. Video servers are used in a number of applications, and often have additional functions and capabilities that address the needs of particular applications. For example, video servers used in security, surveillance and inspection applications typically are designed to capture video from one or more cameras and deliver the video via a computer network. In video production and broadcast applications, a video server may have the ability to record and play recorded video, and to deliver many video streams simultaneously.

Enterprise content management (ECM) extends the concept of content management by adding a timeline for each content item and, possibly, enforcing processes for its creation, approval, and distribution. Systems using ECM generally provide a secure repository for managed items, analog or digital. They also include one methods for importing content to bring manage new items, and several presentation methods to make items available for use. Although ECM content may be protected by digital rights management (DRM), it is not required. ECM is distinguished from general content management by its cognizance of the processes and procedures of the enterprise for which it is created.

Lustre is a type of parallel distributed file system, generally used for large-scale cluster computing. The name Lustre is a portmanteau word derived from Linux and cluster. Lustre file system software is available under the GNU General Public License and provides high performance file systems for computer clusters ranging in size from small workgroup clusters to large-scale, multi-site systems. Since June 2005, Lustre has consistently been used by at least half of the top ten, and more than 60 of the top 100 fastest supercomputers in the world, including the world's No. 1 ranked TOP500 supercomputer in November 2022, Frontier, as well as previous top supercomputers such as Fugaku, Titan and Sequoia.

Hierarchical storage management (HSM), also known as Tiered storage, is a data storage and Data management technique that automatically moves data between high-cost and low-cost storage media. HSM systems exist because high-speed storage devices, such as solid state drive arrays, are more expensive than slower devices, such as hard disk drives, optical discs and magnetic tape drives. While it would be ideal to have all data available on high-speed devices all the time, this is prohibitively expensive for many organizations. Instead, HSM systems store the bulk of the enterprise's data on slower devices, and then copy data to faster disk drives when needed. The HSM system monitors the way data is used and makes best guesses as to which data can safely be moved to slower devices and which data should stay on the fast devices.

The Technology and Engineering Emmy Awards, or Technology and Engineering Emmys, are one of two sets of Emmy Awards that are presented for outstanding achievement in engineering development in the television industry. The Technology and Engineering Emmy Awards are presented by the National Academy of Television Arts and Sciences (NATAS), while the separate Primetime Engineering Emmy Awards are given by its sister organization the Academy of Television Arts & Sciences (ATAS).

IBM Storage Protect is a data protection platform that gives enterprises a single point of control and administration for backup and recovery. It is the flagship product in the IBM Spectrum Protect family.

In broadcasting, channel playout is the generation of the source signal of a radio or television channel produced by a broadcaster, coupled with the transmission of this signal for primary distribution or direct-to-audience distribution via any network. Such radio or television distribution networks include terrestrial broadcasting, cable networks, satellites, IPTV, OTT Video, point-to-point transport over managed networks or the public Internet, etc.

<span class="mw-page-title-main">TDF Group</span>

TDF is a French company which provides radio and television transmission services, services for telecommunications operators, and other multimedia services – digitization of content, encoding, storage, etc.

StorNext File System (SNFS), colloquially referred to as StorNext is a shared disk file system made by Quantum Corporation. StorNext enables multiple Windows, Linux and Apple workstations to access shared block storage over a Fibre Channel network. With the StorNext file system installed, these computers can read and write to the same storage volume at the same time enabling what is known as a "file-locking SAN." StorNext is used in environments where large files must be shared, and accessed simultaneously by users without network delays, or where a file must be available for access by multiple readers starting at different times. Common use cases include multiple video editor environments in feature film, television and general video post production.

<span class="mw-page-title-main">Ocarina Networks</span>

Ocarina Networks was a technology company selling a hardware/software solution designed to reduce data footprints with file-aware storage optimization. A subsidiary of Dell, their flagship product, the Ocarina Appliance/Reader, released in April 2008, uses patented data compression techniques incorporating such methods as record linkage and context-based lossless data compression. The product includes the hardware-appliance-based compressor, the Ocarina Optimizer and a real-time decompressor, the software-based Ocarina Reader.

FlipFactory from Telestream is a video transcoding and workflow automation application. It enables the transfer of media and metadata files between professional video systems, including catch servers, broadcast servers, edit systems, streaming and distribution servers, storage area networks and digital asset management systems.

<span class="mw-page-title-main">TACTIC (web framework)</span> Web-based, open source workflow platform and digital asset management system

TACTIC is a web-based, open source workflow platform and digital asset management system supported by Southpaw Technology in Toronto, ON. Designed to optimize busy production environments with high volumes of content traffic, TACTIC applies business or workflow logic to combined database and file system management. Using elements of digital asset management, production asset management and workflow management, TACTIC tracks the creation and development of digital assets through production pipelines. TACTIC is available under both commercial and open-source licenses, and also as a hosted cloud service through Amazon Web Services Marketplace.

<span class="mw-page-title-main">Active Archive Alliance</span> Trade association

The Active Archive Alliance is a trade association that promotes a method of tiered storage. This method provides users access to data across a virtual file system that migrates data between multiple storage systems and media types including solid-state drive/flash, hard disk drives, magnetic tape, optical disk, and cloud. The result of an active archive implementation is that data can be stored on the most appropriate media type for the given retention and restoration requirements of that data. This allows less time sensitive or infrequently accessed data to be stored on less expensive media and eliminates the need for an administrator to manually migrate data between storage systems. Additionally, since storage systems such as tape libraries have low power consumption, the operational expense of storing data in an active archive is significantly reduced.

An online video platform (OVP) enables users to upload, convert, store, and play back video content on the Internet, often via a private server structured, large-scale system that may generate revenue. Users will generally upload video content via the hosting service's website, mobile or desktop application, or other interfaces (API),and typically provides embed codes or links that allow others to view the video content.

Nirvana was virtual object storage software developed and maintained by General Atomics.

Channel in a box is the name given to an all-in-one playout device for broadcast television. Commonly based on a standard PC, it includes the ability to store content immediately required, add graphics, and deliver it to a transmission chain. Thus it provides an integrated, software playout platform.

References