Cloop

Last updated
cloop
Original author(s) Rusty Russell
Developer(s) Klaus Knopper
Stable release
2.637-1 / 12 January 2011;13 years ago (2011-01-12)
Size 338 KB
Type Loadable kernel module
License GNU GPL v2 [1]
Website knoppix.net/wiki/Cloop

The compressed loop device (cloop) is a module for the Linux kernel. It adds support for transparently decompressed, read-only block devices. It is not a compressed file system: cloop is mostly used as a convenient way to compress conventional file systems onto Live CDs.

Contents

Cloop was originally written for the Levanta Bootable Business Card by Rusty Russell, but is now maintained by Klaus Knopper, the author of Knoppix.

A compression ratio of about 2.5:1 is common for software. The Knoppix cloop image, for example, is 700 MB compressed and around 1.8 GB uncompressed.

Design

cloop images contain:

The data blocks are compressed separately; this makes it possible to seek to individual blocks without having to decompress the entire image from the start, but at the cost of slightly reducing the compression ratio. Live CD images typically use a block size of 256 KB as a compromise between decompression speed and space-efficiency.

Apple uses a similar file format in the compressed variant of its DMG disk images.

Limitations

The design of the cloop driver requires that compressed blocks be read whole from disk. This makes cloop access inherently slower when there are many scattered reads, which can happen if the system is low on memory or when a large program with many shared libraries is starting. A big issue is the seek time for CD-ROM drives (~80 ms), which exceeds that of hard disks (~10 ms) by a large factor. On the other hand, because files are packed together, reading a compressed block may thus bring in more than one file into the cache. The effects of tail packing are known to improve seek times (cf. reiserfs, btrfs), especially for small files. Some performance tests related to cloop have been conducted. [2]

See also

Related Research Articles

zlib DEFLATE codec library

zlib is a software library used for data compression as well as a data format. zlib was written by Jean-loup Gailly and Mark Adler and is an abstraction of the DEFLATE compression algorithm used in their gzip file compression program. zlib is also a crucial component of many software platforms, including Linux, macOS, and iOS. It has also been used in gaming consoles such as the PlayStation 4, PlayStation 3, Wii U, Wii, Xbox One and Xbox 360.

ext2, or second extended file system, is a file system for the Linux kernel. It was initially designed by French software developer Rémy Card as a replacement for the extended file system (ext). Having been designed according to the same principles as the Berkeley Fast File System from BSD, it was the first commercial-grade filesystem for Linux.

In computing, Deflate is a lossless data compression file format that uses a combination of LZ77 and Huffman coding. It was designed by Phil Katz, for version 2 of his PKZIP archiving tool. Deflate was later specified in RFC 1951 (1996).

<span class="mw-page-title-main">Knoppix</span> Linux distribution

Knoppix, stylized KNOPPIX, is an operating system based on Debian designed to be run directly from a CD / DVD or a USB flash drive. It was first released in 2000 by German Linux consultant Klaus Knopper, and was one of the first popular live distributions. Knoppix is loaded from the removable medium and decompressed into a RAM drive. The decompression is transparent and on-the-fly.

The Lempel–Ziv–Markov chain algorithm (LZMA) is an algorithm used to perform lossless data compression. It has been under development since either 1996 or 1998 by Igor Pavlov and was first used in the 7z format of the 7-Zip archiver. This algorithm uses a dictionary compression scheme somewhat similar to the LZ77 algorithm published by Abraham Lempel and Jacob Ziv in 1977 and features a high compression ratio and a variable compression-dictionary size, while still maintaining decompression speed similar to other commonly used compression algorithms.

vmlinux Executable file containing the Linux kernel

vmlinux is a statically linked executable file that contains the Linux kernel in one of the object file formats supported by Linux, which includes Executable and Linkable Format (ELF) and Common Object File Format (COFF). The vmlinux file might be required for kernel debugging, symbol table generation or other operations, but must be made bootable before being used as an operating system kernel by adding a multiboot header, bootsector and setup routines.

Lempel–Ziv–Oberhumer (LZO) is a lossless data compression algorithm that is focused on decompression speed.

<span class="mw-page-title-main">UPX</span>

UPX is a free and open source executable packer supporting a number of file formats from different operating systems.

Executable compression is any means of compressing an executable file and combining the compressed data with decompression code into a single executable. When this compressed executable is executed, the decompression code recreates the original code from the compressed code before executing it. In most cases this happens transparently so the compressed executable can be used in exactly the same way as the original. Executable compressors are often referred to as "runtime packers", "software packers", "software protectors".

In Linux systems, initrd is a scheme for loading a temporary root file system into memory, to be used as part of the Linux startup process. initrd and initramfs refer to two different methods of achieving this. Both are commonly used to make preparations before the real root file system can be mounted.

DriveSpace is a disk compression utility supplied with MS-DOS starting from version 6.0 in 1993 and ending in 2000 with the release of Windows Me. The purpose of DriveSpace is to increase the amount of data the user could store on disks by transparently compressing and decompressing data on-the-fly. It is primarily intended for use with hard drives, but use for floppy disks is also supported. This feature was removed in Windows XP and later.

In Unix-like operating systems, a loop device, vnd, or lofi is a pseudo-device that makes a computer file accessible as a block device.

Squashfs is a compressed read-only file system for Linux. Squashfs compresses files, inodes and directories, and supports block sizes from 4 KiB up to 1 MiB for greater compression. Several compression algorithms are supported. Squashfs is also the name of free software, licensed under the GPL, for accessing Squashfs filesystems.

The following tables compare general and technical information for a number of file systems.

The Calgary corpus is a collection of text and binary data files, commonly used for comparing data compression algorithms. It was created by Ian Witten, Tim Bell and John Cleary from the University of Calgary in 1987 and was commonly used in the 1990s. In 1997 it was replaced by the Canterbury corpus, based on concerns about how representative the Calgary corpus was, but the Calgary corpus still exists for comparison and is still useful for its originally intended purpose.

In computing, a bitmap is a mapping from some domain to bits. It is also called a bit array or bitmap index.

AXFS is a compressed read-only file system for Linux, initially developed at Intel, and now maintained at Numonyx. It was designed to use execute in place (XIP) alongside compression aiming to reduce boot and program load times, while retaining a small memory footprint for embedded devices. This is achieved by mixing compressed and uncompressed pages in the same executable file. AXFS is free software licensed under the GPL.

<span class="mw-page-title-main">Apple Disk Image</span> File format developed by Apple and used by macOS

AppleDisk Image is a disk image format commonly used by the macOS operating system. When opened, an Apple Disk Image is mounted as a volume within the Finder.

Zstandard is a lossless data compression algorithm developed by Yann Collet at Facebook. Zstd is the corresponding reference implementation in C, released as open-source software on 31 August 2016.

EROFS is a lightweight read-only file system initially developed by Huawei for the Linux kernel and now maintained by an open-source community from all over the world.

References

  1. License stated in Readme in the tarball Archived 2012-03-07 at the Wayback Machine .
  2. Cloop performance testing on a ThinkPad T61 [ dead link ] 31-Dec-2011