 | This article has multiple issues. Please help improve it or discuss these issues on the talk page . (Learn how and when to remove these messages)
|
Digital compositing is the process of digitally assembling multiple images to make a final image, typically for print, motion pictures or screen display. It is the digital analogue of optical film compositing. It's part of VFX processing.
The basic operation used in digital compositing is known as alpha blending, where an opacity value, 'α', is used to control the proportions of two input pixel values that end up a single output pixel.
As a simple example, suppose two images of the same size are available and they are to be composited. The input images are referred to as the foreground image and the background image. Each image consists of the same number of pixels. Compositing is performed by mathematically combining information from the corresponding pixels from the two input images and recording the result in a third image, which is called the composited image.
Consider three pixels;
and
Then, considering all three colour channels, and assuming that the colour channels are expressed in a γ=1 colour space (that is to say, the measured values are proportional to light intensity), we have:
Note that if the operations are performed in a colour space where γ is not equal to 1 then the operation will lead to non-linear effects which can potentially be seen as aliasing artifacts (or "jaggies") along sharp edges in the matte. More generally, nonlinear compositing can have effects such as "halos" around composited objects, because the influence of the alpha channel is non-linear. It is possible for a compositing artist to compensate for the effects of compositing in non-linear space.
Performing alpha blending is an expensive operation if performed on an entire image or 3D scene. If this operation has to be done in real time video games, there is an easy trick to boost performance.
By simply rewriting the mathematical expression, one can save 50% of the multiplications required.
When many partially transparent layers need to be composited together, it is worthwhile to consider the algebraic properties of compositing operators used. Specifically, the associativity and commutativity determine when repeated calculation can or cannot be avoided.
Consider the case when we have four layers to blend to produce the final image: F=A*(B*(C*D)) where A, B, C, D are partially transparent image layers and "*" denotes a compositing operator (with the left layer on top of the right layer). If only layer C changes, we should find a way to avoid re-blending all of the layers when computing F. Without any special considerations, four full-image blends would need to occur. For compositing operators that are commutative, such as additive blending, it is safe to re-order the blending operations. In this case, we might compute T=A*(B*D) only once and simply blend T*C to produce F, a single operation. Unfortunately, most operators are not commutative. However, many are associative, suggesting it is safe to re-group operations to F=(A*B)*(C*D), i.e. without changing their order. In this case, we may compute S:=A*B once and save this result. To form F with an associative operator, we need only do two additional compositing operations to integrate the new layer S, by computing F:=S*(C*D). This expression indicates compositing C with all of the layers below it in one step, and then blending all of the layers on top of it with the previous result to produce the final image in the second step.
If all layers of an image change regularly but many layers still need to be composited (such as in distributed rendering), the commutativity of a compositing operator can still be exploited to speed up computation through parallelism even when there is no gain from pre-computation. Again, consider the image F=A*(B*(C*D)). Each compositing operation in this expression depends on the next, leading to serial computation. However, associativity can allow us to rewrite F=(A*B)*(C*D) where there are clearly two operations that do not depend on each other that may be executed in parallel. In general, we can build a tree of pair-wise compositing operations with a height that is logarithmic in the number of layers.
The most historically significant nonlinear compositing system was the Cineon, which operated in a logarithmic color space, which more closely mimics the natural light response of film emulsions (the Cineon System, made by Kodak, is no longer in production). Due to the limitations of processing speed and memory, compositing artists did not usually have the luxury of having the system make intermediate conversions to linear space for the compositing steps. Over time, the limitations have become much less significant, and now most compositing is done in a linear color space, even in cases where the source imagery is in a logarithmic color space.
Compositing often also includes scaling, retouching and colour correction of images.
There are two radically different digital compositing workflows: node-based compositing and layer-based compositing.
Node-based compositing represents an entire composite as a directed acyclic graph, linking media objects and effects in a procedural map, intuitively laying out the progression from source input to final output, and is in fact the way all compositing applications internally handle composites. This type of compositing interface allows great flexibility, including the ability to modify the parameters of an earlier image processing step "in context" (while viewing the final composite). Node-based compositing packages often handle keyframing and time effects poorly, as their workflow does not stem directly from a timeline, as do layer-based compositing packages. Software which incorporates a node based interface include Natron, Shake, Blender, Blackmagic Fusion, and Nuke. Nodes are a great way to organize the effects in a complex shot while maintaining a grip on the smaller details.
Layer-based compositing represents each media object in a composite as a separate layer within a timeline, each with its own time bounds, effects, and keyframes. All the layers are stacked, one above the next, in any desired order; and the bottom layer is usually rendered as a base in the resultant image, with each higher layer being progressively rendered on top of the previously composited of layers, moving upward until all layers have been rendered into the final composite. Layer-based compositing is very well suited for rapid 2D and limited 3D effects, such as in motion graphics, but becomes awkward for more complex composites entailing numerous layers. A partial solution to this is some programs' ability to view the composite-order of elements (such as images, effects, or other attributes) with a visual diagram called a flowchart to nest compositions, or "comps," directly into other compositions, thereby adding complexity to the render-order by first compositing layers in the beginning composition, then combining that resultant image with the layered images from the proceeding composition, and so on.
In computer graphics, alpha compositing or alpha blending is the process of combining one image with a background to create the appearance of partial or full transparency. It is often useful to render picture elements (pixels) in separate passes or layers and then combine the resulting 2D images into a single, final image called the composite. Compositing is used extensively in film when combining computer-rendered image elements with live footage. Alpha blending is also used in 2D computer graphics to put rasterized foreground elements over a background.
2D computer graphics is the computer-based generation of digital images—mostly from two-dimensional models and by techniques specific to them. It may refer to the branch of computer science that comprises such techniques or to the models themselves.
Chroma key compositing, or chroma keying, is a visual-effects and post-production technique for compositing (layering) two or more images or video streams together based on colour hues. The technique has been used in many fields to remove a background from the subject of a photo or video – particularly the newscasting, motion picture, and video game industries. A colour range in the foreground footage is made transparent, allowing separately filmed background footage or a static image to be inserted into the scene. The chroma keying technique is commonly used in video production and post-production. This technique is also referred to as colour keying, colour-separation overlay, or by various terms for specific colour-related variants such as green screen or blue screen; chroma keying can be done with backgrounds of any colour that are uniform and distinct, but green and blue backgrounds are more commonly used because they differ most distinctly in hue from any human skin colour. No part of the subject being filmed or photographed may duplicate the colour used as the backing, or the part may be erroneously identified as part of the backing.
Bit blit is a data operation commonly used in computer graphics in which several bitmaps are combined into one using a boolean function.
Texture mapping is a method for mapping a texture on a computer-generated graphic. "Texture" in this context can be high frequency detail, surface texture, or color.
In mathematics, a differential operator is an operator defined as a function of the differentiation operator. It is helpful, as a matter of notation first, to consider differentiation as an abstract operation that accepts a function and returns another function.
In image processing and photography, a color histogram is a representation of the distribution of colors in an image. For digital images, a color histogram represents the number of pixels that have colors in each of a fixed list of color ranges, that span the image's color space, the set of all possible colors.
Transparency in computer graphics is possible in a number of file formats. The term "transparency" is used in various ways by different people, but at its simplest there is "full transparency" i.e. something that is completely invisible. Only part of a graphic should be fully transparent, or there would be nothing to see. More complex is "partial transparency" or "translucency" where the effect is achieved that a graphic is partially transparent in the same way as colored glass. Since ultimately a printed page or computer or television screen can only be one color at a point, partial transparency is always simulated at some level by mixing colors. There are many different ways to mix colors, so in some cases transparency is ambiguous.
A quadtree is a tree data structure in which each internal node has exactly four children. Quadtrees are the two-dimensional analog of octrees and are most often used to partition a two-dimensional space by recursively subdividing it into four quadrants or regions. The data associated with a leaf cell varies by application, but the leaf cell represents a "unit of interesting spatial information".
In quantum information theory, a quantum channel is a communication channel which can transmit quantum information, as well as classical information. An example of quantum information is the general dynamics of a qubit. An example of classical information is a text document transmitted over the Internet.
The Cineon System was one of the first computer based digital film systems, created by Kodak in the early 1990s. It was an integrated suite of components consisting a Motion picture film scanner, a film recorder and workstation hardware with software for compositing, visual effects, image restoration and color management.
In mathematics, the derivative is a fundamental construction of differential calculus and admits many possible generalizations within the fields of mathematical analysis, combinatorics, algebra, geometry, etc.
Mattes are used in photography and special effects filmmaking to combine two or more image elements into a single, final image. Usually, mattes are used to combine a foreground image with a background image. In this case, the matte is the background painting. In film and stage, mattes can be physically huge sections of painted canvas, portraying large scenic expanses of landscapes.
Connected-component labeling (CCL), connected-component analysis (CCA), blob extraction, region labeling, blob discovery, or region extraction is an algorithmic application of graph theory, where subsets of connected components are uniquely labeled based on a given heuristic. Connected-component labeling is not to be confused with segmentation.
Blend modes in digital image editing and computer graphics are used to determine how two layers are blended with each other. The default blend mode in most applications is simply to obscure the lower layer by covering it with whatever is present in the top layer ; because each pixel has numerical values, there also are many other ways to blend two layers.
Order-independent transparency (OIT) is a class of techniques in rasterisational computer graphics for rendering transparency in a 3D scene, which do not require rendering geometry in sorted order for alpha compositing.
The random walker algorithm is an algorithm for image segmentation. In the first description of the algorithm, a user interactively labels a small number of pixels with known labels, e.g., "object" and "background". The unlabeled pixels are each imagined to release a random walker, and the probability is computed that each pixel's random walker first arrives at a seed bearing each label, i.e., if a user places K seeds, each with a different label, then it is necessary to compute, for each pixel, the probability that a random walker leaving the pixel will first arrive at each seed. These probabilities may be determined analytically by solving a system of linear equations. After computing these probabilities for each pixel, the pixel is assigned to the label for which it is most likely to send a random walker. The image is modeled as a graph, in which each pixel corresponds to a node which is connected to neighboring pixels by edges, and the edges are weighted to reflect the similarity between the pixels. Therefore, the random walk occurs on the weighted graph.
Image editing encompasses the processes of altering images, whether they are digital photographs, traditional photo-chemical photographs, or illustrations. Traditional analog image editing is known as photo retouching, using tools such as an airbrush to modify photographs or edit illustrations with any traditional art medium. Graphic software programs, which can be broadly grouped into vector graphics editors, raster graphics editors, and 3D modelers, are the primary tools with which a user may manipulate, enhance, and transform images. Many image editing programs are also used to render or create computer art from scratch. The term "image editing" usually refers only to the editing of 2D images, not 3D ones.
The Teknomo–Fernandez algorithm , is an efficient algorithm for generating the background image of a given video sequence.
Video matting is a technique for separating the video into two or more layers, usually foreground and background, and generating alpha mattes which determine blending of the layers. The technique is very popular in video editing because it allows to substitute the background, or process the layers individually.
