Livewire Segmentation Technique

Last updated
Example of livewire segmentation on a baby's photo Livewire-Bia.jpg
Example of livewire segmentation on a baby's photo

Livewire, is a segmentation technique which allows a user to select regions of interest to be extracted quickly and accurately, using simple mouse clicks. [1] It is based on the lowest cost path algorithm, by Edsger W. Dijkstra. Firstly convolve the image with a Sobel filter to extract edges. Each pixel of the resulting image is a vertex of the graph and has edges going to the 4 pixels around it, as up, down, left, right. The edge costs are defined based on a cost function. In 1995, Eric N. Mortensen and William A. Barrett made some extension work on livewire segmentation tool, which is known as Intelligent Scissors. [2]

Contents

Livewire segmentation

The user sets the starting point clicking on an image's pixel, known as an anchor. Then, as he starts to move the mouse over other points, the smallest cost path is drawn from the anchor to the pixel where the mouse is over, changing itself if the user moves the mouse. If he wants to choose the path that is being displayed, he simply clicks the image again.

One can easily see in the right image, that the places where the user clicked to outline the desired region of interest are marked with a small square. It is also easy to see that the livewire has snapped on the image's borders.

Livewire algorithm

Convolve the image with a Sobel filter to extract edges. Using this filtered image create a graph using pixels as nodes with edges in four directions (up, down, left right). [1] Edges are weighted with features gathered from the Sobel filter making it less costly to stay on an edge. Several different cost methods are possible but the most important is the gradient magnitude [1]

Live-Wire 2-D DP graph search algorithm in pseudocode [2]

algorithm Livewire isinput:s                       {Start (or seed) pixel.}         l(q, r)                 {Local cost function for link between pixels q and r.}      data structures:         L                       {List of active pixels sorted by total cost (initially empty).}         N(q)                    {Neighborhood set of q (contains 8 neighbors of pixel).}          e(q)                    {Boolean function indicating if q has been expanded/processed.}         g(q)                    {Total cost function from seed point to q.}     output:p                       {Pointers from each pixel indicating the minimum cost path.}      g(s) ← 0; L ← s;        {Initialize active list with zero cost seed pixel.}     while L≠∅ do begin      {While still points to expand.}         q ← min(L);             {Remove minimum cost pixel q from active list.}         e(q) ← TRUE;             {Mark q as expanded (i.e., processed).}         for each r∈N(q) such that not e(r) do begin             gtmp ←g(q) + l(q, r);        {Compute total cost to neighbor.}             if r∈Land gtmp < g(r) then  {Remove higher cost neighbor's from list.}                 r ← L;                                   if r∉L then begin            {If neighbor not on list, }                 g(r) ← gtmp;             {assign neighbor's total cost, }                 p(r) ← q;                {set (or reset) back pointer, }                  L ← r;                   {and place on (or return to)  active list.}              endendend

Extension to 3D

In 2010, Leo Grady extended the Livewire algorithm to 3D. [3] This extension treated the 2D Livewire algorithm as enabling a user to specify a 0-dimensional boundary (two points) and finding the minimal 1-dimensional coboundary (curve) connecting those points, where the minimum is defined in terms of image properties. In order to extend the algorithm to 3D, the user is instead asked to specify one or more 1-dimensional boundaries (closed curves) and the algorithm finds the minimal 2-dimensional coboundary (surface) bounded by the 1-dimensional curves, where the minimum surface is defined in terms of image properties. This 3D extension of Livewire leans heavily on concepts of discrete exterior calculus to reinterpret the 2D Livewire algorithm from the standpoint of boundary/coboundary operators and then apply these concepts in 3D. An efficient algorithm for computing the 3D minimal surface is also provided in the Grady paper.

See also

Related Research Articles

<span class="mw-page-title-main">Minimum spanning tree</span> Least-weight tree connecting graph vertices

A minimum spanning tree (MST) or minimum weight spanning tree is a subset of the edges of a connected, edge-weighted undirected graph that connects all the vertices together, without any cycles and with the minimum possible total edge weight. That is, it is a spanning tree whose sum of edge weights is as small as possible. More generally, any edge-weighted undirected graph has a minimum spanning forest, which is a union of the minimum spanning trees for its connected components.

<span class="mw-page-title-main">Dijkstra's algorithm</span> Graph search algorithm

Dijkstra's algorithm is an algorithm for finding the shortest paths between nodes in a graph, which may represent, for example, road networks. It was conceived by computer scientist Edsger W. Dijkstra in 1956 and published three years later.

In computer science and optimization theory, the max-flow min-cut theorem states that in a flow network, the maximum amount of flow passing from the source to the sink is equal to the total weight of the edges in a minimum cut, i.e., the smallest total weight of the edges which if removed would disconnect the source from the sink.

Digital image processing is the use of a digital computer to process digital images through an algorithm. As a subcategory or field of digital signal processing, digital image processing has many advantages over analog image processing. It allows a much wider range of algorithms to be applied to the input data and can avoid problems such as the build-up of noise and distortion during processing. Since images are defined over two dimensions digital image processing may be modeled in the form of multidimensional systems. The generation and development of digital image processing are mainly affected by three factors: first, the development of computers; second, the development of mathematics ; third, the demand for a wide range of applications in environment, agriculture, military, industry and medical science has increased.

<span class="mw-page-title-main">Maximum flow problem</span> Computational problem in graph theory

In optimization theory, maximum flow problems involve finding a feasible flow through a flow network that obtains the maximum possible flow rate.

<span class="mw-page-title-main">Sobel operator</span> Image edge detection algorithm

The Sobel operator, sometimes called the Sobel–Feldman operator or Sobel filter, is used in image processing and computer vision, particularly within edge detection algorithms where it creates an image emphasising edges. It is named after Irwin Sobel and Gary Feldman, colleagues at the Stanford Artificial Intelligence Laboratory (SAIL). Sobel and Feldman presented the idea of an "Isotropic 3 × 3 Image Gradient Operator" at a talk at SAIL in 1968. Technically, it is a discrete differentiation operator, computing an approximation of the gradient of the image intensity function. At each point in the image, the result of the Sobel–Feldman operator is either the corresponding gradient vector or the norm of this vector. The Sobel–Feldman operator is based on convolving the image with a small, separable, and integer-valued filter in the horizontal and vertical directions and is therefore relatively inexpensive in terms of computations. On the other hand, the gradient approximation that it produces is relatively crude, in particular for high-frequency variations in the image.

<span class="mw-page-title-main">Canny edge detector</span> Image edge detection algorithm

The Canny edge detector is an edge detection operator that uses a multi-stage algorithm to detect a wide range of edges in images. It was developed by John F. Canny in 1986. Canny also produced a computational theory of edge detection explaining why the technique works.

<span class="mw-page-title-main">Image segmentation</span> Partitioning a digital image into segments

In digital image processing and computer vision, image segmentation is the process of partitioning a digital image into multiple image segments, also known as image regions or image objects. The goal of segmentation is to simplify and/or change the representation of an image into something that is more meaningful and easier to analyze. Image segmentation is typically used to locate objects and boundaries in images. More precisely, image segmentation is the process of assigning a label to every pixel in an image such that pixels with the same label share certain characteristics.

The scale-invariant feature transform (SIFT) is a computer vision algorithm to detect, describe, and match local features in images, invented by David Lowe in 1999. Applications include object recognition, robotic mapping and navigation, image stitching, 3D modeling, gesture recognition, video tracking, individual identification of wildlife and match moving.

<span class="mw-page-title-main">Image gradient</span> Directional change in the intensity or color in an image

An image gradient is a directional change in the intensity or color in an image. The gradient of the image is one of the fundamental building blocks in image processing. For example, the Canny edge detector uses image gradient for edge detection. In graphics software for digital image editing, the term gradient or color gradient is also used for a gradual blend of color which can be considered as an even gradation from low to high values, as used from white to black in the images to the right. Another name for this is color progression.

Connected-component labeling (CCL), connected-component analysis (CCA), blob extraction, region labeling, blob discovery, or region extraction is an algorithmic application of graph theory, where subsets of connected components are uniquely labeled based on a given heuristic. Connected-component labeling is not to be confused with segmentation.

As applied in the field of computer vision, graph cut optimization can be employed to efficiently solve a wide variety of low-level computer vision problems, such as image smoothing, the stereo correspondence problem, image segmentation, object co-segmentation, and many other computer vision problems that can be formulated in terms of energy minimization. Many of these energy minimization problems can be approximated by solving a maximum flow problem in a graph. Under most formulations of such problems in computer vision, the minimum energy solution corresponds to the maximum a posteriori estimate of a solution. Although many computer vision algorithms involve cutting a graph, the term "graph cuts" is applied specifically to those models which employ a max-flow/min-cut optimization.

<span class="mw-page-title-main">Watershed (image processing)</span>

In the study of image processing, a watershed is a transformation defined on a grayscale image. The name refers metaphorically to a geological watershed, or drainage divide, which separates adjacent drainage basins. The watershed transformation treats the image it operates upon like a topographic map, with the brightness of each point representing its height, and finds the lines that run along the tops of ridges.

<span class="mw-page-title-main">Seam carving</span>

Seam carving is an algorithm for content-aware image resizing, developed by Shai Avidan, of Mitsubishi Electric Research Laboratories (MERL), and Ariel Shamir, of the Interdisciplinary Center and MERL. It functions by establishing a number of seams in an image and automatically removes seams to reduce image size or inserts seams to extend it. Seam carving also allows manually defining areas in which pixels may not be modified, and features the ability to remove whole objects from photographs.

<span class="mw-page-title-main">Spectral clustering</span> Clustering methods

In multivariate statistics, spectral clustering techniques make use of the spectrum (eigenvalues) of the similarity matrix of the data to perform dimensionality reduction before clustering in fewer dimensions. The similarity matrix is provided as an input and consists of a quantitative assessment of the relative similarity of each pair of points in the dataset.

Image segmentation strives to partition a digital image into regions of pixels with similar properties, e.g. homogeneity. The higher-level region representation simplifies image analysis tasks such as counting objects or detecting changes, because region attributes can be compared more readily than raw pixels.

The random walker algorithm is an algorithm for image segmentation. In the first description of the algorithm, a user interactively labels a small number of pixels with known labels, e.g., "object" and "background". The unlabeled pixels are each imagined to release a random walker, and the probability is computed that each pixel's random walker first arrives at a seed bearing each label, i.e., if a user places K seeds, each with a different label, then it is necessary to compute, for each pixel, the probability that a random walker leaving the pixel will first arrive at each seed. These probabilities may be determined analytically by solving a system of linear equations. After computing these probabilities for each pixel, the pixel is assigned to the label for which it is most likely to send a random walker. The image is modeled as a graph, in which each pixel corresponds to a node which is connected to neighboring pixels by edges, and the edges are weighted to reflect the similarity between the pixels. Therefore, the random walk occurs on the weighted graph.

In applied mathematics, lambda-connectedness deals with partial connectivity for a discrete space.

In image processing, line detection is an algorithm that takes a collection of n edge points and finds all the lines on which these edge points lie. The most popular line detectors are the Hough transform and convolution-based techniques.

Semi-global matching (SGM) is a computer vision algorithm for the estimation of a dense disparity map from a rectified stereo image pair, introduced in 2005 by Heiko Hirschmüller while working at the German Aerospace Center. Given its predictable run time, its favourable trade-off between quality of the results and computing time, and its suitability for fast parallel implementation in ASIC or FPGA, it has encountered wide adoption in real-time stereo vision applications such as robotics and advanced driver assistance systems.

References

  1. 1 2 3 BAGGIO, Daniel L´elis. GPGPU Based Image Segmentation Livewire Algorithm Implementation. 2007. 108f. Thesis of Master in Science – Technological Institute of Aeronautics, S˜ao Jos´e dos Campos. http://gpuwire.googlecode.com/files/Master%20Thesis%20-%20Updated%20February%2015th.pdf Archived 2010-12-17 at the Wayback Machine
  2. 1 2 MORTENSEN, E. N.; BARRETT, W. A. Intelligent scissors for image composition. In: SIGGRAPH ’95: Proceedings of the 22nd annual conference on Computer graphics and interactive techniques. New York, NY, USA: ACM Press, 1995. p. 191–198. ISBN   0-89791-701-4.
  3. Leo Grady, “Minimal Surfaces Extend Shortest Path Segmentation Methods to 3D”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 32, No. 2, pp. 321-334, Feb. 2010