LP-type problem

Last updated

In the study of algorithms, an LP-type problem (also called a generalized linear program) is an optimization problem that shares certain properties with low-dimensional linear programs and that may be solved by similar algorithms. LP-type problems include many important optimization problems that are not themselves linear programs, such as the problem of finding the smallest circle containing a given set of planar points. They may be solved by a combination of randomized algorithms in an amount of time that is linear in the number of elements defining the problem, and subexponential in the dimension of the problem.

Contents

Definition

LP-type problems were defined by Sharir & Welzl (1992) as problems in which one is given as input a finite set S of elements, and a function f that maps subsets of S to values from a totally ordered set. The function is required to satisfy two key properties:

A basis of an LP-type problem is a set BS with the property that every proper subset of B has a smaller value of f than B itself, and the dimension (or combinatorial dimension) of an LP-type problem is defined to be the maximum cardinality of a basis.

It is assumed that an optimization algorithm may evaluate the function f only on sets that are themselves bases or that are formed by adding a single element to a basis. Alternatively, the algorithm may be restricted to two primitive operations: a violation test that determines, for a basis B and an element x whether f(B) = f(B ∪ {x}), and a basis computation that (with the same inputs) finds a basis of B ∪ {x}. The task for the algorithm to perform is to evaluate f(S) by only using these restricted evaluations or primitives.

Examples and applications

Lp Balls

A linear program may be defined by a system of d non-negative real variables, subject to n linear inequality constraints, together with a non-negative linear objective function to be minimized. This may be placed into the framework of LP-type problems by letting S be the set of constraints, and defining f(A) (for a subset A of the constraints) to be the minimum objective function value of the smaller linear program defined by A. With suitable general position assumptions (in order to prevent multiple solution points having the same optimal objective function value), this satisfies the monotonicity and locality requirements of an LP-type problem, and has combinatorial dimension equal to the number d of variables. [1] Similarly, an integer program (consisting of a collection of linear constraints and a linear objective function, as in a linear program, but with the additional restriction that the variables must take on only integer values) satisfies both the monotonicity and locality properties of an LP-type problem, with the same general position assumptions as for linear programs. Theorems of Bell (1977) and Scarf (1977) show that, for an integer program with d variables, the combinatorial dimension is at most 2d. [1]

Many natural optimization problems in computational geometry are LP-type:

Smallest circle problem Smallest circle problem.svg
Smallest circle problem

LP-type problems have also been used to determine the optimal outcomes of certain games in algorithmic game theory, [11] improve vertex placement in finite element method meshes, [12] solve facility location problems, [13] analyze the time complexity of certain exponential-time search algorithms, [14] and reconstruct the three-dimensional positions of objects from their two-dimensional images. [15]

Algorithms

Seidel

Seidel (1991) gave an algorithm for low-dimensional linear programming that may be adapted to the LP-type problem framework. Seidel's algorithm takes as input the set S and a separate set X (initially empty) of elements known to belong to the optimal basis. It then considers the remaining elements one-by-one in a random order, performing violation tests for each one and, depending on the result, performing a recursive call to the same algorithm with a larger set of known basis elements. It may be expressed with the following pseudocode:

function seidel(S, f, X) isR := empty set     B := Xforx in a random permutation of S:         iff(B) ≠ f(B ∪ {x}):             B := seidel(R, f, X ∪ {x})         R := R ∪ {x}     returnB

In a problem with combinatorial dimension d, the violation test in the ith iteration of the algorithm fails only when x is one of the d|X| remaining basis elements, which happens with probability at most (d|X|)/i. Based on this calculation, it can be shown that overall the expected number of violation tests performed by the algorithm is O(d! n), linear in n but worse than exponential in d.

Clarkson

Clarkson (1995) defines two algorithms, a recursive algorithm and an iterative algorithm, for linear programming based on random sampling techniques, and suggests a combination of the two that calls the iterative algorithm from the recursive algorithm. The recursive algorithm repeatedly chooses random samples whose size is approximately the square root of the input size, solves the sampled problem recursively, and then uses violation tests to find a subset of the remaining elements that must include at least one basis element:

function recursive(S, f) isX := empty set     repeatR := a random subset of S with size d√n         B := basis for RX, computed recursively         V := {x | f(B) ≠ f(B ∪ {x})}         X := XVuntilV is empty     returnB

In each iteration, the expected size of V is O(n), [16] and whenever V is nonempty it includes at least one new element of the eventual basis of S. Therefore, the algorithm performs at most d iterations, each of which performs n violation tests and makes a single recursive call to a subproblem of size O(dn).

Clarkson's iterative algorithm assigns weights to each element of S, initially all of them equal. It then chooses a set R of 9d2 elements from S at random, and computes the sets B and V as in the previous algorithm. If the total weight of V is at most 2/(9d 1) times the total weight of S (as happens with constant probability) then the algorithm doubles the weights of every element of V, and as before it repeats this process until V becomes empty. In each iteration, the weight of the optimal basis can be shown to increase at a greater rate than the total weight of S, from which it follows that the algorithm must terminate within O(log n) iterations.

By using the recursive algorithm to solve a given problem, switching to the iterative algorithm for its recursive calls, and then switching again to Seidel's algorithm for the calls made by the iterative algorithm, it is possible solve a given LP-type problem using O(dn + d! dO(1) log n) violation tests.

When applied to a linear program, this algorithm can be interpreted as being a dual simplex method. [17] With certain additional computational primitives beyond the violation test and basis computation primitives, this method can be made deterministic. [18]

Matoušek, Sharir, and Welzl

Matoušek, Sharir & Welzl (1996) describe an algorithm that uses an additional property of linear programs that is not always held by other LP-type problems, that all bases have the same cardinality of each other. If an LP-type problem does not have this property, it can be made to have it by adding d new dummy elements and by modifying the function f to return the ordered pair of its old value f(A) and of the number min(d,|A|), ordered lexicographically.

Rather than adding elements of S one at a time, or finding samples of the elements, Matoušek, Sharir & Welzl (1996) describe an algorithm that removes elements one at a time. At each step it maintains a basis C that may initially be the set of dummy elements. It may be described with the following pseudocode:

function msw(S, f, C) isifS = CthenreturnC     choose a random element x of S \ CB = msw(S \ x, f, C)     iff(B) ≠ f(B ∪ {x}) thenB := basis(B ∪ {x})         B := msw(S, f, B)     returnB

In most of the recursive calls of the algorithm, the violation test succeeds and the if statement is skipped. However, with a small probability the violation test fails and the algorithm makes an additional basis computation and then an additional recursive call. As the authors show, the expected time for the algorithm is linear in n and exponential in the square root of d log n. By combining this method with Clarkson's recursive and iterative procedures, these two forms of time dependence can be separated out from each other, resulting in an algorithm that performs O(dn) violation tests in the outer recursive algorithm and a number that is exponential in the square root of d log d in the lower levels of the algorithm. [19]

Variations

Optimization with outliers

Matoušek (1995) considers a variation of LP-type optimization problems in which one is given, together with the set S and the objective function f, a number k; the task is to remove k elements from S in order to make the objective function on the remaining set as small as possible. For instance, when applied to the smallest circle problem, this would give the smallest circle that contains all but k of a given set of planar points. He shows that, for all non-degenerate LP-type problems (that is, problems in which all bases have distinct values) this problem may be solved in time O(nkd), by solving a set of O(kd) LP-type problems defined by subsets of S.

Implicit problems

Some geometric optimization problems may be expressed as LP-type problems in which the number of elements in the LP-type formulation is significantly greater than the number of input data values for the optimization problem. As an example, consider a collection of n points in the plane, each moving with constant velocity. At any point in time, the diameter of this system is the maximum distance between two of its points. The problem of finding a time at which the diameter is minimized can be formulated as minimizing the pointwise maximum of O(n2) quasiconvex functions, one for each pair of points, measuring the Euclidean distance between the pair as a function of time. Thus, it can be solved as an LP-type problem of combinatorial dimension two on a set of O(n2) elements, but this set is significantly larger than the number of input points. [20]

Chan (2004) describes an algorithm for solving implicitly defined LP-type problems such as this one in which each LP-type element is determined by a k-tuple of input values, for some constant k. In order to apply his approach, there must exist a decision algorithm that can determine, for a given LP-type basis B and set S of n input values, whether B is a basis for the LP-type problem determined by S.

Chan's algorithm performs the following steps:

With the assumption that the decision algorithm takes an amount of time O(T(n)) that grows at least polynomially as a function of the input size n, Chan shows that the threshold for switching to an explicit LP formulation and the number of subsets in the partition can be chosen in such a way that the implicit LP-type optimization algorithm also runs in time O(T(n)).

For instance, for the minimum diameter of moving points, the decision algorithm needs only to calculate the diameter of a set of points at a fixed time, a problem that can be solved in O(n log n) time using the rotating calipers technique. Therefore, Chan's algorithm for finding the time at which the diameter is minimized also takes time O(n log n). Chan uses this method to find a point of maximal Tukey depth among a given collection of n points in d-dimensional Euclidean space, in time O(nd 1 + n log n). A similar technique was used by Braß, Heinrich-Litan & Morin (2003) to find a point of maximal Tukey depth for the uniform distribution on a convex polygon.

The discovery of linear time algorithms for linear programming and the observation that the same algorithms could in many cases be used to solve geometric optimization problems that were not linear programs goes back at least to Megiddo ( 1983 , 1984 ), who gave a linear expected time algorithm for both three-variable linear programs and the smallest circle problem. However, Megiddo formulated the generalization of linear programming geometrically rather than combinatorially, as a convex optimization problem rather than as an abstract problem on systems of sets. Similarly, Dyer (1986) and Clarkson (in the 1988 conference version of Clarkson 1995) observed that their methods could be applied to convex programs as well as linear programs. Dyer (1992) showed that the minimum enclosing ellipsoid problem could also be formulated as a convex optimization problem by adding a small number of non-linear constraints. The use of randomization to improve the time bounds for low dimensional linear programming and related problems was pioneered by Clarkson and by Dyer & Frieze (1989).

The definition of LP-type problems in terms of functions satisfying the axioms of locality and monotonicity is from Sharir & Welzl (1992), but other authors in the same timeframe formulated alternative combinatorial generalizations of linear programs. For instance, in a framework developed by Gärtner (1995), the function f is replaced by a total ordering on the subsets of S. It is possible to break the ties in an LP-type problem to create a total order, but only at the expense of an increase in the combinatorial dimension. [21] Additionally, as in LP-type problems, Gärtner defines certain primitives for performing computations on subsets of elements; however, his formalization does not have an analogue of the combinatorial dimension.

Another abstract generalization of both linear programs and linear complementarity problems, formulated by Stickney & Watson (1978) and later studied by several other authors, concerns orientations of the edges of a hypercube with the property that every face of the hypercube (including the whole hypercube as a face) has a unique sink, a vertex with no outgoing edges. An orientation of this type may be formed from an LP-type problem by corresponding the subsets of S with the vertices of a hypercube in such a way that two subsets differ by a single element if and only if the corresponding vertices are adjacent, and by orienting the edge between neighboring sets AB towards B if f(A) ≠ f(B) and towards A otherwise. The resulting orientation has the additional property that it forms a directed acyclic graph, from which it can be shown that a randomized algorithm can find the unique sink of the whole hypercube (the optimal basis of the LP-type problem) in a number of steps exponential in the square root of n. [22]

The more recently developed framework of violator spaces generalizes LP-type problems, in the sense that every LP-type problem can be modeled by a violator space but not necessarily vice versa. Violator spaces are defined similarly to LP-type problems, by a function f that maps sets to objective function values, but the values of f are not ordered. Despite the lack of ordering, every set S has a well-defined set of bases (the minimal sets with the same value as the whole set) that can be found by variations of Clarkson's algorithms for LP-type problems. Indeed, violator spaces have been shown to exactly characterize the systems that can be solved by Clarkson's algorithms. [23]

Notes

  1. 1 2 3 Matoušek, Sharir & Welzl (1996).
  2. Although the smallest circle problem was first stated to be an LP-type problem by Matoušek, Sharir & Welzl (1996), several earlier papers described algorithms for this problem based on ideas from low-dimensional linear programming, including Megiddo (1983) and Welzl (1991).
  3. Fischer & Gärtner (2004).
  4. Löffler & van Kreveld (2010).
  5. Dyer (1986).
  6. Nielsen & Nock (2008).
  7. Matoušek, Sharir & Welzl (1996); Welzl (1991).
  8. Chan (2004).
  9. Amenta (1994).
  10. Amenta, Bern & Eppstein (1999); Eppstein (2005).
  11. Halman (2007).
  12. Amenta, Bern & Eppstein (1999).
  13. Puerto, Rodríguez-Chía & Tamir (2010).
  14. Eppstein (2006).
  15. Li (2007).
  16. Tail bounds on the size of V are also known: see Gärtner & Welzl (2001).
  17. Kalai (1992).
  18. Chazelle & Matoušek (1996).
  19. Matoušek, Sharir & Welzl (1996). A very similar time bound for linear programming was also given by Kalai (1992).
  20. The LP-type formulation of this problem was given by Chan (2004), but it was studied earlier using other algorithmic methods by Gupta, Janardan & Smid (1996). Chan also cites an unpublished manuscript by Clarkson for an O(n log n) time algorithm, matching the time that can be achieved by the implicit LP-type approach.
  21. Matoušek (2009).
  22. Szabó & Welzl (2001).
  23. Gärtner et al. (2008); Brise & Gärtner (2011).

Related Research Articles

<span class="mw-page-title-main">Decision problem</span> Yes/no problem in computer science

In computability theory and computational complexity theory, a decision problem is a computational problem that can be posed as a yes–no question of the input values. An example of a decision problem is deciding by means of an algorithm whether a given natural number is prime. Another is the problem "given two numbers x and y, does x evenly divide y?". The answer is either 'yes' or 'no' depending upon the values of x and y. A method for solving a decision problem, given in the form of an algorithm, is called a decision procedure for that problem. A decision procedure for the decision problem "given two numbers x and y, does x evenly divide y?" would give the steps for determining whether x evenly divides y. One such algorithm is long division. If the remainder is zero the answer is 'yes', otherwise it is 'no'. A decision problem which can be solved by an algorithm is called decidable.

<span class="mw-page-title-main">Convex hull</span> Smallest convex set containing a given set

In geometry, the convex hull or convex envelope or convex closure of a shape is the smallest convex set that contains it. The convex hull may be defined either as the intersection of all convex sets containing a given subset of a Euclidean space, or equivalently as the set of all convex combinations of points in the subset. For a bounded subset of the plane, the convex hull may be visualized as the shape enclosed by a rubber band stretched around the subset.

<span class="mw-page-title-main">Linear programming</span> Method to solve some optimization problems

Linear programming (LP), also called linear optimization, is a method to achieve the best outcome in a mathematical model whose requirements are represented by linear relationships. Linear programming is a special case of mathematical programming.

<span class="mw-page-title-main">Mathematical optimization</span> Study of mathematical algorithms for optimization problems

Mathematical optimization or mathematical programming is the selection of a best element, with regard to some criterion, from some set of available alternatives. It is generally divided into two subfields: discrete optimization and continuous optimization. Optimization problems arise in all quantitative disciplines from computer science and engineering to operations research and economics, and the development of solution methods has been of interest in mathematics for centuries.

<span class="mw-page-title-main">Tree decomposition</span> Mapping of a graph into a tree

In graph theory, a tree decomposition is a mapping of a graph into a tree that can be used to define the treewidth of the graph and speed up solving certain computational problems on the graph.

<span class="mw-page-title-main">Combinatorial optimization</span> Subfield of mathematical optimization

Combinatorial optimization is a subfield of mathematical optimization that consists of finding an optimal object from a finite set of objects, where the set of feasible solutions is discrete or can be reduced to a discrete set. Typical combinatorial optimization problems are the travelling salesman problem ("TSP"), the minimum spanning tree problem ("MST"), and the knapsack problem. In many such problems, such as the ones previously mentioned, exhaustive search is not tractable, and so specialized algorithms that quickly rule out large parts of the search space or approximation algorithms must be resorted to instead.

Branch and bound is a method for solving optimization problems by breaking them down into smaller sub-problems and using a bounding function to eliminate sub-problems that cannot contain the optimal solution. It is an algorithm design paradigm for discrete and combinatorial optimization problems, as well as mathematical optimization. A branch-and-bound algorithm consists of a systematic enumeration of candidate solutions by means of state space search: the set of candidate solutions is thought of as forming a rooted tree with the full set at the root. The algorithm explores branches of this tree, which represent subsets of the solution set. Before enumerating the candidate solutions of a branch, the branch is checked against upper and lower estimated bounds on the optimal solution, and is discarded if it cannot produce a better solution than the best one found so far by the algorithm.

<span class="mw-page-title-main">Bounding sphere</span> Sphere that contains a set of objects

In mathematics, given a non-empty set of objects of finite extension in -dimensional space, for example a set of points, a bounding sphere, enclosing sphere or enclosing ball for that set is an -dimensional solid sphere containing all of these objects.

<span class="mw-page-title-main">Arrangement of lines</span> Subdivision of the plane by lines

In geometry, an arrangement of lines is the subdivision of the plane formed by a collection of lines. Problems of counting the features of arrangements have been studied in discrete geometry, and computational geometers have found algorithms for the efficient construction of arrangements.

<span class="mw-page-title-main">Radon's theorem</span> Says d+2 points in d dimensions can be partitioned into two subsets whose convex hulls intersect

In geometry, Radon's theorem on convex sets, published by Johann Radon in 1921, states that:

Any set of d + 2 points in Rd can be partitioned into two sets whose convex hulls intersect.

In graph theory, the treewidth of an undirected graph is an integer number which specifies, informally, how far the graph is from being a tree. The smallest treewidth is 1; the graphs with treewidth 1 are exactly the trees and the forests. The graphs with treewidth at most 2 are the series–parallel graphs. The maximal graphs with treewidth exactly k are called k-trees, and the graphs with treewidth at most k are called partial k-trees. Many other well-studied graph families also have bounded treewidth.

In mathematical optimization, the ellipsoid method is an iterative method for minimizing convex functions. When specialized to solving feasible linear optimization problems with rational data, the ellipsoid method is an algorithm which finds an optimal solution in a number of steps that is polynomial in the input size.

<span class="mw-page-title-main">K-set (geometry)</span> Points separated from others by a line

In discrete geometry, a -set of a finite point set in the Euclidean plane is a subset of elements of that can be strictly separated from the remaining points by a line. More generally, in Euclidean space of higher dimensions, a -set of a finite point set is a subset of elements that can be separated from the remaining points by a hyperplane. In particular, when , the line or hyperplane that separates a -set from the rest of is a halving line or halving plane.

<span class="mw-page-title-main">Smallest-circle problem</span> Finding the smallest circle that contains all given points

The smallest-circle problem is a mathematical problem of computing the smallest circle that contains all of a given set of points in the Euclidean plane. The corresponding problem in n-dimensional space, the smallest bounding sphere problem, is to compute the smallest n-sphere that contains all of a given set of points. The smallest-circle problem was initially proposed by the English mathematician James Joseph Sylvester in 1857.

In mathematics, a unique sink orientation is an orientation of the edges of a polytope such that, in every face of the polytope, there is exactly one vertex for which all adjoining edges are oriented inward. If a polytope is given together with a linear objective function, and edges are oriented from vertices with smaller objective function values to vertices with larger objective values, the result is a unique sink orientation. Thus, unique sink orientations can be used to model linear programs as well as certain nonlinear programs such as the smallest circle problem.

<span class="mw-page-title-main">Kenneth L. Clarkson</span> American computer scientist

Kenneth Lee Clarkson is an American computer scientist known for his research in computational geometry. He is a researcher at the IBM Almaden Research Center, and co-editor-in-chief of the Journal of Computational Geometry.

<span class="mw-page-title-main">Criss-cross algorithm</span> Method for mathematical optimization

In mathematical optimization, the criss-cross algorithm is any of a family of algorithms for linear programming. Variants of the criss-cross algorithm also solve more general problems with linear inequality constraints and nonlinear objective functions; there are criss-cross algorithms for linear-fractional programming problems, quadratic-programming problems, and linear complementarity problems.

Emmerich (Emo) Welzl is a computer scientist known for his research in computational geometry. He is a professor in the Institute for Theoretical Computer Science at ETH Zurich in Switzerland.

In the design and analysis of algorithms for combinatorial optimization, parametric search is a technique invented by Nimrod Megiddo (1983) for transforming a decision algorithm into an optimization algorithm. It is frequently used for solving optimization problems in computational geometry.

References