Karmarkar's algorithm

Last updated

Karmarkar's algorithm is an algorithm introduced by Narendra Karmarkar in 1984 for solving linear programming problems. It was the first reasonably efficient algorithm that solves these problems in polynomial time. The ellipsoid method is also polynomial time but proved to be inefficient in practice.

Contents

Denoting by the number of variables, m the number of inequality constraints, and the number of bits of input to the algorithm, Karmarkar's algorithm requires operations on -digit numbers, as compared to such operations for the ellipsoid algorithm. [1] In "square" problems, when m is in O(n), Karmarkar's algorithm requires operations on -digit numbers, as compared to such operations for the ellipsoid algorithm. The runtime of Karmarkar's algorithm is thus

using FFT-based multiplication (see Big O notation).

Karmarkar's algorithm falls within the class of interior-point methods: the current guess for the solution does not follow the boundary of the feasible set as in the simplex method, but moves through the interior of the feasible region, improving the approximation of the optimal solution by a definite fraction with every iteration and converging to an optimal solution with rational data. [2]

The algorithm

Consider a linear programming problem in matrix form:

maximize cTx
subject toAxb.

Karmarkar's algorithm determines the next feasible direction toward optimality and scales back by a factor 0 < γ ≤ 1. It is described in a number of sources. [3] [4] [5] [6] [7] [8] Karmarkar also has extended the method [9] [10] [11] [12] to solve problems with integer constraints and non-convex problems. [13]

Algorithm Affine-Scaling

Since the actual algorithm is rather complicated, researchers looked for a more intuitive version of it, and in 1985 developed affine scaling, a version of Karmarkar's algorithm that uses affine transformations where Karmarkar used projective ones, only to realize four years later that they had rediscovered an algorithm published by Soviet mathematician I. I. Dikin in 1967. [14] The affine-scaling method can be described succinctly as follows. [15] While applicable to small scale problems, it is not a polynomial time algorithm.[ citation needed ]

Input:  A, b, c, ,stopping criterion, γ.
do whilestopping criterionnot satisfiedifthenreturn unbounded     end ifend do

Example

Example solution Karmarkar.svg
Example solution

Consider the linear program

That is, there are 2 variables and 11 constraints associated with varying values of . This figure shows each iteration of the algorithm as red circle points. The constraints are shown as blue lines.

Patent controversy

At the time he invented the algorithm, Karmarkar was employed by IBM as a postdoctoral fellow in the IBM San Jose Research Laboratory in California. On August 11, 1983 he gave a seminar at Stanford University explaining the algorithm, with his affiliation still listed as IBM. By the fall of 1983 Karmarkar started to work at AT&T and submitted his paper to the 1984 ACM Symposium on Theory of Computing (STOC, held April 30 - May 2, 1984) stating AT&T Bell Laboratories as his affiliation. [16] After applying the algorithm to optimizing AT&T's telephone network, [17] they realized that his invention could be of practical importance. In April 1985, AT&T promptly applied for a patent on his algorithm.

The patent became more fuel for the ongoing controversy over the issue of software patents. [18] This left many mathematicians uneasy, such as Ronald Rivest (himself one of the holders of the patent on the RSA algorithm), who expressed the opinion that research proceeded on the basis that algorithms should be free. Even before the patent was actually granted, it was argued that there might have been prior art that was applicable. [19] Mathematicians who specialized in numerical analysis, including Philip Gill and others, claimed that Karmarkar's algorithm is equivalent to a projected Newton barrier method with a logarithmic barrier function, if the parameters are chosen suitably. [20] Legal scholar Andrew Chin opines that Gill's argument was flawed, insofar as the method they describe does not constitute an "algorithm", since it requires choices of parameters that don't follow from the internal logic of the method, but rely on external guidance, essentially from Karmarkar's algorithm. [21] Furthermore, Karmarkar's contributions are considered far from obvious in light of all prior work, including Fiacco-McCormick, Gill and others cited by Saltzman. [21] [22] [23] The patent was debated in the U.S. Senate [ citation needed ] and granted in recognition of the essential originality of Karmarkar's work, as U.S. patent 4,744,028 : "Methods and apparatus for efficient resource allocation" in May 1988.

AT&T designed a vector multi-processor computer system specifically to run Karmarkar's algorithm, calling the resulting combination of hardware and software KORBX, [24] and marketed this system at a price of US$8.9 million. [25] [26] Its first customer was the Pentagon. [27] [28]

Opponents of software patents have further argued that the patents ruined the positive interaction cycles that previously characterized the relationship between researchers in linear programming and industry, and specifically it isolated Karmarkar himself from the network of mathematical researchers in his field. [29]

The patent itself expired in April 2006, and the algorithm is presently in the public domain.

The United States Supreme Court has held that mathematics cannot be patented in Gottschalk v. Benson , [30] In that case, the Court first addressed whether computer algorithms could be patented and it held that they could not because the patent system does not protect ideas and similar abstractions. In Diamond v. Diehr , [31] the Supreme Court stated, "A mathematical formula as such is not accorded the protection of our patent laws, and this principle cannot be circumvented by attempting to limit the use of the formula to a particular technological environment. [32] In Mayo Collaborative Services v. Prometheus Labs., Inc. , [33] the Supreme Court explained further that "simply implementing a mathematical principle on a physical machine, namely a computer, [i]s not a patentable application of that principle." [34]

Applications

Karmarkar's algorithm was used by the US Army for logistic planning during the Gulf War. [1]

Related Research Articles

<span class="mw-page-title-main">Knapsack problem</span> Problem in combinatorial optimization

The knapsack problem is the following problem in combinatorial optimization:

Quadratic programming (QP) is the process of solving certain mathematical optimization problems involving quadratic functions. Specifically, one seeks to optimize a multivariate quadratic function subject to linear constraints on the variables. Quadratic programming is a type of nonlinear programming.

<span class="mw-page-title-main">Linear programming</span> Method to solve optimization problems

Linear programming (LP), also called linear optimization, is a method to achieve the best outcome in a mathematical model whose requirements and objective are represented by linear relationships. Linear programming is a special case of mathematical programming.

<span class="mw-page-title-main">Assignment problem</span> Combinatorial optimization problem

The assignment problem is a fundamental combinatorial optimization problem. In its most general form, the problem is as follows:

An integer programming problem is a mathematical optimization or feasibility program in which some or all of the variables are restricted to be integers. In many settings the term refers to integer linear programming (ILP), in which the objective function and the constraints are linear.

<span class="mw-page-title-main">Combinatorial optimization</span> Subfield of mathematical optimization

Combinatorial optimization is a subfield of mathematical optimization that consists of finding an optimal object from a finite set of objects, where the set of feasible solutions is discrete or can be reduced to a discrete set. Typical combinatorial optimization problems are the travelling salesman problem ("TSP"), the minimum spanning tree problem ("MST"), and the knapsack problem. In many such problems, such as the ones previously mentioned, exhaustive search is not tractable, and so specialized algorithms that quickly rule out large parts of the search space or approximation algorithms must be resorted to instead.

<span class="mw-page-title-main">Set cover problem</span> Classical problem in combinatorics

The set cover problem is a classical question in combinatorics, computer science, operations research, and complexity theory.

Narendra Krishna Karmarkar is an Indian mathematician. Karmarkar developed Karmarkar's algorithm. He is listed as an ISI highly cited researcher.

The Fulkerson Prize for outstanding papers in the area of discrete mathematics is sponsored jointly by the Mathematical Optimization Society (MOS) and the American Mathematical Society (AMS). Up to three awards of $1,500 each are presented at each (triennial) International Symposium of the MOS. Originally, the prizes were paid out of a memorial fund administered by the AMS that was established by friends of the late Delbert Ray Fulkerson to encourage mathematical excellence in the fields of research exemplified by his work. The prizes are now funded by an endowment administered by MPS.

<span class="mw-page-title-main">Interior-point method</span> Algorithms for solving convex optimization problems

Interior-point methods are algorithms for solving linear and non-linear convex optimization problems. IPMs combine two advantages of previously-known algorithms:

Convex optimization is a subfield of mathematical optimization that studies the problem of minimizing convex functions over convex sets. Many classes of convex optimization problems admit polynomial-time algorithms, whereas mathematical optimization is in general NP-hard.

The Frank–Wolfe algorithm is an iterative first-order optimization algorithm for constrained convex optimization. Also known as the conditional gradient method, reduced gradient algorithm and the convex combination algorithm, the method was originally proposed by Marguerite Frank and Philip Wolfe in 1956. In each iteration, the Frank–Wolfe algorithm considers a linear approximation of the objective function, and moves towards a minimizer of this linear function.

In mathematical optimization, the ellipsoid method is an iterative method for minimizing convex functions over convex sets. The ellipsoid method generates a sequence of ellipsoids whose volume uniformly decreases at every step, thus enclosing a minimizer of a convex function.

Semidefinite programming (SDP) is a subfield of mathematical programming concerned with the optimization of a linear objective function over the intersection of the cone of positive semidefinite matrices with an affine space, i.e., a spectrahedron.

Robert J. Vanderbei is an American mathematician and Emeritus Professor in the Department of Operations Research and Financial Engineering at Princeton University.

<span class="mw-page-title-main">Klee–Minty cube</span> Unit hypercube of variable dimension whose corners have been perturbed

The Klee–Minty cube or Klee–Minty polytope is a unit hypercube of variable dimension whose corners have been perturbed. Klee and Minty demonstrated that George Dantzig's simplex algorithm has poor worst-case performance when initialized at one corner of their "squashed cube". On the three-dimensional version, the simplex algorithm and the criss-cross algorithm visit all 8 corners in the worst case.

In applied mathematics, Graver bases enable iterative solutions of linear and various nonlinear integer programming problems in polynomial time. They were introduced by Jack E. Graver. Their connection to the theory of Gröbner bases was discussed by Bernd Sturmfels. The algorithmic theory of Graver bases and its application to integer programming is described by Shmuel Onn.

<span class="mw-page-title-main">Affine scaling</span> Algorithm for solving linear programming problems

In mathematical optimization, affine scaling is an algorithm for solving linear programming problems. Specifically, it is an interior point method, discovered by Soviet mathematician I. I. Dikin in 1967 and reinvented in the U.S. in the mid-1980s.

The configuration linear program (configuration-LP) is a linear programming technique used for solving combinatorial optimization problems. It was introduced in the context of the cutting stock problem. Later, it has been applied to the bin packing and job scheduling problems. In the configuration-LP, there is a variable for each possible configuration - each possible multiset of items that can fit in a single bin. Usually, the number of configurations is exponential in the problem size, but in some cases it is possible to attain approximate solutions using only a polynomial number of configurations.

<span class="mw-page-title-main">Chambolle-Pock algorithm</span> Primal-Dual algorithm optimization for convex problems

In mathematics, the Chambolle-Pock algorithm is an algorithm used to solve convex optimization problems. It was introduced by Antonin Chambolle and Thomas Pock in 2011 and has since become a widely used method in various fields, including image processing, computer vision, and signal processing.

References

  1. 1 2 Arkadi Nemirovsky (2004). Interior point polynomial-time methods in convex programming.
  2. Strang, Gilbert (1 June 1987). "Karmarkar's algorithm and its place in applied mathematics". The Mathematical Intelligencer . 9 (2): 4–10. doi:10.1007/BF03025891. ISSN   0343-6993. MR   0883185. S2CID   123541868.
  3. Karmarkar, N. (1984). "A new polynomial-time algorithm for linear programming". Proceedings of the sixteenth annual ACM symposium on Theory of computing - STOC '84. pp. 302–311. doi:10.1145/800057.808695. ISBN   0897911334. S2CID   13101261.
  4. Karmarkar, N. (1984). "A new polynomial-time algorithm for linear programming". Combinatorica. 4 (4): 373–395. doi:10.1007/BF02579150. S2CID   7257867.
  5. Karmarkar, Narendra K. (1989). "Power Series Variants of Karmarkar-Type Algorithms". AT&T Technical Journal. 68 (3): 20–36. doi:10.1002/j.1538-7305.1989.tb00316.x. S2CID   42071587.
  6. Karmarkar, Narendra (1990). "An interior-point approach to NP-complete problems. I". Mathematical developments arising from linear programming (Brunswick, ME, 1988). Contemporary Mathematics. Vol. 114. Providence, RI: American Mathematical Society. pp. 297–308. doi:10.1090/conm/114/1097880. MR   1097880.
  7. Karmarkar, Narendra (1990). "Riemannian geometry underlying interior-point methods for linear programming". Mathematical developments arising from linear programming (Brunswick, ME, 1988). Contemporary Mathematics. Vol. 114. Providence, RI: American Mathematical Society. pp. 51–75. doi:10.1090/conm/114/1097865. MR   1097865.
  8. Karmarkar N. K., Lagarias, J.C., Slutsman, L., and Wang, P., Power Series Variants of KarmarkarType Algorithm, AT & T technical Journal 68, No. 3, May/June (1989).
  9. Karmarkar, N.K., Interior Point Methods in Optimization, Proceedings of the Second International Conference on Industrial and Applied Mathematics, SIAM, pp. 160181 (1991)
  10. Karmarkar, N. K. and Kamath, A. P., A continuous Approach to Deriving Upper Bounds in Quadratic Maximization Problems with Integer Constraints, Recent Advances in Global Optimization, pp. 125140, Princeton University Press (1992).
  11. 26. Karmarkar, N. K., Thakur, S. A., An Interior Point Approach to a Tensor Optimisation Problem with Application to Upper Bounds in Integer Quadratic Optimization Problems, Proceedings of Second Conference on Integer Programming and Combinatorial Optimisation, (May 1992).
  12. 27. Kamath, A., Karmarkar, N. K., A Continuous Method for Computing Bounds in Integer Quadratic Optimisation Problems, Journal of Global Optimization (1992).
  13. Karmarkar, N. K., Beyond Convexity: New Perspectives in Computational Optimization. Springer Lecture Notes in Computer Science LNCS 6457, Dec 2010
  14. Vanderbei, R. J.; Lagarias, J. C. (1990). "I. I. Dikin's convergence result for the affine-scaling algorithm". Mathematical developments arising from linear programming (Brunswick, ME, 1988). Contemporary Mathematics. Vol. 114. Providence, RI: American Mathematical Society. pp. 109–119. doi:10.1090/conm/114/1097868. MR   1097868.
  15. Robert J. Vanderbei; Meketon, Marc; Freedman, Barry (1986). "A Modification of Karmarkar's Linear Programming Algorithm" (PDF). Algorithmica. 1 (1–4): 395–407. doi:10.1007/BF01840454. S2CID   779577.
  16. Karmarkar Algorithm, IBM Research, retrieved 2016-06-01
  17. Sinha L.P., Freedman, B. A., Karmarkar, N. K., Putcha, A., and Ramakrishnan K.G., Overseas Network Planning, Proceedings of the Third International Network Planning Symposium, NETWORKS' 86, Tarpon Springs, Florida (June 1986).
  18. Kolata, Gina (1989-03-12). "IDEAS & TRENDS; Mathematicians Are Troubled by Claims on Their Recipes". The New York Times .
  19. Various posts by Matthew Saltzman, Clemson University
  20. Gill, Philip E.; Murray, Walter; Saunders, Michael A.; Tomlin, J. A.; Wright, Margaret H. (1986). "On projected Newton barrier methods for linear programming and an equivalence to Karmarkar's projective method". Mathematical Programming. 36 (2): 183–209. doi:10.1007/BF02592025. S2CID   18899771.
  21. 1 2 Andrew Chin (2009). "On Abstraction and Equivalence in Software Patent Doctrine: A Response to Bessen, Meurer and Klemens" (PDF). Journal of Intellectual Property Law. 16: 214–223.
  22. Mark A. Paley (1995). "The Karmarkar Patent: Why Congress Should "Open the Door" to Algorithms as Patentable Subject Matter". 22 Computer L. Rep. 7
  23. Margaret H. Wright (2004). "The Interior-Point Revolution in Optimization: History, Recent Developments, and Lasting Consequences" (PDF). Bulletin of the American Mathematical Society. 42: 39–56. doi: 10.1090/S0273-0979-04-01040-7 .
  24. Marc S. Meketon; Y.C. Cheng; D.J. Houck; J.M.Liu; L. Slutsman; Robert J. Vanderbei; P. Wang (1989). "The AT&T KORBX System". AT&T Technical Journal. 68 (3): 7–19. doi:10.1002/j.1538-7305.1989.tb00315.x. S2CID   18548851.
  25. Lowenstein, Roger (15 August 1988). "AT&T markets problem solver, based on math whiz's find, for $8.9 million" (PDF). Wall Street Journal. Archived from the original (PDF) on 8 June 2016. Retrieved 30 January 2016.
  26. Markoff, John (13 August 1988). "Big A.T.&T. Computer for Complexities". The New York Times.
  27. "Military Is First Announced Customer Of AT&T Software". Associated Press . AP News. Retrieved 2019-06-11.
  28. Kennington, J.L. (1989). "Using KORBX for military airlift applications". Proceedings of the 28th IEEE Conference on Decision and Control. pp. 1603–1605. doi:10.1109/CDC.1989.70419. S2CID   60450719.
  29. "今野浩: カーマーカー特許とソフトウェア – 数学は 特許に なるか (Konno Hiroshi: The Kamarkar Patent and Software – Has Mathematics Become Patentable?)". FFII. Archived from the original on 2008-06-27. Retrieved 2008-06-27.
  30. 409 U.S. 63 (1972). The case concerned an algorithm for converting binary-coded decimal numerals to pure binary.
  31. 450 U.S. 175 (1981).
  32. 450 U.S. at 191. See also Parker v. Flook , 437 U.S. 584, 585 (1978) ("the discovery of a novel and useful mathematical formula may not be patented").
  33. 566 U.S. __, 132 S. Ct. 1289 (2012).
  34. Accord Alice Corp. v. CLS Bank Int’l , 573 U.S. __, 134 S. Ct. 2347 (2014).