Sethi-Skiba point

Last updated

Sethi-Skiba points, [1] [2] [3] [4] also known as DNSS points, arise in optimal control problems that exhibit multiple optimal solutions. A Sethi-Skiba point is an indifference point in an optimal control problem such that starting from such a point, the problem has more than one different optimal solutions. A good discussion of such points can be found in Grass et al. [2] [5] [6]

Contents

Definition

Of particular interest here are discounted infinite horizon optimal control problems that are autonomous. [2] These problems can be formulated as

s.t.

where is the discount rate, and are the state and control variables, respectively, at time , functions and are assumed to be continuously differentiable with respect to their arguments and they do not depend explicitly on time , and is the set of feasible controls and it also is explicitly independent of time . Furthermore, it is assumed that the integral converges for any admissible solution . In such a problem with one-dimensional state variable , the initial state is called a Sethi-Skiba point if the system starting from it exhibits multiple optimal solutions or equilibria. Thus, at least in the neighborhood of , the system moves to one equilibrium for and to another for . In this sense, is an indifference point from which the system could move to either of the two equilibria.

For two-dimensional optimal control problems, Grass et al. [5] and Zeiler et al. [7] present examples that exhibit DNSS curves.

Some references on the applications of Sethi-Skiba points are Caulkins et al., [8] Zeiler et al., [9] and Carboni and Russu [10]

History

Suresh P. Sethi identified such indifference points for the first time in 1977. [11] Further, Skiba, [12] Sethi, [13] [14] [15] and Deckert and Nishimura [16] explored these indifference points in economic models. The term DNSS (Deckert, Nishimura, Sethi, Skiba) points, introduced by Grass et al., [5] recognizes (alphabetically) the contributions of these authors. These indifference points have been also referred to as Skiba points or DNS points in earlier literature. [5]

Example

A simple problem exhibiting this behavior is given by and . It is shown in Grass et al. [5] that is a Sethi-Skiba point for this problem because the optimal path can be either or . Note that for , the optimal path is and for , the optimal path is .

Extensions

For further details and extensions, the reader is referred to Grass et al. [5]

Related Research Articles

<span class="mw-page-title-main">Indifference curve</span> Concept in economics

In economics, an indifference curve connects points on a graph representing different quantities of two goods, points between which a consumer is indifferent. That is, any combinations of two products indicated by the curve will provide the consumer with equal levels of utility, and the consumer has no preference for one combination or bundle of goods over a different combination on the same curve. One can also refer to each point on the indifference curve as rendering the same level of utility (satisfaction) for the consumer. In other words, an indifference curve is the locus of various points showing different combinations of two goods providing equal utility to the consumer. Utility is then a device to represent preferences rather than something from which preferences come. The main use of indifference curves is in the representation of potentially observable demand patterns for individual consumers over commodity bundles.

The calculus of variations is a field of mathematical analysis that uses variations, which are small changes in functions and functionals, to find maxima and minima of functionals: mappings from a set of functions to the real numbers. Functionals are often expressed as definite integrals involving functions and their derivatives. Functions that maximize or minimize functionals may be found using the Euler–Lagrange equation of the calculus of variations.

<span class="mw-page-title-main">Optimal control</span> Mathematical way of attaining a desired output from a dynamic system

Optimal control theory is a branch of control theory that deals with finding a control for a dynamical system over a period of time such that an objective function is optimized. It has numerous applications in science, engineering and operations research. For example, the dynamical system might be a spacecraft with controls corresponding to rocket thrusters, and the objective might be to reach the Moon with minimum fuel expenditure. Or the dynamical system could be a nation's economy, with the objective to minimize unemployment; the controls in this case could be fiscal and monetary policy. A dynamical system may also be introduced to embed operations research problems within the framework of optimal control theory.

Pontryagin's maximum principle is used in optimal control theory to find the best possible control for taking a dynamical system from one state to another, especially in the presence of constraints for the state or input controls. It states that it is necessary for any optimal control along with the optimal state trajectory to solve the so-called Hamiltonian system, which is a two-point boundary value problem, plus a maximum condition of the control Hamiltonian. These necessary conditions become sufficient under certain convexity conditions on the objective and constraint functions.

In mathematics, a Markov decision process (MDP) is a discrete-time stochastic control process. It provides a mathematical framework for modeling decision making in situations where outcomes are partly random and partly under the control of a decision maker. MDPs are useful for studying optimization problems solved via dynamic programming. MDPs were known at least as early as the 1950s; a core body of research on Markov decision processes resulted from Ronald Howard's 1960 book, Dynamic Programming and Markov Processes. They are used in many disciplines, including robotics, automatic control, economics and manufacturing. The name of MDPs comes from the Russian mathematician Andrey Markov as they are an extension of Markov chains.

<span class="mw-page-title-main">Bellman equation</span> Necessary condition for optimality associated with dynamic programming

A Bellman equation, named after Richard E. Bellman, is a necessary condition for optimality associated with the mathematical optimization method known as dynamic programming. It writes the "value" of a decision problem at a certain point in time in terms of the payoff from some initial choices and the "value" of the remaining decision problem that results from those initial choices. This breaks a dynamic optimization problem into a sequence of simpler subproblems, as Bellman's “principle of optimality" prescribes. The equation applies to algebraic structures with a total ordering; for algebraic structures with a partial ordering, the generic Bellman's equation can be used.

In mathematical optimization theory, duality or the duality principle is the principle that optimization problems may be viewed from either of two perspectives, the primal problem or the dual problem. If the primal is a minimization problem then the dual is a maximization problem. Any feasible solution to the primal (minimization) problem is at least as large as any feasible solution to the dual (maximization) problem. Therefore, the solution to the primal is an upper bound to the solution of the dual, and the solution of the dual is a lower bound to the solution of the primal. This fact is called weak duality.

In mathematics, and particularly ordinary differential equations (ODEs), a monodromy matrix is the fundamental matrix of a system of ODEs evaluated at the period of the coefficients of the system. It is used for the analysis of periodic solutions of ODEs in Floquet theory.

The theory of optimal control is concerned with operating a dynamic system at minimum cost. The case where the system dynamics are described by a set of linear differential equations and the cost is described by a quadratic function is called the LQ problem. One of the main results in the theory is that the solution is provided by the linear–quadratic regulator (LQR), a feedback controller whose equations are given below.

In control theory, the linear–quadratic–Gaussian (LQG) control problem is one of the most fundamental optimal control problems, and it can also be operated repeatedly for model predictive control. It concerns linear systems driven by additive white Gaussian noise. The problem is to determine an output feedback law that is optimal in the sense of minimizing the expected value of a quadratic cost criterion. Output measurements are assumed to be corrupted by Gaussian noise and the initial state, likewise, is assumed to be a Gaussian random vector.

The Hamiltonian is a function used to solve a problem of optimal control for a dynamical system. It can be understood as an instantaneous increment of the Lagrangian expression of the problem that is to be optimized over a certain time period. Inspired by—but distinct from—the Hamiltonian of classical mechanics, the Hamiltonian of optimal control theory was developed by Lev Pontryagin as part of his maximum principle. Pontryagin proved that a necessary condition for solving the optimal control problem is that the control should be chosen so as to optimize the Hamiltonian.

Merton's portfolio problem is a problem in continuous-time finance and in particular intertemporal portfolio choice. An investor must choose how much to consume and must allocate their wealth between stocks and a risk-free asset so as to maximize expected utility. The problem was formulated and solved by Robert C. Merton in 1969 both for finite lifetimes and for the infinite case. Research has continued to extend and generalize the model to include factors like transaction costs and bankruptcy.

The sequential probability ratio test (SPRT) is a specific sequential hypothesis test, developed by Abraham Wald and later proven to be optimal by Wald and Jacob Wolfowitz. Neyman and Pearson's 1933 result inspired Wald to reformulate it as a sequential analysis problem. The Neyman-Pearson lemma, by contrast, offers a rule of thumb for when all the data is collected.

The value function of an optimization problem gives the value attained by the objective function at a solution, while only depending on the parameters of the problem. In a controlled dynamical system, the value function represents the optimal payoff of the system over the interval [t, t1] when started at the time-t state variable x(t)=x. If the objective function represents some cost that is to be minimized, the value function can be interpreted as the cost to finish the optimal program, and is thus referred to as "cost-to-go function." In an economic context, where the objective function usually represents utility, the value function is conceptually equivalent to the indirect utility function.

Stochastic control or stochastic optimal control is a sub field of control theory that deals with the existence of uncertainty either in observations or in the noise that drives the evolution of the system. The system designer assumes, in a Bayesian probability-driven fashion, that random noise with known probability distribution affects the evolution and observation of the state variables. Stochastic control aims to design the time path of the controlled variables that performs the desired control task with minimum cost, somehow defined, despite the presence of this noise. The context may be either discrete time or continuous time.

In estimation theory, the extended Kalman filter (EKF) is the nonlinear version of the Kalman filter which linearizes about an estimate of the current mean and covariance. In the case of well defined transition models, the EKF has been considered the de facto standard in the theory of nonlinear state estimation, navigation systems and GPS.

The Sethi model was developed by Suresh P. Sethi and describes the process of how sales evolve over time in response to advertising. The model assumes that the rate of change in sales depend on three effects: response to advertising that acts positively on the unsold portion of the market, the loss due to forgetting or possibly due to competitive factors that act negatively on the sold portion of the market, and a random effect that can go either way.

<span class="mw-page-title-main">Suresh P. Sethi</span> American mathematician

Suresh P. Sethi is Eugene McDermott Chair of operations management and director of the Center for Intelligent Supply Networks at the University of Texas at Dallas.He has contributed in the fields of manufacturing and operations management, finance and economics, marketing, industrial engineering, operations research, and optimal control. He is known for his developments of the Sethi advertising model and DNSS Points, and for his textbook on optimal control.

<span class="mw-page-title-main">Shapley–Folkman lemma</span> Sums of sets of vectors are nearly convex

The Shapley–Folkman lemma is a result in convex geometry that describes the Minkowski addition of sets in a vector space. It is named after mathematicians Lloyd Shapley and Jon Folkman, but was first published by the economist Ross M. Starr.

K-convex functions, first introduced by Scarf, are a special weakening of the concept of convex function which is crucial in the proof of the optimality of the policy in inventory control theory. The policy is characterized by two numbers s and S, , such that when the inventory level falls below level s, an order is issued for a quantity that brings the inventory up to level S, and nothing is ordered otherwise. Gallego and Sethi have generalized the concept of K-convexity to higher dimensional Euclidean spaces.

References

  1. Caulkins, Jonathan P.; Grass, Dieter; Feichtinger, Gustav; Hartl, Richard F.; Kort, Peter M.; Prskawetz, Alexia; Seidl, Andrea; Wrzaczek, Stefan (2021-03-01). "The optimal lockdown intensity for COVID-19". Journal of Mathematical Economics. The economics of epidemics and emerging diseases. 93: 102489. doi:10.1016/j.jmateco.2021.102489. hdl: 10067/1777560151162165141 . ISSN   0304-4068.
  2. 1 2 3 Sethi, Suresh P. (2021). "Optimal Control Theory". Sethi, S.P. (2021). " Optimal Control Theory: Applications to Management Science and Economics". Fourth Edition, Springer Nature Switzerland AG, ISBN 978-3-319-98236-6 . doi:10.1007/978-3-319-98237-3. ISBN   978-3-319-98236-6.
  3. Caulkins, Jonathan P.; Grass, Dieter; Feichtinger, Gustav; Hartl, Richard F.; Kort, Peter M.; Prskawetz, Alexia; Seidl, Andrea; Wrzaczek, Stefan (2022), Boado-Penas, María del Carmen; Eisenberg, Julia; Şahin, Şule (eds.), "COVID-19 and Optimal LockdownStrategies: The Effect of New and MoreVirulent Strains", Pandemics: Insurance and Social Protection, Springer Actuarial, Cham: Springer International Publishing, pp. 163–190, doi: 10.1007/978-3-030-78334-1_9 , hdl: 10419/229887 , ISBN   978-3-030-78334-1
  4. Caulkins, Jonathan; Grass, Dieter; Feichtinger, Gustav; Hartl, Richard; Kort, Peter M.; Prskawetz, Alexia; Seidl, Andrea; Wrzaczek, Stefan (2020-12-02). "How long should the COVID-19 lockdown continue?". PLOS ONE. 15 (12): e0243413. doi: 10.1371/journal.pone.0243413 . ISSN   1932-6203. PMC   7710360 . PMID   33264368.
  5. 1 2 3 4 5 6 Grass, D.; Caulkins, J. P.; Feichtinger, G.; Tragler, G.; Behrens, D. A. (2008). Optimal Control of Nonlinear Processes: With Applications in Drugs, Corruption, and Terror. Springer. ISBN   978-3-540-77646-8.
  6. Caulkins, J. P., Grass, D., Feichtinger, G., Hartl, R. F., Kort, P. M., Prskawetz, A., Seidl, A., Wrzaczek, A. (2020). “When should the Covid-19 lockdown end?”. OR News, Ausgabe 69: 10-13
  7. Zeiler, I., Caulkins, J., Grass, D., Tragler, G. (2009). Keeping Options Open: An Optimal Control Model with Trajectories that Reach a DNSS Point in Positive Time. SIAM Journal on Control and Optimization, Vol. 48, No. 6, pp. 3698-3707.| doi =10.1137/080719741 |
  8. Caulkins, J. P.; Feichtinger, G.; Grass, D.; Tragler, G. (2009). "Optimal control of terrorism and global reputation: A case study with novel threshold behavior". Operations Research Letters. 37 (6): 387–391. doi:10.1016/j.orl.2009.07.003.
  9. I. Zeiler, J. P. Caulkins, and G. Tragler. When Two Become One: Optimal Control of Interacting Drug. Working paper, Vienna University of Technology, Vienna, Austria
  10. Carboni, Oliviero A.; Russu, Paolo (2021-06-01). "Taxation, Corruption and Punishment: Integrating Evolutionary Game into the Optimal Control of Government Policy". International Game Theory Review. 23 (2): 2050019. doi:10.1142/S021919892050019X. ISSN   0219-1989.
  11. Sethi, S.P. (1977). "Nearest Feasible Paths in Optimal Control Problems: Theory, Examples, and Counterexamples". Journal of Optimization Theory and Applications. 23 (4): 563–579. doi:10.1007/BF00933297. S2CID   123705828.
  12. Skiba, A.K. (1978). "Optimal Growth with a Convex-Concave Production Function". Econometrica. 46 (3): 527–539. doi:10.2307/1914229. JSTOR   1914229.
  13. Sethi, S. P. (1977-12-01). "Nearest feasible paths in optimal control problems: Theory, examples, and counterexamples". Journal of Optimization Theory and Applications. 23 (4): 563–579. doi:10.1007/BF00933297. ISSN   1573-2878.
  14. Sethi, S.P. (1979). "Optimal Advertising Policy with the Contagion Model". Journal of Optimization Theory and Applications. 29 (4): 615–627. doi:10.1007/BF00934454. S2CID   121398518.
  15. Sethi, S.P., "Optimal Quarantine Programmes for Controlling an Epidemic Spread," Journal of Operational Research Society, 29(3), 1978, 265-268. JSTOR 3009454 SSRN 3587573
  16. Deckert, D.W.; Nishimura, K. (1983). "A Complete Characterization of Optimal Growth Paths in an Aggregated Model with Nonconcave Production Function". Journal of Economic Theory. 31 (2): 332–354. doi:10.1016/0022-0531(83)90081-9.