Odds algorithm

Last updated

In decision theory, the odds algorithm (or Bruss algorithm) is a mathematical method for computing optimal strategies for a class of problems that belong to the domain of optimal stopping problems. Their solution follows from the odds strategy, and the importance of the odds strategy lies in its optimality, as explained below.

Contents

The odds algorithm applies to a class of problems called last-success problems. Formally, the objective in these problems is to maximize the probability of identifying in a sequence of sequentially observed independent events the last event satisfying a specific criterion (a "specific event"). This identification must be done at the time of observation. No revisiting of preceding observations is permitted. Usually, a specific event is defined by the decision maker as an event that is of true interest in the view of "stopping" to take a well-defined action. Such problems are encountered in several situations.

Examples

Two different situations exemplify the interest in maximizing the probability to stop on a last specific event.

  1. Suppose a car is advertised for sale to the highest bidder (best "offer"). Let potential buyers respond and ask to see the car. Each insists upon an immediate decision from the seller to accept the bid, or not. Define a bid as interesting, and coded 1 if it is better than all preceding bids, and coded 0 otherwise. The bids will form a random sequence of 0s and 1s. Only 1s interest the seller, who may fear that each successive 1 might be the last. It follows from the definition that the very last 1 is the highest bid. Maximizing the probability of selling on the last 1 therefore means maximizing the probability of selling best.
  2. A physician, using a special treatment, may use the code 1 for a successful treatment, 0 otherwise. The physician treats a sequence of patients the same way, and wants to minimize any suffering, and to treat every responsive patient in the sequence. Stopping on the last 1 in such a random sequence of 0s and 1s would achieve this objective. Since the physician is no prophet, the objective is to maximize the probability of stopping on the last 1. (See Compassionate use.)

Definitions

Consider a sequence of independent events. Associate with this sequence another sequence of independent events with values 1 or 0. Here , called a success, stands for the event that the kth observation is interesting (as defined by the decision maker), and for non-interesting. These random variables are observed sequentially and the goal is to correctly select the last success when it is observed.

Let be the probability that the kth event is interesting. Further let and . Note that represents the odds of the kth event turning out to be interesting, explaining the name of the odds algorithm.

Algorithmic procedure

The odds algorithm sums up the odds in reverse order

until this sum reaches or exceeds the value 1 for the first time. If this happens at index s, it saves s and the corresponding sum

If the sum of the odds does not reach 1, it sets s = 1. At the same time it computes

The output is

  1. , the stopping threshold
  2. , the win probability.

Odds strategy

The odds strategy is the rule to observe the events one after the other and to stop on the first interesting event from index s onwards (if any), where s is the stopping threshold of output a.

The importance of the odds strategy, and hence of the odds algorithm, lies in the following odds theorem.

Odds theorem

The odds theorem states that

  1. The odds strategy is optimal, that is, it maximizes the probability of stopping on the last 1.
  2. The win probability of the odds strategy equals
  3. If , the win probability is always at least 1/e = 0.367879..., and this lower bound is best possible.

Features

The odds algorithm computes the optimal strategy and the optimal win probability at the same time. Also, the number of operations of the odds algorithm is (sub)linear in n. Hence no quicker algorithm can possibly exist for all sequences, so that the odds algorithm is, at the same time, optimal as an algorithm.

Sources

Bruss 2000 devised the odds algorithm, and coined its name. It is also known as Bruss algorithm (strategy). Free implementations can be found on the web.

Applications

Applications reach from medical questions in clinical trials over sales problems, secretary problems, portfolio selection, (one way) search strategies, trajectory problems and the parking problem to problems in online maintenance and others.

There exists, in the same spirit, an Odds Theorem for continuous-time arrival processes with independent increments such as the Poisson process (Bruss 2000). In some cases, the odds are not necessarily known in advance (as in Example 2 above) so that the application of the odds algorithm is not directly possible. In this case each step can use sequential estimates of the odds. This is meaningful, if the number of unknown parameters is not large compared with the number n of observations. The question of optimality is then more complicated, however, and requires additional studies. Generalizations of the odds algorithm allow for different rewards for failing to stop and wrong stops as well as replacing independence assumptions by weaker ones ( Ferguson 2008 ).

Variations

Bruss & Paindaveine 2000 discussed a problem of selecting the last successes.

Tamaki 2010 proved a multiplicative odds theorem which deals with a problem of stopping at any of the last successes. A tight lower bound of win probability is obtained by Matsui & Ano 2014.

Matsui & Ano 2017 discussed a problem of selecting out of the last successes and obtained a tight lower bound of win probability. When the problem is equivalent to Bruss' odds problem. If the problem is equivalent to that in Bruss & Paindaveine 2000. A problem discussed by Tamaki 2010 is obtained by setting

Multiple choice problem

A player is allowed choices, and he wins if any choice is the last success. For classical secretary problem, Gilbert & Mosteller 1966 discussed the cases . The odds problem with is discussed by Ano, Kakinuma & Miyoshi 2010. For further cases of odds problem, see Matsui & Ano 2016.

An optimal strategy for this problem belongs to the class of strategies defined by a set of threshold numbers , where .

Specifically, imagine that you have letters of acceptance labelled from to . You would have application officers, each holding one letter. You keep interviewing the candidates and rank them on a chart that every application officer can see. Now officer would send their letter of acceptance to the first candidate that is better than all candidates to . (Unsent letters of acceptance are by default given to the last applicants, the same as in the standard secretary problem.)

When , Ano, Kakinuma & Miyoshi 2010 showed that the tight lower bound of win probability is equal to For general positive integer , Matsui & Ano 2016 proved that the tight lower bound of win probability is the win probability of the secretary problem variant where one must pick the top-k candidates using just k attempts.

When , tight lower bounds of win probabilities are equal to , and respectively.

For further numerical cases for , and an algorithm for general cases, see Matsui & Ano 2016.

See also

Related Research Articles

Shor's algorithm is a quantum algorithm for finding the prime factors of an integer. It was developed in 1994 by the American mathematician Peter Shor. It is one of the few known quantum algorithms with compelling potential applications and strong evidence of superpolynomial speedup compared to best known classical (non-quantum) algorithms. On the other hand, factoring numbers of practical significance requires far more qubits than available in the near future. Another concern is that noise in quantum circuits may undermine results, requiring additional qubits for quantum error correction.

In game theory, the Nash equilibrium is the most commonly-used solution concept for non-cooperative games. A Nash equilibrium is a situation where no player could gain by changing their own strategy. The idea of Nash equilibrium dates back to the time of Cournot, who in 1838 applied it to his model of competition in an oligopoly.

<span class="mw-page-title-main">Dynamic programming</span> Problem optimization method

Dynamic programming is both a mathematical optimization method and an algorithmic paradigm. The method was developed by Richard Bellman in the 1950s and has found applications in numerous fields, from aerospace engineering to economics.

In statistics, gambler's ruin is the fact that a gambler playing a game with negative expected value will eventually go bankrupt, regardless of his betting system.

In the field of mathematical optimization, stochastic programming is a framework for modeling optimization problems that involve uncertainty. A stochastic program is an optimization problem in which some or all problem parameters are uncertain, but follow known probability distributions. This framework contrasts with deterministic optimization, in which all problem parameters are assumed to be known exactly. The goal of stochastic programming is to find a decision which both optimizes some criteria chosen by the decision maker, and appropriately accounts for the uncertainty of the problem parameters. Because many real-world decisions involve uncertainty, stochastic programming has found applications in a broad range of areas ranging from finance to transportation to energy optimization.

In combinatorics, Bertrand's ballot problem is the question: "In an election where candidate A receives p votes and candidate B receives q votes with p > q, what is the probability that A will be strictly ahead of B throughout the count?" The answer is

<span class="mw-page-title-main">Stopping time</span> Time at which a random variable stops exhibiting a behavior of interest

In probability theory, in particular in the study of stochastic processes, a stopping time is a specific type of “random time”: a random variable whose value is interpreted as the time at which a given stochastic process exhibits a certain behavior of interest. A stopping time is often defined by a stopping rule, a mechanism for deciding whether to continue or stop a process on the basis of the present position and past events, and which will almost always lead to a decision to stop at some finite time.

<span class="mw-page-title-main">Secretary problem</span> Mathematical problem involving optimal stopping theory

The secretary problem demonstrates a scenario involving optimal stopping theory that is studied extensively in the fields of applied probability, statistics, and decision theory. It is also known as the marriage problem, the sultan's dowry problem, the fussy suitor problem, the googol game, and the best choice problem. Its solution is also known as the 37% rule.

<span class="mw-page-title-main">Kelly criterion</span> Formula for bet sizing that maximizes the expected logarithmic value

In probability theory, the Kelly criterion is a formula for sizing a bet. The Kelly bet size is found by maximizing the expected value of the logarithm of wealth, which is equivalent to maximizing the expected geometric growth rate. Assuming that the expected returns are known, the Kelly criterion leads to higher wealth than any other strategy in the long run. John Larry Kelly Jr., a researcher at Bell Labs, described the criterion in 1956.

In computer science, the longest increasing subsequence problem aims to find a subsequence of a given sequence in which the subsequence's elements are sorted in an ascending order and in which the subsequence is as long as possible. This subsequence is not necessarily contiguous or unique. The longest increasing subsequences are studied in the context of various disciplines related to mathematics, including algorithmics, random matrix theory, representation theory, and physics. The longest increasing subsequence problem is solvable in time where denotes the length of the input sequence.

The cross-entropy (CE) method is a Monte Carlo method for importance sampling and optimization. It is applicable to both combinatorial and continuous problems, with either a static or noisy objective.

In mathematics, the theory of optimal stopping or early stopping is concerned with the problem of choosing a time to take a particular action, in order to maximise an expected reward or minimise an expected cost. Optimal stopping problems can be found in areas of statistics, economics, and mathematical finance. A key example of an optimal stopping problem is the secretary problem. Optimal stopping problems can often be written in the form of a Bellman equation, and are therefore often solved using dynamic programming.

Stochastic approximation methods are a family of iterative methods typically used for root-finding problems or for optimization problems. The recursive update rules of stochastic approximation methods can be used, among other things, for solving linear systems when the collected data is corrupted by noise, or for approximating extreme values of functions which cannot be computed directly, but only estimated via noisy observations.

The Gittins index is a measure of the reward that can be achieved through a given stochastic process with certain properties, namely: the process has an ultimate termination state and evolves with an option, at each intermediate state, of terminating. Upon terminating at a given state, the reward achieved is the sum of the probabilistic expected rewards associated with every state from the actual terminating state to the ultimate terminal state, inclusive. The index is a real scalar.

Precoding is a generalization of beamforming to support multi-stream transmission in multi-antenna wireless communications. In conventional single-stream beamforming, the same signal is emitted from each of the transmit antennas with appropriate weighting such that the signal power is maximized at the receiver output. When the receiver has multiple antennas, single-stream beamforming cannot simultaneously maximize the signal level at all of the receive antennas. In order to maximize the throughput in multiple receive antenna systems, multi-stream transmission is generally required.

In probability theory, Robbins' problem of optimal stopping, named after Herbert Robbins, is sometimes referred to as the fourth secretary problem or the problem of minimizing the expected rank with full information.

Let X1, ..., Xn be independent, identically distributed random variables, uniform on [0, 1]. We observe the Xk's sequentially and must stop on exactly one of them. No recall of preceding observations is permitted. What stopping rule minimizes the expected rank of the selected observation, and what is its corresponding value?

The exponential mechanism is a technique for designing differentially private algorithms. It was developed by Frank McSherry and Kunal Talwar in 2007. Their work was recognized as a co-winner of the 2009 PET Award for Outstanding Research in Privacy Enhancing Technologies.

In the theory of stochastic processes in discrete time, a part of the mathematical theory of probability, the Doob decomposition theorem gives a unique decomposition of every adapted and integrable stochastic process as the sum of a martingale and a predictable process starting at zero. The theorem was proved by and is named for Joseph L. Doob.

In computer science, an optimal binary search tree (Optimal BST), sometimes called a weight-balanced binary tree, is a binary search tree which provides the smallest possible search time (or expected search time) for a given sequence of accesses (or access probabilities). Optimal BSTs are generally divided into two types: static and dynamic.

In computer science, multiway number partitioning is the problem of partitioning a multiset of numbers into a fixed number of subsets, such that the sums of the subsets are as similar as possible. It was first presented by Ronald Graham in 1969 in the context of the identical-machines scheduling problem. The problem is parametrized by a positive integer k, and called k-way number partitioning. The input to the problem is a multiset S of numbers, whose sum is k*T.

References