Repeated game

Last updated

In game theory, a repeated game (or iterated game) is an extensive form game that consists of a number of repetitions of some base game (called a stage game). The stage game is usually one of the well-studied 2-person games. Repeated games capture the idea that a player will have to take into account the impact of their current action on the future actions of other players; this impact is sometimes called their reputation. Single stage game or single shot game are names for non-repeated games.

Contents

For an example of a repeated game, consider two gas stations that are adjacent to one another. They compete by publicly posting pricing, and have the same and constant marginal cost c (the wholesale price of gasoline). Assume that when they both charge p = 10, their joint profit is maximized, resulting in a high profit for everyone. Despite the fact that this is the best outcome for them, they are motivated to deviate. By modestly lowering the price, either can steal all of their competitors' customers, nearly doubling their revenues. P = c, where their profit is zero, is the only price without this profit deviation. In other words, in the pricing competition game, the only Nash equilibrium is inefficient (for gas stations) that both charge p = c. This is more of a rule than an exception: in a staged game, the Nash equilibrium is the only result that an agent can consistently acquire in an interaction, and it is usually inefficient for them. This is because the agents are just concerned with their own personal interests, and do not care about the benefits or costs that their actions bring to competitors. On the other hand, gas stations make a profit even if there is another gas station adjacent. One of the most crucial reasons is that their interaction is not one-off. This condition is portrayed by repeated games, in which two gas stations compete for pricing (stage games) across an indefinite time range t = 0, 1, 2,....

Finitely vs infinitely repeated games

Repeated games may be broadly divided into two classes, finite and infinite, depending on how long the game is being played for.

Even if the game being played in each round is identical, repeating that game a finite or an infinite number of times can, in general, lead to very different outcomes (equilibria), as well as very different optimal strategies.

Infinitely repeated games

The most widely studied repeated games are games that are repeated an infinite number of times. In iterated prisoner's dilemma games, it is found that the preferred strategy is not to play a Nash strategy of the stage game, but to cooperate and play a socially optimum strategy. An essential part of strategies in infinitely repeated game is punishing players who deviate from this cooperative strategy. The punishment may be playing a strategy which leads to reduced payoff to both players for the rest of the game (called a trigger strategy). A player may normally choose to act selfishly to increase their own reward rather than play the socially optimum strategy. However, if it is known that the other player is following a trigger strategy, then the player expects to receive reduced payoffs in the future if they deviate at this stage. An effective trigger strategy ensures that cooperating has more utility to the player than acting selfishly now and facing the other player's punishment in the future.

There are many results in theorems which deal with how to achieve and maintain a socially optimal equilibrium in repeated games. These results are collectively called "Folk Theorems". An important feature of a repeated game is the way in which a player's preferences may be modelled. There are many different ways in which a preference relation may be modelled in an infinitely repeated game, but two key ones are :

For sufficiently patient players (e.g. those with high enough values of ), it can be proved that every strategy that has a payoff greater than the minmax payoff can be a Nash equilibrium - a very large set of strategies.

Finitely repeated games

Repeated games allow for the study of the interaction between immediate gains and long-term incentives. A finitely repeated game is a game in which the same one-shot stage game is played repeatedly over a number of discrete time periods, or rounds. Each time period is indexed by 0 < t ≤ T where T is the total number of periods. A player's final payoff is the sum of their payoffs from each round. [1]

For those repeated games with a fixed and known number of time periods, if the stage game has a unique Nash equilibrium, then the repeated game has a unique subgame perfect Nash equilibrium strategy profile of playing the stage game equilibrium in each round. This can be deduced through backward induction. The unique stage game Nash equilibrium must be played in the last round regardless of what happened in earlier rounds. Knowing this, players have no incentive to deviate from the unique stage game Nash equilibrium in the second-to-last round, and so on this logic is applied back to the first round of the game. [2] This ‘unravelling’ of a game from its endpoint can be observed in the Chainstore paradox.

If the stage game has more than one Nash equilibrium, the repeated game may have multiple subgame perfect Nash equilibria. While a Nash equilibrium must be played in the last round, the presence of multiple equilibria introduces the possibility of reward and punishment strategies that can be used to support deviation from stage game Nash equilibria in earlier rounds. [2]

Finitely repeated games with an unknown or indeterminate number of time periods, on the other hand, are regarded as if they were an infinitely repeated game. It is not possible to apply backward induction to these games.

Examples of cooperation in finitely repeated games

XYZ
A5 , 41, 12 , 5
B1, 13 , 21, 1

Example 1: Two-Stage Repeated Game with Multiple Nash Equilibria

Example 1 shows a two-stage repeated game with multiple pure strategy Nash equilibria. Because these equilibria differ markedly in terms of payoffs for Player 2, Player 1 can propose a strategy over multiple stages of the game that incorporates the possibility for punishment or reward for Player 2. For example, Player 1 might propose that they play (A, X) in the first round. If Player 2 complies in round one, Player 1 will reward them by playing the equilibrium (A, Z) in round two, yielding a total payoff over two rounds of (7, 9).

If Player 2 deviates to (A, Z) in round one instead of playing the agreed-upon (A, X), Player 1 can threaten to punish them by playing the (B, Y) equilibrium in round two. This latter situation yields payoff (5, 7), leaving both players worse off.

In this way, the threat of punishment in a future round incentivizes a collaborative, non-equilibrium strategy in the first round. Because the final round of any finitely repeated game, by its very nature, removes the threat of future punishment, the optimal strategy in the last round will always be one of the game's equilibria. It is the payoff differential between equilibria in the game represented in Example 1 that makes a punishment/reward strategy viable (for more on the influence of punishment and reward on game strategy, see 'Public Goods Game with Punishment and for Reward').

MNO
C5 , 41, 10, 5
D1, 13 , 21, 1

Example 2: Two-Stage Repeated Game with Unique Nash Equilibrium

Example 2 shows a two-stage repeated game with a unique Nash equilibrium. Because there is only one equilibrium here, there is no mechanism for either player to threaten punishment or promise reward in the game's second round. As such, the only strategy that can be supported as a subgame perfect Nash equilibrium is that of playing the game's unique Nash equilibrium strategy (D, N) every round. In this case, that means playing (D, N) each stage for two stages (n=2), but it would be true for any finite number of stages n. [3] To interpret: this result means that the very presence of a known, finite time horizon sabotages cooperation in every single round of the game. Cooperation in iterated games is only possible when the number of rounds is infinite or unknown.

Solving repeated games

In general, repeated games are easily solved using strategies provided by folk theorems. Complex repeated games can be solved using various techniques most of which rely heavily on linear algebra and the concepts expressed in fictitious play. It may be deducted that you can determine the characterization of equilibrium payoffs in infinitely repeated games. Through alternation between two payoffs, say a and f, the average payoff profile may be a weighted average between a and f.

Incomplete information

Repeated games can include some incomplete information. Repeated games with incomplete information were pioneered by Aumann and Maschler. [4] While it is easier to treat a situation where one player is informed and the other not, and when information received by each player is independent, it is possible to deal with zero-sum games with incomplete information on both sides and signals that are not independent. [5]

Related Research Articles

In game theory, the Nash equilibrium is the most commonly-used solution concept for non-cooperative games. A Nash equilibrium is a situation where no player could gain by changing their own strategy. The idea of Nash equilibrium dates back to the time of Cournot, who in 1838 applied it to his model of competition in an oligopoly.

In game theory, the centipede game, first introduced by Robert Rosenthal in 1981, is an extensive form game in which two players take turns choosing either to take a slightly larger share of an increasing pot, or to pass the pot to the other player. The payoffs are arranged so that if one passes the pot to one's opponent and the opponent takes the pot on the next round, one receives slightly less than if one had taken the pot on this round, but after an additional switch the potential payoff will be higher. Therefore, although at each round a player has an incentive to take the pot, it would be better for them to wait. Although the traditional centipede game had a limit of 100 rounds, any game with this structure but a different number of rounds is called a centipede game.

In game theory, a move, action, or play is any one of the options which a player can choose in a setting where the optimal outcome depends not only on their own actions but on the actions of others. The discipline mainly concerns the action of a player in a game affecting the behavior or actions of other players. Some examples of "games" include chess, bridge, poker, monopoly, diplomacy or battleship.

In game theory, grim trigger is a trigger strategy for a repeated game.

<span class="mw-page-title-main">Solution concept</span> Formal rule for predicting how a game will be played

In game theory, a solution concept is a formal rule for predicting how a game will be played. These predictions are called "solutions", and describe which strategies will be adopted by players and, therefore, the result of the game. The most commonly used solution concepts are equilibrium concepts, most famously Nash equilibrium.

In game theory, a Perfect Bayesian Equilibrium (PBE) is a solution with Bayesian probability to a turn-based game with incomplete information. More specifically, it is an equilibrium concept that uses Bayesian updating to describe player behavior in dynamic games with incomplete information. Perfect Bayesian equilibria are used to solve the outcome of games where players take turns but are unsure of the "type" of their opponent, which occurs when players don't know their opponent's preference between individual moves. A classic example of a dynamic game with types is a war game where the player is unsure whether their opponent is a risk-taking "hawk" type or a pacifistic "dove" type. Perfect Bayesian Equilibria are a refinement of Bayesian Nash equilibrium (BNE), which is a solution concept with Bayesian probability for non-turn-based games.

In game theory, a Bayesian game is a strategic decision-making model which assumes players have incomplete information. Players may hold private information relevant to the game, meaning that the payoffs are not common knowledge. Bayesian games model the outcome of player interactions using aspects of Bayesian probability. They are notable because they allowed, for the first time in game theory, for the specification of the solutions to games with incomplete information.

Backward induction is the process of determining a sequence of optimal choices by reasoning from the endpoint of a problem or situation back to its beginning using individual events or actions. Backward induction involves examining the final point in a series of decisions and identifying the optimal process or action required to arrive at that point. This process continues backward until the best action for every possible point along the sequence is determined. Backward induction was first utilized in 1875 by Arthur Cayley, who discovered the method while attempting to solve the secretary problem.

In game theory, a symmetric game is a game where the payoffs for playing a particular strategy depend only on the other strategies employed, not on who is playing them. If one can change the identities of the players without changing the payoff to the strategies, then a game is symmetric. Symmetry can come in different varieties. Ordinally symmetric games are games that are symmetric with respect to the ordinal structure of the payoffs. A game is quantitatively symmetric if and only if it is symmetric with respect to the exact payoffs. A partnership game is a symmetric game where both players receive identical payoffs for any strategy set. That is, the payoff for playing strategy a against strategy b receives the same payoff as playing strategy b against strategy a.

In game theory, trembling hand perfect equilibrium is a type of refinement of a Nash equilibrium that was first proposed by Reinhard Selten. A trembling hand perfect equilibrium is an equilibrium that takes the possibility of off-the-equilibrium play into account by assuming that the players, through a "slip of the hand" or tremble, may choose unintended strategies, albeit with negligible probability.

In game theory, folk theorems are a class of theorems describing an abundance of Nash equilibrium payoff profiles in repeated games. The original Folk Theorem concerned the payoffs of all the Nash equilibria of an infinitely repeated game. This result was called the Folk Theorem because it was widely known among game theorists in the 1950s, even though no one had published it. Friedman's (1971) Theorem concerns the payoffs of certain subgame-perfect Nash equilibria (SPE) of an infinitely repeated game, and so strengthens the original Folk Theorem by using a stronger equilibrium concept: subgame-perfect Nash equilibria rather than Nash equilibria.

In game theory, a correlated equilibrium is a solution concept that is more general than the well known Nash equilibrium. It was first discussed by mathematician Robert Aumann in 1974. The idea is that each player chooses their action according to their private observation of the value of the same public signal. A strategy assigns an action to every possible observation a player can make. If no player would want to deviate from their strategy, the distribution from which the signals are drawn is called a correlated equilibrium.

In game theory, a subgame perfect equilibrium is a refinement of a Nash equilibrium used in dynamic games. A strategy profile is a subgame perfect equilibrium if it represents a Nash equilibrium of every subgame of the original game. Informally, this means that at any point in the game, the players' behavior from that point onward should represent a Nash equilibrium of the continuation game, no matter what happened before. Every finite extensive game with perfect recall has a subgame perfect equilibrium. Perfect recall is a term introduced by Harold W. Kuhn in 1953 and "equivalent to the assertion that each player is allowed by the rules of the game to remember everything he knew at previous moves and all of his choices at those moves".

Equilibrium selection is a concept from game theory which seeks to address reasons for players of a game to select a certain equilibrium over another. The concept is especially relevant in evolutionary game theory, where the different methods of equilibrium selection respond to different ideas of what equilibria will be stable and persistent for one player to play even in the face of deviations of the other players. This is important because there are various equilibrium concepts, and for many particular concepts, such as the Nash equilibrium, many games have multiple equilibria.

In game theory, a game is said to be a potential game if the incentive of all players to change their strategy can be expressed using a single global function called the potential function. The concept originated in a 1996 paper by Dov Monderer and Lloyd Shapley.

In game theory, an epsilon-equilibrium, or near-Nash equilibrium, is a strategy profile that approximately satisfies the condition of Nash equilibrium. In a Nash equilibrium, no player has an incentive to change his behavior. In an approximate Nash equilibrium, this requirement is weakened to allow the possibility that a player may have a small incentive to do something different. This may still be considered an adequate solution concept, assuming for example status quo bias. This solution concept may be preferred to Nash equilibrium due to being easier to compute, or alternatively due to the possibility that in games of more than 2 players, the probabilities involved in an exact Nash equilibrium need not be rational numbers.

In game theory, a stochastic game, introduced by Lloyd Shapley in the early 1950s, is a repeated game with probabilistic transitions played by one or more players. The game is played in a sequence of stages. At the beginning of each stage the game is in some state. The players select actions and each player receives a payoff that depends on the current state and the chosen actions. The game then moves to a new random state whose distribution depends on the previous state and the actions chosen by the players. The procedure is repeated at the new state and play continues for a finite or infinite number of stages. The total payoff to a player is often taken to be the discounted sum of the stage payoffs or the limit inferior of the averages of the stage payoffs.

<span class="mw-page-title-main">Jean-François Mertens</span> Belgian game theorist (1946–2012)

Jean-François Mertens was a Belgian game theorist and mathematical economist.

The one-shot deviation principle is the principle of optimality of dynamic programming applied to game theory. It says that a strategy profile of a finite multi-stage extensive-form game with observed actions is a subgame perfect equilibrium (SPE) if and only if there exist no profitable single deviation for each subgame and every player. In simpler terms, if no player can increase their expected payoff by deviating from their original strategy via a single action, then the strategy profile is an SPE. In other words, no player can profit by deviating from the strategy in one period and then reverting to the strategy.

The Berge equilibrium is a game theory solution concept named after the mathematician Claude Berge. It is similar to the standard Nash equilibrium, except that it aims to capture a type of altruism rather than purely non-cooperative play. Whereas a Nash equilibrium is a situation in which each player of a strategic game ensures that they personally will receive the highest payoff given other players' strategies, in a Berge equilibrium every player ensures that all other players will receive the highest payoff possible. Although Berge introduced the intuition for this equilibrium notion in 1957, it was only formally defined by Vladislav Iosifovich Zhukovskii in 1985, and it was not in widespread use until half a century after Berge originally developed it.

References

  1. Knight, Vince. "Finitely Repeated Games". Game Theory. Retrieved 6 December 2017.
  2. 1 2 Benoit, J.P. & Krishna, V. (1985). "Finitely Repeated Games". Econometrica. 53 (4): 905–922. doi:10.2307/1912660. JSTOR   1912660.{{cite journal}}: CS1 maint: multiple names: authors list (link)
  3. Levin, Jonathan (May 2006). ""Repeated Games I: Perfect Monitoring"" (PDF). www.stanford.edu. Retrieved 12 December 2017.
  4. Aumann, R. J.; Maschler, M. (1995). Repeated Games with Incomplete Information. Cambridge London: MIT Press. ISBN   9780262011471.
  5. Mertens, J.-F. (1987). "Repeated Games". Proceedings of the International Congress of Mathematicians, Berkeley 1986. Providence: American Mathematical Society. pp. 1528–1577. ISBN   0-8218-0110-4.