Subgame perfect equilibrium

Last updated
Subgame Perfect Equilibrium
A solution concept in game theory
Relationship
Subset of Nash equilibrium
Intersects with Evolutionarily stable strategy
Significance
Proposed by Reinhard Selten
Used for Extensive form games
Example Ultimatum game

In game theory, a subgame perfect equilibrium (or subgame perfect Nash equilibrium) is a refinement of a Nash equilibrium used in dynamic games. A strategy profile is a subgame perfect equilibrium if it represents a Nash equilibrium of every subgame of the original game. Informally, this means that at any point in the game, the players' behavior from that point onward should represent a Nash equilibrium of the continuation game (i.e. of the subgame), no matter what happened before. Every finite extensive game with perfect recall has a subgame perfect equilibrium. [1] Perfect recall is a term introduced by Harold W. Kuhn in 1953 and "equivalent to the assertion that each player is allowed by the rules of the game to remember everything he knew at previous moves and all of his choices at those moves". [2]

Contents

A common method for determining subgame perfect equilibria in the case of a finite game is backward induction. Here one first considers the last actions of the game and determines which actions the final mover should take in each possible circumstance to maximize his/her utility. One then supposes that the last actor will do these actions, and considers the second to last actions, again choosing those that maximize that actor's utility. This process continues until one reaches the first move of the game. The strategies which remain are the set of all subgame perfect equilibria for finite-horizon extensive games of perfect information. [1] However, backward induction cannot be applied to games of imperfect or incomplete information because this entails cutting through non-singleton information sets.

A subgame perfect equilibrium necessarily satisfies the one-shot deviation principle.

The set of subgame perfect equilibria for a given game is always a subset of the set of Nash equilibria for that game. In some cases the sets can be identical.

The ultimatum game provides an intuitive example of a game with fewer subgame perfect equilibria than Nash equilibria.

Example

Determining the subgame perfect equilibrium by using backward induction is shown below in Figure 1. Strategies for Player 1 are given by {Up, Uq, Dp, Dq}, whereas Player 2 has the strategies among {TL, TR, BL, BR}. There are 4 subgames in this example, with 3 proper subgames.

Figure 1 Backwards Induction Example 2.png
Figure 1

Using the backward induction, the players will take the following actions for each subgame:

Thus, the subgame perfect equilibrium is {Dp, TL} with the payoff (3, 3).

An extensive-form game with incomplete information is presented below in Figure 2. Note that the node for Player 1 with actions A and B, and all succeeding actions is a subgame. Player 2's nodes are not a subgame as they are part of the same information set.

Figure 2 A Game of imperfect information with subgames shown..svg
Figure 2

The first normal-form game is the normal form representation of the whole extensive-form game. Based on the provided information, (UA, X), (DA, Y), and (DB, Y) are all Nash equilibria for the entire game.

The second normal-form game is the normal form representation of the subgame starting from Player 1's second node with actions A and B. For the second normal-form game, the Nash equilibrium of the subgame is (A, X).

For the entire game Nash equilibria (DA, Y) and (DB, Y) are not subgame perfect equilibria because the move of Player 2 does not constitute a Nash Equilibrium. The Nash equilibrium (UA, X) is subgame perfect because it incorporates the subgame Nash equilibrium (A, X) as part of its strategy. [3]

To solve this game, first find the Nash Equilibria by mutual best response of Subgame 1. Then use backwards induction and plug in (A,X) → (3,4) so that (3,4) become the payoffs for Subgame 2. [3]

The dashed line indicates that player 2 does not know whether player 1 will play A or B in a simultaneous game.

Subgame 1 is solved and (3,4) replaces all of Subgame 1 and player one will choose U -> (3,4)Solution for Subgame 1 Subgame 1 solved.svg
Subgame 1 is solved and (3,4) replaces all of Subgame 1 and player one will choose U -> (3,4)Solution for Subgame 1

Player 1 chooses U rather than D because 3 > 2 for Player 1's payoff. The resulting equilibrium is (A, X) → (3,4).

Solution of Subgame Perfect Equilibrium Subgame-Perfect-Solution.svg
Solution of Subgame Perfect Equilibrium

Thus, the subgame perfect equilibrium through backwards induction is (UA, X) with the payoff (3, 4).

Repeated games

For finitely repeated games, if a stage game has only one unique Nash equilibrium, the subgame perfect equilibrium is to play without considering past actions, treating the current subgame as a one-shot game. An example of this is a finitely repeated Prisoner's dilemma game. The Prisoner's dilemma gets its name from a situation that contains two guilty culprits. When they are interrogated, they have the option to stay quiet or defect. If both culprits stay quiet, they both serve a short sentence. If both defect, they both serve a moderate sentence. If they choose opposite options, then the culprit that defects is free and the culprit who stays quiet serves a long sentence. Ultimately, using backward induction, the last subgame in a finitely repeated Prisoner's dilemma requires players to play the unique Nash equilibrium (both players defecting). Because of this, all games prior to the last subgame will also play the Nash equilibrium to maximize their single-period payoffs. [4] If a stage-game in a finitely repeated game has multiple Nash equilibria, subgame perfect equilibria can be constructed to play non-stage-game Nash equilibrium actions, through a "carrot and stick" structure. One player can use the one stage-game Nash equilibrium to incentivize playing the non-Nash equilibrium action, while using a stage-game Nash equilibrium with lower payoff to the other player if they choose to defect. [5]

Finding subgame-perfect equilibria

One game in which the backward induction solution is well known is tic-tac-toe Tic tac toe.svg
One game in which the backward induction solution is well known is tic-tac-toe

Reinhard Selten proved that any game which can be broken into "sub-games" containing a sub-set of all the available choices in the main game will have a subgame perfect Nash Equilibrium strategy (possibly as a mixed strategy giving non-deterministic sub-game decisions). Subgame perfection is only used with games of complete information. Subgame perfection can be used with extensive form games of complete but imperfect information.

The subgame-perfect Nash equilibrium is normally deduced by "backward induction" from the various ultimate outcomes of the game, eliminating branches which would involve any player making a move that is not credible (because it is not optimal) from that node. One game in which the backward induction solution is well known is tic-tac-toe, but in theory even Go has such an optimum strategy for all players. The problem of the relationship between subgame perfection and backward induction was settled by Kaminski (2019), who proved that a generalized procedure of backward induction produces all subgame perfect equilibria in games that may have infinite length, infinite actions as each information set, and imperfect information if a condition of final support is satisfied.

The interesting aspect of the word "credible" in the preceding paragraph is that taken as a whole (disregarding the irreversibility of reaching sub-games) strategies exist which are superior to subgame perfect strategies, but which are not credible in the sense that a threat to carry them out will harm the player making the threat and prevent that combination of strategies. For instance in the game of "chicken" if one player has the option of ripping the steering wheel from their car they should always take it because it leads to a "sub game" in which their rational opponent is precluded from doing the same thing (and killing them both). The wheel-ripper will always win the game (making his opponent swerve away), and the opponent's threat to suicidally follow suit is not credible.

See also

Related Research Articles

In game theory, the Nash equilibrium, named after the mathematician John Nash, is the most common way to define the solution of a non-cooperative game involving two or more players. In a Nash equilibrium, each player is assumed to know the equilibrium strategies of the other players, and no one has anything to gain by changing only one's own strategy. The principle of Nash equilibrium dates back to the time of Cournot, who in 1838 applied it to competing firms choosing outputs.

In game theory, the centipede game, first introduced by Robert Rosenthal in 1981, is an extensive form game in which two players take turns choosing either to take a slightly larger share of an increasing pot, or to pass the pot to the other player. The payoffs are arranged so that if one passes the pot to one's opponent and the opponent takes the pot on the next round, one receives slightly less than if one had taken the pot on this round, but after an additional switch the potential payoff will be higher. Therefore, although at each round a player has an incentive to take the pot, it would be better for them to wait. Although the traditional centipede game had a limit of 100 rounds, any game with this structure but a different number of rounds is called a centipede game.

In game theory, a subgame is any part of a game that meets the following criteria :

  1. It has a single initial node that is the only member of that node's information set.
  2. If a node is contained in the subgame then so are all of its successors.
  3. If a node in a particular information set is in the subgame then all members of that information set belong to the subgame.

In game theory, a player's strategy is any of the options which they choose in a setting where the outcome depends not only on their own actions but on the actions of others. The discipline mainly concerns the action of a player in a game affecting the behavior or actions of other players. Some examples of "games" include chess, bridge, poker, monopoly, diplomacy or battleship. A player's strategy will determine the action which the player will take at any stage of the game. In studying game theory, economists enlist a more rational lens in analyzing decisions rather than the psychological or sociological perspectives taken when analyzing relationships between decisions of two or more parties in different disciplines.

<span class="mw-page-title-main">Solution concept</span> Formal rule for predicting how a game will be played

In game theory, a solution concept is a formal rule for predicting how a game will be played. These predictions are called "solutions", and describe which strategies will be adopted by players and, therefore, the result of the game. The most commonly used solution concepts are equilibrium concepts, most famously Nash equilibrium.

In game theory, an extensive-form game is a specification of a game allowing for the explicit representation of a number of key aspects, like the sequencing of players' possible moves, their choices at every decision point, the information each player has about the other player's moves when they make a decision, and their payoffs for all possible game outcomes. Extensive-form games also allow for the representation of incomplete information in the form of chance events modeled as "moves by nature". Extensive-form representations differ from normal-form in that they provide a more complete description of the game in question, whereas normal-form simply boils down the game into a payoff matrix.

In economics and game theory, complete information is an economic situation or game in which knowledge about other market participants or players is available to all participants. The utility functions, payoffs, strategies and "types" of players are thus common knowledge. Complete information is the concept that each player in the game is aware of the sequence, strategies, and payoffs throughout gameplay. Given this information, the players have the ability to plan accordingly based on the information to maximize their own strategies and utility at the end of the game.

In game theory, a Perfect Bayesian Equilibrium (PBE) is an equilibrium concept relevant for dynamic games with incomplete information. It is a refinement of Bayesian Nash equilibrium (BNE). A perfect Bayesian equilibrium has two components -- strategies and beliefs:

In game theory, a Bayesian game is a strategic decision-making model which assumes players have incomplete information. Players hold private information relevant to the game, meaning that the payoffs are not common knowledge. Bayesian games model the outcome of player interactions using aspects of Bayesian probability. They are notable because they allowed, for the first time in game theory, for the specification of the solutions to games with incomplete information.

Backward induction is the process of reasoning backwards in time, from the end of a problem or situation, to determine a sequence of optimal actions. It proceeds by examining the last point at which a decision is to be made and then identifying what action would be most optimal at that moment. Using this information, one can then determine what to do at the second-to-last time of decision. This process continues backwards until one has determined the best action for every possible situation at every point in time. Backward induction was first used in 1875 by Arthur Cayley, who uncovered the method while trying to solve the infamous Secretary problem.

In game theory, trembling hand perfect equilibrium is a type of refinement of a Nash equilibrium that was first proposed by Reinhard Selten. A trembling hand perfect equilibrium is an equilibrium that takes the possibility of off-the-equilibrium play into account by assuming that the players, through a "slip of the hand" or tremble, may choose unintended strategies, albeit with negligible probability.

In game theory, folk theorems are a class of theorems describing an abundance of Nash equilibrium payoff profiles in repeated games. The original Folk Theorem concerned the payoffs of all the Nash equilibria of an infinitely repeated game. This result was called the Folk Theorem because it was widely known among game theorists in the 1950s, even though no one had published it. Friedman's (1971) Theorem concerns the payoffs of certain subgame-perfect Nash equilibria (SPE) of an infinitely repeated game, and so strengthens the original Folk Theorem by using a stronger equilibrium concept: subgame-perfect Nash equilibria rather than Nash equilibria.

In game theory, a repeated game is an extensive form game that consists of a number of repetitions of some base game. The stage game is usually one of the well-studied 2-person games. Repeated games capture the idea that a player will have to take into account the impact of his or her current action on the future actions of other players; this impact is sometimes called his or her reputation. Single stage game or single shot game are names for non-repeated games.

Sequential equilibrium is a refinement of Nash Equilibrium for extensive form games due to David M. Kreps and Robert Wilson. A sequential equilibrium specifies not only a strategy for each of the players but also a belief for each of the players. A belief gives, for each information set of the game belonging to the player, a probability distribution on the nodes in the information set. A profile of strategies and beliefs is called an assessment for the game. Informally speaking, an assessment is a perfect Bayesian equilibrium if its strategies are sensible given its beliefs and its beliefs are confirmed on the outcome path given by its strategies. The definition of sequential equilibrium further requires that there be arbitrarily small perturbations of beliefs and associated strategies with the same property.

In game theory, a Manipulated Nash equilibrium or MAPNASH is a refinement of subgame perfect equilibrium used in dynamic games of imperfect information. Informally, a strategy set is a MAPNASH of a game if it would be a subgame perfect equilibrium of the game if the game had perfect information. MAPNASH were first suggested by Amershi, Sadanand, and Sadanand (1988) and has been discussed in several papers since. It is a solution concept based on how players think about other players' thought processes.

<span class="mw-page-title-main">Non-credible threat</span>

A non-credible threat is a term used in game theory and economics to describe a threat in a sequential game that a rational player would not actually carry out, because it would not be in his best interest to do so.

A Markov perfect equilibrium is an equilibrium concept in game theory. It has been used in analyses of industrial organization, macroeconomics, and political economy. It is a refinement of the concept of subgame perfect equilibrium to extensive form games for which a pay-off relevant state space can be identified. The term appeared in publications starting about 1988 in the work of economists Jean Tirole and Eric Maskin.

<span class="mw-page-title-main">Jean-François Mertens</span> Belgian game theorist (1946–2012)

Jean-François Mertens was a Belgian game theorist and mathematical economist.

In game theory, Mertens stability is a solution concept used to predict the outcome of a non-cooperative game. A tentative definition of stability was proposed by Elon Kohlberg and Jean-François Mertens for games with finite numbers of players and strategies. Later, Mertens proposed a stronger definition that was elaborated further by Srihari Govindan and Mertens. This solution concept is now called Mertens stability, or just stability.

The one-shot deviation principle is the principle of optimality of dynamic programming applied to game theory. It says that a strategy profile of a finite multi-stage extensive-form game with observed actions is a subgame perfect equilibrium (SPE) if and only if there exist no profitable single deviation for each subgame and every player. In simpler terms, if no player can increase their expected payoff by deviating from their original strategy via a single action, then the strategy profile is an SPE. In other words, no player can profit by deviating from the strategy in one period and then reverting to the strategy.

References

  1. 1 2 Osborne, M. J. (2004). An Introduction to Game Theory. Oxford University Press.
  2. Kuhn, Harold William; Tucker, Albert William (2 March 2016). Contributions to the Theory of Games (AM-28), Volume II. Princeton University Press. ISBN   978-1-4008-8197-0.
  3. 1 2 Joel., Watson (2013-05-09). Strategy : an introduction to game theory (Third ed.). New York. ISBN   9780393918380. OCLC   842323069.
  4. Yildiz, Muhamet (2012). "12 Repeated Games". 14.12 Economic Applications of Game Theory. Massachusetts Institute of Technology: MIT OpenCourseWare. Retrieved April 27, 2021.
  5. Takako, Fujiwara-Greve (27 June 2015). Non-cooperative game theory. Tokyo. ISBN   9784431556442. OCLC   911616270.