Equilibrium selection

Last updated

Equilibrium selection is a concept from game theory which seeks to address reasons for players of a game to select a certain equilibrium over another. The concept is especially relevant in evolutionary game theory, where the different methods of equilibrium selection respond to different ideas of what equilibria will be stable and persistent for one player to play even in the face of deviations (and mutations) of the other players. This is important because there are various equilibrium concepts, and for many particular concepts, such as the Nash equilibrium, many games have multiple equilibria.

Contents

Equilibrium Selection with Repeated Games

A stage game is an n-player game where players choose from a finite set of actions, and there is a payoff profile for their choices. A repeated game is playing a number of repetitions of a stage game in discrete periods of time (Watson, 2013). A player's reputation affects the actions and behavior of the other players. In other words, how a player behaves in preceding rounds determines the actions of their opponents in subsequent rounds. An example is the interaction between an employee and an employer where the employee shirks their responsibility for a short-term gain then loses on the bonus that the employer discontinues after observing the employee's behavior (Watson, 2013). The dynamics of equilibrium selection for repeated games can be illustrated with a two-period game. With every action from the players in one period, a new subgame is initiated based on that action profile. For the Nash Equilibrium of the entire game, a subgame perfect equilibrium of every game is required. Hence, in the last period of a repeated game, the players will choose a stage game Nash Equilibrium. Equilibria stipulation actions that are not Nash Equilibrium strategies in the stage game are still supported. This can be achieved by establishing a reputation of "cooperation" in the preceding periods that leads to the opponent selecting a more favorable Nash Equilibrium strategy in the final period. If a player builds a reputation of deviating instead of co-operating, then the opponent can "punish" the player by choosing a less favorable Equilibrium in the final period of the repeated game.

Focal point

Another concept that can help to select an equilibrium is focal point. This concept was first introduced by Thomas Schelling, a Nobel-winning game theorist, in his book The Strategy of Conflict in 1960 (Schelling, 1960). When the participants are in a coordinate game where the players do not have a chance to discuss their strategies beforehand, focal point is a solution that somehow stands out as the natural answer.  

For example, in an experiment conducted in 1990 by Mehta et al. (1994), the researchers let the participants answer a questionnaire, which contained the question "name a year" or "name a city in England". The participants were asked to provide the first answer that came to their minds, and many provided their birth year or hometown city.

However, when they had the incentive to coordinate - the participants were told they would be paid if they managed to answer the question the same way as an anonymous partner - most of them chose 1990 (the year at the time) and London (the largest city in the UK). These are not the first answers that came to their minds, but they are the best bets after deliberation while trying to find a partner without prior discussion. In this case, the year 1990, or the city of London, is the focal point of this game, to help the players to select the best equilibrium in this coordinate game.

Besides, even in the situation where the game players are allowed to communicate with each other, such as negotiation, focal point can still be useful for them selecting an appropriate equilibrium: When the negotiation is about to the end, each player must make the last-minute decision about how aggressive they should be and to what extent they should trust their opponents (Hyde, 2017).

Symmetries

Various authors have studied games with symmetries (over players or actions), especially in the case of fully cooperative games. For example, the team game Hanabi has been considered, in which the different players and suits are symmetric. Relatedly, some authors have considered how equilibrium selection relates between isomorphic games. Generally, it has been argued that a group of players shouldn't choose strategies that arbitrarily break these symmetries, i.e., should play symmetric strategies to aim for symmetric equilibria. [1] Similarly they should play isomorphic games isomorphically. [2] For example, in Hanabi, hints corresponding to the different suits should have analogous (symmetric) meanings. In team games, the optimal symmetric strategy is also a (symmetric) Nash equilibrium. [3] While breaking symmetries may allow for higher utilities, it results in unnatural, inhuman strategies. [1] Every finite symmetric game (including non-team games) has a symmetric Nash equilibrium. [4]

Examples of equilibrium selection concepts

Risk & Payoff dominance

Definition: Consider a situation where a game has multiple Nash equilibria (NE), the equilibrium can be classified into two categories:

Explanation: A risk dominant NE is chosen when the player wants to avoid big losses while a payoff dominant NE is considered for an optimal payoff solution. Note that either the risk dominant or payoff dominant is a typical type of NE.

Example: Take the normal form payoff matrix of a game as an example:

Table1: Normal form of the example game
LR
U10, 100, 9
D9, 05, 5

There are two NEs in this game, i.e. (U, L) and (D,R). Here (U, L) is a payoff dominant NE since such a strategy can return an optimal overall payoff. However, given the uncertainty of an opponent's action, one of the players may consider a more conservative strategy (D for player 1 and R for player two), which can avoid the situation of "a great loss", i.e. getting 0 payoff once the opponent deviates. Hence the (D, R) is a risk dominant NE.

1/2 dominance

See also

Related Research Articles

An evolutionarily stable strategy (ESS) is a strategy that is impermeable when adopted by a population in adaptation to a specific environment, that is to say it cannot be displaced by an alternative strategy which may be novel or initially rare. Introduced by John Maynard Smith and George R. Price in 1972/3, it is an important concept in behavioural ecology, evolutionary psychology, mathematical game theory and economics, with applications in other fields such as anthropology, philosophy and political science.

In game theory, the Nash equilibrium is the most commonly-used solution concept for non-cooperative games. A Nash equilibrium is a situation where no player could gain by changing their own strategy. The idea of Nash equilibrium dates back to the time of Cournot, who in 1838 applied it to his model of competition in an oligopoly.

In game theory, the best response is the strategy which produces the most favorable outcome for a player, taking other players' strategies as given. The concept of a best response is central to John Nash's best-known contribution, the Nash equilibrium, the point at which each player in a game has selected the best response to the other players' strategies.

A coordination game is a type of simultaneous game found in game theory. It describes the situation where a player will earn a higher payoff when they select the same course of action as another player. The game is not one of pure conflict, which results in multiple pure strategy Nash equilibria in which players choose matching strategies. Figure 1 shows a 2-player example.

In game theory, the centipede game, first introduced by Robert Rosenthal in 1981, is an extensive form game in which two players take turns choosing either to take a slightly larger share of an increasing pot, or to pass the pot to the other player. The payoffs are arranged so that if one passes the pot to one's opponent and the opponent takes the pot on the next round, one receives slightly less than if one had taken the pot on this round, but after an additional switch the potential payoff will be higher. Therefore, although at each round a player has an incentive to take the pot, it would be better for them to wait. Although the traditional centipede game had a limit of 100 rounds, any game with this structure but a different number of rounds is called a centipede game.

In game theory, a move, action, or play is any one of the options which a player can choose in a setting where the optimal outcome depends not only on their own actions but on the actions of others. The discipline mainly concerns the action of a player in a game affecting the behavior or actions of other players. Some examples of "games" include chess, bridge, poker, monopoly, diplomacy or battleship.

<span class="mw-page-title-main">Solution concept</span> Formal rule for predicting how a game will be played

In game theory, a solution concept is a formal rule for predicting how a game will be played. These predictions are called "solutions", and describe which strategies will be adopted by players and, therefore, the result of the game. The most commonly used solution concepts are equilibrium concepts, most famously Nash equilibrium.

In game theory, a Perfect Bayesian Equilibrium (PBE) is a solution with Bayesian probability to a turn-based game with incomplete information. More specifically, it is an equilibrium concept that uses Bayesian updating to describe player behavior in dynamic games with incomplete information. Perfect Bayesian equilibria are used to solve the outcome of games where players take turns but are unsure of the "type" of their opponent, which occurs when players don't know their opponent's preference between individual moves. A classic example of a dynamic game with types is a war game where the player is unsure whether their opponent is a risk-taking "hawk" type or a pacifistic "dove" type. Perfect Bayesian Equilibria are a refinement of Bayesian Nash equilibrium (BNE), which is a solution concept with Bayesian probability for non-turn-based games.

Backward induction is the process of determining a sequence of optimal choices by reasoning from the endpoint of a problem or situation back to its beginning using individual events or actions. Backward induction involves examining the final point in a series of decisions and identifying the optimal process or action required to arrive at that point. This process continues backward until the best action for every possible point along the sequence is determined. Backward induction was first utilized in 1875 by Arthur Cayley, who discovered the method while attempting to solve the secretary problem.

In game theory, a dominant strategy is a strategy that is better than any other strategy for one player, no matter how that player's opponent will play. Some very simple games can be solved using dominance.

Rationalizability is a solution concept in game theory. It is the most permissive possible solution concept that still requires both players to be at least somewhat rational and know the other players are also somewhat rational, i.e. that they do not play dominated strategies. A strategy is rationalizable if there exists some possible set of beliefs both players could have about each other's actions, that would still result in the strategy being played.

In game theory, folk theorems are a class of theorems describing an abundance of Nash equilibrium payoff profiles in repeated games. The original Folk Theorem concerned the payoffs of all the Nash equilibria of an infinitely repeated game. This result was called the Folk Theorem because it was widely known among game theorists in the 1950s, even though no one had published it. Friedman's (1971) Theorem concerns the payoffs of certain subgame-perfect Nash equilibria (SPE) of an infinitely repeated game, and so strengthens the original Folk Theorem by using a stronger equilibrium concept: subgame-perfect Nash equilibria rather than Nash equilibria.

In game theory, a repeated game is an extensive form game that consists of a number of repetitions of some base game. The stage game is usually one of the well-studied 2-person games. Repeated games capture the idea that a player will have to take into account the impact of their current action on the future actions of other players; this impact is sometimes called their reputation. Single stage game or single shot game are names for non-repeated games.

In game theory, a subgame perfect equilibrium is a refinement of a Nash equilibrium used in dynamic games. A strategy profile is a subgame perfect equilibrium if it represents a Nash equilibrium of every subgame of the original game. Informally, this means that at any point in the game, the players' behavior from that point onward should represent a Nash equilibrium of the continuation game, no matter what happened before. Every finite extensive game with perfect recall has a subgame perfect equilibrium. Perfect recall is a term introduced by Harold W. Kuhn in 1953 and "equivalent to the assertion that each player is allowed by the rules of the game to remember everything he knew at previous moves and all of his choices at those moves".

Risk dominance and payoff dominance are two related refinements of the Nash equilibrium (NE) solution concept in game theory, defined by John Harsanyi and Reinhard Selten. A Nash equilibrium is considered payoff dominant if it is Pareto superior to all other Nash equilibria in the game.1 When faced with a choice among equilibria, all players would agree on the payoff dominant equilibrium since it offers to each player at least as much payoff as the other Nash equilibria. Conversely, a Nash equilibrium is considered risk dominant if it has the largest basin of attraction. This implies that the more uncertainty players have about the actions of the other player(s), the more likely they will choose the strategy corresponding to it.

<span class="mw-page-title-main">Non-credible threat</span>

A non-credible threat is a term used in game theory and economics to describe a threat in a sequential game that a rational player would not actually carry out, because it would not be in his best interest to do so.

<span class="mw-page-title-main">Simultaneous game</span>

In game theory, a simultaneous game or static game is a game where each player chooses their action without knowledge of the actions chosen by other players. Simultaneous games contrast with sequential games, which are played by the players taking turns. In other words, both players normally act at the same time in a simultaneous game. Even if the players do not act at the same time, both players are uninformed of each other's move while making their decisions. Normal form representations are usually used for simultaneous games. Given a continuous game, players will have different information sets if the game is simultaneous than if it is sequential because they have less information to act on at each step in the game. For example, in a two player continuous game that is sequential, the second player can act in response to the action taken by the first player. However, this is not possible in a simultaneous game where both players act at the same time.

A Markov perfect equilibrium is an equilibrium concept in game theory. It has been used in analyses of industrial organization, macroeconomics, and political economy. It is a refinement of the concept of subgame perfect equilibrium to extensive form games for which a pay-off relevant state space can be identified. The term appeared in publications starting about 1988 in the work of economists Jean Tirole and Eric Maskin.

In game theory, Mertens stability is a solution concept used to predict the outcome of a non-cooperative game. A tentative definition of stability was proposed by Elon Kohlberg and Jean-François Mertens for games with finite numbers of players and strategies. Later, Mertens proposed a stronger definition that was elaborated further by Srihari Govindan and Mertens. This solution concept is now called Mertens stability, or just stability.

In game theory, a multi-stage game is a sequence of several simultaneous games played one after the other. This is a generalization of a repeated game: a repeated game is a special case of a multi-stage game, in which the stage games are identical.

References

  1. 1 2 Hu, Hengyuan; Lerer, Adam; Peysakhovich, Alex; Foerster, Jakob (2020). "'Other-Play' for Zero-Shot Coordination". Proceedings of the 37th International Conference on Machine Learning . arXiv: 2003.02979 .
  2. Harsanyi, John C.; Selten, Reinhard (1988). A General Theory of Equilibrium Selection. MIT Press.
  3. Emmons, Scott; Oesterheld, Caspar; Critch, Andrew; Conitzer, Vincent; Russell, Stuart (2022). "For Learning in Symmetric Teams, Local Optima are Global Nash Equilibria". Proceedings of the 39th International Conference on Machine Learning . pp. 5924–5943.
  4. Nash, John (1951). "Non-cooperative games". Annals of Mathematics. 54 (2): 286–295. doi:10.2307/1969529. JSTOR   1969529.