Equilibrium selection

Last updated

Equilibrium selection is a concept from game theory which seeks to address reasons for players of a game to select a certain equilibrium over another. The concept is especially relevant in evolutionary game theory, where the different methods of equilibrium selection respond to different ideas of what equilibria will be stable and persistent for one player to play even in the face of deviations (and mutations) of the other players. This is important because there are various equilibrium concepts, and for many particular concepts, such as the Nash equilibrium, many games have multiple equilibria.

Contents

Equilibrium Selection with Repeated Games

A stage game is an n-player game where players choose from a finite set of actions, and there is a payoff profile for their choices. A repeated game is playing a number of repetitions of a stage game in discrete periods of time (Watson, 2013). A player's reputation affects the actions and behavior of the other players. In other words, how a player behaves in preceding rounds determines the actions of their opponents in subsequent rounds. An example is the interaction between an employee and an employer where the employee shirks their responsibility for a short-term gain then loses on the bonus that the employer discontinues after observing the employee's behavior (Watson, 2013). The dynamics of equilibrium selection for repeated games can be illustrated with a two-period game. With every action from the players in one period, a new subgame is initiated based on that action profile. For the Nash Equilibrium of the entire game, a subgame perfect equilibrium of every game is required. Hence, in the last period of a repeated game, the players will choose a stage game Nash Equilibrium. Equilibria stipulation actions that are not Nash Equilibrium strategies in the stage game are still supported. This can be achieved by establishing a reputation of "cooperation" in the preceding periods that leads to the opponent selecting a more favorable Nash Equilibrium strategy in the final period. If a player builds a reputation of deviating instead of co-operating, then the opponent can "punish" the player by choosing a less favorable Equilibrium in the final period of the repeated game.

Focal point

Another concept that can help to select an equilibrium is focal point. This concept was first introduced by Thomas Schelling, a Nobel-winning game theorist, in his book The Strategy of Conflict in 1960 (Schelling, 1960). When the participants are in a coordinate game where the players do not have a chance to discuss their strategies beforehand, focal point is a solution that somehow stands out as the natural answer.  

For example, in an experiment conducted in 1990 by Mehta et al. (1994), the researchers let the participants answer a questionnaire, which contained the question "name a year" or "name a city in England". The participants were asked to provide the first answer that came to their minds, and many provided their birth year or hometown city.

However, when they had the incentive to coordinate - the participants were told they would be paid if they managed to answer the question the same way as an anonymous partner - most of them chose 1990 (the year at the time) and London (the largest city in the UK). These are not the first answers that came to their minds, but they are the best bets after deliberation while trying to find a partner without prior discussion. In this case, the year 1990, or the city of London, is the focal point of this game, to help the players to select the best equilibrium in this coordinate game.

Besides, even in the situation where the game players are allowed to communicate with each other, such as negotiation, focal point can still be useful for them selecting an appropriate equilibrium: When the negotiation is about to the end, each player must make the last-minute decision about how aggressive they should be and to what extent they should trust their opponents (Hyde, 2017).

Symmetries

Various authors have studied games with symmetries (over players or actions), especially in the case of fully cooperative games. For example, the team game Hanabi has been considered, in which the different players and suits are symmetric. Relatedly, some authors have considered how equilibrium selection relates between isomorphic games. Generally, it has been argued that a group of players shouldn't choose strategies that arbitrarily break these symmetries, i.e., should play symmetric strategies to aim for symmetric equilibria. [1] Similarly they should play isomorphic games isomorphically. [2] For example, in Hanabi, hints corresponding to the different suits should have analogous (symmetric) meanings. In team games, the optimal symmetric strategy is also a (symmetric) Nash equilibrium. [3] While breaking symmetries may allow for higher utilities, it results in unnatural, inhuman strategies. [1] Every finite symmetric game (including non-team games) has a symmetric Nash equilibrium. [4]

Examples of equilibrium selection concepts

Risk & Payoff dominance

Definition: Consider a situation where a game has multiple Nash equilibria (NE), the equilibrium can be classified into two categories:

Explanation: A risk dominant NE is chosen when the player wants to avoid big losses while a payoff dominant NE is considered for an optimal payoff solution. Note that either the risk dominant or payoff dominant is a typical type of NE.

Example: Take the normal form payoff matrix of a game as an example:

Table1: Normal form of the example game
LR
U10, 100, 9
D9, 05, 5

There are two NEs in this game, i.e. (U, L) and (D,R). Here (U, L) is a payoff dominant NE since such a strategy can return an optimal overall payoff. However, given the uncertainty of an opponent's action, one of the players may consider a more conservative strategy (D for player 1 and R for player two), which can avoid the situation of "a great loss", i.e. getting 0 payoff once the opponent deviates. Hence the (D, R) is a risk dominant NE.

1/2 dominance

See also

Related Research Articles

An evolutionarily stable strategy (ESS) is a strategy that is impermeable when adopted by a population in adaptation to a specific environment, that is to say it cannot be displaced by an alternative strategy which may be novel or initially rare. Introduced by John Maynard Smith and George R. Price in 1972/3, it is an important concept in behavioural ecology, evolutionary psychology, mathematical game theory and economics, with applications in other fields such as anthropology, philosophy and political science.

Game theory is the study of mathematical models of strategic interactions among rational agents. It has applications in many fields of social science, used extensively in economics as well as in logic, systems science and computer science. Traditional game theory addressed two-person zero-sum games, in which a participant's gains or losses are exactly balanced by the losses and gains of the other participant. In the 21st century, game theory applies to a wider range of behavioral relations, and it is now an umbrella term for the science of rational decision making in humans, animals, as well as computers.

In game theory, the Nash equilibrium, named after the mathematician John Nash, is the most common way to define the solution of a non-cooperative game involving two or more players. In a Nash equilibrium, each player is assumed to know the equilibrium strategies of the other players, and no one has anything to gain by changing only one's own strategy. The principle of Nash equilibrium dates back to the time of Cournot, who in 1838 applied it to competing firms choosing outputs.

In game theory, the best response is the strategy which produces the most favorable outcome for a player, taking other players' strategies as given. The concept of a best response is central to John Nash's best-known contribution, the Nash equilibrium, the point at which each player in a game has selected the best response to the other players' strategies.

A coordination game is a type of simultaneous game found in game theory. It describes the situation where a player will earn a higher payoff when they select the same course of action as another player. The game is not one of pure conflict, which results in multiple pure strategy Nash equilibria in which players choose matching strategies. Figure 1 shows a 2-player example.

In game theory, the centipede game, first introduced by Robert Rosenthal in 1981, is an extensive form game in which two players take turns choosing either to take a slightly larger share of an increasing pot, or to pass the pot to the other player. The payoffs are arranged so that if one passes the pot to one's opponent and the opponent takes the pot on the next round, one receives slightly less than if one had taken the pot on this round, but after an additional switch the potential payoff will be higher. Therefore, although at each round a player has an incentive to take the pot, it would be better for them to wait. Although the traditional centipede game had a limit of 100 rounds, any game with this structure but a different number of rounds is called a centipede game.

<span class="mw-page-title-main">Solution concept</span> Formal rule for predicting how a game will be played

In game theory, a solution concept is a formal rule for predicting how a game will be played. These predictions are called "solutions", and describe which strategies will be adopted by players and, therefore, the result of the game. The most commonly used solution concepts are equilibrium concepts, most famously Nash equilibrium.

In game theory, a Perfect Bayesian Equilibrium (PBE) is a solution with Bayesian probability to a turn-based game with incomplete information. More specifically, it is an equilibrium concept that uses Bayesian updating to describe player behavior in dynamic games with incomplete information. Perfect Bayesian equilibria are used to solve the outcome of games where players take turns but are unsure of the "type" of their opponent, which occurs when players don't know their opponent's preference between individual moves. A classic example of a dynamic game with types is a war game where the player is unsure whether their opponent is a risk-taking "hawk" type or a pacifistic "dove" type. Perfect Bayesian Equilibria are a refinement of Bayesian Nash equilibrium (BNE), which is a solution concept with Bayesian probability for non-turn-based games.

Backward induction is the process of determining a sequence of optimal choices by employing a type of reasoning. This reasoning is from the end of a problem or situation to its beginning, point by point. It involves examining the last point at which a decision is to be made and identifying the most optimal process of action from that point. Using this information, one can then determine what to do at the second-to-last point of decision. This process continues backward until the best action for every possible point along the sequence is determined. Backward induction was first utilized in 1875 by Arthur Cayley, who discovered the method while attempting to solve the secretary problem.

In game theory, strategic dominance occurs when one strategy is better than another strategy for one player, no matter how that player's opponents may play. Many simple games can be solved using dominance. The opposite, intransitivity, occurs in games where one strategy may be better or worse than another strategy for one player, depending on how the player's opponents may play.

In game theory, trembling hand perfect equilibrium is a type of refinement of a Nash equilibrium that was first proposed by Reinhard Selten. A trembling hand perfect equilibrium is an equilibrium that takes the possibility of off-the-equilibrium play into account by assuming that the players, through a "slip of the hand" or tremble, may choose unintended strategies, albeit with negligible probability.

In game theory, folk theorems are a class of theorems describing an abundance of Nash equilibrium payoff profiles in repeated games. The original Folk Theorem concerned the payoffs of all the Nash equilibria of an infinitely repeated game. This result was called the Folk Theorem because it was widely known among game theorists in the 1950s, even though no one had published it. Friedman's (1971) Theorem concerns the payoffs of certain subgame-perfect Nash equilibria (SPE) of an infinitely repeated game, and so strengthens the original Folk Theorem by using a stronger equilibrium concept: subgame-perfect Nash equilibria rather than Nash equilibria.

In game theory, a repeated game is an extensive form game that consists of a number of repetitions of some base game. The stage game is usually one of the well-studied 2-person games. Repeated games capture the idea that a player will have to take into account the impact of their current action on the future actions of other players; this impact is sometimes called their reputation. Single stage game or single shot game are names for non-repeated games.

In game theory, a subgame perfect equilibrium is a refinement of a Nash equilibrium used in dynamic games. A strategy profile is a subgame perfect equilibrium if it represents a Nash equilibrium of every subgame of the original game. Informally, this means that at any point in the game, the players' behavior from that point onward should represent a Nash equilibrium of the continuation game, no matter what happened before. Every finite extensive game with perfect recall has a subgame perfect equilibrium. Perfect recall is a term introduced by Harold W. Kuhn in 1953 and "equivalent to the assertion that each player is allowed by the rules of the game to remember everything he knew at previous moves and all of his choices at those moves".

Risk dominance and payoff dominance are two related refinements of the Nash equilibrium (NE) solution concept in game theory, defined by John Harsanyi and Reinhard Selten. A Nash equilibrium is considered payoff dominant if it is Pareto superior to all other Nash equilibria in the game. When faced with a choice among equilibria, all players would agree on the payoff dominant equilibrium since it offers to each player at least as much payoff as the other Nash equilibria. Conversely, a Nash equilibrium is considered risk dominant if it has the largest basin of attraction. This implies that the more uncertainty players have about the actions of the other player(s), the more likely they will choose the strategy corresponding to it.

<span class="mw-page-title-main">Non-credible threat</span>

A non-credible threat is a term used in game theory and economics to describe a threat in a sequential game that a rational player would not actually carry out, because it would not be in his best interest to do so.

<span class="mw-page-title-main">Simultaneous game</span>

In game theory, a simultaneous game or static game is a game where each player chooses their action without knowledge of the actions chosen by other players. Simultaneous games contrast with sequential games, which are played by the players taking turns. In other words, both players normally act at the same time in a simultaneous game. Even if the players do not act at the same time, both players are uninformed of each other's move while making their decisions. Normal form representations are usually used for simultaneous games. Given a continuous game, players will have different information sets if the game is simultaneous than if it is sequential because they have less information to act on at each step in the game. For example, in a two player continuous game that is sequential, the second player can act in response to the action taken by the first player. However, this is not possible in a simultaneous game where both players act at the same time.

A Markov perfect equilibrium is an equilibrium concept in game theory. It has been used in analyses of industrial organization, macroeconomics, and political economy. It is a refinement of the concept of subgame perfect equilibrium to extensive form games for which a pay-off relevant state space can be identified. The term appeared in publications starting about 1988 in the work of economists Jean Tirole and Eric Maskin.

In game theory, Mertens stability is a solution concept used to predict the outcome of a non-cooperative game. A tentative definition of stability was proposed by Elon Kohlberg and Jean-François Mertens for games with finite numbers of players and strategies. Later, Mertens proposed a stronger definition that was elaborated further by Srihari Govindan and Mertens. This solution concept is now called Mertens stability, or just stability.

In game theory, a multi-stage game is a sequence of several simultaneous games played one after the other. This is a generalization of a repeated game: a repeated game is a special case of a multi-stage game, in which the stage games are identical.

References

  1. 1 2 Hu, Hengyuan; Lerer, Adam; Peysakhovich, Alex; Foerster, Jakob (2020). "'Other-Play' for Zero-Shot Coordination". Proceedings of the 37th International Conference on Machine Learning . arXiv: 2003.02979 .
  2. Harsanyi, John C.; Selten, Reinhard (1988). A General Theory of Equilibrium Selection. MIT Press.
  3. Emmons, Scott; Oesterheld, Caspar; Critch, Andrew; Conitzer, Vincent; Russell, Stuart (2022). "For Learning in Symmetric Teams, Local Optima are Global Nash Equilibria". Proceedings of the 39th International Conference on Machine Learning . pp. 5924–5943.
  4. Nash, John (1951). "Non-cooperative games". Annals of Mathematics. 54 (2): 286–295. doi:10.2307/1969529.