Extensive-form game

Last updated

An extensive-form game is a specification of a game in game theory, allowing (as the name suggests) for the explicit representation of a number of key aspects, like the sequencing of players' possible moves, their choices at every decision point, the (possibly imperfect) information each player has about the other player's moves when they make a decision, and their payoffs for all possible game outcomes. Extensive-form games also allow for the representation of incomplete information in the form of chance events modeled as "moves by nature".

Contents

Finite extensive-form games

Some authors, particularly in introductory textbooks, initially define the extensive-form game as being just a game tree with payoffs (no imperfect or incomplete information), and add the other elements in subsequent chapters as refinements. Whereas the rest of this article follows this gentle approach with motivating examples, we present upfront the finite extensive-form games as (ultimately) constructed here. This general definition was introduced by Harold W. Kuhn in 1953, who extended an earlier definition of von Neumann from 1928. Following the presentation from Hart (1992), an n-player extensive-form game thus consists of the following:

A play is thus a path through the tree from the root to a terminal node. At any given non-terminal node belonging to Chance, an outgoing branch is chosen according to the probability distribution. At any rational player's node, the player must choose one of the equivalence classes for the edges, which determines precisely one outgoing edge except (in general) the player doesn't know which one is being followed. (An outside observer knowing every other player's choices up to that point, and the realization of Nature's moves, can determine the edge precisely.) A pure strategy for a player thus consists of a selection choosing precisely one class of outgoing edges for every information set (of his). In a game of perfect information, the information sets are singletons. It's less evident how payoffs should be interpreted in games with Chance nodes. It is assumed that each player has a von Neumann–Morgenstern utility function defined for every game outcome; this assumption entails that every rational player will evaluate an a priori random outcome by its expected utility.

The above presentation, while precisely defining the mathematical structure over which the game is played, elides however the more technical discussion of formalizing statements about how the game is played like "a player cannot distinguish between nodes in the same information set when making a decision". These can be made precise using epistemic modal logic; see Shoham & Leyton-Brown (2009 , chpt. 13) for details.

A perfect information two-player game over a game tree (as defined in combinatorial game theory and artificial intelligence) can be represented as an extensive form game with outcomes (i.e. win, lose, or draw). Examples of such games include tic-tac-toe, chess, and infinite chess. [1] [2] A game over an expectminimax tree, like that of backgammon, has no imperfect information (all information sets are singletons) but has moves of chance. For example, poker has both moves of chance (the cards being dealt) and imperfect information (the cards secretly held by other players). ( Binmore 2007 , chpt. 2)

Perfect and complete information

A complete extensive-form representation specifies:

  1. the players of a game
  2. for every player every opportunity they have to move
  3. what each player can do at each of their moves
  4. what each player knows for every move
  5. the payoffs received by every player for every possible combination of moves
A game represented in extensive form Extensive-form tree.svg
A game represented in extensive form

The game on the right has two players: 1 and 2. The numbers by every non-terminal node indicate to which player that decision node belongs. The numbers by every terminal node represent the payoffs to the players (e.g. 2,1 represents a payoff of 2 to player 1 and a payoff of 1 to player 2). The labels by every edge of the graph are the name of the action that edge represents.

The initial node belongs to player 1, indicating that player 1 moves first. Play according to the tree is as follows: player 1 chooses between U and D; player 2 observes player 1's choice and then chooses between U' and D' . The payoffs are as specified in the tree. There are four outcomes represented by the four terminal nodes of the tree: (U,U'), (U,D'), (D,U') and (D,D'). The payoffs associated with each outcome respectively are as follows (0,0), (2,1), (1,2) and (3,1).

If player 1 plays D, player 2 will play U' to maximise their payoff and so player 1 will only receive 1. However, if player 1 plays U, player 2 maximises their payoff by playing D' and player 1 receives 2. Player 1 prefers 2 to 1 and so will play U and player 2 will play D' . This is the subgame perfect equilibrium.

Imperfect information

An advantage of representing the game in this way is that it is clear what the order of play is. The tree shows clearly that player 1 moves first and player 2 observes this move. However, in some games play does not occur like this. One player does not always observe the choice of another (for example, moves may be simultaneous or a move may be hidden). An information set is a set of decision nodes such that:

  1. Every node in the set belongs to one player.
  2. When the game reaches the information set, the player who is about to move cannot differentiate between nodes within the information set; i.e. if the information set contains more than one node, the player to whom that set belongs does not know which node in the set has been reached.

In extensive form, an information set is indicated by a dotted line connecting all nodes in that set or sometimes by a loop drawn around all the nodes in that set.

A game with imperfect information represented in extensive form Extensive-form tree with uncertainty.svg
A game with imperfect information represented in extensive form

If a game has an information set with more than one member that game is said to have imperfect information . A game with perfect information is such that at any stage of the game, every player knows exactly what has taken place earlier in the game; i.e. every information set is a singleton set. [1] [2] Any game without perfect information has imperfect information.

The game on the right is the same as the above game except that player 2 does not know what player 1 does when they come to play. The first game described has perfect information; the game on the right does not. If both players are rational and both know that both players are rational and everything that is known by any player is known to be known by every player (i.e. player 1 knows player 2 knows that player 1 is rational and player 2 knows this, etc. ad infinitum), play in the first game will be as follows: player 1 knows that if they play U, player 2 will play D' (because for player 2 a payoff of 1 is preferable to a payoff of 0) and so player 1 will receive 2. However, if player 1 plays D, player 2 will play U' (because to player 2 a payoff of 2 is better than a payoff of 1) and player 1 will receive 1. Hence, in the first game, the equilibrium will be (U, D' ) because player 1 prefers to receive 2 to 1 and so will play U and so player 2 will play D' .

In the second game it is less clear: player 2 cannot observe player 1's move. Player 1 would like to fool player 2 into thinking they have played U when they have actually played D so that player 2 will play D' and player 1 will receive 3. In fact in the second game there is a perfect Bayesian equilibrium where player 1 plays D and player 2 plays U' and player 2 holds the belief that player 1 will definitely play D. In this equilibrium, every strategy is rational given the beliefs held and every belief is consistent with the strategies played. Notice how the imperfection of information changes the outcome of the game.

To more easily solve this game for the Nash equilibrium [3] , it can be converted to the normal form [4] . Given this is a simultaneous/sequential game, player one and player two each have two strategies [5] .

Players 1 \ 2Up' (U')Down' (D')
Up (U)(0,0)(2,1)
Down (D)(1,2)(3,1)

We will have a two by two matrix with a unique payoff for each combination of moves. Using the normal form game, it is now possible to solve the game and identify dominant strategies for both players.

These preferences can be marked within the matrix, and any box where both players have a preference provides a nash equilibrium. This particular game has a single solution of (D,U’) with a payoff of (1,2).

In games with infinite action spaces and imperfect information, non-singleton information sets are represented, if necessary, by inserting a dotted line connecting the (non-nodal) endpoints behind the arc described above or by dashing the arc itself. In the Stackelberg competition described above, if the second player had not observed the first player's move the game would no longer fit the Stackelberg model; it would be Cournot competition.

Incomplete information

It may be the case that a player does not know exactly what the payoffs of the game are or of what type their opponents are. This sort of game has incomplete information . In extensive form it is represented as a game with complete but imperfect information using the so-called Harsanyi transformation. This transformation introduces to the game the notion of nature's choice or God's choice. Consider a game consisting of an employer considering whether to hire a job applicant. The job applicant's ability might be one of two things: high or low. Their ability level is random; they either have low ability with probability 1/3 or high ability with probability 2/3. In this case, it is convenient to model nature as another player of sorts who chooses the applicant's ability according to those probabilities. Nature however does not have any payoffs. Nature's choice is represented in the game tree by a non-filled node. Edges coming from a nature's choice node are labelled with the probability of the event it represents occurring.

A game with incomplete and imperfect information represented in extensive form Extensive form game 3.1.JPG
A game with incomplete and imperfect information represented in extensive form

The game on the right is one of complete information (all the players and payoffs are known to everyone) but of imperfect information (the employer doesn't know what nature's move was.) The initial node is in the centre and it is not filled, so nature moves first. Nature selects with the same probability the type of player 1 (which in this game is tantamount to selecting the payoffs in the subgame played), either t1 or t2. Player 1 has distinct information sets for these; i.e. player 1 knows what type they are (this need not be the case). However, player 2 does not observe nature's choice. They do not know the type of player 1; however, in this game they do observe player 1's actions; i.e. there is perfect information. Indeed, it is now appropriate to alter the above definition of complete information: at every stage in the game, every player knows what has been played by the other players. In the case of private information, every player knows what has been played by nature. Information sets are represented as before by broken lines.

In this game, if nature selects t1 as player 1's type, the game played will be like the very first game described, except that player 2 does not know it (and the very fact that this cuts through their information sets disqualify it from subgame status). There is one separating perfect Bayesian equilibrium; i.e. an equilibrium in which different types do different things.

If both types play the same action (pooling), an equilibrium cannot be sustained. If both play D, player 2 can only form the belief that they are on either node in the information set with probability 1/2 (because this is the chance of seeing either type). Player 2 maximises their payoff by playing D' . However, if they play D' , type 2 would prefer to play U. This cannot be an equilibrium. If both types play U, player 2 again forms the belief that they are at either node with probability 1/2. In this case player 2 plays D' , but then type 1 prefers to play D.

If type 1 plays U and type 2 plays D, player 2 will play D' whatever action they observe, but then type 1 prefers D. The only equilibrium hence is with type 1 playing D, type 2 playing U and player 2 playing U' if they observe D and randomising if they observe U. Through their actions, player 1 has signalled their type to player 2.

Formal definition

Formally, a finite game in extensive form is a structure where:

, the restriction of on is a bijection, with the set of successor nodes of .

Infinite action space

It may be that a player has an infinite number of possible actions to choose from at a particular decision node. The device used to represent this is an arc joining two edges protruding from the decision node in question. If the action space is a continuum between two numbers, the lower and upper delimiting numbers are placed at the bottom and top of the arc respectively, usually with a variable that is used to express the payoffs. The infinite number of decision nodes that could result are represented by a single node placed in the centre of the arc. A similar device is used to represent action spaces that, whilst not infinite, are large enough to prove impractical to represent with an edge for each action.

A game with infinite action spaces represented in extensive form Extensive form game 4.JPG
A game with infinite action spaces represented in extensive form

The tree on the left represents such a game, either with infinite action spaces (any real number between 0 and 5000) or with very large action spaces (perhaps any integer between 0 and 5000). This would be specified elsewhere. Here, it will be supposed that it is the former and, for concreteness, it will be supposed it represents two firms engaged in Stackelberg competition. The payoffs to the firms are represented on the left, with and as the strategy they adopt and and as some constants (here marginal costs to each firm). The subgame perfect Nash equilibria of this game can be found by taking the first partial derivative [ citation needed ] of each payoff function with respect to the follower's (firm 2) strategy variable () and finding its best response function, . The same process can be done for the leader except that in calculating its profit, it knows that firm 2 will play the above response and so this can be substituted into its maximisation problem. It can then solve for by taking the first derivative, yielding . Feeding this into firm 2's best response function, and is the subgame perfect Nash equilibrium.


See also

Related Research Articles

Minimax is a decision rule used in artificial intelligence, decision theory, game theory, statistics and philosophy for minimizing the possible loss for a worst case scenario. When dealing with gains, it is referred to as "maximin"—to maximize the minimum gain. Originally formulated for two-player zero-sum game theory, covering both the cases where players take alternate moves and those where they make simultaneous moves, it has also been extended to more complex games and to general decision-making in the presence of uncertainty.

In game theory, the Nash equilibrium, named after the mathematician John Forbes Nash Jr., is a proposed solution of a non-cooperative game involving two or more players in which each player is assumed to know the equilibrium strategies of the other players, and no player has anything to gain by changing only their own strategy.

In theoretical physics and mathematical physics, analytical mechanics, or theoretical mechanics is a collection of closely related alternative formulations of classical mechanics. It was developed by many scientists and mathematicians during the 18th century and onward, after Newtonian mechanics. Since Newtonian mechanics considers vector quantities of motion, particularly accelerations, momenta, forces, of the constituents of the system, an alternative name for the mechanics governed by Newton's laws and Euler's laws is vectorial mechanics.

Automatic differentiation

In mathematics and computer algebra, automatic differentiation (AD), also called algorithmic differentiation or computational differentiation, is a set of techniques to numerically evaluate the derivative of a function specified by a computer program. AD exploits the fact that every computer program, no matter how complicated, executes a sequence of elementary arithmetic operations and elementary functions. By applying the chain rule repeatedly to these operations, derivatives of arbitrary order can be computed automatically, accurately to working precision, and using at most a small constant factor more arithmetic operations than the original program.

Virtual work arises in the application of the principle of least action to the study of forces and movement of a mechanical system. The work of a force acting on a particle as it moves along a displacement is different for different displacements. Among all the possible displacements that a particle may follow, called virtual displacements, one will minimize the action. This displacement is therefore the displacement followed by the particle according to the principle of least action. The work of a force on a particle along a virtual displacement is known as the virtual work.

Solution concept formal rule for predicting how a strategic game will be played

In game theory, a solution concept is a formal rule for predicting how a game will be played. These predictions are called "solutions", and describe which strategies will be adopted by players and, therefore, the result of the game. The most commonly used solution concepts are equilibrium concepts, most famously Nash equilibrium.

In game theory, a Perfect Bayesian Equilibrium (PBE) is an equilibrium concept relevant for dynamic games with incomplete information. A PBE is a refinement of both Bayesian Nash equilibrium (BNE) and subgame perfect equilibrium (SPE). A PBE has two components - strategies and beliefs:

In game theory, a Bayesian game is a game in which players have incomplete information about the other players. For example, a player may not know the exact payoff functions of the other players, but instead have beliefs about these payoff functions. These beliefs are represented by a probability distribution over the possible payoff functions.

In game theory, normal form is a description of a game. Unlike extensive form, normal-form representations are not graphical per se, but rather represent the game by way of a matrix. While this approach can be of greater use in identifying strictly dominated strategies and Nash equilibria, some information is lost as compared to extensive-form representations. The normal-form representation of a game includes all perceptible and conceivable strategies, and their corresponding payoffs, for each player.

In game theory, trembling hand perfect equilibrium is a refinement of Nash equilibrium due to Reinhard Selten. A trembling hand perfect equilibrium is an equilibrium that takes the possibility of off-the-equilibrium play into account by assuming that the players, through a "slip of the hand" or tremble, may choose unintended strategies, albeit with negligible probability.

In game theory, folk theorems are a class of theorems about possible Nash equilibrium payoff profiles in repeated games. The original Folk Theorem concerned the payoffs of all the Nash equilibria of an infinitely repeated game. This result was called the Folk Theorem because it was widely known among game theorists in the 1950s, even though no one had published it. Friedman's (1971) Theorem concerns the payoffs of certain subgame-perfect Nash equilibria (SPE) of an infinitely repeated game, and so strengthens the original Folk Theorem by using a stronger equilibrium concept subgame-perfect Nash equilibria rather than Nash equilibrium.

In game theory, a repeated game is an extensive form game that consists of a number of repetitions of some base game. The stage game is usually one of the well-studied 2-person games. Repeated games capture the idea that a player will have to take into account the impact of his or her current action on the future actions of other players; this impact is sometimes called his or her reputation. Single stage game or single shot game are names for non-repeated games.

In game theory, a correlated equilibrium is a solution concept that is more general than the well known Nash equilibrium. It was first discussed by mathematician Robert Aumann in 1974. The idea is that each player chooses their action according to their observation of the value of the same public signal. A strategy assigns an action to every possible observation a player can make. If no player would want to deviate from the recommended strategy, the distribution is called a correlated equilibrium.

Hamiltons principle formulation of the principle of stationary action

In physics, Hamilton's principle is William Rowan Hamilton's formulation of the principle of stationary action. It states that the dynamics of a physical system is determined by a variational problem for a functional based on a single function, the Lagrangian, which contains all physical information concerning the system and the forces acting on it. The variational problem is equivalent to and allows for the derivation of the differential equations of motion of the physical system. Although formulated originally for classical mechanics, Hamilton's principle also applies to classical fields such as the electromagnetic and gravitational fields, and plays an important role in quantum mechanics, quantum field theory and criticality theories.

In game theory, a subgame perfect equilibrium is a refinement of a Nash equilibrium used in dynamic games. A strategy profile is a subgame perfect equilibrium if it represents a Nash equilibrium of every subgame of the original game. Informally, this means that if the players played any smaller game that consisted of only one part of the larger game, their behavior would represent a Nash equilibrium of that smaller game. Every finite extensive game with perfect recall has a subgame perfect equilibrium.

In game theory, an epsilon-equilibrium, or near-Nash equilibrium, is a strategy profile that approximately satisfies the condition of Nash equilibrium. In a Nash equilibrium, no player has an incentive to change his behavior. In an approximate Nash equilibrium, this requirement is weakened to allow the possibility that a player may have a small incentive to do something different. This may still be considered an adequate solution concept, assuming for example status quo bias. This solution concept may be preferred to Nash equilibrium due to being easier to compute, or alternatively due to the possibility that in games of more than 2 players, the probabilities involved in an exact Nash equilibrium need not be rational numbers.

In game theory, a stochastic game, introduced by Lloyd Shapley in the early 1950s, is a dynamic game with probabilistic transitions played by one or more players. The game is played in a sequence of stages. At the beginning of each stage the game is in some state. The players select actions and each player receives a payoff that depends on the current state and the chosen actions. The game then moves to a new random state whose distribution depends on the previous state and the actions chosen by the players. The procedure is repeated at the new state and play continues for a finite or infinite number of stages. The total payoff to a player is often taken to be the discounted sum of the stage payoffs or the limit inferior of the averages of the stage payoffs.

A continuous game is a mathematical concept, used in game theory, that generalizes the idea of an ordinary game like tic-tac-toe or checkers (draughts). In other words, it extends the notion of a discrete game, where the players choose from a finite set of pure strategies. The continuous game concepts allows games to include more general sets of pure strategies, which may be uncountably infinite.

Mertens stability is a solution concept used to predict the outcome of a non-cooperative game. A tentative definition of stability was proposed by Elon Kohlberg and Jean-François Mertens for games with finite numbers of players and strategies. Later, Mertens proposed a stronger definition that was elaborated further by Srihari Govindan and Mertens. This solution concept is now called Mertens stability, or just stability.

Network games of incomplete information represent strategic network formation when agents do not know in advance their neighbors, i.e. the network structure and the value stemming from forming links with neighboring agents. In such a setting, agents have prior beliefs about the value of attaching to their neighbors; take their action based on their prior belief and update their belief based on the history of the game. While games with a fully known network structure are widely applicable, there are many applications when players act without fully knowing with whom they interact or what their neighbors’ action will be. For example, people choosing major in college can be formalized as a network game with imperfect information: they might know something about the number of people taking that major and might infer something about the job market for different majors, but they don’t know with whom they will have to interact, thus they do not know the structure of the network.

References

  1. 1 2 https://www.math.uni-hamburg/Infinite Games, Yurii Khomskii (2010) Infinite Games (section 1.1), Yurii Khomskii (2010)
  2. 1 2 "Infinite Chess, PBS Infinite Series" PBS Infinite Series. Perfect information defined at 0:25, with academic sources https://arxiv.org/abs/1302.4377 and https://arxiv.org/abs/1510.08155.
  3. Watson, Joel. Strategy : an introduction to game theory. pp. 97–100. ISBN   978-0-393-91838-0. OCLC   1123193808.
  4. Watson, Joel. Strategy : an introduction to game theory. pp. 26–28. ISBN   978-0-393-91838-0. OCLC   1123193808.
  5. Watson, Joel. Strategy : an introduction to game theory. pp. 22–26. ISBN   978-0-393-91838-0. OCLC   1123193808.

Further reading

Historical papers