Markov perfect equilibrium

Last updated
Markov perfect equilibrium
A solution concept in game theory
Relationship
Subset of Subgame perfect equilibrium
Significance
Proposed by Eric Maskin, Jean Tirole
Used for tacit collusion; price wars; oligopolistic competition

A Markov perfect equilibrium is an equilibrium concept in game theory. It has been used in analyses of industrial organization, macroeconomics, and political economy. It is a refinement of the concept of subgame perfect equilibrium to extensive form games for which a pay-off relevant state space can be identified. The term appeared in publications starting about 1988 in the work of economists Jean Tirole and Eric Maskin. [1] [2] [3] [4]

Contents

Definition

In extensive form games, and specifically in stochastic games, a Markov perfect equilibrium is a set of mixed strategies for each of the players which satisfy the following criteria:

Focus on symmetric equilibria

In symmetric games, when the players have a strategy and action sets which are mirror images of one another, often the analysis focuses on symmetric equilibria, where all players play the same mixed strategy. As in the rest of game theory, this is done both because these are easier to find analytically and because they are perceived to be stronger focal points than asymmetric equilibria.

Lack of robustness

Markov perfect equilibria are not stable with respect to small changes in the game itself. A small change in payoffs can cause a large change in the set of Markov perfect equilibria. This is because a state with a tiny effect on payoffs can be used to carry signals, but if its payoff difference from any other state drops to zero, it must be merged with it, eliminating the possibility of using it to carry signals.

Examples

For examples of this equilibrium concept, consider the competition between firms which have invested heavily into fixed costs and are dominant producers in an industry, forming an oligopoly. The players are taken to be committed to levels of production capacity in the short run, and the strategies describe their decisions in setting prices. The firms' objectives are modelled as maximizing the present discounted value of profits. [6]

Airfare game

Often an airplane ticket for a certain route has the same price on either airline A or airline B. Presumably, the two airlines do not have exactly the same costs, nor do they face the same demand function given their varying frequent-flyer programs, the different connections their passengers will make, and so forth. Thus, a realistic general equilibrium model would be unlikely to result in nearly identical prices.

Both airlines have made sunk investments into the equipment, personnel, and legal framework, thus committing to offering service. They are engaged or trapped, in a strategic game with one another when setting prices.

Consider the following strategy of an airline for setting the ticket price for a certain route. At every price-setting opportunity:

This is a Markov strategy because it does not depend on a history of past observations. It satisfies also the Markov reaction function definition because it does not depend on other information which is irrelevant to revenues and profits.

Assume now that both airlines follow this strategy exactly. Assume further that passengers always choose the cheapest flight and so if the airlines charge different prices, the one charging the higher price gets zero passengers. Then if each airline assumes that the other airline will follow this strategy, there is no higher-payoff alternative strategy for itself, i.e. it is playing a best response to the other airline strategy. If both airlines followed this strategy, it would form a Nash equilibrium in every proper subgame, thus a subgame-perfect Nash equilibrium. [note 1]

A Markov-perfect equilibrium concept has also been used to model aircraft production, as different companies evaluate their future profits and how much they will learn from production experience in light of demand and what others firms might supply. [7]

Discussion

Airlines do not literally or exactly follow these strategies, but the model helps explain the observation that airlines often charge exactly the same price, even though a general equilibrium model specifying non-perfect substitutability would generally not provide such a result. The Markov perfect equilibrium model helps shed light on tacit collusion in an oligopoly setting, and make predictions for cases not observed.

One strength of an explicit game-theoretical framework is that it allows us to make predictions about the behaviours of the airlines if and when the equal-price outcome breaks down, and interpret and examine these price wars in light of different equilibrium concepts. [8] In contrast to another equilibrium concept, Maskin and Tirole identify an empirical attribute of such price wars: in a Markov strategy price war, "a firm cuts its price not to punish its competitor, [rather only to] regain market share" whereas in a general repeated game framework a price cut may be a punishment to the other player. The authors claim that the market share justification is closer to the empirical account than the punishment justification, and so the Markov perfect equilibrium concept proves more informative, in this case. [9]

Notes

  1. ↑ This kind of extreme simplification is necessary to get through the example but could be relaxed in a more thorough study. A more complete specification of the game, including payoffs, would be necessary to show that these strategies can form a subgame-perfect Nash equilibrium. For illustration let us suppose however that the strategies do form such an equilibrium and therefore that they also constitute a Markov perfect equilibrium.

Related Research Articles

In game theory, the Nash equilibrium, named after the mathematician John Forbes Nash Jr., is the most common way to define the solution of a non-cooperative game involving two or more players. In a Nash equilibrium, each player is assumed to know the equilibrium strategies of the other players and no player has anything to gain by changing only their own strategy. The principle of Nash equilibrium dates back to the time of Cournot, who in 1838 applied it to competing firms choosing outputs.

In game theory, the best response is the strategy which produces the most favorable outcome for a player, taking other players' strategies as given. The concept of a best response is central to John Nash's best-known contribution, the Nash equilibrium, the point at which each player in a game has selected the best response to the other players' strategies.

In game theory, grim trigger is a trigger strategy for a repeated game.

In game theory, the battle of the sexes (BoS) is a two-player coordination game that also involves elements of conflict. The game was introduced in 1957 by Luce and Raiffa in their classic book, Games and Decisions.

Solution concept

In game theory, a solution concept is a formal rule for predicting how a game will be played. These predictions are called "solutions", and describe which strategies will be adopted by players and, therefore, the result of the game. The most commonly used solution concepts are equilibrium concepts, most famously Nash equilibrium.

In game theory, a Perfect Bayesian Equilibrium (PBE) is an equilibrium concept relevant for dynamic games with incomplete information. It is a refinement of Bayesian Nash equilibrium (BNE). A perfect Bayesian equilibrium has two components -- strategies and beliefs:

Backward induction is the process of reasoning backwards in time, from the end of a problem or situation, to determine a sequence of optimal actions. It proceeds by examining the last point at which a decision is to be made and then identifying what action would be most optimal at that moment. Using this information, one can then determine what to do at the second-to-last time of decision. This process continues backwards until one has determined the best action for every possible situation at every point in time. Backward induction was first used in 1875 by Arthur Cayley, who uncovered the method while trying to solve the infamous Secretary Problem.

In game theory, rationalizability is a solution concept. The general idea is to provide the weakest constraints on players while still requiring that players are rational and this rationality is common knowledge among the players. It is more permissive than Nash equilibrium. Both require that players respond optimally to some belief about their opponents' actions, but Nash equilibrium requires that these beliefs be correct while rationalizability does not. Rationalizability was first defined, independently, by Bernheim (1984) and Pearce (1984).

In game theory, trembling hand perfect equilibrium is a refinement of Nash equilibrium due to Reinhard Selten. A trembling hand perfect equilibrium is an equilibrium that takes the possibility of off-the-equilibrium play into account by assuming that the players, through a "slip of the hand" or tremble, may choose unintended strategies, albeit with negligible probability.

In game theory, folk theorems are a class of theorems describing an abundance of Nash equilibrium payoff profiles in repeated games. The original Folk Theorem concerned the payoffs of all the Nash equilibria of an infinitely repeated game. This result was called the Folk Theorem because it was widely known among game theorists in the 1950s, even though no one had published it. Friedman's (1971) Theorem concerns the payoffs of certain subgame-perfect Nash equilibria (SPE) of an infinitely repeated game, and so strengthens the original Folk Theorem by using a stronger equilibrium concept: subgame-perfect Nash equilibria rather than Nash equilibria.

In game theory, a repeated game is an extensive form game that consists of a number of repetitions of some base game. The stage game is usually one of the well-studied 2-person games. Repeated games capture the idea that a player will have to take into account the impact of his or her current action on the future actions of other players; this impact is sometimes called his or her reputation. Single stage game or single shot game are names for non-repeated games.

In game theory, the war of attrition is a dynamic timing game in which players choose a time to stop, and fundamentally trade off the strategic gains from outlasting other players and the real costs expended with the passage of time. Its precise opposite is the pre-emption game, in which players elect a time to stop, and fundamentally trade off the strategic costs from outlasting other players and the real gains occasioned by the passage of time. The model was originally formulated by John Maynard Smith; a mixed evolutionarily stable strategy (ESS) was determined by Bishop & Cannings. An example is a second price all-pay auction, in which the prize goes to the player with the highest bid and each player pays the loser's low bid.

An Edgeworth price cycle is cyclical pattern in prices characterized by an initial jump, which is then followed by a slower decline back towards the initial level. The term was introduced by Maskin and Tirole (1988) in a theoretical setting featuring two firms bidding sequentially and where the winner captures the full market.

In game theory, a subgame perfect equilibrium is a refinement of a Nash equilibrium used in dynamic games. A strategy profile is a subgame perfect equilibrium if it represents a Nash equilibrium of every subgame of the original game. Informally, this means that at any point in the game, the players' behavior from that point onward should represent a Nash equilibrium of the continuation game, no matter what happened before. Every finite extensive game with perfect recall has a subgame perfect equilibrium. Perfect recall is a term introduced by Harold W. Kuhn in 1953 and "equivalent to the assertion that each player is allowed by the rules of the game to remember everything he knew at previous moves and all of his choices at those moves".

Risk dominance and payoff dominance are two related refinements of the Nash equilibrium (NE) solution concept in game theory, defined by John Harsanyi and Reinhard Selten. A Nash equilibrium is considered payoff dominant if it is Pareto superior to all other Nash equilibria in the game. When faced with a choice among equilibria, all players would agree on the payoff dominant equilibrium since it offers to each player at least as much payoff as the other Nash equilibria. Conversely, a Nash equilibrium is considered risk dominant if it has the largest basin of attraction. This implies that the more uncertainty players have about the actions of the other player(s), the more likely they will choose the strategy corresponding to it.

Drew Fudenberg is the Paul A. Samuelson Professor of Economics at MIT. His extensive research spans many aspects of game theory, including equilibrium theory, learning in games, evolutionary game theory, and many applications to other fields. Fudenberg was also one of the first to apply game theoretic analysis in industrial organization, bargaining theory, and contract theory. He has also authored papers on repeated games, reputation effects, and behavioral economics.

Jean-François Mertens Belgian game theorist (1946–2012)

Jean-François Mertens was a Belgian game theorist and mathematical economist.

Mertens stability is a solution concept used to predict the outcome of a non-cooperative game. A tentative definition of stability was proposed by Elon Kohlberg and Jean-François Mertens for games with finite numbers of players and strategies. Later, Mertens proposed a stronger definition that was elaborated further by Srihari Govindan and Mertens. This solution concept is now called Mertens stability, or just stability.

The one-shot deviation principle is the principle of optimality of dynamic programming applied to game theory. It says that a strategy profile of a finite extensive-form game is a subgame perfect equilibrium (SPE) if and only if there exist no profitable one-shot deviations for each subgame and every player. In simpler terms, if no player can increase their payoffs by deviating a single decision, or period, from their original strategy, then the strategy that they have chosen is a SPE. As a result, no player can profit from deviating from the strategy for one period and then reverting to the strategy.

Squential bargaining is a structured form of bargaining between two participants, in which the participants take turns in making offers. Initially, person #1 has the right to make an offer to person #2. If person #2 accepts the offer, then an agreement is reached and the process ends. If person #2 rejects the offer, then the participants switch turns, and now it is the turn of person #2 to make an offer. The people keep switching turns until either an agreement is reached, or the process ends with a disagreement due to a certain end condition. Several end conditions are common, for example:

References

  1. ↑ Maskin E, Tirole J. A Theory of Dynamic Oligopoly, I: Overview and Quantity Competition with Large Fixed Costs. Econometrica 1988;56:549.
  2. ↑ Maskin and Maskin E, Tirole J. A Theory of Dynamic Oligopoly, II: Price Competition, Kinked Demand Curves, and Edgeworth Cycles. Econometrica 1988;56:571
  3. ↑ Maskin E, Tirole J. Markov Perfect Equilibrium. J Econ Theory 2001;100:191–219.
  4. ↑ Fudenberg D, Tirole J. Game Theory. 1991:603.
  5. ↑ We shall define a Markov Perfect Equilibrium (MPE) to be a subgame perfect equilibrium in which all players use Markov strategies. Eric Maskin and Jean Tirole. 2001. Markov Perfect Equilibrium Archived 2011-10-05 at the Wayback Machine . Journal of Economic Theory 100, 191-219. doi : 10.1006/jeth.2000.2785
  6. ↑ Tirole (1988), p. 254
  7. ↑ C. Lanier Benkard. 2000. Learning and forgetting: The dynamics of aircraft production. American Economic Review 90:4, 1034–1054. (jstor)
  8. ↑ See for example Maskin and Tirole, p.571
  9. ↑ Maskin and Tirole, 1988, p.592

Bibliography

This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.