# Tit for tat

Last updated

Tit for tat is an English saying meaning "equivalent retaliation". It developed from "tip for tap", first recorded in 1558. [1]

## Contents

It is also a highly effective strategy in game theory. An agent using this strategy will first cooperate, then subsequently replicate an opponent's previous action. If the opponent previously was cooperative, the agent is cooperative. If not, the agent is not.

## Game theory

Tit-for-tat has been very successfully used as a strategy for the iterated prisoner's dilemma. The strategy was first introduced by Anatol Rapoport in Robert Axelrod's two tournaments, [2] held around 1980. Notably, it was (on both occasions) both the simplest strategy and the most successful in direct competition.

An agent using this strategy will first cooperate, then subsequently replicate an opponent's previous action. If the opponent previously was cooperative, the agent is cooperative. If not, the agent is not. This is similar to reciprocal altruism in biology.

## Implications

The success of the tit-for-tat strategy, which is largely cooperative despite that its name emphasizes an adversarial nature, took many by surprise. Arrayed against strategies produced by various teams it won in two competitions. After the first competition, new strategies formulated specifically to combat tit-for-tat failed due to their negative interactions with each other; a successful strategy other than tit-for-tat would have had to be formulated with both tit-for-tat and itself in mind.

This result may give insight into how groups of animals (and particularly human societies) have come to live in largely (or entirely) cooperative societies, rather than the individualistic "red in tooth and claw" way that might be expected from individuals engaged in a Hobbesian state of nature. This, and particularly its application to human society and politics, is the subject of Robert Axelrod's book The Evolution of Cooperation .

Moreover, the tit-for-tat strategy has been of beneficial use to social psychologists and sociologists in studying effective techniques to reduce conflict. Research has indicated that when individuals who have been in competition for a period of time no longer trust one another, the most effective competition reverser is the use of the tit-for-tat strategy. Individuals commonly engage in behavioral assimilation, a process in which they tend to match their own behaviors to those displayed by cooperating or competing group members. Therefore, if the tit-for-tat strategy begins with cooperation, then cooperation ensues. On the other hand, if the other party competes, then the tit-for-tat strategy will lead the alternate party to compete as well. Ultimately, each action by the other member is countered with a matching response, competition with competition and cooperation with cooperation.

In the case of conflict resolution, the tit-for-tat strategy is effective for several reasons: the technique is recognized as clear, nice, provocable, and forgiving. Firstly, it is a clear and recognizable strategy. Those using it quickly recognize its contingencies and adjust their behavior accordingly. Moreover, it is considered to be nice as it begins with cooperation and only defects in following competitive move. The strategy is also provocable because it provides immediate retaliation for those who compete. Finally, it is forgiving as it immediately produces cooperation should the competitor make a cooperative move.

The implications of the tit-for-tat strategy have been of relevance to conflict research, resolution and many aspects of applied social science. [3]

## Mathematics

Take for example the following infinitely repeated prisoners dilemma game:

CD
C6, 62, 9
D9, 23, 3

The Tit for Tat strategy copies what the other player previously chose. If players cooperate by playing strategy (C,C) they cooperate forever.

1234...
p1CCCC...
p2CCCC...

Cooperation gives the following payoff (where ${\displaystyle \delta }$ is the discount factor):

${\displaystyle 6+6\delta +6\delta ^{2}+6\delta ^{3}...,}$

a geometric series summing to

${\displaystyle {\frac {6}{1-\delta }}}$

If a player deviates to defecting (D), then the next round they get punished. Alternate between outcomes where p1 cooperates and p2 deviates, and vice versa.

1234...
p1CDCD...
p2DCDC...

Deviation gives the following payoff:

${\displaystyle 9+2\delta +9\delta ^{2}+2\delta ^{3}+9\delta ^{4}+2\delta ^{5}...,}$

a sum of two geometric series that comes to

${\displaystyle {\frac {9}{1-\delta ^{2}}}+{\frac {2\delta }{1-\delta ^{2}}}}$

Expect collaboration if payoff of deviation is no better than cooperation.

${\displaystyle {\frac {6}{1-\delta }}\geq {\frac {9}{1-\delta ^{2}}}+{\frac {2\delta }{1-\delta ^{2}}}}$

${\displaystyle {\frac {6}{1-\delta }}\geq {\frac {9+2\delta }{1-\delta ^{2}}}}$

${\displaystyle {\frac {1-\delta ^{2}}{1}}\cdot {\frac {6}{1-\delta }}\geq {\frac {9+2\delta }{\cancel {1-\delta ^{2}}}}\cdot {\frac {\cancel {1-\delta ^{2}}}{1}}}$

${\displaystyle {\frac {(1+\delta ){\cancel {(1-\delta )}}}{1}}\cdot {\frac {6}{\cancel {1-\delta }}}\geq 9+2\delta }$

${\displaystyle 6+6\delta \geq 9+2\delta }$

${\displaystyle 4\delta \geq 3}$

${\displaystyle \delta \geq {\frac {3}{4}}}$

Continue cooperating if, ${\displaystyle \delta \geq {\frac {3}{4}}}$

Continue defecting if, ${\displaystyle \delta <{\frac {3}{4}}}$

## Problems

While Axelrod has empirically shown that the strategy is optimal in some cases of direct competition, two agents playing tit for tat remain vulnerable. A one-time, single-bit error in either player's interpretation of events can lead to an unending "death spiral": if one agent defects and the opponent cooperates, then both agents will end up alternating cooperate and defect, yielding a lower payoff than if both agents were to continually cooperate. This situation frequently arises in real world conflicts, ranging from schoolyard fights to civil and regional wars. The reason for these issues is that tit for tat is not a subgame perfect equilibrium, except under knife-edge conditions on the discount rate. [4] While this sub-game is not directly reachable by two agents playing tit for tat strategies, a strategy must be a Nash equilibrium in all sub-games to be sub-game perfect. Further, this sub-game may be reached if any noise is allowed in the agents' signaling. A sub-game perfect variant of tit for tat known as "contrite tit for tat" may be created by employing a basic reputation mechanism. [5]

Knife-edge is "equilibrium that exists only for exact values of the exogenous variables. If you vary the variables in even the slightest way, knife-edge equilibrium disappear." [6]

Can be both Nash equilibrium and knife-edge equilibrium. Known as knife-edge equilibrium because the equilibrium "rests precariously on" the exact value.

Example:

Left Right (X, X) (0, 0) (0, 0) (−X, −X)

Suppose X = 0. There is no profitable deviation from (Down, Left) or from (Up, Right). However, if the value of X deviates by any amount, no matter how small, then the equilibrium no longer stands. It becomes profitable to deviate to up, for example, if X has a value of 0.000001 instead of 0. Thus, the equilibrium is very precarious. In its usage in the Wikipedia article, knife-edge conditions is referring to the fact that very rarely, only when a specific condition is met and, for instance, X, equals a specific value is there an equilibrium.

Tit for two tats could be used to mitigate this problem; see the description below. [7] "Tit for tat with forgiveness" is a similar attempt to escape the death spiral. When the opponent defects, a player employing this strategy will occasionally cooperate on the next move anyway. The exact probability that a player will respond with cooperation depends on the line-up of opponents.

Furthermore, the tit-for-tat strategy is not proved optimal in situations short of total competition. For example, when the parties are friends it may be best for the friendship when a player cooperates at every step despite occasional deviations by the other player. Most situations in the real world are less competitive than the total competition in which the tit-for-tat strategy won its competition.

Tit for tat is very different from grim trigger, in that it is forgiving in nature, as it immediately produces cooperation, should the competitor chooses to cooperate. Grim trigger on the other hand is the most unforgiving strategy, in the sense even a single defect would the make the player playing using grim trigger defect for the remainder of the game. [8]

## Tit for two tats

Tit for two tats is similar to tit for tat, but allows the opponent to defect from the agreed upon strategy twice before the player retaliates.  This aspect makes the player using the tit for tat strategy appear more “forgiving” to the opponent.

In a tit for tat strategy, once an opponent defects, the tit for tat player immediately responds by defecting on the next move. This has the unfortunate consequence of causing two retaliatory strategies to continuously defect against each other resulting in a poor outcome for both players. A tit for two tats player will let the first defection go unchallenged as a means to avoid the "death spiral" of the previous example. If the opponent defects twice in a row, the tit for two tats player will respond by defecting.

This strategy was put forward by Robert Axelrod during his second round of computer simulations at RAND. After analyzing the results of the first experiment, he determined that had a participant entered the tit for two tats strategy it would have emerged with a higher cumulative score than any other program. As a result, he himself entered it with high expectations in the second tournament. Unfortunately, owing to the more aggressive nature of the programs entered in the second round, which were able to take advantage of its highly forgiving nature, tit for two tats did significantly worse (in the game-theory sense) than tit for tat. [9]

## Real-world use

### Explaining reciprocal altruism in animal communities

Studies in the prosocial behaviour of animals have led many ethologists and evolutionary psychologists to apply tit-for-tat strategies to explain why altruism evolves in many animal communities. Evolutionary game theory, derived from the mathematical theories formalised by von Neumann and Morgenstern (1953), was first devised by Maynard Smith (1972) and explored further in bird behaviour by Robert Hinde. Their application of game theory to the evolution of animal strategies launched an entirely new way of analysing animal behaviour.

Reciprocal altruism works in animal communities where the cost to the benefactor in any transaction of food, mating rights, nesting or territory is less than the gains to the beneficiary. The theory also holds that the act of altruism should be reciprocated if the balance of needs reverse. Mechanisms to identify and punish "cheaters" who fail to reciprocate, in effect a form of tit for tat, are important to regulate reciprocal altruism. For example, tit-for-tat is suggested to be the mechanism of cooperative predator inspection behavior in guppies.

### War

The tit-for-tat inability of either side to back away from conflict, for fear of being perceived as weak or as cooperating with the enemy, has been the cause of many prolonged conflicts throughout history.

However, the tit for tat strategy has also been detected by analysts in the spontaneous non-violent behaviour, called "live and let live" that arose during trench warfare in the First World War. Troops dug in only a few hundred feet from each other would evolve an unspoken understanding. If a sniper killed a soldier on one side, the other expected an equal retaliation. Conversely, if no one was killed for a time, the other side would acknowledge this implied "truce" and act accordingly. This created a "separate peace" between the trenches. [11]

## Related Research Articles

An evolutionarily stable strategy (ESS) is a strategy that is impermeable when adopted by a population in adaptation to a specific environment, that is to say it cannot be displaced by an alternative strategy which may be novel or initially rare. Introduced by John Maynard Smith and George R. Price in 1972/3, it is an important concept in behavioural ecology, evolutionary psychology, mathematical game theory and economics, with applications in other fields such as anthropology, philosophy and political science.

The Evolution of Cooperation is a 1984 book by political scientist Robert Axelrod that expanded upon a highly influential paper of the same name written by Axelrod and evolutionary biologist W.D. Hamilton. It details a theory on the emergence of cooperation between individuals, drawing from game theory and evolutionary biology. Since 2006, reprints of the book have included a foreword by Richard Dawkins and been marketed as a revised edition.

The prisoner's dilemma is a standard example of a game analyzed in game theory that shows why two completely rational individuals might not cooperate, even if it appears that it is in their best interests to do so. It was originally framed by Merrill Flood and Melvin Dresher while working at RAND in 1950. Albert W. Tucker formalized the game with prison sentence rewards and named it "prisoner's dilemma", presenting it as follows:

Two members of a criminal organization are arrested and imprisoned. Each prisoner is in solitary confinement with no means of communicating with the other. The prosecutors lack sufficient evidence to convict the pair on the principal charge, but they have enough to convict both on a lesser charge. Simultaneously, the prosecutors offer each prisoner a bargain. Each prisoner is given the opportunity either to betray the other by testifying that the other committed the crime, or to cooperate with the other by remaining silent. The possible outcomes are:

In game theory, the Nash equilibrium, named after the mathematician John Forbes Nash Jr., is the most common way to define the solution of a non-cooperative game involving two or more players. In a Nash equilibrium, each player is assumed to know the equilibrium strategies of the other players, and no player has anything to gain by changing only their own strategy. The principle of Nash equilibrium dates back to the time of Cournot, who in 1838 applied it to competing firms choosing outputs.

Evolutionary game theory (EGT) is the application of game theory to evolving populations in biology. It defines a framework of contests, strategies, and analytics into which Darwinian competition can be modelled. It originated in 1973 with John Maynard Smith and George R. Price's formalisation of contests, analysed as strategies, and the mathematical criteria that can be used to predict the results of competing strategies.

In game theory, the centipede game, first introduced by Robert Rosenthal in 1981, is an extensive form game in which two players take turns choosing either to take a slightly larger share of an increasing pot, or to pass the pot to the other player. The payoffs are arranged so that if one passes the pot to one's opponent and the opponent takes the pot on the next round, one receives slightly less than if one had taken the pot on this round, but after an additional switch the potential payoff will be higher. Therefore, although at each round a player has an incentive to take the pot, it would be better for them to wait. Although the traditional centipede game had a limit of 100 rounds, any game with this structure but a different number of rounds is called a centipede game.

In game theory, a non-cooperative game is a game with competition between individual players, as opposed to cooperative games, and in which alliances can only operate if self-enforcing.

In game theory, grim trigger is a trigger strategy for a repeated game.

In game theory, the stag hunt, sometimes referred to as the assurance game, trust dilemma or common interest game, describes a conflict between safety and social cooperation. The stag hunt problem originated with philosopher Jean-Jacques Rousseau in his Discourse on Inequality. In Rousseau's telling, two hunters must decide separately, and without the other knowing, whether to hunt a stag or a hare. However, both hunters know the only way to successfully hunt a stag is with the other's help. One hunter can catch a hare alone with less effort and less time, but it is worth far less than a stag and has much less meat. Rousseau therefore posits it would be much better for each hunter, acting individually, to give up total autonomy and minimal risk, which brings only the small reward of the hare. Instead, each hunter should separately choose the more ambitious and far more rewarding goal of getting the stag, thereby giving up some autonomy in exchange for the other hunter's cooperation and added might. Commentators have seen the situation as a useful analogy for many kinds of social cooperation, such as international agreements on climate change.

In game theory, a trigger strategy is any of a class of strategies employed in a repeated non-cooperative game. A player using a trigger strategy initially cooperates but punishes the opponent if a certain level of defection is observed.

In game theory, folk theorems are a class of theorems describing an abundance of Nash equilibrium payoff profiles in repeated games. The original Folk Theorem concerned the payoffs of all the Nash equilibria of an infinitely repeated game. This result was called the Folk Theorem because it was widely known among game theorists in the 1950s, even though no one had published it. Friedman's (1971) Theorem concerns the payoffs of certain subgame-perfect Nash equilibria (SPE) of an infinitely repeated game, and so strengthens the original Folk Theorem by using a stronger equilibrium concept: subgame-perfect Nash equilibria rather than Nash equilibria.

In game theory, a repeated game is an extensive form game that consists of a number of repetitions of some base game. The stage game is usually one of the well-studied 2-person games. Repeated games capture the idea that a player will have to take into account the impact of his or her current action on the future actions of other players; this impact is sometimes called his or her reputation. Single stage game or single shot game are names for non-repeated games.

In game theory, the purification theorem was contributed by Nobel laureate John Harsanyi in 1973. The theorem aims to justify a puzzling aspect of mixed strategy Nash equilibria: that each player is wholly indifferent amongst each of the actions he puts non-zero weight on, yet he mixes them so as to make every other player also indifferent.

Peace war game is an iterated game originally played in academic groups and by computer simulation for years to study possible strategies of cooperation and aggression. As peace makers became richer over time it became clear that making war had greater costs than initially anticipated. The only strategy that acquired wealth more rapidly was a "Genghis Khan", a constant aggressor making war continually to gain resources. This led to the development of the "provokable nice guy" strategy, a peace-maker until attacked. Multiple players continue to gain wealth cooperating with each other while bleeding the constant aggressor. The Hanseatic League for trade and mutual defense appears to have originated from just such concerns about seaborne raiders.

In game theory, an epsilon-equilibrium, or near-Nash equilibrium, is a strategy profile that approximately satisfies the condition of Nash equilibrium. In a Nash equilibrium, no player has an incentive to change his behavior. In an approximate Nash equilibrium, this requirement is weakened to allow the possibility that a player may have a small incentive to do something different. This may still be considered an adequate solution concept, assuming for example status quo bias. This solution concept may be preferred to Nash equilibrium due to being easier to compute, or alternatively due to the possibility that in games of more than 2 players, the probabilities involved in an exact Nash equilibrium need not be rational numbers.

The Complexity of Cooperation, by Robert Axelrod, 0691015678 is the sequel to The Evolution of Cooperation. It is a compendium of seven articles that previously appeared in journals on a variety of subjects. The book extends Axelrod's method of applying the results of game theory, in particular that derived from analysis of the Prisoner's Dilemma (IPD) problem, to real world situations.

A continuous game is a mathematical concept, used in game theory, that generalizes the idea of an ordinary game like tic-tac-toe or checkers (draughts). In other words, it extends the notion of a discrete game, where the players choose from a finite set of pure strategies. The continuous game concepts allows games to include more general sets of pure strategies, which may be uncountably infinite.

Mertens stability is a solution concept used to predict the outcome of a non-cooperative game. A tentative definition of stability was proposed by Elon Kohlberg and Jean-François Mertens for games with finite numbers of players and strategies. Later, Mertens proposed a stronger definition that was elaborated further by Srihari Govindan and Mertens. This solution concept is now called Mertens stability, or just stability.

Subjective expected relative similarity (SERS) is a normative and descriptive theory that predicts and explains cooperation levels in a family of games termed Similarity Sensitive Games (SSG), among them the well-known Prisoner's Dilemma game (PD). SERS was originally developed in order to (i) provide a new rational solution to the PD game and (ii) to predict human behavior in single-step PD games. It was further developed to account for: (i) repeated PD games, (ii) evolutionary perspectives and, as mentioned above, (iii) the SSG subgroup of 2×2 games. SERS predicts that individuals cooperate whenever their subjectively perceived similarity with their opponent exceeds a situational index derived from the game's payoffs, termed the similarity threshold of the game. SERS proposes a solution to the rational paradox associated with the single step PD and provides accurate behavioral predictions. The theory was developed by Prof. Ilan Fischer at the University of Haifa.

Reciprocal altruism in humans refers to an individual behavior that gives benefit conditionally upon receiving a returned benefit, which draws on the economic concept – ″gains in trade″. Human reciprocal altruism would include the following behaviors : helping patients, the wounded, and the others when they are in crisis; sharing food, implement, knowledge.

## References

1. Shaun Hargreaves Heap, Yanis Varoufakis (2004). Game theory: a critical text. Routledge. p. 191. ISBN   978-0-415-25094-8.
2. Forsyth, D.R. (2010) Group Dynamics
3. Gintis, Herbert (2000). Game Theory Evolving. Princeton University Press. ISBN   978-0-691-00943-8.
4. Boyd, Robert (1989). "Mistakes Allow Evolutionary Stability in the Repeated Prisoner's Dilemma Game". Journal of Theoretical Biology. 136 (1): 47–56. CiteSeerX  . doi:10.1016/S0022-5193(89)80188-2. PMID   2779259.
5. "Knife-Edge Equilibria – Game Theory 101" . Retrieved 2018-12-10.
6. Dawkins, Richard (1989). The Selfish Gene. Oxford University Press. ISBN   978-0-19-929115-1.
7. Axelrod, Robert (2000-01-01). "On Six Advances in Cooperation Theory". Analyse & Kritik. 22 (1). CiteSeerX  . doi:10.1515/auk-2000-0107. ISSN   2365-9858.
8. Axelrod, Robert (1984). . Basic Books. ISBN   978-0-465-02121-5.
9. Cohen, Bram (2003-05-22). "Incentives Build Robustness in BitTorrent" (PDF). BitTorrent.org. Retrieved 2011-02-05.
10. Nice Guys Finish First . Richard Dawkins. BBC. 1986.