Goodhart's law

Last updated

Goodhart's law is an adage named after British economist Charles Goodhart, who advanced the idea in a 1975 article on monetary policy in the United Kingdom, Problems of Monetary Management: the U.K. Experience: [1] [2]

Contents

Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.

It later became used to criticize the British Thatcher government for trying to conduct monetary policy on the basis of targets for broad and narrow money. [3]

Generalization by Marilyn Strathern

In a paper published in 1997, Anthropologist Marilyn Strathern generalized Goodhart's law beyond statistics and control to evaluation more broadly. The phrase commonly referred to as Goodhart's law comes from Strathern's paper, not from any of Goodhart's writings:

When a measure becomes a target, it ceases to be a good measure. [4]

One way in which this can occur is individuals trying to anticipate the effect of a policy and then taking actions that alter its outcome. [5]

Priority and background

There are numerous concepts related to this idea, at least one of which predates Goodhart's statement of 1975. [6] Notably, Campbell's law likely has precedence, as Jeff Rodamar has argued, since various formulations date to 1969. [7] Other academics had similar insights during this time period. Jerome Ravetz's 1971 book Scientific Knowledge and Its Social Problems [8] also predates Goodhart, though it does not formulate the same law. He discusses how systems in general can be gamed, focuses on cases where the goals of a task are complex, sophisticated, or subtle. In such cases, the persons possessing the skills to execute the tasks properly are instead able to achieve their own goals to the detriment of the assigned tasks. When the goals are instantiated as metrics, this could be seen as equivalent to Goodhart and Campbell's claim.

Shortly after Goodhart's publication, others suggested closely related ideas, including the Lucas critique (1976). As applied in economics, the law is also implicit in the idea of rational expectations, a theory in economics that states that those who are aware of a system of rewards and punishments will optimize their actions within that system to achieve their desired results. For example, if an employee is rewarded by the number of cars sold each month, they will try to sell more cars, even at a loss.

While it originated in the context of market responses, the law has profound implications for the selection of high-level targets in organizations. [2] Jon Danielsson quotes the law as "Any statistical relationship will break down when used for policy purposes", and suggests a corollary to the law for use in financial risk modelling: "A risk model breaks down when used for regulatory purposes." [9] Mario Biagioli has related the concept to consequences of using citation impact measures to estimate the importance of scientific publications: [10]

All metrics of scientific evaluation are bound to be abused. Goodhart's law [...] states that when a feature of the economy is picked as an indicator of the economy, then it inexorably ceases to function as that indicator because people start to game it.

The law is illustrated in the 2018 book The Tyranny of Metrics by Jerry Z. Muller. [11]

See also

Related Research Articles

Macroeconomics Branch of economics that studies aggregated indicators

Macroeconomics means using interest rates, taxes and government spending to regulate an economy’s growth and stability. It is a branch of economics dealing with the performance, structure, behavior, and decision-making of an economy as a whole. This includes regional, national, and global economies. Macroeconomists study topics such as GDP, unemployment rates, national income, price indices, output, consumption, unemployment, inflation, saving, investment, energy, international trade, and international finance.

Post-Keynesian economics

Post-Keynesian economics is a school of economic thought with its origins in The General Theory of John Maynard Keynes, with subsequent development influenced to a large degree by Michał Kalecki, Joan Robinson, Nicholas Kaldor, Sidney Weintraub, Paul Davidson, Piero Sraffa and Jan Kregel. Historian Robert Skidelsky argues that the post-Keynesian school has remained closest to the spirit of Keynes' original work. It is a heterodox approach to economics.

This aims to be a complete article list of economics topics:

In the economic study of the public sector, economic and social development is the process by which the economic well-being and quality of life of a nation, region, local community, or an individual are improved according to targeted goals and objectives.

The Lucas critique, named for Robert Lucas's work on macroeconomic policymaking, argues that it is naive to try to predict the effects of a change in economic policy entirely on the basis of relationships observed in historical data, especially highly aggregated historical data. More formally, it states that the decision rules of Keynesian models—such as the consumption function—cannot be considered as structural in the sense of being invariant with respect to changes in government policy variables. The Lucas critique is significant in the history of economic thought as a representative of the paradigm shift that occurred in macroeconomic theory in the 1970s towards attempts at establishing micro-foundations.

Gunnar Myrdal Swedish economist

Karl Gunnar Myrdal was a Swedish economist and sociologist. In 1974, he received the Nobel Memorial Prize in Economic Sciences with Friedrich Hayek for "their pioneering work in the theory of money and economic fluctuations and for their penetrating analysis of the interdependence of economic, social and institutional phenomena." He is best known in the United States for his study of race relations, which culminated in his book An American Dilemma: The Negro Problem and Modern Democracy. The study was influential in the 1954 landmark U.S. Supreme Court Decision Brown v. Board of Education. In Sweden, his work and political influence were important to the establishment of the Folkhemmet and the welfare state.

The green economy is defined as economy that aims at making issues of reducing environmental risks and ecological scarcities, and that aims for sustainable development without degrading the environment. It is closely related with ecological economics, but has a more politically applied focus. The 2011 UNEP Green Economy Report argues "that to be green, an economy must not only be efficient, but also fair. Fairness implies recognizing global and country level equity dimensions, particularly in assuring a Just Transition to an economy that is low-carbon, resource efficient, and socially inclusive."

Scientometrics is the field of study which concerns itself with measuring and analysing scientific literature. Scientometrics is a sub-field of bibliometrics. Major research issues include the measurement of the impact of research papers and academic journals, the understanding of scientific citations, and the use of such measurements in policy and management contexts. In practice there is a significant overlap between scientometrics and other scientific fields such as information systems, information science, science of science policy, sociology of science, and metascience. Critics have argued that over-reliance on scientometrics has created a system of perverse incentives, producing a publish or perish environment that leads to low quality research.

Post-normal science

Post-normal science (PNS) represents a novel approach for the use of science on issues where "facts [are] uncertain, values in dispute, stakes high and decisions urgent". PNS was developed in the 1990s by Silvio Funtowicz and Jerome R. Ravetz. It can be considered as a reaction to the styles of analysis based on risk and cost-benefit analysis prevailing at that time, and as an embodiment of concepts of a new "critical science" developed in previous works by the same authors. In a more recent work PNS is described as "the stage where we are today, where all the comfortable assumptions about science, its production and its use, are in question".

Charles Goodhart

Charles Albert Eric Goodhart, is a British economist. He was a member of the Bank of England's Monetary Policy Committee from June 1997 to May 2000 and a professor at the London School of Economics. He is the developer of Goodhart's law, an economic law named after him. He is the son of Arthur Lehman Goodhart, and the brother of William Goodhart and Sir Philip Goodhart.

Vladimir Martynenko

Vladimir Vladimirovich Martynenko is a Russian sociologist, economist, and political scientist; Doctor of political sciences, Professor, Chief Scientific Officer, Institute of Socio-Political Studies under the Russian Academy of Sciences.

Campbell's law is an adage developed by Donald T. Campbell, a psychologist and social scientist who often wrote about research methodology, which states:

"The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor."

Karl Brunner was a Swiss economist.

Jerome Ravetz

Jerome (Jerry) Ravetz is a philosopher of science. He is best known for his books analysing scientific knowledge from a social and ethical perspective, focussing on issues of quality. He is the co-author of the NUSAP notational system and of Post-normal science. He is currently an Associate Fellow at the Institute for Science, Innovation and Society, University of Oxford.

Gaming the system can be defined as using the rules and procedures meant to protect a system to, instead, manipulate the system for a desired outcome.

The McNamara fallacy, named for Robert McNamara, the US Secretary of Defense from 1961 to 1968, involves making a decision based solely on quantitative observations and ignoring all others. The reason given is often that these other observations cannot be proven.

The first step is to measure whatever can be easily measured. This is OK as far as it goes. The second step is to disregard that which can't be easily measured or to give it an arbitrary quantitative value. This is artificial and misleading. The third step is to presume that what can't be measured easily really isn't important. This is blindness. The fourth step is to say that what can't be easily measured really doesn't exist. This is suicide.

Market monetarism is a school of macroeconomic thought that advocates that central banks target the level of nominal income instead of inflation, unemployment, or other measures of economic activity, including in times of shocks such as the bursting of the real estate bubble in 2006, and in the financial crisis that followed. In contrast to traditional monetarists, market monetarists do not believe monetary aggregates or commodity prices such as gold are the optimal guide to intervention. Market monetarists also reject the New Keynesian focus on interest rates as the primary instrument of monetary policy. Market monetarists prefer a nominal income target due to their twin beliefs that rational expectations are crucial to policy, and that markets react instantly to changes in their expectations about future policy, without the "long and variable lags" postulated by Milton Friedman.

Sensitivity auditing is an extension of sensitivity analysis for use in policy-relevant modelling studies. Its use is recommended - e.g. in the European Commission Impact assessment guidelines and by the European Science Academies - when a sensitivity analysis (SA) of a model-based study is meant to demonstrate the robustness of the evidence provided by the model, but in a context where the inference feeds into a policy or decision-making process.

<i>Uncertainty and Quality in Science for Policy</i>

Uncertainty and Quality in Science for Policy is a 1990 book by Silvio Funtowicz and Jerome Ravetz, in which the authors explain the notational system NUSAP and applies it to several examples from the environmental sciences. The work is considered foundational to the development of post-normal science.

<i>Science on the Verge</i>

Science on the verge is a book written in 2016 by group of eight scholars working in the tradition of Post-normal science. The book analyzes the main features and possible causes of the present science's crisis.

References

  1. Goodhart, Charles (1975). "Problems of Monetary Management: The U.K. Experience". Papers in Monetary Economics. 1. Sydney: Reserve Bank of Australia.CS1 maint: date and year (link)
  2. 1 2 Goodhart, Charles (1975). "Problems of Monetary Management: The U.K. Experience". In Courakis, Anthony S. (ed.). Inflation, Depression, and Economic Policy in the West. Totowa, New Jersey: Barnes and Noble Books (published 1981). p. 116. ISBN   0-389-20144-8.
  3. Smith, David (1987). The Rise And Fall of Monetarism. London: Penguin.
  4. Strathern, Marilyn (1997). "'Improving ratings': audit in the British University system". European Review. John Wiley & Sons. 5 (3): 305–321. doi:10.1002/(SICI)1234-981X(199707)5:3<05::AID-EURO184>3.0.CO;2-4.
  5. Manheim, David; Garrabrant, Scott (2018). "Categorizing Variants of Goodhart's Law". arXiv: 1803.04585 [cs.AI].
  6. Manheim, David (29 September 2016). "Overpowered Metrics Eat Underspecified Goals". ribbonfarm. Retrieved 26 January 2017.
  7. Rodamar, Jeffery (28 November 2018). "There ought to be a law! Campbell versus Goodhart". Significance. 15 (6): 9. doi: 10.1111/j.1740-9713.2018.01205.x .
  8. Ravetz, Jerome R. (1971). Scientific knowledge and its social problems. New Brunswick, New Jersey: Transaction Publishers. pp. 295–296. ISBN   1-56000-851-2. OCLC   32779931.
  9. Daníelsson, Jón (July 2002). "The Emperor has no Clothes: Limits to Risk Modelling". Journal of Banking & Finance. 26 (7): 1273–1296. CiteSeerX   10.1.1.27.3392 . doi:10.1016/S0378-4266(02)00263-7.
  10. Biagioli, Mario (12 July 2016). "Watch out for cheats in citation game" (PDF). Nature. 535 (7611): 201. Bibcode:2016Natur.535..201B. doi:10.1038/535201a. PMID   27411599.
  11. Muller, Jerry Z. (2018). The Tyranny of Metrics. Princeton University Press. ISBN   978-0-691-19126-3.

Further reading