Goodhart's law

Last updated
Charles Goodhart, for whom the adage is named, delivering a speech in 2012 Charles Goodhart delives the 2012 Long Finance conference keynote speech.JPG
Charles Goodhart, for whom the adage is named, delivering a speech in 2012

Goodhart's law is an adage often stated as, "When a measure becomes a target, it ceases to be a good measure". [1] It is named after British economist Charles Goodhart, who is credited with expressing the core idea of the adage in a 1975 article on monetary policy in the United Kingdom: [2]

Contents

Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes. [3]

It was used to criticize the British Thatcher government for trying to conduct monetary policy on the basis of targets for broad and narrow money, [4] but the law reflects a much more general phenomenon. [5]

Priority and background

Numerous concepts are related to this idea, at least one of which predates Goodhart's statement. [6] Notably, Campbell's law likely has precedence, as Jeff Rodamar has argued, since various formulations date to 1969. [7] Other academics had similar insights at the time. Jerome Ravetz's 1971 book Scientific Knowledge and Its Social Problems [8] also predates Goodhart, though it does not formulate the same law. He discusses how systems in general can be gamed, focuses on cases where the goals of a task are complex, sophisticated, or subtle. In such cases, the persons possessing the skills to execute the tasks properly seek their own goals to the detriment of the assigned tasks. When the goals are instantiated as metrics, this could be seen as equivalent to Goodhart and Campbell's claim.

Shortly after Goodhart's publication, others suggested closely related ideas, including the Lucas critique (1976). As applied in economics, the law is also implicit in the idea of rational expectations, a theory in economics that states that those who are aware of a system of rewards and punishments will optimize their actions within that system to achieve their desired results. For example, if an employee is rewarded by the number of cars sold each month, they will try to sell more cars, even at a loss.

While it originated in the context of market responses, the law has profound implications for the selection of high-level targets in organizations. [3] Jon Danielsson states the law as

Any statistical relationship will break down when used for policy purposes.

Jon Danielsson

He suggested a corollary for use in financial risk modelling:

A risk model breaks down when used for regulatory purposes. [9]

Jon Danielsson

Mario Biagioli related the concept to consequences of using citation impact measures to estimate the importance of scientific publications: [10] [11]

All metrics of scientific evaluation are bound to be abused. Goodhart's law [...] states that when a feature of the economy is picked as an indicator of the economy, then it inexorably ceases to function as that indicator because people start to game it.

Mario Biagioli

The law is illustrated in the 2018 book The Tyranny of Metrics by Jerry Z. Muller. [12]

Generalization

Later writers generalized Goodhart's point about monetary policy into a more general adage about measures and targets in accounting and evaluation systems. In a book chapter published in 1996, Keith Hoskin wrote:

'Goodhart's Law' – That every measure which becomes a target becomes a bad measure – is inexorably, if ruefully, becoming recognized as one of the overriding laws of our times. Ruefully, for this law of the unintended consequence seems so inescapable. But it does so, I suggest, because it is the inevitable corollary of that invention of modernity: accountability. [13]

In a 1997 paper responding to the work of Hoskin and others on financial accounting and grades in education, anthropologist Marilyn Strathern expressed Goodhart's Law as "When a measure becomes a target, it ceases to be a good measure," and linked the sentiment to the history of accounting stretching back into Britain in the 1800s:

When a measure becomes a target, it ceases to be a good measure. The more a 2.1 examination performance becomes an expectation, the poorer it becomes as a discriminator of individual performances. Hoskin describes this as 'Goodhart's law', after the latter's observation on instruments for monetary control which led to other devices for monetary flexibility having to be invented. However, targets that seem measurable become enticing tools for improvement. The linking of improvement to commensurable increase produced practices of wide application. It was that conflation of 'is' and 'ought', alongside the techniques of quantifiable written assessments, which led in Hoskin's view to the modernist invention of accountability. This was articulated in Britain for the first time around 1800 as 'the awful idea of accountability' (Ref. 3, p. 268). [1]

Examples

See also

Related Research Articles

<span class="mw-page-title-main">Macroeconomics</span> Study of an economy as a whole

Macroeconomics is a branch of economics that deals with the performance, structure, behavior, and decision-making of an economy as a whole. This includes regional, national, and global economies. Macroeconomists study topics such as output/GDP and national income, unemployment, price indices and inflation, consumption, saving, investment, energy, international trade, and international finance.

<span class="mw-page-title-main">Economic development</span> Process and policies to improve economic well-being

In the economics study of the public sector, economic and social development is the process by which the economic well-being and quality of life of a nation, region, local community, or an individual are improved according to targeted goals and objectives.

<span class="mw-page-title-main">Money supply</span> Total value of money available in an economy at a specific point in time

In macroeconomics, money supply refers to the total volume of money held by the public at a particular point in time. There are several ways to define "money", but standard measures usually include currency in circulation and demand deposits. Money supply data is recorded and published, usually by the national statistical agency or the central bank of the country. Empirical money supply measures are usually named M1, M2, M3, etc., according to how wide a definition of money they embrace. The precise definitions vary from country to country, in part depending on national financial institutional traditions.

<span class="mw-page-title-main">Monetary policy</span> Policy of interest rates or money supply

Monetary policy is the policy adopted by the monetary authority of a nation to affect monetary and other financial conditions to accomplish broader objectives like high employment and price stability. Further purposes of a monetary policy may be to contribute to economic stability or to maintain predictable exchange rates with other currencies. Today most central banks in developed countries conduct their monetary policy within an inflation targeting framework, whereas the monetary policies of most developing countries' central banks target some kind of a fixed exchange rate system. A third monetary policy strategy, targeting the money supply, was widely followed during the 1980s, but has diminished in popularity since that, though it is still the official strategy in a number of emerging economies.

<span class="mw-page-title-main">Unit of account</span> Money function

In economics, unit of account is one of the functions of money. A unit of account is a standard numerical monetary unit of measurement of the market value of goods, services, and other transactions. Also known as a "measure" or "standard" of relative worth and deferred payment, a unit of account is a necessary prerequisite for the formulation of commercial agreements that involve debt.

The Lucas critique argues that it is naïve to try to predict the effects of a change in economic policy entirely on the basis of relationships observed in historical data, especially highly aggregated historical data. More formally, it states that the decision rules of Keynesian models—such as the consumption function—cannot be considered as structural in the sense of being invariant with respect to changes in government policy variables. It was named after American economist Robert Lucas's work on macroeconomic policymaking.

<span class="mw-page-title-main">Post-normal science</span> Approach to the use of science on urgent issues involving uncertainty in facts and moral values

Post-normal science (PNS) was developed in the 1990s by Silvio Funtowicz and Jerome R. Ravetz. It is a problem-solving strategy appropriate when "facts [are] uncertain, values in dispute, stakes high and decisions urgent", conditions often present in policy-relevant research. In those situations, PNS recommends suspending temporarily the traditional scientific ideal of truth, concentrating on quality as assessed by internal and extended peer communities.

<span class="mw-page-title-main">Charles Goodhart</span> British economist

Charles Albert Eric Goodhart, is a British economist. He worked at the Bank of England on its public policy from 1968–1985; and worked at the London School of Economics from 1966–1968 and 1986–2002. Charles Goodhart's work focuses on central bank governance practices and monetary frameworks. He also conducted academic research into foreign exchange markets. He is best known for formulating Goodhart's Law, which states: "When a measure becomes a target, it ceases to be a good measure."

<span class="mw-page-title-main">Aggregate data</span> Data combined from several measurements

Aggregate data is high-level data which is acquired by combining individual-level data. For instance, the output of an industry is an aggregate of the firms’ individual outputs within that industry. Aggregate data are applied in statistics, data warehouses, and in economics.

Campbell's law is an adage developed by Donald T. Campbell, a psychologist and social scientist who often wrote about research methodology, which states:

The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.

Reification is a fallacy of ambiguity, when an abstraction is treated as if it were a concrete real event or physical entity. In other words, it is the error of treating something that is not concrete, such as an idea, as a concrete thing. A common case of reification is the confusion of a model with reality: "the map is not the territory".

<span class="mw-page-title-main">Sustainability measurement</span> Quantitative basis for the informed management of sustainability

Sustainability measurement is a set of frameworks or indicators to measure how sustainable something is. This includes processes, products, services and businesses. Sustainability is difficult to quantify. It may even be impossible to measure. To measure sustainability, the indicators consider environmental, social and economic domains. The metrics are still evolving. They include indicators, benchmarks and audits. They include sustainability standards and certification systems like Fairtrade and Organic. They also involve indices and accounting. And they can include assessment, appraisal and other reporting systems. These metrics are used over a wide range of spatial and temporal scales. Sustainability measures include corporate sustainability reporting, Triple Bottom Line accounting. They include estimates of the quality of sustainability governance for individual countries. These use the Environmental Sustainability Index and Environmental Performance Index. Some methods let us track sustainable development. These include the UN Human Development Index and ecological footprints.

The McNamara fallacy, named for Robert McNamara, the US Secretary of Defense from 1961 to 1968, involves making a decision based solely on quantitative observations and ignoring all others. The reason given is often that these other observations cannot be proven.

But when the McNamara discipline is applied too literally, the first step is to measure whatever can be easily measured. The second step is to disregard that which can't easily be measured or given a quantitative value. The third step is to presume that what can't be measured easily really isn't important. The fo[u]rth step is to say that what can't be easily measured really doesn't exist. This is suicide.

<span class="mw-page-title-main">Jon Danielsson</span> Icelandic economist

Jón Danielsson is an icelandic economist teaching at the London School of Economics. Danielssons work focuses on artificial intelligence, risk forecasting, international finance, cryptocurrencies, and systemic causes of financial instability. He has written several books on finance and risk analysis, and is active in both domestic and international policy debates on financial regualtion.

Market monetarism is a school of macroeconomic thought that advocates that central banks target the level of nominal income instead of inflation, unemployment, or other measures of economic activity, including in times of shocks such as the bursting of the real estate bubble in 2006, and in the financial crisis that followed. In contrast to traditional monetarists, market monetarists do not believe monetary aggregates or commodity prices such as gold are the optimal guide to intervention. Market monetarists also reject the New Keynesian focus on interest rates as the primary instrument of monetary policy. Market monetarists prefer a nominal income target due to their twin beliefs that rational expectations are crucial to policy, and that markets react instantly to changes in their expectations about future policy, without the "long and variable lags" postulated by Milton Friedman.

The zero lower bound (ZLB) or zero nominal lower bound (ZNLB) is a macroeconomic problem that occurs when the short-term nominal interest rate is at or near zero, causing a liquidity trap and limiting the central bank's capacity to stimulate economic growth.

Sensitivity auditing is an extension of sensitivity analysis for use in policy-relevant modelling studies. Its use is recommended - i.a. in the European Commission Impact assessment guidelines and by the European Science Academies- when a sensitivity analysis (SA) of a model-based study is meant to demonstrate the robustness of the evidence provided by the model in the context whereby the inference feeds into a policy or decision-making process.

<span class="mw-page-title-main">Silvio Funtowicz</span> Philosopher of science

Silvio O. Funtowicz is a philosopher of science active in the field of science and technology studies. He created the NUSAP, a notational system for characterising uncertainty and quality in quantitative expressions, and together with Jerome R. Ravetz he introduced the concept of post-normal science. He is currently a guest researcher at the Centre for the Study of the Sciences and the Humanities (SVT), University of Bergen (Norway).

The Barnett critique, named for the work of William A. Barnett in monetary economics, argues that internal inconsistency between the aggregation theory used to produce monetary aggregates and the economic theory used to produce the models within which the aggregates are used are responsible for the appearance of unstable demand and supply for money. The Barnett critique has produced a long and growing literature on monetary aggregation and index number theory and the use of the resulting aggregates in econometric modeling and monetary policy.

Metric fixation refers to a tendency for decision-makers to place excessively large emphases on selected metrics.

References

  1. 1 2 Strathern, Marilyn (1997). "'Improving ratings': audit in the British University system". European Review. John Wiley & Sons. 5 (3): 305–321. doi:10.1002/(SICI)1234-981X(199707)5:3<305::AID-EURO184>3.0.CO;2-4. S2CID   145644958.
  2. Goodhart, Charles (1975). "Problems of Monetary Management: The U.K. Experience". Papers in Monetary Economics. Papers in monetary economics 1975; 1; 1. - [Sydney]. - 1975, p. 1-20. Vol. 1. Sydney: Reserve Bank of Australia.
  3. 1 2 Goodhart, Charles (1975). "Problems of Monetary Management: The U.K. Experience". In Courakis, Anthony S. (ed.). Inflation, Depression, and Economic Policy in the West. Totowa, New Jersey: Barnes and Noble Books (published 1981). p. 116. ISBN   0-389-20144-8.
  4. Smith, David (1987). The Rise And Fall of Monetarism. London: Penguin Books. ISBN   9780140227543.
  5. Manheim, David; Garrabrant, Scott (2018). "Categorizing Variants of Goodhart's Law". arXiv: 1803.04585 [cs.AI].
  6. Manheim, David (29 September 2016). "Overpowered Metrics Eat Underspecified Goals". ribbonfarm. Retrieved 26 January 2017.
  7. Rodamar, Jeffery (28 November 2018). "There ought to be a law! Campbell versus Goodhart". Significance. 15 (6): 9. doi: 10.1111/j.1740-9713.2018.01205.x .
  8. Ravetz, Jerome R. (1971). Scientific knowledge and its social problems. New Brunswick, New Jersey: Transaction Publishers. pp. 295–296. ISBN   1-56000-851-2. OCLC   32779931.
  9. Daníelsson, Jón (July 2002). "The Emperor has no Clothes: Limits to Risk Modelling". Journal of Banking & Finance. 26 (7): 1273–1296. CiteSeerX   10.1.1.27.3392 . doi:10.1016/S0378-4266(02)00263-7.
  10. Biagioli, Mario (12 July 2016). "Watch out for cheats in citation game" (PDF). Nature. 535 (7611): 201. Bibcode:2016Natur.535..201B. doi: 10.1038/535201a . PMID   27411599.
  11. Varela, Diego; Benedetto, Giacomo; Sanchez-Santos, Jose Manuel (30 December 2014). "Editorial statement: Lessons from Goodhart's law for the management of the journal". European Journal of Government and Economics. 3 (2): 100–103. doi: 10.17979/ejge.2014.3.2.4299 . hdl: 2183/23376 . S2CID   152551763 . Retrieved 8 February 2022.
  12. Muller, Jerry Z. (2018). The Tyranny of Metrics. Princeton University Press. ISBN   978-0-691-19126-3.
  13. Hoskin, Keith (1996). The 'awful idea of accountability': inscribing people into the measurement of objects.
  14. Koltun, V; Hafner, D (2021). "The h-index is no longer an effective correlate of scientific reputation". PLOS ONE. 16 (6): e0253397. arXiv: 2102.03234 . Bibcode:2021PLoSO..1653397K. doi: 10.1371/journal.pone.0253397 . PMC   8238192 . PMID   34181681. Our results suggest that the use of the h-index in ranking scientists should be reconsidered, and that fractional allocation measures such as h-frac provide more robust alternatives. Companion webpage
  15. Mooers, Arne (2022-05-23). "When is a species really extinct?". The Conversation. Retrieved 2023-06-23.
  16. Martin, T. E.; Bennett, G. C.; Fairbairn, A.; Mooers, A. O. (March 2023). "'Lost' taxa and their conservation implications". Animal Conservation. 26 (1): 14–24. doi:10.1111/acv.12788. ISSN   1367-9430. S2CID   248846699.
  17. Chivers, Tom; Chivers, David (2021). "22: Goodhart's Law". How to Read Numbers . Weidenfeld & Nicolson. ISBN   9781474619974.

Further reading