Superforecasting: The Art and Science of Prediction

Last updated

Superforecasting
Superforecasting.jpg
First edition
Author Philip E. Tetlock
Published2015 (Crown Publishers)
Pages340
ISBN 9780804136693
LC Class HB3730 .T47X 2015

Superforecasting: The Art and Science of Prediction is a 2015 book by Philip E. Tetlock and Dan Gardner. It details findings from The Good Judgment Project.

Contents

Reviews

The Economist reports that superforecasters are clever (with a good mental attitude), but not necessarily geniuses. It reports on the treasure trove of data coming from The Good Judgment Project, showing that accurately selected amateur forecasters (and the confidence they had in their forecasts) were often more accurately tuned than experts. [1] According to The Wall Street Journal , Superforecasting is "The most important book on decision making since Daniel Kahneman's Thinking, Fast and Slow ." [2] The Harvard Business Review paired it to the book How Not to Be Wrong: The Power of Mathematical Thinking by Jordan Ellenberg. [3] On the other hand, Klaus Solberg Søilen reviewing the book for the Journal of Intelligence Studies in Business was very critical, saying that the book did not give enough information on how the Good Judgment Project beat other forecasters. [4]

See also

Related Research Articles

<span class="mw-page-title-main">Cognitive bias</span> Systematic pattern of deviation from norm or rationality in judgment

A cognitive bias is a systematic pattern of deviation from norm or rationality in judgment. Individuals create their own "subjective reality" from their perception of the input. An individual's construction of reality, not the objective input, may dictate their behavior in the world. Thus, cognitive biases may sometimes lead to perceptual distortion, inaccurate judgment, illogical interpretation, and irrationality.

A heuristic, or heuristic technique, is any approach to problem solving that employs a practical method that is not fully optimized, perfected, or rationalized, but is nevertheless sufficient for reaching an immediate, short-term goal or approximation. Where finding an optimal solution is impossible or impractical, heuristic methods can be used to speed up the process of finding a satisfactory solution. Heuristics can be mental shortcuts that ease the cognitive load of making a decision.

<span class="mw-page-title-main">Daniel Kahneman</span> Israeli-American psychologist and economist (1934–2024)

Daniel Kahneman was an Israeli-American author, psychologist, and economist notable for his work on hedonism, the psychology of judgment, and decision-making. He is also known for his work in behavioral economics, for which he was awarded the 2002 Nobel Memorial Prize in Economic Sciences shared with Vernon L. Smith. Kahneman's published empirical findings challenge the assumption of human rationality prevailing in modern economic theory. Kahneman became known as the "grandfather of behavioral economics."

Prediction markets, also known as betting markets, information markets, decision markets, idea futures or event derivatives, are open markets that enable the prediction of specific outcomes using financial incentives. They are exchange-traded markets established for trading bets in the outcome of various events. The market prices can indicate what the crowd thinks the probability of the event is. A typical prediction market contract is set up to trade between 0 and 100%. The most common form of a prediction market is a binary option market, which will expire at the price of 0 or 100%. Prediction markets can be thought of as belonging to the more general concept of crowdsourcing which is specially designed to aggregate information on particular topics of interest. The main purposes of prediction markets are eliciting aggregating beliefs over an unknown future outcome. Traders with different beliefs trade on contracts whose payoffs are related to the unknown future outcome and the market prices of the contracts are considered as the aggregated belief.

There are two main uses of the term calibration in statistics that denote special types of statistical inference problems. Calibration can mean

<span class="mw-page-title-main">Planning fallacy</span> Cognitive bias of underestimating time needed

The planning fallacy is a phenomenon in which predictions about how much time will be needed to complete a future task display an optimism bias and underestimate the time needed. This phenomenon sometimes occurs regardless of the individual's knowledge that past tasks of a similar nature have taken longer to complete than generally planned. The bias affects predictions only about one's own tasks. On the other hand, when outside observers predict task completion times, they tend to exhibit a pessimistic bias, overestimating the time needed. The planning fallacy involves estimates of task completion times more optimistic than those encountered in similar projects in the past.

The overconfidence effect is a well-established bias in which a person's subjective confidence in their judgments is reliably greater than the objective accuracy of those judgments, especially when confidence is relatively high. Overconfidence is one example of a miscalibration of subjective probabilities. Throughout the research literature, overconfidence has been defined in three distinct ways: (1) overestimation of one's actual performance; (2) overplacement of one's performance relative to others; and (3) overprecision in expressing unwarranted certainty in the accuracy of one's beliefs.

<i>The Hedgehog and the Fox</i> 1953 book by Isaiah Berlin

The Hedgehog and the Fox is an essay by philosopher Isaiah Berlin that was published as a book in 1953. It was one of his most popular essays with the general public. However, Berlin said, "I meant it as a kind of enjoyable intellectual game, but it was taken seriously. Every classification throws light on something". It has been compared to "an intellectual's cocktail-party game".

Reference class forecasting or comparison class forecasting is a method of predicting the future by looking at similar past situations and their outcomes. The theories behind reference class forecasting were developed by Daniel Kahneman and Amos Tversky. The theoretical work helped Kahneman win the Nobel Prize in Economics.

<span class="mw-page-title-main">Philip E. Tetlock</span> Canadian-American political scientist

Philip E. Tetlock is a Canadian-American political science writer, and is currently the Annenberg University Professor at the University of Pennsylvania, where he is cross-appointed at the Wharton School and the School of Arts and Sciences. He was elected a Member of the American Philosophical Society in 2019.

<i>Thinking, Fast and Slow</i> 2011 book by Daniel Kahneman

Thinking, Fast and Slow is a 2011 popular science book by psychologist Daniel Kahneman. The book's main thesis is a differentiation between two modes of thought: "System 1" is fast, instinctive and emotional; "System 2" is slower, more deliberative, and more logical.

Illusion of validity is a cognitive bias in which a person overestimates their ability to interpret and predict accurately the outcome when analyzing a set of data, in particular when the data analyzed show a very consistent pattern—that is, when the data "tell" a coherent story.

The Good Judgment Project (GJP) is an organization dedicated to "harnessing the wisdom of the crowd to forecast world events". It was co-created by Philip E. Tetlock, decision scientist Barbara Mellers, and Don Moore, all professors at the University of Pennsylvania.

An information technology generalist is a technology professional proficient in many facets of information technology without any specific specialty. Furthermore, an IT generalist is generally considered to possess general business knowledge and soft skills allowing them to be adaptable in a wide array of work environments. The IT Generalist is often able to fulfill many different roles within a company depending on specific technology needs. In a small business environment, budgets often delegate many different facets of technology to a single individual, especially considering a small business will often require an individual proficient in desktop support, web page design, databases, phone systems, and even server administration. The role of the IT Generalist within a larger company, however, often becomes more of a project leader or integrations specialist due to a project team consisting of a varying degree of IT specialists and interfacing with end-users requiring soft-skills.

<span class="mw-page-title-main">Francesca Rossi</span> Italian computer scientist

Francesca Rossi is an Italian computer scientist, currently working at the IBM Thomas J. Watson Research Center as an IBM Fellow and the IBM AI Ethics Global Leader.

Debiasing is the reduction of bias, particularly with respect to judgment and decision making. Biased judgment and decision making is that which systematically deviates from the prescriptions of objective standards such as facts, logic, and rational behavior or prescriptive norms. Biased judgment and decision making exists in consequential domains such as medicine, law, policy, and business, as well as in everyday life. Investors, for example, tend to hold onto falling stocks too long and sell rising stocks too quickly. Employers exhibit considerable discrimination in hiring and employment practices, and some parents continue to believe that vaccinations cause autism despite knowing that this link is based on falsified evidence. At an individual level, people who exhibit less decision bias have more intact social environments, reduced risk of alcohol and drug use, lower childhood delinquency rates, and superior planning and problem solving abilities.

Barbara Ann Mellers is the I. George Heyman University Professor of Psychology at the University of Pennsylvania. Her research focuses on decision processes.

A superforecaster is a person who makes forecasts that can be shown by statistical means to have been consistently more accurate than the general public or experts. Superforecasters sometimes use modern analytical and statistical methodologies to augment estimates of base rates of events; research finds that such forecasters are typically more accurate than experts in the field who do not use analytical and statistical techniques. The term "superforecaster" is a trademark of Good Judgment Inc.

Joshua Frankel is an American contemporary artist and director who makes work in many different media, including animation, film, opera, drawing, printmaking and public art.

<i>Noise: A Flaw in Human Judgment</i> 2021 book by Daniel Kahneman, Olivier Sibony, and Cass Sunstein

Noise: A Flaw in Human Judgment is a nonfiction book by professors Daniel Kahneman, Olivier Sibony and Cass Sunstein. It was first published on May 18, 2021. The book concerns 'noise' in human judgment and decision-making. The authors define noise in human judgment as "undesirable variability in judgments of the same problem" and focus on the statistical properties and psychological perspectives of the issue.

References

  1. "Unclouded vision". The Economist. September 26, 2015. Retrieved September 26, 2015.
  2. Zweig, Jason. "Can You See the Future? Probably Better Than Professional Forecasters". The Wall Street Journal. Retrieved September 25, 2015. I think Philip Tetlock's "Superforecasting: The Art and Science of Prediction," ..., is the most important book on decision making since Daniel Kahneman's "Thinking, Fast and Slow."
  3. Frick, Walter. "Question Certainty". Harvard Business Review. Retrieved September 26, 2015.
  4. Søilen, Klaus Solberg (2018). "Book review: Superforecasting: The art and science of prediction. Crown Publishers, New York, NY. Tetlock. E. Philip, Gardner, Dan (2015)". Journal of Intelligence Studies in Business. 8 (1): 46–53. Retrieved April 19, 2024.