Surprisingly popular

Last updated

The surprisingly popular answer is a wisdom of the crowd technique that taps into the expert minority opinion within a crowd. [1] For a given question, a group is asked both "What do you think the right answer is?" and "What do you think the popular answer will be?" The answer that maximizes the average difference between the "right" answer and the "popular" answer is the "surprisingly popular" answer. [2] The term "surprisingly popular" was coined in a 2017 paper published in Nature entitled "A solution to the single-question crowd wisdom problem", which outlined the technique. [2] [3]

Contents

Example

Suppose the question to be determined is: Is Philadelphia the capital of Pennsylvania? The two questions asked of the group, and the numbers of responses, are:

Is Philadelphia the capital of Pennsylvania?
  • Yes: 65%
  • No: 35%
What do you think most people will respond to that question?
  • Yes: 75%
  • No: 25%

The difference between the answers to the right question and the popular question:

Thus, the No answer is surprisingly popular (10% > −10%). Because of the relatively high margin of 10%, there can be high confidence that the correct answer is No. (The capital is indeed not Philadelphia, but Harrisburg.)

An illustrative breakdown of this follows. There are four groups of people.

This technique causes groups A and C to be eliminated from consideration and measures the difference in size between groups B and D.

Both groups B and D think they know something others do not, but B is wrong and D is right. In cases where people feel like they have "inside" knowledge, it is more often the case that it is because they are correct and knowledgeable (group D), not because they are misled (group B). [3]

For m>2 candidates, the Surprisingly Popular Algorithm requires votes from an infinite number of voters on all possible ranked permutations (m!) of the alternatives to recover the ground-truth ranking with complete certainty, as discussed in the Nature article. Hosseini et al. (2021) extended the previous work to recover rankings using various elicitation formats. [4]

See also

Related Research Articles

In complexity theory and computability theory, an oracle machine is an abstract machine used to study decision problems. It can be visualized as a Turing machine with a black box, called an oracle, which is able to solve certain problems in a single operation. The problem can be of any complexity class. Even undecidable problems, such as the halting problem, can be used.

<span class="mw-page-title-main">Wisdom</span> Ability to think and act using knowledge, experience, understanding, common sense and insight

Wisdom, sapience, or sagacity is the ability to contemplate and act productively using knowledge, experience, understanding, common sense, and insight. Wisdom is associated with attributes such as unbiased judgment, compassion, experiential self-knowledge, self-transcendence, and non-attachment, and virtues such as ethics and benevolence.

Knights and Knaves is a type of logic puzzle where some characters can only answer questions truthfully, and others only falsely. The name was coined by Raymond Smullyan in his 1978 work What Is the Name of This Book?

Prediction markets, also known as betting markets, information markets, decision markets, idea futures or event derivatives, are open markets that enable the prediction of specific outcomes using financial incentives. They are exchange-traded markets established for trading bets in the outcome of various events. The market prices can indicate what the crowd thinks the probability of the event is. A typical prediction market contract is set up to trade between 0 and 100%. The most common form of a prediction market is a binary option market, which will expire at the price of 0 or 100%. Prediction markets can be thought of as belonging to the more general concept of crowdsourcing which is specially designed to aggregate information on particular topics of interest. The main purposes of prediction markets are eliciting aggregating beliefs over an unknown future outcome. Traders with different beliefs trade on contracts whose payoffs are related to the unknown future outcome and the market prices of the contracts are considered as the aggregated belief.

<span class="mw-page-title-main">Question</span> Request for information

A question is an utterance which serves as a request for information. Questions are sometimes distinguished from interrogatives, which are the grammatical forms typically used to express them. Rhetorical questions, for instance, are interrogative in form but may not be considered bona fide questions, as they are not expected to be answered.

<i>Protagoras</i> (dialogue) Platonic dialogue

Protagoras is a dialogue by Plato. The traditional subtitle is "or the Sophists". The main argument is between Socrates and the elderly Protagoras, a celebrated sophist and philosopher. The discussion takes place at the home of Callias, who is host to Protagoras while he is in town, and concerns the nature of sophists, the unity and the teachability of virtue. A total of twenty-one people are named as present.

Tomorrow were an English musical group active in the 1960s, whose music touched on psychedelic rock, pop and freakbeat. Despite critical acclaim and support from DJ John Peel, who featured them on his "Perfumed Garden" radio show, the band was not a great success in commercial terms. They were among the first psychedelic bands in England, along with Pink Floyd and Soft Machine. Tomorrow recorded the first John Peel show session on BBC Radio 1 on 21 September 1967. The band included Keith West of "Excerpt from A Teenage Opera" fame on vocals and Steve Howe on guitars, who would later join the British progressive rock band Yes.

<i>The Wisdom of Crowds</i> 2004 book by James Surowiecki

The Wisdom of Crowds: Why the Many Are Smarter Than the Few and How Collective Wisdom Shapes Business, Economies, Societies and Nations, published in 2004, is a book written by James Surowiecki about the aggregation of information in groups, resulting in decisions that, he argues, are often better than could have been made by any single member of the group. The book presents numerous case studies and anecdotes to illustrate its argument, and touches on several fields, primarily economics and psychology.

The Hardest Logic Puzzle Ever is a logic puzzle so called by American philosopher and logician George Boolos and published in The Harvard Review of Philosophy in 1996. Boolos' article includes multiple ways of solving the problem. A translation in Italian was published earlier in the newspaper La Repubblica, under the title L'indovinello più difficile del mondo.

The wisdom of the crowd is the collective opinion of a diverse and independent group of individuals rather than that of a single expert. This process, while not new to the Information Age, has been pushed into the mainstream spotlight by social information sites such as Quora, Reddit, Stack Exchange, Wikipedia, Yahoo! Answers, and other web resources which rely on collective human knowledge. An explanation for this phenomenon is that there is idiosyncratic noise associated with each individual judgment, and taking the average over a large number of responses will go some way toward canceling the effect of this noise.

In computability theory and computational complexity theory, an undecidable problem is a decision problem for which it is proved to be impossible to construct an algorithm that always leads to a correct yes-or-no answer. The halting problem is an example: it can be proven that there is no algorithm that correctly determines whether an arbitrary program eventually halts when run.

<i>Symmorium</i> Extinct genus of cartilaginous fishes

Symmorium is a dubious genus of extinct stethacanthid holocephalian from the Devonian and Carboniferous of the United States (Illinois) and Russia. The type species, Symmorium reniforme, was named by Edward Drinker Cope in 1893 and several other species were originally classed under this genus, but they have since been classified into other genera such as Petalodus. Symmorium bears close similarity in size and appearance to Stethacanthus but the former is missing the "spine and brush" on its back. Some paleontologists think that the two forms are simply the males and females of related species, while other scientists think they were distinct genera.

In linguistics, a yes–no question, also known as a binary question, a polar question, or a general question, is a question whose expected answer is one of two choices, one that provides an affirmative answer to the question versus one that provides a negative answer to the question. Typically, in English, the choices are either "yes" or "no". Yes–no questions present an exclusive disjunction, namely a pair of alternatives of which only one is a felicitous answer. In English, such questions can be formed in both positive and negative forms

Convergent thinking is a term coined by Joy Paul Guilford as the opposite of divergent thinking. It generally means the ability to give the "correct" answer to questions that do not require novel ideas, for instance on standardized multiple-choice tests for intelligence.

<span class="mw-page-title-main">Ornithoscelida</span> Clade of reptiles

Ornithoscelida is a proposed clade that includes various major groupings of dinosaurs. An order Ornithoscelida was originally proposed by Thomas Henry Huxley but later abandoned in favor of Harry Govier Seeley's division of Dinosauria into Saurischia and Ornithischia. The term was revived in 2017 after a new cladistic analysis by Baron et al.

Social information seeking is a field of research that involves studying situations, motivations, and methods for people seeking and sharing information in participatory online social sites, such as Yahoo! Answers, Answerbag, WikiAnswers and Twitter as well as building systems for supporting such activities. Highly related topics involve traditional and virtual reference services, information retrieval, information extraction, and knowledge representation.

Drazen Prelec is a professor of management science and economics in the MIT Sloan School of Management, and holds appointments in the Department of Economics and in the Department of Brain and Cognitive Sciences at MIT as well. He is a pioneer in the field of neuroeconomics.

<span class="mw-page-title-main">Global justice movement</span> Network of organized efforts around international justice

The global justice movement is a network of globalized social movements demanding global justice by opposing what is often known as the “corporate globalization” and promoting equal distribution of economic resources.

<span class="mw-page-title-main">Brainly</span> Educational technology company

Brainly is an education company based in Kraków, Poland, with headquarters in New York City. It is an AI-powered homework help platform targeting students and parents. As of November 2020, Brainly reported having 15 million daily active users, making it the world's most popular education app. In 2024, FlexOS reported Brainly as the #1 Generative AI Tool in the education category and the #6 Generative AI Tool overall. Also in 2024, Andreessen Horowitz reported Brainly as #6 in the Top 50 Gen AI Mobile Apps by monthly active users.

<span class="mw-page-title-main">Proteoarchaeota</span> Proposed kingdom of archaea

"Proteoarchaeota" are a proposed archaeal kingdom thought to be closely related and possibly ancestral to the Eukaryotes.

References

  1. Akst, Daniel (February 16, 2017). "The Wisdom of Even Wiser Crowds". The Wall Street Journal. Retrieved 16 May 2018.
  2. 1 2 Dizikes, Peter (January 25, 2017). "Better wisdom from crowds". MIT News. Retrieved 16 May 2018.
  3. 1 2 Prelec, Dražen; Seung, H. Sebastian; McCoy, John (2017). "A solution to the single-question crowd wisdom problem". Nature. 541 (7638): 532–535. Bibcode:2017Natur.541..532P. doi:10.1038/nature21054. ISSN   1476-4687. PMID   28128245. S2CID   4452604.
  4. Hosseini, Hadi; Mandal, Debmalya; Shah, Nisarg; Shi, Kevin (2021). "Surprisingly Popular Voting Recovers Rankings, Surprisingly!". Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence: 245–251. doi:10.24963/ijcai.2021/35.

Further reading

Prelec, Dražen; Seung, H. Sebastian; McCoy, John (25 January 2017). "A solution to the single-question crowd wisdom problem". Nature. 541 (7638): 532–535. Bibcode:2017Natur.541..532P. doi:10.1038/nature21054. PMID   28128245. S2CID   4452604.