Wiki survey

A view of the 'opinion space' visualization tool for a Polis wiki survey. Each user can see how similar their voting behavior is to other voters based on their closeness to one another within the two-dimensional space.

Wiki surveys or wikisurveys are a software-based survey method with similarity to how wikis evolve through crowdsourcing. In essence, they are surveys that allow participants to create the questions that are being asked. ^{[1] [2] [3]} Other names include bridging systems and collective response systems. ^[4] As participants engage in the survey they can either vote on a survey question or create a survey question. A single open-ended prompt written by the creator of the survey determines the topic the questions should be on. The first known implementation of a wiki survey was in 2010, ^[5] and they have been used since then for a variety of purposes such as facilitating deliberative democracy, crowdsourcing opinions from experts and figuring out common beliefs on a given topic. ^{[6] [7] [8]} A notable usage of wiki surveys is in Taiwan's government system, where citizens can participate in crowdsourced lawmaking through Polis wiki surveys. ^{[9] [10] [11]}

Wiki surveys facilitate collective intelligence by allowing users to both contribute and respond to the survey, as well as see the results of the survey in real time. They can be seen in a more general sense as a tool for establishing consensus in large volumes of people. Wiki surveys mainly differ from consensus-building in comment sections by using a heuristic which determines the order of questions for each participant that aims to maximize consensus, not allowing replies to questions and providing visualization tools to better understand consensus.

Implementations

All Our Ideas

A view of the user interface for seeing in real-time the results of an All Our Ideas wiki survey. The top 10 ranked items are shown in this view of the interface.

A view of the user interface for casting votes on and submitting items to an All Our Ideas wiki survey. After casting a vote, a heuristic determines the next item that will be shown. The heuristic favors showing items with relatively low numbers of votes.

All Our Ideas may have been the first ever wiki survey. ^{[1] [12]} Its focus is on ranking the favorability of each 'item' that users submit to the survey. Each question presented asks the participant to rank the best of two items. At any point in time, participants can view a ranking of the items in order of their score. The score for an item is the estimated probability that it would be favored over another randomly chosen item. It is considered a 'pairwise wiki survey'. The code for All Our Ideas is open source. ^{[13] [ non-primary source needed ]}

Decidim

Decidim has been used by governments throughout Spain and Europe to help with participatory budgeting and other public policy decisions. ^{[14] [15]}

Polis

Polis (also known as Pol.is) was developed in 2012. ^[2] The focus of Polis is to project participants into an 'opinion space' where they can see how their voting behavior compares to other participants. The opinion space clusters participants into groups of similar opinion and is designed in a way to avoid tyranny of the majority by being able to include groups that have small numbers of participants. The questions participants are presented with are a simple agree/disagree/pass on a single 'comment' submitted by a participant. The code for Polis is open source. ^[16]

Remesh

Remesh was founded in 2013 ^[17] and has partnered with the United Nations ^[18] and Alliance for Middle East Peace ^[14] efforts to bring peaceful resolutions to conflicts. Participants are anonymous and the algorithm can be fine-tuned to better understand local dialects in specific regions. ^[18]

Examples

• PlaNYC used All Our Ideas to gather ideas on how to establish New York City's sustainability plan ^[1]
• vTaiwan, a citizen-lead government process in Taiwan, uses Polis for enabling large amounts of citizens to deliberate and consequently provide input on Taiwan's legislative decisions ^[2]
• OECD used All Our Ideas to gather ideas from the public prior to meeting for a forum and meeting on which skills are most important to invest in for the 21st century ^[1]
• March On, an offshoot of the Women's March Movement, used Polis to understand the opinions of people wanting to support the movement ^[19]
• AARP used All Our Ideas to better understand the concerns its members have about acquiring and using big data regarding people's health and wellness ^{[20] [21]}
• Bowling Green used Polis to host a virtual town hall for its residents to better inform its community leaders on how to improve the quality of life of its residents ^[22]
• Residents of Harrogate use Polis to debate issues in their community, with the results being released publicly to everyone ^[23]

Characteristics

Wiki surveys have the following three defining characteristics: ^{[1] [6] [2]}

Collaborativeness

Wiki surveys allow participants to contribute questions, as well as answer questions created by its participants.

Adaptivity

Wiki surveys adapt to elicit the most useful information from its participants. This is done by changing the ordering of questions based on the voting behavior of previous participants so as to maximize consensus. The heuristic determining the ordering of questions highly values showing the comments that have been voted on the least.

Greediness

Although being greedy typically has a negative connotation, it is used in a positive manner for wiki surveys. They are 'greedy' because they make full use of information that participants are willing to provide. Wiki surveys do not require participants to answer a fixed amount of questions, participants can answer as little or as much as they want.

Traditional survey methods vs. wiki surveys

Advantages of wiki surveys

Questions in traditional survey methods fall into two categories: Open and closed questions. Open questions ask the person taking the survey to write an open response while closed questions give a fixed set of responses to select from. ^[24] Wiki surveys are like a hybrid of the two, enabling insightful consensus in certain situations where traditional survey methods may lack. Closed questions are easy to analyze quantitively, but the limited options to select from for a given question may cause bias. Open questions are not as subject to bias, but are difficult to analyze quantitatively at scale. Wiki surveys allow for open responses by the users' contribution of survey questions (also called 'items'), and uses machine learning techniques to automatically quantitative analyze the responses to those questions.

The 'greediness' characteristic of wiki surveys is thought to also be advantageous, as it allows for gathering more data per participant. ^[1] Data from real-world usage of wiki surveys shows that there are a relatively small amount of participants that answer a relatively large amount of questions compared to most participants. In traditional surveys, participants who want to provide more information than required are typically not allowed to do so, thus missing out on potentially useful information.

See also

• Online deliberation

References

1. Salganik, Matthew J.; Levy, Karen E. C. (2015-05-20). Helleringer, Stephane (ed.). "Wiki Surveys: Open and Quantifiable Social Data Collection". PLOS ONE. 10 (5): e0123483. Bibcode: 2015PLoSO..1023483S . doi: 10.1371/journal.pone.0123483 . ISSN   1932-6203. PMC   4439069 . PMID   25992565.
2. Small, Christopher; Bjorkegren, Michael; Erkkilä, Timo; Shaw, Lynette; Megill, Colin (2021-07-22). "Polis: Scaling deliberation by mapping high dimensional opinion spaces". Democracy in the Age of Artificial Intelligence. 26 (2). doi: 10.6035/recerca.5516 . hdl: 10234/196348 . ISSN   2254-4135. S2CID   242371333 .
3. Bass, Theo (28 January 2019). "Crowdsourcing for democracy using Wikisurveys". nesta. Retrieved 2022-02-28.
4. "Plurality: The Future of Collaborative Technology and Democracy | Augmented Deliberation". www.plurality.net. Retrieved 2025-05-13.
5. "About this project". All Our Ideas. Retrieved 2022-02-28.
6. French, Megan; Hancock, Jeff (2017-02-02). "What's the Folk Theory? Reasoning About Cyber-Social Systems". SSRN. Rochester, NY. SSRN   2910571.
7. Wong, Arnold YL; Lauridsen, Henrik H.; Samartzis, Dino; Macedo, Luciana; Ferreira, Paulo H.; Ferreira, Manuela L. (2019-01-15). "Global Consensus From Clinicians Regarding Low Back Pain Outcome Indicators for Older Adults: Pairwise Wiki Survey Using Crowdsourcing". JMIR Rehabilitation and Assistive Technologies. 6 (1): e11127. doi: 10.2196/11127 . PMC   6350088 . PMID   30664493. Archived from the original on Jun 16, 2022.
8. "Featured Case Studies". The Computational Democracy Project. Retrieved 2022-02-28.
9. Horton, Chris (August 21, 2018). "The simple but ingenious system Taiwan uses to crowdsource its laws". MIT Technology Review. Retrieved 2022-02-28.
10. BBC Click (Oct 29, 2019), "Can Taiwan Reboot Democracy?", YouTube, retrieved 2022-02-28
11. Harris, Tristan; Raskin, Aza; Tang, Audrey (July 23, 2020). "22 – Digital Democracy is Within Reach with Audrey Tang". Your Undivided Attention. Center for Humane Technology. Retrieved 2022-03-18.
12. Garber, Megan (August 10, 2010). "All Our Ideas facilitates crowdsourcing — of opinions". Nieman Lab.
13. "All Our Ideas". GitHub. Retrieved 2022-03-20.
14. Schirch, Lisa (2025-07-07). "Social media can support or undermine democracy – it comes down to how it's designed". The Conversation. Retrieved 2025-08-21.
15. Graham, Thomas (May 18, 2018). "Barcelona is leading the fightback against smart city surveillance". Wired. ISSN   1059-1028 . Retrieved 2025-08-21.
16. Polis, The Computational Democracy Project, 2022-03-15, retrieved 2022-03-20
17. Marcia Pledger, The Plain Dealer (2018-04-12). "Plug and Play makes first investment in Cleveland-founded company Remesh". cleveland. Retrieved 2025-08-21.
18. "The United Nations is turning to artificial intelligence in search for peace in war zones". The Washington Post. 2021-04-23. ISSN   0190-8286 . Retrieved 2025-08-21.
19. Ruiz, Rebecca (2018-01-20). "Nasty women (and men) to crowdsource 'marching orders' ahead of midterm elections". Mashable. Retrieved 2022-03-20.
20. "AARP and All Our Ideas". congress.crowd.law. Retrieved 2022-03-20.
21. "Testimony of Beth Simone Noveck, Professor and Director, The Governance Lab, New York University U.S. House Select Committee on the Modernization of Congress Recommendations for the Committee's agenda for the 117th Congress" (PDF). US House of Representatives. March 25, 2021.
22. "Civic Assembly". Bowling Green Daily News. Retrieved 2022-03-21.
23. "Anonymous voting tool aims to find Harrogate's consensus on key issues". York Press. March 2022. Retrieved 2022-03-25.
24. Schuman, Howard; Scott, Jacqueline (1987-05-22). "Problems in the Use of Survey Questions to Measure Public Opinion" . Science. 236 (4804): 957–959. Bibcode:1987Sci...236..957S. doi:10.1126/science.236.4804.957. PMID   17812751. S2CID   42371722.