Crowdsource (app)

Last updated

Crowdsource
Developer(s) Google
Initial releaseAugust 29, 2016;7 years ago (2016-08-29)
Stable release(s)
2.27.604817332 [1] / February 7, 2024;2 months ago (2024-02-07)
Operating system Android, web
Platform Android 5.0+
Size 43.1 MB
Type Crowdsourcing platform
Website crowdsource.google.com   OOjs UI icon edit-ltr-progressive.svg

Crowdsource is a crowdsourcing platform developed by Google intended to improve a host of Google services through the user-facing training of different algorithms. [2]

Contents

Crowdsource was released for the Android operating system on the Google Play store on August 29, 2016, and is also available on the web. Crowdsource includes a variety of short tasks users can complete to improve many of Google's different services. Such tasks include image label verification, sentiment evaluation, and translation validation. By completing these tasks, users provide Google with data to improve services such as Google Maps, Google Translate, and Android. [3] As users complete tasks, they earn achievements including stats, badges, and certificates, which track their progress.

Crowdsource was released quietly on the Google Play store, with no marketing from Google. [4] It received mixed reviews on release, with many reviews stating that its lack of monetary rewards is unusual, as similar platforms, such as Google Opinion Rewards, often reward users with Play credits. [5] [4] [6]

Features

Crowdsource includes different types of tasks, and these each provide Google with different information that it can give as training data to its machine learning algorithms. In the app's description on Google Play, Google refers to these tasks as "microtasks" which should take "no more than 5-10 seconds" to complete. [7]

Tasks

Upon launch, the Crowdsource Android application presented users with 5 different tasks: image transcription, handwriting recognition, translation, translation validation, and map translation validation. [6] The most recent version of the app includes 11 tasks: Image Label Verification, Sentiment Evaluation, Audio Validation, Smart Camera, Glide Type, Handwriting Recognition, Reading Charts, Trust in Charts, Translation, Translation Validation, and Image Capture.

Translation and translation validation

Translation related tasks (translation and translation validation) are only shown to users who have selected more than one language they are fluent in. [7] [5] [4] [6] While Maps Translation validation is no longer a task in the Crowdsource Android and web apps, users can still complete translation and translation validation tasks. [7] Translation presents the user with one of the languages they listed themselves as fluent in, and asks them to translate it into another language they are fluent in. [7] Translation validation presents users with a list of translations submitted by other users, and asks them to categorize them as correct or incorrect. [7] Both of these tasks help improve Google's translating capabilities, most notably in Google Translate, and any other Google app with translated content, including Google Maps. [3] [8]

Image Label Verification

Image Label Verification allows users to select a word from a list of topics, such as "Cars, and then presents the user with a picture, asking the question "Does this image contain cars?". Users can select "Yes", "No", or "Skip" if they are unsure. [9] [4] The data gained from this task is used to help read text within images for services like Google Street View. [4]

A similar task is available in the Google Photos mobile and web apps. However, unlike Crowdsource, the photos that are presented there are the user's own photos. [10]

Handwriting recognition

Handwriting recognition relies on users to read handwritten words and transcribe them to text. According to Google, completing this task helps improve Gboard's handwriting feature. [3]

Sentiment evaluation

Sentiment evaluation presents the user with various reviews and comments, and asks them to describe the statement as "positive", "neutral", or "negative". [11] Alternatively, users can skip a question if they are unsure. [3] These evaluations by the users of Crowdsource help with various recommendation-based technologies that Google uses on platforms like Google Maps, the Google Play Store, and YouTube. [3] [11]

Landmark identification

This task asks users to confirm if a specific landmark is visible in pictures shown in the app. This task is designed to help ensure businesses and landmarks are recognizable in applications such as Google Maps. This task is now complete and no longer available. [12]

Audio Validation

Audio Validation helps improve Google's Text-to-Speech technology. [9] The user is presented with a short audio clip in which a computer tries to read a word out loud. The user can then specify whether this is how said word would be correctly pronounced.

Image Caption

This task helps improve an algorithm to create captions for online images. According to the Google Crowdsource web app, "Verifying machine generated captions will help make images more accessible to people with visual and cognitive impairment". [13]

Image Capture and Smart Camera

Contributed via Crowdsource, this Likoni Ferry picture now illustrates several Wikipedia articles. MV Likoni in Mombasa 01 (cropped).jpg
Contributed via Crowdsource, this Likoni Ferry picture now illustrates several Wikipedia articles.

When working on the Image Capture task, the user is presented with an interface to upload an image and add tags to it.

When using Smart Camera, the user can point the devices camera to an object, similar to how Google Lens is used. A dialogue opens when an item is detected. The app then says what it thinks it sees. The user can then confirm or deny this description. If the user confirms, the photo will be uploaded and you return to the beginning. If the user denies, an option to select why you denied and a place to add tags to the image pops up.

Glide Type

This task was added on May 17, 2020, to help improve the algorithm for Gboard's glide typing feature.

Reading and Trust in Charts

These two tasks were also added on May 17, 2020, as a collaboration with King's College and University of Vienna to measure the community's understanding and trustworthiness of different graph types. The task was completed the next day, May 18th, and closed.

Achievements

Beyond the tasks that users can complete, the Crowdsource app has an "Achievements" section that shows users stats and badges which they earn through completing different tasks in Crowdsource. [7] [3] [14] [15]

Stats

When users contribute to Crowdsource by completing tasks, Crowdsource tracks their total number of contributions, as well as metrics like "upvotes" which show how many of a user's answers are "in agreement with answers from the Crowdsource community," [14] and "accuracy" which shows what percentage of the user's answers have been accepted as correct. [14]

Badges

As users complete tasks, they also receive badges. There are badges for each type of task, which track progress along that particular task (such as translation validation), as well as badges for other milestones, like completing a task while offline or completing a task given through a push notification. [15] [7]

Updates

An April 2018 update to Crowdsource included a new "Image Capture" task in which users can take photos, tag them, and upload them to Crowdsource. [4] Users can choose to open-source their images, as well, to share them with researchers and developers not exclusive to Google. [16] In an interview with Wired Magazine, Anurag Batra, a product manager at Google who leads the Crowdsource team, said that the data gained from users completing this image capture task could improve Google Images, Pixel Camera, and Google Lens. [17] The latest update adds a 'back' button to change your previous response.

On January 21, 2021, an update introduced dark theme to the mobile app. [9] This update also changed the overall user interface significantly. A new task, Audio Validation, was also added. [9]

Platforms

Crowdsource is also available as a web application. It offers many of the same tasks, such as Image Label Verification, translation, and translation validation, and includes a page for users to view their achievements, much like the Android application. Unlike the Android version, the Crowdsource website includes a task for validation image captions, and evaluating facial expressions. However, it does not have the sentiment evaluation, image capture, smart camera, and audio validation tasks available. [13] As of March 18,2021, The Crowdsource website also does not have dark theme.

Reception

On release, many reviewers found the app's lack of monetary rewards unusual, due to the fact that Google has a similar app, Google Opinion Rewards, which offers Google Play store credits after completing short surveys. [6] [4] [5] An August 2016 review from Android Pit noted that "this reliance on altruism [is] a little strange given that Google already has an app, Google Opinion Rewards, which has a financial incentive for user feedback. It works slightly differently, but I don't see why the same reward scheme could not be applied." The review also expressed concern with this model, citing that users would be less likely to complete these tasks without more substantial rewards, writing that Crowdsource is "banking on the kind nature of its users", and asking "How are users, who generally want everything free and without adverts, going to respond to this in the long-term? Why would they rate this app highly, when the results are so nebulous?" [6] A review from Wired shared similar concerns, writing "while Google is being open about its motivations, it will be difficult for users to know what difference their contributions make." [17]

On Crowdsource's FAQ page, Google addresses this question of "Will I get paid for my answers?", answering, "No. Crowdsource is a community effort – we rely on the goodwill of community members to help improve the quality of services such as Google Maps, Google Translate, and others, so that everybody in the world can benefit". [18] In an August 2016 review, CNET noted that Google's statement in Crowdsource's description, "Every time you use it, you know that you've made the internet a better place for your community.", is not accurate, stating that Google does not offer free access to Google Maps and Google Translate data. [19] A review from TechCrunch also noted that Crowdsource is "solely focused on helping Google improve its own services," contrasting it with Amazon Mechanical Turk, which focuses on tasks from third parties. [4]

Applications

An April 2018 interview in Wired stated that Google's machine learning algorithms work best in the United States and Western Europe, but are less effective in less prosperous countries. In this interview Anurag Batra, a product manager at Google who leads the Crowdsource team, shared Google's motivations behind the Crowdsource app, stating that Google has "very sparse training data set from parts of the world that are not the United States and Western Europe," [17] According to Wired, Google has a team that promotes the Crowdsource app in India and throughout Asia at colleges, and will likely expand to Latin America later in 2018. [17] In a blog post on Local Guides Connect, Batra explains why Crowdsource is helpful to Google, detailing that the questions that Crowdsource asks users are designed to collect better samples of data to feed their machine learning algorithms. [3]

Google uses the answers provided by users of Crowdsource, and validates them by showing them anonymously to other Crowdsource users. [20] According to Google, once answers are validated, they are used to "train computer algorithms that run services such as Translate, Maps, Gboard, and others." [20]

A brief on Cio Dive stated that an "accurate data set is critical" to the success of new technologies such as voice assistants and autonomous vehicles. [21] The brief also notes that companies like Google and IBM are well positioned in the fields of artificial intelligence and machine learning due to the volume of data available to them to train and develop advanced artificial intelligence. [21]

See also

Related Research Articles

<span class="mw-page-title-main">Google Search</span> Search engine from Google

Google Search is a search engine operated by Google. It allows users to search for information on the Internet by entering keywords or phrases. Google Search uses algorithms to analyze and rank websites based on their relevance to the search query. It is the most popular search engine worldwide.

<span class="mw-page-title-main">Gmail</span> Email service provided by Google

Gmail is an email service provided by Google. As of 2019, it had 1.5 billion active users worldwide, making it the largest email service in the world. It also provides a webmail interface, accessible through a web browser, and is also accessible through the official mobile application. Google also supports the use of third-party email clients via the POP and IMAP protocols.

<span class="mw-page-title-main">Google Maps</span> Googles web mapping service (launched 2005)

Google Maps is a web mapping platform and consumer application offered by Google. It offers satellite imagery, aerial photography, street maps, 360° interactive panoramic views of streets, real-time traffic conditions, and route planning for traveling by foot, car, bike, air and public transportation. As of 2020, Google Maps was being used by over one billion people every month around the world.

<span class="mw-page-title-main">Google Translate</span> Multilingual neural machine translation service

Google Translate is a multilingual neural machine translation service developed by Google to translate text, documents and websites from one language into another. It offers a website interface, a mobile app for Android and iOS, as well as an API that helps developers build browser extensions and software applications. As of 2022, Google Translate supports 133 languages at various levels; it claimed over 500 million total users as of April 2016, with more than 100 billion words translated daily, after the company stated in May 2013 that it served over 200 million people daily.

<span class="mw-page-title-main">Crowdsourcing</span> Sourcing services or funds from a group

Crowdsourcing involves a large group of dispersed participants contributing or producing goods or services—including ideas, votes, micro-tasks, and finances—for payment or as volunteers. Contemporary crowdsourcing often involves digital platforms to attract and divide work between participants to achieve a cumulative result. Crowdsourcing is not limited to online activity, however, and there are various historical examples of crowdsourcing. The word crowdsourcing is a portmanteau of "crowd" and "outsourcing". In contrast to outsourcing, crowdsourcing usually involves less specific and more public groups of participants.

A human-based computation game or game with a purpose (GWAP) is a human-based computation technique of outsourcing steps within a computational process to humans in an entertaining way (gamification).

Google Drive is a file storage and synchronization service developed by Google. Launched on April 24, 2012, Google Drive allows users to store files in the cloud, synchronize files across devices, and share files. In addition to a web interface, Google Drive offers apps with offline capabilities for Windows and macOS computers, and Android and iOS smartphones and tablets. Google Drive encompasses Google Docs, Google Sheets, and Google Slides, which are a part of the Google Docs Editors office suite that permits collaborative editing of documents, spreadsheets, presentations, drawings, forms, and more. Files created and edited through the Google Docs suite are saved in Google Drive.

<span class="mw-page-title-main">Google Play</span> Digital distribution service by Google

Google Play, also known as the Google Play Store or Play Store and formerly Android Market, is a digital distribution service operated and developed by Google. It serves as the official app store for certified devices running on the Android operating system and its derivatives, as well as ChromeOS, allowing users to browse and download applications developed with the Android software development kit (SDK) and published through Google. Google Play has also served as a digital media store, offering games, music, books, movies, and television programs. Content that has been purchased on Google Play Movies & TV and Google Play Books can be accessed on a web browser and through the Android and iOS apps.

Habitica, formerly HabitRPG, is an online task management application developed by HabitRPG, Inc. It was founded on January 30, 2013. The application uses the format of a role-playing game to organize tasks. Habitica is an open source project. It has attracted the interest of scientific research into the efficacy of habit-forming.

<span class="mw-page-title-main">Google Sheets</span> Cloud-based spreadsheet software

Google Sheets is a spreadsheet application included as part of the free, web-based Google Docs Editors suite offered by Google. Google Sheets is available as a web application; a mobile app for: Android, iOS, and as a desktop application on Google's ChromeOS. The app is compatible with Microsoft Excel file formats. The app allows users to create and edit files online while collaborating with other users in real-time. Edits are tracked by which user made them, along with a revision history. Where an editor is making changes is highlighted with an editor-specific color and cursor. A permissions system regulates what users can do. Updates have introduced features that use machine learning, including "Explore", which offers answers based on natural language questions in the spreadsheet. Sheets is one of the services provided by Google that also includes Google Docs, Google Slides, Google Drawings, Google Forms, Google Sites and Google Keep.

<span class="mw-page-title-main">Mapillary</span> Swedish service for sharing crowdsourced geotagged photos

Mapillary is a service for sharing crowdsourced geotagged photos, developed by remote company Mapillary AB, based in Malmö, Sweden. Mapillary was launched in 2013 and acquired by Meta Platforms in 2020. It offers street level imagery similar to Google Street View.

Influenster is a product discovery and reviews platform for consumers. Influenster uses social media analytics to measure its users’ influence on social media. Influenster launched in 2010. An iOS app for Influenster launched in 2014 while an Android app launched in 2015. As of January 2018, Influenster's community consisted of more than 4 million users. Influenster was acquired by Bazaarvoice in 2019, and the app was relaunched in October 2022.

<span class="mw-page-title-main">Gboard</span> Virtual keyboard app for Android and iOS

Gboard is a virtual keyboard app developed by Google for Android and iOS devices. It was first released on iOS in May 2016, followed by a release on Android in December 2016, debuting as a major update to the already-established Google Keyboard app on Android.

<span class="mw-page-title-main">Google Assistant</span> AI-powered digital assistant from Google

The Google Assistant is a virtual assistant software application developed by Google that is primarily available on mobile and home automation devices. Based on artificial intelligence, The Google Assistant can engage in two-way conversations, unlike the company's previous virtual assistant, Google Now.

<span class="mw-page-title-main">Google Allo</span> Instant messaging app by Google

Google Allo was an instant messaging mobile app by Google for the Android and iOS mobile operating systems, with a web client available in some web browsers. It closed on March 12, 2019.

<span class="mw-page-title-main">Google Opinion Rewards</span> Mobile survey app

Google Opinion Rewards is a rewards-based program developed by Google. It was initially launched as a survey mobile app for Android and iOS developed by Google. The app allows users to answer surveys and earn rewards. On Android, users earn Google Play credits which can be redeemed by buying paid apps from Google Play. On iOS, users are paid via PayPal. Users in the available countries who are over 18 years old are eligible. Google Opinion Rewards works with Google Surveys, market researchers make the survey through Google Surveys and answers are received through Google Opinion Rewards by app users. This process provides surveyors with a large pool of surveyees quickly. This "fast and easy" surveying process has been criticized due to contention over the validity of results as well as concern over the privacy and security of the app users' data.

Toloka is a crowdsourcing platform and microtasking project launched by Yandex in 2014 to quickly markup large amounts of data, which are then used for machine learning and improving search algorithms. The proposed tasks are usually simple and do not require any special training from the performer. Most of the tasks are designed to improve algorithms that are used by modern technologies spanning self-driving vehicles, smart web searches, advanced voice assistants and e-commerce. Upon completion of each task the performer receives a reward based on the volume of images, videos, and unstructured text. The service has two app versions – for Android and iOS.

References

  1. ↑ "Crowdsource APKs". APKMirror. Retrieved April 10, 2024.
  2. ↑ Sarin, Supheakmungkol; Pipatsrisawat, Knot; Pham, Khiem; Batra, Anurag; Valente, Luis (2019). "Crowdsource by Google: A Platform for Collecting Inclusive and Representative Machine Learning Data" (PDF). AAAI Hcomp 2019.
  3. 1 2 3 4 5 6 7 AnuragB (February 6, 2017). "Try the new Crowdsource app for Android" . Retrieved October 24, 2018.
  4. 1 2 3 4 5 6 7 8 Perez, Sarah (August 29, 2016). "Google's new app Crowdsource asks users to help with translation, transcription & more". TechCrunch. Retrieved October 23, 2018.
  5. 1 2 3 Whitwam, Ryan (August 29, 2016). "New Google Crowdsource app asks you to help with translation and text transcription a few seconds at a time". Android Police. Retrieved October 23, 2018.
  6. 1 2 3 4 5 Gordon, Scott (September 6, 2016). "Google Crowdsource: what you need to know". AndroidPIT. Retrieved October 23, 2018.
  7. 1 2 3 4 5 6 7 "Crowdsource 1.2.0.215240185 APK Download by Google LLC". APKMirror. October 3, 2018. Retrieved October 23, 2018.
  8. ↑ "Translate words & phrases - Crowdsource Help". support.google.com. Retrieved October 26, 2018.
  9. 1 2 3 4 "Crowdsource - Apps on Google Play". play.google.com. Retrieved March 22, 2024.
  10. ↑ Li, Abner (November 9, 2020). "Google Photos 5.18 lets you improve app's machine learning". 9to5Google. Retrieved March 18, 2021.
  11. 1 2 "Evaluate sentiment - Crowdsource Help". support.google.com. Retrieved October 26, 2018.
  12. ↑ "Identify landmarks - Crowdsource Help". support.google.com. Retrieved November 8, 2018.
  13. 1 2 "Crowdsource by Google". crowdsource.google.com. Retrieved October 26, 2018.
  14. 1 2 3 "Achievements (Stats) - Crowdsource Help". support.google.com. Retrieved October 23, 2018.
  15. 1 2 "Achievements (Badges) - Crowdsource Help". support.google.com. Retrieved October 23, 2018.
  16. ↑ "Choose & tag images - Crowdsource Help". support.google.com. Retrieved October 23, 2018.
  17. 1 2 3 4 Simonite, Tom (April 5, 2018). "Google Turns to Users to Improve Its AI Chops Outside the US". WIRED. Retrieved October 23, 2018.
  18. ↑ "FAQ - Crowdsource Help". support.google.com. Retrieved October 23, 2018.
  19. ↑ Shankland, Stephen (August 29, 2016). "New Crowdsource app lets your work for Google for free". CNET. Retrieved October 23, 2018.
  20. 1 2 "How does Google use my answers? - Crowdsource Help". support.google.com. Retrieved October 23, 2018.
  21. 1 2 Hickey, Alex (April 6, 2018). "Google counters western bias with global image 'crowdsourcing'". CIO Dive. Retrieved October 24, 2018.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.