Panagiotis G. Ipeirotis | |
|---|---|
| Παναγιώτης Ηπειρώτης | |
| Born | May 3, 1976 |
| Citizenship | Greek, American |
| Alma mater | University of Patras (BSc) Columbia University (MSc, PhD) |
| Known for | Crowdsourcing; human computation; Data quality; Text mining |
| Awards | Lagrange Prize (2015) [2] ACM SIGKDD Test of Time Award (2020) [3] |
| Scientific career | |
| Fields | Computer Science |
| Institutions | New York University Stern School of Business |
| Thesis | Classifying and Searching Hidden-Web Text Databases (2004) |
| Doctoral advisor | Luis Gravano |
Panagiotis G. Ipeirotis (aka Panos Ipeirotis) (born May 3, 1976, in Serres, Greece) is a Greek-American computer scientist and the Merchants' Council Professor of Technology and Business at the New York University Stern School of Business. [4] [5] His research focuses on data mining, crowdsourcing, human computation, and the economics of online information systems. [6]
Ipeirotis is a recipient of the Lagrange Prize in Complex Systems (2015) [2] and the ACM SIGKDD Test of Time Award (2020). [3] He is known for research on Amazon Mechanical Turk and crowdsourcing quality management, work that has been covered in publications including The New York Times , [7] [8] The Economist , [9] Financial Times , [10] The Washington Post , [11] MIT Technology Review , [12] TIME , [13] and Bloomberg Businessweek . [14] [15] He has also been profiled in Greek media, including a 2014 interview with Kathimerini discussing Greek higher education and his career. [16]
In addition to his academic career, Ipeirotis co-founded the AI consulting firm Detectica in 2015, which was acquired by Compass, Inc. in 2019. [17] He has also held research positions at Google (2013–2014), where he conducted research on crowdsourcing methods to improve the Google Knowledge Graph, [18] [19] and at Meta's Reality Labs (2024–2025), where he has worked on machine learning deployment for wearable devices. [4] [20]
He is also the author of the blog "A Computer Scientist in a Business School," [21] which covers topics in crowdsourcing, data science, and academia; posts from the blog have been cited in academic papers and media coverage. [22]
Ipeirotis was born on May 3, 1976, in Serres, Greece [1] . In 1994, he won a Gold Medal at the 8th National Greek Competition in Chemistry. [23] Later that year, he represented Greece at the 26th International Chemistry Olympiad in Oslo, Norway. [24]
Ipeirotis earned his Diploma in Computer Engineering and Informatics (CEID) from the University of Patras in 1999. [4] He pursued graduate studies at Columbia University under the supervision of Luis Gravano, receiving his M.Sc. in 2001, M.Phil. in 2003, and Ph.D. in Computer Science in 2004. [4]
Ipeirotis began his academic career as a graduate research assistant at Columbia University (1999–2004). In 2004, he joined the Department of Information, Operations, and Management Sciences at the New York University Stern School of Business as an Assistant Professor. He was promoted to Associate Professor in 2010 and Full Professor in 2016. He holds a courtesy appointment at the Courant Institute of Mathematical Sciences [25] and is an associated faculty member at the NYU Center for Data Science. [26]
Ipeirotis has held leadership roles in major academic conferences and journals. He served as General Co-Chair of the AAAI Conference on Human Computation and Crowdsourcing (HCOMP) in 2015 [27] and as Technical Program Co-Chair for The Web Conference 2018 (WWW 2018). [28] He has served on the editorial boards of Management Science and IEEE Transactions on Knowledge and Data Engineering, and was a founding co-Editor-in-Chief of the journal Collective Intelligence launched in 2022 as a collaboration between SAGE Publications, the Association for Computing Machinery, and Nesta. [29] [30]
His research has received coverage in Bloomberg Businessweek , which in 2011 featured his work on combining human and machine intelligence in crowdsourcing systems [31] and in 2013 profiled him as the "data dude" of business analytics. [15]
Ipeirotis completed his PhD at Columbia University under the supervision of Luis Gravano, himself a student of Hector Garcia-Molina at Stanford. According to the Mathematics Genealogy Project, Ipeirotis has advised seven doctoral students at NYU and Columbia. [32]
His former students include Beibei Li, now a professor at Carnegie Mellon University's Heinz College and recipient of the 2019 INFORMS ISS Sandra A. Slaughter Early Career Award, [33] and Marios Kokkodis, recipient of the 2020 INFORMS ISS Gordon B. Davis Young Scholar Award. [34] Former postdoctoral researcher Djellel Difallah is now Assistant Professor and Program Head at New York University Abu Dhabi and previously served as a Research Scientist at the Wikimedia Foundation. [35]
Ipeirotis's work explores the intersection of computer science and economics, an approach he and collaborators have termed "EconoMining." His research has been published in venues such as Management Science , Information Systems Research , and IEEE Transactions on Knowledge and Data Engineering. [4]
Ipeirotis is known for his studies on Amazon Mechanical Turk (MTurk). His 2010 paper "Running Experiments on Amazon Mechanical Turk," [36] co-authored with Gabriele Paolacci and Jesse Chandler, provided methodological guidance for using crowdsourcing platforms in behavioral research. The paper has been characterized as a "landmark" study that significantly expanded the use of crowdsourced experiments in social science research. [37]
His research on MTurk data quality gained significant media attention. Studies revealing that approximately 40% of MTurk responses contained spam or low-quality content were covered by TIME , [13] Business Insider, MIT Technology Review, and The Washington Post. His 2010 demographic analysis of the MTurk workforce has been frequently cited by researchers studying crowdsourcing platforms, providing foundational data on worker motivations, geographic distribution, and income levels. [38] [37] His expertise on crowdsourcing data quality continued to be sought by media outlets; in 2018, Wired consulted Ipeirotis for an article about suspected bot infiltration of MTurk, where he noted that "thousands of published social science studies use MTurk each year" and advised that proper survey design could maintain data reliability. [39] In 2019, The New York Times quoted him in an in-depth investigation of working conditions on Mechanical Turk. [8]
This work led to the development of quality management techniques for crowdsourcing, including the "Get Another Label" framework for improving data quality using multiple noisy labelers, which received the SIGKDD Test of Time Award in 2020. [3] His research on crowdsourcing best practices also influenced journalism; ProPublica published a "Guide to Mechanical Turk" for investigative reporters that drew directly on Ipeirotis's research and blog insights, crediting him as an authority on ensuring data quality through multiple worker verification. [40]
Ipeirotis has conducted extensive research on duplicate record detection and data quality. His 2007 survey "Duplicate Record Detection: A Survey" in IEEE Transactions on Knowledge and Data Engineering, co-authored with Ahmed Elmagarmid and Vassilios Verykios, has been widely cited in the database research community. [41]
His collaborative work with Anindya Ghose on the economic value of textual content in product reviews quantified the pricing power derived from user-generated content. [42] A related 2011 study demonstrated that the quality of spelling and grammar in product reviews significantly affects perceived helpfulness and product sales. [43] This finding was featured in Forbes , Harvard Business Review , Slate , and Freakonomics . [44] [45] [46] [47]
Beginning in 2009, Ipeirotis was part of the founding data science team at Integral Ad Science (originally AdSafe Media), where he helped develop machine learning systems for detecting inappropriate web content and advertising fraud, using crowdsourcing to generate training data for the models. [48] In 2011, working with AdSafe engineers, he uncovered an elaborate click fraud scheme that used hidden iframes to generate fraudulent ad impressions, which he termed "traffic laundering." [49] The investigation, which estimated the scheme generated hundreds of thousands of dollars monthly, was reported by The Wall Street Journal and MIT Technology Review , and led to an FBI referral. [50] [49]
In 2011, Ipeirotis co-founded Tagasauris, a media annotation startup where he served as Chief Scientist until 2013. [15] The company combined crowdsourcing with semantic graph technology to tag photo archives. A notable accomplishment was the discovery of approximately two dozen "lost" photographs from the 1973 filming of "American Graffiti" in the Magnum Photos archive; the system identified the photos by using crowdworkers to tag individuals and machine learning to connect them via shared film appearances. [51] [52]
He also served as Academic-in-Residence at Upwork (then oDesk) in 2012 and as a Visiting Scientist at Google from 2013 to 2014. [4]
In 2015, Ipeirotis collaborated with the World Bank on a report titled "The Global Opportunity in Online Outsourcing," which examined the potential of digital labor markets to provide employment pathways in developing nations. [53] Drawing on his empirical research from Mechanical Turk and oDesk, the report estimated the online outsourcing market would grow to $15–25 billion by 2020 and identified barriers preventing workers in the Global South from accessing these jobs, including lack of reliable internet infrastructure and the "reputation cold start" problem. The report has been cited in subsequent policy frameworks by the International Labour Organization and the RAND Corporation. [54]
In 2015, Ipeirotis co-founded Detectica with Foster Provost and Josh Attenberg, offering AI strategy consulting and machine learning solutions for business applications. [55] The company developed AI-driven compliance monitoring systems for financial institutions. Prior to founding Detectica, the team had designed and built the founding data science architecture for Integral Ad Science. [55]
Detectica was acquired by Compass, Inc. in November 2019. [17] At Compass, the Detectica team developed "Likely to Sell," a predictive analytics system that identifies properties likely to enter the market; the company cited the tool as a significant revenue contributor in earnings calls. [56] [57]
In July 2011, Ipeirotis published a blog post titled "Why I will never pursue cheating again," describing his experience catching 22 of 108 students plagiarizing in his "Information Technology in Business and Society" course using Turnitin software. [58] The post, which criticized the administrative burden placed on faculty who enforce academic integrity policies, went viral and was temporarily removed. [59] The incident sparked national debate about the costs of enforcing academic honesty. Business Insider reported that after Ipeirotis reported the widespread plagiarism, his student evaluations declined, costing him part of his annual raise. [60] The case was subsequently cited in Nature in a 2012 commentary on plagiarism detection in academia. [61]
In April 2012, Ipeirotis accidentally discovered a vulnerability in Google Spreadsheets that he termed a "Denial of Money" attack. While running a crowdsourcing experiment that required image classification, he uploaded 25,000 image URLs (hosted on his Amazon S3 bucket) into a Google Spreadsheet. Google's Feedfetcher bot aggressively re-downloaded all images from his S3 bucket every time the spreadsheet was processed, generating massive outgoing bandwidth. Ipeirotis reported receiving an Amazon Web Services bill of $1,177.76—approximately ten times his normal monthly rate. [62]
The incident highlighted a novel attack vector: a malicious actor could create a Google Sheet with thousands of links to a victim's large files, causing Google's servers to hammer the victim's cloud service with requests, resulting in either financial damage or a denial of service. The discovery was covered by The Economist , Wired , Business Insider, and The Hacker News. [63] [62] [64] [65] Amazon subsequently forgave the charges, and Google investigated the behavior. [62]
In December 2025, Ipeirotis developed an AI-driven oral examination system using ElevenLabs voice agents to address concerns about students using generative AI tools to submit perfect assignments while lacking deeper understanding. The system, which cost approximately $0.42 per student to administer, used a voice AI agent to call students and ask personalized questions about their submitted work. [66] The experiment was covered by Business Insider, The Decoder, and educational technology publications [67] and generated substantial online discussion about the future of educational assessments.
They designed and built the founding data science architecture for Integral Ad Science, originally founded as AdSafe Media.
In 2021, $151 million gross commission revenue was from listings that the likely to sell tool recommended to our agents before the listing was created. We expect this number to exceed $400 million in 2022.
Since launching in 2020, we have handed out millions of recommendations that be attributed more than $100 million of annual incremental revenue to these recommendations over each of the last two years.
