Project LISTEN

Last updated

Project LISTEN (Literacy Innovation that Speech Technology ENables) was a 25-year research project at Carnegie Mellon University to improve children's reading skills. Project LISTEN. The project created a computer-based Reading Tutor that listens to a child reading aloud, corrects errors, helps when the child is stuck or encounters a hard word, provides hints, assesses progress, and presents more advanced text when the child is ready. The Reading Tutor has been used daily by hundreds of children in field tests at schools in the United States, Canada, Ghana, and India. Thousands of hours of usage logged at multiple levels of detail, including millions of words read aloud, have been stored in a database that has been mined to improve the Tutor's interactions with students. An extensive list of publications (with abstracts) can be found at Carnegie Mellon University. [1]

Contents

Project LISTEN’s Reading Tutor is now being transformed into "RoboTutor" by Carnegie Mellon’s team competing in the Global Learning XPRIZE. [2] The goal of the Global Learning XPRIZE is to develop open-source Android tablet apps, in both English and Swahili, that enables children in developing countries who have little or no access to schooling to teach themselves basic reading, writing and arithmetic without adult assistance. RoboTutor is an integrated collection of intelligent tutors and educational games implemented on an Android tablet, and is now being field-tested in Tanzania.

History

Project LISTEN was led by (now Emeritus) Professor David 'Jack' Mostow, [3] who currently leads Carnegie Mellon's "RoboTutor" team in the Global Learning XPRIZE competition. [4] Project LISTEN was supported by the Defense Advanced Research Projects Agency through DARPA Order 5167, the National Science Foundation under ITR/IERI Grant No. IEC-0326153, [5] Grant No. REC-0326153 from the U.S. Department of Education’s Institute of Education Sciences [6] and under Grants R305B070458, R305A080157 and R305A080628, and by the Heinz Endowments. [7] Project LISTEN's purpose was to develop, evaluate, and refine an intelligent tutor to listen to children read aloud, and help them learn to read.

As part of the research and testing, Project LISTEN's Reading Tutor has been used with positive results by hundreds of children in the United States, Canada, and other countries. [8] (See Prototype Testing below.) Results indicated that often the students whose initial proficiency was lowest benefited most from the Reading Tutor. Of particular interest was the strong performance of the Reading Tutor for English Language Learners. [9]

Project Listen fits well into Carnegie Mellon University's Simon Initiative, whose goal is to use learning science research to improve educational practice. As noted in the History of the Simon Initiative, "The National Science Foundation included Project LISTEN’s speech recognition system as one of its top 50 innovations from 1950-2000." [10]

How it works

The goal of the Reading Tutor is to make the student experience of learning to read using it as effective or more effective than being tutored by a human coach - for example, as described at the Intervention Central website. [11] A child selects an item from a menu listing texts from a source such the Weekly Reader or authored stories. The Reading Tutor listens to the child read aloud using Carnegie Mellon’s Sphinx – II Speech Recognizer [12] [13] to process and interpret the student's oral reading. When the Reading Tutor notices a student misread a word, skip a word, get stuck, hesitate, or click for help, it responds with assistance modeled in part on expert reading teachers, adapted to the capabilities and limitations of technology. [14]

The Reading Tutor dynamically updates its estimate of a student’s reading level and picks stories a bit harder (or easier) according to the estimated level; this approach allows the Reading Tutor to aim for the zone of proximal development, that is, to expand the span of what a learner currently can do without help, toward what he or she can do with help.

The Tutor also scaffolds (provides support for) key processes in reading. It explains unfamiliar words and concepts by presenting short factoids (that is, comparisons to other words). It can provide both spoken and graphical assistance when the student has a problem. The Tutor represents Visual speech using talking mouth video clips of phonemes. It assists word identification by previewing new words, reading hard words aloud, and giving rhyming and other hints.

Detailed data on the interactions is saved in a database, and data-mining has been used to improve the Reading Tutor and investigate research questions.

Prototype Testing

Project LISTEN trials demonstrated usability, user acceptance, effective assistance, and pre-to-post-test gains. A number of controlled studies extended over several months, with student use of 20 minutes per day. Use of the Reading Tutor produced higher comprehension gains than current methods. To ensure there was no third variable involved, different treatments were compared within the same classrooms, with randomized assignment of children to treatments, stratified by pretest scores. Valid and reliable measures (Woodcock.1998) [15] were used to measure gains between pre and post test. [16] Data gathered during each trial was used to improve the efficacy of the tutor.

Various controlled studies were carried out as the Reading Tutor evolved, for example,

  1. Pilot Study(1996–97) [17]
  2. Within-classroom comparison(1998) [18] [19]
  3. Comparison to human tutors(1999-2000) [20] [21] [22]
  4. Equal-time comparison to Sustained Silent Reading (2000-2001) [23]

Since 2005, researchers both within and outside Project LISTEN have conducted and published controlled studies of the Reading Tutor. (See list in [24] ).

Awards

Project Listen has received global recognition and many awards:

See also

Related Research Articles

<span class="mw-page-title-main">Carnegie Mellon University</span> Private research university in Pittsburgh, Pennsylvania, U.S.

Carnegie Mellon University (CMU) is a private research university in Pittsburgh, Pennsylvania. The institution was originally established in 1900 by Andrew Carnegie as the Carnegie Technical Schools. In 1912, it became the Carnegie Institute of Technology and began granting four-year degrees. In 1967, it became the current-day Carnegie Mellon University through its merger with the Mellon Institute of Industrial Research, founded in 1913 by Andrew Mellon and Richard B. Mellon and formerly a part of the University of Pittsburgh.

<span class="mw-page-title-main">Carnegie Mellon School of Computer Science</span> School for computer science in the United States

The School of Computer Science (SCS) at Carnegie Mellon University in Pittsburgh, Pennsylvania, US is a school for computer science established in 1988. It has been consistently ranked among the top computer science programs over the decades. As of 2022 U.S. News & World Report ranks the graduate program as tied for second with Stanford University and University of California, Berkeley. It is ranked second in the United States on Computer Science Open Rankings, which combines scores from multiple independent rankings.

<span class="mw-page-title-main">Raj Reddy</span> Indian-American computer scientist (born 1937)

Dabbala Rajagopal "Raj" Reddy is an Indian-born American computer scientist and a winner of the Turing Award. He is one of the early pioneers of artificial intelligence and has served on the faculty of Stanford and Carnegie Mellon for over 50 years. He was the founding director of the Robotics Institute at Carnegie Mellon University. He was instrumental in helping to create Rajiv Gandhi University of Knowledge Technologies in India, to cater to the educational needs of the low-income, gifted, rural youth. He is the chairman of International Institute of Information Technology, Hyderabad. He is the first person of Asian origin to receive the Turing Award, in 1994, known as the Nobel Prize of Computer Science, for his work in the field of artificial intelligence.

A cognitive tutor is a particular kind of intelligent tutoring system that utilizes a cognitive model to provide feedback to students as they are working through problems. This feedback will immediately inform students of the correctness, or incorrectness, of their actions in the tutor interface; however, cognitive tutors also have the ability to provide context-sensitive hints and instruction to guide students towards reasonable next steps.

<span class="mw-page-title-main">Carnegie Mellon Silicon Valley</span> Branch campus in California

Carnegie Mellon Silicon Valley is a degree-granting branch campus of Carnegie Mellon University located in the heart of Silicon Valley in Mountain View, California. It was established in 2002 at the NASA Ames Research Center in Moffett Field.

An intelligent tutoring system (ITS) is a computer system that aims to provide immediate and customized instruction or feedback to learners, usually without requiring intervention from a human teacher. ITSs have the common goal of enabling learning in a meaningful and effective manner by using a variety of computing technologies. There are many examples of ITSs being used in both formal education and professional settings in which they have demonstrated their capabilities and limitations. There is a close relationship between intelligent tutoring, cognitive learning theories and design; and there is ongoing research to improve the effectiveness of ITS. An ITS typically aims to replicate the demonstrated benefits of one-to-one, personalized tutoring, in contexts where students would otherwise have access to one-to-many instruction from a single teacher, or no teacher at all. ITSs are often designed with the goal of providing access to high quality education to each and every student.

CMU Sphinx, also called Sphinx for short, is the general term to describe a group of speech recognition systems developed at Carnegie Mellon University. These include a series of speech recognizers and an acoustic model trainer (SphinxTrain).

Kenneth R. Koedinger is a professor of human–computer interaction and psychology at Carnegie Mellon University. He is the founding and current director of the Pittsburgh Science of Learning Center. He is widely known for his role in the development of the Cognitive Tutor software. He is also widely published in cognitive psychology, intelligent tutoring systems, and educational data mining, and his research group has repeatedly won "Best Paper" awards at scientific conferences in those areas, such as the EDM2008 Best Paper, ITS2006 Best Paper, ITS2004 Best Paper, and ITS2000 Best Paper.

The CMU Pronouncing Dictionary is an open-source pronouncing dictionary originally created by the Speech Group at Carnegie Mellon University (CMU) for use in speech recognition research.

John E. Laird is a computer scientist who, with Paul Rosenbloom and Allen Newell, created the Soar cognitive architecture at Carnegie Mellon University. Laird is a Professor of the Computer Science and Engineering Division of the Electrical Engineering and Computer Science Department of the University of Michigan.

The Language Technologies Institute (LTI) is a research institute at Carnegie Mellon University in Pittsburgh, Pennsylvania, United States, and focuses on the area of language technologies. The institute is home to 33 faculty with the primary scholarly research of the institute focused on machine translation, speech recognition, speech synthesis, information retrieval, parsing, information extraction, and multimodal machine learning. Until 1996, the institute existed as the Center for Machine Translation, which was established in 1986. Subsequently, from 1996 onwards, it started awarding degrees, and the name was changed to The Language Technologies Institute. The institute was founded by Professor Jaime Carbonell, who served as director until his death in February 2020. He was followed by Jamie Callan, and then Carolyn Rosé, as interim directors. In August 2023, Mona Diab became the director of the institute.

<span class="mw-page-title-main">Jaime Carbonell</span> American computer scientist (1953–2020)

Jaime Guillermo Carbonell was a computer scientist who made seminal contributions to the development of natural language processing tools and technologies. His extensive research in machine translation resulted in the development of several state-of-the-art language translation and artificial intelligence systems. He earned his B.S. degrees in Physics and in Mathematics from MIT in 1975 and did his Ph.D. under Dr. Roger Schank at Yale University in 1979. He joined Carnegie Mellon University as an assistant professor of computer science in 1979 and lived in Pittsburgh from then. He was affiliated with the Language Technologies Institute, Computer Science Department, Machine Learning Department, and Computational Biology Department at Carnegie Mellon.

<span class="mw-page-title-main">Manuela M. Veloso</span> Portuguese-American computer scientist

Manuela Maria Veloso is the Head of J.P. Morgan AI Research & Herbert A. Simon University Professor in the School of Computer Science at Carnegie Mellon University, where she was previously Head of the Machine Learning Department. She served as president of Association for the Advancement of Artificial Intelligence (AAAI) until 2014, and the co-founder and a Past President of the RoboCup Federation. She is a fellow of AAAI, Institute of Electrical and Electronics Engineers (IEEE), American Association for the Advancement of Science (AAAS), and Association for Computing Machinery (ACM). She is an international expert in artificial intelligence and robotics.

<span class="mw-page-title-main">Christopher G. Atkeson</span> American roboticist

Christopher Granger Atkeson is an American roboticist and a professor at the Robotics Institute and Human-Computer Interaction Institute at Carnegie Mellon University (CMU). Atkeson is known for his work in humanoid robots, soft robotics, and machine learning, most notably on locally weighted learning.

Vincent Aleven is a professor of human-computer interaction and director of the undergraduate program at Carnegie Mellon University's Human–Computer Interaction Institute.

<span class="mw-page-title-main">Bruce M. McLaren</span> American researcher, academic and author (born 1959)

Bruce Martin McLaren is an American researcher, scientist and author. He is an Associate Research Professor at Carnegie Mellon University and a former President of the International Artificial Intelligence in Education Society (2017-2019).

<span class="mw-page-title-main">Hideto Tomabechi</span> Japanese cognitive scientist and computer scientist

Hideto Tomabechi is a Japanese cognitive scientist computer scientist.

<span class="mw-page-title-main">Carnegie Mellon University Africa</span> University in Rwanda

Carnegie Mellon University Africa, in Kigali, Rwanda, is a global location of Carnegie Mellon University. CMU-Africa offers master's degrees in Information Technology, Electrical and Computer Engineering, and Engineering Artificial Intelligence. CMU-Africa is part of the Carnegie Mellon College of Engineering. The College of Engineering is top-ranked. In U.S. News & World Report's 2023 graduate rankings, the College of Engineering was ranked #4.

References

  1. ↑ Project LISTEN Publications
  2. ↑ ‘’Global Learning XPRIZE’’
  3. ↑ "Professor Jack Mostow"
  4. ↑ "Carnegie Mellon University's RoboTutor XPRIZE Team"
  5. ↑ “Integrating Speech and User Modeling in a Reading Tutor that Listens"
  6. ↑ Institute of Education Sciences
  7. ↑ Heinz Endowments
  8. ↑ "Project LISTEN A summary". Project Listen summary. February 7, 2013.
  9. ↑ "A Magic Reading Box: New literacy software delivers amazing results among Vancouver grade schoolers who speak English as a second language". UBC Reports. 51 (8). August 4, 2005.
  10. ↑ "The Simon Initiative"
  11. ↑ Assisted reading example
  12. ↑ "CMU Sphinx Speech Recognition Toolkit". CMU Sphinx.
  13. ↑ "CMU Sphinx-II User Guide". CMU Sphinx.
  14. ↑ "Project Listen Description". HCI Project Listen.
  15. ↑ "WOODCOCK, R.W. 1998. Woodcock Reading Mastery Tests - Revised (WRMT-R/NU). American Guidance Service, Circle Pines, Minnesota".
  16. ↑ "Project Listen Research Basis".
  17. ↑ "When Speech Input is Not an Afterthought: A Reading Tutor that Listens, In Workshop on Perceptual User Interfaces, Banff, Canada, October, 1997" (PDF).
  18. ↑ Within-classroom comparison(1998)
  19. ↑ Mostow, Jack; Aist, Greg; Burkhead, Paul; Corbett, Albert; Cuneo, Andrew; Eitelman, Susan; Huang, Cathy; Junker, Brian; Sklar, Mary Beth; Tobin, Brian (22 July 2016). "Evaluation of an Automated Reading Tutor That Listens: Comparison to Human Tutoring and Classroom Instruction". Journal of Educational Computing Research. 29 (1): 61–117. doi:10.2190/06AX-QW99-EQ5G-RDCF. S2CID   46073812.
  20. ↑ Jack Mostow; Greg Aist; Paul Burkhead; Albert Corbett; Andrew Cuneo; Susan Rossbach; Brian Tobin, Independent practice versus computer-guided oral reading: Equal-time comparison of sustained silent reading to an automated reading tutor that listens (PDF)
  21. ↑ "A controlled evaluation of computer- versus human-assisted oral reading" (PDF).
  22. ↑ A controlled evaluation of computer-versus human-assisted oral reading. Tenth Artificial Intelligence in Education (AI-ED) Conference, J.D. MOORE, C.L. REDFIELD and W.L. JOHNSON, Eds. Amsterdam: IOS Press, San Antonio, Texas, 586-588.
  23. ↑ Mostow, Jack; Nelson-Taylor, Jessica; Beck, Joseph E. (2013). "Computer-Guided Oral Reading versus Independent Practice: Comparison of Sustained Silent Reading to an Automated Reading Tutor That Listens". Journal of Educational Computing Research. 49 (2): 249–276. doi:10.2190/EC.49.2.g. S2CID   62382102.
  24. ↑ "Published Studies of Project Listen".
  25. ↑ Jack Mostow; Steven F. Roth; Alexander G. Hauptmann; Matthew Kane. "(Outstanding Paper) A Prototype Reading Coach that Listens".
  26. ↑ The Simon Initiative
  27. ↑ "Nifty 50".
  28. ↑ "Helping Children Learn Vocabulary during Computer-Assisted Oral Reading".
  29. ↑ "Project Listen Videos on Reading Rockets PBS Series".
  30. ↑ Beck, J. E., Chang, K.-m., Mostow, J., & Corbett, A. (2008). Introducing the Bayesian Evaluation and Assessment methodology. Programming and Software Engineering. Springer. pp. 383–394. ISBN   9783540691303.{{cite book}}: CS1 maint: multiple names: authors list (link)
  31. ↑ "(Best Paper Award) Does help help? Introducing the Bayesian Evaluation and Assessment methodology" (PDF).
  32. ↑ Beck, Joseph E.; Mostow, Jack (2008), "How Who Should Practice: Using Learning Decomposition to Evaluate the Efficacy of Different Types of Practice for Different Types of Students", Intelligent Tutoring Systems, Lecture Notes in Computer Science, vol. 5091, pp. 353–362, doi:10.1007/978-3-540-69132-7_39, ISBN   978-3-540-69130-3
  33. ↑ "(Best Paper Nominee) How who should practice: Using learning decomposition to evaluate the efficacy of different types of practice for different types of students" (PDF).
  34. ↑ Jack Mostow; Kai-min Chang; Jessica Nelson (January 2013). "Toward Exploiting EEG Input in a Reading Tutor". International Journal of Artificial Intelligence in Education. 22 (1–2): 230–237. doi:10.3233/JAI-130033.
  35. ↑ "(Best Paper Award) Toward Exploiting EEG Input in a Reading Tutor, Mostow, J., Chang, K.-m., & Nelson, J., Proceedings of the 15th International Conference on Artificial Intelligence in Education, Auckland, NZ, 230-237, 2011" (PDF).
  36. ↑ Sunayana Sitaram; Jack Mostow. "Mining Data from Project LISTEN's Reading Tutor to Analyze Development of Children's Oral Reading Prosody". Proceedings of the Twenty-Fifth International Florida Artificial Intelligence Research Society Conference, AASI Digital Library.
  37. ↑ "(Best Paper Award) Mining Data from Project LISTEN's Reading Tutor to Analyze Development of Children's Oral Reading Prosody" (PDF).
  38. ↑ Xu, Y., Mostow, J. "Comparison of methods to trace multiple subskills: Is LR-DBN best?" (PDF). Proceedings of the Fifth International Conference on Educational Data Mining, Chania, Crete, Greece.{{cite journal}}: CS1 maint: multiple names: authors list (link)
  39. ↑ "(Best Student Paper Award) Comparison of methods to trace multiple subskills: Is LR-DBN best?" (PDF).
  40. ↑ Lallé, S., Mostow, J., Luengo, V., & Guin, N.page=161-170 (22 June 2013). "Comparing Student Models in Different Formalisms by Predicting their Impact on Help Success". Proceedings of the 16th International Conference on Artificial Intelligence in Education, Memphis, TN 2013. Springer US. ISBN   9783642391125.{{cite book}}: CS1 maint: multiple names: authors list (link)
  41. ↑ "(Finalist for Best Paper Award) Comparing Student Models in Different Formalisms by Predicting their Impact on Help Success" (PDF).
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.