Lip reading

Last updated

Lip reading, also known as speechreading, is a technique of understanding a limited range of speech by visually interpreting the movements of the lips, face and tongue without sound. Estimates of the range of lip reading vary, with some figures as low as 30% because lip reading relies on context, language knowledge, and any residual hearing. [1] Although lip reading is used most extensively by deaf and hard-of-hearing people, most people with normal hearing process some speech information from sight of the moving mouth. [2]

Contents

Process

Although speech perception is considered to be an auditory skill, it is intrinsically multimodal, since producing speech requires the speaker to make movements of the lips, teeth and tongue which are often visible in face-to-face communication. Information from the lips and face supports aural comprehension [3] and most fluent listeners of a language are sensitive to seen speech actions (see McGurk effect). The extent to which people make use of seen speech actions varies with the visibility of the speech action and the knowledge and skill of the perceiver.

Phonemes and visemes

The phoneme is the smallest detectable unit of sound in a language that serves to distinguish words from one another. /pit/ and /pik/ differ by one phoneme and refer to different concepts. Spoken English has about 44 phonemes. For lip reading, the number of visually distinctive units - visemes - is much smaller, thus several phonemes map onto a few visemes. This is because many phonemes are produced within the mouth and throat, and are hard to see. These include glottal consonants and most gestures of the tongue. Voiced and unvoiced pairs look identical, such as [p] and [b], [k] and [g], [t] and [d], [f] and [v], and [s] and [z]; likewise for nasalisation (e.g. [m] vs. [b]). Homophenes are words that look similar when lip read, but which contain different phonemes. Because there are about three times as many phonemes as visemes in English, it is often claimed that only 30% of speech can be lip read. Homophenes are a crucial source of mis-lip reading.

The legend to this puzzle reads "Here is a class of a dozen boys, who, being called up to give their names were photographed by the instantaneous process just as each one was commencing to pronounce his own name. The twelve names were Oom, Alden, Eastman, Alfred, Arthur, Luke, Fletcher, Matthew, Theodore, Richard, Shirmer, and Hisswald. Now it would not seem possible to be able to give the correct name to each of the twelve boys, but if you practice the list over to each one, you will find it not a difficult task to locate the proper name for every one of the boys." Sam Loyd lipreading puzzle.jpg
The legend to this puzzle reads "Here is a class of a dozen boys, who, being called up to give their names were photographed by the instantaneous process just as each one was commencing to pronounce his own name. The twelve names were Oom, Alden, Eastman, Alfred, Arthur, Luke, Fletcher, Matthew, Theodore, Richard, Shirmer, and Hisswald. Now it would not seem possible to be able to give the correct name to each of the twelve boys, but if you practice the list over to each one, you will find it not a difficult task to locate the proper name for every one of the boys."

Co-articulation

Visemes can be captured as still images, but speech unfolds in time. The smooth articulation of speech sounds in sequence can mean that mouth patterns may be 'shaped' by an adjacent phoneme: the 'th' sound in 'tooth' and in 'teeth' appears very different because of the vocalic context. This feature of dynamic speech-reading affects lip-reading 'beyond the viseme'. [5]

How can it 'work' with so few visemes?

While visemes offer a useful starting point for understanding lipreading, spoken distinctions within a viseme can be distinguished and can help support identification. [6] Moreover, the statistical distribution of phonemes within the lexicon of a language is uneven. While there are clusters of words which are phonemically similar to each other ('lexical neighbors', such as spit/sip/sit/stick...etc.), others are unlike all other words: they are 'unique' in terms of the distribution of their phonemes ('umbrella' may be an example). Skilled users of the language bring this knowledge to bear when interpreting speech, so it is generally harder to identify a heard word with many lexical neighbors than one with few neighbors. Applying this insight to seen speech, some words in the language can be unambiguously lip-read even when they contain few visemes - simply because no other words could possibly 'fit'. [7]

Variation in readability and skill

Many factors affect the visibility of a speaking face, including illumination, movement of the head/camera, frame-rate of the moving image and distance from the viewer (see e.g. [8] ). Head movement that accompanies normal speech can also improve lip-reading, independently of oral actions. [9] However, when lip-reading connected speech, the viewer's knowledge of the spoken language, familiarity with the speaker and style of speech, and the context of the lip-read material [10] are as important as the visibility of the speaker. While most hearing people are sensitive to seen speech, there is great variability in individual speechreading skill. Good lipreaders are often more accurate than poor lipreaders at identifying phonemes from visual speech.

A simple visemic measure of 'lipreadability' has been questioned by some researchers. [11] The 'phoneme equivalence class' measure takes into account the statistical structure of the lexiconand can also accommodate individual differences in lip-reading ability. [12] [13] In line with this, excellent lipreading is often associated with more broad-based cognitive skills including general language proficiency, executive function and working memory. [14] [15]

Lipreading and language learning in hearing infants and children

The first few months

Seeing the mouth plays a role in the very young infant's early sensitivity to speech, and prepares them to become speakers at 1 – 2 years. In order to imitate, a baby must learn to shape their lips in accordance with the sounds they hear; seeing the speaker may help them to do this. [16] Newborns imitate adult mouth movements such as sticking out the tongue or opening the mouth, which could be a precursor to further imitation and later language learning. [17] Infants are disturbed when audiovisual speech of a familiar speaker is desynchronized [18] and tend to show different looking patterns for familiar than for unfamiliar faces when matched to (recorded) voices. [19] Infants are sensitive to McGurk illusions months before they have learned to speak. [20] [21] These studies and many more point to a role for vision in the development of sensitivity to (auditory) speech in the first half-year of life.

The next six months; a role in learning a native language

Until around six months of age, most hearing infants are sensitive to a wide range of speech gestures - including ones that can be seen on the mouth - which may or may not later be part of the phonology of their native language. But in the second six months of life, the hearing infant shows perceptual narrowing for the phonetic structure of their own language - and may lose the early sensitivity to mouth patterns that are not useful. The speech sounds /v/ and /b/ which are visemically distinctive in English but not in Castilian Spanish are accurately distinguished in Spanish-exposed and English-exposed babies up to the age of around 6 months. However, older Spanish-exposed infants lose the ability to 'see' this distinction, while it is retained for English-exposed infants. [22] Such studies suggest that rather than hearing and vision developing in independent ways in infancy, multimodal processing is the rule, not the exception, in (language) development of the infant brain. [23]

Early language production: one to two years

Given the many studies indicating a role for vision in the development of language in the pre-lingual infant, the effects of congenital blindness on language development are surprisingly small. 18-month-olds learn new words more readily when they hear them, and do not learn them when they are shown the speech movements without hearing. [24] However, children blind from birth can confuse /m/ and /n/ in their own early production of English words – a confusion rarely seen in sighted hearing children, since /m/ and /n/ are visibly distinctive, but auditorially confusable. [25] The role of vision in children aged 1–2 years may be less critical to the production of their native language, since, by that age, they have attained the skills they need to identify and imitate speech sounds. However, hearing a non-native language can shift the child's attention to visual and auditory engagement by way of lipreading and listening in order to process, understand and produce speech. [26]

In childhood

Studies with pre-lingual infants and children use indirect, non-verbal measures to indicate sensitivity to seen speech. Explicit lip-reading can be reliably tested in hearing preschoolers by asking them to 'say aloud what I say silently'. [27] In school-age children, lipreading of familiar closed-set words such as number words can be readily elicited. [28] Individual differences in lip-reading skill, as tested by asking the child to 'speak the word that you lip-read', or by matching a lip-read utterance to a picture, [29] show a relationship between lip-reading skill and age. [30] [31]

In hearing adults: lifespan considerations

While lip-reading silent speech poses a challenge for most hearing people, adding sight of the speaker to heard speech improves speech processing under many conditions. The mechanisms for this, and the precise ways in which lip-reading helps, are topics of current research. [32] Seeing the speaker helps at all levels of speech processing from phonetic feature discrimination to interpretation of pragmatic utterances. [33] The positive effects of adding vision to heard speech are greater in noisy than quiet environments, [34] where by making speech perception easier, seeing the speaker can free up cognitive resources, enabling deeper processing of speech content.

As hearing becomes less reliable in old-age, people may tend to rely more on lip-reading, and are encouraged to do so. However, greater reliance on lip-reading may not always make good the effects of age-related hearing loss. Cognitive decline in aging may be preceded by and/or associated with measurable hearing loss. [35] [36] Thus lipreading may not always be able to fully compensate for the combined hearing and cognitive age-related decrements.

In specific (hearing) populations

A number of studies report anomalies of lipreading in populations with distinctive developmental disorders. Autism: People with autism may show reduced lipreading abilities and reduced reliance on vision in audiovisual speech perception. [37] [38] This may be associated with gaze-to-the-face anomalies in these people. [39] Williams syndrome: People with Williams syndrome show some deficits in speechreading which may be independent of their visuo-spatial difficulties. [40] Specific Language Impairment: Children with SLI are also reported to show reduced lipreading sensitivity, [41] as are people with dyslexia. [42]

Deafness

Debate has raged for hundreds of years over the role of lip-reading ('oralism') compared with other communication methods (most recently, total communication) in the education of deaf people. The extent to which one or other approach is beneficial depends on a range of factors, including level of hearing loss of the deaf person, age of hearing loss, parental involvement and parental language(s). Then there is a question concerning the aims of the deaf person and their community and carers. Is the aim of education to enhance communication generally, to develop sign language as a first language, or to develop skills in the spoken language of the hearing community? Researchers now focus on which aspects of language and communication may be best delivered by what means and in which contexts, given the hearing status of the child and her family, and their educational plans. [43] Bimodal bilingualism (proficiency in both speech and sign language) is one dominant current approach in language education for the deaf child. [44]

Deaf people are often better lip-readers than people with normal hearing. [45] Some deaf people practice as professional lipreaders, for instance in forensic lipreading. In deaf people who have a cochlear implant, pre-implant lip-reading skill can predict post-implant (auditory or audiovisual) speech processing. [46] In adults, the later the age of implantation, the better the visual speechreading abilities of the deaf person. [47] For many deaf people, access to spoken communication can be helped when a spoken message is relayed via a trained, professional lip-speaker . [48] [49]

In connection with lipreading and literacy development, children born deaf typically show delayed development of literacy skills [50] which can reflect difficulties in acquiring elements of the spoken language. [51] In particular, reliable phoneme-grapheme mapping may be more difficult for deaf children, who need to be skilled speech-readers in order to master this necessary step in literacy acquisition. Lip-reading skill is associated with literacy abilities in deaf adults and children [52] [53] and training in lipreading may help to develop literacy skills. [54]

Cued Speech uses lipreading with accompanying hand shapes that disambiguate the visemic (consonant) lipshape. Cued speech is said to be easier for hearing parents to learn than a sign language, and studies, primarily from Belgium, show that a deaf child exposed to cued speech in infancy can make more efficient progress in learning a spoken language than from lipreading alone. [55] The use of cued speech in cochlear implantation for deafness is likely to be positive. [56] A similar approach, involving the use of handshapes accompanying seen speech, is Visual Phonics, which is used by some educators to support the learning of written and spoken language.

Teaching and training

The aim of teaching and training in lipreading is to develop awareness of the nature of lipreading, and to practice ways of improving the ability to perceive speech 'by eye'. [57] While the value of lipreading training in improving 'hearing by eye' was not always clear, especially for people with acquired hearing loss, there is evidence that systematic training in alerting students to attend to seen speech actions can be beneficial. [58] Lipreading classes, often called lipreading and managing hearing loss classes, are mainly aimed at adults who have hearing loss. The highest proportion of adults with hearing loss have an age-related, or noise-related loss; with both of these forms of hearing loss, the high-frequency sounds are lost first. Since many of the consonants in speech are high-frequency sounds, speech becomes distorted. Hearing aids help but may not cure this. Lipreading classes have been shown to be of benefit in UK studies commissioned by the Action on Hearing Loss charity [59] (2012).

Trainers recognise that lipreading is an inexact art. Students are taught to watch the lips, tongue and jaw movements, to follow the stress and rhythm of language, to use their residual hearing, with or without hearing aids, to watch expression and body language, and to use their ability to reason and deduce. They are taught the lipreaders' alphabet, groups of sounds that look alike on the lips (visemes) like p, b, m, or f, v. The aim is to get the gist, so as to have the confidence to join in conversation and avoid the damaging social isolation that often accompanies hearing loss. Lipreading classes are recommended for anyone who struggles to hear in noise, and help to adjust to hearing loss.

Tests

Most tests of lipreading were devised to measure individual differences in performing specific speech-processing tasks and to detect changes in performance following training. Lipreading tests have been used with relatively small groups in experimental settings, or as clinical indicators with individual patients and clients. That is, most lipreading tests to date have limited validity as markers of lipreading skill in the general population. [60]

Lipreading and lip-speaking by machine

Automated lip-reading has been a topic of interest in computational engineering, as well as in science fiction movies. The computational engineer Steve Omohundro, among others, pioneered its development. In facial animation, the aim is to generate realistic facial actions, especially mouth movements, that simulate human speech actions. Computer algorithms to deform or manipulate images of faces can be driven by heard or written language. Systems may be based on detailed models derived from facial movements (motion capture); on anatomical modelling of actions of the jaw, mouth and tongue; or on mapping of known viseme- phoneme properties. [61] [62] Facial animation has been used in speechreading training (demonstrating how different sounds 'look'). [63] These systems are a subset of speech synthesis modelling which aim to deliver reliable 'text-to-(seen)-speech' outputs. A complementary aim—the reverse of making faces move in speech—is to develop computer algorithms that can deliver realistic interpretations of speech (i.e. a written transcript or audio record) from natural video data of a face in action: this is facial speech recognition. These models too can be sourced from a variety of data. [64] Automatic visual speech recognition from video has been quite successful in distinguishing different languages (from a corpus of spoken language data). [65] Demonstration models, using machine-learning algorithms, have had some success in lipreading speech elements, such as specific words, from video [66] and for identifying hard-to-lipread phonemes from visemically similar seen mouth actions. [67] Machine-based speechreading is now making successful use of neural-net based algorithms which use large databases of speakers and speech material (following the successful model for auditory automatic speech recognition). [68]

Uses for machine lipreading could include automated lipreading of video-only records, automated lipreading of speakers with damaged vocal tracts, and speech processing in face-to-face video (i.e. from videophone data). Automated lipreading may help in processing noisy or unfamiliar speech. [69] Automated lipreading may contribute to biometric person identification, replacing password-based identification. [70] [71]

The brain

Following the discovery that auditory brain regions, including Heschl's gyrus, were activated by seen speech, [72] the neural circuitry for speechreading was shown to include supra-modal processing regions, especially superior temporal sulcus (all parts) as well as posterior inferior occipital-temporal regions including regions specialised for the processing of faces and biological motion. [73] In some but not all studies, activation of Broca's area is reported for speechreading, [74] [75] suggesting that articulatory mechanisms can be activated in speechreading. [76] Studies of the time course of audiovisual speech processing showed that sight of speech can prime auditory processing regions in advance of the acoustic signal. [77] [78] Better lipreading skill is associated with greater activation in (left) superior temporal sulcus and adjacent inferior temporal (visual) regions in hearing people. [79] [80] In deaf people, the circuitry devoted to speechreading appears to be very similar to that in hearing people, with similar associations of (left) superior temporal activation and lipreading skill. [81]

Related Research Articles

<span class="mw-page-title-main">Tadoma</span>

Tadoma is a method of communication utilized by deafblind individuals, in which the listener places their little finger on the speaker's lips and their fingers along the jawline. The middle three fingers often fall along the speaker's cheeks with the little finger picking up the vibrations of the speaker's throat. It is sometimes referred to as tactile lipreading, as the listener feels the movement of the lips, the vibrations of the vocal cords, expansion of the cheeks and the warm air produced by nasal phonemes such as 'N' and 'M'. Hand positioning can vary, and it is a sometimes also used by hard-of-hearing people to supplement their remaining hearing.

<span class="mw-page-title-main">McGurk effect</span> Perceptual illusion

The McGurk effect is a perceptual phenomenon that demonstrates an interaction between hearing and vision in speech perception. The illusion occurs when the auditory component of one sound is paired with the visual component of another sound, leading to the perception of a third sound. The visual information a person gets from seeing a person speak changes the way they hear the sound. If a person is getting poor-quality auditory information but good-quality visual information, they may be more likely to experience the McGurk effect. Integration abilities for audio and visual information may also influence whether a person will experience the effect. People who are better at sensory integration have been shown to be more susceptible to the effect. Many people are affected differently by the McGurk effect based on many factors, including brain damage and other disorders.

A viseme is any of several speech sounds that look the same, for example when lip reading.

The two-streams hypothesis is a model of the neural processing of vision as well as hearing. The hypothesis, given its initial characterisation in a paper by David Milner and Melvyn A. Goodale in 1992, argues that humans possess two distinct visual systems. Recently there seems to be evidence of two distinct auditory systems as well. As visual information exits the occipital lobe, and as sound leaves the phonological network, it follows two main pathways, or "streams". The ventral stream leads to the temporal lobe, which is involved with object and visual identification and recognition. The dorsal stream leads to the parietal lobe, which is involved with processing the object's spatial location relative to the viewer and with speech repetition.

Speech perception is the process by which the sounds of language are heard, interpreted, and understood. The study of speech perception is closely linked to the fields of phonology and phonetics in linguistics and cognitive psychology and perception in psychology. Research in speech perception seeks to understand how human listeners recognize speech sounds and use this information to understand spoken language. Speech perception research has applications in building computer systems that can recognize speech, in improving speech recognition for hearing- and language-impaired listeners, and in foreign-language teaching.

Bimodal bilingualism is an individual or community's bilingual competency in at least one oral language and at least one sign language, which utilize two different modalities. An oral language consists of a vocal-aural modality versus a signed language which consists of a visual-spatial modality. A substantial number of bimodal bilinguals are children of deaf adults (CODA) or other hearing people who learn sign language for various reasons. Deaf people as a group have their own sign language(s) and culture that is referred to as Deaf, but invariably live within a larger hearing culture with its own oral language. Thus, "most deaf people are bilingual to some extent in [an oral] language in some form". In discussions of multilingualism in the United States, bimodal bilingualism and bimodal bilinguals have often not been mentioned or even considered. This is in part because American Sign Language, the predominant sign language used in the U.S., only began to be acknowledged as a natural language in the 1960s. However, bimodal bilinguals share many of the same traits as traditional bilinguals, as well as differing in some interesting ways, due to the unique characteristics of the Deaf community. Bimodal bilinguals also experience similar neurological benefits as do unimodal bilinguals, with significantly increased grey matter in various brain areas and evidence of increased plasticity as well as neuroprotective advantages that can help slow or even prevent the onset of age-related cognitive diseases, such as Alzheimer's and dementia.

Auditory processing disorder (APD), rarely known as King-Kopetzky syndrome or auditory disability with normal hearing (ADN), is a neurodevelopmental disorder affecting the way the brain processes sounds. Individuals with APD usually have normal structure and function of the outer, middle, and inner ear. However, they cannot process the information they hear in the same way as others do, which leads to difficulties in recognizing and interpreting sounds, especially the sounds composing speech. It is thought that these difficulties arise from dysfunction in the central nervous system. This is, in part, essentially a failure of the cocktail party effect found in most people.

Aural rehabilitation is the process of identifying and diagnosing a hearing loss, providing different types of therapies to clients who are hard of hearing, and implementing different amplification devices to aid the client's hearing abilities. Aural rehab includes specific procedures in which each therapy and amplification device has as its goal the habilitation or rehabilitation of persons to overcome the handicap (disability) caused by a hearing impairment or deafness.

Speech shadowing is a psycholinguistic experimental technique in which subjects repeat speech at a delay to the onset of hearing the phrase. The time between hearing the speech and responding, is how long the brain takes to process and produce speech. The task instructs participants to shadow speech, which generates intent to reproduce the phrase while motor regions in the brain unconsciously process the syntax and semantics of the words spoken. Words repeated during the shadowing task would also imitate the parlance of the shadowed speech.

The motor theory of speech perception is the hypothesis that people perceive spoken words by identifying the vocal tract gestures with which they are pronounced rather than by identifying the sound patterns that speech generates. It originally claimed that speech perception is done through a specialized module that is innate and human-specific. Though the idea of a module has been qualified in more recent versions of the theory, the idea remains that the role of the speech motor system is not only to produce speech articulations but also to detect them.

<span class="mw-page-title-main">Superior temporal sulcus</span> Part of the brains temporal lobe

In the human brain, the superior temporal sulcus (STS) is the sulcus separating the superior temporal gyrus from the middle temporal gyrus in the temporal lobe of the brain. A sulcus is a deep groove that curves into the largest part of the brain, the cerebrum, and a gyrus is a ridge that curves outward of the cerebrum.

Prelingual deafness refers to deafness that occurs before learning speech or language. Speech and language typically begin to develop very early with infants saying their first words by age one. Therefore, prelingual deafness is considered to occur before the age of one, where a baby is either born deaf or loses hearing before the age of one. This hearing loss may occur for a variety of reasons and impacts cognitive, social, and language development.

Auditory feedback (AF) is an aid used by humans to control speech production and singing by helping the individual verify whether the current production of speech or singing is in accordance with his acoustic-auditory intention. This process is possible through what is known as the auditory feedback loop, a three-part cycle that allows individuals to first speak, then listen to what they have said, and lastly, correct it when necessary. From the viewpoint of movement sciences and neurosciences, the acoustic-auditory speech signal can be interpreted as the result of movements of speech articulators. Auditory feedback can hence be inferred as a feedback mechanism controlling skilled actions in the same way that visual feedback controls limb movements.

Speech acquisition focuses on the development of vocal, acoustic and oral language by a child. This includes motor planning and execution, pronunciation, phonological and articulation patterns.

Forensic speechreading is the use of speechreading for information or evidential purposes. Forensic speechreading can be considered a branch of forensic linguistics. In contrast to speaker recognition which is often the focus of voice analysis from an audio record, forensic speechreading usually aims to establish the content of speech, since the identity of the talker is usually apparent. Often, it involves the production of a transcript of lipread video-records of talk that lack a usable audiotrack, for example CCTV material. Occasionally, 'live' lipreading is involved, for example in the Casey Anthony case. Forensic speechreaders are usually deaf or from deaf families (CODA), and use speechreading in their daily lives to a greater extent than people with normal hearing outside the deaf community. Some speechreading tests suggest deaf people can be better lipreaders than most hearing people.

Phonemic restoration effect is a perceptual phenomenon where under certain conditions, sounds actually missing from a speech signal can be restored by the brain and may appear to be heard. The effect occurs when missing phonemes in an auditory signal are replaced with a noise that would have the physical properties to mask those phonemes, creating an ambiguity. In such ambiguity, the brain tends towards filling in absent phonemes. The effect can be so strong that some listeners may not even notice that there are phonemes missing. This effect is commonly observed in a conversation with heavy background noise, making it difficult to properly hear every phoneme being spoken. Different factors can change the strength of the effect, including how rich the context or linguistic cues are in speech, as well as the listener's state, such as their hearing status or age.

Language acquisition is a natural process in which infants and children develop proficiency in the first language or languages that they are exposed to. The process of language acquisition is varied among deaf children. Deaf children born to deaf parents are typically exposed to a sign language at birth and their language acquisition follows a typical developmental timeline. However, at least 90% of deaf children are born to hearing parents who use a spoken language at home. Hearing loss prevents many deaf children from hearing spoken language to the degree necessary for language acquisition. For many deaf children, language acquisition is delayed until the time that they are exposed to a sign language or until they begin using amplification devices such as hearing aids or cochlear implants. Deaf children who experience delayed language acquisition, sometimes called language deprivation, are at risk for lower language and cognitive outcomes. However, profoundly deaf children who receive cochlear implants and auditory habilitation early in life often achieve expressive and receptive language skills within the norms of their hearing peers; age at implantation is strongly and positively correlated with speech recognition ability. Early access to language through signed language or technology have both been shown to prepare children who are deaf to achieve fluency in literacy skills.

Language deprivation in deaf and hard-of-hearing children is a delay in language development that occurs when sufficient exposure to language, spoken or signed, is not provided in the first few years of a deaf or hard of hearing child's life, often called the critical or sensitive period. Early intervention, parental involvement, and other resources all work to prevent language deprivation. Children who experience limited access to language—spoken or signed—may not develop the necessary skills to successfully assimilate into the academic learning environment. There are various educational approaches for teaching deaf and hard of hearing individuals. Decisions about language instruction is dependent upon a number of factors including extent of hearing loss, availability of programs, and family dynamics.

<span class="mw-page-title-main">Quentin Summerfield</span> British psychologist

Quentin Summerfield is a British psychologist, specialising in hearing. He joined the Medical Research Council Institute of Hearing Research in 1977 and served as its deputy director from 1993 to 2004, before moving on to a chair in psychology at The University of York. He served as head of the Psychology department from 2011 to 2017 and retired in 2018, becoming an emeritus professor. From 2013 to 2018, he was a member of the University of York's Finance & Policy Committee. From 2015 to 2018, he was a member of York University's governing body, the Council.

Deaf and hard of hearing individuals with additional disabilities are referred to as "Deaf Plus" or "Deaf+". Deaf children with one or more co-occurring disabilities could also be referred to as hearing loss plus additional disabilities or Deafness and Diversity (D.A.D.). About 40–50% of deaf children experience one or more additional disabilities, with learning disabilities, intellectual disabilities, autism spectrum disorder (ASD), and visual impairments being the four most concomitant disabilities. Approximately 7–8% of deaf children have a learning disability. Deaf plus individuals utilize various language modalities to best fit their communication needs.

References

  1. ↑ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3155585/
  2. ↑ Woodhouse, L; Hickson, L; Dodd, B (2009). "Review of visual speech perception by hearing and hearing-impaired people: clinical implications". International Journal of Language and Communication Disorders. 44 (3): 253–70. doi:10.1080/13682820802090281. PMID   18821117.
  3. ↑ Erber, NP (1969). "Interaction of audition and vision in the recognition of oral speech stimuli". J Speech Hear Res. 12 (2): 423–5. doi:10.1044/jshr.1202.423. PMID   5808871.
  4. ↑ Sam Loyd's Cyclopedia of Puzzles, 1914
  5. ↑ Benguerel, AP; Pichora-Fuller, MK (1982). "Coarticulation effects in lipreading". J Speech Hear Res. 25 (4): 600–7. doi:10.1044/jshr.2504.600. PMID   7162162.
  6. ↑ Files, BT (2015). "Visual speech discrimination and identification of natural and synthetic consonant stimuli". Front. Psychol. 6: 878. doi: 10.3389/fpsyg.2015.00878 . PMC   4499841 . PMID   26217249.
  7. ↑ Auer, ET (2010). "Investigating speechreading and deafness". Journal of the American Academy of Audiology. 21 (3): 163–8. doi:10.3766/jaaa.21.3.4. PMC   3715375 . PMID   20211120.
  8. ↑ Jordan, TR; Thomas, SM (2011). "When half a face is as good as a whole: effects of simple substantial occlusion on visual and audiovisual speech perception". Atten Percept Psychophys. 73 (7): 2270–85. doi: 10.3758/s13414-011-0152-4 . PMID   21842332.
  9. ↑ Thomas, SM; Jordan, TR (2004). "Contributions of oral and extraoral facial movement to visual and audiovisual speech perception". J Exp Psychol Hum Percept Perform. 30 (5): 873–88. doi:10.1037/0096-1523.30.5.873. PMID   15462626.
  10. ↑ Spehar, B; Goebel, S; Tye-Murray, N (2015). "Effects of Context Type on Lipreading and Listening Performance and Implications for Sentence Processing". J Speech Lang Hear Res. 58 (3): 1093–102. doi:10.1044/2015_JSLHR-H-14-0360. PMC   4610295 . PMID   25863923.
  11. ↑ Files, BT; Tjan, BS; Jiang, J; Bernstein, LE (2015). "Visual speech discrimination and identification of natural and synthetic consonant stimuli". Front Psychol. 6: 878. doi: 10.3389/fpsyg.2015.00878 . PMC   4499841 . PMID   26217249.
  12. ↑ Auer, ET; Bernstein, LE (1997). "Speechreading and the structure of the lexicon: computationally modeling the effects of reduced phonetic distinctiveness on lexical uniqueness". Journal of the Acoustical Society of America. 102 (6): 3704–10. Bibcode:1997ASAJ..102.3704A. doi:10.1121/1.420402. PMID   9407662.
  13. ↑ Feld J1, Sommers M 2011 There Goes the Neighborhood: Lipreading and the Structure of the Mental Lexicon. Speech Commun. Feb;53(2):220-228
  14. ↑ Tye-Murray, N; Hale, S; Spehar, B; Myerson, J; Sommers, MS (2014). "Lipreading in school-age children: the roles of age, hearing status, and cognitive ability". J Speech Lang Hear Res. 57 (2): 556–65. doi:10.1044/2013_JSLHR-H-12-0273. PMC   5736322 . PMID   24129010.
  15. ↑ Feld, JE; Sommers, MS (2009). "Lipreading, processing speed, and working memory in younger and older adults". J Speech Lang Hear Res. 52 (6): 1555–65. doi:10.1044/1092-4388(2009/08-0137). PMC   3119632 . PMID   19717657.
  16. ↑ "HuffPost - Breaking News, U.S. and World News". HuffPost. Retrieved 2020-10-11.
  17. ↑ Meltzoff, AN; Moore, MK (1977). "Imitation of facial and manual gestures by human neonates". Science. 198 (4312): 74–8. doi:10.1126/science.897687. PMID   897687.
  18. ↑ Dodd B.1976 Lip reading in infants: attention to speech presented in- and out-of-synchrony. Cognitive Psychology Oct;11(4):478-84
  19. ↑ Spelke, E (1976). "Infants intermodal perception of events". Cognitive Psychology. 8 (4): 553–560. doi:10.1016/0010-0285(76)90018-9. S2CID   1226796.
  20. ↑ Burnham, D; Dodd, B (2004). "Auditory-visual speech integration by prelinguistic infants: perception of an emergent consonant in the McGurk effect". Developmental Psychobiology. 45 (4): 204–20. doi:10.1002/dev.20032. PMID   15549685.
  21. ↑ Rosenblum, LD; Schmuckler, MA; Johnson, JA (1997). "The McGurk effect in infants". Percept Psychophys. 59 (3): 347–57. doi: 10.3758/BF03211902 . PMID   9136265.
  22. ↑ Pons, F; et al. (2009). "Narrowing of intersensory speech perception in infancy". Proceedings of the National Academy of Sciences. 106 (26): 10598–602. Bibcode:2009PNAS..10610598P. doi: 10.1073/pnas.0904134106 . PMC   2705579 . PMID   19541648.
  23. ↑ Lewkowicz, DJ; Ghazanfar, AA (2009). "The emergence of multisensory systems through perceptual narrowing". Trends in Cognitive Sciences. 13 (11): 470–8. CiteSeerX   10.1.1.554.4323 . doi:10.1016/j.tics.2009.08.004. PMID   19748305. S2CID   14289579.
  24. ↑ Havy, M., Foroud, A., Fais, L., & Werker, J.F. (in press; online January 26, 2017). The role of auditory and visual speech in word-learning at 18 months and in adulthood. Child Development. (Pre-print version)
  25. ↑ Mills, A.E. 1987 The development of phonology in the blind child. In B.Dodd & R.Campbell(Eds) Hearing by Eye: the psychology of lipreading, Hove UK, Lawrence Erlbaum Associates
  26. ↑ Lewkowicz, DJ; Hansen-Tift, AM (Jan 2012). "Infants deploy selective attention to the mouth of a talking face when learning speech". Proceedings of the National Academy of Sciences. 109 (5): 1431–6. Bibcode:2012PNAS..109.1431L. doi: 10.1073/pnas.1114783109 . PMC   3277111 . PMID   22307596.
  27. ↑ Davies R1, Kidd E; Lander, K (2009). "Investigating the psycholinguistic correlates of speechreading in preschool age children". International Journal of Language and Communication Disorders. 44 (2): 164–74. doi:10.1080/13682820801997189. hdl: 11858/00-001M-0000-002E-2344-8 . PMID   18608607.{{cite journal}}: CS1 maint: numeric names: authors list (link)
  28. ↑ Dodd B. 1987 The acquisition of lipreading skills by normally hearing children. In B.Dodd & R.Campbell (Eds) Hearing by Eye, Erlbaum NJ pp163-176
  29. ↑ Jerger, S; et al. (2009). "Developmental shifts in children's sensitivity to visual speech: a new multimodal picture-word task". J Exp Child Psychol. 102 (1): 40–59. doi:10.1016/j.jecp.2008.08.002. PMC   2612128 . PMID   18829049.
  30. ↑ Kyle, FE; Campbell, R; Mohammed, T; Coleman, M; MacSweeney, M (2013). "Speechreading development in deaf and hearing children: introducing the test of child speechreading". Journal of Speech, Language, and Hearing Research. 56 (2): 416–26. doi:10.1044/1092-4388(2012/12-0039). PMC   4920223 . PMID   23275416.
  31. ↑ Tye-Murray, N; Hale, S; Spehar, B; Myerson, J; Sommers, MS (2014). "Lipreading in school-age children: the roles of age, hearing status, and cognitive ability". J Speech Lang Hear Res. 57 (2): 556–65. doi:10.1044/2013_JSLHR-H-12-0273. PMC   5736322 . PMID   24129010.
  32. ↑ Peelle, JE; Sommers, MS (2015). "Prediction and constraint in audiovisual speech perception". Cortex. 68: 169–81. doi:10.1016/j.cortex.2015.03.006. PMC   4475441 . PMID   25890390.
  33. ↑ Campbell, R (2008). "The processing of audio-visual speech: empirical and neural bases". Philosophical Transactions of the Royal Society B. 363 (1493): 1001–1010. doi:10.1098/rstb.2007.2155. PMC   2606792 . PMID   17827105.
  34. ↑ Sumby, WH; Pollack, I (1954). "Visual contribution to speech intelligibility in noise". Journal of the Acoustical Society of America. 26 (2): 212–215. Bibcode:1954ASAJ...26..212S. doi:10.1121/1.1907309. hdl: 11858/00-001M-0000-002C-6664-3 .
  35. ↑ Taljaard, Schmulian; et al. (2015). "The relationship between hearing impairment and cognitive function: A meta-analysis in adults" (PDF). Clin Otolaryngol. 41 (6): 718–729. doi:10.1111/coa.12607. hdl: 2263/60768 . PMID   26670203. S2CID   5327755.
  36. ↑ Hung, SC; et al. (2015). "Hearing Loss is Associated With Risk of Alzheimer's Disease: A Case-Control Study in Older People". J Epidemiol. 25 (8): 517–21. doi:10.2188/jea.JE20140147. PMC   4517989 . PMID   25986155.
  37. ↑ Smith, EG; Bennetto, L.J (2007). "Audiovisual speech integration and lipreading in autism". Journal of Child Psychology and Psychiatry. 48 (8): 813–21. doi:10.1111/j.1469-7610.2007.01766.x. PMID   17683453.
  38. ↑ Irwin, JR; Tornatore, LA; Brancazio, L; Whalen, DH (2011). "Can children with autism spectrum disorders "hear" a speaking face?". Child Dev. 82 (5): 1397–403. doi:10.1111/j.1467-8624.2011.01619.x. PMC   3169706 . PMID   21790542.
  39. ↑ Irwin, JR; Brancazio, L (2014). "Seeing to hear? Patterns of gaze to speaking faces in children with autism spectrum disorders". Front Psychol. 5: 397. doi: 10.3389/fpsyg.2014.00397 . PMC   4021198 . PMID   24847297.
  40. ↑ Böhning, M; Campbell, R; Karmiloff-Smith, A (2002). "Audiovisual speech perception in Williams syndrome". Neuropsychologia. 40 (8): 1396–406. doi:10.1016/s0028-3932(01)00208-1. PMID   11931944. S2CID   9125298.
  41. ↑ Leybaert, J; Macchi, L; Huyse, A; Champoux, F; Bayard, C; Colin, C; Berthommier, F (2014). "Atypical audio-visual speech perception and McGurk effects in children with specific language impairment". Front Psychol. 5: 422. doi: 10.3389/fpsyg.2014.00422 . PMC   4033223 . PMID   24904454.
  42. ↑ Mohammed T1, Campbell R; Macsweeney, M; Barry, F; Coleman, M (2006). "Speechreading and its association with reading among deaf, hearing and dyslexic individuals". Clinical Linguistics and Phonetics. 20 (7–8): 621–30. doi:10.1080/02699200500266745. PMID   17056494. S2CID   34877573.{{cite journal}}: CS1 maint: numeric names: authors list (link)
  43. ↑ "Hands & Voices :: Articles".
  44. ↑ Swanwick, R (2016). "Deaf Children's bimodal bilingualism and education" (PDF). Language Teaching. 49 (1): 1–34. doi:10.1017/S0261444815000348. S2CID   146626144.
  45. ↑ Bernstein, LE; Demorest, ME; Tucker, PE (2000). "Speech perception without hearing". Perception & Psychophysics. 62 (2): 233–52. doi: 10.3758/bf03205546 . PMID   10723205.
  46. ↑ Bergeson TR1, Pisoni DB; Davis, RA (2005). "Development of audiovisual comprehension skills in prelingually deaf children with cochlear implants". Ear & Hearing. 26 (2): 149–64. doi:10.1097/00003446-200504000-00004. PMC   3432935 . PMID   15809542.{{cite journal}}: CS1 maint: numeric names: authors list (link)
  47. ↑ Pimperton, Hannah; Ralph-Lewis, Amelia; MacSweeney, Mairead (2017). "Speechreading in Deaf Adults with Cochlear Implants:Evidence for Perceptual Compensation". Front. Psychol. 8: 106. doi: 10.3389/fpsyg.2017.00106 . PMC   5294775 . PMID   28223951.
  48. ↑ "Communication support for deaf people". 2015-11-24.
  49. ↑ "Lipspeaker UK - Communication services for deaf & hard of hearing people".
  50. ↑ "Reading and dyslexia in deaf children | Nuffield Foundation". 19 November 2019.
  51. ↑ Mayer, C. (2007). "What really matters in the early literacy development of deaf children". J Deaf Stud Deaf Educ. 12 (4): 411–31. doi: 10.1093/deafed/enm020 . PMID   17566067.
  52. ↑ Mohammed, Tara; Campbell, Ruth; MacSweeney, Mairéad; Barry, Fiona; Coleman, Michael (2006). "Speechreading and its association with reading among deaf, hearing and dyslexic individuals". Clinical Linguistics & Phonetics. 20 (7–8): 621–630. doi:10.1080/02699200500266745. PMID   17056494. S2CID   34877573.
  53. ↑ Kyle, F. E.; Harris, M. (2010). "Predictors of reading development in deaf children: a 3-year longitudinal study" (PDF). J Exp Child Psychol. 107 (3): 229–243. doi:10.1016/j.jecp.2010.04.011. PMID   20570282.
  54. ↑ Kyle, Fiona E.; Campbell, Ruth; Mohammed, Tara; Coleman, Mike; MacSweeney, Mairéad (2013). "Speechreading Development in Deaf and Hearing Children: Introducing the Test of Child Speechreading". Journal of Speech, Language, and Hearing Research. 56 (2): 416–426. doi:10.1044/1092-4388(2012/12-0039). PMC   4920223 . PMID   23275416.
  55. ↑ Nicholls, GH; Ling, D (1982). "Cued Speech and the reception of spoken language". J Speech Hear Res. 25 (2): 262–9. doi:10.1044/jshr.2502.262. PMID   7120965.
  56. ↑ Leybaert, J; LaSasso, CJ (2010). "Cued speech for enhancing speech perception and first language development of children with cochlear implants". Trends in Amplification. 14 (2): 96–112. doi:10.1177/1084713810375567. PMC   4111351 . PMID   20724357.
  57. ↑ "Lipreading Alphabet: Round Vowels". Archived from the original on 2014-06-23. Retrieved 2014-06-23.
  58. ↑ Bernstein, LE; Jordan, N; Auer, ET; Eberhardt, SP (2022). "Lipreading: A review of its Continuing Importance for Speech Recognition with an Acquired Hearing Loss and Possibilities for Effective Training". American Journal of Audiology. 31 (2): 453–469. doi:10.1044/2021_AJA-21-00112. PMC   9524756 . PMID   35316072.
  59. ↑ "Campaigns and influencing".
  60. ↑ Altieri, NA; Pisoni, DB; Townsend, JT (2011). "Some normative data on lipreading skills". Journal of the Acoustical Society of America. 130 (1): 1–4. doi:10.1121/1.3593376. PMC   3155585 . PMID   21786870.
  61. ↑ "Home > Rachel-Walker > USC Dana and David Dornsife College of Letters, Arts and Sciences" (PDF).
  62. ↑ "Rule-Based Visual Speech Synthesis". 1995. pp. 299–302.
  63. ↑ Bosseler, Alexis; Massaro, Dominic W. (2003). "Development and Evaluation of a Computer-Animated Tutor for Vocabulary and Language Learning in Children with Autism". Journal of Autism and Developmental Disorders. 33 (6): 653–672. doi:10.1023/B:JADD.0000006002.82367.4f. PMID   14714934. S2CID   17406145.
  64. ↑ "Visual Speech Synthesis - UEA".
  65. ↑ "Lip-reading computer can distinguish languages".
  66. ↑ Archived at Ghostarchive and the Wayback Machine : "Video to Text: Lip reading and word spotting". YouTube .
  67. ↑ Hickey, Shane (2016-04-24). "The innovators: Can computers be taught to lip-read?". The Guardian.
  68. ↑ "Google's DeepMind AI can lip-read TV shows better than a pro".
  69. ↑ Petajan, E.; Bischoff, B.; Bodoff, D.; Brooke, N. M. (1988). "An improved automatic lipreading system to enhance speech recognition". Proceedings of the SIGCHI conference on Human factors in computing systems - CHI '88. pp. 19–25. doi:10.1145/57167.57170. ISBN   978-0201142372. S2CID   15211759.
  70. ↑ Luettin, Juergen; Thacker, Neil A.; Beet, Steve W. "Speaker Identification by Lipreading" (PDF). Applied Science and Engineering Laboratories.
  71. ↑ http://www.planetbiometrics.com-article-details-i-2250%5B%5D
  72. ↑ Calvert, GA; Bullmore, ET; Brammer, MJ; et al. (1997). "Activation of auditory cortex during silent lipreading". Science. 276 (5312): 593–6. doi:10.1126/science.276.5312.593. PMID   9110978.
  73. ↑ Bernstein, LE; Liebenthal, E (2014). "Neural pathways for visual speech perception". Front Neurosci. 8: 386. doi: 10.3389/fnins.2014.00386 . PMC   4248808 . PMID   25520611.
  74. ↑ Skipper, JI; van Wassenhove, V; Nusbaum, HC; Small, SL (2007). "Hearing Lips and Seeing Voices: How Cortical Areas Supporting Speech Production Mediate Audiovisual Speech Perception". Cerebral Cortex. 17 (10): 2387–2399. doi:10.1093/cercor/bhl147. PMC   2896890 . PMID   17218482.
  75. ↑ Campbell, R; MacSweeney, M; Surguladze, S; Calvert, G; McGuire, P; Suckling, J; Brammer, MJ; David, AS (2001). "Cortical substrates for the perception of face actions: an fMRI study of the specificity of activation for seen speech and for meaningless lower-face acts (gurning)". Brain Res Cogn Brain Res. 12 (2): 233–43. doi:10.1016/s0926-6410(01)00054-4. PMID   11587893.
  76. ↑ Swaminathan, S.; MacSweeney, M.; Boyles, R.; Waters, D.; Watkins, K. E.; Möttönen, R. (2013). "Motor excitability during visual perception of known and unknown spoken languages". Brain and Language. 126 (1): 1–7. doi:10.1016/j.bandl.2013.03.002. PMC   3682190 . PMID   23644583.
  77. ↑ Sams, M; et al. (1991). "Aulenko et al. 1991 Seeing Speech: visual information from lip movements modifies activity in the human auditory cortex". Neuroscience Letters. 127 (1): 141–145. doi:10.1016/0304-3940(91)90914-f. PMID   1881611. S2CID   9035639.
  78. ↑ Van Wassenhove, V; Grant, KW; Poeppel, D (Jan 2005). "Visual speech speeds up the neural processing of auditory speech". Proceedings of the National Academy of Sciences. 102 (4): 1181–6. Bibcode:2005PNAS..102.1181V. doi: 10.1073/pnas.0408949102 . PMC   545853 . PMID   15647358.
  79. ↑ Hall DA1, Fussell C, Summerfield AQ. 2005 Reading fluent speech from talking faces: typical brain networks and individual differences.J. Cogn Neurosci. 17(6):939-53.
  80. ↑ Bernstein, LE; Jiang, J; Pantazis, D; Lu, ZL; Joshi, A (2011). "Visual phonetic processing localized using speech and nonspeech face gestures in video and point-light displays". Hum Brain Mapp. 32 (10): 1660–76. doi:10.1002/hbm.21139. PMC   3120928 . PMID   20853377.
  81. ↑ Capek, CM; Macsweeney, M; Woll, B; Waters, D; McGuire, PK; David, AS; Brammer, MJ; Campbell, R (2008). "Cortical circuits for silent speechreading in deaf and hearing people". Neuropsychologia. 46 (5): 1233–41. doi:10.1016/j.neuropsychologia.2007.11.026. PMC   2394569 . PMID   18249420.

Bibliography

Further reading

See also

This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.