Visual capture

Last updated
Vision capture aids in the illusion that a dummy is talking in ventriloquism. To the right of it is Terry Fator. Comedian Terry Fator on stage.jpg
Vision capture aids in the illusion that a dummy is talking in ventriloquism. To the right of it is Terry Fator.

In psychology, visual capture is the dominance of vision over other sense modalities in creating a percept. [1] In this process, the visual senses influence the other parts of the somatosensory system, to result in a perceived environment that is not congruent with the actual stimuli. Through this phenomenon, the visual system is able to disregard what other information a different sensory system is conveying, and provide a logical explanation for whatever output the environment provides. Visual capture allows one to interpret the location of sound as well as the sensation of touch without actually relying on those stimuli but rather creating an output that allows the individual to perceive a coherent environment.

Contents

One example of visual capture is known as the ventriloquism effect which refers to the perception of speech sounds as coming from a direction other than their true direction, due to the influence of visual stimuli from an apparent speaker. [2] Thus, when the ventriloquism illusion occurs, the speaker's voice is visually captured at the location of the dummy's moving mouth (rather than the speaker's carefully unmoving mouth). [3]

Another example of visual capture occurs when a sound that would normally be perceived as moving from left to right is heard while a person is viewing a visual stimulus that is moving from right to left; in this case, both sound and stimulus appear to be moving from right to left. [4]

Defining criteria

Theory

When two sensory stimuli are presented simultaneously, vision is capable of dominating and capturing the other. This occurs as visual cues can distract from other sensations, causing the origin of the stimulus to appear as if it is being produced by the visual cue. Therefore, when an individual is in an environment, and multiple stimuli reach the brain at once, there is a hierarchy that vision will guide the rest of the somatosensory cues to be perceived as though they align with the visual experience, despite where their original source may be. Research has found that the visual and auditory reflexive spatial orienting are controlled through a common underlying neural substrate. [5] Furthermore, studies have shown that vision has an effect in cognitive neuroscience, and provides for a significant effect when visually attended to. This dominance is seen again through a visual-haptic task that vision is capable of making better judgements of an object that physically touching it. [6] It has also been determined, that there are certain amounts of visual capture that occur depending on the task, sometimes allowing the visual system to be entirely dominant, while others provide haptic cues to be prominent. [7]

Brain regions

The thalamus is a section of the brain responsible for relaying sensory and motor signals to the cerebral cortex. As stimuli pass through the thalamus, there are specific regions dedicated to each sense, and therefore is able to sort out the multiple parts of an environment an individual experiences in a given moment. Two of these regions are specific to vision and hearing respectively, which may be responsible for the order in which sensory information is coded and then perceived within the cerebral cortex.[ citation needed ]

The retina at the back of the eye is what perceives stimuli, allowing them to travel through the occipital tract to the lateral geniculate nucleus (LGN) within the thalamus. The data is then transmitted to the occipital lobe where the orientation and other recognizable factors are processed.[ citation needed ]

The LGN is located near the medial geniculate nucleus (MGN) which is responsible for organizing auditory stimuli after one hears a specific sound. Because these two systems are closely located to each other, research has shown that this might be where vision is responsible for taking over the perception of an environment and resulting in visual capture. As the multiple senses are organized and the response is sent further into the brain for processing, it is possible that the visual cues were recorded stronger, and therefore everything is perceived in a way that all other senses are a function of this visual cue, resulting in a cohesive experience for the individual, driven by the visual system, therefore fitting the definition of visual capture.[ citation needed ]

Origin of research

This phenomenon was first demonstrated by Frenchman J. Tastevin in 1937, after studying the tactile Aristotle illusion in 1937. This illusion produces the sensation of touching two objects by crossing one's fingers and then holding a spherical object between them. Visual capture was used to explain how vision could overcome this effect and determine what is actually going on. [8] [9]

Attention was again tied to visual cues during an experiment conducted by Michael Posner in 1980. [10] By indicating visually in which direction a stimulus will appear, response time will improve (decrease) if the correct direction is attended to. (Conversely, if the indicator is misleading, response time increases.) This ability to attend to a specific direction allows for a faster reaction time, despite the participant not physically shifting their visual focus during the pre-stimulus indicator.

The evidence that vision has an impact on reaction time demonstrates that vision has a neurological effect on the attentional process[ citation needed ]. Thus, it is clear that vision is capable of manipulating the perception an individual has of an environment —— this perceptual manipulation is what Tastevin considered visual capture.[ citation needed ]

Examples

Examples in research

A number of studies have demonstrated the visual capture effect. For example, Alais and Burr (2004) using the ventriloquism effect, found that vision is capable of taking over auditory senses, specifically with well-localized visual stimuli. [2] This means that when the stimuli producing the sound as well as vision are close together, there seems to be a direct relationship formed in the perception of these separate stimuli, that correlate them into the same sensation. [11]

Another example of visual capture comes from Ehrsson, Spense, & Passingham (2004) who used a rubber hand to prove that vision is capable of determining how other senses react. As participants watched a rubber hand be stroked, their hand was also stroked in a similar fashion, allowing the individual to attribute their own sensation to what they were watching rather than what was happening to their own body. Therefore, when the rubber hand was then manipulated, for example hitting it with a hammer, the participant feels an immediate shock and pain as they fear that it is their own hand that is in danger. This serves as evidence that the visual system is capable of not only manipulating where an individual perceives another sense to be coming from, but can actually manipulate how one reacts to an experience as well, given that it is vision that is taking charge. [12] [13]

A study by Remington, Johnston, & Yantis (1992) found that attention is involuntarily drawn away from a given task when a visual stimulus interferes. In this study, participants were presented with four boxes; they were told that an image would precede a letter that they were to memorize. The conditions were either to attend to the same box, a different one, all four, or to focus on the center. However, even though they were told to not attend to a certain box, the participant was consistently drawn to the image before the letter in all cases, resulting in a longer response time in all conditions except for the same. The results prove that there is a consistent need for vision to dominate the other senses, and attention is immediately drawn away by it in a controlled setting. [14]

The research in visual capture does not all work in the favor of vision being constantly dominant, as Shams, Kamitani, & Shimojo in 2000 found that visual illusion can be induced by sound in a controlled environment. When a flash of light is accompanied by a series of auditory beeps, the results show that the participant views the flash to be a series of flashes corresponding with the beeps. Because in this experiment hearing seemed to be the dominant sense, it is clear that there is still plenty to be determined about visual capture, although this like the other studies, proves that there is a connection between these two senses when it comes to integrating the perception of an environment. [15]

Everyday examples

An example of visual capture experienced in daily life is the ventriloquism effect. [2] This is when ventriloquists make their speech appear to be coming from their puppet rather than their own mouths. In this situation, visual capture allows the audio stimuli to be controlled by the vision system and produce a congruent experience that the sound is coming from the puppet. Another popular example of visual capture happens while watching a movie in a theater, and the sound appears to be coming from the actors lips. Although this may seem true, the sound is actually coming from the speakers, often spread out across the theater rather than directly behind wherever the character's mouth may be.

There is also a phenomenon known that while crossing a street, an individual can hear the sound of an oncoming car. However, when they look to the left the next car is a few blocks away so it is safe to cross. But when they look to the right, there is a car that is passing them that they did not even notice before. This occurs because the individual attributes the sound of oncoming traffic to the first car because they were unaware of the other, closer car. This therefore is an example of visual capture reassigning the audio cue to the incorrect visual cue, resulting in a mistake that could be far more costly than expected.[ citation needed ]

Applications

Mirror box technique

A phantom limb is the sensation that an amputated limb is still attached. This can cause pain and distress amongst many amputees, and was thought to be incurable. However, in 1998, Vilayanur S. Ramachandran created a mirror box, which allows for an amputee to place their intact limb on one side of the box, and observe their amputated limb by looking at the mirror image of their actual limb. [16] Through visual capture, the visual system is able to override the somatosensory system and send feedback to the brain that the arm is actually okay and not in any specific pain. This has resulted in numerous solutions to problems that individuals with phantom limb pain were having as they could now train their brain via visual capture that the limb was not actually cramped in the position it was when amputated, but rather free to move around and act as a normal limb.

McGurk effect

The McGurk effect is a phenomenon that occurs when the reception of an auditory stimulus is determined by the visual system. For example, when the syllable “ba” is repeated over and over, and one sees an individual saying this, then the individual is perceived to be saying “ba”. However, when the same audio is played over a person saying the word “fa”, the fact that the utterance is completely forgotten, and the person will hear the word “fa”. This is once again because vision is able to dominate the auditory system and produce a response that is guided strictly by vision. Because the auditory system is silenced, visual capture is evident and the visual system is able to reorganize the environmental stimuli to produce a cohesive explanation for what would make the most sense by combining the different stimuli. [17]

Implications

Understanding visual capture has the potential to lead to numerous benefits in the future. Beyond solving people's pain in phantom limb syndrome, there are numerous potential applications for visual capture. Already, there have been surround-sound systems built to provide unique listening experiences, that “put you right in the middle of the action”. However, it is more than just having sound come from every direction, but the improvements in visual quality of movies, and where sound and vision can be localized best to provide a coherent movie-going experience.[ citation needed ]

See also

Related Research Articles

<span class="mw-page-title-main">Perception</span> Interpretation of sensory information

Perception is the organization, identification, and interpretation of sensory information in order to represent and understand the presented information or environment. All perception involves signals that go through the nervous system, which in turn result from physical or chemical stimulation of the sensory system. Vision involves light striking the retina of the eye; smell is mediated by odor molecules; and hearing involves pressure waves.

An illusion is a distortion of the senses, which can reveal how the mind normally organizes and interprets sensory stimulation. Although illusions distort the human perception of reality, they are generally shared by most people.

<span class="mw-page-title-main">Optical illusion</span> Visually perceived images that differ from objective reality

In visual perception, an optical illusion is an illusion caused by the visual system and characterized by a visual percept that arguably appears to differ from reality. Illusions come in a wide variety; their categorization is difficult because the underlying cause is often not clear but a classification proposed by Richard Gregory is useful as an orientation. According to that, there are three main classes: physical, physiological, and cognitive illusions, and in each class there are four kinds: Ambiguities, distortions, paradoxes, and fictions. A classical example for a physical distortion would be the apparent bending of a stick half immerged in water; an example for a physiological paradox is the motion aftereffect. An example for a physiological fiction is an afterimage. Three typical cognitive distortions are the Ponzo, Poggendorff, and Müller-Lyer illusion. Physical illusions are caused by the physical environment, e.g. by the optical properties of water. Physiological illusions arise in the eye or the visual pathway, e.g. from the effects of excessive stimulation of a specific receptor type. Cognitive visual illusions are the result of unconscious inferences and are perhaps those most widely known.

A tactile illusion is an illusion that affects the sense of touch. Some tactile illusions require active touch, whereas others can be evoked passively. In recent years, a growing interest among perceptual researchers has led to the discovery of new tactile illusions and to the celebration of tactile illusions in the popular science press. Some tactile illusions are analogous to visual and auditory illusions, suggesting that these sensory systems may process information in similar ways; other tactile illusions don't have obvious visual or auditory analogs.

Auditory illusions are false perceptions of a real sound or outside stimulus. These false perceptions are the equivalent of an optical illusion: the listener hears either sounds which are not present in the stimulus, or sounds that should not be possible given the circumstance on how they were created.

<span class="mw-page-title-main">McGurk effect</span> Perceptual illusion

The McGurk effect is a perceptual phenomenon that demonstrates an interaction between hearing and vision in speech perception. The illusion occurs when the auditory component of one sound is paired with the visual component of another sound, leading to the perception of a third sound. The visual information a person gets from seeing a person speak changes the way they hear the sound. If a person is getting poor-quality auditory information but good-quality visual information, they may be more likely to experience the McGurk effect.

Stimulus modality, also called sensory modality, is one aspect of a stimulus or what is perceived after a stimulus. For example, the temperature modality is registered after heat or cold stimulate a receptor. Some sensory modalities include: light, sound, temperature, taste, pressure, and smell. The type and location of the sensory receptor activated by the stimulus plays the primary role in coding the sensation. All sensory modalities work together to heighten stimuli sensation when necessary.

Multisensory integration, also known as multimodal integration, is the study of how information from the different sensory modalities may be integrated by the nervous system. A coherent representation of objects combining modalities enables animals to have meaningful perceptual experiences. Indeed, multisensory integration is central to adaptive behavior because it allows animals to perceive a world of coherent perceptual entities. Multisensory integration also deals with how different sensory modalities interact with one another and alter each other's processing.

Sensory substitution is a change of the characteristics of one sensory modality into stimuli of another sensory modality.

Sensory processing is the process that organizes and distinguishes sensation from one's own body and the environment, thus making it possible to use the body effectively within the environment. Specifically, it deals with how the brain processes multiple sensory modality inputs, such as proprioception, vision, auditory system, tactile, olfactory, vestibular system, interoception, and taste into usable functional outputs.

The tau effect is a spatial perceptual illusion that arises when observers judge the distance between consecutive stimuli in a stimulus sequence. When the distance from one stimulus to the next is constant, and the time elapsed from one stimulus to the next is also constant, subjects tend to judge the distances, correctly, as equal. However, if the distance from one stimulus to the next is constant, but the time elapsed from one stimulus to the next is not constant, then subjects tend to misperceive the interval that has the shorter temporal interval as also having a shorter spatial interval. Thus, the tau effect reveals that stimulus timing affects the perception of stimulus spacing. Time is also a perceived quantity and subject to its own illusions; research indicates that in the tau effect, perceived stimulus spacing follows perceived (phenomenal) time rather than actual (physical) time.

In perceptual psychology, a sensory cue is a statistic or signal that can be extracted from the sensory input by a perceiver, that indicates the state of some property of the world that the perceiver is interested in perceiving.

The cutaneous rabbit illusion is a tactile illusion evoked by tapping two or more separate regions of the skin in rapid succession. The illusion is most readily evoked on regions of the body surface that have relatively poor spatial acuity, such as the forearm. A rapid sequence of taps delivered first near the wrist and then near the elbow creates the sensation of sequential taps hopping up the arm from the wrist towards the elbow, although no physical stimulus was applied between the two actual stimulus locations. Similarly, stimuli delivered first near the elbow then near the wrist evoke the illusory perception of taps hopping from elbow towards wrist. The illusion was discovered by Frank Geldard and Carl Sherrick of Princeton University, in the early 1970s, and further characterized by Geldard (1982) and in many subsequent studies. Geldard and Sherrick likened the perception to that of a rabbit hopping along the skin, giving the phenomenon its name. While the rabbit illusion has been most extensively studied in the tactile domain, analogous sensory saltation illusions have been observed in audition and vision. The word "saltation" refers to the leaping or jumping nature of the percept.

Extinction is a neurological disorder that impairs the ability to perceive multiple stimuli of the same type simultaneously. Extinction is usually caused by damage resulting in lesions on one side of the brain. Those who are affected by extinction have a lack of awareness in the contralesional side of space and a loss of exploratory search and other actions normally directed toward that side.

A sense is a biological system used by an organism for sensation, the process of gathering information about the surroundings through the detection of stimuli. Although, in some cultures, five human senses were traditionally identified as such, many more are now recognized. Senses used by non-human organisms are even greater in variety and number. During sensation, sense organs collect various stimuli for transduction, meaning transformation into a form that can be understood by the brain. Sensation and perception are fundamental to nearly every aspect of an organism's cognition, behavior and thought.

Chronostasis is a type of temporal illusion in which the first impression following the introduction of a new event or task-demand to the brain can appear to be extended in time. For example, chronostasis temporarily occurs when fixating on a target stimulus, immediately following a saccade. This elicits an overestimation in the temporal duration for which that target stimulus was perceived. This effect can extend apparent durations by up to half a second and is consistent with the idea that the visual system models events prior to perception.

Pre-attentive processing is the subconscious accumulation of information from the environment. All available information is pre-attentively processed. Then, the brain filters and processes what is important. Information that has the highest salience or relevance to what a person is thinking about is selected for further and more complete analysis by conscious (attentive) processing. Understanding how pre-attentive processing works is useful in advertising, in education, and for prediction of cognitive ability.

The Colavita visual dominance effect refers to the phenomenon in which study participants respond more often to the visual component of an audiovisual stimulus, when presented with bimodal stimuli.

During every moment of an organism's life, sensory information is being taken in by sensory receptors and processed by the nervous system. Sensory information is stored in sensory memory just long enough to be transferred to short-term memory. Humans have five traditional senses: sight, hearing, taste, smell, touch. Sensory memory (SM) allows individuals to retain impressions of sensory information after the original stimulus has ceased. A common demonstration of SM is a child's ability to write letters and make circles by twirling a sparkler at night. When the sparkler is spun fast enough, it appears to leave a trail which forms a continuous image. This "light trail" is the image that is represented in the visual sensory store known as iconic memory. The other two types of SM that have been most extensively studied are echoic memory, and haptic memory; however, it is reasonable to assume that each physiological sense has a corresponding memory store. For example, children have been shown to remember specific "sweet" tastes during incidental learning trials but the nature of this gustatory store is still unclear. However, sensory memories might be related to a region of the thalamus, which serves as a source of signals encoding past experiences in the neocortex.

Interindividual differences in perception describes the effect that differences in brain structure or factors such as culture, upbringing and environment have on the perception of humans. Interindividual variability is usually regarded as a source of noise for research. However, in recent years, it has become an interesting source to study sensory mechanisms and understand human behavior. With the help of modern neuroimaging methods such as fMRI and EEG, individual differences in perception could be related to the underlying brain mechanisms. This has helped to explain differences in behavior and cognition across the population. Common methods include studying the perception of illusions, as they can effectively demonstrate how different aspects such as culture, genetics and the environment can influence human behavior.

References

  1. Grünwald, Martin. Human haptic perception. Birkhauser, 2008. 657. Print.
  2. 1 2 3 "APA Dictionary of Psychology". dictionary.apa.org. Retrieved 16 January 2022.
  3. Wright, Richard, and Lawrence Ward. Orienting of attention. Oxford University Press, USA, 2008. 215. Print. Retrieved Aug. 28, 2010, from .
  4. Soto-Faraco et al. "When does visual perceptual grouping affect multisensory integration?." (2004): n. pag. Web. 28 Aug 2010. <http://cabn.psychonomic-journals.org/content/4/2/218.full.pdf>.
  5. Santangelo, V., Belardinelli, M.O. (2007). The suppression of reflexive visual and auditory orienting when attention is otherwise engaged. Journal of Experimental Psychology-Human Perception and Performance, 33, 137-148.
  6. Ernst, M.O., Banks, M.S., (2002). Humans integrate visual and haptic information in a statistically optimal fashion. Nature, 415, 429-433.
  7. Fishkin, S.M., Pishkin, V., Stahl, M.L. (1975). Factors involved in visual capture. Perceptual and Motor Skills, 40, 427-434.
  8. J. Tastevin (Feb 1937). "En partant de l'expérience d'Aristote: Les déplacements artificiels des parties du corps ne sont pas suivis par le sentiment de ces parties ni par les sensations qu'on peut y produire" [Starting from Aristotle’s experiment: The artificial displacements of parts of the body are not followed by feeling in these parts or by the sensations which can be produced there]. L'Encephale  [ fr ] (in French). 32 (2): 57–84.  (English abstract)
  9. J. Tastevin (Mar 1937). "En partant de l'expérience d'Aristote". L'Encephale (in French). 32 (3): 140–158.
  10. Posner, M. I. (1980). "Orienting of attention". The Quarterly journal of experimental psychology 32 (1): 3–25. doi:10.1080/00335558008248231
  11. Alais, D., Burr, D. (2004). The ventriloquist effect results from near-optimal bimodal integration. Current Biology, 14, 257-262.
  12. Ehrsson, H.H., Spence, C., Passingham, R.E., (2004). That's my hand! Activity in premotor cortex reflects feeling of ownership of a limb. Science, 305, 875-877.
  13. Pavani, F., Spence, C., Driver, J. (2000). Visual capture of touch: out-of-the-body experiences with rubber gloves. Psychological Sciences, 11, 353-359.
  14. Remington, R.W., Johnston, J.C, Yantis, S. (1992). Involuntary attentional capture by abrupt onsets. Perception & Psychophysics, 51, 279-290.
  15. Shams, L., Kamitani, Y., Shimojo, S. (2000). Visual illusion induced by sound. Cognitive Brain Research, 14, 147-152.
  16. Ramachandran, V.S., Rodgers-Ramachandran, D., (1996). Synaesthesia in phantom limbs induced with mirrors. Proceedings of the Royal Society B-Biological Sciences, 263, 377-386
  17. de Gelder, B., Vroomen, J., (2000). The perception of emotions by ear and by eye. Cognition & Emotion, 14, 289-311.