Crossmodal

Last updated

Crossmodal perception or cross-modal perception is perception that involves interactions between two or more different sensory modalities. [1] Examples include synesthesia, sensory substitution and the McGurk effect, in which vision and hearing interact in speech perception.

Perception organization, identification, and interpretation of sensory information in order to represent and understand the environment

Perception is the organization, identification, and interpretation of sensory information in order to represent and understand the presented information, or the environment.

Synesthesia neurological condition

Synesthesia is a perceptual phenomenon in which stimulation of one sensory or cognitive pathway leads to automatic, involuntary experiences in a second sensory or cognitive pathway. People who report a lifelong history of such experiences are known as synesthetes.

Sensory substitution is a change of the characteristics of one sensory modality into stimuli of another sensory modality.

Contents

Crossmodal perception, crossmodal integration and cross modal plasticity of the human brain are increasingly studied in neuroscience to gain a better understanding of the large-scale and long-term properties of the brain. [2] A related research theme is the study of multisensory perception and multisensory integration.

Cross modal plasticity

Cross modal plasticity is the adaptive reorganization of neurons to integrate the function of two or more sensory systems. Cross modal plasticity is a type of neuroplasticity and often occurs after sensory deprivation due to disease or brain damage. The reorganization of the neural network is greatest following long-term sensory deprivation, such as congenital blindness or pre-lingual deafness. In these instances, cross modal plasticity can strengthen other sensory systems to compensate for the lack of vision or hearing. This strengthening is due to new connections that are formed to brain cortices that no longer receive sensory input.

Brain organ that serves as the center of the nervous system in all vertebrate and most invertebrate animals

The brain is an organ that serves as the center of the nervous system in all vertebrate and most invertebrate animals. The brain is located in the head, usually close to the sensory organs for senses such as vision. The brain is the most complex organ in a vertebrate's body. In a human, the cerebral cortex contains approximately 14–16 billion neurons, and the estimated number of neurons in the cerebellum is 55–70 billion. Each neuron is connected by synapses to several thousand other neurons. These neurons communicate with one another by means of long protoplasmic fibers called axons, which carry trains of signal pulses called action potentials to distant parts of the brain or body targeting specific recipient cells.

Neuroscience scientific study of the nervous system

Neuroscience is the scientific study of the nervous system. It is a multidisciplinary branch of biology that combines physiology, anatomy, molecular biology, developmental biology, cytology, mathematical modeling and psychology to understand the fundamental and emergent properties of neurons and neural circuits. The understanding of the biological basis of learning, memory, behavior, perception, and consciousness has been described by Eric Kandel as the "ultimate challenge" of the biological sciences.

As a cultural movement

Described as synthesizing art, science and entrepreneurship. Crossmodialism as a movement started in London in 2013. [3] [4] The movement focuses on bringing together the talents of traditionally distinct disciplines to make cohesive works. Crossmodalism has been compared to the Dadaist art movement of the 20th century, as well as other avant garde styles like futurism and surrealism. [5]

Futurism artistic and social movement

Futurism was an artistic and social movement that originated in Italy in the early 20th century. It emphasised speed, technology, youth, violence, and objects such as the car, the airplane, and the industrial city. Its key figures were the Italians Filippo Tommaso Marinetti, Umberto Boccioni, Carlo Carrà, Gino Severini, Giacomo Balla, and Luigi Russolo. It glorified modernity and aimed to liberate Italy from the weight of its past. Cubism contributed to the formation of Italian Futurism's artistic style. Important Futurist works included Marinetti's Manifesto of Futurism, Boccioni's sculpture Unique Forms of Continuity in Space, Balla's painting Abstract Speed + Sound, and Russolo's The Art of Noises.

Surrealism international cultural movement that began in the early 1920s

Surrealism is a cultural movement that began in the early 1920s, and is best known for its visual artworks and writings. Artists painted unnerving, illogical scenes with photographic precision, created strange creatures from everyday objects, and developed painting techniques that allowed the unconscious to express itself. Its aim was to "resolve the previously contradictory conditions of dream and reality into an absolute reality, a super-reality".

See also

Crossmodal attention refers to the distribution of attention to different senses. Attention is the cognitive process of selectively emphasizing and ignoring sensory stimuli. According to the crossmodal attention perspective, attention often occurs simultaneously through multiple sensory modalities. These modalities process information from the different sensory fields, such as: visual, auditory, spatial, and tactile. While each of these is designed to process a specific type of sensory information, there is considerable overlap between them which has led researchers to question whether attention is modality-specific or the result of shared "cross-modal" resources. Cross-modal attention is considered to be the overlap between modalities that can both enhance and limit attentional processing. The most common example given of crossmodal attention is the Cocktail Party Effect, which is when a person is able to focus and attend to one important stimulus instead of other less important stimuli. This phenomenon allows deeper levels of processing to occur for one stimulus while others are then ignored.

Ideasthesia Idea in psychology

Ideasthesia is defined as a phenomenon in which activations of concepts (inducers) evoke perception-like experiences (concurrents). The name comes from the Ancient Greek ἰδέα (idéa) and αἴσθησις (aísthēsis), meaning "sensing concepts" or "sensing ideas". The main reason for introducing the notion of ideasthesia was the problems with synesthesia. While "synesthesia" means "union of senses", empirical evidence indicated that this was an incorrect explanation of a set of phenomena traditionally covered by this heading. Syn-aesthesis denoting also "co-perceiving", implies the association of two sensory elements with little connection to the cognitive level. However, according to others, most phenomena that have inadvertently been linked to synesthesia in fact are induced by the semantic representations. That is, the meaning of the stimulus is what is important rather than its sensory properties, as would be implied by the term synesthesia. In other words, while synesthesia presumes that both the trigger (inducer) and the resulting experience (concurrent) are of sensory nature, ideasthesia presumes that only the resulting experience is of sensory nature while the trigger is semantic. Meanwhile, the concept of ideasthesia developed into a theory of how we perceive and the research has extended to topics other than synesthesia — as the concept of ideasthesia turned out applicable to our everyday perception. Ideasthesia has been even applied to the theory of art. Research on ideasthesia bears important implications for solving the mystery of human conscious experience, which according to ideasthesia, is grounded in how we activate concepts.

Molyneuxs problem

Molyneux's problem is a thought experiment in philosophy concerning immediate recovery from blindness. It was first formulated by William Molyneux, and notably referred to in John Locke's An Essay Concerning Human Understanding (1689). The problem can be stated in brief, "if a man born blind can feel the differences between shapes such as spheres and cubes, could he, if given the ability to see, distinguish those objects by sight alone, in reference to the tactile schemata he already possessed?"

Related Research Articles

Sensory nervous system part of the nervous system responsible for processing sensory information

The sensory nervous system is a part of the nervous system responsible for processing sensory information. A sensory system consists of sensory neurons, neural pathways, and parts of the brain involved in sensory perception. Commonly recognized sensory systems are those for vision, hearing, touch, taste, smell, and balance. In short, senses are transducers from the physical world to the realm of the mind where we interpret the information, creating our perception of the world around us.

The McGurk effect is a perceptual phenomenon that demonstrates an interaction between hearing and vision in speech perception. The illusion occurs when the auditory component of one sound is paired with the visual component of another sound, leading to the perception of a third sound. The visual information a person gets from seeing a person speak changes the way they hear the sound. If a person is getting poor quality auditory information but good quality visual information, they may be more likely to experience the McGurk effect. Integration abilities for audio and visual information may also influence whether a person will experience the effect. People who are better at sensory integration have been shown to be more susceptible to the effect. Many people are affected differently by the McGurk effect based on many factors, including brain damage and other disorders.

Parietal lobe part of the brain

The parietal lobe is one of the four major lobes of the cerebral cortex in the brain of mammals. The parietal lobe is positioned above the temporal lobe and behind the frontal lobe and central sulcus.

Stimulus modality, also called sensory modality, is one aspect of a stimulus or what is perceived after a stimulus. For example, the temperature modality is registered after heat or cold stimulate a receptor. Some sensory modalities include: light, sound, temperature, taste, pressure, and smell. The type and location of the sensory receptor activated by the stimulus plays the primary role in coding the sensation. All sensory modalities work together to heighten stimuli sensation when necessary.

Multisensory integration, also known as multimodal integration, is the study of how information from the different sensory modalities, such as sight, sound, touch, smell, self-motion and taste, may be integrated by the nervous system. A coherent representation of objects combining modalities enables us to have meaningful perceptual experiences. Indeed, multisensory integration is central to adaptive behavior because it allows us to perceive a world of coherent perceptual entities. Multisensory integration also deals with how different sensory modalities interact with one another and alter each other's processing.

Prof. Charles Spence is an experimental psychologist at the University of Oxford. He is the head of the Crossmodal Research group which specializes in the research about the integration of information across different sensory modalities He also teaches Experimental Psychology to undergraduates at Somerville College. He is currently a consultant for a number of multinational companies advising on various aspects of multisensory design. He has also conducted research on human-computer interaction issues on the Crew Work Station on the European Space Shuttle, and currently works on problems associated with the design of foods that maximally stimulate the senses, and with the effect of the indoor environment on mood, well-being, and performance. Charles has published more than 500 articles in top-flight scientific journals over the last decade. Charles has been awarded the 10th Experimental Psychology Society Prize, the British Psychology Society: Cognitive Section Award, the Paul Bertelson Award, recognizing him as the young European Cognitive Psychologist of the Year, and, most recently, the prestigious Friedrich Wilhelm Bessel Research Award from the Alexander von Humboldt Foundation in Germany.

Sensory processing is the process that organizes sensation from one’s own body and the environment, thus making it possible to use the body effectively within the environment. Specifically, it deals with how the brain processes multiple sensory modality inputs, such as proprioception, vision, auditory system, tactile, olfactory, vestibular system, interoception, and taste into usable functional outputs.

Body schema is a concept used in several disciplines, including psychology, neuroscience, philosophy, sports medicine, and robotics. The neurologist Sir Henry Head originally defined it as a postural model of the body that actively organizes and modifies 'the impressions produced by incoming sensory impulses in such a way that the final sensation of [body] position, or of locality, rises into consciousness charged with a relation to something that has happened before'. As a postural model that keeps track of limb position, it plays an important role in control of action. It involves aspects of both central and peripheral systems. Thus, a body schema can be considered the collection of processes that registers the posture of one's body parts in space. The schema is updated during body movement. This is typically a non-conscious process, and is used primarily for spatial organization of action. It is therefore a pragmatic representation of the body’s spatial properties, which includes the length of limbs and limb segments, their arrangement, the configuration of the segments in space, and the shape of the body surface. Body schema also plays an important role in the integration and use of tools by humans.

Extinction is a neurological disorder that impairs the ability to perceive multiple stimuli of the same type simultaneously. Extinction is usually caused by damage resulting in lesions on one side of the brain. Those who are affected by extinction have a lack of awareness in the contralesional side of space and a loss of exploratory search and other actions normally directed toward that side.

Michael Steven Anthony Graziano is an American scientist and novelist who is currently a professor of Psychology and Neuroscience at Princeton University. His scientific research focuses on the brain basis of awareness. He has proposed the "attention schema" theory, an explanation of how, and for what adaptive advantage, brains attribute the property of awareness to themselves. His previous work focused on how the cerebral cortex monitors the space around the body and controls movement within that space. Notably he has suggested that the classical map of the body in motor cortex, the homunculus, is not correct and is better described as a map of complex actions that make up the behavioral repertoire. His publications on this topic have had a widespread impact among neuroscientists but have also generated controversy. His novels rely partly on his background in psychology and are known for surrealism or magic realism. Graziano also composes music including symphonies and string quartets. He is an avid practitioner of ventriloquism and uses a puppet orangutan "Kevin" as sidekick.

Body transfer illusion is the illusion of owning either a part of a body or an entire body other than one's own, thus it is sometimes referred to as "body ownership" in the research literature. It can be induced experimentally by manipulating the visual perspective of the subject and also supplying visual and sensory signals which correlate to the subject's body. For it to occur, bottom-up perceptual mechanisms, such as the input of visual information, must override top-down knowledge that the certain body does not belong. This is what results in an illusion of transfer of body ownership. It is typically induced using virtual reality.

Berit Brogaard philosopher

Berit Oskar Brogaard is a Danish and American philosopher specializing in the areas of cognitive neuroscience, philosophy of mind, and philosophy of language. Her recent work concerns synesthesia, savant syndrome, blindsight and perceptual reports. She is Professor of Philosophy and runs a perception lab at the University of Miami in Coral Gables, Florida. She is also co-editor of the Philosophical Gourmet Report.

The Colavita visual dominance effect refers to the phenomenon where participants respond more often to the visual component of an audiovisual stimulus, when presented with bimodal stimuli. Research has shown that vision is the most dominant sense for human beings who do not suffer from sensory difficulties. Theorists have proposed that the Colavita visual dominance effect demonstrates a bias toward visual sensory information, because the presence of auditory stimuli is commonly neglected during audiovisual events.

Gemma Calvert, FRSA is a British neuroscientist and pioneer of Neuromarketing. She is the Founder of Neurosense Limited, the world’s first neuromarketing agency established in 1999, and in 2016 she co-Founded Split Second Research, a company which provides implicit research for companies worldwide. Calvert is a Professor of Marketing at the Nanyang Business School at the Nanyang Technological University in Singapore.

Sensory processing disorder is a condition where multisensory integration is not adequately processed in order to provide appropriate responses to the demands of the environment.

Medial pulvinar nucleus

Medial pulvinar nucleus is one of four traditionally anatomically distinguished nuclei of the pulvinar of the thalamus. The other three nuclei of the pulvinar are called lateral, inferior and anterior pulvinar nuclei.

Multisensory learning is the theory that individuals learn better if they are taught using more than one sense (modality). The senses usually employed in multisensory learning are visual, auditory, kinesthetic, and tactile – VAKT. Other senses might include smell, taste and balance.

References

  1. ↑ Lalanneab, Christophe; Lorenceaua, Jean (2004). "Crossmodal integration for perception and action". Journal of Physiology. 98 (1): 265–279. doi:10.1016/j.jphysparis.2004.06.001.
  2. ↑ Shams, Ladan; Kima, Robyn (September 2010). "Crossmodal influences on visual perception". Physics of Life Reviews. 7 (3): 269–284. doi:10.1016/j.plrev.2010.04.006.
  3. ↑ "Crossmodalism". CROSSMODALISM.
  4. ↑ Michalina (9 November 2017). "Crossmodalism: Mysterious Events that will take you by Surprise". Heart Berlin. Retrieved 16 August 2018.
  5. ↑ "Is Crossmodalism the Dadaism of the 21st century?". TOA.life. Tech Open Air. Retrieved 16 August 2018.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.