Dual-coding theory

Last updated
Visual Storytelling by Suhani Gowan. Visual-Storytelling.jpg
Visual Storytelling by Suhani Gowan.

Dual-coding theory is a theory of cognition that suggests that the mind processes information along two different channels; verbal and nonverbal. It was hypothesized by Allan Paivio of the University of Western Ontario in 1971. In developing this theory, Paivio used the idea that the formation of mental imagery aids learning through the picture superiority effect. [2]

Contents

According to Paivio, there are two ways a person could expand on learned material: verbal associations and imagery. Dual-coding theory postulates that both sensory imagery and verbal information is used to represent information. [3] [4] Imagery and verbal information are processed differently and along distinct channels in the human mind, creating separate representations for information processed in each channel. The mental codes corresponding to these representations are used to organize incoming information that can be acted upon, stored, and retrieved for subsequent use. Both imagery and verbal codes can be used when recalling information. [4] For example, say a person has stored the stimulus concept "dog" as both the word 'dog' and as the image (appearance, sound, smell, and other sensory information) of a dog. When asked to recall the stimulus, the person can retrieve either the word or the image individually, or both simultaneously. If the word is recalled, the image of the dog is not lost and can still be retrieved at a later point in time. The ability to code a stimulus two different ways increases the chance of remembering that item compared to if the stimulus was only coded one way.

There has been controversy to the limitations of the dual-coding theory. Dual-coding theory does not take into account the possibility of cognition being mediated by something other than words and images. Not enough research has been done to determine if words and images are the only way we remember items, and the theory would not hold true if another form of codes were discovered. [5] Another limitation of the dual-coding theory is that it is only valid for tests on which people are asked to focus on identifying how concepts are related. [2] If associations between a word and an image cannot be formed, it is much harder to remember and recall the word at a later point in time. While this limits the effectiveness of the dual-coding theory, it is still valid over a wide range of circumstances and can be used to improve memory. [2]

Types of codes

Analogue codes are used to mentally represent images. Analogue codes retain the main perceptual features of whatever is being represented, so the images we form in our minds are highly similar to the physical stimuli. They are a near-exact representation of the physical stimuli we observe in our environment, such as trees and rivers. [4]

Symbolic codes are used to form mental representations of words. They represent something conceptually, and sometimes, arbitrarily, as opposed to perceptually. Similar to the way a watch may represent information in the form of numbers to display the time, symbolic codes represent information in our mind in the form of arbitrary symbols, like words and combinations of words, to represent several ideas. Each symbol (x, y, 1, 2, etc.) can arbitrarily represent something other than itself. For instance, the letter x is often used to represent more than just the concept of an x, the 24th letter of the alphabet. It can be used to represent a variable x in mathematics, or a multiplication symbol in an equation. Concepts like multiplication can be represented symbolically by an "x" because we arbitrarily assign it a deeper concept. Only when we use it to represent this deeper concept does the letter "x" carry this type of meaning.

Support

Evidence from psychological research

Many researchers have agreed that only words and images are used in mental representation. [5] Supporting evidence shows that memory for some verbal information is enhanced if a relevant visual is also presented or if the learner can imagine a visual image to go with the verbal information. Likewise, visual information can often be enhanced when paired with relevant verbal information, whether real-world or imagined. [6] This theory has been applied to the use of multimedia presentations. Because multimedia presentations require both spatial and verbal working memory, individuals dual code information presented and are more likely to recall the information when tested at a later date. [7] Moreover, studies that have been conducted on abstract and concrete words have also found that the participants remembered concrete words better than the abstract words. [8] [9] [10]

Paivio found that participants when shown a rapid sequence of pictures as well as a rapid sequence of words and later asked to recall the words and pictures, in any order, were better at recalling images. Participants, however, more readily recalled the sequential order of the words, rather than the sequence of pictures. These results supported Paivio's hypothesis that verbal information is processed differently from visual information and that verbal information was superior to visual information when sequential order was also required for the memory task. [11] Lee Brooks conducted an experiment that provided additional support for two systems for memory. He had participants perform either a visual task, where they had to view a picture and answer questions about the picture, or a verbal task, where they listened to a sentence and were then asked to answer questions pertaining to the sentence. To respond to the questions, participants were asked to either respond verbally, visually, or manually. Through this experiment, Brooks found that interference occurred when a visual perception was mixed with manipulation of the visual task, and verbal responses interfere with a task involving a verbal statement to be manually manipulated. This supported the idea of two codes used to mentally represent information. [4]

Working memory as proposed by Alan Baddeley includes a two-part processing system with a visuospatial sketchpad and a phonological loop which essentially maps to Paivio's theory.

Dual-coding theories complement a dual-route theory of reading. When people read written information, dual-route theory contends that the readers access orthographic and phonological information to recognize words in the writing.

Paivio's work has implications for literacy, visual mnemonics, idea generation, HPT, human factors, interface design, as well as the development of educational materials among others. It also has implications for, and counterparts in, cognitive sciences and computational cognitive modeling (in the form of dual process cognitive models and so on [12] [13] [14] ). It also has had implications for cognitive robotics.

Cognitive neuroscience support

Two different methods have been used to identify the regions involved in visual perception and visual imagery. First, functional magnetic resonance imaging (fMRI) is used to measure cerebral blood flow, which allows researchers to identify the amount of glucose and oxygen being consumed by a specific part of the brain, with an increase in blood flow providing a measure of brain activity. Second, an event-related potential (ERP) can be used to show the amount of electrical brain activity that is occurring due to a particular stimulus. Researchers have used both methods to determine which areas of the brain are active with different stimuli, and results have supported the dual-coding theory. Other research has been done with positron emission tomography (PET) scans and fMRI to show that participants had improved memory for spoken words and sentences when paired with an image, imagined or real. Those participants also showed an increase in brain activation that processes abstract words not easily paired with an image. [15]

Alternative theory

Dual-coding theory is not accepted by everyone. John Anderson and Gordon Bower proposed an alternative method – the propositional theory – of how knowledge is mentally represented. The propositional theory claims that mental representations are stored as propositions rather than as images. Here, proposition is defined as the meaning that underlies the relationship between concepts. The propositional theory is able to explain the basic concept of an idea without needing images or verbal information. It is able to take the complex and break it down more into the differing components of the idea or concept. [16] This theory states that images occur as a result of other cognitive processes because knowledge is not represented in the form of images, words, or symbols. This theory is also related to A Natural Deduction Systems model. This type of model allows for both feed forward and its reverse.

The common coding theory has also been proposed as an alternative to dual coding theory. The common coding theory looks at how things we see and hear are connected to our motor actions. It claims that there is a common code that is shared between perceiving something and the respective motor action.

See also

Related Research Articles

<span class="mw-page-title-main">Cognitive science</span> Interdisciplinary scientific study of cognitive processes

Cognitive science is the interdisciplinary, scientific study of the mind and its processes with input from linguistics, psychology, neuroscience, philosophy, computer science/artificial intelligence, and anthropology. It examines the nature, the tasks, and the functions of cognition. Cognitive scientists study intelligence and behavior, with a focus on how nervous systems represent, process, and transform information. Mental faculties of concern to cognitive scientists include language, perception, memory, attention, reasoning, and emotion; to understand these faculties, cognitive scientists borrow from fields such as linguistics, psychology, artificial intelligence, philosophy, neuroscience, and anthropology. The typical analysis of cognitive science spans many levels of organization, from learning and decision to logic and planning; from neural circuitry to modular brain organization. One of the fundamental concepts of cognitive science is that "thinking can best be understood in terms of representational structures in the mind and computational procedures that operate on those structures."

Cognitive psychology is the scientific study of mental processes such as attention, language use, memory, perception, problem solving, creativity, and reasoning.

<span class="mw-page-title-main">Cognition</span> Act or process of knowing

Cognition is the "mental action or process of acquiring knowledge and understanding through thought, experience, and the senses". It encompasses all aspects of intellectual functions and processes such as: perception, attention, thought, imagination, intelligence, the formation of knowledge, memory and working memory, judgment and evaluation, reasoning and computation, problem-solving and decision-making, comprehension and production of language. Cognitive processes use existing knowledge and discover new knowledge.

Visual thinking, also called visual or spatial learning or picture thinking, is the phenomenon of thinking through visual processing. Visual thinking has been described as seeing words as a series of pictures. It is common in approximately 60–65% of the general population. "Real picture thinkers", those who use visual thinking almost to the exclusion of other kinds of thinking, make up a smaller percentage of the population. Research by child development theorist Linda Kreger Silverman suggests that less than 30% of the population strongly uses visual/spatial thinking, another 45% uses both visual/spatial thinking and thinking in the form of words, and 25% thinks exclusively in words. According to Kreger Silverman, of the 30% of the general population who use visual/spatial thinking, only a small percentage would use this style over and above all other forms of thinking, and can be said to be true "picture thinkers".

In the philosophy of mind, neuroscience, and cognitive science, a mental image is an experience that, on most occasions, significantly resembles the experience of "perceiving" some object, event, or scene but occurs when the relevant object, event, or scene is not actually present to the senses. There are sometimes episodes, particularly on falling asleep and waking up, when the mental imagery may be dynamic, phantasmagoric, and involuntary in character, repeatedly presenting identifiable objects or actions, spilling over from waking events, or defying perception, presenting a kaleidoscopic field, in which no distinct object can be discerned. Mental imagery can sometimes produce the same effects as would be produced by the behavior or experience imagined.

<span class="mw-page-title-main">Picture superiority effect</span> Psychological phenomenon

The picture superiority effect refers to the phenomenon in which pictures and images are more likely to be remembered than are words. This effect has been demonstrated in numerous experiments using different methods. It is based on the notion that "human memory is extremely sensitive to the symbolic modality of presentation of event information". Explanations for the picture superiority effect are not concrete and are still being debated.

<span class="mw-page-title-main">Visual memory</span> Ability to process visual and spatial information

Visual memory describes the relationship between perceptual processing and the encoding, storage and retrieval of the resulting neural representations. Visual memory occurs over a broad time range spanning from eye movements to years in order to visually navigate to a previously visited location. Visual memory is a form of memory which preserves some characteristics of our senses pertaining to visual experience. We are able to place in memory visual information which resembles objects, places, animals or people in a mental image. The experience of visual memory is also referred to as the mind's eye through which we can retrieve from our memory a mental image of original objects, places, animals or people. Visual memory is one of several cognitive systems, which are all interconnected parts that combine to form the human memory. Types of palinopsia, the persistence or recurrence of a visual image after the stimulus has been removed, is a dysfunction of visual memory.

<span class="mw-page-title-main">Subvocalization</span> Internal process while reading

Subvocalization, or silent speech, is the internal speech typically made when reading; it provides the sound of the word as it is read. This is a natural process when reading, and it helps the mind to access meanings to comprehend and remember what is read, potentially reducing cognitive load.

The Levels of Processing model, created by Fergus I. M. Craik and Robert S. Lockhart in 1972, describes memory recall of stimuli as a function of the depth of mental processing. Deeper levels of analysis produce more elaborate, longer-lasting, and stronger memory traces than shallow levels of analysis. Depth of processing falls on a shallow to deep continuum. Shallow processing leads to a fragile memory trace that is susceptible to rapid decay. Conversely, deep processing results in a more durable memory trace. There are three levels of processing in this model. Structural processing, or visual, is when we remember only the physical quality of the word E.g how the word is spelled and how letters look. Phonemic processing includes remembering the word by the way it sounds. E.G the word tall rhymes with fall. Lastly, we have semantic processing in which we encode the meaning of the word with another word that is similar or has similar meaning. Once the word is perceived, the brain allows for a deeper processing.

The cognitive revolution was an intellectual movement that began in the 1950s as an interdisciplinary study of the mind and its processes, from which emerged a new field known as cognitive science. The preexisting relevant fields were psychology, linguistics, computer science, anthropology, neuroscience, and philosophy. The approaches used were developed within the then-nascent fields of artificial intelligence, computer science, and neuroscience. In the 1960s, the Harvard Center for Cognitive Studies and the Center for Human Information Processing at the University of California, San Diego were influential in developing the academic study of cognitive science. By the early 1970s, the cognitive movement had surpassed behaviorism as a psychological paradigm. Furthermore, by the early 1980s the cognitive approach had become the dominant line of research inquiry across most branches in the field of psychology.

Memory has the ability to encode, store and recall information. Memories give an organism the capability to learn and adapt from previous experiences as well as build relationships. Encoding allows a perceived item of use or interest to be converted into a construct that can be stored within the brain and recalled later from long-term memory. Working memory stores information for immediate use or manipulation, which is aided through hooking onto previously archived items already present in the long-term memory of an individual.

Creative visualization is the cognitive process of purposefully generating visual mental imagery, with eyes open or closed, simulating or recreating visual perception, in order to maintain, inspect, and transform those images, consequently modifying their associated emotions or feelings, with intent to experience a subsequent beneficial physiological, psychological, or social effect, such as expediting the healing of wounds to the body, minimizing physical pain, alleviating psychological pain including anxiety, sadness, and low mood, improving self-esteem or self-confidence, and enhancing the capacity to cope when interacting with others.

In psychology, a dual process theory provides an account of how thought can arise in two different ways, or as a result of two different processes. Often, the two processes consist of an implicit (automatic), unconscious process and an explicit (controlled), conscious process. Verbalized explicit processes or attitudes and actions may change with persuasion or education; though implicit process or attitudes usually take a long amount of time to change with the forming of new habits. Dual process theories can be found in social, personality, cognitive, and clinical psychology. It has also been linked with economics via prospect theory and behavioral economics, and increasingly in sociology through cultural analysis.

Concept learning, also known as category learning, concept attainment, and concept formation, is defined by Bruner, Goodnow, & Austin (1967) as "the search for and listing of attributes that can be used to distinguish exemplars from non exemplars of various categories". More simply put, concepts are the mental categories that help us classify objects, events, or ideas, building on the understanding that each object, event, or idea has a set of common relevant features. Thus, concept learning is a strategy which requires a learner to compare and contrast groups or categories that contain concept-relevant features with groups or categories that do not contain concept-relevant features.

<span class="mw-page-title-main">Outline of thought</span> Overview of and topical guide to thought

The following outline is provided as an overview of and topical guide to thought (thinking):

Allan Urho Paivio was a professor of psychology at the University of Western Ontario and former bodybuilder. He earned his Ph.D. from McGill University in 1959 and taught at the University of Western Ontario from 1963 until his retirement.

Guided imagery is a mind-body intervention by which a trained practitioner or teacher helps a participant or patient to evoke and generate mental images that simulate or recreate the sensory perception of sights, sounds, tastes, smells, movements, and images associated with touch, such as texture, temperature, and pressure, as well as imaginative or mental content that the participant or patient experiences as defying conventional sensory categories, and that may precipitate strong emotions or feelings in the absence of the stimuli to which correlating sensory receptors are receptive.

<span class="mw-page-title-main">Functional specialization (brain)</span> Neuroscientific theory that different regions of the brain are specialized for different functions

In neuroscience, functional specialization is a theory which suggests that different areas in the brain are specialized for different functions.

<span class="mw-page-title-main">Bilingual memory</span>

Bilingualism is the regular use of two fluent languages, and bilinguals are those individuals who need and use two languages in their everyday lives. A person's bilingual memories are heavily dependent on the person's fluency, the age the second language was acquired, and high language proficiency to both languages. High proficiency provides mental flexibility across all domains of thought and forces them to adopt strategies that accelerate cognitive development. People who are bilingual integrate and organize the information of two languages, which creates advantages in terms of many cognitive abilities, such as intelligence, creativity, analogical reasoning, classification skills, problem solving, learning strategies, and thinking flexibility.

Seductive details are often used in textbooks, lectures, slideshows, and other forms of educational content to make a course more interesting or interactive. Seductive details can take the form of text, animations, photos, illustrations, sounds or music and are by definition: (1) interesting and (2) not directed toward the learning objectives of a lesson. John Dewey, in 1913, first referred to this as "fictitious inducements to attention." While illustrated text can enhance comprehension, illustrations that are not relevant can lead to poor learning outcomes. Since the late 1980s, many studies in the field of educational psychology have shown that the addition of seductive details results in poorer retention of information and transfer of learning. Thalheimer conducted a meta-analysis that found, overall, a negative impact for the inclusion of seductive details such as text, photos or illustrations, and sounds or music in learning content. More recently, a 2020 paper found a similar effect for decorative animations This reduction to learning is called the seductive details effect. There have been criticisms of this theory. Critics argue that seductive details do not always impede understanding and that seductive details can sometimes be motivating for learners.

References

  1. ↑ Asch, Solomon E.; Ebenholtz, Sheldon M. (1962). "The Principle of Associative Symmetry". Proceedings of the American Philosophical Society. 106 (2): 135–163. JSTOR   985378.
  2. 1 2 3 Reed, Stephen K. (12 April 2012). Cognition : theories and applications. Wadsworth, Cengage Learning. ISBN   978-1-133-49228-3. OCLC   1040947645.
  3. ↑ Sadoski, Mark; Paivio, Allan, "A Dual Coding Theoretical Model of Reading", Theoretical Models and Processes of Reading, DE: International Reading Association, pp. 1329–1362, doi:10.1598/0872075028.47 , retrieved 2022-04-23
  4. 1 2 3 4 Sternberg, Robert J (January 2016). Cognitive psychology. Cengage Learning. ISBN   978-1-305-64465-6. OCLC   1037299606.
  5. 1 2 Pylyshyn, Zenon W. (1973), "What the Mind's Eye Tells the Mind's Brain: A Critique of Mental Imagery", Images, Perception, and Knowledge, Springer Netherlands, pp. 1–36, doi:10.1007/978-94-010-1193-8_1, ISBN   978-94-010-1195-2
  6. ↑ Anderson, John R. (2014). Human Associative Memory. Taylor and Francis. ISBN   978-1-317-76988-0. OCLC   871224620.
  7. ↑ Brunyé, Tad T.; Taylor, Holly A.; Rapp, David N. (2007). "Repetition and dual coding in procedural multimedia presentations". Applied Cognitive Psychology. 22 (7): 877–895. doi:10.1002/acp.1396. ISSN   0888-4080. S2CID   1853751.
  8. ↑ Hargis, Gickling, Charles H, Edward E (May 1978). "The Function of Imagery in Word Recognition Development". The Reading Teacher. 31: 870–874 – via JSTOR.{{cite journal}}: CS1 maint: multiple names: authors list (link)
  9. ↑ Sadoski, Mark; Willson, Victor L.; Holcomb, Angelia; Boulware-Gooden, Regina (2004). "Verbal and Nonverbal Predictors of Spelling Performance". Journal of Literacy Research. 36 (4): 461–478. doi: 10.1207/s15548430jlr3604_2 . ISSN   1086-296X.
  10. ↑ Yui, Lin; Ng, Roslin; Perera-W.A., Hiran (2017-01-14). "Concrete vs abstract words – what do you recall better? A study on dual coding theory". doi: 10.7287/peerj.preprints.2719v1 .{{cite journal}}: Cite journal requires |journal= (help)
  11. ↑ Paivio, Allan (1969). "Mental imagery in associative learning and memory". Psychological Review. 76 (3): 241–263. doi:10.1037/h0027272. ISSN   0033-295X.
  12. ↑ Anderson, John Robert. (2005). Cognitive psychology and its implications : John R. Anderson. Worth Publishers. ISBN   0-7167-0110-3. OCLC   587804014.
  13. ↑ Just, Marcel Adam; Newman, Sharlene D; Keller, Timothy A; McEleney, Alice; Carpenter, Patricia A (2004). "Imagery in sentence comprehension: an fMRI study". NeuroImage. 21 (1): 112–124. doi:10.1016/j.neuroimage.2003.08.042. ISSN   1053-8119. PMID   14741648. S2CID   2912716.
  14. ↑ Sun, Ron, 1960- (2012). Duality of the mind : a bottom-up approach toward cognition. Routledge. ISBN   978-0-8058-3880-0. OCLC   858009136.{{cite book}}: CS1 maint: multiple names: authors list (link) CS1 maint: numeric names: authors list (link)
  15. ↑ Crosson, Bruce; Ford, Anastasia; McGregor, Keith M.; Meinzer, Marcus; Cheshkov, Sergey; Li, Xiufeng; Walker-Batson, Delaina; Briggs, Richard W. (2010). "Functional imaging and related techniques: An introduction for rehabilitation researchers". The Journal of Rehabilitation Research and Development. 47 (2): vii–xxxiv. doi: 10.1682/jrrd.2010.02.0017 . ISSN   0748-7711. PMC   3225087 . PMID   20593321.
  16. ↑ Friedenberg, Jay. (2016). Cognitive science : an introduction to the study of mind. Sage. ISBN   978-1-4833-4741-7. OCLC   989659597.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.