Parallel processing (psychology)

Last updated

In psychology, parallel processing is the ability of the brain to simultaneously process incoming stimuli of differing quality. [1] Parallel processing is associated with the visual system in that the brain divides what it sees into four components: color, motion, shape, and depth. These are individually analyzed and then compared to stored memories, which helps the brain identify what you are viewing. [2] The brain then combines all of these into the field of view that is then seen and comprehended. [3] This is a continual and seamless operation. For example, if one is standing between two different groups of people who are simultaneously carrying on two different conversations, one may be able to pick up only some information of both conversations at the same time. [4] Parallel processing has been linked, by some experimental psychologists, to the stroop effect (resulting from the stroop test where there is a mismatch between the name of a color and the color that the word is written in). [5] In the stroop effect, an inability to attend to all stimuli is seen through people's selective attention. [6]

Contents

Background

In 1990, American Psychologist David Rumelhart proposed the model of parallel distributed processing (PDP) in hopes of studying neural processes through computer simulations. [7] According to Rumelhart, the PDP model represents information processing as interactions between elements called units, with the interactions being either excitatory or inhibitory in nature. [8] Parallel Distributed Processing Models are neurally inspired, emulating the organisational structure of nervous systems of living organisms. [9] A general mathematical framework is provided for them.

Parallel processing models assume that information is represented in the brain using patterns of activation. Information processing encompasses the interactions of neuron-like units linked by synapse-like connections. These can be either excitatory or inhibitory. Every individual unit's activation level is updated using a function of connection strengths and activation level of other units. A set of response units is activated by the propagation of activation patterns. The connection weights are eventually adjusted using learning. [10]

Serial vs parallel processing

In contrast to parallel processing, serial processing involves sequential processing of information, without any overlap of processing times. [11] The distinction between these two processing models is most observed during visual stimuli is targeted and processed (also called visual search).

In case of serial processing, the elements are searched one after the other in a serial order to find the target. When the target is found, the search terminates. Alternatively, it continues to the end to ensure that the target is not present. This results in reduced accuracy and increased time for displays with more objects.

On the other hand, in the case of parallel processing, all objects are processed simultaneously but the completion times may vary. This may or may not reduce the accuracy, but the time courses are similar irrespective of the size of the display. [12]

However, there are concerns about the efficiency of parallel processing models in case of complex tasks which are discussed ahead in this article.

Aspects of a parallel distributed processing model

There are eight major aspects of a parallel distributed processing model: [8]

Processing units

These units may include abstract elements such as features, shapes and words, and are generally categorised into three types: input, output and hidden units.

Activation state

This is a representation of the state of the system. The pattern of activation is represented using a vector of N real numbers, over the set of processing units. It is this pattern that captures what the system is representing at any time.

Output functions

An output function maps the current state of activation to an output signal. The units interact with their neighbouring units by transmitting signals. The strengths of these signals are determined by their degree of activation. This in turn affects the degree to which they affect their neighbours.

Connectivity patterns

The pattern of connectivity determines how the system will react to an arbitrary input. The total pattern of connectivity is represented by specifying the weights for every connection. A positive weight represents an excitatory input and a negative weight represents an inhibitory input.

Propagation rule

A net input is produced for each type of input using rules that take the output vector and combine it with the connectivity matrices. In the case of a more complex pattern connectivity, the rules are more complex too.

Activation rule

A new state of activation is produced for every unit by joining the net inputs of impinging units combined and the current state of activation for that unit.

Learning rule

The patterns of connectivity are modified using experience. The modifications can be of three types: First, the development of new connections. Second, the loss of existing connection. Last, the modification of strengths of connections that already exist. The first two can be considered as special cases of the last one. When the strength of a connection is changed from zero to a positive or negative one, it is like forming a new connection. When the strength of a connection is changed to zero, it is like losing an existing connection.

Environmental representation

In PDP models, the environment is represented as a time-varying stochastic function over the space of input patterns. [13] This means that at any given point, there is a possibility that any of the possible set of input patterns is impinging on the input units.   [9]

An example of a parallel distributed processing (PDP) model Parallel Processing Model.png
An example of a parallel distributed processing (PDP) model

An example of the PDP model is illustrated in Rumelhart's book 'Parallel Distributed Processing' of individuals who live in the same neighborhood and are part of different gangs. Other information is also included, such as their names, age group, marital status, and occupations within their respective gangs. Rumelhart considered each category as a 'unit' and an individual has connections with each unit. For instance, if more information is sought on an individual named Ralph, that name unit is activiated, revealing connections to the other properties of Ralph such as his marital status or age group. [8]

Depth

To sense depth, humans use both eyes to see three dimensional objects. This sense is present at birth in humans and some animals, such as cats, dogs, owls, and monkeys. [14] Animals with wider-set eyes have a harder time establishing depth, such as horses and cows. A special depth test was used on infants, named The Visual Cliff. [15] This test consisted of a table, half coated in a checkerboard pattern, and the other half a clear plexiglass sheet, revealing a second checkerboard platform about a foot below. Although the plexiglass was safe to climb on, the infants refused to cross over due to the perception of a visual cliff. This test proved that most infants already have a good sense of depth. This phenomenon is similar to how adults perceive heights.

Certain cues help establish depth perception. Binocular cues are made by humans' two eyes, which are subconsciously compared to calculate distance. [16] This idea of two separate images is used by 3-D and VR filmmakers to give two dimensional footage the element of depth. Monocular cues can be used by a single eye with hints from the environment. These hints include relative height, relative size, linear perspective, lights and shadows, and relative motion. [15] Each hint helps to establish small facts about a scene that work together to form a perception of depth. Binocular cues and monocular cues are used constantly and subconsciously to sense depth.

Limitations

Limitations of parallel processing have been brought up in several analytical studies. The main limitations highlighted include capacity limits of the brain, attentional blink rate interferences, limited processing capabilities, and information limitations in visual searches.

There are processing limits to the brain in the execution of complex tasks like object recognition. All parts of the brain cannot process at full capacity in a parallel method. Attention controls the allocation of resources to the tasks. To work efficiently, attention must be guided from object to object. [17]

These limits to attentional resources sometimes lead to serial bottlenecks in parallel processing, meaning that parallel processing is obstructed by serial processing in between. However, there is evidence for coexistence of serial and parallel processes. [18]

Feature integration theory

The feature integration theory by Anne Treisman is one of the theories that integrates serial and parallel processing while taking into account attentional resources. It consists of two stages-

  1. Detection of features- This stage occurs instantaneously and uses parallel processing. In this step, all the basic features of a display are picked up simultaneously, even if attention is being paid to a specific object.
  2. Integration of features- This step is more time-consuming and uses serial processing. It leads to the perception of whole objects and patterns. [19]

See also

Related Research Articles

<span class="mw-page-title-main">Attention</span> Psychological process of selectively perceiving and prioritising discrete aspects of information

Attention is the concentration of awareness on some phenomenon to the exclusion of other stimuli. It is a process of selectively concentrating on a discrete aspect of information, whether considered subjective or objective. William James (1890) wrote that "Attention is the taking possession by the mind, in clear and vivid form, of one out of what seem several simultaneously possible objects or trains of thought. Focalization, concentration, of consciousness are of its essence." Attention has also been described as the allocation of limited cognitive processing resources. Attention is manifested by an attentional bottleneck, in terms of the amount of data the brain can process each second; for example, in human vision, only less than 1% of the visual input data can enter the bottleneck, leading to inattentional blindness.

<span class="mw-page-title-main">Connectionism</span> Cognitive science approach

Connectionism is the name of an approach to the study of human mental processes and cognition that utilizes mathematical models known as connectionist networks or artificial neural networks. Connectionism has had many 'waves' since its beginnings.

The consciousness and binding problem is the problem of how objects, background and abstract or emotional features are combined into a single experience.

<span class="mw-page-title-main">Stroop effect</span> Effect of psychological interference on reaction time

In psychology, the Stroop effect is the delay in reaction time between congruent and incongruent stimuli.

<span class="mw-page-title-main">James McClelland (psychologist)</span>

James Lloyd "Jay" McClelland, FBA is the Lucie Stern Professor at Stanford University, where he was formerly the chair of the Psychology Department. He is best known for his work on statistical learning and Parallel Distributed Processing, applying connectionist models to explain cognitive phenomena such as spoken word recognition and visual word recognition. McClelland is to a large extent responsible for the large increase in scientific interest in connectionism in the 1980s.

<span class="mw-page-title-main">Feedforward neural network</span> One of two broad types of artificial neural network

A feedforward neural network (FNN) is one of the two broad types of artificial neural network, characterized by direction of the flow of information between its layers. Its flow is uni-directional, meaning that the information in the model flows in only one direction—forward—from the input nodes, through the hidden nodes and to the output nodes, without any cycles or loops, in contrast to recurrent neural networks, which have a bi-directional flow. Modern feedforward networks are trained using the backpropagation method and are colloquially referred to as the "vanilla" neural networks.

<span class="mw-page-title-main">Neural network (biology)</span> Structure in nervous systems

A neural network, also called a neuronal network, is an interconnected population of neurons. Biological neural networks are studied to understand the organization and functioning of nervous systems.

Holonomic brain theory is a branch of neuroscience investigating the idea that human consciousness is formed by quantum effects in or between brain cells. Holonomic refers to representations in a Hilbert phase space defined by both spectral and space-time coordinates. Holonomic brain theory is opposed by traditional neuroscience, which investigates the brain's behavior by looking at patterns of neurons and the surrounding chemistry.

Affective neuroscience is the study of how the brain processes emotions. This field combines neuroscience with the psychological study of personality, emotion, and mood. The basis of emotions and what emotions are remains an issue of debate within the field of affective neuroscience.

<span class="mw-page-title-main">David Rumelhart</span> American psychologist (1942–2011)

David Everett Rumelhart was an American psychologist who made many contributions to the formal analysis of human cognition, working primarily within the frameworks of mathematical psychology, symbolic artificial intelligence, and parallel distributed processing. He also admired formal linguistic approaches to cognition, and explored the possibility of formulating a formal grammar to capture the structure of stories.

In cognitive psychology, the word superiority effect (WSE) refers to the phenomenon that people have better recognition of letters presented within words as compared to isolated letters and to letters presented within nonword strings. Studies have also found a WSE when letter identification within words is compared to letter identification within pseudowords and pseudohomophones.

Visual search is a type of perceptual task requiring attention that typically involves an active scan of the visual environment for a particular object or feature among other objects or features. Visual search can take place with or without eye movements. The ability to consciously locate an object or target amongst a complex array of stimuli has been extensively studied over the past 40 years. Practical examples of using visual search can be seen in everyday life, such as when one is picking out a product on a supermarket shelf, when animals are searching for food among piles of leaves, when trying to find a friend in a large crowd of people, or simply when playing visual search games such as Where's Wally?

Memory has the ability to encode, store and recall information. Memories give an organism the capability to learn and adapt from previous experiences as well as build relationships. Encoding allows a perceived item of use or interest to be converted into a construct that can be stored within the brain and recalled later from long-term memory. Working memory stores information for immediate use or manipulation, which is aided through hooking onto previously archived items already present in the long-term memory of an individual.

TRACE is a connectionist model of speech perception, proposed by James McClelland and Jeffrey Elman in 1986. It is based on a structure called "the TRACE," a dynamic processing structure made up of a network of units, which performs as the system's working memory as well as the perceptual processing mechanism. TRACE was made into a working computer program for running perceptual simulations. These simulations are predictions about how a human mind/brain processes speech sounds and words as they are heard in real time.

The logogen model of 1969 is a model of speech recognition that uses units called "logogens" to explain how humans comprehend spoken or written words. Logogens are a vast number of specialized recognition units, each able to recognize one specific word. This model provides for the effects of context on word recognition.

Network of human nervous system comprises nodes that are connected by links. The connectivity may be viewed anatomically, functionally, or electrophysiologically. These are presented in several Wikipedia articles that include Connectionism, Biological neural network, Artificial neural network, Computational neuroscience, as well as in several books by Ascoli, G. A. (2002), Sterratt, D., Graham, B., Gillies, A., & Willshaw, D. (2011), Gerstner, W., & Kistler, W. (2002), and Rumelhart, J. L., McClelland, J. L., and PDP Research Group (1986) among others. The focus of this article is a comprehensive view of modeling a neural network. Once an approach based on the perspective and connectivity is chosen, the models are developed at microscopic, mesoscopic, or macroscopic (system) levels. Computational modeling refers to models that are developed using computing tools.

Catastrophic interference, also known as catastrophic forgetting, is the tendency of an artificial neural network to abruptly and drastically forget previously learned information upon learning new information. Neural networks are an important part of the network approach and connectionist approach to cognitive science. With these networks, human capabilities such as memory and learning can be modeled using computer simulations.

In psychology, the transposed letter effect is a test of how a word is processed when two letters within the word are switched.

In neuroscience, predictive coding is a theory of brain function which postulates that the brain is constantly generating and updating a "mental model" of the environment. According to the theory, such a mental model is used to predict input signals from the senses that are then compared with the actual input signals from those senses. With the rising popularity of representation learning, the theory is being actively pursued and applied in machine learning and related fields.

Artificial neural networks (ANNs) are models created using machine learning to perform a number of tasks. Their creation was inspired by neural circuitry. While some of the computational implementations ANNs relate to earlier discoveries in mathematics, the first implementation of ANNs was by psychologist Frank Rosenblatt, who developed the perceptron. Little research was conducted on ANNs in the 1970s and 1980s, with the AAAI calling that period an "AI winter".

References

  1. ↑ LaBerge, David; Samuels, S.Jay (1974). "Toward a theory of automatic information processing in reading". Cognitive Psychology. Elsevier BV. 6 (2): 293–323. doi:10.1016/0010-0285(74)90015-2. ISSN   0010-0285.
  2. ↑ Hinton, Geoffrey (2014). Parallel models of associative memory. New York: Psychology Press. ISBN   978-1-315-80799-7.
  3. ↑ Wässle, Heinz (2004). "Parallel processing in the mammalian retina". Nature Reviews Neuroscience. 5 (10): 747–757. doi:10.1038/nrn1497. ISSN   1471-003X. PMID   15378035. S2CID   10518721.
  4. ↑ Cohen, J. D.; Dunbar, K.; McClelland, J. L. (16 June 1988). On the Control of Automatic Processes: A Parallel Distributed Processing Model of the Stroop Effect (Report). Fort Belvoir, VA: Defense Technical Information Center. doi:10.21236/ada218914.
  5. ↑ Baghdadi, Golnaz; Towhidkhah, Farzad; Rajabi, Mojdeh, eds. (2021), "Chapter 7 – Assessment methods", Neurocognitive Mechanisms of Attention, Academic Press, pp. 203–250, doi:10.1016/B978-0-323-90935-8.00005-6, ISBN   978-0-323-90935-8
  6. ↑ Cohen, Jonathan D.; Dunbar, Kevin; McClelland, James L. (1990). "On the control of automatic processes: A parallel distributed processing account of the Stroop effect". Psychological Review. 97 (3): 332–361. doi:10.1037/0033-295x.97.3.332. PMID   2200075.
  7. ↑ Rogers, Timothy T.; McClelland, James L. (December 2008). "Précis of Semantic Cognition: A Parallel Distributed Processing Approach". Behavioral and Brain Sciences. 31 (6): 689–714. doi:10.1017/S0140525X0800589X. ISSN   0140-525X. S2CID   145402215.
  8. 1 2 3 Rumelhart, David E. (1986). Parallel distributed processing : explorations in the microstructure of cognition. James L. McClelland, San Diego. PDP Research Group University of California. Cambridge, Mass.: MIT Press. ISBN   0-262-18120-7. OCLC   12837549.
  9. 1 2 Rumelhart, David E.; McClelland, James L.; PDP Research Group (1986). Parallel distributed processing: explorations in the microstructure of cognition. Cambridge, Mass.: MIT Press. ISBN   0-262-18120-7. OCLC   12837549.
  10. ↑ Holyoak, Keith J. (1987). Rumelhart, David E.; McClelland, James L.; Group, PDP Research (eds.). "A Connectionist View of Cognition". Science. 236 (4804): 992–996. doi:10.1126/science.236.4804.992. ISSN   0036-8075. JSTOR   1699673. PMID   17812774.
  11. ↑ Townsend, James T. (January 1990). "Serial vs. Parallel Processing: Sometimes They Look like Tweedledum and Tweedledee but they can (and Should) be Distinguished". Psychological Science. 1 (1): 46–54. doi:10.1111/j.1467-9280.1990.tb00067.x. ISSN   0956-7976. S2CID   51773777.
  12. ↑ Dosher, Barbara Anne; Han, Songmei; Lu, Zhong-Lin (2010). "Information-limited parallel processing in difficult heterogeneous covert visual search". Journal of Experimental Psychology: Human Perception and Performance. 36 (5): 1128–1144. doi:10.1037/a0020366. ISSN   1939-1277. PMC   3929106 . PMID   20873936.
  13. ↑ Snodgrass, Joan Gay; Townsend, James T.; Ashby, F. Gregory (1985). "Stochastic Modeling of Elementary Psychological Processes". The American Journal of Psychology. 98 (3): 480. doi:10.2307/1422636. ISSN   0002-9556. JSTOR   1422636.
  14. ↑ Nityananda, Vivek; Read, Jenny C. A. (2017-07-15). "Stereopsis in animals: evolution, function and mechanisms". Journal of Experimental Biology. 220 (14): 2502–2512. doi:10.1242/jeb.143883. ISSN   0022-0949. PMC   5536890 . PMID   28724702.
  15. 1 2 Myers, David G. (16 September 2021). Exploring Psychology. ISBN   978-1-319-42980-5. OCLC   1273977481.
  16. ↑ W., Eysenck, Michael (2020). Cognitive psychology : a student's handbook. ISBN   978-1-138-48221-0. OCLC   1150999656.{{cite book}}: CS1 maint: multiple names: authors list (link)
  17. ↑ Wolfe, Jeremy M. (August 1992). "The Parallel Guidance of Visual Attention". Current Directions in Psychological Science. 1 (4): 124–128. doi:10.1111/1467-8721.ep10769733. ISSN   0963-7214. S2CID   15808488.
  18. ↑ Sigman, Mariano; Dehaene, Stanislas (2008-07-23). "Brain Mechanisms of Serial and Parallel Processing during Dual-Task Performance". Journal of Neuroscience. 28 (30): 7585–7598. doi:10.1523/JNEUROSCI.0948-08.2008. ISSN   0270-6474. PMC   6670853 . PMID   18650336.
  19. ↑ "Features and Objects in Visual Processing", Foundations of Cognitive Psychology, The MIT Press, 2002, doi:10.7551/mitpress/3080.003.0025, ISBN   9780262278263 , retrieved 2022-02-16
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.