Spreading activation

Last updated

Spreading activation is a method for searching associative networks, biological and artificial neural networks, or semantic networks [1] . The search process is initiated by labeling a set of source nodes (e.g. concepts in a semantic network) with weights or "activation" and then iteratively propagating or "spreading" that activation out to other nodes linked to the source nodes. Most often these "weights" are real values that decay as activation propagates through the network. When the weights are discrete this process is often referred to as marker passing. Activation may originate from alternate paths, identified by distinct markers, and terminate when two alternate paths reach the same node. However brain studies show that several different brain areas play an important role in semantic processing. [2]

Contents

Spreading activation in semantic networks as a model were invented in cognitive psychology [3] [4] to model the fan out effect.[ citation needed ]

Spreading activation can also be applied in information retrieval, [5] [6] by means of a network of nodes representing documents and terms contained in those documents.

Cognitive psychology

As it relates to cognitive psychology, spreading activation is the theory of how the brain iterates through a network of associated ideas to retrieve specific information. The spreading activation theory presents the array of concepts within our memory as cognitive units, each consisting of a node and its associated elements or characteristics, all connected together by edges. [4] A spreading activation network can be represented schematically, in a sort of web diagram with shorter lines between two nodes meaning the ideas are more closely related and will typically be associated more quickly to the original concept. In memory psychology, the spreading activation model holds that people organize their knowledge of the world based on their personal experiences, which in turn form the network of ideas that is the person's knowledge of the world. [3]

When a word (the target) is preceded by an associated word (the prime) in word recognition tasks, participants seem to perform better in the amount of time that it takes them to respond. For instance, subjects respond faster to the word "doctor" when it is preceded by "nurse" than when it is preceded by an unrelated word like "carrot". This semantic priming effect with words that are close in meaning within the cognitive network has been seen in a wide range of tasks given by experimenters, ranging from sentence verification to lexical decision and naming. [7]

As another example, if the original concept is "red" and the concept "vehicles" is primed, they are much more likely to say "fire engine" instead of something unrelated to vehicles, such as "cherries". If instead "fruits" was primed, they would likely name "cherries" and continue on from there. The activation of pathways in the network has everything to do with how closely linked two concepts are by meaning, as well as how a subject is primed.

Algorithm

A directed graph is populated by Nodes[ 1...N ] each having an associated activation value A [ i ] which is a real number in the range [0.0 ... 1.0]. A Link[ i, j ] connects source node[ i ] with target node[ j ]. Each edge has an associated weight W [ i, j ] usually a real number in the range [0.0 ... 1.0]. [8]

Parameters:

Steps:

  1. Initialize the graph setting all activation values A [ i ] to zero. Set one or more origin nodes to an initial activation value greater than the firing threshold F. A typical initial value is 1.0.
  2. For each unfired node [ i ] in the graph having an activation value A [ i ] greater than the node firing threshold F:
  3. For each Link [ i, j ] connecting the source node [ i ] with target node [ j ], adjust A [ j ] = A [ j ] + (A [ i ] * W [ i, j ] * D) where D is the decay factor.
  4. If a target node receives an adjustment to its activation value so that it would exceed 1.0, then set its new activation value to 1.0. Likewise maintain 0.0 as a lower bound on the target node's activation value should it receive an adjustment to below 0.0.
  5. Once a node has fired it may not fire again, although variations of the basic algorithm permit repeated firings and loops through the graph.
  6. Nodes receiving a new activation value that exceeds the firing threshold F are marked for firing on the next spreading activation cycle.
  7. If activation originates from more than one node, a variation of the algorithm permits marker passing to distinguish the paths by which activation is spread over the graph
  8. The procedure terminates when either there are no more nodes to fire or in the case of marker passing from multiple origins, when a node is reached from more than one path. Variations of the algorithm that permit repeated node firings and activation loops in the graph, terminate after a steady activation state, with respect to some delta, is reached, or when a maximum number of iterations is exceeded.

Examples

In this example, spreading activation originated at node 1 which has an initial activation value of 1.0 (100%). Each link has the same weight value of 0.9. The decay factor was 0.85. Four cycles of spreading activation have occurred. Color hue and saturation indicate different activation values. Spreading-activation-graph-1.png
In this example, spreading activation originated at node 1 which has an initial activation value of 1.0 (100%). Each link has the same weight value of 0.9. The decay factor was 0.85. Four cycles of spreading activation have occurred. Color hue and saturation indicate different activation values.

See also

Notes

  1. Fähndrich, J. (2018). Semantic decomposition and marker passing in an artificial representation of meaning. Technische Universitaet Berlin (Germany).
  2. Karalyn Patterson, Peter J. Nestor & Timothy T. Rogers: "Where do you know what you know? The representation of semantic knowledge in the human brain"
  3. 1 2 Collins, Allan M.; Loftus, Elizabeth F. (1975). "A spreading-activation theory of semantic processing". Psychological Review. 82 (6): 407–428. doi:10.1037/0033-295X.82.6.407. ISSN   0033-295X. S2CID   14217893.
  4. 1 2 Anderson, John R. (1983). "A spreading activation theory of memory". Journal of Verbal Learning and Verbal Behavior. 22 (3): 261–295. doi:10.1016/S0022-5371(83)90201-3. ISSN   0022-5371.
  5. S. Preece, A spreading activation network model for information retrieval. PhD thesis, University of Illinois, Urbana-Champaign, 1981.
  6. Fabio Crestani. "Application of Spreading Activation Techniques in Information Retrieval". Artificial Intelligence Review, 1997
  7. Chwilla, Dorothee J.; Hagoort, Peter; Brown, C. M., "The mechanism underlying backward priming in a lexical decision task: spreading activation versus semantic matching", The Quarterly Journal of Experimental Psychology, 1998, 51A (3), 531-560
  8. Boosting item keyword search with spreading activation Aswath, D.; Ahmed, S.T.; Dapos;cunha, J.; Davulcu, H., Web Intelligence, 2005. Proceedings. The 2005 IEEE/WIC/ACM International Conference on Volume, Issue, 19-22 Sept. 2005 Page(s): 704 - 707

Related Research Articles

<span class="mw-page-title-main">Semantic network</span> Knowledge base that represents semantic relations between concepts in a network

A semantic network, or frame network is a knowledge base that represents semantic relations between concepts in a network. This is often used as a form of knowledge representation. It is a directed or undirected graph consisting of vertices, which represent concepts, and edges, which represent semantic relations between concepts, mapping or connecting semantic fields. A semantic network may be instantiated as, for example, a graph database or a concept map. Typical standardized semantic networks are expressed as semantic triples.

Recall in memory refers to the mental process of retrieval of information from the past. Along with encoding and storage, it is one of the three core processes of memory. There are three main types of recall: free recall, cued recall and serial recall. Psychologists test these forms of recall as a way to study the memory processes of humans and animals. Two main theories of the process of recall are the two-stage theory and the theory of encoding specificity.

Semantic memory refers to general world knowledge that humans have accumulated throughout their lives. This general knowledge is intertwined in experience and dependent on culture. New concepts are learned by applying knowledge learned from things in the past.

Semantic similarity is a metric defined over a set of documents or terms, where the idea of distance between items is based on the likeness of their meaning or semantic content as opposed to lexicographical similarity. These are mathematical tools used to estimate the strength of the semantic relationship between units of language, concepts or instances, through a numerical description obtained according to the comparison of information supporting their meaning or describing their nature. The term semantic similarity is often confused with semantic relatedness. Semantic relatedness includes any relation between two terms, while semantic similarity only includes "is a" relations. For example, "car" is similar to "bus", but is also related to "road" and "driving".

The Levels of Processing model, created by Fergus I. M. Craik and Robert S. Lockhart in 1972, describes memory recall of stimuli as a function of the depth of mental processing. More analysis produce more elaborate and stronger memory than lower levels of processing. Depth of processing falls on a shallow to deep continuum. Shallow processing leads to a fragile memory trace that is susceptible to rapid decay. Conversely, deep processing results in a more durable memory trace. There are three levels of processing in this model. Structural processing, or visual, is when we remember only the physical quality of the word E.g how the word is spelled and how letters look. Phonemic processing includes remembering the word by the way it sounds. E.G the word tall rhymes with fall. Lastly, we have semantic processing in which we encode the meaning of the word with another word that is similar or has similar meaning. Once the word is perceived, the brain allows for a deeper processing.

Neurophilosophy or philosophy of neuroscience is the interdisciplinary study of neuroscience and philosophy that explores the relevance of neuroscientific studies to the arguments traditionally categorized as philosophy of mind. The philosophy of neuroscience attempts to clarify neuroscientific methods and results using the conceptual rigor and methods of philosophy of science.

Tip of the tongue is the phenomenon of failing to retrieve a word or term from memory, combined with partial recall and the feeling that retrieval is imminent. The phenomenon's name comes from the saying, "It's on the tip of my tongue." The tip of the tongue phenomenon reveals that lexical access occurs in stages.

Memory has the ability to encode, store and recall information. Memories give an organism the capability to learn and adapt from previous experiences as well as build relationships. Encoding allows a perceived item of use or interest to be converted into a construct that can be stored within the brain and recalled later from long-term memory. Working memory stores information for immediate use or manipulation, which is aided through hooking onto previously archived items already present in the long-term memory of an individual.

<span class="mw-page-title-main">Negative priming</span> Initial stimulus inhibits response to subsequent stimulus

Negative priming is an implicit memory effect in which prior exposure to a stimulus unfavorably influences the response to the same stimulus. It falls under the category of priming, which refers to the change in the response towards a stimulus due to a subconscious memory effect. Negative priming describes the slow and error-prone reaction to a stimulus that is previously ignored. For example, a subject may be imagined trying to pick a red pen from a pen holder. The red pen becomes the target of attention, so the subject responds by moving their hand towards it. At this time, they mentally block out all other pens as distractors to aid in closing in on just the red pen. After repeatedly picking the red pen over the others, switching to the blue pen results in a momentary delay picking the pen out. The slow reaction due to the change of the distractor stimulus to target stimulus is called the negative priming effect.

Psi-theory, developed by Dietrich Dörner at the University of Bamberg, is a systemic psychological theory covering human action regulation, intention selection and emotion. It models the human mind as an information processing agent, controlled by a set of basic physiological, social and cognitive drives. Perceptual and cognitive processing are directed and modulated by these drives, which allow the autonomous establishment and pursuit of goals in an open environment.

Priming is the idea that exposure to one stimulus may influence a response to a subsequent stimulus, without conscious guidance or intention. The priming effect is the positive or negative effect of a rapidly presented stimulus on the processing of a second stimulus that appears shortly after. Generally speaking, the generation of priming effect depends on the existence of some positive or negative relationship between priming and target stimuli. For example, the word nurse might be recognized more quickly following the word doctor than following the word bread. Priming can be perceptual, associative, repetitive, positive, negative, affective, semantic, or conceptual. Priming effects involve word recognition, semantic processing, attention, unconscious processing, and many other issues, and are related to differences in various writing systems. Onset of priming effects can be almost instantaneous.

SAC is a computational model of memory encoding and retrieval. It has been developed by Lynne M. Reder at Carnegie Mellon University. It shares many commonalities with ACT-R.Ilyes le bosse

Bilingual interactive activation plus (BIA+) is a model for understanding the process of bilingual language comprehension and consists of two interactive subsystems: the word identification subsystem and task/decision subsystem. It is the successor of the Bilingual Interactive Activation (BIA) model which was updated in 2002 to include phonologic and semantic lexical representations, revise the role of language nodes, and specify the purely bottom-up nature of bilingual language processing.

Memory gaps and errors refer to the incorrect recall, or complete loss, of information in the memory system for a specific detail and/or event. Memory errors may include remembering events that never occurred, or remembering them differently from the way they actually happened. These errors or gaps can occur due to a number of different reasons, including the emotional involvement in the situation, expectations and environmental changes. As the retention interval between encoding and retrieval of the memory lengthens, there is an increase in both the amount that is forgotten, and the likelihood of a memory error occurring.

The mental lexicon is a component of the human language faculty that contains information regarding the composition of words, such as their meanings, pronunciations, and syntactic characteristics. The mental lexicon is used in linguistics and psycholinguistics to refer to individual speakers' lexical, or word, representations. However, there is some disagreement as to the utility of the mental lexicon as a scientific construct.

Anthony Marcel is a British psychologist who contributed to the early debate on the nature of unconscious perceptual processes in the 1970s and 1980s. Marcel argued in favour of an unconscious mind that "…automatically re-describe(s) sensory data into every representational form and to the highest levels of description available to the organism.” Marcel sparked controversy with his claim to have demonstrated unconscious priming. As of 2013 Marcel was working at the University of Hertfordshire and Cambridge University where his research focused on consciousness and phenomenological experience.

Conceptual combination is a fundamental cognitive process by which two or more existing basic concepts are mentally synthesized to generate a composite, higher-order concept. The products of this process are sometimes referred to as "complex concepts." Combining concepts allows individuals to use a finite number of concepts which they already understand to construct a potentially limitless quantity of new, related concepts. It is an essential component of many abilities, such as perception, language, synthetic reasoning, creative thought and abstraction.

This glossary of artificial intelligence is a list of definitions of terms and concepts relevant to the study of artificial intelligence, its sub-disciplines, and related fields. Related glossaries include Glossary of computer science, Glossary of robotics, and Glossary of machine vision.

A semantic decomposition is an algorithm that breaks down the meanings of phrases or concepts into less complex concepts. The result of a semantic decomposition is a representation of meaning. This representation can be used for tasks, such as those related to artificial intelligence or machine learning. Semantic decomposition is common in natural language processing applications.

<span class="mw-page-title-main">Knowledge graph</span> Type of knowledge base

In knowledge representation and reasoning, a knowledge graph is a knowledge base that uses a graph-structured data model or topology to represent and operate on data. Knowledge graphs are often used to store interlinked descriptions of entities – objects, events, situations or abstract concepts – while also encoding the semantics or relationships underlying these entities.

References