Prefrontal cortex basal ganglia working memory

Last updated

Prefrontal cortex basal ganglia working memory (PBWM) is an algorithm that models working memory in the prefrontal cortex and the basal ganglia. [1]

Contents

It can be compared to long short-term memory (LSTM) in functionality, but is more biologically explainable. [1] [2]

It uses the primary value learned value model to train prefrontal cortex working-memory updating system, based on the biology of the prefrontal cortex and basal ganglia. [3]

It is used as part of the Leabra framework and was implemented in Emergent in 2019.

Abstract

The prefrontal cortex has long been thought to subserve both working memory (the holding of information online for processing) and "executive" functions (deciding how to manipulate working memory and perform processing). Although many computational models of working memory have been developed, the mechanistic basis of executive function remains elusive.

PBWM is a computational model of the prefrontal cortex to control both itself and other brain areas in a strategic, task-appropriate manner. These learning mechanisms are based on subcortical structures in the midbrain, basal ganglia and amygdala, which together form an actor/critic architecture. The critic system learns which prefrontal representations are task-relevant and trains the actor, which in turn provides a dynamic gating mechanism for controlling working memory updating. Computationally, the learning mechanism is designed to simultaneously solve the temporal and structural credit assignment problems.

The model's performance compares favorably with standard backpropagation-based temporal learning mechanisms on the challenging 1-2-AX working memory task, and other benchmark working memory tasks. [1] [ third-party source needed ]

Model

First, there are multiple separate stripes (groups of units) in the prefrontal cortex and striatum layers. Each stripe can be independently updated, such that this system can remember several different things at the same time, each with a different "updating policy" of when memories are updated and maintained. The active maintenance of the memory is in prefrontal cortex (PFC), and the updating signals (and updating policy more generally) come from the striatum units (a subset of basal ganglia units). [3]

PVLV provides reinforcement learning signals to train up the dynamic gating system in the basal ganglia.

Sensory input and motor output

The sensory input is connected to the posterior cortex which is connected to the motor output. The sensory input is also linked to the PVLV system.

Posterior cortex

The posterior cortex form the hidden layers of the input/output mapping. The PFC is connected with the posterior cortex to contextualize this input/output mapping.

PFC

The PFC (for output gating) has a localist one-to-one representation of the input units for every stripe. Thus, you can look at these PFC representations and see directly what the network is maintaining. The PFC maintains the working memory needed to perform the task.

Striatum

This is the dynamic gating system representing the striatum units of the basal ganglia. Every even-index unit within a stripe represents "Go", while the odd-index units represent "NoGo." The Go units cause updating of the PFC, while the NoGo units cause the PFC to maintain its existing memory representation.

There are groups of units for every stripe.

In the PBWM model in Emergent, the matrices represent the striatum.

PVLV

All of these layers are part of PVLV system. The PVLV system controls the dopaminergic modulation of the basal ganglia (BG). Thus, BG/PVLV form an actor-critic architecture where the PVLV system learns when to update.[ citation needed ]

SNrThal

SNrThal represents the substantia nigra pars reticulata (SNr) and the associated area of the thalamus, which produce a competition among the Go/NoGo units within a given stripe and mediates competition using k-winners-take-all dynamics. If there is more overall Go activity in a given stripe, then the associated SNrThal unit gets activated, and it drives updating in PFC. For every stripe, there is one unit in SNrThal.[ citation needed ]

VTA and SNc

Ventral tegmental area (VTA) and substantia nigra pars compacta (SNc) are part of the dopamine layer. This layer models midbrain dopamine neurons. They control the dopaminergic modulation of the basal ganglia.[ citation needed ]

See also

Related Research Articles

Working memory is a cognitive system with a limited capacity that can hold information temporarily. Working memory is important for reasoning and the guidance of decision-making and behavior. Working memory is often used synonymously with short-term memory, but some theorists consider the two forms of memory distinct, assuming that working memory allows for the manipulation of stored information, whereas short-term memory only refers to the short-term storage of information. Working memory is a theoretical concept central to cognitive psychology, neuropsychology, and neuroscience.

Putamen Round structure at the base of the forebrain

The putamen is a round structure located at the base of the forebrain (telencephalon). The putamen and caudate nucleus together form the dorsal striatum. It is also one of the structures that compose the basal nuclei. Through various pathways, the putamen is connected to the substantia nigra, the globus pallidus, the claustrum, and the thalamus, in addition to many regions of the cerebral cortex. A primary function of the putamen is to regulate movements at various stages and influence various types of learning. It employs GABA, acetylcholine, and enkephalin to perform its functions. The putamen also plays a role in degenerative neurological disorders, such as Parkinson's disease.

Striatum Nucleus in the basal ganglia of the brain

The striatum, or corpus striatum, is a nucleus in the subcortical basal ganglia of the forebrain. The striatum is a critical component of the motor and reward systems; receives glutamatergic and dopaminergic inputs from different sources; and serves as the primary input to the rest of the basal ganglia.

Basal ganglia Group of subcortical nuclei involved in the motor and reward systems

The basal ganglia (BG), or basal nuclei, are a group of subcortical nuclei, of varied origin, in the brains of vertebrates. In humans, and some primates, there are some differences, mainly in the division of the globus pallidus into an external and internal region, and in the division of the striatum. The basal ganglia are situated at the base of the forebrain and top of the midbrain. Basal ganglia are strongly interconnected with the cerebral cortex, thalamus, and brainstem, as well as several other brain areas. The basal ganglia are associated with a variety of functions, including control of voluntary motor movements, procedural learning, habit learning, conditional learning, eye movements, cognition, and emotion.

The mesolimbic pathway, sometimes referred to as the reward pathway, is a dopaminergic pathway in the brain. The pathway connects the ventral tegmental area in the midbrain to the ventral striatum of the basal ganglia in the forebrain. The ventral striatum includes the nucleus accumbens and the olfactory tubercle.

Dopaminergic pathways Projection neurons in the brain that synthesize and release dopamine

Dopaminergic pathways, in the human brain are involved in both physiological and behavioral processes including movement, cognition, executive functions, reward, motivation, and neuroendocrine control. Each pathway is a set of projection neurons, consisting of individual dopamine neurons.

Ventral tegmental area Group of neurons on the floor of the midbrain

The ventral tegmental area (VTA), also known as the ventral tegmental area of Tsai, or simply ventral tegmentum, is a group of neurons located close to the midline on the floor of the midbrain. The VTA is the origin of the dopaminergic cell bodies of the mesocorticolimbic dopamine system and other dopamine pathways; it is widely implicated in the drug and natural reward circuitry of the brain. The VTA plays an important role in a number of processes, including reward cognition and orgasm, among others, as well as several psychiatric disorders. Neurons in the VTA project to numerous areas of the brain, ranging from the prefrontal cortex to the caudal brainstem and several regions in between.

Prefrontal cortex Part of brain largely responsible for personality, decision making, and social behaviour

In mammalian brain anatomy, the prefrontal cortex (PFC) is the cerebral cortex which covers the front part of the frontal lobe. The PFC contains the Brodmann areas BA8, BA9, BA10, BA11, BA12, BA13, BA14, BA24, BA25, BA32, BA44, BA45, BA46, and BA47.

Executive functions Cognitive processes necessary for control of behavior

Executive functions are a set of cognitive processes that are necessary for the cognitive control of behavior: selecting and successfully monitoring behaviors that facilitate the attainment of chosen goals. Executive functions include basic cognitive processes such as attentional control, cognitive inhibition, inhibitory control, working memory, and cognitive flexibility. Higher-order executive functions require the simultaneous use of multiple basic executive functions and include planning and fluid intelligence.

Medium spiny neuron Type of GABAergic neuron in the striatum

Medium spiny neurons (MSNs), also known as spiny projection neurons (SPNs), are a special type of GABAergic inhibitory cell representing 95% of neurons within the human striatum, a basal ganglia structure. Medium spiny neurons have two primary phenotypes : D1-type MSNs of the direct pathway and D2-type MSNs of the indirect pathway. Most striatal MSNs contain only D1-type or D2-type dopamine receptors, but a subpopulation of MSNs exhibit both phenotypes.

Leabra stands for local, error-driven and associative, biologically realistic algorithm. It is a model of learning which is a balance between Hebbian and error-driven learning with other network-derived characteristics. This model is used to mathematically predict outcomes based on inputs and previous learning influences. This model is heavily influenced by and contributes to neural network designs and models. This algorithm is the default algorithm in emergent when making a new project, and is extensively used in various simulations.

Reward system Group of neural structures responsible for motivation and desire

The reward system is a group of neural structures responsible for incentive salience, associative learning, and positively-valenced emotions, particularly ones involving pleasure as a core component. Reward is the attractive and motivational property of a stimulus that induces appetitive behavior, also known as approach behavior, and consummatory behavior. A rewarding stimulus has been described as "any stimulus, object, event, activity, or situation that has the potential to make us approach and consume it is by definition a reward". In operant conditioning, rewarding stimuli function as positive reinforcers; however, the converse statement also holds true: positive reinforcers are rewarding.

Synaptic gating

Synaptic gating is the ability of neural circuits to gate inputs by either suppressing or facilitating specific synaptic activity. Selective inhibition of certain synapses has been studied thoroughly, and recent studies have supported the existence of permissively gated synaptic transmission. In general, synaptic gating involves a mechanism of central control over neuronal output. It includes a sort of gatekeeper neuron, which has the ability to influence transmission of information to selected targets independently of the parts of the synapse upon which it exerts its action.

Blocqs disease Loss of memory of specialized movements causing the inability to maintain an upright posture

Blocq's disease was first considered by Paul Blocq (1860–1896), who described this phenomenon as the loss of memory of specialized movements causing the inability to maintain an upright posture, despite normal function of the legs in the bed. The patient is able to stand up, but as soon as the feet are on the ground, the patient cannot hold himself upright nor walk; however when lying down, the subject conserved the integrity of muscular force and the precision of movements of the lower limbs. The motivation of this study came when a fellow student Georges Marinesco (1864) and Paul published a case of parkinsonian tremor (1893) due to a tumor located in the substantia nigra.

The biology of obsessive–compulsive disorder (OCD) refers biologically based theories about the mechanism of OCD. Cognitive models generally fall into the category of executive dysfunction or modulatory control. Neuroanatomically, functional and structural neuroimaging studies implicate the prefrontal cortex (PFC), basal ganglia (BG), insula, and posterior cingulate cortex (PCC). Genetic and neurochemical studies implicate glutamate and monoamine neurotransmitters, especially serotonin and dopamine.

Inhibitory control Cognitive process

Inhibitory control, also known as response inhibition, is a cognitive process – and, more specifically, an executive function – that permits an individual to inhibit their impulses and natural, habitual, or dominant behavioral responses to stimuli in order to select a more appropriate behavior that is consistent with completing their goals. Self-control is an important aspect of inhibitory control. For example, successfully suppressing the natural behavioral response to eat cake when one is craving it while dieting requires the use of inhibitory control.

The 1-2-AX working memory task is a task which requires working memory to be solved. It can be used as a test case for learning algorithms to test their ability to remember some old data. This task can be used to demonstrate the working memory abilities of algorithms like PBWM or Long short-term memory.

A Bayesian Confidence Propagation Neural Network (BCPNN) is an artificial neural network inspired by Bayes' theorem, which regards neural computation and processing as probabilistic inference. Neural unit activations represent probability ("confidence") in the presence of input features or categories, synaptic weights are based on estimated correlations and the spread of activation corresponds to calculating posterior probabilities. It was originally proposed by Anders Lansner and Örjan Ekeberg at KTH Royal Institute of Technology. This probabilistic neural network model can also be run in generative mode to produce spontaneous activations and temporal sequences.

The neurocircuitry that underlies executive function processes and emotional and motivational processes are known to be distinct in the brain. However, there are brain regions that show overlap in function between the two cognitive systems. Brain regions that exist in both systems are interesting mainly for studies on how one system affects the other. Examples of such cross-modal functions are emotional regulation strategies such as emotional suppression and emotional reappraisal, the effect of mood on cognitive tasks, and the effect of emotional stimulation of cognitive tasks.

Cortico-basal ganglia-thalamo-cortical loop System of neural circuits in the brain

The cortico-basal ganglia-thalamo-cortical loop is a system of neural circuits in the brain. The loop involves connections between the cortex, the basal ganglia, the thalamus, and back to the cortex. It is of particular relevance to hyperkinetic and hypokinetic movement disorders, such as Parkinson's disease and Huntington's disease, as well as to mental disorders of control, such as attention deficit hyperactivity disorder (ADHD), obsessive–compulsive disorder (OCD), and Tourette syndrome.

References

  1. 1 2 3 O'Reilly, R.C & Frank, M.J. (2006). "Making Working Memory Work: A Computational Model of Learning in the Frontal Cortex and Basal Ganglia". Neural Computation. 18 (2): 283–328. doi:10.1162/089976606775093909. PMID   16378516. S2CID   8912485.
  2. Jeevanandam, Nivash (2021-09-13). "Underrated But Fascinating ML Concepts #5 – CST, PBWM, SARSA, & Sammon Mapping". Analytics India Magazine. Retrieved 2021-12-04.
  3. 1 2 "Leabra PBWM". CCNLab.