Three-factor learning

Last updated

In neuroscience and machine learning, three-factor learning is the combinaison of Hebbian plasticity with a third modulatory factor to stabilise and enhance synaptic learning. [1] This third factor can represent various signals such as reward, punishment, error, surprise, or novelty, often implemented through neuromodulators. [2]

Description

Three-factor learning introduces the concept of eligibility traces, which flag synapses for potential modification pending the arrival of the third factor, and helps temporal credit assignement by bridging the gap between rapid neuronal firing and slower behavioral timescales, from which learning can be done. [3] Biological basis for Three-factor learning rules have been supported by experimental evidence. [4] [2] This approach addresses the instability of classical Hebbian learning by minimizing autocorrelation and maximizing cross-correlation between inputs. [1]

Related Research Articles

Computational neuroscience is a branch of neuroscience which employs mathematics, computer science, theoretical analysis and abstractions of the brain to understand the principles that govern the development, structure, physiology and cognitive abilities of the nervous system.

Hebbian theory is a neuropsychological theory claiming that an increase in synaptic efficacy arises from a presynaptic cell's repeated and persistent stimulation of a postsynaptic cell. It is an attempt to explain synaptic plasticity, the adaptation of brain neurons during the learning process. It was introduced by Donald Hebb in his 1949 book The Organization of Behavior. The theory is also called Hebb's rule, Hebb's postulate, and cell assembly theory. Hebb states it as follows:

Let us assume that the persistence or repetition of a reverberatory activity tends to induce lasting cellular changes that add to its stability. ... When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A’s efficiency, as one of the cells firing B, is increased.

Spike-timing-dependent plasticity (STDP) is a biological process that adjusts the strength of connections between neurons in the brain. The process adjusts the connection strengths based on the relative timing of a particular neuron's output and input action potentials. The STDP process partially explains the activity-dependent development of nervous systems, especially with regard to long-term potentiation and long-term depression.

Neuroplasticity, also known as neural plasticity or brain plasticity, is the ability of neural networks in the brain to change through growth and reorganization. It is when the brain is rewired to function in some way that differs from how it previously functioned. These changes range from individual neuron pathways making new connections, to systematic adjustments like cortical remapping or neural oscillation. Other forms of neuroplasticity include homologous area adaptation, cross modal reassignment, map expansion, and compensatory masquerade. Examples of neuroplasticity include circuit and network changes that result from learning a new ability, information acquisition, environmental influences, pregnancy, caloric intake, practice/training, and psychological stress.

Neural coding is a neuroscience field concerned with characterising the hypothetical relationship between the stimulus and the neuronal responses, and the relationship among the electrical activities of the neurons in the ensemble. Based on the theory that sensory and other information is represented in the brain by networks of neurons, it is believed that neurons can encode both digital and analog information.

<span class="mw-page-title-main">Spiking neural network</span> Artificial neural network that mimics neurons

Spiking neural networks (SNNs) are artificial neural networks (ANN) that more closely mimic natural neural networks. These models leverage timing of discrete spikes as the main information carrier.

Pendleton Read Montague, Jr. is an American neuroscientist and popular science author. He is the director of the Human Neuroimaging Lab and Computational Psychiatry Unit at the Fralin Biomedical Research Institute at VTC in Roanoke, Virginia, where he also holds the title of the inaugural Virginia Tech Carilion Vernon Mountcastle Research Professor. Montague is also a professor in the department of physics at Virginia Tech in Blacksburg, Virginia and professor of Psychiatry and Behavioral Medicine at Virginia Tech Carilion School of Medicine.

Bienenstock–Cooper–Munro (BCM) theory, BCM synaptic modification, or the BCM rule, named after Elie Bienenstock, Leon Cooper, and Paul Munro, is a physical theory of learning in the visual cortex developed in 1981. The BCM model proposes a sliding threshold for long-term potentiation (LTP) or long-term depression (LTD) induction, and states that synaptic plasticity is stabilized by a dynamic adaptation of the time-averaged postsynaptic activity. According to the BCM model, when a pre-synaptic neuron fires, the post-synaptic neurons will tend to undergo LTP if it is in a high-activity state, or LTD if it is in a lower-activity state. This theory is often used to explain how cortical neurons can undergo both LTP or LTD depending on different conditioning stimulus protocols applied to pre-synaptic neurons.

In neuroscience and computer science, synaptic weight refers to the strength or amplitude of a connection between two nodes, corresponding in biology to the amount of influence the firing of one neuron has on another. The term is typically used in artificial and biological neural network research.

A Bayesian Confidence Propagation Neural Network (BCPNN) is an artificial neural network inspired by Bayes' theorem, which regards neural computation and processing as probabilistic inference. Neural unit activations represent probability ("confidence") in the presence of input features or categories, synaptic weights are based on estimated correlations and the spread of activation corresponds to calculating posterior probabilities. It was originally proposed by Anders Lansner and Örjan Ekeberg at KTH Royal Institute of Technology. This probabilistic neural network model can also be run in generative mode to produce spontaneous activations and temporal sequences.

The network of the human nervous system is composed of nodes that are connected by links. The connectivity may be viewed anatomically, functionally, or electrophysiologically. These are presented in several Wikipedia articles that include Connectionism, Biological neural network, Artificial neural network, Computational neuroscience, as well as in several books by Ascoli, G. A. (2002), Sterratt, D., Graham, B., Gillies, A., & Willshaw, D. (2011), Gerstner, W., & Kistler, W. (2002), and David Rumelhart, McClelland, J. L., and PDP Research Group (1986) among others. The focus of this article is a comprehensive view of modeling a neural network. Once an approach based on the perspective and connectivity is chosen, the models are developed at microscopic, mesoscopic, or macroscopic (system) levels. Computational modeling refers to models that are developed using computing tools.

Many experiments have been done to find out how the brain interprets stimuli and how animals develop fear responses. The emotion, fear, has been hard-wired into almost every individual, due to its vital role in the survival of the individual. Researchers have found that fear is established unconsciously and that the amygdala is involved with fear conditioning.

In neuroscience, predictive coding is a theory of brain function which postulates that the brain is constantly generating and updating a "mental model" of the environment. According to the theory, such a mental model is used to predict input signals from the senses that are then compared with the actual input signals from those senses. Predictive coding is member of a wider set of theories that follow the Bayesian brain hypothesis.

<span class="mw-page-title-main">Kathryn Mary Murphy</span> Canadian neuroscientist

Kathryn Mary Murphy is a Canadian neuroscientist and professor who studies development and plasticity of the brain.

Claudia Clopath is a Professor of Computational Neuroscience at Imperial College London and research leader at the Sainsbury Wellcome Centre for Neural Circuits and Behaviour. She develops mathematical models to predict synaptic plasticity for both medical applications and the design of human-like machines.

Tim P. Vogels is a professor of theoretical neuroscience and research leader at the Institute of Science and Technology Austria. He is primarily known for his scholarly contributions to the study of neuronal plasticity related to learning and memory in the brain.

Ilana B. Witten is an American neuroscientist and professor of psychology and neuroscience at Princeton University. Witten studies the mesolimbic pathway, with a focus on the striatal neural circuit mechanisms driving reward learning and decision making.

<span class="mw-page-title-main">Wulfram Gerstner</span> German neuroscientist

Wulfram Gerstner is a German and Swiss computational neuroscientist. His research focuses on neural spiking patterns in neural networks, and their connection to learning, spatial representation and navigation. Since 2006 Gerstner has been a full professor of Computer Science and Life Sciences at École Polytechnique Fédérale de Lausanne (EPFL), where he also serves as a Director of the Laboratory of Computational Neuroscience.

The spike response model (SRM) is a spiking neuron model in which spikes are generated by either a deterministic or a stochastic threshold process. In the SRM, the membrane voltage V is described as a linear sum of the postsynaptic potentials (PSPs) caused by spike arrivals to which the effects of refractoriness and adaptation are added. The threshold is either fixed or dynamic. In the latter case it increases after each spike. The SRM is flexible enough to account for a variety of neuronal firing pattern in response to step current input. The SRM has also been used in the theory of computation to quantify the capacity of spiking neural networks; and in the neurosciences to predict the subthreshold voltage and the firing times of cortical neurons during stimulation with a time-dependent current stimulation. The name Spike Response Model points to the property that the two important filters and of the model can be interpreted as the response of the membrane potential to an incoming spike (response kernel , the PSP) and to an outgoing spike (response kernel , also called refractory kernel). The SRM has been formulated in continuous time and in discrete time. The SRM can be viewed as a generalized linear model (GLM) or as an (integrated version of) a generalized integrate-and-fire model with adaptation.

<span class="mw-page-title-main">Alexei Koulakov</span>

Alexei Koulakov is a theoretical physicist and a neuroscientist. He is the Charles Robertson Professor of Neuroscience at the Cold Spring Harbor Laboratory.

References

  1. 1 2 Porr, Bernd; Wörgötter, Florentin (October 2007). "Learning with "Relevance": Using a Third Factor to Stabilize Hebbian Learning". Neural Computation. 19 (10): 2694–2719. doi:10.1162/neco.2007.19.10.2694. ISSN   0899-7667. PMID   17716008.
  2. 1 2 Gerstner, Wulfram; Lehmann, Marco; Liakoni, Vasiliki; Corneil, Dane; Brea, Johanni (2018-07-31). "Eligibility Traces and Plasticity on Behavioral Time Scales: Experimental Support of NeoHebbian Three-Factor Learning Rules". Frontiers in Neural Circuits. 12: 53. doi: 10.3389/fncir.2018.00053 . ISSN   1662-5110. PMC   6079224 . PMID   30108488.
  3. Frémaux, Nicolas; Gerstner, Wulfram (2016-01-19). "Neuromodulated Spike-Timing-Dependent Plasticity, and Theory of Three-Factor Learning Rules". Frontiers in Neural Circuits. 9: 85. doi: 10.3389/fncir.2015.00085 . ISSN   1662-5110. PMC   4717313 . PMID   26834568.
  4. Kuśmierz, Łukasz; Isomura, Takuya; Toyoizumi, Taro (2017-10-01). "Learning with three factors: modulating Hebbian plasticity with errors". Current Opinion in Neurobiology. Computational Neuroscience. 46: 170–177. doi: 10.1016/j.conb.2017.08.020 . ISSN   0959-4388. PMID   28918313.