CoDi

Last updated
A CA-space with 64x64 cells in the signaling phase with the CoDi-model with axonal (red) signal trails and dendritic (green) and neuron bodies (white). CoDi CA signaling.png
A CA-space with 64x64 cells in the signaling phase with the CoDi-model with axonal (red) signal trails and dendritic (green) and neuron bodies (white).

CoDi is a cellular automaton (CA) model for spiking neural networks (SNNs). [1] CoDi is an acronym for Collect and Distribute, referring to the signals and spikes in a neural network.

Contents

CoDi uses a von Neumann neighborhood modified for a three-dimensional space; each cell looks at the states of its six orthogonal neighbors and its own state. In a growth phase a neural network is grown in the CA-space based on an underlying chromosome. There are four types of cells: neuron body, axon, dendrite and blank. The growth phase is followed by a signaling- or processing-phase. Signals are distributed from the neuron bodies via their axon tree and collected from connection dendrites. [1] These two basic interactions cover every case, and they can be expressed simply, using a small number of rules.

Cell interaction during signaling

Codi signaling: the arrows inside the axonal (red) signal trails and dendritic (green) signal trails indicate the direction of information flow during the signaling phase. CoDi CA signaling rules.png
Codi signaling: the arrows inside the axonal (red) signal trails and dendritic (green) signal trails indicate the direction of information flow during the signaling phase.

The neuron body cells collect neural signals from the surrounding dendritic cells and apply an internally defined function to the collected data. In the CoDi model the neurons sum the incoming signal values and fire after a threshold is reached. This behavior of the neuron bodies can be modified easily to suit a given problem. The output of the neuron bodies is passed on to its surrounding axon cells. Axonal cells distribute data originating from the neuron body. Dendritic cells collect data and eventually pass it to the neuron body. These two types of cell-to-cell interaction cover all kinds of cell encounters.

Every cell has a gate, which is interpreted differently depending on the type of the cell. A neuron cell uses this gate to store its orientation, i.e. the direction in which the axon is pointing. In an axon cell, the gate points to the neighbor from which the neural signals are received. An axon cell accepts input only from this neighbor, but makes its own output available to all its neighbors. In this way axon cells distribute information. The source of information is always a neuron cell. Dendritic cells collect information by accepting information from any neighbor. They give their output, (e.g. a Boolean OR operation on the binary inputs) only to the neighbor specified by their own gate. In this way, dendritic cells collect and sum neural signals, until the final sum of collected neural signals reaches the neuron cell.

Each axonal and dendritic cell belongs to exactly one neuron cell. This configuration of the CA-space is guaranteed by the preceding growth phase.

Synapses

The CoDi model does not use explicit synapses, because dendrite cells that are in contact with an axonal trail (i.e. have an axon cell as neighbor) collect the neural signals directly from the axonal trail. This results from the behavior of axon cells, which distribute to every neighbor, and from the behavior of the dendrite cells, which collect from any neighbor.

The strength of a neuron-neuron connection (a synapse) is represented by the number of their neighboring axon and dendrite cells. The exact structure of the network and the position of the axon-dendrite neighbor pairs determine the time delay and strength (weight) of a neuron-neuron connection. This principle infers that a single neuron-neuron connection can consist of several synapse with different time delays with independent weights.

Genetic encoding and growth of the network

CA-space with chromosome during growth phase with two randomly positioned neuron cells (white) with two dendrites and two axons each. (Right) The beginning of the growth phase, after three CA-steps. Codi chromosome start growth.png
CA-space with chromosome during growth phase with two randomly positioned neuron cells (white) with two dendrites and two axons each. (Right) The beginning of the growth phase, after three CA-steps.

The chromosome is initially distributed throughout the CA-space, so that every cell in the CA-space contains one instruction of the chromosome, i.e. one growth instruction, so that the chromosome belongs to the network as a whole. The distributed chromosome technique of the CoDi model makes maximum use of the available CA-space and enables the growth of any type of network connectivity. The local connection of the grown circuitry to its chromosome, allows local learning to be combined with the evolution of grown neural networks.

A growing neuron in the CoDi-model with two dendrites and two axons. The arrows inside the axonal and dendritic signal trails indicate the direction of information flow during the growth phase. Codi growth chromosome.png
A growing neuron in the CoDi-model with two dendrites and two axons. The arrows inside the axonal and dendritic signal trails indicate the direction of information flow during the growth phase.

Growth signals are passed to the direct neighbors of the neuron cell according to its chromosome information. The blank neighbors, which receive a neural growth signal, turn into either an axon cell or a dendrite cell. The growth signals include information containing the cell type of the cell that is to be grown from the signal. To decide in which directions axonal or dendritic trails should grow, the grown cells consult their chromosome information which encodes the growth instructions. These growth instructions can have an absolute or a relative directional encoding. An absolute encoding masks the six neighbors (i.e. directions) of a 3D cell with six bits. After a cell is grown, it accepts growth signals only from the direction from which it received its first signal. This reception direction information is stored in the gate position of each cell's state.

Implementation as a partitioned CA

State representation in the CoDi model. During the growth phase 6 of the bits are used to store the chromosome's growth instructions. The same 6 bits are later used to store the activity of a neuron cell during the signaling phase. Codi 16bit cell.png
State representation in the CoDi model. During the growth phase 6 of the bits are used to store the chromosome's growth instructions. The same 6 bits are later used to store the activity of a neuron cell during the signaling phase.

The states of our CAs have two parts, which are treated in different ways. The first part of the cell-state contains the cell's type and activity level and the second part serves as an interface to the cell's neighborhood by containing the input signals from the neighbors. Characteristic of our CA is that only part of the state of a cell is passed to its neighbors, namely the signal and then only to those neighbors specified in the fixed part of the cell state. This CA is called partitioned, because the state is partitioned into two parts, the first being fixed and the second is variable for each cell.

The advantage of this partitioning-technique is that the amount of information that defines the new state of a CA cell is kept to a minimum, due to its avoidance of redundant information exchange.

Implementation in hardware

Since CAs are only locally connected, they are ideal for implementation on purely parallel hardware. When designing the CoDi CA-based neural networks model, the objective was to implement them directly in hardware (FPGAs). Therefore, the CA was kept as simple as possible, by having a small number of bits to specify the state, keeping the CA rules few in number, and having few cellular neighbors.

The CoDi model was implemented in the FPGA based CAM-Brain Machine (CBM) by Korkin. [2]

History

CoDi was introduced by Gers et al. in 1998. [1] A specialized parallel machine based on FPGA Hardware (CAM) to run the CoDi model on a large scale was developed by Korkin et al. [2] De Garis conducted a series of experiments on the CAM-machine evaluating the CoDi model. The original model, where learning is based on evolutionary algorithms, has been augmented with a local learning rule via feedback from dendritic spikes by Schwarzer. [3]

See also

Related Research Articles

<span class="mw-page-title-main">Axon</span> Long projection on a neuron that conducts signals to other neurons

An axon or nerve fiber is a long, slender projection of a nerve cell, or neuron, in vertebrates, that typically conducts electrical impulses known as action potentials away from the nerve cell body. The function of the axon is to transmit information to different neurons, muscles, and glands. In certain sensory neurons, such as those for touch and warmth, the axons are called afferent nerve fibers and the electrical impulse travels along these from the periphery to the cell body and from the cell body to the spinal cord along another branch of the same axon. Axon dysfunction can be the cause of many inherited and acquired neurological disorders that affect both the peripheral and central neurons. Nerve fibers are classed into three types – group A nerve fibers, group B nerve fibers, and group C nerve fibers. Groups A and B are myelinated, and group C are unmyelinated. These groups include both sensory fibers and motor fibers. Another classification groups only the sensory fibers as Type I, Type II, Type III, and Type IV.

<span class="mw-page-title-main">Dendrite</span> Small projection on a neuron that receives signals

A dendrite or dendron is a branched protoplasmic extension of a nerve cell that propagates the electrochemical stimulation received from other neural cells to the cell body, or soma, of the neuron from which the dendrites project. Electrical stimulation is transmitted onto dendrites by upstream neurons via synapses which are located at various points throughout the dendritic tree.

<span class="mw-page-title-main">Neuron</span> Electrically excitable cell found in the nervous system of animals

Within a nervous system, a neuron, neurone, or nerve cell is an electrically excitable cell that fires electric signals called action potentials across a neural network. Neurons communicate with other cells via synapses, which are specialized connections that commonly use minute amounts of chemical neurotransmitters to pass the electric signal from the presynaptic neuron to the target cell through the synaptic gap.

The development of the nervous system, or neural development (neurodevelopment), refers to the processes that generate, shape, and reshape the nervous system of animals, from the earliest stages of embryonic development to adulthood. The field of neural development draws on both neuroscience and developmental biology to describe and provide insight into the cellular and molecular mechanisms by which complex nervous systems develop, from nematodes and fruit flies to mammals.

<span class="mw-page-title-main">Dendritic spine</span> Small protrusion on a dendrite that receives input from a single axon

A dendritic spine is a small membranous protrusion from a neuron's dendrite that typically receives input from a single axon at the synapse. Dendritic spines serve as a storage site for synaptic strength and help transmit electrical signals to the neuron's cell body. Most spines have a bulbous head, and a thin neck that connects the head of the spine to the shaft of the dendrite. The dendrites of a single neuron can contain hundreds to thousands of spines. In addition to spines providing an anatomical substrate for memory storage and synaptic transmission, they may also serve to increase the number of possible contacts between neurons. It has also been suggested that changes in the activity of neurons have a positive effect on spine morphology.

<span class="mw-page-title-main">Pyramidal cell</span> Projection neurons in the cerebral cortex and hippocampus

Pyramidal cells, or pyramidal neurons, are a type of multipolar neuron found in areas of the brain including the cerebral cortex, the hippocampus, and the amygdala. Pyramidal cells are the primary excitation units of the mammalian prefrontal cortex and the corticospinal tract. One of the main structural features of the pyramidal neuron is the conic shaped soma, or cell body, after which the neuron is named. Other key structural features of the pyramidal cell are a single axon, a large apical dendrite, multiple basal dendrites, and the presence of dendritic spines.

<span class="mw-page-title-main">Golgi's method</span>

Golgi's method is a silver staining technique that is used to visualize nervous tissue under light microscopy. The method was discovered by Camillo Golgi, an Italian physician and scientist, who published the first picture made with the technique in 1873. It was initially named the black reaction by Golgi, but it became better known as the Golgi stain or later, Golgi method.

Synaptogenesis is the formation of synapses between neurons in the nervous system. Although it occurs throughout a healthy person's lifespan, an explosion of synapse formation occurs during early brain development, known as exuberant synaptogenesis. Synaptogenesis is particularly important during an individual's critical period, during which there is a certain degree of synaptic pruning due to competition for neural growth factors by neurons and synapses. Processes that are not used, or inhibited during their critical period will fail to develop normally later on in life.

Holonomic brain theory is a branch of neuroscience investigating the idea that human consciousness is formed by quantum effects in or between brain cells. Holonomic refers to representations in a Hilbert phase space defined by both spectral and space-time coordinates. Holonomic brain theory is opposed by traditional neuroscience, which investigates the brain's behavior by looking at patterns of neurons and the surrounding chemistry.

An apical dendrite is a dendrite that emerges from the apex of a pyramidal cell. Apical dendrites are one of two primary categories of dendrites, and they distinguish the pyramidal cells from spiny stellate cells in the cortices. Pyramidal cells are found in the prefrontal cortex, the hippocampus, the entorhinal cortex, the olfactory cortex, and other areas. Dendrite arbors formed by apical dendrites are the means by which synaptic inputs into a cell are integrated. The apical dendrites in these regions contribute significantly to memory, learning, and sensory associations by modulating the excitatory and inhibitory signals received by the pyramidal cells.

A neurite or neuronal process refers to any projection from the cell body of a neuron. This projection can be either an axon or a dendrite. The term is frequently used when speaking of immature or developing neurons, especially of cells in culture, because it can be difficult to tell axons from dendrites before differentiation is complete.

Neuromorphology is the study of nervous system form, shape, and structure. The study involves looking at a particular part of the nervous system from a molecular and cellular level and connecting it to a physiological and anatomical point of view. The field also explores the communications and interactions within and between each specialized section of the nervous system. Morphology is distinct from morphogenesis. Morphology is the study of the shape and structure of biological organisms, while morphogenesis is the study of the biological development of the shape and structure of organisms. Therefore, neuromorphology focuses on the specifics of the structure of the nervous system and not the process by which the structure was developed. Neuromorphology and morphogenesis, while two different entities, are nonetheless closely linked.

The synaptotropic hypothesis, also called the synaptotrophic hypothesis, is a neurobiological hypothesis of neuronal growth and synapse formation. The hypothesis was first formulated by J.E. Vaughn in 1988, and remains a focus of current research efforts. The synaptotropic hypothesis proposes that input from a presynaptic to a postsynaptic cell eventually can change the course of synapse formation at dendritic and axonal arbors. This synapse formation is required for the development of neuronal structure in the functioning brain.

<span class="mw-page-title-main">Synaptic pruning</span> Process of synapse elimination that occurs between early childhood and the onset of puberty

Synaptic pruning, a phase in the development of the nervous system, is the process of synapse elimination that occurs between early childhood and the onset of puberty in many mammals, including humans. Pruning starts near the time of birth and continues into the late-20s. During pruning, both the axon and dendrite decay and die off. It was traditionally considered to be complete by the time of sexual maturation, but this was discounted by MRI studies.

Neural backpropagation is the phenomenon in which, after the action potential of a neuron creates a voltage spike down the axon, another impulse is generated from the soma and propagates towards the apical portions of the dendritic arbor or dendrites. In addition to active backpropagation of the action potential, there is also passive electrotonic spread. While there is ample evidence to prove the existence of backpropagating action potentials, the function of such action potentials and the extent to which they invade the most distal dendrites remain highly controversial.

<span class="mw-page-title-main">Dendritic spike</span> Action potential generated in the dendrite of a neuron

In neurophysiology, a dendritic spike refers to an action potential generated in the dendrite of a neuron. Dendrites are branched extensions of a neuron. They receive electrical signals emitted from projecting neurons and transfer these signals to the cell body, or soma. Dendritic signaling has traditionally been viewed as a passive mode of electrical signaling. Unlike its axon counterpart which can generate signals through action potentials, dendrites were believed to only have the ability to propagate electrical signals by physical means: changes in conductance, length, cross sectional area, etc. However, the existence of dendritic spikes was proposed and demonstrated by W. Alden Spencer, Eric Kandel, Rodolfo Llinás and coworkers in the 1960s and a large body of evidence now makes it clear that dendrites are active neuronal structures. Dendrites contain voltage-gated ion channels giving them the ability to generate action potentials. Dendritic spikes have been recorded in numerous types of neurons in the brain and are thought to have great implications in neuronal communication, memory, and learning. They are one of the major factors in long-term potentiation.

<span class="mw-page-title-main">Nonsynaptic plasticity</span> Form of neuroplasticity

Nonsynaptic plasticity is a form of neuroplasticity that involves modification of ion channel function in the axon, dendrites, and cell body that results in specific changes in the integration of excitatory postsynaptic potentials and inhibitory postsynaptic potentials. Nonsynaptic plasticity is a modification of the intrinsic excitability of the neuron. It interacts with synaptic plasticity, but it is considered a separate entity from synaptic plasticity. Intrinsic modification of the electrical properties of neurons plays a role in many aspects of plasticity from homeostatic plasticity to learning and memory itself. Nonsynaptic plasticity affects synaptic integration, subthreshold propagation, spike generation, and other fundamental mechanisms of neurons at the cellular level. These individual neuronal alterations can result in changes in higher brain function, especially learning and memory. However, as an emerging field in neuroscience, much of the knowledge about nonsynaptic plasticity is uncertain and still requires further investigation to better define its role in brain function and behavior.

<span class="mw-page-title-main">Neuronal self-avoidance</span>

Neuronal self-avoidance, or isoneural avoidance, is an important property of neurons which consists in the tendency of branches arising from a single soma to turn away from one another. The arrangements of branches within neuronal arbors are established during development and result in minimal crossing or overlap as they spread over a territory, resulting in the typical fasciculated morphology of neurons.

Dendrodendritic synapses are connections between the dendrites of two different neurons. This is in contrast to the more common axodendritic synapse (chemical synapse) where the axon sends signals and the dendrite receives them. Dendrodendritic synapses are activated in a similar fashion to axodendritic synapses in respects to using a chemical synapse. An incoming action potential permits the release of neurotransmitters to propagate the signal to the post synaptic cell. There is evidence that these synapses are bi-directional, in that either dendrite can signal at that synapse. Ordinarily, one of the dendrites will display inhibitory effects while the other will display excitatory effects. The actual signaling mechanism utilizes Na+ and Ca2+ pumps in a similar manner to those found in axodendritic synapses.

<span class="mw-page-title-main">Neurotubule</span>

Neurotubules are microtubules found in neurons in nervous tissues. Along with neurofilaments and microfilaments, they form the cytoskeleton of neurons. Neurotubules are undivided hollow cylinders that are made up of tubulin protein polymers and arrays parallel to the plasma membrane in neurons. Neurotubules have an outer diameter of about 23 nm and an inner diameter, also known as the central core, of about 12 nm. The wall of the neurotubules is about 5 nm in width. There is a non-opaque clear zone surrounding the neurotubule and it is about 40 nm in diameter. Like microtubules, neurotubules are greatly dynamic and the length of them can be adjusted by polymerization and depolymerization of tubulin.

References

  1. 1 2 3 Gers, Felix; Hugo Garis; Michael Korkin (1998). "CoDi-1Bit : A simplified cellular automata based neuron model" . Artificial Evolution. Lecture Notes in Computer Science. Vol. 1363. pp.  315–333. CiteSeerX   10.1.1.2.17 . doi:10.1007/BFb0026610. ISBN   978-3-540-64169-8.
  2. 1 2 de Garis, Hugo; Michael Korkin; Gary Fehr (2001). "The CAM-Brain Machine (CBM): An FPGA Based Tool for Evolving a 75 Million Neuron Artificial Brain to Control a Lifesized Kitten Robot". Autonomous Robots. 10 (3): 235–249. doi:10.1023/A:1011286308522. ISSN   0929-5593. S2CID   28589336.
  3. ↑ Schwarzer, Jens; Müller-Schloer, Christian (2004-08-05). Lernverfahren für evolutionär optimierte Künstliche Neuronale Netze auf der Basis Zellulärer Automaten. Logos Verlag Berlin. pp. 125–. ISBN   9783832506285 . Retrieved 7 January 2013.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.