Novelty detection

Last updated

Novelty detection is the mechanism by which an intelligent organism is able to identify an incoming sensory pattern as being hitherto unknown. If the pattern is sufficiently salient or associated with a high positive or strong negative utility, it will be given computational resources for effective future processing.

The principle is long known in neurophysiology, with roots in the orienting response research by E. N. Sokolov [1] in the 1950s. The reverse phenomenon is habituation, i.e., the phenomenon that known patterns yield a less marked response. Early neural modeling attempts were by Yehuda Salu. [2] An increasing body of knowledge has been collected concerning the corresponding mechanisms in the brain. [3] [4] In technology, the principle became important for radar detection methods during the Cold War, where unusual aircraft-reflection patterns could indicate an attack by a new type of aircraft. Today, the phenomenon plays an important role in machine learning and data science, where the corresponding methods are known as anomaly detection or outlier detection. An extensive methodological overview is given by Markou and Singh. [5] [6]

See also

Related Research Articles

<span class="mw-page-title-main">Nervous system</span> Part of an animal that coordinates actions and senses

In biology, the nervous system is the highly complex part of an animal that coordinates its actions and sensory information by transmitting signals to and from different parts of its body. The nervous system detects environmental changes that impact the body, then works in tandem with the endocrine system to respond to such events. Nervous tissue first arose in wormlike organisms about 550 to 600 million years ago. In vertebrates it consists of two main parts, the central nervous system (CNS) and the peripheral nervous system (PNS). The CNS consists of the brain and spinal cord. The PNS consists mainly of nerves, which are enclosed bundles of the long fibers, or axons, that connect the CNS to every other part of the body. Nerves that transmit signals from the brain are called motor nerves or efferent nerves, while those nerves that transmit information from the body to the CNS are called sensory nerves or afferent. Spinal nerves are mixed nerves that serve both functions. The PNS is divided into three separate subsystems, the somatic, autonomic, and enteric nervous systems. Somatic nerves mediate voluntary movement. The autonomic nervous system is further subdivided into the sympathetic and the parasympathetic nervous systems. The sympathetic nervous system is activated in cases of emergencies to mobilize energy, while the parasympathetic nervous system is activated when organisms are in a relaxed state. The enteric nervous system functions to control the gastrointestinal system. Both autonomic and enteric nervous systems function involuntarily. Nerves that exit from the cranium are called cranial nerves while those exiting from the spinal cord are called spinal nerves.

<span class="mw-page-title-main">Attention</span> Psychological process of selectively perceiving and prioritising discrete aspects of information

Attention is the concentration of awareness on some phenomenon to the exclusion of other stimuli. It is a process of selectively concentrating on a discrete aspect of information, whether considered subjective or objective. William James (1890) wrote that "Attention is the taking possession by the mind, in clear and vivid form, of one out of what seem several simultaneously possible objects or trains of thought. Focalization, concentration, of consciousness are of its essence." Attention has also been described as the allocation of limited cognitive processing resources. Attention is manifested by an attentional bottleneck, in terms of the amount of data the brain can process each second; for example, in human vision, only less than 1% of the visual input data can enter the bottleneck, leading to inattentional blindness.

The development of the nervous system, or neural development (neurodevelopment), refers to the processes that generate, shape, and reshape the nervous system of animals, from the earliest stages of embryonic development to adulthood. The field of neural development draws on both neuroscience and developmental biology to describe and provide insight into the cellular and molecular mechanisms by which complex nervous systems develop, from nematodes and fruit flies to mammals.

<span class="mw-page-title-main">Phi phenomenon</span> Optical illusion of apparent motion

The term phi phenomenon is used in a narrow sense for an apparent motion that is observed if two nearby optical stimuli are presented in alternation with a relatively high frequency. In contrast to beta movement, seen at lower frequencies, the stimuli themselves do not appear to move. Instead, a diffuse, amorphous shadowlike something seems to jump in front of the stimuli and occlude them temporarily. This shadow seems to have nearly the color of the background. Max Wertheimer first described this form of apparent movement in his habilitation thesis, published 1912, marking the birth of Gestalt psychology.

<span class="mw-page-title-main">Olfactory bulb</span> Neural structure

The olfactory bulb is a neural structure of the vertebrate forebrain involved in olfaction, the sense of smell. It sends olfactory information to be further processed in the amygdala, the orbitofrontal cortex (OFC) and the hippocampus where it plays a role in emotion, memory and learning. The bulb is divided into two distinct structures: the main olfactory bulb and the accessory olfactory bulb. The main olfactory bulb connects to the amygdala via the piriform cortex of the primary olfactory cortex and directly projects from the main olfactory bulb to specific amygdala areas. The accessory olfactory bulb resides on the dorsal-posterior region of the main olfactory bulb and forms a parallel pathway. Destruction of the olfactory bulb results in ipsilateral anosmia, while irritative lesions of the uncus can result in olfactory and gustatory hallucinations.

Habituation is a form of non-associative learning in which an innate (non-reinforced) response to a stimulus decreases after repeated or prolonged presentations of that stimulus. Responses that habituate include those that involve the intact organism or those that involve only components of the organism. The broad ubiquity of habituation across all biologic phyla has resulted in it being called "the simplest, most universal form of learning...as fundamental a characteristic of life as DNA." Functionally-speaking, by diminishing the response to an inconsequential stimulus, habituation is thought to free-up cognitive resources to other stimuli that are associated with biologically important events. For example, organisms may habituate to repeated sudden loud noises when they learn these have no consequences. A progressive decline of a behavior in a habituation procedure may also reflect nonspecific effects such as fatigue, which must be ruled out when the interest is in habituation. Habituation is clinically relevant, as a number of neuropsychiatric conditions, including autism, schizophrenia, migraine, and Tourette's, show reductions in habituation to a variety of stimulus-types both simple (tone) and complex (faces).

<span class="mw-page-title-main">Place cell</span> Place-activated hippocampus cells found in some mammals

A place cell is a kind of pyramidal neuron in the hippocampus that becomes active when an animal enters a particular place in its environment, which is known as the place field. Place cells are thought to act collectively as a cognitive representation of a specific location in space, known as a cognitive map. Place cells work with other types of neurons in the hippocampus and surrounding regions to perform this kind of spatial processing. They have been found in a variety of animals, including rodents, bats, monkeys and humans.

<span class="mw-page-title-main">Motion perception</span> Inferring the speed and direction of objects

Motion perception is the process of inferring the speed and direction of elements in a scene based on visual, vestibular and proprioceptive inputs. Although this process appears straightforward to most observers, it has proven to be a difficult problem from a computational perspective, and difficult to explain in terms of neural processing.

The orienting response (OR), also called orienting reflex, is an organism's immediate response to a change in its environment, when that change is not sudden enough to elicit the startle reflex. The phenomenon was first described by Russian physiologist Ivan Sechenov in his 1863 book Reflexes of the Brain, and the term was coined by Ivan Pavlov, who also referred to it as the Shto takoye? reflex. The orienting response is a reaction to novel or significant stimuli. In the 1950s the orienting response was studied systematically by the Russian scientist Evgeny Sokolov, who documented the phenomenon called "habituation", referring to a gradual "familiarity effect" and reduction of the orienting response with repeated stimulus presentations.

Schaffer collaterals are axon collaterals given off by CA3 pyramidal cells in the hippocampus. These collaterals project to area CA1 of the hippocampus and are an integral part of memory formation and the emotional network of the Papez circuit, and of the hippocampal trisynaptic loop. It is one of the most studied synapses in the world and named after the Hungarian anatomist-neurologist Károly Schaffer.

<span class="mw-page-title-main">Neural binding</span>

Neural binding is the neuroscientific aspect of what is commonly known as the binding problem: the interdisciplinary difficulty of creating a comprehensive and verifiable model for the unity of consciousness. "Binding" refers to the integration of highly diverse neural information in the forming of one's cohesive experience. The neural binding hypothesis states that neural signals are paired through synchronized oscillations of neuronal activity that combine and recombine to allow for a wide variety of responses to context-dependent stimuli. These dynamic neural networks are thought to account for the flexibility and nuanced response of the brain to various situations. The coupling of these networks is transient, on the order of milliseconds, and allows for rapid activity.

<span class="mw-page-title-main">Olga Vinogradova</span> Russian neuroscientist

Professor Olga S. Vinogradova (1929–2001) was a specialist in Russian cognitive neuroscience. In 1969 she founded the Laboratory of Systemic Organization of Neurons in the Institute of Biological Physics, Russian Academy of Sciences (Pushchino) and headed this Laboratory till the end of her life.

Neural coding is a neuroscience field concerned with characterising the hypothetical relationship between the stimulus and the individual or ensemble neuronal responses and the relationship among the electrical activity of the neurons in the ensemble. Based on the theory that sensory and other information is represented in the brain by networks of neurons, it is thought that neurons can encode both digital and analog information.

The mismatch negativity (MMN) or mismatch field (MMF) is a component of the event-related potential (ERP) to an odd stimulus in a sequence of stimuli. It arises from electrical activity in the brain and is studied within the field of cognitive neuroscience and psychology. It can occur in any sensory system, but has most frequently been studied for hearing and for vision, in which case it is abbreviated to vMMN. The (v)MMN occurs after an infrequent change in a repetitive sequence of stimuli For example, a rare deviant (d) stimulus can be interspersed among a series of frequent standard (s) stimuli. In hearing, a deviant sound can differ from the standards in one or more perceptual features such as pitch, duration, loudness, or location. The MMN can be elicited regardless of whether someone is paying attention to the sequence. During auditory sequences, a person can be reading or watching a silent subtitled movie, yet still show a clear MMN. In the case of visual stimuli, the MMN occurs after an infrequent change in a repetitive sequence of images.

<span class="mw-page-title-main">Anomaly detection</span> Approach in data analysis

In data analysis, anomaly detection is generally understood to be the identification of rare items, events or observations which deviate significantly from the majority of the data and do not conform to a well defined notion of normal behaviour. Such examples may arouse suspicions of being generated by a different mechanism, or appear inconsistent with the remainder of that set of data.

Neurorobotics is the combined study of neuroscience, robotics, and artificial intelligence. It is the science and technology of embodied autonomous neural systems. Neural systems include brain-inspired algorithms, computational models of biological neural networks and actual biological systems. Such neural systems can be embodied in machines with mechanic or any other forms of physical actuation. This includes robots, prosthetic or wearable systems but also, at smaller scale, micro-machines and, at the larger scales, furniture and infrastructures.

Coincidence detection is a neuronal process in which a neural circuit encodes information by detecting the occurrence of temporally close but spatially distributed input signals. Coincidence detectors influence neuronal information processing by reducing temporal jitter and spontaneous activity, allowing the creation of variable associations between separate neural events in memory. The study of coincidence detectors has been crucial in neuroscience with regards to understanding the formation of computational maps in the brain.

Feature detection is a process by which the nervous system sorts or filters complex natural stimuli in order to extract behaviorally relevant cues that have a high probability of being associated with important objects or organisms in their environment, as opposed to irrelevant background or noise.

<span class="mw-page-title-main">Catastrophic interference</span> AIs tendency to abruptly & drastically forget old info after learning new info

Catastrophic interference, also known as catastrophic forgetting, is the tendency of an artificial neural network to abruptly and drastically forget previously learned information upon learning new information. Neural networks are an important part of the network approach and connectionist approach to cognitive science. With these networks, human capabilities such as memory and learning can be modeled using computer simulations.

During every moment of an organism's life, sensory information is being taken in by sensory receptors and processed by the nervous system. Sensory information is stored in sensory memory just long enough to be transferred to short-term memory. Humans have five traditional senses: sight, hearing, taste, smell, touch. Sensory memory (SM) allows individuals to retain impressions of sensory information after the original stimulus has ceased. A common demonstration of SM is a child's ability to write letters and make circles by twirling a sparkler at night. When the sparkler is spun fast enough, it appears to leave a trail which forms a continuous image. This "light trail" is the image that is represented in the visual sensory store known as iconic memory. The other two types of SM that have been most extensively studied are echoic memory, and haptic memory; however, it is reasonable to assume that each physiological sense has a corresponding memory store. For example, children have been shown to remember specific "sweet" tastes during incidental learning trials but the nature of this gustatory store is still unclear. However, sensory memories might be related to a region of the thalamus, which serves as a source of signals encoding past experiences in the neocortex.

References

  1. Sokolov, E.N. (1960). "Neuronal models and the orienting reflex". The Central Nervous System and Behavior. Josiah Macy, Jr. Foundation. pp. 187–276. OCLC   222201512.
  2. Salu, Y. (1988). "Models of neural novelty detectors, with similarities to cerebral cortex". BioSystems. 21 (2): 99–113. doi:10.1016/0303-2647(88)90003-2. PMID   3355886.
  3. Tiitinen, H.; May, P.; Reinikainen, K.; Näätänen, R. (1994). "Attentive novelty detection in humans is governed by pre-attentive sensory memory". Nature. 372 (6501): 90–92. Bibcode:1994Natur.372...90T. doi:10.1038/372090a0. PMID   7969425. S2CID   4255887.
  4. Duncan, K.; Ketz, N.; Inati, S.J.; Davachi, L. (2012). "Evidence for area CA1 as a match/mismatch detector: A high-resolution fMRI study of the human hippocampus". Hippocampus. 22 (3): 389–398. doi:10.1002/hipo.20933. PMC   3529001 . PMID   21484934.
  5. Markou, M.; Singh, S. (2003). "Novelty detection: a review — Part 1: statistical approaches". Signal Processing. 83 (12): 2481–97. doi:10.1016/j.sigpro.2003.07.018. S2CID   17490415.
  6. Markou, M.; Singh, S. (2003). "Novelty detection: a review — Part 2: neural network based approaches". Signal Processing. 83 (12): 2499–2521. doi:10.1016/j.sigpro.2003.07.019.