Ideal observer analysis

Last updated

Ideal observer analysis is a method for investigating how information is processed in a perceptual system. [1] [2] [3] It is also a basic principle that guides modern research in perception. [4] [5]

Contents

The ideal observer is a theoretical system that performs a specific task in an optimal way. If there is uncertainty in the task, then perfect performance is impossible and the ideal observer will make errors.

Ideal performance is the theoretical upper limit of performance. It is theoretically impossible for a real system to perform better than ideal. Typically, real systems are only capable of sub-ideal performance.

This technique is useful for analyzing psychophysical data (see psychophysics).

Definition

Many definitions of this term have been offered.

Geisler (2003) [6] (slightly reworded): The central concept in ideal observer analysis is the ideal observer, a theoretical device that performs a given task in an optimal fashion given the available information and some specified constraints. This is not to say that ideal observers perform without error, but rather that they perform at the physical limit of what is possible in the situation. The fundamental role of uncertainty and noise implies that ideal observers must be defined in probabilistic (statistical) terms. Ideal observer analysis involves determining the performance of the ideal observer in a given task and then comparing its performance to that of a real perceptual system, which (depending on the application) might be the system as a whole, a subsystem, or an elementary component of the system (e.g. a neuron).

Sequential ideal observer analysis

In sequential ideal observer analysis, [7] the goal is to measure a real system's performance deficit (relative to ideal) at different processing stages. Such an approach is useful when studying systems that process information in discrete (or semi-discrete) stages or modules.

Natural and pseudo-natural tasks

To facilitate experimental design in the laboratory, an artificial task may be designed so that the system's performance in the task may be studied. If the task is too artificial, the system may be pushed away from a natural mode of operation. Depending on the goals of the experiment, this may diminish its external validity.

In such cases, it may be important to keep the system operating naturally (or almost naturally) by designing a pseudo-natural task. Such tasks are still artificial, but they attempt to mimic the natural demands placed on a system. For example, the task might employ stimuli that resemble natural scenes and might test the system's ability to make potentially useful judgments about these stimuli.

Natural scene statistics are the basis for calculating ideal performance in natural and pseudo-natural tasks. This calculation tends to incorporate elements of signal detection theory, information theory, or estimation theory.

Normally distributed stimuli

Das and Geisler [8] described and computed the detection and classification performance of ideal observers when the stimuli are normally distributed. These include the error rate and confusion matrix for ideal observers when the stimuli come from two or more univariate or multivariate normal distributions (i.e. yes/no, two-interval, multi-interval tasks and general multi-category classification tasks), the discriminability index of the ideal observer (Bayes discriminability index) and its relation to the receiver operating characteristic.

Notes

  1. ↑ Tanner Jr, Wilson P.; Birdsall, T. G. (1958). "Definitions of d′ and η as Psychophysical Measures". Journal of the Acoustical Society of America . 30 (10): 922–928. doi:10.1121/1.1909408. Archived from the original on February 26, 2013. Retrieved August 19, 2012.
  2. ↑ Tanner Jr, W. P.; Jones, R. Clark (1960). "The ideal sensor system as approached through statistical decision theory and the theory of signal detectability". Visual search techniques: proceedings of a symposium, held in the Smithsonian Auditorium, Washington, D. C., April 7 and 8, 1959. United States National Academies. pp. 59–68. Retrieved August 19, 2012.
  3. ↑ W. P. Tanner Jr. (1961). "Physiological implications of psychophysical data" (PDF). Annals of the New York Academy of Sciences . 89 (5): 752–65. doi:10.1111/j.1749-6632.1961.tb20176.x. hdl: 2027.42/73966 . PMID   13775211. S2CID   7135400.
  4. ↑ Knill, David C.; Whitman, Richards (1996). Perception as Bayesian Inference. Cambridge University Press. ISBN   9780521461092 . Retrieved August 19, 2012.
  5. ↑ Pelli, D. G. (1993). "The quantum efficiency of vision". In Blakemore, Colin (ed.). Vision: Coding and Efficiency. Cambridge University Press. pp. 3–24. ISBN   9780521447690 . Retrieved August 19, 2012.
  6. ↑ Geisler, Wilson S. (2003). "Ideal Observer Analysis". In Chalupa, Leo M.; Werner, John S. (eds.). The Visual Neurosciences. MIT Press. pp. 825–837. ISBN   9780262033084 . Retrieved August 19, 2012.
  7. ↑ W. S. Geisler (1989). "Sequential ideal-observer analysis of visual discriminations". Psychological Review . 96 (2): 267–314. doi:10.1037/0033-295x.96.2.267. PMID   2652171.
  8. ↑ Das, Abhranil; Geisler, Wilson (2020). "A method to integrate and classify normal distributions". arXiv: 2012.14331 [stat.ML].

Related Research Articles

Attention Psychological process of selectively concentrating on a discrete aspect of information

Attention is the behavioral and cognitive process of selectively concentrating on a discrete aspect of information, whether considered subjective or objective, while ignoring other perceivable information. William James (1890) wrote that "Attention is the taking possession by the mind, in clear and vivid form, of one out of what seem several simultaneously possible objects or trains of thought. Focalization, concentration, of consciousness are of its essence." Attention has also been described as the allocation of limited cognitive processing resources. Attention is manifested by an attentional bottleneck, in term of the amount of data the brain can process each second; for example, in human vision, only less than 1% of the visual input data can enter the bottleneck, leading to inattentional blindness.

Categorization is the ability and activity to recognize shared features or similarities between the elements of the experience of the world, organizing and classifying experience by associating them to a more abstract group, on the basis of their traits, features, similarities or other criteria. Categorization is considered one of the most fundamental cognitive abilities, and as such it is studied particularly by psychology and cognitive linguistics.

Computer science is the study of the theoretical foundations of information and computation and their implementation and application in computer systems. One well known subject classification system for computer science is the ACM Computing Classification System devised by the Association for Computing Machinery.

Psychophysics

Psychophysics quantitatively investigates the relationship between physical stimuli and the sensations and perceptions they produce. Psychophysics has been described as "the scientific study of the relation between stimulus and sensation" or, more completely, as "the analysis of perceptual processes by studying the effect on a subject's experience or behaviour of systematically varying the properties of a stimulus along one or more physical dimensions".

In the study of vision, visual short-term memory (VSTM) is one of three broad memory systems including iconic memory and long-term memory. VSTM is a type of short-term memory, but one limited to information within the visual domain.

Detection theory or signal detection theory is a means to measure the ability to differentiate between information-bearing patterns and random patterns that distract from the information. In the field of electronics, the separation of such patterns from a disguising background is referred to as signal recovery.

The kappa effect or perceptual time dilation is a temporal perceptual illusion that can arise when observers judge the elapsed time between sensory stimuli applied sequentially at different locations. In perceiving a sequence of consecutive stimuli, subjects tend to overestimate the elapsed time between two successive stimuli when the distance between the stimuli is sufficiently large, and to underestimate the elapsed time when the distance is sufficiently small.

Filling-in

In vision, filling-in phenomena are those responsible for the completion of missing information across the physiological blind spot, and across natural and artificial scotomata. There is also evidence for similar mechanisms of completion in normal visual analysis. Classical demonstrations of perceptual filling-in involve filling in at the blind spot in monocular vision, and images stabilized on the retina either by means of special lenses, or under certain conditions of steady fixation. For example, naturally in monocular vision at the physiological blind spot, the percept is not a hole in the visual field, but the content is “filled-in” based on information from the surrounding visual field. When a textured stimulus is presented centered on but extending beyond the region of the blind spot, a continuous texture is perceived. This partially inferred percept is paradoxically considered more reliable than a percept based on external input..

Ideal tasks arise during task analysis. Ideal tasks are different from real tasks. They are ideals in the Platonic sense of a circle being an ideal whereas a drawn circle is flawed and real. The study of theoretically best or “mathematically ideal” tasks, has been the basis of the branch of stimulus control in psychology called Psychophysics as well as being part of Artificial Intelligence. Such studies include the instantiation of such ideal tasks in the real world. The notion of the ideal task has also played an important role in information theory. Tasks are defined as sequences of contingencies, each presenting stimuli and requiring an action or a sequence of actions to occur in some non-arbitrary fashion. These contingencies may not only provide stimuli that require the discrimination of relations among actions and events but among task actions themselves. Again, Task actions, E, are actions that are required to complete tasks. Properties of tasks are varied, and responses to them can be measured and analyzed.

Two-alternative forced choice (2AFC) is a method for measuring the sensitivity of a person, child or infant, or animal to some particular sensory input, stimulus, through that observer's pattern of choices and response times to two versions of the sensory input. For example, to determine a person's sensitivity to dim light, the observer would be presented with a series of trials in which a dim light was randomly either in the top or bottom of the display. After each trial, the observer responds "top" or "bottom". The observer is not allowed to say "I do not know", or "I am not sure", or "I did not see anything". In that sense the observer's choice is forced between the two alternatives.

Scene statistics is a discipline within the field of perception. It is concerned with the statistical regularities related to scenes. It is based on the premise that a perceptual system is designed to interpret scenes.

Basic science (psychology)

Some of the research that is conducted in the field of psychology is more "fundamental" than the research conducted in the applied psychological disciplines, and does not necessarily have a direct application. The subdisciplines within psychology that can be thought to reflect a basic-science orientation include biological psychology, cognitive psychology, neuropsychology, and so on. Research in these subdisciplines is characterized by methodological rigor. The concern of psychology as a basic science is in understanding the laws and processes that underlie behavior, cognition, and emotion. Psychology as a basic science provides a foundation for applied psychology. Applied psychology, by contrast, involves the application of psychological principles and theories yielded up by the basic psychological sciences; these applications are aimed at overcoming problems or promoting well-being in areas such as mental and physical health and education.

Perceptual learning

Perceptual learning is learning better perception skills such as differentiating two musical tones from one another or categorizations of spatial and temporal patterns relevant to real-world expertise. Examples of this may include reading, seeing relations among chess pieces, and knowing whether or not an X-ray image shows a tumor.

In modern psychology, vigilance, also termed sustained concentration, is defined as the ability to maintain concentrated attention over prolonged periods of time. During this time, the person attempts to detect the appearance of a particular target stimulus. The individual watches for a signal stimulus that may occur at an unknown time.

In the psychology of perception and motor control, the term response priming denotes a special form of priming. Generally, priming effects take place whenever a response to a target stimulus is influenced by a prime stimulus presented at an earlier time. The distinctive feature of response priming is that prime and target are presented in quick succession and are coupled to identical or alternative motor responses. When a speeded motor response is performed to classify the target stimulus, a prime immediately preceding the target can thus induce response conflicts when assigned to a different response as the target. These response conflicts have observable effects on motor behavior, leading to priming effects, e.g., in response times and error rates. A special property of response priming is its independence from visual awareness of the prime.

The Troland Research Awards are an annual prize given by the United States National Academy of Sciences to two researchers in recognition of psychological research on the relationship between consciousness and the physical world. The areas where these award funds are to be spent include but are not limited to areas of experimental psychology, the topics of sensation, perception, motivation, emotion, learning, memory, cognition, language, and action. The award preference is given to experimental work with a quantitative approach or experimental research seeking physiological explanations.

Haptic memory is the form of sensory memory specific to touch stimuli. Haptic memory is used regularly when assessing the necessary forces for gripping and interacting with familiar objects. It may also influence one's interactions with novel objects of an apparently similar size and density. Similar to visual iconic memory, traces of haptically acquired information are short lived and prone to decay after approximately two seconds. Haptic memory is best for stimuli applied to areas of the skin that are more sensitive to touch. Haptics involves at least two subsystems; cutaneous, or everything skin related, and kinesthetic, or joint angle and the relative location of body. Haptics generally involves active, manual examination and is quite capable of processing physical traits of objects and surfaces.

In cognitive psychology, intertrial priming is an accumulation of the priming effect over multiple trials, where "priming" is the effect of the exposure to one stimulus on subsequently presented stimuli. Intertrial priming occurs when a target feature is repeated from one trial to the next, and typically results in speeded response times to the target. A target is the stimulus participants are required to search for. For example, intertrial priming occurs when the task is to respond to either a red or a green target, and the response time to a red target is faster if the preceding trial also has a red target.

Perceptual load theory is a psychological theory of attention. It was presented by Nilli Lavie in the mid-nineties as a potential resolution to the early/late selection debate.

Human performance modeling (HPM) is a method of quantifying human behavior, cognition, and processes. It is a tool used by human factors researchers and practitioners for both the analysis of human function and for the development of systems designed for optimal user experience and interaction. It is a complementary approach to other usability testing methods for evaluating the impact of interface features on operator performance.

This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.