Filling-in

Last updated
When steadily fixating the central dot for many seconds, the peripheral annulus will fade and will be replaced by the colour or texture of the background. Troxler fading.svg
When steadily fixating the central dot for many seconds, the peripheral annulus will fade and will be replaced by the colour or texture of the background.

In vision, filling-in phenomena are those responsible for the completion of missing information across the physiological blind spot, and across natural and artificial scotomata. There is also evidence for similar mechanisms of completion in normal visual analysis. Classical demonstrations of perceptual filling-in involve filling in at the blind spot in monocular vision, and images stabilized on the retina either by means of special lenses, or under certain conditions of steady fixation. For example, naturally in monocular vision at the physiological blind spot, the percept is not a hole in the visual field, but the content is “filled-in” based on information from the surrounding visual field. When a textured stimulus is presented centered on but extending beyond the region of the blind spot, a continuous texture is perceived. This partially inferred percept is paradoxically considered more reliable than a percept based on external input. (Ehinger et al. 2017).

Contents

A second type of example relates to entirely stabilized stimuli. Their colour and lightness fade until they are no longer seen and the area fills in with the colour and lightness of the surrounding region. A famous example of fading under steady fixation is Troxler's fading. When steadily fixating on the central dot for many seconds, the peripheral annulus will fade and will be replaced by the colour or texture of the background. Since the adapted region is actively filled-in with background colour or texture, the phenomenon cannot be fully explained by local processes such as adaptation.

There is general agreement that edges play a central role in determining the apparent colour and lightness of surfaces through similar filling-in mechanisms. However, the way in which their influence is performed is still unclear. Two different theories have been put forward to explain the filling-in completion phenomenon.

One theory, addressed as the "isomorphic filling-in theory" according to the definition of Von der Heydt, Friedman et al. (2003), postulates that perception is based on an image representation held in a two dimensional array of neurons, typically arranged retinotopically, in which colour signals spread in all directions except across borders formed by contour activity. The process is thought to be analogous to physical diffusion, with contours acting as diffusion barriers for the colour and brightness signals. An alternative hypothesis is that image information is transformed at the cortical level into an oriented feature representation. Form and colour would be derived at a subsequent stage, not as the result of an isomorphic filling-in process, but as an attribute of an object or proto-object. This theory is called the symbolic filling-in theory.

According to the isomorphic filling-in theory, colour is represented by the activity of cells whose receptive fields point at the surface, but it is assumed that these cells receive additional activation through horizontal connections that keeps their activity level high despite mechanisms of lateral inhibition tending to suppress surface activity and despite the transient nature of the afferent signals. The lateral activation comes from receptive fields at contrast borders. These signals are strong because receptive fields are exposed to contrast, and reliable because the border produces continuous light modulation even during fixation, due to small residual eye movements. In the alternative symbolic hypothesis, there is no spreading of activity, but all the information would be carried by the relevant features, that would be tagged with information on contrast polarity, colour and lightness of the surfaces they enclose. Despite the many attempts to verify the two different models by psychophysical and physiological experiments, the mechanisms of colour and lightness filling-in are still debated.

Isomorphic filling-in

There are at least three different kinds of experiments whose results support the idea that a real spreading of neural activity in early visual areas is the basis for filling-in of visual information.

Recordings from cells of the blind spot representation in monkey striate cortex

Komatsu and colleagues (Komatsu et al., 2000) recorded activity of cells of the blind spot representation in monkey striate cortex (area V1) and found some cells, in layers 4–6, that responded to large stimuli covering the blind spot (the condition under which filling-in is perceived), but not to small stimuli near the blind spot. A neuronal circuitry seems to exist that elaborates and transmits colour and brightness information through the blind region.

Though intriguing, these results cannot be easily generalized to similar phenomena, such as the filling-in of illusory contours or the filling-in through artificial scotomata or adapted edges (such as in the Troxler's effect). All these phenomena are indeed similar, and probably rely on similar neural circuitries but they are not identical. For instance, an obvious difference between filling-in across the blind spot and filling-in of occluded edges is that filling-in across the blind spot is modal (i.e. you literally see the filled-in section), while filling-in across occluders is amodal. Filling-in across the blind spot was found to be different also from filling-in across cortical scotomata in two patients examined by V. S. Ramachandran (Ramachandran 1992; Ramachandran, Gregory et al. 1993). In these subjects, some features filled in the scotoma faster than others, and in some circumstances filling-in took some seconds before it was completed (while filling-in across the blind spot is immediate). Together these data suggested that mechanisms for the filling-in of colours, motion and texture can be dissociated and may correspond to processes in higher-order areas that are specialized for these attributes.

Delayed masking psychophysical experiments

Striking evidence implying a spreading of neural activity like the one postulated by isomorphic filling-in theory is given by experiments of backward masking after brief presentations of uniform surfaces or textures. The working hypothesis of these experiments is that if a response initially biased toward the boundaries fills-in to represent the interiors of uniform surfaces, it may be possible to interfere with the filling-in process and leave the percept at an incomplete stage.

Paradiso and Nakayama (1991) performed an experiment to verify this hypothesis. They presented a large disk of uniform brightness on a black background. The stimulus was briefly flashed and, after a variable stimulus offset asynchrony, a masking stimulus was presented. The mask consisted of a circle on a black background with the masking contours positioned within the boundaries of the large uniform disk. This experiment is grounded on the assumption that filling-in consists of a spreading of neural activity from the boundaries of luminance and through the surfaces, that is stopped when another luminance-contrast border is reached (this is proposed by many models of brightness perception, see for example Walls 1954, Gerrits and Vendrik 1970, Cohen and Grossberg 1984), and that the process takes some time to be completed.

Subjects were asked to match the brightness at the centre of the disk with a palette of grey scales. When the delay between target and mask presentation was long enough, the mask had no effect on the apparent brightness of the stimulus, but for stimulus offset asynchronies of 50–100 ms, the surface of the disk inside the masking annulus appeared unfilled. Moreover, the minimum target-mask delay at which the masking was effective increased with target size, suggesting that there would be a spreading phenomenon and that the farther the features delimiting a region, the more time is necessary for the filling-in to be completed. These results are supported also by further experiments on temporal limits of brightness induction in simultaneous contrast (De Valois, Webster et al. 1986; Rossi and Paradiso 1996; Rossi, Rittenhouse et al. 1996), as well as by a similar experiment performed by Motoyoshi (1999) on filling-in of texture.

Recordings from striate cortex of cat

An isomorphic filling-in theory calls for the existence of surface responsive neurons in early retinotopic visual areas. The activity of such neurons would be raised by elements capable of responding to the luminance of the surface also in the absence of edges; and would be strongly modulated by spreading of activity from the luminance borders enclosing the surface.

Electro-physiological recordings in retinal ganglion cells, LGN and primary visual cortex showed that neurons of these areas responded to luminance modulation within the receptive field even in the absence of contrast borders.

In a second condition, a uniform grey patch was placed on the receptive field (extending 3–5 degrees beyond the receptive field boundary on either side), and two flanking patches modulated sinusoidally in time from dark to light. With such stimuli, the brightness of the central patch appears to modulate, despite the absence of luminance change. In this condition, cat retinal ganglion cells and lateral geniculate nucleus cells, having their receptive fields centred in the uniform grey patch, did not respond; on the other hand, primary visual cortex neurons were modulated by luminance changes far outside their receptive fields. Together, these results suggest that neurons in the retina and LGN are responsive to luminance modulation, but their response does not correlate with perceived brightness. On the other hand, striate neurons responded to stimulus conditions producing changes in brightness in the area corresponding to the receptive field.

The behaviour of primary visual cortex neurons seems to be in agreement with the one hypothesized by an isomorphic filling-in theory in that they both respond to luminance of the surfaces also in the absence of borders, and their activity is modulated by that of edges far outside the receptive field. Moreover, when the temporal frequency of luminance modulation in the surrounding patches exceeded a threshold value, the induced response disappeared, suggesting that it was the result of a spreading of activity, taking a finite time to happen, likely explainable in the context of isomorphic filling-in.

Symbolic filling-in

"Perceptual filling-in", in its simplest definition, is simply the filling-in of information that is not directly given to the sensory input. The missing information is inferred or extrapolated from visual data acquired in a different part of the visual field. Examples of filling-in phenomena include lightness assignment to surfaces from information of contrast across the edges and completion of features and textures across the blind spot, based on the features and textures that are detected in the visible part of the image. In this definition, it is clear that a filling-in process involves a rearrangement of visual information, in which activity in one region of the visual field (i.e. edges) is assigned to other regions (surfaces). In any event, the total amount of information available is not increased, being determined by the retinal input, and any rearrangement of information is useful only if it brings the information contained in the image into a form that is more easily analyzed by our brain.

Dennett and Kinsbourne (Dennett 1992; Dennett and Kinsbourne 1992) opposed to the idea that an active filling-in process would take place in our brain on philosophical grounds. They argued that such an idea would be the result of the false belief that in our brain there is a spectator, a sort of homunculus similar to ourselves, needing a filled-in image representation. From a scientific viewpoint, Dennett's homunculus may correspond to higher-order scene representation or decision-making mechanisms. The question is whether or not such mechanisms need a filled in, gap free representation of the image to function optimally (Ramachandran 2003).

The symbolic filling-in theory postulates that such a "homunculus" need not exist, and that image information is transformed at the cortical level into an oriented feature representation. Surface form and colour are not coded at this stage, but would be derived only at a symbolic level of representation, as attributes of objects or proto-objects.

Electrophysiology recordings from monkey primary visual cortex

When experiencing perceptual filling-in, the colour of an adapted surface is gradually replaced by the colour or texture of outside the surface. Friedman et al. (Friedman 1998; Friedman, Zhou et al. 1999) performed an experiment aimed at determining if surface activity of cells in monkey primary visual cortex changed in accordance with perceptual change or simply followed the modulation of the colour presented to the retina. Stimuli consisted of a disk-ring configuration similar to that illustrating the Troxler effect, but where the inner and outer part of the annulus have two physically different colours. After a few seconds of (peripheral) fixation, the disk tends to disappear, whereas the outer contour of the ring is perceived much longer, and the area of the disk is filled-in with the colour of the ring (Krauskopf 1967). These stimuli where intermixed with control stimuli, in which the physical colour of the disk was gradually changed to that of the ring. The animals were instructed to signal a colour change, and their responses to control stimuli and to test stimuli were compared in order to determine if monkeys perceive colour filling-in under steady fixation like humans.

The authors recorded the activity of surface- and edge-cells (cells whose receptive fields pointed either to the filled-in surface or to the border between the disk and the ring) in the visual cortices V1 and V2 while the monkey was performing the filling-in task. The activity of surface-cells correlated with the physical stimulus change in both areas V1 and V2, but not with the perceived colour change induced by filling-in. The activity of edge-cells followed the stimulus contrast when the disk colour changed physically; when the colours were constant, the edge signals also decayed, but more slowly. Together, the data was incompatible with the isomorphic filling-in theory, which assumes that colour signals spread from the borders into uniform regions.

fMRI experiments in human subjects

The neuronal activity in different brain areas can be recorded in humans through non-invasive techniques, like fMRI (functional magnetic resonance imaging). Perna et al. (2005) used fMRI to investigate the neuronal mechanisms responsible for the Craik–O'Brien–Cornsweet illusion. These authors recorded the activity in different brain areas when observers were presented with a Cornsweet visual stimulus, and compared the activities with those elicited by a similar image, which however did not elicit any brightness filling-in.

Contrary to the predictions of isomorphic filling-in, these authors found an identical response to the stimulus that induced filling-in and to the control stimulus in early visual cortex.

Recently, Cornelissen et al. (2006) performed a similar experiment involving the simultaneous contrast illusion. These authors presented observers in an fMRI scan with simultaneous contrast stimuli composed of a central circle of uniform luminance and a peripheral region whose luminance was modulated in time (and also tested other conditions where the modulated and the constant regions were inverted). The brain activity was recorded in primary visual cortex in a retinotopic position corresponding to the perceptually filled-in region. Also in this condition, no activity was found at this level in response to the filled-in signal.

See also

Related Research Articles

<span class="mw-page-title-main">Perception</span> Interpretation of sensory information

Perception is the organization, identification, and interpretation of sensory information in order to represent and understand the presented information or environment. All perception involves signals that go through the nervous system, which in turn result from physical or chemical stimulation of the sensory system. Vision involves light striking the retina of the eye; smell is mediated by odor molecules; and hearing involves pressure waves.

<span class="mw-page-title-main">Visual cortex</span> Region of the brain that processes visual information

The visual cortex of the brain is the area of the cerebral cortex that processes visual information. It is located in the occipital lobe. Sensory input originating from the eyes travels through the lateral geniculate nucleus in the thalamus and then reaches the visual cortex. The area of the visual cortex that receives the sensory input from the lateral geniculate nucleus is the primary visual cortex, also known as visual area 1 (V1), Brodmann area 17, or the striate cortex. The extrastriate areas consist of visual areas 2, 3, 4, and 5.

<span class="mw-page-title-main">Sensory nervous system</span> Part of the nervous system

The sensory nervous system is a part of the nervous system responsible for processing sensory information. A sensory system consists of sensory neurons, neural pathways, and parts of the brain involved in sensory perception and interoception. Commonly recognized sensory systems are those for vision, hearing, touch, taste, smell, balance and visceral sensation. Sense organs are transducers that convert data from the outer physical world to the realm of the mind where people interpret the information, creating their perception of the world around them.

The receptive field, or sensory space, is a delimited medium where some physiological stimuli can evoke a sensory neuronal response in specific organisms.

<span class="mw-page-title-main">Motion perception</span> Inferring the speed and direction of objects

Motion perception is the process of inferring the speed and direction of elements in a scene based on visual, vestibular and proprioceptive inputs. Although this process appears straightforward to most observers, it has proven to be a difficult problem from a computational perspective, and difficult to explain in terms of neural processing.

Multisensory integration, also known as multimodal integration, is the study of how information from the different sensory modalities may be integrated by the nervous system. A coherent representation of objects combining modalities enables animals to have meaningful perceptual experiences. Indeed, multisensory integration is central to adaptive behavior because it allows animals to perceive a world of coherent perceptual entities. Multisensory integration also deals with how different sensory modalities interact with one another and alter each other's processing.

<span class="mw-page-title-main">Illusory contours</span> Visual illusions

Illusory contours or subjective contours are visual illusions that evoke the perception of an edge without a luminance or color change across that edge. Illusory brightness and depth ordering often accompany illusory contours. Friedrich Schumann is often credited with the discovery of illusory contours around the beginning of the 20th century, but they are present in art dating to the Middle Ages. Gaetano Kanizsa’s 1976 Scientific American paper marked the resurgence of interest in illusory contours for vision scientists.

<span class="mw-page-title-main">Chubb illusion</span> Optical illusion

The Chubb illusion is an optical illusion or error in visual perception in which the apparent contrast of an object varies substantially to most viewers depending on its relative contrast to the field on which it is displayed. These visual illusions are of particular interest to researchers because they may provide valuable insights in regard to the workings of human visual systems.

Repetition priming refers to improvements in a behavioural response when stimuli are repeatedly presented. The improvements can be measured in terms of accuracy or reaction time and can occur when the repeated stimuli are either identical or similar to previous stimuli. These improvements have been shown to be cumulative, so as the number of repetitions increases the responses get continually faster up to a maximum of around seven repetitions. These improvements are also found when the repeated items are changed slightly in terms of orientation, size and position. The size of the effect is also modulated by the length of time the item is presented for and the length time between the first and subsequent presentations of the repeated items.

Flash suppression is a phenomenon of visual perception in which an image presented to one eye is suppressed by a flash of another image presented to the other eye.

<span class="mw-page-title-main">Motion-induced blindness</span> Optical illusion

Motion Induced Blindness (MIB), also known as Bonneh's illusion is a visual illusion in which a large, continuously moving pattern erases from perception some small, continuously presented, stationary dots when one looks steadily at the center of the display. It was discovered by Bonneh, Cooperman, and Sagi (2001), who used a swarm of blue dots moving on a virtual sphere as the larger pattern and three small yellow dots as the smaller pattern. They found that after about 10 seconds, one or more of the dots disappeared for brief, random times.

<span class="mw-page-title-main">Neural correlates of consciousness</span> Neuronal events sufficient for a specific conscious percept

The neural correlates of consciousness (NCC) are the minimal set of neuronal events and mechanisms sufficient for the occurrence of the mental states to which they are related. Neuroscientists use empirical approaches to discover neural correlates of subjective phenomena; that is, neural changes which necessarily and regularly correlate with a specific experience. The set should be minimal because, under the materialist assumption that the brain is sufficient to give rise to any given conscious experience, the question is which of its components are necessary to produce it.

Feature detection is a process by which the nervous system sorts or filters complex natural stimuli in order to extract behaviorally relevant cues that have a high probability of being associated with important objects or organisms in their environment, as opposed to irrelevant background or noise.

Transsaccadic memory is the neural process that allows humans to perceive their surroundings as a seamless, unified image despite rapid changes in fixation points. Transsaccadic memory is a relatively new topic of interest in the field of psychology. Conflicting views and theories have spurred several types of experiments intended to explain transsaccadic memory and the neural mechanisms involved.

<span class="mw-page-title-main">Phantom contour</span> Type of illusory contour

A phantom contour is a type of illusory contour. Most illusory contours are seen in still images, such as the Kanizsa triangle and the Ehrenstein illusion. A phantom contour, however, is perceived in the presence of moving or flickering images with contrast reversal. The rapid, continuous alternation between opposing, but correlated, adjacent images creates the perception of a contour that is not physically present in the still images. Quaid et al. have also authored a PhD thesis on the phantom contour illusion and its spatiotemporal limits which maps out limits and proposes mechanisms for its perception centering around magnocellularly driven visual area MT.

<span class="mw-page-title-main">Visual tilt effects</span>

Due to the effect of a spatial context or temporal context, the perceived orientation of a test line or grating pattern can appear tilted away from its physical orientation. The tilt illusion (TI) is the phenomenon that the perceived orientation of a test line or grating is altered by the presence of surrounding lines or grating with a different orientation. And the tilt aftereffect (TAE) is the phenomenon that the perceived orientation is changed after prolonged inspection of another oriented line or grating.

Biased competition theory advocates the idea that each object in the visual field competes for cortical representation and cognitive processing. This theory suggests that the process of visual processing can be biased by other mental processes such as bottom-up and top-down systems which prioritize certain features of an object or whole items for attention and further processing. Biased competition theory is, simply stated, the competition of objects for processing. This competition can be biased, often toward the object that is currently attended in the visual field, or alternatively toward the object most relevant to behavior.

Binocular neurons are neurons in the visual system that assist in the creation of stereopsis from binocular disparity. They have been found in the primary visual cortex where the initial stage of binocular convergence begins. Binocular neurons receive inputs from both the right and left eyes and integrate the signals together to create a perception of depth.

Surround suppression is where the relative firing rate of a neuron may under certain conditions decrease when a particular stimulus is enlarged. It has been observed in electrophysiology studies of the brain and has been noted in many sensory neurons, most notably in the early visual system. Surround suppression is defined as a reduction in the activity of a neuron in response to a stimulus outside its classical receptive field.

<span class="mw-page-title-main">Russell L. De Valois</span>

Russell L. De Valois was an American scientist recognized for his pioneering research on spatial and color vision.

References