Workshop 6: Information Processing in the Visual System: Abstracts and Lecture Materials
In area V1 and other visual cortical areas, the response of neurons to stimuli within the classical receptive field (RF) are modulated by stimuli outside the classical RF (in the "surround"). Surround modulation is likely to be the neural correlate of context-dependent perception. We have investigated the neural circuits underlying surround modulation in macaque V1.
In a first set of studies, we quantitatively compared the spatial dimensions of V1 neurons RF center and surround with the visuotopic extent of geniculocortical feedforward, intra-V1 horizontal and extrastriate feedback connections to V1. We found that the surround of V1 neurons can extend well beyond the monosynaptic range of feedforward (Angelucci & Sainsbury, J Comp. Neurol. 2006) and horizontal connections (Angelucci et al., J. Neurosci. 2002), but well within the range of feedback connections (Angelucci et al., J. Neurosci. 2002). These results and evidence from other laboratories on the slow propagation of signals along horizontal axons, suggested that feedforward connections and polysynaptic chains of horizontal connections cannot underlie the fast onset of modulation arising from the "far" surround of V1 neurons. Feedback connections are thus the most likely substrate for far surround modulation. Feedforward and horizontal connections, however, are likely to underlie "near" surround modulation.
We have used an anatomically and physiologically constrained recurrent network model of macaque V1, to investigate the relative contribution of horizontal and feedback connections to the size tuning properties of V1 neurons, and to surround modulation in V1 (Schwabe et al., J. Neurosci. 2006). In the model, both horizontal and feedback connections contribute to contrast-dependent increases in RF size and to near surround modulation, whereas feedback connections underlie far surround modulation. To account for the anatomical observation that excitatory feedback axons target almost exclusively excitatory neurons in V1, in our model feedback neurons in the far surround exert their suppressive influence via contacts with excitatory neurons in the near surround sending monosynaptic horizontal connections to excitatory and inhibitory neurons in the RF center. Our model can account for a wide range of physiological data and generates several testable predictions. We are currently working to incorporate surround suppression of LGN afferents into our network model, so as to investigate the specific contribution of feedforward connections to surround modulation in V1.
A central prediction of our feedback model is that the "suppressive" far surround of V1 neurons can be facilitatory under conditions that weakly activate neurons in the RF center. We have tested this model prediction (Ichida et al., 2007) by recording the response of single neurons in macaque V1 to a center-surround stimulus in which the surround grating does not directly activate feedforward and monosynaptic horizontal connections to the RF center. This stimulus allowed us to isolate the modulatory signals arising from the far surround. As predicted by the model, we found iso-orientation far surround facilitation when the RF center was driven by a low-contrast stimulus, and the far surround by a small annular stimulus. Suppression occurred when the center stimulus contrast or the size of the surround stimulus were increased. This suggests that center-surround interactions result from excitatory and inhibitory mechanisms of similar spatial extent, and changes in their balance determine whether facilitation or suppression occurs.
In layer 4C, the main target of geniculocortical afferents which lacks long-range intra-cortical connections, far surround facilitation was rare and large surround fields (suppressive and facilitatory) were absent. In contrast, near surround facilitation was strongest in layer 4C and other layers of V1 that receive direct LGN input. This suggests that while feedforward connections may contribute to near surround facilitation, they do not contribute to far surround facilitation or suppression; the latter is thus generated by intra-cortical mechanisms, likely involving top-down feedback.
Work done in collaboration with J.S. Lund, J.B.Levitt, L. Schwabe, J.M. Ichida, and S. Shushruth.
Many receptive fields in lateral geniculate nucleus (LGN) are biphasic in time, i.e. a bright (dark) excitatory phase is followed by a dark (bright) excitatory phase. We describe a hierarchical model of predictive coding and simulations that capture these changing neuronal response properties. The model is composed of two areas, resembling the LGN and primary visual cortex (V1). Model V1 attempts to predict its LGN inputs, while neurons in LGN signal the difference between actual input and the V1 predictions. After training on natural images, model V1 receptive fields resemble simple cell receptive fields. In addition, the spatio-temporal response profile of LGN model neurons is biphasic in structure, resembling the biphasic response structure of neurons in cat LGN.
Furthermore we show that special care must be taken in using signed line codings in such circuits. The biophysics of nerve cells does not allow for a straightforward implementation of simple signed algebraic relationships, but these can still be implemented if sufficient care is taken.
The model predicts a specific pattern of influence of feedback, where LGN receptive fields that are aligned over a simple cell receptive field zone of the same polarity decrease their responses and neurons of opposite polarity increase their responses due to feedback. This phase-reversed pattern of influence was recently confirmed in neurophysiology. These results corroborate the idea that predictive feedback is a general coding strategy in the brain.
Primate visual cortex contains multiple functionally specialized areas which are linked by feedforward (FF) and feedback (FB) connections within a hierarchical network. We have found that mouse visual cortex has a similar organization in that it contains at least ten visuotopically organized areas which are interconnected by reciprocal FF and FB circuits. Our anatomical studies have shown that interareal FF and FB connections are made almost exclusively by pyramidal neurons which in the target area form excitatory synapses onto pyramidal cells and GABAergic interneurons. Both pathways provide 12-15% of inputs to interneurons, whereas the rest goes to pyramidal cells. Whole cell patch clamp recordings in acute slices have further shown that FF and FB circuits contain similar distributions of fast spiking (45-48%), regular spiking nonpyramidal cells (31-33%), irregular spiking (4-16%) and low threshold spiking (6-7%) neurons and that these cells provide disynaptic 'feedforward' inhibition to pyramidal cells. Each of these cell types receives approximately 10 to 16 FF and FB inputs. Despite these similarities, we have found that FB inputs generate smaller fast GABA-A receptor-mediated postsynaptic inhibitory currents than FF inputs. One reason for this is that FB synapses are more strongly depressing than FF synapses and that they deplete more quickly during repetitive stimulation. Another reason is that fast spiking neurons in the FB pathway have a more positive spike threshold (-36 mV) than in the FF pathway (-44 mV), which may decrease the number of simultaneously active interneurons and lower the inhibitory output of the FB circuit. In addition to the pathway-differences in fast synaptic inhibition, we found that slow GABA-B receptor-mediated inhibition is much more powerful in the FF than in the FB circuit. Strong GABA-B receptor-mediated inhibition was also found after stimulation of thalamocortical (TC) afferents and horizontal connections (HC) within V1. As a result of the strong slow inhibition, FF, TC and HC circuits generate much less recurrent polysynaptic excitation than FB circuits.
Contextual information is represented by influences of the receptive field surround on neuronal firing evoked by visual stimulation of the receptive field center. It is thought that surround inputs represent modulatory HC and/or FB influences on TC responses. We have simulated these interactions by co-activating HC+TC and FB+TC inputs in cortical slices. The results show that in both conditions the combined inputs produce stronger monosynaptic excitation, but the combined responses were always smaller that the arithmetic sum of the separate responses. In contrast, polysynaptic excitation evoked by co-activation of FB+TC was larger than the arithmetic sum, whereas recurrent excitation due to stimulation of HC+TC was suppressed by strong activation of GABA-B receptor-mediated slow inhibition. These results suggest that the reported suppressive and facilitating effects of the receptive field surround on its center are due to diverse 'feedforward' inhibitory mechanisms in TC, HC, FF and FB circuits.
We will discuss data showing how salient tuning properties of V1 cells are achieved, including contrast-invariant orientation tuning, cross-orientation suppression and surround suppression (suppression evoked by stimuli outside the classical receptive field). We then discuss models showing that the surround suppression data can be understood if V1 is an inhibition-stabilized network: one in which recurrent excitation alone is strong enough to produce instability, but in which feedback inhibition stabilizes the network. Finally we discuss how a similar network architecture may explain the findings of Grinvald and colleagues on the structure of V1 spontaneous activity.
Development of the selective response properties that define columns in sensory cortex is thought to begin early in cortical maturation, without the need for visual experience. My talk will focus on recent experiments that explore the development of direction selectivity in visual cortex of the ferret using intrinsic signal and in vivo 2-photon calcium signal imaging. Our results indicate that direction columns, unlike other columnar systems, emerge a few days after eye opening and this emergence depends on visual experience. The impact of visual experience on the emergence of direction columns can be visualized in individual visually naive animals that are exposed to a motion training stimulus. After 10-12 hours of stimulation, direction columns become evident and gradually strengthen. These effects of visual experience are strikingly limited to the cortical neurons that are activated by the training stimulus, and reflect a rapid increase in the direction selectivity of individual layer 2/3 neurons. Taken together, these results suggest that visual experience plays a critical role in the emergence of direction selective cortical responses.
Several investigations in the visual system have established that coarse spatial features are processed before those of fine detail. Although it is generally assumed that this is a cortical function, there are features of early visual pathways that provide the basis for a coarse-to-fine sequence. We have investigated this possibility by neurophysiological studies in the lateral geniculate nucleus. Our findings show clear coarse-to-fine properties for nearly all LGN neurons in our sample. By use of a model, we estimate that the known temporal dynamic features in the visual cortex may be accounted for by a feed-forward process.
This talk will review recent evidence from my lab, based on intracellular recordings of cortical cells, showing that the neural code in V1 seems to be optimized for the viewing of natural scenes through natural eye-movement dynamics. We propose that there exists for any V1 cortical cell a fit (acquired through learning) between the spatio-temporal organization of its subthreshold (nCRF) and spiking (CRF) receptive fields with the dynamic features of the retinal flow produced by specific classes of eye-movements (saccades and fixation).
Top-down processes such as attention and memory enable selective filtering of sensory information in order to meet behavioral demands, but the mechanisms underlying this process remain unknown. Most evidence supports the idea that attention highlights attended locations or features by enhancing neuronal responses but does not change stimulus selectivity. However, some theoretical work suggests that attention might alter the way neurons encode visual information. In two experiments we have demonstrated that feature-based attention alters the multidimensional tuning curves of many V4 neurons, thereby dynamically changing the way that shape is represented in this area. This attentional modulation enhances responses to attended features, consistent with a matched filter model of visual attention. Our results demonstrate that some V4 neurons do not act as "labeled lines," and they have important implications for the way the visual network must be designed in order to permit correct decoding and interpretation of the information in V4 by higher visual areas. Taken together, our results suggest that memory and decision-making processes required for natural vision are integrated into the basic processes of visual representation and distributed widely across the neocortex.
The most common paradigm for studying visual cortical processing has been to examine the activity of single neurons in response to artificial stimuli such as bars and gratings. While this approach has been highly successful, very little has been learned about how groups of neurons jointly respond to natural scenes. To address this issue, we have recorded spike activity from small groups of 3-10 well isolated, single units in the striate cortex of anesthetized cats and alert monkeys and analyzed their responses to the repeated presentation of short episodes of time-varying natural scenes (movies). We find that the responses of striate neurons to movies are brief, decorrelated, and exhibit high population sparseness. Adjacent neurons differ significantly in their peak firing rates, even when they responding to the same frames of a movie. Pairs of adjacent neurons recorded by the same probe exhibit as much heterogeneity in their responses as pairs recorded by different probes. During periods of joint activity, some cell pairs exhibit transient episodes of synchronous firing indicating a time- and stimulus-dependence on the occurrence and strength of synchronous activity. These findings demonstrate that complex natural scenes evoke highly heterogeneous, but sparse, responses within local populations that can be transiently synchronized, and thereby reveal features of visual cortical dynamics not readily apparent in response to simple stimuli.
Recurrent feedback in the visual cortex can potentially be conceptualized as a mechanism for mediating the influence of prior beliefs in a hierarchical Bayesian inference framework. We consider the computational problem of 3D shape inference based on monocular and binocular cues. We present evidence suggesting that neuronal tuning and neuronal interaction in the primary visual cortex encode ecological statistical priors between 3D scene structures and 2D images relevant for 3D inference. These sensitivities, together with evidence on the response dynamics of neurons in the early visual cortex, are consistent with the hierarchical Bayesian perspective on visual processing.
A border between two image regions normally belongs to only one of the regions; determining which one it belongs to is essential for surface perception and figure-ground segmentation. Von der Heydt and colleagues have observed that border ownership is signaled by a class of V2 neurons, even though the ownership value depends on information coming from well outside the classical receptive fields of the cells. I use a model of V2 to show that this visual area is able to generate the ownership signal by itself, without requiring any top-down mechanism or external explicit labels for figures, T junctions, or corners. In the model, neurons have spatially local classical receptive fields, are tuned to orientation, and receive information (from V1) about the location and orientation of borders. Border ownership signals that model physiological observations arise through finite range, intraareal interactions. Additional effects from surface features and attention are discussed. The model licenses testable predictions.
The primary visual cortex can be modeled computationally as a self-organizing map in a dynamic equilibrium with afferent, lateral, and feedback input. Simulated experiments with such a unified model, LISSOM, demonstrate how a wide variety of phenomena follow from this principle. The map organizes into orientation, direction, ocular dominance, and color selective patches, and develops selective patchy lateral connections between them. The model demonstrates how the map can recover from retinal and cortical injury, as well as how psychophysical phenomena such as tilt aftereffects and contour integration may arise in it. For instance, the model suggests how Kanizsa-type illusory contours can arise based on feedback from a higher area, a prediction that was subsequently verified through VSD optical imaging on macaque V1. In this manner, the model can be used to gain a precise computational understanding of existing data, and to make specific predictions for future biological experiments.
I will present some of our work on modeling binocular depth perception based on physiological properties of binocular cells in the visual cortex. I will show that binocular disparity maps can be effectively computed from stereograms with a population of complex cells, without explicit feature matching. I will describe a unified theory for understanding depth effects of both horizontal and vertical disparities, and will show that our stereo model can be naturally combined with the motion-energy model to explain a family of Pulfrich depth illusions. I will also address some recent criticisms of our models. Finally, I will discuss da Vinci stereopsis and present a simple model for determining the location and ocularity of monocularly occluded regions using disparity-boundary-selective cells.
I will describe a local circuit computational model of a patch of the input layer 4Ca of the primary visual cortex (V1) of the macaque monkey. The model neurons are integrate-and-fire neurons with biologically plausible synaptic conductances. This model can account for the distributions of orientation and spatial frequency selectivity across the population of V1, and also the relative prevalence of linear and nonlinear spatial summation in V1 neurons. The crucial controlling variable appears to be the relative strength of cortico-cortical inhibition relative to excitation, the local circuit. Experiments on responses to spatially extended stimuli indicate that there is also a strong influence of long-distance, possibly feedback, interactions on V1 neurons.
Computational modeling has played an important role in the dissection of the biophysical basis of rhythmic oscillations in thalamus that are associated with sleep and certain forms of epilepsy. In contrast, the dynamic filter properties of thalamic relay nuclei during states of arousal are not well understood. Here we present two modeling studies of the throughput properties of the visually driven dorsal lateral geniculate nucleus (dLGN) in the presence of feedback inhibition from the perigeniculate nucleus (PGN). In the first study we employ thalamocortical (TC) and thalamic reticular (RE) versions of a minimal integrate-and-fire-or-burst type model and a one-dimensional, two-layered network architecture. Potassium leakage conductances control the neuromodulatory state of the network and eliminate rhythmic bursting in the presence of spontaneous input (i.e., wake up the network). The aroused dLGN/PGN network model is subsequently stimulated by spatially homogeneous spontaneous retinal input or spatio-temporally patterned input consistent with the activity of X-type retinal ganglion cells during full-field or drifting grating visual stimulation. The throughput properties of this visually-driven dLGN/PGN network model are characterized and quantified as a function of stimulus parameters such as contrast, temporal frequency, and spatial frequency. In the second study we show how all-to-all coupled TC-RE networks can be described by a multivariate probability density function that satisfies a conservation equation with appropriately defined probability fluxes and boundary conditions. Consistent with the IFB model, the independent variables of these densities are the membrane potential of both cell types and the inactivation gating variable of the low-threshold Ca current IT. The synaptic coupling of the populations and external excitatory drive are modeled by instantaneous jumps in the membrane potential of postsynaptic neurons. When the aroused population density model is stimulated with constant retinal input (10-100 spikes/sec), a wide range of responses are observed depending on cellular parameters and network connectivity. These include asynchronous burst and tonic spikes, sleep spindle-like rhythmic bursting, and oscillations in population firing rate that are distinguishable from sleep spindles due to their amplitude, frequency, or the presence of tonic spikes. Supported by NSF grants IBN 0228273, MCB 0133132, and DMS/NIGMS 0443843.
At the heart of the retinogeniculocortical pathway are neurons in the lateral geniculate nucleus (LGN) of the thalamus. LGN neurons receive feedforward input from retinal ganglion cells and feedback input from the primary visual cortex. Because LGN neurons are strongly driven by the retina and have receptive fields much like those of their retinal afferents, the LGN is generally regarded as a structure that simply relays retinal activity without doing much in terms of visual processing. Closer examination, however, reveals a more complex picture of LGN function, as LGN neurons can dynamically filter their retinal input. Results will be presented from experiments in anesthetized and alert animals that examined the role of feedforward and feedback pathways in the spike transfer, surround suppression and attentional modulation of LGN neurons.
A number of phenomena in visual perception can be related to the assignment of 'border ownership', a hypothetical process of detecting contours of objects and assigning them to the corresponding image regions. Border ownership assignment relates to the interpretation of the image in terms of a 3D layout of objects and the perception of shadows and transparent overlay; it affects recognition of form and deployment of attention; figures draw attention, while shapes of the ground tend to be ignored. In this lecture I will review evidence for border ownership coding in the visual cortex of macaques and discuss recent neurophysiological experiments on the relationship between figure-ground organization and attention. These new results show that border ownership coding and voluntary (top-down) attention influences are combined in single neurons of area V2. Border ownership coding was found for attended as well as for ignored figures, indicating that figure-ground organization occurs preattentively, and in parallel across the image. Tests with two overlapping figures revealed spatial asymmetry of the attention effect relative to the receptive field of the individual neuron. This asymmetry was correlated with border ownership preference, indicating that border ownership and attention mechanisms share critical neural circuitry. A possible interpretation is that the neural network that creates figure-ground organization also provides the interface for the top-down selection process. Thus, the phenomena of figure-ground organization may reflect the general process of recoding a visual image representation into a more efficient data structure that enables further, object-based processing.
