Our lab contributed to this year’s Systems Vision Science Summer School in Tübingen.
Our contribution was led by Carlotta Trottenberg, Matthias Baumann, and Tong Zhang, and it covered our experiments on the active vision loop (image below), with a focus on the superior colliculus (the topic of choice for our session at the summer school).
Carlotta started by considering the visual processes that might lead to saccade generation. She, therefore, investigated the forward arrow in the above diagram. Her work described how a temporal neural code (the time of first stimulus-evoked action potential) might be at play in the superior colliculus when it comes to dictating saccadic reaction times. That is, if the same stimulus shown twice and the superior colliculus first responds to it at different times, then eye movement reaction times will also be different. However, she also showed that a rate code is probably more important for this brain structure, favoring the integration of sensory evidence before committing to moving the eyes. This evidence integration idea makes sense because the superior colliculus is a potent structure when it comes to driving eye movements, with every single spike by a given neuron making an impact on what the eye does. Therefore, it makes intuitive sense to make saccade generation a bit more immune to individual spiking, and this is achieved by the idea of a rate code (evidence integration). Interestingly, Carlotta also found that primary visual cortex neurons are very differently related to saccade generation. Thus, there is a clear dissociation between what the superior colliculus and primary visual cortex do when it comes to active vision.
Matthias then described our results on sensory tuning in the superior colliculus movement commands. That is, once a commitment to move the eye is made, what happens in the superior colliculus at the time of saccade generation? Through a series of beautiful studies, he showed how the saccade motor commands of the superior colliculus are not “pure” motor commands. That is, we can get the very same movement being executed even if the collicular motor commands are different. If so, then what is encoded in the movement commands? It turns out that these movement commands are sensory-tuned, reflecting the visual appearance of the saccade target. This signal could jumpstart the feedback pathway in the active perception loop shown above: that is, as the superior colliculus issues the movement command to trigger an eye movement, this command itself contains a sensory prediction about the visual appearance of the target that will soon be foveated by the eye movement; this prediction could be used by the feedback part of the active vision loop shown above, in which the visual system is trying to predictively handle the sensory consequences of the eye movement (a kind of image stabilization mechanism as in modern-day hand-held camera systems).
Finally, Tong was motivated by this idea about the feedback pathway, and by Matthias’ discovery. She, therefore, asked whether foveal neurons in the superior colliculus (neurons that would “see” the target after the eye movement has ended) receive a kind of “prediction” signal about how the target should appear after the saccade. She performed gaze-contingent display update experiments, in which the pre-saccadic image (which the foveal neurons did not see before the eye movements) was changed mid-flight during the saccades. Thus, the post-saccadic image (which the foveal neurons saw) was different from the pre-saccadic image (which the same neurons did not see). If the foveal neurons receive a sensory prediction about the appearance of the saccade target, then they might signal a “prediction error” when the post-saccadic image is different from the pre-saccadic one. This is indeed what she found.
In all, our lab’s sessions covered big parts of the active vision loop, describing neural mechanisms not just for the forward process of saccade generation, but also for the feedback process of handling the sensory consequences of moving the eyes.