Participants donning thick-lensed stereo glasses grab virtual shapes in front of them, gesturing as if the boxes were real. The subjects place their heads in chin rests to keep their heads from moving, and soon, green and red boxes flash on the screen. Meanwhile, a motorized arm works with the computer program, moving a real, physical block to the same location that the virtual reality shows on the screen. When subjects reach out a hand to grab the virtual object, they feel and grab the real object instead.
This scene is common in the visual perception lab run by Professor of Cognitive, Linguistic and Psychological Sciences Fulvio Domini, who researches how the human visual system processes information.
“We try to find a mathematical model to explain how 3D visual information is processed by the brain,” Domini said. “Then I test this model with experiments I do with human subjects.”
Domini has devoted two decades to his research, and now he has help from three graduate students, as well as two undergraduates, who work with him in his lab. The lab tries to disprove a common cognitive neuroscience theory called the dual visual streams hypothesis, which suggests human beings have separate visual streams for perception and action.
“We find that errors in perception transfer to error in action in a very clear way,” Domini said.
Bethany Hung ’19, who worked in the lab during her first year at the University, helped perform and build experiments to disprove the dual visual stream hypothesis.
“They posit that the visual streams for your eyes and for your hands are separate,” Hung said about the hypothesis, which was first proposed by David Milner and Melvyn Goodale in 1992. The theory, which is still taught in most introductory cognitive neuroscience classes, proposes that the stream for perception is more susceptible to illusions and the one for action is more in line with the real world.
“Every textbook has got this stuff,” Domini said of the dual visual streams hypothesis. “When theories are simple and taken almost for granted, it’s difficult to go against those.”
Domini, who grew up in Italy, has worked on numerous papers that challenge the hypothesis, helping to rewrite scientists’ understanding of how the brain processes visual inputs. He said that while the dual streams hypothesis is still the most mainstream theory of visual perception and action, more labs are starting to do research on how the visual system may process the same information for both perception and action. One paper in particular, which Domini, Hung, Carlo Campagnoli ScM ’13 PhD ’17 and Evan Cesanek ScM ’15 PhD ’19 co-authored, shows that action and perception are affected by the same visual biases, even with physical objects.
“We claim that you don’t have these two (visual streams),” Domini said. “We only have one representation, and if there are distortions (in how the brain perceives an object), your grasp is affected.”
While trying to dispute the dual streams hypothesis is one of the primary research functions for the lab, researchers in the lab are also exploring other experiments and hypotheses.
Abdul Deeb GS is a second-year PhD student whose research focuses on the role that Newtonian physics may play in the visual system in addition to how physics might help “to make predictions and disambiguate scenes when it’s not clear what’s going on,” he said. “In the experiments, I show people physically impossible events and I test to see if the visual system tries to correct to the Newtonian correct event,” Deeb said, describing a physically impossible game of pool, where balls shoot off in the wrong direction. “Even if we don’t know any physics at all, our visual system might know it.”
Jovan Kemp GS, also a second-year PhD student, studies cue combinations, or how the visual system creates models of a 3D world from various 2D cues, such as texture and perspective. “We have a lot of freedom with what we do,” Kemp said.
Ailin Deng GS is a third-year PhD student who studies how sensory feedback from the environment around us, such as visual changes and tactile touch, influences responses to perception and action.
As for practical applications of the research, Domini said that eventually the science could be used in robotics and virtual reality applications, but science for science’s sake is the primary motivation for now.
“The basic research in our field is still very much needed,” Domini said. “We are far from understanding how the visual system, and the mind, works.”
