Dr. Nadine Dijkstra is a cognitive neuroscientist at the Wellcome Centre for Human Neuroimaging at University College London where she investigates the neural mechanisms of mental imagery, perception and how the brain dissociates between the two.
A post by Nadine Dijkstra
Did you really just see something appear in the corner of your eye or was it just your imagination? For most of us, the difference between what we imagine and what we see seems very clear. However, the more we learn about the neuroscience of imagination, the more puzzling it is that we don’t confuse our internally generated experiences with reality more often.
Over the last decades, the development of neuroimaging methods suitable for use in humans - such as functional magnetic resonance imagining (fMRI) and magneto- and electroencephalography (M/EEG) – have made it possible to actually look into people’s heads while they are imagining. This research has revealed that when we imagine something, many of the same brain areas get activated as when we perceive that same thing.
For example, when we imagine the face of a loved one, the same parts of the high-level fusiform gyrus – a brain area located in the lower lobe of the cortex, sitting close to ear level in the skull – become activated as when we would actually see that person in real life (Ishai, 2002; Ishai et al., 2000). Similarly, when we imagine alternating black and white lines of a certain orientation (i.e. ‘a grating stimulus’, the classical image used in visual neuroscience research), the low-level primary visual cortex (V1) – the neural entry point for signals coming from the eyes, located all the way at the back of the brain – becomes activated in a similar way as when we would actually see those same lines (Albers et al., 2013; Harrison & Tong, 2009; Rademaker et al., 2019). Dozens of neuroimaging studies have shown neural overlap between imagined and perceived stimuli of all kinds, from simple shapes and lines to objects to full scenes (for reviews, see Dijkstra et al., 2019; Pearson, 2019). This poses a fundamental question: given that the brain signals of imagery and perception are similar, how are we able to keep apart imagination and reality?
The first possibility is that there actually is not really a problem. This could for example be the case if the neural overlap that we observe with neuroimaging is an artefact of the limited resolution of these methods. While fMRI can capture neural responses with millimeter precision, a millimeter of cortex still consists of tens of thousands of neurons. This means that it is theoretically possible that despite the fact that our brain measurements of imagery and perception look similar, they do in fact rely on different neural populations.
Another option is that the brain uses the fact that imagery tends to be done on purpose. At least in all of the studies that I mentioned above, participants were always explicitly instructed to imagine stimuli. This means that, in contrast to perception, the sensory signals during imagery are accompanied by an intention to imagine. Perhaps the brain could utilize this for reality monitoring so that when sensory signals match an intention signal, they are discounted to be the result of our own imagination.
Anecdotal evidence in line with this idea comes from the famous Perky effect. In her seminal experiment from 1910, Mary Cheves West Perky asked her participants to imagine various objects, like an apple, while looking at a certain location on the wall. At the same time, Perky projected congruent shapes, like a reddish circle, at exactly the same location on the wall using a magic projection lantern that she had hidden away in the experimental room (see below). What happened was that all participants confused the sensory signals from the projection lantern as being the result of their imagination, saying things like “I never knew I could do this [imagine so vividly], but then again, I never really tried” or “If I hadn’t known I was imagining, I would have thought it real” (Perky, 1910). This finding can be explained by the idea that signals in line with the imagery intention – in this case the project shapes - were tagged as imaginary.
Perky effect. In her seminal 1910 study, Mary Cheves West Perky made her participants believe that externally projected shapes were the consequence of their own imagination. To do this, she used a device called a ‘magic projection lantern’ which she hid away in the room so that participants did not know real images could be shown.
The intention hypothesis would suggest that if sensory signals are not accompanied by a clear intention signal, we will assume that they reflect reality. However, imagery is often triggered without a clear conscious intention, and still not mistaken for reality. For example, a mental image of a tasty doughnut might pop-up in your mind quite automatically around 4 o’clock when your blood sugar levels are decreasing. In most cases, we still do not confuse these mental images for reality (and start grabbing in the air for doughnuts that aren’t there).
One other factor that might be used to distinguish reality and imagination is sensory strength or vividness[1]. Due to the way our visual system is wired, signals coming from the outside world tend to be stronger and more precise than signals that are triggered within the brain (Dijkstra et al., 2022; Koenig-Robert & Pearson, 2021). This means that while imagery and perception activate similar sensory representations, the imagery ones tend to be weaker. This might also be the reason why the phenomenological experience of imagery tends to be less vivid than that of perception. Following from this, one other option is that perceptual reality monitoring is done based on vividness: the more vivid the sensory signal, the more likely we think it’s real.
We recently executed a large-scale online psychophysics experiment to arbitrate between these different options (Dijkstra & Fleming, 2023). We wanted to design a modern-day version of Perky’s experiment in which participants are unaware that external stimuli might be presented to them. This is important, because if participants would know that external stimuli could be presented, they might look for them and start exhibiting response biases. To accomplish this without hiding away magic projection lanterns, we had each participant only perform one critical trial. Participants imagined a simple stimulus (a grating) and indicated how vivid their imagery was a few trials in a row. Then on the last, critical trial, we secretly presented either the same stimulus or a different stimulus. Participants again had to rate their vividness and after that, we asked, ‘on the previous trial, was a real stimulus presented or was anything you saw only imagined?’
What we found was exactly in line with the sensory strength idea: when the same stimulus was presented, participants’ imagery vividness became higher and they were more likely to say they saw a real stimulus, even when no real stimulus was presented. We also found that participants who had higher imagery vividness overall, over all trials, were more likely to make reality monitoring errors: indicating a real stimulus was there when there wasn’t or indicating that everything was imagined when a real stimulus was in fact presented.
How do we distinguish imagination from reality? These images are generated by MidJourney, an AI which uses a combination of large language and diffusion models to generate images given language prompts. The prompt I used was ‘imagination versus reality’. All of these images seem to capitalize on vividness to portray this idea.
So how does our brain tell apart imagination and reality? Our results suggest that vividness is one very important factor. This idea is in line with what David Hume already said in 1739 in his Treatise on Human Nature: “The idea [imagination] of red which we form in the dark differs only in degrees [vividness], not in nature, from the impression [perception] of red which strikes our eyes in sunlight.” However, most likely, perceptual reality monitoring is a complicated decision process that also takes into account other factors like intention, context and background knowledge (Dijkstra et al., 2022). Otherwise, if we accidentally imagine that doughnut a bit too vividly, we’ll be very disappointed.
Notes
[1]I am defining vividness very loosely here, as something about the amount of sensory details and strength in a mental image or how ‘perception-like’ it is. One important avenue for future perceptual reality monitoring research is to more precisely define what kind of vividness is important for reality monitoring. For more discussion on the concept of vividness in imagination research, see Kind, A. (2017) Imaginative vividness. Journal of the American Philosophical Association, 3(1), 32-50.
References
Albers, A. M., Kok, P., Toni, I., Dijkerman, H. C., & De Lange, F. P. (2013). Shared representations for working memory and mental imagery in early visual cortex. Current Biology : CB, 23(15), 1427–1431. https://doi.org/10.1016/j.cub.2013.05.065
Dijkstra, N., Bosch, S. E., & van Gerven, M. A. J. (2019). Shared Neural Mechanisms of Visual Perception and Imagery. Trends in Cognitive Sciences, 23, 18–29. https://doi.org/10.1016/j.tics.2019.02.004
Dijkstra, N., Bosch, S., & van Gerven, M. A. J. (2017). Vividness of Visual Imagery Depends on the Neural Overlap with Perception in Visual Areas. Journal of Neuroscience, 37(5), 1367–1373. https://doi.org/10.1523/JNEUROSCI.3022-16.2016
Dijkstra, N., & Fleming, S. M. (2023). Subjective signal strength distinguishes reality from imagination. Nature Communications, 14, 1627. https://doi.org/10.1038/s41467-023-37322-1
Dijkstra, N., Kok, P., & Fleming, S. M. (2022). Perceptual reality monitoring: Neural mechanisms dissociating imagination from reality. In Neuroscience and Biobehavioral Reviews (Vol. 135, p. 104557). Pergamon. https://doi.org/10.1016/j.neubiorev.2022.104557
Harrison, S. A., & Tong, F. (2009). Decoding reveals the contents of visual working memory in early visual areas. Nature, 458(7238), 632–635. https://doi.org/10.1038/nature07832
Ishai, A. (2002). Visual Imagery of Famous Faces: Effects of Memory and Attention Revealed by fMRI. NeuroImage, 17(4), 1729–1741. https://doi.org/10.1006/nimg.2002.1330
Ishai, A., Ungerleider, L. G., & Haxby, J. V. (2000). Distributed neural systems for the generation of visual images. Neuron, 28, 979–990. https://doi.org/10.1016/S0896-6273(00)00168-9
Koenig-Robert, R., & Pearson, J. (2021). Why do imagery and perception look and feel so different? Philosophical Transactions of the Royal Society B: Biological Sciences, 376(1817), 20190703. https://doi.org/10.1098/rstb.2019.0703
Pearson, J. (2019). The human imagination: the cognitive neuroscience of visual mental imagery. Nature Reviews Neuroscience. https://doi.org/10.1038/s41583-019-0202-9
Perky, C. W. (1910). An Experimental Study of Imagination. The American Journal of Psychology, 21(3), 422. https://doi.org/10.2307/1413350
Rademaker, R. L., Chunharas, C., & Serences, J. T. (2019). Coexisting representations of sensory and mnemonic information in human visual cortex. Nature Neuroscience, 22(8), 1336–1344. https://doi.org/10.1038/s41593-019-0428-x
Ragni, F., Tucciarelli, R., Andersson, P., & Lingnau, A. (2020). Decoding stimulus identity in occipital, parietal and inferotemporal cortices during visual mental imagery. Cortex, 127, 371–387. https://doi.org/10.1016/j.cortex.2020.02.020