How the brain constructs perception — and why illusions happen

The brain constructs perception every time we open our eyes. What feels like a continuous, detailed view of reality is not simply a copy of the image projected onto the retina. The brain combines incoming signals with context, previous experience and expectations to arrive at the most plausible interpretation of what is happening around us.

We rarely notice this construction because the result is usually seamless. Each eye, for example, has a blind spot where the optic nerve leaves the retina and there are no photoreceptors. Yet we do not normally see a hole floating in our visual field. Information from the surrounding scene and from the other eye makes the missing region largely disappear from awareness.

The same problem exists throughout perception. Sensory information is often incomplete, noisy or ambiguous. Lighting changes, objects obscure one another, and every eye movement shifts the image across the retina. Despite this, the world does not appear as a fragmented collection of signals.

Perception is not a photograph

The eyes provide information about light, colour, contrast, motion and other visual features. Those signals alone, however, do not uniquely specify what is out there.

Similar retinal patterns can be produced by different objects under different lighting conditions. The visual system therefore has to interpret its input. Context helps determine which edges belong to the same object, what is near or far, whether a dark region is part of an object or a shadow, and what is likely to continue behind something that blocks our view.

Visual illusions expose this process particularly well. Two physically identical elements can look different because of their surroundings. A stationary image may appear to move. An ambiguous figure can suddenly switch between two interpretations even though nothing in the image itself has changed.

In such cases, the external stimulus remains the same while its interpretation changes.

This does not mean that the brain simply invents reality. Sensory evidence strongly constrains what can be perceived. But perception emerges from an interaction between incoming information and what the nervous system already knows about the probable structure of the world.

Expectations reach early visual cortex

A 2012 experiment by Peter Kok, Janneke Jehee and Floris de Lange provides a useful example of how deeply expectations can influence visual processing.

The researchers recruited 20 healthy volunteers, with data from 18 included in the final analysis. Participants viewed simple grating patterns with different orientations. Before each image, an auditory tone predicted the likely orientation of the upcoming gratings with 75% validity.

The researchers used functional MRI to examine activity in early visual cortex. Rather than measuring only how strong the overall response was, they also analysed the spatial pattern of activity to determine how much information about stimulus orientation could be extracted from the signal.

The result initially looked paradoxical. When a grating had the expected orientation, the overall response in primary visual cortex, or V1, was lower. Yet the neural representation of its orientation became more informative.

The expected stimulus therefore produced less overall activity but a sharper representation.

Participants were also better at discriminating small orientation differences when the stimuli matched expectations. The behavioural improvement was related to the improvement in the neural representation measured in V1.

The authors interpreted this pattern as evidence that expectation can sharpen sensory representations, reducing less informative activity while making relevant stimulus features more distinct. The experiment was small and used simple visual stimuli, however, so it cannot establish one universal mechanism for all forms of human perception.

A predictive brain

These findings fit with an influential family of ideas known as predictive processing.

In simplified form, predictive-processing models propose that the brain does not passively wait for a complete sensory description of the world. Instead, it generates expectations about what it is likely to encounter and compares those expectations with incoming information.

The difference between prediction and sensory evidence — often called prediction error — becomes especially informative. When events closely match expectations, less processing may be required for predictable aspects of the input. Unexpected information creates a mismatch that can drive further processing and revision of the brain's current model.

This strategy makes sense for an organism that almost never receives perfect information. Quickly recognizing a face, an approaching object or a possible threat can be more useful than reconstructing every scene from scratch as if the brain were analysing millions of independent pixels.

Predictive processing, however, is a theoretical framework rather than a completed theory of the brain. Scientists continue to debate how broadly it applies and exactly how predictive signals are implemented across different neural systems.

Understanding illusions does not require assuming that every aspect of perception follows a single predictive mechanism. The broader point is more secure: perception depends not only on sensory input but also on context, attention, experience and expectation.

This produces the apparent paradox. The same system that allows us to navigate the world efficiently can sometimes mislead us.

Illusions are therefore not simply evidence that vision is badly designed. Many reveal assumptions that are normally useful because they help the brain find the most plausible interpretation of incomplete information. An unusual image can exploit those assumptions, causing us to perceive a feature that is not literally present or to misjudge one that is.

What reaches awareness is therefore not the raw stream of light striking the retina. It is the result of continuous selection, comparison and interpretation by the nervous system.

Most of the time that reconstruction works so well that we never notice it. Illusions are interesting precisely because, for a moment, they make the hidden work of perception visible.