The Visible and The
We only see what we believe, we see our meanings. In his article, Pancho Tolchinsky describes that all we see is a controlled hallucination. This is the first chapter of the book we have edited and designed for OFFF Barcelona 2018.
While an undergraduate, the cognitive science researcher Alva Noë was chatting with an artist, curious about what was going on in the science of visual perception. Noë was telling him that, at the time, vision scientists were seeking to understand “how it is we see so much—the colorful and detailed world of objects spread out around us in space—when what we are given are tiny distorted upside-down images in the eyes.”
That is,—“How do we see so much on the basis of so little?”
To which the artist replied: —“Nonsense! That’s not the question we should ask. The important question is this: why are we so blind, why do we see so little, when there is so much around us to see?”
This story nicely frames one of the paradoxes of perception. We have this vivid and rich experience of the world around us that feels so much more than the sum of the stimuli that reach our bodies. Yet, so many times we find ourselves, and especially others, so frustratingly blind. Sometimes we simply fail to see what later, in retrospect, sits just in front of our eyes. This may be literal seeing, as when the car keys on the hallway table become invisible right when you need them most, or metaphorical seeing, as when your partner frustratingly tries to explain to you why adding 20 percent to 80 gives you 96.
We live by our predictions. We can only see what we expect to see.
Like in a very bad thriller, the short explanation for this mystery is that we don’t see reality, we imagine it: “Sorry, it was all a dream!” Until recently, neuroscience continued with the classical view that our perceptions are reconstructing the present world around us. That is, we take all the inputs, shake them in very sophisticated ways, and then redraw our own version of what is happening outside. According to more recent evidence, however, this is not at all what’s happening. We’re not reacting to outside stimuli, we’re actually predicting them. In other words, your perceptions are envisioning the future world you’re about to meet. And they’re pretty good at it. This is why it feels as if we really see things as they are. But it’s an illusion we take at face value. It’s what the philosopher Merleau-Ponty, to whom we owe the title of this book, called “perceptual faith”. But perception is far from infallible, and that’s our way in to expose its dirty doings. To situate yourself in this story, let’s first pull you back towards the edge of the future, right where it meets the present. No fancy tricks yet, you only have to drive an unfamiliar car.
Your friend has lent you her car to pick up your parents from the airport. You take the keys, get in the car and oh! The keyhole is not exactly where you expected. No big deal, you sort of know where it might be and indeed you find it. You put the key in and start the engine. With a sense of expectation you wait to see that everything is in place. OK, now it’s time to reverse. How do you reverse this car?… Done, you’re out of the parking lot but still feeling sort of anxious as you try to get the car under control. Boom! You pressed the brake pedal and the car stopped completely. You nervously press the accelerator hoping that the car behind you won’t bump into you at the same time as realizing that you have no idea how the car will respond to a fully pressed accelerator. Fortunately there are no more surprises. Gradually, you’re getting a grip on the car. The initial tension is slowly melting away. You can now put your mind beyond the next few milliseconds and beyond the boundaries of your borrowed car. Your horizons have expanded, the world has become predictable again. This helps understand why learner drivers fix their gaze just in front of the car as they take a corner while more expert drivers can look further ahead, up to 3 seconds ahead of the speed they’re driving.
A reactive organism can never go beyond those next few milliseconds. Without predictions we would be more like plankton, at the mercy of the tides and currents of the sea. Animals like us, able to display such a sophisticated range of behaviours, strongly rely on their guesses. Think about it. Does it make sense to be surprised if you’re not making any predictions? Right now, for instance, probably nothing around you is all that surprising, other than this book, of course. It’s not because the world around you has nothing to surprise you with. Rather, whatever you’re aware of falls within your predictions, everything else doesn’t make it into your perception. It’s invisible. If you want to be surprised, try engaging with the world by being more curious, by making more predictions. Apparently, feeling awe frequently is healthy.
A case in point would be the poet Lucretius, who more than two thousand years ago was fascinated by the fluctuations of tiny particles visible when sunbeams penetrate a “shadowy place”. He pondered that these mingling particles dance because they’re hit by even smaller particles, by Democritus’s envisioned atoms. It took about 2000 years of mathematics and physics, and another great poet to provide the exact measure of those atoms. It was Einstein, aged 25, who observed how small particles drifted from their position, not in a shadowy place but in fluid. The next time you encounter sunbeams lighting up those dancing particles, you’ll probably see something more than just suspended dust.
Without concepts we are blind.
You may be more comfortable calling these predictions assumptions, but they are not, they are more than assumptions. Your brain is making informed guesses based on your past experience. When those guesses fail dramatically, that is when your predictions become unreliable, you are liable to become anxious or even very anxious. It may be driving an unfamiliar car in heavy traffic or any uncomfortable set of uncertainties that shrink your space-time horizon and suffocate you in a very narrow present. Of course, you may think you can enjoy this thrilling moment of uncertainty if you go into it with the right attitude. This is a very good point worth exploring in the next self-help book. But, if your entire life, every millisecond of it, is like driving an unfamiliar car it may not be that easy to keep up this positive attitude for too long, or indeed to have much of a life for that matter.
So we live by our predictions, but does it mean we’re seeing the future? It doesn’t, but yes, in a way, we sort of do. Let’s dive in again to where the future meets the present.
Suddenly a systematic and repetitive sound in the background vanishes and it’s only then that you realise it was there all along. You weren’t taken by the presence of a new sound but by its very absence. Similarly, when you’re listening to a song you know pretty well, you would obviously be more surprised by the sudden absence of an expected beat than by its presence. Slightly more dramatically, imagine you’re going down the stairs while chatting with a friend and the following step takes just half a second more than you expect. Now that’s probably a bit more surprising. And we can continue dramatising this idea as far as we want. This is only to show that the unexpected absence of an input is as perceptually prominent as any concrete event. What captures our attention is not the stimuli as such, so much as a mismatch between predicted and incoming inputs. Every time there is such a mismatch, your brain fires a prediction error.
What neuroscience is now proposing is that our brain is continuously firing prediction errors that make all kinds of amendments readjusting our perceptions. According to this view, higher parts of our brain generate plausible sensory stories for us to perceive. While, at the same time, the lower parts of our brain are checking whether the proposed stories make sense with respect to the incoming inputs. If a story makes sense, for instance your foot does indeed touch the step when expected, you continue with your own personal play, otherwise, a prediction error is fired to make the appropriate adjustments. If there is an important mismatch between expected and incoming inputs the prediction error will escalate and you’ll experience surprise. But, if the prediction error is minor, the amendment takes place in the basement and you’re completely oblivious to any changes made to the imagined story. The end product of this process is a fully fledged sensory story that you yourself have generated but readjusted so as to make it consistent with the incoming inputs. This end product is what we typically call reality. Others have proposed calling it controlled hallucination.
Before we continue, it is worth having a glance at the hardware of the brain. There we find that for every single wire that brings information from your eyes to the higher parts of the brain, there are about ten wires that go in the opposite direction. That is, from this hardware perspective, 90% of what you see is generated from within, while only 10% is reacting to what’s in front of your eyes. We are hardwired to predict the incoming inputs not to react to them.
What neuroscience is now proposing is that our brain is continuously firing prediction errors that make all kinds of amendments readjusting our perceptions.
If this story is right, it means that you can only perceive what your brain, using all its past experiences, is able to generate for you to see, hear, sense, smell, savour, etc. It is hard to imagine that you haven’t already browsed all the images in this chapter, but let’s suppose you haven’t. Most likely, you will only see blobs and stains that don’t really make any particular sense to you. You can view them as abstract images for you to explore your imagination. While you are enjoying these abstract images, the higher parts of your brain are working like crazy making all kinds of guesses in order to provide you with something useful, something less abstract. All they get in return however, is a burst of prediction errors from the lower parts of your brain. If this is true, then at this moment you are experientially blind. Your brain can’t provide you with anything useful given the experiences you’ve been exposed to. Please enjoy this moment. Little is stopping you from using all your creative powers to imagine everything that your past experience lets you. Once you reveal the hidden image the chances are high that you won’t be able to go back and re-experience this freedom again. Go on, you can see the hidden image now. Be a slave to your own experiences, once again.
Now, all those previously abstract images should appear somewhat more concrete. Your perception experience has changed: what you see now makes sense to you. And this is because the hidden image has changed your brain by modifying the firing of your neurons. While your old brain could only provide you with stains and blobs, your new and updated brain can now provide you with a clear and meaningful shape. So, congratulations, you’re no longer experientially blind. You’re the owner of a new concept that has made visible what was once invisible.
A concept is precisely that: something that allows you to engage with the world outside your skull. It turns bubbles and stains into something that has meaning for you. This means that we perceive the world around us, and within us, through our own concepts. Because, let me tell you, the meaningful shape you so clearly see now in all those images isn’t physically there. It’s the new concept you own (or that owns you) that you have to thank for this persistent and compelling hallucination.
Have you ever been sitting in a cafeteria wondering what the couple behind you are talking about? A key moment is when you identify which language they’re talking in. Then, those strange sounds start to build a character. Those Italian words and tempos become clearer and clearer until your friend tells you you’re completely crazy: they are Australians. At that moment your brain dismisses your Italian concepts and tries for better luck by recruiting more exotic English sounding concepts. Suddenly, those sound patterns are building a completely different character from before. On occasions, you actually understand what they’re saying. This, I’m afraid is another clear case of hallucination. Studies have shown that in a typical conversation about 50% of the uttered sounds are incomprehensibly out of context. The context provides your brain with clues about which concepts to recruit to make better predictions. If the context is not deceiving your brain and you have the appropriate concepts, you can have a normal conversation without noticing that 50% of what your friend is saying sounds more like gibberish. That is the essence of perception: turning environmental noise into meaning. Concepts are the bridges that connect what is outside your head to what’s inside. They turn the meaningless stimuli into meaningful perceptions. One takeaway message would be: the more concepts you cultivate, the wider and richer your perceptual world will be.
Perceiving is already a form of understanding. We perceive our own meanings.
Next time you meet someone who can’t see (metaphorically or not) what you’re so frustratingly trying to show them, just think that it may be that the two of you don’t share the same concepts. You may have had two different lives, different experiences. It doesn’t matter how clearly you see what you perceive it’s always a product of your concepts. Neither of you is seeing reality: you’re simply not sharing the same hallucination. This is not carte blanche for stupidity. Just as some predictions are better than others (e.g. predicting that Barça will defeat Real Madrid 50-0 is a better prediction than expecting Barça to win by π goals to √2), some hallucinations are also better than others. However, this doesn’t turn them into objective reality.
But why do we hallucinate in the first place? Why can’t we just see the world as it is? Basically, because the raw data that we receive as input is in itself completely meaningless. It’s, and sorry to repeat myself, even less than blobs and stains. However, because we perceive the world through our concepts we are able to engage with our surroundings in terms of our needs, fears, and desires. Perceiving, thus, already involves a form of understanding the world around us. Our hallucinations are, if you may, a product of an augmented reality through which we can only perceive the meaning of things, or rather, what those things mean to us. Because meaning doesn’t lie outside of us. What we see, hear, sense, smell or savour is tailored to us by our memories, our body, our needs, and our expectations. We create meaning in this very act of going beyond the information given; that is to say, in the very act of hallucinating.
So, why do we see so much on the basis of so little? Because we imagine the world around us through our own meanings. Why are we so blind? Because we imagine the world around us through our own meanings.
Cover illustration: © Mariano Pascual
Last illustration: © Ricard Jorge
