This is the inaugural post in our series “Spotlights on Computational Neuroscience”. Spotlight posts are invited submissions from leading neuroscientists that highlight both cutting edge research and general conception issues arising within the neurosciences. We are thrilled to have A. David Redish kick off this series with a great post about our understanding of cognition as a computational process.
– Trey Boone, Associate Editor
The world we live in is partially predictable, that is, it is neither truly random, nor is it deterministic (at least from the perspective of an internal observer). This is true whether we are navigating the complex social dynamics of a faculty meeting, foraging for ripe berries on trees, hunting prey (or escaping from predation), or even just walking from one place to another. Partial predictability means that one can gather information, store and process that information, and do better at achieving one’s agentic goals (whatever those are) by acting on that information.
Information measures how much knowing about one thing tells us about another. For example, the American Revolutionaries in 1775 needed to know whether the British army stationed in Boston would be crossing the Charles River by sea or coming by land over the bridge at Boston Neck. Moreover, they needed to know when the army would move. They couldn’t split the revolutionary militia across both routes, nor could they sit and wait for the army to show up. Instead, lanterns were hung in the steeple of the Old North Church (“one if by land, two if by sea”) to indicate which route the British were taking and when they were on the march. The militia responded to the lanterns, and in doing so, responded correctly to the British army’s route, even though they couldn’t actually see the British army at all. The lanterns shared mutual information with the route taken by the army.
Similarly, neural signals contain information about the world. Some of that information is due to sensory systems, some of it has been encoded in memory, either from previous sensory experiences or from evolutionary selection processes. But the effectors (muscles, viscera) in agents respond, not to the actual things in the world, but rather to the neural signals that share mutual information with that world. For example, the Mauthner cell in fish is active when it detects a looming stimulus on one side, and the muscles in the fish respond in such a way to move it away from that looming stimulus very quickly. Importantly, the muscles of the fish are not responding to the presence of a predator — they are responding to the Mauthner cell, which shares mutual information with a looming stimulus (which is likely a diving predator).
One of the most important discoveries of the past hundred years has been that how one stores information changes what one can do with it. For example, most English dictionaries are sorted alphabetically, which makes it easy to find words. One can make a good guess about where to open the dictionary to find the word, and then move forward or back to find it. This storage strategy is particularly useful if you know the spelling of a word, but don’t know what it means. Imagine, in contrast, that the dictionary was organized by sounds, such that similar sounds were put together. This organization would be useful if you heard a word but didn’t know how to spell it.
This concept — that how one stores information changes what one can do with it — occurs in both symbolic realms (as in the dictionary example, above, where one is manipulating symbols) and also in subsymbolic realms. Subsymbolic realms entail distributed representations, where each element contains partial information about some broad condition, but the mapping from elements to meaning can only be seen in the combination of those subsymbolic elements. The current working hypothesis is that human cognition entails symbolic manipulations instantiated through subsymbolic means. We know that human cognition can manipulate symbols. (This short essay is a bunch of symbols [letters, words, phrases].) But purely symbolic models are generally unable to accomplish many of the tasks humans routinely do. Moreover, they break down in ways that are fundamentally different from the mistakes humans make in their cognition. Models that are built from subsymbolic architectures show much more human-like patterns of success and failure.
Fundamentally, we talk of both the symbolic and the subsymbolic processes as computation — components have mutual information with the world, that information is combined with other information, transformed, and reintegrated into new structures. The questions become: What information is there? How is it stored? How is it transformed? How is it used?
We don’t perceive pixels of green and brown and red, we see a rosebush. We don’t perceive specific frequencies, we hear music. Perception is a hypothesis about the world, a computation about what is most likely out there. This is why illusions exist, because the underlying computations sometimes find an answer that does not reflect reality. Similarly, the multiple human decision making systems represent information about the past (memory), the present (perception), and the future (agentic goals) in different ways, leading to different probabilities of taking a given action. Systems that contain information about the consequences of one’s actions can be used for planning, but the computation to reason through those plans is slow, while systems that learn a mapping from perception to action based on past successes can execute actions quickly, but are inflexible.
The cognition as computation hypothesis does not mean that we are von Neumann computers, like a typical desktop computer with a central processing unit and separate memory chips. Nor does it mean that we are generative data models (aka “large language models”, being currently sold as “AI”) with chains of subsymbolic pattern matching units. But these systems (von Neumann computers, generative data models, and biological [including human] cognition) are best understood in terms of the computations they are performing. Importantly, the cognition as computation hypothesis does not mean that all computation is cognition; cognition is a specific set of computational processes that enable us to deal with our particularly complex partially predictable world. Articulating exactly what those computational processes are is the central challenge of modern cognitive neuroscience.
Further reading
Churchland PS, Sejnowski TS. The Computational Brain. Cambridge MA: MIT Press; 1994.
McClelland JL, Rogers TT. The parallel distributed processing approach to semantic cognition. Nature Reviews Neuroscience 2003;4: 310–322.
Redish AD. The Mind within the Brain: How we make decisions and how those decisions go wrong. Oxford; 2013.
Mitchell KJ. Free Agents: How Evolution Gave Us Free Will. Princeton University Press; 2023.
I can see main emphasis here is the nature of (un/partial)predicatbility of systems and how cognition/computation overlaps in the area of prediction.
By this token, when we have simpler systems lets say with binary variables, and artificial models predict the outcomes accurately, can such computation can be called Cognition? or what if the computational capabilities of the artificial agents enhances to such a degree that they could predict complex equations accurately, will it then be called cognition?
From Embodied Cognition paradigm, we learn that it is a ‘cognitive mechanism’ which involves sensory-motor cortices and neural regions responsible for affect which result in generation of “experience”. It’s not merely “information” …say, ‘rose is red’, but also that rose is ‘beautiful’, or ‘apple is sweet’, but also that apple is ‘tasty’.
Why to say that such cognitive mechanism is just a computation on information…?! Characterizing in this manner was the compulsion of computer scientists because computer doesn’t have consciousness and works based on mere computations on data. But Cognitive Neuroscience of Humans (or animals) seems to have mistakenly adopted this language of computationalism… Why not to say that it is a cognitive mechanism (not computation) which actively involves affective processes, e.g in the works of Walter Freeman, Damasio, etc. ?