One of the anonymous reviewers of my book manuscript remarked, with approval, that it contained very little discussion of embodied, extended and enactive (EEE) cognition. Probably this omission stems from my Kantian gut feeling that an explanation of mind and cognition must appeal only to what happens after sensory input hits the senses, and the view that the prediction error minimization (PEM) scheme accomplishes this explanation. But does PEM indeed make the extremely influential current EEE-debates obsolete? Can one buy into PEM and EEE at the same time? This post suggests that PEM is indeed cast in an unfashionable Cartesian and Kantian mould, which is anathema to EEE. Yet, though deeply inferentialist, PEM does give a central role to the body, to some objects, and to action.
According to PEM, perception and cognition arises in a process where an internal model predicts the sensory input and thereby infers the hidden causes of that sensory input. The only thing that matters is the ability to suppress prediction error on average and over time. This means that what matters is a process on the “inside”, between the sensory input and the internal model. The focus is on keeping the sensory input predictable. This can be difficult in a world with many interacting causes that produce twists and turns in the sensory input at many time scales. This difficulty calls for all the statistical PEM-tools, employed in a hierarchical model, as outlined in the previous post.
If all that matters is a relation between sensory input and internal model, then there is little reason to think that things on the “outside” such as the body and specific objects matter to an explanation of the mind. More accurately, the body and things in the environment matter only insofar as they have the potential to influence the flow of sensory input and thus mandate representation in some parameter of the internal model.
The body is hugely important here, since the way we move around in the environment causes massive changes in the sensory input. Therefore we had better be able to form pretty much all our expectations of sensory input conditional on what the body is like and what it is doing. If not then it would be difficult to keep prediction error under control. This means that the mind is ‘embodied’ only in the sense that the body is an internally represented cause of sensory input. In principle the body is no different from other objects that may cause various degrees of havoc to the flow and thus the predictability of sensory input.
This inferential story immediately incorporates action, since action is part of what makes the body move around; perception is thus ‘enactive’ because action causes changes in sensory input. I also think that the mind is ‘extended’ only in the sense that some inferred external objects are associated with high precision prediction error, in particular in our inferences about action. Though more is needed to fill this out, I think the wholly internal PEM story can account for the intuitions and cases that drive the EEE-views.
The upshot is a very deflated conception of the role of body, objects and action (including of self and of other people). All this is just representations that we harbor because they best explain away the sensory input. If some other representation could better explain away sensory input, then we would jettison the old ones immediately (to illustrate how fickle our representations are recall the study linked to in the first post, where Bryan and I show how readily we experience the world as having supernatural phenomena). I rather like this solipsistic, minimalist, opportunistic, almost nihilist landscape of our mental lives—Scandinavian noir for the mind. Andy Clark, on the other hand, sees PEM as much more amenable to some (but not all) the EEE trends; in Andy’s more sunny perspective PEM is a more ‘jazzy’, rich framework for understanding our fluent interactions with the world.
My interpretation of PEM is driven by the fact that PEM has to be strongly internalist, simply because PEM must be inferential. There must be something doing the inference and something, on the other side of an ‘evidential boundary’, which is being inferred. The more of the occurrent evidence that the agent’s model can explain away the more the agent gains evidence for itself—creatures are in other words self-evidencing. There is growing awareness that this evidential boundary is a Markov blanket for the internal states, such that the behavior of the brain can be understood once the brain’s sensory and active states are known. This of course allows Cartesian skepticism, a total seclusion from the external world: we have an epistemically motivated sensory veil and we have even said that the evidence for the agent’s existence is the model’s prowess at anticipating the activity at its sensors. Truly, I think (=minimize prediction error), therefore I am. (See papers by Friston that Bryan linked to in an earlier post, as well as this one; much of this is developed in my paper here).
All of this suggests that PEM comes with a principled way of drawing the boundary between mind and world, and a clear agenda for understanding the mind in neurocentric terms. Importantly, PEM appropriates body, special objects, and action into this internalist scheme. So it is anti-EEE without ignoring the explanatory challenges posed by the EEE-literature.
Having said all this, PEM also sees us as embedded in the causal structure of the world. We are physical structures in the world, who for some span of years manage to use action to maintain our bodily integrity and thus withstand dispersion. If it wasn’t for the causal impact of the world on our senses PEM would have no purpose. In this setting, PEM is aligned with older notions of self-organisation as well as newer, enactivist notions of autonomy, autopoeisis, and self-enabling (see Evan Thompson and co-authors). What intrigues me is that here we need a marriage of a strongly anti-inferentialist, anti-internalist EEE-style view with the unapologetically inferentialist and internalist PEM. I suspect a profound conception about the mind will come to light once we figure out how to reconcile these views.
Hi Jakob,
Thank for this fascinating post. I agree with you that your book is written largely along Kantian lines rather than the Heideggerian “in-the-world” lines that EEE people favor.
However, I was struck by one passage in your book that strikes me as very pro-Heideggerian, pro-Gibsonian. On page 49 you write:
“We find the required additional constraints, the knowledgeable supervisor, in the world itself. And of course, this is the optimal solution because, to put it in a glib slogan: the world is the truth. The feedback signal is the actual statistical regularities caused in us by the world itself.”
What this striking passage reminded me of was Rodney Brooks, darling of EEE, and his famous anti-representational statement: “The world is it’s own best model”.
What’s interesting then is that your book cuts a middle path between Kantianism and Heideggerianism. Kant would never license himself to talk about the “world itself” and Heidegger/Brooks would loathe talk of internal models. As would Gibson. But as you acknowledge in this passage the world is the “ground truth” so-to-speak – that’s the core message of ecological psychology. The environment is the milieu we are born into and we can never escape it – so of course any prediction machine will take advantage of the fact that the body and the world are always there.
Take gravity for instance. It’s always there as a background condition. So there’s little need to build a complex representational model of it – rather we simply sample gravity when needed. In Bayesian language the presence of gravity has a very high prior, about as high as anything in the world of Earthly creatures. It’s factored into our predictions for bodily actions but I imagine the generative model is very basic because as you say the supervisor is the world itself.
Thanks again for this interesting post and your willingness to consider approaches that seem prima facie opposed to your entire book. That takes intellectual openness, humility, and a willingness to learn from others, virtues much needed in today’s battlefield that is philosophy of mind.
Hi Gary – I think you’re right in highlighting the idea that prediction error is minimized under guidance from the world. That is where the interface between world and mind is, so if we understand what is going on there we might be able to grasp how PEM and EEE may connect.
One way to put that idea in context is to consider the debate about supervised and self-supervised systems. It seems clear that a supervised system will not be able to explain perception, since the problem just moves to the perception of the supervisor. But the solution cannot be a literally self-supervised system, since it would be magic if such an insulated system could latch on to real things in the world.
It seems to me PEM is what offers the middle ground, something that is self-supervised, but given sensory input. Here it matters too that we are talking about empirical Bayes, where priors are shaped over time to reflect frequencies in the input.
I am not confident I ever really understood the idea that the world is its own best model. Maybe I am being overly literal here but the world is not a model, it just is. And I have trouble with the general idea of something being its own model. Specifically, I don’t think the way empirical Bayes relies on sampling the world is what Brooks and others had in mind. The sensory samples from the world play a role in PEM only against the explicit, deeply hierarchical, generative model.
Very often, the more seamless and fluent the interaction with the world, the more it relies on controlled, long-term, deep modeling. Similarly, active inference (sampling the world to make the input conform with expectations) only makes sense if it happens on the basis of an internal representation of the world.
The gravity example is good to think about. Gravity is a constant interacting factor, so it would be informing most of perceptual inference; it is more general than say the expectation that light comes from above. I guess though that it would not take much to undermine this prior, if relying on it begins to generate much prediction error (e.g., going to space). The only priors that we don’t deviate from are interoceptive ones, relating to homeostasis and such. That is our ground truth and we act to make it a self-fulfilling prophecy. The more expansive, exteroceptive model of the world comes along as a ‘bonus’ in our attempt at maintaining ourselves within these expected homeostatic states.
This means that representations are central to understanding how PEM works, even though PEM is not primarily designed to be an explanation of representation of the world. I find this very attractive: we discover that this system, which acts to maintain itself, must represent the world. (From a computational point of view, there are other ways of approaching the question of representation, which are more friendly to the Gibsonian view, see in particular Nicoletta Orlandi’s work).
(There are however some further, complicating moves here, since the reason we engage in PEM is that we cannot know our model directly, which means that actually PEM is aimed at a representational task of sorts. PEM allows us to approximate the true model (i.e., our expected states relative which surprisal is measured) because the sum of prediction error (i.e., the free energy) bounds the surprise).