Reconstructing the movie in your head

Looking at MRI signals in visual cortex, researchers were able to reconstruct, with varying degrees of success, the movie clips people were watching (examples below in movie). The most obvious applications we could extrapolate to are for folks who are paralyzed, or perhaps even during dreaming.

What I find most interesting starts around 30 seconds into the movie, when it shows the training movies with the highest posterior probabilities. The influence of the limited training data can clearly be seen. Coming up with an optimal set of training data will likely be a very active area of research in the future.



The study experimentally verifies the Cartesian Theater theory of mind, proving that conscious experiences are simply movies that play in our brains.


  1. Tad

    Really? I don’t see this at all! Nobody doubts that different parts of the brain carry information about different parts of a perceived scene, and that a “film” can be reconstructed from this information. How does that prove that there is something (someone?) in your brain watching such a film in ordinary, everyday perception? I mean how do video cameras work? You can take the information in some medium (e.g., film, or flash memory) and reconstruct the scene that gave rise to it with a projector. Does that mean there’s a little homunculus in the video camera watching the scene while it’s being recorded?

    In any case, those reconstructed films are so unlike my visual phenomenology, that I actually took them to show how unlikely the Cartesian Theater model is!

  2. Eric Thomson

    Arnold: the study was on visual cortex, my hunch is that they would reconstruct the exact same movies even if they played them to people anesthetized with their eyes propped open. They didn’t come close to getting at questions about consciousness or attention. That was what I was joking about, as this study doesn’t address the meaty issues at all. Not to fault it for this: that was not its goal. That they were able to pull out information about dynamic stimuli (using these slow MRI signals) was impressive enough for me.

    Incidentally, supplementing MRI with more temporally-sensitive measures (e.g., EEG, MEG) might help with the reconstruction efforts for these rapidly-changing stimuli.

    However, like you, I do not share the philosophers’ strong aversion to Cartesian Theaters. I should qualify that by saying it depends on how they define the theater, as there are a few different definitions out there.

    For instance, I am happy with thinking that experiences happen in some temporal order. That is, I take it that there is a “literal fact of the matter” about whether I had experience X before experience Y.  That is enough, on some interpretations, to make me think there is a Cartesian Theater: if so, buy me a ticket baby!

    Another version of the Cartesian Theater involves invoking dreaded homunculi. Then there is the anatomical version, in which there is supposed to be an anatomical locus where “it all comes together” (I discussed the anatomical Cartesian theater here).

    I think the notion of a Cartesian Theater has been largely useless for framing debates about consciousness. I brought it up in the context of this study just to be a troublemaker.

  3. Forget about “an anatomical *locus* where it all comes together”. Think instead of a neuronal *system* where it all comes together within an egocentric frame. See my responses in your discussion of the anatomical Cartesian theater.

  4. Eric Thomson

    I think we are on the same page, Arnold, as is verified in the comment thread at that post on the anatomical theater. A good discussion there, actually, that I had forgotten about!

    My response to Marcin basically endorses the Theater metaphor (and yours does the same).

    The impression I get is that philosophers are loathe to endorse the Cartesian Theater, but most actually believe that one exists. This suggests sociological forces at work within professional philosophy. Perhaps the wrath of Dennett? I don’t know. I’m just shooting from the hip here obviously.

  5. Thank you, Richard (I think), for calling my attention to this very interesting discussion. I really like the idea of using a version of this task to predict fMRI responses (or local field potentials in animals, or even single units in epileptics and animals). Then doing a kind of Bayesian analysis. Very cool idea.
    I should mention quickly that I’ve advocated a NON-Cartesian “theater” metaphor for a long time, and that Dan Dennett and other philosophers have been very clear about that. So it’s not susceptible to the old Cartesian theory critique.

    It’s also worth saying (again) that Global Workspace Theory (GWT) is derived from Alan Newell (et al) work in Artificial Intelligence, which has never been vulnerable to theater critiques. Nevertheless, GWT shows both convergence onto a set of integrated “gestalts” that correspond well to the contents of consciousness (given certain conditions), and which are plausibly distributed to unconscious networks in the brain. In that sense the theater metaphor provides a handy thinking tool for understanding, e.g., brain processes that are likely to be unconscious, and those that demonstrably underlie conscious contents. The quick version is that the ventral stream of vision enables visual features and objects that can be described and “pointed to” with extraordinary accuracy, that the dorsal stream does not, and that subcortical regions (like cerebellum) don’t either. After a lot of years of thinking about the brain-consciousness relationship, I think we’re finally settling on a satisfactory AND TESTABLE interpretation.

    I do NOT think that any of this has a bearing on the mind-body problem when it is phrased METAPHYSICALLY, as a relationships between a physical and a non-physical universe of discourse. What we do have is two (or more) perspectives on the same reality, one of which is private and the other can be shared with others. As far as I can see this viewpoint raises no metaphysical or logical problems at all. Which is a good thing, because the science on all this is still so early that we’re not ready to tackle the equivalent of quantum mechanics. If you don’t have to tackle the dinosaurs, so much the better.

    Thank you again for letting me know about this very interesting thread.

Comments are closed.