I am glad to kick off our latest Mind & Language symposium on Jean-Rémy Martin and François Le Corre‘s “Sensory Substitution Is Substitution ,” from the journal’s April 2015 issue, with commentaries by Kevin Connolly (Mellon Postdoctoral Fellow, Penn), Ophelia Deroy (Center for the Study of the Senses/Institute of Philosophy, University of London), Julian Kiverstein (Amsterdam), and Michael Proulx (Bath).
Sensory substitution devices (SSDs) convert images from a video camera into patterns of vibrotactile stimulation (White et al. 1970; Bach-y-Rita 1972, 2004), electrotactile stimulation (Sampaio et al. 2001, Ptito et al. 2005, Chebat et al. 2011), or auditory stimulation (Meijer 1992, Capelle et al. 1998, Renier et al. 2005, Amedi et al. 2007, Kim & Zatorre 2008) that visually-impaired individuals can use to perform tasks ordinarily guided by non-prosthetic vision, including object recognition and navigation. In addition to these abilities, SSD-users often report a dramatic experiential shift, sometimes referred to as “distal attribution” (Loomis 1982, Epstein 1986): they come to experience the stimulations produced by the SSD as transparently revealing the spatial layout of objects in distal, 3D space. Bach-y-Rita et al., discussing their experiments on visual-to-tactile sensory substitution, write: “Our subjects spontaneously report the external localization of stimuli in that sensory information seems to come from in front of the camera, rather than from the vibrotactors on their back” (1969, p. 964; quoted by Martin & Le Corre 2015, p. 210).
Martin and Le Corre distinguish between two main approaches to understanding the representational end-product(s) of training with an SSD. According to what they call the “Deference View”, SSDs that substitute for vision give rise in trained users to modally visual perceptual states (e.g. Bach-y-Rita et al.1969; Hurley & Noë, 2003; Noë, 2004; O’Regan & Noë 2001; Renier et al. 2006). By contrast, according to the “Dominance View,” these devices give rise to perceptual states that belong to the substituting modality, e.g., touch in the case of a visual-to-tactile SSD.
Martin and Le Corre propose an alternative that they refer to as the “Metamodal Spatial View.” Like proponents of Deference and Dominance, they accept that the substituting modality plays a causal role in producing the experiential shift characteristic of distal attribution. However, they argue, contrary to both views, that the resulting phenomenology belongs neither to the substituted (deference) nor to the substituting (dominance) modality.
There is a large body of evidence that many processing areas in the brain perform metamodal rather than modality-specific perceptual functions, that is, they perform the same types of information-processing tasks in response to sensory information received from different peripheral input systems (Pascual-Leone & Hamilton, 2001, Merabet et al. 2007). Increasingly, researchers have defended the view that the brain, in general, is metamodal. On this view, as Michael Proulx writes in his commentary, “the brain is not a sensory-organized information processing device, but a task-based computational device.”
Martin and Le Corre argue that just as certain brain areas can be properly characterized as metamodal, certain kinds of information can be characterized as metamodal as well. Metamodal information, as they put it, “is a kind of information that is potentially accessible by any modality.” Information about distal 3D layout, in particular, is not, strictly speaking, visual information, on their view, but rather metamodal information. Hence, the fact that trained subjects experientially access such spatial information, Martin and Le Corre maintain, doesn’t favor the Deference View. On the other hand, they argue, the phenomenology of distal attribution speaks against the Dominance View: “if SSD-users after the experiential shift have distal experiences and do not feel the substituting stimulations anymore, then Dominance seems to be in a delicate position. It is indeed difficult to claim that tactile stimulations, in the case of a visuo-tactile device, are projected in the external space” (p. 211). This leaves the Metamodal Spatial View, which Martin and Le Corre defend on the basis of phenomenological, psychological, and neuroscientific grounds.
(I might note that Martin and Le Corre’s use of the terms “deference” and “dominance” in their article departs somewhat from that of Susan Hurley and Alva Noë in their influential “Neural Plasticity and Consciousness” [Biology & Philosophy 18: pp. 131–168, 2003]. Hurley and Noë use these terms to distinguish between cases in which the qualitative character of a subject’s experience in SSD-perception is determined by the area of cortex that is activated by input from the SSD [this is what they call “cortical dominance”] and cases in which it is determined instead by the source of input to that cortical area [this is what they call “cortical deference”]. According to Hurley and Noë, in visual-to-tactile sensory substitution, activity in somatosensory cortex takes its qualitative expression from the character of the “externally rerouted” visual input that it receives. Hence, visual-to-tactile sensory substitution, as they see it, is a case of somatosensory cortical deference. By contrast, if occipital processing areas are activated by inputs from SSDs as suggested by current research [for a review, see section 3.2 of the target article] and visual phenomenology results, then this would be a case of cortical dominance as Hurley and Noë use the term.)
Comments on this post will be open for at least a couple of weeks. Many thanks to Jean-Rémy, François, Julian, Kevin, Michael, and Ophelia for all their hard work. I’m also very grateful to Sam Guttenplan, the other Minds & Language editors, and the staff at Wiley-Blackwell (especially Imogen Clarke) for their continued support of these symposia!
Below there are links to the target article, commentaries, and Jean-Rémy and François’ replies.
Target Article: Martin and Le Corre, “Sensory Substitution Is Substitution”
- Kevin Connolly, “Just How Metamodal Is Sensory Substitution?”
- Ophelia Deroy, “Why go meta-?”
- Julian Kiverstein, “In Defence of Deference”
- Michael Proulx, “The brain is not just a sensory machine: how one can see with sound using sensory substitution”
Jean-Rémy Martin and François Le Corre’s replies:
Cover image: Pietro Paolini’s “Allegory of the Five Senses”, via Wiki Commons