I am delighted to announce the next symposium in our series on articles from Neuroscience of Consciousness. We have two types of symposia. For primarily theoretical articles, we will have several commentators from a variety of theoretical perspectives. For novel empirical research, we will have single commentators whose goal is to bring out the theoretical challenges and import of the results. This symposium is based on Haun et al.’s compelling theoretical paper, “Are we underestimating the richness of visual experience?” We have excellent commentaries from Nico Orlandi, Ian Phillips, and Cristian Giron, Hakwan Lau, and J.D. Knotts. These are followed by a response from Haun et al.
***
How rich is visual experience? Answering this question is important for theories of (perceptual) consciousness. A growing trend in consciousness science suggests that subjects’ impressions of a rich, detailed visual world are incorrect. Phenomena of change blindness and inattentional blindness—on which subjects fail to report aspects of the visual scene that should be clearly visible if perception were rich—are often taken to show that perception is not, in fact, rich. Similarly, subjects do not report a lack of color detail in their peripheral visual field, despite the lack of cones in peripheral, as opposed to foveal, retinal areas. If one believe that consciousness is not rich, then one needs a theory that explains subjects’ reports of rich experiences. One proposal, recently summarized by Cohen, Dennett, and Kanwisher (2016), is that perception represents ensemble statistics. Rather than representing each detail, perception exploits “structure, regularity, and redundancy” in the environment to provide a summary of the scene. Subjects then introspectively mistake this summary for a detailed representation.
In the target paper for this symposium, Haun, Tononi, Koch, and Tsuchiya respond to this trend, arguing in favor of rich experience. If experience is indeed rich, then we need no error theory for introspective reports of richness—we can take perceivers at their word. On the other hand, we must account for subjects’ failures to report on all of the visual details in the scene. Haun et al. suggest that, rather than stemming from sparse perception, failure to report on detail is an artifact of standard experimental setups. First, they argue that experiments on peripheral color do not show lack of color perception in the periphery, but only lack of resolution. If a uniform field of small colored dots is presented to a subject, they will be less able to report on the color of the peripheral dots. However, if the dots are scaled in size to capture differences in resolution between foveal and peripheral vision, then subjects can in fact report on peripheral color.
Second, and more generally, Haun et al. suggest that standard psychophysical methods, such as two-alternative forced choice, artificially restrict the kinds of contents that subjects might report on, rather than capturing the aspects of their experience that are in fact rich. These methods “create the illusion that phenomenology is made up entirely of high-level categories that are experimentally convenient to construct and analyze, such as letters or digits, nameable objects, and statistical summaries” (p. 3). But simply asking what letters are present, for instance, in a Sperling task, fails to address the many aspects of the stimulus that subjects do in fact consciously experience, such as their complex shapes and spatial relations. If this is right, then we need new methods to assess the actual limitations of subjects’ experience—candidates include studying subjects’ natural speech about their experiences, as well as studying the “drawn reports” of artistically talented subjects. If such methods uncover more richness than standard psychophysical measures, it will go a long way towards substantiating everyday introspection about visual experience.
References
Cohen, M. A., Dennett, D. C., & Kanwisher, N. (2016). What is the bandwidth of perceptual experience? Trends in Cognitive Sciences, 20(5), 324-335.
***
Thanks very much to our contributors for participating. Thanks also to Jakob Hohwy and the other editors of Neuroscience of Consciousness, and to Oxford University Press. Please feel free to comment in the discussion board below!
***
[expand title=”Nico Orlandi: Symposium on ‘Are we underestimating the richness of visual experience?’ by A.W. Haun, G. Tononi, C. Koch and N. Tsuchiya.”]
Nico Orlandi – UC Santa Cruz[1]
According to the authors of this compelling piece, we are indeed underestimating the richness of visual experience. Solid psychophysical evidence – Haun et al. argue – supports subjective reports of a richly detailed experiential phenomenology, while studies reporting poor performance employ a problematic methodology. The requirement that experimental participants generate only pre-defined, high-level categorical responses is questionable, according to the authors. For example, the classic Sperling experiment that tests subjects on arrays of letters where some arrays are primed with an audible sound (Sperling 1960), introduces “strictures” that bind experiential bandwidth. Subjects are limited to reporting what letters they saw (Haun et al, p. 3). More open-ended questions about visual arrays would better bring out the richness of perceptual experience.
Here, I leave aside the issue of methodology, noting only in passing that there are studies that stand in tension with the authors’ thesis while following the methodology they recommend. Simons et al.’s (2000) experiment using gradual change is one example. In this study, observers are asked to report any change they notice in a presented scene. This study shows significant blindness to change even though it uses more open-ended questions. Similarly, the Sperling paradigm has been repeated employing less ‘high-level’ categories. In one study, subjects have to signal if there is a change in segment orientation between two displays (Vandenbroucke et al. 2011). These studies can be taken to speak against the richness of visual content despite following Haun et al’s methodological guidelines.
In what comes next, however, I focus on the psychophysical evidence that the authors mention in support of their view. This is because I am not clear on how their argument is supposed to work. According to Haun et al., the reliable correspondence between phenomenological report and “first order” psychophysics is worth deeper consideration (p. 1) and it supports the view that visual experience is rich.
How does evidence from psychophysics bear on the issue of the richness of experiential content? Consider psychophysical evidence that concerns color vision, and that speaks in favor of the view that Haun et al. oppose. The human retina is such that cone density varies across the surface of the retina. Cones – photoreceptors that are sensitive to color – are much more densely assembled in the fovea. As we move away from the central fovea, cone density decreases. The impression of a colorful periphery in the visual field, the reasoning goes, must then be illusory. First-person reports of colorful periphery should be explained either as effects of memory, or as effects of some type of “filling-in”.
In contrast to this type of reasoning, Haun et al. cite three studies that they take to show that peripheral color experience is not essentially different from foveal color experience (Anstis, 1998; Block 2007; Tyler, 2015). The studies scale stimuli according to cortical magnification factor (more below).
Of the three studies cited, only one (Tyler 2015) explicitly argues for some type of continuity of color sensitivity across the retina. Part of the argument is based on the consideration that “despite the high concentration of cones in the fovea, even the central 5° of the retina contains only about 50,000 cones (1% of the total), while the remainder of the total population of about 5 million cones is distributed throughout the peripheral retina with an average density of about 5,000 cones/mm2 (beyond about 10° eccentricity)” (Tyler 2015, p. 1). This type of evidence, however, seems to only show that peripheral color vision is possible because of the presence of cones at the periphery. The evidence does not show that peripheral color vision is of the same type as foveal color vision. Presumably density is the variable that makes the difference.
Of course, depending on what Haun et al. mean by essentially different they may be right that peripheral color experience is not essentially different from foveal color experience. Since cones are present in the periphery, there is no essential difference. But this is something that the opposition can grant, while insisting that, because of a difference in density, reports of continuous color that extends to the periphery are, in some sense, illusory. The use of scaling of the stimuli according to cortical magnification factor confirms, rather than contradicting, this position.
From Anstis (1998) we learn that “The visual field is mapped topographically on the surface of the striate cortex in humans; the projection is large for the central visual field and is progressively compressed toward the periphery. Visual acuity decreases with distance from the fovea in proportion to the estimated cortical magnification factor, CMF.” (ibid, p. 817) CMF is the number of millimetres on the cortex to which one degree on the retina projects. The central retina is so dominant “that about 25% of striate cortex is devoted to processing the central 2.5 deg of the visual scene.” (ibid, p. 818).
So, given these psychophysical facts, how do we get subjects to see what is in the periphery in a similar way to how they see what is in the center of the visual field? We magnify the stimulus in the periphery. When stimuli are “M-scaled”, the near periphery resembles foveal vision. An example of an M-scaled stimulus is an eye-chart with small letters near the middle and progressively larger letters further out (Anstis 1998, p. 818).
Tyler (2015) uses M-scaling of color patches where small patches in the center are surrounded by much larger patches in the periphery. In this way, Tyler aims to show the efficacy of peripheral color vision.
But this type of method, and this type of evidence, does not to support the richness of ordinary visual experience. This is because, in ordinary visual experience, objects in the periphery are not M-scaled. To the contrary, the evidence seems to support the view that ordinary perceptual experience is sparse. As Anstis notes, for some visual tasks, such as flicker frequency tasks, two types of scaling are required: M-scaling and F-scaling in which illuminance is multiplied (Anstis 1998, p. 818). Neither one of these two types of scaling ordinarily occur. And for some visual competences, M-scaling does not matter. Performance on some tasks falls off with eccentricity, even after stimuli have been M-scaled: “Examples are the detection of circular disks (Bijl et al, 1992), [and] the detectability of mirror symmetry in line-segment patterns (Saatinen 1987) (…)” (ibid p. 818). This leads Anstis to wonder why the whole visual field normally looks equally sharp all over despite there being “an enormous degradation of the visible detail in peripheral vision” (ibid, p. 820). His answer is not that there is no such degradation, but that the question is ill-posed because peripheral vision cannot detect such degradation, and so we cannot become aware of it.
In sum, the core of the evidence provided by Haun et al. seems to leave untouched the idea that perceptual experience is sparse, particularly in the periphery. Contrary to what the authors aim to establish, perceptual reports of continuous vision throughout the visual field do not appear to be psychophysically plausible. There is some sense, of course, in which peripheral vision is not essentially different from foveal vision, but this is something that champions of the limitations of perceptual content can readily admit.
References
Anstis, S. (1998) “Picturing Peripheral Acuity,” Perception, volume 27, pp. 817—825.
Simons, D. J. Franconeri, S. L. Reimer, R.L. (2000) “Change Blindness in the Absence of a Visual Disruption.” Perception, vol 29, Issue 10.
Sperling G. (1960) “The information available in brief visual presentations.” Psychol Monogr Gen Appl;74:1–29.
Tyler CW. (2015) “Peripheral color demo.” I-Perception; 6: pp. 1–5.
Vandenbroucke, A., Sligte, I., and Lamme, V. (2011) “Manipulations of attention dissociate fragile Visual Short-Term Memory from Visual Working Memory.” Neuropsychologia, 49, 1559-68.
[1] Thanks to Dan Burnston for comments on an early draft of this critique.
[/expand]
[expand title=”Ian Phillips: Commentary on Commentary on Haun et al. (2017) ‘Are we underestimating the richness of visual experience?'”]
Ian Phillips
University of Birmingham and Princeton University
Two issues are often elided in work on phenomenal consciousness. First, whether experience is rich or sparse. Second, whether phenomenal experience overflows cognitive access (e.g. Block 2007). Many theorists assume that experience could only be rich if it overflowed cognitive access. Though they do not put it in these terms, Haun et al. (2017) raise an important challenge to this elision. They also call for a reorientation of the field in which experimentalists no longer “treat phenomenology as a doubtful hypothesis, but as a thing to be ‘explained’”. I wholeheartedly agree. We should take seriously how experience seems to us from the inside, and we also take seriously the idea that conscious experience as it seems is inextricably bound-up with cognitive function. Here I expand on this reading of Haun et al.’s paper, whilst raising two sets of critical issues.
Reclaiming richness
Rich and sparse are relative terms. Any theorist using them ought to explain their meaning: rich in these respects; richer than this. Haun et al. are less clear than one might hope here. Rich could mean: richer than what is available to cognition. Yet I doubt Haun et al. would espouse any such gloss. Much of their argument is that richness is supported by psychophysics. Psychophysical methods presume that the contents of experience can be exploited by subjects and so reveal themselves in non-chance performance in relevant discrimination tasks. Such methods thus assume that conscious experience is minimally accessible to cognition. Thus, if (as is argued) psychophysics supports “naïve phenomenology”, the prospect emerges of rich and accessible phenomenal consciousness.
Still we need to know what is meant by richness. Haun et al. rightly insist that we can see more than three or four selectively attended items. However, they find fault with appeals to ensemble statistics as a supplement to such sparse contents (e.g. Cohen et al. 2016). I remain unpersuaded. Haun et al. describe ensemble statistics as “features of visual experience that are about detailed structure in the world” (p. 1). Are such contents not precisely of the required richness, e.g. to explain the data in De Gardelle et al. 2009? In any case, Haun et al.’s main complaint against the ensemble statistics programme strikes me as unfair. This is that “when the emphasis is on experimental rigor it does not appear that any known summary statistic is capable of capturing the appearance of peripheral vision” (p. 2). In support they cite Wallis et al. 2016. Yet, Wallis et al. consider just two statistical image transformations: a Gaussian blur manipulation (Geisler and Perry 1998) and a texture synthesis manipulation (Portilla and Simoncelli 2000). Furthermore, they put these manipulations to an extremely rigorous test. We should not then be surprised that these two models require refinement and modification. Vision is extremely complex. Note that Wallis et al. are themselves perfectly content with thinking of peripheral vision as relatively “compressed or lossy” (p. 1). The issue is simply: how so? In short, it seems precipitous to dismiss the appeal to ensemble statistics on the basis of early failures to pass extremely rigorous tests.
Returning to richness, Haun et al. talk about visual experience as having “a very densely detailed structure”, and urge us to take seriously a subject’s reports “of colorful, clear experience extending across her visual field” (p. 2). However, beyond (quite rightly) insisting that colour is not absent from peripheral vision, it is not obvious what this amounts to. We are told that “peripheral color experience is not essentially different from foveal color experience” and has no more to do “with memory, expectation or selective attention” (ibid.). We are also told that it is not “blurry” (ibid.). Yet it is also acknowledged that there is a dramatic difference in objective resolution (and recognitional capacity) in the periphery. So there is plainly a difference in degree if not kind. Where does this leave us? Haun et al. conclude that the work they cite on peripheral vision “undermine[s] a popular criticism of introspective reports as illusory or even delusional” (ibid.). Yet they cite Anstis who, in his discussion, notes that although “a large page of text, fixated at its centre, creates the illusion of being equally sharp and equally legible all over … when one attends carefully to the peripheral parts of the page, while maintaining strict fixation … one becomes aware that these parts are illegible” (1998, p. 821). This sounds much closer to Cohen et al. (2016) than Haun et al. allow.
Finally, whilst Haun et al. focus on the positive, it is worth noting that several of the studies they cite precisely aim to show a gap between what is seen and what we think we see (e.g. Ward et al. 2016 and De Gardelle et al. 2009). It would be good to know what Haun et al. make of such studies, and so how naïve they think we can really be.[1]
Loosening the straightjacket
A second key message of Haun et al.’s piece is that experimentalists must “loosen[…] the straightjacket of simple discrimination and identification tasks” if we are to gauge the true richness of consciousness. Such tasks “fail to adequately deal with the complexity of experience, and create the illusion that phenomenology is made up entirely of high-level categories that are experimentally convenient to construct and analyze, such as letters or digits, nameable objects, and statistical summaries” (pp. 2-3).
To illustrate, they consider Sperling’s famous partial report task. According to Haun et al., this is often cited “to emphasize the limited capacity of phenomenal vision, since subjects can report at most 3-4 of 12 briefly flashed letters” (p. 3). Against this they argue that if “only they were asked, subjects could report much more” even of a sterile static letter array than a mere clutch of letters. They would report “that there [were] many black marks … arranged in rows and columns, in a rectangular array, without depth [or] motion, within a rectangular display, against a bright homogeneous background that [was] spatially extended, being composed of a multitude of distinguishable locations” etc. Similarly, they continue, “subjects can report, with high confidence, what they did not see—such as the burning Twin Towers, a Bernese Mountain dog, the President, and so on.” (ibid.)
I found this discussion puzzling for two reasons. First, pace Haun et al., Sperling is not cited “to emphasize the limited capacity of phenomenal vision”. Sperling’s partial report methodology was intended, and is widely cited, to argue that subjects see much more than they can report (e.g. Block 2007, but see Phillips 2011). Given this, it would have been a distraction to consider whether subjects saw and/or could recall other aspects of the display. Sperling’s focus on the letter array allowed him (and subsequent researchers) to probe various aspects of partial report superiority with great precision and build models of visual processing and storage accordingly. It would be just a confusion to infer from this methodology “that phenomenology is made up entirely of high-level categories that are experimentally convenient to construct and analyze”.
Second, great care is needed in moving between what subjects correctly report and what they see. For example, plausibly, we don’t notice everything we see. So, where failures to report are due to failures to notice, they will not show that nothing was seen. This is a familiar criticism of inattentional blindness studies (for a pertinent such study see Persuh and Melara 2016, for the familiar criticism see e.g. Block 2011). There are also well-known issues concerning responses bias which may lead subjective report to under- or overstate the contents of experience. Haun et al. are well aware of this of course. Yet it is somewhat misleading to say that “exactly [these] challenges [are] faced by traditional … approaches to consciousness science” (p. 3). Simplified tasks are used because it is vastly easier to analyse the decision space in such tasks. So whilst there may be real merit in expanding the range of tasks and questions asked of subjects it is vital that this be done in conjunction with rigorous analysis to avoid misinterpreting the data. Relatedly, it should be noted that whilst Haun et al. complain about the straightjacket of traditional psychophysics, many of the studies they cite to support their viewpoint fit tightly within it. Thus, Wallis et al. 2016 explicitly adopt a forced-choice paradigm arguing that it is “preferable to a rating method (…) to measure objective sensitivity” (p. 3).
Conclusion
Despite these various concerns, there is much to be enthusiastic about in Haun et al.’s paper. It is just that our enthusiasm, be it for richness or for revolutionary techniques, must be tempered—especially given the troubled history of consciousness science in both respects.
References
Anstis, S. 1998. Picturing peripheral acuity. Perception 27: 817–26.
Block, N. 2007. Consciousness, accessibility, and the mesh between psychology and neuroscience. Behavioral and Brain Sciences 30: 481–499.
Block, N. 2011. Perceptual consciousness overflows cognitive access. Trends in Cognitive Sciences December 15(12): 567–575.
Cohen, M. A., Dennett, D. C., and Kanwisher, N. 2016. What is the bandwidth of perceptual experience? Trends in Cognitive Sciences 20: 324–35.
De Gardelle, V., Sackur, J., and Kouider, S. 2009. Perceptual illusions in brief visual presentations. Consciousness and Cognition 18: 569–77.
Geisler, W. S., and Perry, J. S. 1998. Real-time foveated multiresolution system for low-bandwidth video communication. In B. E. Rogowitz and T. N. Pappas (eds) Proceedings of the SPIE, 3299, human vision and electronic imaging III. San Jose, CA: SPIE, pp. 294–305.
Haun, A. M., Tononi, G., Koch, C. and Tsuchiya, N. 2017. Are we underestimating the richness of visual experience? Neuroscience of Consciousness 3(1): 1–4.
Persuh, M., and Melara, R. D. 2016. Barack Obama Blindness (BOB): Absence of Visual Awareness to a Single Object. Frontiers in Human Neuroscience 10(118): 1–6.
Phillips, I. B. 2011. Perception and iconic memory: what Sperling doesn’t show. Mind & Language 26(4): 381–41.
Portilla, J., and Simoncelli, E. P. 2000. A parametric texture model based on joint statistics of complex wavelet coefficients. International Journal of Computer Vision 40(l): 49–70.
Vandenbroucke, A. R. E., Sligte, I. G., Barrett, A. B., Seth, A. K., Fahrenfort, J. J., and Lamme, V. A. F. 2014. Accurate metacognition for visual sensory memory representations. Psychological Science 25: 861–73.
Wallis, T. S. A., Bethge, M., and Wichmann, F. A. 2016. Testing models of peripheral encoding using metamerism in an oddity paradigm. Journal of Vision 16(4): 1–30.
Ward, E. J., Bear, A., and Scholl, B. J. Can you perceive ensembles without perceiving individuals?: the role of statistical perception in determining whether awareness overflows access. Cognition 152: 78–86.
[1] In defence of naïveté, Haun et al. vaunt a study by Vandenbroucke et al. (2014) said to “show that attention-bound working memory and pre-attentive sensory (iconic) memory are equally accessible to introspection, debunking the claim that the latter is unconscious or illusory” (Haun et al. 2017, p. 2). Yet Vandenbroucke et al.’s study is problematic. The basic issue is that although metacognitive accuracy is found to be roughly the same across iconic, fragile and working memory, metacognition was only measurable for cued representations. Yet such representations are those which, in Vandenbroucke et al.’s words, have been “made robust and available for report and for cognitive manipulations” (2014, p. 861). So all parties to the debate will predict this data. The critical issue is whether we can generalize from cued to uncued representations. But to generalize would, in effect, be to beg the question by assuming that such representations are conscious independently of being made robust and available for report.
[/expand]
[expand title=”Cristian G. Giron, Hakwan Lau, & J.D. Knotts: Are Open Interviews Superior to Button Presses? A Commentary on Haun et al. (2017)”]
Cristian G. Giron 1 , Hakwan Lau 1,2,3 , & J.D. Knotts 1
1 – Department of Psychology, UCLA, Los Angeles, 90095, USA.
2 – Brain Research Institute, UCLA, Los Angeles, 90095, USA.
3 – Department of Psychology, University of Hong Kong, Pokfulam Road, Hong Kong
Correspondence: C.G. (giron@ucla.edu) and J.D.K. (jeffereydknotts@gmail.com).
In their recent commentary, Haun et al. (2017) argue that, despite the recent arguments for a ‘sparse’ interpretation of visual phenomenology based on the peripheral ensemble statistics hypothesis (Cohen et al., 2016; Ward, Bear, & Scholl, 2016; Odegaard & Lau, 2016), the ‘rich’ interpretation of visual phenomenology is still the one that best fits the empirical evidence. More specifically, they first argue that the ensemble statistics hypothesis inherently discredits introspection more than experimental evidence suggests it should, therefore an alternative hypothesis that assumes greater reliability in introspection, e.g., Block’s phenomenological overflow (1995, 2007), should be preferred. They then suggest that alternative experimental methods that do more to take advantage of this reliable introspection (e.g., by allowing study participants to freely describe what they see) will ultimately be more useful than traditional forced-choice psychophysics paradigms in settling the debate about the richness of phenomenology. We disagree with these main points of Haun et al. (2017).
To argue that the reliability of introspection is unfairly undermined by the ensemble statistics hypothesis, Haun et al. (2017) direct our attention to peripheral color perception. In a recent review on ensemble statistics, Cohen et al. (2016) describe an exercise [Box 2 of their paper] whereby it is easy to show that there are points in our periphery at which we can detect moving objects, but cannot identify their color. Despite this compelling demonstration, the fact remains that we do not typically perceive the periphery to be colorless. This is not to say reliable introspection concerning phenomenology always has to conform to our perceptual abilities; there can be unconscious perceptual abilities that are unreflected by introspection (Lau & Rosenthal, 2011). But here the perceptual ability seems limited, suggesting that color representation is weak. If, introspectively, one thinks there is more richness, this is unlikely based on actual detailed color representations. This is in line with a long tradition of research, including a classic study by Nisbett and Wilson (1977) leading to the established consensus within experimental psychology that introspective reports are often based on unrealistic confabulations. Often, such reports could not possibly be based on factual knowledge.
Haun et al. (2017) argue that the reason we cannot identify the color of certain objects (e.g., a playing card or an orange, both examples used in Cohen et al., 2016) in the periphery despite detecting their motion is simply that those objects are too small for the peripheral color system, with its lower density of cones and smaller proportion of dedicated cortex, to resolve. That is to say, if the objects were enlarged sufficiently to account for receptive field size differences between the center and the periphery, then we would be able to identify color in the periphery (Tyler 2015). While this observation is true, and is useful for understanding color sensitivity mechanisms and physiology, this does not change the fact that color sensitivity is poor in the periphery in real life. Objects in the real world do not, on their own, enlarge upon entering our periphery to compensate for differences in receptive field sizes. The mismatch between introspection and color sensitivity we have been discussing so far concerns these realistic cases. This mismatch does not go away just because in contrived settings the color sensitivity in the periphery can be compensated for.
There is also substantial empirical evidence that is not discussed in Haun et al. (2017), which similarly suggests that the phenomenological richness that is generally inferred from visual introspection is unreliable. Change and inattentional blindness are two of the most famous examples that challenge this intuition. For example, in a classic inattentional blindness paradigm, subjects can fail to detect a salient black square presented at the fovea while attending to a peripheral size discrimination task (Mack & Rock, 1999). Also relevant here are experiments that clearly demonstrate subjective filling-in of color in peripheral vision (Balas & Sinha, 2007). One could argue that these introspective reports of uniform colorfulness and detail in the unattended background may be phenomenologically real. But again, at least in the case of color, it is unclear if such large regions of the visual field can be ‘filled in’ via top-down mechanisms, in the sense of being enriched with actual colorful details at every peripheral location. Unlike the case of the blindspot, it is unlikely that the brain can figure out such a large amount of details instantly. If the details aren’t represented, but one introspectively thinks they are, such introspection cannot be truly reliable.
There is also the tendency for subjects to use relatively liberal detection criteria for peripheral or minimally attended vision compared to central or attended vision, despite matched detection sensitivity (i.e., peripheral inflation – Rahnev et al., 2011; Solovey et al., 2014). Indeed, it has been speculated that this form of peripheral inflation may explain the mismatch between the feeling of richness gained from introspection and the suboptimal performance that is consistently demonstrated in peripheral tasks (Cohen et al., 2016; Lau and Brown, in press; Rahnev et al., 2011).
However, there has also been an unfortunate tendency to focus on metacognitive sensitivity measures, and to ignore the kind of biases reported by Rahnev et al. (2011) and Solovey et al. (2014). For example, based on a lack of difference in metacognitive sensitivity, it was argued that introspective phenomenology under some conditions of inattention is real (Vandenbroucke et al., 2014). But the same study reported evidence in support of subjective inflation too, also in the form of detection criterion bias. It is unclear why such biases are ignored, as they can capture important aspects of subjective perception (Witt et al., 2015; Peters, Ro, & Lau, 2016). Of course, if one systematically ignores such critical psychophysical measures, it is not surprising that there would be an underappreciation of the usefulness of forced-choice measurements.
Finally, an interesting question arises from the hypothetical free-form report provided by Haun et al. (2017) in the second half of their commentary where they promote new, less constrained paradigms for probing visual introspection. Their hypothetical subject reports that in an array found in a typical Sperling task (e.g., a 3 x 4 array of black letters on a white background (Sperling, 1960)), “there are many black marks, that they are arranged in rows and columns, in a rectangular array, without depth nor [sic] motion, within a rectangular display, against a bright homogeneous background that is spatially extended, being composed of a multitude of distinguishable locations, each with its specific neighbors”. Such a report, they argue, would provide evidence for the phenomenal representation of the Sperling array being richly detailed as opposed to being partially composed of summaries or gists, à la ensemble statistics. However, if this hypothetical report does not constitute a gist, then it is on the authors to explain where they think the line between gist perception and rich perception should be drawn. Drawing this line is clearly not a trivial task, and this difficulty is essentially just one more example of how hard it is to measure the contents of phenomenology in a way that is actually meaningful for distinguishing between the rich and sparse hypotheses. So while we support the forward-thinking nature of Haun’s et al. proposal, more work is needed to characterize the details of these reports, and their potential mismatch from reality. Ironically, such work will likely depend on the very psychophysical analyses that Haun et al. criticize.
[/expand]
[expand title=”Andrew Haun, Giulio Tononi, Christof Koch, and Naotsugu Tsuchiya: Response to Commentaries“]
Andrew M. Haun1, Giulio Tononi1, Christof Koch2, and Naotsugu Tsuchiya3,4
1 – Department of Psychiatry, University of Wisconsin-Madison, USA
2 – Allen Institute for Brain Science, Seattle, USA
3 – School of Psychological Sciences, Monash University, Australia
4 – Monash Institute of Cognitive and Clinical Neuroscience, Monash University, Australia
Our response is divided into a few common threads that appeared in the three commentaries:
1. Color in the visual periphery
“But here [in the periphery] the perceptual ability seems limited, suggesting that color representation is weak. If, introspectively, one thinks there is more richness, this is unlikely based on actual detailed color representations… The mismatch between introspection and color sensitivity … does not go away just because in contrived settings the color sensitivity can be compensated for.” –Giron et al
Between the three commentaries there is a common resistance to our argument as it regards the nature of color experience. This resistance is exemplified by Giron et al’s comments. The difficulty is part of what we hoped to dispel with the article: in talking about the contents of consciousness, we must distinguish between what those contents are like for us, and what some outside observer might deduce by e.g. doing psychophysics experiments using physical stimuli. As Orlandi says:
“…because of a difference in density, reports of continuous color that extends to the periphery are, in some sense, illusory.”
This “in some sense” is exactly the sense that matters for quantifying perceptual experience (its ‘bandwidth’). If we have an ‘illusory’ experience, it is still a real experience. Illusion is a mismatch between subjective experience and objective stimulus (e.g. “I see color but there is no colored stimulus there; I see no color but there is some colored stimulus there”). Cases like the experiment of Balas and Sinha (Balas & Sinha, 2007), mentioned by Giron et al, are especially interesting in this respect, since we cannot take the colorfulness of the stimulus as a ground truth for the contents of experience: the stimulus may be colorless, but if the subject claims it was colorful, we must not immediately discount their report just because it is objectively incorrect.
Color perception experiments have shown over and over that peripheral color experience is qualitatively similar to foveal color experience (e.g. Gordon & Abramov, 1977; Parry, McKeefry, & Murray, 2006; Sakurai, Ayama, & Kumagai, 2003; Webster, Halen, Meyers, Winkler, & Werner, 2010): peripheral and foveal color-stimuli are matched with similar primary mixtures, and are identified with similar labels (‘similar’ is important here – there is gradual, small drift with eccentricity in the hue/brightness/saturation structure of color perception; e.g. see (Abramov, Gordon, & Chan, 1991; Mullen & Kingdom, 2002)). Our subjective impression of colorful peripheral experiences is backed up by objective, psychophysical data. Such impressions can therefore be taken as an informative and, potentially, accurate observations.
This all means that, if one wants to argue that introspection about vision is unreliable and subject to e.g. confabulation, one at least cannot use the comparison of foveal and peripheral color perception, because there is no relevant difference there as far as color experience is concerned. The differences between foveal and peripheral vision concern the relationship between physical stimuli and visual experience (“sensitivity”, in Giron et al’s terms): among other things, peripheral vision has lower spatial resolution. This is what Tyler demonstrates so clearly in his short review of the issue (Tyler, 2015). Another recent review (Rosenholtz, 2016) outlines a comparable case for the richness of the visual periphery. Rosenholtz subscribes to a ‘summary statistic’ view of peripheral vision, but unlike Cohen et al (2016) she finds much support for a spatially-extended visual field that persists beyond the attentional spotlight. She also demonstrates how wrongheaded the notion of the “blurry periphery” is, from a similar psychophysical basis as Tyler.
Ecological validity of foveal/peripheral comparison
Orlandi and Giron et al both take issue with the seeming artificiality of conditions that permit comparison of the qualitative aspects of peripheral color vision:
“… in ordinary visual experience, objects in the periphery are not M-scaled.” – Orlandi
“Objects in the real world do not, on their own, enlarge upon entering our periphery to compensate for differences in receptive field sizes.” – Giron et al
There is a mistaken notion at work here that objects in the world are somehow best-fit by foveal vision, and poorly fit by peripheral vision; but it is well-known that the spatial structure of natural scenes (i.e. how patches of color and brightness, and their boundaries, are distributed in a scene) is scale invariant (Atick & Redlich, 1992; Field, 1987; Ruderman & Bialek, 1994) – i.e., in the real world the amount of ‘stuff’ that a receptive field is presented with is more-or-less independent of the receptive field’s size. Foveal vision is better for seeing small stimuli, but there is nothing special about small stimuli in the natural world, and no relationship between smallness and colorfulness. It follows that, in viewing a broadband, wide-field color stimulus (e.g. the colorful natural world), the color vision system across the entire visual field should be expected to get sufficient input for eliciting color experiences.
- Visual experience does have limits
While we are arguing that ‘naïve’ introspective reports should be taken more seriously, we hope not to be interpreted as going too far: clearly there is much in every visual situation that could be seen but that is, in some real sense, not seen; for example, “a gorilla walking through the room” will not be reported by some subjects in the famous experiment, no matter how they generate their reports, whether it is a 2AFC judgment or a free verbal report. There are certainly limits to the contents of visual experience, and depending on what part of experience we look at, these limits can be surprisingly narrow. So, we do not argue that e.g. inattentional blindness is an artifact of experiment design, and that it would go away if reports were less restricted.
However, even inattentional blindness experiments have understated the contents of what a subject actually experiences during a critical display. In the gorilla video, subjects are experiencing some moving people in black and white, roughly in the center of a computer display, itself roughly in the center of the visual field. The experimenter intentionally places the focus on detection of certain events or objects, or changes – and as interpreters of the result we are drawn to surprising lapses of perception of the experimenter-defined target. But at the same time as these lapses occur, the subject still experiences many other things that experimenters rarely or never think to ask about (e.g., the color of the background, the shape of the display, the spatial relations between objects and their relations to the background; the spatial structure or extent of the surfaces that constitute those objects, and the contours that bind them; the larger spatial field that contains all of these, and its apparent relation to the viewer, etc). If all such contents are considered as a part of a visual experience, then lapses may be less surprising, and a high estimate of “visual bandwidth” seems more warranted.
2.1 Reports about perception can be susceptible to bias
Giron et al refer to several papers showing that SDT bias is more liberal for peripheral detection relative to more conservative for foveal detection (or for unattended vs attended), meaning strictly that in a peripheral detection task, subjects are more prone to false alarms, i.e. deciding that something is there that is not there. However, such bias is virtually irrelevant for perception that is amply suprathreshold, which is what really matters regarding the reported appearance of perceived stimuli (i.e. a stimulus must be perceptible for us to say anything about the perceptual properties it evokes). Visual experience of a stimulus is suprathreshold by definition because subthreshold means nonconscious. In amply suprathreshold experience, detection bias is hardly relevant, or at least, not the central issue. If a patch of color, or an edge, is visible on every trial, as it typically is by design in a matching or identification task, then the ‘signal distribution’ is so far from the ‘noise distribution’ that false alarms are exceedingly (or unmeasurably) rare regardless of bias (Haun et al, in preparation). So we do not see a strong connection here to our argument that the clearly-visible content of normal visual experience is much as it seems to be, but we can concede that there may be a general predisposition towards incorrectly believing we see near-threshold peripheral or unattended stimuli.
- What do we take ‘richness’ and ‘sparseness’ to mean
As Ian Phillips notes, our commentary paper did not give a detailed treatment of our take on ‘richness’ and ‘sparseness’, instead only alluding to these ideas, which leaves much to the reader’s interpretation. Since our piece was inspired by Cohen et al’s opinion article (Cohen, Dennett, & Kanwisher, 2016), let us take their Figure 2 and associated discussion as a case in point.
The figure shows examples of experiment stimuli consisting of arrays of many items, including the 5×5 arrays of colored dots in the top row (reproduced here). In the original experiment by (Brady & Tenenbaum, 2013), subjects made a ‘same or different’ judgment about a pair of stimuli. When certain statistical properties of the array were maintained (the left-side pair above), subjects distinguished pairs of arrays as though they saw only a ‘handful’ of items at a time (i.e. 4 or 5 colored dots); however, when the statistical properties were not maintained (the right-side pair above), subjects distinguished arrays as though they saw almost all of the items. Cohen et al review several similar findings. The explanation that resolves these findings is that, as Cohen et al say:
“… observers encoded a few individual items along with a summary of the statistics of the entire display.” – Cohen et al (2016), page 327
According to their account, this explains why there are conflicting sparse/rich claims regarding visual experience: it’s sparse in the sense of number of distinct things that are perceived (a handful of items/summaries), it’s rich in that summary statistics tell the observer about:
“…the vast amount of information observers have access to at a given instant.” – Cohen et al (2016), page 328.
In other words, some believe human experience is rich because, thanks to the summary statistics, subjects know there is much that they could perceive (by using attention to promote items to “high resolution representations”, in Cohen et al’s terms). But while they have access to this vast amount of information, they do not experience it (Cohen et al appear to use the term “access” to refer to the relation between an experienced summary statistic and the unexperienced information that it is ‘about’, in a kind of end-run around Block’s (Block, 1996) overflow formulation).
However, in our view, Cohen et al’s view of ‘richness’ is miserly. Consider what ‘summary statistics and a few items’ means for phenomenology. When the stimulus is presented, the observer is believed to see a few – maybe three or four – colored discs, and to have a ‘statistical feeling’ that there are other discs, that they are arranged in a square shape, and that they are colored. But, it is believed, there are no actual discs experienced (except the few high-resolution ones), and so they certainly do not have actual colors – and if they did, they would be indeterminate “statistical” colors (“if there were a disc, it would be either red or blue”). What would it be like to experience “probabilities of possible arrangements of probabilistically-colored discs – and a few actual discs”? We find this statistical description to be literally inconceivable in terms of phenomenology – utterly alien to our own definite experience of “an array of many colored discs”.
[/expand]
