93

Rich conscious perception outside focal attention Ned Block New York University, New York, NY, USA Can we consciously see more items at once than can be held in visual working memory? This question has eluded resolution because the ultimate evidence is subjects’ reports in which phenomenal consciousness is filtered through working memory. However, a new technique makes use of the fact that unattended ‘ensemble properties’ can be detected ‘for free’ without decreasing working memory capacity. Efforts to pinpoint consciousness in the brain have foundered on the issue of distinguishing phenomenal consciousness (the experience of the ‘redness of red’) from access consciousness in which a phenomenally conscious percept is made available to cognitive mechanisms of reporting, reasoning, evaluating, planning, and the like. The problem is that the ultimate basis of evidence about consciousness is first-person reports in which phenomenal consciousness is filtered through our cognitive access to it via perceptual working memory. Those who like Stanislas Dehaene and his colleagues have embraced the intractability of empirically separating phenomenal from access consciousness have identified frontal–parietal ‘global workspace’ networks as the basis of phenomenal consciousness. In Dehaene’s 2014 book, he says that when ‘the prefrontal cortex does not gain access to…[a] message, it cannot be broadly shared and therefore remains unconscious’ [1]. However, those who have favored trying to separate phenomenal from access consciousness have found evidence for a neural basis of consciousness in perceptual areas in the back of the head [2,3]. One locus of controversy has concerned the issue of the ‘richness’ of perception. ‘Change blindness’ and ‘inattentional blindness’ phenomena in which subjects fail to notice objects in front of their face seem to support sparse perception conscious perception is limited to the contents of visual working memory, roughly three or four things at a time in many standard paradigms. However ‘iconic memory’ supports richer perception. In the Sperling phenomenon, subjects are shown a brief presentation of a grid of letters, say a four-by-three array. They say they see all or almost all of them but can report only three or four items. However, if subjects are cued after the stimulus is gone regarding which row to report, they can report three or four from any cued row, arguing for rich perception. In this and similar paradigms, the seemingly irresolvable issue has been whether the rich perception is conscious rather than unconscious [4,5]. However, several new experimental paradigms have shown how to distinguish empirically between phenomenal consciousness and the cognitive basis of reports of phenomenal consciousness. One dramatic advance is reported in Bronfman et al. [6] that takes advantage of a well-known phenomenon. In a task that demands focal attention, subjects show little or no decrement in performance from a secondary task in which they report ‘ensemble properties’ or ‘gists’ concerning items that are not focally attended. Bronfman et al. used stimuli of 24 letters in four rows with colors that varied in diversity in both the cued and uncued rows. The diversity level could be high, covering samples from the whole color wheel, or low, covering about one-third of the color wheel. One row was pre-cued (attracting focal attention), an array was briefly presented, and the subject had to first report a letter (indicated by a box) from the previously cued row and then decide between high and low color diversity in either the cued or the uncued row (as specified in advance of the trial) (Figure 1). Naming the letters was described as the main task and it was emphasized that in the color-diversity judgment there was no right answer. Feedback was provided for the letter task but not for the color-diversity task. The astonishing result was that the accuracy of colordiversity judgments was almost as high for the uncued and therefore not focally attended rows as for the cued rows. Furthermore, these color-diversity judgments came for free in that the subjects could retain about three items from the cued row independently of whether they were reporting color diversity for the cued or uncued rows. Could it be that the subjects were ignoring letters on some trials and ignoring colors on other trials? No, because there was no negative correlation between tasks. Could it be that color-diversity judgments were based on unconscious color perception? There were two different manipulations intended to exclude that possibility. First, subjects were asked to press an escape button if they did not see colors in the uncued rows, and there were catch trials with colorless uncued rows. Subjects were 93% accurate on the catch trials but no subject pressed the escape button when the uncued rows were colored. In another variant, the presentation of the array was reduced from 300 ms to 16.7 ms and masks were introduced to decrease the visibility of the array. In addition, the subjects were asked to give a visibility rating just before giving the diversity judgment. There was a strong correlation between visibility ratings and accuracy on the diversity judgment. At the lowest visibility level, diversity judgments were at chance; at the highest visibility level, diversity judgments were 80% accurate. Further, judgments of color averages could be made with the lowest visibility ratings (i.e., unconscious perception), but color-diversity judgments required conscious perception. A simulation of color-average judgments compared with color-diversity judgments in conditions of varying noise Spotlights 1364-6613/ 2014 Published by Elsevier Ltd. http://dx.doi.org/10.1016/j.tics.2014.05.007 Trends in Cognitive Sciences, September 2014, Vol. 18, No. 9 445showed that color-average judgments were not affected much by increasing levels of noise. Thus, color-average judgments would be expected to be robust to the degradation involved in unconscious perception. However, colordiversity judgments were strongly impacted by noise to about the same degree as judgments of single colors. The fact that subjects have almost as much awareness of color diversity in uncued rows as in cued rows suggests awareness of individual colors that are not focally attended above the capacity of visual working memory, supporting the rich view of visual consciousness. It may be said that awareness of the unattended colors is of low precision, but several lines of evidence suggest that, although decreasing attention decreases accuracy, it does not substantially decrease precision [7,8]. How does this experiment get around the problem that evidence of phenomenal consciousness is always filtered through the global workspace? Ensemble perception of the kind reported by Bronfman reveals that there must have been conscious awareness of specific colors beyond the limits of the global workspace because a trace of that conscious awareness in the form of a diversity judgment can enter the global workspace for free. See Box 1 for another paradigm that suggests contents of phenomenal consciousness beyond the global neuronal workspace. References 1 Dehaene, S. (2014) Consciousness and the Brain: Deciphering How the Brain Codes Our Thoughts, Viking Box 1. Passive Responding. Another paradigm that disentangles the neural basis of conscious perception from the neural basis of report involves binocular rivalry, a visual phenomenon in which different images are presented to the two eyes, resulting in a percept in which the whole visual field alternates between the two images. For example, if one eye is presented with a grid moving left and the other with a grid moving right, the subject experiences first one direction then the other, repeating indefinitely (Figure 2) Many experiments have shown that, as subjects report their changing percepts, activations change both in visual areas in the back of the head and in global broadcasting areas in the front, and this is taken to support the global broadcasting theory. Wolfgang Einha¨user devised a method that did not require reports [9]. Small eye movements called ‘nystagmus’ indicate which direction the subject was experiencing. (Nystagmus also occurs in binocular rivalry in low doses of ‘dissociative anesthesia’ [10], but in conscious perception nystagmus indicates conscious rivalry.) Nystagmus correlated 83% with reports and better than reports with the shifting neural networks underlying binocular rivalry. Subjects’ judgments validate the nystagmus method, but once the method was validated the experimenters put subjects in a scanner viewing rivalrous images for 61-s periods with no task at all. The result was that with no task there was no differential frontal brain activity. The authors conclude that previous results that showed frontal global workspace changes in binocular rivalry reflected the self-monitoring required to make a response, but that when no response was required there was little or no monitoring and so no differential frontal activity despite the shifting conscious states. This experiment provides evidence that the contents of perceptual phenomenal consciousness are based in the perceptual areas that analyze the stimuli without substantial involvement of shifting global neuronal workspace neurons as the basis of the shifting percepts. What is not ruled out by this type of experiment are shifting insubstantial frontal activations such as ‘pointers’ to perceptual contents. (A) (C) (B) Non-cued rows ‘low’ Relevant-row cue 300 ms 4X6 le er array 300 ms Blank interval 900 ms Spa al le er-cue un l response Diversity esma on un l response Color-diversity? Low/high ? Cued-rows ‘low’ Cued-rows ‘high’ Non-cued rows ‘high’ TRENDS in Cognitive Sciences Figure 1. (A) High and low levels of color diversity, high being all 19 colors and low being a chunk of six colors. (B) Four examples of combinations of high and low diversity in both the cued and uncued rows. (C) The order of events. First, a cue that specifies a row, then an array, then the subject names one of the letters in the cued row (indicated by a box), then (in some blocks of trials) a color-diversity judgment. In all but one of the experiments, naming the letters was specified to be the main task (for which feedback was provided to the subjects about their performance) whereas the color-diversity judgments were treated as purely subjective with no right answer and no feedback. Diversity estimations in some blocks of trials were of cued rows and some of uncued rows. The author thanks Marius Usher for permission to use this figure. Le eye Time Time Gaze x-pos Gaze x-pos Right eye TRENDS in Cognitive Sciences Figure 2. The top row shows the stimuli a green grid moving leftward that is fed to one eye and a red grid moving rightward that is fed to the other eye. The bottom row shows the nystagmus patterns that indicate which direction of motion the subject experiences. Reproduced with the permission of the Society for Neuroscience. Spotlights Trends in Cognitive Sciences September 2014, Vol. 18, No. 9 4462 Vandenbroucke, A.R.E. et al. (2014) Seeing without knowing: neural signatures of perceptual inference in the absence of report. J. Cogn. Neurosci. 26, 955–969 3 Block, N. (2011) Perceptual consciousness overflows cognitive access. Trends Cogn. Sci. 15, 567–575 4 Cohen, M. and Dennett, D. (2011) Consciousness cannot be separated from function. Trends Cogn. Sci. 15, 358–364 5 Phillips, I.B. (2011) Perception and iconic memory: what Sperling doesn’t show. Mind Lang. 26, 381–411 6 Bronfman, Z. et al. (2014) We see more than we can report: ‘cost free’ color phenomenality outside focal attention. Psychol. Sci. 25, 1394–1403 7 Asplund, C. et al. (2014) The attentional blink reveals the probabilistic nature of discrete conscious perception. Psychol. Sci. 25, 824–831 8 Ling, S. et al. (2009) How spatial and feature-based attention affect the gain and tuning of population responses. Vision Res. 49, 1194–1204 9 Fra¨ssle, S. et al. (2014) Binocular rivalry: frontal activity relates to introspection and action but not to perception. J. Neurosci. 34, 17381747 10 Leopold, D. et al. (2002) Visual processing in the ketamineanesthetized monkey: optokinetic and blood oxygenation leveldependent responses. Exp. Brain Res. 143, 359–372 Inferior frontal junction biases perception through neural synchrony Yaoda Xu Harvard University, 33 Kirkland Street, Room 780, Cambridge, MA 02138, USA How the primate attentional control network interacts with posterior sensory regions to bias perception is not fully understood. Using magnetoencephalography (MEG) supplemented by functional magnetic resonance imaging (fMRI), a recent study reported that human inferior frontal junction (IFJ) could play a key role in biasing perception through neural synchrony with posterior sensory regions. Our visual world is filled with a continuous influx of visual information. The ability to attend, select, and encode what is most relevant to the task at hand is vital to everyday visual cognition. For example, when driving on a busy street, we need to attend constantly to road signs, moving cars, and pedestrians while ignoring a host of other nonessential visual inputs. Decades of cognitive neuroscience research have provided us with rich details of the neural mechanisms mediating attention and selection. In their influential review work, Corbetta and Shulman [1] outlined two parietal-frontal circuits that are involved in the control of top-down and bottom-up visual attention. Later work by Yantis and Serences [2] highlighted the role of superior parietal lobule (SPL) in initiating attentional switching between different locations, different features of the same attended object, and different input modalities. More recently, Xu and Chun [3] showed that goal-directed object representation can be achieved by distinct parietal mechanisms in a two-step process with the inferior intraparietal sulcus (IPS) selecting objects via their locations and the superior IPS encoding the detailed features of the selected objects. In recent monkey neurophysiological studies [4,5], neural synchronies among frontal, parietal, and sensory regions have been shown to play an important role in attentional modulation of sensory processing. Despite these advances, little research has examined in humans how the attentional control network interacts with posterior sensory regions to bias perception, and whether neural synchrony plays a role in this process. In a recent study, using MEG supplemented by fMRI to optimize both temporal and spatial resolution, Baldauf and Desimone [6] addressed this pivotal question. They presented human subjects with two streams of images, one containing a sequential presentation of faces and the other houses. The two streams were presented at slightly different temporal frequencies to allow each stream to be tagged by a unique frequency. Baldauf and Desimone overlapped the two image streams at the exact same spatial location and asked subjects to attend either the faces or the houses, and detect a one-back image repetition in the attended stream. They then used fMRI to localize two sensory regions that show preference to the processing of faces and houses, which are the fusiform face area (FFA) [7] and the parahippocampal place area (PPA) [8], respectively. Baldauf and Desimone also localized a brain region involved in non-spatial attention in the IFJ. Using these regions of interests to guide the analysis of MEG signals, they examined the power of MEG signals at the tagging frequency for faces and houses. They found that FFA was more responsive to the face tagging frequency only when faces were attended and, similarly, PPA was more responsive to the house tagging frequency only when houses were attended. Thus, sensory responses in FFA and PPA were modulated by top-down attention according to the task demand. Interestingly, the IFJ responded to the tagging frequency of the attended objects regardless of whether faces or houses were attended. In other words, IFJ was synchronized with FFA when faces were attended, and with PPA when houses were attended (Figure 1). By analyzing the phase-lags between IFJ and FFA/PPA, Baldauf and Desimone further discovered that IFJ was leading FFA/PPA with a constant time-lag of about 20 ms. The IFJ thus appeared to be the driver of the synchrony. Additional analysis with diffusion tensor imaging confirmed that IFJ is connected to both FFA and PPA, providing anatomical support for its role in biasing perception in posterior regions through neural synchrony.