101

The Effect is Perceptual Although there has been some controversy over what exactly these results show (Anton-Erxleben, Abrams, & Carrasco, 2010; Carrasco, Fuller, & Ling, 2008; Prinzmetal, Long, & Leonhardt, 2008; Schneider, 2006; Schneider & Komlos, 2008), it has been settled beyond any reasonable doubt that the effect is a genuine perceptual effect rather than any kind of cognitive effect. To begin, consider the idea that there is a cognitive bias to categorize the side where the dot appears as higher in contrast or that there is a tendency to find the orientation of the cued patch easier to judge. Both are made improbable by the fact that the boost to the attended side persisted to the same degree even when the instruction was to report the orientation of the lower contrast patch rather than the higher contrast patch (i.e. the ISI or interstimulus interval). Further, the attentional boost disappeared when the temporal gap between the dot and the patch was increased from 53 ms to 500 ms, which is what one would expect from the decay of transient attention in a perceptual effect but not what one would expect if a slow cognitive effect is governing the result.38 NedBlock Figure 6. if the subject adapts to A (by looking at it for a minute), then looks at the vertical grating in B, it will look slightly tilted to the left as depicted in an exaggerated form in C. Attention has in general been shown to operate via a number of neural effects in the perceptual system. Attention to a stimulus boosts the firing rate of neurons that respond to that spatial area and suppresses the firing rate of neurons that respond to other spatial areas. Further, the degree of enhancement and suppression reflects the degree of attention (Chen et al., 2008; Kastner, 2009; Reynolds, 2008). In general, increasing attention has a similar effect in the visual cortex as increasing contrast itself. This picture is further supported by purely psychological experiments. One impressive psychological effect involves something called the “tilt aftereffect”: if one looks at a visible tilted grating for, say, one minute, then gratings presented immediately afterward that are close in orientation to the original grating appear somewhat rotated away from the original grating. Oneversion is diagrammed in Figure 6. The subject looks at the tilted grating in Figure 6A for a period, say one minute. When the subject looks at the vertical grating of Figure 6B, it will look slightly tilted in a counter-clockwise direction as depicted in an exaggerated form in Figure 6C. The explanation is to be found in the perceptual phenomenon known as “adaptation”, present in all or virtually all sensory systems. In the first few minutes of exposure to a stimulus, receptor response decreases to the orientation, contrast, direction of motion and other perceptual parameters of the stimulus, decreasing the perceptual performance on subsequent stimuli with respect to these parameters (Seri´ es, Stocker, & Simoncelli, 2009). In the case of tilt, receptors sensitive to the orientation of the stimulus undergo a raising of the thresholdAttention and Mental Paint 39 of activation. When the subject adapts to 6A, then looks at the vertical bars (in B), receptors that register a slight tilt to the right are firing more weakly than receptors that register a slight tilt to the left, making the vertical bars look titled to the left. One interesting feature of adaptation to a grating is that within limits, a higher contrast grating raises the threshold for firing more and, further, that raised threshold lasts longer than would be produced by adaptation to a lower contrast grating (Langley, 2002). So, within limits, the higher the contrast of A, the bigger and longer lasting the effect on the tilt depicted in C. Ling and Carrasco (2006) compared the increased size and length of the tilt after-effect accomplished by increasing the contrast of the adapter grating with the increased size and length of the tilt after-effect accomplished by directing voluntary sustained attention to the adapter grating (for periods up to 16 seconds). They found that attention to a 70% contrast grating increased the size and length of the tilt-after-effect just the same as if the actual contrast of the grating had been increased between 11% and 14%. (Different subjects showed slightly different sizes of effect.) That is, a 70% grating plus focal attention had the equivalent adapting effect as an 81%84% grating without focal attention. Increasing the contrast of a particular orientation of a grating has a benefit in making that orientation easier to see, but then after a few seconds it has a cost in making that orientation harder to see because of adaptation. And so Ling and Carrasco were able to show that sustained attention confers a benefit in the first few seconds, then a cost. The article is called “When sustained attention impairs perception” because it demonstrates, surprisingly, that a focally attended stimulus can be harder to see (after a few seconds) because of adaptation. The upshot is that since the attentional effect on perceived contrast affects adaptation, it is a perceptual effect.13 Another study I find impressive in demonstrating the perceptual nature of the effect involves the basic Carrasco paradigm with one major change: attention is cued by a sound on one side or the other instead of by a visible event (St¨ormer, McDonald, & Hillyard, 2009). A sound cue rather than a visual cue also vitiates the allegation that the effect is due to a sensory interaction (Schneider, 2006). Using this paradigm with some other minor changes, St¨ormer et. al. got slightly smaller effects than Carrasco (5% rather than 6%elevation of perceived contrast at the 22% level). They found that the effect correlated with increased activity in the first stages of visual processing (using a brain imaging technique involving a mesh of wires on the scalp). Further, the increase in cortical activity occurred in the same cortical areas found to be sensitive to actual differences in contrast in the stimuli. And the larger the cortical effect, the larger the effect on subjects’ judgments of increased contrast. To sum up: there are strong reasons to suppose that the effects of attention on contrast documented by Carrasco and her colleagues are perceptual.40 NedBlock Where are we? Does the attended Gabor patch look to be higher in contrast as a result of the attention or not? The upshot of the studies I have been talking about is that the effect of attention on perceived contrast is a perceptual effect, not a cognitive effect or due to a response bias. So what? Given that the effect is perceptual, one can ask whether it is a conscious perceptual effect or an unconscious effect or has both a conscious and an unconscious component. One way to get evidence on whether an effect is conscious is to ask the subjects. That works for the Tse Illusion mentioned earlier and indicated in Figure 2. But there is a problem that will be familiar to anyone who has thought about experimental work on consciousness: one cannot simply ask subjects to compare an attended with an unattended stimulus since the question itself would induce attention to the supposedly unattended stimulus. There has been controversy about the issue of whether or to what extent the Carrasco effect is a conscious effect (though the participants are not always as clear as they should be that this is the issue they are discussing), centering around methodological problems involved in asking subjects to make same/different judgments as opposed to “forced choice” judgments in which subjects indicate which item is higher on a given parameter (Schneider, 2006; Schneider & Komlos, 2008; Turatto, Vescovi, & Valsecchi, 2007). This issue has now been resolved by Anton-Erxleben, Abrams and Carrasco (2010) which explores the methodological issues involved in same/different judgments, showing that when properly done, the perceptual effect can be demonstrated with both methodologies. However, the possibility still exists that the effect, though perceptual, is partly unconscious. In order to control the subjects’ attention and eye movements, experimenters have tended to use very brief stimuli (40 msecs100 msecs), which may decrease the conscious component of the effect relative to the unconscious component. Wecan get a bit of illumination on this issue using a different paradigm. There is considerable evidence that when events are simultaneous, attention to one of them can make it appear to occur before the other (Spence & Parise, 2009; Stelmach & Herdman,1991;VanDerBurg,Olivers,Bronkhorst, & Theeuwes, 2008)14. Steimach & Herdman showed subjects a screen containing 3 sets of 4 dots arranged in a square. They asked the subjects to attend to the left, or alternatively to the right, or to the center, always fixating the central square. Dots then appeared inside the 4 dot outline in the leftmost square and the rightmost square. The dots were sometimes simultaneous and other times slightly asynchronous (separated by a brief period). In one experiment, subjects were asked to choose which dot appeared earlier. This was a forced choice between left earlier and right earlier with no option for saying they occurred simultaneously. They found that the dot on the attended side was perceived to occur about 40 msecs earlier than the other dot. For example, the subjects’ point of maximum temporaluncertainty—that is, whenAttention and Mental Paint 41 they were unable to discriminate which came first—came when the dot on the unattended side led by about 40 msecs. Then the experimenters added a third possible response—simultaneity. They found that when attention was to the center, subjects tended to judge simultaneity accurately, but when they attended to either the right or to the left, their judgments of simultaneity dropped precipitously both in absolute number (35% to 4%) and although they trended in the direction of the earlier judgments, the results were weak. This result suggests that there is a substantial unconscious component to the forced choice result. But what about a conscious component? The experimenters then asked the subjects to adjust the difference between times of appearance of the dots in 5 msec increments. They were allowed to go back and forth as many times as they wished until they were satisfied that the dots appeared simultaneously. I really like this procedure because it allows for subjects to make considered phenomenological judgments. Anyone who has been a subject in an experiment in which brief stimuli have to be compared knows that subjects are often uncertain—the stimulus goes by so fast the subjects often feel as if their responses are partly a matter of guessing. Allowing the subjects to try the judgment repeatedly with the ability to change the timing is a way of boosting the credibility of the conscious source of the judgment. The result was that subjects regarded the attended side as simultaneous with the unattended side when the unattended side led by about 40 msecs. The authors give a model of subjects’ behavior which I won’t describe, but the upshot for our purposes is that same/different judgments provide a less adequate indicator of conscious perceptual state than other methods such as the ones described that indicated the 40 msec difference. The issue of the size of a conscious effect for these very brief stimuli can, however, be sidestepped because the attentional effect is visible in the sample stimuli printed on the page. You can see it yourself using Figure 4! In the case of a one-off illusion like the Tse Illusion, the scientific community accepts the illusion because 16 subjects all said they got it and because the reviewers for Vision Research got it, but for a general phenomenon—attention alters perceived contrast—journals have a different perspective. In what follows, I will simply assume that attention does in fact affect consciously perceived contrast, gap size, etc., as Carrasco’s paradigm suggests. Is the phenomenal effect of attention a matter of salience or vividness rather than perceived contrast? First, as the Stelmach experiment just described shows, attention can affect perceived simultaneity as well as contrast. And as I also mentioned, perceived gap size, speed, flicker rate, speed, color saturation and spatial frequency are also affected. The attended item looks bigger, faster, earlier, more saturated, stripier. No doubt increased salience is a result of these changes, but it is not a substitute.42 NedBlock 7. The Argument How can the opponents of mental paint explain these results? I’ll start with the issue as it affects direct realism, though as we will see, the arguments are similar for representationism. Let us begin by assuming that there is no relevant illusion, that is, that the two different percepts of the 22% patch in Figure 4 (the patch on the left) are both veridical or at least non-illusory. I will argue that on the assumption of no illusion, direct realism has no reasonable way of accomodating the results. Then I will argue that there is no relevant illusion. It will be useful to run the entire argument with respect to Figure 4 (and a slight modification of it in Figure 7). Let me first review the facts. The subject’s eyeballs are pointed at the fixation point in the center and in the Carrasco experiments described earlier, the subject attends either to the fixation point or to the figure on the left. (Let’s ignore the cases in which the subject attends to the right.) When the subject attends to the fixation point, the experience of the relative contrasts—that the figure on the right has higher contrast—is veridical. When the subject attends to the figure on the left, the contrast of the figure on the left looks—non-veridically—to be the same in contrast as the figure on the right. There is no difference in the spatial position of the subject with respect to the two patches and no difference in the state of the sense organs or the relation between the sense organs and the layout. The difference between these two percepts of the 22% patch is entirely mental. Consider the subject’s perceptual experience of the 22% patch itself (the one on the left) in the two cases (rather than the perceptual experience of the comparison between the two patches). You may wonder whether the effect of attention really applies to a single patch as opposed to a comparison, where there is competition for the viewer’s attention. Carrasco (Carrasco, Cigdem, & Eckstein, 2000) shows that the increase in sensitivity that underlies the effects reported occurs—though slightly decreased in magnitude—for a single Gabor patch rather than two competing patches.15 To repeat: we are talking about the percept of the 22% Gabor patch itself, not the comparative percept. In the two cases (attention to the left, attention to the center) there are two different phenomenal experiences of the same item with the same relevant instantiated property, i.e. 22% contrast, yet the experiences are different. This is the challenge to direct realism. Attentively seeing and less attentively seeing the same thing—the 22% patch—are experiences that differ phenomenally but not in the item seen or in its instantiated properties. Earlier, I mentioned cases in which opponents of mental paint appealed to differences in actually instantiated properties selected by attention, e.g. face vs house in Figure 1, axes of symmetry in the square/diamond case, rows vs columns in the matrix case. It is not obvious how any such selection of properties can apply here, but it is worth considering whether there areAttention and Mental Paint 43 properties in the display of different levels of specificity, grain or determinacy that could be appealed to (cf. Chalmers, 2004, and Tye, 2005, 2006). For example, any scarlet patch is also red, and perhaps a perceiver can have different experiences of the same color patch depending on which is attended to, red or scarlet. A shift between awareness of red and awareness of scarlet might be phenomenologically significant. In the case of contrast, one could take the corresponding determinable/determinate relation to be, for example, that between medium-low contrast and 22% contrast. However, the results cannot be explained by appeal to a shift between direct awareness of 22% contrast in one case and medium-low contrast in another. For the subject experiences the difference as a difference in contrast at the same level, for example, as a higher specific contrast or specific size, rather than as a move to a more generic or abstract level of contrast or size. There is a related objection in which the two percepts involve different ranges of vagueness. I will take that up in due course. Another line of objection would be to suppose that the subject is directly aware of the focus of attention in the layout, and since the focus of attention shifts, so does the phenomenology. In speaking of the focus of attention, I am indicating the thing or area of space that is attended to, and suggesting that it can be seen as a property of the layout. I think that allowing awareness of such mind-dependent properties of the layout would be a significant concession in direct realist thinking. Direct realists often try to capture aspects of appearance by appeal to a variety of types of relations to the subject. But these are normally characterized non-mentally. (For example, Campbell’s 2002 book mentions spatial relations and jaundice. No¨ e (2004) treats appearances as spatial projections on a plane perpendicular to the line of sight.) Relations to a focus of attention or to a center of salience might fit the letter but not the spirit of these views. Recall the summaries quoted earlier by Brewer (“the subjective qualities of experience…are constituted by the actual spatial distribution of the various displays as these are accessible to the subject,”) and Campbell (“the phenomenal character of your experience, as you look around the room, is constituted by the actual layout of the room itself: which particular objects are there, their intrinsic properties, such as colour and shape, and how they are arranged in relation to one another and to you”). It would be as if they were allowing one kind of mental paint, attentional mental paint. A better form of the idea for the direct realist would be to think of attention as coming into the direct awareness relation itself. But this idea in either of its forms will not help the direct realist, for a highly significant reason, one that I will be returning to again and again. Recall that subjects experience the effects of attention as alterations of perceived contrast, earliness, size, speed, flicker rate and spatial frequency. Appealing to direct awareness of the focus of attention or, alternatively, attentional vs less attentional awareness of the patch is inadequate to explaining that44 NedBlock phenomenology. Attending makes the item attended look, for example, bigger, so it is inadequate to say that it looks more attended to even if it does look more attended to. The same point applies to salience conceived as a phenomenal property of the layout—as mentioned earlier. It is certainly true that an increase in perceived earliness, size, flicker rate, spatial frequency and contrast bring with them an increase in salience. But looking bigger is not just a matter of looking more salient even if it involves looking more salient. Earlier I said that the positive thesis of Moorean transparency was important to my case. This is one place where it comes in. The effect of attention is experienced in terms of appearance of contrast, speed, size, color saturation, etc. Attended things look bigger, faster, more saturated and higher in contrast even if they also look more attended and more salient. So far, I have been bracketing the issue that is most significant, whether one or both experiences of the 22% patch are illusory. Just what this comes to and how it avoids the problem depends on the direct realist’s theory of illusion, which is the topic of the next section. The discussion to follow concerns the question of whether there is an illusory percept of the 22% patch when the subject attends to the fixation point—or, alternatively to the 22% patch itself. I always mean the issue of whether there is an illusory percept of the 22% patch (not the other patch). As before, I am ignoring the cases in which the 28% patch is attended, and I will speak of the attended patch and the unattended patch for brevity when what I really have in mind is the more attended vs less attended patch. There are a number of different views that finger one or the other patch as seen illusorily. The percept of the unattended patch can be accused of being illusory, or the percept of the unattended patch can be exonerated, the percept of the attended patch being accused instead. There is a straightforward problem with both views, namely that they only make sense on what one might call a “steady spotlight” model of attention and that model is false. The model has both a spatial and a temporal dimension. I’ll start with the spatial aspect, the idea of attention as a spotlight (Posner, Snyder, & Davidson, 1980). The spotlight model was known to be oversimple in the 1980s because it was shown that attentional effects are substantial far from the center of one’s attentional field, but it was thought that attentional effects could be thought of in terms of a “gradient of attention” (Downing & Pinker, 1985), which would be compatible with a fuzzy spotlight. However, more recent work shows that the spotlight model is in far worse shape than previously thought. One basic failure is that there are two attentional fields, an excitatory field and a suppressive field. A second basic failure is that both fields are quite large and irregular (Datta & DeYoe, 2009; Hopf et al., 2006). How large? Hopf, et al. show an excitatory effect from targets in one quadrant of the visual field (varying in strength) over perhaps half of the visual cortex and a suppressive field, also quiteAttention and Mental Paint 45 variable, that is only slightly smaller. Datta & DeYoe were able to tell which of 18 areas subjects were attending to with 100% accuracy just by eyeballing the scans, but for some areas, attention spread radially for distances up to more than half the width of the stimulus grid, which itself occupied 56◦ of visual angle. (The fovea, the central high density part of the retina, responds to only about 2◦ of visual angle.) Datta & DeYoe note that “…attention is actively modulating visual processing throughout the field of view, not just in the local vicinity of the target” (2009, p. 1044). They suggest that the replacement of the spotlight metaphor by the “gradient of attention” metaphor is inadequate, and we should think in terms of the “landscape of attention.”