74

Responses to my critics Ned Block 1. Adaptation, signal detection and the purposes of perception: reply to Ian Phillips and Chaz Firestone Ian Phillips and Chaz Firestone have written a wonderful article on the rationale for adaptation as an indicator of perception, and more generally, on the purpose of perception, full of insights and challenges. 1.1 Adaptation The issue they raise that I find the most interesting and challenging, and that I didn’t say enough about in the book, is whether there is any independent justification for adaptation as an indicator of perception or whether my reliance on phenomenology (and also retinotopy) to ground adaptation makes adaptation superfluous. I will approach the issue by reminding the reader of my three-layer methodology as explained in Chapter 1. Here are the three layers: (i) use armchair criteria of perception and of cognition to roughly delineate the categories of perception and cognition. (ii) Use those categories to isolate scientific indicators. In particular, I chose perceptual adaptation, rivalry, pop-out, illusory contours and speed of processing, but as I indicated, I could have picked many other indicators. (iii) Use the scientific indicators to isolate the underlying constitutive features of perception and of cognition. As I also explain, the use of a variety of scientific indicators raises a problem of circularity. The problem is that the justification of any given indicator depends on invocations of other indicators. I argued that the circularity is benign so long as the indicators converge on the same results and those results match up better with the armchair criteria than they would have with alternatives. My case for benign circularity is threatened by the issue raised by Phillips and Firestone of whether some of the indicators play no real role at all. In particular, do I validate adaptation by appealing to retinotopy and phenomAnalysis Vol. 83 | Number 3 | July 2023 | 575–588 https://doi.org/10.1093/analys/anac058 © The Author(s) 2023. Published by Oxford University Press on behalf of The Analysis Trust. All rights reserved. For permissions, please email: journals.permissions@oup.com Downloaded from https://academic.oup.com/analysis/article/83/3/575/7441531 by New York University Libraries user on 09 February 2024576 | book symposium enology, where retinotopy and phenomenology just stand on their own? If so, they argue, adaptation has no independent significance as an indicator. I’ll start with a case that fits their criticism in which an indicator has no independent significance: in binocular rivalry, a type of eye movement called optokinetic nystagmus is used as an indicator of which stimulus the subject is consciously aware of. Of course, the eye movements’ role as indicators is validated by the subject’s phenomenological judgements, but the eye movements are nonetheless incredibly useful in circumstances in which we do not want the subject to make such judgements, because these judgements would register on brain scans and make it more difficult to isolate the neural basis of the experience itself (this methodology is known as the ‘no report’ paradigm). The general point is that we can have an extremely useful nonphenomenological indicator that is entirely based on phenomenology. Still, optokinetic nystagmus has no independent significance as an indicator. Is adaptation in the same derivative boat as optokinetic nystagmus? I will explain why adaptation is different. To approach this issue, we need to distinguish between high-level and low-level perceptual phenomenology. Recall that low-level perception is the immediate product of sensory transduction and is the causal basis of high-level perception, that is, perception that is not the immediate product of sensory transduction but rather depends on further processing. Notable low-level perceptual representations represent shape, size, texture, hue, motion, depth, contrast and brightness. High-level perceptions mentioned in the book are perceptions of faces, emotional expressions, causation and numerosity. Look at Figure 1 in Phillips’ and Firestone’s article (reproduced here for convenience), specifically at the top halves of parts A, B and C. In A(i), it is perfectly clear from the first person that one is perceiving colours, with hues of red, yellow, blue and green. In B(i), one perceives slightly clockwise tilts on the top and slightly counter-clockwise tilts on the bottom. And in C(i), one perceives red and black vertical stripes and green and black horizontal stripes. We can be sure from the phenomenology that we visually attribute these qualities. If one stares at the (i) parts of these figures and then at the (ii) parts, one experiences the adaptation phenomena in which the perceptual attributions change, but the change in perceptual attributions carries no force that is not already carried by the phenomenology of the pre-adaptation perception. So far, adaptation is playing no real role, just as Phillips and Firestone suggest. Although the phenomenology of perception is sufficiently recognizable to tell us that we are perceiving, it isn’t much use in distinguishing low-level from high-level perception or either one from the non-ampliative perceptual judgement that is most directly based on perception this is minimal, immediate, direct perceptual judgement; see the critique by Steven Gross in this issue for more discussion of this kind of perceptual judgement. When I say ‘perceptual judgement’ in what follows, I mean minimal immediate direct perceptual judgement). Something can look expensive or look like a piano or a pine tree (examples I use in the book). But one cannot tell from the phenomenology whether Downloaded from https://academic.oup.com/analysis/article/83/3/575/7441531 by New York University Libraries user on 09 February 2024book symposium | 577 when something looks expensive; that stems from perceptually representing it as expensive – or whether what one is experiencing is the phenomenology of the colour, shape, texture and other low-level properties, together with the perceptual judgement that it is expensive (or is a piano or a pine tree). (This issue is also discussed in (Begby 2011, Block 2014, Siegel and Byrne 2016)). That is, introspection is useless in deciding between (i) high-level perception and (ii) low-level perception without high-level perception but with high-level perceptual judgement. Of course, there will be high-level perceptual judgement in both (i) and (ii), but in (ii), the high-level perceptual judgement is based on low-level perception. Without appealing to adaptation, we can’t use introspection to distinguish high-level phenomenology from low-level phenomenology, and we can’t use introspection to distinguish either one from perceptual judgement. So how exactly does adaptation help with the problem? Look at the adaptation stimuli in Figure 1D. After staring at a fixation point in the top image A B C (i) (ii) D (i) (ii) (i) (ii) (i) (ii) (iii) E (i) (ii) Downloaded from https://academic.oup.com/analysis/article/83/3/575/7441531 by New York University Libraries user on 09 February 2024 Figure 1 From Phillips and Firestone.578 | book symposium for 30–60 seconds, and then looking at the releaser stimulus in the bottom part of the image, what you experience is that the left cloud of dots on the bottom briefly looks to have fewer dots than the right cloud of dots. Then, after a fraction of a second, they look to have the same number of dots. The explanation is that we have spatiotopic visual channels for high numerosity and low numerosity. Focusing on the top of D decreases the sensitivity for high numerosity on the left and decreases the sensitivity for low numerosity on the right, yielding the repulsive effect when looking at the bottom of D. As I mentioned, students in my classes who view these stimuli often suppose that I have surreptitiously changed the stimulus. The phenomenology changes as a result of adaptation even though the stimulus stays the same. But why is this change in phenomenology any more significant than the static phenomenology prior to the change? We can rule out the most significant low-level explanation of the change due to adaptation: it is clear from introspection that dots do not seem to appear or disappear despite the changes in apparent numerosity. After staring at the top part of D and then looking at the bottom part, it just looks from the get-go that there are more dots on the right and fewer on the left. And then it seems the numerosities are equal on the two sides, without any appearance or disappearance of dots (see for yourself!). The one potential low-level confound is whether the operative change might be in texture density rather than numerosity. This is an empirical issue that has been explored with great ingenuity by creating stimuli in which texture density and numerosity are varied independently. One particularly impressive experiment, summarized in Block 2019, added lines connecting dots, increasing the texture density while decreasing the apparent numerosity. The adaptation effects depended on the numerosity, not the density. The phenomenology of adaptation is part of the package of evidence that shows that numerosity perception is at least in part a high-level perceptual phenomenon. But the fact that it is the phenomenology of adaptation that I am appealing to does not show adaptation is playing no real role, since static phenomenonology – without adaptation – would not suffice for establishing the result. A top-down source of the change in perceptual judgement in D is very unlikely since one has no tendency to believe that the paper stimulus actually changes. So the change in perceptual judgement is plausibly due to the change in high-level perception. And that suggests that we don’t have to appeal to any supposed phenomenology of high-level perceptual judgement to explain the effect. And the reasoning just discussed depends essentially on adaptation, so adaptation is playing a real role. In sum, in distinguishing low- from high-level perception and distinguishing both from perceptual judgement, adaptation plays a real role. I haven’t said anything yet, though, about whether adaptation plays an independent role in ruling out ‘criterion’ effects. Decision criteria influence how much perceptual information the subject needs to respond to in a Downloaded from https://academic.oup.com/analysis/article/83/3/575/7441531 by New York University Libraries user on 09 February 2024book symposium | 579 certain way. Consider a ‘detection’ task in which the subject is supposed to press one button if there is a stimulus and another if there is no stimulus. If there are very many no-stimulus trials, subjects will tend towards a ‘conservative’ criterion in which they must be quite sure there is a stimulus to push the yes button. And if you reward the subject for detecting the stimulus without penalizing them for ‘false alarms’ (incorrect yes button presses), the subjects will respond by liberalizing detection, pressing the yes button without being very sure there is a stimulus. Can a ‘criterion’ effect explain away the evidence for numerosity perception? No, because the changes in phenomenology of first seeing the bottom part of D as having more dots on the right and then equal numbers on both sides is clearly a change in perceptual phenomenology and not at all like the phenomenology of a changing criterion of the sort described in the last paragraph. Note that although the appeal here is to phenomenology, it is the phenomenology of adaptation that plays a real role. 1.1.1 Retinotopy and spatiotopy Moving to retinotopy and spatiotopy, retinotopic effects depend on where light falls on the retina, and they typically move with the eye. Spatiotopic effects preserve retinal neighbourhood relations but do not move with the eye. The adaptation effects in D and E are spatiotopic because they depend on whether the adaptor and the releaser are on the left or right side of space. The left cloud of dots in D(ii) looks to be fewer than they are after the adaptation because the left cloud of dots in D(i) was numerous. So why do we need adaptation at all in reasoning about numerosity perception, given that we know which dots are on which side of space and which dots fall on which parts of the retina? The answer is that these facts tell us nothing about numerosity perception, whether there is numerosity perception at all or alternatively, whether vision just attributes low-level properties and we make judgements about numerosity. The reasoning I went through a few paragraphs ago that distinguishes low-level from high-level perception and both from perceptual judgement all depended on adaptation. 2. Signal detection theory I will now move to a disagreement between me and Phillips and Firestone on the proper role of signal detection theory. Signal detection theory is pervasive in their reply though it is only discussed explicitly near the end. Their discussion makes heavy use of the signal detection theory notion of a ‘criterion’. I’ll start with the issue they raise at the end of their article since it reflects in a simple and straightforward manner why I think the signal detection theory framework is problematic in application to the issues of my book. The issue is my thesis that perception is a winner-takes-all faculty. I argue that there is always a dominant perception and that a likely explanation is that the Downloaded from https://academic.oup.com/analysis/article/83/3/575/7441531 by New York University Libraries user on 09 February 2024580 | book symposium purpose of perception encourages the design of a perceptual system that does not wallow in ambiguity. As I note in Chapter 4, the dominant percept can take a number of forms. In one, the rival percepts merge, and in another, the rival percepts compete. In binocular rivalry, there is an interplay of local and global features in determining winners. Two faces – one projected to each eye – can be sufficiently ‘compatible’ in a sense described in Chapter 4 for representations of face stimuli to merge when the subject is attending holistically to the face but to compete when the subject is attending to local features. In particular, a masculine and feminine face (one projected to each eye, as shown in Chapter 4) results in a percept of an androgynous face when the subject attends to the face as a whole. But when the subject attends to parts, the two faces alternate. See Figure 2, reproduced from Chapter 4. As Phillips and Firestone note, it is not easy to see how a winner-takesall nature of perception is compatible with a signal detection framework in which there is a continuous balance between distributions representing different percepts. This shows an inadequacy in signal detection theory: it concerns the information content of perception, not the percept itself. Let me explain. Figure 2 Binocular rivalry stimuli. The masculine face is presented to the left eye and the feminine face to the right eye. If the subject is attending to local features or parts such as the eyes, standard binocular rivalry ensues. This is indicated by the label ‘alternating percepts’. If the subject is attending to holistic features such as gender or the identity of the person, the subject sees a morphed androgynous face, as pictured. Thanks to Chris Klink for this figure. See Klink et al. 2017. Downloaded from https://academic.oup.com/analysis/article/83/3/575/7441531 by New York University Libraries user on 09 February 2024book symposium | 581 As Phillips and Firestone note, in a stimulus situation in which stimuli are degraded, there is substantial information in subjects’ second and even third ‘guesses’. But the second and subsequent choices reflect the information in a non-dominant percept that is not conscious, or as I will say here with misgivings that I cannot go into, not present to the person. It is true that the information reflected in the second and subsequent choices are represented in the visual system. When a subject has a percept as of an androgynous (merged) face, the information from the masculine and feminine faces is present in an implicit form. When the subject attends to the parts of the face, and when the feminine face is dominant, there is no hint of the masculine face at the personal level (although there can be patchy versions of both that briefly appear in the transition). The dominant face is the only one present in the person-level percept, although the information about the suppressed face is still represented in the perceptual system. Signal detection theory concerns the information represented in the visual system, but without emendation, it does not concern the percept itself. It is a defect in signal detection theory that it has no notion of what is present at the personal level, and this failure stems from the lack of thresholds in signal detection frameworks. Matthias Michel and Hakwan Lau suggest, following Michel and Lau 2021, Witt et al. 2015, a modification of signal detection theory in which thresholds are introduced as ‘perceptual criteria’, as distinct from the report or decision criteria usually understood as part of signal detection theory. Decision criteria influence how much perceptual information the subject needs to respond to in a certain way. To use a variant of the previous example, if in a detection task you reward the subject for correct no-stimulus judgements without penalizing them for incorrect no-stimulus judgements, the effect will be to shift the subject temporarily towards a ‘conservative’ bias in deciding on and reporting a stimulus, that is, the subject will only say the stimulus was there when they are pretty sure it was. The idea of a perceptual criterion is that signals whose strength is less than the perceptual criterion are unconsciously perceived.1 Second choice accuracy can be accounted for by such a model. A low-threshold model in which the noise distribution is sometimes over the threshold would also explain confident false alarms since confidence judgements could be based in part on the noise distribution (I am indebted to Matthias Michel and Wixted 2020, but see Phillips 2021). Note that the dominant perception issue is not just about phenomenology and consciousness: it is about the dominant percept, and that is a wider 1 Perhaps there should be another ‘perceptual criterion’ above which is unconscious perception and below which is no perception at all. A criterion below which there is no perception at all is required to account for binocular rivalry in unconscious perception. Downloaded from https://academic.oup.com/analysis/article/83/3/575/7441531 by New York University Libraries user on 09 February 2024582 | book symposium notion than just the notion of a conscious percept. Adaptation effects can occur in unconscious perception of tilt, as noted in Chapter 2. As I also note in Chapter 2, binocular rivalry can occur in unconscious perception. And fruit flies show binocular rivalry though they may not be conscious creatures. In the book, I made a suggestion concerning why we have dominant percepts – that since the perceiving subject often has to act quickly, ‘it won’t do for perception to wallow in ambiguity’. As Stephen Jay Gould noted, evolutionary accounts are often ‘just so stories’ (Gottlieb 2012). They may sound plausible, but other stories that conflict with them can also sound plausible. While mentioning Gould, Phillips and Firestone counter with another evolutionary suggestion: But in any event, since in the case of non-reflex actions there must always be a decision to act or not, it is unclear why ambiguity precludes quick action. If one sets a criterion which says: ‘Run if there is even a 1% chance of a tiger being present’, one does not need a winner-takesall tiger percept to run quickly. Of course, complication in perception would presumably come at a cost. For whatever reason, the visual system apparently does not follow the advice of Phillips and Firestone and does not use signal detection parameters in determining perceptual content, even though they play a role in confidence judgements and in guessing. One might speculate (of course, just another just so story) that adding probabilities to perceptual contents would excessively complicate the visual system (there has been a debate about whether perception is indeed probabilistic. See Morrison 2016, Munton 2016, Denison 2017, Block 2018).