Mental Imagery Mental imagery can be defined as pictures in the mind or a visual representation in the absence of environmental input. Not everybody can conjure up mental images at will. Sir Francis Galton discovered this in 1883 when he asked 100 people, including prominent scientists, to form an image of their breakfast table from that morning. Some had detailed images, others reported none at all. What is the easiest way to make visual imagery stronger? Everybody has mental imagery during dreams, including people who go blind at an early age. Some individuals are capable of deep levels of hypnosis in which they can have visual hallucinations of dreamlike clarity, but this is quite unusual. For most of us, mental imagery during states of wakefulness is faint or difficult to manipulate. The best way to make it more vivid is to imitate the conditions of sleep. When one is relaxed or half asleep, mental imagery can be quite vivid. What do brain scans show about brain areas involved in mental imagery? An abundance of evidence from brain scanning research shows that the same areas of the brain used for normal perception are also activated by mental imagery. (Miyashita, 1995). In general, imagination activates some of the same brain areas as normal perception For example, "thinking about a telephone activates some of the same brain areas as seeing a telephone." (Posner, 1993) Early, important studies of mental imagery came from Roger Shepard of Stanford University and various colleagues. He used computer-generated block shapes similar to these:
One shape is different from the others.
Three of the shapes are the same as each other, only rotated. The fourth is different; it is a mirror image of the others. Can you find the one that is a mirror image? To determine this, most subjects must mentally rotate the figures, much as they would rotate a three-dimensional block model, to see if each matches the others. Why was the Cooper and Shepard research influential? Following up on the first experiments with mental rotation, Cooper and Shepard (1973) found that the time required for mental rotations depended upon the amount of rotation. This was a very important finding, because it implied that mental images could be manipulated as if real.
Mental Maps and Images Steven Kosslyn of Harvard is also famous for studies of mental imagery. Kosslyn found that the size of an imagined image influenced how quickly subjects could "move around" the image in memory. If subjects memorized a map, the time it took them to make a mental jump from one location to another depended upon the distance on the imagined map. What did Kosslyn find, concerning scanning of mental maps? When he had subjects imagine a rabbit next to a fly?
In one experiment, Kosslyn (1975) asked subjects to imagine animals standing next to one another, such as a rabbit next to an elephant or a rabbit next to a fly. Then subjects were asked questions such as, "Does the rabbit have two front paws?" People took longer to answer such questions when the rabbit was imagined next to an elephant, because the rabbit's image was so small. When the rabbit was imagined next to a fly, its imagined image was large, and subjects were quicker to answer questions about the image. Kosslyn concluded that visual imagination produces "little models, which we can manipulate much like we do actual objects."
When asked to imagined a rabbit next to a fly, people were quick to answer questions about the rabbit's appearance. They were slower when first asked to imagine it next to an elephant.
What are two types of mental imagery? What is evidence that they are distinct? Researchers have identified two types of mental imagery, one for pictures (for example, visualizing the rabbit next to the fly) and one for spatial representation (for example, rotating shapes in imagination). Kosslyn's work focused on imagined pictures. Shepard and colleagues focused on imagined rotation of shapes in space. The two involve different brain areas, and the two skills can be doubly dissociated by brain injury (you can lose one but not the other). The ability to imagine pictures can be lost after damage to the back of the brain, near the occipital lobe. The ability to imagine space is lost after damage to the middle of the right hemisphere, near the parietal lobe.
Memory Mostly for Meaning Although mental imagery can be vivid and detailed, most people have rather poor memory for the details of a picture. We remember mostly the meaning of a picture, not details. Baggett (1975) showed that detailed picture memory fades over several days. She showed subjects a short cartoon in one of two versions. One she called the explicit version. It showed a longhaired person getting hair cut. The other she called the implicit version. It showed no actual hair cutting.
However, it did show a person walking out of a barbershop with shorter hair. After showing one or the other version to her subjects, Baggett waited various lengths of time then asked subjects whether a test picture, showing hair being cut, appeared in the original sequence. How did Baggett investigate memory for pictures? What did she discover?
The "implicit" version did not show hair being cut in the study sequence.
For up to three days, subjects who received the implicit version knew they had not seen the test picture. However, after three days, they were likely to believe they had seen the hair being cut, even though they had not. Why? It was consistent with the story told by the pictures. When someone walks out of a hair salon with shorter hair, one infers that hair was cut. Baggett concluded that after three days the subjects lost their memories for the images themselves. They mostly remembered the meaning of those images and found the test item familiar because it was consistent with the meaning of the images. In summary, it appears that most people can generate mental images and manipulate them like "little models" as Kosslyn put it. However, mental images are usually lost after a few days, and we remember only the meaning of an image.
Summary: Visual Information Processing Visual scene analysis is a computer-based effort to simulate visual perception. It requires the computer to identify which parts of a scene "belong together" to form objects. The key process is called constraint satisfaction. The computer locates edges and corners where lines come together (called vertexes). The assignment of meaning to each line and vertex has the effect of limiting or constraining the interpretation put on other parts of the scene. The constraints "propagate." Each part of the scene, when interpreted, helps to limit possible interpretations of other parts of the scene. Constraint-propagation eventually produces one interpretation of the whole scene that is consistent with all the evidence. Classic work on visual scene analysis in the mid-60s culminated in a successful program by the 1980s. Success, in this case, was defined by a computer's ability to start with a visual image from a camera and identify boundaries of all objects in the scene, while specifying which lines, edges, and corners belonged to particular objects. Humans and computers both use a combination of bottom-up (data driven) and top-down (schema-driven) processing. Bottom-up processing is the flow of raw data into the system, which determines activity at higher levels. Top-down processing is the influence of organized knowledge upon the interpretation of data. A schema is a pattern or piece of knowledge based on past experience. The phenomena of set and expectancy, illustrated by the "rat man" example, show the effect of schemas on top-down processing. Cartoonists exploit top-down processing by using tiny cues to suggest emotions, movements, and mental states. Mental imagery takes place in imagination. Cooper and Shepard showed that the speed of rotating an imagined shape varied with the amount of rotation required, suggesting mental images had characteristics similar to physical images. Kosslyn did research showing picture-like qualities of mental images. Brain scans show that the same areas involved in normal perception are involved in mental imagery. However, picture memory and spatial memory are distinct, involving different areas of the brain. Memory of images is not exact. We remember the meaning of an image, not the pictorial detail. The "haircut" experiment by Baggett showed that people lost memory for details of an image within about three days.
Mental Imagery Phenomenology of Mental Images
When we form a mental image our experience seems much like seeing something in our mind. It seems a lot like vision. When we form a mental image we seem to be able to manipulate them and we seem to be solving problems some times by means of manipulating them. Mental images can be quite detailed but they tend to be less detailed than actual perception. Question: How accurate is our phenomenology at describing what is really going on in our heads?
How is imagery like vision? Can be used to recognize properties of objects. o o o
Subtle visual properties Properties that had not previously been labeled Properties which can't easily be deduced.
Can be used for pretend navigation
What do people use imagery for (Kosslyn, 1990)
Memory retrieval Problem solving Producing descriptions Mental practice Motivational states Daydreaming and association
Evidence for imagery based representations Scanning visual images Kosslyn, Ball & Reiser (1978): Scanning mental maps
Parallel vs. Sequential Processing(Nielsen & Smith,1970) o o o o
Schematic faces developed which varied on 5 features: Eyebrows, Eyes, Nose, Mouth, Ears Subjects asked to memorize picture or memorize verbal description. All subjects shown a target picture and asked if it matched while response time was taken. Number of relevant features was varied (3,4,5)
Mental Rotation (Shepard & Metzler, 1971)
Neurological evidence (Bisiach & Luzzatti, 1978) o o o
Brain damaged patients with unilateral visual neglect Asked to image a scene that was familiar before their stroke Patients only report things on one side or the other
Image size and ease of inspection (Kosslyn, 1975) o o o o
Ss asked to imagine two different sized animals side by side. A rabbit next to an elephant should be imagined smaller than a rabbit next to a fly. Ss asked about some detail of the animal (Does the rabbit have red eyes?) Ss respond faster when the imagined animal is large than when it is small
Limitations to mental imagery Memory for a common object (Nickerson & Adams, 1979) o o o
Subjects asked to draw a penny or recognize a penny from an array. Most subjects did poorly at this Shows that images are often lacking in details.
Mental rotation by the congenitally blind
o o
Even congenitally blind show mental rotation effects Shows that mental imagery may be spatially not specifically perceptual
Meanings and Connotations of ‘Mental Imagery’ Mental imagery is a familiar aspect of most people's everyday experience (Galton, 1880; Betts, 1909; Doob, 1972; Marks, 1972, 1999). A few people may insist that they rarely, or even never, consciously experience imagery (Galton, 1880, 1883; Faw, 1997, 2009; but see Brewer & Schommer-Aikins, 2006), but for the vast majority of us, it is a familiar and commonplace feature of our mental lives. The English language supplies quite a range of idiomatic ways of referring to visual mental imagery: „visualizing,‟ „seeing in the mind's eye,‟ „having a picture in one's head,‟ „picturing,‟ „having/seeing a mental image/picture,‟ and so on. There seem to be fewer ways to talk about imagery in other sensory modes, but there is little doubt that it occurs, and the experiencing of imagery in any sensory mode is often referred to as „imagining‟ (the appearance, feel, smell, sound, or flavor of something). Alternatively, the quasi-perceptual nature of an experience may be indicated merely by putting the relevant sensory verb („see,‟ „hear,‟ „taste,‟ etc.) in actual or implied “scare quotes.” Despite the familiarity of the experience, the precise meaning of the expression „mental imagery‟ is remarkably hard to pin down, and differing understandings of it have often added considerably to the confusion of the already complex and fractious debates, amongst philosophers, psychologists, and cognitive scientists, concerning imagery's nature, its psychological functions (if any), and even its very existence. In the philosophical and scientific literature (and a fortiori in everyday discourse), the expression „mental imagery‟ (or „mental images‟) may be used in any or all of at least three different senses, which are only occasionally explicitly distinguished, and all too often conflated: {1} {2} {3}
quasi-perceptual conscious experience per se; hypothetical picture-like representations in the mind and/or brain that give rise to {1}; hypothetical inner representations of any sort (picture-like or otherwise) that directly give rise to {1}.
Far too many discussions of visual mental imagery fail to draw a clear distinction between the contention that people have quasi-visual experiences and the contention that such experiences are to be explained by the presence of representations, in the mind or brain, that are in some sense picture-like. This picture theory (or pictorial theory) of imagery experience is deeply entrenched in our language and our folk psychology. The very word „image,‟ after all, suggests a picture. However, although the majority of both laymen and experts probably continue to accept some form of picture theory, many 20th century philosophers and psychologists, from a variety of theoretical traditions, have argued strongly against it, and, in several cases they have developed quite detailed alternative, non-pictorial accounts of the nature and causes of imagery experiences (e.g., Dunlap, 1914; Washburn, 1916; Sartre, 1940; Ryle, 1949; Shorter, 1952; Skinner, 1953, 1974; Dennett, 1969; Sarbin & Juhasz, 1970; Sarbin, 1972; Pylyshyn, 1973, 1978, 1981, 2002a, 2003a, 2005; Neisser, 1976; Hinton, 1979;
Slezak, 1991, 1995; Thomas, 1999b, 2009). Others, it should be said, have developed and defended picture theory in sophisticated ways in the attempt to meet these critiques (e.g., Hannay, 1971; Kosslyn, 1980, 1983,1994; von Eckardt, 1988, 1993; Tye, 1988, 1991; Cohen, 1996). However, despite these developments, much philosophical and scientific discussion about imagery and the cognitive functions it may or may not serve contines to be based on the often unspoken (and even unexamined) assumption that, if there is mental imagery at all, it must consist in inner pictures. Consider, for example, the title of the book The Case for Mental Imagery, by Kosslyn, Thompson & Ganis (2006). In fact the book is an extended and quite polemical defense of the much disputed view that visual mental imagery consists in representational brain states that are, in some significant and important ways, genuinely picture-like (see supplement: The Quasi-Pictorial Theory of Imagery, and its Problems). That is to say, the contents suggest that the title should be understood as intending "imagery" in sense {2}. However, it would also be very natural (and, very possibly, in accord with the authors' intentions – compare Kosslyn, Ganis & Thompson, 2003) to understand the title as implying that the book's deeper purpose is to refute the view that imagery, even in sense {1}, does not really exist (or, at least, that the concept of imagery will find no place in a properly scientific ontology). Although this denialist view of imagery has few, if any, supporters today, it is well known that not so very long ago, in the era of Behaviorist psychology, it had great influence. The book's title thus (intentionally or otherwise) invites us to conflate the (now) very controversial view that mental images are picture-like entities, with what is, today, the virtual truism that people really do have quasi-perceptual experiences, and that our science of the mind owes us some account of them. Another way in which the expression „mental imagery‟ (together with many of its colloquial near-equivalents) may be misleading, is that it tends to suggest only quasi-visual phenomena. Despite the fact that most scholarly discussions of imagery, in the past and today, do indeed focus mainly or exclusively upon the visual mode, in fact, quasi-perceptual experience in other sensory modes is just as real, and, very likely, just as common and just as psychologically important (Newton, 1982). Contemporary cognitive scientists generally recognize this, and interesting studies of auditory imagery, kinaesthetic (or motor) imagery, olfactory imagery, haptic (touch) imagery, and so forth, can be found in the recent scientific literature (e.g., Segal & Fusella, 1971; Reisberg, 1992; Klatzky, Lederman, & Matula, 1991; Jeannerod, 1994; Bensafi et al., 2003). Although such studies are still vastly outnumbered by studies of visual imagery, „imagery‟ has become the generally accepted term amongst cognitive scientists for quasi-perceptual experience in any sense mode (or any combination of sense modes).
1.1 Experience or Representation? In the introduction to this entry, in order to avoid making a premature commitment to the picture theory, and in accordance with definitions given by psychologists such as McKellar (1957), Richardson (1969), and Finke (1989), mental imagery was characterized as a form of experience (i.e., as {1}). However, this itself is far from unproblematic. Evidence for the occurrence of any experience is necessarily subjective and introspective, and, because of this,
those who have doubts about the validity of introspection as a scientific method, may well be led to question whether there is any place for a concept such as imagery within a truly scientific world view. J.B. Watson, the influential instigator of the Behaviorist movement that dominated scientific psychology (especially in the United States) for much of the 20th century, questioned the very existence of imagery for just these sorts of reasons (Watson, 1913a, 1913b, 1928 – see supplement; see also: Thomas, 1989, Berman & Lyons, 2007). Although few later Behaviorist psychologists (or their philosophical allies) expressed themselves on the matter in quite the strong and explicit terms sometimes used by Watson, the era of Behaviorist psychology is characterized by a marked skepticism about imagery (if not its existence, at least its psychological importance) amongst both psychologists and philosophers. Imagery did not become widely discussed again among scientific psychologists (or philosophers of psychology) until around the end of the 1960s, when Behaviorism began to be displaced by Cognitivism as the dominant psychological paradigm. Most informed contemporary discussions of imagery, amongst both philosophers and psychologists, are still very much shaped by this recent history of skepticism about imagery (or iconophobia, as it is sometimes called), and the subsequent reaction against it. By contrast with their Behaviorist predecessors, most cognitive psychologists today hold that imagery has an essential role to play in our mental economy. Many may share some of the reservations of their Behaviorist predecessors about the place of introspection and subjectivity in science, but they take the view that imagery must be real (and scientifically interesting) because it is explanatorily necessary: The results of many experiments on cognitive functioning, they hold, cannot be satisfactorily explained without making appeal to the storage and processing of imaginal mental representations. The belief that such mental representations are real is justified in the same sort of way that belief in the reality of electrons, or natural selection, or gravitational fields (or other scientifically sanctioned “unobservables”) is justified: Imagery is known to exist inasmuch as the explanations that rely upon imaginal representations are known to be true. From this perspective, some theorists recommend that the term „imagery‟ should not be understood to refer to a form of subjective experience, but, rather, to a certain type of “underlying representation” (Dennett, 1978; Block, 1981a, Introduction; Block, 1983a; Kosslyn, 1983; Wraga & Kosslyn, 2003; Kosslyn, Thompson & Ganis, 2006). Such representations are “mental” in the sense now commonplace in cognitive science: i.e., they are conceived of as being embodied as brain states, but as individuated by their functional (and computational) role in cognition. As Block (1981a, 1983a) points out, an advantage of defining mental imagery in this way (i.e., as an unspecified form of representation, as {3} rather than {2}) is that it does not beg the controversial question of whether the relevant representations are, in any interesting sense, picture-like. However, if it is not because they are picture-like, what is it that makes these mental representations mental images? Presumably the idea is that a mental representation deserves to be called an image if it is of such a type that its presence to mind (i.e., its playing a role in some currently occurring cognitive process) can give rise to a quasi-perceptual experience of whatever is represented. But this move relies upon our already having a grasp of the experiential conception of imagery, which must, therefore, be more fundamental than the representational conception just outlined. Furthermore, to define imagery in the way that
Block, Kosslyn etc. suggest, as first and foremost a form of representation (as explanans rather than explanandum), is to beg more basic and equally controversial questions about the nature of the mind and the causes of quasi-perceptual experiences. A number of scientists and philosophers, coming from a diverse range of disciplinary and theoretical perspectives, do not accept that imagery experiences are caused by the presence to mind of representational tokens (e.g., Sartre, 1940; Ryle, 1949; Skinner, 1953, 1974; Sarbin, 1972; Thomas, 1999b, 2009; O'Regan & Noë, 2001; Bartolomeo, 2002; Bennett & Hacker, 2003; Blain, 2006). It should be admitted, however, that focusing too narrowly on the experiential conception of imagery has its own potential dangers. In particular, it may obscure the very real possibility, foregrounded by the representational conception, that importantly similar underlying representations or mechanisms may sometimes be operative both when we consciously experience imagery and sometimes when we do not. Some evidence, such as Paivio's (1971, 1983a, 1991) work on the differential memorability of words with different “imagery values” (see section 4.2, below), suggests that this is indeed the case. In practice, both the experiential and the representational conceptions of imagery are frequently encountered in the literature of the subject. Unfortunately, it is often hard to tell which is intended in any particular case. Even where they are not actually conflated, confusion can arise when one conception is favored over the other without this ever being made sufficiently clear or explicit. Although it would be pedantic and potentially confusing to insist on explicitly drawing the distinction everywhere, where it seems important or helpful to do so this entry will refer to imagery experiences (or quasi-perceptual experiences) on the one hand, and imagery representations (or imagery processes) on the other.
1.2 The Relation to Perception There are further potential problems, however, with the brief characterization of imagery given in our introduction. Not only does what is said there duck the difficult (and rarely considered) task of specifying what dimensions and degrees of similarity to perception are necessary for an experience to count as imagery; it also elides the controversial question of whether, despite the surface resemblance, imagery is a sui generis phenomenon, conceptually quite distinct from true perceptual experience, or whether imagery and perception differ only in degree rather than in kind. Some, such as Hume (1740), hold that percepts (impressions in his terminology) and images (ideas) do not differ in kind, but only in their causal history and their degree of “vivacity”or vividness. This view has frequently been criticized, however, most recently by McGinn (2004; cf. Reid, 1764 II.5, VI.24; Savage, 1975; Warnock, 1976). An alternative view, explicitly defended by some (e.g., Jastrow, 1899; Savage, 1975; Thomas, 1997a), and implicit in much of the other relevant literature, is that imagery lies at one end of a spectrum stretching from veridical, highly stimulus-driven and stimulus-constrained perception at one end, to “pure” imagery (where the content of the experience is generated entirely by the subject, and is quite independent of any current stimulus input) at the other. Several varieties of imaginative preceptual experience may be taken to fill in the continuum between these
extremes: mistaken or illusive perceptions (imagining, for instance, that the bush seen indistinctly in the darkness is a bear), and various types of non-deceptive seeing as or seeing in (such as imagining a cloud to have the shape of a camel, weasel, or whale; seeing a Laughing Cavalier in paint on canvas; seeing someone's sadness in their eyes; or seeing the notorious duck-rabbit figure as a duck [or rabbit]).
Figure 1.2_1 The Duck-Rabbit Others, however, notably Reid (1764 II.5), Sartre (1936), Wittgenstein (1967 §621 ff.), and McGinn (2004), argue that there is a sharp conceptual and phenomenological distinction to be drawn between imagery and perception proper. After all, it is argued, our imagination, unlike our perception, is under the control of our will (and experienced as such). Provided I know what an elephant looks like, I can choose to imagine one wherever and whenever I want to, but I cannot choose to see an elephant unless one actually happens to be present. By contrast, if an elephant is present before my open eyes, I cannot help but see it, whether I will or no.[1] Furthermore, it is claimed that (in sharp contrast to perception) we can derive no new information about the world from our imagery: No image can contain anything except what the imager put there, which must already have been in his or her mind (Sartre, 1940; Wittgenstein, 1967 §§627, 632). However, this negative view of the epistemological value of imagery is rejected by Kosslyn (1980, 1983), and the arguments of Sartre and Wittgenstein on this point have been rebutted in some detail by Taylor (1981).[2] McGinn (2004 p. 19ff) argues that although Sartre and Wittgenstein overstate their point, there is a genuine and important insight underlying what they say: The information we can derive from our imagery is of a different sort, and is derived in a different way, from that which we get from perception.
1.3 The Intentionality of Imagery On a more consensual note, with only rare exceptions (e.g. Wright, 1983; Martin, 2008 p. 160) nearly all serious discussions of imagery take it for granted that it bears intentionality in the sense of being of, about, or directed at something (Harman, 1998): A mental image is
always an image of something or other (whether real or unreal), in the same sense that perception (whether veridical or not) is always perception of something (see Anscombe, 1965). It is in virtue of this intentionality that mental imagery may be (and usually is) regarded as a species of mental representation that can, and often does, play an important role in our thought processes. It is also generally accepted that imagery is, for the most part, subject to voluntary control. Although it is true that images often come into the mind unbidden, and sometimes it is hard to shake off unwanted imagery (for instance, a memory of some horrible sight that one cannot get out of one's mind), most of us, most of the time can quite freely and voluntarily conjure-up and manipulate imagery of whatever we may please (provided, of course, that we know what it looks like). There are quasi-perceptual experiences, such as afterimages, that are not subject to this sort of direct voluntary control, and indeed, that do not seem to bear intentionality, but these are usually (at least implicitly) understood to be phenomena of a distinctly different type from mental imagery proper (see supplement). Further discussion of phenomena akin to, or sometimes confused with, mental imagery:
4.2 Mnemonic Effects of Imagery Despite the developments outlined above, interest in imagery amongst experimental psychologists remained at a fairly low level until the mid to late 1960s. It was the recognition, in that period, of the powerful mnemonic effects of imagery that changed the situation, leading to a thriving tradition of experimental research, and securing imagery a firm place in cognitive theory. These mnemonic effects, it turned out, could be clearly demonstrated in readily repeatable experiments that did not rely in any way upon introspective reports. According to Bugelski (1977, 1984), an important stimulus to the flowering of experimental research on imagery and memory[23] was the 1966 publication of Frances Yates' celebrated and widely read historical study, The Art of Memory. Yates details how imagery based mnemonic techniques, particularly versions of the so-called method of loci, were in widespread use amongst European intellectuals, educators, and orators from classical Greek through to early modern times, and she argues that the knowledge and use of these techniques may have had quite significant effects on the development of Western philosophical, theological, and early scientific thought. (see Supplement: Ancient Imagery Mnemonics). Around the same time, Soviet psychologist Alexander Luria's (1960, 1968) extensive case study of the “mnemonist” Shereshevskii first became available in English, and well known amongst English-speaking psychologists. Shereshevskii's truly prodigious feats of memory were apparently made possible by an abnormally vivid visual imagination, often harnessed to his own version of the method of loci. Experiment soon confirmed that the imagery method of
loci, as described by Yates and Luria, was extremely effective in enhancing memory performance in ordinary people (Ross & Lawrence, 1968). In fact, Canadian psychologist Allan Paivio was already quietly working on the mnemonic effects of imagery in the early 1960s, before either Yates or Luria had published their books (Paivio, 1963, 1965). (His interest in the subject was apparently sparked by a demonstration of imagery mnemonics he witnessed at a public speaking course (Marks, 1997).) However, the considerable attention that Paivio's theoretical speculations and painstaking quantitative experiments were attracting by the end of the decade surely owed much to the interest aroused by the more eye-catching historical and anecdotal studies of Yates and Luria. By the later 1960s, and in the 1970s, many other psychologists would take up research in this area, but Paivio was well established as the field's leading figure, and discussion centered largely upon the implications and merits, or otherwise, of the Dual Coding (imagery code and verbal code) theory of mental representation that Paivio proposed to explain his results (Paivio, 1971, 1986, 1991, 1995, 2007). Although they inevitably became entangled, this contoversy between the Dual Coding and the rival Common Coding theories of memory (see Supplement: Dual Coding and Common Coding Theories of Memory) should not be confused with the better known debate between analog (pictorial) and propositional (descriptive) theories of imagery (see section 4.4). The former debate is about the function of imagery in cognition, the latter is about the nature and mechanism of imagery itself. The findings of this extensive experimental research program on the mnemonic effects of imagery, can be crudely summarized as the discovery of two principal effects. First of all, it was demonstrated quite incontrovertibly that subjects who follow explicit instructions to use simple imagery based mnemonic techniques to memorize verbal material (typically lists of apparently random words, or word pairs) remember it very much better than subjects who do not use such techniques (Bower, 1970, 1972; Bugelski, 1970; Paivio, 1971; Neisser & Kerr, 1973). Secondly, and somewhat more controversially, Paivio and others claim to have shown that imagery plays a large role in verbal memory even when the experimental subjects are not given explicit instructions to form imagery, and make no deliberate effort to do so. To demonstrate this, Paivio and his associates initially determined quantitative imagery values for each of a long list of nouns: that is to say, the relative ease with which subjects could generate a mental image appropriate to the word, or the likelihood that an image would spontaneously be evoked by the word in question (Paivio, Yuille, & Madigan, 1968).[24] (On the whole, concrete nouns such as „cat‟ have high imagery values, and abstract nouns such as „truth‟ have low ones, although there are exceptions to this rule.) Once these quantitative imagery values were established, Paivio was able to show, in various experimental designs, that words with high imagery values were consistently remembered significantly better than those with lower ones, quite regardless of any conscious intent on the subjects' part to form relevant images (Paivio, 1971, 1983, 1991).[25] Further discussion of theories of the mnemonic properties of imagery: Supplement: Dual Coding and Common Coding Theories of Memory Supplement: Conceptual Issues in Dual Coding Theory
4.3 The Spatial Properties of Imagery By the end of the 1960s, the work of Paivio and others on the mnemonic properties of imagery had established a strong empirical case for the functional importance of imagery in cognition. The phenomenon could no longer be ignored by psychologists, or treated as a mere subjective epiphenomenon of no scientific interest, as it had been in the Behaviorist era. However, this work had done very little to illuminate the nature of imagery itself, or of the cognitive mechanisms that generate it. This began to change in the early 1970s when Roger Shepard and his students began to publish experimental demonstrations of the “mental rotation” of images (e.g., Shepard & Metzler, 1971; Shepard & Cooper et al., 1982). Soon after, Stephen Kosslyn and his collaborators produced experimental evidence for the “mental scanning” of visual images, showing that it took longer for subjects to consciously scan between image features that were relatively further apart than between those that appeared close together (Kosslyn, 1973, 1980; Kosslyn, Ball, & Reiser, 1978; Pinker & Kosslyn, 1978; Pinker, 1980; Finke & Pinker, 1983; Pinker, Choate, & Finke, 1984; Borst & Kosslyn, 2008). (There were also related claims that “the visual angle of the mind's eye” could be experimentally measured (Kosslyn, 1978a; Finke & Kosslyn, 1980; Finke & Kurtzman, 1981a,b).) Kosslyn also demonstrated that the subjective sizes of visual mental images (and the relative sizes of their sub-parts) measurably affect the times it takes to inspect and report on particular details of imagined objects. The presence of larger features of an object could be reported more quickly (from an image of the object) than smaller features. As this size-time relation, did not appear unless the subjects used imagery to do the task (i.e., to confirm that some named object type has some particular feature or sub-part), these experiments provided further evidence for the notion that imagery is a sui generis form of mental representation, with distinct properties from linguistic or purely conceptual representations (Kosslyn, 1975, 1976a, 1976b, 1980). It should be noted that the methodology of many of these experiments leaves them vulnerable to the charge, pressed by several critics, that the results reflect not so much the normal workings of cognition, and the properties of the representational structures (such as mental images) that enable them, but, rather, what psychologists call the demand characteristics of the experimental situation (Orne, 1962; Rosnow, 2002) (see supplement). This is a wellknown pitfall of psychological experimentation with human subjects, and experimenters are, for the most part, well aware of it, and take what precautions they can to rule out the possibility that demand characteristics might significantly influence their findings. Nevertheless, certain types of imagery experiment, including most of those discussed in this section, appear to be particularly susceptible to such influence (Intons-Peterson, 1983), and it may sometimes be effectively impossible to rule out the possibility that demand characteristics have played a large, or even predominant, role in determining the results. Thus, some of the results obtained in this area of research remain open to question. Despite this, however, converging evidence from several different types of experiment has been enough to convince a majority of cognitive scientists that processes like mental scanning,
mental rotation, and size effects in image inspection are real and significant components of cognition. Shepard, Kosslyn, and others argued that these results show that visual mental imagery has inherently spatial properties, and represents in an “analog” fashion that is quite different to the way that language and other symbolic systems represent (Shepard & Chipman, 1970; Shepard, 1975, 1978b, 1981; 1984; Kosslyn, 1975, 1980, 1981, 1983; Kosslyn, Pinker, Smith, & Shwartz, 1979). However, others, particularly among those who were strongly committed to a (digital) computational view of the mind, firmly rejected this conception of imagery (Simon, 1972; Anderson & Bower, 1973; Baylor, 1973; Moran, 1973; Pylyshyn, 1973, 1978, 1981, 1984; Hinton, 1979; Slezak, 1995). A lively and high-profile theoretical debate ensued, about the nature of mental imagery and of mental representation in general. Further discussion of experiments on the spatial properties of imagery: Supplement: Mental Rotation Supplement: The Problem of Demand Characteristics in Imagery Experiments
4.4 The Analog-Propositional Debate The analog-propositional debate, occasionally also called the picture-description debate, or sometimes just the imagery debate (as if there were no other debatable or hotly debated issues about imagery) is an ongoing and notoriously irreconcilable dispute within cognitive science about the representational format of visual mental imagery. The huge impact it had on the early development of the field appears to have resulted in a widespread belief, amongst both philosophers and cognitive scientists, that analog and propositional theories (those terms being understood in the rather special senses that they have acquired in this context), together, perhaps, with hybrid theories incorporating elements of both (e.g., Chambers, 1993), completely exhaust the space of possible or empirically plausible scientific accounts of imagery (Thomas, 2002). That is not the case, as we shall see in section 4.5. To a first approximation, the analog side of the debate holds that the mental representations that we experience as imagery are, in some important sense, like pictures, with intrinsically spatial representational properties of the sort that pictures have (i.e., pictures do not just represent spatial relationships between the objects they depict, but represent those relationships, at least in part, via actual spatial relationships on the picture surface). The propositional side, by contrast, holds the relevant mental representations to be more like linguistic descriptions (of visual scenes), without inherently spatial properties of their own. Although it began as a dispute amongst scientists, the debate clearly touches on fundamental issues about the nature of mind and thought, and perhaps the nature of science too, so it soon attracted a good deal of interest from philosophers as well. It is good to be aware that the terms analog and propositional, although they have become entrenched usage in this context, are both potentially misleading. On the one hand the propositions which are supposed by some to constitute the descriptions that constitute
imagery, are not really propositions at all in the established philosophical sense of the word: rather they are a sort of sentence (albeit not sentences natural language, but of mentalese).[26] On the other hand, the “quasi-pictorial” theory of Kosslyn (1980, 1983, 1994, 2005), which has become very much the dominant theory on the other side of the debate, in fact models mental images as digitized pictures generated within a simulation program running on a digital computer (Kosslyn & Shwartz, 1977; Kosslyn, 1980).[27] Although the debate began, and was at its fiercest, during the formative years of the discipline of cognitive science in the 1970s, it has yet to reach a generally accepted resolution. Despite Kosslyn's (1994) unilateral declaration of victory for the analog side, controversy has flared up anew, and continues in the 21st century (Slezak, 1995, 2002; Thomas, 1999b, 2002, 2003, 2009; Pylyshyn, 2002a,b, 2003a,b,c, 2004; Kosslyn, Ganis & Thompson, 2003, 2004; Kosslyn, 2005; Grueter, 2006; Kosslyn, Thompson, & Ganis, 2006; Dulin et al., 2008). The Behavioral and Brain Sciences target article by Kosslyn, Pinker, Smith, & Shwartz (1979), together with the appended commentaries, provides a good sense of the debate at its height, and many of the most important philosophically oriented contributions to its early stages can be found in two collections edited by Block (1981a,b). However, it may be difficult to understand the scientific and philosophical issues at stake unless one has a sense of the historical and intellectual context in which the dispute originated. Matters became so hotly contested during the 1970s that some participants, most notably Anderson (1978) and Palmer (1975b, 1978), came to the conclusion that the disagreement was quite impossible to resolve by the methods of scientific psychology, or perhaps at all. Indeed, Anderson (1978) offered a formal proof purporting to show that the two main contending theories are empirically equivalent. Anderson's arguments in particular aroused much interest at the time, and were themselves vigorously disputed (Hayes-Roth, 1979; Pylyshyn, 1979b; Cohen, 1996) and defended (Anderson, 1979). However, the main debate continued, and it is probably fair to say that most observers have come to the conclusion that the empirical equivalence claimed by Anderson is ultimately not particularly interesting or important. It can probably be regarded as just a special case of the well known Duhem-Quine underdetermination of theory by evidence: many philosophers of science hold that any theory can be made to fit any evidence provided one is allowed freely to supplement the theory with arbitrary (and perhaps ad hoc, complex, and implausible) auxiliary hypotheses, which is essentially what Anderson was doing. Nevertheless, the fact that such claims could be seriously proposed and discussed is testament to just how acrimonious and intractable this debate about imagery had come to seem at the time, and how important it was to those involved. The very possibility of a science of cognition seemed to be under threat. Despite this, the debate's focus has, in practice, been quite narrow. Although it is often understood to be a debate about the nature of imagery per se, it may more truly be seen as about what theory of imagery will best account for the facts within the parameters of a computational functionalist theory of the mind, i.e., a theory that holds that mental states in general are to be identified with states of the brain as individuated in terms of their computational/functional role in cognition. Although this computational functionalism still has many adherents, it no longer dominates the philosophy of mind and cognition to the
extent that it once did, but in the early 1970s it was new and exciting, and the computational approach to cognitive theory that it sanctioned was being taken up with great enthusiasm by many psychologists (Baars, 1986; Gardner, 1987). Both the cognitive psychologists and the functionalist philosophers drew much of their inspiration from Artificial Intelligence research in the "physical symbol systems" tradition of what Haugeland has called GOFAI (Good Old Fashioned AI) (Newell, 1981; Haugeland, 1985). Initially the concurrent rise of imagery research and computational psychology played mutually reinforcing roles within the cognitivist revolution against Behaviorism, because they both implied that the concept of mental representation should play a central role in the science of the mind. However, a tension soon became apparent between the symbolic and syntactic concept of mental representation that came from Artificial Intelligence and the, on the face of it, very different concept implicit in the work of the imagery researchers. The analog-propositional debate, and much of the passion and partisanship it aroused, grew out of that tension, and more particularly, the desire to bring imagery within the fold of computational functionalism. 4.4.1 Pylyshyn's Critique, and Description Theory The analog-propositional debate may be said to have begun when this tension found its first clear expression, in a very influential article by Zenon Pylyshyn (1973).[28] Since then, Pylyshyn has continued to extend and defend his critique of pictorial (or analog) theories of imagery in many subsequent publications (e.g., Pylyshyn, 1978, 1981,1984, 2002a,b,2003a,b, 2005). Although a number of other cognitive scientists and philosophers have taken positions, and made empirical and theoretical arguments, similar to and supportive of those of Pylyshyn (e.g., Anderson & Bower, 1973; Reed, 1974; Palmer, 1975a, 1977; Kieras, 1978; Hinton, 1979; Lang, 1979; Slezak, 1991, 1992, 1993, 1995, 2002), and others continue to reject picture theories for different reasons (e.g., Neisser, 1979; Heil, 1982; White, 1990; Thomas, 1999b, 2009; O'Regan & Noë, 2001; Bartolomeo, 2002; Bennett & Hacker, 2003), Pylyshyn remains indisputably the most important, influential, and trenchant critic of pictorial theories of imagery. Clearly Pylyshyn objects, as many philosophers have before him, to the notion of inner mental pictures that are somehow called to mind and reperceived by the “mind's eye”. In his 1973 article he raised a number of objections to this notion, some of which have withstood criticism better than others, but the underlying worry was clearly that the inner-picture theory of imagery inevitably commits the homunculus fallacy: it implicitly relies on the assumption that there is a little man (or rather, something that is the functional equivalent of a fullfledged visual system, including eyes), or, at the very least, something with inexplicable mental powers, inside the head to reperceive, experience, and interpret the image. The broad functional architecture of Kosslyn's theory, in fact, closely parallels that of Descartes' account of imagery (see section 2.3.1, and Supplement: The Quasi-Pictorial Theory of Imagery), and, of course, Descartes notoriously relied upon a conscious homunculus, the immaterial soul, that is placed forever beyond the reach of natural science. Modern defenders of the pictorial/analog theory protest that they cannot have committed the homunculus fallacy (let alone committed themselves to Cartesian dualism) because a computer model of the
theory has been implemented, and they have outlined an account of how picture-like representations, formed at an early stage of visual processing in the brain, are subject to several more stages of neural processing before they give rise to visual knowledge and experience (Kosslyn, 1980, 1994; Kosslyn, Thompson, & Ganis, 2002, 2006). However, their critics remain unconvinced that they have truly successfully avoided these pitfalls (Slezak, 1993, 1995; Thomas, 1999b, 2003, 2009; Pylyshyn, 2002a,b, 2005). For instance, in the computer simulation of pictorial imagery due to Kosslyn & Shwartz (1977, 1978; see also Kosslyn, Flynn, et al., 1990) the role of the homunculus is played by the humans who program and/or operate the computer. The program constructs and displays pictures (for an example, see figure 2 in the Supplement: The Quasi-Pictorial Theory of Imagery, and its Problems), and transforms them in various ways in order to model image transformations such as mental rotation and mental scanning. However, nothing in the program implements, or even simulates or models, conscious awareness of these pictures (despite a handwave towards a “mind's eye function”), and nothing in the program is even intended to correspond to a grasp of what it is that the pictures represent (or even that they are supposed to be representations). Only the human onlooker consciously experiences the pictures, knows that they are pictures, and can tell what they might be pictures of (Thomas, 2009). If quasi-pictorial theory is capable of accounting for the representational and conscious nature of imagery without appealing to a homunculus, then the Kosslyn-Shwartz program might constitute a useful model of how images are built up and transformed. However, despite vehement claims to the contrary (e.g., Kosslyn, Thompson, & Ganis, 2006 p. 41), nothing about the program in any way shows that the theory can avoid making such an appeal. In a subsequent paper, Pylyshyn (1978) introduced an important new argument against pictorialism, based on the concepts (which he introduced) of cognitive penetrability and impenetrability. The distinction between cognitively impenetrable and cognitively penetrable processes closely parallels Fodor's later distinction between modularized and unmodularized, “central” cognitive systems (Fodor, 1983). Cognitive processes are said to be cognitively penetrable if their workings can be affected by the beliefs and goals of the person, and cognitively impenetrable if they cannot be. Pylyshyn argues (and Fodor (1983) concurs) that there are good reasons to believe that “early” visual processing, i.e., the processes by which visual inputs give rise to beliefs about our surroundings, is cognitively impenetrable. For example, he points out that well known visual illusions, such as the Ponzo and Müller-Lyer illusions, continue to deceive us even when we know perfectly well that they are illusions (figure 4.4.1_1). The two horizontal lines in both the Ponzo and the Müller-Lyer figures continue to look as though they are of different lengths even when we have measured them and are quite convinced that they are, in fact, the same.
Figure 4.4.1_1 The Ponzo Illusion (left) and the Müller-Lyer Illusion (right). In both cases, the two horizontal lines are the same length, but appear to be different lengths. As Pylyshyn sees it, the view that he rejects, the view that visual mental imagery involves a representational format that is peculiarly visual and is distinct from the format in which beliefs and propositional attitudes in general are represented, amounts to the claim that images are generated within this cognitively impenetrable, “early visual processing” module. If that were the case, imagery should be a cognitively impenetrable phenomenon, but it is not.[29] The way we experience our mental imagery is clearly affected by our beliefs and goals. Not only do we have a large degree of voluntary control over the content of our imagery experiences, but it has also been experimentally demonstrated that extra-visual beliefs can influence the course of supposed imagery processes. For example, the times taken to “mentally scan” between different landmarks on a mental image of a map (see section 4.3 above) are affected not only by the actual relative distances between the points on the map, but also by information that is verbally given about those distances. Subjects tend to take longer to “mentally scan” over a distance marked as representing 80 miles than over a distance marked as 20 miles, even though the actual distances depicted on the map (which they have learned and are supposedly imagining) are the same (Richman, Mitchell, & Reznick, 1979a,b; see also Mitchell & Richman, 1980; Goldston et al., 1985; Reed, Hock & Lockhead, 1983). Pylyshyn's views about the cognitive impenetrability of “early vision”, the relevance of this to our understanding of imagery, and the concept of cognitive penetrability/impenetrability itself, all remain controversial (see, for example, the commentaries published with Pylyshyn (1999)), but he has continued to develop and refine the concept (which also has applications outside the imagery debate), as well as the associated argument, over many years (e.g., Pylyshyn, 1984, 1999, 2002a, 2002b, 2003b). Pylyshyn also argues (1981, 2002a) that most if not all of the experimental evidence that is supposed to show that imagery has inherently spatial properties (such as Kosslyn's work on mental scanning (Kosslyn, Ball, & Reiser, 1978; Kosslyn, 1980)) can be explained away as the result of the interaction of the experimental subject's tacit knowledge of the properties of visual experience and the experimental instructions. For instance, he holds that if subjects are
asked to scan their mental gaze from one point to another on a mental image of a map, what they interpret these instructions to be asking them to do is to behave as if they were actually looking at the relevant map and scanning their gaze between the points. Because they know from their history of ordinary visual experience that it takes longer to scan between points that are further apart, this will be reflected in their performance. Thus, the fact that people instructed to scan across their images take longer to scan longer distances is not evidence of the existence of some inner, mental, image-space. Rather, it is a reflection of people's implicit understanding (which they may not necessarily always be able to articulate) of the visual properties of the actual space around them (cf, Morgan, 1979). Pylyshyn's critics have often been inclined to conflate his tacit knowledge theory with the view that the results of imagery experiments may be fatally contaminated by the effects of experimental demand characteristics (see supplement). However, although he clearly does think experimental demand plays a large role in determining many of the results on imagery, he resists this interpretation of his position. He is not saying (as is sometimes implied) that experimental subjects are, as it were, consciously faking their performances in order to please the experimenter. Rather they are doing their best to comply with experimental instructions that turn upon the slippery concept of mental imagery. The fact that the subjects and the experimenter may share similar assumptions about the causes of quasi-visual experience (the “folk theory” of images as inner pictures), so that both interpret the experiment in terms of operations on inner pictures in an inner space, is not evidence that those assumptions are correct. It is generally assumed that problems caused by demand characteristics can be avoided or minimized by careful and ingenious experimental design, or by such tactics as post-experimental questioning of subjects to see if they have guessed the experimental hypothesis (Kosslyn, for one, routinely throws out data from subjects who guess this correctly). However, Pylyshyn is not saying that what might otherwise be meaningful experimental results are “contaminated” by the effects of demand characteristics, so such “decontamination” tactics are not very relevant. Rather, the problem lies with the basic conceptualization of the phenomena and the experimental tasks (by experimenter and subject alike). Of course, Pylyshyn was far from the first person to raise objections to the idea of inner pictures, or to criticize the standard interpretations of imagery experiments. What made his critique particularly effective was that he began, both in his 1973 article and in subsequent writings (e.g., Pylyshyn, 1978, 1981, 1984, 2003b), to sketch out an alternative, non-pictorial account of the nature of mental imagery. Instead of being like pictorial representations of a visual scene, he suggested, images might better be thought of as being a sort of description (sometimes referred to as a “structural description”) of that scene. Perhaps for the first time in the history of thought, there now seemed to be a viable alternative to the pictorial conception of imagery that had dominated folk, philosophical, and psychological thinking about imagery since ancient times.[30] Philosophers such as Shorter (1952) and Dennett (1969) (who both base their position on the oft repeated, but almost certainly unsound, argument from image indeterminacy, or “tiger's stripes” argument) had anticipated Pylyshyn in suggesting that mental images might be more like descriptions than pictures.[31] However, Pylyshyn was able to make the idea much more concrete and plausible by linking it with concepts of the nature of mental representation that were emerging from Artificial
Intelligence research. Particularly important for Pylyshyn was the work of Simon (1972) and Newell (1972), and their students Baylor (1973) and Moran (1973), who had already made some progress in devising symbol systems suitable for the computational representation of the spatial structure of simple layouts or objects (such as rectangular blocks). They explicitly presented these representations as models of the image representations that people use in doing certain visuo-spatial cognitive tasks.[32] By presenting these ideas as the germ of an alternative to the venerable but highly problematic conception of images as inner pictures, Pylyshyn was able to make a powerful case. In his original (1973) article, Pylyshyn alludes to a number of disparate schemes developed by various computer scientists, for the computational representation of visual information, and it is perhaps not altogether clear what these have in common as alternatives to a pictorial conception of imagery. This was very much clarified, however, when Fodor (1975) introduced the hypothesis of mentalese, the “language of thought” (see Language of Thought Hypothesis), a syntactically structured representational system innate to the human brain, which Fodor argued, was, in some form, implicitly required by all coherent computational functionalist theories of cognition. The vocabulary and syntax of mentalese (if it exists) remain unknown, but they are likely to be very different from those of English or any other natural (actually spoken) language. Nevertheless, Fodor holds that any cognitive theory capable of accounting for the full range of human mental capacities is bound to make appeal to some such inner language. Pylyshyn has embraced this hypothesis (Fodor & Pylyshyn, 1988), and, in the light of that, we can say that his view of the nature of imagery is that it consists of descriptions, in mentalese, of visual objects or scenes. (This means, of course, that Pylyshyn's positive view of imagery – description theory – is viable only if the controversial language of thought hypothesis is true. However, many of Pylyshyn's objections to pictorial theories of imagery might still stand even if this were not the case.) Fodor did not, however, embrace Pylyshyn's objections to pictorial mental imagery. Although he holds that pictorial representations are not sufficient to support cognition on their own, and are probably dependent on associated mentalese representations for playing any cognitive roles that they do play, he nevertheless argues against some philosophical objections to mental pictures,[33] and thinks that the empirical evidence (from psychologists such as Paivio and Shepard) suggests that images do in fact play a real role in our cognitive processes (Fodor, 1975 pp.174–194). It is sometimes objected that a description theory, like Pylyshyn's, is incompatible with the phenomenology of imagery (e.g., Fodor, 1975 p. 188). After all, having a mental image of a cat does not seem anything like reciting a description of a cat to oneself. However, this seems to be based on having drawn too close an analogy between the mentalese descriptions intended by the theory and descriptions in English (or other natural languages). In the first place, although we can be conscious of English sentences as such, we are (pretty much ex hypothesis) never conscious of our mentalese representations as such, but only (at most) of what they represent. Thus there is no reason to expect that entertaining a mentalese description would subjectively seem anything at all like reciting, or reading, or otherwise thinking of a description in English. In the second place, Pylyshyn presumably holds (quite consistently with mainstream “information processing” theories of perception (e.g., Marr,
1982)) that percepts, the end products of visual processing in the brain, are also mentalese descriptions. Thus his theory readily accounts for the phenomenological similarity between imagery and perceptual experience. (If, as seems likely, the perceptual descriptions are typically more detailed than those of imagery, this might also account for any phenomenological differences between imagery and perception.) This introspectively based argument against description theory, weak though it is, is often prelude to the even stronger claim that the phenomenology of imagery directly supports the view that mental images are inner pictures. After all, it is said, in contrast to reciting a description to oneself, having a mental image of (say) a cat does seem very much like seeing a picture of a cat. Although some people seem to find this argument tempting (e.g., Sterelny, 1986), it does not stand up to much examination (Block,1983a; Tye, 1991; Thompson, 2007). There is very little to the alleged similarity between having a mental image of a cat and seeing a picture of one, apart from the fact that both these experiences in some ways resemble the experience of actually seeing a cat, and both differ from it in that no cat need actually be present. It is true that pictures (paintings, drawings, photographs, videos, etc.) provide a familiar and relatively well understood example of how we can have an experience as of seeing something that is not actually present. Indeed, pictures (and sculptures) may be our only familiar example of this, apart from mental imagery itself. However, it does not follow that mental images must therefore be a species of picture. The easy analogy may be a false one. Mental images, after all, are not similar to pictures in many other respects: you cannot turn them over and look at the back of them; they do not need to be in front of your eyes for you to see them; they do not normally seem to be located on a surface; and there is little reason to think that they are normally flat (and good reason, even beyond introspection, to think otherwise (Shepard & Metzler, 1971; Pinker & Kosslyn, 1978; Pinker, 1980; Pinker & Finke,1980; Kerr, 1987)). 4.4.2 The Defense of Analog Imagery In fact, neither Paivio nor Shepard, who were undoubtedly the best known imagery researchers at the time Pylyshyn published his initial (1973) critique, were committed to the straightforward picture theory of imagery that he seemed to be criticizing. Paivio, in response to Pylyshyn, quite explicitly rejected the picture metaphor (and related ones, such as the photograph and the wax impression), and suggested, instead, that imagery is “a dynamic process more like active perception than a passive recorder of experience” (Paivio, 1977).[34] (Unfortunately, however, that is about as explicit as he ever gets about his positive view of the nature of imagery.) Shepard, although he continued to write about the “analog” nature of image representations, and to insist on the “second order isomorphism” between objects and the brain processes that constitute mental images of them (Shepard, 1975, 1978b, 1981, 1984), was also very wary of the picture metaphor, suggesting instead that imagery was related to perceptual anticipation or readiness to recognize (Shepard, 1978b; cf. Cooper, 1976). Two other important early critics of Pylyshyn's position were Ulric Neisser and Ronald Finke. Neisser developed the notion of imagery as perceptual readiness or anticipation into a theory of imagery explicitly opposed to both the picture and description theories (Neisser,
1976, 1978a, b; 1979 – see section 4.5 below). Finke's experimental work on visual illusions and aftereffects induced by imagery (Finke, 1979, 1989 ch. 2; Finke & Schmidt, 1977, 1978) suggested that Pylyshyn was wrong to argue that imagery does not make use of the “cognitively impenetrable” mechanisms of “early” visual processing,[35] and he argued that there is evidence for functional equivalences between imagery and perception (i.e., shared mechanisms) at multiple levels or stages of perceptual processing (Finke, 1980, 1985, 1986, 1989). Neither Paivio nor Shepard, nor, indeed, Neisser, shared the assumptions of the computational functionalist framework from within which Pylyshyn was arguing: Paivio developed a metatheoretical framework that remained rooted in Behaviorist empiricism, and which he called “neomentalism” or “behavioral mentalism” (1975c, 1986); Shepard speculated about the neural basis of imagery in a style reminiscent of the speculative neuroscience of Gestalt field theory[36] (Shepard, 1981, 1984); and Neisser aligned himself with the ecological psychology of J.J. Gibson (1966, 1979).[37] In this regard, Pylyshyn was much more in step than they were with the direction in which cognitive science was going at the time. However, Stephen Kosslyn soon intervened in the debate, proposing a theory of visual imagery that was both explicitly computational and overtly pictorialist (or, as he prefers, quasi-pictorial), based on an analogy with computer graphics programs (which were a fairly new thing back then) (Kosslyn, 1975).[38] Before long, a flood of theoretical and empirical publications from Kosslyn and his collaborators, including several books (Kosslyn, 1980, 1983, 1994; Kosslyn & Koenig, 1992), had established him as clearly the pre-eminent figure of the debate, alongside Pylyshyn. As the leading figures on each side were now both firmly wedded to the theoretical framework of computational functionalism,[39] the debate's scope was, in practice, greatly narrowed by this development, even as its intensity and contentiousness (and notoriety) grew. It developed not into an open ended inquiry into the nature and causes of imagery, but a manichean struggle between the computational pictorialism championed by Kosslyn and his supporters, and the computational description theory still most ably and enthusiastically represented by Pylyshyn. In what remains one of the most effective critiques of Pylyshyn's position, Kosslyn & Pomerantz (1977; see also Kosslyn, 1980 ch. 1), besides giving a point-by-point rebuttal of Pylyshyn's original (1973) arguments, go on to compare how a description theory and a quasi-pictorial theory like that proposed by Kosslyn (1975) might respectively explain various alleged “imagery effects” such as mental rotation, selective interference, and the mental scanning and size/inspection time effects discovered by Kosslyn himself. Their general conclusion was that description (propositional) theories could only explain these effects by making ad hoc auxiliary assumptions about how the propositional (mentalese) code is organized and processed, whereas the explanations of quasi-pictorial theory flow naturally from the core theory itself. Indeed, they pointed out, it was pictorial conceptions of imagery, not “propositional” ones, that had suggested to psychologists that these effects might exist, and that it would be worthwhile devising experiments to confirm them. Pictorial theories had shown themselves to be scientifically fruitful in a way that description theories had not.
On the face of things, description theory predicts that imagery should depend upon the mechanisms and brain structures that subserve conceptual, non-imaginal thought, and not those that subserve perception. Indeed, one of Pylyshyn's favorite arguments against pictorial images turns on his view that perception (but not imagery) depends upon a “cognitively impenetrable,” highly modularized cognitive system (see section 4.4.1). Thus it can be (and has been) argued that it is difficult, or at the least very awkward, for the description theorist to account for a wealth of empirical findings from neuroscience research that indicate that there is a good deal of overlap between the neural structures and cognitive mechanisms involved in imagery and those involved in perception (Kosslyn, 1994, 2005; Kosslyn, Thompson, & Ganis, 2006; Kosslyn & Thompson, 2003; Bartolomeo, 2002; Kosslyn, Ganis, & Thompson, 2001; Kreiman, Koch, & Freid, 2000; Bisiach & Berti, 1990; Farah, 1988). Although a strong case can be made that (despite superficial appearances) these neuroscientific findings do not provide strong evidence in favor of the quasi-pictorial (or any other pictorial) theory of imagery (Thomas, 1999b; Abell & Currie, 1999; Pylyshyn, 2002a,b, 2003a,b; Bartolomeo, 2002), it does not follow that description theory can readily assimilate them. (Incidentally, although it was once widely believed that visual imagery in humans was primarily a function of the right hemisphere of the brain (e.g., Ley, 1983), more recent research contradicts this. It now appears that imagery involves structures on both sides of the brain, with, if anything, the left hemisphere playing a slightly more extensive role (Ehrlichman & Barrett, 1983; Farah, 1984, 1995; Sergent, 1990; Tippett, 1992; Trojano & Grossi, 1994; Loverock & Modigliani, 1995; Michimata, 1997).) As well as having continued to duel with Pylyshyn and other critics, Kosslyn has continued to develop his quasi-pictorial theory of imagery, initially as a computational model (Kosslyn & Shwartz, 1977, 1978; Kosslyn, Pinker, Smith, & Shwartz, 1979: Kosslyn, 1980, 1981), and latterly as a neurological one (Kosslyn, 1988, 1994, 2005; Kosslyn, Ganis, & Thompson, 2001; Kosslyn, Thompson, & Ganis, 2006). He calls the theory quasi-pictorial, to avoid the implication that he thinks images are pictures in too literal and implausible a sense. Quasipictures are not the sort of thing that can be hung on a wall (Kosslyn, 1978b), and you do not need actual eyes inside the head looking at them in order to experience them. Nevertheless, they remain like pictures in many important respects. It remains controversial whether there can be a coherent notion of a quasi-picture that both retains the explanatorily useful properties of true pictures (such as their inherent spatiality, and their capacity to cause visual experiences as of what they represent) and, at the same time, lacks those properties that make it impossible for true pictures to be mental, or even neural, representations (such as needing to be illuminated and before our eyes in order to be experienced). It is clearly Pylyshyn's opinion that there is no such notion, and that much of the superficial plausibility of quasi-pictorial theory depends upon an equivocation between the relatively well understood concept of a picture in the everyday sense, and the essentially non-pictorial notion of an array data structure (Pylyshyn, 1981, 2002a, 2003b). The literal picture in the head theory appeals to our folk-theoretical intuitions, makes interesting predictions, and has the resources to be genuinely explanatory, but it is demonstrably false. On the other hand, the data-structure theory (to which quasi-pictorialists retreat when literal pictorialism is
challenged) is really just a version of Pylyshyn's own description theory, and, properly understood, has none of the special intuitive, explanatory, or predictive advantages that picture theorists claim for their views. Any apparent similarity between pictures and twodimensional array data structures is, according to Pylyshyn, no more than an artifact of the way we customarily present such arrays on paper (or screen) for the benefit of human eyes. It has nothing to do with their actual mathematical properties, or with how they might function in cognition. Despite Pylyshyn's criticisms, however, many philosophers have clearly been impressed by Kosslyn's work. Some have directly defended the quasi-pictorial theory of imagery, attempting to clarify the notion of a quasi-picture, and to show that it has real content (von Eckardt, 1984, 1988, 1993; Tye, 1988, 1991; Cohen, 1996). Others are more circumspect, or less committed to the specifics of Kosslyn's theory, but are now persuaded to countenance the possibility of picture-like mental representations of some sort (e.g. Sober, 1976; Block, 1981a, 1981b, 1983a, 1983b; Bower, 1984; Sterelny, 1986; Rollins, 1989, 2001; Mortensen, 1989; Dennett, 1991; Brann, 1991). Yet others, however, for reasons discussed in the previous section (and the following supplement), remain entirely unpersuaded (e.g., Heil, 1982, 1998; White, 1990; Slezak, 1993, 1995, 2002; Thomas, 1997a, 1999b, 2002, 2003, 2009; Bennett & Hacker, 2003). Further discussion of the analog-propositional debate: Supplement: The Quasi-Pictorial Theory of Imagery, and its Problems
4.5 Beyond Pictures and Propositions Reading most of the recent philosophical literature on imagery (and, it must be admitted, most of the broader cognitive science literature, especially textbooks) one might easily form the impression that quasi-pictures and "propositional" descriptions are the only possible theoretical models for imagery, or, at least, the only ones ever seriously proposed or considered. This is not the case, however.The analog-propositional debate arose, and was at its height, in the 1970s, when cognitive theories based upon symbolic computation seemed to many to be “the only game in town” in cognitive science (Fodor, 1975; Haugeland, 1978), and the quasi-pictorial and propositional models are both products of this mileu. Even in the 70s, however, a number of alternative, non-computational accounts of imagery were being put forward. On the one hand, Taylor (1973) and Skinner (1974) looked for ways to assimilate imagery into Behaviorism. On the other, several cognitive psychologists suggested versions of what may be called enactive (or sensorimotor, or perceptual activity)[40] imagery theories (Hochberg, 1968; Hebb, 1968, 1969; Gibson, 1970, 1979; Juhasz 1969, 1972; Sarbin & Juhasz, 1970; Sarbin, 1972; Neisser, 1976, 1978a, b).[41] Although this work had little impact at the time, more recently, spurred by related developments in perceptual theory, there has been some revival of interest in theories of this type. 4.5.1 Enactive Theories of Perception and Imagery
Enactive theories of imagery may be seen as modern successors to the motor theories of the early twentieth century (see Supplement: The American Response: Behaviorist Iconophobia and Motor Theories of Imagery). They depend the idea that perception is not mere passive receptivity (or even receptivity plus inner processing), but a form of action, something done by the organism. The perceiving organism is not merely registering but exploring and asking questions of its environment (Ellis, 1995), actively and intentionally (though not necessarily with conscious volition) seeking out the answers in the sensory stimuli that surround it.[42] Imagery is then experienced when someone persists in acting out the seeking of some particular information even though they cannot reasonably expect it to be there. We have imagery of, say, a cat, when we go through (some of) the motions of looking at something and determining that it is a cat, even though there is no cat (and perhaps nothing relevant at all) there to be seen. Visually imagining a cat is seeing nothing-in-particular as a cat (Thomas, 1999b, 2003, 2009; cf. Ishiguro, 1967). Farley (1974, 1976) developed a computer simulation inspired by Hochberg's version of enactive theory, and Hampson & Morris (1978, 1979; Morris & Hampson, 1983) discussed and critiqued Neisser's version (which was undoubtedly the most detailed). However, with those exceptions, in the 1970s and 80s the enactive approach to imagery attracted very little attention. It was not just that these non-computational theories seemed irrelevant to psychologists and philosophers whose focus was on integrating imagery into the prevalent (symbolic, GOFAI (Haugeland, 1985)) computational model of the mind. More specifically, the enactive theories do not fit comfortably, if at all, into the framework of computational information processing theory that shaped most scientists' thinking about perception and perceptual experience. Information processing theories come in many varieties, but they all, broadly speaking, depict the sense organs as passive transducers of stimulus energies (light, sound, etc.), whose outputs are then computationally processed and enriched, in the brain, into meaningful mental representations (Lindsay & Norman, 1972; Haber, 1974; Frisby, 1979; Marr, 1982; Pylyshyn, 2003b; Boothe, 2002).[43] Kosslyn's quasi-pictorial theory and Pylyshyn's description theory of imagery were both designed to fit this framework. They differ merely in that Kosslyn holds that the representations comprising imagery are formed at an early stage of visual processing, whereas Pylyshyn holds that they are formed at a late stage. This difference, however, gave rise to the impassioned analog-propositional debate, whose sound and fury only served to further distract attention from theoretical alternatives that did not fit the information processing paradigm.[44] But it is well known that GOFAI style symbolic computationalism did not remain the “only game in town” in cognitive science for long. Since the mid 1980s its hegemony has been repeatedly challenged, first by connectionism (e.g., Rumelhart, McClelland et al., 1986; Clarke, 1989), then by various versions of situated or embodied approaches to cognition (e.g., Varela et al., 1991; Smith, 1991; Clancey, 1997; Clark, 1997), by dynamical systems theory (e.g., Freeman & Skarda, 1990; Port & van Gelder, 1995; van Gelder, 1995; Garson, 1996), and by cognitive neuroscience (e.g., Kosslyn & Koenig, 1992; Gazzaniga, 2004). Connectionism did not challenge the information processing view of perception, however, and thus proved of little significance for imagery theory, inspiring little more than a handful of variants of quasi-pictorial array theory (Julstrom & Baron, 1985; Mel, 1986, 1990; Stucki
& Pollack, 1992). The robotic system Murphy, designed by Mel (1990), has some interesting features in that it combines such a connectionist model of visual imagery with a model of trial-and-error learning of motor control, wherein information in the putative image is used to control the reaching behavior of a robotic arm (although it is not obvious that imagery, as distinct from visual perception, plays any such role in human reaching). Grush (2004) adopts this model as the basis of his own account of visual mental imagery, within the wider context of his “emulation” theory of cognition. Nevertheless, Mel and Grush continue to conceive of the image itself as being a two dimensional array of elements, just as the quasi-pictorial theory of Kosslyn does, and, indeed, in support of their models both Mel and Grush follow Kosslyn in appealing to evidence about the spatial properties of imagery and about the involvement of visual areas of the brain in imagery. Thus, despite the fact that Mel and Grush situate their accounts of imagery in the context of motor control rather than of visual cognition, they remain quasi-pictorial accounts, and are, in most respects, considerably less developed than (though perhaps consistent with) the version of quasi-pictorial theory developed by Kosslyn. As such, they share most of the virtues of Kosslyn's version, and are subject to the same objections (see supplement: The Quasi-Pictorial Theory of Imagery, and its Problems). Dynamical systems theory has also had relatively little to say about imagery, although Freeman (1983) has sketched an account of olfactory imagery in terms of neural dynamics. He explicitly distances himself from both sides of the analog-propositional debate, and makes appeal instead to the concept of search image as used in the science of Behavioral Ecology. A search image is (to a first approximation) a specific, learned recognitional capacity, or a form of selective attention, that leads a predator species to recognize and preferentially prey upon members of the more abundant prey species in its environment, whilst largely failing to notice less abundant types of potential prey (Tinbergen, 1960; Atema et al., 1980; Lawrence & Allen, 1983; Langley, 1996; Blough, 2002). However, it is less than clear that Freeman is justified in conflating this concept of search image with that of mental image, as used in folk psychology and cognitive science. It is mainly the rise of situated and embodied approaches to cognition that has challenged the information processing approach to perception, and enabled the re-emergence and further development of enactive imagery theory. During the 1980s, robotics researchers interested in creating robots to operate in real wold environments were finding that getting a machine to process information from sensory transducers into an internal representation of its surroundings that would provide a suitable basis for action planning was a very difficult computational problem. Indeed, some became convinced that, even if it could be done in principle, in practice the process would be unacceptably slow, unreliable, and computationally expensive (by the time the robot knew what was going on, things would have changed). Thus, there was a turn toward “active” (or “animate”) techniques in robotic perception. Instead of attempting to build up detailed internal representations of their environment, robots began to be designed to deploy their sensors purposively, to actively seek out just the specific information needed at that particular moment for making an impending behavioral decision (e.g., Bajcsy, 1988; Ballard, 1991; Blake & Yuille, 1992; Aloimonos, 1993; Swain & Stricker, 1993; Landy et al.,1996; Davison, 2003; Lungarella & Sporns, 2006).
At around the same time, a number of neuroscientists, perceptual psychologists, and philosophers began, for diverse reasons, to converge on a similar view of human vision (Ramachandran, 1990; O'Regan, 1992; Churchland et al., 1994; Akins, 1996; Cotterill, 1997; Thomas, 1999b, 2009; Hayhoe, 2000; O'Regan & Noë, 2001; Noë, 2002, 2004, 2009). Studies of (amongst other things) exploratory perceptual behaviors such as eye movements (Yarbus, 1967; Noton & Stark, 1971a,b; Landy et al., 1996; Hayhoe & Ballard, 2005), and recently recognized perceptual effects such as change blindness (Grimes, 1996; Simons & Levin, 1997; O'Regan, 2003) and inattentional blindness (Neisser & Becklen, 1975; Mack & Rock, 1998, 1999; Simons & Chabris, 1999), cast doubt on the traditional idea that a rich and detailed inner representation of the visual scene mediates our visual consciousness. Instead, some now argue that perception depends on a multitude of special purpose neural and behavioral structures and/or routines (Ullman, 1984; Ramachandran, 1990; Thomas, 1999b, 2009; Roelfsema et al., 2000; Hayhoe, 2000; Roelfsema, 2005), each of which actively utilizes the sensory transducers (eyes, ears, etc.) in a different way in order to obtain specific types of information as and when needed. We do not have our sense of the immediate perceptual presence of the world because we have a representation of it in our heads, but rather because these routines operate (for the most part) so quickly and effortlessly that virtually as soon as we want to know some perceptually available fact, we are able to discover it.[45] Although this way of thinking about perception remains a minority view, and certainly does not dominate perceptual theory in the way that information processing theory once did, it has nevertheless created a theoretical space in which enactive/motor theories of imagery can be more plausibly entertained, and the theory has recently been broached again by an assortment of philosophers, neuroscientists, psychologists, and computer scientists (Thomas, 1987, 1997b, 1999b, 2009; Newton, 1993, 1996; Ellis, 1995; Ramachandran & Hirstein, 1997 p. 442; Marks, 1990, 1999; Bartolomeo, 2002; Bartolomeo & Chokron, 2002; Blain, 2006). Thomas (1999b) argues that enactive theory can explain experimental cognitive psychology's findings about imagery (see sections 4.2 and 4.3 above) at least as well as the better-known quasi-pictorial and propositional/description theories, and, indeed, that it handles the facts about imagery in the blind and image reconstrual (see Supplement: The Quasi-Pictorial Theory of Imagery) in a more principled and plausible way than they do. It has also been argued that enactive theory can provide a more satisfactory explanation of the neurological evidence about imagery (i.e., the ways imagery experience and abilities may be impacted by various forms of brain damage), and, in particular, the syndrome of representational neglect (see Supplement: Representational Neglect) than other theories can (Bartolomeo, 2002; Bartolomeo & Chokron, 2002; Dulin et al., 2008). Other relevant evidence comes from studies of eye movements during imagery. Saccades are quick, mostly unconscious, flicks of the eyes, which are now known to play an important role not only in vision, but in visual imagery as well. In normal human vision they occur, on average, about three times every second, and play a vital role in our visual system's exploration of the visual world, and the extraction of information from it. The pattern of our saccadic movements is purposeful, under cognitive control, and depends both on what we are looking at, and on what visual information we hope to obtain, on the purpose behind our looking (Yarbus, 1967; Noton & Stark, 1971a,b; Stark & Ellis, 1981; Findlay & Gilchrist,
2003; Hayhoe & Ballard, 2005; Rucci et al., 2007; Rothkopf et al., 2007; Martinez-Conde & Macknik, 2007; Trommershäuser et al., 2009). Although the scientific study of saccades began well over a century ago, in recent years technological advances in eye-tracking technology have led to a rapid growth in understanding and appreciation of the large role that they play in human vision (Richardson & Spivey, 2004; Wade & Tatler, 2005). It has also become apparent that saccades (and perhaps other types of eye movement too) play a significant role in visual mental imagery. Laboratory studies have shown that, when people hold a visual image in mind, they spontaneously and unconsciously make saccadic eye movements that (at least partially) enact the stimulus-specific pattern of such movements that they would make if actually looking at the equivalent visual stimulus (Brandt & Stark, 1997; Demarais & Cohen, 1998; Spivey & Geng, 2001; Laeng & Teodorescu, 2002; de'Sperati, 2003; Johansson et al., 2005, 2006; see also Jacobson, 1932). Furthermore, imagery is disrupted (to a greater or lesser degree) if someone who is holding an image in their mind either restrains themselves (to the limited degree that this is possible) from making eye movements, or else deliberately moves their eyes in an image-irrelevant way, thus disrupting the spontaneous saccadic pattern (Antrobus et al., 1964; Singer & Antrobus, 1965; Andrade et al., 1997; Ruggieri, 1999; Laeng & Teodorescu, 2002; Barrowcliff et al., 2004; Gunter & Bodner, 2008). The psychotherapeutic technique known as EMDR (Eye Movement Desensitization and Reprocessing) may perhaps owe its effectiveness largely to this fact. EMDR is widely used in the treatment of Post-Traumatic Stress Disorder, and studies of therapeutic outcomes have found evidence of its effectiveness (Van Etten & Taylor, 1998; Shepherd et al., 2000; APA, 2006; Bisson et al., 2007; Högberg et al., 2007). In EMDR treatment, patients are induced to deliberately move their eyes back and forth whilst visually recalling the events that have traumatized them; it is claimed that this leads to a significant reduction in the vividness of their memories of those events, and of the distress, and consequent symptoms, that those memories cause (Shapiro & Forrest, 1997; Shapiro, 2001; Mollon, 2005). Although the mechanisms and real therapeutic effectiveness of EMDR remain controversial (Herbert et al., 2000; Davidson & Parker, 2001; Perkins & Rouanzoin, 2002), the disruptive effect of deliberate eye movement upon visual imagery appears to be well established, and it implies that the eye movements that spontaneously occur when people visualize things (or, at the least, the brain processes that initiate and control these movements) are not mere accompaniments or epiphenomena of the imagery, but are (as enactive theory would lead one to expect) a true, functionally significant part of the physiological process that embodies it.[46] (However, Mast & Kosslyn (2002b) argue that the eye-movement evidence can also be accommodated to quasi-pictorial theory.[47]) Kosslyn, Thompson, Sukel, & Alpert (2005; see also Kosslyn, Thompson, & Ganis, 2006 pp. 90–92) report an experiment in which subjects were asked to recall mental images of simple geometrical arrangements while PET scans of their brains were taken. Although all subjects formed images of the same figures, some originally formed them on the basis of verbal descriptions, whereas others were shown separate segments of the entire structure to be visualized, and asked to assemble them mentally into the complete figure. The PET scan was not taken at the time the images were originally formed in one or other of these ways, but
when they were later recalled. According to the experimenters, enactive theory holds that when someone recalls a mental image they re-enact what they did at the time of its original formation, and since the two subject groups originally formed their images in very different ways, the theory predicts that the two groups should display radically different patterns of brain activation at the time of recall. In fact, however, no marked differences were seen. This is claimed to constitute a refutation of the enactive theory. It rests, however, on a demonstrable misunderstanding of the theory. No version of enactive imagery theory holds (either explicitly or implicitly) what these experimenters claim it holds: that recall of mental imagery is constituted by re-enactment of whatever was the original act of image formation. What enactive theory in fact holds is that imagery (recalled or otherwise) is constituted by (partial) enactment of the perceptual acts that would be carried out if one were actually perceiving whatever is being imagined. It is true that in the most straightforward and paradigmatic case of mental image formation – the direct recall of an earlier perceptual experience of something – enactment of what one would be doing if actually perceiving that thing is equivalent to re-enactment of what one did during the original perceptual episode. However, this equivalence clearly breaks down in most other circumstances, including those of the experiment in question. Since both groups of subjects in the experiment under discussion were supposed to be recalling an image of the same geometrical pattern when their brains were scanned, enactive theory actually predicts that the neural activity due to the recalled image should be much the same in each group, just as was found. Quite apart from empirical evidence, certain distinctively philosophical advantages have been claimed for enactive theory. It has been suggested that it is better able than its rivals to explain imaginal consciousness (Ellis, 1995; Thomas, 1999b, 2001, 2009; Bartolomeo, 2002), and Thomas (1987, 1997a, 1999b) argues that enactive theory can provide the basis for an understanding of the concept of imagination, whereas quasi-pictorial theory and description theory cannot.[48] Traditionally, both philosophers and the folk have thought of the imagination as a mental faculty responsible both for mental imagery, and for the most admired forms of artistic (and other) creativity.[49] Unfortunately, neither picture nor description theories of imagery seem capable of providing a satisfactory account of how one mental faculty could be responsible for both these things (which may go some way toward explaining why many recent philosophers doubt whether there is any such faculty[50]). However, Thomas (1997a; 1999b) argues that enactive theory depicts both imagery and creative thinking as manifestations of the more basic imaginative capacity of intentionalistic perception (or “seeing as”). It has also been suggested (Newton, 1993, 1996; Thomas, 1999b, 2003, 2009; see also Heil, 1998 ch. 6) that, because it regards imagery not as a form of representational inscription (whether pictorial or descriptive), but as a form of action, enactive theory may be able to account for the intentionality of imagery without making appeal either to the controversial language of thought hypothesis, or to the widely discredited (see section 3.3) resemblance theory of representation.[51] However, if mental images are (as just about everybody believes) a species of mental representation, these latter claims are at odds with the idea that mental representations are
token identical to brain states. The majority of cognitive scientists (and sympathetic philosophers) remain firmly committed to that idea, and perhaps it is largely for that reason that enactive theory remains a minority viewpoint. Certainly it has yet to receive anything like the amount of attention (either supportive or critical) that experimenters and theorists have devoted to quasi-pictorial and description theories. Further discussion: Supplement: Representational Neglect
4.6 The Return of the Imagery Theory of Cognition? The analog-propositional debate and the enactive theory of imagery concern themselves primarily with the nature and underlying mechanisms of the phenomenon, and have thus had relatively little direct impact on views about the function of imagery in cognition. In fact, both of the best known cognitive theories of imagery, the quasi-pictorial theory of Kosslyn (1980, 1994, 2005; Kosslyn, Thompson, & Ganis, 2006) (especially as philosophically glossed by Tye (1991)), and the description theory of Pylyshyn (1973, 1978, 2003b), portray imagery as embedded within and dependent upon a more fundamental, language-like mental representational system, mentalese, from which it derives much or all of its semantic content. Thus neither of these theories did much to challenge the post-Wittgensteinian consensus (see section 3.3) that continues to give imagery, at most, a minor, auxiliary role in cognition, with most of the burden being carried by either natural language or the more basic and more flexible representations of the hypothetical mentalese. Some neuroscientists and psychologists have been little moved by this consensus. Damasio (1994), for example, takes it for granted that mental representations are imagistic; Bisiach & Berti (1990) and Edelman (1992) argue that mentalese (but not image) representations are neuroscientifically implausible; and Paivio (e.g., 1971, 1986, 2007; Paivio & Begg, 1981; Sadoski & Paivio, 2001) elaborates a comprehensive theory of cognition entirely in terms of image and natural language representations, and holds that that the representational power of language derives from that of imagery (see Supplement: Dual Coding and Common Coding Theories of Memory). However, as these authors did rather little to address the arguments that have led most contemporary philosophers to think that imagery cannot be representationally basic, their views (in this regard) have had relatively little impact on philosophy. However, more recently those arguments have been challenged by philosophers such as Lowe (1995, 1996, 2005), Nyíri (2001), and Ellis (1995). Ellis outlines a theory of how the meaningfulness of language may be grounded in imagery that appears to meet at least some of the stock objections (see section 3.3, and Thomas, 1997b). The arguments are also addressed, at least in part, by Barsalou and his collaborators, who have proposed a theory of what they call “perceptual symbol systems” as an alternative to the language-like, “amodal” (mentalese) symbol systems of traditional cognitive science (Barsalou, 1993, 1999; Barsalou & Prinz, 1997; Barsalou et al., 2003; Kan et al., 2003). Although Barsalou denies that the perceptual symbols of his theory can be straightforwardly equated with mental images
(mainly because he holds that they may sometimes be active in cognitive processes without our being conscious of them)[52] he clearly conceives of them in a way very close to traditional conceptions of imagery, and certainly as being the immediate causes of our imagery experience when we actually do have it. Barsalou holds that the neural basis of his perceptual symbols is a neural “simulation” of the brain processes that would be involved in the actual perception of whatever it is that is being symbolized. Others, such as Currie (1995; Currie & Ravenscroft, 1997; Abell & Currie, 1999) and Hesslow (2002), have also suggested that imagery is best understood as a simulation of perception. However, quasi-pictorial, enactive, and probably even propositional/descriptional theories of imagery can all reasonably be classed as simulative theories in the relevant sense (see Nichols et al., 1996), so it is not clear that this suggestion advances our understanding of the nature of imagery very much.[53] In any case, Barsalou's main interest is not in the nature of imagery, but in how perceptual symbols might function in cognition to do the jobs that others have thought can only be done by a more language-like system of representations, such as representing logical relations and propositions (as opposed to just representing things). His detailed suggestions about these questions have aroused much interest. Perhaps inspired by Barsalou's work, Prinz (2002; see also Gallese & Lakoff, 2005) has recently made a detailed defense of something very like the traditional Empiricist theory of concepts (usually, although not invariably, interpreted as the view that concepts are images (see section 2.3.3)). Like Barsalou (and, indeed, Locke), he does not take any strong position as to the inherent nature of images or perceptual symbols (and thus avoids embroiling himself in the analog-propositional debate and its aftermath). Instead, he confines himself to trying to show that it is plausible that our fundamental concepts are perceptual in their genesis and character, a view that he is quite happy to acknowledge is very close to the traditional imagery theory of cognition. Prinz deals ingeniously with many of the standard philosophical objections to theories of this sort, and he sidesteps what has been the main philosophical objection to image theories of concepts by avoiding committing himself to the resemblance theory of representation (see section 3.3 above). Instead, he suggests that his account of perceptual representations can be combined with a version of the causal (or covariation) theory of intentional content developed by Fodor (1990, 1994), Dretske (1995, 2000), and others.[54] However, it remains open to question whether such a causal theory can work (Cummins, 1997; Horst, 1996, 1999). Other recent work has sought to explore the relationship between current conceptions of mental imagery and the more resonant, but more nebulous, notion of imagination (and related concepts such as insight and creativity) (White, 1990; Brann, 1991; Finke et al., 1992; Thomas, 1997a,b, 1999a,b, 2006; Kind, 2001; McGinn, 2004; Blain, 2006). Perhaps the most ambitious claims in this regard are those of Arp (2005, 2008), who comes at the matter from the controversial perspective of evolutionary psychology. Arp suggests that an innate, evolved capacity for what he calls scenario visualization is unique to the human species, and is the crucial factor that has made our high-level creative problem-solving abilities possible. From this perspective, it is in large part thanks to our capacity to form and manipulate mental
imagery that humankind has been able to out-compete rival species, and develop our complex cultures and technologies.
