Lecture 4: From the Linguistic Model to Semiotic Ecology: Structure and Indexicality in Pictures and in the Perceptual World

The argument of our third lecture showed that iconicity could only be saved from the critical arguments advanced by Bierman and Goodman by means of introducing a properly structured common sense world. In this lecture, we will first consider to what extent the linguistic model may still be helpful, and in which respects it is misleading. Then the necessary furnishing of the common sense world, which is also the basis of picture interpretation, will be discussed in its own right. In this connection, the importance of indexicality to perception, in itself and as it carries over to pictorial representation, will be demonstrated. This will also prompt a return to the theory of indexicality, inspired, once again, in a close reading of Peirce, but developed on the bases of more recent psychological findings. The function of structural opposition will be discussed in contrast to the perceptual logic of indexicality.

Contents
4.1. The linguistic metaphor revisited: The semiotic function according to Hjelmslev 5 Presuppositions of the linguistic model The principle of pertinence 8 10 12 13 6

Beyond linguisticism: The symptom model Some simple systems of visual signification Pictorial semiogenesis as resemanticization Visuality as a Hjelmslevean form 15

The dimensions of pictorial specificity Summary 19

17

4.2. Beyond density and repleteness 19 The linguistic model regained: hyletic issues 20

The linguistic model regained: semantic issues 25 Mount Fujiyama and the electrocardiogram Exhibitive import and pictorial semiosis 29 Summary 30 4.3. Similarity and contiguity of verbal and visual semiosis 31 In defence of the linguistic model fallacy, malgr tout Structuralism without a structure 33 The resources of verbal and visual semiosis Linguistic and logical oppositions 43 Oppositions in semantics and semiotics 45 Presence and different kinds of absence 47 Summary 48 4.4. Figures of perception: wholes and neighbourhoods 49 Three (or perhaps four) kinds of perceptual theory 49


26

31

37

The furniture of the Lifeworld 54 Semiotics as the general theory of categories Prototypes, idealtypes, and antitypes 67 74 59

Two kinds of wholes: Structure and configuration Parts and relations to wholes 80 Summary 86

4.5. From the world to the picture and back again 87 Exhibit import and complex ideas 87

First rhetoric: Shifting the levels of intension and extension 93 The chunks of the visual world Charlie Brown in the Lifeworld References 113 96 101

The third lecture was concerned to show, not only that pictures are iconical signs, but also that they are a very peculiar kind of icons, in which the similarity is of the kind known as depiction. In the process of the argument, we could not avoid attending in passing to the linguistic metaphor, which consists in treating pictures (just as film, architecture, dance, etc.) as being organized in the same way as verbal language, in particular as having what is known in linguistics as double articulation or duality of patterning. In this lecture, I will first show that the linguistic metaphor is unhelpful, not only in its structuralist version, which is geared to identity, but also as it has been used in the form of a contrastive argument by, notably, Nelson Goodman. In spite of the shortcomings of the linguistic metaphor, it will be suggested that is still valid to some extent on a secondary level, which is equivalent to the kind of oppositions found in mythic consciousness by Lvi-Strauss. Then I will suggest that there are


indeed ways in which pictures may be divided into elements, also on the basic level, but not at all of the kind found in language. This will bring us to considerations of perceptual theory and the organisation of the world taken for granted. And it will permit us to return to a classical issue of semiotic theory, first broached in a systematic way by Lessing, when he opposed the pregnant moment of pictures to the characteristic properties of language. Indeed, as I will show, there are more true empirical observations of the distinction between semiotic resources in the work of Lessing, in spite of the preconceptions of his age, then in the more recent contributions to the contrastive study of pictures and language by, notably, Kress & van Leeuwen. Ever since Lessing and other Enlightenment semioticians, pictures have been taken, as a matter of course, to be more strongly material than literature, the more transparent, or subtle, expression plane of which is made up of articulate tones, i.e. linguistic sounds, or phonemes (cf. Wellbery 1984; Sonesson 1988:105ff). Yet there has long existed a discipline focused on the material character of language, that part by means of which language is given to perception, phonetics, and even a sub-field consecrated to the linguistic expression plane in its function as a carrier of meaning, phonology or phonemics, whereas we have hardly begun to consider seriously the material, and therefore perceptual, nature of pictorial meaning, let alone the perceptual organisation specifically characterising the pictorial expression plane. Even the Prague school model, which is the only one in classical semiotics to insist on the peculiar perceptual nature of the process by which artefacts are turned into signs, was

principally conceived to account for literary and other verbal meanings, rather than pictures. The general study of pictures, or pictorial semiotics, may have to start out from something equivalent to phonetics, although it will only attain maturity, as linguistics did, when it is able to create a domain parallel to phonology which is not to say a discipline copied on phonology. In the early sixties, the art historian E.H. Gombrich (1960:7) declared the creation of a linguistics of the visual image, separate from art history, to be an urgent task; and yet, in the late seventies, the psychologist James Gibson (1978:228) was still complaining over the fact that nothing even approximating a science of depiction, comparable to the science of language, had been developed. The analogy to linguistics (and, more particularly, to phonology) is valid as far as the type of science and the nature of the research interest are concerned. But precisely because of these parallels, the divergent character of both semiotic resources will naturally make these sciences very different. In the meantime, and since then, Gibson and his disciples and colleagues, such as Hochberg, Kennedy, and Hagen, have been making important contributions towards a psychosemiotics of picture perception. More explicitly semiotic work on pictorial meaning has been accomplished, with some influence from Gombrich, but without any connection to Gibson, by the French Structuralists. Some of the more innovative thinkers of that group, such as Lindekens and Tardy, already devoted some attention to the perceptual peculiarities of pictures, repeating and applying the findings of Gestalt psychology and the


studies of eye-movements. In his critique of the iconicity of pictorial signs, Eco insisted on the correlation, not between the picture and the thing itself, but between the former and the perceptual model, yet failed to give any substance to the latter notion. Clinging to the autonomy postulate, the representatives of the Greimas school denied any relevance of perceptual psychology to their study, refusing to take account, also in any other way, of the specificity of iconicity (in the Peircean sense, not, of course, the quite different Greimasean one; cf. Lecture 3), let alone picturehood. Fernande Saint-Martin, and more recently the Belgian Groupe , take for granted the importance of perceptual organisation to pictures, yet both are explicitly determined to dissolve pictorality into the more general sphere of visual semiosis, without asking any question about the relation of pictorality to the features of the (mostly visually experienced) perceptual world. In the following, we will investigate the more distinctly perceptual character of pictures, as opposed, not only to linguistic and other non-visual signs, but also to visual, non-pictorial ones. After considering the semiotic function as conceived by Saussure and amplified by Hjelmslev, we will proceed to an appraisal of such properties as density and repleteness, taken by Nelson Goodman to be the defining characters of picturehood. As we go along, we will suggest that the perceptual charFr. mutton living animal adult non-adult sheep lamb animal for eating mutton lamb

acter of pictures must be allowed to have a much deeper influence on the way pictorial signs are analysed, and that it is rather on the basis of a closer scrutiny of the linguistic model, than from its outright rejection, that we may hope to regain the perceptual model.

4.1. The linguistic metaphor revisited: The semiotic function according to Hjelmslev
A central part of the second lecture was dedicated to the discussion of what Piaget termed the semiotic function. We now turn to the second classical locus of the concept of semiotic function: the work of Louis Hjelmslev. To Hjelmslev, as to his predecessor Saussure, the sign is made up of two units, the signifier and the signified, with a third item, the referent, being relegated outside the sign. The semiotic function is that which joins the signifier and the signified inextricably together. That particular kind of function obtaining between the signifier and the signified is a solidarity, that is, a mutual implication. That is to say that, given a particular signifier, one and only one signified is implied (in a particular semiotic system), and vice-versa. This is seen in the commutation test: when, in the word pier, the p is exchanged for a b (or the feature voiceless is exchanged for voiced), we get beer, which has an entirely different meaning. And when the feature non-adult replaces adult in

Fig. . Saussures example of the change of meaning of mutton from French to English, when opposed to sheep, complemented with the oppositions to non-adult animals

sheep, we obtain lamb. As Saussure himself noted, mutton in English has the additional trait food stuff, which is not relevant for the similarly-looking French word: le mutton sur la table is ambigous, which mutton on the table is not (see Fig.1). In spite of their reference to a common Saussurean heritage, Hjelmslev and Piaget certainly use the term function to mean different things. To Hjelmslev, it simply is a relation, a mapping of one object onto another. Piaget, however, would seem to locate functionality in a wider context: as an ability to put something into use. At the same time, concerning the limited domain which is his theme, Hjelmslev is much more explicit, and much more precise. It is possible to ask, then, if there could be a unitary semiotic function, also in Hjelmslevs sense, notably as applying to verbal language and pictures alike. More in particular, we have to investigate, in the following, whether the Hjelmslevian semiotic function may apply also to pictures. Presuppositions of the linguistic model Some of the presuppositions of the semiotic function, as conceived by Hjelmslev, are the following: Expression and content are connected by a mutual implication (a solidarity); they are mental or, more precisely, intersubjective entities. The relation between expression and content is arbitrary, and is merely determined by a convention: that is, the expression is not similar to, or in some other way intrinsically motivated by the content. The referent, which may be a ma The basic sources here are of course Hjelmslev 943; 959; 973.

terial object, lies outside the sign. The entire sign is arbitrary in relation to the referent, that is, the way a particular language divides up the world does not reproduce physical reality. Only some features that usually accompany the expression are indispensable for the same content to result, and viceversa; these features make up the form, as against the substance, of the sign. There is a principle of pertinence, or relevance, which determines which features are required. This is what is seen in the commutation test. A sign may be divided into small-

Expression
Content

Fig. . The simplest example of double articulation: the first division is parallell on both sides of the sign, the second one does not coincide.

er signs, as long as each separation on the expression plans is paralleled on the content plane; in this case, the larger sign will be called a statement. But there may come a point in the division procedure, when the only repeatable units that are to be found are such that they have no correspondence on the other plane of the sign, and must be determined separately for the expression


plane and the content plane. In this case, the iterable units are called figurae (phonemes and distinctive features in linguistics). A semiotic system having both an articulation into signs and an articulation into figurae is said to be doubly articulated. Thus, if there is a unitary semiotic function, in Hjelmslevs sense, then all kinds of semiosis would require the presence of an arbitrary relation between sign and reality, and between expression and content; and a double articulation, involving both features and signs (cf. Fig. 2.). Indeed, Umberto Eco, along with many other semioticians, originally argued that the picture was as arbitrary as the verbal sign, and was doubly articulated. Later, however, he claimed, rather similarly to Nelson Goodman, that pictures were indeed arbitrary, but could not be dissolved into features (which should in fact make double articulation impossible). On the other hand, I have tried to show, relying both on a critique of numerous theories of pictorial meaning, and on the evidence from perceptual psychology, that pictures must be feature-based and motivated by similarity (see Sonesson 1989a,III.). Still, there is something to be learnt from the Hjelmslevean semiotic function: the importance of allo-functionality, the mutual dependence of expression and content. Allo-functionally, however, depends of pertinence. The principle of pertinence There is every reason to retain the principle
 Ignoring the expression plane, this clearly has something to do with what in modern philosophy and linguistics is known as the principle of compositionality, according to which the meaning of a complex sign (here the statement) is determined by the meanings of the constituent signs and the rules for their combination. Cf. the discussion of Goodman in the next section.

Fig. . The map of France, a staircase and French population statistics in diagram form in the same picture but with different principles of pertinence

of pertinence: not everything on the pictorial expression plane is equally relevant to content, and vice-versa, although, as we shall see, the limits may be less absolute than in verbal signs. Indeed, in some pictures, various principles of pertinence are at work at the same time, and the precise requirement that these different contents place on the expression differ. This may be seen in figure 3, which is at the same time a map of France, a picture of a staircase, and a diagram. More importantly, however, all pictures would seem to involve at least two principles of pertinence, commonly termed the iconic (or, more exactly, pictorial) and the plastic ones (see Sonesson 1992a, c). On the pictorial (iconic) level, the picture stands for some object recognizable from the ordinary perceptual Lifeworld; whereas, on the plastic level, expression is mostly conveyed by simple qualities of the picture thing itself, which tend to cor

Fig. . Different renderings of noses, eyes, ears, and mouths, some of which need more context to be recognizable than the others - that is, in order to be resemanticized.

respond to increasingly abstract concepts (see Groupe 978; 979; 985; Floch 985: 5; 986a, passim; 986b. 6ff, and passim). When pictorial semiotics was first launched, everybody seemed to believe in the existence of some kind of minimal unit of pictorial meaning, sometimes termed an iconeme. This is true of such authors as Eco, Koch, Floch, Gauthier, Thrlemann, Lindekens, Groupe , Gubern, Tardy, Vilchez, Paromio, etc. Iconic codes, according to Eco, possess their figurae, signs, and statements, just like verbal language. Although Eco was later to reject, ever more emphatically, the existence of pictorial features, he retains, at least as late as in 976, his idea, according to which films are organized into three articulations, which supposes the double articulation model for pictures, according to which pictures are built up of distinctive traits, with no independent meaning, forming together autonomous signs.3
3 In his Platypus book, Eco (2000:337ff) regrets having allowed his essay on triple filmic articulation to be reproduced and translated after he stopped believing in the theory, but he says nothing about the inherent contradiction in his Theory (Eco 1976). A very different kind of feature is proposed by Saint-Martin (1987). For some

In spite of the arguments of Barthes, Metz, and the second Eco, there is every reason to accept, along with such psychologists of perception as Gibson, Kennedy, and Hochberg, the existence of pictorial features. But these features differ in important respects from those of linguistics. Like all features that pertain to signs, pictorial features must be allo-functionally defined, that is, they derive their identity from their relation to the other plane of the sign; and although there are probably not just a small number of them, they could scarcely be infinite. On the other hand, pictorial features are not meaningless, at least not in the way phonemes are. Structure in general, and binary, privative oppositions in particular, do not seem to be fundamentally involved in the constitution of pictorial meaning (cf. Sonesson 1989a; 1992a, c). While pictorial features do seem to be categorical in themselves, their relation to the other plane of the sign is merely probabilistic. Indeed, the pictorial sign contains many redundant expressions for one content, but also a cumulation of contents conveyed by a single expression. In this way, they are similar to
problems posed by this feature concept, cf. my review, Sonesson 993a. 

the features present in the perceptual Lifeworld, but, for the same reason, they allow for rhetorical modifications of our Lifeworld experience. Ecos contention that the cinema possesses three articulations, which is based on the double articulation of static pictures, clearly derives from an at least threefold confusion about the import of the notion of articulation in linguistics (cf. Sonesson 1989a,III.4.; 1992d). Most importantly, Eco confounds levels of configuration, where a whole adds further meaning to its parts, and levels of appresentation, where there is a passage to a quite different realm of reality. As we saw in lecture , appresentation is Husserls term for the semiotic function, that is, the sign; thus, if there are several levels of appresentation, the signified of one sign will be the signifier of another, and so on. The configuration is the perceptual whole, the Gestalt, which emerges out of the conjunction of numerous smaller parts; therefore, there will be more than one configurational level to the extent that a number of wholes are integrated into a single, more comprehensive whole.4 In fact, although linguists have never presented a satisfactory definition of articulation, it is clear form their analytical practice that they take it to imply a concurrent shift in level of appresentation and configurational
4 The difference is illustrated with a drawing by Magritte (fig.14 of Lecture 3) discussed later on in this lecture

level. Thus, there cannot be any triple articulation in the cinema. Nor can Ecos three stratum model of pictorial meaning be sustained, for, as soon as we attend to the definitions given, the figurae level merges with the sign level, and the sign level with the semata level. Although pictorial features are really meaningless in isolation, they acquire specific meanings as parts of a signifying configuration, and thus they do not form anything comparable to the second linguistic articulation. If there is something like the first linguistic articulation in pictures, then it appears on different configurational and extensional levels of the picture, in different kinds of picture. A nose is a nose is a nose but it may appear as a feature or a sign, depending on the pictorial style chosen (Fig. 4). Beyond linguisticism: The symptom model Louis Porcher (1976) and Guy Gauthier (1979) both tried to apply the commutation test in the analysis of pictures, with strange results. Porcher took away ever larger portions of an advertisement picture, such as the steering-wheel, the interior of the car, the road outside, the others cars visible on the road, and so on, concluding that there was no way of getting rid of the meaning voyage. Of course, if voyage is a meaning of the picture, it is a very compound sign indeed, and if effacing the

Fig. . The Peanuts code, according to Gauthier, as analyzed in Sonesson a (a-d Charlie Brown; e a Brtcher face as comparison) 

steering wheel does not remove the content voyage, we should at least expect it to nullify the content steering-wheel. But it does not: the contextual constraints, the indexicalities, are much more heavily determinate than in verbal language. As for Gauthier, he claimed that the water drop on its own accounted for the meaning freshness attributed to the tomato appearing inside a bottle in a publicity picture. In fact, of course, it can do so only when seen as a water drop on a tomato, with the additional assumption that it is more probably a dew drop than some sprinkle of water remaining after the tomato has been washed. In any case, the same water drop is a carrier of many other meanings: it shows that the tomato has recently been outside the bottle; the light reflected in it indicates at light source, and so on. In his analysis of Schultzs comic strip Peanuts, Gauthier (1976) observes that there is no separate expression for such meanings as eye, opening/shutting of eye, the eye in profile/from in front, or even the parts of the eye (cf. Fig. 5). The same line may participate in conveying the contents eye, openness of eye, profile view, smile, happiness, and so on. While this is true, many different things seem to be confounded in this example. The line that makes up part of the configuration that means eye may possibly participate in the expression of the meanings smile and happiness, in the same way as the real eye in the real world participates in the expression of these real-world meanings. Thus, like in a compound word, the eye, together with other signs, builds up a larger unit, but unlike the case of verbal language, the eye retains its own meaning. Then, in the second place, we encounter one of the pe-

culiarities of the Peanuts code: that there is no separate expression for parts of the eye. Third, and more interestingly, there are clearly features that must go together, if the result should be what is considered a prototypical picture: a visual display which has separate expressions for the eye as such, its state of openness, and the point of view from which the observer confronts it, could be some kind of notation (such as those used by Birdwhistell and Kendon to describe occurrences of gestural communication), but it would no longer be a picture, simply because such a dissociation would not respect the conditions of Lifeworld experience. Simple modifications of such traits, however, may be the signs of pictorial rhetoric (as in the case of some Cubistic pictures). In conclusion, the picture not only turns out to be more highly redundant than the linguistic sign, it also tends to cumulate meanings in ways making them hard to dissociate. Rather than the linguistic model, we should perhaps look for a medical model of pictorial meaning. Inside medicine the study of symptoms has long been designated in many parts of the world with the term semiology or semiotics. In the sixties, when all respectable signs where thought to possess double articulation, Barthes (1972) tried to show that even the medical symptom had it. He undoubtedly was mistaken, but the nature of his error is instructive. When scurvy was first identified 150 years ago, it was believed to possess four symptoms, all of which separately indicate other diseases. Rather than resembling meaningless figurae adding up to a sign, as Barthes claims, this may at first seem to parallel the linguistic compound, in which several
0

Shape

Colour
yellow (in Sweden), white (in most other countries) with red border lines

Import

s [l t] l [n] nlzu

warning

yellow with red border blue (many shades), green, yellow, etc.

prohibition decree, command

Fig. 6. The organization of traffic signs - not a good exemple of either first or second articulation.

meaningful units combine into a higher-order entity. In fact, symptoms combine to diseases in a way which is different also from that of compound words, because one single symptom can already have the entire disease as its meaning, though given in a dubitative mode, whereas a part of a verbal compound corresponds merely to part of the meaning of the whole. Here the expression is probabilistically, rather than deterministically or conventionally, coupled to the content; the more expressions that are produced for the same content, the more firmly the content is established; and when all possible expressions for a particular content have been given, the complete series of expressions is indistinguishable from the content. In fact, the symptom simply repeats the logic of all ordinary perception: it adds its hypotheses to the information given, and must then confirm them, or let them be falsified. The symptom is a sign only because medical semiotics has objectified it; expression and content are in fact only provisionally differentiated. The picture would seem to function in the same way, only that its content is heterogeneous to its expression. This is possible, I contend, because the picture is an iconical sign.

Some simple systems of visual signification The interplay of categoricalness and density points to a way in which pictures are different from verbal signs. In fact, as against Ecos idea of there being a pictorial double articulation, it is instructive to ask why there can be no such thing. In order to do that, we shall first follow Prieto in characterizing the first and the second articulation; then we shall see how pictures differ from both. A semiotic system having only the first articulation is, according to Prieto (1966), a system the recurrent units of which are already signs, that is, units such that their separation from each other on one plane of the sign entails a parallel division on the other plane. This would be the division of a verbal statement into words, if the words could not be further divided into phonemes, phonological and semantic features. Is has sometimes been suggested that the traffic signs could be, at least partially, such a code having a visual expression plane (Fig. 6). This is, however, problematic: almost any shape may stand for the category of warning, prohibition and, in particular, command, but certain shapes


Fig. 7. Instructions for the cleaing of garments: a single abstract picture corresponds to a given meaning and is combined with other pictures to give compund meanings.

are preferred in each of these categories. An element of probability thus enters their composition, making them, in this respect, more closely akin to real pictorial signs. A much better example of a code having only the first articulation is constituted by a variety of the instructions for the treatment of garments, current in Sweden (fig.7). On the other hand, we will have a code possessing only the second articulation, to the extent that the recurrent units of the system are all of such a nature, that the division on one plane of the system is not paralleled on the other plane. An excellent example of this is the flag code used by sailors in the last century (fig.8); although the circle, the rectangle, and the triangle recur on the expression plane, this does not tell us anything about what units will appear on the content plane. Indeed, there may be recurrent units on the content plane also, but these do not imply, nor are they implied by, the presence of particular units on the expression plane. Put in another way, there are two

ways in which signs may be divided into smaller, recurrent, units: either in a way the result of which are still signs, having both expression and content; or in away in which those recurrent units which may be found are not signs any longer, but only appear on the level of expression, or on the level of content. Some semiotic systems may be divided in both ways. A case in point is of course verbal language, where the division of the statement first gives rise to smaller signs, i.e. words, and then to figurae, i.e. phonemes or graphemes, which only appear on the expression plane, or semantic features, which only appear on the content plane. Some systems only have the second articulation, that is, they can only be analysed into recurrent units that are not parallel on the levels of expression and content. We have seen that one case is the sailors flag code. In other cases, semiotic systems may only be analyzed into the first articulation, that is, the recurrent units that can be found in them only appear in parallel at the levels of expression and content. The


instructions for the treatment of garments are, as we have seen, of this kind. I have been unable to find any visual system having both articulations; in any case, double articulation, outside verbal language, is certainly found in the system of telephone numbers, where some combinations correspond to different countries, other to different regions in a country, and still others to the telephone station. This example is given by Prieto, as is another one involving hotels in which the first digit of the room number indicates the floor level. Eco certainly claimed in his early work that pictures were doubly articulated, and although he has now long since abandoned this conception, he has not given any arguments for doing so. This is why it is worthwhile understanding the reason because of which this conception cannot be true. Pictorial semiogenesis as resemanticization Two points must be made here. The first is that, in claiming triple articulation for the cinema, and thus assuming the double articulation of pictures, Eco actually confuses two meanings of meaning, the sign and the configuration. In the second place, there is nothing similar to figurae in pictures, for while all pictorial features are meaningless in isolation, they all start to carry meaning once they are seen as part

of the whole. According to Eco, the admittedly paradoxical concept of triple articulation supposes a code made up of figurae, which build up signs, but are not parts of their meanings; and whose signs, in turn, form some larger unit X, without being parts of its meaning. The iconic statement, equivalent to a photogram, a tall, blond gentleman wearing a bright suit forms, together with a number of other such iconic statements, the cinematographic scene in which the teacher is talking to the children in the classroom. In this argument, many confusions are contained; the essential one, however, is that in arguing for the existence of a second sign level, or, as we shall say, level of appresentation, Eco is really quoting evidence for there being another level of configuration: that is, not that the teacher in his classroom is something quite different from the man in the bright suit, but that he is something more.5 The general idea behind double articulation is that, at some point in the shift from one configurational level to another, there is also a change of appresentational level, or its first emergence. Indeed, there may be a secondary level of appresentation in the cinema, in the montage, as practised by Eisenstein: but then again, both artic And of course another intensional level, as we shall see in 4..

sl ln nls lns lss

We are starving to death Help, fire or leakage! Do you have telegrams or news for us? Yes No Fig. . Code possessing just the second articulation Sailors flag code during last century 

ulations are really of the first type, since the secondary meaning only emerges from a clear recognition of the meaning on the primary one, as is the case of all metaphorical meanings. The level of configuration at which appresentation occurs is movable in pictures, as we saw when considering Magrittes Le Viol (fig. 14 and 15 of Lecture 3) which may be seen both as a face and as a womans trunk; it is precisely because of this double, contradictory appresentation that it is instructive. Depending on the level at which the interpretation passes to the other plane of the sign, the same strokes and dots will be seen as breasts or eyes, as nose or navel, as a mouth or as female pubic hair. This is quite unlike what happens in language. Though phonemes and graphemes may be combined to different words, they do not carry any of the partial meanings of the resulting combinations. Like the phonemes /m/, //, and /n/, forming the word /mn/, the strokes and dots making up the picture of a man are in themselves meaningless even when considered in their particular spatial location; however after having been put together, the phonemes continue to be deprived of meaning as such, whereas the strokes and the dots begin to take on the aspects of different proper parts and attributes of the man they contribute to form. In lecture 3, the term resemanticization was used for describing the process by means of which the meaning of the whole is redistributed to its parts.6 All that has been said so far also applies to the supposedly most realistic
6 As pointed out in Lecture 3, this is not simply the hermeneutic circle, as has been claimed, but at least a very particular variety of it, which involves the level of expression as well as the content.

Fig. . The dancer becomes impossible to recoconize in this photograph when only a small (but essential) part is subtracted from the surface.

type of pictures, photographs, as is readily seen from this picture of a dancer (fig.9), where a part has been cut away.7 This example also shows that some parts of a picture carry more weight than others within the play of probabilities. However trivial this observation may seem, it epitomizes the peculiar character of pictorial semiosis, and so must be taken to underlay the numerous differences between pictorial and verbal meanings. Visuality as a Hjelmslevean form From one point of view, then, pictorial semiotics should be a part of visual semiotics if this is a meningful way of dividing up semiotics. There certainly are precedents for this division of the field of semiotics: Roman Jakobson (1964) has treated of the differences between visual and auditory signs, and Thomas Sebeok (1976) has divided up semiotics according to the sense modalities. Indeed, this conception is at the heart of Lessings seminal discussion
7 Photography is no doubt peculiar in other ways, for which cf. Sonesson 989b, and Lecture 9. 

of the differences between painting and literature. Kmmel (1969) is perhaps the real pioneer of this domain, but his book mostly reads as a catalogue of visually conveyed (sichtbare) phenomena. Preziosi (1983) has conceived of architecture as being a kind of visual semiosis, which he then opposes to linguistic meanings, identified with auditory semiosis, and Saint-Martin (1987) has been very explicit in choosing visuality as her domain. The title of Groupe :s (1992) latest book, Trait du signe visuel, and many of its headings and subheadings, suggest that it is concerned, not with pictures, but with something more general called visual semiosis, or visual meaning; in fact, however, if is almost only about pictures. Also Kress & van Leeuwen (1996) oppose linguistic structures to visual ones, but talks almost exclusively about pictures. In more recent years, a number of authors have claimed for themselves the amorphous field (or whatever they think it is) of visual culture or visual studies (cf. Mirzoeff 999; Howles 003; Bal 2003), struggling in vain to determine what it is all about. The case of visual culture is actually even more problematic than that of visual semiotics, both because visual things only have in common being visual (which is true of most things we are aware of in the human Lifeworld), but visual signs are at least also signs; and because the scholars involved in the study of visual culture always seem to argue that it is about more than mere visual things. The most paradoxical case is that of Mieke Bal (2003), who, directly after having pointed out some of the (most obvious) contradictions of the earlier authors, then goes on to

accept the label.8 On the other hand, from the point of view of Hjelmslevean semiotics, we would normally not expect visuality, being a mere substance or even matter, to determine any relevant categorisations of semiotic means. In their dictionary, Greimas & Courts actually claim that sense modalities, identified with the expression substance, are not pertinent for semiotics, and this is no doubt the reason for visuality being one of the many layers between the unique picture and signification per se being left out of consideration in the picture analyses offered by Greimas disciple Floch. As I have argued elsewhere (in Sonesson 1988; 1993a; 1995a, 1996a), this type of argument is based on a confusion between the terms substance and matire, as employed by Hjelmslev, and in their ordinary usage. Thus, the term matire, to Hjelmslev, is simply that which is unknowable, and, as a consequence, not susceptible of being analysed; that is, it is the residue of the analysis; and substance, which, in the earlier texts, is the term used for matire in the above-mentioned sense, stands, in the later works, for the combination of matire and form. Thus, substance, in the early works, and matter later, simply means that which is not pertinent relative to the other plane of the sign (see discussion in Sonesson 1989a,II.4. and 1988); it does not necessarily stand for matter in the sense of ordinary language, that is, the material of which something is made, or the sense modality. If the material or the
8 As both Mirzoeff (2003) and Bal (2003) point out, the problem is compounded by the ambiguities of the notion of culture. We will return to this problem in Lecture 8 when discussing the Tartu school of cultural semiotics. 

sense modality turns out to be relevant in relation to the other plane of signification, it becomes form (from Hjelmslevs standpoint, this is what happens in connotational language; cf. Lecture 5). In an early article, Groupe (1979) appeared to make precisely this error, when making allomateriality into one of the possible characterising traits of the collage; but this analysis has recently (1992:331ff) been partly rephrased as heterogeneity of textures, and although some residues of the earlier analysis lingers on (1992:333ff), this is not as disturbing as it once was, since the Hjelmslevean framework is now largely dispensed with. The psychology of perception certainly seems to suggest the existence of some common organisation that puts all or most visually conveyed meanings on a par. If, as we have argued, all signs must also be objects of perception, there is every reason to believe that the modality according to which they are perceived determines at least part of their nature. This is indeed the position taken by Groupe (1992:58f), who goes on to compare this conception to the one favoured by such linguists as Saussure, Martinet, and Bloomfield, according to which the vocal character of language is one of its defining characteristics. More to the point, they observe that the linearity of verbal language is a constraint imposed on linguistic form by the characteristics of the vocal channel by which it was once exclusively conveyed. That is, the qualities of the visual sense modality are of interest to semiotics, to the extent that they specify formal properties embodied in each system addressed to that particular sense. Hjelmslev (1954) does not reason differently when he posits different forms for written and spoken language. The example

of linearity, adduced by Groupe , is interesting in its own right, however, for about the only thing Saussure (1974:39) has to say about pictures, or, to be precise, paintings, is that they are multi-dimensional semiotical systems (une sme multi-spatiale), in opposition to verbal language, which has a unique spatial dimension, that of temporal extension, or linearity. Nevertheless, it is not certain that the visual mode, although imposed by the vehicle of communication, defines the most fundamental domain of which pictures form a part. There may be other, perhaps more important division blocks of semiosis, to which pictures and some other visual signs pertain, such as, for instance, that of iconicity. Indeed, I have suggested that pictorality must be considered a very particular variety of iconicity. To the extent that there is a legitimate domain of visual semiotics, furthermore, it should undoubtedly comprehend much more than pictures, buildings, and sculptures, which are the only visual signs discussed by Saint-Martin and Groupe . Curiously, in spite of the promise made in the introduction to ignore received categories such as art, Groupe (1992:12ff), just like SaintMartin (1987), would seem to be the victims of the sacred trinity of art history, painting (to which drawing, photography, and so on, are conveniently assimilated), sculpture, and architecture. As soon as we leave the traditional divisions of art history behind, this trichotomy turns out to have a very limited value. Instead, sculpture should be compared to semiotically similar objects like the tailors dummy, and the like. At one point, Groupe (1992:405f) actually mentions marionettes as being a kind of sculpture to which movement has


been added. But why not also add the ballet dancer, whose art is certainly visual? There are also significations that are only partly visual, such as those of theatre communication. Others might be considered not to have an intrinsically visual organisation, such as writing, the conformation of which depends in part on spoken language. But all kinds of gestures and bodily postures, objects, dummies, logotypes, clothing, and many other phenomena must be counted as visual signs and significations. In fact, even visual perception per se supposes a pick-up of meaning of sorts. Not only should we therefore have to go through the arduous task of determining the ways in which the various kinds of visual semiosis, beyond those of pictures, architecture, and sculpture, differ, but it also remains to be shown that they all have sufficient properties in common to be considered visual signs (or at least visual significations), in the sense of this property being relevant to their form. The dimensions of pictorial specificity In differentiating pictorial meaning from other meanings, we should be particularly interested in knowing, not only how they are distinguished from other kinds of visual signification, but also how they differ from other iconic signs, that its, from other signs motivated by similarity or identity (see Sonesson 1989a,II.2.2. and III.6.). Moreover, not all signs that are visual and iconic would ordinary be described as pictures. Something more would seem to be needed in order to characterise picturehood. Most semioticians, even those who have used the picture as the principal whipping boy of their critique of iconicity, such as Eco, have simply ignored the need for

further characterisation. In other cases, the peculiarities of the picture sign have been addressed in oblique ways only: by Peirce in terms of qualities and exhibitive import, and by Saussure with reference to spatial dimensions. Husserl, as we say in Lecture 3, describes pictorial consciousness as something that is perceptually imagined. The most radical stance has been taken by Nelson Goodman, who simply rejects the ordinary sense of picture, in order to introduce his own. It might be argued, however, that, as he becomes prescriptive rather than descriptive, Goodman ceases to be of interest to semiotics, whatever may be the value of his theory to philosophy. The image is one of the three subtypes of iconical signs mentioned by Peirce: the one in which the iconical relation is assured, not by relations, as in the diagram, nor by relations between relations, as in the metaphor, but by simple qualities. The opposition between image and diagram, in this sense, echoes Degrandos (800: I,153ff, 262ff, II:302ff) distinction between sensuous and logical analogy. It must be noted that the image so defined, is not necessarily a picture in the ordinary sense: it may be addressed to other senses than the visual one (onomatopoetic words might thus be described as acoustic images). In addition, there are, as we have seen, a number of reasons, stemming from semiotics and psychology alike (notably Gibsonian psychology), to think that pictures, in the ordinary sense, are not based on simple qualities, whatever that may mean; in fact, they must rather be Peircean diagrams or metaphors (although Peirce must have though otherwise), in the sense of supposing an identity between relations, and relations between relations, present on


the expression and content planes. Pictures do, however, convey an illusion of there being a similarity of qualities, which is not true of the diagrams. Therefore, the difference between qualities and relations may have less to do with the difference of iconic ground joining expression and content together than with the effect produced by the two types of iconic signs. As we noted above, Saussure (1974:39) observes that, whereas language is unidimensional, painting depends on a semiotic system deployed in multiple dimensions. This does not appear to be something peculiar to pictures, however, for clothing certainly supposes at least two combinatory dimensions (or syntagmatic axes) the slots of which are defined by the body parts and the layers of closeness to the body, respectively (cf. Sonesson 988; 1991; 1992e; 1993c); and, if suprasegmatic features are taken into account, even verbal language will have to be considered multidimensional (as claimed by Jakobson). The question is, therefore, to what extent multidimensionality is a relevant property of the pictorial expression plane, that is, a property of the pictorial matter which is also a property of the corresponding form. It certainly does not define the order in which units are put together, according to rules of ordering (i.e. syntagms), as is the case with clothing and language. Closeness to the body and body part location do determine together the positional meaning of a piece of clothing: on the contrary, in a picture, in the core sense of a sign depicting a real-world scene, things are not basically defined by their horizontal and vertical position. It is true, of course, that horizontal and vertical position, just as position with respect to the harmonic or disharmonic

diagonal and other spatial axes defined by Saint-Martins (1987) analytical scheme, may add shades of meaning, and even essential building blocks, to pictorial signification, but they are not the defining characteristics of pictorality, because they may distinguish many visual signs which are not pictures, not only in the sense of the core meaning, but even in an extended sense. Nevertheless, spatial dimensions may turn out to be important to the picture sign in more supple ways: the projection of the, ordinarily three-dimensional, content plane onto the twodimensionality of the expression plane is one of the spatial characteristics of picturehood. We can grasp the nature of the picture sign, only once we have discovered and characterised a number of neighbouring sign types, those, for instance, which are conveyed visually, like pictures, but differ in other respects; and those which are iconically grounded, as pictures most certainly are, but which are different as to the peculiar character of their iconicity, or in other ways, from the picture sign. It will then be seen that the particular character of pictorial iconicity, unlike most others, involves what Gibson and Husserl alike have termed indirect perception, and that it is to account for this peculiarity of pictures that a perceptual model urgently needs to be reconstructed. This issue will be addressed in the next section, in the form of a critique of Nelson Goodmans iconicity critique. Summary In discussing the notion of semiotic function, this time as defined, not by Piaget, but by Hjelmslev, we have discovered that what may still be generalised from linguistic


theory, is not the arbitrarity of the relation between the relata of the sign, nor the rules for combination and selection along different dimensional axes, but the principle of pertinence or relevance, more well-known in (structural) linguistics as the opposition between form and substance (or, more strictly, matter). Although pictures (unlike many other kinds of meaning) function like language in this respect, pertinence is regulated more by probability than discreetness, as epitomized in the symptom model. The way in which parts are related to wholes in pictures is quite different from both the first and the second linguistic articulation. This lack of discreetness, however, many not go as far as suggested by thinkers like Goodman, as we shall see in the following.

4.2. Beyond density and repleteness

For a long time, semioticians tried to demonstrate the existence of some kind of minimal unit of pictorial meaning, sometimes termed iconeme, which was supposed to have no meaning of its own, but to discriminate the meanings of larger wholes, just as phonemes do in relation to words or morphemes. As we have seen, Umberto Eco (1968), who was an early proponent of this conception, later retracted himself completely, arguing that there could be no distinctive features in pictures (1976); even more recently (1984), he has claimed that the very question lacks pertinence to semiotics. Yet, curiously, to the extent that he continues to defend his earlier theory of the triple articulation in the cinema, Eco is still committed to a feature theory of pictures. Goodmans position is more unambiguous. Although he would deny the relevance of the common-sense notion of pic-

ture, prototypical cases of pictures in the latter sense are among those signs he would qualify as being analogous, or semantically and syntactically dense and replete. Density is a property of sign systems the possession of which implies that no matter how close a division of the signs is made into smaller parts, it will always be possible to proceed with the division, introducing a third unit between each earlier couple of items, and so on indefinitely. Density is semantic when it applies to content units (to referents, in Goodmans nominalist terms), and syntactic as far at it involves the varieties of expression (Goodmans marks). A dense system is replete when its signs can be divided from many different, perhaps an infinite number, of viewpoints. Density and repleteness, in Goodmans view, apply to pictures both as carrier of reference and as exemplifications, that is, in terms more familiar to semioticians, as iconic (or, in our terms, pictorial) and plastic language respectively. As we will see, this means that pictures are semiotic atoms, in the original sense, i.e. not susceptible of being divided in any non-arbitrary way into smaller units. Both Goodman and Eco relied in their arguments on a comparison with verbal language, Eco originally to underline the parallel, and Goodman and the later Eco because they wanted to emphasise the difference. On both counts they were wrong, however, because they had erroneous ideas about ordinary language, as reconstructed in modern linguistics. The linguistic model regained: hyletic issues The central concept of Goodmans theory is the symbol system, which henceforth


Fig. 0, Visualization of Goodmans syntactic criteria as a double system of projections

will be termed sign system, and which we, stripping it as far as possible of Goodmans nominalist metaphysics, will conceive to be made up of an expression scheme and a content scheme. (cf. Goodman 968; 984. Elgin 1983, etc.).9 Sign systems differ, on this account, according to the way marks or inscriptions are assigned to the units of the expression scheme, called characters, and according to the way in which the schemes themselves are correlated. The latter preoccupation is of course semantic; that Goodman should consider the former problem to pertain to the domain of syntax is, on the other hand, curious, for rather than having to do, in Carnaps (1958:79) terms, with the ways expressions are constructed out of signs in determinate order, it involves the reduction of variants to invariants, i.e., of substance to form, characteristic of the linguistic domains of phonematics and graphematics, for which van Kesteren (1984: 54), in the somewhat different context of theatre semiotics, has

proposed the term hyletics.0 Goodman uses a particular variant of a sign system, a notational system, as a prototype, or idealtype, in relation to which other sign systems are defined as approximations. A notational system, simply put, is made up of a series of separate, discontinuous characters, which are correlated with a content scheme, equally segregated, in such a way that each character in the system isolates the object(s) to which it corresponds and, inversely, so that each object isolates the characters correlated with it. A musical score comes close to being a notational system, and verbal language approaches the ideal in some respects, whereas pictures are found at the other extreme. We will first have a look at what Goodman terms the syntactic aspects, and which should, more properly, be assigned to hyletics. A notational scheme is an expression scheme which is syntactically (or,
10 van Kesteren (1984: 54) opposes hyletics, as the study of form, to syntactics, as the study of structure (where I would have preferred to talk more generally about organization for which see below), as well as to sigmatics, semantics, and pragmatics (concerned, respectively, with content, reference and use. 0

9 For a more complete treatment of the issues involved, cf. Sonesson 989a,III...4-5. and III.6..

have at least one inscription in common. The real issue, however, hinges on the comparison to verbal language, which Goodman calls discursive language and considers to be possessed of an expression scheme that is disjoint and finitely differentiated. According to Goodman, the expression scheme is made up of the characters integrating the sign system, with no regard to the corresponding content scheme. Formally, a character is defined by the class of all inscriptions or utterances corresponding to it: thus, a, A, and a , are all a-inscriptions. Similarity, in Goodmans view, cannot be used to relate all inscriptions to their corresponding character (cf. Goodman 1970). Not even topological similarity is required: of two a-inscriptions, one may be closed and the other open (see Fig. 11a); and there may be topological similarity between two inscriptions that do not belong to the same character (the middle one and the rightmost in Fig. 11b). The only way of defining a character must therefore be by enumeration of its marks. In practice, Goodman always starts from a given alphabet, but even so, the study of disjointness and finite differentiation supposes the inscriptions to be listed. This is certainly a language of Goodmans own invention. The description, already at this point, is incompatible with the findings of modern linguistics. In the case of ordinary language, no listing of its inscriptions is feasible, since the latter must be infinite in number (because a given phoneme or grapheme will be realised differently by different persons, and by the same person on different occasions). This is connected with another finding, according to which the characters forming the expression plane cannot be conceived independ

Fig. . Inscriptions: a-d from Goodman, e from Naus & Shillman (cf. text) more correctly, hyletically) disjoint, i.e. none of its marks belongs to more than one character; and finitely differentiated, i.e. it is theoretically possible, in the case of every two characters of the scheme, and of every one of its marks which does not belong to both characters, to determine to which one it does not belong (cf. Fig. 10.). For a notational scheme to become a notational system it must be adequately correlated with a content scheme, which is a semantic question, to which we will turn below. Our familiar alphabet, the Arabic numerals, the binary, telegraphic, and basic musical notations meet the syntactic (i.e. hyletic) requirements. Pictures, on the other hand, are syntactically (that is, hyletically) nondisjoint and dense throughout, which means that a single mark may belong to several characters, and that between any two characters of the system a third can be introduced. It should be noted that the requirements for a notational scheme are mutually independent: a scheme in which every difference of the length of straight marks, however small, counts as a difference of character, is disjoint but not finitely differentiated; and there will be finite differentiation but not disjointness in every scheme in which all marks are conspicuously different, but some two characters

ently of whatever serves a the content, the former being allo-functionally defined in relation to the latter (in the sense of picking out form from mere substance or matter). In the case of actual language, in the third place, the alphabet, i.e. the repertory of elementary signs, is not given beforehand, but must be recovered from the texts, which means that segmentation must precede, or coincide with, the classification of inscriptions into real characters. If we pay attention to these observations, we will find that verbal language (as opposed to the logicians constructed language) is not so different from pictures after all: indeed, on at least some interpretations of the notion of inscription, verbal language is also syntactically non-disjoint. In order to accomplish the listing of the inscriptions, Goodman would first have to reduce the tokens to types; but it is exactly for this purpose he needs the listing. The prospects for listing the features of which phonemes, graphemes, and so on, are made up may seem more promising: but even these, as conceived in phonology (for instance, traditionally, as voiced vs. mute, or more abstractly, as diffuse vs. compact, etc.) or in graphemics (for instance, the circle, strokes and points imagined by Mounin 1970) are types, not tokens. What is needed is a principle of relevance, relying, on the one hand, on oppositions inside the expression scheme, and, on the other hand, on relation to the content scheme. In Goodmans example (reproduced as Fig.11b), the B and the O with an intermediary case, are really two prototypes, linked by a third case close to the limit of both. In the case of the pairs V/Y, C/F, and U/
 And thus also, as many linguistics and philosophers would argue today, not fully compositional. Cf. note  above!

H, Naus & Shillman (1976) demonstrated the existence of a trajectory passing over a determined point of transition (Fig. 11e). Over and above that, however, the differences that matter are those that make a difference to meaning: a and a are both a-inscriptions, because the exchange of one for the other in a word does not change the content of the word. Goodman all the time reasons as if two inscriptions could be separated from each other, before being ascribed to different characters, but this is clearly impossible in the case of hand-writing, and even more so as far as spoken language is concerned: there is simply no point where, from the physical point of view, the realisation of one phoneme comes to a close without the realisation of the next having already begun. This means that the string of inscriptions cannot be divided up into parts, before each inscription has been attributed to a type, and they can only be associated with expression types once a correlation with the content plane has been established. Thus, both the idea of listing an alphabet beforehand and the resolution to treat expression separately are plainly wrongheaded. There is a sense, we noted above, in which ordinary language is clearly nondisjoint. Consider Goodmans (1968:137f) claim that two inscriptions which look alike may, because of the context, be determined to be at one time a d, at another an a (Fig. 11c), but that, when the same occurrence has to be taken in two ways, in must belong to a third character (Fig 11d). In fact, of course, as long as the notion of context has not been specified, it is not obvious why it should not be possible to dissolve all cases of nondisjointness, not only that


of the letter cross, where the time-slice, in terms of reading time, is unequivocal, but also the parallel case of pictures. It is never clear, to be sure, whether Goodman intends inscriptions to be unique timespaced occurrences, or some kind of subcategories. However, in Elgins (1983:98f) authorised version of Goodmans theory, the reasoning is clearly in terms of subcategories, when a system in which is substitutable for A, and for H, without A being substitutable for H, is said to be non-disjoint, for in the opposite case, there would be no possibility of substitution. If so, however, the trivial fact of neutralisation, well known to linguistics, will make all ordinary language syntactically non-disjoint. A rather familiar example of neutralization are the sounds /d/ and /t/, which both become /t/ at the end of German words, but not in other positions (e.g. Rad and Rat). Nevertheless, it will turn out to be more rewarding to take the clue from Goodmans letter cross and consider inscriptions to be time-space slices. Each segment of spoken language, which manifests a phoneme, necessarily manifests at the same time some part of the intonation contour, along with a number of expressive values and other paralinguistic features. Similar observations are valid for handwriting, and even for printed letters. But this means that ordinary language is non-disjoint, each occurrence being a member of a number of characters. No notion of context will save language from non-disjointness in this case, because it is the same time-slice that must be multiply classified. There is, however, a fairly traditional way of resolving this problem, which consists in saying that the types to which the same occurrence

belongs are members of different sign systems or, in other cases, of different parts of the same sign system: thus, in the case of spoken language, paralinguistic features would normally be considered to form a system independent of the phonemes, whereas intonation is considered a part of language proper (cf. Trubetzkoy 1939). In the latter case, then, inscriptions would be nondisjoint inside language as a whole, but disjoint, once an adequate partition of language has been made. This really brings us to the heart of the matter: we have been studying the ordinary act of speech long enough to know, more or less, what an adequate ascription of its properties to sign systems and their parts should be like, but since we have only recently taken up a serious study of pictures, we are hardly in a position to know, whether it derives from a single system which is non-disjoint, or whether instead it realises types from several sign systems, and perhaps from more sign systems, and system parts, than verbal language. Goodman never encounters this problem, because he does not stop to consider syntax, in the received sense: the ordering of units which supposes segmentation, which, in turn, cannot be realised without the ascription of units to sign systems. This is why he discovers no system in pictures.3 The linguistic model regained: semantic issues A sign system the expression scheme of
 And this is of course the basis of the distinction between denotational and connotational language according to Hjelmslev. Cf. Lecture 5. 3 Further issues pertaining to Goodmans identification of the properties manifested by the picture as an exemplificational sign, and as a vehicle of reference, as well as the nature of diagrams, are considered in Sonesson 989a:33ff. 

Fig. . Results of density which are not analogy (in any interesting sense)

which is notational will also be a notational system, if the way in which the expression is related to the content answers to three requirements: it must be unambiguous, semantically disjoint, and finitely differentiated from a semantic point of view. Thus, no expression is allowed to stand for more than one (type of?) referent; no referent must be apt to be described by different labels in the system, most notably those which partly overlap, such as phenomenon, animate being, mammal, gorilla, heavier than an acorn, etc.; and it should always be possible to decide which of several particular labels, such as purple, dark purple or halfway between dark purple and deep purple, apply in each particular case. Not only pictures but also verbal language turn out to fail all these requirements. Both are semantically nondisjoint, dense throughout, and ambiguous. Curiously then, Goodmans theory makes pictures and verbal language appear to be similar semantically that is, exactly where our intuitive notions about these sign systems makes us anticipate the fundamental difference.4 Or at least it will seem so, once we re4 For a thorough analysis of these requirements, as well as of Goodmans failing to reconstruct analogy, cf. Sonesson 989a,III..5.

alise what Goodmans criteria involve. Given a referent, it is not possible, in the case of nondisjointness and nonfinite differentiation, to predict which content, forming part of the sign system, will be applied to it, first, because there is a choice between different ways of construing the referent, and, in the second place, because the level of delicacy of the description is optional. It is true, of course, that, when faced with a gorilla, we are free to describe it from different points of view, and to take more or less details into account, no matter whether we are making a drawing or writing a verbal report. But the interesting thing certainly seems to be the different constraints that are imposed on our options in the two cases. Instead of attending to this question, Goodman is content to reduce the difference between pictures and verbal language to what he terms syntax, i.e. the different relations obtaining between tokens and types. Actually, Goodman does intend to contribute to the description of these constraints, for, by implication, he acknowledges their existence in verbal language, but not in pictures. Taken together, semantic and syntactic density are supposed to give rise to analogy, which is taken in


the familiar sense in which it is opposed to the digital, that is, as something which varies continuously. In fact, Goodman fails to reconstruct this familiar sense of analogy. The syntactic and semantic requirements really amounts to a double system of projections, from the token to the type, and from the referents to the contents (cf. Fig. 12.). There must, however, be an infinite number of ways of projecting two (potentially) infinite series, such as the syntactic and the semantic schemes, onto each other, giving rise to many syntactically and semantically dense systems which combine the drawbacks of the systems being dense in only one way, and which appear alongside with those systems which are analogue in the common sense of the term. It is not enough for both expression and content to be continuous, but it must be possible to map the two continuous series onto each other. Thus, analogy has not been reconstructed, but is simply introduced from the outside. The point is not merely that analogy, in the familiar sense, is not the same thing as semantic and syntactic density (with or without the corresponding repleteness, to which we will turn below): the mapping of one continuous series onto another is not really the essence of picturehood, without which there would be no way of determining what the picture is a picture-of. Indeed, we know from the findings of psychologists such as Gibson, Hochberg, and Kennedy,

that the referents of pictures cannot simply be appointed, as Goodman would claim. Mount Fujiyama and the electrocardiogram A black wiggly line on a white background may, in Goodmans (1968:229ff) view, either be a Hokusai drawing of Mt Fujiyama, or a momentary electrocardiogram: in the first case, it would be a picture in the ordinary sense, and in the second a diagram, again in the sense of ordinary language (i.e. not, specifically, in Peirces wider sense). According to Goodman, both the diagram and the picture are members of dense systems, in the sense discussed above, but the latter, in addition, is also replete. This is to say that, whereas, in a diagram, only changes along one dimension, i.e. the spatial co-ordinates, are relevant, in a picture many more types of variation may be so, e.g. the relative thickness of the line, its colour and intensity, its absolute size, and so on. The picture is like a multidimensional diagram! What Goodman says about diagrams appears to be true, to some extent, about the electrocardiogram, but it certainly does not apply to all diagrams (cf. Fig. 13.). Thus, if the black wiggly line represents the amount of cars sold during different years, only the fragment of the line positioned exactly above the indication of the year is relevant, which means that the system is not dense. In fact, even the density of the electrocardiogram is limited by the

Fig. . Mount Fujiyama as different kinds of diagrams 

technical capacity of the measuring device. On the other hand, it would seem that some diagrams are replete, in Goodmans sense of the word: according to Bertin (1967), the relevant unit of the diagram is susceptible of variation along eight dimensions: location more or less up or down, location more or less to the left or the right, shape, size, colour, clarity, coarseness, and orientation. Sometimes, it is true, it seems that Goodman (1984:57f) would really want us to take repleteness to mean, not that comparatively more dimensions are relevant, but that any variation will turn out to constitute a relevant dimension. But in this sense, not even pictures can be replete: thus, for instance, the thickness of the line with which Hokusai represents the contour of mount Fujiyama in undoubtedly irrelevant for its capacity to refer to the mountain, as is the whole inner contour of the line; they convey no meaning in the drawing considered as a picture of Mount Fujiyama.5 It will be noted that the diagram brings its key with it: the terms and numbers inscribed along its vertical and horizontal axes serves to separate out the form from the substance, and to assign a meaning to the units so delimited. Thus, the meaning of the diagram is really appointed, as is that of a droodle, for instance, Carracis key, or Arnheims droodle, which may be seen as an olive dropping into a Martini glass, or a Close-up of a girl in scanty bathing suit (Fig. 9b of Lecture 3). The characteristic property of a droodle is that
5 These properties may of course be relevant when the picture is considered as exemplifying aesthetic properties, i.e. as a plastic sign, in the sense of Groupe , which shows that Goodman (1968:233) and Elgin (1983:121f) are wrong in thinking that exemplification and reference give rise to the same segmentation. Cf. Sonesson 989a:33f.

Fig. a. Kennedys house: features ) occluding bound with background air; ) occluding bound with background surface; ) occluding edge with background air; ) occluding edge with background surface; 5) concave corner; 6) convex corner; ) crack. b: the common denominator of Chaplin (c) and Hitler (d).

the sign function is sparked off, and meanings distributed to the parts, only once a verbal label has been attributed to the figure. In the case of a proper picture, on the other hand, we are immediately able to see into the expression plane, and project as its content, some part of the perceptual world, without receiving any further indication on how it should be taken. Thus, the droodle is like a picture, in that it refers to the perceptual world, but it is similar to the diagram, in that it requires a label. Goodmans black wiggly line, when separated from its contexts, is in fact either a droodle or an unlabelled diagram. These observations take on a new importance, when we consider Goodmans suggestion that the picture is comparable, and in fact not coherently to be distinguished from, an ungraduated thermometer. Actually, like an unlabelled diagram, an ungraduated thermometer is a curious, and largely useless, object: since no degrees are inscribed on it, we would be at a loss to know whether this particular ther

mometer is of the kind used for taking the temperature, baking a cake, or heating the oil. Apart from the physiognomic qualities of up and down, and perhaps more and less, it would carry no meaning. Yet no picture is meaningless in this way (cf. Sonesson 1989a,239ff, 324ff). The diagram only conveys the variant information, the invariant one being reserved for the labels, as Bertin (1967) observes, and this also applies to droodles; but in pictures, both the object and its particular properties are given together. The real issue may thus be to account for the fact that some marks on paper are pictures, while other are only droodles. Contrary to Goodman and Eco, those psychologists who have been concerned with picture perception emphasise the pervasiveness of features determining the identity of that which is perceived. For instance, Kennedy would show, using a familiar landscape scene (Fig. 14a), that certain constellations of lines meeting at particular angles, stand for bounds, edges, surfaces, corners and cracks of the threedimensional perceptual world. Features like those discussed by Kennedy have been implemented in computer programs, and work out quite nicely. It does not follow, however, that this is really the essence of picture perception. Hochberg observes that there are, in addition to the spatial layout features, so-called canonical features, which account for the minimal opposition between Hitler and Chaplin in some caricatures (Fig. 14b). These caricatures, it will be noted, are not droodles: they are in need of no labels. In fact, canonical features may turn out to be much less marginal than Kennedy, and even Hochberg, imply. Indeed, even in Kennedys picture, the sea

and the clouds are not seen because of any layout features, but are clearly conveyed simply by means of prototypical forms. In fact, even the drawing of the house, with the same spatial configuration, could represent any number of other cubic objects, if it were not for its prototypical house features. It may be suggested, then, that canonical features are really pervasive in pictorial perception. Just as the meaning of a sentence may be grasped directly, independently of the details of syntax, there may be a direct perception of gist in the picture, in some cases eventually supplemented by the registration of spatial layout. What Hermern (1983:101) wanted us to see as a jar (Cf. Fig 9 of Lecture 3) is much more naturally seen as a face. There is nothing accidental, I submit, to those limitations of human imagination invoked by Hermern: they are imposed by the Lifeworld hierarchy of prototypical things. Indeed, there must be an infinity of objects whose light pattern, in a static view, fit much better to the square pattern on Fig.9b than a face, and yet we cannot help seeing it. And although it is possible to impose the jar reading suggested by Hermern on Fig. 9b it is only there in the droodle fashion, once a key has been given, and it is all the time being disturbed, and in fact overridden, by the more natural face interpretation. It seems, then, that we come to the task of picture interpretation equipped with certain expectancies to encounter those objects which are normally close at hand in our everyday Lifeworld, such as faces and human bodies, in addition to such objects which, like Chaplin and Hitler, which are introduced to us by our particular culture, with its peculiar his

torical experience. Most or all objects and scenes may certainly be depicted, but if they rank below the apex of the hierarchy built of our Lifeworld expectancies, many more details are necessary, for the object or scene to be recognisable. Exhibitive import and pictorial semiosis In spite of the existence of pictorial features, Goodmans observations on density are not entirely off the mark. Indeed, once we have determined whether a particular scribble realises the category of a tree or of a womans profile (e.g. Fig. 12a-b of Lecture 3), the drawing will tell us a lot about the particular conformation of the crown, or the nose, the hair-cut, and so on. Similarly, once we realise that the black wiggly line does indeed represent Mount Fuji, its minute details and variations will inform us about the particular shape of this mountain ridge. But not indefinitely: only up to a point set by the principle of pertinence embodied in the pictorial medium. The term exhibitive import was introduced by Greenlee as a label for a peculiar property of iconical signs observed by Peirce: that truths concerning their object not determining the construction of the sign can be discovered by direct observation of the sign vehicle: thus, for instance, a map can be drawn by means of two photographs. Greenlee erroneously proceeds to dispute the specificity of this great distinguishing property claiming that it is found also in novels, in respect of human situations, and this entirely independent of the perceptual qualities of the vehicle, in contrast to the imports of a lyric poem (1973: 80). If we attend to Peirces examples (which include algebraic formulae), we will realise that exhibitive import does

Fig. . Mother and Child by Paul Klee.

depend on the perceptual properties of the vehicle: it is that which is shown, in addition to being signified. This, in turn, may have something to do with the fact that pictures simulate perceptual experience, and thus are able to build on the interpretative schemes used in the on-going practice of everyday life. This is reminiscent of Wollheims notion of seeing-in: they involves a peculiar visual experience in which a state of affairs can be seen in a particular, with attention being distributed evenly between the particular, i.e. the expression, and the state of affairs, i.e. the content or referent. And it reminds us of the Husserls pictorial consciousness. Although Peirce would have ascribed exhibitive import (if he had used that term) also to algebraic formulae, it should be possible to characterise a specifically pictorial exhibitive content: it permits us to see in a drawing of a human face (e.g. Fig 12c of Lecture 3) those facial traits, such as the forehead, the cheeks, etc., which are rendered by blank spaces between the lines and surfaces; and those features, e.g. the ears, which are not even marked in the


drawing. In the second place, those traits which have no expression proper may be at least roughly located in relation to the rest of the facial traits: that is, the ears which are not drawn can be shown to be lacking at a particular place. This is of course another aspects of what has been called resemanticisation beforehand. It will be remembered that in Gauthiers (1976) analysis of Schultzs comic strip Peanuts, the same line may participate in conveying the contents eye, openness of eye, profile view, smile, happiness, and so on (Fig. 5). In Goodmans parlance, all these would seem to be examples of a mark being ascribed to several different characters. Yet, as we saw, many different things are confounded in this example. The line that makes up part of the configuration that means eye may possibly participate in the expression of the meanings smile and happiness, in the same way that the real eye in the real world participates in the expression of these real-world meanings. Thus, like in a compound word, the eye, together with other signs, builds up a larger unit, but unlike the case of verbal language, the eye retains its own meaning. In other respects, however, the Peanuts code cumulates meanings that are separate in the perceptual world; thus, there is no separate expression for parts of the eye. There are limits to such variations, however, because some features clearly have to go together, if the result is to be a prototypical picture: a visual display which has separate expressions for the eye as such, its state of openness, and the point of view from which the observer confronts it, could be some kind of notation (such as those used by Birdwhistell and Kendon to describe occurrences of gestural commu-

nication), but it would no longer be a picture, simply because such a dissociation would not respect the conditions of Lifeworld experience. Simple modifications of such traits, however, may be the signs of a pictorial rhetoric, as a coalescence of characters in one mark that does not coincide with those of visual reality. Thus, in Klees Mother and child (Fig. 15), in which a continuous contour line enclose both part of the mothers face and that of the child, the process of resemanticisation, which is characteristic of pictures, concurrently starts out from two centres of attention, corresponding to the sub-whole of the womans face, and the sub-whole of the child, and then spread their associated features outwards until these enter into conflict with each other, giving rise to a zone of indecision, which may only be resolved in the droodle mode, alternatively according to one or the other interpretation. This is similar to what happens with the devils turning fork and other impossible figures, although in this case the feature invariants that enter into conflict do not concern the spatial layout of the objects involved (see Sonesson 1989a,II.3.4.). Summary The problem with Goodmans opposition between language and pictures is twofold: first, the notion of language which he avails himself of involves a logicians reconstructed language, not a natural language as spoken by ordinary people; and, second, he so completely deprives the picture of organization that its ability to carry meaning becomes mysterious. Real language (including graphic marks, which are Goodmans example) is clearly hyletically (syntactically) non-disjoint in Good

mans sense, if we take it to be governed by one single system. Since it is also, on Goodmans own accord, semantically nondisjoint, dense, and ambiguous it does not seem to be very different from what Goodman claims about pictures. In fact, there is something strange to the suggestion that pictures and language are most similar semantically, which is where we would expect most of the difference. However, as is seen in the discussion of Mount Fujiyama and the diagram, syntactic and semantic density cannot explain analogy, contrary what Goodman intimates, and pictures cannot really be totally dense and replete, in case of which they would not signify anything. In fact, there must be one system, that of pictorality, which is at least to some degree disjoint.

It is quite another issue to determine whether language and pictures are in some ways similar to each other, i.e. whether their respective sign structures mirror one other. Paradoxically, Umberto Ecos wellknown refutation of the existence of iconic signs (signs based on similarity) itself was based on a postulated similarity (i.e. iconicity) between language and picture: just like verbal signs, pictorial signs were, in Ecos original view, conventional and based on features which had no meaning in themselves. In defence of the linguistic model fallacy, malgr tout According to one simplistic view, as we have seen, semiotics really consists of two traditions, which run parallel to each other: the Peircean school, which starts out from a general, philosophically grounded, theory; and the Saussurean one, which tends to construe all semiotic phenomena according to the model of verbal language, particularly, as the latter was conceived by the Structuralist schools in linguistics. In fact, many followers of Saussure, such as the Prague school, and the tradition from Buyssens to Prieto, make very few and only very abstract analogies to verbal language. And those who explicitly claimed to apply the Saussurean language model to all phenomena, the French structuralists, were very rapidly disenchanted with the linguistic model, and repudiated it as rashly as they had once embraced it. Indeed, it is seldom appreciated that the outright rejection of the linguistic model must be at least as naive, and as epistemologically unsound, as its unqualified acceptance; for, the use of one science as a metaphor for another involves such a long
0

4.3. Similarity and contiguity of verbal and visual semiosis

There are numerous ways in which language and pictures, verbal and pictorial texts, may be related. One question which may be posed concerns their modes of coexistence: the ways in which a linguistic item may be contiguous too, or form part of, a pictorial representation, or the reverse, giving rise to particular forms of semiotic interaction, which involve indexicality. The issue of verbal/visual interactions can also, and in fact frequently has been, formulated, in terms of rules and principles rather the singular token: whether our interpretation of pictures is always mediated by our linguistic competence, or the reverse. The first thesis was defended by the French structuralist; the opposite conception, which is actually somewhat more reasonable, has so far, I believe, never been formulated.

series of choices and comparisons, on different levels of abstraction and analysis, that there can be no rational way of undoing them all at one stroke (cf. Sonesson 1989a,I.1.2. and 1992b). The validity of the linguistic analogy must be appreciated separately for different levels of abstraction pertaining to the object of study, and when it comes to the nature of semiotics as a science we are faced with a quite different question. If we take semiotics to be something more than just a cover term for a series of traditional endeavours, such as art history and the history of literature, then (as I suggested in Lecture 1) we can reasonably claim that, like linguistics, it must be a nomothetic science, which, just like linguistics, but contrary to the natural sciences and the social sciences, is concerned with qualities, rather than quantities. Thus, semiotics should be concerned to ascertain general laws and regularities, but it should do so in terms of meaningful categories, not in statistical form (e.g. what is true of all pictures, or all kinds of music, etc., and of some particular sub-categories of these, not of individual objects). It is in this sense that we may claim that, even in the study of pictures, there is a possibility of quadrating the hermeneutic circle: of finding regularities, categories which are repeated from one instance to another, rules, usually not of combination, as in the case of linguistic syntax, but often of transformation, and of abstraction, which may serve to reconstruct the individual task of interpretation. Only in this way does it make sense to talk about pictorial texts (gestural texts, spatial texts, and so on), alongside the familiar verbal ones. To admit such a parallel to linguistics

is not properly speaking to embrace the linguistic model, which is ordinarily understood to consist in transposing concepts and terms derived form the (structural) study of language to the analysis of other phenomena. For the last 0-5 years, numerous students of other semiotic domains have marked their distance to the linguistic model, but this has often meant a return to a pre-structuralist (sometimes, paradoxically, termed poststructuralist), and even pre-theoretical, stage of reflection, as is the case of the late Barthes, and in part of the work of Damisch, Marin, Schefer, and Lyotard. Even during the heyday of French structuralism, Hubert Damisch (1979) quoted numerous reasons for thinking that the picture was quite differently organised from verbal language, and Christian Metz (1968) argued against positing something like a language system behind the meaning production of the cinema. In this feud of language, as Pavel (1989) has called it, both structuralists and their critics may well be accused of ignoring the stakes involved; but it should be important to distinguish, more clearly than Pavel does, their separate responsibilities. Most one-time structuralists abandoned the linguistic model like a whim of fashion, just as naively as they had once adopted it: the exact way in which the linguistic analogy did not fit in with the nature of music, pictures, or whatever, was never spelled out. It is true that Metz and Damisch tried to adduce reasons for rejecting the model: however, it is clearly the intuitive, pre-theoretical notions of film and picture, respectively, which are here compared to the concept of language, as reconstructed by linguistic theory, in fact, by a particular linguistic theory, that of the Saussure/Hjelmslev tra

dition. But the comparison of a folk notion and a concept forming part of a scientific theory can never yield any valid result (cf. Sonesson 1989,I.1.2.). This state of the case explains that, more recently, postmodernist critics like Boit (1992) and Krauss (1992) have had no difficulty in resurrecting the linguistic analogy, albeit only in the particular case of Cubist painting. No matter which may be the deformations that the linguistic model imposes on other kinds of signification, they stem less from the linguistic terms as such, than from the distortions which the latter have suffered at the hands of scholars not versed in linguistics. With few exceptions, linguists cannot legitimately be accused of having imposed their model on other brands of meaning. On the contrary, they should be held responsible for having treated the analysis of all non-linguistic significations as something spurious, either denying the interest of their study altogether, or citing these meaning types only in the guise of elementary examples at the beginning of introductory courses to linguistics then to abandon them for ever. As we have seen above (in Lecture 2), this parti pris of the linguists merits at least as much censure as that of the structuralist laymen, because the former are guilty of impeding the introduction of a truly comparative, developmental and evolutionary perspective on semiotic resources. Structuralism without a structure There is every reason to doubt that, in a deeper sense, there has even been a linguistic model in semiotics. Barthes, LviStrauss, Greimas, and many of their followers did certainly have recourse, in their attempts to analyse non-linguistic

objects, to a number of terms taken over from Saussure, Hjelmslev, and Jakobson. When closely scrutinised, these analyses generally turns out to be concerned with very abstract notions like connectedness (in the guise of syntagms, syntax, and metonymies) and categorical identity (termed paradigms and metaphors). Thus the same thing could have been said without referring to linguistic terms. In the second place, most semioticians are really too ignorant of the concepts of linguistics to be able to apply its model (cf. Pavel 1989). Even connotational language, as it is misinterpreted by Barthes, is introduced as a means of establishing complex networks of meaning, not to do the business intended by Hjelmslev (see Sonesson 1989a,II.1. and Lecture 5). The notion of sign itself is never highlighted, although the terms expression and content appear abundantly the resulting confusion of which we have taken stock of in Lecture . The term, if not the concept, of structure is essential to Lvi-Strauss (1975) work, but the actual procedure really involves putting the concept of structure defined by linguistic structuralism on its head. Paradoxically, the French structuralists, Lvi-Strauss disciples, never understood the concept of structure. This fact may be gathered from Lvi-Strauss (1975) analysis of a couple of ritual masks stemming from the American Northwest Coast, which, to me, has the advantage of involving visual artefacts (cf. Fig. 6 and 7; Lvi-Strauss 975 and Sonesson 1989a; 1992a, c). From the observation of the properties of the first mask, the Swaihw mask, Lvi-Strauss claims to derive not only the existence, but also the relevant properties, of another one, the so

Fig. . The Swaihw mask, analysed by LviStrauss

called Dzonokwa mask. But in linguistic structuralism, one item is not derived from another one, but the properties of several items, known to exist within the system, are re-described from the point of view of their mutual opposition within a whole.16 According to Lvi-Strauss (975:, I: 32ff) description, the Swaihw mask has a wide-open mouth, its lower jaw is dropping with an enormous tongue lolling out, the eyes are protuberant, the predominant colour is white, and its decoration consists in bird feathers. This mask only acquires
6 Structure, in this sense, should not be confused with Gestalt or configuration: in the case of the latter, the parts get lost in the greater whole, but in the former, they stand out because of the relation within the whole. Cf. 4.4 below!

meaning in relation to another one, LviStrauss assures us, and the properties of the missing mask can be deduced from those of the one observed: it will be black, and instead of feathers it will have hair; its eyes should be sunken, and the mouth must have a shape which does not permit the tongue to show (p. 102f). It so happens that this mask can be found among the members of a neighbouring tribe: it is the Dzonokwa mask (cf. Fig. 18). Structural thinking consists in inquiring about the properties of A, drawing, from the known fact of the inclusion of A in the couple A vs. B, the conclusion that the relevant property is x, one of the properties of A, which is the only property


Fig. . The Dzonokwa mask, discovered by Lvi-Strauss starting out from the Swaihw mask.

which is not shared by B. Lvi-Strauss, however, starts out from the observation that A has the properties x, y, z, positing that there must somewhere be an item having the opposite properties, from which follows, according to his thinking, the existence of B within the couple A vs. B. The inversion of the structural operation is not necessarily unreasonable: but it would only be possible within a domain which is already given (that is, that structure in which A is opposed to B). The existence of such a domain, however, is what Lvi-Strauss believes he is proving. Thus, the result of Lvi-Strauss analysis is precisely the kind of description that would have been censored by the lin-

guistic structuralist: one which is not true to the internal workings of the sign system. Applying Lvi-Strauss reasoning to language, we would be able to demonstrate, much to the surprise of all Japanese, that there is a distinction between the sounds r and l in their language, or that there exists an opposition between r and rr in English, just as in Spanish. Or, following more strictly the parallel drawn from LviStrauss analysis, we would have to say that there must be a non-aspirated p in English, only because there is an aspirated one in other languages. This is exactly the kind of thinking which was anathema to the linguistic structuralists. The real interests of many structuralist analyses may actually


Swaihw dominant colour white decorated with feathers dropping jaw open mouth with tongue lolling out

Dzonokwa dominant colour black decorated with hair closed jaw mouth shape which impedes the tongue from showing eyes protuberant vs. sunken eyes (actually holes) Convexity vs. Concavity

vs. vs. vs. vs. vs.

Features not mentioned by Lvi-Strauss no ears vs. protruding ears crooked nose vs. straight nose round eyes vs. oval (normal) eyes Plastic differences all dominant axes exaggerated vs. normal facial proportions symmetrisation geometrisation vs. realistic representation all facial elements rendered vs. the face rendered only as surface (holes for eyes, mouth, etc.) imitated material vs. partly real material (hair) Order (culture) vs. Disorder (nature)
Fig. . A comparison between the features of the two masks (as suggested by Lvi-Strauss, with supplements from Sonesson 1989a) be discovered once we realise that they are not at all talking about the same phenomena as in linguistics, which is for instance true of the notion of opposition, as we will see below (cf. Sonesson 1989a,I.1.3/5). The pan-linguisticism characteristic of French structuralism appears to be of at least two kinds. While the Greimas school would seem to adopt, to some extent, the linguistic model, because all meaning is considered to be similar to the linguistic kind, or to admit of the same treatment, that is, for ontological reasons, the justifications Barthes appears to have for the same choice could rather be termed epistemological, and they are basically opposed to those of the Greimas school. Barthes gives the impression of thinking that semiotical systems other than verbal language are inaccessible to analysis, and thus can only be attained indirectly, through the way language refers to them and describes them. Probably, Barthes really holds both positions, in different articles, and the same could be true of the Tartu school with respect to the curious notion of secondary model-building systems7. To reject ontological pan-linguisticism, we will have to show that other sign
17 These and other notions of the Tartu school will be further discussed in Lecture 8. 

types or other meanings are, in some essential respects, fundamentally different from verbal language. To reject epistemological pan-linguisticism, on the other hand, it is necessary to demonstrate that there are meanings that are accessible to us independently of verbal language, for instance before it is even acquired (cf. Sonesson 1993a). There are, however, some levels on which the linguistic model may actually be adequate. It may apply on very high levels of generality. For instance, if, following Halliday, we distinguish three functions of communication, the ideational, the interpersonal, and the textual, it is reasonable to claim that they will also be found in visual semiotics (cf. OToole 1994). And, even more generally (as I argued in Lecture 2), the sign, defined by differentiation and thematisation, could be said to be a feature of many kinds of meaning (though not all), including language and pictures. The resources of verbal and visual semi-

osis Curiously, the most enlightening observations on literature (that is, most of the time, verbal language in general) and painting (pictures and, to some extent, other visual modes of mediation) are still those made by Gottfried Ephraim Lessing, in his book Laokoon, first published in 1766 at least if we take into account the comments offered on this book more recently by Wellbery, Bayer and the present author. According to Lessings conception, paintings use signs the expressions of which are shapes and colours in space; and which have an iconic (motivated) relation to their contents; whereas literature uses sounds in time and has an arbitrary relation to the content. Contrary to contemporary semioticians, Lessing does not bother to separate questions of fact from normative issues: he stipulates that art must be iconic (Todorov 1977:169ff). Therefore, pictures can only signify objects in space and literature only objects in time. This description can obvi-

Fig. . Reconstruction (from Sonesson ) of Wellberys analys of Lessings distinction between pictorial art and literature


ously not apply to non-artistic pictures, nor to modernist art (which by definition overrides norms created by earlier art), and in fact in some ways is not respected by mediaeval pictures either. Yet it may be true that some semiotic vehicles are better adapted to certain purposes. Wellbery (1984) has reformulated Lessings analysis in terms taken over from Hjelmslev, unfortunately abusing this terminology (cf. Fig. 9; Sonesson 988; 1996e; 2003d). Thus, while the terms content and expression are correctly used, the use to which Wellbery puts terms such as material, substance and form is quite foreign to Hjelmslevs intent, because the difference observed by Lessing has nothing to do with relevance. Instead we shall from now on talk about resources, units, and constraints. Resources are what are at hand. Units are the principles of individuation, corresponding to actions in time, and to bodies in space. The constraints, finally, are rules, principles, and regularities of the respective semiotic resources. The content resources seem to be equivalent to what Benveniste (1969) has called the domain of validity of a sign system, and the expression resources are his mode of operation. Verbal language apparently can talk about everything, (it is a pass-key language, as Hjelmslev said), while pictures must make do with everything visible, or everything having visible homologues. The expression resources are Lessings articulate tones, now called phonemes, etc., again opposed to anything visible (limited to static and bi-dimensional visuality in prototypical pictures). Since time is not well rendered in pictures, visual art should ideally pick up one single moment, and, in a parallel

fashion, literature, which it not very conversant with space, should be content to describe a unique attribute. Then, according to Lessing, an extension to the whole will take place in the imagination, spatially in language and temporally in pictures, that is, in the domain that the system cannot adequately render.8 The property that most easily allows such an extension to the whole of the (spatial) object is called the sensate quality; and the phase which best permits the anticipation of the complete temporal succession is called the pregnant moment. If we are to believe Lessing (and, in fact, many others who have written about pictures since then, including Goodman 1968), visual art is not only able to describe the whole of space, but it cannot avoid doing so: pictures have to show fully determinate entities. Taken literally, this must mean that pictures are unable to pick up sensate qualities. Even if we limit this claim, as is no doubt intended, to sensate qualities in the visual modality, this is certainly not true: as I have shown elsewhere (above and in Sonesson 1989a; 1995a), notably against Goodman, the density of pictures is only relative, and all kinds of abstraction are found in them.9 This applies to the expression plane, in the case of more or less schematic pictures: but is also applies to the content plane of some pictures the expression plane of which is fully
8 This is Wellberys conclusion, but in actual fact, the extension in time is the one most important to Lessing, as shown by his negative view of the possibilities of pictures. Cf. Sonesson, in press b. 9 Simply put, density to Goodman means that, no matter how fine the analysis of something (e.g. a picture) into meaningful units, it will always be possible to posit another unit between each two of those already given, and so on indefinitely. 

Fig. 0, Bayers analysis of Lessings system (as systematized in Sonesson ) dense. Thus, for all practical purposes, many pictures are not about a particular person in one or other disguise, but about more or less abstract roles in relatively generic situations. Deriving his inspiration from Peirce, Bayer (1975; 1984) formulates Lessings problem differently: it concerns the relation between the scheme of distribution for the expressions and the scheme of extensions for the referents (Cf. Fig. 20). Bodies are carriers of actions, i.e. they are presupposed by them. Actions are continuous, but can only be rendered iconically as discrete states. The distribution scheme of pictures does not allow for succession, only for actions rendered indirectly by means of bodies and collective actions where several persons act together.0 It will be noted
0 It is strange that Lessing as well as Bayer claim that collective actions are different in this respect from individual ones: the former are made up of individual ones, which simply are distributed in space, and they therefore cannot be rendered if the latter cannot.

that Bayer supposes all continuous objects to be temporal. But, clearly, space is also continuous from the point of view of our perception, so there should also be spatial continua. Pictures actually render certain spatial continua better than language in fact, this is the other side of what was called fully determinate objects above. However, since spatial objects are (potential) carriers of actions, all spatial details serve to suggest potential stories, in particular if they are sufficiently familiar to us to fit in with many action schemes. Thus, it seems to me that, everything else being equal, a picture containing more spatial details will evoke more virtual courses of action. In terms of contemporary narratology, pictures actually contains a larger amount of disnarrated elements, that is, alternative courses of actions starting out from the given moment and in this respect (though of course not in many others), they actually are better than verbal language at telling a story (cf. Sonesson


1997c). The difficulty posed by narrativity in pictures, as Bayer reads Lessing, is that the picture is unable to abstract: Homer may show the gods drinking and discussing at the same time, but that is too much information to put into a single picture. Actually, it is not the amount of information that is crucial (the picture may easily carry more) but the possibility to organise it: verbal language has fixed means for conveying relative importance, newness, focus, etc. The picture, however, in the prototypical sense of the term, may possess some corresponding mechanisms which are not sufficiently known, but hardly any systematic and content-neutral means for organising such information: that is, in Hallidays (1967-68) terms, there are no fixed devices for separating that which is given from that which is new, and that which is the theme (what we talk about) form the rheme (what is said about it). Indeed, although background, as applied to language, is originally a visual metaphor, just as is perspective, that which the picture places in front is not always the most weighty element, with importance decreasing according to increasing apparent distance; nor is necessarily the central figure the most semantically prominent one. One of the principal difficulties is that, in the ordinary picture, the space of representation is, at the same time, a representation of the space of ordi Kress & van Leeuwen (1996) make a lot of unsubstantiated claims of this kind (also as applied to the left and right side). Curiously, although they declare Halliday to be one of their principal sources of inspiration, they do not even differentiate between given and new, on one hand, and theme and rheme, on the other.

nary human perception, which impedes an organisation by other systems. In the history of art, these difficulties were at least partially overcome by Cubism, Matisse, as well as some forms of collages and synthetic pictures, and it has been even more radically modified by visual systems of information, logotypes, Blissymbolics, traffic signs, etc. (cf. Sonesson 1988; 1992a; 2004b). Yet it remains true that pictorial representations lack systematic means for rendering what Halliday has termed information structure. There is another sense, in which pictures, differently from language, have been claimed, notably by Lessing as systematized by Wellbery, to lack the necessary organizational devices: those which permit the distinction between ontological regions, such as that which is real, in opposition to that which is simply possible, or imaginary. In pictures, there is supposed to be a single ontological region, where reality is found on the same level as possibility, imagination, etc. On the contrary, in language, there are many ontological regions, permitting the separation of reality from possibility, imagination, etc. This is was is often called modality, or modus, in linguistics and logic.22 Clearly, language does not possess this property at the level of words (there is not a different genre, or classifier, in any language, as far as I know, for things like
22 OToole (1994), using the notion of communicative functions as characterized by Halliday, talks about modalities in pictures, but these turn out to be semantically entirely distinct from linguistic modalities such as those mentioned above. 

Expression
Pictures Language Linguistic system Pictures

Content
Language Everything which can be thought about (as construed by verbal language) All units, but only rendered as a limited number of properties abstracted from the wholes of the Lifeworld; events (temporal continua) and bodies (spatial continua) but only indirectly rendered as pregnant moment and sensate qualities, in different combinations Minimal chunks, abstracted properties Abundant means for reorganzing things according to prominence (values imposed by the sender, such as given/new, theme/ rheme)

Resources

Static visibility

Everything visual (In prototypical pictures from one point of view) Bodies (spatial continua) directly rendered, that is, as relatively determinate entities; and events (temporal continua) rendered indirectly by traces left on bodies (spatial continua)

All resources

Whole texts

Units
Spatial deployment, relatively impermeable and overlapping organisation of signs (iconic relation to the content), only a limited communication structure (in the case of prototypical pictures)

Constraints

Temporal deployment (or quasi-spatial in written form), successive organisation of signs (independence of content, except at very high levels of organisation or in special cases), developed communication structure

Big, continuous chunks picked up from the Lifeworld, containing irreducible connection to the rest of the Lifeworld. Limited means for reorganising things according to prominence (values imposed by the sender).

Fig. 21. Revised analysis of Lessings system unicorn are opposed to horse). Pictures must however be compared to statements, or rather to paragraphs, chapters, and the like; and considered in that way, language has many means for separating that which is taken to be real from that which is on some other level of reality. While it might be true that there is nothing comparable to modality in pictures, the distinction between levels of reality can however sometimes be made in the whole picture, which is comparable to an entire text. Rather than claiming
0

an impossibility for separating different ontological regions in pictures, we therefore have reason to posit an impossibility (or at least a difficulty) in imposing an ordering according to prominence.23 In the present context, it will be sufficient to spell out two conclusions (cf. Fig. 21): although pictures do not render the world in the form of fully determinate entities, they have to divide up the world in bigger chunks in order to convey information about it than is the case with verbal language, and they lack any general means for imposing an internal structuring on these chunks, apart from the one given in perception. In terms of more modern cognitive linguistics, the same two points might be driven home by saying that pictures cannot pick one image scheme without also having to choose several others, and they are unable to organised these schemes in order of relative importance.4 A further observation pertaining to the ability of language to render temporal objects must also be made. Lessings claim, upheld by Bayer and Wellbery, that language is somehow more capable or rendering temporal continuity than pictures, depends on the idea that linguistic expression, unlike pictures, is itself an action (where of course oral expression is taken
23 This will be shown in Lecture 7, where we take up the discussion of pictorial rhetoric. 4 The term image scheme is used by such linguists as Lakoff, Langacker, Talmy, and many others, but I have nowhere seen any clear definition of what it means. It clearly implies that linguistic meaning is different from what logicians call propositions but is in some way more similar to pictures. The visual representations used, in particular, by Langacker and Talmy, suggests that images schemes are some very abstract kinds of pictures corresponding to a single or a very limit number of objects or events. For discussion, now see Zlatev in press.

as the prototypical case): however, except for a small set of particular cases such as onomatopoetic words, performatives, quotations, and some cases of preferred word order, the action accomplished by the linguistic expression very rarely is the same as the one rendered by its content. This means that, as a general case, language is no better at rendering temporal continuity than pictures are.5 Thus, language does not only have to isolate the sensate quality, but must also, just like pictures, pick up the pregnant moment. Much more low-grained differences between linguistic and visual structures (which most of the time are taken to be pictures) have been suggested by Kress & van Leeuwen (1996: 75ff): thus, for instance, they claim that an affirmation such as Mary gave him a book must be expressed in pictures as Mary bookgave him. In their terminology, inspired in Hallidays linguistic theory, processes with three participants, the third of which is a beneficiary (often equivalent to the indirect object of traditional grammar), are transformed into processes with only two participants, actor and goal. This seems to me to be a completely arbitrary claim. It is true that some languages have specific grammatical constructions that express the part of participant, but they
5 In the semiotical sense, linguistic actions are not iconic of the actions they talk about. Of course, film, and even theatre, as Lessing himself recognised, are able to render temporality in an iconic way: they are moving pictures. At some very high and abstract level, the words, sentences or at least the paragraphs used by the radio journalist describing a horse race at the same time as it occurs also are linguistic actions standing iconically for the actions accomplished by the horses. But this is of course a fairly marginal case, even though it may be more common now than at Lessings time. 

also have such constructions for actor and goal.6 The picture has no specific resources for expressing any of these parts. But, just as in perceptual reality, all the parts of participation can be projected onto the picture. Perhaps Kress & van Leeuwen want to suggest that the relation between actor and goal may be more directly seen in the picture than that to the beneficiary. Indeed, they follow the Gestalt psychologist Arnheim in supposing the presence of vectors (some kind of directional indications) in pictures. Even supposing such vectors to exist, and to be as abundantly present as suggested by Arnheim and Kress & van Leeuwen, there is really no reason to accept curious entities such as book-give. It could be argued, of course, that the act of giving is not as analytically distinct from the book in the picture (and then also in perceptual reality) than in language. However, other things than books may be given, and books may be the vehicle of other actions than giving. This would simply be a particular case of the fact that pictures present reality in bigger and more intricated chunks (Lessings wholly determinate entities) than language. Again, this would also be a contrast between perception generally and language. In the following, I mean to suggest that the opposition functions, in pictures, as a thematic device, permitting pictorial texts to acquire the force of statements. Thus the basic function of the opposition is quite different in pictures from that found
6 It is perhaps only a curiosity that exactly this kind of example is used by Peirce to claim that triadic relations cannot be reduced to dyadic relations which would imply that neither pictures nor other semiotic resources could accomplish this feat.

in verbal language. While the linguistic opposition is mainly pre-semantic, its counterpart in pictures concerns extended chunks of texts: it is found on the rhetorical level (cf. Sonesson 996a; 997a; 00a; 2004a). Linguistic and logical oppositions The notion of opposition has a double origin in semiotics, from philosophy, and logic in particular, and from Saussurean linguistics, particularly as developed in the phonology of the Prague school. Logically, the important distinction is that between contradictory (white vs. non-white) and contrary terms (white vs. black, which allows for all the intermediaries of grey-scale). In linguistics, the opposition in closely wedded to the notion of structure. Saussure famously argued that in the language system, there are only differences without positive terms. Every element derives its identity from its distinction to other elements in the same system. The phonemes, in particular, Saussure said, are units that are purely oppositive, relative, and negative. In his pioneering study of phonology, Trubetzkoy (1939:59ff) distinguished different types of oppositions from several points of view. These distinctions are based on his important insight, often forgotten in later semiotics, that an opposition between several terms must suppose some kind of similarity, a base of comparison, as well as properties which are different. Thus, an opposition is one-dimensional, if the base of comparison is only found in two items, but otherwise multi-dimensional (e.g. the common factor in the Latin letters E vs. F is not found elsewhere, but the one present in P vs. R also appears in B). On the


other hand, an opposition is proportional if the distinction between the terms is found in other pairs of elements, or else isolated (some irregular plurals, like goose/geese and tooth/teeth are proportional, as are even more obviously the regular ones). In privative oppositions, one of the terms simply consists in the absence of the trait found in the other term (in phonetics, unvoiced sounds as opposed to voiced ones, in semantics the plural s opposed to the lack of it). An equipollent opposition, on the other hand, involves two terms both of which are something in themselves (irregular singular/plural modification like foot vs. feet, where the singular is not just the absence of plurality marking). In gradual oppositions, finally, some feature is present in different degrees in several terms (an example is the traditional phonetic description of the degree of aperture in vowels). This latter distinction would seem to correspond to the logical one between contradictory and contrary terms, adding the case in which some points between the extremes are singled out for consideration. In the final case, the opposition is not binary: it has more than two terms. Oppositions in semantics and semiotics Roman Jakobsons (1942) heritage is, in this domain, extremely ambiguous: he was the first one to show that, at least in phonology, all oppositions may be reduced to the binary, privative kind. This supposes the resolution of one non-binary, equipollent opposition into a set of binary, privative ones, itself based on a redefinition of the categories entering the opposition. In the case of phonological features, Jakobson, Fant, and Halle (1952) have shown that these categories may be justified from

an acoustic point of view; whether they are also perceptually relevant is an open question. In any case, it does not follow that the reduction to binary, privative oppositions in adequate outside the domain of linguistic expression. Paradoxically, it was Jakobson (1976) himself who, in his 94 lectures at the New School of Social Research in New York, countered Saussures idea that also semantic oppositions were purely negative: contrary to the Saussurean claim, not the whole meaning of the words night and day is derived from their opposition. Yet, Claude Lvi-Strauss, who listened to these lectures, later brought the idea of oppositions being purely negative, binary and privative to what would seem to be an even more saturated domain, myths, and also, in his mask analyses, to visual semiotics. Jakobson and Lvi-Strauss together heavily influenced what is known as French structuralism into conceiving all oppositions as being purely privative, and this idea still lingers on in the work of the Greimas School. In fact, the kind of oppositions discovered by Structuralism in myths, literary works, pictures, and cultures, are, on many counts, very different from those present in the expression system of verbal language. Even Trubetzkoys classifications turn out to be of little help when trying to understand these differences. First of all, oppositions may be constitutive of the identity of signs and/or their parts, as the features of phonology, or they may be merely regulative in relation to an already constituted identity, which would seem to be true of many other cases, such as two pictures, or two objects in a picture, already identified as representing some

Fig. . The Kindy publicity (a) and its two intertextual correspondents, from the Marily Monroe film The seven year itch. I.3.3.,I.3.5.; 1992a). Presence and different kinds of absence Oppositions may be in absentia, or true oppositions, or in praesentia, or contrasts. Thus, in pictures there is no obvious equivalent to the system of (constitutive) oppositions present in the phonological and semantic organisations of verbal language. Rather than deriving from the system, oppositions are created on the spot, i.e. in a given text. Most oppositions found by Structuralists in poetry, visual art, advertisements, myths, and so on, are really of this kind, in other words, they are oppositions in praesentia, or contrasts. In visual semiotics, it is in particular the Greimas school which posits a model according to which all pictures are organised into two parts, one of which realises the features opposed to those present in the other (e.g.


thing. Thus, Lvi-Strauss (1975) is certainly wrong in arguing that the meaning of the Swaihw and Dzonokwa masks derives entirely from their mutual opposition: this opposition, if opposition there is, is only secondary to our recognition of both as (aberrant) faces (cf. Sonesson 1989a,I.1.3.,

Marilyn
with dcolletage tight-fitting naked shoulders lifted by air stream in circle shape (film) or in the shape of a scroll (poster)

dress vs vs vs vs body position vs vs vs vs vs vs

vs

Kindy girl
without dcolletage loose shoulders covered hanging straight down

shoulders lifted head (chin) lowered face somewhat from the side chin touches shoulder body in 3/4 profile legs pressed together and bent at knee level (film) or in the shape of inverted V (poster) hands joined in front of lower body in vs order to press down the skirt embarrassed giggle vs location in space somewhat in front of the man (film) vs or far in front of the man (poster)

sloping shoulders head (chin) lifted up face in profile chin does not touch shoulder body seen from the front straight legs hands joined in front of lower body without instrumental function good laugh with open mouth behind the man

clear colours in one field, and dark colours in the other). However, it should be noted that oppositions in absentia are not necessarily systemic: they may refer to another text. An advertisement, or a postmodern artwork, may make use of the fact that there is a large stock of pictures which we, as members of Western culture, tend to recognise, and position itself as a set of op-

positions and identities in relations to one such picture. Using a familiar but vague term, this kind of oppositions in absentia could be called intertextual (cf. Sonesson 1989a;I.3.3.;1992c). Thus, for instance, the advertisement for a brand of socks called Kindy could at first be mistaken for the poster, or a still, from the well-known Marilyn Monroe movie The seven year itch: in parFig. . Feature oppositions between the Marilyn picture and the Kindy advertisement (a above: Marily vs the Kindy girl; b to the left: the men with Marily and the Kindy girl)


body position back from the left vs front and right side head (chin) lowered vs head (chin) lifted up hands in pockets hands in pockets without any instrumental to lift the trouser legs function no influence of air stream vs tie lifted by the air stream location in space somewhat behind the girl (film) vs in front of the girl or far behind the girl (poster)

Man with Marilyn

vs

Man with Kindy girl

I. Norm: schema, isotopy II. Normal and expected transgression of the norm Transgression of the norm which creates new norm III. Mitigation of the transgression Modification of the new norm

The body (except certain parts such as the hinds, the face, etc.) should be covered. A woman, whose body in our culture (as opposed to what appears to have been the case in Ancient Greece) has a heavier erotic load, shows some part of her body, notably some part which is particularly erotically loaded

a) The result of the transgression is only that some lower layer of those which cover the body (the underwear) becomes visible (which supposes are modern Western model of garments semiotics, in which there are several layers of clothing). b) The body part being shown (here the legs) is not in itself heavily erotically loaded, but it is the neighbourhood of parts which are. In both cases, the erotic load only remains present indexically, as an effect of the contiguity relation

IV. Double transgression of the expected and doubly mitigated transgression: IVa. First transgression of the normatively reinstated transgression R e v e r s a l of gender roles IVb. Second transgression of the normatively reinstated transgression D i s p l a c e m e n t of erotic loads
The act of showing is transferred from the woman to the man, itself a fundamental anthropological opposition of all societies until the present

The act of showing is transferred to a body part which is not even very close to any part which is heavily erotically loaded (the feet)

Fig. . Example of the series of transgression based on system oppositions in the Kindy advertisement ticular, it reminds us of the familiar scene in which Marilyns skirt is lifted by the stream coming from the air-valve (cf. Fig. 22 and Sonesson 1989a and 1992c). But there are many, more or less notable, differences: thus, Marilyns dress has a deep dcolletage, is tight-fitting, displays naked shoulders, and is lifted by the air stream in a circular shape (in the still) or in the shape of a scroll (on the poster). The dress of the Kindy girl, on the other hand, shows no dcolletage, is rather loose, covers the shoulders, and hangs straight down. Marilyns shoulders are lifted, but those of the Kindy girl sloping; the chin of the first is lowered touching the shoulder, that of the other uplifted without contact with the shoulder. Marilyn body is seen in 3/4 profile, that of the other is frontal. Marilyns legs are pressed together and bent at knee level (in the still) or in the shape of an inverted V (on the poster), whereas those of the Kindy girl are straight. Both have the hands joined in front of the lower part of


body, in Marilyns case in order to press down the skirt, but in the other case without any instrumental function. Marilyns embarrassed giggle contrast with the full open-mouthed laugh of the other girl (cf. Fig. 23). In both still and poster, Marilyn is placed more or less in front of the man, while the Kindy girl stands behind him. Marilyns companion is seen from the back left, his chin is lowered, and he is in no way influenced by the air stream. The Kindy man, on the other hand, is seen from the front right, his chin is up lifted, and his tie has been blown up by the air stream. Both have the hands in their pockets, without any instrumental function in the first case, but in order to lift the trouser legs in the second case. Thus, the meaning of the picture may
constitutive oppositions regulative oppositions oppositions in praesentia (= contrasts) oppositions in absentia

be derived from its opposition to the Marilyn Monroe picture. At least in this case, part of the meaning may also stem from its opposition to expectancies engendered by our cultural system: in contemporary Western culture (contrary to, for instance, that of the Ancient Greeks) the female, but not the male, nude is considered to be an object of (sexual, aesthetic, and so on) value; and body parts close to the sex organs are particularly heavily loaded. But here it is the man who is at risk of showing his body, and the body part in question is simply his feet. These, then, would be systemic oppositions, but they are in no way specific to the picture, but pertain to culture as a whole (cf. Fig. 4 and 5 and Sonesson 1992a, c). Summary Given the same state of the world, pictures

create the items which they define (which are nothing else but the poles of the oppositions) transfer new meaning to units which are already constituted as such both poles of the opposition are present in the perceived artefact only one of the poles of the opposition is present in the perceived artefact, but the other poles is present in the consciousness of a group of people which thereby are abler to interpret the artefact and attribute an meaning to it.

systemic oppositions intertextual oppositions structural oppositions abductive oppositions

exist before the artefact, or independent of it, in the form of a system, i.e. a set if units and the rules for their realisation and combination. exist in relation to some other artefact the meaning of the opposition is entirely exhausted by the poles between which it obtains. the meaning of the opposition can only be grasped if the poles are considered on the background of some generally accepted rules or regularity, that is, if one makes an abduction (in the sense of Peirce)

Fig. . Different kinds of oppositions



and verbal language are able to convey its properties to different extents, and in different respects. The best analysis of this difference remains the classical work of Lessing, in particular as reviewed by more recent semioticians. However, we have to conclude, in part against Lessing, that not only is the picture forced to select one pregnant moment of many existing ones from the state of the world described, but language is obliged to make the same reduction, in addition to picking up a limited amount of characteristic properties. Put in other words, pictures analyse the world in bigger and more intricate chunks that language. Language therefore is more capable of choosing just some properties out of those present in the real world scene, and it has more systematic devices for telling us what is important and what can be considered less significant. In language, (binary) oppositions are constitutive of the very means making up the semiotic resources; in the case of pictures, however, they are secondary to meanings already given in the depictions, and thus may serve as a substitute for the lacking thematic devices.

vant in the analysis of pictures than in that of literature, for which it was essentially developed. Fernande Saint-Martin, as well as (more indirectly) Groupe , are right, I believe, in claiming that pictorial signs are inherently perceptual, that is, visual, in nature. Not only must such a claim be justified from a discussion of perceptual theory, but some criteria must also be proposed for choosing as a foundation one among the several conflicting theories occupying the contemporary scene of perceptual psychology. Moreover, if all perception turns out to carry meaning, we are faced with the further task of determining in what way visual signs, such as pictures, differ from mere meanings conveyed be visual means. We have of course already anticipated on the answer to this question, but it will only be fully elucidated once we have acquainted ourselves a little more with perceptual meaning. Three (or perhaps four) kinds of perceptual theory Even granted that the pictorial sign is an object of perception, it remains to be determined whether it is a Gibsonian object of perception, a Gestaltist one, a constructivist one, or perhaps even something else. The choice of a perceptual theory that is to serve as a foundation to pictorial semiotics is by no means as easy to accomplish as Saint-Martin or Groupe would like to think. There are, in present-day psychology, basically three ways of conceiving the relationship between that which is perceived and the cause of the perception: and the three corresponding theories are those of constructivism, Gestalt psychology, and direct registration theory, or Gibsonianism


4.4. Figures of perception: wholes and neighbourhoods

Visuality, as a mode of perception, it certainly a Hjelmslevean form: it is pertinent to the conformation of perceptual objects such as pictures, even if a perceptual explanation will not be sufficient to account for them. The state of the world, which is subject to the varying descriptions given expression with the aid of pictorial or verbal means, is a world of (mainly visual) perception. The model of the sign as an object of perception deployed by the Prague school seems much more obviously rele-

(Hagen 980:4ff; 979; Winner 98:84ff; Sonesson 1989a,III.3.3.), which Gibson himself calls ecological psychology. It is the contention of the latter theory, that all information needed is available directly in the light coming from the environment, and is determined by this light, although only if we take into account all the higher-order variables of the environment and their invariants over time. According to Hagen, constructivists like Gregory and Gombrich claim that reality lacks all intrinsic organization, and so must be set in order by a hypothesis on the part of the perceiving subject; but the resulting arrangement is only given with a certain degree of probability, and may have to be further revised. Again according to Hagen, Gestaltists such as Arnheim and Hochberg would agree with the constructivists in affirming that reality is fundamentally ambiguous, and so must be supplemented by the beholders share, but, in their view, the perceived organization results deterministically from the Gestalt laws, built into the human mind. Also, while the Gestalt laws, or at least the simplicity principle on which they are based, are supposedly innate, constructivists rather tend to suppose that the hypotheses employed in perception are either explicitly posited as conventions, or derive in a more tacit fashion from earlier experience of the world (cf. Winner 1982:108).7 Hagen maintains that all three theories are descriptively inadequate: constructivism because no criteria have been proposed for when a hypothesis is confirmed; Gestalt psychology, because its laws are mysterious; and Gibsonianism, because no list of the invariants picked up from the en7 Surprisingly, Saint-Martin (1990:86) affirms, on unclear evidence, that no such innateness is required by Gestalt theory.

vironment can at present be given (p.21ff). In spite of these observations, however, Hagen herself clearly remains within the bounds of direct registration theory. This is precisely the theory that Winner (1982:98ff) declares to be descriptively inadequate. On the other hand, she argues that there are cases in pictorial perception, in which simplicity may be shown to override familiarity, thus favouring Gestalt psychology, as well as other cases in which familiarity gains the upper hand, which is a result favouring constructionism. Contrary to Hagen, Winner thus concludes that reality is ambiguous, but may be supplemented in various ways. It is natural that, in her book on Gestalt theory, Saint-Martin should neglect constructionism and direct registration theory in favour of the Gestalt school. However, it is as pity that these movements are never ever mentioned as such, and thus are never presented as the alternative conceptions that in fact they are. Gibson is even introduced as a continuator of Gestalt theory (Saint-Martin 1990:58), only to be later attacked as the apostate he must thus appear to be. Having never tired, in his numerous publications, of criticizing Gestalt psychology, Gibson would have been surprised and shocked by this suggestion. On the other hand, it is true that he has always recognized in Koffka one of the most important influences on his thinking. In fact, there seems to be no real Gestalt psychologists left, except for those who are rather to be counted among the students of pictorial art, as, for instance, Arnheim. There is undoubtedly an array of phenomena, discovered by the Gestalt psychologists, which are still with us, but which now are in need of new explanations.


The theoretical stance taken by Hochberg, whom Hagen treats as a Gestaltist, is, in actual fact, that of constructivism, as he himself affirms. He has, on the other hand, undertaken a critical appraisal of the Gestalt tradition. Sometimes, he claims, the most natural three-dimensional interpretation of a picture is not the simplest one, as Gestalt theory would make us expect (Hochberg 1972:59f). Gestalt phenomena are really peculiar cases of Helmholtz law, according to which we perceive that which is most probable, given the pattern of stimulation (Hochberg 1980:58f; cf. 1974;196ff). The minimum principle cannot be due to a built-in perceptual mechanism that makes us perceive always the simplest object fitting the overall stimulus pattern, Hochberg (1978) claims, going on to suggest that instead it may be an effect of putting together fragmentary sensory data, in a manner corresponding to the most likely object, or that it may result from the arrangement that has the best chance of being seen and remembered from one momentary glance to another. Indeed, Hochberg (1972:60) even claims that the very fact of perceptual objects having to be grasped in a long series of momentary glances imposes limitations on the validity of Gestalt organization, since different parts of the whole will fall on the fovea at different moments. SaintMartin (1990:28f), who notes this last point, takes Hochberg to task for neglecting the coherence that Gestalt psychology takes to persist from one glance to another, and for supposing only that portion which is reflected in the fovea to be actually perceived. In other writings of his, not quoted by Saint-Martin, Hochberg certainly gives due attention to peripheral seeing. Yet it

may be true, as I have argued in discussing pictures of impossible objects (in Sonesson 1989a,III.3.4.), that Hochberg exaggerates the importance of foveal perception. In perceptual psychology, the really interesting discussion nowadays takes place between constructionism and direct registration theory, which first of which has newly come into the fore thanks to cognitive science (cf. notably, Hoffman 1998). One may wonder, however, how it is possible for Gregory, Hochberg, Hoffman and others to think that inferences are necessary to explain what is actually perceived, when Gibson, Kennedy, and Hagen feel they can dispense with them altogether. Among the facts to be explained by perceptual psychology figure prominently, in Gregorys view (1966:1974), such things as the pick-up of non-optical properties, gaps in the stimuli, visual illusions, ambiguities, illusory contours, and the perception of logically impossible objects. To Gibson, on the other hand, most of these phenomena are simple curiosities, of very little weight to everyday perception, and therefore to perceptual psychology. Thus, one of the differences between the theories lies in the choice of facts that they consider worth-while explaining. Yet, it is perhaps not beside the point to argue about which facts we should care to explain. One of the pioneers of constructionism, Ulrich Neisser, has recognized, in his later work, the necessity of accounting for the fact that ordinary perception usually proves right. Just like Gibson claims, information is picked up from light, Neisser (1976: 16, 20ff) grants, but this pick-up only serves to start a perceptual cycle taking place in time: anticipatory schemes generate generic, rather than specific, hy0

pothesis which are modified by the information available, engendering subsequently more detailed schemes, which guide the further exploration of the optic array. In a late publication, Neisser (1987) seems even more convinced of the fact that, as Gibson affirms, information for that which is perceived is present in the array of light available to the eye, as soon as we attend to higher-order variables, and to their modification over time. Even categorisation is now said to be ecologically grounded, though somewhat less direct. There thus seems to be a fourth alternative in perceptual psychology, not recognized neither by Hagen nor by Winner, which amounts to a blend of direct registration theory with some facets of constructionism. This is Hochbergs position, as it was that of Neisser at least as late as in 976. Although Saint-Martin often quotes Piaget to help buttressing the common bias of constructivism and Gestalt theory, the latters general conception, according to which both assimilation and accommodation are involved in our ordinary experience of reality, certainly seems to suggest that he, too, would favour a mixed approach. There are differences between the constructivists, the Gestaltists, and the Gibsonians, which have to do with which experiments they consider relevant, and to which properties of the experimental results they attribute most importance, but this in turn must be due to the way they, as laymen, inhabit our common Lifeworld, and how consciously they relate to it. This does not necessarily mean that the differences between these psychologists are mere disparities of taste and personal predispositions; for they could as well be explained by their different aptness for the difficult

task of doing phenomenology. Indeed, the numerous similarities between Husserlean phenomenology and Gibsons ecological psychology are not merely of anecdotal interest. For, whatever we may think of its ultimate philosophical postulates, phenomenology constitutes an exceptionally careful description of reality as it appears to us, when closely scrutinized. Thus far, it will be noted, we have been mainly concerned with the perception of the world, and with the relation between the assumed physical cause and the resulting percept, not with picture perception. No doubt, the whole issue appears to be roughly analogous to that of pictorial iconicity, the relation between the picture and ordinary perceptual reality reproducing, in that order, that between the ordinary percept and its physical cause. We should therefore expect constructionists and Gestaltists to favour a version of a conventionalist theory of picture perception, and Gibson to defend a similarity theory, but the opposite turns out to be closer to the truth. Not only do Gestaltists and constructivists (with the exception of Hochberg) treat pictures and reality as being of a kind, but most of their reasoning is based on pictorial examples, although their conclusions concern the perception of the real, threedimensional world. Gibson actually argues that their theories are artefacts of their having studied pictures rather than reality. And he goes on to claim that pictures are not at all based on similarity. Yet he certainly does not want to maintain that they are conventional, in the way semioticians would use that term: instead, because of being so different from the perceptual environment, they must render the invariants of perception, and convey them to us, in a


very different way from that in which they become manifest in the real world. It is thus misleadingly when Saint-Martin (1990:15) quotes Gibson as saying, like Piaget, that depicted objects are not perceived: they are indirectly perceived, as he continues the phrase elsewhere. It is precisely this phenomenal observation, to the effect that perceptual objects, rather than piecemeal perceptions, are that which is perceived, which Saint-Martin (1990:58ff) finds unacceptable. Not surprisingly, she finds the same faults (p. 27) with a disciple of Husserl, Merleau-Ponty, who also claims that perception is of the object, not of its appearances. In a way, she is right in affirming that this proposition cannot be verified in a laboratory; but it is verified by each and every instance of human perception. Childrens drawings at first render things (and abstract properties) instead of their appearances, because the capacity to see appearances must be laboriously acquired (cf. Sonesson 1989a,III.3.2.). In the naive attitude, Gibson (1971:31f) affirms, we look through a series of perspectives in movement to the invariant features of the object, while in the perspectival attitude, we fix a single perspectival view in order to consider it in its own right. Roughly the same opposition exists between the natural attitude and phenomenological reflection in Husserls work.8 Moreover, Husserl would argue, just like Gibson (quoted by Lombardo 1987: 350), that what is seen-now and seen-from-here specify the self rather than the environment.
8 Saint-Martin (1990:60ff) criticizes an earlier variant of this distinction, in terms of the visual world and the visual field, because it seems to suppose that sensations persist to no purpose whatsoever in the human organism, but she fails to note that Gibson himself rejected the distinction in this form in his later work.

Saint-Martin (1990:11) it mistaken to believe that Gibson supposes there to be any kind of pre-established similarity between human knowledge and the objects of this knowledge. This description is based on the idea, defended by constructivism and Gestalt theory alike, that the communication between the world and the mind is somehow interrupted. Constructivists and Gestaltists assumed that something must be added to the information given, because it seemed to them that only impoverished information could be available. It is the merit of Gibson (and here he goes well beyond phenomenology) to have shown that all the information needed is actually there to be picked up, once we realize that the perceptual system is able to attend to higher-order properties of the array of light, in particular as they change over time. In terms more familiar to semioticians, it is a question of determining what kinds of units form the pertinent input to the perceptual system. After some hundred odd years of discussion about what must be added to the stimuli, in order for perception to occur, the claim that reality is perceived directly may appear much too Gordian a solution. Also, Husserls position is, on the face of it, more sophisticated, since what is directly perceived in his view is some kind of object internal to consciousness (although the difference is reduced away), in fact a Lifeworld object, not a physical one. Indeed, Gibson (1982:106) observes that he is concerned with properties noticed by phenomenologists, but that he assumes them also to be real. On the other hand, although he was certainly committed to some kind of psychophysical parallelism in the earlier versions of his theory, he later (1982:217) argued that ecological physics must be


distinct from the ordinary one, and that its invariants were of a quite different order. Indeed, the kinds of implicitly known regularities prevailing in the world of Gibsonian ecology are not very different from the customary ways things have of behaving in Husserls Lifeworld.9 The real problem with Gibsons and Husserls conceptions is that they do not take the argument far enough. Not only do we not see sensations, but real objects, but we do not perceive geometrical volume as such, but a cultural-laden object, not a cube but a dice, not the tea cup formula but the tea cup itself, not the cat as a geometrical shape in movement, but that peculiar domestic animal of the Occidental Lifeworld (cf. Sonesson 1989a,I.2.2. and III.3.2.). In an interesting discussion of the changing meaning given by Gibson to the notion of direct perception through the years, Costall (1989:10ff) makes a similar observation, concluding that no example of human perception could ever count as direct on Gibsons terms. Yet, the only world we could ever directly perceive is the world of our own culture. Just as some disciples of Husserl, as for instance Schtz, discovered the sociocultural character of the Lifeworld, Costall thus points to the cultural overlay of the Gibsonian environment. In a way, therefore, constructions and unconscious inferences are really there: they are only much more deeply embedded. The furniture of the Lifeworld So far, I have suggested that the kind of common sense world required by a semiotic theory, or at least by any viable theory of iconicity, is somehow akin to Husserls
9 An expression which is remniscent of the turn of phrase used by Peirce to introduce the concept of abduction.

proposal for a science of the Lifeworld, which was given some more empirical substance by Gibson, and, in a parallel line of development, received some rudiments of semiotical trappings from Greimas. As no one can ignore, the history of philosophy is littered with ontologies but we are here engaged in a different enterprise which, long ago, I suggested should be called folk ontology (Sonesson 1978), imitating then fashionable expressions such as folk ethnology and folk taxonomy; and which, quite independently, was baptised in the same way by Smith (1995), on the analogy of folk psychology. However, this enterprise is probably better know as descriptive metaphysics (Strawson 1957): the reconstruction of the world which is taken for granted in the ongoing business of everyday life, not the least part of which involves the use of signs and other meanings. However, there is nothing very new about folk ontologies either: apart from the authorities mentioned before, and also from those representatives of cognitive science who have resurrected the idea under the heading nave physics, there is a long, though apparently somewhat disconnected, tradition, at least since the end of the 9th century (cf. Smith & Casati 1994), which busies itself with the reconstruction of common sense. So, in the end, the originality which I may hope to claim for this proposal must rest on not trying to reconstruct folk ontology simply from our intuitions (or rather our meta-intuitions) about how the common sense world is made up, but simply delineate some of the features which must be present in the common sense world, in order to explain our intuitions about how signs are different from nonsigns (and proto-signs), and how primary


iconicity is possible. This also means, on the negative side, that all we can hope for is a partial ontology. Something must undoubtedly be said about what kind of reality I claim for the common sense world, as compared to the world described by scientific physics. Smith (199b; & Varzi 1999) invokes ecological terminology to describe the Lifeworld as the niche in which human beings stake out their life. It is found on a mesoscopic level, in between the microscopic and the macroscopic levels described by physics, but it is real in the same sense as the latter two. According to Smith, this is Gibsons view, which he opposes to that of Husserl, for whom only the Lifeworld, not that of the physical sciences, is real. This is not how I read Husserl; and even if it should turn out that I am wrong in my interpretation of Husserl, I should still prefer to see the relationship otherwise. To take a familiar though perhaps somewhat old-fashioned example, physics may describe light as being at the same time a series of waves and a conjunction of bodies. This is nonsense to common sense, and rightly so: for, clearly, this must mean that light is really some third kind of thing, which happens to share some properties with the common sense objects called waves and bodies. So, the language that physics uses to describe the physical world is approximate and metaphoric (Husserls Ideenkleid). This does not mean that the world that it tries to approach is not real. But the Lifeworld is the only world to which we have direct access and which may be described in its own language. Even in folk ontology, there is no starting from scratch. So perhaps, like so many others, we should take our departure

in Aristotle, certainly in some respects a good folk ontologist. Smith (1995), for one, suggests we should retain the opposition between substances, defined roughly as that which can exist on its own, and which admit contrary accidents at different moments in time; and accidents, which corresponds to qualities, actions, etc., and which are said to inhere in substances.30 It should be mentioned right away that Smith makes an important addition to traditional Aristotelian ontology: he allows for relational accidents, which are, so to speak, attached to several carriers, i.e. substances. But whatever the intrinsic merits of these terms, at least the first one is not very convenient to use in semiotics, since the same word has there been employed, following Saussure and Hjelmslev, in quite a different sense: i.e. to signify that which is not relevant (notably, in a sign, in relation to the other plane), as opposed to form.3 Elsewhere, however, Smith (1999a) uses other terms to express his bicategorial ontology: continuants, which, in addition to Aristotelian substances, include media such as air and water; and occurrents, which correspond to Aristotelian accidents but also to events and processes (which Smith 995 already assimilated to the accidents). Unfortunately, I still find this terminology somewhat misleading. After all, it makes perfectly good sense to speak about continuity in space as well as
30 This is of course only a small part of the properties Smith casually attributes to substances, and it is quite different from his formal definition, which is expressed in terms of boundaries. 3 The confusion between the Aristotelian and the Saussurean sense of substance has actually taken place, in an otherwise excellent book about Lessings semiotics written by Wellbery (984; cf. criticism in Sonesson 988 and above). 

in time (cf. Sonesson 988 and the discussion of Lessing above). On the other hand, spatial occurrents certainly sounds like pure nonsense (or magic?). But my misgivings go further: I do not think folk ontology would generally accept the postulated equivalence between events and processes, on one hand, and qualities of things, on the other. If we take the view that things are in some sense carriers of both properties and events, then the general feeling would no doubt be that the connection between the former and its carrier is closer to being intrinsic than that between the latter and its carrier. Of course, properties may change, but precisely as a result of a process: and the result, as opposed to the process, is intrinsic to the thing. In the last paragraph, I reverted to a more common sense terminology, which opposes things (or objects) to events. Actually, it may be more convenient at times to talk about spatial and temporal objects, respectively (while still allowing the term objects, without qualification, to be equivalent to spatial objects). This, I would take to be the basic common sense opposition: objects that are (prominently) in space, as distinct from objects that are (prominently) in time. As for the properties of things (and of events, which I will not discuss further here), I think it would be more proper to derive them mereologically, that is, as parts of the whole making up the object. While this piece of folk ontology is introduced here only as stemming from my intuition as a member of the universal Lifeworld, it could have been derived as a requirement from my work of visual rhetoric (cf. Sonesson 989a; 996a, b, c; 997a; 2001a; 2004a; 2005a; and Lecture 7). In

this context, I have suggested that there are three ways of dividing a piece of cake, or any other object: into its proper parts (e.g. the head if the whole making up a human body); into its properties (being male as opposed to being female, or being an adult as opposed to being a child, with reference to the same whole); and into its perspectives or adumbrations (the body seen from the back, the head seen in a three quarter view, etc.). In my earlier writings, I talked about three kinds of factorality (which, along with contiguity, makes up indexicality, not as a kind of sign, but as a ground). A more well-know term for what I was there talking about, I have since learnt, is mereology, which is the theory of parts and wholes, derived from Husserls early work, but apparently given this name by the logician Lesniewski (cf. Smith 1994; 1995). It should be noted that, within this repertory of three kinds of mereological principles of division, the third kind would, if it was described as an Aristotelian accident, be relational, in the sense of Smith: it would inhere in two objects. But that is not the only way in which it is special. One of the objects in which it inheres is a subject in other words, a person, an ego. In Peircean terms, perceptual terms are primarily indexical. According to the interpretation of Peirce that I have defended elsewhere (Sonesson 989a; 996a, b, in press a, d, e), indexicality can be conceived as a property that makes something which is a sign into an index. However, by a slight shift of emphasis, it could be construed as a property which, when added to the sign function, creates an index, but which, in addition, may have other parts to play in the constitution of meaning. Such a conception might account for the ambiguities


of the Peircean notion, as well as for some of the uses to which it has been put subsequently.3 Generally put, then, an indexical ground, or indexicality, would then involve two things that are apt to enter, in the capacity of being its expression and content (i.e. representamen and object), into a semiotic relation forming an indexical sign, due to a set of properties which are intrinsic to the relationship between them, such as it is independently of the sign relation. This kind of ground, which is a relation, is best conceived in opposition to an iconic ground, which really consists of two sets of properties which happen to be of the same kind, and the symbolic ground, which is a non-entity, since the motivation of the sign has no existence independently of the sign itself. This is the sense in which indexicality is Secondness, iconicity Firstness, and symbolicity Thirdness. Just as indexicality is conceivable, but is not a sign, until it enters the sign relation, iconicity has some kind of being, but does not exist, until a comparison takes place. In this sense, if indexicality is a potential sign, iconicity is only a potential ground. In sum, then, iconicity begins with the single object; indexicality starts out as a relation. The problem, therefore, consists in determining what kind of relation it is. Such a view of indexicality as the one reconstructed above best fits in with the most general formulations given by Peirce, according to which it depends on there
3 Given the long period through which Peirces thinking evolved, and the state in which it came down to the public, it is not surprising that indexicality, like so many Peircean notions, should be so variously, and inconsistently, defined, and that many of the examples given hardly fit in with the definitions (cf. Goudge 1965; Sonesson 1989a; 1994a, 1995b; 1998a).

being a real connection, an existential relation, a dynamical (including spatial) connection and even, in one of its many conceivable senses, a physical connection between the items involved (Peirce 1.558; 1.196; 2:305; 3.361; 8.335). From this point, it seem natural to go on to argue that indexicality is involved with spatiotemporal location (Burks 1949:683ff), which underlies the indices of such logicians as Bar-Hillel and Montague, the egocentric particulars of Russell and the shifters of Jespersen and Jakobson. In fact, however, as Savan (1976:25ff) observes, location in time and space will only result, to the extent that some system of co-ordinates has been conveyed by other types of signs or, as I would add, to the extent that it can be presupposed by the ongoing practice of the ordinary world of our experience, the world taken for granted, our common Lifeworld. More generally, many of the examples adduced by Peirce would justify us in going along with Jakobson (1979), when he claims that indexicality is based on real contiguity, and is connected with the syntagmatic axis of language, and the rhetorical figures of metonymy. To Jakobson, however, metonymy actually involves, not only the relation of contiguity of traditional rhetoric, but also that of part to whole, known in rhetoric as synecdoche. This distinction may be re-established inside the category of indexicality (cf. Nth 1975:20f), and could be described more generally in terms of contiguity and factorality (cf. Sonesson 1989a:40ff).33
33 There are of course many other, more widely quoted, definitions by Peirce that suggest that indexicality is defined by causality. We will discuss this issue, and some other problems with indexicality, when we turn to the photographic 

Fig. . An elementary model of the Lifeworld Moreover, if, as Peirce would have it, the world is profused with signs (which I would prefer to call, with a more general term, significations), then this world is highly organised, according to a scale of values. In a deeper sense than that of Gibson, our folk ontology is ecological: it is centred on some objects, in relation to which everything else is given in adumbrations. It is, so to speak, irreducibly contextual (cf. Sonesson 1978). In other words, the common sense world follows at least two principles of ordering: that of directness and that of centrality. In relation to the subject, the first depends on contiguity, and the second may (in a vaguer sense) have something to do with factorality. Inside ecological physics, in Gibsons sense, there must be some kind of social physics, not exactly in the Durkheimian sense, but on the micro-level. Schtz and Mead have talked about the array of things of the human world which
sign, in Lecture 9.

are peculiar in being at hand, occupying the manipulatory sphere; and Wallon has discussed the ultra-choses, which are outside this sphere, but are seen from there. Even these humble things do not only have a use, but are also there, as LviStrauss would have said, to think with. On a smaller scale, we may say that the things which serve as expressions in sign functions are part of those things at hand, while the things which serve as contents are more akin to the ultra-things. Indeed, as I suggested in my investigations concerning cultural semiotics, we may imagine further concentric circles around the ego, analogous to those of proxemics, but serving less for protection than for recognisance. These are the spheres were we find those things which are relatively more well-known, more understandable, more similar to the subject itself, and even, in a sense more real (Fig. 6 and Sonesson 1997a; 2000b; 2001e). We may think of these spheres as those to which the subject


has direct access, and which serves as stepping-stones to the wider Lifeworld. However, most things close to the subject, in this sense, do not make up expressions forming part of sign functions. Most of them are valued in themselves. As a contrast, though directly accessible, expressions are non-autonomous entities. They are, in Husserls (1913) sense, dependant objects (also cf. Smith 1994). They are not, as we shall see, parts of larger wholes, but objects in their own right. Nor are they simply dispensable. It is at some general level of Lifeworld ranking that they may be considered dependant objects. However, as part of the pair of objects making up a sign function they are interdependent, as are their contents. There is no sign without both expression and content. But the best way to think of this relationship is again to consider it from the point of view of the perceiving subject: the expression is dependant in the sense that the subject only focuses on it in order to reach further out to the content. It only enjoys a mediate focus. Semiotics as the general theory of categories If categories are the tools by means of which the world of our experiences is carved into pieces (segmented as linguists used to say), then the theory of categorization is the most elementary layer of semiosis. The French 8th century School of ideologues studied extant classification systems without divorcing this study from the search for the best classification. As different presentday semioticians as Lvi-Strauss and Prieto have made the analysis of categorizations central to their theories. In many versions of semiotics, moreover, the relations obtaining between categories are also stud-

ied, and sometimes, in structuralism, the categories themselves are derived from the relations. In the latter case, the elementary relations are taken to be oppositions, but in many semiotic theories relations like similarity, identity, and contiguity, often come to play an important role, though a less well-defined one. If instead the relations of superordination and subordination are seen as the most important, the elaboration of taxonomical systems will result, as is particularly the case in ethnolinguistics. Among the elementary relations, in particular if these are oppositions, some are often thought to precede the others, in a historical or in a systematic sense, or in both senses concurrently, as in Jakobsons phonological theory. Again, the elementary relations may be considered to form more complex systems, perhaps relations between relations, which are the condition of possibility of all possible meaning, as is for instance the case in the theories of Greimas and Lvi-Strauss. Finally, categorization systems are sometimes seen as being derived from our experience with things and to be in need of being reapplied to things whose meaning somehow exceeds them as is claimed in the very different epistemologies of Husserl, Piaget and, maybe, Lvi-Strauss. Knowing the category of a thing seems essential to all understanding. From the first, the Prague school was preoccupied with the very general categories of literarity, theatrality, pictorality, and so on, but then concentrated on the specific categories of the traditional genres (See Striedters essay in Vodika 1976); more recently, Todorov (1978) has remade the same itinerary. Hirsch (1967) and Gombrich (1960; 1963) have claimed that understand

ing of a work of art is only possible inside he given genre. On the other hand, Geertz (1983) has called for the elucidation of the most general categories of humanist experience, now re-emerging as metaphors in the social sciences. Categories of different levels of generality are, so it seems, worthy of investigation; and besides the categories of the works of arts and of other signs and meanings, the categories rendered by them have to be studied in particular in pictorial semiotics because, as Arnheim (1969) has recognized in his study of childrens drawings, pictorial concepts may well join other instances together than verbal equivalents would. The study of categories goes at least as far back as the ideologues. When Itard tried to teach language to Victor, the speechless boy found running around in the woods of Aveyron in 799, he wrote down a word on a paper and placed the corresponding object beside the word on the same paper (as seen in Truffauts wellknown film Le sauvage and described by Itard himself in Malson 964; see also Lane 1976). The cognitive psychologist Roger Brown (1958:3ff), who has pondered Itards experience, points to the difficulties in learning how to distinguish a book from a magazine using this method: while dateof-publication could possibly be the criterial attribute, it is certainly not accessible to immediate perception (p 11). No doubt it would be even more difficult to bring home the meaning of such complex notions as literature and art to our wild boy from Aveyron. More light is shed on the importance of categories by the philosopher Arthur Danto (1979:4 ff), who makes the interesting observation that, if an object exactly

identical to the Manhattan telephone directory for 978 is presented as a work of art, it will be absolutely essential to know if it is supposed to be a work of literature or a sculpture, before we complain about the exiguities of the plot, which will be a remark of some relevance only in the former case. This suggests that what the category may do for a work of art or any other object is to define its principles of relevance (Dantos rules of the genre). In fact, there are innumerable ways of writing a novel, like there are an infinite variety of pronunciations for each single phoneme of a language, but in order for something to be a novel, or the same phoneme, its variations must be variations along a limited number of predetermined dimensions. Narratology is presently looking for these dimensions in stories, like phonology sought them out in the sounds of language. A category, therefore, does not only serve to separate an object pertaining to it from the objects of other categories; it also informs us about the dimensions of variation permitted within the category. This is not to deny that there are also what Brown called quiet attributes, by which he meant properties which vary independently of the category but then again, these variations could very well be due to other categories, stemming from a different, overlapping segmentation of reality. Categories are usually thought to be defined by a set of sufficient and necessary properties. This has indeed been the presupposition underlying most studies of spontaneous categorization, from Itard to Lvi-Strauss and Piaget. Actually, categories of this kind are probably the outcome of the Galilean revolution of the sciences, or maybe of Greek mathematics and philoso

phy. Natural categories anyhow seem to be built up in a rather different way. There are semiotical as well as psychological and phenomenological evidence for this contention. The concept of dominant, fundamental to the Prague school (cf. Jakobson 963; Mukaovsk 974; Matejka & Titunik 1976), can be reinterpreted as a kind of category. According to Jakobsons (p 09 ff) well-know analysis, the communicative situation involves six factors: context or referent, sender, receiver, message, contact, and code, and as each one of these factors comes to the fore, a particular function of the message will be emphasized. The poetical function, which consists in emphasizing the message itself, or more exactly, the palpable aspects of the sign, is widely present outside of poetry, for instance in publicity and political propaganda, and will define poetry only to the extent that all other functions are found to be subordinated to it and hierarchically organized around it. Now, this means that the poetic function is a necessary but not a sufficient criterion of poetry; what is sufficient is the saliency of the necessary property. All or most of the functions must be present in any message, if I understand Jakobson correctly: but they must not predominate over the poetic function, if poetry is to result. To Jakobson and Mukaovsk, poetic language, standard language, and so on, are dominance concepts; but if I am not mistaken, language itself, as a reunion of the six or four factors, respectively, will be an ordinary logical concept. In the Chinese encyclopaedia described by Borges (1974), animals are classified according to various categories among which are: those belonging to the

emperor; embalmed animals; fabulous animals; wild dogs; animals painted with a very fine brush of camel hairs; and those which seem to be flies when observed from a distance. The psychologist Eleanor Rosch (1978) tells us a classification like this could never be conceived and used by any real human group, but the ethnolinguist Roy Ellen (1979:6), who again refers to Borges encyclopaedia but not to Roschs appreciation, notes that the arrangement of the categories has a familiar ring for an anthropologist. At least one of the traits that makes Borgess classification seem so peculiar at first can be discovered in the ethnography of any kitchen, for instance in the way the kitchen utensils are stored in order to be ready at hand. As an example, in one drawer are found the forks, knives, and spoons, maybe together with other small-size instruments like the cork-screw and the canopener; in a second drawer may be found all instruments made of wood, i.e. ladles and similar utensils; in a third one may be put bigger sized metal instruments, such as bread and carving knives, scissors, and whisks, the fact that many of these instruments have wooden handles here being of no avail to the classification; and lastly, in a fourth drawer, could be found all the instruments needed to bake a cake, whether of wood, metal, or plastic. Each category is here defined by a property situated on a new dimension (with the exception of the wood vs. metal distinction) each one of which is a quiet attribute of other categories, and salient for one category only. Although abstractly considered it seems confused, this arrangement is really very handy. So we cannot judge the Chinese Encyclopaedia either before knowing its purpose. But Lifeworld concepts may still be
0

more unlike what logicians like us to believe, as suggested not only by Husserls, Gurwitschs and Schtzs theories of typification, but also by Eleanor Roschs experiments on natural categories. These categories, according to Rosch (1975d:l78), are not logical conjunctions of discrete criterial attributes but have an internal organization consisting in a core meaning, identified with the prototype, the clearest cases, and the best examples, which are then surrounded by other members of decreasing degrees of membership in the category. The prototypes of a given category are said to be maximally similar to each other and maximally different from the prototypes of other categories, but the limits between the categories are thought to be fuzzy, running continuously into each other.34 It is not quite clear if the prototypical core of the category consists of a collection of concrete cases, the best examples mentioned above, or if it should rather be considered as the abstract representation of a category (Rosch & Mervis 1975:575). A typical bird, e. g. a robin, will lay eggs, have wings and feathers, but not all birds possess all these properties, and some animals having one or other of these properties are not birds (e. g. a bat) but thanks to the high correlational structure of the world, the property of having wings, for instance, tends to co-occur with the property of having feathers rather than furs (Rosch et al. 1976a:429). There is ample evidence for Roschs theory, most of it stemming from her experiments and those of her collaborators. Persons asked to choose the typical exam34 As I pointed out already in Sonesson 989a, I dont think the category boundaries can really be fuzzy, or metaphors become impossible. More will be said about this below.

ples of a category will find the question meaningful and tend to select the same items (Rosch 1973). Labov asked people to classify a series of cuplike objects: they found more or less typical ones, besides bowls, glasses, and vases (cited in Glass et al. 1979; 331 f). In fact, the same type of experiment was made much earlier, in the case of chairs and armchairs, by Gipper (1959). Another linguist, Lakoff (1972) has noted that languages are full of hedges, i.e. words whose job it is to make things fuzzier or less fuzzy (p 195), or to indicate the degree of membership of an object in a category: e. g. phrases like sort of, kind of, loosely speaking, essentially, par excellence, in a real sense, in a manner of speaking, technically, virtually, nominally, etc. (cf. the adjuster-words of Austin 1962b:73). Rosch (1975b) asked experimental subjects to place words in hedged frames and found that the resulting propositions could not be inverted: only prototypes can be cognitive reference points (Cf. Tversky 1977). When asked to place objects on a table in relation to a centrally fixed object, subjects could also comply meaningfully with the task. In other experiments, it was found that propositions relating a category name and a prototypical example could be judged to be true or false more rapidly than a proposition containing an untypical instance, and that priming with the category name speeded responses of same to the good example members but delayed the responses for the untypical members (Cf. Rosch 975 a, b, c, d; 978; Glass et al. 1979; 333 ff). One may of course still doubt that all categories are of this form, but it certainly seems a natural way of building categories. The illusion as to the existence of


Fig. . Details of a Le Bry engraving commented on by Bucher (cf. text)

criterial attributes for categories derives, according to Rosch & Mervis (1975:582), from the tendency to consider only the most prototypical members of a category, which in fact can be shown to have many attributes in common. It should be noted that what is here described as a delusion is the same procedure which Husserl and Gurwitsch have termed idealization, exemplified by the invention of geometry out of the spirit of land-surveying. In fact, logical categories would seem to be necessary for rational thinking, and it is the paradox of semiotics as well as of the humanities and the social sciences in general that, being knowledge about knowledge, as Prieto puts it (Cf. Lecture 2), the knowledge that they produce must be logical, whereas the

knowledge which they are about will inevitably most of the time be prototypical. The difficulty of maintaining this distinction may be illustrated by examples from Buchers (1977) study of Le Brys New world engravings (cf. Sonesson 1989a). These engravings, Bucher rightly observes, inform us less about the customs of cannibals and other savages than about the ideology of Le Bry himself and his compatriots. Possibly, Buchers study tells us more about her ideology than about that of Le Bry, thus cannibalizing Le Brys mental cannibalization of the cannibals. Anyhow, at the beginning of her book, Bucher (1977:46, 58 ff), using one of the engravings which shows a group of cannibals roasting human body parts and


distributing them to eat, forms an interesting pictorial concept, named already in the title: la sauvage aux seins pendants, which then later turns out to be of the prototypical kind. Most of the natives have been given bodies of perfectly classical proportions, but the motive that is of interest to Bucher constitutes an exception. Among the four women present at the cannibalistic orgy, only one is, according to Buchers description, a young person with a classical body form, firm bosom, long hair combed flatly over the head and joined in a braid at the back, and wearing necklaces and bracelets, whereas the other three are old hags deprived of all physical charms, their breasts hanging down vertically, while their foreheads are full of wrinkles, the disorderly tufts of their hair fall down on their shoulders, and they lack all adornments. In the engraving considered (here reproduced as Fig. 27), this difference in appearance is said to be correlated with another one: the young woman devours a human body part, whereas the three old hags are just licking their fingers voraciously. And then Bucher goes on to invent a complex system of classification incorporating Lvi-Strausss culinary triangle, which distributes different body parts and body fluids to the men, the young women, the old hags, and the children. Bucher (1977:52) tells us she only wants to retain two traits of the linguistic model: the oppositions and the notion of pertinence. As to the latter, it is clear that she must be applying some principle of pertinence but less clear which it is: why, one may ask, are not the men also separated into those which have a classical body form and those who are hairy all over their

bodies. Buchers answer, I think, would be that, contrary to the two types of men, the two types of women can be correlated with different cannibalistic foodstuff, which is a separate code, and with the presence and absence of adornment, which is also supposed to be a separate code. What is arbitrary, in the end, is the delimitation of codes, which is never justified (Cf. Bucher 1977:37 ff). Interestingly, Buchers correlations are really contiguities between elements in the picture, more exactly between the expressions of the signs, perhaps most similar in that respect to the attributes placed in contiguity to saints and other iconographically prominent figures. But the contents of the signs are here also of the kind that might probably easily be found in contiguity in ordinary life, if we admit the cannibalistic vraisemblable so the contiguity in question could simply be an iconical sign of real world contiguity. Admittedly, to the extent that other combinations are possible, the contiguity could perhaps be connotatively overdetermined (Cf. Lecture 5.). Bucher apparently supposes these indexicalities to be incorporated into the sign type, so that their meanings are carried over into pictures where there is no longer any correlation with foodstuff, adornments, and so on similar to LviStrausss (1979:44ff; 1983) Wagner interpretation, according to which the contiguities of musical themes and events create a system of cross-references which continues to cumulate meaning as events and themes separate. In fact, all the distinctive features isolated by Bucher turn out to be of this provisional kind. Before we go on to consider this strange dialectic, we shall however have a look of our own at the Le


Bry engraving which is central to Buchers argument. The basis of Buchers generalizations seems to be rather feeble: true enough, only three of the women are seen to lick their fingers, but one of the finger-licking ladies also holds a leg in one arm, which she cannot be eating, at least for the moment, because she is holding it at arms length. What, then, are the two other hags doing with their free arms? In one case, the second arm disappears behind something, which is probably the smoke from the fire; in the other case, the whole figure is cut by the picture frame, and there is no contiguity cue for the position of the non-visible arm. Looked at without Bucherian preconceptions, the four women of the engraving, seem to be different variations on a common type, which is perhaps Le Brys pictorial concept of woman. These variations are partly due to contextual constraints, caused by the way the women are placed in relation to each other and to other objects present in the picture: only one woman, the young one according to Bucher, is almost entirely visible, while the others hide each other from view, or are hidden by the fire and the grill or cut by the picture frame. There is also a stylistic variation, Le Bry choosing to present his dames from varying perspectives and in different corporeal positions. Bucher would have to show that there is variability not explainable by contextual or stylistic variation or demonstrate that what we have taken to be free, stylistic, variation can be reduced to one that is constrained by subcategories to the pictorial concept woman. Only the supposedly young woman is seen in full, and what can be apprehended of the second womans legs seems as

classical as the body of the woman in the foreground. It is impossible to establish if the latter woman has wrinkles on her forehead, because she is seen from the side, and there is nothing to indicate if the facial features are those of a young person or an old one. Because of the relative positions of the women, only two of the bosoms can be observed in the picture, one of which is clearly hanging down slackly. As to the difference in hairdo, it seems to be purely imaginary: what difference there is can be entirely explained from the varying perspectives, and the long hair of the woman in the front is clearly seen to fall freely on her back, instead of being joined in a braid, as Bucher tells us it should, and as it is seen to do in another engraving (fig. 3, in Buchers book). As to the bony chest, said to be particularly visible on the woman to the left, it can at least not be seen on the version of the engraving reproduced in Buchers book. Lest it should be concluded that Buchers theory is now completely disproved, it should be noted that details not visible on the reproduction in Buchers book may have been so in the original engraving she studied, and that the varying perspectives and positions may themselves have been chosen in order to suggest a subcategorization of the pictorial concept woman. Suppose, however, that Bucher is right in thinking that young woman and old hag are subcategories to the pictorial concept woman in Le Brys engravings. Being found in contiguity to different cannibalistic foodstuff in just one engraving or, in the case of the old hag, three times on one engraving the pictorial subconcepts are supposed to acquire further meanings which follow them through the rest of the engravings a number of


occurrences which would be of no value to a linguistic distributionalist. What is more disturbing is that even the features thought to characterize the pictorial concept old hag start to vary uncontrollably in the following engravings: the woman presenting fruits to the conquerors (Buchers fig. 8) is richly adorned, and the woman outside the house in the background, who, according to Bucher (p33) is the same woman in an earlier phase of her labour, has a bosom as firm as the woman in the foreground of the first engraving and like her, she is pictured in a frontal view. In the engravings showing some pagan idols, the only feature retained is the sagging bosom (Buchers fig. 6-7.), but if this sole feature defines the pictorial concept in question, it is not a distinctive feature, as Bucher repeatedly tells us, but a sign of its own, and the other features mentioned would be simply redundant. Even at the beginning of the book, a more correct term for the kind of entity envisaged by Bucher would be complex sign: the hanging bosom, sometimes together with the wrinkles on the forehead (p 62; 98 f) stands for old age and decrepitude, which then together with lack of adornment, etc., comes to designate the complex meaning sought by Bucher. However, the sagging breasts suddenly acquire a signification independent of age, when they appear on young, rather classical, bodies with no sign of boniness (Buchers fig. 10-13 and 15-16). It also seems relevant to ask to what degree the breasts must be directed to the ground in order to be considered to be sagging. While a bosom in one or another shape is present in most of the cases, it does not always hang down, and rarely to an appreciable degree, and it is almost

never bony. Most of the women are not old; yet decrepitude may well be the only common factor in some of the cases. Very rarely do the women lack all adornment and have disorderly tufts of hair failing over their shoulders.35 Bucher talks about a system of transformations and acknowledges some of the differences we have noted, but she hardly realizes the consequences: if la femme sauvage aux seins pendants is a visual sign, equivalent to Ecos iconical sema (p 198f), it is certainly not easy to identify, contrary to what Bucher maintains, because its features may change in an arbitrary manner. As in the chain-concepts of children studied by Vygotsky and the family concepts of Wittgenstein, each instance of the concept has at least one trait in common with each other instance, but no
35 In fig. 10, the bosom hardly points downwards and its direction can be fully explained by the forward lean of the woman; the same goes for the upper woman in fig. 11-12. The second woman in fig. 11-12 and the women in fig. 15 and 6 have breasts which are literally vertical, and even more so the women in fig. 21 and 22. The breasts of the girl in fig. 13 hardly seem to sag, whereas the breasts of the woman in fig. 14 are hidden by her arms but would hang down on any woman given her position. Buchers affirmation (p 122) that the woman in fig. 14 is similar in other respects to the cannibal women of the first engraving has no foundation whatsoever: the hair is more disorderly, the body more muscular, and so on. Since the breasts are hidden, the only trait uniting this woman with the cannibals of the first engraving is her rather monstrous appearance. The breasts of the women in fig. 18 and 21, far from being bony, appear to be pulled down by their own weight (in one case, the position may also be a contributory factor); but in fig. 11-12, 15, 16 and 22, the rather flat bosom possibly only serves to indicate the presence of breasts, using the perspective from which their shape is most easily identified. In fig. 18, the womans perfect body is, according to Bucher (p 177) disfigured by a sagging bosom (one suspects the readers of pornographic magazines would judge otherwise) so that the terms of the original opposition in the cannibalistic orgy, the statue like body and the hanging breasts, are now joined in one figure. 

trait is common to all instances. However, Buchers insistence on the old hag as she first appears in the cannibalistic orgy indicates that this family concept, like many other family concepts studied by Rosch & Mervis (1975), has a central prototype, to which all the other instances may be referred for the determination of degree of membership. The prototype concept which Bucher derives from the series of engravings by Le Bry could of course coincide with the prototype concept once used by Le Bry and his contemporaries but that would be a mere coincidence, and there is really no way of knowing if the concept was like this or quite different. Prototypes, idealtypes, and antitypes We can again take Rosch as our guide, when we proceed to investigate what the elementary forms of real pictorial concepts would be like. Simple configurations in the sense of the Gestalt school were used by Rosch (1973) in some of her prototype experiments, e.g. the square, the circle, and the triangle; and verticals, horizontals, and diagonals, were found by Rosch (1975 b) to function like cognitive reference points. Some squares are more square than others, i. e. better examples, prototypes, and can thus be used in the comparison of other squares and square-like shapes. This, as Rosch (1975b:192) notes, is what Wertheimer called Prgnanz, and, in fact, the Leipzig school of Ganzheitspsychologie termed the same phenomenon Ausgeprgtheitsgrad der Gestalt (Sander & Volkelt 1962:78). What von Ehrenfels (cf. Weinhandl 1960:44ff), who first discovered the Gestaltqualitten, described as the relative purity of the configuration (Gestaltreinheit) is perhaps the same thing.

Rosch (1975 b; 1978) seems to identify her prototype concept with what Weber has called an idealtype. According to von Scheltings study (1922; 1934) of Webers concept, the term is used by Weber for many different phenomena, so that it will be necessary for us to obtain an idealtypical (or maybe rather a prototypical) concept of the idealtype if we are to be able to make the necessary comparison with the prototype (also cf. Aron 938; Nyman 95; Weber 1964). On the face of it, however, there seems to be many differences between the Weberian idealtype and what Rosch terms a prototype. To begin with, Weber insists that the idealtype is the result of a very artificial type of scientific reasoning, whereas Rosch pretends to have found the internal organization of natural categories. But there are other differences, more directly pertaining to the nature of the category itself. To begin with, the idealtype is said to be unreal, i. e. not found as such in empirical reality, but the prototype is at least sometimes identified with concrete cases, i. e. the best examples, and is thought to be similar to an image. As described by von Schelting and Nyman alike, the first step in the production of an idealtype consists in extracting a salient property from an empirical phenomenon and then exaggerating it beyond what is empirically possible: in that way, the rationality of economic man is exaggerated by Weber. Against this, the prototype, it will be remembered, uses the extreme cases furnished by reality itself. In further steps, hypothetical consequences are then derived from the exaggerated feature, these are confronted with the events actually taking place, it is concluded that other factors must also have been at work, and these factors are searched for. If we


generalize this description beyond the causal links it suggests, it is reminiscent of the expected co-occurrence of features mentioned by Rosch, as for instance wings and feathers going together. Again, the last steps, rather than characterizing the idealtype, pertain to its possible use as a cognitive reference point. However, in Webers view, it seems, an idealtype is unreal, not just because it exaggerates a feature beyond empirical feasibility, but also because it may be contradictory. Von Schelting (1922:729ff) criticizes Weber because he takes notions like the Middle Ages or primitive Christianity (Urchristentum) to be idealtypes, while von Schelting considers them to be real historical entities. However he fails to note that even so, they may be conceived idealtypically, and Weber actually tells us how: by joining together salient features of, for instance, the Middle Ages, which in actual fact could never be found together, for instance one trait from the beginning of the Middle Ages, and one from the end. Thus, the idealtype is different from the prototype, at least because it exaggerates a feature beyond what can be found in reality, and because it can join together features that were never so found in actual fact. In the opinion of Nyman (1951:179), Wlfflins five conceptual pairs are idealtypes in this sense. However, the psychologist Sander (in Sander & Volkelt 1962:383ff) identifies Wlfflins distinction between classical and baroque with prototype coincidence, and small deviations from the prototype, respectively. On the first interpretation, the concept is constructed by the scholar. On the second interpretation, on the other hand, it is also present in the experience of the connoisseur. For the time

being, we shall only note that perhaps idealtypes should not be thought of as unreal in an absolute sense: we know, for instance, that a drawing expressing in an exaggerated way the features typical of babyness causes more clear-cut reactions of parental tenderness than does the shape of real babies heads (Hckstedt 1965), and many animals have been shown by ethologists to react in the same way. Now, since the drawing is a real object of the Lifeworld, viz. of the world of psychological experiments, the once unreal shape has become real; and even the contradictory idealtype of the Middle Ages may come into being, if Hollywood gets its way. When Merleau-Ponty (1960:51) compares the structural whole according to Saussure to celle des lments dune vote qui sepaulent lun lautre, the comparison is really very fitting; indeed, wouldbe structuralists sometimes prefer to put up an armature on which to hinge the vault before even beginning to lay the stones. As we saw in the earlier discussion of LviStrauss mask analysis, something else may often be behind what structuralists take to be structure. Often, I will suggest, this is the antitype, which is the opposite of the prototype (or perhaps of the corresponding idealtype). What we shall term the LviStraussian proportionality is a relation, more exactly a similarity, between two other relations, which themselves seem to be oppositions, normally contradictions. The Oedipus myth, for instance, is said to be based on such a proportionality (LviStrauss 1958:237ff): overvalued kinship relates to undervalued kinship as the negation of the terrestrial origin of man relates to the persistence of this terrestrial origin. Much


later LviStrauss (interview with Bellour, in Bellour & Clment 1979:158 f) says that while, in Le cru et le cuit, he wanted to show how mythical thinking makes use of elementary sense qualities as some kind of symbolic tokens which permit the formulation of certain logical propositions, he then in the later volumes of Mythologiques, when more myths were integrated into the system, came to take an interest not so much in the relations between the terms as in those between the relations. This is actually true even of the Oedipus analysis. Even in these cases LviStrauss seems to think that some proposition is expressed by the formula in the Oedipus myth, for instance the incomprehensibility of the fact that we have our origin in man and woman, and not in the Earth, and that we come from two elements, not just one (1958:237 ff). Later when the Oedipus myth and the Grail myth are declared to be the two principal types of myths, both types are considered to pose the problem of communication and fail to resolve it, the first because of excessive communication, whose extreme case is incest, and the second because of a lack of communication, as exemplified by the question never being asked (LviStrauss 1983:301 ff; 1979). Thus, the myth is less a proposition than a whole piece of reasoning; in Peircean terms, it is an argument rather than a dicent, and, as Gardner (1973b) says about works of art, an act of problem-solving. But in both types of myths, we are told that the problem remains unresolved, because no mediation can be obtained between the terms. Though a disciple of Greimas, Floch finds proportionality in the pictures he analyses, as we shall see later (Lecture 5); however, these proportionalities, unlike

those of LviStrausss own mask analyses, or Buchers similar work, are located on the most directly perceivable level of the pictures, at least in part, and thus do not exclude the existence of a semiotic square (see below) on a deeper level of analysis. Also, unlike at least some of LviStrausss proportionalities, those of Floch are read more as propositions than as arguments. Even to LviStrauss himself, the proportionality is apparently only one alternative form of possible meanings: for triads and analogical models are also cited in the Bellour interview (Bellour & Clment 1979:181), of which the former was used by Lvi Strauss (1966) himself for his culinary triangle, which has thus nothing to do with Jakobsons phonological one, which is no precondition for the perception of meaning but a record of precedence relations (Cf. Sonesson 1989a; 1.3.4.). Greimas, on the other hand, really seems to think that all meaning is somehow reducible to the semiotic square: two terms in a contrary opposition, to which are added their contradictory terms organized in such a manner that each of the first two terms will imply (or be implied by) the contradictory term of its contrary term (Cf. Greimas 1970:136ff). Thus, the terms life and death will be implied by nonlife and non-death respectively. Between the two contrary terms there is a complex term which unites them, and between their contradictory terms there is a neutral term which, since it units the subcontraries, goes beyond both contraries. Two operations can also be defined on the square: the conjunction, which brings two terms together, and the disjunction, which serves to separate terms. There is an extensive literature about


the semiotic square, which will not be discussed in the present context. The square, like the proportionality, undoubtedly has some heuristic value. However, it is not clear why we should suppose all meanings to take these forms. Pending further investigation we will consider the square and the proportionality to be two possible forms of higherorder semiotical organization among many others. A third model, the semantic differential, designed by Osgood, has also been used in semiotics, mainly in linguistics, but also in some experimental investigations in architectural semiotics by Krampen (1979), and in the pictorial semiotics of Espe (983 a, b) and Lindekens (1971; 1976). Between two opposite terms, which are often contraries, a scale of numbered units is indicated; since the qualities at the poles are not of the kind that can be measured and weighted, the numbers can only stand for physiognomic quantities. Often three scales will be put together to form a three dimensional space, in which words or concepts are located on different dimensions at once. The literature about the semantic differential is even more extensive than the one about the semiotic square, so we prefer to be very brief. Suffice it to say that in one way, this model is less reductionist than the other two: as in the Lifeworld, properties may be possessed to different degrees; but its employment of numbers is metaphorical, since real quantities are made to stand for physiognomic ones. Any one of these scales, also when the numbers are even, must have a middle, a point of rest between the two opposite terms. Thus, the middle is not a result of the numbers as such. This brings us back to the problem of mediation.

We have seen that Greimas model includes both neutral and complex terms. As far as I have been able to determine, the mediations according to LviStrauss are rather like the complex terms, but they are not outright contradictions like the paradoxes of Taoism, the Freudean Urworte, or some cases of surrealist poetry. The two simplest ways in which mediation may obtain without there resulting any logical contradiction between the terms are exemplified by LviStrauss (1983:181 ff) analysis of la bonne conseillre in the Kwakiutl myths, who is said to mediate between the earth and the underworld, being either a human figure half buried in the earth, or a mouse running up and down between the worlds. Here, properties, which are contradictory as such, are possessed in one case by different parts of the same object, in the other case by different temporal slices of the object. There is no contradiction, since the attributes taken from different poles of the same dimension pertain to separate proper parts and temporal noemata of the object, respectively. But other cases are more complex. Thrlemann (1982:54ff), in his analysis of Klees Pflanzen-Analytisches, distinguishes two kinds of schma de transition, without however referring to LviStrauss: one of them, graduation, exemplified by a saturated and a nonsaturated colour being mediated by a halfsaturated one, brings to mind the middle term of the semantic differential; if anything, this will be a neutral term. In the other transitional scheme, the mediating term shares a property in one dimension with one of the opposite terms and in another dimension with the second term. For instance, the geometrically cut trees of the Versailles garden share their


Human beings

Europeans Indians

Human beings

Europeans Indians

Human beings Monsters/Animals

Monsters Animals Visual experience Animals Phase I Phase II

Fig. . The categorization of monsters (see text). regular shape with the building and their material with the freely growing woods. Again, in Klees painting the mediational term between the circle and the rectangle would be the square, which is dimensionally symmetrical like the circle but not the rectangle, while being straight like the rectangle rather than round like the circle. In these cases there are really two scales, which turn out to vary independently. In this case, it is surprising that the third term is thought to mediate between the other two. This meaning can only result if the scales are expected to vary concomitantly. For example, it is clear that the wood, which is made up of living material having irregular shape, is a typical instance of Nature, while the building, with its regular shapes and inanimate material (if made of stone), instantiates Culture. Similarly Zurlo (1976:436ff), in his excellent analysis of the Western using Lvi-Straussian mediation, suggest there is a primary opposition between the Indian and the Immigrant, which dominates a series of secondary axes, concerned with behaviour, clothing, arms, ways of using arms, etc. A person having the role of a mediator in the Western could thus take on the value associated with an Indian in his clothing, but behave in other ways like an Immigrant. Hence, there is no contradiction between the attributes of the scales concerned, but only between the associated values of the dominating scale. The precondition for the emergence of the mediational term is clearly the existence of two multidimensionally characterized opposite terms: the typical Indian, as well as the typical Immigrant, will be different on a number of scales. The similarity between these opposite terms and the prototypes of Rosch, recognized by their high correlational structure, should be obvious: like the wings and feathers of the birds, the typical behaviour, clothing, and arms of the Immigrant tend to cooccur, and to contrast maximally with those of the Indian. What the mediational term, which we may now call the antitype, contradicts, is the expectancies stemming from these correlational structures. Interestingly, LviStrauss (1984:91) points out that certain deviant animals are particularly apt to serve as mediational terms. Bulmer (1979:58) observes that animals that are typical representatives of their groups, as well as the anomalous ones, are ritually marked, both being equally good to think. Thus, prototypes and antitypes are treated alike which should not be surprising, since antitypes may well become
0

prototypes of new categories, once less perfect versions crystallize around them. But as long as the antitype persists as such, it will be hierarchically subordinated to the corresponding prototype, or to the two prototypes between which it has been developed. Given three elements, not just any distribution of the roles of prototype, opposing prototype, and antitype, is feasible, partly because some prototypes, or rather the tendencies behind them, are innate (Cf. Rosch 1973; 1975 c; etc.) and partly because any culture will form its own prototype hierarchy. For instance, the square mediating between the circle and the rectangle, as suggested by Thrlemann, seems highly improbable, since the square and the circle appear to constitute the best forms, thus being highly prototypical, while the rectangle may be less so. It is arguable that the context of Klees painting changes this hierarchy. Perhaps the antitype may become independent, forming an autonomous prototype, or at least an opposing prototype. According to Bucher (1977:193ff), la femme sauvage aux seins pendants, once created to signify an anomalous type of Indian woman, is then used to stand for any female Indian. If we accept Buchers premise, the standard pictorial concept of a woman in Le Brys engravings is the classical anatomy study, some exotic details of costume, arms, ornaments, and so on being added to express the subcategory of Indian woman (p 42ff). But from the class of Indian women, a group is separated out through being designated by the pictorial concept formed by a non-classical body shape with hanging bosom. There is some indication (p 50 ff; 170 f) that la sauvage aux seins pendants

should be considered to combine human and animal traits, thus being monstrous. To the Catholics, Nature and Culture are acceptable; only their hybrid form is not. But to the Protestants, all of Nature is degraded because of the Fall. When the Protestants set out to colonize the New World, they assimilated all Indians to the monstrous type (p 154 f; 196 f). All these facts are of doubtful truth and of doubtful relevance, but let us accept them here at their face value; we are interested in the resulting conceptual organization. First of all, we assist at a conceptual reshuffling: what is biologically two human races is in the first phase reinterpreted as three separate categories, two of which are subsumed under the notion of Humankind; and then, when in the second phase the segmentation regains its motivation, the visually noticeable differences have been exchanged for new, arbitrary traits. In fact, if we also take account of the opposing category, animality, even the second phase amounts to a different segmentation. In the real world of our experience, Europeans and Indians are human beings and are as such opposed together to Animals. Later on, monsters are introduced as a third category, opposed to humans and animals. The monsters, among which is found the savage with a hanging bosom, will be an antitype in Phase I if, on some of the scales dominated by Nature vs. Culture, they occupy the position expected of animals, while on other scales occupying the position considered normal for human beings. In Phase II, however, when Nature is itself considered to be degraded, monsters become prototypical, as they best express the monstrosity of Nature in her state of decadence (Cf. Fig. 28). Not only would


this explain the presence of the savage with her sagging bosom as a representative of all Indian women (and men!), but also the abundance of hybrid animals (Cf. Bucher 1977:170f). The problem is that monstrosity is itself defined as the illicit combination of traits from human beings and animals, Culture and Nature. But perhaps it is against a background of a preFall animal prototype that the present one appears to be monstrous. If this is the case, the antitype of Phase  has now become the opposing prototype to prototypical Humankind, as exemplified in Europeans! Two kinds of wholes: Structure and configuration It was suggested above that there are two kinds of indexical relationships: contiguity, which is a relation between independent objects (or substances, as Gibson would have said), and factorality, which is a relation which is internal to a whole, that is, a relation between parts and the of which they are parts. The notion of whole is itself ambiguous. Different notions of wholeness, viz. structure and configuration, as conceived by structural linguistics and Gestalt psychology respectively, are often confused, so for instance by MerleauPonty (1960; 1969; and in Bastide 1962). As early as 947, Mukaovsk (1974:7 ff) insisted on the importance of distinguishing the structure from the kind of wholes conceived by holism, observing that while a structural whole results from the mutual relations between its components, including negative ones, a holistic whole is primarily a delimitation made in the field, a setting up of borders, from which an inner differentiation may later ensue.36
36 In another context, Mukaovsk

Fig. 29a: A configuration, which in 29b is put under the influence of a structure It will be useful to start from the difference between the interplay of relations typical of the structure and the demarcational virtues of the configuration. In both cases, the whole is really something more than its parts, as the Gestaltist saying goes, but in the structure it is the network of relations which is central, and the elements connected by the relations will thus appear to be more distinct (though sometimes identical) to each other; in the configuration, however, the general idea of wholeness and of all the elements belonging together predominates, and the elements themselves are only secondarily apprehended as separate parts. Arnheim (1969:60ff) who, like many followers of the Gestalt school, sometimes uses structure in the sense given here to configuration, tells us the square in fig. 9a will seem somewhat less straight because of the influence from the reclining V in which it has been inscribed; this, I submit, is a typical configurational effect. But when a second square is added, as in fig. 29b, the relationship between the two squares will stand out, creating a structural effect. Another way of obtaining a structure that more decisively destroys the
(1974:20ff) suggests a more complex, but, it seems to me, less coherent distinction between the configuration as described by Gestalt psychology, the structure, the context, and the composition. 

configuration, would be to apply a ruler to the borders of the square, thus introducing a continuous series of relations between points on the ruler and points on the contours of the square. In fig. 29b, the structure is only possible because of a similarity of position and overall configuration of the two squares; secondary differences as in the oppositions could be introduced. This is exactly the mechanism of the paired drawings of the Western lady and the Vietnamese (Fig. 30): here, the red areas are lost in their respective configurations and modified by them (that, they are resemanticized) to stand in one case for painted lips and a lipstick, in the other for blood stains. When the two faces are presented side by side, a structure results thanks to the similarity in the colour, the localization, and the directional axes of the two areas. From the interaction of the configuration and the structure, complex meanings may be derived. Similarly, Groupe (1980:267f) tells us, the waves and Mount Fuji in Hokusais The Wave (Fig. 31) are seen as different when they are interpreted as such, but on another isotopy they are identified because of the similarity of their triangular shape, both with the point turned upwards, and of their colour, which is blue stained with white spots. It is of course possible to perceive this structure, but, contrary to the one present in the two womens faces, it is deeply embedded into configurations. Gestaltqualitten were supposedly discovered by Christian von Ehrenfels in 1890 (reprinted in Weinhandl 1960:11 ff) and investigated by him in the particular example of melodies. A melody may be transposed without any of its elements, the notes, being constant. To von Ehrenfels and

Fig. 30. Front and back of the review Bohemia, by Fremez. his immediate followers in the Graz school (Meinong, Bernussi, Witasek), the configurational qualities are simply added to the elementary sensations, which are considered to be their foundational layer (Cf. Weinhandl 1960; Gurwitsch 1957:54ff).37 According to Khler, Wertheimer, Koffka, and others members of the Berlin school, as well as Arnheim, Gurwitsch, and Merleau-Ponty, the configuration is what is immediately given, whereas the presumed elementary sensations have to be constructed out of this whole on a posterior level of abstraction. To the Berlin school, including Arnheim, the biological predetermination of perception thought to derive from electromagnetical brain fields, is considered proven by the priority of configurational qualities. This issue is irrelevant to Gurwitsch, who however takes configurational priority to demonstrate the presence of intrinsic principles of organization in the field of perception. Besides such criteria of a configuration as demarcation and closure, Khler introduced over-summativity (discussed by Rausch and Wellek in Weinhandl 1960:334 ff, 384 ff). If neither the part taken away nor the part remaining
37 Indeed, one of the ideologues, Daube already in 805 recognized that one kind of whole must be apprehended prior to the perception of its parts that is, it is rather the parts which are founded on the whole. 

Fig. . Mount Fujiyama, by Hokusai

changes because of the separation, we will have a sum, not a configuration. But it is rare to hear of any sums in the world of Gestalt psychology. It has usually been overlooked that von Ehrenfels (in Weinhandl 1960:29 ff) establishes a distinction between what he takes to be two kinds of configurational qualities: those which, like the melody and the square, can be directly perceived; and relations, like the similarities and differences discovered between two notes, which depend on our own initiative for their existence. Not only does this suggest that von Ehrenfels favoured a Berlin school interpretation of melodies and squares, as early as 1890, but it opposes these configurations to examples of what we have called structures. Generalizing the other term, Piaget (1972a:l 37f; 1972b:47) tells us there are two types of structures, or rather two extreme cases with many intermediary variants: the perceptual Gestalts, which are non-additive and non-reversible, and the operative structures of intelligence, elsewhere termed schemes, which are additive and based on two kinds of reversibility, i. e. negation and reciprocity. Though all psychological structures define laws for

the whole not present in the parts, all are not non-additive, Piaget assures us, contradicting the presumption of Wertheimer and Khler. If three elements are presented in a row, the middle element will be perceived to be larger than the first but smaller than the third, and this clearly excludes additivity and the conservation of quantities. On the other hand, classification, seriation, and natural numbers are additive, as the child comes to learn as he goes through the different stages discovered in Piagets experimental tasks. Unfortunately, there are problems with these observations. First, if non-additivity is taken to be the same as non-summativity, as suggested by the tradition, it is not true that natural numbers and classifications are nonadditive, since their parts would not only change but completely cease to exist if the other elements where taken away. Secondly, if conservation in general is supposed to be impossible in configurations, then how is the melody, usually given as a typical case of a configuration, susceptible of being transposed? The first problem may be easily resolved. It is not in the structure itself, the whole which interdefines the elements, that


these elements are non-summative, but in the combinations of elements which can be derived from the structure; not in the repertory of potentialities, the paradigm in Hjelmslevs terminology, but in the realizations of this system, in the syntagm. To choose a straight-forward example, the numbers owe all of their identity to their relations to other numbers in the potentially infinite series of numbers, as has long since been recognized in the philosophy of mathematics. However, in an arithmetical example, which may be perceived or only conceived, at least ideally, the numbers do not influence each other, but  is separately conceived, and so is , before being added to 3. In the case of a configuration, however, there is no clear-cut distinction between the potential whole and its realizations: if the perceived figure fails to coincide with the good form, it will be unconsciously completed or else considered defective or at least less perfect. This, I submit, is because the general idea of wholeness predominates over the network of relationships. We are thus brought back to the problem of transposability, which supposes some conservation in Piagets sense, also necessary to a form taking on varying substances, in the sense of Hjelmslev. Bartlett (1958:38ff), though not directly concerned with our problem, gives a perfect example, again using numbers, but not in their mathematical function. Given the sets of numbers 34; 34; 43, the task is to complete the series, using the rules implied by the permutations. The most simple solution confines changes of position to first and last pairs only, thus terminating the series in the following way: ... ; 43; 34. Many people discovered this

or another more complex solution involving additional interchanges of the middle pair of numbers. But when the figure below (fig. 32) was presented to the same persons, not even those who had found the rules in the numerical example discovered that the rules were the same. Not only is there no transfer from the first example, as Bartlett (1958:39f) notes, but there is no transfer, transposition, or conservation of the structural relations of the design either probably because, as Bartlett (p 40) remarks, the design is seen as a whole! The whole to which Bartlett refers here is of course a configuration. The transpositions with which we have been concerned so far concern the relations separately but maybe Bartletts configuration (fig. 32) could be transposed, like a melody, as a whole. Now, it seems we tend to remember well-known melodies in terms of precise intervals, as for instance +5+4-3+-4, while new tunes are stored in the form of a general pattern, i. e. as +++- (Winner 1982:206f). We should be able to get something more out of this example, if we take account of the more subtle distinctions introduced by the third current of holistic psychology, the Leipzig school. As early as 906, Krueger (as cited by Wellek in Weinhandl, ed. 1960:385) criticizes the all too general use of the term Gestalt to designates all kinds of wholes and proposes a distinction between wholes distinctly moulded to a particular shape and wholes in a more general sense (Ganzheit). Emotions, as well as the experiences of small children, are non-configurational wholes. All wholes are oversummative, Wellek suggests, but only configurations are transposable. Again, we may wonder if the wholeness itself, i.


e. the atmosphere, could not be transposed. Other criteria are proposed by Volkelt (in Sander & Volkelt 1962:43ff): a typical configuration stands out from a background and is internally articulated (gegliedert), but other holistic properties may well be externally and internally diffuse (aussenund binnendiffus; p 41f). In his studies of childrens drawings, Volkelt came upon holistic properties, more obviously so than emotions, which are not configurational, for instance the closure and angularity of the cube (Fig. 33.). In this and similar drawings, non-configurational, holistic properties like angularity and closure have been transposed, contrary to Welleks suggestion; the more specific, inner and outer organization is of course not transposed. The newly-learned melody is, in that respect, a less extreme example. As we shall see, Volkelt and Sander recognize many degrees of demarcation (Absetziichkeit) and articulation (Gegliedertheit). The typical configuration occupies a middle position between diffuseness (Diffusitt) and dismemberment (Zerstcktheit), Volkelt (p 45) observes. Later, however, he claims these are two different scales, and Chaos is both diffuse and dismembered. If so, becoming

more diffuse, a percept does not have to result less dismembered, and vice-versa. Extreme diffuseness, Sander says (p 77), produces a non-configurational whole, the extreme case of dismemberment being the breaking up of a whole into many separate objects. But the multiplication of parts in a configuration will never lead to its dismemberment, because the multiplicity and the unity will grow simultaneously (p 45). If so, there clearly must be two scales, and a configuration requires a relatively low degree of dismemberment as well as diffuseness. The idea that unity augments with multiplicity and that Zerstcktheit will result from effacing order, suggests the second dimension is really concerned with holarchy, in Koestlers sense (Cf. next section). So a configuration should be highly holarchic and highly demarcated. Corresponding to inner and outer diffuseness, there is inner and outer demarcation: the latter, of which the most important form is the contour (p 44), precedes the former, at least in childrens drawings (p 70; p 40; p 289). Thus, there will be two scales of demarcation. Outer demarcation is the most important, at least in the sense that only a relatively high degree of outer demarcation will permit the development of inner demarcation. Again, the contour is said to have different degrees of distinctiveness (Grad der Abgehobenheit; possibly the same as the varying clearness of borders, Grenzklarheit; p 44) and different degrees of closure (Geschlossenheit oder Offenheit des Konturs; p 75 f). Putting all this together, a configuration must have a high degree of holarchy and demarcation, the latter resulting from different degrees of inner and outer demar

Fig. 32. Bartletts configuration

Fig. . The essential cube (according to Volkelt) cation, at least the latter of which follows from relatively high degrees of distinctiveness and closure of the contour. If so, not only are typical configurations like good forms prototype categories, but the concept of configuration is itself a prototype concept: a given shape may be more or less configurational. But the configuration is not really the extreme opposite pole to the structure, because it supposes inner differentiation and holarchy. It is in the non-configurational whole that the feeling of wholeness will completely gain the upperhand.38
38 More recently, Garner and his collaborators have investigated experimentally the nature of configurations, using a technique that Gurwitsch would have found very Graz school style: using different constellations of parentheses, they try to find out when these are seen as forming a whole (Cf. Garner 976; Pomerantz & Garner 973; Pomerantz, Sager & Stoever 1977). According to Garner (1976), there are four different types of interaction between the dimensions on which the features characterizing a unit of perception are located: integral, if redundant dimensions facilitate discrimination, i. e. the reaction time is shorter, and if selective attention to the dimensions is impossible; configural, when there is neither any facilitation with redundant dimensions, nor any possibility of selective attention; separable, if selective attention is possible, but there is no advantage to redundant dimensions; and separable asymmetric, if redundant dimensions facilitate discrimination,

If there is a series of constitutive oppositions, maybe we can organize them in a system following temporal and/or some other kind of precedence relations, like Jakobson (1942) did with the phonological distinctions of all the worlds languages. Jakobsons theory is really three different theories: about the childs language learning, the aphasics language loss, and the common traits of all languages. None of these systems of precedence relations necessarily coincides with the one implied by the grown-up persons perception of the unities of his own language, in particular if this is a system o tout se tient, as the structuralists like to cite Meillet. The same thing is true about the temporal precedence relations in the development of the childs drawing ability, hinted at by Lurat (1968; 1970; 1974), Gardner (1980), and Volkelt (in Sander & Volkelt 1962:197ff), and, in a particularly systematic fashion, by Olivier (1974). However, it is possible that in the case of drawing there is less difference between the childs system and the one used by adults in a particular
and selective attention is possible only to one of the dimensions. Garner speculates there may also be an asymmetric configural interaction, where no facilitation results from redundant dimensions, but one of the dimensions is susceptible of selective attention. Pomerantz et al. (1977) demonstrate that, at least in some cases, the discrimination of parts is facilitated inside configurations. Although promising, these investigations can still not be directly used in semiotic analysis. But they serve to suggest that features may be relevant to analysis, even if what appears to consciousness are configurations, not only, as the Graz school would have it, because the configurations can be explained from the interaction of the features, but because sometimes selective attention is possible to one or more of the dimensions, and sometimes the effect of redundancy shows that on some level of consciousness the features are attended to. Thus, there may really be constitutive, structural oppositions in absentia in pictures. 

culture, at least if they interpret rather than create works of art. While this idea must remain speculative here, there is some evidence for a parallelism between the childs drawing ability and the adults perceptual process, when the latter is slowed down by the method termed Aktualgenese by Sander. Extremely diminished outline figures are shown to the experimental subject in a dark room and are then gradually enlarged. The subjects draw what they see at each phase, and the experiment is terminated when the shape of the configuration has come to a rest. From a diffuse whole emerges a contour in the shape of a circle, which is later articulated, usually in some regular shape. The field inside the contour will continue to be diffuse for a long time, and is at first filled in with holistic, nonconfigurational properties like outspreadness (Sperrigkeit), which are then located in particular parts of the figure and configurationally developed (Sander & Volkelt 96:0 ff; cf. Arnheim 974:63 ff where reproductions from memory are made to yield similar, if less systematic results). It should be noted that these are a series of inclusive oppositions, like the ones in Jakobsons phonological system: the circle at first signifies all kinds of closure, but is then differentiated. Exactly in the same way, the child chooses to represent the closure of the cube as a circle, because as yet he only has undifferentiated closure-signs in his repertory (Cf. Sander & Volkelt 1962:198f; cf. p 331 ff). Parts and relations to wholes To find the parts of the whole, the configuration apparently would have to be treated as a structure. It has generally been supposed that this business is accomplished

by language. According to structuralist dogma, the world before language, like the experience of the newborn as described by William James, is a buzzing confusion with no particular organization. Saussure tells us the sign is created by a simultaneous cut through two amorphous masses, the one of sounds and the one of meanings. Humboldt, Sapir, and Whorf exemplified the differing ways in which the same experience may be organized by distinct languages, and Hjelmslev (1943:50; 1959:113), in a now famous chart, compared the way the ideas of a tree, of the configuration resulting from a combination of many trees, and of the materials obtained from trees, were treated in Danish, German, and French, whilst Eco (1968:77; 1976:141), employing the same chart, added Italian examples. This case may serve to illustrate the problem of categorization, i. e. of determining the dimensions on which two instances are permitted to differ while still pertaining to the same category (as discussed in the preceding section). But there is also the problem of segmentation (which could perhaps be illustrated by another classical example, the colour spectrum), i. e. the problem of establishing the limits between the instances of two categories in a continuous medium. Both these problems are well-known from the analysis of the linguistic expression plane, but they are equally relevant in the case of a picture. When Goodman tells us pictorial sign systems are dense, he seems to be rejecting the possibility of both our operations at once and when Barthes (1964b) argues the picture has to be anchored by a linguistic message, he directly translates the structuralist dogma from the perceptual world onto the picture, at least if we admit that


the profusion of meanings suggested by Barthes (p 31) in the end comes to the same thing as the lack of meaning closer to the linguists conception. In both cases, there is no organization, no definite structure in the visible world. If structuralist methods are applied to the comparison between languages, three dimensions of variations seem to emerge from Hjelmslevs chart, one of them taking three values. Adding a few more languages, we get the following (Fig. 34). There is certainly no suggestion of buzzing confusion here; rather, inside a common continuum, variations along different dimensions are given varying weight. Arguments for linguistic relativism tend to oppose physical reality and the content form of language, but that is to forget that between them, there is the Lifeworld, the world of lived experience, which may derive its meanings from many sources apart from language. As Prieto (1975b: 95 ff) rightly observes, in the Panzani publicity studied by Barthes it is really the text, not the picture, which will be deprived of meaning if isolated; and it should be added

that the kind of meanings Barthes finds in the picture, are, contrary to what his theory pretends, neither described by lexical items in the French language nor designated in any other way by the text accompanying the picture rather, they are, as we will see (Lecture 5), units of Lifeworld experience. Even if the Lifeworld is itself organized, language, or any other semiotical system, may be arbitrary in a less radical sense: in the levels of organization it considers relevant. For instance, all the languages on our chart find it worth while separating the isolated tree from a conjunction of trees, but they differ in their interest for recognizing new physiognomic quantities inside the category of more than one tree. In his latest book, Eco (2000), recognises that the ability of language (and no doubt other semiotic systems) to redraw the borders of the world, so fundamental to structuralist semiotics, is limited by the grain intrinsic to the world itself, or at least to its perception by human beings. This was actually a basis point of my own critique of structuralism (Sonesson 1989a),

Fig. . The segmentation of the tree domain in different languages



where I relied on Roschs concept of prototype to show that the world, as least as we perceive it, is highly organised and only susceptible of being reorganised on a secondary, rhetorical, level. Mereology, which is the science of the whole and its parts, is inspired by Edmund Husserls early works, notably by the third study contained in the second book of Logische Untersuchungen (Husserl 93, 225-293). It owes its name, however, to the logician Lesniewski who gave it its logical formulation (cf. Smith 994; 995; Stjernfelt 2000). The task of the mereology is not only to account for the relations between the whole and its parts, but also to explain the difference between various kinds of totalities. Husserl opposes configurations to aggregates, and we find attempts of the same kind, but sometimes more developed, in the work of various representative of Ganzheitspsychologie (cf. Sonesson 989a, I.3.4). Peirce even wrote a very long but rather disorganised list of various kinds of totalities (quoted in Stjernfelt 2000). I shall retain here the first lesson of Husserls study, which consists in putting the emphasis, not on the way the whole results from the addition of its various fragments, or, in a parallel fashion, the way in which the part is derived from the division of the totality; but rather, on the relations of mutual or one-sided dependence (among which are to be found the counterpart and the autonomous object) which exist between the parts and the totality which they establish together. In this sense, the mereological model is not equivalent, in the linguistic domain, to a constituent structure grammar, of the kind envisaged by Chomsky, but to a grammar of dependence, such as is conceived by Tesnire, or a categorical

grammar, in the sense of Montague (which moreover is inspired, through several intermediaries, by Husserls study; Cf. Sonesson 1989a III.5.1).39 According to Arthur Koestler (1978:27ff), reality is a multi-levelled, stratified hierarchy of sub-wholes, where each sub-whole or holon is, in relation to higher levels, a dependant part and, in relation to its own parts, a whole of remarkable self-sufficiency. Benvenistes (1966) observation, transferred from linguistics to narratology by Barthes (1966:5), that each elements gets its meaning in part from its distributional relations to other elements on the same level, and in part from its integration into elements of a higher level, also presupposes a hierarchy, where the wholes of one level are the parts of a higher one. If there is indeed a holarchy like this, to continue using Koestlers term, then different semiotical systems can establish their categories on different holarchical levels. However, there is some psychological evidence to suggest that one level of the hierarchy is privileged, is in fact the basic level, made up of intrinsically separate things (Rosch et al 1976:383). All parts may still be wholes, but some wholes are more wholes than others. Categories, according to Rosch et al. (1976:383ff), serve to reduce the infinite variety of real-world experience to a manageable degree, recognizing distinctions only where these seem important. Features tend to come in pairs, triplets, and n-tuples, the appearance of one of them serving to predict the others, as for instance an animal with wings will be expected to
39 Without reference to Husserl, however, Hjelmslev bases his glossematics on the same minimal system of dependencies between the whole and its parts. 0

have feathers that is, in our terms, categorization is based on indexicality, in particular abductive contexts. At the basic level cue validity is maximal, giving rise to information-rich bundles of perceptual and functional attributes. A basic object like a chair is found at the most inclusive level at which there are attributes common to all or most members of the category. A category at a superordinate level, like furniture, contains objects having fewer attributes in common, and a category at a subordinate level, e. g. kitchen chair, shares most of its attributes with contrasting subordinate levels. Many experiments support Roschs theory: subjects list more attributes for terms on the basic level; this level is the highest on which motor movements are recognized as defining a category; averaged shapes combined from pictures of objects at the basic level are identified as readily as those of objects at a subordinated level and much better than those at superordinate levels; basic level categories help in detecting a picture in visual noise, and priming with basic level categories facilitates the same-response as applied to physically identical stimuli; objects are recognized more rapidly in pictures at the basic level than either at the subordinate or the superordinate levels; classification of pictures on the basic level is practically the same for three year olds as for adults, but when it comes to superordinate categories, the results differ widely; small children are found to be using almost exclusively basic level terms (Cf. Rosch et al. 1976). It should be noted that the existence of a basic level, in some of Roschs experiments demonstrated for pictures, justifies the much-criticized notion of a pre-icono-

graphical level on which, according to Panofsky (1955:28, 33), lines and colours are directly seen as human beings, and the equivalent level of immediate perception in Barthes (1964b) pictorial semiotics. Both Floch (1978) and Larsen (1976) take exception to these models, which they erroneously identify (cf. Lecture 5), because such a level is incompatible with a conventionalist theory of iconical signs (although Eco 968:30, 34f, 59 fails to note the contradiction). In order to explain her result according to which objects in pictures are identified more rapidly at the basic level, Rosch (p 414) speculates that superordinate categories are derived by inference from the class membership of the basic object, whereas subordinate categories must be determined from additional attributes present in the picture. As applied to one of Panofskys (1955:28f) examples, a shape on the canvas is immediately seen to be a male figure, thus perhaps inferred to be a human being, and with the aid of another shape contiguous to the first and immediately seen to be a knife, at last identified as being St. Bartholomew. In the kind of cases considered by Panofsky, subcategorization is normally obtained through objects found in contiguity with the object, usually a human being, which has to be further determined, and the meaning of these contiguous objects is conventionally and/or indexically fixed (i. e. an abductive as well a performative index). In other cases, for instance when a shape seen as a human being is subcategorized as a well-known person or a friend, we will have to attend to cues internal to the original shape, and even to higher-order features, not, for instance, the shape identified as a nose, but the peculiar


curvature designating it as being a nose of a particular kind. However, there are really two kinds of hierarchies, or holarchies: analytic ones, of the man-arm-hand-type, and synthetic ones, of the man-mammal-vertebrate type (cf. Ellen 1977:344ff).40 These terms, unfortunately, are misleading, since at least the second kind of hierarchy has its basic level somewhere in the middle, and thus will proceed by aggregation from this level upwards but through analysis from the same level downwards. Nevertheless, it is true that all levels and elements in the first type of hierarchy, unlike those in the second, have a concrete existence (p 345) in fact, as we go further down in the hierarchy, the space or the extension occupied by the elements will be smaller in the first hierarchy, but there is no change in the second type. For instance, the old hag, the female cannibal, the cannibal, and the human being are equally space-filling, but whether we apply our ordinary body scheme, or the one Europeans projected onto South American cannibals (if we are to believe Bucher 1977), each step down the hierarchy gives us a smaller portion of space. Adopting a traditional logical distinction, we will therefore distinguish extensional hierarchies, where subcategories are less space-consuming, and intensional hierarchies, where extension is held constant. We have seen that the levels introduced by Panofsky are positions on the
40 Groupe (1970; 97ff ) refers to this distinction more obliquely, in terms of the material and the conceptual decompositions, respectively, exemplified by the three being divided into stem, branches, leaves, roots, etc, and, strangely, poplars, birches, oaks, etc.. They latter are of cause different choices on the same intensional level, dominated by the basic object tree (Cf. Sonesson 1989a: 44f)

intensional hierarchy; but the problem of segmentation of either a body, text or picture, is a question pertaining to the extensional hierarchy. The question thus becomes whether there is a basic level also in the extensional hierarchy. Unfortunately, Rosch et al. (1976) do not make any distinction between these two types of hierarchies, but it is recorded in a footnote (p 388), that all categories bearing a part-whole relationship to the superordinate have been eliminated from the experiments along with a few other types of hierarchies. Intuitively, it seems much more obvious that there is a privileged level in an extensional hierarchy: the body appears to have precedence over the arm as well as the couple and the group. However, the characteristics of the privileged level are perhaps different in the case of the extensional hierarchy: while superordinate categories may still have fewer attributes in common (e. g. group) than basic level categories (e. g. body), subordinate categories (e. g. arm) appear to possess many further attributes not present at the basic level. It would be interesting to repeat some of Roschs experiments for extensional hierarchies. It seems probable that averaged shapes as well as figures hidden in visual noise would continue to be more easily identified at the basic level than at superordinate levels; and that basic level objects will even in this case be more rapidly categorized than objects at any other level. But maybe quite different criteria can be used to determine the basic level of an extensional hierarchy: the Gestalt factors of common fate in movement, perfect closure, etc. Here we will suppose that a basic extensional level can be found. In order to go beyond the above-men

tioned, rather negative characterization of intensional levels, it would help to consider some examples. Darcy (1963), in his analysis of human acts, points out that exactly the same happening may be described as tensing ones forefinger, pressing a piece of metal, releasing a spring, pulling the trigger of a gun, firing a gun, firing a bullet, shooting at a man, shooting towards a man, shooting a man, killing a man, committing judicial murder, and saving four lives. While to Darcy, this example shows the difficulty of separating an act from its consequences, to us it suggests that the very same event (or, in other cases, object), while continuing to be thematized, can be redescribed at a different intensional level, as it is embedded in a wider context of which it is seen to form a part. That is to say, as we step down the intensional ladder, we have to take a wider extension into account, just as we do when we climb the extensional hierarchy, but the theme of the category, that which is to be characterized is all the time the same. When a young girl is seen in the wider context of a sword, a charger with the head of a beheaded man, and a maid, she may be redescribed on another intensional level as Judith; but if the same girl is presented in the context of a charger with a head of a beheaded man and, for instance, her parents, then she should properly be redescribed as Salome (Cf. Panofsky 1955:36f). Again, contrary to the contention of Goodman (1968:27 ff), in a picture of the first Duke of Wellington as a soldier, the Duke of Wellington is just as present in the picture itself as the-Duke-as-soldier, and, in a quite parallel fashion, both the unicorn and the unicorn-as-putting-his-hornin-the-virgins-lap are found in the picture

only on distinct intensional levels. To localize the Duke in reality and the Dukeas-solider merely in the pictorial sign is quite arbitrary just as would be the case with the unicorn as compared to the unicorn-signifying-virginity: both meanings can be connected with their peculiar features of expression in the picture, though those of the second intensional level may well be mere modifications in the facture of the features of the first level, as noted above (Cf. Lecture 1). In fact, Panofskys pre-iconographical level does not seem to correspond perfectly to our basic intensional level. The source of pre-iconographical interpretation, Panofsky (1955:40 f) tells us, is practical experience (familiarity with objects and events), while knowledge about specific literary themes and concepts is necessary for iconographical analysis (also cf. Kaemmerling, ed. 1979). However, since the distinction is introduced by the discussion of how a real act accomplished in the world of our experience, a greeting, is interpreted, it is seen to oppose more generally naturally motivated meanings to conventional ones (though present-day scholars are not so sure about the conventional character of greetings, cf. the work of Eibl-Eibesfeldt in particular). Interestingly, perceptual psychologists like Hochberg, Gibson, and Kennedy have noted that experience with the world, rather than experience with pictures, is what is needed in order to interpret at least some simple types of pictures (Cf. Lecture 3). However, experience with the world will be sufficient also to interpret some cases of intensional subcategorization: the Duke-soldier as well as the Duke, our friend as well as a man but not St. Bartholomew as well as the man with a


knife. Panofskys distinction is thus seen to cut across ours. Sometimes, Panofsky (1955:33ff) tells us, even the apprehension of figures at the pre-iconographical level will be problematic. In Roger van der Weydens painting Three Magi, the apparition of a small child in the sky, iconographically identified with the Infant Jesus, is not recognized as such because of the golden halo, which a putatively real Infant Jesus may have, nor because of any non-existent cues to the child hovering in the air, but because the child is suspended in mid-air whereas the rest of the representation, contrary to what happens in Ottonian miniatures, respects the laws of gravity. Not only is the preiconographical level here no more coextensive with our basic intensional level, that which is most directly apprehended, but it is based on knowledge going beyond practical experience, being as conventional as the level of iconography, so that the only difference seems to be whether the source of the convention is found inside painting itself or in literary sources. Quite consistently, Panofsky also locates the history of styles on the pre-iconographical level. However, from our point of view, Panofskys exception to the straightforward mode of pre-iconographical analysis is interesting: we can conjecture, though it is certainly particularly difficult to prove, that an apparition in the real world would have a basic level of its own, forming with human beings, angels, and devils (as Swedenborg might have argued) a superordinate category, while in the picture, it appears as a kind of man, so that the pictorial space must be searched for further details in order to identity some basic level men as apparitions. That is, the basic object of

the picture is not necessarily the same as the basic object of the world. Indeed, it is by means of the manipulation of basic intensional and extensional levels that pictures, like language and all other semiotic systems, can reorganize the world of our experience, segmenting it in their own peculiar manner. Summary Before we can even start to compare pictures to the perceptual world, we need to know how the latter is organized, but there is no single scientific answer to the question how this organization comes about. Different perceptual theories centres on different facts, but from an phenomenological point of view, a least, Gibsons conception seems more adequate than the others, at least if complemented, along the lines of Neisser, with some amount of hypothesis testing. At the same time, like Husserls phenomenology, Gibsons psychology needs to be provided with a cultural level of interpretation. Yet, some phenomena observed by Gestalt psychology, but perhaps less by the familiar Berlin school than by the two other currents, may help us distinguish different kinds of wholes, such as structure and configuration. But it is not sufficient to understand wholes: we also need to elucidate parts, and in fact the whole nexus connecting parts and whole at different levels, which may be extensional (proper parts) or intensional (properties or perspectives). In both the extensional and the intensional hierarchies, basic levels must be posited, along the lines suggested by Rosch, and these may change from the depicted world to the picture. Categorization, as well as segmentation, accounts for the relation between pictures and oth

er semiotic resources and the Lifeworld: these categories, it turns out, are very rarely based on sufficient and necessary criteria, but rather depend on dominance or on prototypes. The latter, however, may be exaggerated into idealtypes, and given their fictional counterparts, the least probable co-occurrence of features, in the antitypes, exemplified by monsters, which, as Lvi-Strauss would have said, are particular good to think with.

ten requires us to pick another: the picture, unlike language, points to reality in extricable chunks. Exhibit import and complex ideas After giving to Peirces great distinguishing property (1932, II:279) of iconical signs the name exhibitive import, Greenlee (1973:79) curiously proceeds to dispute its specificity, claiming it to be present also, for instance, in novels, in respect of human situations, and this entirely independent of the perceptual qualities of the vehicle, in contrast to the imports of a lyric poem (p. 80). Here, I submit, Greenlee seriously misinterprets Peirces point, which is actually concerned with that property of icons which guarantees their cognitive value, as Maldonado would have said (Cf. Lecture 3), and which was singled out, in connection with mathematical and logical signs, already in the semiotical theories of Leibniz and Lambert (cf. Holenstein 976: 151 ff; Dascal 1978). It is important to note that this, at first sight, curious rapprochement between pictures and logico-mathematical formulae is a constant theme, also of Peirces theory of iconicity. But before we can elucidate what may well be termed the classical conception of iconicity, it will be useful to try to reconstruct Greenlees, in itself very reasonable line of thinking. For what indeed can it mean, that, in Peirces words, by direct observation of the sign, other truths about its object can be discovered than those which suffice to determine its construction? A novel is entirely made up of the resources of verbal language, the words of its vocabulary, whose received meanings are listed in any lexicon; and yet the novel is undoubtedly able to recount experiences not already


4.5. From the world to the picture and back again

In spite of their basic iconicity, pictures are also very different from perceptual reality, and yet they convey a lot of information about the latter. When Peirce claimed that iconicity was distinguished by the fact of other truths being extracted from their observation than those put into their construction, he was not only thinking about pictures, but also about mathematical formalism. There is in fact a tradition for making this parallel, going back to Leibniz and the French ideological school. In both case, it seems, there is a fundamental difference between the sign itself and the object in the world to which it points, that is, in Husserlean terms, as applied to pictures, between the picture object and the picture subject. Perhaps this distinction could more readily be construed as consisting of different levels within the sign, such as Sinn and Bedeutung, or levels of intentions (to which must be added the levels of extension). As a complex idea, pointing to reality, the picture may very well shift the basic level of both intensional and extensional hierarchies. To some extent, these ideas are already familiar from Panofskys iconology. However, picking a feature of meaning of-

A ..........................................a B......................b C.c

Fig. 35. The exhibitive import of logic contained in an adequate feature analysis of the vocabulary of the language in question. In itself this is certainly an interesting property of signs (captured in Benvenistes distinction between le smiotique and le smantique), but something more is required for the sign to have exhibitive import: that this additional meaning can somehow be directly observed. Thus when Greenlee cites, in support of the novel having exhibitive import, the fact that the additional meaning engendered in no way depends on the perceptual qualities of the vehicle, this is really an argument against his claim, and poetry is in fact a better candidate for iconicity (cf. Jakobson 1965a, b). In spite of Greenlees own interpretation of the term he has coined, the exhibition of the import must here be taken in the sense in which Wittgenstein says that certain properties are shown by the sign, rather than signified (cf. also Rcanati 1979); indeed, exhibitive import would seem to be that part of what is shown which is also signified.4 For instance, what is iconic about mathematical formulae is, according to Peirce (1932: ll: 282), that, each time, the same letters are used to stand for such unknown terms that are in analogous relation to the problem. The same is true about the existential graphs. When Eco (1976: 333ff) denies the iconicity of Peirces existential graphs, he
4 Cf. Goodmans notion of exemplification, which is a reference to properties also possesses by the object used to refer; however, pictures do not literally possess the properties they exhibit.

once again misses the point: visuality has never been a criterial attribute of icons, although some icons are visual (cf. and Peirce would certainly not want to claim that the parts of which his graphs are made up are motivated in themselves, any more than he would have said so about the numbers and letters of the mathematical formulae. Only the relationships are exhibited. Long before Peirces graphs and Eulers circles, Lambert invented a logikalische Zeichenkunst, in which logical relations could simply be read off (cf. Eisenring 1942: 22 ff). Thus, for instance, if we know that all B are A, and that some C are B, and if we transcribe these two relations on paper, we are able to see directly, rather than conclude by reasoning, that at least some C are A (Fig. 35). In this sense, the identical letters in the mathematical formulae are perhaps the irreducible residues of exhibitive import. The question then becomes whether ordinary pictures possess even more exhibitive import than existential graphs and the like. Peirce certainly seems to suggest so, when he tells us that a map may be drawn by means of two photographs. However, we will first consider a more straightforward example: in Picassos print (Fig. 36), it is conceivable that only the relation between the eye and the brow and between the brow and the hair were necessary to determine the construction of the upper right-hand part, and yet the relation between, for instance, the eye and the hair can also be discovered in the picture. Since the order in which the elements of a picture have been drawn is rarely relevant, or even known, this fact is of very slight interest. But there are two other properties of pictorial exhibitive import that are fundamental. First,


exhibitive import permits us to see into Picassos print also such parts of a human head as possess no expression features of their own in the picture, as for instance the cheeks, the forehead, etc. What is more, these parts can be at least roughly localized in relation to the other parts of the head, exactly as those parts that do possess expression features. And this brings us to the second point: that which is exhibited in an ordinary picture is not just any kind of relations, but the spatial relationships prevailing in our common Lifeworld, that is, a series of locations. The question is, of course, where the information, which is not contained in the principles of construction determining the picture, comes from, and the only reasonable answer, I believe, is that it derives from the interpretational schemes of the Lifeworld. But so much, it seems, could also be claimed for Greenlees novels. The difference, then, is that, through the projection of the interpretational schemes onto the picture, it is possible to locate details of the depicted object in relation to each other in the picture, also when these details are not marked on the pictorial expression plane. Even the photographic lens is selective, and can be made more so by special techniques, so even here, interpretational schemes can make their contribution. Nevertheless, it might be argued that, as the principles of construction of the photograph are those

physico-technical processes making the camera work, no other truths than those permitted by the processes in question can be observed in the photograph. However, Peirces claim regarding the possibility of deriving a map from two photographs could be taken to suggest that, given two perspectival adumbrations of an object, we are able to construct, by means of the Dingschema of the perceptual world, an invariant view of this same object. At least under some circumstances, one photograph should be enough. And in a more restricted way, the picture could also be derived from the map. Before delving deeper into the peculiarities of pictorial exhibitive import, a few more remarks on the iconicity of mathematical formalism are in order. In Artificial Intelligence, there have been some recent efforts by Sloman and Hayes to establish a general distinction between pictorial and verbal modes of representation. Here, logical and mathematical formalism is taken to be even more clearly opposed to pictures than verbal language. It is interesting to note, that Janlert (985: 98, 0 f, 106), in his critical remarks on Slomans and Hayes distinctions, repeatedly quotes mathematical examples in order to show that verbal representations possess the properties supposedly reserved for pictorial ones, apparently without realizing that it would be much harder, at least in some of the cases, to find verbal counterexamples in the strict sense. Thus Janlert (p. 101 f) rightly observes that, contrary to Slomans claim, the structure of the mathematical formula, just as the structure of the picture, could well contain information about the structure of what is represented. But Janlerts implication that what a formula


Fig. . Head in Picasso drawing

such as x3 represents is 6, i.e. its sum, appears confused to me, for this is a meaning relation of a secondary nature, in the formula, not from its expression plane to its content plane. Rather, each one of the numerals and signs stands for a number or a mathematical operation, and they do so, as Degrando has shown, by means of logical analogy (cf. also see Prieto 966:0 ff). Indeed, each single numeral being a sign, not a figura, as Prieto (1966: 159 f) observes, there is isomorphy between the two semiotic planes, and this isomorphy also extends to the syntax of the formula. Although isolated instances of this kind can also be found in verbal language (cf. Jakobson 1965a, b), its dominant principle is certainly very different. Therefore, we may conclude, with Leibniz, Lambert, Degrando, and Peirce, that mathematics has more in common with pictures than with verbal language. And so, in discussing the interpretational schemes rendering exhibitive import possible, we will pursue the comparison with mathematics. According to Degrando (1800, 1: 180ff), complex ideas of the second order are formed out of simple ideas in alternative ways; thus a, b, and c, d may add up to A and B, respectively, which together form X; or else, a joins with c, and b with d, to form A and B, respectively, which again result in X; or it is a, d, and b, c which form K and B, respectively, again adding up to X. The concept of complex ideas of the second order is introduced by Degrando, in order, to solve the muchdiscussed problem concerning the interest of a proposition such as 5+ = 4+3; for one would naturally think, Degrando suggests, that either the formulae 5+ and 4+3 are really identical, in which case the

fact of their equivalence is trivial, or they are not identical, which means the formulae cannot be exchanged for each other; but in fact, they can be so exchanged, and the result is informative. While it may be surprising to learn that this was a well-known problem even as early as in Degrandos times, we immediately recognize Freges (1966: 41) famous question how it is possible for expressions like a=b to carry any information (also cf. Carnaps 958:38 idea of intensional isomorphism). In a more general way, we are also reminded of the idea, current in communication theory and psycholinguistics (cf. Miller 967; Clark & Clark 977; Chafe 1972; 1977; etc.), that information may be handled and stored in memory in differentsized chunks. Indeed, Degrando (800, 1: 165tf) claims complex ideas are needed because we are unable to comprehend more than 4-5 units at a time, and Daube (1805: 21 f) thinks ltendue du jugement spans at the most 5-6 objects, which comes close to a more recent bet, Millers magical number 7. The solution to the puzzle, Degrando says, is that, while the same primitive elements are found each time in the complex idea, they are not envisage dans la mme manire par lesprit. If I have 5 pence in one pocket and  in the other, or 4 pence in one pocket and 3 in the other, there is a difference, for mon esprit peut apercevoir, un suivant lune, des rapports quil ne remarque point en sattachant lautre (p. 180f). The complex idea seven may perhaps be conceived in the following way (Fig. 37). In a sense, then, there is identity on the highest and the lowest level, but not on the intermediate level. This is only a small


fragment of what Arnheim (1969: 218ff) terms the inner structure of mathematics, even as applied to the number seven, for there are numerous ways in which seven may be organized into chunks; but the essential point is that the identity lost at intermediate levels, re-emerges at both extremes of the hierarchy. But so far, we have only been concerned with quantities; the question now is if something similar holds true of the qualitative types of complex concepts. In the famous equation Hesperus = Vesperus, and in other cases that proved of interest to Frege and Quine, there is an identity at a higher level (except for the person who is ignorant of this astronomical equivalence, to whom there are really two cultural objects, as suggested by Eco 1976: 129); indeed, there is identity all through the hierarchical levels of constituency, but the thematic organizations of the elements differ, just as in the noematic matrix of the perceptual object. Also different are the classical cases of intensional hierarchies we have considered above: in Panofskys example, Judith is intensionally included in young girl, and in the case cited by Goodman, the duke of Wellington intensionally overlaps the warrior; thus, in both instances, there is only partial identity of elements. But if we now consider what we have called extensional hierarchies, a parallel to the mathematical example can be found. A case in point is the body scheme. Here, an obvious division into the head and the central body can be made, followed by the separation of the latter into upper and lower body, comprising chest and arms, and belly and legs, respectively. An alternative analysis would distinguish the central part, the trunk, and different appendages,

 5

ll  7

= =

 4

 3 7

Fig. . Complex ideas according to Dgerando. such as the head, the arms, and the legs. Both these divisions, and a number of other ones, apply to an identical whole, and have identical ultimate constituents, such as fingers, toes, etc. If verbal language is our sole source for the semiotics of the body, as Ellen (1977) would seem to believe, then the fact that some languages use the same term to mean hand and arm, certainly implies that also the ultimate constituents of the body scheme may turn out to be different from one classification to another. However, it would seem that in such models of the body that are derived from gestures, dance, work, and other corporeal activities, in which the hand, and the fingers, are directly pertinent units of signification, lower-order distinctions of this type have to be made. In any case, we have no trouble admitting that, in some instances, the identity between the variant analyses of a complex idea is found only on the highest level of the extensional hierarchy, in contradistinction to what seems to be the case in mathematics. But this would mean that, contrary to what Degrando supposes, the variant analyses of the same complex idea are not necessarily based on a simple rearrangement of the identical constituents, but presuppose different elementary segmentations of the perceptual world. A more important difference to mathematics, from our present point of view, is that the different variant analyses of the same extensional hierarchy do not seem to


have, in general, the same status; and it is not only that one version is explicitly given, and the others implicitly, but some variants are clearly granted more importance, and even correctness, than the others. In Occidental cultures, at least one of the relevant bodily schemes looks much like Fig. 38. As here presented, the diagram applies to the body the kind of analysis which linguistics has employed with the sentence, the tree diagram, and actually, it combines two traditional approaches to such an analysis, constituency grammar and dependency grammar. In the first case, as in the base component of a transformational grammar, the body, just like the sentence, is divided into its proper parts. The format which I have (rather arbitrarily) adopted is that of binary contrasts; and although this is not always linguistically manifested (arm, arm, etc.), all contrasts are based on inclusive oppositions; e.g. the arm both includes the hand and is opposed to it, as that part (of the arm, in the first sense) which is not the hand. In order to express such relations I also have recourse to dependency grammar, where parts which are somehow more central, and on which the others depend, occupy a privileged position (cf. Tesnire 1959). Instead of using two tree diagrams, I

have chosen to employ brackets to stand for constituency, and arrows for dependency, on the same diagram. The arrow goes from the dependent element to that on which it is dependent; a double arrow expresses mutual dependence, and no arrow stands for simple coexistence (cf. Hjelmslevs concepts of determination, interdependence, and constellation). Perhaps these relations of dependence and dominance can be identified with what was termed thematic relations above (as in Hesperus/Vesperus case, but here combined with differences in constituency). First rhetoric: Shifting the levels of intension and extension Before we proceed to investigate (in the next section) the interesting case, in which a picture operates a reanalysis of such a complex idea as the body, it remains for us to consider the possibility of a pictures pruning some of the levels in the hierarchy corresponding to its picture subject, so as to establish a base figure level different from the base object level characteristic of the object in the Lifeworld. Since we know that there are at least two different kinds of hierarchies, the extensional and intensional hierarchies, the issue must be examined

Fig. . Canonical Body Scheme

0

separately for the two cases. In a strict sense, there has been a modification of the base level in a picture when the original base object level does not appear; in a looser sense, there is such a modification, when another level is seen to be intrinsically more dominant (in the Prague school sense) in the picture. Since one of Roschs criteria for determining the intensional base level is that it is the last level on which an averaging of all pictures will still yield a recognizable object, it seems obvious that there can be no upward shift of base level in a picture, not even in the weak sense. Thus, while it is not true, as is often suggested, that pictures are inextricably bound up with singularity, they cannot go beyond the perceptual base level, or rather, just as verbal language, they are limited to doing this by means of a semantic rule. Traffic signs, for instance, contain pictures whose dominant level is human being, but it has been impossible to make a picture corresponding to the meaning wild animals crossing, which must instead be synecdochally implied by a picture showing an elk or some other particular animal; for the averaged shape resulting from an elk, a bear, a boar, and so on, would be impossible to identity. In a semiotic system less geared to easy readability, it would of course be possible to make a composite animal out of the most characteristic parts of the different species (which is in fact the origin of monsters), but then each part, considered as such, would be at the base level, and a semantic rule would again have to account for the meaning. A more intricate question is if there can be a downward shift of the intensional base level in pictures. This would require

us to depict Judith, to use Panofskys example, in a way not permitting the meaning woman to be seen in the picture, or at least in such way that the meaning Judith remains more dominant; or, to use Goodmans example, we would have to convey the meaning the duke of Wellington, without goring through the meaning man. To gain some perspective on this issue, it will be useful to consider first the more general question, whether it is possible, in the rendering of an object in a picture, to select the base level and the lower intensional levels separately. According to Gombrich (1963: 2), the wax-doll at Mme. Tussauds, which represents a guard, is as particularized as the other, more well-known wax-dolls in the same location, which are made in the likenesses of famous people; it is not a generalized, but only an anonymous guard. Armed with the concept of base level, we are now able to point out that the guard, just as those famous people, is first of all seen as a human being, then as a guard, and much later as an individual person. But it may still be true that there is no possible pictorial rendering, in which the intensional levels below the base level are not also included. While Gombrich is probably right about wax-dolls, it may seem that schematic pictures (to use Wallis 975 term) like traffic signs fail to posit any intensional levels below the base level: the man, it seems, is simply the man. This, however, is only apparently so. First, then, interpretational schemes permit us to add to the picture subject parts and properties which are in no way indicated in the picture thing, and thus in the picture object. Strictly speaking, no eyes can, for instance, be seen in the figure


of a man depicted on the traffic signs: but, given our body scheme, they are tacked on to the picture subject as a matter of course. And so is the case for all other details. As such, this is of course a question of extensional levels, going from the whole of the body to its proper parts. What is not given, however, is the exact way in which we imagine the eyes (and so forth): for we will most probably conceive of these eyes as they are most likely to look in our culture, and this is something which is most certainly not given by the principles of construction of the picture. In this connection, it is interesting to note that in the Middle Ages, painted figures were supposed to be, in Baxandalls (1972: 47) words, generalized and yet massively concrete, in order to permit the viewer to project onto their faces those of familiar people, thereby fortifying his memory for the teachings of the picture. To some extent, such processes still seem vital to pictorial interpretation. There is however a second, more fundamental reason for claiming that no picture is capable of rendering the mere base level, to the exclusion of all lower levels; and this is that, in all pictorial systems, some different categories of meanings can only be conveyed together, in the same information package. In none of the well-known traffic signs is there a mere depiction of a man; it is a man digging, or walking, or riding a motorcycle and so on. But apart from such features, which are part of what was meant to be conveyed by the traffic signs, each man-depiction also presupposes a number of choices as to how tall the man is to be, how fat, how wellproportioned and so on. Another case in point is the well-known pictogram that often indicates the whereabouts of the ladies

washroom: although the intended message, of the sign is simply woman (from which, by synecdoche and convention, we arrive at ladies washroom), the pictorial system forces the draughtsman to decide also on such trifle matters as if the woman wears a skirt (and she must wear one, lest she be confused with a man, if we are not prepared to permit the representation of more obligatory sexual characteristics) and even on its length.4 Thus, the pictorial system often forces the artist to convey more information than he is supposed to, and/ or which he possesses. Traffic signs and toilet pictograms are supposed to depict generalized human beings; but other pictures are meant to render concrete situations about which we have an insufficient knowledge. The draughtsman, who is asked to visualize a particular landscape as it will look in the future, may be furnished with information as to how many trees are to be planted there, but he has to decide arbitrarily on the appearances these trees will assume as they grow. Another draughtsman, who is given the task to describe, in a comic strip, the flight from prison of a well-known criminal, may supplement the lacking information by using the format established by Dick Tracy and similar comic strips, as Fresnault-Deruelle (1977: 79 f) suggests, but he also has to invent a number of minute details that neither he nor anybody else was there to see: for instance, the exact angle between the criminals legs at the moment he fired his first shot. From these observations it follows, not that pictures must render all levels be4 Thus, some of the pictograms in Aicher & Krampen 1977 clearly show women of the fifties, and others women of the sixties. 

low the base level, but that something more than the base level must probably be included, and how much that is depends on the particular pictorial resources, traffic signs, toilet pictograms and cartoons being, generality speaking, less restrictive than more elaborate kinds of drawings and photographs. Perhaps it is not possible to prune the base level in what was above called the strong sense, but it certainly is possible to make other levels more intrinsically dominant in the picture. Thus, for instance, in an analysis I made of some comic strips by the French draughtsman (based strips published in Nouvel Observateur no 931-945, September to October 1982), hierarchically lower levels such as young, beautiful woman and old, ugly woman were found to dominated by far the putative base level woman. Indeed, from the point of view of the expression plane, there is a primary opposition between young, beautiful woman on the one hand, and man and old, ugly woman, on the other. The existence of an extensional base level has not been established by Rosch, but what is at stake, when Piaget and Bower argue over the innateness of the object concept, is obviously not just any object, but such a base level, which is appears in the concept of independent object or substance central to Gibsonian perceptual psychology. There seems to be no difficulty in having pictures in which there is an upward shift of the extensional base level: a drawing may render a group of people in such a way that there is no possibility of separating out the signs for the different individuals that make it up; and, even more obviously, the group level may be given more prominence in the picture than the

body level. As Lindekens (1973) observes, the same effect is easily obtained in a highly contrasted photograph; and, it might be added, in a partially blurred one. Traffic signs and other pictographs, it would seem, rarely indicate any inner detail.43 In other cases, however, only partially drawn lines, inside the contours of a drawing, may serve to indicate the limits between the body parts. Another interesting device consists in transforming repeated parts into the corresponding holistic property; thus, for instance, as in an example referred to by Matisse, leaves may not rendered as such, but instead a kind of leaf-ness is expressed by the repetition of the digit 3. The possibility of adjusting the extensional base level downwards is no doubt more controversial. However, as we will see in Lecture 7, it is possible to use body parts to create an onion, and parts of a Greek column to suggest a bottle of Absolut Vodka, and this supposes the body parts and the parts of the column to be recognizible as such without the presence of the extensional base level to which they would normally refer. The chunks of the visual world Pictorial meanings, it seems, tend to come in informational packages. Put simply, this means that, in a given pictorial system, the decision to render one particular semantic feature forces us to render also another particular semantic feature, or, perhaps more commonly, to choose among a range
43 In the scale of abstraction that Aicher & Krampen 977: 9 ff apply to different pictographs a decrease of inner details seems to be a contributing factor; however, increased regularity and symmetry would appear to be more fundamental in their examples.



of alternatives pertaining to another feature type (cf. the implicational universals of Husserl and Jakobson, as discussed by Holenstein 1974: 50f). Not all these mutual implications between semantic features are of the same kind, for the picture shares some of them with the Lifeworld. In any case, the really interesting implications are not of the strictly logical kind, but are intrinsic to particular semiotic resources (or to the Lifeworld). Hayes (as quoted in Janlert 1985: 106) claims that information cannot be added piecemeal in pictures, as is typically the case in verbal language, because we cannot add the distance between city A and city B on a map, without adding other distances simultaneously if other cities are included in the representation. Janlert thinks that analogous properties are present in mathematics, but this time he seems to me to be mistaken: adding +=3, he claims, also involves adding, however implicitly, + =3 but these are, as we saw above, two different analyses of the same complex idea. Its parallel in the map would be the impossibility of adding the distance between city A and city B, without also adding the distance between city B and city A. Hayess point is undoubtedly that the positing of a content, in this case a distance on a map, forces us to posit (indeed, as Janlert 985: 07 rightly observes, automatically posits) another content, i.e. another distance; and, as far as I understand, there is nothing similar to that in mathematics. On the other hand, the very property here attributed to pictures seems to be claimed by Prieto (1966:144; 1975a:123) to be a peculiarity of verbal language. Admittedly what Prieto (966:7, 9; 975b: 122 ff) actually says is that only verbal

language, among really existing codes, is capable of adaptation to circumstances, because it is the only one which possesses signs whose contents are not in mutual exclusion, but include and intersect each other.44 Thus, consider the kind of example referred to in all of Prietos books (966:7ff, 34f; 975a: 4ff; 975b:88ff, 03ff, 57 ff, 239 ff, etc.): in phrases such as Give my book to me!, Give the book to me, Give it to me, and so on, there is partial overlap of semantic features, which permits us to choose each time that particular constellation of features which is adequate to the circumstances. In order to examine one of Prietos (1975b: 103 ff) examples in detail, we will present it in a format that is easier to survey (Fig. 39). The point here is not only that facts, known beforehand or from the situation, may be explicitly included or not in the verbal sentence, but also that there are certain types of features that cannot be chosen independently of each other. As for the latter, Prieto notes three combinatorial impossibilities, two of which are really classes of impossibilities, since nothing is said about the presence or absence of the last four features; but many other impossible combinations are readily generated from our matrix above. This seems to be exactly the same case as Hayes points to in his analysis of the map: the choice of some features, or feature types, is not independent of other choices, but implies them and/or is implied by them. Thus Hayes seems to be wrong in thinking that this is a property which distinguishes pictorial modes from verbal ones; and even in the attenuated form giv44 As the saw above, this is exactly what is wrong with verbal language, and pictures, according to Goodman, if we take him to be referring to a comparable level of content 

en to it by Janlert (1985: 108), according to which a pictorial information chunk is at least as big as, and often bigger than, the corresponding verbal information chunk, the claim refers to too many unknown elements to be evaluated at this point. But what about Prietos reverse claim? Interestingly, in a later text, Prieto (975b: 159f) observes that in claiming this property exclusively for verbal language, he was thinking only about utilitarian codes, the artistic ones, like the cinema, the comics, and figurative painting being as yet insufficiently known. This is certainly a prudent position; but in the case of another kind of picture, the traffic sign, Prieto (1966: 130f) has not hesitated to demonstrate that there can be no intersection, or inclusion, of the contents of its signs. Before we proceed to examine this analysis, we should ask ourselves if it is possible to have a semiotic system, in which some semantic features must be chosen together, and which yet does not contain contents that intersect or include other contents. The only conceivable case seems to be a system, in which all such implications are mutual and valid in all cases, so that there is in fact no criteria for individuating the features chosen together and consequently must be considGive Imp 3III Pl IDO Ip Sg Donnez-moi mon livre! + Donnez-moi le mien! + Donnnez moi le livre + Donnez-le-moi! + Donnez-moi + + + Impossible combinations + Poss Ip

ered single features. Pictures do not belong to such a system, however, although there is of course no finite number of pictorial signs. Prieto suggests we imagine what it would mean for traffic signs to possess contents in inclusion. Fig. 40a (from Prieto 1966: 130) is an existing sign which signifies prohibition for vehicles the width of which exceeds 3 meters to cross the bridge; but fig. 40b shows an imaginary sign which, according to circumstances, is supposed to take on the meaning already mentioned (when a bridge is visible, for instance), or the meaning Prohibition to pass below the viaduct for vehicles the height of which exceeds 3 meters (when a viaduct spanning the road is in sight) and so on. Prieto now claims that there is no traffic sign like this in existence, and so the corresponding code does not contain any contents that overlap or include others. However, Gambarara (1979: 280) cites a putative counter-example: prohibition to overtake another vehicle is included in end of prohibition to overtake another vehicle; and Prieto (1975a: 125) curiously accepts this counter-example as such, but argues that it is too isolated a case to be of importance. It should be observed, first, that in both these
Sg of DO DO Sg M/F Book

+ + + + +

+ + + +

+ + + +

+ + + + +

+ + + +

+ + + +

+ + + +

+ +

Fig. : How to ask for a book, according to Prieto (from Sonesson a)


Fig. 40a. Real an imaginary traffic signs (from Prieto 1966) cases, the relation of inclusion between the contents is accompanied by a parallel relation of inclusion between the expressions, and this is not necessarily the case in the linguistic examples; and second, that, whatever the merits of Gambararas example, it is not really comparable, neither to Prietos linguistic examples, nor to his traffic signs, for unlike these, the signs referred to by Gambarara cannot, in different circumstances, be made to stand for the same meaning, considered in conjunction with the circumstances, but of necessity signify quite opposite things and thus the inclusion does not lead to an adaptability to circumstances, as Prieto insists it should. In any case, the real reason why a traffic sign like fig. 40b is impossible is that no ambiguity and no vagueness can be tolerated in this kind of code; but, in the case of other pictures, even those intended to be informative, this is no great trouble, as long as there is ample time to examine them in detail. The Peanuts code would seem to contain at least one instance directly parallel to Prietos imaginary traffic sign: brows are ordinarily not marked as parts of the eye, unless they are called upon to manifest a content from the expression paradigm; but this is not meant to indicate that brows are absent from the faces of the Peanuts heroes the rest of the time. Indeed, sometimes not even the mouth is drawn, without there being any suggestion that it periodically disappears. Also, Matisses Nu bleu as well as a simple line figure manifest at least the features woman and person sitting on the ground but there seems to be total exclusion between their respective features of expression. And although there may be circumstances, in which they could be used to mean the same thing, this would not be a typical use of them, in particular not of Matisses cut-out What we see into Nu bleu IV (Fig. 41) is, of course, a human being, and more particularly a woman; and we also see that she is sitting on the ground, holding one of her legs with her left hand, while the right arm is raised above the head; and we could continue for a very long time describing the details of her position. All this is exhibited thanks to the projection onto the picture of the body scheme. And yet, in a very obvious and immediate way, the picture is made up of seven independent units or elements, none of which is directly comparable to divisions in the body scheme. Since Nu bleu is clearly perceived as (the picture of) a woman, it would be important to find out in what way its picture object still differs from the impression en

gendered by a real woman, and even by a more straightforward portrait; in other words: how it modifies the conventional body scheme. Clearly, the organization, in terms of constituency as well as dependency, inside each of the elements, is very different from that of the body scheme. A possible analysis is the one given in Fig. 4.45 Not only is there very little conformity between this analysis of that of the conventional body scheme, but the constituent elements are different also, being generally roundish but with numerous jags, none of which are property of the real body. Under these circumstances, we must ask what remains of exhibitive import. First, let us consider to what extent the woman sitting on the ground can be seen in the cut-out. The configurational level at which appresentation takes place must no doubt be the picture as a whole. The elements 5-, and 4-3- are resemanticized as being the legs; 7a is the head; 7b-c is one arm, and 6d-e-f is the other; 6b stands for the bosom; and 6a-c must represent the upper part of the trunk. Other details must be added to the picture subject to produce a possible perception; the woman must have eyes, hair, etc, and we can roughly place them, but we cannot decide on the colour of her eyes, or the length of her hair. It is on the basis of the body scheme that 6a-c, in spite of its curious shape, can be seen as the upper trunk, and that the lower part of the trunk can be intercalated between 6ac and 5-4. There is no shift of extensional base level, since none of the body parts is recognizable in itself: if anything the base
45 Arguments for this particular analysis are given in Sonesson 989a:30ff. In this context, the details are not important, only that both constituency and dependency are deviant in relation to the body scheme

Fig. . Nu bleu IV, by Henri Matisse level becomes even more privileged as the locus of interpretation. As for the intensional base level, it remains woman (as it is in everyday life, for people we do not know personally);46 however, as it tends to be the case in pictures (except perhaps for very schematized pictures), the intensional base level is actually shifted down, not only to sitting woman, but to a woman sitting in a particular position. The picture thing here is a series of jumbled jigsaw-pieces which do not fit together and whose boundaries are largely rounded interrupted by sharp angles and jags. The picture object, that which might be seen in, has some local suggestions of being a sitting woman, but is has to be complemented by common interpretational schemes to really result in such a picture subject. Curiously, much of what is actually exhibited by the picture would make the
46 Perhaps we could say young woman, for an old woman would probably not sit in such a position. 

Fig. . Analysis of the body scheme in Nu bleu body depicted an impossible object of sorts: thus, one arm is seen to emerge from the head; the legs would seem to be joined to each other, and to the bottom, in an anatomically impossible way; and the whole body is cut into pieces at places that would be surprising even to a cannibal. Indeed, in order for the picture subject woman sitting to override the information directly contained in the picture, many adjustments are required, some of which are as follows: the parallel borders of 7a and 7g, as well as those of 6a and 6b, must be taken to be subdivisions inside one single object; but the space separating 7b and 5c has to be considered a pocket of air, the proximity of the two elements being an accidental effect of the sitting position. And while we cannot accept the idea that 5a and 4, and 7a and 7b-c, are related as shown in the picture, their proximity cannot be denied, as cannot, in the first case, the presence of a direct corporal link. Although its a photograph, Billy Brandts Nude (Fig. 43) is similar in many ways. However, it allows for more interpretation at levels below the extensional base level, in particular for the face, and the internal boundaries of the body parts are not as deviant as in Matisses cut-out. The similarity is mostly on the level of deviant dependency. Charlie Brown in the Lifeworld Quite apart from the difference of aesthetic value, the Charlie Brown figure the Charlie Brown figure in Schulzs Peanuts is distinguished from Matisses cut-out by at least three characteristics. To begin with, Charlie Brown is not a unique pictorial statement, but a ready-made formula, used to form complete and more varying messages, i.e. to act out some scene or other, in an innumerable series of comic strips. Thus, whatever message is conveyed by the hierarchical organization of Charlie Browns body, this is not the primary message, but its condition of possibility in the particular code employed by Schulz that is, the connotation (cf. Lecture 5): In the second place, Charlie Brown is made all in one piece, not cut up into elements separated by blanks, as is Nu bleu IV, and so we can dispense with the primary task of joining the elements together; by-passing synthesis, we go directly to analysis. Thirdly, Charlie Brown is an outline figure, not simply a set of surfaces; as Volkelt (963: 8 f) would have said, its surfaces are konturiert, not just begrenzt, i.e. limited by


Fig. . Nude, by Billy Brandt.

a pigment line facing on to two directions, instead of a simple change of pigment colour; and thus it may also possess inner details and parts. Even a superficial comparison of the Charlie Brown hierarchy (fig. 44) with the common body scheme (fig. 38) immediately points to a number of deviations. Most notably, the head completely dominates the Charlie Brown hierarchy. On the other hand, Charlie is made up of three immediate division blocks, as against only two in the body scheme (there may be a variant of the latter scheme that gives of four, but not three, primary units: head, trunk, arms, legs). Indeed, the distinction between arms and body is made on a lower extensional level than that of the legs, because the arms are partly fused with the trunk (this is not true of all Charlie Brown variants, however), whereas the legs are clearly separate

elements. Moreover, legs and feet are treated as equivalent elements, as to size and shape, and maybe their distinction should really be raised one more step, to the primary level, for there is not much to unite them except their very similarity. There seems to be the same identity of size and general shape between arms and hands, but this is less clear, since the arms merge with the trunk at their upper limits. The elements, as we have distinguished them, can be further divided into parts, where lines stand for edges of objects and surfaces, or into details, where the pigment itself, of whatever form, represents the object; and a last suggestion would therefore be that elements divided into details are better integrated, and therefore more dominant, than both elements divided into parts and elements lacking subdivision. Whatever the Charlie Brown formula


Fig. 44. The Charlie Brown hierarchy (The abbreviations stand for principles of division which will be discussed in Lecture 6). does to the body scheme, it certainly modifies the relative importance of the body parts: notably in the case of the head, the feet and the upper trunk. Here, it would be profitable to make a comparison with other comic strip figures, but such a task can only be cursorily hinted at in the present context. Brtechers drawings in Les frustrs would seem to respect much more closely common understandings of constituency and dependency in the body scheme (at least the strips from Nouvel Observateur 963 and 964, April 983, which we have scrutinized) but with one interesting exception: the parts of the head. Indeed, the nose would seem to be the most dominant part of the head; however, its limits in relation to the rest of the face are unclear, parts of its contour must at the same time stand for the upper lip, and the chin would seem to be non-existent, while the forehead dissolves into the root of the nose, where also the eyes are placed. Even more hypertrophied is the nose found in the personages of another French draughtsman, Reiser (as studied in Nouvel Observateur 93-945, September to December 1982): here, the head is made up of an enormous nose with an eye placed on the upper limit, and a few strokes added to indicate the hair. The body itself is not very different in shape and size from the nose; the division line between them is the mouth, and the facial region below the latter is drawn as a part of the body. Arms, and even legs, are diminutive appendages. All this is true of male personages, and also of ugly, old women. On the contrary, young, beautiful girls are drawn as a quite different species. Their hierarchical organization would seem to reproduce the common body scheme even more closely than Brtechers personages, with one exception: the breasts are depicted as concentric semicircles seemingly independent of the rest of the body. At least three semioticians have engaged in the study of the well-known comic-strip Peanuts, by Charles Schulz: Oomen (1975) and Kloepfer (1977), who refers to Oomen; and Gauthier (1976), who neither refers to, nor is referred to by any of the others. This, it would seem, is a good starting point for our meta-analytic considerations. Perhaps there is some particular characteristic of the strip itself that makes it more amenable to semiotic analysis than
00

most other comic strips. Kloepfer (977: 133), who admits Ecos description of iconic languages as being a weak code, thinks Schulz has managed, through abstraction and reduction, to transform his still predominantly iconic code (p. 131) into a system of discrete signs. Also Gauthier (1976: 116) believes there is something particular to Schulzs drawing technique that makes the analysis workable, but he adds that numerous contemporary strips would lend themselves as easily to the same type of analysis. Whatever may be the truth of this, it seems probable that a comparative study of these two or three different approaches to Peanuts will have something to tell us about the intermingling of iconic and categorical elements in pictures - or, rather, in some pictures. For, once again, what we have to propose here is more an illustration than a demonstration. According to Oomen (1975: 254), the visual code in Peanuts carries a predominantly expressive function, while it is the linguistic code that conveys the bulk of the information. Gauthier (1976: 121), on the other hand, distinguishes, in the pictures alone, elements pertaining to a code of identification, a code of emotions and a code of movements.47 The code of emotions, it would seem, is the expressive function of Oomen. Also Kloepfer (1977: 131) cites those features that serve to identify Charlie Brown, corresponding to Gauthiers code of identification. In his article, Gauthier limits himself to an examination of Charlie Browns head seen from the right side. In his opinion, the expressions of the emo47 He also adds a code of positions, but this is a confusion, for the units assigned to this code are mere elements of expression; they concern the location of a stroke, not of something in the depicted world.

tional code differ, not only for different personages (p. 122), but also for four different viewing positions (p 7f; intermediate positions are non-pertinent). Oomen and Kloepfer, on the contrary, consider the emotional code to be identical for all the personages, and to possess variants for only two viewing positions, the profile and the front view (Kloepfers Vor-Zeichen, p. 133; the rear view is ignored). Although Gauthier studies a particular comics book, while Oomen and Kloepfer refer to Peanuts in general, there is no doubt that they all intend to describe the Peanuts code, so it is disturbing to find their conclusions to be so divergent. Without having access to the material studied, we can of course not even begin to determine who is right. However, it is conceivable that the great number of free variants for each one of the emotional expressions, which Oomen (1975:251 ff) discovers, results from her ignoring the differences due to the personages having the emotions. But, on the other hand, there are incoherences, or at least a lack of clarity, also in Gauthiers (1976:122 ff) argument. Although he says the emotional expression of the eye is different for different characters, in one particular case, that a Charlie Brown and Lucy, he explains the difference as a result of their having different temperaments; also the mouth expressions of different personages differ, but again, in the case of Lucy, this is said to be due to a difference in what there is to be expressed. It is not at all clear from where Gauthier derives this information. Originally, Gauthier clearly supposed the contents to be identical for all individuals: indeed, no structural procedure can be used, if at least the content substance, in Hjelmslevs sense, is not the
0

same. This means that it is possible that not just the pictorial labels of the emotions differ from one personnage to another, but even the range of expressions included in one category, and thus the limits between the categories (as dicussed in our segmentation chapter above); but the comparison breaks down, if also the content continuum itself differs in the several cases. This does not show that Gauthier is wrong in his description, only that he employs information from sources which he fails to mention: probably some synthesized readers knowledge about the personality of Lucy. According to Kloepfer (1977: 133), there is a mouth paradigm in Peanuts, of the same discrete type as a phoneme system. Curiously, it is Gauthier, and not Kloepfer, who sets out to demonstrate the existence of such a system in practice. In the context runder Kreis (sic!), Kloepfer tells us, no matter what is put inside the lower half of it is seen as a mouth; and Gauthier (1976: 129) claims that any line inside the contour will do as well. Something like this was found by Goodnow (1977) to be literally true, when small children were doing the interpreting. It is less obvious, however, that adults are equally liberal, and even if they were, this would not demonstrate the discreteness of the mouth paradigm. In Kloepfers view, it is because we have repeatedly seen Charlie Brown with a particular mouth shape as he is saying something that has an identifiable emotional content that we come to associate each mouth shape with its peculiar emotion that is, Kloepfer thinks, indexicalities transform iconicities into conventional signs. This is reminiscent of Lotmans notion of inner transformation of a code, and of Lvi-Strausss Wagner interpreta-

tion, and it is perhaps the kind of operation that may be able to justify Gauthiers feelings about Lucy, but it is most certainly not needed at this elementary level. Cceloglu (1970) generated 60 even more abstract heads than those in Peanuts, using three types of eyes, four types of brows and five mouth types, and Americans, Japanese, and Turks agreed on the whole on how much these faces expressed pleasantness, irritation and non-receptivity. Contrary to Michottes well-known hypothesis, according to which movement is primary in the expression of emotion, Thayer & Shiff (1969) found that aggressive movements were reinterpreted when executed by happy faces. In the present context, the first study may be taken to suggest that no meaning transference is needed in order to make abstract faces interpretable; and since the second study shows that the emotional meanings of such abstract faces are not overridden by movements, it seems very doubtful that static cues could accomplish this. But, of course indexicalities may extend and specify meanings already contained in the faces (as, perhaps, in the case of Lucy). Under such circumstances, there can hardly be any justification for talking, as Gauthier does, about the features (p. 113), the digital nature (p. 114) and even the double articulation (p. 126) of the Peanuts code. And it is enough to have a look at Gauthiers own schemes (p. 132ff) to discover that the options of the Peanuts code are very rarely dependent on binary choices, as Gauthier (p. 118) wants us to believe. And yet, there is a sense in which comic strips (and, I believe, all kinds of pictures) are digital, or rather categorical. Let us begin by defending Gauthier against
0

his own self-understanding: he does apply a principle of pertinence to his text. At one point (p. 117), he tells us for instance, that he is going to consider the graphism, without taking account of the corresponding content. If he had done that, he would never have discovered any mouth paradigm, for, graphically, the different mouth shapes have nothing in common (shapes such as nothing, stroke, two strokes forming an angle, a half-circle). In spite of his intentions, he thus avoids making what Jakobson calls a numismatic, and therefore completely uninteresting, analysis. A more serious obstacle to categoricality in the Peanuts strips is their redundancy. Gauthiers assumption (p. 123) that only the mouth opening is a pertinent feature, while the variations of the nose, the forehead, and the chin are contextually determined, is completely gratuitous. If Gauthier is right when he later claims (p. 128) that there are many expressions for a single content, then the categorical, or digital, sign model, cannot be correct. But we need not conclude that pictures are irrevocably dense, in Goodmans sense. Even if something similar to the symptom model applies to pictures, there may be points at which the accumulation of evidence is, for all practical purposes, determinate. There is, however, yet another way in which categoricality sets limits to density or analogy, at least in Peanuts and many other comic strips. Or rather, there are three ways. 1) there is a lower threshold of iconicity, below which resemanticization cannot proceed; 2) there is also an upper threshold of iconicity, above which equivalence can only be globally established (which does not mean it must

be conventional below); 3) and, finally, the very form in which similarity is conveyed is categorical, i.e. the invariants are embedded in graphic material possessing its own organization and therefore offering its own possibilities and its own constraints (comparable to the case of the concurrent features choices which we quoted from Prieto above). We will have a look at these different limiting factors of iconicity in turn. The explanation of the lower threshold is the most straightforward. Quoting Matisses (1972: 171) critical remarks about painters using repeated 33s to indicate foliage, Gauthier (976: 0,5; 98:8) claims that, in such a case, there is not one feature of the picture answering to each leaf. But we already know that, strictly speaking, there never is (of course, we suppose that the requirement for features to be minimal, in some sense or other, is retained). The real point, however, is that, in a case like this, it is impossible to correlate each feature with a leaf depicted, even when the features have been integrated into the larger whole, i.e. resemanticized. The foliage is rendered only on the extensional level foliage, to the exclusion of all lower levels; but contrary to what happens in the common lavatory logotype, where there is nothing to indicate the levels below that of man and woman, the parts are here represented, not only by their common borders (indeed, these are left out), but by their holistic properties, which they produce together: something like the irregular lattice made up of numerous overlapping small and rounded shapes.48 In any case,
48 Matisses dissatisfaction with painters using this device is, I assume, grounded in his opinion that this way of establishing the 0

when parts are signified through their holistic properties, there is no way of pointing to the parts; and so we cannot distinguish individual leaves, and not even groups of leaves, in the pictures considered. What has here been termed the lower threshold of iconicity should not be confused with intensional and extensional levels and their possible downward shift. Below the threshold, the details cannot be given any interpretation in the Lifeworld. Intensional and extensional level determine at what level of typicality is situated the meaning which is conveyed and/or given prominence in the picture. Gauthier (1976: 121, 125) contends that, in Peanuts, hair is rendered in the same way as the foliage criticized by Matisse. The different hair-signs can only be further analyzed into features having no meaning of their own, Gauthier claims, for each stroke does not signify a single strand of hair. There are several problems with this parallel. To begin with, no characters hair in Peanuts is an autonomous sign, but must be referred for resemanticization, at least to the higher configurational level of the entire head. This is probably also true of the foliage made up of 33s. In the second place, that to which the picture refers is a perceptual object, and not, as Hermern would say, some elusive Dingan-sich, and the assumption that a persons hair, as a perceptual object, is made up of a determinate number of strands of hair has not much to commend itself, if we except the case of the old gentleman having
equivalence of perceptual invariants has grown all too commonplace, and that new signes plastiques are called for. He would probably welcome the use of other invariants of the same perceptual object, or even other plastic variants of the same invariant.

only one or two wisps of hair left. Seeing things configurationally is a fact of ordinary life, not only of pictures. And as soon as we abandon the conception that hair is made up of single strands of hairs, we can see that the strokes standing for hair (in the collective sense) in Peanuts do carry meaning below that extensional level (Cf. Fig. 45). There is no problem identifying the parts of these hairs, even if the limits between them are scarcely clear-cut. We can point to the fringe of all the personages, and see how the hair is arranged around the ears. The hair of personage c is combed upwards towards the top of the head. It is also possible to assign a number of properties to the different hairs: the hair of c is well-ordered, that of d is dishevelled, while bs hair could be ranged somewhere in between; they all have a lot of hair, whereas a has a small amount, but probably more than one or two tufts. And a lot more could be said, in particular, if we take the girls hair also into account. Therefore, the depiction of hair in Peanuts is not a good example of the lower threshold of iconicity; but there are such examples, for instance the eyes, to which we will return later in this section. As compared to the lower threshold, the upper threshold of iconicity is much more difficult to establish, and even to explain. It can be demonstrated in Peanuts, as long as we limit our attention to single figures: the apex of the hierarchy, where the threshold must be situated, is the personage, not the frame. Gauthier (1976: 117) observes, that dans la configuration profil droit de Charlie Brown, la tte peut approximativement sinscrire dans un cercle, le torse dans un trapze, les jambes dans
0

Fig. . Abstracted hairdo of the different Peanuts personages

un rectangle ou dans un polygone plus complexe. In spite of differences of size, proportions are constant (cf. p. 131). At another point, Gauthier (p. 120f) calls attention to the fact that the content profil droit de la tte de neuf personages intervenant dans 4 sries de Peanuts does not possess any single graphic expression; and yet all the heads are, as he goes on to say, the result of a process taking its point of departure from the circle. But then, of course, there is a common graphic expression for all the heads; as a good form, and thus a prototype, it is implicitly present (cf. Rosch 1973; 1975b). If the circle is the cognitive reference point, then it should also be significant that the heads of Charlie Brown and, perhaps, Linus, come so much closer to the ideal than the others. But in order to appreciate the meaning of this, we must first know what the circle itself stands for. For, of course the different degrees of deviation from a perfect circle in the drawings of the heads are not meant to indicate the extent to which the corresponding real heads diverge from circularity; no actual human heads are circular, in any stricter sense of the term. More plausibly, the different degrees of circularity present in the Peanuts heads are metaphors for mental properties, as is often the case in caricatures (according to Worth 98; but cf. Perkins 1975; & Hagen 1980): The problem then amounts to finding the plastic meaning of

the circle. Groupe (980: 49f gives as connotations of the circle such contents as happiness, God, and formal perfection (cf. 11.3.1.); and Gauthier (1982: 147ff), in a different context, also opts for forme parfaite. Should we then conclude that Charlie Brown is somehow (perhaps in an ironical sense) more perfect than the others? or closer to God? or simply more happy? None of these interpretations sounds particularly convincing. Another line of reasoning would point to the working of biological releasing mechanisms. According to our first variant, the circle is reminiscent of the underlying biological facial scheme, which has been found to elicit the smiling response in neonates (cf. Argyle & Cook 1976; Schaffer 1971; E. Gibson 1969). However, it would seem that even from the beginning, children pay even more attention to real faces (see E. Gibson 1969: 347ff), the dummy faces actually used were more often oval than circular, and there is no evidence that the scheme is innate (cf. E. Gibson 1969: 356). Therefore, the comparison could scarcely be revealing. A second version relates the Peanuts heads to that releasing mechanism, which, according to Konrad Lorenz, is responsible for adults finding children so cute, or herzig, and which was experimentally demonstrated to work by Hckstedt (1965). Although all the Peanuts personages have comparatively short arms and legs, and generally roundish forms (but only as far as their heads are
0

concerned), they do not seem to possess any of the four other more specific features characteristic of the Kindchenschema, notably the factor investigated by Hckstedt, viz. the greater height and convexity of the forehead, which exaggerate those of real babies. Moreover, since it is clear from the results of Hckstedt (1965: 433ff) and Fullard & Reiling (1976), that the supranormal head shapes start to elicit reactions at moments which are socially, rather than biologically important (for men, for instance, when they marry, and even more when they have their own children), there is really nothing to prove that such reactions are biologically programmed, rather than socially and culturally inculcated. The importance of this point should be clear. If the Peanuts heads could be related to an underlying biological mechanism, of whatever type, we would not know exactly what is signified by the relative closeness of the heads to a perfect circle; but we could assume the heads to be more or less reminiscent of primordial, elementary forms (of course, the circle may be an Urform for quite other reasons): The less iconical the head, in this case, that is, the less similar to an actual head, the more it would approach the head idealtype, and the more it would be natural. For when we reach the upper iconical threshold, those meanings that emerge may be biologically grounded or conventional; the important point is that they admit no resemanticization. That is to say, whereas small parts and details of the drawing can readily be taken to stand densely, in Goodmans sense, for the position of an arm, or the degree of aperture of the mouth (but see below), the general configuration, made up of a circle, a trapezium and a rectangle, must be ap-

prehended categorically, as an equivalent of a human body more precisely, as equivalent to the three categories head, trunk, and legs of the body. Even subtler is the notion of iconicity itself having a categorical form. By now we are familiar with the idea, expressed also by Gauthier (1976: 113f), that certain semantic features cannot be manifested in an expression without certain other semantic features being chosen concurrently. We know that to Prieto (1975b: 103ff), such an implicational relationship between features, which he calls a noema (in a sense rather different from that of Husserl), is a particularity of verbal language; but that, in Artificial intelligence, the same phenomenon has been claimed to be characteristic of pictures (as noted above). Gauthier, who ignores these parallels, finds a number of examples of such implicational relationships, or, as we said, informational packages, in Peanuts; but it seems that he is really confusing at least three very different things: a) That a configuration of lines stands for the eye, without there being any particular lines corresponding to the different parts of the eye (p. 118). This simply means that, in the Peanuts code, we can make our choice at the extensional level eye, not that we have to choose all the features corresponding to the different parts of the eye together. In fact, only the eye as a general shape is depicted, and the information about the parts must be supplied from our knowledge about the picture subject. The only thing new here is that the extensional level of the eye is not necessarily the same as that of other parts of the body, so that we may have various extensional levels in the same picture.
0

Fig. 46. The Peanuts mouth paradigm, according to Oomen. b) Quite another thing is that the pictorial expression for eye must, at the same time, convey such features as oeil ouvert vu de face regarde devant lui expression lgrement ahurie (p. 118). Here, the different features are all in the picture object, that is they can be seen in the picture. But the examples are yet different in other ways, so we will first consider the first two. It would seem that some implicational relationships are valid, not only for the Peanuts code and other pictures, but also for the perceptual world. One of the most fundamental essential relations which Husserl has formulated, and which Gurwitsch has insisted upon, is that which makes it impossible to perceive an object, without perceiving it from a particular point of view. Thus, it would seem, it is not only impossible (but only in a sense, as we shall see) to depict something without depicting it from one particular side (or a few particular sides, as in Cubism and split representation), but there is no other way it can be perceived either. The implications here are simply taken over from the Lifeworld. Not that there are not some less typical kinds of pictures (such as the notation systems used for gestures), in which a thing and the aspect it presents are dissociated; but the more a picture respects the conditions of spatiality in the Lifeworld, the closer it will be to realize the prototypical concept of a picture. The case of the open eye is somewhat different. Like Mussets proverbial door, an eye must be open or closed. In a picture, however, we may choose to render the eye as a simple dot (which means it is an element of a larger whole, in which appresentation takes place at a higher extensional level, perhaps that of the head), in which case we do not have to decide if we want the depicted eye to be open or closed (this is like case a above). At lower levels, however, the decision has to be made. It will be noted (somewhat in Prietos spirit), that we cannot speak about the openness of the eye without also mentioning the eye, unless it is clear from the context that it is the eye we are concerned with. We can, however, mention the eye without specifying if it is open or closed. In a picture, on the other hand, once we have opted for a certain level of detail, the eye cannot be shown without being shown as open or closed; thus, there is mutual implication of the features concerned. In the second place, each time we want to mention its openness, we are compelled to repeat the eye, even if it is absolutely obvious from the context (there are rare exceptions to this, but they are rhetorically marked). All this is to suggest that the prototypical picture is bound up with the essential relations of the Lifeworld; and no doubt this was what Husserl meant, when he said that at least some of the similarities in a picture must be anschaulich. Although some elements in Klees pictures can be described, according to Thrle0

Fig. . The complete mouth paradigm, as analysed in Sonesson a (neutralized variants below the thick line)

mann (1982), as things, pine-trees, and minerals, the pictures that contain them are clearly much further from the picture prototype than Rembrandts Bathscheba, or even Matisses cut-out Nu bleu IV. c) Then there is the fact, that the same configuration may serve to convey meanings such as the eye of a person and his excitement; or a persons identity and the movement he is presently accomplishing (cf. Gauthier 1976: 118). This is not quite the same case as the foregoing: the strokes that complete the sign of the eye are usually only part of the reason we have for attributing excitement to the personage (and only part of the strokes making up the eye may be relevant to the meaning excitement). Unlike what happens in verbal language, signs are embedded into other signs, and this is really only a reflection of the contextuality already found in the perceptual world. Note that this is different from compound words as found in verbal language, for the units which are put together to form complex signs at the same time function independently, to convey their own meaning; in addition, they often contribute concurrently to a number of different complex signs. Neither is this like the phenomenon called connotation by Eco, and sym-

bols by Todorov and Sperber, and which we have preferred to term contextual implication (cf. Lecture 5), for the composite sign is in no sense more indirectly given, but can be directly pointed to, and resemanticized, on the expression plane of the picture, as can the identity between it and the elementary signs making it up. Again, the more closely a picture respects these Lifeworld conditions, the more prototypical it is as a picture. Of course, many pictures deviate from this: the difference between Chaplin and Hitler, which, even as appearance is concerned, accrues from an infinite number of details, is reduced by the cartoonist to the binary opposition of the bowler hat and the fringe (cf. Fig. 14.). On the other hand, Reiser would seem to exaggerate symptoms of sexual difference, when he attributes an entirely different body scheme to the beautiful young woman and to the man (together with the old hag): Going in the opposite direction, lavatory logotypes usually reduce sexual difference to a question of hairdo and the wearing of a skirt. At least the last two factors considered must really be said to favour iconicity, in spite of the possibility of exceptions. In order to see how the form of iconicity
0

may yet be categorical, we will now return to the mouth paradigm, our point of departure in this section. In fig. 46., the members of this mouth paradigm, which Oomen (1975: 250f) claims to have found in Peanuts, have been listed (those of the side view, which are the only ones we will discuss here). From the point of view of the emotional code, Oomens mouth paradigm only comprises four categories: positive, negative and neutral feelings, and effort. Moreover, variants c, d, and e designate a speaker, e at the same time one who shouts, and f is a zero form (apparently also from the point of view of content). Gauthier (1976:118, 132, 138), who exhaustively presents the analysis of Charlie Brown only, and whose classification relies on the degrees of opening, discovers five types, corresponding to f, b, c, and e, together with an even more extreme variant. If we ignore type g, which is too special, the whole mouth paradigm can apparently be reduced to the result of combining two dimensions, one comprising neutral, positive and negative feelings, and the other made up of the five degrees of aperture, including the zero position. Since Gauthier (p. 132) does not attend to the first of these dimensions, our reanalysis must also involve substituting fig. 46d for fig. 46b as the neutral variant of degree one aperture. Of course, as can be seen from fig. 47 above, the distinction between the different positions on the dimension of feelings is neutralized at all other degrees of aperture, which explains that Gauthier ignores the former dimension. Viewed in this way, the mouth paradigm is really similar to a linguistic paradigm, not, as Kloepfer suggests, to that of phonemes, but rather to that of grammati-

cal endings, as for instance in the case systems of Latin or Russian, with their largely neutralized expressions for combinations of features. At degree one aperture, the two dimensions are independent, in the sense that they can be selected by separate choices. That is, both meanings must be carried by the single stroke in the head circle, so that they are spatially inseparable; but it is the fact of it being a single stroke that conveys the degree one aperture, while the orientation of the stroke (from the left above to the right below, or the reverse) and the direction of the curvature together mark the quality of the feeling involved. Perhaps this can be compared to the case of suprasegmental traits in verbal language, which are functionally independent of the phonemes, although temporally fused with them in the speech chain. In the case paradigm, such independent features would of course follow each other in time, and so be more obviously distinct. On the other hand, just as in the case paradigm, the values of the two dimensions can be separated throughout the mouth paradigm, only because they do have independent expression somewhere in the paradigm. If we were to believe Oomen and Gauthier, however, this only happens once in our paradigm: for the other degrees of aperture, only one variant each is attested. Therefore, we are forced to conclude that but for the degree one aperture the emotional dimension is neutralized everywhere. It would seem that this neutralization has nothing to do with content, i.e. real facial expression, but must be entirely explained from the properties of the graphic expression plane. As for degree zero, it is obvious that any variation on nothing must give nothing. In the case of the other de0

grees, our hypothetical forms show that, when the expression features of emotion, viz. curvature and orientation, are combined with the features standing for the different degrees of aperture, the result is increasingly similar, and in the end indistinguishable. In reality, the openness of the mouth cannot vary so widely as suggested by the cartoon heads. And although there is a movement which could be described as pulling up the corners of ones mouth, and which is taken to convey happiness, to pull the corners down would require the assistance of both hands, and even so it is impossible to bring them much further down. Perhaps, then, this is one of those cases referred to by Darwin and Jakobson, where one sign is motivated, and the other is created to stand for the opposite value, out of the negated features of the first sign. Cceloglu (1970: 98), in his test using schematic heads (only in front view, admittedly), found that Americans, Japanese, and Turks agree in attributing pleasantness to mouths having the corners turned upwards; and while they also concur in taking the absence of corners turned upwards to express the opposite of pleasantness, only the Turks attribute a specific meaning, viz. the negation of irritation (!), to corners turned downwards. On the other hand, it would be reasonable to suppose the mouth to participate in a more general, figurative dimension, in which the direction upward stands for happiness, and its inversion for depression (cf. Arnheim 1966: 70f). Then question then becomes how far the degrees of aperture of the mouth can be taken to be iconical. To begin with, the variants -4 do share one topological property with the real mouth, i.e. openness,

only transferred to a graphic expression substance. Furthermore, when the degree of aperture of the cartoon mouth augments, this is clearly meant to indicate an augmentation of the same general kind in a real mouth. But while a real mouth increases continuously its degree of opening, the Peanuts code only provides for four (or five) degrees of aperture; thus, it digitalizes real-world density. In addition, the zero form, which Gauthier (1976: 119) explains from child logic, which leaves out that which is not important for the moment, does not depict any possible state of a real mouth, and the degree one forms only acquire that property of closure, which they probably express, in structural opposition to the other variants. There are also curious modifications in the mouth sign itself; for in type , the pigment is the mouth, with the contours forming the outer limits of the lips, but in type , there is a pigment line with two contours for each one of the lips. Nor do the increasing roundness of the contours, and their perfect circularity in type 4, correspond to reality. It is true that, seen from the side, the mouth may take on a more rounded character when the degree of opening increases; but when it approaches the point of maximal aperture, the side view on the contrary becomes flat. In the rendering of the mouth, as it appears in the Peanuts code, iconicity and categoricality can scarily be dissociated; and when the similarity to the real mouth is diminished, this is in part because the graphic means do not permit a closer resemblance (as in the case of the many neutralized variants), and in part because the code relies on other, more physiognomic or figurative iconicities. Before closing our discussion of the
0

categorical form of iconicity in Peanuts, we will now return to Gauthiers claim that features like eye and front view cannot be expressed separately in the Peanuts code. We said (under point b above) that this was due to the Lifeworld principle, according to which anything perceived must be perceived from a particular point of view. Indeed, in real perception, the eye seen from the side does not differ from an eye seen from the front in a few details, or in having some particular tag, but the noema of the one is a complete transposition of all the elements of the other. But is this really true about the Peanuts code? In fact, when arguing for his positional code, Gauthier (1976: 121) says it is needed, because the nose is drawn identically from the front and in profile, and only differs through its position from the ear. Thus, the thing and the perspective on it would be separable in the case of the nose and the ear, but not in that of the eye. But in order to understand what is really at issue here, we must begin from another end. Gauthiers positional code is, as we indicated above, not a code in the same sense as his other codes, for its contents are not features of perceptual reality, but of the pictorial expression plane; they only serve to tell us how the expression planes of the other codes are to be taken. Moreover, the code cannot accomplish that for which it was designed, for position as such is unable to disambiguate the Peanuts drawings; only relative position, which means position relative to a higher configuration, will do, that is, what we earlier termed resemanticization. Indeed, both fig. 5a and 5b are ambiguous: the former may show a nose in a face seen from the front or an ear in profile; and the latter may show the

ear of a face seen from the front or a nose in profile; these possibilities are combined differently in fig. 5c-d (cf. also the frames reproduced in Gauthier 1976: 131). It would be too simple to say, then, that in the Peanuts code, ear and nose have the same expression plane (which in itself would be very counter-iconic), but that they are distinguished by the additional feature of position, the ear being outside the contour and the nose in the middle of the circle (or the reverse). In fact, both identity and perspective of nose and ear can only be determined in relation to all the other dots and strokes standing for the other facial traits. The case of the eye does not seem to be very different: most dots in the head circle are eyes, and front and profile view are distinguished by the former containing two dots, which are relatively more central than the single dot of the latter case. Consider an apparently more subtle case: the characteristic female head of Brtechers drawings (fig. 5e). Although they look very different, the Brtecher face as seen directly from the side and in a view (the frontal view is almost non-existent) actually only differ in the latter presenting two circles, rather than one, to indicate the eyes, and the distance between the front and the back outlines increasing. The nose and other facial traits are identical in the two perspectives; there is a zero form of the mouth for non-speakers, and different degrees of aperture, but these seem to be identical for both perspectives. More importantly, the outlines of the head (and at least sometimes of the entire upper body) are identical in the two variants, only that the distance between them is modified. It could therefore be said that, while a particular, comparatively iconical silhouette line


is a sign of a head, and of a particular personage, the distance between the outlines, together with the number of small circles inside the figure, stand quite independently for the perspective from which the head is seen. It is true that in real-life perception too, there is more chance of seeing two eyes from a angle than from the side; and that, when projected onto a flat surface, a threedimensional body in perspective will occupy more space, but while these facts are in themselves iconical in the picture, they are used categorically, to the exclusion of the infinite number of other modifications which follow upon a shift in perspective, to indicate personal and bodily identity as a different phenomenon from that of viewing distance. Once again, we have seen that the very form of iconicity may be categorical. On the other hand, we could perhaps say that this categorical form may, in turn, be iconical. It is not true, strictly speaking, that the Peanuts code or the Brtecher drawings override the Lifeworld postulate, which says that all perception is perception from a particular point of view. In both cases, all noses are rendered from a particular point of view: from the side. But while this side view is the only one that appears in the expression, and in the picture object, it is made to represent all conceivable referential noses, in all perspectives that may be given to the picture subject. Taking our point of departure from the valuable analyses of the Peanuts strip by Gauthier, Oomen, and Kloepfer, we have delved deeper, in this section, into the intricacies of the interplay of iconical and categorical elements in pictures. We found there were both an upper and a lower threshold of iconicity, at least in some pic-

tures like Peanuts, and also a categorical form given to the iconical elements themselves. The latter case, as the most complex, was thoroughly discussed using as an illustration the mouth paradigm in Peanuts, as well as some material from Brtechers drawings. The mutual implication of some features in the perceptual Lifeworld was shown to suffer derogation in some pictures, by being transformed into categorical signs, which can be independently chosen, although they have to be realized in a common graphic substance as part of the same configuration. Just as the results of our Matisse study, the conclusions of this section cannot easily be generalized beyond the material from which they have been derived. Both studies, however, serve to direct attention to some of the complexities of pictorial signs. Summary Starting out from that great distinguishing property of iconicity described by Peirce as the conveying of more information than is put into it, termed exhibitive import by Greenlee, I showed against the latter than this is indeed a property not normally found in verbal semiosis. However, that which renders possible exhibitive import also puts limitations on it. Indeed, pictures at the same time render and transform the perceptual reality of the Lifeworld: through the pruning of some parts of the intensional and extensional hierarchies, the downward shift of both base levels, and the upward shift of the extensional base level. I have tried to illustrate the way in which pictures sometimes reanalyze, or resegment, the common objects of the perceptual world, as for instance the human body.


After a rather thorough examination of Matisses cut-out Nu bleu IV, a few more superficial considerations on Charlie Brown and on the personages of Brtecher and Reiser were added. Moreover, we explored the parallels between the ways in which some content features implied, and were implied by, other content features in the make-up of pictures and verbal language. Such coming together of content features, variously termed informational packages, chunks and noemata were not, it turned out, restricted only to verbal language, as Prieto claimed, nor to pictures, as has been taken for granted in Artificial Intelligence. What is characteristic of pictures, or at least of prototypical pictures, however, is that they follow as closely as possible the rules of feature coexistence prevailing in the perceptual Lifeworld. But at the same time as we come across this essential addition to the traditional criteria of iconicity and density, we discover important limits to both the latter criteria, in the upper and lower thresholds of iconicity, and in the categorical form given to iconicity itself. These concepts are elucidated in our discussion of the three analyses of the Peanuts code, suggested by Gauthier, Oomen, and Kloepfer. The interest of this discussion has so far only been to suggest procedures of analysis that may be applied to other (kinds of) pictures, and to indicate the types of differences that we may expect to find between them. The real result of this section is, from a semiotic point of view, the principles that we have formulated, in order to account for the intuitive evaluations made; but these will only acquire value, as they are tried out on other pictures and shown to be able to account for them.

References
Aicher, Otl, & Krampen, Martin, (1977) Zeichensysteme der visuellen Kommunikation. Stuttgart: Verlagsanstalt Alexander Koch. Argyle, Michael, & Cook, Mark, (1976) Gaze and mutual gaze. Cambridge: Cambridge University Press. Arnheim, Rudolf (1969) Visual thinking. Berkeley & Los Angeles: University of California Press. _ (1974) Art and visual perception. New edition. Berkeley & Los Angeles: University of California Press. Aron, Raymond, (1938) Essai sur la thorie de lhistoire dans lAllemagne contemporaine. Paris: Vrin. Austin, J.L. (1962a) How to do things with words. London: Oxford University Press. (1962b) Sense and sensibilia. London: Oxford University Press. Bal, Mieke (2003) Visual essentialism and the object of visual culture. In Journal of visual culture 2(1), 5-32. Barthes, Roland (1964a) Rhtorique de limage, in Communications 4, pp. 40-5. Also in Barthes 98, pp. 5- 4. (1964b) Elments de smiologie , in Communications 4, pp. 9-35. Republished in Barthes, Roland, Le degr zro de lcriture - Elments de smiologie, Gonthier, Paris 965. (1966) Introduction lanalyse structurale des rcits, in Communications 8, -7. (1972): Smiologie et mdecine, in Les sciences de la folie, Bastide, Roger, ed., 3746. Paris & Le Haye: Mouton.. Bartlett, F. C: (1958) Thinking. An experimental and social study. London: Allen & Unwin Ltd. Bastide, Roger, ed. (1962) Sens et usages du terme structure. The Hague: Mouton Baxandall, Michael, (1972) Painting and experience in fifteenth century Italy. Oxford: Clarendon press. Bayer, Udo, (1975). Lessings Zeichenbegriffe und Zeichenprozesse im `Laokoon und ihre Analyse nach der modernen Semiotik. Diss., Stuttgart. (1984) Laokoon - Momente einer semiotischen sthetik, in Das Laokoon-Projekt. Gebauer, Gunter, ed.,.58-0. Stuttgart Metzlersche Verlagsbuchhandlung.


Bellour, Raymond, & Clment, Catherine, eds., Claude Lvi-Strauss. Paris: Gallimard. Benveniste, mile, (966) Problmes de linguistique gnrale. Paris: Gallimard. (1969). Smiologie de la langue, in Semiotica I:, pp. -; and I., pp. 7-35. Bertin, Jacques (1967) Smiologie graphique. Paris & La Haye: Gauthier-Villars/Mouton. Second edition 974. Bierman, Arthur K. (1963). That there are no iconic signs, Philosophy and phenomenological research, XXIII:: 43-49. Bois, Yves-Alain (1992). The Semiology of Cubism, in Picasso and Braque. A symposium. Zelevansky, Lynn (ed.). 169-208. The Museum of Modern Art, New York. Borges, Jorge Luis (1974) Obras completas. Buenos Aires: Emec. Brown, Roger, (1958) Words and things. New York: The Free Press. (1976) Reference, in Cognition 4, 125-153. Bucher, Bernadette, (1977) La sauvage aux seins pendants. Paris: Hermann. Bulmer, Ralph, (1979) Mystical and mundane in Kalam classification of birds, in Classifications in their social context, Ellen, R., & Reason, D., eds., 57-79. London, New York, and San Francisco: Academic Press. Burks, Arthur, (1949) Icon, index, symbol, in Philosophy and phenomenological research, IX:4, 673-689. Carnap, Rudolf (1958) Introduction to symbolic logic. New York: Dover Publications. Cceloglu, D. M., (1970) Perception of facial expressions in three different cultures, in Ergonomics, 3, , 93-00. Clark, Herbert, & Clark, Eve, (1977) Psychology and language. New York: Harcourt Brace Jovanovich Inc. Costall, Alan (1989). A closer look at direct perception. In Cognition and social worlds, Angus Gellatly, Don Rogers, & John Sloboda, (eds.), 10-21..Oxford: Clarendon Press. Danto, Arthur (1979) Pictorial representation and work of art, in In Perception and pictorial representation, C. Nodine, & D. Fisher, (eds.), 4-16. New York: Praeger Publishers. Damisch, Hubert (1979) Sur la smiologie de la peinture. In A semiotic landscape/Panorama smiotique. Actes du premier congrs de lAssociation internationale de smiotique, Milan 1974., S. Chatman, U.

Eco, U., & J.M. Klinkenberg, (eds.), 128136. The Hague, Paris, & New York: Mouton. Darcy, Eric, (1963) Human acts. Oxford The Clarendon Press. Dascal, Marcelo, (1978) La smiologie de Leibniz. Paris: Aubier Montaigne. Degerando, J.M. (1800). Des signes et de lart de penser considrs dans leurs rapports mutuels. Paris: Goujon fils Eco, Umberto (1968). La struttura assente. Milan: Bompiani. Swedish translation: Den frnvarande strukturen. Lund: Cavefors 97. (1976). A theory of semiotics. Bloomington: Indiana University Press. - (1984). Perspectivas de una semitica de las artes visuales, in Revista de esttica, 2, 54. (2000) Kant and the platypus. New York: Harcourt Brance & Co. Eisenring, Max, (1942) Johann Heinrich Lambert und die wissenschaftilche Philosophie der Gegenwart. Zrich; Mller, Werder & Co AG. Elgin, Catherine (1983) With reference to reference. Indianapolis: Hacekell Pub. Co. Ellen, Roy, (1977) Anatomical classification and the semiotics of the body, in The anthropology ot the body, Blacking, John, ed., 343-373. London, New York & San Francisco Academic press. (1979) Introductory essay, in Classifications in their social context, Ellen, R., & Reason, D., eds., 1-32. London, New York, and San Francisco: Academic press. Espe, Hartmut, (1983a) Empirische Analyse visueller Zeichen: der Einfluss der Belichtungsdauer bei der Vergrsserung auf die affektive Bedeutung von Schwarz-WeissFotografien, in Visuelle Kommunikation und/oder verbale Kommunikation. Krampen, M ed., 9-. Hildesheim & Berlin: Olms Verlag/Hochschule der Knste. (1983b) Realism and some semiotic functions of photographs, in Semiotics unfolding. Proceedings of the second congress of the Internatlonal association for semiotic studies, Vienna, July 979. Borb, Tasso ed., Volumes III, 1435-1442. Berlin, New York & Amsterdam: Mouton, Floch, Jean-Marie, (1978) Roland Barthes: Smiotique de limage, in Bulletin du Groupe de recherches smio-linguistiques


4/5, pp. 7-3. (1985): Petites mythologies de lil et lesprit. Paris: Hads. 986a: /entries in/ Greimas, A.J., & Courts J., eds. Smiotique - Dictionnaire raisonn de la thorie du langage. Volume II. Paris: Hachette. 986b: Les formes de lempreinte. Prigueux: Pierre Fanlac. Frege, Gottlob, (1966) Funktion, Begriff, Bedeutung. Gttingen: Vanderhoeck & Rupprecht. Fresnault-Deruelle, Pierre, (1977) La chambre bulles. Paris: Union gnrale dditions. Fullard, W., & Reiling, A., (1976) An investigation of LorenzBabyness in Child development, 47;4, 9-93. Gambarara, Danile, (1979) Lobjet de la smiologie et lobjet de la linguistique in A semiotic landscape/Panorama smiotique. Actes du premier congrs de lAssociation internationale de smiotique, Milan 1974., S. Chatman, U. Eco, U., & J.M. Klinkenberg, (eds.), 278-281. The Hague, Paris, & New York: Mouton. Gardner, Howard (1973a) The quest tor mind. Piaget, Levi-Strauss, and the structualist movement. New York: Knopf. (1973b) The arts and human development. New York, London, Sidney, Toronto: Wiley & Sons. (1980) Artful scribbles. New York: Basic Books. Garner, Wendell, (1974) The processing ot information and structure. New York: Wiley & Sons. (1976) Interaction of stimulus dimensions in concept and choice processes, in Cognitive psychology, 8:, 98-3. Gauthier, Guy (1976:) Les Peanuts: une graphisme idiomatique, in Communications, 24, 08-39. (1979) Initiation la smiologie de limage. Paris: Les cahiers de laudiovisuel. (1982) Vingts leons sur limage et le sens. Paris: Edilig. Geertz, Clifford, (1983) Local knowledge. Further essays in Interpretative anthropology. New York: Basic books. Gibson, Elenor (1969) Principels of perceptual learning and development. New York: Appleton-Century-Crafts. Gibson, James (1966). The senses considered as perceptual systems. Bonston: Houghton Mifflin Co.

(1971). The information available in pictures, Leonardo, 4:, 7-35. (1978). The ecological approach to visual perception of pictures, Leonardo, 4:: 735. (1982). Reasons for realism. Selected essays of James J. Gibson, Edward Reed, & Rebecca Jones, (eds.). Hillsdale, New Jersey, & London: Lawrence Erlbaum Ass. Gipper, Helmut, Sessel oder Stuhl, in Gipper, H., ed., Sprache Schlssel zur Welt. Schwann, Dsseldorf 959; pp 7-9. Glass, A. L., Holyoak, K. J., & Santa, J. L., (1979) Cognition. Reading, Mass.: Addison-Wesley Publishing Co. Gombrich, E-H. (1960). Art and illusion. London: Phaidon Press. (1963) Meditations on a hobby horse. London: Phaidon. Goodman, Nelson (1968). Languages of art. London: Oxford University Press. - (1970): Seven strictures on similarity, in Experience and Theory, Foster, L., & Swanson, J.W., (eds.), 19-29: Cambridge. Mass., University of Massachussets Press. - (1984) Of mind and other matters. Cambridge. Mass., University of Massachussets Press. Goodnow, Jacqueline, (1977) Childrens drawing. Glasgow: Fontana/Open books, 977. Goudge, Thomas A. (1965) Peirces Index, in Transactions of the Charles S. Peirce Society, Amherst, Mass.; 5-70. Greenlee, Douglas (1973). Peirces concept of sign. The Hague & Paris: Mouton. Gregory, R.L. (1966). Visual illusions. In New Horizons in pychology, 1, Brian Foss, (ed.), 68-96. Harmondsworth, Penguin. Second edition 973. Greimas, A.J. (1970) Du sens. Paris: Seuil. & Courts, Joseph, (1979) Smiotique: Dictionnaire raisonn de la thorie du langage. Paris: Hachette. Groupe (Dubois, J., Edeline, Fr., Klinkenberg, J.M., Minguet, Ph., etc.) Rhtorique gnrale. Paris: Larousse. (1978): Douce bribes pour dcoller en 40 000 signes, in Revue dsthtique, 3-4, pp. 4 (1979) Iconique et plastique: sur un fondement de la rhtorique visuelle, in Revue dsthtique, -, 73-9. (1980) Plan diun rhtorique de limage, in Kodikas/Code, 3, 980; pp 49-6. (1985) Structure et rhtorique du signe ico

nique, in Parret, H., & Ruprecht, H-G., eds., Exigences et perspective de la smiotique. Recueil dhommages pour A.J. Greimas. Volume I, pp.449-46. Amsterdam: Benjamins Publishing Co. (1992). Trait du signe visuel. Pour une rhtorique de limage. Paris: Seuil. Gurwitsch, Aron (1957) Thorie du champs de la conscience. Bruges: Descle de Brouwer. Hagen, Margaret (1979). A new theory of the psychology of representation art. In Perception and pictorial representation, C. Nodine, & D. Fisher, (eds.), 196-212. New York: Praeger Publishers. (1980). Generative theory: a perceptual theory of pictorial representation. In The perception of pictures, Volume II: Drers devices, Margaret Hagen, (ed.), 3-46. New York: Academic Press. Halliday, M.A.K., (1967-68): Notes on transitivity and theme in English , in Journal of linguistics, III:, pp. 37-8; III:, pp. 9944; IV:, pp. 79-5. Hermern, Gran (1983). Aspects of aesthetics. Lund: Gleerups. Hirsch, E. D, (1967) Validity In Interpretatlon. New Haven and London: Yale university press. (1976) The aims of interpretations. Chicago and London: University of Chicago press. Hjelmslev, Louis (1943). Omkring sprogteoriens grundlggelse. Copenhagen. New edition: Copenhange: Akademisk forlag 966. (1953). Introduction la discussion gnrale des problmes realtifs la phonologie des langues mortes. In Acta Congressus Madvigiani. Proceedings of the Second International Congress of Classical Studies. Copenhagen 954, 0-3. (1959). Essais linguistiques. Copenhagen. Second edition: Paris, Minuit 97 (1973) Essais linguistiques II. Copenhangen: Nordisk sprog- och kulturforlag. Hochberg, Julian (1972). The representation of things and people. In Art, perception, and reality, E.H. Gombrich et al., (eds.), 47-94. Baltimore and London: John Hopkins University Press. (1974). Organization and the Gestalt tradition. In Handbook of perception, Volume I, E. Carterette, & M. Friedman, (eds)., 180-210. New York & London, Academic

Press. (1978). Perception. Second, revised edition. Englewood Cliffs, Prentice Hall, Inc. (1980). Pictorial functions and perceptual structures. In The perception of pictures, Volume II: Drers devices, Margaret Hagen, (ed.), 47-93. New York: Academic Press. Hoffman, Donald (1998) Visual Intelligence. How We Create What We See. New York: Norton. Holenstein, Elmar, (1974) Jakobson ou le structuralleme phnomenologique. Paris: Seghers. (1976) Linguistik/Semiotik/Hermeneutik. Frankfurt/M. :Suhrkamp. Howles, Richard (2003) Visual culture. Oxford: Blackwell. Husserl, Edmund (1913) Logische Untersuchungen. Tbingen: Niemeyer. Fnfte Auflage 1968. (1939). Erfahrung und Urteil. Prag: Academia Verlagsbuchhandlung. (1980) Phantasie, Bildbewusstsein, Erinnerung. Husserliana XXIII. The Hague: Nijhoff. Hckstedt, Brbel, (1965) Experimentelle Untersuchungen zum Kindchenschema, in Zeitschrift fr experimentelle und angewandte Psychologie, Xl:3, 965; pp 4450. Jakobson, Roman (1942). Kindersprache, Aphasie, und allgemeine Lautgesetze. Uppsala: Uppsala University. (1963) Essais de linguistique gnrale. Paris: Minuit. (1964). De la relation entre signes visuels et auditifs. In Essais de linguistique gnrale II. Minuit: Paris; 04-. (1965a) la recherche de lessence du langage. In Diogne 5, 965: -38 (1965b) Poesie der Grammatik und Grammatik der Poesie. In Mathematik und Dichtung, Gunzenhuser, R, & Kreuzer, H, (eds.), 21-32. Mnchen: Nymphenburger Verlagsbuchhandlung. (1976). Six leons sur le son et le sens. Paris: Minuit. (1979) Coup dil sur le dveloppment de la smiotique, in A semiotic landscape/ Panorama smiotique. Actes du premier congrs de lAssociation internationale de smiotique, Milan, juin 1974. Chatman, S, Eco, U., & Klinkenberg, J.M., (eds.); The


Hague, Paris & New York Mouton 1979; pp 3-8. , Fant, Gunnar, & Halle, Morris (95) Preliminaries to Speech Analysis. Cambridge, Mass.: The MIT Press. Janlert, Jan-Erik, (1985) Studies in knowledge representatlon. Ume: Institute of information processing. Kaemmerling, Ekkat, (1979) ed., Bildende Kunst als Zeichensystem l: Ikonographie und Ikonologie, Kln: Dumont Buchverlag. Kennedy, John M. (1974). A psychology of picture perception. San Francisco: JosseyBas, Inc. Kloepfer, Rolf, (1977) Komplentaritt von Sprache und Bild am Beispiel von Comic, Karikatur und Reklame, in Zeichenprozessese Posner, Roland, & Reinecke, Hans-Peter, eds, 9-45. Wiesbaden: Athenaion. Koestler, Arthur, (1964) The act ot creation. London: Hutchinson & Co. Pan books Ltd., London 966. (1978) Janus. A summing up. London: Hutchinson & Co Kolers, Paul (1977). Reading pictures and reading text, in In The arts and cognition, D. Perkins, & B. Leondar, (eds.), 136-164.. Baltimore & London: John Hopkins University Press. Krampen, Martin (1979) Meaning in the urban environment. London: Pion Ltd. Krauss, Rosalind (1992). The Motivation of the Sign, in Picasso and Braque. A symposium. Zelevansky, Lynn (ed.). 262-288. The Museum of Modern Art, New York. Kress, Gunther, & van Leeuwen, Theo (1996): Reading images. The Grammar of Vusual Design. London & New York: Routledge. Kmmel, Peter (1969) Struktur und Funktion sichtbarer Zeichen. Quickborn: Schnelle. Lakoff, George (1972) Hedges: a study in meaning criteria and the logic of fuzzy concepts, in Petanteau et al., eds. Papers trom the eighth regional meeting ot the Chicago linguistic society, April 4-6, pp 83-8. (1987). Cognitive models and prototype theory. In Concepts and conceptual development: ecological and intellectual factors in categorization, Ulric Neisser, (ed.), 63-100. Cambridge & London: Cambridge University Press;

& Turner, Mark (1989). More than cool reason. A field guide to poetic metaphor. Chicago & London: University of Chicago Press. Lane, Harlan, (1976) The wild boy of Aveyron. Cambridge, Mass.: Harvard university press. Larsen, Peter, (1976) Billedanalyse, in Olivarius, P. Rasmussen, O., & Rugholm, P., eds., Massekommunkation. Dansklrerfreningen, Kpenhavn; pp. 7-8. Lessing, G. E., (1766). Laokoon oder ber die Grenzen der Malerei und Poesie. Berlin. Philipp Reclam Jun., Stuttgart 964. Lvi-Strauss, Claude (1958) Anthropologie structurale, Paris Plon. (1966) Le triangle culinaire, in LArc, 26, 199. (1975), La voie de masques. Genve: Skira. (1979) Myth and meaning. University of Toronto press, Toronto 978. Schocken books, New York (1983) Le regard loign. Paris: Plon. (1984) Paroles donnes. Paris: Plon. Lindekens, Ren (1971) Elments pour une smiotique de la photographie. Didier/Aimav, Paris & Bruxelles. (1976) Elments de smiotique visuelle. Paris: Klincksieck. Lombardo, Thomas (1987). The reciprocity of perceiver and environment. Hilldale, New Jersey, & London: Lawrence Erlbaum Ass. Lurat, Liliane, (1968) Evolution du graphisme entre quatre et cinque ans: les figurations, in Journal de psychologie normale et pathologique, LXV: 4, 433-448. - (1970) Evolution de lactivit graphique entre 5 et 6 ans: multiplications et diversifications des ralisations: in Journal de psychologie LVII: 3, 35-34. - (1974) Etudes de lacte graphique. Paris & La Haye: Mouton. Malson, Lucien, (1964) Les enfants sauvages. Paris: Union gnrale dditions. Matejka, L., & Titunik, l., eds. (1976) Semiotics of art. Prague School contributions. Cambridge, Mass.: The MIT press. Matisse, Henri, (1972) crits et propos sur lart. Texte, notes et index tablis par Dominique Fourcade. Nouvelle dition revue et corrige. Paris: Hermann. Merleau-Ponty, Maurice, (1960) Signes. Paris: Gallimard.


 (1969) La prose du monde. Paris: Gallimard. Metz, Christian (1968), Essais sur la signification au cinma /I/. Paris: Klincksieck. Miller, G. A., (1967) The psychology of communication. New York & London: Basic books, Inc. & Johnson-Laird, Ph., (1976) Language and perception. Cambridge. Mass., The Belknap press. Mirsoeff, Nichaolas (1999) Introduction to visual culture. London: Routledge. Mounin, Georges (1970) Introduction la smiologie. Paris: Minuit. Mukaovsk, Jan, (1974) Studien zur strukturalistischen sthetik und Poetik. Mnchen: Hanser Verlag. (1976) The essence of the visual arts, in Semiotics of art. Prague School contributions. Matejka, L., & Titunik, l., eds., 944. Cambridge, Mass.: The MIT press. Naus, M. & Shilman, R, (1976) Why a Y is not a V. A new look at the distintive features of letters, in Journal of experimental psychology: human perception and performance, :3, ss 394-400. Neisser, Ulric (1967). Cognitive psychology. New York: Appleton-Century-Crofts. (1976). Cognition and reality. San Fransisco, Freeman & Co. (1987). From direct perception to conceptual structure. In Concepts and conceptual development: ecological and intellectual factors in categorization, Ulric Neisser, (ed.), -4. Cambridge & London: Cambridge University Press. Norrick, Neal, (1983) Principles of motivation, in Semiotics unfolding. Proceedings of the Second congress of the International Association for Semiotic Studies, Vienna, July 1979, Borb, Tasso, ed., Berlin, New York & Amsterdam: Mouton 983; volume I, pp. 9-36. Nyman, Alf, (1951) Grnsbegrepp och renodling inom vetenskapen. Avec un rsum en francais. Lund: Gleerups. Nth, Winfried, (1975) Semiotik. Tbingen: Niemeyer Verlag. (1977) Dynamik semiotischer Systeme. Stuttgart: Metzlersche Verlagsbuchhandling. Olivier, Fernand, (1974) Le dessin enfantin estil une criture?, in Enfance, 3-4-5, 83-6. Oomen, Ursula, (1975) Wort-Bild - Nachricht: semiotische Aspekte des Comic Strip Pea-

nuts, in Linguistik und Didaktik, 6,, 4759. OToole, Michael (1994), The language of displayed art. London: Leicester University Press.. Panofsky, Erwin, (1955) Meaning in the visual arts. New York: Doubleday Anchor. Pavel, Thomas (1989), The Feud of language, Oxford: Blackwell. Peirce, Charles Sanders, (1931-58) Collected Papers I-VIII. Hartshorne, C, Weiss, P, & Burks, A, (eds.). Cambridge, Mass. Harvard University Press. Perkins, D., (1975) A definition of caricature and caricature and recognition, in Studies in the anthropology of visual communication :, -4. & Hagen, M., (1980) Convention, context, and caricature, in The perception of pictures, Volume I: Albertis window, Margaret Hagen, (ed.), 257-286. New York: Academic Press. Piaget, Jean (1972a) Problmes de psychologie gntique. Paris: Denol. (1972b) The concept of structure, in Scientific thought. 35-56 . The Hague & Paris: Mouton/Unesco. Pomerantz, James, & Garner, W. R., (1973) Stimulus configuration in selective attention tasks, in Perception & psychophysics, 4:3, 565-569. , Sager, L., & Stoever, R., (1977) Perception of wholes and of their component parts, in Journal ot experimental psychology: human perception and performance, 3:3, 4-435Porcher, Louis, (1976) Introduction une smiotique des images. Paris: Crdif/Didier. Preziosi, Donald (1983). Advantages and limitations of visual communication. In Visuelle Kommunikation und/oder verbale Kommunikation, Martin Krampen, (ed.), -34. Berlin: Olms Verlag, Hildesheim/ Hochschule der Knste. Prieto, Louis, (1966): Messages et signaux. Paris: PUF. (1975a). Pertinence et pratique. Essai de smiologie. Paris: Minuit. (1975b). Essai de linguistique et smiologie gnrale. Genve: Droz. Rcanati, Franois, (1979) La transparence et lnonciation. Paris: Seuil. Reed, Edward S. (1988). James J. Gibson and the psychology of perception. New Haven


& London, Yale University Press. Rosch, Eleanor (1973) On the internal structure of perceptual and semantic categories in Moore, Thomas E., ed., Cognitive development and the acquisition of language, Academic press, New York & London; pp -44. (1975a) Cognitive representations of semantic categories, in Journal of experimental psychology: general, 04: 3, 975, pp 933. (1975b). Cognitive reference points, in Cognitive psychology, 7,4: 53-547. (1975c) The nature of mental codes for color categories, in Journal of experimental psychology: human perception and performance, :4,; pp 303-3. (1975d) Universals and cultural specifics in human categorization, in Brislin, R. W., Bochner, S., & Lonner, W. J., eds., Crosscultural perspectives on learning. Wiley & Sons, New York; pp 177-206. (1978) Principles of categorization, in Cognition and categorization, Rosch, E., & Lloyd, B., eds., 7-48. Hillsdale: Lawrence Erlbaum Ass. , & Mervis, C. B., (1975). Family Ressemblances: studies in the structure of categories, in Cognitive Psychology 7:4, pp 38439. & Mervis, C. B., Gray, W., Johnson, D., & Boyes-Braem, P., (1976a) Basic objects in natural categories, in Cognitive psychology, 8:3, pp 38-439. &., Simpson, C., & Miller, R. S., (1976b) Structural bases of typicality effects, in Joumal of experimental psychology: human perception and performance, :4; pp 49-50. Saint-Martin, Fernande (1987). Smiologie du langage visuel. Qubec: Presse de lUniversit du Qubec. (1990). La thorie de la Gestalt et lart visual. Qubec: Presse de lUniversit du Qubec. Sander, Friedrich & Volkelt, Hans (1962). Granzheitspsychologie. Mnchen: Verlag C.H. Beck. Saussure, Ferdinand de (1968/1974). Cours de linguistique gnrale I-II. Edition critique par Rudolf Engler. Wiesbaden. Harrossowitz. Savan, David, (1976) An introduction to C.S. Peirces semiotics. Toronto: Toronto Semi-

otic Circle. Schaffer, H. R., (1971) The growth of sociability. Harmondsworth: Penguin. Sebeok, Thomas A., (1976). Contributions to the doctrine of signs. Bloomington & Lisse: Indiana University Press/Peter de Ridder Press. Smith, Barry (1994) Topological foundations of cognitive science, in Topological Foundations of Cognitive Science, Echenbach, C., Habel., C, & Smith, B, (eds), 3-22: Hamburg: Graduiertenkolleg Kognitionswissenschaft. (1995) Formal ontology, common sense, and cognitive science. In International Journal of Human-Computer Studies, 43, 64-667. (1997) On substances, accidents and universals. In Philosophical Papers 6, 057. (1999a) Les objets sociaux, in Philosophiques 6/, 35-347 (1999b) Truth and the visual field, to appear in Naturalizing Phenomenology. Petitito, J, Varela, F.J., Pachoud, B & Roy, J.-M. (eds). Stanford: Stanford University Press. Smith, Barry & Casati, Roberto (1994) Nave Physcis: An essay in ontology, Philosophical Psychology, 7/, 5-44. Smith, Barry, & Varzi, Achille (1999), The niche, in Nos, 33:, 98-. Sonesson, Gran, (1978) Tecken och handling. Doxa: Lund. (1988) Methods and models in pictorial semiotics. Report of the Semiotics, Project, Lund University. (1989a). Pictorial concepts. Inquiries into the semiotic heritage and its relevance for the analysis of the visual world. Lund: Aris/ Lund University Press. (1989b). Semiotics of photography. On tracing the index. Report from the Semiotics project. Lund: University. (1990). Rudimentos de una retrica de la caricatura. In Investigaciones semiticas III. Actas del III simposio internacional de la Asociacin espaola de semitica, Madrid - de diciembre de ; Volumen II, 389-400. Madrid: UNED. (1991). Kroppens byggnad och bruk. Ngra semiotiska modeller. In Att tala utan ord. Mnniskan icke-verbala uttrycksformer, Gran Hermern, (ed.), 83-100. Stockholm: Almqvist & Wiksell International. (1992a) Bildbetydelser. Inledning till bild

semiotiken som vetenskap. Lund: Studentlitteratur. (1992b). The semiotic function and the genesis of pictorial meaning. In Center/Periphery in representations and institutions. Proceedings from the Conference of The International Semiotics Institute, Imatra, Finland, July -, 0., Tarasti, Eero (ed.), 211-156. Imatra: Acta Semiotica Fennica. (1992c). Comment le sens vient aux images. Un autre discours de la mthode. In De lhistoire de lart la smiotique visuelle. Carani, Marie(ed.), 29-84. Qubc: Les ditions du Septentrion/CLAT. (1992d). Le mythe de la triple articulation. Modles linguistiques et perceptifs dans la smiotique des images. In Signs of Humanity/Lhomme et ses signes. Proceedings of the Fourth congress of the International Association for Semiotic Studies, Barcelona/Perpignan, Mars-April , Balat, Michel, Deledalle-Rhodes, Janice, & Deledalle, Grard, (eds.), Volume I, 14956. Berlin: Mouton de Gruyter. (1992e). Bodily semiotics and the extensions of man. In Center/Periphery in representations and institutions. Proceedings from the Conference of The International Semiotics Institute, Imatra, Finland, July -, 0., Tarasti, Eero (ed.), 185-210. Imatra: Acta Semiotica Fennica. (1993a). Pictorial semiotics, Gestalt psychology, and the ecology of perception. Review of Saint-Martin, Fernande, La thorie de la Gestalt et lart visuel. In Semiotica 99: 3/4, 39-399. (1993b) Die Semiotik des Bildes, Zeitschrift fr Semiotik 5: -, 3-64. (1993c) The multiple bodies of man. Project for a semiotics of the body. in Degr, 74, d-d4. (1994a). Prolegomena to the semiotic analysis of prehistoric visual displays. In Semiotica, 00: 3/, 67-33. 47-64. (1994b) Smiotique visuelle et cologie smiotique, in RSSI, 4, -, printemps 994, pp. 3-48. 47-64. (1995a) On pictorality. The impact of the perceptual model in the development of visual semiotics, in The semiotic web /: Advances in visual semiotics, Sebeok, Th., & Umiker-Sebeok, J., eds., 67-08. Mouton de Gruyter, Berlin & New York.

(1995b) Indexicality as perceptual mediation, in Indexicality. Pankow. Christiane (ed.), 7-43. SSKKII Report 9604.Gothenburg: Gothenburg University. (1995c): Livsvrlden mediering. Kommunikation i en kultursemiotisk ram. I Holmberg, Claes-Gran, & Svensson, Jan, utg., Medietexter och mediatolkningar. Nora: Nya Doxa, 33-78. (1996a) An essay concerning images. From rhetoric to semiotics by way of ecological physics. Review of Groupe , Trait du signe visuel. Semiotica, 09-/, 4-40. (1996b) Le silence parlant des images, Prote 4:, 37-46. (1996c). De la retrica de la percepcin a la retrica de la cultura/Frn varseblivningens till kulturens retorik, in Heterognesis 5, -. (1996d) Iconicit de limage Imaginaire de liconicit. De la ressemblance la vraisemblance. Actes du Premier Congrs de lAssociation internationale de smiologie de limage, Blois, novembre de 990. In VISIO, , , pp. 3-34. (1996e) The quadrature of the hermeneutic circle. In LSP and theory of translation. Acts of the XVI Vakki symposion, Text and Image, Vjri, February 0-,  Vaasa 996; 9-33. (1997a) Approaches to the Lifeworld core of visual rhetoric. VISIO , 3, 49-76. (1997b) The ecological foundations of iconicity, in Semiotics Around the World: Synthesis in Diversity. Proceedings of the Fifth International Congress of the IASS, Berkeley, June -, . Rauch, Irmgard, & Carr, Gerald F., eds., 739-74. Berlin & New York: Mouton de Gruyter. (1997c) Mute narratives. New issues in the study of pictorial texts. In Interart Poetics. Acts of the congress Interart Studies: New Perspectives, Lund, May 1995. Lagerroth, Ulla-Britta, Lund, Hans, & Hedling, Erik, (eds.). Rodophi, Amsterdam & Atlanta 997; 43-5. (1998a) /Entries/ in Encyclopaedia of Semiotics. Bouissac, Paul, ed., New York & London: Oxford University Press. (1998b) That there are many kinds of pictorial signs, VISIO, 3, , 33-54 (1999) Post-photography and beyond. From mechanical reproduction to digital production. In VISIO 4, , -36.
0

 (2000a) Iconicity in the ecology of semiosis, In Iconicity - A Fundamental Problem in Semiotics. Troels Deng Johansson, Martin Skov & Berit Brogaard (eds.), 59-80. Aarhus: NSU Press. (2000b) Ego meets Alter: The meaning of otherness in cultural semiotics, in Semiotica, 8-3/4, 537-559. (2001a) La retorica del mondo della vita. In Modo dellimmagine. Bologna: Esculapio, Pierluigi Basso (ed.), 93-112. Actes of the 5th Congress of the International Association for Visual Semiotics, Siena, Italy, 1998). (2001b) De liconicit de limage liconicit des gestes. In Oralit et gestualit. Actes du colloque ORAGE 2001, Cav, Christian, Guatella, Isabelle, Santi, Serge (eds), 47-58. Paris. LHarmattan. (2001c) From semiosis to ecology. On the theory of iconicity and its consequences for an ontology of the lifeworld. In VISIO, , -3, 85-0. (2001d) The pencils of nature and culture: New light on and in the Lifeworld. In Semiotica 36-/4, 7-53. (2003a) The Symbolic Species revisited. Considerations on the semiotic turn in biology and cognitive science. Workings Papers of the SGB poject. In http://www.arthist.lu.se/ kultsem/pro/SGBWP3.pdf (2003b) Why the mirror is a sign and why the television picture is no miror. Two episodes in the critique of the iconicity critique. In Special issue. Iconicity. European journal for semiotic studies. 5-/4, 003, 7-3. (2003c) Uber die Mglichkeit von bildhaften Metaphern. In Zeitschrift fr Semiotik, 5: -, 003: Metaphern in Bild und Film, Gestik, Theater und Musik, Johansen, Jrgen Dines, & Posner, Roland, (Hrsg.), 2538 (2003d) Resurserna i bildens och sprkets semiotik. Frn Lessing to Kress & van Leeuwen. In Frn Nrpesdialekt till EUsvenska. Festskrift till Kristina Nikula, Lnnroth, Harry (ed.), 225-258. Tampere: Tampere Univesity Press. (2004a) Rhtorique du monde de la vie. In Ateliers de smiotique visuelle, Henault, Anne, & Beyaert, Anne (eds), 83-100. Paris: PUF. (2004b) Perspective from a semiotical perspective. In Essays on fiction and perspec-

tive. Rossholm, Gran, (ed), 255-292. Bern: Peter Lang. Papers from the symposion Fiction and perspective, organized by Stockholm University and Kungliga Vitterhets-, Historie- och Antikvitetsakademien, October 3 to 7, 00 (2005a) De la mtaphore lallgorie dans la smiotique cologique. In Prote, 33:, printemps 005, 77-9. (2005b) La semitica de imgenes de la reproduccin mecnica a la produccin digital. In: Semitica de los Mass-media: Discurso de la comunicacin visual. Espinosa Vera, Pablo, (comp.), 209-236. Monterrey, Mxico: Universidad Autnoma de Nuevo Len. (in press a) From Iconicity to Pictorality. A view from ecological semiotics. To appear in VISIO: thematic issue: Iconicity revisited/Iconicit revisit, Sonesson, Gran (ed), Spring 005 (in press b) Iconicity strikes back: the third generation or why Eco still is wrong. To be published in Actes of the th congress of the International Association for visual semiotics, Quebec City, 00 (in press c) La rhtorique de la perception. Recherche de mthode. To be published in the Acts of the Urbino symposion on rhetorics, Urbino 00, Badir, Smir, / Klinkenberg, Jean-Marie (eds.) (in press d) The extensions of man revisited. From Primary to Tertiary Embodiment, in Krois, John, ed., The body and embodiment. London & New York: Rouledge. (in press e) From the meaning of embodiment to the embodiment of meaning. In: Zimke, T., Zlatev, J. and R. Frank (eds.) Body, Language and Mind. Vol . Embodiment. Berlin: Mouton. Stjernfelt, Fredrik (2000), Mereology and Semiotics, in Sign Systems Studies, vol. 8, Tartu, 73-98 Strawson, P. F., (1957) Individuals. London: Meuthen. Tesnire, Lucien, (1959) lments de syntaxe structurale. Paris: Klincksieck, 959, 969. Thayer, S. & Schiff, W., (1969) Stimulus factors in observer judgment of social interaction: facial expression and motion pattern, in American Journal of psychology, 8:, 73-85. Thrlemann, Flix, (1982) Paul Klee. Analyse


smiotique de trois peintures. Lausanne: LAge dHomme. Trubetzkoy, N.S. (1939) Grundzge der Phonologie. Prague. 4th edition: Gttingen: Vanderheock & Ruprecht 958. Tversky, Amos (1977). Features of similarity, Psychological Review, 84,4: 37-35. Todorov, Tzvetan (1977) Thories du symbole. Paris: Seuil. (1978) Les genres du discours. Paris: Seuil. van Kesteren, Aloysius, (1984) Theatre and drama research: an analytical proposition, in Semiotics of drama and theatre. Schmidt, Herta, & van Kesteren, Aloysius, eds., 9-66. Amsterdam & Philadelphia: Benjamins. Vodika, Felix, (1976) Die Struktur der literarischen Entwicklung. Mnchen: Fink. Von Schelting, Alexander, (1922) Die logische Theorie der historischen Kulturwissenschaften von Max Weber und im besonderen sein Begriff des Idealtypus in Archiv fr Sozialwissenschaft und Sozialpolitik, 49, 63 75. (1934) Max Webers Wissenschaftslehre. Tbingen: Mohr. Wallis, Mieczyslaw, (1975) Arts and signs. Lisse & Bloomington: Peter de Ridder Press/lndiana university press, 975. Weber, Max, (1913) Uber einige Kategorien der verstehenden Soziologie, in Logos IV, 93, pp 53-94 (page references to Weber, M., Soziologie. 4th edition, Krner, Stttgart 1968; pp 97-150). (1964) Wirtschaft und Gesellschaft. Kln & Berlin: Kiepenhauer & Witsch. Weinhandl, Ferdinand (ed.) (1960). Gestalhaftes Sehen. Zum hundertjhrigen Geburtstag von Christian von Ehrenfels. Darmstadt: Wissenschaftliches Buchgesellschaft. Wellbery, David E. (1984) Lessings Laocoon. Semiotics and aesthetics in the Age of Reason. Cambridge: Cambridge University Press. Winner, Ellen (1982). Invented worlds. The psychlogy of art. Cambridge, Mass., Harvard University Press. Wollheim, Richard (1980) Art and its objects. Cambridge: Cambridge University Press. Worth, Sol, (1981) Studying visual communication. Philadelphia: University of Pennsylvania press. Zlatev, Jordan, (in press) Bodily mimesis and the grounding of language. In: Hampe,

B. (ed.), From Perception to Meaning: Image Schemas in Cognitive Linguistics. Berlin: Mouton. Zurlo, Georg, (1976) Strukturelle Untersuchungen zu Western Texten (Film und Literatur), in Textsemiotik und strukturelle Rezeptionstheorie, Koch, Walter, ed., 407567. Hildesheim & New York Olms Verlag.