Semiotics and Intertextuality of Videogames

CONTENTS INTRODUCTION .................................................................3

A SHORT PRIMER .................................................................6

PLAYING PL AYING WITH SEMIOTICS

................................................................11

CODES CO DES IN VIDEOGAMES ................................................................19

INTERTEXTUALITY ................................................................23 CONTINUE /CONCLUSION

BIBLIOGRAPHY

................................................................27 ................................................................29

2

Semiotics and Intertextuality of Videogames

3

There is terrible irony present in the scholarly study of videogames: though the medium is an effective communicator, communicator, it requires a tremendous amount of communication to convince those that hold it in derision, and think of it only as a child’s play-thing. The videogame is a medium that speaks to its generation, its culture. It possesses a language all its own, and yet due to its progenitors, it can speak to the same discourses of traditional media. Videogames are a model of expression expression that utilizes several several modes of communication within itself. The visual: text, words and narratives borrowed and derived from a great wealth of literature and invention; computer-generated imagery, evocative of animation and lm techniques and styles. The aural: sounds, both in the realm of diegesis and beyond, used to manipulate ambience, depth, and function; more recently, human voice acting, further blurring the line between the videogame medium and one of its antecessors, the lm. The haptic: the tactile sensations between a player — the medium’s actor — the physical topology of the controller, controller, and the medium itself. These modes coalesce to provide the user feedback at prescribed moments when meaning can be inferred. The videogame is a profoundly sophisticated means of communication, communication, even more so than aforementioned. “Games are models of experiences rather than textual descriptions or visual depictions of them. When we play games, we operate those models, our actions constrained by their rules...Videogames are a medium that lets us play a role within the constraints constraints of a model model world,” (Bogost, (Bogost, How to do things 4). While I would be cautious to downplay the signicance of the textual and visual elements of the videogvideogame, it is impossible to ignore the intellectual signicance of the medium’s gestalt, the encapsulation encapsulation of distilled experience. And yet we do. Or at least, we have. The academic inquiry of new media, and consequently, videogames, is dwarfed by the stalwart canons of liberal studies. Due in part to its relative adolescence, with the earliest non-commernon-commercial games dating back to the 1960’s1, and perhaps as well, its general perception among the masses. “(Videogames) are a part of the ‘entertainment software’ software’ industry, and 1

There are examples of computer games (we cannot really call them digital because of the analog computers used) being created from the early 1950’s, like the tennis simulator, Tennis enni s for Two. Two. This inspection of the medium examines more commercialized titles, beginning with Pong, released for the Magnavox Odyssey in 1972.

4


they are generally considered a leisure practice by players and the general public alike. Videogame play is considered considered an unproductive unproductive expenditure of time, time, time that lls the the breaks between works,” (Bogost, (Bogost, “Rhetoric of Videogames” Videogames” 120). Regardless Regardless of the validity of the medium, medium, videogames are almost almost exclusively exclusively seen as a form of of entertainment, predominantly skewed towards children children — not something to be studied by academics. However, I, along with others, do not hold that to be true. If McLuhan was correct in his magnanimous afrmation that the “medium is the message” (McLuhan 7), and that the delicious, meaty morsels of content are communicatively superuous, then it does not matter that the medium is routinely used to slay dragons and play war; if anything, the distractive tendencies of its content plays perfectly with McLuhan’s model, as really any “gamer” would tell you, it is easy to become lost in those “model worlds.” As I have expressed, I feel the videogame’s penultimate function is as a mode of expression, of communication. But what is it to communicate with one another? How are ideas formed, framed, and rendered from one mind to another? Because of the videogame’s close close relationship to the the differing media and and modes that compose compose it, I propose that we can often use the same points of analysis to better understand its parts. Umberto Eco wrote that “every act of communication to or between human beings — or any other intelligent biological or mechanical apparatus — presupposes presupposes a signication system as its necessary condition,” (Eco 9). This, I believe, is the essence of communication, a system of signications, signications, called semiotics, semiotics, which decode abstractions that we have previously given meaning to, and encode further interpretations. “At the heart of semiotics is the realization that the whole of human experience, without exception, is an interpretive structure structure mediated and sustained by signs,” (Deely 6). The study of semiotics, a contentious eld of interdisciplinary interdisciplinary research that spans across the social sciences — moreover linguistics, psychology, and philosophy — attempts to explain the “action of signs” (Deely 22). Early 20th century linguist, Ferdinand de Saussure wrote of and


5

created the term, semiology2, a synonym for the eld to describe what he foresaw as a eld of academic interest. “A science that studies the life of signs within society is conceivable; it would be a part of social psychology and consequently of general psychology; I shall call it ‘semiology.’ Semiology would show what constitutes signs, what laws govern them” them” (Leeds-Hurwitz (Leeds-Hurwitz 4). Saussure went went on to write of critical critical xtures of semiotic studies that will be later described within a videogame context. Additionally, Saussure’s contemporaries contemporaries were also breaching the eld. Charles Sanders Peirce, an American philosopher, philosopher, wrote wrote of nearly identical identical relationships relationships3, albeit more explicitly, and declared his study “semiotic.” From these two scholars alone comes a tree of academics interpretively dening communication. communication. There are several different families of semiotic research. research. The one investigated in this paper is anthropsemiosis, anthropsemiosis, a system used to describe “all of the sign processes that human beings are directly involved in, and, looked at another way, names those sign processes which are species, specically human,” and as Deely puts it, “the one closest to us,” (28). Can a relatively young academic eld be used to explain an even younger medium like like the videogame? Absolutely, Absolutely, almost almost shockingly shockingly so. The videogame, as I said, is a valuable method of communication, communication, and as such, is a veritable trove of signs — some obvious and familiar, familiar, and others foreign, but all work together together through their different modes to signify meaning. Much like the gestalic models that Bogost earlier mentioned, “meanings in multimedia are not xed and additive...but multiplicative...making multiplicative...making a whole far greater than the simple sum of its parts,” (Lemke 72). And the agency of communication in videogames is not derived from a disorderly mess of signs, but rather a hierarchy. When Michel Foucault spoke the relationship of the verbal (referring to text he observed) and the visual, he said, “what is essential is that verbal signs and and visual representation representation are never given at once. An order always

2 3

From the Greek semeion:, a sign, mark. Without knowledge or collaboration with Saussure’s Saussure’s work.

6


hierarchizes them, running from the gure to discourse or from discourse to the gure,” (33). As I will describe, signs work in the videogame in much the same way Foucault described paintings: in order to establish discursive, rhetorical structures, meaningful signs must combine to form systems within videogames — these systems combining to make larger systems, which in turn beget even larger systems. I will demonstrate, starting with the medium’s basic semiotics units, that this organization creates a grid of intertextuality, encompassing the videogame medium, the sub-culture surrounding the medium, and the culture enclosing that — all of which is a part of the paramount semiotic structure, human experience.

A Short Primer In order to analyze the role of semiotics in videogames, we rst must have a solid understanding understanding of the subject’s fundamentals. As mentioned earlier the most succinct denition of semiotics is a eld in which examines the actions of signs. Interesting, you may say, but what are signs? What sort of actions can they exhibit, and why should we care? The two aforesaid scholars, Saussure and Peirce, developed two distinct but similar methods for analyzing semiotic structures. A sign is something that represents something else. I say “something” not out of a lack of words or an inability to nd an appropriate one, but rather because what I am presenting is more an equation; a proposition that states that any variable material or immaterial concept or object is representative, signicant, of another. Saussure in his book Course book Course in General Linguistics offered not only what a sign could be, but more specically what constitutes a sign. Saussure interpreted signs as a dichotomy, comprised comprised of two parts, the signied and the signier. The signied is the “visible part”, or rather, the most obvious part, as it is the “the explicit aspect of a sign, present during the interaction, a material presence


7

of some sort,” (Leeds-Hurwitz 9). The signied is what you literally see when you gaze upon the sign; literally, meaning not at all guratively, as it is the gurative which is the signied (even if there is a great similarity between the two at the time). The signied is the “invisible part”, or the “tacit element of a sign, what might be termed an ‘immaterial’ ‘immaterial’ presence, something literally absent yet functionally present because it has been invoked,” (Leeds-Hurwitz 9). I feel that it is difcult to understand the concept without the use of a visual aid, so let us briey consult a sign that we are all familiar with. The object in Figure 1A is a cartoonish depiction depiction of a tree, made with simple lines, curves, and angles, but when it is looked upon and registered, the individual components that comprise comprise it go unnoticed, and instead you think, “tree” — oak, evergreen, sumac leaves green with rough, driedout bark, earthy smells. As the viewer of this sign, you play the role of the “social actor”, one whom uses their knowledge of contextual evidence to extrapolate the action of the sign. When you see the tree sign in Figure 1A, what you are viewing is the active signier: representing representing what you can gleam out of your cultural knowledge of what the object should mean, the signied. Leeds-Hurwitz suggests perhaps a better example example in her description description of Saussure’s relationship: relationship: instead of a tree, let us imagine a white wedding dress dress — an actual dress, dress, and not just a cartoon depiction depiction as I presented presented before, because a sign is a function, function, and not married married to the mode of the image. The wedding dress signies a wedding (that is, the dress meaning the signier, while the concept and practice of the wedding is the signied). But there’s more to

Figure 1A

8


this particular sign that could be said. Depending on the cultural situation of the social actor that is interpreting the sign, a wedding dress could be representative representative of many things — in some cultures the white color of the dress is indicative of purity which extends into an idea of a virgin bride, while in other circumstances the wedding dress is symbolic of the marriage itself and not just the wedding ceremony, and furthermore by other’s interpretations, interpretations, the dress could be representative of an industry geared to take monetary advantage of lovelorn brides who want a perfect wedding (22-27). There are many semiotic principles at work within this simple, conjured image. First, the sign is dependent on context, whether it is contextually contextually from the location of its introduction, or the awareness of the interpreter. “The sign rst of all depends on something other than itself. It is representative but only in a derivative way, in a subordinate capacity. The moment a sign slips out from under this subordination, as frequently happens, at just that moment does it cease for a while to be a sign... Thus on its own, it is a mere object or thing become object, waiting to become a sign,” (Deely 35). Deely continues to say that without this content, or in my own preferred terms, the context, “the sign ceases to be a sign.” He explains that it is still something, often an “object”, as it cannot become inexistent (or if it can, that is a stasis more suited towards philosophers like Deely to contend with), but instead it must wait to be re-contextualized, re-contextualized, to be given further further meaning. What is important to realize realize here is that signs cannot develop meaning on their own, “that objects, images and patterns of behavior can signify, and do so on a large scale, but never autonomously.” (Barthes, Elements of Semiology Semiology 10). Signs require human intervention and invention in order to function and to be understood. Second, it is necessary to note how many different interpretations of the wedding dress that I alone was able to produce. As I have shown, signs are not created from a void. They are reliant reliant upon the context context of their environments environments and intentions, intentions, and allow


9

for further denition by semioticians. Because the wedding dress serves as a signier to several different signieds4, we can declare the wedding dress a polysemic sign. Alternatively, if a signied signied is tied to multiple signiers, signiers, it is polysemic polysemic as well, and signs can often be both. There are other terms that dictate how a sign is interpreted. Referring to Figure 1B, notice how the same sign trying to evoke the tree is now

Figure 1B

displayed in color, with a green top and a brown trunk. “Motivation...refers “Motivation...refers to the degree in which the signied determines the signier,” (Lee-Hurwitz 26). Because the sign more closely illustrates what an actual tree looks like, the signier is more closely constrained to the signied, and is described as “being highly

motivated or highly constrained.” We can continue to use the tree example to describe a sign’s attribute of “convention.” Convention is used to refer to a sign’s sense of tradition, or how it is often used in a particular system. If one views Figure 1, they are likely to see “tree” because that is what they are conditioned to do. Lee-Hurwitz explains that highly conventionalized conventionalized signs are often ignored due to their ubiquity, and include other popular signs such as the male and female symbols. Also, when analyzing signs, it is imperative to be able to distinguish distinguish “detonation” from “connotation”. “connotation”. “Denotation “Denotation refers to the the explicit, obvious, straightforward, straightforward, rst meaning of a sign; the related term connotation refers to the implicit, conventional, second meaning of a sign, imposed by a specic culture,” (LeeHurwitz 27). If we reuse the example of the wedding dress from earlier, it is simple to see that the dress denotes clothing, but it connotes wedding. Or at least, it connotes wedding for the culture of the intended audience of this paper. “Denotation often crosses cultural boundaries; connotation connotation almost almost never does,” (Lee-Hurwitz (Lee-Hurwitz 27). 4 Which will differ depending on the type of reading occurring: occurr ing: dominant, sub-ordinate, sub-ordinate, oppositional, etc.

10


Of these semiotic principles that have been laid out, the most pertinent one to grasp and isolate is the sign’s dependence upon human direction — an interpreter must be present in order for a sign to be interpreted, but even more so is the sign’s dependence upon other signs (which were earlier signied by people). Umberto Eco describes this relationship as unlimited semiosis. semiosis. “In order to establish what the interpretant of a sign is, it is necessary to name it by means of another sign which in turn has another interpretant to be named by another sign and and so on. At this point point there begins as process process of unlimited unlimited semiosis, which, paradoxical as it may be, is the only guarantee for the foundation of a semiotic system capable of checking itself entirely by its own means...the very denition of ‘sign’ implies a process of unlimited semiosis...semiosis explains itself by itself,” (Eco 69-71). In other words, every sign relies upon another in order to be interpreted by the social actors within its realm. Eco, you may have noticed, uses unfamiliar terminology within his description of the semiotic process, which he actually borrows from the aforementioned forerunner forerunner of semiotics, Peirce. I chose to introduce Saussure rst because understanding understanding his methodology allows us in turn understand Peirce’s viewpoints5. Peirce believed the sign to instead be a trichotomy: trichotomy: “the sign or the representatum, representatum, the object, and the interpretant,” (Leeds-Hurwitz 23). The representatum representatum corresponds directly to Saussure’s signier and acts in the same way. However, instead of presenting the signied as a complete unit of expression, Peirce splits it into two — the object, the idea or gure that the representatum reects, and the interpretant, “over and above the unique essential structure that makes signication possible in the rst place,” (Deely 25), the invisible component that represents represents the meaning generated by the representatum representatum in reference to the object. The process that a social actor uses to 5

A phenomenon not unlike that which was discussed with the interconnectivity of signs.


11

interpret, to go from the representatum representatum to gather the interpretant interpretant about the object, Peirce calls semiosis — a term now commonly accepted as a synonym for signication. As Leeds-Hurwitz Leeds-Hurwitz explains, Saussure Saussure and Peirce’s differing differing ideas are not grounds grounds for contradiction, contradiction, “Peirce was simply slightly more explicit than Saussure,” (23). It is because of the validity validity of both of these methods methods — or maybe maybe as Deely would argue, argue, points of view — that I choose to examine the semiotic structure structure of videogames in accordance with Saussure’s more direct system, but with the accoutrements of Peirce’s further inspection of sign variations.

Playing with Semiotics With this primer of semiotic theory under our belts, we nally have a working knowledge of fundamentals that can be used to analyze the semiology in videogames. But again we should address: can we, and if so, should we? I feel that the disconnect between new and and old media is not a drastic drastic one, and that all media (media in the the McLuhan sense, that is, almost everything) have a common lineage. “Rhetorics inevitably vary by their substance (here articulated sound, there image, gesture, or whatever) but not necessarily necessarily by by their form; it is even probable that there there exists a single rhetorical form, common for instance to dream, literature, and image,” (Barthes, “Rhetoric of the Image” 161-162). That rhetorical singularity singularity Barthes references is what I am referring to myself (because what is a rhetoric if not a chain of signiers of an “ideology”, as he would put it). These commonalities commonalities allow human communication communication to be analyzed with with generally the same same methods — just just as we analyzed analyzed a tree or a wedding dress, we could analyze the latest Call of Duty. Duty. So we are able to apply elder methods to new analyzes, but is there a point to it? Do we stand to learn anything from close readings that we could not have gathered from extraneous study of the medium? I believe there is still much to learn from individual elds like semiotics through the inspection of unfamiliar environments. environments.

12


As Bogost would retort, retort, “The content content and context of of a media artifact is not not as inessential inessential as McLuhan would have it. The medium is the message, but the message is the message, too,” (How to do things 5). For the sake of simplicity along with establishing a temporal continuity of sign play in videogames, it is interesting to begin discussing semiotics with a game from early in the creation of the videogame industry. Older videogames and those created by noncommercial commercial developers have a long history of using representational visuals to exhibit meaning. Adventure meaning. Adventure (1979), a game created for the Atari 2600, allows the player — a term specic to videogame culture and conveniently coincides with our semiotic idea of the social actor — to control a small, colored square through a digital environment. environment. This square whom you control starts out in front of a similarly colored structure structure featuring a forward-facing black grid and it all is surrounded by a semi-permeable semi-permeable perimeter of rectangles (Figure 2). That’s how the game begins. There is no foreword in-game text that provides context to your objectives, surroundings, surroundings, or even who you are, and yet without direct effort effort you are able to collect collect this context context through semiosis. semiosis. As you direct direct your avatar throughout throughout this game environment, environment, you notice notice things; you realize realize that when you go through openings openings between the perimeter’s perimeter’s boundaries boundaries the environment environment changes its appearance either through its layout or the color (Figure 3). This signies that your

Figure 2: Starting point o Adven o Adventure. ture.

Figure 3: Advent 3: Adventure ure - Environmental diferences


13

character, this square, square, is able to traverse this area and you are able to differentiate between various locations. locations. But then then upon your wandering wandering you discover objects objects familiar to you within the negative space that you walk in: door keys, arrows (resembling (resembling directional arrows, like on a keyboard) and goblets. In this realm of abstraction where all you can infer are shapes, you nally come upon something of substance, a key. As a social actor from a system that understands what that shape shape confers, you know know what a key is and what what it can do. Adding Adding to the signicance of this object is a relationship unique to the videogame, the ability to interact, manipulate the object. Your character picks up the key and carries it along with them (Figure 4) while while exploring the the environment, but but you decide instead to equip equip yourself with the the arrow to further further investigate its uses, uses, wherein you note note that it points towards you when you hold it. Along the way you encounter encounter an object resembling resembling a serpentine dragon, moving aggressively unlike anything that you have seen before. Just as you knew what the key was, you are additionally able to decipher this sign to be a dragon dragon — a beast not known for

Figure 4: Advent 4: Adventure ure - Interacting with ound artiacts.

its benevolency — and this suspicion is conrmed when it opens its jaws to attack, followed with a harsh, deep rumble. You accidentally accidentally run forward directly into the path of the dragon forcing it to produce a defeated noise signifying its demise. It takes a new position symbolizing its condition (Figure 5).

ure - Found objects interacting with others Figure 5: Advent 5: Adventure through semiosis.

14


Using your innate semiotic skills you are able to gather many things about this videogame from the the encounter: rst, rst, you are in a world where where dragons exist and and the tool you possess seemed to enable you to slay them. This jumpstarts a series of signications much like the unlimited semiosis effect Eco described earlier. Using your knowledge as the intended social actor for this game, you realize the arrow was a sword the whole time, because what other than a sword to slay a dragon in popular culture? This secondly, in turn, compels you to examine the sword more closely. The arrow that we earlier saw is the the signier to the sword’s sword’s signied — the the reason we did not realize realize it sooner was because the motivation of the sign was not closely constrained. Regardless, due to the convention of the sword being a symbol of dragon slaying and knighthood, we are able to identify identify the sign accurately accurately within the game game without being given further subtext. Lastly, by knowing this we are better able to signify the rest of the signs that we have come into contact contact with thus thus far, we allow chains chains of semiosis semiosis to take effect. The The structure we started out in front of in the game’s beginning now more obviously is a signier for a castle — thereby connotating a medieval aesthetic — the keys we discover open the corresponding color’s castle gates, and the character in which you play — the square-shaped congregation of yellow pixels — now resembles a knight. All of this can be inferred within within a minuscule period period of time due to semiotics semiotics at play. I mentioned earlier that there was no “in-game text” provided for context for Adventure, Adventure, but what I did not mention was that was that there was accompanying accompanying literature with the game that did so. Often instruction manuals and booklets would provide a deeper backstory and explain game mechanics in games that did not have the time or graphical (or even narrative) resources to do so — for instance, there was little indication that Super that Super Mario Brothers Brothers 2’s 2’s (1988) narrative was set within a dream, without having read read the manual where where it was much more more obvious. But that system, system, of reading supplementary literature, does not break the semiotic structures that we


15

have supposed in Adventure in Adventure,, nor any other game that we could analyze, but rather it makes for a more descriptive experience — one would understand signs more readily having have read all the available context, just as a social actor would better understand particular signs having understood the cultural system that the signs came from. Luckily Adventure’s Adventure’s designers understood the mechanics of semiotics enough to imbibe the game with enough meaning to allow players over 30 years in the future to comprehend its gameplay. Modern games appear to have moved away from the model where an instruction manual is necessary, instead relying on other forms of media to provide backstory, like books, websites, websites, and animations animations that may or may may not be canonical, canonical, and rely furthermore furthermore on semiosis within the game itself. In contemporary videogames these moments can be juxtaposed between between playable moments moments like in loading loading screens (Figure (Figure 6), or outside of those times (Figure 7). Notice in Figure 7 that I am intentionally describing describing the words under the images as signs. More specically, words are a part of a semiotic and linguistic system in which language is composed of groups of signs we call words, words, and words are composed of units of signs called letters. letters. Semiosis still occurs when meaning is derived — the idea of a sign simply being a “picture” is not accurate as we can see with this display as well as with the sound of the dragon’s roar from before. The representational

Figure 6: Braid - Instructional loading screens.

Figure 7: Braid - Diegetic, active in-game instructions.

16


style shown in Figure 7 is especially interesting because the player is able to actively test out what the signier, the instruction suggesting to jump with the spacebar, is supposed to signify — even without knowledge of the higher systems at work here, a player could interpret the sign’s meaning along with consecutive tutorial signs. When we discuss the role of semiotics in more current games like Braid like Braid in in Figure 6 and 7, we are invoking more complicated relationships relationships than what was observed in Adventure in Adventure.. Videogames, just like signs, grow exponentially in complexity and sophistication, sophistication, and necessitate the use of more specic types of signs. In his scholarly research of semiotics, Peirce “identied 66 potential varieties of signs, 3 of which have gained wide acceptance: the concepts of icon, index, and symbol,” (Lee-Hurwitz 23). Because we are able to treat the videogame as a serious medium worthy of semiotic study, it is possible to locate these types of signs within games, and often due to the graphical delity, it’s easier to do so in slightly more modern games than Adventure than Adventure where we rst identied identied Saussure’s Saussure’s principles. To further further prove the validity of of the assertion that classical semiotic thought is active in videogames, as well as these specic varieties identied identied by Peirce, let us look briey briey at a slightly older title, The Legend of Zelda for the Nintendo Entertainment System (1986) — a spiritual successor to Adventure. Adventure. The Legend of Zelda plays out much like how Adventure Adventure started: you begin the game with little narrative context of what you are supposed supposed to do or who you are, and you are tasked with completing your objectives by manipulating objects objects that interact with one another. But when compared to Adventure this game is relatively more realistic (gure 8).

Figure 8: Te Legend o Zelda


17

The Nintendo Entertainment System was capable of producing more robust 8-bit graphics as opposed to the Atari 2600 because of expanded memory, which allowed it to produce detailed and vibrant virtual environments. This also meant that signs intended for use on games on this console could also be more detailed. Peirce’s icon “has a relationship of similarity or resemblance... resemblance... (and displays) a similarity between the present and the absent components (of the sign),” (Lee-Hurwitz 23). An icon is a sign that closely resembles what it is intended to represent; it is a highly-motivated sign. Refer Figure 9: Te Legend o Zelda - Comparison o ingame representation v. extratextual representation.

to Figure 9 for example: the green and brown sprite that you control serves as an icon to what to what you are supposed to see as according to the game’s literature (situated to the right), because of the close proximity of its representation. With improvements of graphical integrity it has become increasingly easy to denote icons in videogames, and extragame content often has a 1:1 signication with in-game content (Figure 10: Assassin’s Creed 2 comparison). That is not to say that games of

Figure 10: Assassin 10: Assassin’’s Creed 2 - Comparison o in-game representation v. extratextual representation (o a modern game).

higher graphical quality are intrinsically intrinsically better at at communicating communicating through visual semiosis, however. Describing cartooning, Scott McCloud sees simplication as more a way of amplication. “When we abstract an

18


image through cartooning, we’re not so much eliminating in details as we are focusing on specic details, (McCloud 201). Additionally, realism, through its emphasis on denotation as opposed to connotation, makes it more difcult for “the artist to make his point symbolically,” symbolically,” (Arnheim 141). Iconography viewed through the lens of semiotics can make these distinctions. distinctions. Indices use a part of something to stand for the whole (Lee-Hurwitz (Lee-Hurwitz 23). A common trope in The Legend of Zelda series is the character’s representation representation of vitality. As depicted in Figure 11, the character’s health is represented by an orderly group of hearts. Upon injury, the player is alerted by a disastrous noise and the

Figure 11: Te Legend o Zelda

visual of the heart being being halved or quartered, quartered, signifying that they have taken damage. damage. The index present here is subtle and more complex than anything we would have seen in Adventure in Adventure;; the heart works as an index by representing a vital part of the body — not the whole body, as it is an index, but enough to make it obvious to what it represents — the body, as an idea, then acts as signier to the signifed of the concept of health, a small intricate subset of semiosis. Perhaps the most important sign that Peirce identied was the symbol, a structure so prevalent it has nearly become synonymous with the entirety of semiotics, and is often the one of most interest in the communication eld. “A symbol has the relationship of arbitrariness...any arbitrariness...any sign using an arbitrary connection between the present and absent components is a symbol... (and can often include) objects, behaviors, texts, ideas, and people,” (Lee-Hurwitz 23 and 30). It is this arbitrary aspect, despite its pervasiveness, pervasiveness, that makes the symbol difcult to isolate and signify. The arbitrariness of a symbol speaks to its situation in the culture in which it was created, or was intended to


19

be used in — without being privy to that system system it is initially difcult difcult to interpret these these particular signs. I feel the best example of symbols in the videogame is the “health bar” and its multitudinous variations. The health bar, as I will touch on when speaking of medium intertextuality, intertextuality, is present in several games of varying scopes and time periods. The health bar is in a way represented by the heart containers of The of The Legend of Zelda series earlier discussed, but it appears in more generic terms such as in game series like Mega Man (1987), Mortal (1987), Mortal Kombat (1992), Kombat (1992), and the even the rst Halo rst Halo (2001) game6. There is no health bar for real life, nor in any other variety of media — only videogames. “Symbols “Symbols are a form of of shorthand; encapsulating encapsulating cultural cultural knowledge knowledge in particular ways, they serve a valuable role in deliberate passing on of traditions... People use symbols as a way of conveying considerable considerable amounts of information in a small space or short time,” (Lee-Hurwitz 31-34). The health bar is shorthand unique to the videogame, and can only be “picked up through osmosis” as Lee-Hurwitz would say, through contact in that medium. I will soon suggest, however, that because of the transcendal properties of both semiotics and videogames, other social actors could become of aware aware of videogame symbols symbols and other signs signs without ever playing playing them.

Codes in Videogames The word “code” in videogames connotes several different signieds: primarily cheat codes, like the strings of characters and commands used to in order to unlock special privileges allowed by the game’s designer, and the videogame’s coding, the machine language that controls and shapes the medium’s visuals, sounds, and mechanics. But in semiotics, codes are something else altogether. Like with the language analogy presented earlier, signs congregate into increasingly larger systems. A group or system of signs is called a code code by semioticians. semioticians. “Placement of signs signs into appropriate grouping stresses that meaning arises not solely, not even primarily, from 6

Subsequent titles in that series did employ a similar mechanic, but the first most closely identifies with the aesthetic I am speaking towards).

20


the relationship of signier to signied but from relations between signs,” (Lee-Hurwitz 51). Semiotic codes work the same way in videogames just as they would in other forms of media. While we could easily reanalyze the videogames examined before, it would be more prudent to introduce an even more recent game to illustrate coding systems. Mass systems. Mass Effect 3 (2012) is a third-person action/role-playing action/role-playing game developed to function in a much more fast-paced manner than its predecessors, and substantially retooled the series’ game interfaces to reect this change. These alterations required that the player be able to receive a wealth wealth of information information in a short period period of time — just what what signs were made to do. To meet meet these ends, groups groups of signs are often often used in conjunction conjunction with one another in what is called a “heads-up “heads-up display”. Not unlike unlike the dashboard of your car, a heads-up display provides the user a quick synopsis of necessary necessary information to them; in Mass in Mass Effect 3, 3, this often occurs during combat (gure 12).

Eect 3 - Gameplay Heads-up Display Figure 12: Mass Eect


21

The image in Figure 12 reects a singular moment7 of this code system, where the focal point in the middle of the image features a segmented circle with several different different signs on it. Some of these signs can innately go through semiosis; they are icons and obvious, some color coded to signify futility or blank to reect unavailability, but still others are symbols and require an intimate knowledge of this particular videogame to understand. This circle within Mass within Mass Effect 3’s 3’s mechanics is called the Power the Power Wheel , that is, a code developed by the game’s designers for users to be able to quickly identify what actions they can take. This type of constructed system is called a digital code because the signs that make the code up are clearly identiable and distinguished by the social actor, the player. In the lower right hand corner, however, we see a compass. This, we may not notice, is a code as well. The individual signs consist of red dots to signify hostile nonplayable characters, characters, blue ones representing friendly ones (including the central most one that represents your own avatar), and the arrow pointing due to the next objective and its distance in meters consolidating and signifying that this is indeed a radar within the realm of this videogame. But the the average player, or even the above average player, would not see these parts — they would only see the radar. Analogic radar. Analogic codes, codes, of which this is, contain contain “signs that that run together, being being separated only only by the analyst (or (or determined semiotician) for the purpose of interpretation,” (Lee-Hurwitz 52-53). While the radar is an analogic code, it’s not the only apt determiner we can assign to it. It is also a paradigm. A paradigm, paradigm, as dened by Saussure, is a code in which social actors only choose one central sign to display, or focus upon — Lee-Hurwitz compares it to us wearing different pieces of clothing, but calling the ensemble one outt. The power wheel is a paradigm, paradigm, the health and shield shield bars below it is a paradigm, as well well as the enemy health bar above it8.

7

This type of analysis is called synchronic analysis; if we observed the code’s evolution evolution through time it would be called a diachronic analysis (Lee-Hurwitz 64).

8

As I elaborated before, the health bar motif is as prevalent as the use of arbitrary, numerical scoring in this medium.

22


There are more codes here, though, and larger ones at that. The largest one shown in Figure 12 is the aforementioned heads-up display, another common element in games. We saw the beginnings of this code in games like The Legend of Zelda, Zelda, with all the pertinent signs relaying information information at the top of the screen, and this is the natural evolution of that idea; signs, and codes as a result, are always growing. The HUD in Mass Effect 3 is a special type of code, also identied by Saussure, called a syntagm. syntagm. “A syntagm is that new set relating from the combination of elements drawn from different paradigms...meaning paradigms...meaning is primarily located at the level of the syntagm,” (Lee-Hurwitz (Lee-Hurwitz 55). Meaning is located here, but it is still necessary to break it down to the paradigmatic unit in order to be situationally comprehended, comprehended, as Barthes presents the syntagm as a fundamental “chain...which must be carved up,” ( Elements ( Elements of Semiology Semiology 65). Mass Effect 3’s 3’s HUD is a syntagm composed of several active paradigms, but as Barthes suggests as well as practice dictates, the paradigms must be interpreted individually so the syntagm’s meaning can be deciphered. It is at the syntagmatic level where the breadth of the videogame’s meaning is derived, often displayed through the same long, overarching methods: methods: systems of rules delineated by Bogost in his analysis of the videogame’s operative mechanics; perceptions perceptions of space, whether “hyper-realized” in sports or racing games, or “highly abstract” in Super Mario Bros (Wade 78); genre, used not unlike its application in traditional media, as well as narrative; and perspective, whether it be top-down or isometric, rst-person rst-person or third, the perspective is perhaps the most complex system of signieds of them all. Perspective is a syntagm that inuences how an individual game will function: its play mechanics and interface, visual style, and even inuence how the social actor — the player — perceives themselves, their avatar, or both as one. All videogames communicate via a cocktail of syntagmatic clusters, a language, which varies in paradigmatic paradigmatic composition, composition, like dialects, dialects, from game-to-game, game-to-game, but they all all


23

must singularly communicate from these previously introduced high-level syntagms. All videogames are kin to to one another.

Intertextuality As we know, high functioning functioning syntagms syntagms and lesser paradigms form larger systems, which beget even larger structures until they encompass a language completely their own. But these languages are bound to context and proximity. Even at the highest levels of semiotic processes within a domain, be it videogames or anything else, a foreign social actor will not be able to interpret every sign or relationship, while at the same time they could in another domain, or culture, from which they are a part of. A culture is essentially a grouping of symbolic codes (Leeds-Hurwitz 17), and is the highest term given to semiotic structures. The whole of videogames — the medium, its games, its social actors and all that surrounds them — is a culture. Videogame culture viewed from the outside stereotypes its users as “those who forgo all other cultural, social or even hygienic activities in favor of videogames,” (Bogost, Unit Operations 52). But as the consumer base expands and record nancials are set each sales cycle, more and more uninitiated actors become acquainted with the culture, disregard or even reshape those stereotypes, and thus they become more familiar with the medium and its peculiarities and its specicities — they learn l earn to interpret the signs and structures that were once unintelligible to them. “Every time we make meaning by reading a text or interpreting a graph or picture we do so by connecting the symbols at hand to other texts and other images read, heard, seen, or imagined on other occasions...which connections connections we make (are) characteristic of our society and our place in it...”(Lemke 73). Participants in this culture are able to understand individual videogames because they have played other videogames, representatives representatives of the genre. genre.

24


It is Hodge and Kress’ metasign, metasign, “markers of social allegiance which permeate the majority of texts” (Leeds-Hurwitz 27), that allows social actors to interpret other signs with signs. In our discussion of other other sign types we have already touched upon examples of metasigns in videogames, most notably and

Kombat Figure 13: Mortal 13: Mortal Kombat

ubiquitous, I believe, the health bar (reiterated in Figure 13), a sort of volumetric percentage representation representation of an abstract quantity. As a metasign, which are mainly comprised of important codes and some individual structures, the health bar is dispersed evenly throughout the medium, and even though it can be represented represented different visually, its meaning is universally understood. The most important semiotic aspect in the videogame medium involves the use of metasigns and their ability to speak across discrete units. The breadth of videogames’ ability to communicate comes from its intertextual qualities. Intertextuality qualities. Intertextuality is a concept derived from Mikhail Bakhtin, describing describing the ability of a text to make reference to another or to several others (Leeds-Hurwitz 41). Intertextuality allows games to speak amongst themselves through semiotic means, transcending time, bureaucracy, and physical media. Several intertextual elements have already been introduced: of course, any metasign like the recently mentioned health bar, but any other shared trope in videogames as well well — numerical numerical scoring systems, systems, narrative narrative clichés9, HUD elements, and environment depiction standards. Additionally, Additionally, there are videogames that include less than subtle homages to previous titles. Software development itself has a rich history of its developers inserting messages into their coding, and videogames are no exception to this. Many videogames contain Easter eggs, included materials hidden by developers and often meant to be humorous to or lightly editorialize the 9

Many already shared with classic literature, such as Joseph Campbell’s monomyth.


25

game (Bogost, How (Bogost, How to do things 37). They are often obscurely located within the game environment and difcult to view, but often make meta-jokes, about the game itself, other games, or even people or events (Figure 14). Intertextual elements are not only within and around around videogames themselves, themselves, but also manifest manifest themselves as interlopers exploring other semiotic domains, who then retrieve new elements. As the videogame industry continues to grow, it begins to adopt business practices increasingly similar Figure 14: Grand Tef Auto : San Andreas - In-game Easter egg

to that of traditional media and that of the

the entertainment complex. complex. To increase awareness of the product, videogame icons and symbols now saturate various markets — clothing, food, ringtones, toys, bumper stickers, nearly any sort of merchandise you can think of has had a Call of Duty or Halo or Halo emblem on it. You see trailers for videogames on television just as you would lms before, and print advertisements advertisements everywhere else. else. This is intertextual intertextual and extratextual extratextual because it illustrates illustrates the videogame’s videogame’s ability to make make reference to itself itself even while being being outside of its original text. At the same time, time, videogames show the the capacity of recursiveness, recursiveness, mirroring mirroring its ability of impacting surrounding cultures by displaying the action of cultures in itself. In-game real world advertisements exist in videogames where it makes diegetic sense, ones that emulate a sort of factual reality like in sports or racing games. These games can display advertisements from actual companies, updated dynamically through constant internet connection, and adjusted to suit the advertiser’s target demographic. For example, in 2008, the Barack Obama presidential campaign campaign paid for advertisements advertisements

26


to be displayed in games like NBA like NBA Live 0810 (Figure 15). Advertisements Advertisements especially legitimatize the videogame as an intertextual medium, because as Barthes would point out, the semiotic properties of the advertisement are purely intentional, “formed with a view to the optimum reading,” (“Rhetoric of the Figure 15: NBA Live 08 - In-game advertisments

Image” 152). This (along with the millions of dollars spent to produce such exchanges) signies that the videogame medium is mature enough to clearly accommodate its own semiotic processes, processes, an individual game simultaneously referring referring to itself and games before it, in addition to to incorporating incorporating outside signications. signications. However, not all intertextual elements are as obvious nor as integral as others. The most subtle element in videogames, gameplay, is its most inuential and also unique to the medium. The process of gameplay — that is, the gestalic culmination of elements that compose the play mechanics mechanics of a videogame — is made up of interactions of codes; codes in the semiotic-sense, semiotic-sense, and the coding of machine languages. Videogames, just as other completely completely digital media, media, are created from a myriad of computer languages, complicated structures complete with their own systems of signiers and signieds. To create a videogame from basal codes is complicated, time consuming, and scally unadvisable. In order to circumvent a portion of the menial labor in the creation of videogames, many developers use pre-congured software frameworks called game engines. engines. Game engines are provided to developers by other videogame developers developers — thus establishing establishing semiotic sub-culture systems systems amongst amongst themselves — to aid in the production of their medium. One engine can create several different videogames. The engines are often presented as proprietary or open source software suites containing development tools to render graphics, alter in-game physics or articial intelligence, to script events, sounds, or any number of manipulable 10

Source


27

elements. “Game engines move far beyond literary devices and genres. Unlike cultural categories like the modern novel or lm noir, game engines regulate individual videogame’s artistic, cultural and narrative expression,” (Bogost, Unit Operations 57). Bogost seems to suggest that while game engines take the “drudgery out of game development”, they are

Figure 16a: Star Wars Republic Commando

also “partly responsible for the massive growth of the game industry”, allowing developers to focus more on discursive opportunities rather than the mechanical agency of a game. Because of the intrinsically similar language, and there by semiotic structures, structures, that game game developers use for particular titles, we can observe clear, intertextual commonalities in gameplay. Figures 16 depicts different videogames

Figure 16b: Lineage 2

that have been produced from the same game engine, the Unreal Engine 2.While these games do not belong to the same genres, or share a great deal of high level syntagmatic codes, we can observe basal similarities in these images, namely perspective.

Figure 16c: Deus Ex: InvisibleWar

28


While game engines, to a degree, reign in artistic expression expression within the constraints constraints of the software (in the same way iMovie does lm for novices), they are still an efcient method of videogame production, of progenerating videogame symbols, to sustain and disseminate those signs, and display the most lucid inuence of semiotic structures in videogames, along with the intertextual relationships they evoke. Videogame developers developers have their own rhetorical rhetorical purposes, purposes, outside of their their individual games, when creating a game engine. Videogames have great communicative properties, but to the average social actor, signication is only for their own reception. Some game engines address this by using graphical user interfaces to allow one to manipulate game mechanics without working knowledge of a programming language, and some engines simplify this even more by being geared toward beginners.

Click Here to Continue


29

Bibliography McLuhan, Marshall. Understanding Media The Extensions of Man . Cambridge: MIT Press, 1964. Print. Bogost, Ian. How to do things with videogames. Minneapolis: University of Minnesota Press, 2011. Print. Bogost, Ian. Unit Operations. Cambridge: MIT Press, 2008. Print. Eco, Umberto. Theory of Semiotics. Bloomington: Indiana University Press, 1976. Print. Lemke, J.L. “Metamedia Literacy: Transforming Meanings and Media.” V isual Rhetoric in a Digital World . Ed. Carolyn Handa. Boston: Bedford/St. Martin’s, 2004. 71-93. Print. Kress, Gunther. “Multimodality, Multimedia, and Genre.” Visual Rhetoric in a DigitalWorld . Ed. Carolyn Handa. Boston: Bedford/St. Martin’s, 2004. 38-54. Print. Arnheim, Rudolph. “Pictures, Symbols and Signs”. Visual Rhetoric in a DigitalWorld . Ed. Carolyn Handa. Boston: Bedford/St. Martin’s, 2004. 137-151. Print. Barthes, Roland. “Rhetoric of the Image”. Visual Rhetoric in a DigitalWorld . Ed. Carolyn Handa. Boston: Bedford/St. Martin’s, 2004. 152-163. Print. Barthes, Roland. Elements of Semiology . New York: Hill H ill and Wang, Wang, 1964. Print Prin t McCloud, Scott. “Understanding Comics: The Invisible Arts”. Visual Rhetoric in a DigitalWorld . Ed. Carolyn Handa. Boston: Bedford/St. Martin’s, 2004. 195-208. Print. Semiotici,ed.5 2009, Print. Wade, Alex. “Spatial Typolog Typologies ies of o f Games”. Gam es”. Associazione Italiana di Studi Semiotici

Foucault, Michel. This is not a pipe. Berkeley: University University of California Press, 1983. Print. Leeds-Hurwitz, Leeds-H urwitz, Wendy. endy. Semiotics and Communication: Signs, Codes, Cultures. Hillsdale: Hove and London, 1993. Print. Deely, John. Basics of Semiotics. Bloomington: Indiana University Press, 1990. Print. The digital work produced in this paper was created with the Construct 2 engine by Scirra Inc. (2012), operating under the free license.

http://jarrodt.me/thesisgame

Games Robinett, Robinet t, Warren Warren.. Adventure. Atari Inc., 1979. Atari 2600. Ubisoft Montreal. Assassin’s Assassin’s Creed 2. Ubisoft, 2009. Multiplatform Number None, Inc. Braid . Number None, Inc.. 2009. PC. Rockstar North. Grand Theft Auto: San Andreas. Rockstar Games, 2004. Multiplatform. Bungie. Halo: Combat Evolved. Microsoft Game Studios, 2001. Xbox. Nintendo. The Legend of Zelda. Nintendo, 1986. Nintendo Entertainment System. NCsoft. Lineage 2. NCsoft, 2004. 2 004. Windows/PC. Bioware. Mass Effect 3. Electronic Arts, 2012. Multiplatform. Capcom. Mega Man. Capcom, 1987. Nintendo Entertainment System. Midway. Mortal Kombat. Midway, 1992. Arcade. EA Canada. NBA Live 2008. EA Sports, 2007. Multiplatform.

Semiotics and Intertextuality of Videogames

Recommend Documents