Literature has always involved aspects that exceed the linguistic art constituting its primary focus. We need only remember the gestural and aural aspects of oral poetry, the gorgeous visuality of medieval illuminated manuscripts, the iconicity of Renaissance emblem books, and the illustrated books of such writers as William Blake, William Morris, and Lewis Carroll to appreciate the integrative aspects of the art forms we call "literature." The twentieth century has seen the blossoming of artists' books, a genre that uses a wide variety of forms to interrogate the book as a technical, material, and aesthetic production. In the last two decades electronic literature – that is, literature that is "digital born," created and read on a computer – has continued to expand the scope of literary art by combining language with animation, graphics, visual images and sonic landscapes. Dating back at least to Ted Nelson’s vision of hypertext, early practitioners worked with DOS commands in the 1960s but it was not until the 1980s and the advent of desktop computers that the genre took off, aided by the development of Storyspace, a hypertext authoring program developed by Mark Bernstein of Eastgate Systems in conjunction with Jay Bolter and Michael Joyce, also the author of Afternoon (1987), the first widely read and discussed literary hypertext. In the 1990s and beyond, the explosion of artistic creativity fostered by the information age is transforming what it means to read and write. In the process, works of significant literary interest are becoming so deeply multimodal that they could equally well be considered digital art as electronic literature. This essay discusses a range of works that inhabit the borderland between digital art and electronic literature, including installations created for immersive projective environments, works that combine text with graphics, animation and sound, and “code work” that joins natural language with expressions drawn from programming language.
One of the signs literature has become a player in the arena of digital arts are works that function as site-specific installations, giving the lie to the assumption that literature resides only in books. Good realistic fiction is often called "immersive," but the works discussed below are literally so, in the sense that they are performed in three-dimensional CAVE environments.1 The CAVE uses high-speed computers to generate an immersive environment that employs virtual reality goggles and data glove(s) (or in some cases, data wands or joysticks) to mark the position of the user's gaze, hand position, and location within the CAVE, typically a four-surface room that includes three walls and a floor display. When the user moves her head, the computers generate the calculations that change the display accordingly, so she is given the impression that she is moving (or flying) within a complex three-dimensional environment.
Screen by Noah Wardrip-Fruin and his collaborators, including an introduction narrated by Robert Coover, transports literature into the CAVE environment.2 As the work begins, the user hears Coover read the words "In a world of illusions, we hold ourselves in place by memories" and sees text displayed on the three vertical CAVE walls in billboard fashion. This theme becomes enacted in a startlingly literal way when words suddenly begin peeling away from the walls and moving in the three-dimensional space. [Figure 1] The user can try to bat them back into place with the data glove, but as she interacts with the words, they peel off faster than she can put them back, despite her best efforts. Moreover, the batted words move along trajectories difficult to control, creating neologisms, nonsense words, and chaotic phrases that further make the text difficult to read. Eventually all the words lay jumbled on the floor, the text now impossible to recover for "normal" reading. In another sense, of course, the work has re-defined what it means to read, so that reading becomes, as Rita Raley has pointed out, a kinesthetic, haptic, and proprioceptively vivid experience, involving not just the cerebral activity of decoding but bodily interactions with the words as perceived objects moving in space.3 Entering the narrative in this case does not mean leaving the surface behind, as when a reader plunges into an imaginative world and finds it so engrossing that she ceases to notice the page. Rather, the "page" is transformed into a complex topology that rapidly transforms from a stable surface into a "playable" space in which she is an active participant. "Playable media," a term coined by Noah Wardrip-Fruin to denote computer games and other interactive works such as Screen, accurately expresses the user's engagement with the game-like aspects of the work.4
When we encounter the playable (and playful) complex surfaces of New Media, we do not leave behind our long experience with print. In his recent work, John Cayley has focused on the ways in which our intuitive knowledge of letter forms can define space and inflect time. Working in the CAVE environment at Brown University, he and his collaborator Dmitri Lammerman created Torus, a virtual reality installation in which sixteen vanes of text are arranged like slices through a doughnut.5 This torus shape is doubly virtual, for it is not imaged as such but rather is brought into existence by the text-slices that implicitly define it for the user. The play between what the user's imagination constructs and what is actually visible transforms the typical situation in literature, in which the user decodes words to create an imaginative space in which the action takes place. By contrast, the CAVE room actually exists in space, and the user explores it through embodied actions such as walking, turning, and listening. At the same time, the user can also read the text and re-create for herself the imagined world of Proust's Remembrance of Time Past that appears on the torus vanes.
Further complicating the writing surfaces is yet another dynamic, the relation between the virtual text and the massive computations generating it. Unlike durable ink inscription, the text here is a virtual image and so is capable of transformations impossible for print. In this installation, the text demonstrates its agency by moving in space, responding to the user's spatial orientation by always turning to face the viewer. Through this motion, the user experiences a temporal dimension of the text – its motion so that it is always right-reading – acting in complex synchrony with the time of reading and the time of spatial exploration within the CAVE environment. These temporal interactions, as well as the virtual / actual spatiality of the textual surfaces, create an enriched sense of embodied play that complicates and extends the phenomenology of reading.
Influence can flow in the opposite direction as well, from the phenomenology of reading back into the installation. This effect was discovered when a glitch in the program caused letters that were proportionately smaller, and thus perceived as farther away, to be rendered over larger letters in the foreground. To reconcile the contradiction, users perceived the smaller letters as if they were inscribed on the back wall at the end of a corridor – a corridor that did not exist except in the user's perception. Cayley theorizes that our extensive experience with letter forms subconsciously affects perception, so users struggle to create a scene that preserves the integrity of letter forms while still making phenomenological sense. In this understanding, Torus becomes not only an experiment in the enriched phenomenology of reading but also in the complex interplay between our traditional experience with letter forms and our much more recent understanding of computation. Reading in this view becomes a complex performance in which agency is distributed between the user, the interface, and the active cognitions of the networked and programmable machine (or in Cayley’s preferred terminology, the programmaton).
As these works suggest, the boundaries of literary art have become porous membranes. Rather than trying to police them to ensure "the literary" remains pure and undiluted, it is more productive to regard such works as creating a nexus in which literary material synergistically cooperates (and occasionally competes) with other elements from the visual, sonic, digital, and graphic arts. Examples include Dan Waber's Strings, Flash animations in which strings are used as verbal / visual signifiers. [ Figure 3] Sometimes the strings bunch up to form words; at other times they perform gestures in a series of fluid transformations that create mini-narratives, flirting, arguing, making up, making out.6 Whereas Waber's work subordinates the linguistic to the visual (in the sense that the stories are minimal and the content banal), Robert Kendall's Faith, an exemplum of the rapidly growing genre of Flash poetry (created by means of the Macromedia Flash software), uses the opposite strategy of employing visual and sonic effects to enhance a verbally dense literary work rich with ambiguities.
Poets have always employed strategies to make words do double, triple, and quadruple duty, using such effects as assonance, rhythm, enjambment and a host of other techniques to create resonances within and between lines so that multiple significations come into play simultaneously. In Faith, these traditional effects are combined with color, music, and animated behaviors to generate a host of multiple meanings. This kinetic poem reveals itself in five successive states. As the poem's speaker struggles to resolve the conflict between logic and faith, each state is overlaid onto the previous one, with color coding (from orange for the first, then red, burgundy, and gray for successive states) reminding the reader how the letters and words are being recycled. Each time a word or letter is re-used it changes meaning but nevertheless retains resonances from earlier usages. For example, in the second red state, "edge" appears in the sentence "I edge / logic / out"; in the third state, a burgundy "h" appears and converts "edge" to "hedge," while the red letters in "out" disperse themselves in "no, I just can’t ," so that even as the speaker recognizes the call of logical reasoning, the imperative of faith lingers in red colors as one strata of signification is overlaid onto another.
Of course, the synergistic combination of linguistic effects with the affordances of digital media is possible only because layers of code, functioning as commands to the computer, generate the surface visible on the screen. This creates a situation in which the legacy systems of speech and writing, the home provinces of literary effects, are in dynamic interplay with the new language system of computer code. Increasingly, writers who work in networked and programmable media are insisting that the underlying computer code must be considered part of the literary work, a position that Loss Pequeño Glazier, among others, has eloquently argued.
Just as artists working in visual and plastic media have created works that interrogate the materiality of the medium, so literary authors working in digital media have created a genre called "codework," characterized by a creolized discourse in which expressions from coding languages inflect and infect the so-called "natural" language that has traditionally been the native tongue of literature. A creole, defined as a distinct new language created by blending the grammar, syntax, and lexicon of two original languages, normally emerges when speakers from different geographic areas and linguistic traditions engage in daily discourse with one another (for example, "Spanglish" in Southern California). Two eminent practitioners of codework are Talan Memmott (notably in the digital work Lexia to Perplexia9), and MEZ (aka Maryanne Breeze), whose Web piece ][ad][Dressed in a Skin C.ode shows her "mezanglled" prose at work.
TA section entitled "_Back-and-Foregrounding_" in plaintext (that is, natural English language before it is mashed into a creole with programming language) describes the transformation of sensibility that occurs when the persona encounters the computer and is forever transformed:
From an institutional perspective, exhibiting interactive art and other works that invite the visitor’s touch could also be justified by commercial reasons. In today’s hyper-competitive marketplace, the art museum has to compete for "customers" with institutions such as theme parks, science centers, sports stadiums, concert halls, shopping malls and, increasingly, technologically saturated private environments. The museum must be able to relate the experiences it offers to already familiar forms of behavior and culture. At the same time, it has to guard its own territory and maintain its identity as distinct from the “experience industry." Emphasizing tactile communication may be a way of attracting new visitors to the art museum, as was demonstrated by the success of Game on, an exhibition of fully playable commercial videogames, organized by the Barbican Gallery (London) some years ago. Yet, combining behavioral modes derived from everyday life with unfamiliar ones is challenging. Working in a stressful situation where the stakes are high, the museum may find it difficult to draw a line between aesthetically and conceptually uncompromising experiences and pre-digested “art entertainment.” Before opening the gates to “anything tactile,” it is necessary to ask some basic questions. What is essential in the art experience? Silent contemplation? Forms and colors? Themes and stories? “Touch of eternity?” Sensational attractions? Visiting a “place” with family or friends? Or, perhaps, the discovery of something new and unprecedented through interaction and touching?
A Mezzian Flesh-Mote enters a library. In a networked sense this library is cold; binary data advancements are yet to make any perceivable impact on itsmanifest functions. A silvered sliver-glint pulls the Mezzian Mote forward to the only technoniche available – a computer laboratory, used primarily for word-processing tasks. It also has an Internet connection. A Datadervish [E-Mote] is born, and a Flesh-Mote is extinguished.
A tale of transformation, the story can only be told from a retrospective view (for it is only after the fact that the transformation can be recognized as such), and this angle of vision is reflected in the vocabulary. "E-mote" is a formation born of the Web, a verb transformed into a noun by the interjection of a dash that references the electronic ("E") world and the subjects who emerge from it.11 Through back-formation the subject prior to her electrification is named a "Flesh-mote," a word that already recognizes the individual will exist in a haze of networked others as soon as it transforms into an "E-mote." Pulled forward by the gleam of the screen, the Flesh-Mote finds the means of her transformation in the computer, primitive though this particular laboratory is.
Now consider the "mezangelled" text, which compresses the plaintext and, paradoxically, through compression extends its implications.
.a mezzian flesh-mote enters.
.the libr][bin][ary is cold. a s[]l][i][ver glint pulls the mote 4wards.
.4warding][ing of the datadervish][in2 the][comp][lab lair.
At first it appears that the prose of the plaintext has been converted into poetic lines, a transformation that brings into play the traditional poetic tension between the ending of one line and the beginning of another. However, in the programming language Perl, the dot is a concatenation operator used to add strings together, so the lines now exist both as discrete units and additive lines, with the dot signaling division when read as a period ending a sentence; and addition when read as a concatenation operator preceding the string. The second line, typical in its use of interjected square brackets, shows how mezangelling works. "Library" can be recovered as a word, but only after encountering "][bin][ary," the binary code still largely missing from this "cold" library. "Binary" is in a sense now hidden or found to be concealed within "library," a form of reading that anticipates the coming transformation of this institution as it creeps into the information age, a process already begun in its primitive word-processing laboratory. Read as operators, the brackets in this mezangelled word do not make sense, for there is no opening bracket for the initial right bracket, and no closing bracket for the final left bracket. Despite its violation of normal syntax, “][“ has a polysemy that draws MEZ to it, for it resembles "I," the nomination of selfhood, and also "H," which by back-formation can often be read as "I" in her texts. Although the brackets can be broken apart, "][" often functions as a symbol in its own right. That the "bin" of binary should be surrounded by this symbol suggests the implication of the subject "I”"in the discovery of the binary within the library, an association that the plaintext makes clear in other words.
The "silvered sliver-glint" of the plaintext is now compressed into a mezangelled word that folds "silvered" and "sliver" into one through the interjection of brackets, a process that also twice creates the "][" symbol and so interjects the "Mezzian" of the plaintext into the middle of the word, so that now "mote" appears without the preceding adjective. "Forward" becomes "4wards," a word homophonically recoverable as the plaintext term but also visually contaminated by a number combined with an English syllable in a creole that signals the in-mixing of numerical code with language. In the mezangelled text, the "Datadervish" is moved "4ward][ing" into a lab, a prescient anticipation of the transformation already encoded by the interjection of the "][" symbol into the motion of moving forward. "Computer laboratory" in the plaintext becomes "][comp][lab lair," with the "]["symbol now surrounding "comp," emphasizing that the "I" and "computer" have now joined in a space that has also become a "lair," with the connotation of secrecy, protection, and most of all habitation.
The transformation, in the plaintext performed by the assertion that "A Datadervish [E-mote] is born, and a Flesh-mote is extinguished," is now dramatically enacted by a visual and verbal full stop, punningly performed by bolded dots and the word "stop."
>>stop<<
In older programming languages such as Basic, "stop" interrupts the currently running program. Here, however, it is not the program that ends but a certain kind of pre-electronic subjectivity. The dots above and below this process serve both as dividers and connectors (when read as concatenation operators), thus marking the splice from one kind of subjectivity to another.
The interior voice associated with traditional print texts, particularly the novel, helped to create a kind of subject appropriate to the rise of the mass medium that transformed social and economic structures throughout Europe, namely the print book. Now, with the advent of digital media, new kinds of subjects are being performed and created by digital art works that can, in some respects, still be seen as "literary." At their core is a blending of human and machine cognition, the counterpart in subject formation to the linguistic mixing of natural and computer languages in creoles such as MEZ's "mezangelled" prose. The stakes are much higher than the question whether literature will continue to exist as a discrete art form specializing in linguistic effects. Seen in this context, the blending of literature and other art forms in digital media is part of a larger transformation in which language and code, human subjects, and intelligent machines are co-producing the art works that probe the implications of what it means to be human in our computationally intensive culture.
Will works such as those discussed above still be called literature? Yes and no – yes by those, like me, who count themselves among the stakeholders of a literary tradition that stretches back for thousands of years and continues to inform the linguistic effects on which these works depend; no, perhaps, by young readers who surf the Web and see in these works affinities with computer games, Web videos, and animations from blockbuster films. However, the one outcome of which we can be certain is that these works, by whatever name they will become known, will be a vitally important component of the continuing innovation and creativity that inform the digital arts of the twenty-first century.