Glitches and Black Boxes

In this blog post I would like to expand upon my practicum exercise from week 9. In playing with all three of the course tools for digital deformance I was struck by the layers that they added to artifacts in the process of deforming them. The GIFmelter and Image Glitcher felt as if they were applying filters (Lisa suggested as much in her practicum). While these are certainly new ways of looking at these texts, they don't seem quite aligned with Bogost's model of carpentry discussed in notes towards a deformed humanities.

What I didn’t note in My practicum but which bothered me about these tools was the degree to which I was unsure if what I was seeing was an accident, an aberration in the coded structure of the texts being glitched or if they were simply processes crafted to give the appearance of a technical mishap. This recalls the 5^th point raised by Rosa Menkman in “The Glitch Manifesto,” “Realize that the gospel of glitch art also tells about new standards implemented by corruption. Not all glitch art is progressive or something new. The popularization and cultivation of the avant-garde of mishaps has become predestined and unavoidable. Be aware of easily reproducible glitch effects automated by softwares and plug-ins. What is now a glitch will become a fashion.” (11) These tools were aesthetically and maybe formally glitchy but they didn't seem to be a function of noise introduced in their transmission but rather were applied after the fact. I suppose this is a matter of identifying where in the Sender->Reciever chain noise is introduced. But these technologies seemed more in line with the Databendign Menkman experiments with around file formats than a glitch produced in a signal during transmission.

These might all be the same kind of glitches in the sense of the “Glitch Manifesto’s” 7^th point that we ought to “rejoice in the critical trans-media aesthetics of glitch artifacts. Utilize glitches to bring any medium into a critical state of hypertrophy, to (subsequently) criticize its inherent politics.” (11) This seems to welcome an expanded sense of the glitch. Certainly the data-bending that Menkman does depends on the introduction of noise, but I wonder what the ontological difference between this glitch and a photo filter is then? Is it the unexpected or random nature of it? Is it still random if you've written a program with the intention of making it random? The databent images in Menkman’s work look very much like the images produced by the Image Glitcher software, but Menkman’s operate by sending the user into the code where the Image Glitcher simply puts a new black-box in the mix. Menkman explains at length the processes by which she achieved particular glitches, by copying code, by introducing random code, etc. Each of these produces a different effect, and these differences reveal different operations at work in making image data readable to a user. This brings us back to the questions of Bogost’s carpentry in Mark Sample’s “Notes Towards a Deformed Humanities.” The quote that Sample has cited (and which I’ll borrow) defines carpentry as “making things that explain how things make their world.” (Bogost, 93) 

    

Certainly the products of Glitchers, Databending, Melters, and Meaning-Eaters are new texts, but they don’t all necessarily point us what is going on in the machine, at least not in the same ways. The glitched image may recall it; I had a text message image download improperly on my phone once and the image glitcher resembles that, just like it resembles the databent images from Menkman’s work. However, I don’t understand the glitched image produced by the Image Glitcher and there doesn’t seem to be much cause for me to direct my attention towards the internal processes that have produced it. It’s the intended result of the tool - where my phone corrupting image data is cause of confusion or Menkman’s databending is cause for experimentation, the tool produces a planned outcome. The tool is disruptive to the images formal coherence - I can think think about the image and what (in a humanistic sense) is the code of an image, of its ineligibility, of its form. But with these black boxes I’m not necessarily thinking of the machine and of noise in the same ways. So, in a wider view of defeomance all these technologies might cause us to think about the cognitive mechanisms we've developed for receiving images, gifs or online articles. However, the process seems to matter for the critical capabilities of these tools. The objective of Bogost’s carpentry is to turn our attention to technical process, to world-making. Likewise the Glitch Manifesto’s first point poses glitch as a critique of a fetishization of coherence, or noiseless transmission. It seems to me that these projects take a critical edge when we are encouraged by this deformance/transformation to think about the process, in addition to the aesthetic effects.  

At any rate, all of this reminded me of my object from my practicum in week 8, the shoe-box of NES games on my coffee table.  Below are two videos of one of these games glitching. Where the deformance tools seem to layer code onto their objects producing a new text in the process, this seems to function through the loss of some information. What is going on on my TV screen is a hardware glitch, somewhere in the machine  (in this case the pins that connect the game cartridge to the game console) information is lost or corrupted. My Nintendo is trying to make sense of the games program despite some information not getting passed to the machine. The result is a visually baffling but still playable version Ducktales 2.

After digging this out of my bedroom a few weeks ago and moving it to my apartment I've encountered a number of these glitches. Sometimes the game won’t start, or often static runs through the screen. In a particularly interesting case, I encountered a level in Super Mario 3 that was missing part of a hillside that caused Mario to hover as if he were permanently mid-jump. In considering the implications of the hardware glitch, it’s not simply the textual object and our relationship to it that is changed, but in the process the underlying technological processes that produce the text which are made apparent. In order to rectify the glitches that appear on my NES I’m forced to break with the game’s play interfaces, the TV screen and the controller. I turn my attention to the hardware interface. I try the reset button and the power button, but these don't solve the problem. The signal is disrupted somewhere deeper in my machine. I could jiggle the cartridge inside the machine to try to establish a better connection. Or, I could take the game cartridge out and blow on the connector pins on the game-board protruding from the bottom of the cartridge, to remove any dust. In either case this interface isn’t made for me, the pins form an analogue connection that allow the NES to read the information on the game cartridge. By blowing on the pins I recognize this point of technical interfacing as the site at which the noise is introduced, I’m forced to think through the operations inside my NES and intervene in the process of transmission. I become aware of the world making of my NES rather than simply taking pleasure in the aesthetic of garbled information on my TV.

The NES Cleaning Kit,
Licensed Nintendo product for cleaning game connectors.

What is especially interesting about this version of deformance is that the reading technologies necessary to produce the view of Ducktales2 that has appeared on my screen are only really available on the original hardware. The glitch is integral to the process of reading done by the NES’s original hardware, it’s a pattern that relies on an analogue disruption of the NES’s internal system. This means that there isn't a way to accurately emulate this kind of deformance/glitch with layers of new code. It’s here that I feel like we are seeing an ontological difference between active tools deformance and the intervention of noise more organically.

Thinking about this distinction between the two is best illustrated by NO-CARRIER's NESglitch tool. This program tries to reproduce familiar NES glitches by writing an executable NES ROM File that approximates the aesthetic of fuzzy connection inside a Nintendo. The NES glitch is so integral to the console and the experience of playing that it takes on a kind of nostalgic cache and an aesthetic of its own. The Angry Nerd videogame, a tribute to a popular YouTube channel that reviews classic games, features a level that centers on the glitch such that the word Glitch appears onscreen. The letters are jumbled to look as if they were produced accidentally on an old NES. NO-CARRIER’s tool aims at a similar effect. Without the analogue disconnect between metal connecting pins inside the NES, NO-CARRIER’s game scripts patterns that look like these glitches. Using the button inputs on a controller the user can cycle through a combination of patterns that approximate the look of a hardware glitch. However, a comparison between NO-CARRIER’s tool and a glitched NES game reveals important dissimilarities.

The first image below is Produced by NO-CARRIER's ROM file, loaded into an emulator (at least that's how I used it, you could also load it onto an emulator cartridge and play it on an NES) while the second is a hardware glitch produced in Snakes Revenge. They have similar elements, but some of the game world in Snakes Revenge remains visible below the garbled pixels. The Image produced by NO-CARRIER’s tool contains some of the same formal properties, but without the background as a point of reference. We see pixelated and scrambled cells but there isn’t a guiding pattern behind them. They are spread across the screen in a too-random checkerboard pattern. This dissimilarity results because they are produced in very different ways. This, I would like to argue is an ontological difference and produces a difference in what they allow us to read. The glitched NES image reminds us that there is a coherent text somewhere behind the illegible flurry of stay letters and pixels. We know that something responsible for bringing this coherent text to us has failed and the result is a game that causes its reader to reflect on the technical process that makes an NES cartridge playable. NO-CARRIER’s tool can’t point us to that internal operation. Playing with patterns produced intentionally is an interesting experimental practice in glitch aesthetics, but the process that it recalls does less to foreground technology, putting stock in the nostalgic aesthetic instead.

NO-CARRIER GlitchNES

Snake's Revenge

Bogost, Ian. Alien Phenomenology, or, What It’s Like to Be a Thing. Minneapolis: University of Minnesota Press, 2012. Print.

Menkman, Rosa. The Glitch Moment(um). Amsterdam: Network Notebooks, 2011

Sample, Mark, "Notes Towards a Deformed Humanities," Sample Reality, 2012

Games and Map Analysis

In Graphs, Maps, and Trees Moretti explains, "you reduce a the texts to a few elements, and abstract them from the narrative flow, and construct a new, artificial object... and with a little luck, these maps will be more than the sum of their parts: they will possess "emerging" qualities, which were not visible at the lower level." (53) For literary texts like Moretti's  village stories this is a form breaking exercise, it transforms a story, words, ideas into something visible, and in this visualization a pattern becomes identifiable. The map produced allows a visualization of the world of the stories that must otherwise be imagined by the reader. For movies and games, this visibility is already part of the medium's essential quality. More, for videogames, the map is a frequently employed device for understanding the players location within a textual world. These are rendered in little bubbles in the bottom corner of the screen, found in pause menus, and sometimes integrated into the dieresis of the game as PDA screens or pip-boys. This might be as simple as hi-lighting formal distinctions between these media, but it also might be cause for thinking about narrative arrangement in games relative to DH mapping for literary works.

A Google Image Search of “Controller Mapping” returns a page of schematics.

The ‘map’ is a recurring motif in video gaming’s formal imaginary. We might think of the multiplicity of ways that the term is deployed. Levels are alternatively called maps so that when a new set of levels for a game are released the bundle is called a map-pack. Button functions on a game controller are discussed as controller mapping- a representation of the virtual effects of haptic inputs. The process for pinning details, texture, or images to a 3D Model is called texture-mapping. And most obviously there is the game’s world-map which we might identify in two ways. The representation of the game-world that the player moves through. This is roughly level-map association, a visualized representation of the player’s position relative to the objects in the game. Alternatively we may think of maps for navigation as they appear in games in pause menus, HUDs, and even as separate texts. These are an abstraction of the world that highlights only certain information relevant to play. The Grand Theft Auto series is an example of this.

These are three kinds of maps for GTA 3: the pause menu,

 in the HUD, and in the printed game manual.

Pausing the game brings up a world map complete with a legend and a changing set of objectives and location's marked on the map. This is an abstraction of the world that the player moves through but it responds to the players’ location in the game. During regular play a small portion of this map (the area directly around the player) is visible in the lower left corner of the screen; this helps the player locate himself in the level relative to the larger game-world. Finally, printed in the game’s manual is a large fold-out map of the world this functions similarly to the pause menu map, only it’s not interactive.

Unlike the literary maps, that Moretti’s work produces these maps do very little for text analysis. Although the in-game map of GTA 3 resembles, for example, Moretti’s figure “Protagonists of Parisian Novel’s, and objects of their desire,” (55) there is comparatively very little that is transformed by an attention to GTA 3’s map. Perhaps this is because the DH maps that Moretti produces challenge the formal structure of the texts that they analyze where the game map is already so integral to the medium. For players of GTA (or Fallout, Far Cry, Final Fantasy, Etc.) the map doesn’t reveal an element of the narrative world otherwise difficult to visualize, it simply points players to new objectives and streamlines their travel to these points. Many of these games even use the map as a way of excising the experience of distance in the game world- you may select and travel from distant locations in order to arrive more quickly at waypoints. So, unlike the DH map which aims at something transformative, the deployment of game maps seem to work in the opposite direction. They simplify and consolidate the formal experience with the games listed here.

More than illuminating otherwise easy-to-miss structural elements these maps work in the opposite direction. They tell “white lies” as Jason Farman citing Mark Monmonier explains, “The various ways by which maps arrive to users in locative media projects serve either to obscure the sedimentation of historical and cultural forces that constrain how we interact with maps or, conversely, serve to expose those forces as a form of critique.” (89) In either instance, Former would like to suggest that this is a generalization of information in the interest of utility. For games this goes a bit further. They operate not simply as a generalization of the world but in many respects build into this representation operational functions that facilitate the players’ movement through the text. More than simply telling players how to get to an area, it tells them which areas are priorities, using the map in GTA or Fallout players can transport themselves instantly to these locations. What’s crucial about these functions is that they direct the players’ attention – inward and towards key locations. This discourages play that finds the user bumping up against the edges of the game world or wandering into areas where the action isn’t. It’s here also that the alignment between these functional maps and texture mapping and controller mapping seem most complete. The texture map obscures the rough 3D model beneath the games visual appearance. Likewise, the controller map streamlines an interface. Configuring inputs in such a way that they become invisible to the player once they are familiar allows for a haptic experience at the interface that feels natural. Ideally the controller fades but even failing this, a controller hides the computational processes that transform physical inputs to digital action.

The question that this begs is whether this kind of mapping can be disrupted and if this level of disruption might allow for something transformative in the sense of Moretti’s DH maps. One instance we might consider is failed texture mapping. The image below is the result of a missing texture in the game Team Fortress 2, the purple checkerboard is the grid onto which the texture ought to be placed and in its absence it is exposed to the player.

  

A Missing Texture in Team Fortress 2

This example is more the work of a glitch than a map deployed as its intended. However, the result is that the undergirding structure of the game becomes visible. The grid and the 3D wireframe peeks through in the absence of the texture. It’s a failure of code, or more precisely missing assets, and its effect is to direct the player’s attention away from the seamlessness of the game’s world. This is a crack in the map’s totality but the critique that it enables is less a matter of production than Moretti’s maps, which facilitate new critical visualizations.   

A hopeful, proof of concept for games and critical mapping might be the story flowcharts produced by fans for visual novels. These games adopt a stripped-down ludic structure where players make a series of decisions to advance the story, rather than taking control over an avatar or units. There is no world map for players of these games. Rather they unfold as a series of comic book panels with certain choices opening certain paths and closing others. The flowchart then allows players to spatialize and visualize choices that are otherwise rendered invisible by the game code. Unlike glitching which peals back the seamless exterior of TF2, this flowchart lifts the curtain on the narrative arrangement of this game. It transforms a text which resists a visual map (there is no-world map for this game) into one that makes the game easily navigable. Where this does align with the TF2 Glitch or Moretti’s maps is in its exposure of code and form. The narrative is laid bare here but only by work of identifying where decisions branch into different directions. This revelation re-values the choices offered to the player in game. It demonstrates that some choices are empty while others are of vital importance to the story that the player reads.

  

A Flowchart for Katawa Shoujo

This kind of critical analysis allows us to turn back towards literary work. Particularly, around Choose Your Own Adventure (CYOA) books. KS and most visual novels adopt this CYOA format and both are made accessible by this type of visualization. Samizdat Drafting Co.’s visual analysis of these books demonstrates this compatibility.

Samizdat Drafting Co.’s Story Map for The Cave of Time by Edward Packard

Here we begin to see a formal/structural congruence between the visual novel and the CYOA book. In both instances the map disrupts the code of the text by making its branching paths totally visual. In this respect perhaps it is possible to imagine mapping in games not simply as a tool for producing seamlessness but also for pealing at these seems.  

Farman, Jason. "Map Interfaces and the Production of Locative Media Space." Locative Media. Ed. Rowan Wilken and Gerard Goggin. (New York: Routledge, 2015)

Moretti, Franco. Graphs, Maps, Trees. (London: Verso, 2005)

Samizdat Drafting Co., "COYA," http://samizdat.cc/cyoa/#/_