The Metainterface Part 1 PDF

Contents
© 2018 Massachusetts Institute of Technology Acknowledgments vii
All rights reserved. No part of this book may be reproduced in any form by any electronic or
Introduction 1
mechanical means (including photocopying, recording, or information storage and retrieval)
without permission in writing from the publisher. 1 Interface Criticism: Why a Theory of the Interface? 15
2 The Metainterface Industry: New Platforms for Culture 39
This book was set in ITC Stone Serif Std by Toppan Best-set Premedia Limited. Printed and bound 3 The Urban Metainterface: Territorial Interfaces 81
in the United States of America. 4 The Cloud Interface: Experiences of a Metainterface World 121
5 Interface Criticism by Design 157
Library of Congress Cataloging-in-Publication Data
Names: Andersen, Christian Ulrik, author. I Pold, S0ren1 author. Notes 183
Title: The metainterface : the art of platforms, cities, and clouds / Bibliography 215
Christian Ulrik Andersen and S0ren Bro Pold. Index 233
Description: Cambridge, MA : The MIT Press, [2018] I Includes bibliographical
references and index.
Identifiers: LCCN 2017042793 j ISBN 9780262037945 (hardcover: alk. paper)
Subjects: LCSH: User interfaces (Computer systems)--Philosophy. I Application
software--Social aspects. I Human-computer interaction--Psychological
aspects. j Computer art.
Classification: LCC QA76.9.U83 A524 2018 I DDC 005.4/37--dc23 LC record available at https://
lccn.loc.gov/2017042793
10 9 8 7 6 5 4 3 2 1
Acknowledgments
This book is the result of years of research at the Department of Digital Design
and Information Studies, School of Communication and Culture, Aarhus University.
A special thanks to good colleagues and support from Aarhus Institute of Advanced
Studies, Center for Literature Between Media, and Center for Participatory IT. We also
owe great thanks to readers and reviewers who have provided invaluable suggestions
and comments, including Geoff Cox and the anonymous peer reviewers. Also thanks
to the great editorial team at MIT Press, Doug Sery, Katherine A. Almeida, and Noah J.
Springer. Finally, thanks to our wives and families for sticking up with us during the
long hours of writing and editing.
Fragments of the book are rewritten from published material in the anthologies The
Imaginary App, Critical Theory and Interaction Design, and Disrupting Business and have
been presented at conferences and festivals, including Electronic Literature Organi-
zation Conference, Transmediale festival for art and digital culture, Interface Politics
1st International Conference, Hybrid City Conference, and the Media Architecture
Biennale.
Introduction
Art and Interfaces
To many people, computer interfaces relate to graphical user interfaces, and an implicit
understanding of user friendliness and easy access to functionality. It is less known
that since the early 1960s, there has been a tradition of artistic experimentation with
computers, and this continuously informs a cultural and aesthetic critique-from the
underbelly of digital culture, so to speak. These curatorial and artistic practices were
first presented as part of an art scene that attempted to mix software with conceptual
art. Exhibitions such as Cybernetic Serendipity at the Institute of Contemporary Arts
in London in 1968, Software at the Jewish Museum in New York in 1970, and Comput-
ers and Visual Research in Zagreb in 1968 are all well-known examples of this. Within
computer music, literature, games, and visuals, there are numerous illustrations of even
earlier experimentation crossing computation with various forms of representation and
performance, and today festivals such as transmediale in Berlin, Ars Electronica in Linz,
and numerous other events attract tens of thousands of spectators and practitioners.
This book focuses on this art scene along with its relationship to interfaces, software,
and computers, and more specifically the book's examples are commonly referred to
as net art, software art, and electronic literature. In the words of Geert Lovink, such
art forms "blow up the walls of the white cube ... in such a systematic manner that it
moved itself outside of the art system altogether." 1 So although one may refer to the
works presented in this book as art, art is in this case to be understood in its broadest
sense. It may include what is recognized by the arts institutions and exhibited at muse-
ums, for instance, but it may also include what Gilbert Seldes in the 1920s referred to as
the "lively arts"-meaning a scene that (like jazz, burlesque shows, or comics, as Seldes
describes) finds its institutions outside high culture. 2 This by no means implies that the
book rejects qualitative assertions about the works it presents, but only that the living
practice is not necessarily performed by people who are trained at an art academy, and
Introduction Introduction 3
2
conversely that the artists in this field sometimes make works that are not necessarily An example of this relationship between market innovation and artistic practice can
recognized by arts institutions. be found in the works of net artist Christophe Bruno. In 2001, he launched the Internet
As indicated by concepts such as the postdigital, post-Internet, and new aesthet- installation Fascinum (figure 1) which displayed the ten most searched images on Yahoo
ics, this breakdown between high art and a lively art of electronic networks and soft- portals in different countries in real time. 6 Already in 2004, Fabrica (Benetton's com-
ware also has to do with a normalization of these technologies. 3 The digital, online, munication research center) had produced a similar installation (lOxlO), which ironi-
networked, and so on, have become an intrinsic part of everyday life and routines, cally was later sold to Yahoo. 7 And in the French presidential campaign in 2007, the
and it makes little sense to maintain that they have their own art form: all kinds of socialist candidate Segolene Royal's website had a feature similar to Fascinum. 8 What is
art-high or lively-now relate to and use digital technologies and platforms. What at stake for Bruno, however, is neither the alleged plagiarism nor the innovative poten-
remains, as a cardinal point for this book, is the assertion that contemporary media tial of his art, but rather the ability of his practice to address its own subsumption. In
and network technologies should be discussed as a cultural construct, and conversely his work Artwar(e) from 2010 (figure 2), he launches his own service that by means of
that contemporary culture should be explored on technical terms. This is why this a network analysis, creates artwork hype cycles as a form of "artistic risk management"
book concentrates on the relations between art and interfaces. The interface can be and "computer-assisted" curating. 9 From the perspective of networks, Fascinum, lOxlO,
seen as an intrinsic and important part of the production of a contemporary condition and Segolene Royal's website belong to the same hype cycle of "digital panopticism. 1110
where the capture and flow of data, information, and media is everywhere and part In other words, Bruno speculates in the particular networked mode of production, or
of everything. The interface as a technological and cultural construct is what brings inscription of his own work in a contemporary, global, and networked economy, which
about this contemporary world, and what makes it up to date and present at hand, real enables the transformation from "work" to "ware," and which he witnessed with Fasci-
time, global, and so forth. In every sense, the development of tablets, fast networks, num. By parodying and scrutinizing the language of the network, he further transforms
cloud computing, and more are not only technological advancements; they also seri- this into a kind of "war" (as the title suggests).
ously disturb and alter everyday cultural practices. The artistic practices that are exam- The arts that deal with interfaces are, in other words, not just innovative. They
4
ined in this book all relate to the production of this new reality. But how does this do not belong in the realm of commercial products and services that people usually
take place? associate with computer interfaces, nor do museums or libraries institutionalize them.
Historically, artistic experimentation with computers and software has often been Rather, they are part of an arts scene that receives attention from both sides, and dem-
considered to be innovative. Though they do not always invent new technologies, art- onstrate an ability to reflect the larger conditions of a new regime of production. Bor-
ists may demonstrate the technologies' aesthetic and cultural value in new ways. For rowing a term from Walter Benjamin, one might say that they express a "tendency," as
instance, in the 1990s with the advent of the World Wide Web, people experienced will be explained in chapter 1. Instead of using tendency as a reference to the banality
how text, music, and images were no longer protected cultural objects, but instead of digital production, it can be associated with certain artistic practices' ability to show
could be remixed and distributed freely in the network-rendering old technologies the networked computer's wider cultural implications on not only curating, market-
such as the landline telephone or music studio if not obsolete, then forced to undergo ing, and politics (as in the case of Bruno) but also reading, writing, working, know-
severe changes. Often one could see how the strategies applied in marketing or how the ing, and much more. Through their playfulness as well as interaction with interface
formation of new cultural products had been present in earlier examples of, say, net art. enterprises such as Apple, Google, Facebook, or Amazon, they critically demonstrate
In this way, net art, software art, and other practices relate to what has been referred to how the networked computer imposes hypes, creates value, organizes labor, distributes
as "disruptive innovation" and "creative destruction." 5 Whereas the mainframe com- cultural content, or inflicts new symbolic systems. Hence, this book seeks to bring this
puter and the Internet of the eighties were expensive and exclusive technologies, the critique to the fore, and in its conclusion, reflect on how critique can be used in the
PC and the WWW made them much cheaper and easy to access for ordinary people. design of interfaces, too.
This is common knowledge, but it is less known how artistic practices have sometimes In l>rief terms, this book is about interface aesthetics and culture, and as an analytic
led the way into new understandings and developments of this new consumer culture's strategy, it focuses on the tendency in art that reflects the contemporary interface-
expressions, preferences, productions, and other inner operations. that is, on readings of artworks such as Bruno's. In this sense, it presents contemporary
4
FR DE T ES UK IN
HYPE ANALYSIS IN ART
N:,,,2r; -Dc-c,;,2;:no
tte,..,~:nt·e:. :"ltc~n'"tbn~let
/'::,··1,-Sc,i,1 M;:;;:J<id,
Ce~tie P:>rn;,i:!:,c1,
bre;:kt'KJ.J,}11S ,1 'l1Ut.1C"rl~t;:~I
l:;,gi:
ARTISTIC RISK MANAGEMENT TRAINING N;:t.i-J~ B~~;f,"I

Cr,rj~\::,::,he 8•">->'JJ
e,..,,ae1
Hype cyd~ of dlg.ital p:.,r1optkism
G~::,,g;:,. w. 8.1s'l
Eti21,;,-:: C!,;iuct
Figure 2
Artwar(e) (2012-) by Christophe Bruno. Web page that offers to calculate and map the circulation
of value in the arts, and predict the hypes and trends of artworks and concepts. Courtesy of Chris-
tophe Bruno. Screen shot from web page.
artworks, but also reflects on the current challenges of contemporary interface culture
in a situation where the computer's interface seemingly both becomes omnipresent
and invisible, and where it at once is embedded in everyday objects and characterized
by hidden exchanges of information between objects, or what it conceptualizes as a
Figure 1 metainterface. The book, in other words, presents a new interface paradigm of cloud
Fascinum (2001-) by Christophe Bruno. Real-time display of the most viewed news images (ranked
services, smartphones, data capture, and the like, and how particular art forms seek to
from 1 to 10) on different national news feeds (Yahoo or Google). Courtesy of Christophe Bruno.
reflect and explore this new paradigm. By bringing the tendency in artworks forward,
Screen shot from web page.
the book aims to demonstrate how certain critical interfaces have an ability to reflect
the deeper fissures within new technologies and the production of the work of art
6
itself as well as show us an interface, after the interface has seemingly disappeared into creatures navigate by the light of the screen, and interact through conductive material
"smart" futures and new promises of anticipation, participation, and emancipation. in their "clicking" legs and "swiping" tails. The work is designed to "complicate the
Like Fascinum and Artwar(e), these works frequently belong to an underbelly of digital relationship between systems," and the direct interaction between the bots and tablets
culture, and their tendency is not just a banal expression of a digital realm (as seen in is in every way absurd. Their dance shows the functioning of the interface by disrupt-
Segolene Royals website or lOxlO) but rather a critical exploration of a mode of producing the data capture of the websites they visit such as Google Maps. As Andaluz writes,
tion that feeds on networked circulations of phenomena, and this production's wider "They provide wrong information for tracking website location, [and] fighting against
the design homogenization and GUI standards." 11 In fact, they do so by letting one
cultural consequences.
Another example of an artist who looks at the cultural consequences of the interface meet another. Besides conductive material, Inter_fight consists of vibrators,
metainterface is Cesar Escudero Andaluz. Andaluz is part of the Interface Culture LAB at engines, photocells, and a microprocessor controlling the robotic creatures' behavior.
Kunstuniversitat Linz in Austria. His 2015 work Inter_fight (figure 3), which was devel- In this way, they become interfaces that are added to the tablet's interface in a kind
oped at the Hangar.org visual art center in Barcelona, addresses the increased use of of closed cybernetic feedback loop. The apparent natural and invisible touch interface
touch interfaces that like Bruno's distributed and "scale-free" networks, seem to be a is revealed as a technical artifice that depends on signs and acquired gestures, and its
core characteristic of contemporary interface culture's way of producing signification. capturing of behavioral data is demonstrated as a simple technique. Attaching an artis-
Inter_fight consists of miniature robotic creatures that move around on tablets. The tic interface to the interface and turning it into a closed cybernetic loop illustrates the
artificiality of both the interaction and data capture that takes place on the Internet
and in the metainterface.
Whereas both Bruno and Andaluz work on a formal-material level of the networked
interface, and address production's inscription in networks on a coded or social level,
the tendency can also be expressed in other ways, and address many different aspects
of the material production of interfaces. As has also been pointed out by Matthew
G. Kirschenbaum, interfaces have a "forensic" materiality that refers to how data are
materially inscribed in computers in ways that cannot be deleted. 12 These layers can
be brought to the fore, or excavated, in forensic readings of electronic literature (as
Kirschenbaum does), and point to the interface's material topology (hard drives, cables,
and so forth). As an alternative to practices that highlight the histories of media tech-
nologies, and practices that tum the material architecture into a formality (that car-
ries content in new ways, as digital or new media), these archaeological and forensic
practices point to the traces and fractures that media technologies leave behind. 13 They
reflect the legacies of computing systems (file formats, protocols, etc.) as remains or
ruined monuments that may, for instance, bear witness to preferences, values, ide-
ologies, (geo)politics, and so on. Contemporary metainterfaces also have legacies or
a "prehistory," as, for example, Tung-Hui Hu argues in relation to cloud computing.
This prehistory of the cloud points to not only the legacy of former infrastructures and
spatial evidences of state or corporate power (e.g., how Cold War bunkers were turned
Figure 3
Inter_fight (2015-) by Cesar Escudero Andaluz. Robotic creatures interacting with touch screens into data centers) but also how this history is displaced. 14 As much as the cloud is a
(sweeping, clicking, typing, etc.), and providing "false" information for website tracking. Courtesy new service in contemporary computing, it is also an interface construct that pro-
of Cesar Escudero Andaluz. Photo by Cesar Escudero Andaluz. duces a distance to the material architectures of the network, a particular "clouded"
8
perception of the contemporary world. This construct is a central characteristic of the perform with them as immaterial, they clearly have material consequences: when peo-
metainterface. ple search on Google, or use other networked services like Dropbox or Facebook, they
The Barcelona-based artist Joana Moll's project C02GLE from 2015 is a good exam- pollute. As the actual emission might happen far away in a different country, people do
ple of a work that points to the material evidences of interfaces (figure 4). It specifically not necessarily sense the effects in their immediate environment. This, however, only
alludes to the geologic age of the Anthropocene, and how the use of interfaces, as a points to a shift of scale that might be difficult to comprehend, but that is no less real.
human activity, influences the climate and environment. One second after opening, To put it another way, the material workings of the Internet are often displaced, but
the web page C02GLE simply states that "Google.com emitted 510.49 kg of CO2 since the work foregrounds the networked interface's infrastructure and global implications
you opened this page," with simple black text on a white background, and it keeps on the climate. Even though these climatic consequences of the networked interface
adding 510.49 kilograms every second. 15 Google.com is the most visited website on the happen somewhere else and, so to speak, are concealed by the interface, C02GLE dem-
Internet, and according to Moll, the Internet is responsible for 2 percent of the global onstrates that they are still real and significant.
16
carbon dioxide emissions (which by comparison is equivalent to aviation). In other Parody, intervention, and excavation, along with many other artistic modes of
words, though the perception of the online as a virtual cyberreality is persistent, and exploration, function as ways to reflect the work's material levels and production, and
often people tend to think of interfaces to web services and the kinds of activities they through this, also the wider cultural consequences of the interface's new regime of pro-
duction. Bruno, Andaluz, and Moll each in their own way address contemporary inter-
face technologies and aesthetics, which are at once symbolic and material as well as
8 " hUp:/twww.janBvirgin.<:orn/C02/ undergoing severe changes. Finally, this book is not only about the art of contemporary
~ 1'N1'v,janavirgin.comfC02i
interface culture that reflects such cultural changes. It is also a design book that actively
seeks to speculate on the potential of applying a critical perspective in interface design,
and how to actively relate to and reflect the fissures of production in interface design.
The Metainterface
In the 1990s and early 2000s, when the interface to many was still considered a work-
related phenomenon (a legacy of the 1980s' introduction of interfaces into the work-
GOOGLE.COM EMITTED 53601.60 KG OF CO2 SINCE YOU place), net and software art enabled understandings of how desktops, web browsers,
OPENED THIS PAGE image editors, and so forth, were also cultural constructs. In other words, net and
software art showed how the interface did not just disappear behind office and work
metaphors along with desires of seamless interaction but resurfaced as a new milieu for
cultural practice, too. Many artworks pointed to how values and conduct were embed-
ded in technical infrastructures, and how PCs and the World Wide Web had become
not only new production and dissemination systems for existing cultural objects (text,
images, movies, etc.) but also the conveyor of new cultural practices that included shar-
ing, remix, and much more. Fascinum, and the ways it presents visual culture, is a good
Figure 4 illustration of this. Other works deconstructed the usability of image editors (such as
C02GLE (2015-) by Joana Moll. A real-time, net-based installation that displays the amount Adrian Ward's Auto-Illustrator [2000-2002]), or pointed to new commons-based cultural
of CO 2 emitted each second by Google.com. Courtesy of Joana Moll. Screen shot from project production that challenged existing notions of origin and copyright (such as Cornelia
website. Sollfrank's Net.art Generator [1997-]).
10 Introduction Introduction
11
In many ways, the interfaces of today are much more embedded in the kinds of cul- Second, the industry around the metainterface presents this as a new reality of
tural practices that were highlighted in net and software art. To put it simply, though
the desktop and PC still exist, the interface has moved from the office into culture.
The new platforms for this culture are the ever-present media devices and apps, and
displaced networks of clouds and data streams. It is this shift, and how the interface dis-
I smooth access and smart interaction. Yet its protocols also hold within them recon-
figurations of the everyday production, distribution, and consumption of culture as
well as the reorganizations and renegotiations of urban space, and profound changes
in the relation to material infrastructures that also affect sense perception itself-that
appears, not into seamless work-related activities, but into the environment and every- is, the perception of a globalized and real-time world.
day cultural practices, that is the topic of this book, conceptualized as a metainterface. Third, the notion therefore points to metainterfacial artworks along with ways
Whereas the work-related interface was designed to disappear as windows into virtual of analyzing them as critical explorations of their own material conditions that not
and immaterial realities and workspaces, today's cultural interfaces disappear by blend- only reflect the metainterface industry's corporate production of new realities but also
ing immaculately into the environment via what is commonly referred to as mobile depicts alternative ways of constructing and designing the metainterface. One may, for
computing, ubiquitous computing, and the Internet of things, among other terms. instance, experience how the interface resurfaces as absurd predictions, as in the case in
The same way seamless immersion is the cultural promise of the screen-based interface Artwar(e); the surreal use of maps and exchanges of data, as in Inter_fight; or displaced
technologies, social promises such as the networked, participatory, or open (which material infrastructures as in C02GLE. The works, each in their own way, point to the
were once solemnly associated with the underbelly of net culture) have now become reality that metainterfaces produce, and how they reconfigure a range of domains from
intrinsic to the development of interface technologies. The promise to deliver interac- cultural production to urban space as well as the perception of a globalized world.
tion spaces where apps, smart objects, and mobile computers blend flawlessly into the Such explorations may potentially lead the way to the design of new interface para-
environment is a promise to deliver a smart, networked, participatory, and open future. digms, too.
The central claim of this book, however, is that despite the attempts to make the In other words, the artworks at the core of this book reflect the fissures and crises
interface disappear, and conceal it behind a mask of smoothness and real-time infor- of production of an interface that moves beyond the networked PC, and they each
mation flows that all seem to be for the social, individual, or functional good, it gradu- highlight three different kinds of material symbolism of this new interface. Besides a
ally resurfaces. 17 Although the interface may seem to evade perception, and become theoretical introduction, and a concluding reflection on how to apply this in critical
global (everywhere) and generalized (in everything), it still holds a textuality: there still interface design, the book's chapters address these three material tendencies, including
is a metainterface to the displaced interface. Artistic practices may help us to see this by a semantic capitalization of language (as seen in the apps and devices of a new interface-
reflecting the fissures in the kinds of realities that the metainterface produces. That is, based culture industry, and illustrated by Bruno's focus on the networked character of
they may reflect their own production, and in this way, challenge the disappearance of cultural production and consumption), semantic territorial control (as seen in urban
the interface, and make it much less innocent than corporate rhetoric otherwise sug- interfaces, and shown by Andaluz's emphasis on the inscription of space), and a vir-
gests: the ubiquity, pervasiveness, and real-time smoothness of the interface are loaded tualization of the material architectures of the network that has deeper effects on the
with worldviews, values, ideologies, politics, regulation, and conflicts, and thus also experience of reality (as seen in the cloud interface, and characterized by Moll's focus
hold within them potential new forms of expressiveness. The concept of metainterface on the Anthropocene). Though these tendencies seem dispersed (and the chapters are
consequently performs three different, but related, functions. also readable independent of each other), this book argues that they are related at the
First, it is the description of a contemporary interface paradigm. One may argue level of interface aesthetics: they are related at the level of the language and grammar
that the layered structure of the computer (from assembly language to graphical user of a new material textuality. The chapters will progressively explain how this textual-
interface) makes any interface a metainterface, but what is specific about the current ity functions as a formal inscription of reading and writing (in cultural devices and
interface paradigm is the universally dispersed, omnipresent nature of this. As such, it platforms) that unfolds as a spatial and territorial inscription (in cities) that assumes
is an interface to the many hidden interfaces and clouded exchanges of data and sig- immense and incomprehensible material scale with consequences for sense perception
nals in a series of platforms that connects the everyday use of apps on a smartphone to (in the cloud). The final chapter will discuss how this critical understanding of the
large-scale, globally networked infrastructures. metainterface's material textuality is not only a way to discuss art and interfaces but
12 Introduction Introduction
13
also a way to conceptualize and design, beyond a simple belief in the disappearance of ongoing process of territorialization in the urban. Naturally, changes in the linguistic
the interface. apparatus, as those appearing with the metainterface and its language-based indus-
The first chapter describes the theoretical framework of the book. This includes an try (also known as semio-capitalism), will also affect this process. The chapter thus
elaboration of the book's key notions, notably the interface itself, and an explanation explores how the metainterface industry disrupts the everyday practices of urban life,
of why it is an important beginning for an aesthetic critique; an elaboration of "ten- and how the city is read, perceived, inhabited, and organized in new ways. Immediate
dency" as a key concept that address material production and textuality, and enables examples of this are phenomena like Airbnb, Uber, and TripAdvisor, which all point to
new understandings of artistic practice with interfaces; and finally, an analysis of how how anything potentially can signify anything: an apartment can be a hotel, a pass-
the interface is changing and becoming a metainterface. From being a relatively self- ing car can be a taxi, and so forth. To read and use the city, one only needs the right
contained entity, it is now mobile, ubiquitous, and deeply entangled with massive data app. The chapter will discuss how the city is opened up by the metainterface industry,
capture and techniques that make the interface appear intelligent. Artistic practices but also how this implies a new organization of the city, and how the mechanisms of
can be regarded as an occupation that reflects and reflects on this tendency, and more semiotization (which determines how the city should be read, interpreted, and used)
specifically, how the interface reappears as a metainterface, with a material textuality become increasingly opaque as well as layered with an algorithmic grammar and way
and particular grammar that reflects linguistic, territorial, and experiential crises in of seeing. The process of opening up the urban sphere with the interface hence also
production. The following chapters will analyze these changed material relations and implies a new mode of control where alternatives to "the open" are needed.
crises in the production of the artwork, and relate them to a wider cultural critique of The fourth chapter addresses the clouds of the contemporary metainterface, and
interface culture. how cloud computing influences the experience of a world where the interface is in
The second chapter addresses the new platforms of a cultural metainterface industry everything (generalized) and everywhere (global). In other words, how the interface
(the smartphone, e-reader, etc.). These platforms, which are central to the metainter- becomes a metainterface, or how the interface seemingly disappears but resurfaces as
face, are built on a capitalization of net culture. As much as this opens up new pos- a metainterface with its own textuality and grammar. If the desktop was the metaphor
sibilities for cultural production, it is also experienced as a crisis of black boxing and of interfaces in the 1990s, the cloud is emblematic of the cultural interfaces of today. 18
tracking. More specifically, the metainterface industry is based on a "semio-capitalism" Cloud computing typically covers services that offer data storage (including e-mails,
that involves a formal inscription of not only production (e-books, streaming of mov- photo libraries, text documents, and much more), and also the power to process data,
ies and music, etc.) on the platforms but also consumption. What people look at, read, which are left "out there," "in the cloud." What were once network architectures and
listen to, and so on, becomes data that can be used to anticipate user behavior. In other material inscriptions that the users who built the World Wide Web cared for as well
words, the loss of access to the media technologies that characterizes the metainterface as cherished, have now become standardized and displaced out of human reach. The
is a loss of ownership and privacy. This is of political importance, but it also relates main objective of this chapter is to explore how this material displacement influences
to what writing and reading is and becomes, as a partly nonhuman activity. The the experience of the interface as an alienating smoothness and phantasmagoria, and
metainterface industry transforms reading and writing: consumption, which tradition- how this experience of the metainterface affects sense perception at a more general
ally has been considered passive, becomes a productive inscription of behavioral data, level, too. With the war on terror and climate change as examples, the chapter thus
and the production of culture becomes a kind of consumption. The interface itself is examines how the metainterface becomes emblematic of the experience, knowledge,
no longer just a consumer product that can be bought or sold in a traditional sense, and actions of a global, synchronous, and real-time world.
but is a networked streaming service that is financed by the capturing and inscription The final chapter concludes by discussing how to extend an interface criticism into
of user behavior. a new tradition of design, or how to design with an awareness of the language and
The third chapter examines the urban metainterface. Apart from houses, facto- grammar of interfaces. Although the metainterface is intrinsic to new cultural indus-
ries, offices, streets, electricity networks, sewage, and so on, a city can be defined as a tries, a new de- and reterritorialization of the urban, and new experiences of realities,
"semiotization" of space. Media and language-including street signs, advertisements, there is no reason to take it for granted. As the examples in this book hopefully will
and more-also inscribe space. In other words, language is intrinsically related to an demonstrate, seeing and writing with the metainterface can allow for experiences of
Introduction
14
surrealisms, beauty, anxieties, laughter, and much more. In a contemporary technology 1 Interface Criticism: Why a Theory of the Interface?
critique, it is well worth knowing how the smoothness, phantasmagorias, and prophe-
cies of openness and participation function as interface constructions that combine
semantic processes of signification with machinic processes of signals. Nevertheless,
giving way for such freedom is not only an analytic insight; it can also fuel new tech-
nological imaginaries, and may function as a catalyst for a design paradigm. In other
words, the subtitle of this book (The Art of Platforms, Cities, and Clouds) points not
only to how computing today makes way for new artistic expressions but also to how
computing and interface design itself may be practiced, "tactically" as "not-just-art" (as
Matthew Fuller once characterized software art). 19 The final chapter thus presents two
case studies and discusses this interface criticism by design in relation to a larger his- Any understanding of the metainterface's cultural and aesthetic significance must
tory of critical design, particularly the Scandinavian political tradition of participatory begin with a clarification of what an interface is, and why it is an important notion:
codesign and Anglo-Saxon tradition of critical design. why the interface is inevitable for the networked computer, and why there always will
The selection of the cultural metainterface industry, urban interface, cloud interface, be an interface-even in a situation where it has become displaced (seamless), global
and critical design of interfaces as the main foci in the book by no means expresses (everywhere), and generalized (in everything). Interfaces and their aesthetics have been
an intention to exhaust an interface criticism. Naturally, many other areas could have the object of our studies for more than a decade. We, as authors of this book, and
been included as they too are affected by the metainterface (e.g., the body, logistical like many others in the late 1990s and early 2000s, initially set out with the intent to
infrastructures, networked activism, etc.). By presenting a perspective, analytic strategy, research the relationship between literature and computers. With its algorithmic text
and practical tactics, however, this book will hopefully open up the field of interface processing, and complex relations between code and text, the computer in many ways
aesthetics and criticism, rather than restrain it to certain phenomena. As much as it seemed to offer new perspectives on, for instance, textuality, and also instigated new
seeks to cover important domains and tendencies that interest its authors, it hopefully forms of text, such as electronic literature and computer games with epic narrative
outlines a methodology for a contemporary cultural critique and design of interface elements. Neither electronic literature nor computer games compare to printed books,
culture in a time where interfaces are ever more present, yet ever more unintelligible. however, and conventional literary textual analysis does not apply easily. There was a
new materiality to the textual that called for new theoretical approaches.
This discrepancy between textual organization and its analytic apparatus was also
coined by Espen Aarseth in his groundbreaking work Cybertext: Perspectives on Ergodic
Literature from 1997. 1 According to Aarseth, the new formats for electronic literature
that were found in hypertext fictions or Multi-User Dungeons were not manifestations
of a postmodern notion of the text as a rhizome, as much poststructuralist hypertext
theory argued. 2 For instance, the user of a cybertext was a "more integrated figure than
even reader-response theorists would claim. 113 The kinds of interaction that cybertexts
offered were different from the traditional conventions of literature and the printed
book. Whereas the flipping of pages in a book is trivial next to the interpretative inter-
action when reading a book, it can almost be the other way around when interact-
ing with hypertext and cybertext. In cybertexts, the interaction became "ergodic" as a
way of working out one's path through the text, and the cybertext literally becomes a
"machine for the production of variety of expression." 4 As the term "cyber" indicates
16 Chapter 1 Interface Criticism 17
(originating in the Greek Kybemetes, which means steersman), navigation is in many production, they have the potential to reflect how interfaces and networked computers
cases crucial. For example, to many computer gamers the narrative is only interesting change culture in a larger perspective, and the purpose of this book is to discuss these
insofar as it explains what to do next. As Aarseth pointed out, such text forms do have aspects of the interface and interface culture. In other words, this chapter explains
a history and tradition, as seen, say, in the writings of the French literary movement why and how artistic practices express this tendency. Furthermore, it includes a reflec-
Oulipo (Ouvroir de litterature potentielle, or workshop of potential literature), which tion on how the interface is changing from being relatively contained in software and
used conceptual or material constraints in its writing and produced cybertexts such as hardware, to being mobile, limitless, and almost reckless and dissolute in the ways it
Raymond Queneau's Cent mille milliards de poemes (Hundred thousand billion poems), incorporates a constant flow of data from other interfaces. This change fundamentally
but this understanding of text and its history had usually been marginalized. Both presents a new mode of production that is, as will be presented in the following chap-
electronic literature and computer games were, in other words, not just new media for ters, setting the stage for the tendency in contemporary art that deals with interfaces as
narratives but also presented a type of text that was characterized by a schism between well as networked computers and platforms.
doing and reading, or instrument and media. These texts offered literary expressions
that were constantly interrupted by the presence of ergodic action. This influence of Theories of the Interface
the organization of the text had rarely been an issue in literary analysis and textual
theory, but Aarseth made it clear why cybertexts and ergodic literature called for new A theory of the interface is of course not a new thing, and the textual assemblage of
theories. representation and organization that Aarseth also described has received attention in
Rather than inventing new names for this particular type of text and its artistic several different ways throughout the past decades. In their seminal book Remediation
genres, one may choose to stick to the colloquial term interface that most people are from 1999, Jay David Bolter and Richard Grusin depicted the assemblage as a percep-
familiar with, and refer to the genres and art forms by the names that have most often tual friction between the logics of "hypermediacy" and "transparency," or "looking at"
been used, such as computer games, electronic literature, net art, and software art. and "looking through," which have origins stretching back to the Renaissance and the
All these arts are interface based, and frequently also follow parallel historical devel- invention of a linear perspective in Western visual culture. 5 A few years later, in 2001,
opments of exploration of the nonlinear or computer-generated expression. They Lev Manovich in The Language of New Media explained how the language of cultural
take advantage of multimedia, too, and blend into one another to the point where it interfaces draws on traditions of media such as movies and written texts as well as
becomes difficult and sometimes useless to sustain the borders between the fields of instruments and interfaces to machines. 6 In a wider perspective, all software interfaces
literature, art, and music: images include texts, poetry is performed as readings with are dealing with the kinds of composition that Aarseth, Bolter and Grusin, and Manov-
music, and so forth. In addition to existing aesthetic theories, there was, as Aarseth ich have described in their own ways: software interfaces generally convey meaning
noted, and still is, a need for conceptual and material understandings of these artistic and action, representation and computation, media and instrument, and so forth. An
interfaces along with their formal expressions, histories, and interplay with broader icon on a desktop is, for instance, at once an image and an access to an operation in
cultural characteristics. This book is a contribution to this field, and provides an insight the computer.
into the arts of interfaces, and how they relate to contemporary developments in an The various theories of interfaces from the late 1990s and first decade of the new
information technology that by all means is a technology that both builds on and millennium all reflect how media theory adopts concepts from computer science
transgresses contemporary aesthetics and culture. (such as the interface) as central categories, and how media theory becomes software
This chapter sets the theoretical frame of this book and delves into what interfaces studies. This development was coined in a US tradition by Manovich, and in Europe,
are as cultural-technological constructs. It also explains how they can be critiqued, predominantly by an arts scene. The most prominent examples of this scene were per-
not just for being good or bad, useful or useless, right or wrong, beautiful or ugly, but haps witnessed at the Readme software art festivals organized by Alexei Shulgin and
for their own literary value (as Aarseth recommends). Beyond the moral of what they Olga Goriunova together with a wider international group in Moscow (2002), Hel-
convey, their ease of use, and the logic of their operations, they have an aesthetics that sinki (2003), Aarhus (2004), and Dortmund (2005), but were also an important part
calls for a new literary interface critique, too: through reflection of their own means of of the Transmediale festival in Berlin under the leadership of Andreas Broeckmann
(2001-2007). Several important texts and books were published at the time, such as the contextual, and technical materiality of the algorithmic text. In this sense, the theori-
2008 Software Studies: A Lexicon, which was edited by Matthew Fuller. 7 This tradition zation of the interface presented in this book differs from poststructuralist semiology
also laid the foundation for our attempt to coin an interface criticism in our 2011 book, and semiotics' conception of a generalized textuality of signs without materiality. 13
Interface Criticism: Aesthetics beyond Buttons. 8 Instead, interfaces are discussed as a new form of textuality with other grammars and
In a time of smartphones, smart cities, and many other phenomena, the interface inscriptions than traditional text. As the practice-oriented research professor for new
has seemingly become ubiquitous and networked in the sense that it is increasingly communication technologies Florian Cramer has pointed out, computers can be seen
everywhere and limitlessly connected. This outlines the continued necessity of devel- as textual machines whose material properties and contexts differ from other texts and
oping an understanding of the interface, and recently there have been several efforts textual machines in that they run on "alphabetic code." 14 Following on from Cramer,
to do this. For instance, there is Alexander R. Galloway's political perspective on the it should be discussed how the interface has changed the material, concept, and role
processes of interfaces, Lori Emerson's literary perspective on how writers address the of text, rather than whether interfaces fit within existing notions of text and literature.
interfaces of reading and writing from the book and typewriter to the tablet, and Bran- The interface thus becomes a text in an analytic strategy, but a text that is material,
den Hookway's philosophical perspective on the interface as a relationship with tech- technological, and part of more extensive political and social contexts than what exist-
nology. 9 The interface is also conceptualized in different ways-notably as "stacks" ing notions of text traditionally point at.
by Benjamin Bratton, signifying the layered structure of an interface that is at once Without sacrificing notions of representation and textuality, the understanding of
technical, cultural, and part of a new political reality. 10 In addition to such efforts, this text applied in this book is fundamentally to be understood in its relation to the mate-
book focuses on the interface as a material and technical format that juxtaposes the rial processes of the machine. The approach is related to current theoretical interests
operational with the representational, and thus is deeply entangled with the cultural in materialism such as new materialism as well as post- and nonhuman theories, but
and aesthetic domain. Wendy Chun's account of metaphors, direct manipulation, ide- this book insists on stressing how representational and material dimensions are inter-
ologies, and real-time adaptation in interfaces also seems particularly useful in this twined in interfaces. A critical interface theory relies on the understanding of discrete
context. 11 This juxtaposition of instrument and media, and how it is played out in ideological and political mechanisms that are often hidden within the implicit rep-
artistic practice, is core to the understanding of interfaces that are applied in this book. resentations and grammars of the interface. As argued by David Berry, and also oth-
But it also expands the notion of interfaces into an understanding of current develop- ers, contemporary materialism such as speculative realism or object-oriented ontology
ments that are exemplified by the phenomena of platforms such as the e-reader and frequently tend to disregard such dimensions. 15 In contrast, the materialism that this
smartphone-interfaces to the urban and cloud computing, where interfaces become book seeks to assert is inherently dialectical, and explores the relations between mate-
networked in ways that are sometimes hard to comprehend. rial and mediation, including the ideological, political, and cultural dimensions of this
This book goes beyond its literary starting point, and includes the different art forms relation. 16
and studies of software that relate to the interface; still, it is the literary perspective that Critical analyses of technology, including analyzing technology's cultural, ideologi-
guides its interests. Though text is fundamentally changing when it becomes compu- cal, and social dimensions, are important parts of this book's approach. Accordingly, it
tational, there are still textual structures such as inscriptions, grammars, scripts, and shares many interests with science and technology studies. Although the French soci-
codes, and it is therefore relevant to draw on not only the literary theories of new ologist and philosopher Bruno Latour has been criticized for pluralizing materialism to
textual understandings (such as Aarseth, N. Katherine Hayles, and Manuel Portela) but a level of abstraction, his discussions of how phenomena become "Things" or "mat-
also the more classical materialist literary theories of Walter Benjamin and others. 12 Like ters of concern," as presented in his collaboration with the German artist and curator
these theoreticians, we, as authors, do not confine ourselves to the field of literature Peter Weibel at ZKM / Zentrum fur Kunst und Medien (Center for Art and Media) in
but instead see literature as part of a broader field of artistic and material production Karlsruhe, are good examples of this. 17 The book is without question also related to
that includes artistic practices with networked computers. In a theoretical discussion of Donna Haraway's political, feminist voice in science and technology studies, especially
digital media text forms, the challenge is hence to transgress the limited understanding her suggestion of a mediated perspective that includes the technological body as a
of text as a formal system of representation, and pay attention to the particular formal, "material-semiotic actor" and avoids reducing technology to a mere material object of
observation. 18 Haraway's material-semiotic actor relates to the observer (as a techno- at all levels it is saturated by choices, conduct, language, values, and worldviews, and
material cyborg), but also to the media, instruments, and technologies through which consequently, also an aesthetics that reflect these saturations critically.
we observe, and in this sense it corresponds closely with interface criticism, which like Though a symbolic machine, however, the computer processes signs in a far differ-
Haraway's feminist objectivity is a material and situated knowledge. ent way than a human interpreter. Frieder Nake, who is not only known for being a
Taking as its starting point a literary approach to the material semiotics of interfaces, pioneer in computer art but also a computer semiotician, has explained the difference
the theoretical outline necessarily begins with theorizing the networked computer's like this: "Whenever we program a computer, what we are really doing is mechanizing
fundamental ability to combine processes of signification with the signal processes. mental labor. In order to mechanize a specific mental labor, it must be generalized in
Although the book's literary materialist approach also differs from computer semiotics' the broadest sense possible, it must be formalized, and finally turned into computable
understanding of interfaces, there is still something to be gained from the field and form .... The mental labor remains with the human as their genuine capacity.... But
its understanding of how the interface combines representational and instrumental besides this human capacity, we encounter its algorithmic form. 1120 This distinction
dimensions. between mental labor and its formalization is crucial. All programing of software is
always an expression of a process that reflects a formalized relation to labor-that is,
Computer Semiotics and Interface Design a translation of mental labor into a computer program. I, the human being, can inter-
pret the world as signs. The computer, on the other hand, is stupid and only capable
In the theories of human-computer interaction (HCI), the juxtaposing of instrument of determining, not understanding. Everything must be computable, and hence the
and media has historically been considered a fundamental challenge for interface computer can only interpret things at this basic level.
design. To overcome the challenge, interface design has often aimed at concealing not Following this line of thinking, the computer is conceptual by nature. Any com-
only the inner processes of interfaces (e.g., the running code and its command lines) puter program is a manifestation of a concept: I "think" searching, sorting, listing, and
that makes the interfaces inaccessible to ordinary users but also its hidden processes so on, and I make it into a machine that can do it indefinitely. Labor therefore has both
of labor. a mental form (meaning a tacit knowledge of the process of, for instance, searching,
Computers are not ordinary machines; they are symbolic ones, as computer semio- sorting, or creating) and an algorithmic form (searching, sorting's symbolic representa-
tician Peter B0gh Andersen claimed in the late 1980s: they are constructed and con- tion, expressed in formal instructions of, say, sequencing, iterating, and selecting, as
trolled by means of signs, or expressed differently, they are constructed by interfaces. they are often applied in programming).
The graphical user interface is an obvious case of how machine signals are controlled At the level of the graphical user interface, the formalization of labor gets doubled
by means of signs, but underneath the graphical interface there are other interfaces. and is represented to the user as signs that can be used to manipulate mental labor. For
There is, for instance, a program text that contains signs that stand for possible pro- example, an interface for searching a file in an archive is based on both the formaliza-
gram executions. The executions also involve a compiler, or interpreter. A compiler is a tion of the mental labor process of searching in the form of algorithms and a repre-
program that transforms source code into computer language, or object code, that can sentation of this to the user in the form of a text field along with an icon displaying a
run on the computers central processing unit. In other words, it enables the execution looking glass.
of the source code by means of text. As Andersen explains, "Everything in a computer This sort of interface design (understood as the successful mapping of the formal-
system, from top to bottom, is used as signs by some group of professionals. At each ized labor process to the user's world of signs) will in many cases cause a forgetting of
level, there are texts that must be interpreted a statements or prescriptions about some the interpretation of reality involved in the programmers' formalization of the world
present or future state of the system." 19 To put it briefly, a basic assumption in this book as well as the computing and signal processing that takes place within the computer
is that there is no computer without an interface. The interface can be the well-known when executing a search. Even the simplest act "sets into motion huge piles of frozen
graphical user interface between user and computer, but it exists at other levels, too: mental labor that others-system and software designers and programmers have done.
between different programs, and even between hardware components. Yet there is no ... The user can hardly escape the impression it is the computer acting all by itself, in
privileged interface either: at all levels, the interface combines signs with signals, and particular producing signs," Nake writes. 21
In other words, when users search using Google or other search interfaces, they acti- what fascinates the user, and is also seen today in, for instance, the marketing of the
vate labor processes. Rather than systematically looking through and organizing the 3D virtual reality headset Oculus Rift or other similar inventions.
archive, they (in order to save time) rely on a predefined formalization, and a machine Much digital aesthetic theory (including that of Aarseth), software studies, and
that can interpret this language and determine an outcome. In this process, they often interface aesthetics can be seen in opposition to this. 26 Rather than developing tech-
tend to forget the conceptual and performative elements that are involved in actions, niques to overcome the interface, this book regards aesthetics as a reflection of the
and that do not strictly speaking make it the user's search; it is rather the software interface's juxtaposition of sign and signal, or media and instrument. For instance, in
and its technical infrastructures that search. Google's search algorithms are in fact also electronic literature the assemblage becomes the center of attention, in the sense that
renowned for their obscurity and continual updates, and likewise most people know it is a literary form that often experiments with what kinds of actions and mediations
little of their servers and the network that brings the results to them. there potentially could be. In this way, you find literary expressions that reflect the
organization of the text, and textual organizations that become important parts of the
Interface Aesthetics expression. One example of this (which will be addressed in the book's final chapter)
is the Poetry Machine, which investigates how readers become involved in the ergodic
Following on from computer semiotics and the field of HCI, interface aesthetics is fre- cybertext production, and thus gain a new role and responsibility for what Aarseth
quently reduced to a pragmatic purpose, or an unreflexive realism, as a means to create has labeled "the scripton," meaning the strings of text that appear to the reader, while
correspondence between the mental, formal, and operational labor processes. Nake writers become producers of a "textonic" landscape, meaning the strings of text as they
expresses it as a situation where "signs representing things and signs representing oper- exist in the text. 27 Another example (also used in the second chapter) is The Project
ations merge." 22 In other words, it is a form of interaction that strives toward an ideal Formerly Known as Kindle Forkbomb by Ubermorgen (2011-2013); it investigates the
mapping of the domains that can transgress the arbitrariness of the sign. Though often networked production of text along with publishing models related to YouTube and
building on a deep understanding of the users' perception of the world as expressed Amazon. 28
in their language, this design approach also uses aesthetics to support the mapping Electronic literature's reflection on relations between reading and writing, and tex-
and generate seamless interaction. 23 Andersen has, for instance, studied how aesthetic tural organization, compares well to Bolter and Grusin's description of how visual cul-
properties from movies, such as cuts, scenes, or sequences, can be used to design pro- ture always has had art forms that reflect the particular distribution of the logics of
cess control in a user interface-among other things, by creating comprehensible transparency and hypermediacy in a given time. In other words, only in a limited
relations between sound signals and events, or creating thematic structures in the understanding does aesthetics refer to the style and beauty of interfaces. In a criti-
interaction. 24 cal perspective, it concerns the ways in which the interface reflects new perspectives
The dream of seamless interaction, embedded in the user's reality, is historically a as well as new ways of perceiving, organizing, and thinking brought about by media
driving force of much HCI design. Although both Nake and most interaction designers technological changes. Fundamentally, as Nake has explained, all these visual, textual,
are well aware that seamlessness is an illusion, it is a logic of transparency (to use Bolter sensual, organizational, and other strategies depend on a particular relation to labor.
and Grusin's term) that persists in images of "Man-Computer Symbiosis" (as expressed An aesthetic critique of the interface, and aesthetic interface theory, could begin by
by Joseph Licklider in his visions of computer systems), "synthetic telepathy" (such as understanding this relation to labor as a reading of the artwork's tendency.
Edmond Dewans's experiments with mind control in order to send Morse code), and
"ubiquitous" or "calm" computing (where information technology leaves the graphical Tendency within the Artwork
user interface to become part of a tangible environment, as a silent hum in the back-
ground, as articulated by Mark Weiser). 25 Indeed, there are endless high-tech myths The aesthetics of artistic practice considered in this book is neither concerned with
that address the disappearance of the interface. In other words, interface design often the beautiful, trendy, or engaging interactive experience, nor a modernistic media
has an embedded prophesy of pure, unnoisy, and seamless signal processes where the specificity (as in Clement Greenberg's understanding of art's autonomy and division
problems of representation are overcome in various ways. This has frequently been into media specific subgenres like oil on canvas, bronze, etc.). Instead, it references a
L
criticism that appears through the material explorations of technology that are found "What is its position in them?" This question directly concerns the function the work has within
the literary relations of production of its time. It is concerned, in other words, directly with the
in the arts practices that deal with interfaces, and how they critically reflect the ways
literary technique of works. 30
they themselves are challenged by the interface and its operations. Or put inversely,
if the interface challenges the existing understanding of art, design, and culture, it is Benjamin's highlight of the author's concern for the technique, as opposed to the
because it includes the material and technology as carriers of meaning in a more gen- attitude of the writing, also makes way for a different understanding of tendency than
eral perspective. When the artistic practices induce critical reflections on their way of the one found in contemporary autonomist Marxist thinking (e.g., Franco "Bifo" Berardi
being art and their material basis, they address even deeper political questions of how and Antonio Negri). As noted by Benjamin Noys (in a critique of Berardi), tendency
the interface challenges conceptions of society, politics, and culture. Subsequently, an in both autonomous and classical Marxism (e.g., Georg Lukacs) points to the "demise
interface aesthetics and critique is intimately linked with how the artistic interface of capitalism under the pressure 'of the potency of productive forces. Instead of this
111
administers its own technology, and it strives to illuminate how users, more gener- "apocalyptic" and "eschatological" understanding of tendency, Noys argues (with Fred-
ally through computers, manage the world and produce the reality they are part of. ric Jameson) for a "more nuanced realism about the contemporary conjuncture" where
As such, interface aesthetics expresses what the German Jewish critical theorist Walter "the aim of critical intellectuals should be to present or represent the contradiction of
Benjamin also refers to as a tendency (or Tendenz in German) in his seminal essay "The the time, even to sharpen them ... by practicing a method of the tendency that more
Author as Producer" from 1934. closely aligns base and superstructure in our analyses, that permits a closer grasp of the
Tendency is typically associated with a somewhat-banal expression of the common- failures, tensions and contradictions of this order." 31 Although Noys does not mention
place and contemporary, but a tendency can, according to Benjamin, also have a differ- Benjamin, such "ways of practicing a method of the tendency" seem aligned with his
ent and deeper meaning related to a material understanding of technology and media. call for a preoccupation with a literary technique as a way of not relating to material
Already in 1927, he writes in a short text that media revolutions lead to fissures in art and political conditions (good, bad, right, wrong, etc.) but instead positioning oneself
where deeper tendencies are exposed, and that the art that investigates this-and in dialectically within them. Without question, "The Author as Producer" is one of the
this way, uses its medium and technology critically-is the art that contains a deeper texts that most clearly presents Marxist visions (not to mention given the context in
political tendency: "But just as deeper rock strata emerge only where the rock is fis- which it was delivered), and it also aims to unravel the eschatological dogmatism of the
sured, the deep formation of 'political tendency' likewise reveals itself only in the fis- Marxist tendency toward an analytic, dialectic material exploration, which is neverthe-
sures of art history (and works of art). The technical revolutions are the fracture points less still political.
of artistic development; it is there that the different political tendencies may be said In other words, the essay expresses Benjamin's materialistic and technocritical way
to come to the surface." 29 Art is in Benjamin's view a probe to the fracture points cre- of thinking about the relation between art, technology, and politics with a focus on
ated by technical revolutions. This is not because artists have a better knowledge of literature, and he shows how literature, through its political and literary tendency,
politics than others, but because artistic production is a material exploration of its own can demonstrate general social and technical relations concerning the conditions of
technological means of production, and how these constantly change. Consequently, production. Its readers should in this way understand the essay's proposal of seeing
tendency is related to a dialectic material examination of production and technology the author as producer-meaning both a producer of their own text, the apparatus it is
through artistic production, rather than an abstract ideology or immediate attitude produced and published through, and a producer of writing in general. To Benjamin,
of the work, as Benjamin detects in both fascist and Marxist thinking (the essay was the author is more than just a writer: "An author who teaches writers nothing teaches
given as a speech in a specific Communist context: the Paris Institute for the Study of no one," he states in his argument for a technocritical dimension to writing. 32 To be an
Fascism): author means to recognize the opportunity that through their work and experiences
with changes in a work's own relations of production (i.e., the literary technique), the
Instead of asking, "What is the attitude of a work to the relations of production of its time? Does
autho~ can reflect on how production is related to ideology, politics, morality, soci-
it accept them, is it reactionary? Or does it aim at overthrowing them, is it revolutionary?"-in-
stead of this question, or at any rate before it, I would like to propose another. Rather than asking, etal organization, and so on. Literature-and more generally, art-is therefore not just
"What is the attitude of a work to the relations of production of its time?" I would like to ask, something elevated and spiritual, above the level of the technological and productive,
but a kind of technology in itself, too, which is engaged in and part of society. If the The point here is that aesthetics has a further role than simply providing nice design
author does not recognize this challenge, they will merely be working for the powers that involves users or facilitates innovative production. The need to engage critically
that be. Benjamin also cites Bertolt Brecht's critique of the cultural producers who do with the kinds of formalizations of labor that the interface implies is related to broader
not understand the apparatus they themselves are part of. Rather than controlling the political issues. The interface conceals labor and production processes, including its
apparatus of production, they are naively controlled by it. Consequently, art is not only users' own production of data. Aesthetics is hence a question of exploring the fissures
based on a processing of the individual's own existential experiences but also on a con- and deeper tendencies of the production conditions, and how the artistic production
current desire to engage oneself in-and even change-the institutions and apparatus with interfaces continuously reflects the inadvertent result of the interface's modes of
of production: the artist needs to act as an engineer. As Benjamin notes, there is in this operation (the combination of signs with signals) along with what lies behind, within,
a decisive difference between merely supplying a productive apparatus and seeking its ahead of, and beyond them. Informed by both computer semiotics and interface aes-
transformation: "The proposition that to supply a productive apparatus without-to thetics, this book's aim is to present a dialectic materialism that can be used in con-
the utmost extent possible-changing it would still be a highly censurable course, even temporary studies of software and interfaces, and how they produce realities that are
if the material with which it is supplied seemed to be of a revolutionary nature." 33 In entangled with the everyday; the analysis of software's and interfaces' art forms as well
its broadest meaning, this productive apparatus encloses not only the author but the as the kinds of wonderful technological imaginaries they may produce; and the inno-
reader, too, and the art itself. vative and transformative design of the productive apparatus of the interface in a time
Even though "The Author as Producer" primarily refers to authors, Benjamin was of metainterfaces.
widely interested in the new media of his time, such as film and photography, and how A contemporary example of an artistic approach relevant to Benjamin's thinking
they could relate to political realities and the potential for change. Similarly, this book is "The Critical Engineering Manifesto" of the Critical Engineering Working Group
is attempting to cover these underlying tendencies in the arts of interfaces. 34 Techno- (2011-2015). This manifesto argues for a critical engineer who "looks beyond the 'awe
logical revolutions-such as the changes seen during the last five decades with the of implementation' to determine methods of influence and their specific effects," and
appearance of computers in the military, PCs, supercomputers, tablets, smartphones, "raises awareness that with each technological advance our techno-political literacy is
Internet of things, and so on-lead in all certainty to fissures within art, and these challenged." 36 Echoing Benjamin and the materialist approach present in this book,
fissures reflect broader issues in society. Consider, for instance, how the networked engineering is considered a form of writing or language that shapes its users, and
distribution of art, music, and literature has challenged the old culture industry as well should be studied and exploited in ways that expose its influence. The people behind
as the traditional understandings of property, copyright, ownership, and privacy, and the Critical Engineering Working Group are also part of the Weise7 studio in Berlin,
how this has led to a new digital culture industry based on streaming and licenses. and their works are good illustrations of critical engineering. In 2012, they created the
These new models not only for production but also morals, laws, politics, and much Weisel book, which unlike other books, is a wireless independent server that runs from
more fundamentally function in an interplay with media technological developments. a computer inside the book. By opening the book, "readers" can join the book's wireless
Moreover, the culture industry is only the precursor of what is happening to other network and browse its contents (which includes the images and other material from
industries where social media not only prescribes social relations but also becomes a Weise7's archive). 37 In 2014, they further developed the concept and created a black
platform for marketing, distribution, and a whole economy of sharing (such as Uber or book of cloth, The Little Black Book of Wireless, which includes "a complete history of
Airbnb), or where open access to data becomes not only a question of public transpar- 802.11 and GSM wireless communications, exploits and vulnerabilities," and simulta-
ency but the basis for innovation and new business opportunities, too (e.g., in urban neously plays with the reader's own vulnerability by harvesting hostnames, usernames,
development, as will be discussed in chapter 3). In relation to work and social life, unencrypted passwords, HTML, and much more. These fragments are published on
people generally are organized (in work as well as leisure) markedly different than dur- the book's website, but deleted when closing the book. 38 Through the way they work
ing the capitalism of industrialization, and the ways value is produced, consumed, and with technology, Weise7 members explore the tendency of interface production, such
exchanged are undergoing deep changes described by concepts such as, for instance, as the i;:onstruction and conflicts of the inscription of reading and writing (this will be
semantic or semio-capitalism. 35 addressed in the following chapter).
As Weise? demonstrates, the interface is undergoing severe changes. The excessive

In this way, the interface has not only become adapted to us by way of technologi-
use of "cookies" on the World Wide Web, apps such as Uber, and many more all depend
cal development but has also internalized a changed perspective on the world, where
not only on interactions in a self-contained system but on a real-time flow of data
the online versus off-line, or virtual versus real-life dichotomies, are undermined and
between devices and services, too. The signals that users emit along with the capturing
hardly make sense anymore in developed societies. The "interface culture," described
of user behavior and consumption patterns become valuable in the sense that they
by Steven Johnson at the end of the millennium, is almost trivial today. 39 This is, for
both serve as the impetus for smart metainterface services, and a business model for
instance, also demonstrated by the US artist Benjamin Grosser's Touching Software short
offering such services for free. In general, streaming services for cultural consumption,
film, in which he compiled the numerous times the famous political Washington, DC,
social services, and much more function as smart networked services that financially
drama series House of Cards displays interactions with interfaces: characters touching
as well as functionally build on the capturing and processing of data-rather than the
screens, texting each other, reading e-mails, taking photos with their phones, and so
interface itself as a consumer product that can be bought or sold in a traditional sense.
on. Touch interfaces are ubiquitous in the series, and constantly intertwined in ordi-
In other words, the labor and modes of production that the author/producer is
nary activities; in fact, Grosser's edit also documents that interfaces of all kinds become
engaging with, and the tendency reflected within this production, are changing. The
the main plotting device in the many intrigues and conspiracies, and how that fuels
interface, understood as a new form of textuality with other grammars and modes of
our attention as spectators. 40 Yet with the prevalent use of the interface, it becomes
inscription than traditional text, is entering new contexts that change its material, con-
masked in new ways that reconfigure interface culture as well.
ceptual, and societal role (e.g., in relation to cultural production, urban settings, and
Notably, the human-computer interface does not merely involve individual clicks
perceptions of climate change or the war on terror, as will be examined in the following
on a laptop or desktop screen that address a script in a piece of relatively self-contained
chapters). At the same time as the interface is the catalyst for HCI and textual inscrip-
software. The ways users experience and interact with computing, and the ways users
tions of sign and signal processes (i.e., self-contained systems), it is increasingly build-
become inscribed in interfaces by means of reading, writing, and executing the soft-
ing on and feeding into a network of interfaces. In this network, the textual inscription
ware, are developing beyond the well-known desktop-based graphical user interface. As
does not merely include the formal representation of labor in an algorithmic form
the interfaces increasingly become mobile, embedded, ubiquitous, and pervasive, they
(as Nake described) but also an incomprehensible and invisible system that captures
also become part of a network of interfaces whose inner functions are less transpar-
and quantifies signals emitted from the use of interfaces, location devices, sensors,
ent than the everyday interfaces of cell phones, entertainment devices, and so forth.
and much more. The constant and dissolute exchange and calculation of these data
The interesting aspect is how the everyday human-computer interface is intrinsically
make the signs of the interface appear as a new and smart mode of production. How
linked to a constant flow of signals emitted from users or the environment, and con-
is one to conceptually understand this change of the interface? And what could the
versely, that it itself emits signals that can be captured and processed. This means that
beginning of an aesthetic critique of an interface that conceals itself in significant new
users rarely interact with isolated computers, but instead with networks of data floating
ways be?
in "intelligent" software systems.
On this coded and networked level of interfaces, the everyday interfaces become
The Metainterface's Inscriptions of Behaviors
smart interfaces that make the HCI apt and swift: it becomes a metainterface. This
metainterface is based on a constant flow of signals. Examples of these signals range
With the metainterface, the computer interface today is seemingly omnipresent. Wher-
from electromagnetic waves such as a Bluetooth device to other kinds of motions like
ever, there is usually some kind of computer interface available, be it the touch inter-
the transfer of real-time data on the Internet. Even nondigital phenomena such as the
face of a smartphone or a sensor embedded in the environment. Human-computer
weather or flow of traffic in a city emit signals, but in either case signals are always con-
interfaces pervade all aspects of everyday life. From being restrained to desktops, they
sidered to carry value: in order for them to be computed, they must be quantified. In
are now everywhere: in pockets and entertainment devices, on walls and buildings,
this way, the smartness of the human-computer interface also points to how surround-
and hidden in infrastructure and environmental sensors. They become smaller and big-
ings are increasingly turned into data that can be used to determine and anticipate
ger, more mobile and less stationary, than the interfaces of PCs or other workstations. events, and how areas that have not been previously thought of as quantifiable are now
30 Interface Criticism 31
Chapter 1
regularly quantified and computed. Data are captured and stored in so diverse areas as To Agre, the use of data capture is not only an extension of human activity through
cultural consumption (what people look at or listen to on their devices), cities (from technology but also a structural condition for the use of computers that reflects val-
CCTV or the tracking of devices), or weather (rainfall, temperatures, and so forth). ues and worldviews. In other words, systems of data capture (despite being concealed)
In other words, at the same time as the interface seems to transcend perception, change the ways of interacting with interfaces and condition perceptions of the world.
it is omnipresent: it is increasingly difficult to avoid interaction with the networked This condition is not an indisputable imperative of computational management, but
computers. With this omnipresence, the users' behaviors constantly feed into the as it is embedded within an everyday use, it is hard to imagine alternatives. Agre, for
smart system of the interface as a cybernetic feedback loop. Moving through traffic example, points to the way activities are reorganized in order to become traceable as
with a location service, paying with a credit card, browsing the Internet, communicat- perhaps the most important effect of datafication. This reorganization involves "repre-
ing via e-mail and social media, reading on an e-reader, watching movies, listening sentation schemes" that use linguistic metaphors and formal "languages" for represent-
to music via streaming services, playing games online, navigating through web pages ing human activities. "Human activity is thus effectively treated as a kind of language
with cookies, and so on, are all examples of activities with networked interfaces that itself." 43 The notion of a grammar of language is traditionally restricted to speech or
themselves emit signals that can be quantified and fed into the system. All the user writing as a system of inscription into alphabetic and phonetic systems that can exter-
sees is the metainterface to these displaced processes. This data production is deeply nalize and communicate thoughts. The externalization of thoughts into systems and
dependent on an increased use of interfaces, such as social media, smartphone apps, procedures, however-or as Nake has explained, a formalization of labor and its formal
search engines, streaming media services, GPS navigators, and the so-called Internet of instrumentalization in the user interface-extends the notion of grammar into human
things, to name only a few instances. It is hard to imagine any "activity-system" that activity. Echoing not Nake but instead the well-known HCI theorists Terry Winnograd
is not thoroughly integrated with distributed computational processes in "smart" and and Fernando Flores, Agre describes how this "grammar of action" has a five-stage cycle
"predictive" ways. consisting of analysis, articulation, imposition, social and technical instrumentation,
Illustrations of how the everyday and HCI relate to a networked level of signal- and elaboration.
computer-interaction are numerous. Most people have probably noticed how interfaces In other words, already in 1994, Agre sees "the tendency toward ever 'deeper'
become customized in smart ways, and how displays and functions-ranging from articulation and capture of activities" as the most significant technical trend, but
search results to route planning-are variables of signals emitted by the user and other also insists that it is impossible to "remove the elements of interpretation, strategy,
users. Broadly speaking, but also fundamentally, the metainterface inscriptions of and institutional dynamics." 44 Data might seem raw and immanent to the world and
behavior build on two aspects: data capture and smartness (and techniques of machine people's activities, but this only happens through initial grammatizing and reorganiz-
learning). ing, and such systems are not totalitarian mechanisms but rather leave language for
Datafication, or inscription of the world into computational systems, which is interpretation.
implied in the signal-computer interface, has been discussed in various ways previ- Agre's arguments for a grammar of action is not only restricted to the organization
ously (though not always in relation to an art context). Data analytics, for instance, of labor at the workplace. With its widespread use, it is part of the everyday inscription
is far from a new thing in computing, and many corporations and public institutions of behaviors that happens with the proliferation of interfaces and grand-scale inclusion
have gathered and analyzed data for many years, too. Already in 1994, the informa- of signal-computer interfaces into the human-computer interface. Furthermore, the
tion theorist Philip E. Agre discussed how the capture of data is part of "a tradition grammar not only concerns actions that are performed to become traceable (or subject
of applied representational work that has informed organizational practice the world to data mining) but also a grammar built on the imperative of smartness.
41
over." Agre argues that "as human activities become intertwined with the mecha- The smartness of the networked level of interfaces, generally speaking, relates to
nisms of computerized tracking, the notion of human interactions with a 'computer'- processes of machine learning that are concerned with predicting what people want
understood as a discrete, physically localized entity-begins to lose its forcei in its place (rather than just knowing what they want). To give an example, services such as Face-
we encounter activity-systems that are thoroughly integrated with distributed compu- book or Amazon offer interfaces that appear as customized to the individual user's
tational processes. "42 needs and preferences. Yet the selection of items in the interface, and statistical models
32 Chapter 1 33
Interface Criticism
51
that decide what the user is likely to like, are not just the results of tracking the user's replaces the subject, and the subject's narrative is replaced by the capturing of actions.
own preferences and interactions (friends, likes, purchases, browsing, scrolling behav- This change in formality is also visible to the user, and seen, say, in the ways maps and
ior, etc.) but also the massive data capture of many users' behaviors along with complex dashboards (graphs, analytics, etc.) have replaced the subject and three-dimensional
algorithms that compare them. As pointed out by Chun, this movement from analysis imagery as an origin of perspective in interface design. In other words, interface design
to prediction builds (among other things) on collaborative filtering algorithms that signifies the ways individuals, objects, and their relations are processed by neighbor-
create "neighborhoods" based on similarities and differences between user responses to hoods and other predictors, and inscribe the user in a grammatical system. Subject
recommendations. 45 This means that user data are not just kept as a record of an indi- formation is an outcome of a complex process that inscribes "actors" and "actions" in
vidual customer that the system may respond intelligently to but instead gain value by the interface, and where the smart processing of actors and actions is as important as
their contribution to a large statistical body that the user responds to. 46 Hence, in these their traceability.
smart systems, the human is not the only standard for intelligence (as often imagined Like data capture, the inscriptions of behaviors that take place in machine learn-
in theories of HCI). The kinds of intelligence that machine learning processes imply are ing processes participate in world construction as well as, for instance, the constitu-
different from the artificial intelligence of expert systems, as they were envisioned in tion of the subject's identity. Yet the relation between processes of individuation and
the 1980s, and the Turing test (testing the ability to distinguish between the actions of inscription is naturally also a complex philosophical issue that has been brought up
humans and computers) does not seem to comply any longer, or rather, the Turing test by, for one, Jacques Derrida, who describes it as an exteriorization of consciousness
only complies insofar as the human is acting more like the machine. 47 and memory. 52 Both Chun and Mackenzie seem to suggest that this externalization,
In addition to being smart, predictor systems are also all encompassing, and include which lies in the inscription's transformation of thought and memory into discrete
all aspects of the world. As the science and technology sociologist Adrian Mackenzie units, depends on technical systems that are undergoing severe changes. The construc-
points out in his elaboration of machine learning processes, machine learning, like tions of writing (called "grammatology" in Derrida's terms) is not just dependent on a
data capture, has a long history within computer science dating back to the 1960s, formal alphabetic system with a grammar but also on material constructions, or tex-
mostly in specific settings such as credit risk assessment. Today's techniques include tual machines, that run on alphabetic code (as formerly explained), data capture, and
decision trees, neural networks, association rules, k-nearest neighbors, expectation smart algorithms. Subsequently, interfaces, and their encouragement to compulsively
maximization, latent semantic analysis, and much more. 48 The general idea is known "like," "retweet," or in other ways inscribe their actors and actions in a smart system,
from statistics' linear modeling of the relationship between dependent variables (such are not innocent activities, nor are they extensions of human activities. Instead, they
as the price one is willing to pay for an item) and independent explanatory variables inscribe the human in a system with a grammar where they are not always in control
(such as income, gender, number of children, etc.). As Mackenzie explains, classical sta- or conscious of writing but nevertheless can leave signs for human interpretation and
tistics is limited in its predictions because of the difficulty in calculating large matrices critique.
of numbers, and it typically depends on a few variables such as gender, age, income, The grammars of these inscriptions that take place through the combination of HCI
or occupation. By contrast, today almost anything can be considered a variable, and with signal-computer interaction (i.e., data capture and predictive smart modeling)
almost anything is juxtaposed and associated in the statistical regression models that have a certain persistence. Antoinette Rouvroy, a doctor of law at Namur University,
form the consumers' or users' neighborhoods. The predictive models of machine learn- has argued that "data-behaviorism" (as she labels the grammar) and its "statistical
ing explore high-dimensional patterns rather than just associations between limited inference operated on the basis of correlations" is validated "through a kind of 'back-
variables. 49 ward performativity': anything that would happen and be recorded ... will contribute
53
Not unlike Agre's hypothesis that data capture is a language with a grammar, Mack- to the refinement and improvement of the 'statistical body."' When data regarding
enzie notices that predictive modeling gives rise to new forms of agency, and "pro- consumption, transactions, social relations, and so forth, becomes indicative of the
gressively interpolates and interpellates subjects." 50 Similarly, Chun has argued that reality to come (which cannot yet be felt), it has an impact on how technical infra-
neighborhood predictors dissolve a postmodern disorientation and subject incapable structures are constructed. The wide proliferation of interfaces encourages the users to
of apprehending the multiple relations that construct their identity. Instead, an actor act. This may happen consciously as when they willingly express their preferences on
34
Chapter 1 Interface Criticism 35
Table 1.1
Elements of the human-computer interface and the signal-computer interface hard time grasping or do not really want. Critique may also be of a different kind, how-
Human-Computer Interface ever. Rather than the validity and logic of data capture and predictive modeling, one
Signal-Computer Interface
may address their operationality as such, and how the textual machinery not only is
Browsing, scrolling behavior, likes, reviews, posts, Techniques for tracking behavior a functional entity but also a kind of linguistic formalization of labor with a particular
and other user responses to the metainterface that contributes to the statistical
Sensors, cookies, Bluetooth, etc. grammar that is veiled, though nevertheless possible to address. This knowledge is the
body (data capture)
Recommendations, requests, posts, routes, and other aim of aesthetics. But where does an aesthetic critique begin?
Machine learning and collaborative
experiences of personalization and customization filtering of data (smartness) As previously discussed, artistic production with interfaces has traditionally been
concerned with an interface that disappears behind a veil of immersive and trans-
parent actions that generally speaking, not only convey meaning but meaning and
social media, but also unconsciously when they are tracked by cookies or positioning
action simultaneously, too. This juxtaposition of processes of signification with signal
systems in their devices. These actions and behaviors in the metainterfaces (platforms,
processes is often unveiled as a language with specific properties that refer to a system
urban interfaces, and more) are thus part of a signal-computer interface and contribute
of production. By reflecting its own textual production of inscription, as an encoded
to the statistical body that forms the basis for the users' interfaces in a feedback loop
textual machinery, the artwork (and "author as a producer" of interfaces) potentially
(table 1.1). Users, however, also dislike, avoid suggestions, take different routes, or in
reflects the larger conditions of production along with their influence on cultural val-
other ways misbehave. Such disruptions of the loop do not dismiss the modeling and
ues such as reading, writing, sharing, producing, and so on.
its grammar; they merely cause an adjustment. They may contribute to the formation
The interface has seemingly functioned as an intangible window into an alternative,
of new neighborhoods and are even actively encouraged by the systems. 54 The system,
virtual reality that could be operated with buttons, menus, pointers, and so forth, and
its processes of interpellation, and so forth, are still in place, and continually becom-
this operational realism and reality effect of the interface has been countered by inter-
ing ever more differentiated. This strange temporal feedback loop is what creates the
face arts that in different ways have reflected its operationality as a particular mode of
product of anticipation in the metainterface.
production. Put differently, the artistic productions have functioned as realism in its
literary sense, reflecting the visible/invisible procedures of the interface as a tendency
Interface Criticism
rather than producing reality effects. 55
When it comes to the metainterface, the coupling of human-computer interfaces
Clearly, many things are at stake in the metainterface when the human-computer
with signal-computer interfaces, and the textual production and processes of inscrip-
interface gets coupled with a signal-computer interface. Hence, the interface's inscrip-
tion involved in data capture and the smartness of machine learning, this realism and
tions of behavior through data capture and machine learning can be critiqued in many
tendency in the artwork is of a different kind. As Rouvroy has argued, "Raw data func-
different ways. Generally speaking, they can be critiqued at the levels of ethics, logics,
tion as de-territorialized signals, inducing reflex responses in computer systems, rather
and aesthetics.
than as signs carrying meanings and requiring interpretation." 56 In other words, in
Evidently, it is not difficult to question their validity. To many, the excessive data
the coupling of human-computer interfaces with signal-computer interfaces, knowl-
capture should not be officially and legally acceptable, and it is also common knowl-
edge is seemingly not produced about the world anymore but instead from the world.
edge that it takes place on grounds that are opaque and not always entirely honest.
The epistemic knowledge produced in the metainterface (and that is intrinsic to the
This goes for the wide use of cookies on the World Wide Web to the tracking of loca-
intelligence of contemporary interface culture) has lost its contact with the world it is
tions and user data by apps. As smart as interfaces may seem, they obviously also have
aimed at representing, yet it simultaneously presents itself as an ontology. This means
their logical limit: they can be difficult to use, or produce outcomes that are useless
that the substitution of interpretation with computed and automated reflex responses
and simply not rational. It is hard to make up for human experience and easy to put
is not an overcoming of a problem of representation. Whereas the design of human-
users in the wrong "neighborhood," and the smartness of Google, Netflix, Amazon,
computer interface traditionally attempts to overcome the interface by displacing its
and others may turn out to be a well-programmed presentation of things users have a
written and constructed character, the coupling to the signal-computer interface in
the metainterface transforms the entire world into (the production of) writing or data,
but displaces its own grammar of selection, processing, and inscription: on top of the Like · Comment · Share 1 c
illusionism and transparency of traditional interface realism, a data realism is added. ~ 10 people like this.
In their realism and way of relating to the real, the metainterfaces of search engines,
D 3 shares
weather apps, location services, and so on, are not just seamless representations of
virtual realities but rather augmented ones. They are no longer just windows to other 0 View all 7 comments
worlds that can be manipulated; instead, they appear to be enhancements of real-
ity. These enhancements are the result of intelligent processes that capture and pro-
cess data, and are presented to the user as overviews that filter, quantify, update, and ~ people like this.
prompt news, traffic, messages, advertisements, playlists, social relations, and the like.
Following this, the new challenge for an aesthetics of the metainterface is not only
D shares
to discuss the fusion of physical and virtual reality as an aesthetics inscribed in the 0 View all comments
human-computer-interface (control and media, opaqueness and transparency, etc.)
Facebook Demetricator Demetricating Likes, Shares, Comments, and illimestamps
but instead to address this concealed production of reality in the metainterface: how it Original (top), Demetricated (bottom)
reappears and leaves signs of its own process of inscription. Interface aesthetics implies
an ability to question how signals in the form of captured data (supposedly immanent Figure 1.1
in the world) and their intelligent processing are inscribed into the world as language Facebook Demetricator (2012-) by Ben Grosser. Browser extension that allows the user to experience
with a grammar that affects perceptions of reading and writing, spatial properties, and Facebook without quantifications such as time stamps, number of likes, requests, etc. Courtesy of
Ben Grosser. Screen capture.
even sense perception (as the following chapters will address). The metainterface is a
new production mode that increasingly transgresses the relatively closed system of self-
contained software and HCI. It incorporates a signal-computer interface that quantifies become a visible interface or graphopticon, and why metrics is not always hidden as an
and datafies, and ultimately turns the whole world into an interface: a large statistical invisible part of data capture and smart algorithms. Given this, Facebook Demetricator
body whose reality deeply depends on the processing and visualizations of data. demonstrates a tendency: "Through its metrics, Facebook imposes patterns of interac-
One example of a practice that addresses the concealed production of reality in tion on us, changing what we say to each other and guiding how we think about each
the metainterface is found in the works of Ben Grosser, and also reflected in his own other. Demetricator, through its removal of the metrics, both reveals and eases these
writings. Through the removal of all metrics on Facebook's interface (numbers of likes, prescribed patterns of sociality. It shows us what the metrics want. The metrics want
friends, shares, comments, requests, etc.), the browser extension Facebook Demetricator more. 1159 It is the logic of quantification, the wish for "more" that makes it necessarily
(figure 1.1) is an exploration of how the seemingly innocent and smooth social interface visible as an interface element users see and react to-a grammar of action that entices
leaves signs of its own process of inscription as well as contains particular grammars of users to participate in the quantification of the social and self. 60
57
action. The extension lets the users experience the effects on their own social individ- The following three chapters track the development of the metainterface through
uation processes on Facebook. In Grosser's own words, such processes depend on what artistic productions and practices that deal with, respectively, the platforms that con-
he labels a "graphopticon": a "form of self-induced audit within social networks like vey interfaces for cultural consumption (e-readers, smartphones, and tablets), spatial
Facebook where the many watch the metrics of the many." As such, graphopticons- inscriptions of urban interfaces, and clouds of today's cultural interfaces. Through anal-
the overwhelming elements of metrification on the Facebook interface-is a parallel ysis of specific artworks and their tendency, they discuss how these new inscriptions
to Agre's grammars of actions, yet Grosser's work demonstrates how the grammar is and grammars of the metainterface appear at the linguistic formal, spatial, and material
produced through concrete interface elements that relate to the quantification of indi- levels, and eventually how to design with this critical awareness, as addressed in the
vidual and social behavior. 58 In fact, it points out exactly why quantification elements final chapter.
2 The Metainterface Industry: New Platforms for Culture
Rea/Beat
RealBeat is an iOS app by the Austrian media artist and poet Jorg Piringer (figure 2.1). It
is a small sampler and sequencer made for the iPhone and iPad that lets the user record
and transform everyday sounds into musical compositions. RealBeat transforms the
iPhone into a mobile recording studio, and demonstrates the potential of the smart-
phone as at once a recorder, studio with effects, live processor, and playback device. By
allowing users to easily record, transform, and compose with the sounds of their every-
day environment, it points to how the metainterface of the smartphone potentially
changes our way of relating to and being in our (sonic) environment, and makes it rela-
tively easy to compose beat-driven electronic music that is otherwise quite technical.
In several ways, RealBeat utilizes the device to fulfill many of the twentieth-century
aesthetic sonic visions. Already in 1913, the futurist Luigi Russolo argued that we
should use the noise of the city as an instrument in music, the 1940s' and 1950s'
musique concrete developed the recording and composition of everyday sound, and in
the 1960s and 1970s, R. Murray Schafer and other composers maintained that we need
to care about and listen to the soundscape and acoustic ecology of the environment. 1
Also in popular music, recorded sound, sampling, sequencing, and noise have become
part of a musical repertoire. Clearly, RealBeat is an example of how musical aesthetics,
once reserved for the avant-garde and owners of expensive studios and equipment, has
become a cheap, commonplace accessory, readily available in our pockets. It demon-
strates how smartphones potentially transform experience into an active recording,
sampling, and manipulating activity while waiting for the bus or cycling along in the
city. It allows for a new kind of active listening as a way of fulfilling Russolo's predic-
tion that in the future, we would be able to combine "thirty thousand different noises
... according to our artistic fantasy." 2 As such, RealBeat follows a trajectory seen within
visual culture, too, where mobile cameras and social media have allowed their users to
40
Chapter 2 The Metainterface Industry 41
does not influence the aesthetic form, medium, or content. Typical illustrations of
this are streaming music services, streaming film and TV services, and e-books. Yet the
metainterface industry is also an opportunity for more innovative aesthetics. Within
electronic literature, a number of projects have related to both smartphones and tab-
lets. Interesting example are Jason Edward Lewis's and Piringer's poetry apps that use
the touch screen interface to allow readers to "touch" letters and words in concrete
poetry universes, or narrative apps like Danny Cannizzaro and Samantha Gorman's
Pry from 2014. Also within art and music, a rich variety of apps critically explore all
aspects of the new platforms for interface culture along with their discreet networks,
displays, location systems, cameras, accelerometers, microphone, and much more. The
metainterface industry is obviously an opportunity for software artists-with a stable
distribution system, and even a business model that allows content creators to create a
small income (after giving Apple or Google their share), but it also presents challenges
and problems for both art and computing.
This chapter focuses not only on how computing turns cultural with the metainter-
Figure 2.1
face but also how this implicitly involves a computation of culture with automated and
Rea/Beat (2011) by Jorg Piringer. A sampler and sequencer made for the iPhone and iPad that lets
software-driven means that, for instance, automatize the process of curation. Seemingly
the user record and transform everyday sounds into musical compositions. Courtesy of Jorg Pir-
inger. Screen shot from instruction video. the new platforms empower both user and producer. As it will be argued, however, the
new model for cultural production, distribution, and consumption that comes with
the metainterface embeds technocratic protocols that disguise particular perspectives
photograph, filter, manipulate, or in other ways interact with the immediate environ-
ment, and distribute their daily experiences. on culture. More specifically, the chapter looks at how consumption becomes heavily
monitored and datafied, and hence also a new mode of production in a metainterface
Rea/Beat is an example of how a software-based art practice, with clear references
industry. The aim of the chapter, in other words, is to provide a characterization of
to an art historical context, has eventually found a stable platform for production,
this new industry, reflect on how it relates to former culture industries, and envisage
distribution, and consumption. One may even argue that the sonic practice presented
how cracks in the system allow for new imaginations as a tendency within the cultural
in Rea/Beat has become mainstream. Given that the app-based platforms also feature
expression. For instance, the chapter will explore how particular apps and projects
streaming services for music, books, and movies as well as numerous social media apps
innovatively expose, utilize, and subvert the metainterface industry's inherent proce-
along with thousands of games, it is apparent that these platforms-which include
dures and restrictions to exemplify its ambiguities, and offer imaginary and concrete
smartphones, tablets, watches, e-readers, TV and music streamers from companies such
alternatives in a legal gray zone, too.
as Apple, Google, and Amazon, and so on-represent a cultural turn in computing.
From the avant-garde to the mainstream, software-based aesthetic production and dis-
In the Office
tribution seems to drive the market for the metainterface and its platform computing.
Or put differently, the platforms have become a new conveyor belt for the networked
The present cultural turn in computing rests on an awareness of how the often-
computer as a cultural industry and the functional basis of the metainterface's produc-
tion system. bureaucratic and office-related procedures of computing constitute a new language,
and thereby also lead to new cultural expressions. In computing, this aspect has histori-
Besides a steady flow of games, most of the content for cultural devices are digital
cally received little attention, but is nevertheless essential in an understanding of the
versions of more or less traditional cultural content where the digitization seemingly
present cultural tum.
42 Chapter 2 The Metainterface Industry 43
Throughout the history of human-computer interaction (HCI), the preference in for word processing much to the handicap of other word processors." 7 Or as Geoff
interface design has always been user-friendly office work. In the 1980s and 1990s, Cox states, "The writer becomes part of the machine, thoroughly embedded in the
computing was seen as a revolution in office work. The operating systems of first Apple choice of computer and software program." 8 To understand how software and interface
(Mac OS, 1984) and later Microsoft (Windows, 1985) was structured around the desk- become cultural, one needs to understand this compliance between the author and
top metaphor, with folders, documents, and so forth, and PCs were widely marketed the bureaucratic machinery of the computer, including the ambiguities of this rela-
through software packages for office work, such as Microsoft's Office package (1988) tion. The cultural turn is then not just a history of pragmatist assimilation but also the
with word processing, spread sheets, and slide shows. As Warren Sack states, "[That] creative development of uses of the software and cultures of writing, as demonstrated
these operations correlate almost exactly with what the bureaucrat does with his file by Matthew Kirschenbaum in his Track Changes: A Literary History of Word Processing. "9
cabinets, desk, and trash can is no coincidence." It is therefore "clear that shuffling Both Fuller and Piringer have been active in producing cultural software that
through, stacking, listing, and filing were the ideals of 'memory' and 'thought' admired explores the operationality of the computer, or software as an aesthetic phenomenon.
and implemented by the founders of computer science and interface design." 3 In a In relation to Microsoft Word, Fuller suggests that users, for example, "cut the word up,
larger genealogy, these ideals also originate from the military system that developed open, and into process." 10 In his own installation A Song for Occupations, for which his
modern computing and programming. This has been elaborated by Wendy Chun, who text on Microsoft Word was originally conceived, he pedantically maps the software
also argues that the very programmability of the computer depends on an ability to "to take apart a component of this body, that part that 'produces' writing, and to find
store programs and information in the computer's memory, and that its genealogy out how it is composed as a sensorium, how it machines language," as he explains. 11
exposes an idiosyncratic image of memory as storable. 4 Piringer's earlier work, the visual poetry generator nam shub, can be seen as taking
In HCI, there has historically been only a limited interest in how computing has up this challenge. 12 If A Song for Occupations and nam shub explore the sensorium of
developed as a cultural phenomenon. If at all, the interest has been in how the field machinic language, RealBeat is a sonic practice that follows the same avant-garde tradi-
could learn from, for instance, computer games. As Ben Shneiderman, a key figure tion of cutting up, opening up, and putting into process.
in HCI, wrote in the early 1980s, where video games had experienced an immense In the present cultural turn in computing, though, one may argue that such aes-
popularity, games are "perhaps the most exciting, well-engineered-certainly, the most thetic practices not only have become mainstream but also subjugated. Apps for sam-
successful-application of direct manipulation." Joysticks, buttons, and other physical pling, editing, and displaying recordings of everyday experiences are among the most
actions contain principles that "can be applied to office automation, personal comput- popular in Apple's App Store, though with a preference for visual rather than sonic
ing, and other interactive environments." 5 The present cultural turn in computing, recordings (popular apps such as Afterlight or Facetune are good examples of this). 13 In
however, differs from Shneiderman's pragmatist interest in user-friendliness that wants the inclusion of cultural computing, the App Store and other similar phenomena also
to do away with the syntax errors of the command line interface and the like so as to significantly change and bureaucratize the possible compliance between the author
create a more human-centered interface design. Rather than seeing the interface as and the machinery of the computer.
something disappearing in the use situation, the turn is founded on the language of
the interface-in its grammars of action, and aesthetic and cultural implications. This In the Office of Culture
language is, for instance, brought forward by Matthew Fuller in his seminal article "It
Looks Like You're Writing a Letter: Microsoft Word"; Microsoft Office's design is not If user-friendliness in office-based interface design brought the computer to the masses
only a useful design for office work but also a design of a user, and particular bureau- in the 1990s, the current cultural turn in computing presents software interfaces that
cratic and businesslike ways of being active and laborious. 6 In other words, the soft- are cultural by default. Like sound has music, and writing has literature, software has its
ware interface contains a range of predefined functions whose purpose is not simply own cultural expressions. Whereas books, music, movies, and so on, have had solid plat-
to empower the user (as imagined by Shneiderman, and others) but also to regulate forms for production, distribution, and consumption (publishers, stores, libraries, etc.),
the user's writing and render them "a subject under the word processer" who is not though, cultural software has not had its platforms until now. With the metainterface
just writing texts but Word documents, too, and thereby creating a "de facto standard (i.e., Apple's App Store, Google Play, Amazon, and other services), producers of cultural
45
44 Chapter 2 The Metainterface Industry
software now have an interface to the new consumers of computer culture-with a Hence, just as much as computing has turned cultural, culture is increasingly com-
well-defined business model and on hardware platforms readily available to billions puted and bureaucratized. The new culture industry has, in other words, subsumed
of users. If the 1980s and 1990s were dominated by the PC interface, the 2000s have the kinds of software that explore the aesthetic and cultural implications of software.
become dominated by cultural computing: devices, software, and interfaces related to Seemingly, the aimless cultural occupation of messing with text, sound, and images has
cultural content and social media are constantly developed, updated, and integrated beaten office work. Yet this turn also implies that "office work"-and its bureaucratic
into everyday spaces, interests, and tasks. procedures of monitoring, storing, reporting, and so on-has now become applied to
Mainstream interface design's interest in computing and software as a cultural phe- cultural production and consumption. The scale of this, and how the cultural com-
nomenon is obviously not restricted to experimental software on the iPhone. With puter is used to compute and predict cultural, social, and individual behavior, is exorbi-
streaming services for music, movies, audiobooks and e-books, games, social media, tant. The implications of monitoring user behavior and other forms of datafication is,
and many other apps, it is evident that the metainterface is a new culture industry. on the one hand, a sense of "ease of use" where networks of other computers and users
Yet as will be discussed in more depth later in this chapter, this new culture industry serve cultural products intelligently on a plate, and on the other, a sense of discomfort.
differs in several ways from the industry of mass media. The easy access to apps and As an echo of Fuller's plea to address the use of office software design critically,
networks provided by the new computer platforms of the metainterface increases the the aim must be to cut up and take apart the machine that produces and distributes
consumption of culture (music, video, games, text, etc.), but implies that independent writing, or any other kind of cultural expression, and explore how the metainterface
and niche productions (such as Piringer's RealBeat) can reach a dispersed audience (also functions as a sensorium that "machines" language and designs its consumers-or
framed by Chris Anderson as the theory of the "Long Tail"). 14 In other words, the in Walter Benjamin's term, explore its tendency. The challenge that Fuller proposes
market for culture is changing dramatically, and in ways that benefit both mainstream is not only analytic but also practical. Whereas A Song for Occupations and nam shub
and niche. Though there are of course both winners and losers (including content reflect the operations of word processing and a work-based office industry, the aim is to
producers that get a tiny revenue), it still seems that "the sky is rising" according to a present how a practice may relate to a cultural metainterface industry, which automa-
widely referred US technology report: people are generally increasing their spending tizes, bureaucratizes, and instrumentalizes cultural production, distribution, and con-
on culture. 15 sumption. The following parts of this chapter will explore the new cultural industry of
Any kind of cultural critique of metainterfaces and their industry must include not metainterfaces, first by reflecting on platforms of the metainterface and their control
only a general critique of the interface but also a critique of the general user-friendliness mechanisms, then on how the platforms reconfigure everyday life (e.g., produces new
of the interface-meaning a critique of the bureaucratization of cultural software that users along with new forms of writing and reading), and finally on how this industry
takes place in the metainterface. In contemporary interface culture, cultural production both resembles and distinguishes itself from a former culture industry.
is tailored to the distribution platforms, and to a large extend becomes" shrink-wrapped"
and fitted-like office work-into particular formats and predefined platforms. In this Software Art Platforms-from Curating to Autocurating
way, cultural production and consumption become connected as input and output
in the same metainterface, often blurring the differences. Reading books, listening to To begin an exploration of how platform control is exercised in the metainterface, it
music, or watching movies have hitherto been considered a passive engagement, but is fruitful to return to a situation before the platforms settled. RealBeat is an example
is now an invaluable part of a chain of production: the successful prediction of what of how software art and aesthetic explorations of interfaces no longer are restricted
people will consume deeply depends on processes of monitoring, quantifying, and to repositories such as runme.org, where Piringer previously has advertised his visual
calculating consumption in controlled environments that can predict general behav- poetry generator nam shub.
iors. As behavioral monitoring, profiling, and production of predictions work behind Runme.org (figure 2.2) was launched in 2003 and developed with strong ties to
the simulated pages of e-books or streaming of music, the human-computer interfaces the Readme festivals, organized by the net and software artist Alexei Shulgin, Olga
merge with signal-computer interfaces in the metainterface, and the functional basis Goriunova
,, I
and others. The Readme festivals consisted of four consecutive events in
of this is the creation of a new cultural metainterface industry. Moscow (Readme 1.2, 2002), Helsinki (Readme 2.3, 2003), Aarhus (Readme 3.4, 2004),
words, their aim was explicitly to frame the field of ongoing artistic activities with soft-
runme.org - say It with software, '"'. ware (i.e., software art), and build a theoretical framework around them that included
scholars from aesthetics, sociology of technology, literary theory, cultural studies, phi-
losophy, and so on. As such, they appeared as part of a larger trend that included
the Berlin-based Transmediale Festival's software art award, art.bit in Tokyo, Liverpool
Biennal's Generator, Whitney Museum's code.doc in New York, Electrohype in Malmo,
!!.lgprHtn:nlc; .iu,...mes:Ia.tlon (2)
> QQ...~:.~.affi.._d (1)
> pseudo-quines (0)
Q.bQ!Q9EMll!Y
QQ.llSClQUSne~ and the 2003 edition of Ars Electronica that featured "CODE" as its main topic. At the
Q.L!;ltLJ.(Pj.ogmj.oLrn..a.ll.sJl.q
agprop1iaUon. a.nd.J,lli!.qlarism (5)
> stealing (0) SMec6{e$haring time, they represented rare opportunities for scholars to gather, and an audience to
acti.fi£!.@!JntJ1!~rJ.!=.~ (10)
access and experience software art.
~rtllit1tlQQ.! (38)
> a!ld.iQILi&y_,;iJ (28)
> !l.Y.!'.filiyg(4) Whereas other cultural expressions (like literature, music, movies, etc.) had solid
> !J.$§~$;~ (1)
!;t~_iQc!.@g~(tl§ (16) platforms for production, distribution, consumption, and critique (e.g., literature has
J)_r.QW.§,gr_art(21)
had a print industry with publishers, libraries, stores, and professional critics), it was
£Q.c!ftil.tl(24)
> .99£!!':!.Q.Q!'ilt,'. (9)
> n:iinimal .~ode (3) ~fl~B~&s fl.m~9.hr2Ilifillr;; significant that platforms for cultural software other than games were only known in
> .obfu?c;~tion (3) antmoomorp]li~
> [email protected].!S.L~g!l~li. (7)
> SYJ!!.§§(1) L~~l!!m!'! ~.Q.&.\:.d..ol@m narrow circles, and were largely collective, self-organizing, and in flux. Yet at the same
cqnce.,ptJJal $OftwJ:ire (31)
> w.iHl9.YLhardware.-jormatinstr\Ji::t/Jm (4) time, they sought to mime the procedures of an art field, such as festivals and critical
Social Bits!
Three new Twlttery/Facebooky projects on
Runme. How Hetero by Stockholm Pride -
f(l,1.§!@ff!..Q peer reviews. As stated on runme.org,
uses artificial ignorance to analyze @_Qg~
heterosexuality based on language, Evil by
Tom Scott • reminds us that Facebook Software art gets its lifeblood and its techniques from living software culture and represents ap-
might just be the world's larges!
phonebook. And you C O D E me by
youandme takes a poet's-eye view of
proaches and strategies similar to those used in the art world. Software culture lives on the Inter-
Twitter.
f01 Jun 201 OJ net and is often presented through special sites called software repositories. Art is traditionally
Sneak Preview of Runrne's New Website
Design
intQI~~ presented in festivals and exhibitions. Software art on the one hand brings software culture into
Chee~ out a_ preview of runme's new -~1JL8.l1!i.Q §i!~n_gg y_g_Qt.91@!
the art field, but on the other hand it extends art beyond institutions. 17
In the spirit of Internet culture, runme.org is an open database where any user can
submit projects they consider interesting examples of software art. This freedom, inher-
Figure 2.2 ent to a free and open-source software culture more generally, is followed by a particu-
Runme.org (2003-) by Amy Alexander, Olga Goriunova, Alex McLean, and Alexei Shulgin. An lar mode of regulation known from the art world's curatorial practices. Though the
online and open repository where everyone can submit examples of software art. All entries ap-
invitation to submit is open, and all entries appear in a keyword cloud, they are, for
pear in the database (in lists, keyword clouds, etc.)-some are also featured with expert reviews.
Screen shot from website. example, also listed in predefined categories, and only some of them will get reviewed
as recognition of their quality. These reviewers (Amy Alexander, Florian Cramer, Fuller,
Goriunova, Thomax Kaulmann, Alex McLean, Pit Schultz, Shulgin, the Yes Men, Hans
and Dortmund (Readme 100, 2005). 16 Already in 1997, Shulgin had launched a "Form
Bernhard, and Alessandro Ludovico) were part of the community and functioned-
Art Competition" for the big Austrian festival for electronic art, Ars Electronica, that
with a term from an arts context-as "curators."
hitherto had been focused on art produced with computers and electronics, and less on
With the metainterface, in the new interface industry, this relation between content
computers and software as expressions in themselves. The submissions featured drop-
production and curation no longer implies a peer relationship, and this considerably
down menus, radio buttons, text fields, and window frames as interactive images-
changes the nature of the platform. The cultural industries have had a hard time adapt-
turning the otherwise-functional metaphors for interaction on the World Wide Web
ing to a living software culture of rips and remixes (which RealBeat alludes to as well),
into conceptual art. The Readme festivals served as platforms for the critical research of
but the platforms seem to offer new and viable solutions for cultural production, dis-
software as a cultural phenomenon rather than a mere tool for solving tasks. In other
tribution, and consumption. As much as they allow for a new interface-based cultural
industry to flourish, however, they also inflict censorship and control. Although you will also know it when you cross it .... We view Apps different than books or songs, which
runme.org exercises control as a form of curating, the curating of the platforms of a we do not curate. If you want to criticize a religion, write a book. If you want to describe sex,
new metainterface industry differs significantly from this. If one compares App Store's write a book or a song, or create a medical App .... If you run to the press and trash us, it never
helps. 22
curation of RealBeat to the kinds of curating on Piringer's nam shub, submitted to
runme.org, it becomes evident how cultural critique increasingly becomes administra- This can only be read as a call for self-censorship, an explicit evasion of the free-
tive. Although it has never been featured and reviewed by peers, nam shub is readily dom of expression, which even warns against raising a public debate. It reveals that
identifiable and available as a software art project on runme.org. Contrary to nam shub, Apple's curatorial control is not restricted to technical issues (bugs, use of file formats,
RealBeat has been reviewed and found suitable for the cultural platform. Yet instead human interface guidelines, etc.) but also includes larger cultural debates, even though
of employing cultural experts, Apple has strict protocols for the review process, which it shows a lack of understanding of critical, artistic content. Unlike the major record
allows for automated and large-scale reviews, including not only bureaucrats but also labels, publishers, and so forth, in the former culture industry, it has monopolistic
software analysis and user report systems. In this sense, the new cultural platforms control over the market in its App Store: if an iOS app is rejected, it is cut off from
implicitly render expert critique abundant, and Apple has neither category nor appre- mainstream distribution to Apple's devices.
ciation for art in its App Store. Phone Story (figure 2.3), from the independent game producer Paolo Pedercini and
Examples of how Apple's assessment exceeds the app's functionality are numerous, his company Molleindustria, perfectly illustrates how apps that critically reflect the
and most often follow rules that are not always explicit but instead hidden within the cultural and political reality of their own production-that is, their tendency-may end
system as well as its software and hardware. For instance, apps need to serve a clear up in trouble when faced with Apple's curatorial control. Phone Story is a game made for
purpose; they need to submit themselves to a strict moral codex that excludes the use Android and iOS that "attempts to provoke a critical reflection on its own technologi-
of explicit lyrics, visuals, and so on, and they need to follow the codex of copyright. 18 cal platform." 23 It does so by letting the user play small episodes that demonstrate the
All these protocols by definition rule out most cultural and artistic practices. Artistic production and marketing of smartphones, including the exploitative mining of coltan
practice does not have a clear purpose; it may also involve explicit expressions, and in Congo, suicidal conditions of sweatshop workers in China, manufacturing of a con-
software art often violates copyright. If sampler-based music was (in)famous for the stant desire for new products through obsolescence, and problems of e-waste when
ways it violated copyrights and reused musical tracks, it is therefore remarkable that the old products are discarded. The game's website elaborates how the overall idea of
RealBeat does not provide access to the user's music library. There might be artistic rea- the game is to collect money for nongovernmental organizations that fight corporate
sons for this, as Piringer's artistic production is characterized by an interest in concrete abuse. The game, however, was only allowed in the App Store for a few hours, and less
sound and poetry, and he is also part of the Vegetable Orchestra (which performs on than a thousand users managed to buy Phone Story before Apple decided to withdraw
self-made instruments made out of vegetables). Yet there is a technical reason for this I it. 24 Though Apple has rejected Phone Story for various reasons, including its way of
too. As Apple states on its development website, "Apps that do not use the MediaPlayer generating money for charitable organizations, it is hard not to see it as one example
framework to access media in the Music Library will be rejected." 19 Or put differently, among many of how Apple does not tolerate criticism of its platform in the App Store.
"Every App is an Island," and any access to the system directory is off-limits to the Consequently, it is difficult to expose and critique the cultural and political reality of
developer. 20 the metainterface industry from within its own platforms.
There are numerous stories about how apps have been delayed or rejected by Apple's In other words, the automated protocols for curating cultural software-and indeed
gatekeeping because they violate the developer guidelines (their functionality is mis- any media distributed on the platforms-reflect a particular control of cultural con-
leading, they display nudity, they have a "wrong" business model. etc.). 21 Apple has, sumption (the enforcement of copyright, protection of brands and business models,
like Facebook and other companies in the metainterface industry, extremely vague etc.) that is embedded within the technical infrastructures of the prevailing cultural
guidelines. As it states on its review guidelines site, platforms of the metainterface: the smartphones, phablets, tablets, and e-readers along
We will reject Apps for any content or behavior that we believe is over the line. What line, you
with their controlled environments of "stores" and "apps." 25
ask? Well, as a Supreme Court Justice once said, "I'll know it when I see it." And we think that
The Control of Production, Distribution, and Consumption
The kinds of bureaucratic and automated control mechanisms that are involved in the
large-scale curating of cultural content in, for instance, the App Store reflects a more
general practice. In his seminal book Everyday Life in the Modern World from 1968, the
French sociologist Henri Lefebvre proposes the idea that human needs and desires are
defined by bureaucracy, and not by the individual. Or put differently, the modern capi-
talist society programs consumer practices into everyday life, through signs. In a con-
sumer society, products are not only consumed for their practical use but also as signs.
Lefebvre gives the example of a car as a "status symbol, it stands for comfort, power,
authority and speed." 26 Subsequently, not having a car inevitably implies a lack of,
say, authority, while having particular brands of cars inflicts different levels of author-
ity and power (which are frequently negotiated on the highway). In other words, the
bureaucracy that created the needs and desires of everyday life constantly assesses and
answers how well the consumer "buys into" a lifestyle, with specific hierarchies and
codes.
Lefebvre's thinking is not unlike the analyses of Word discussed above, where the
tool for word processing simultaneously designs its users to write proficient business-
like English, for instance. If the users do not use the right tool (with the best auto-
correct function and file format), they not only fail but are also failures; bureaucracy
defines not only human desire but human existence, too. Or as Gilles Deleuze asserted,
"Machines are social before they are technical." 27 Being human depends on a bureau-
cracy that encompasses consumer society as such, but in particular manifests in today's
platforms for social and cultural software. From quoting text and posting images, to
liking, reposting, or disliking them, bureaucracy controls not only the curation and
distribution of content but also the very need to produce and consume content. To
free oneself from the quotidian, by producing or consuming differently, as Lefebvre
Figure 2.3
proposed, no longer seems a productive escape. Rather, it is somewhat expected and
Phone Story (2011) by Molleindustria. A game made for Android and iOS that lets the user play
small episodes that demonstrate the production of smart phones, including the exploitative min- encouraged by a bureaucracy that performs predictive analyses of user-generated data
ing of coltan in Congo (illustration). The game was banned from Apple's App Store. Courtesy of (they simply form new neighborhoods, as described in chapter 1).
Molleindustria. Screen shot from app. The US cultural theorist Ted Striphas has applied Lefebvre's ideas of "controlled con-
sumption" in-depth to contemporary publishing and book trade. Striphas summarizes
his analysis into four principles:
1. A cybernetic industrial infrastructure that integrates and handles production, distri-
bution, exchange, and consumption.
2. Programming that closely monitors and tracks consumer behavior along with the and simultaneously, where their behaviors are tracked (what they write, what they like,
effects of marketing, and thereby also reconfigures the relations between consump- what they share, etc.). This not only inscribes users into an infrastructure that designs
tion and production. them as producers and consumers but also generates large parts of the financial basis of
3. Obsolescence that is programmed into the product, limiting its functionality and the metainterface industry. It leads to what Shoshana Zuboff, with reference to Google
durability. and other "hyperscale" companies, has characterized as "surveillance capitalism." Sur-
veillance capitalism is defined as a "new form of information capitalism [that] aims
4. A disruption and reorganization of everyday life practices. 28
to predict and modify human behavior as a means to produce revenue and market
Even though Striphas uses Lefebvre's ideas to characterize the US book trade, they control." 30 The monitoring of behavior is not only an added bonus for the marketing
can also be extended to describe what happens in and around the app interface as well department but becomes the essential production as well. It is therefore no coincidence
as the metainterface industry. that companies like Uber have been portrayed as a big data company that gathers
First, the metainterface industry is characterized by the integration of hardware, valuable information on user and traffic behavior, and evidently the sharing economy
software, and distribution network. Not only Amazon's Kindle platform but also plat- often relies on a simple process of exchange, where access to services is traded for vol-
forms from Apple, Google, Microsoft, Sony, and others all feature an ease of use that untary data capture. 31 Free cloud services and online tools such as Gmail, Google Docs,
depends on an insular management system that prevents exchange operations with Google Drive, or Dropbox as well as free social media platforms, free search engines,
other, related systems. For example, with an iOS device it is extremely easy to access free media streams, and much more, all come at the same price. 32 In fact, what the user
apps and their content, but it is virtually impossible to download software or media at their human-computer interface experiences as sharing (and as such, something
files in other ways than through the App Store or authorized apps. The transfer of regarded positive) is transformed into the dominating business model: users share, but
files from one platform's information silo to another therefore becomes a convoluted platforms collect and possess the data in the metainterface (through tracking, datafica-
process that quite possibly involves a jailbreaking of the device to remove the soft- tion, or signal computer interaction, as described in the introduction). 33
ware restrictions and permit root access to the system directory. This conveniently Following Lefebvre's depiction of everyday life in the bureaucracy of controlled con-
ensures control over the publication of content, and the exchange of content that is sumption, the imperative logic to share and consume in an interface culture reverses
both monitored and monetized. The physical device is eventually a storefront for the the relation between production and consumption: consumption, otherwise perceived
metainterface industry, and if the company behind it should for some reason terminate as a passive engagement, becomes productive, and conversely, the making and sharing
its activities, the device becomes virtually useless. of cultural content, otherwise perceived as an active engagement, becomes integrated
As discussed in relation to RealBeat, this integration generally prohibits access to into the platform. As the not-for-profit, anticonsumerist, proenvironment organiza-
the root directories of the devices, and in many ways these platforms are more closed tion Adbusters have argued, "Your living room is the factory, [and] the product being
and controlled than ordinary proprietary PC software. The devices function more like manufactured is you." 34 Their claim was broadcast in an "un-commercial" and directed
advanced media players, or "IT appliances" that hide essential parts of their function- against commercial television, but in a metainterface industry where many services
ality for the user, as Cory Doctorow argues. 29 They partly lack the universality and come with little or no content, this seems even truer: user-generated content is often a
open dynamics of the general-purpose computer, and only run authorized programs in clear sign that production and sharing comes as a service that is exchanged for the data
protected sandbox environments. As a consequence, software culture becomes limited tracking of consumption patterns (the core business of the platform). Subsequently, the
in developing new, innovative ways of using and understanding the computer, and social media platform, cloud service, search engine and streaming service all become
ultimately in developing new forms of software. important factories of profiling and datafication, although some companies seem more
Second, the business model surrounding the integration of hardware, software, and willing to protect users' privacy than others, depending on their reliance on surveil-
network changes the relations between production and consumption. The products of lance capitalism as business model.
the metainterface industry are not only consumer products to be bought and sold but The reconfiguration of production and consumption also applies to services where
also metainterfaces to networked services where users contribute and share information, users pay for content. 35 Amazon's Kindle e-reader tracks reader behaviors, and the data
54 55
Chapter 2 The Metainterface Industry
becomes part of its marketing strategy, or more specifically, prediction of user taste and The disruption and reorganization of everyday life caused by a new bureaucratic
preferences. The same principle applies to Netflix and many other services that offer and instrumental control of production, distribution, and consumption is reflected as
cultural content for a revenue, and in some cases this allows the service providers to a tendency within the artworks (following Benjamin's understanding as elaborated in
become actual producers, too (e.g., Netflix not only distributes but also produces tele- chapter 1). If Fuller's A Song for Occupation and Piringer's nam shub engaged in an inter-
vision series). The distribution of cultural content is an especially well-suited area to face culture defined by office work, a similar interest in the metainterface industry that
collect data on cultural and aesthetic values, including political preferences and judg- integrates culture in bureaucratic computational platforms can be found today. This
ments of taste. tendency, which in many ways follows former critiques of consumer society, often pre-
Third, the metainterface industry is characterized by the well-known experience of sents a critical attitude toward the new culture that manages to produce ambiguities or
having to endure constant software and hardware updates. The introduction of new file subversive alternatives. More important, however, it seeks to take apart the machinery
formats, standards, and protocols frequently result in a situation where, for example, of the new cultural platforms-by reflecting the production system of the metainter-
newer versions of apps are incompatible with old devices, or the other way around. In face that produces writing and reading, how it "machines" language and composes a
other words, updating controls the functionality and durability of cultural computing sensorium that is at once human and nonhuman, representational and computational,
devices of the metainterface, and this underlines the fact that owners do not indepen- or sign and signal.
dently control their platforms. Molleindustria's aforementioned Phone Story is a good illustration of how the plat-
Finally, the integration of production, distribution, exchange, and consumption form is critically reflected within the game, and how this potentially challenges Apple's
into software and hardware platforms, the tracking of user data that reverses consump- brand and business model, which builds on the principle of integrating hardware
tion and production, and the programmed obsolescence of software and hardware that and software (the first principle), and the constant need for updates of software and
limits their functionality and durability all influence cultural practices and everyday hardware (the third principle). Such behavior reveals the reconfiguration of cultural
life. Collecting, sharing, and creative reconfiguration of cultural content, say-which curation (the fourth principle)-before exercised by publishers and curators, and now
are all well-known practices with former platforms (books, records, and cassette tapes), embedded in vague developer guidelines and internal company bureaucracies. Yet the
and became extensively popular in personal computing's remix and mash-up culture- bureaucratic and automated control mechanisms that are involved in large-scale curat-
have become not only illegal but also almost impossible with the platforms of the ing and content production, and are programmed into the platforms as surveillance
metainterface. When licensing and streaming replaces ownership, users change their capitalism (the second principle) also reveal other reconfigurations. More specifically,
everyday behaviors. Instead of lending a friend a good book or piece of music, users are the following examples each present aspects of this in their exploration of a machine
referred to sharing links to services, for instance. In this way, sharing, which was once language that is not only used by humans (as Word, similarly, is not only a tool for
seen as a shared human experience of enjoying literature or music, has turned into writing but simultaneously produces writing and reading as a designed, instrumental,
platform advertisement for Spotify, Amazon, Facebook, and so on (in fact, often several and industrialized activity):
of these platforms at once when a user "shares" a Spotify link on Facebook). In addi-
1. A writing machine that transforms the HCI of word processing into a global machin-
tion, this changes the cultural and democratic institutions around culture. The library,
ery of text.
for example, is not only threatened by the digitization of cultural content but more so
2. A reading machine that reads how the reader reads this global text.
by the privatization of digital culture, where phenomena such as Amazon's streaming
services aim to take over the former role of libraries as collectors of information made 3. A general control of language that has little interest in what is written or read, but
accessible to a defined community, yet only to those who pay the license fee. Clearly, great capital interest in the system that produces text and language.
the difference between collecting and lending as a public institution, and publishing 4. A body machine that represents the outcome of this process: a body that is not only
and licensing as a private business, is a matter of public interest. Even the concepts of designed for consumption to fulfill its existence but also to share and be part of the
reading and literacy are at stake, as discussed below. 36 machinery of the metainterface industry.
Each of these secondary principles describes the metainterface industry that but it can also be seen as a continuation of the printing press's technologizing and
(through, say, surveillance capitalism) causes a reorganization of everyday life, and capitalization of the word.
will in the following be explained through artistic practices that in various ways asso- Walter J. Ong, a historian of print, argues that Johannes Gutenberg's printing press
ciate them with the metainterface industry's particular platforms. This by no means was a groundbreaking technology that led to increased literacy, new modes of read-
implies that the activities of the platforms are restricted to, or have a monopoly on, ing (silent reading), and renewal of institutions around the study and production of
the particular secondary principles. As the artworks exemplify, the platforms are text-such as the library and university. He further asserts that the printing press was
merely proponents of the principles. Ubermorgen's The Project Formerly Known as Kindle the first carrier of industrialized capitalism, leading to a new business around the pub-
Forkbomb addresses Amazon's print press and the production of writing; John Cay- lishing press, book trade, and newspapers and magazines, with their journalists and
ley and Daniel Howe's The Readers Project focuses on Google along with the changed critics: "Alphabet letterpress printing, in which each letter was cast on a separate piece
notion of reading, and their How It Is in Common Tongue looks at the capitalizations of metal, or type, marked a psychological breakthrough of the first order. It embed-
of language; and JODI's ZYX and Erica Scourti's Body Scan address the iPhone's and ded the word itself deeply in the manufacturing process and made it into a kind of
Google's grammatization of the user's body as a new combined and instrumental commodity. "37
sensorium. If we compare the printing press with our contemporary textual machinery of the
metainterface and a production system like Amazon's, it is clear that the commodi-
Amazon Writing: The Project Formerly Known as Kindle Forkbomb fication of words not only happens through their reproduction. Instead, the manu-
facturing of words has been completely absorbed by complex business structures and
Apple's iPhone is obviously an important device for both the metainterface industry computational processes. Amazon is an example of how language itself has become the
and turning cultural computing into an industrialized entity, but it is not the only fuel of a new mode of capitalism. Thee-reader-as well as social media, search engines,
one. E-readers such as Amazon's Kindle combine old traditions of literature, reading, hypertext, programming languages, and so forth-are all illustrations of how the cur-
and book production with the metainterface industry's new traditions of licensing and rent ecology of text is expanded, and no longer includes just the writing and print
integrating hardware, software, and distribution networks. Amazon's many activities of text.
are good examples of how literature and book production become part of a wider field In The Project Formerly Known as Kindle Forkbomb (figure 2.4), the artist duo Ubermor-
of textual machineries. The company began as a book retailer in the early days of the gen was inspired by the negative comments on Rebecca Black's Friday video-a viral
World Wide Web, but soon expanded its business. Today, it includes its own media music video on YouTube that received massive attention for its poor quality, with hun-
platforms: the Kindle e-book reader and tablets as well as print on demand and publish- dreds of thousands of hateful comments and numerous popular parodies. In 2012,
ing services with direct sales through to the Amazon bookstore. In addition, Amazon Ubermorgen collaborated with Luc Gross and Bernhard Bauch to build an Internet
is a leading provider of cloud services as well as streaming music and video (Prime), robot that could automatically generate books on the basis of YouTube comments on
retailer of consumer goods, provider of crowdsourced production (Mechanical Turk), videos, and then upload them as e-books to Amazon's Kindle bookstore-producing
and more. a whole literary ecology including crowds, authors, books, titles, accounts, pricing,
A central element of Amazon's business is to direct its sales strategically. Amazon is and a defense system against erasure. In this way, the system emulates Amazon's own
renowned for including readers' responses (customer reviews) to the retail experience platform by making a parasitic system inside of it, and it functions as a demonstration
on its website-a model that has been widely adopted by not only retailers but also of the mechanisms of today's production of text. 38 It might even be seen as a perverted
other services such as YouTube where users replace the professional critic. Amazon's version of the ideal postprint capitalism that delivers lots of cheap goods in the form
manufacturing of words has been completely absorbed by complex business structures of e-books to fill the endless shelves at Amazon's electronic warehouses. Just like the
and machine learning processes that present the newest and most helpful reviews to books by Philip M. Parker (who is currently the author of 127,174 books on Amazon),
each customer. This textual business machine might seem advanced compared to the produced from databases and Internet searches, The Project Formerly Known as Kindle
persistency of the traditional printed book, despite the many predictions of its death, Forkbomb demonstrates a new fully automated book production, which can fill the
58
In other words, reading the text is a complex affair. Dramas are renowned for their
structuring plot, and one's ability to read reflects the ability to detect this layer of the
text. In Ubermorgen's drama, though, it is almost impossible to find the structuring
layer. Behind every veiled sign there is another veil of implicit references to network
culture. The Project Formerly Known as Kindle Forkbomb reflects- almost as a mirror or
YOUTUBE kind of realism-the production of language and text in the metainterface industry's
popular culture: What makes the text in Ubermorgen's e-book is neither the video of
Bieber on YouTube nor its original context at funnyordie .com, nor even the video of
the anesthetized boy on his way home from the dentist. It is the relations between
these texts and the different levels of experiences that they reflect. Subsequently, the
POLYMORPHI C
ENGINE ideal reading experience does not find a structuring plot but rather an ecology of texts
AMAZON ELASTIC
COMPUTE CLOUD EC2 and relations that produces meaning. These relations must be seen as a textual and
semiotic manifestation of a human reality, yet whereas the production of meaning
SERVER_,,,,...
CLUSTER '---==--....J traditionally is understood as related to a human condition (language as a human
Figure 2.4 activity), You Funny Get Car reveals relations to a text machinery: if symbol activity was
The Project Formerly Known as Kindle Forkbomb (2011-2013) by Ubermorgen . A hybrid printing once a human affair, Ubermorgen demonstrates how it is embedded in a machinery
press system that automatically generates e-books on Amazon's Kindle bookstore. The texts are that commodities words.
created on the basis of YouTube comments on videos. Courtesy of the artist and Carroll/Fletcher In Ubermorgen's work, the production of text is both the effect of human and
London. Diagram by Ubermorgen.
machine agencies. As a fractured expression of a new kind of text production, which
mimics the text machinery of the networked computer, the work reveals an inner ten-
still-growing storage space on the consumers' Kindles. While Parker produces titles like dency. The project works as a "forkbomb" in the textual machinery of the metainter-
The Official Patient's Sourcebook on Narcolepsy with infinite boring reports, though, The face industry. In computing, a forkbomb is a denial-of-service attack, meaning it is a
Project Formerly Known as Kindle Forkbomb produces a new kind of staged drama. process that continually replicates itself inside the system, draining it of its resources
The books produced by the system appear as dramas with actors performing (or like a parasite and ultimately causing the system to crash. These automations con-
rather producing) social media. In the book You Funny Get Car by "Nrlnick Kencals," we stantly evolve around the semio-capitalist text machinery that produces our reality, but
meet thirteen characters discussing a video in which the teenage pop idol Justin Bieber Ubermorgen automatizes the networked processes that produce text today and turns
appears anesthetized in the back seat of a car. 39 As often happens on YouTube, the com- text against its own textual machine, as a forkbomb .
ments are hateful (the comments to Rebecca Black's Friday video is another example of Ubermorgen's forkbomb can be seen as a contemporary form of realism through
this), and sexually harassing toward Bieber ("when you press eight, thats when biber the way it explores how current expressions of text (e.g., comments on YouTube) are
has a dick rammed up his ass!"). On a closer look, however, the dialogue is much more intrinsically linked to the production of reality embedded in the machinery (e.g., the
complex. Apparently Bieber's video is part of another phenomenon, Funny or Die. Fun- processes of harvesting text and embedding it in new contexts), and in this way it con-
nyordie.com, a website founded by Will Ferrell and Adam McKay, which allows famous nects to a literary history of realism going back to classic French realism of the 1800s.
people to upload their own funny videos. The videos are rated by the users: the funny The Marxist literary critic Georg Lukacs has pointed out how literary realism explores
ones stay, and the others are archived in the "crypt." To further complicate the dia- the ways that technological developments affect culture and how art can respond to
logue in the drama, the characters discuss the sophistication of Bieber's prank, hinting this. He did this in a reading of the French nineteenth-century realist Honore de Bal-
that he is not acting anesthetized but instead reenacting another YouTube hit video in zac's Lost Illusions, which he read as a novel about how literature became commodified
which a young boy is on his way home from the dentist.40 in a capitalistic production system, and how this lead to a capitalization of ideology
and thinking.41 To Lukacs, Balzac's novel is an example of a conscious exploration of to the Amazon platform and stores reading data while the user is reading, including
a new discourse economy, and it shows how material changes influence the formal what, when, and where the user reads, and which notes and underlinings are made.
conditions of the artwork. 42 This kind of data capture is no longer only used to offer products on its website, but
The Project Formerly Known as Kindle Forkbomb might not look like typical realistic is increasingly -integrated into publishing and distribution as prediction models for
literature in the manner of French literary realism, but Lukacs's reading of Balzac nev- reader interests and purchases. So not only does Amazon provide a "one-click" shop-
ertheless makes it possible to discuss it as realism. It reflects how current expressions of ping experience and recommend products to the users, it also delivers fast and exempli-
text (e.g., comments on YouTube) are intrinsically linked to the production of a reality fies how machine learning can be implemented in business, but it is produced through
embedded in a language machinery (e.g., the processes of harvesting text and embed- monitoring reading behavior on the web and Kindle.
ding it in new contexts). But its realism is also different from Balzac's. Though partly a
work of popular culture, You Funny Get Car is not a popular work. The video comments Google Reading: The Readers Project
are of limited interest outside their original context, and the characters appear as one
blurry, homophobic, judgmental crowd. The drama is present, but appears as if it was Text production has always been a central part of the development of the World Wide
painted with one color only. Instead of selling critique as literary consumption, it shifts Web and its infrastructures. This text production not only includes hypertext in its pur-
focus and sells it as literary production- a literature machine. It turns the bureaucrati- est sense but is a complex conglomerate of references and citations spun together by a
zation of culture (e.g., in Amazon's replacement of expert critics with user recommen- machinery of writing as well; YouTube comments and reference to other social media
dations, algorithmic profiling, and recommendations) into the product of its literary may suddenly appear in e-books, as Ubermorgen demonstrates. A central factor in the
machine and text of its dramas, thereby also showing how this text is constituted in text machinery is Google. Google is naturally renowned for its search engine, web
networks and contains its downsides, as when judgments of taste are replaced by rac- browser, Android operating system, and many cloud-based web services that include
ist and homophobic flaming. In this sense, acknowledging that it is not only human maps, e-mail, word processing, file sharing, and more. It also owns YouTube, market
language that produces our reality but also the ecology of the networked computer's its own hardware platforms, and develops designs for imaginary futures, such as its
text machinery, the literary project is to develop a poiesis of the textual machine, to extensive development of self-driving cars. As a large-scale service provider on the Web,
produce a language reality with the machine. The forkbomb at once mimics and mocks advertisement agency, and information technology developer, it is a key player in a
the textual machine of the metainterface industry; it produces an automated and end- metainterface industry. Despite the scale and variety of Google's activities, its control
less long tail of more or less unreadable books, echoing Philip M. Parker's extensive of production, distribution, and consumption of the reading of text in particular seems
book production- books that probably nobody reads, writes, or even discovers, since to be a core activity at a scale that far exceeds Amazon's. For instance, the reading of a
they are strategically kept secret from Amazon in order to keep the forkbomb running Google search result is a controlled experience based on a complex reading of how and
inside the system as a hidden parasite. It explores the tendency of Amazon's printing what people read that Google captures from its various services.
press through the parasitic production system of the Kindle forkbomb, and in this This double-sided reading that characterizes the metainterface- where machines
way, like Balzac's novels, demonstrates how the functional basis of the metainterface's read how humans read- is not new since interaction has always conditioned what com-
production system and the cultural and literary, artistic superstructure of writing are puters register and how they react to users' behavior at the interface, but it has spread
related. and been increasingly intensified with the metainterface industry and surveillance capi-
As much as Amazon is a post-Gutenberg printing press that prints and distributes talism. We increasingly read texts that-like You Funny Get Car or a search result-are
books on demand, it is also a system of reading. If Amazon is renowned for its inclu- rewritten by algorithms programmed to mimic and manage our reading. Or put differ-
sion of customer reviews in the retail experience, the store is renowned, too, for using ently, the texts we read integrate a large body of text, and the scripts that control this
behavioral data along with peoples' taste and preferences to generate consumer pat- integration are (in more or less sophisticated ways) based on scripts that monitor read-
terns and predictions. Today, Amazon even tracks patterns of reading. The users' reading behaviors. The conditions of reading are in this way significantly reconfigured by
ing patterns are handled through Amazon's Whispernet, a cloud service that connects the metainterface industry. In his book Scripting Reading Motions, the Portuguese digital
literary theorist Manuel Portela analyses and discusses digital literature and literary software, or programmed readers, that read texts, rewrite the texts, and present them
interfaces that explore how human reading motions are being scripted into the inter- to human readers, thereby making their reading visible and readable for the human
faces. These self-reflexive interfaces mimic how algorithmic reading simulates, tracks, reader. In other words, their literary interfaces visualize the programmed readers'
and rewrites human reading, and demonstrate a battleground between human and reading. The programmed readers include Simple, Perigram, Spawning, Mesostic, and
algorithmic reading where readers are confronted with the effects as well as politics of Grammatical Lookahead Readers, and they can be downloaded as Java Archives. Their
the changed conditions of reading. As Portela describes it, "The material instantiation reading patterns are inspired by cognitive studies of human reading, and range from
of text, and our perceptual consciousness of it, respond to and reflect our own haptic something close to standard Western human reading (from left to right and top to
and visual interaction with the interface." 43 One of his examples is The Readers Project bottom in the Simple Readers), to reading across what Cayley and Howe define as the
created by the artists, writers, and theorists John Cayley and Daniel Howe. 44 typographic neighborhood and page (Perigram Reader), to readers looking for specific
The Readers Project (figure 2.5) consists of a series of ongoing experiments, installa- letters in order to form words (Mesostic Reader), and readers following the grammatical
tions, and performances that relate to reading. These experiments are based on literary structure of the text and finding alternative words to fit this (Grammatical Lookahead
Reader). 45 Consequently, the different vectors of reading create routes through the text
based on algorithmic rules, typographic neighborhoods, or grammatical and semantic
S\\~mmin g bark alonr to 1hr ba1hing rork, h ead undrr, he rrilchcs oul to grmq1 the familiar lrdge~ n fold in 1hc structures. As Portela writes, "The physical, cognitive, and social dimensions of the pro-
Rosc~tingcd grnni teju<rt aho,·e thr .surface oft hr wai~l•dccp watrr at its edge, hy the .c;l'onr whirh hr ca n ·°'er rlcarly
though unfocus.Edthrough thr lake water. but he has nol reached it yet. his expec ta nt h and brcak"i the surface, cess of reading are algorithmically simulated in a way that invites readers to see their
down through 'cmpty'w..\tcr .inclhiS knuckles graze th e rock. his face will not rise up, dripping a nd ga!-.ping. out of
t.hc wat er. instead, it 'falls' fonrnrDancl, momentarily, down , into the shallows, stumbles, breathes a choking own act of reading in action." 46
mouthful, which he c-_xl'cctcd to be alr. he fmcb: his foct, the ledge, a moment later. a child lea rning to 1-wim, hark
to this sa me rock. from tip-toe six ya rcl~out, the.'\" anxious half-flailing dog-paddle hark to th e !1,mdy shallows. In some of the interfaces, the human readers can only read the texts through the
mi~~ ng the kclgc andchokinG.comfort ed after her first .\·w im. his h a nd hovns on:r smoot h forhidclen fie.sh.
imagined orhrrs. to 1011rhll1cmis assured cli.-mstcr, waking nightmare. ine,itahle misundersta nding and 1 finally.
programmed readers' reading and rewriting of the texts; in others, the programmed
betrayal. hare island fir.sh. torcad-1 this shore. 10 rome hcsidr. islanrl<'d. ncurath '.s sailor on the mming island, readers' routes are highlighted and obviously influence the human reading. The human
\\"J.tching it ;; h",1kt· physirill knowlrdgr - and wondering (in pirturr.s), ·,\·hy is it that language
wi:1-he_(; mr '. w r<•.' on i,l:mdOf~mn,' hemlock, of pine and green mos.,, lloor ofthr woocl1;, l.ight lacing the readers thereby not only become conscious of their own reading process (including the
,!1.tllm,,.. ! word~ drili.ing L"nckr moon, on the sea oftcxtuality. lcucrs lacing the surface of its waters,
like that li1!lt .. l.rndin~--- ti.tt iuG tcxl~ in other languages for other islanders. hut mr gra ndfolhcr's grammars, habits, and materials governing it) but also the algorithmic readers' gram-
boat is ~ nkin g-. an)inorc, those selves. a nd my gra ndmotJ1cr's hoat is sinking and i
cannol reach that island anymore; those!-ieLvc.sof ours. or llu• c-tt-.hion-sh.apccl slone i asked for, or the sloping rock
mars and (re)writing of the text. This double reading-writing ultimately becomes vis-
where anodtcr father ca.,; t for snrnll-mouth h.Ass and other happy fish - trailing a silent line. the ~igh of th e \\':Hers ible through the way that the human reading is blocked by the programmed readers'
pulled ha rk hy th r packUc in th e only island ' j' ca~ mon·. swim ming hack (alonr?) to th e bathing rock radt night,
h rad 1.mdrri he rrar.hcs 011t to grasp the fomiliarlcDgc, a folrl in th<' rosc-tingrcl flesh ju.st brlow thr srnfacr ofhrr performance. Since it is only partly visible and gets disturbed by the programmed read-
, ..·ai.~t 1 but still somehow nea r her face whlch he scr clrarly through the dark \\'atcr. hm h e has not rcarh cd her yet.
his expectant hand lucak-; the surfarc,dow~ through ·dry' water and hi.~ knuckles graze the rock. his farr ,,ill not ers' zigzagging through the text, it is often difficult to read the original text. Instead,
1isc up, dripping andGasping,out of the water. instead, he 'falls' forward and, momcnta1ily, clown, imo the
shallows, stumbles, brcathcSa rhoking mouthful, which he cxpccl<:d to he swccL, dcliC'imt,; darkness. he find~ his
the human reader reads how the text is generated and rewritten by the programmed
sleep, slides orrrhelcdge, a moment later. nctu-:HJ1 '.s pilgrim, 1620, on Lhc mO\in.~ island, lcc:ning: chc old world ,mcl readers. The grammars of literacy literally-displayed in the interface-clash with the
sailing to tHr new. unacrountahly on clcC'k •jn a miglllic~.'itonu' wh en the ship pitrlwd, he was thrown imo the .'ica,
hut caught hold of a top-sail hali"'Rdwhich hung oYr.rhmud ,111cl ·ranr out at length·. h e kept hi, hold ' though h <' grammars of the reading algorithms. HCI becomes rewritten by signal computer inter-
was .sunchicfaclOmr.s unclcr water' until he was hauled ha ck to the ~11rfacc~th en dragged on board with a hoat
hook. th r hocly is lost, givrn over to a rlor. k that gh-cs a new name to rHry srparatr moment. the hody is gi\'rn m·rr action in the metainterface.
to entropy, the sra. yoUcannot reach tha( shore, with .'-raGuUscirrling. turning and turnin g, thc isla nd rnm s: in 1hr.
waler and your hand slips oft~a notHcrbloatcd corpse. In this way, the mainly invisible algorithmic paratextual framing by algorithmic
agents (such as Google's spell and grammar checking, indexing and search bots, con-
textual advertising, other people's popular highlights, etc.) becomes visible. Cayley and
Figure 2.5 Howe specifically mention the writing of their own text in the Google Docs interface as
The Readers Project (2009-) by John Cayley and Daniel Howe. A series of installations and software an example of this paratextual framing by algorithmic agents. 47 In this way, the project
that makes programmed reading visible and readable to the human reader, including a "peri- demonstrates and radicalizes the way paratextual framing agents influence our reading.
gram reader" that is defined by a typographic neighborhood and a "mesostic reader" that follows
As Portela argues, the project is "scripting the act of reading .... Since these readings of
specific letters that form words. Courtesy of John Cayley and Daniel Howe. Screen capture from
the machine are offered as writing to human readers, the writing of reading and the
software.
reading of writing contained in The Readers Project turn readers into metareaders who his2d4 aid sits a2d51ittle aloof he2d6 announces briefld 7 movements
are forced to read their own act of reading the program reading."48 In the programmed of the lower face the2d8 aid enters it2d9 in his ledger
readers' interfaces and real time, the human reader is able to read the comprehensible
my hand2d" won't come words won't come2db no word not even 2dc
and incomprehensible mechanisms that complicate the textual dynamics, and see how
soundless I'm in2dd need of a word of!d• my hand dire2df need I
reading and the readable is not only governed by the reader but also increasingly by can't2•0 they won't that too2• 1
programmed dynamic agents creating alternative connections and neighborhoods.
The human reader thereby sees that they are not the only reader, and that the readable deterioration of the sense of humour2e2 fewer tears too2• 3 that
too they are failing2• 4 too and there another2•5 image yet an-
is cowritten as well as governed by networked software and corporations in the cloud
other2•6 a boy sitting on a bed2• 7 in the dark or a small2•8 old
that are mimicking, tracking, and rewriting the reading. The human reader ultimately man I can't see2• 9 with his head be it2•• young or be it old2•h his
metareads, and realizes that his or her reading is enmeshed in a networked cybertext head in his hands pee appropriate that heart2•d
where reading is tracked and used to generate writing in an endless data loop that we
question am I happy in the2•• present still such2•f ancient things
also know from social media, yet rarely are able to read directly.
a little210 happy on and ofFfl part one before212 Pim brie£2f3 void
Through the production of reading, as an inner tendency, the work reflects the and barely2f4 audible no no !215 would feel it and2f6 brie£2f7 apos-
changed conditions of reading. This is revealed as cracks and tensions within the
work where passive reading becomes an active form of rewriting. In the metainterface "''www.niemanlab.org/author/kdoctor/ (id. 548000)
'"'www.fo!d3.com/document/84474314/ (id. 54) "'ca,anctuary.org/freedom/ (id. 187)
industry, reading (and consuming, more generally) becomes a production, but in The "'www.shakermakerpr.com/post/25379254492 (id. 26400) 2"www.ask.com/Lowcr Face Lift
Readers Project, the human reader experiences how the text becomes controlled, too, (id. 1) '"'forums.soompi.com/discussion/136622/bitcxme/p25 (id. 5)
'"'issuu.com/lsmedia/docs/hasc_low_res (id. 1) 2dbcatmp3.com/Sherald.html (id. 10)
and how this challenges his or her reading. The Readers Project lets us read how our "~n.wikiquote.org/wiki/The_Dark_Towcr_(series) (id. 664000)
'"'www.swjfj.com/archives/120 (id. 9) "'wwwJstor.org/stable/1487868 (id. 5570000) ''"fr-
reading becomes productive as rewriting, and how this production becomes part of fr.facebook.com/isaure.palfroy (id. 4)
'"'www.youtube.com/all_comments?v,,gm8TITthKzQ&page=l (id. 55900)
the text and textual business of big software companies such as Google. The reader '"wtmcclendon.wordpress.com/page/11/ (id. 6) '"wikia!ity.wikia.com/Zeezumrazzumprofcn
(id. 1) '"neurotalk.psychcentral.com/threadlOl407-32.html (id. 366)
is able to see, explore, and read the bureaucratization and instrumentalization of 2
"letters.mobile.salon.com/opinion/fearure/2009/06/08/wingnut/permalink/99ff69f.l844981
60c4b223b75de38e99.html (id. 1) 2'\veb.stagram.com/n/meggmack/ (id. 1430000)
reading. '''waxy.org/links/archivc/2004/01/ (id. 2160) " 7www.greglast.com/?page_id=156 (id.
222000) "'chasethetruth.blogspot.com/ (id. 2) "'thebarefoot.wordpress.com/2012/03/ (id.
18200) 2"scoopssn.blogspotcom/ (id. 3) 2<bairmax2010good.blogspot.com/ (id. 36) '"'i-gavc-
Google Language: How It Is in Common Tongues up.coml?s= (id. 932000) '"'www.ocezine.com/page/page/list (id. 63)
2
ttwww.amichopine.com/blog/?p=140 (id. 10) "'www.mtb-
bg.com/forum/viewtopic.php?f=6&t=310 (id. 10)
''"s-clothing.blogspot.com/2010/04/70s-clothing.html (id. 6) '"www.mgoblue.com/ (id.
The changed conditions of reading along with the encounter between human and 46600) 20nongcavenderphotography.com/?p=3066 (id. 760000)
'"www.microform.co. uk/guides/R97280.pdf (id. 5)
machine reading is also a political battle that extends a former copyright conflict of the "'andywhittnan.blogspot.com/2009_11_01_archive.html (id. 7) 2f''www.arnazon.com/ (id. 2)
'"www.ziglar.com/newslctter/?p=1595 (id. 270000) '-"bricf.moulev.org/ (id. 398000000)
World Wide Web into the metainterface industry. As part of The Readers Project, Cayley
and Howe have produced the artists' book How It Is in Common Tongues (figure 2.6). 49 [ 30]
The book is generated through a script that uses Google to search for the longest strings
Figure 2.6
of text that match Samuel Beckett's How It Is, while avoiding text actually written by
How It Is in Common Tongues (2012) by John Cayley and Daniel Howe. A book generated by a script
Beckett. The result is a book with the same text as Beckett's original, but quoted from that uses Google to search for the longest strings of text that matches Samuel Beckett's How it is.
other texts written by other people, and not least an enormous amount of footnotes In other words, Beckett's text is regenerated from quotes that are not written by himself, but are
with URL references to these texts. In other words, the text is cut up in short sequences "common tongue." Courtesy of John Cayley and Daniel Howe. Screen capture of e-book.
of typically two to five words caused by the longest portion of Beckett's text that could
be found in other texts. Interestingly, this is equal to the way humans read texts in
chunks of words. This was also emphasized by Cayley and Howe in the exhibition
Common Tongues, where the book was displayed next to an interface showing a per- say but rather how we use language. Mapping language (langue) is its business, which
ceptual reader reading through parts of How It Is . is obviously efficient for forming neighborhoods based on how language is used and
Yet Cayley and Howe furthermore used the sequences of words to search online for how it is biased, but (as it will be discussed below in relation to Scourti) these biases also
neighboring texts not associated with Beckett.50 In this way, they not only displayed a gets reproduced by Google's (and others') language machine, such as when it suggests
parallel between algorithmic and human reading (or signal-computer interaction and searches or search results, or when Facebook's algorithms produces newsfeeds accord-
HCI in the metainterface) but also demonstrated how Beckett's How It Is relates to a ing to its user profiles. 55
network of free text and language- a "common tongue" that exists outside Beckett's Conceptualizing, mapping, and organizing the network of texts through techniques
ownership and copyright. The issues of copyright and ownership of language are fur- of referencing and indexing is not a new development in literary culture. References,
thermore reflected in the licensing of the book as "Copyleft, hors de commerce": it is an associations, and intertextuality add new dimensions to a text, and attach the text
artwork "generated from orthothetic links into publically accessible records from the to greater networks of literature and language. Vannevar Bush famously argued that
great inalienable linguistic commons," as Cayley and Howe explain.5 1 They explicitly knowledge is produced through association, and described his conceptual hypertext
describe their work as conceptual algorithmic art built on an original concept, and not system, the Memex, as a support mechanism for associative reading and writing. 56
as a work of literature, which would violate Beckett's copyright. When reading How It Is in Common Tongues, one is confronted with the multidimen-
Put differently, Cayley and Howe point to how even copyrighted work speaks in sionality of Beckett's text, how an endless paradigm of other texts is folded into Beck-
common tongues, and how contextual work is always composed by and related to a ett's, and his into theirs. As a reader, one gradually unfolds one's reading of the text as
common text (to reference Roland Barthes's famous essay From Work to Text). 52 In this a paradigmatic networked text rather than as a sequential and syntagmatic work, and
way, How It Is in Common Tongues demonstrates that words are not only part of the Howe and Cayley thus demonstrate that Google does something similar through its
arrangement of sentences, and thereby the copyrighted work, but also of language as indexing of the World Wide Web. Beckett's original text has no traditional punctuation,
a network of common text. To be more precise, it points out how there is a difference but is in Howe and Cayley's version punctuated by footnotes with URL references that
between language as a common abstract and paradigmatic system where words can be act as fractures, illuminating how Beckett's text is part of a common tongue. It shows
changed and reused, and the particular syntagmatic instantiations of language that how Beckett's text is a network that feeds on and feeds into other texts. This unfolding
forms the positions of words in a sentence. As emphasized by the French founder of of language and limitless networking of How It Is is a poetic and potentially sublime
modern linguistics and semiotics, Ferdinand de Saussure, there would be no meaning- experience, setting the work free as text, equal to the way that Cayley describes search-
ful speaking (parole) without a language (langue), but in the metainterface industry, ing as "literary sublime" and "an encounter with overwhelming quantities of language,
language-and not only its particular instantiations-is increasingly capitalized. 53 arguably beautiful." 57 How It Is in Common Tongues makes the reader glimpse this sub-
If How It Is in Common Tongues illustrates that Beckett does not control his work as lime network of tongues, language, and people writing. Besides their own reading, the
text, Cayley and Howe also show that Google currently controls the whole paradig- reader sees the global reading of the common tongues. Yet the reader also experiences
matic network of text. Google's business concept is, in other words, closely related to how this network has become technologized, bureaucratized, and instrumentalized.
managing the paradigmatic network of language. This does not mean that Google and Both Beckett's How It Is and Cayley and Howe's How It Is in Common Tongues simul-
the metainterface industry control meaning in a totalitarian or indoctrinating way; taneously point to the interruption of language along with the impossibility of expres-
rather, it points to how Google manages, tracks, and indexes the paradigmatic system, sion: "my hand2da won't come words won't come 2db no word not even 2dc soundless I'm
and thus capitalizes language. In her discussion of Google, Zuboff characterizes this in 2dd need of a word of2de my hand dire 2df need I can't2eo they won't that too 2e1 " 58 To
change as follows: "What matters is quantity not quality. Another way of saying this Beckett's narrator, creative expression is problematic, but the footnotes generated by
is that Google is 'formally indifferent' to what its users say or do, as long as they say it Cayley and Howe's script add another layer to this. As observed by Ong in relation to
and do it in ways that Google can capture and convert into data . ... Individual users' the printing press, the "technologizing of the word," including the construction of
meanings are of no interest to Google or other firms in this chain." 54 Google (and other indexes,•·references, and notes, shows the "disengagement of words from discourse." 59
similar big software companies like Facebook and Amazon) is not interested in what we The deployment of algorithmic agents and network services to language continues this
indexing further, and develops a language that is less discursive and more controllable. Cayley ends his poem "Pentameters" by stating that "to make art on terms? Impos-
Ong also remarks that already the printing press and typography "made the word into sible. / For the sake of art and for the sake / Of every cultural institution and their
a commodity. 11 60 This commodification is unquestionably intensified with agents and futures / We must find a way to refuse such / Terms of use. "64 Nevertheless, Cayley
networked services such as the ones actuated by Google. How It Is in Common Tongues and Howe do not refuse the terms-neither of language nor technical protocols, algo-
demonstrates how Google's commodification adds an interface between the writing rithmic agents and big software-but use them in order to consider the conditions of
(Saussure's parole) and the language (Saussure's langue), or the work and the text. It language. Through its own writing, as an inner tendency, The Readers Project mirrors
thereby interrupts the individual reading through the networked reading, and ulti- the algorithmic agents and networked text that condition reading and language in the
mately changes the character of language. metainterface.
As Howe and Cayley argue, the "algorithmic, compositional, and configurative
agents of big software's network services" have changed the nature of reading and iPhone Bodies: ZYX and Body Scan
writing "far more deeply than even the development of hypertext in the 1990s."61 How
It Is in Common Tongues demonstrates how hypertext theorists' (such as Ted Nelson's) As both Molleindustria's Phone Story and Cayley and Howe's Readers Project demon-
dreams of a sublime connected language system has led to the Google's technologiz- strate, it can be difficult to expose and critique the cultural as well as political reality
ing of the word. This process is not only a continuation of former control mechanisms of the metainterface industry from within its own platforms. Attempts are consistent,
from the printing press but also continues with the programmed machine reading and however. Another exemption that confirms this rule is the app ZYX, developed by Joan
rewriting of our language. How It Is in Common Tongues illustrates how Google manages Hemskerk and Dirk Paesmans, also known by their artist group name, JODI, which
this network of language by tracking, saving, and licensing it back to us through their adds an embodied perspective to the critique. 65
terms of use. With programmed readers that have developed into an underlying net- JODI is considered a pioneer in net art as well as for its experiments with video
work, Google is hypertext's shadow: a metainterface industry that manages the unruly, games, maps, social media, and other everyday software interfaces. Often JODI's work
rhizomatic network of hypertext and turns it into a thriving business; a capitalistic explores the codes, platforms, and technologies that are normally hidden behind the
network machine reading our reading and writing, and furthermore this machine is metaphors, such as the page metaphor of the World Wide Web or immersive three-
not openly readable in return. dimensional environments of first-person games. JODI's deconstruction of the inter-
In a poem and article related to The Readers Project, Cayley offers the critique that face is never trivial or self-contained. Rather than being modernistic or media centric,
he is not allowed to make algorithmic agents search or read Google in the same way it looks at the tendencies hidden within the artwork's own production, and in a pre-
that Google makes algorithmic agents read his texts. 62 Not unlike Apple's control of cise and concrete way, points to the culture, politics, and technologies surrounding
the iOS platform, and general strategies of the metainterface industry, Google prevents interfaces. This kind of deconstruction of the interface is also applied in its app from
this by enforcing strict terms of use and licenses supported by technical constraints. 2012, ZYX.
In this way, How It Is in Common Tongues is not only a potential violation of Beckett's ZYX has two user interfaces: the smartphone app and a corresponding website
old-fashioned copyright but also a potential violation of Google's terms of use and its (figure 2. 7). The app can be characterized as a performance tool asking the user to, with-
amassed text. As Cayley writes, out further explanation, do repetitive tasks in a series- for instance, to spin around,
What if I, a good human, write (that is create or compose) a program that acts like a bad robot for jump, or blow into the microphone. When the tasks are completed, an alarm goes off
good reasons, for aesthetic, culturally critical reasons, or simply to recapture and reclaim some and images taken during the performance are shown on the screen. In this way, the
of that superb big data that lies on the other side of the mouth-threshold where the powerful app demonstrates how, with a reference to Agre, smartphones imply particular "gram-
indexes dwell? Well, if I do that, it's pretty bad, and it's against most terms of use. Big software mars of actions. 11 66 By overexposing the normally functional and unnoticed physical
can, it seems-via innovation, hyper-historical momentum, and force majeure-deploy whatever
relationship between interface and user, the app shows how the interface inscribes
robots it wishes-to index the web pages that humans have written or to police human access to
the movements and positions of its users, and how apps constantly respond to these
its services .. .. But any robot that you or I build and that interacts with these services is "bad" by
default, guilty until proven innocent. 63 movements by measuring and counting them. Moreover, as demonstrated by videos
a. on the website, interfacing with the app becomes a performance. Without question,
the behaviors of the users, who navigate the virtual realm of the app, look bizarre to
bystanders. They "see the user as performing a strange dance," as JODI phrases it.67 As
JODI also argues, however, this strange dance has become an everyday phenomenon
performed by millions of smartphone users who are desperately trying to find a signal
by holding their phone into the air, taking panorama shots, and so on. 68 These strange
behaviors often occur because the apps take advantage of the built-in motion coproces-
sor, which tracks user movement through and collects sensor data from the integrated
accelerometer, gyroscope, compass, and GPS.
Already in 1934, the French sociologist Marcel Mauss observed that the girls in Paris
had started walking the same way as in New York thanks to Hollywood cinema: "The
positions of the arms and hands while walking form a social idiosyncracy, they are not
simply a product of some purely individual, almost completely psychical arrangements
and mechanisms."69 With smartphones, lots of apps (maps, games, fitness trackers,
and other apps in the quantified-self genre) track and control user movement, and the
"social idiosyncracy" is increasingly integrated into the metainterface industry. ZYX
b. might be seen as a critical deconstruction of how such apps interface with users-a
way of exploring their strange dance and how the interface controls their movement.
Whereas the app deconstructs the interface of the iPhone, the website continues
se the Resolution center to correspond with App Review until all issues with
this deconstruction with a view into the normally hidden aspects of the interface. This
our app version have been resolved .
includes not only descriptions and illustrations of the app but also bits of its program
Binary Rejected Mar 1, 2012 03: 11 PM
Reasons for Rejection:
• 10 . 6: Apple and our customers place a high value on simple, refined, code that exemplify how the app has addressed the iPhone interfaces (e.g., the sensors,
creative, well thought through interfaces . They take more work but are worth
it. Apple sets a high bar. If your user interface is complex or less than camera, and microphone). Interestingly, it reports on the interaction between JODI
very good it may be rejected
ar 1, 2012 03:11 PM. From Apple .
10.6
and Apple's App Store as well. ZYX was first rejected because of "poor user experience"
e found the user interface of your app is not of sufficient quality to be and only later approved-this time without further explanation. 70 Both the program
appropriate for the App Store. Apps that provide a poor user experience are not
in compliance with the App Store Review Guidelines.
code and correspondence reveal the existence of other interfaces to the metainterface
specifically, we noticed the user interface of your app is not intuitive and
could not be navigated. For example, it asks the user to turn right ten times.
simply turning the phone to the right did not increase the count. After moving
industry than the app itself. These interfaces are usually not visible to users, but JODI
the phone in several directions it began counting, but repeating the same task
ould only allow the count to go up to 4, at which point it looped back to exposes them as an integral part of the industry's production chain, including how the
rovide the same command to 'turn right ten times' .
Please evaluate whether you can make the necessary revisions to i.mprove the user
app needs to be registered and accepted by the App Store. In this way, they demonstrate
experience of your app.
the bureaucratization of interface culture that follows with the metainterface and its
platforms.
Scourti, a Greek artist based in Britain, also explores how users may express them-
selves and write with interfaces even as they are simultaneously written by the inter-
faces. In this sense, her work is an examination of a process of gendering, individuation
Figure 2.7 and subjectivization in the metainterface industry. Among other things, she has pub-
ZYX (2012) by Jodi. (a) A smart phone app that asks the user to do repetitive tasks in a series (spin
lished the "ghostwritten memoir" The Outage "based on her digital footprints" and
around, jump, etc.). The app over-exposes a normally functional and unnoticed physical relation-
ship between interface and user. Photo by Jodi. (b) ZYX's corresponding website documents the
"told by J. A. Harrington."71 With a feminist perspective, she often focuses on the body
apps hidden interface, including the correspondence with Apple. Screen capture of project web-
site. Courtesy of Jodi.
22:so 22 :50 -, o3%c> I space and time. " 74 Naturally this technological integration of the body is not just some-
< I
Pink Lips [!:: <Back Related Images I thing experienced by the perceiving body but also "a reality of life, which profoundly
I affects the ways female bodies are mediated and experienced." 75
I
> I
Similar toJODI's ZYX, Body Scan demonstrates how the interface has become mobile
i and intimate, and almost a part of the user's body-defining its very mobility and
···: 2 Woburn Place appearance in space. The works indicate how the bodily effect of media technology
'r' London. WC 1H OLH >
~ yelp~: 20 m.:ilci'C?$ 'oo'ld nearby is not limited to the perceiving body but instead is part of a larger media system. Or
Pink lips \ Tips to perfect the pink lip ... put differently, the body is part of a sensorium, and defined by an interface that is at
All 1hri trp!- for achlcvino the pink l1ps ss1:1 beauty
PirJH !lend. plus more beauty idec1s and p1oduct 1e11!ews Ir...
\'. ,\ WJGOO~t"Ol'0!.111
once human and nonhuman, representational and computational, or sign and signal.
3 Ways to Get Pink Lips - wikiHow
Whereas JODI seems to create a distance that reveals the body as almost surreal and
How· lo Get Pink Ups Prmty pmk hps me an atlract,vc
lac1al leature. panicu!arly for girls and wome-n. Yet ms allows for a distanced political critique, Scourti's body appears overly intimate (exposed
and photographed naked at close range), and yet at the same time, her voice is overly
How to get pink lips ..? - Yahoo
Y~ah you c;1n get pink lips easily by honey and suger
Honey moisturi.zes our skin and helps !O p1oduce new ...
distanced from the object. In this way, her work creates a particular humorous experi-
in.11Y..,1;er~;,lhoo>-orn
ence of the machine-body sensorium. She shows how the body is read, profiled, and
The best hot pink lips picked by Red ...
Hot Pink Ups. Om ten best bnght p1rk 1.p51icks and interpreted as a data structure and commercial entity in the metainterface. This writing
g,os~es thai w,n help you mak~ o bold be,1ut\' statcm ...
't<N,...,,l~J!,,.,co.u~ and reading of the body is a dynamic, continual, and endless stream. Like rapture, it
no longer seems to matter whether Google, or any other service providers that employ
Figure 2.8
such scripts, manage to profile the user's body successfully or not. The video displays a
Body Scan (2014) by Erica Scourti. In the video, Scourti uses the app CamFind to photograph parts
of her body and search for similar images on the World Wide Web. In the corresponding audio strange mixture of her body, the way she is read and profiled, and all the cultural preju-
track, Scourti reads from the search results. Courtesy of Erica Scourti. Screen captures from video. dices and commercial models she is subjected to. It thereby illuminates how prejudices,
gender stereotypes, and commercial biases are projected back to her (and everybody)
along with the ways it becomes profiled, programmed, and gendered through, for even through intimate, bodily interaction.
example, Google and smartphones, or how Mauss's "social idiosyncracy" is integrated The body has always been enmeshed in the "new" technologies and economic struc-
into the metainterface industry. In her video work Body Scan (figure 2.8), she uses the tures of its time (as indicated by both Lefebvre, Mauss, and Hayles), and Body Scan
app CamFind to photograph parts of her body with an iPhone and search for similar therefore belongs to a history of works that address the (female) body's representation
images on the World Wide Web. The video work displays the photographed images and in media. Also broadcast media-like film, TV, magazines, and radio-represent gen-
the images returned from the search. In the background, Scourti reads from the search der in particular ways and have mediated the capitalist consumer subject as part of a
results: "Identifying human stomach. A hollow muscular organ. Forms gastric acid. Is it "controlled consumption society" (in Lefebvre's terms). As Scourti explains, Export's
cancer? And, how to survive another human. Human belly. Human belly button. Why performance Tap and Touch Cinema from 1968, in which male participants are invited
do I have a belly button?" 72 to stick their hands into "the cinema" and touch the artist's breast (a box with a cur-
In her thesis "The Female Fool: Subversive Approaches to the Techno-Social Media- tain attached to her chest), reenacts how the female body is objectified and gendered
tion of Femininity," she analyses the work of classic feminist performance artists from by media. Though Export subjects herself to the female image, she makes it literal, too.
1960s until now (Valie Export, Martha Rosler, Pipilotti Risti, etc.), and the role of media This act of designation is in every way humorous, or as Gilles Deleuze describes humor,
technologies in their representation-how their "embodied subjectivity [is] insepara- it works by way of a designation that destroys meaning: she replaces the idea with the
ble from technology." 73 She continues with a quote by Katherine Hayles that almost example, behind which there is no hidden platonic "essence." 76
echoes Mauss: "When changes in incorporating practices take place they are often Whereas Export exemplifies the Hollywood body of advertisement and the movie
linked with new technologies that affect how people use their bodies and experience industi;y, Scourti exemplifies the body of the metainterface industry. It is a body that
The Metainterface Industry
75
74 Chapter 2
not only comes into existence by subjecting itself to a consumer logic that objectifies chapter in their seminal book Dialectic of Enlightenment, written in Los Angeles, while
it (through the movie industry, for instance) but also comes into existence by allowing they were in exile from wartime Nazi Germany. Given both the Hollywood industry
itself to be read as a signal and be part of a metainterface industry's textual machinery. and Nazi Germany's wide use of media to propagate values and behaviors, this seems
If the metainterface industry is characterized by a writing machine that transform the no coincidence. The chapter describes how a rising culture industry's increasing man-
HCI of word processing into a global machinery of text-that is, by a reading machine agement of culture is deceiving the masses through a media system of film, radio, and
that reads how the reader reads this global text, and by a general control of language magazines, which is "infecting everything with sameness." 78 In general, the essay is a
that has little interest in what is written, yet great capital interest in the system that cutting criticism of the US and increasingly global popular culture that the two Ger-
produces text and language- then this also transforms the bodily existence of its users: man Jewish intellectuals were surrounded by in the booming West Coast metropolis,
to come into existence, the user does not need to become a particular consumerist including film, technical broadcast media, and popular music.
body but simply be read as a body. The body in Scourti's Body Scan is thus significantly Horkheimer and Adorno describe the growing importance of culture for under-
different from the mass media body that is stereotypical and inflicted with sameness. standing capitalism; they see culture as an industry that lays the ideological foundation
Though arguably mass media still play a role in defining the body as a consuming for mass consumption.79 And their critique is relentless. The culture industry is built
subject, the neoliberal metainterface body is less dependent on replying to the image around a totalitarian necessity
of a consuming body; instead, its subjectivity is greatly dependent on the body's read- inherent in the system, of never releasing its grip on the consum-er, of not for a moment allow-
ability, which can be measured, calculated, and assessed in a million different ways. As ing him or her to suspect that resistance is possible. This principle requires that while all needs
argued by the French philosopher Bernard Stiegler, the individual is "grammatized" should be presented to individuals as capable of fulfilment by the culture industry, they should
be so set up in advance that individuals experience themselves through their needs only as eternal
and "transindividuated" through what he terms "tertiary retentions" or "technical sup- 80
consumers, as the culture industry's object.
ports of various kinds." 77 Another concept for these technical supports and their ter-
tiary retentions would be the culture industry. In Horkheimer and Adorno's view, the technological development of mass media is
propagating a quantifiable sameness through pacifying broadcasts that eradicates indi-
The New Culture Industry vidual, reflective perceptions and interpretations. The result is a hegemony of capital-
ism and consumerism, and an industrialized lifestyle that integrates work and leisure.
If the metainterface industry represents a new culture industry, it both resembles and Fleeing from totalitarian Nazism, they were faced with a totalitarian capitalistic culture
81
differs from its traditional conceptions. Or put differently, though Lefebvre's critique of industry that manufactured business and consumers as its ideology.
a controlled consumption society applies to a contemporary condition, there are sub- When the computer became a mainstream cultural platform (beginning in the
stantial differences and additions to be made. As argued above, the metainterface indus- 1970s and 1980s with home computers and gaming consoles), the industrial era was
try not only controls production but also distribution and consumption, and it does so allegedly over. As described by Stiegler,
in ways that are not authoritarian. The metainterface's reconfiguration of everyday life A fable has dominated the last decades, and to a large extent deluded political and philosophical
does not have a consumerist imperative but rather-through an instrumentalization thought. Told after 1968, it wanted to make us believe that we have entered the age of "free time,"
of writing, reading, language, and bodies- capitalizes on the very being and behavior. "permissiveness" and the "flexibility" of social structures, in short, the society of leisure and in-
dividualism. Theorised under the name of the post-industrial society, this tale notably influenced
The remaining part of this chapter will discuss these similarities and differences in more 82
and weakened "postmodern" philosophy.
depth, and furthermore reflect on the current conditions of critique in a contemporary
interface culture that bureaucratizes not only culture but also its critique (where critical In other words, the post-Fordist and immaterial information society and knowledge
peer review of cultural software is replaced by protocols and automatization) . economy has become part of the theorization of societal developments in the West.
In 1944, the leading members of the German Frankfurt school, Max Horkheimer Also within cultural criticism, the focus has been more on media theory in the line of
and Theodor W. Adorno, first introduced the term "culture industry." "The Culture Marshall McLuhan's "understanding media" than on understanding the material con-
Industry: Enlightenment as Mass Deception" is a prominent and often-referenced ditions of culture as an industry.83
77
76 Chapter 2 The Metainterface Industry
To many, the modern PC is regarded as a technology to end the standardization manifestations of political opposites" offer no redemption, but "proclaim the same
of the industrial conveyor belt of capitalism. With the networked computer, indus- inflexible rhythm." 86 In popular, industrialized culture, there seems to be no possibility
trial standardization is replaced with the flexible customization and modularization for art to explore a critical tendency. Apart from their criticism, Horkheimer and Ador-
of production processes, and leaky borders between producers and consumers broader no's arguments are tainted by an elitist and culturally conservative contempt for the US
supply and demand-also coined in the business term "prosumption," and promoted culture that surrounds them, and often their writing seems like an antidote to Benja-
87
as a "long tail" theory by former Wired editor Chris Anderson. 84 Industrial passive con- min's dialectical understanding of the relations between art and technological media.
sumption is now regarded as more active, "interactive," "participatory," "produsing," Even though it is tempting to disregard Horkheimer and Adorno's points and move
or "coproducing." Yet as argued in this chapter, this apparent liberation from industrial on to more postmodern media theory, it would miss the opportunity to understand
standardization leads to controlled consumption through a new metainterface industry. the current interface industrialization of culture and how it relates to developments in
The metainterface immediately looks flexible, open, and smart to the users. Never capitalism. There is still reason for critical discussions of the culture industry system,
has so much cultural content been so readily available on so many platforms, but the but there is also reason to critically look at Horkheimer and Adorno's understanding of
amazing efficiency of the new metainterface industry comes at the expense of moni- media and technology.
toring, control, and strict licensing. If the conveyor belt produces standard goods in Moreover, Stiegler acknowledges Horkheimer and Adorno as "at once lucid (if not
big numbers and is relatively inflexible toward individual consumer demands, the new prophetic) and erroneous (if not reactionary)," but he critically points to how Hork-
industry thrives on individual choice, consumption, and coproduction that feed back heimer and Adorno exclude the retentive dimensions of reproductive technologies
through a metainterface monitoring every consumer behavior in detail. Agent-based (tertiary retention) in their own thinking. They do not reflect that "the very possibility
11 88
monitoring loops increasingly replace the conveyer belts, and online apps, music, or of 'culture,' and thus of 'spirit,' relies on technics. Consequently, their own criti-
bookstores and streaming services replace suburban malls. Building a business infra- cism is also technical-for instance, it presupposes the creation of a critical distance
structure around the metainterface allows for a flexible, fine-grained, and intimate frequently related to writing and academia-but their critique does not reflect on the
culture industry that can change its public appearance according to demands and technological system of writing critique or possible new forms of criticism evolving
trends, while at the same time experiment with new commercial models behind the with (and through) technological activities. As Stiegler points out, in relation to their
screen. In this way, the metainterface industry holds the technological infrastructures writing on the phonograph's repetitive reproduction, "Such repetition is possible only
of an experience economy; it holds the technology that can exploit and capitalize through technical recording, only through this technologico-industrial reproducibility
our very experience. With the smart city's urban interface and cloud computing, still that is the objective and infrastructural foundation of Horkheimer and Adorno's analy-
new infrastructures are rolled out to monitor, service, and capitalize on behavior (as sis of culture industries, echoing Walter Benjamin but, despite Benjamin, still failing to
will be argued in the following chapters) . Consequently, Horkheimer and Adorno have think them through." 89 Horkheimer and Adorno fail to understand that new media are
become relevant again. As the British digital media theorist David Berry writes, "The not only objects of perception but also integrated into, and thus part of, perception and
challenge for a critical theory of the digital is to critique what Adorno calls identity memory. Cinema develops a cinematic perception, too, or a "kino-eye" (to paraphrase
thinking and a form of thinking that is highly prevalent in computational rationalities the Soviet filmmaker Dziga Vertov), and Stiegler wonders why Horkheimer and Adorno
and practices." 85 There are, however, also reasons to critique Horkheimer and Adorno's does not take Husserl and Benjamin into account "ten years after Benjamin writes
industry determinism. his famous text, 'The Work of Art in the Age of Mechanical Reproducibility,' whose
immense importance completely escapes them." 90 Stiegler argues with Benjamin that
Metainterface Industry Literacy reproduction does not just create copies of the real; it adds something constitutive to
it: "It is this constitutivity of the technics of reproduction, developed through the cul-
To Horkheimer and Adorno, the culture industry system is "never releasing its grip" on ture industries that, according to Benjamin, confer on cinema, for example, its analytic
the individual but instead renders them "eternal consumers, as the culture industry's force, which goes beyond its power of alienation-a force that seems to completely
object." Resistance is either impossible or will get appropriated. Even "the aesthetic .
escape Hor kh eimer an d Ad orno. ,, 9 1
78
Adamo's lack of understanding of technological perception is already expressed in a ordinary people take up this impoverished language. 96 In a computer semiotic perspec-
letter to Benjamin where Adorno comments on the first version of "The Work of Art" tive, though, the signalization of signification is carried out by signal- computer inter-
essay, and criticizes its positive discussion of popular cinema such as Charlie Chap- action in the metainterface (as argued in chapter 1) whose infrastructures are delivered
lin and Mickey Mouse ("dependent art," in Adamo's words), instead of high art such by a rising metainterface industry. The presence of a linguistic system also implies that
as Franz Kafka and Arnold Schoenberg.92 In the final, published, third version of the there still is a grammar at place (following Agre's "grammars of action") .
essay, this discussion is cut out, according to Adamo's advice. Yet throughout the essay, The grammar of the metainterface works through the automated capturing of the
Benjamin maintains an interest in "redeeming the reified images of mass culture and everyday behaviors of writing, reading, language, and bodies, but the grammatization
modernity for a theory and politics of experience." Hence, the discussion of popular does not work automatically. Rather, it is performed through social, cultural, and semi-
cinema cannot be left out without altering the signification of the essay. It allows, otic processes, which it furthermore performs back to us and reinforces through instru-
for instance, for a distinction between cinemas, where "certain films might function mentalization. Even the code execution in interfaces can be seen in relation to political
as a kind of psychic vaccination," and "the cinema becomes an object-as well as a processes (as the discussion of Nake in chapter 1 pointed out as well). With a reference
medium- of 'redemptive criticism,"' as Miriam Hansen, a US-based scholar of Benja- to Michel Foucault, Chun contends that code execution can be compared to neoliberal-
min, argues.93
ism in the way that it stabilizes and privatizes enforcement as self-enforcing law: "from
Instead of just denouncing technology and media, it can be explored via certain police and judicial functions to software functions, 11 code execution expands judicial
works of art that have a knowledge of their own production, or tendency, in Benjamin's interventions and reduces law "to the rules for a game in which each remains master
terms. Art is subject to the metainterface industry, but art can also critically explore the regarding himself and his part. 11 Chun concludes that "this wish for a simpler map of
metainterface industry. Following Benjamin, the political critique is, however, not just power- indeed power as mappable- drives not only code as automatically executable,
a question of emitting attitudes in the artwork. If art is a probe into the fissures created but also ... interfaces more generally. This wish is central to computers as machines
by technical revolutions, it is not because artists have better insights into society and that enable users/programmers to navigate neoliberal complexity.1197 In this way, Agre,
politics. Rather, it is because artistic production can be a material exploration of its Nake, and Chun all maintain that with interfaces and computation, it is not primar-
own technological means of production, and how these constantly change as well as ily our immediate language that is influenced but instead our abilities to interact with
create crises in corporate relations, the use of language and copyrights, bodily percep- reality through interfaces (e.g., through their combination of HCI and signal- computer
tions, and so on (as demonstrated in the artworks brought forward in this chapter) . interaction in the metainterface) .
To Benjamin, media revolutions lead to fissures in art, and by reflecting its medium To sum up, the assumption in this chapter is that there is a functional basis of the
and technology critically, artistic practice can expose these fissures as a deeper politi- metainterface's production system, and that this industry echoes some of the central
cal tendency within the artwork. Critical and artistic practice is hence more related to features described by Horkheimer and Adorno, including effects on capitalism, culture,
a material exploration of production and technology, than to an abstract ideology or and language. Yet there is also a need to consider what it means when the culture
immediate attitude of the work. Consequently, interface art can be used to delve into industry becomes technologized and digitized through metainterfaces. The postindus-
a critical understanding of technology informed by material exploration and analysis, trial has been followed by the metainterface-industrial culture as a general paradigm,
which is more nuanced than Adorno and Horkheimer's distanced critical perspective.94 and this is explored artistically and critically across platforms. The new metainterface
Despite regarding the political infrastructures of the culture industry as a totalitarian industry, at first glance, seems more open and flexible than the old culture industry
power that mainstreams and standardizes everything through industrialization, Hork- described by Horkheimer and Adorno, but the efficiency of the metainterface indus-
heimer and Adorno also acknowledge that something more fine-tuned and flexible is try comes at the expense of other kinds of control, which are difficult-although not
at stake in the process. At the end of their text, they discuss how the culture industry impossible- to address from within the platforms. In opposition to Horkheimer and
influences language, and argue that by circulating linguistic models, signification is Adorno, theoreticians such as Benjamin and Stiegler argue for a critical, redemptive
completed by turning signs into signals. 95 They do not discuss the technicalities, but potential in certain works of technological art that the artworks presented in this
mainly how advertising, branding, and trademarks influence public discourse, and how chapter'"seem to follow. Through the exploration of tendencies, the artworks discussed
80
Chapter 2
in this chapter demonstrate how the metainterface industry produces control and
3 The Urban Metainterface: Territorial Interfaces
language, and how managing systems of language is an essential part of its business
model. To rephrase Ong, if letterpress printing and the old culture industry embedded
the word and culture in the industrial manufacturing process, making it into a kind of
commodity, then the metainterface industry to some extent frees the word and culture
again, but by capturing the system it commodities and grammatizes reading, language,
and the user's body in return, as shown in the works by Ubermorgen, Cayley and
Howe, JODI, and Scourti. These artworks teach us to read these effects; they train our
metainterface industry literacy.
Spoken Streets
In the app Las calles habladas, or Spoken Streets, by Clara Boj and Diego Diaz (figure
3.1), the user is offered a random map and walking path around their location. The
app seemingly functions like an audio guide, where the user listens to news or facts
about places along the way. Unlike most audio guides, however, the narrative appears
fragmented. The audio track does not contain the voices of the people living there but
rather an automated text-to-speech function. The route planning includes not only a
suggested path but also Google searches related to the location. Each user movement
generates a search and is answered by the reading of debris from the World Wide Web's
enormous body of text: phrases from websites, Twitter feeds, or Facebook events
appear together with symbols, numbers, and URLs. Sometimes there is a direct and
visible linkage to the user's location, as when the user is near a shop that also is listed
on a website. At other times the relation is more abstract, and invites the user to specu-
late on potential narratives, as when a place is linked to a Facebook event or Twitter
feed. On rare occasions, there is only a semantic linkage, as when the name of a street
or shop is common, and perhaps located in hundreds of cities. The relation between
sense and nonsense, between potential narrative and raving incoherent jabber, seems
to be central to the experience of using the app. The confrontation with Spoken Streets
(i.e., how the web speaks the streets) appears absurd to a human being. lt functions
almost like a joke or a parody of the ways that sense making functions in other, more
conventional applications that, for instance, augment reality by linking locations to
the real-time generation of information.
Spoken Streets points to how the city-apart from being a functional entity with hous-
ing, fact~ries, offices, streets, electricity networks, sewage, and so forth-can be defined
as a "semiotization" of space. From the tourist industry's promotional narratives, per-
sonal narratives, and media discourses, to traffic warnings, banners, billboards, street

The Metainterface Part 1 PDF

Uploaded by

Copyright:

Available Formats

The Metainterface Part 1 PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

The Metainterface Part 1 PDF

Uploaded by

Copyright:

Available Formats

Contents

© 2018 Massachusetts Institute of Technology Acknowledgments vii

Art and Interfaces

HYPE ANALYSIS IN ART

ARTISTIC RISK MANAGEMENT TRAINING N;:t.i-J~ B~~;f,"I

As Weise? demonstrates, the interface is undergoing severe changes. The excessive

The Control of Production, Distribution, and Consumption

You might also like