Irvine SSA Peirce Computation Expanded Version
Irvine SSA Peirce Computation Expanded Version
Irvine SSA Peirce Computation Expanded Version
A celebrated treatise [by Hobbes] is entitled Logic, or Computation, and although not
all reasoning is computation, it is certainly true that numerical computation is
reasoning. But calculating machines are in everyday use; and Babbage's analytical
engine would perform considerable feats in mathematics. Other logical machines
have been constructed. All those instruments perform inferences; and those
inferences are subject to the rules of logic. -- C. S. Peirce, “Minute Logic” (1901)
1
cumulative combinatorial development. Further, ongoing research in many
sciences provides cross-domain confirmation for hypotheses for understanding
human consciousness, cognition, and computation as kinds of information and
sign-code processes writ large throughout organic and inorganic structures. In this
emerging view, human semiosis (OS Alpha evidenced in generative sign actions)
underlies the discoverability and interpretability of information structures in all
systems within which we are observer-interpreters.
My argument is also framed by a specific definition of semiotic technologies
as social-technical systems employing material-physical symbolic substrates as
cognitive artefacts. 2 We have a long cumulative history of technologies developed
for supporting meaning systems in our inherited technologies for writing, imposing
images on material substrates, reproducing and storing all kinds of symbolic
representation, sending and receiving symbolic expression in communication
media, and in recent semiotic technologies for digital remediation and combining
multiple levels of abstraction and materialization in automatable information
processes. These material systems constitute a special class of technologies,
symbolic-cognitive technologies, and have always had a special status in our larger
social-technical system of artefacts, machines, and industrial products. Symbolic-
cognitive technologies, of course, require general technologies used in instrumental
ways (e.g., electricity, developments in materials science, component and
supporting technologies), and have been part of long histories of cross-domain
developments, but semiotic technologies must be studied and explained with a
specificity beyond general technologies and all other kinds of machines. Calling
computers or communication devices with microprocessors and network
connections “machines” only reifies them as black boxes and treats computing
devices merely as empirical facts rather than artefacts that can be interpreted,
explained, and understood on semiotic principles.
To reveal the deeper continuum leading to our current semiotic
technologies, I will trace the trajectory of symbolic thought that enabled the designs
for correlating electronics with symbolic values in our era, a trajectory that
connects the ideas of Samuel Morse and C. S. Peirce to Alan Turing and Claude
Shannon, and all the way through the recent development of pixel-mapped
graphical computing interfaces for all sign systems in digital media.
I will be focusing on Peirce’s work from the last productive decade of his life,
1902-1912, when he was working on his major synthesis of “logic as semeiotic.”
Over the past 4 years, I’ve been doing archival research on Peirce’s unpublished
papers from this period. I have discovered new sources for developing a more
complete picture of Peirce’s insights on semiotic technologies and the logic of signs
and operations, which also strongly supports the extensibility of his semiotic
philosophy and his integrated view of the sciences, logic, philosophy, and
2
humanistic fields that have important consequences for our current moment. Had
these writings been recovered and published earlier in the reception of Peirce in
the 1930s-1960s, Peirce and semiotics in general would be in a very different
position today.
For the scope of this introductory paper, I will focus on four related concepts
in Peirce’s theory that are extensible to the physical and symbolic architecture of
computing systems and digital media: 3
1) The material structured substrate function in any sign system: the
necessity of structured physical-material-perceptible substrates with affordances
for inferring significant features (sounds, substrates for visual representations,
materials for 3D artefact, etc.). The substrate function is a necessary condition for
signs and symbols to be instantiated as such, and thus forms a continuum across
physical media at any historical moment of development.
Peirce re-described the necessary material-perceptible structures of signs
in three phases of his philosophy, first as feature-bearing Representations, then as
Representamens (whatever performs the function of representing) that supply
percepts correlatable to objects and interpretants, and then simply as the material-
perceptible feature-bearing component of signs, the technically mediated forms of
which can produce actions and systemic agency. All sign systems are unified by
requiring a structured, symbolically motivated, material substrate capable of
eliciting percepts by sign-cognizers in a community. Structured semiosic substrates
are required for supporting feature extraction and pattern recognition that enable
semiotic agents to “go beyond the information given” (Peirce and Bruner) by
enacting the first level of correlations that enable us to convert percepts into signs.
(The contemporary development of fields devoted to signal detection, feature
extraction, and pattern recognition in information and computer sciences confirms
the necessity of this structure.) The material sign substrate, which can be modelled
as a stack of material-perceptible features enabling perceptible and
phenomenologically based percepts to be interpreted as, or “resolve into,” tokens
(specific instances) of symbolic types (whether as sequential strings or
compositional clusters). 4
2) The function of indexical signs (indices) in the combined material-
symbolic-cognitive processes of semiosis (sign actions as a dynamic processes over
intervals of time) in the overall symbolic architecture of computation and digital
media objects. Computation and digital media processing are designed to depend
on many levels of indexical mappings and physical retokenization across
substrates.
3) Extending Peirce’s well-known Type/Token distinction (abstract forms
generative of replicable material instances) as used in computation for ongoing re-
tokenization of forms in and across physical substrates. The type/token distinction
3
and the materially correlated index provides a model for the decomposability and
re-composability of triadic sign structures in digital electronic states, and for
mapping pixel arrays to and from memory states.
4) Sign agency delegated to automating two classes of sign structures
(symbols that mean and symbols that perform operations, data structures and
program structures) in a common representational form (binary, base-2 number
values correlated to electronic states) and internal substrate (digital electronics).
The understanding and specification of different classes of signs and sign functions
has a long history, which I cannot outline here, but the key to understanding
automatable symbolic processes, as Peirce discovered, is seeing that what can be
represented as formally necessary can, in principle, be automated. Mathematics and
logic provide methods for generalization and abstraction of invariant patterns, we
can formally designate symbols and operations in signs for meanings and signs for
operations that yield necessary logical results within a formal sign system, and the
merging of these in necessary processes for enacting interpretations and
transformations across sign structures is the foundation or platform for
automatable semiosis.
The explicit understanding of signs and operations was central to 19th
century mathematics, especially as implemented by Boole and Babbage, and Peirce
was a major contributor to the tradition of Boolean mathematical logic that
Shannon, Turing and modern computational theory is founded on. As George Dyson
described the ideas behind modern computing, “The stored-program computer, as
conceived by Alan Turing and delivered by John von Neumann, broke the
distinction between numbers that mean things and numbers that do things. Our
universe would never be the same.” 5
The trajectory described by Dyson began in Peirce’s era: an algebraic system
for representing sign values and operations, and definitions for correlating sign
meaning and sign actions or agency were at the center of Peirce’s later semiotic
philosophy, much of which was inspired by Boole’s algebra of logic. In modern
computation, binary numbers represented in symbol units can be assigned two
functions: value-representing symbols (units of meaning representations) or
operation-and-relation symbols (symbols representing rules and mathematical-
logical functions, patterns of interpretation assigned to actions, transformation
operations, over the value-representing symbols in the computing architecture).
Computing, at whatever scale or degree of complexity, implements logical
architectures for combining symbols that mean (hold values) and symbols that do
(invoke actions and operations), both simultaneously communicated and
differentiated throughout the same physical, electronic substrate.
C. S. Peirce is in the middle of the story of these foundational ideas,
historically and intellectually. Peirce provides the most usefully heuristic and
4
extensible set of terms and concepts for describing sign functions as correlated
material-cognitive processes that occur as bundles of symbolic representations,
operations, and time transitions, bundles which are themselves symbolically
representable in physical media. 6 However, recovering Peirce’s ideas from his final
formulations and syntheses is not simply an archival or historical project: Peirce
provides us with a rich and still unsurpassed vocabulary for describing the semiotic
structure of computation and digital representation. As John Sowa, the CS pioneer
who championed Peirce’s logical graphs as models for knowledge representation
put it, “Peirce improved on his successors.”
We will begin with two exemplary cases in the foundations of electronic
communication and in the proto-history of computing -- Samuel Morse’s
electromagnetic “system of signs” and C. S. Peirce’s diagrams for electrical logic
circuits using Boolean algebra -- and then consider Peirce’s treatment of sign
processes in a framework for understanding computation as a design for
automatable semiosis. The proto-history of electronic computing began when
electrical circuits were first conceived as structures for symbolic correlations—
sign abstraction and retokenization (Morse) and relations in an algebra of logic
(Peirce). The work and careers of Samuel Morse (1791-1872) and C. S. Peirce
(1839-1914) span the great “century of invention” for the semiotic technologies
that are still with us, and they represent important nodal points in the history of
modern computing, nodal intersections with links to George Boole and Charles
Babbage in England, to the developments in electronics by Claude Shannon, and to
the traditions in logic and mathematics used by Alan Turing. Nearly everything we
take for granted in modern telecommunications, computation, and semiotic
technologies has foundations in this era.
On his return home to New York from Paris in 1832, the same year that
Charles Babbage began designing his first Difference Engine, Samuel Morse was
preoccupied by two concerns. He was carrying with him his famous metapainting
of The Gallery of the Louvre, which he hoped would confirm his career as a history
painter and mediator of European high culture, and he was also thinking about an
idea for controlling an electrical circuit designed to represent a sign system for
sending messages across any distance. 7 Morse began his career as a painter after
graduating from Yale, and his knowledge of what was happening in the sciences,
begun while he was a student, also made him fascinated about electricity and
electromagnetism. On this now-famous ship voyage in 1832, Morse recounted how
he first connected the ideas for his model of the electromagnetic telegraph [image
of painting and text parallel]:
5
traveled through the whole circuit in a time not appreciable, but
apparently instantaneous. It immediately occurred to me that, if the
presence of electricity could be made VISIBLE in any desired part of this
circuit, it would not be difficult to construct a SYSTEM OF SIGNS by which
intelligence could be instantaneously transmitted. The thought, thus
conceived, took strong hold of my mind… and I planned a system of signs,
and an apparatus to carry it into effect. 8 [Emphases as in original text]
[Morse’s biographer continues:]
The spark shall be one sign; its absence another; the time of its absence
another. Here are three signs to be combined into the representation of
figures or letters. They can be made to form an alphabet. Words may thus
be indicated. A telegraph, an instrument to record at a distance, will be
the result. Continents shall be crossed. This great and wide sea shall be
no barrier…. As he sat upon the deck [of the ship] after the conversation
at dinner, he drew from his pocket one of [his sketch books], and began
to make marks to represent letters and figures to be produced by the
agency of electricity at a distance from the place of action. 9
6
telegraphic ticker tape systems, forms of electromechanical teletype in the 20th
century, and, more recently, logic switches implemented in transistors, byte code
standards for ASCII and Unicode characters, and the entire structure of binary
electronics are direct descendants of Morse’s proof of concept.
In this foundational moment for electronic communications, we can
reinterpret Morse’s discovery in Peirce’s terms: using electricity as the agency of
sign actions, the units of electricity “made visible in any desired part of [the] circuit”
could be organized in rule-governed patterns for re-tokening the sign values of
typographic symbols in a physical substrate, a substrate both materially and
temporally different from the message units themselves, whether written on paper
or spoken orally for encoding. By abstracting sign types (the set of alphanumeric
characters and symbols) from specific material token instances, Morse discovered
a way to decompose and recompose signs by translation into electro-magnetic
structures for interpretable sign units. The Morse implementation is the first
extensible method for using patterns of electronic signal switches as indexical signs
for symbolic tokens, by causing electro-magnetic actions in telegraphic devices
wired to send, receive, and register the patterns of switched current; that is, a
design for instantiating sign types by de- and re-composable physical electronic
structures mappable to n-instances (tokens) of typographic symbols, which, by
concatenation, represent the token strings of the interpretable message units that
motivated the transmission.
I would like to point out some of Peirce’s revealing comments about
telephones, telegraphs switches, and logic machines as a context for reconsidering
his semiotic philosophy for modern computing systems.
Peirce was born in Cambridge the same year that Morse met Louis Daguerre
in Paris (1839). Morse brought the method of Daguerre’s photography back to New
York and set up a famous studio there. Peirce’s father, Benjamin Peirce, was a
leading mathematician and scientist and well-connected to Brahmin society. 14
Peirce was no armchair academic semiotician. His semiotic philosophy developed
from several decades of work in science, mathematics, logic, and the history of
philosophy, and began his scientific career at the Harvard Observatory and in the
US Coastal and Geodetic Survey, of which his father was director. He lived through,
experienced, and participated in the scientific and technical advances of his own
era (electricity, telegraph, telephone, logic machines, and early computing
machines). By the time Peirce was a student at Harvard, an international
telecommunications network was already in place (he sent and received telegraph
messages often), and telephones were beginning to take over the telecom network
that was already in place. He attended, with his father and the intelligentsia of
Boston, Alexander Graham Bell’s famous demonstration of the telephone -- “voice
over telegraph” -- in Boston in 1877. Peirce’s first job was as a “computor,” that is,
7
someone skilled in making exact mathematical calculations for logarithm tables
and other kinds of scientific measurement. His father was head of a committee that
bought and brought a Scheutz difference engine, a successor design after Babbage,
to the observatory in Albany to aid in astronomy calculations.
After around 1903, Peirce redefined his project of “logic as semeiotic” to
include sign systems in all kinds of material or technical mediations. In 1904, he
stated: “ideas cannot be communicated at all except through their physical effects.
Our photographs, telephones, and wireless telegraphs, as well as the sum total of
all the work that steam engines have ever done, are, in sober common sense and
literal truth, the outcome of the general ideas that are expressed in the first book of
the Novum Organum (MS 774, 1904, “Ideas, Stray or Stolen;” EP 2.327). (Peirce is
referring to the treatment of inductive logic and scientific method in Francis
Bacon’s famous work.) Earlier in this paper, Peirce states:
8
Every thought, or cognitive representation, is of the nature of a sign.
“Representation” and “sign” are synonyms. The whole purpose of a sign is
that it shall be interpreted in another sign; and its whole purport lies in the
special character which it imparts to that interpretation. When a sign
determines an interpretation of itself in another sign, it produces an effect
external to itself, a physical effect, though the sign producing the effect may
itself be not an existent object but merely a type. It produces this effect, not
in this or that metaphysical sense, but in an indisputable sense….
Consequently, the whole purport of any sign lies in the intended character
of its external action or influence. Some signs are interpreted or reproduced
by a physical force or something analogous to such a force, simply by
causing an event; as sounds spoken into a telephone effect variations or the
rate of alternation of an electric current along the wire, as a first
interpretation, and these variations again produce new sound-vibrations by
reinterpretation. Another case is where a sign excites a certain quality of
feeling, simple or complex, which quality of feeling is a sign of anything that
partakes of it, as the sound of the word “red” may make us imagine the color
red. (MS 1476, 1904, pp. 5-6; p.4, second sequence of drafts.)
There are several drafts of this argument in the manuscript; in another, probably
earlier, draft Peirce words it this way:
In this fascinating text, Peirce describes how the design for using electrical energy
states generated within the technical system itself are assigned semiosic actions
directed toward physical interpretability (reproduction of voice sounds). In a letter
to Victory Welby in 1909, Peirce explains his kinds of interpretants: “My Immediate
Interpretant is implied in the fact that each Sign must have its peculiar
Interpretability before it gets any Interpreter. My Dynamical Interpretant is that
which is experienced in each act of Interpretation and is different in each from that
of any other; and the Final Interpretant is the one Interpretative result to which
every Interpreter is destined to come if the Sign is sufficiently considered. The
9
Immediate Interpretant is an abstraction, consisting in a Possibility” (Welby
Correspondence, March 14-15, 1909; SS 108-119). This seems to be the way he
would frame the teleological semiosis in the electrical transmission of interpretable
signals, the very structure which contemporary electronic digital computing
exploits at any scale.
10
I think you ought to return to the problem [of the design for the logic
machine], especially as it is by no means hopeless to expect to make
a machine for really very difficult mathematical problems. But you
would have to proceed step by step. I think electricity would be the
best thing to rely on.
Peirce proposes using electromagnetic circuits like those used in telegraphy for
Boolean logic gates that map symbolic logic onto the states of the circuit
(open/closed). Peirce combined the technologies of the telegraph, logic machines,
and Babbage’s engines redesigned with an electrical mapping of Boolean algebra
for what could have been used for an electronic analytical engine. Allan Marquand
did work out an electrical switch design for his logic machine (though not
implemented), 19 [diagrams] and we can wonder why Peirce didn’t continue this
development further. Peirce’s papers are scattered with notations and calculations
in binary mathematics, and he even developed a binary cypher code for encrypting
telegraphic signs around 1902. 20 [Image of MS page]
Peirce considered all machines to be logic machines since they implemented
abstract concepts and regularized natural forces through mathematical
engineering principles. Peirce wrote in 1906:
11
whatever is a logical machine working out incessantly new conclusions
from premisses as long as the steam is kept up. (MS 498, 1906, p. 883)
In his many references to logic and reasoning machines, Peirce is clear that
automation is possible if machine signs could be assigned logical self-control, that
is, rational, intentional regulation:
it is quite true that we cannot make a machine that will reason as the
human mind reasons until we can make a logical machine (logical
machines, of course, exist) which shall not only be automatic, which is a
comparatively small matter, but which shall be endowed with a genuine
power of self-control; and we have as little hopes of doing that as we have
of endowing a machine made of inorganic materials with life. Indeed, it
shall be shown in a future article that these two attributes, growth and
self-control, are confronted with closely analogous difficulties, and
further that if we could endow a system of signs with self-control, there
is very strong reason to believe that we should thereby have conferred
upon it a consciousness… (MS 283, 1906, “The Basis of Pragmaticism,”
EP 2.387)
12
Over the past 15 years, there has been an explosion of interest in Peirce’s
work across many sciences and disciplines, and I couldn’t hope to summarize it
here. Peirce’s triadic models of sign functions and progression of interpretant
processes in semiosis are also well-known, and I won’t rehearse the foundational
assumptions here. 23 Rather, I’d like to propose a new synthetic model of sign
processes based on his later writings and unpublished comments on reasoning
machines. This is an interdisciplinary work in progress for developing a new
heuristics for a detailed semiotic description of computing architectures, digital
information and media, and the design of our main semiotic interface, the graphical
user interface (GUI) in pixel grid screens and displays.
All diagrams of semiosis fail unless they capture durations and sequences in
time, but this is a snapshot view of the stacks of processes that Peirce describes in
various ways in his mature writings:
13
Figure 1
Sign/semiosis: From percepts and reactions to sign-mediations
14
Figure 2
A Model of Peirce’s Semiosis Extensible to Computing
15
is, feeling or the peculiar common quality of all our feeling should be
concerned. But it is necessary that there should be two, if not three, quasi-
minds, meaning things capable of varied determinations as to forms of
the kind communicated.
As a medium the Sign is essentially in a triadic relation, to its Object which
determines it, and to its Interpretant which it determines….
That which is communicated from the Object through the Sign to the
Interpretant is a Form; that is to say, it is nothing like an existent, but is
a power, is the fact that something would happen under certain
conditions. (MS 793, 1906, pp.1-3)
In this period, Peirce frequently elaborates on the sign as medium and how sign
effects can be described as determinations on quasi-minds. 24 This description is
really one or two steps away from describing a programming language and designs
for allocating memory for operations on sign units.
16
individually and distributed in groups) using software in common, and between
individual cognitive agents and the whole computational substrate for
representing signs and symbols and receiving actions for operations. 25
Peirce correctly observed that the physical-perceptible ground of any kind
of sign or symbol is revealed in the capability and necessity of repetition—iteration,
replication, re-presentation in any material medium providing affordances for
physical realization or instantiation. The repetition in iterable patterns is also a
function of the bi-directionality of token to type mappings, either in interpretation
or realizations in new expressions. Considering how we begin interpreting signs
through a representamen (whatever functions as an interpretable form):
The mode of being of a representamen is such that it is capable of
repetition. … It is the same with a diagram or picture. It is the same with
a physical sign or symptom. If two weathercocks are different signs, it is
only in so far as they refer to different parts of the air. A representamen
which should have a unique embodiment, incapable of repetition, would
not be a representamen, but a part of the very fact represented. This
repetitory character of the representamen involves as a consequence
that it is essential to a representamen that it should contribute to the
determination of another representamen distinct from itself. (MS 312,
1903, Fifth Harvard Lecture, EP 2.203)
Indices
Peirce expanded the concept of the index, or indexical function, throughout
the last decade of his life. He described many kinds and subclasses signs used with
an indexical function: (1) deictic signs (pointing and indicating through gestures
for joint and focused attention; linguistic indicators in context like calling out “hey”
or “over there;” personal pronouns that reference or indicate antecedents). 26 (2)
Causally connected correlated indications, either in humanly designed instruments
or as inferences drawn from regularities in nature (thermometer gauge read as
temperature in a scale of values; smoke fire; row of bricks stood on end
communicating a “domino effect” action, falling brick prior brick). (3) Abstract
and conceptual signs used in mathematics (the conventional letters used to label
the angles of a triangle, letters and notational symbols used in any formal system).
(4) Maps, which are designed to have indexical, iconic, and symbolic functions.
Peirce also included expressions in the imperative mood as examples of
indexical sign agency: “Imperative signs have the person addressed & his conduct
as their field of interpretation. The action sought to be produced is the
Interpretant.” (MS 339, 1905, f. 254r). The meaning of expressions like “do x,” (e.g.,
“stop,” “close the window,” “please be seated”), which normally assume one or
more addressees, is taken as a signal for a kind of action; that is, we translate the
17
sign tokens through the semantic-symbolic gateway into indexicals for actions
beyond a semantic value (EP 2.288, 1906). In interactive computing, imperative
sign structures underlie all interface components down though many linked
software layers. “Do” is a universal software program imperative function with
many forms.
Indexical sign uses require an interpretive correlation, interpretants, which
are usually handed off to symbolic interpretation. A representation on a
thermometer or car speedometer can’t mean anything (cannot be interpretable)
without a semiotic agent, Peirce’s quasi-mind, supplying learned rules of
correspondence which involve symbolic values correlated to the representations
for states or conditions indexed.
In an unpublished draft of a dictionary definition on “Exact Logic,” Peirce
describes how instruments combine indices, icons, and symbols:
18
(Figure, Robinson Anemometer)
In Peirce’s logical scheme for classes of signs in 1903-04, he explained how some
indexicals have cross-mappings to other sign functions. Indices can be reflexive,
“An Index can very well represent itself. Thus, every number has a double; and thus
the entire collection of even numbers is an Index of the entire collection of numbers,
and so this collection of even numbers contains an Index of itself. But it is
impossible for an Index to be its own Interpretant” (MS 478, 1903, EP 2.276). What
Peirce termed a “Dicent Sinsign,” a single information-announcing sign, affords
information of an actual fact (like weather vanes and all measuring instruments),
by using both iconic and indexical functions (resemblances and interpretable
correlations of value) (MS 540, 1903, EP 2.294; MS 800, p.2, 1903).
Peirce also described the agency of signs and sign actions. “The agency of
the sign” results from the perceptible-cognitive correlations that generate
Interpretants as actions, mental and physical, and further developed responses
from enacted behavior. 27 “[W]e may take a sign in so broad a sense that the
interpretant of it is not a thought, but an action or experience…” (1904, Welby
Letter, CP 8.332). Recall Peirce’s description of interpretants in the variations of
current in telephone voice signals. Peirce held that self-controlled thought and
reasoning – logic and mathematics – are actions with and in sign users, and he also
understood sign agency as parallel and isomorphic actions delegated to logic
machines and any device designed to register physical states.
19
vocal utterer, but putting forth the sign in any way), the other the sign’s
interpreter. Indeed, a mind may, with advantage, be roughly defined as a
sign-creatory in connection with a reaction-machine. (MS 318, 1907, from
version f., sheets 18-22, MS sequence 425-434, p. 18, seq. 425)
“Creatory” is here used with analogy to repository, and refers to how any “quasi-
mind” develops meanings from sensory information by adding Interpretants.
Computer science has a rich repertoire of “index”-like terms and metaphors,
such as index registers, addresses, indexed variables, pointers, calls, lookups,
directories, file names, and many other related concepts. “Index” is a common term
used in databases, allocations of memory (hardware), and references to logical data
“positions” in programming and database code (software). In Web search engine
schemes, an index is the complete table of Web addresses that can be pointed to in
search results. A URL or Web address is an index to a file location on Internet-
connected server (real or virtual).
Peirce explains the future conditional of interpretants when a semiotic
agent is situated to develop higher-order general concepts from compound
structures or sequences of indices and icons representable in further sign patterns.
The being of a symbol consists in the real fact that something surely
will be experienced if certain conditions be satisfied. Namely, it will
influence the thought and conduct of its interpreter. Every word is a
symbol. Every sentence is a symbol. Every book is a symbol. Every
Representamen depending upon conventions is a symbol. Just as a
photograph is an index having an icon incorporated into it, that is,
excited in the mind by its force, so a symbol may have an icon or an
index incorporated into it, that is, the active law that it is may require
its interpretation to involve the calling up of an image, or a composite
photograph of many images of past experiences, as ordinary common
nouns and verbs do… (MS 492, c. 1903, “Logical Tracts, 2, ‘On
Existential Graphs, Euler’s Diagrams, and Logical Algebra’.”) 28
Semiotic agents are inference engines for associating sign token features
with multiple levels of abstraction in sign types. Peirce’s distinction between type
and token is now commonly adopted in various ways in philosophy, linguistics, and
computer science. Tokens are the material-perceptible instances of signs and
symbols, and, as such, exist in unlimited repetition, iteration, replication, or
reproduction. As Peirce summarizes in 1906:
20
By a Type, I mean a general form which can be repeated indefinitely and
is in all its repetitions one and the same sign. Thus the word the is a Type.
It is likely to occur over a score of times on a page of an English book;
but it is only one word twenty times repeated. The distinction between
a Type and a Token is obvious. (MS 399, 1906, f. 276r [dated in Peirce’s
hand “1906 April 2”])
21
automated sequences over intervals in time. The semiosic processes of operations
is marked by symbols representing logically necessary relations among and between
other symbols that determine outcomes. 30 The semiotic leap in electronic digital
design is found in directing the correlations between ways to represent the
necessary rule-governed transitions of logic and mathematics (the building blocks
of algorithms) and embody them in completely regular and consistent time-
stepped transitions in the discrete architecture of a computing system. The whole
physical architecture is predicated on one macro design concept: using symbols-to-
operations correlations for generating interpretations on symbols-as-information.
This is more precise way of describing what we call the distinction between data
(or “content” representations) and code (software).
To make executable code from the texts of our high-level programing
languages (in, say, Java or Python today), we write interpretant programs,
compilers, translate the written text of a program into binary machine code
(executable, “runnable” code physically indexable to memory locations). Compilers
are metaprograms, sequences of abstract interpretants designed to direct the
semiosic process for mapping one set of symbolic representations into others that
can physically enact the rules-to-actions transitions in a physical machine. As Peirce
noted, the interpretants of one class of signs can be in another class of signs.
Likewise, an algorithm is a model of an interpretant machine, an abstract
symbolic representation of these two kinds or levels of code and encoding. An
algorithm is an abstract structure of relation-making and necessary entailment
requiring sequences of time, state transitions, to transform input representations
into output representations. Algorithms are metasymbolic at high orders of
abstraction, and they must be designed to do what’s computational possible in
finite time. Since these abstract models represent patterns of necessary transitions,
they can be automated.
22
and standard for all digital media have enabled us to design a meta summation of
all prior 2D sign systems, a substrate for the history of representational substrates.
The orchestrated complexity of abstraction layers in contemporary
graphical systems sum up into pushing interpretable pixel arrays out to user-
interpreters who motivate the semiosic loops and cycles of interactive computing
that output displayable/playable digital media artefacts with their many symbolic
dimensions. The visible tokens appearing in our physical screen substrates are
composed of observable and unobservable material display layers, unobservable
electrical current that powers the pixels, and many equally unobservable layers of
software for maintaining indexed memory locations and pixel coordinates in the
architecture of the computing device.
Today’s standard interactive GUI pixel grid is designed as a Janus-face bi-
directional interpretant, displaying software outputs of interpretable
representations and for communicating human agency and intentions to other
software interpretants that mediate agency to many levels of the physical
architecture. 32 The pixel grid in any device screen is thus not only an interface in
the ordinary user-facing GUI sense; it is an indexical interface to the whole
computing system designed to produce outputs as projections into screen pixel
arrays for interpretable token instances of sign types and complex combinations of
all representable symbol structures. Software and systems design return
interpretable symbolic clusters for which one kind of interpretive response is an
action with the interface for both inserting new representations (texts, images, etc.)
and directing actions into the cycle of background computations in the software
layers that chain indices and abstract icons for internal interpretants and returning
further symbolic representations.
A grid is a well-known mathematical object of a type Peirce frequently
explained: a grid represents a finite section of the infinite 2-dimensional plane
mapped out in x/y axes with specified scales of units. We can make an iconic sign
of the abstract mathematical relations in a graph (points or locations on an x/y axis,
like lines on a sheet of graph paper), but the abstract iconic pattern is useless
without corresponding indices to physical locations that produce arrays of
symbolic patterns. A pixel grid is translated from an abstract icon into indexical
signs with potential symbolic value precisely when activated as an assignment of
physical indices in a screen material substrate (like the standard 1920 x 1080 pixel
dimensions for an HD screen, image, or video frame). A pixel is yet another
extrapolation of Morse’s “electricity made visible in any desired part of a circuit”
with assigned values. A computer screen, and any defined section of it (e.g., a
“window” displaying a photo, a chunk of text, or video frame) are pixel arrays
representing a few million measurements of the colors and intensity of visible light
assigned to coordinate locations in the x/y axis. (A virtual z axis can be added to the
23
grid for simulating 3D effects, objects, shadow effects, and window layers.) A
specific device screen with its aspect ratio of pixels is designed as a tokenization of
a mathematical grid linked to the physical architecture of graphics controllers and
memory combined with the software interpretant layers that project and manage
the final assignments of specific coordinate locations for rendering the
interpretable pixel arrays.
As Alan Kay and early designers of screen interfaces discovered, the
computer screen with underlying software controls could become a metamedium,
a medium or substrate for representing and interactively transforming other
media. 33 The GUI platform design could only have developed at the moment of
digital convergence, when graphical software could be designed to create a
summation and integration of the whole history of human sign systems (translated
to their digital types) on 2D substrates, especially those using the affordances of
paper. The early designers of the GUI interface features at Engelbart’s lab and Xerox
PARC were focused on the engineering problem of simulating the representational
features of paper as a 2D substrate in the raster display. The main designer of the
influential GUI interactive system, the Xerox Alto, states: “Only one technique is
known for approximating all these properties of paper in a computer-generated
medium: a raster display in which the value of each picture element is
independently stored as an element in a two-dimensional array called a bitmap or
frame buffer.” 34 The term “bitmap” refers to the design technique of “mapping” the
visible pixel array to a chunk of contiguous memory (in bytes). The bitmap-to-pixel
array is thus a two-way index, depending on what view of the system you are taking
(the visible screen or the memory cells assigned by software).
The whole system design -- from the level of representations interpreted by
users as cognitive agents to the unobservable symbolic processes instantiated in
the physical substrates of a computing device -- is a model of multidimensional
semiosis in distributed and delegated processes with physical substrates: sign
processes, sign agency, and sign tokens in materialized instances enacted through
multiple interpretant pathways correlated to types of symbolic relations through
physical intervals in time. Interactive computing exemplifies Peirce’s notion that
interpretants are always in the future, projections of past and present
configurations by signs mediating into a potential future when conditions are such
that sign agency can be completed in an interpretant response. As we say, it’s
semiosis all the way down.
The GUI interactive semiotic leap was discovering that since software can
control both side of the bitmap, any coordinates of the x/y locations of the pixel
grid can be configured with recognizable indexical, iconic, and symbolic functions,
and also defined in interpretable contexts as interactive “channels” for
communicating user intentions for ongoing software processes (links, icons, or any
24
pixel region designed for pushing pixels). 35 Identical designs for actions are also
directed touch screens: the “presence of electricity made visible in any part of a
circuit” is now reengineered to be correlated to human touch, movements, and
gestures. The human body itself is the source of voltage changes in the electrical
capacitance layers overlaying the pixel coordinates. Our fingers modify the
electrical charges running across a grid indexed to the pixels that are in turn
indexed to symbols-that-mean and symbols-that-do.
In our current system architecture with pixel-based screen, all screen
representations are interpreted, rendered, as graphical information regardless of
the data types and background processes that have created their outputs. I find it
conceptually useful to consider the arrays of pixels activated in our screen
substrates as patterns projected from the system’s graphics processor. A type
“font” is a digital file in memory with encoded definitions for correlating abstract
character code values (typically Unicode vales) to stylized pixel arrays interpreted
through the graphics processor from other software layers. In semiotic terms, a
computer “font” is a set of iconic types (the abstract, platform-independent
definitions) designed to be interpretable for assignment to memory indices and
translation, in any software context, to pixel token instances. Our interpretation of
displayed token instances of pixel patterns enacts the standard semiotic feature
extraction to pattern recognition process for inferring regularities and continuities
as types. We find many internal indexical structures and software interpretants for
pattern correlations rendered as token instances of the characters in the indexed
typographic “font” style. (It’s revealing that both Apple and Google are recently
reinforcing the 2D paper and materials metaphors in standards for app interface
designs.)
In short, we can split signs: re-tokenize across substrates, decompose
features, distribute tokens as objects filling variables in software process, and
recompose units in their combined indexical, iconic, and symbolic functions in
display interfaces. Peirce frequently discussed derived and reduced signs (using
the term “degenerate” from geometry). Signs can be derived from, split off from,
complex symbolic structures to form simpler signs like the operations in geometry
for representing 2D planes as intersections or slices of 3D objects; that is,
decomposing a sign structure into simpler constituent elements open for
recomposition in an interpretation in another form. For example, it’s easy to trace
a line drawing from a photograph, by hand or in software, and form a simpler
contour drawing minus the photographic information. (Andy Warhol’s many
screen print images derived from photographs are just such a technique, prompting
multiple interpretant paths when viewed as paintings, prints, and photo-derived
images.) A line drawing or monochrome print is not a photograph, but it can
preserve interpretable (and re-composable) structures by feature selection.
25
Indexical signs can be decomposed (lose a present mental correlation) into
motivated signals and still function as signs by interpretive recomposition. In
computation and digital media, we split and transcode signs as structure-
preserving morphisms, in Joseph Goguen’s category-theoretic terms, translations
that preserve the features of symbolic potential for recomposition in other defined
interpretive contexts. 36
This splitting of sign indexical components from more complete sign
structures yields “a representamen which represents a single object because it is
factually connected with it, but which conveys no information whatever” (1903, EP
2:171–172) (i.e., no information in the Peircean, not Shannon, sense). Peirce’s
terms usefully describe how we can map binary patterns, units of abstract types of
mathematical values, “factually” onto electronic states in actual physical
architectures; that is, we can explain semiotically how and why mathematical
values can be correlated to patterns of physical voltage states that are in turn
interpretable as units of symbolic value when supplied sign-constructing
interpretants. Physical states in themselves are not signs, but can represent them
by instantiating indexed patterns in a computing substrate (memory locations and
transition states in processing), patterns which are held as recomposable in pixel
representations of symbolic, interpretable units. This intra-indexical structure is
required and implemented millions of times in any software process at multiple
meta and physical component levels (like registers and operating system
controlled assignment of memory space and dedicated memory in graphics
microprocessor chips). The specific computing product implementations that we
use are based on multiple combinatorial layers of sign functions and the ability to
use indexical functions for producing actions in a system through a cascade of
linked software processes which return recomposed symbolic representations.
Digital user interfaces to the internal computational symbolic states are thus
designed to correlate meanings, actions, and material structures. Computation and
correlated interfaces allow us to enact the dual functions of semiosis by specifying
symbols that mean things and assigning actions to symbols that do things, in which
the combined process, over any number of iterations, is designed to return
potentially unlimited chains of further interpretable symbolic representations.
Programming, software, and digital media object representations are based
on recursively chaining indexical and iconic signs split off from composite sign
structures (e.g., letter forms, graphic elements, sections of a digital photograph)
into structure-preserving states that can be recomposed and completed as unified
symbolic structures by human cognizers supplying interpretants for recognizable
representations. Human symbolic cognition always holds half the program, as it
were, and motivates both the design architecture for, and creation of, individual
token instances in and across the physical substrates. In all programming and
26
digitization methods, these electronic, material sign components in binary
representations follow end-directed logic for “returning values” as interpretant
outputs in the symbolic representations projected into the grid locations of our
interpretable pixel arrays. (Again, combining symbols that mean and symbols that
do, signs for cognitive-abstractive-conceptual meanings and signs for actions).
By extending Peirce’s descriptions of the material-conceptual correlates in
symbolic activity, we have a more complete set of concepts for productive
explanations of computation and digitization as designs for mediated symbolic
action through motivated sign distributions in re-presentable instances across
multiple material substrates in 2, 3 and 4 dimensions. Since both individual sign
units and complex symbolic structures (e.g., connected discourse, musical forms,
images, film) are decomposable and recomposable at multiple levels of constituent
forms, we are able to produce an unlimited series of retokenizations (new instances
of symbolic types or sets/classes of types in other material forms) across
instantiations in different material substrates in time and place, the whole
architecture of networked and distributed computing.
Conclusion
27
mathematical, and scientific bases of symbolic thought for reappropiating
computation, software, and digital media in the context of human interests.
The deeper origins of computing emerged from a synthesis of thought and
human interests that knew no boundaries or partitions in knowledge domains,
whether cultural, philosophical, mathematical, or scientific. “Machine” is only a
metaphor for automation, not a reduction to instrumental mechanization.
Computation and code for automated semiosis use general technologies
instrumentally in actual computers for implementing sign processes, but
computation as a modelling system for embodying semiotic actions and symbolic
media representations is a symbolic-cognitive artefact, not a machine product.
By repositioning symbolic mediation and the computational automation of
symbolic actions in this longer continuum of cumulative cultural, cognitive,
material, and social implementations, we have a far more productive way to
embrace and incorporate computational knowledge in research, theory, and
practical expertise in all disciplines. Further, this repositioning enables non-
technical people to reclaim ownership over our cognitive symbolic technologies,
not as alienating machine products controlled by specialists or as determinist
instruments of our recent political economy, but as implementations of core human
cognitive-symbolic capacities in live social-material situations.
We can reposition the intellectual and technical history of the ideas that
motivated the designs for computing and technical mediation in the longer
continuum that begins with humanists like Morse and polymaths like Peirce. A
different trajectory, a longer continuum in the code base, emerges. Computation
and digital media are artefacts of human symbolic cognition implemented in
physical substrates designed to use “electricity visible in desired parts of circuit” as
energy for instantiating and enacting patterns of automatable semiosis. In short,
we own this, in every sense of the term. Computation and the open possibilities for
information representation and interpretation are co-extensive with our core
human operating system, collective symbolic cognition. Computation is a human
design for complex, multileveled automated semiosis, distributed through digital
material substrates and their structured time-state transitions, motivated by
returning values for human interpretation. Computation and digital media
processes are designed as structures of symbols that mean things inter-specified
with symbols that do things in the most sophisticated way we’ve yet invented.
And in every age, it can only be the philosophy of that age, such as it
may be, which can animate the special sciences to any work that shall
really carry forward the human mind to some new and valuable truth.
Because the valuable truth is not the detached one, but the one that
goes toward enlarging the system of what is already known.
28
C. S. Peirce, MS 442, 1898, “The First Rule of Logic,” CP 5.583.
The valuable truth is recovering the position of computation and digital media that
has been hiding in plain sight in the longer continuum. Recovering this tradition of
thought provides a new foundation for unifying humanistic, scientific, and technical
concerns. All computing, being fundamentally semiosis-dependent, is, finally,
humanistic computing.
29
Select Bibliography
Peirce, Charles Sanders. Collected Papers of Charles Sanders Peirce, 8 Volumes. Edited by Charles
Hartshorne, Paul Weiss, and A. W. Burks. Cambridge, MA: Harvard University Press, 1931-
1966. (=CP)
Peirce, Charles S. The Essential Peirce: Selected Philosophical Writings (1867–1893), Vol. 1. Edited by
Nathan Houser and Christian J. W. Kloesel. Bloomington, IN: Indiana University Press,
1992. (=EP 1)
——. The Essential Peirce: Selected Philosophical Writings (1893–1913), Vol. 2. Edited by Nathan Houser
and Christian J. W. Kloesel. Bloomington, IN: Indiana University Press, 1998. (=EP 2)
Peirce, Charles S., and Victoria Welby. Semiotic and Significs: The Correspondence Between Charles S. Peirce
and Lady Victoria Welby. Edited by Charles S. Hardwick. Bloomington, IN: Indiana
University Press, 1977. (= SS)
Peirce, Charles S. Writings of Charles S. Peirce: A Chronological Edition. Edited by Max Fisch, Nathan
Hauser, Edward C. Moore, and Christian J. W. Kloesel. 6 vols. to date. Bloomington, IN:
Indiana University Press, 1982-2009. (= W)
Peirce, Charles S. C.S. Peirce Contributions to The Nation, 2 vols. Edited by Kenneth L. Ketner and James
E. Cook. Lubbock, TX: Texas Tech University Press, 1978.
Peirce, Charles S. The New Elements of Mathematics. Edited by Carolyn Eisele. 4 vols. The Hague:
Mouton Publishers / Humanities Press, 1976. (= NEM)
Peirce, Charles Sanders. Reasoning and the Logic of Things: The Cambridge Conferences Lectures of 1898.
Edited by Kenneth Ketner. Cambridge, Mass: Harvard University Press, 1992.
Peirce, Charles S. Pragmatism As a Principle and Method of Right Thinking: The 1903 Harvard Lectures on
Pragmatism. Edited by Patricia Ann Turrisi. Albany: State University of New York Press,
1997.
Andersen, Peter Bøgh. A Theory of Computer Semiotics: Semiotic Approaches to Construction and Assessment
of Computer Systems. Cambridge; New York, NY: Cambridge University Press, 1997.
Denning, Peter J. “What Is Computation?” Ubiquity, ACM, August 26, 2010.
Denning, Peter J., and Craig H. Martell. Great Principles of Computing. Cambridge, MA: The MIT Press,
2015.
Goguen, Joseph. “An Introduction to Algebraic Semiotics, with Application to User Interface
Design.” In Computation for Metaphors, Analogy, and Agents, edited by Chrystopher L. Nehaniv,
242–91. Springer Berlin Heidelberg, 1999.
Gomes, Antônio, Ricardo Gudwin, Charbel Niño El-Hani, and João Queiroz. “Towards the
Emergence of Meaning Processes in Computers from Peircean Semiotics.” Mind & Society
6, no. 2 (November 1, 2007): 173–87.
Nadin, Mihai. “Information and Semiotic Processes: The Semiotics of Computation.” Cybernetics &
Human Knowing 18, no. 1–2 (January 1, 2011): 153–75.
Nöth, Winfried. “Representation in Semiotics and in Computer Science.” Semiotica 115, no. 3/4
(August 1997): 203–13.
30
Nöth, Winfried. “Semiotic Machines.” SEED 3, no. 3 (December 2003).
Rocchi, Paolo. Logic of Analog and Digital Machines. Hauppauge, New York: Nova Science Publishers,
2013.
———. “Ubiquity Symposium: What Is Information?: Beyond the Jungle of Information
Theories.” Ubiquity, ACM 2011, no. March (March 2011): 1:1–1:9.
Smith, Brian Cantwell. “Age of Significance.” 2010.
http://www.ageofsignificance.org/aos/en/toc.html.
Sowa, John, ed. Principles of Semantic Networks: Explorations in the Representation of Knowledge. San Mateo,
CA: Morgan Kaufmann, 1991.
Sowa, John F. Conceptual Structures: Information Processing in Mind and Machine. Reading, MA: Addison-
Wesley, 1983.
———. Knowledge Representation: Logical, Philosophical, and Computational Foundations. Pacific Grove:
Brooks / Cole, Thomson Learning, 1999.
Tanaka-Ishii, Kumiko. Semiotics of Programming. New York: Cambridge University Press, 2010.
Peircean Semiotics
Freadman, Anne. The Machinery of Talk: Charles Peirce and the Sign Hypothesis. Stanford, CA: Stanford
University Press, 2004.
Misak, Cheryl, ed. The Cambridge Companion to Peirce. Cambridge, UK: Cambridge University Press,
2004.
Kockelman, Paul. “The Semiotic Stance.” Semiotica 2005, no. 157 (November 2005): 233–304.
———. “Information Is the Enclosure of Meaning: Cybernetics, Semiotics, and Alternative
Theories of Information.” Language & Communication 33, no. 2 (April 2013): 115–27.
———. “Semiotics: Interpretants, Inference, and Intersubjectivity.” In The SAGE Handbook of
Sociolinguistics, edited by Ruth Wodak, Barbara Johnstone, and Paul E. Kerswill, 165–78.
Thousand Oaks, CA: SAGE Publications, 2010.
———. Agent, Person, Subject, Self: A Theory of Ontology, Interaction, and Infrastructure. Oxford; New
York: Oxford University Press, 2012.
Kockelman, Paul, and Anya Bernstein. “Semiotic Technologies, Temporal Reckoning, and the
Portability of Meaning. or: Modern Modes of Temporality – Just How Abstract Are They?”
Anthropological Theory 12, no. 3 (September 1, 2012): 320–48.
Ransdell, Joseph. “The Relevance of Peircean Semiotic to Computational Intelligence
Augmentation.” SEED 3, no. 3 (2003): 5–36.
Rapaport, William J. “Semiotic Systems, Computers, and the Mind: How Cognition Could Be
Computing.” International Journal of Signs and Semiotic Systems 2, no. 1 (January 2012): 32–
Short, T. L. Peirce’s Theory of Signs. Cambridge, UK: Cambridge University Press, 2009.
Skagestad, Peter. “Peirce’s Inkstand as an External Embodiment of Mind.” Transactions of the
Charles S. Peirce Society 35, no. 3 (July 1, 1999): 551–61.
———. “The Mind’s Machines: The Turing Machine, the Memex, and the Personal Computer.”
Semiotica 111, no. 3/4 (October 1996): 217–43.
———. “Thinking with Machines: Intelligence Augmentation, Evolutionary Epistemology, and
Semiotic.” Journal of Social and Evolutionary Systems 16, no. 2 (1993): 157–80.
31
Brownlee, Peter John, ed. Samuel F. B. Morse’s “Gallery of the Louvre” and the Art of Invention. New
Haven: Yale University Press, 2014.
Prime, Samuel Irenæus. The Life of Samuel F. B. Morse, LL.D.: Inventor of the Electro-Magnetic Recording
Telegraph. New York: D. Appleton and Company, 1875.
Shannon, Claude E. “A Mathematical Theory of Communication.” The Bell System Technical Journal
27 (October 1948): 379–423, 623–56.
32
Stiegler, Bernard. Technics and Time, 1: The Fault of Epimetheus. Translated by Richard Beardsworth and
George Collins. Stanford, CA: Stanford University Press, 1998.
Notes
1 I presented an earlier version of this paper at The Digital Subject: Codes conference, University of
Paris 8, November, 2015. I am grateful for discussions with colleagues at this conference, which
helped me improve the argument and motivated further research.
2 I will develop this conception in a book-length project, which will involve a synthesis of research
in complex systems theory, linguistics, cognitive science, semiotics and sign systems research,
theory of computation and information, and HCI and design theory.
3 Many of Peirce’s concepts for describing meaning and information processes are being
productively reinterpreted in many contemporary research communities. For framing my
approach, I’m indebted to the work of many colleagues working at an interdisciplinary nexus of
interests in semiotics, cognitive science, linguistics, and computing: Anderson, Goguen, Sowa,
Denning, Gomes, Gudwin, El-Hani, and Queiroz in computing and cognitive science; Skagestad,
Tanaka-Ishii, Nöth, and Nadin (in philosophy, logic, and computational semiotics); and
Kockelman and Enfield (in linguistic anthropology).
4 As early as 1873, Peirce noted: “A sign is an object which stands for another to some mind. I
propose to describe the characters of a sign. In the first place like any other thing it must have
qualities which belong to it whether it be regarded as a sign or not. Thus a printed word is black,
has a certain number of letters and those letters have certain shapes. Such characters of a sign I
call its material quality” (W 3.66). A sign “must have characters which shall enable us to
distinguish it from other objects” (W 3.82). In 1897, the features are what enables sign/symbol
instances to known as such: “A sign, or representamen, is something which stands to somebody
for something in some respect or capacity. It addresses somebody, that is, creates in the mind of
that person an equivalent sign or perhaps a more developed sign. That sign which it creates I call
the interpretant of the first sign” (CP 2.228).
6 Peirce’s work is continually being rediscovered for insights in other fields, and valuable ground-
work has been done in applications of Peirce’s semiotics and algebra of relations in theories of
computation, knowledge representation, cognitive artefacts, and HCI and digital media.
33
Representative works include: Andersen, A Theory of Computer Semiotics; Andersen, “A Semiotic
Approach to Programming”; Brier, Cybersemiotics; de Souza, “Semiotic Approaches to User
Interface Design”; Nadin, “Information and Semiotic Processes: The Semiotics of Computation”;
Goguen, “An Introduction to Algebraic Semiotics, with Application to User Interface Design”;
Goguen and Harrell, “Information Visualization and Semiotic Morphisms”; Gomes, Gudwin, and
Queiroz, “On a Computational Model of the Peircean Semiosis”; Gomes, Gudwin, and Queiroz,
“Towards Meaning Processes in Computers from Peircean Semiotics”; Kockelman and Bernstein,
“Semiotic Technologies, Temporal Reckoning, and the Portability of Meaning. Or”; Meystel,
“Intelligent Systems”; Nadin, “Semiotic Machine”; Nöth, “Semiotic Machines”; Nöth,
“Representation in Semiotics and in Computer Science”; Queiroz, Emmeche, and El-Hani, “A
Peircean Approach to ‘Information’ and Its Relationship with Bateson’s and Jablonka’s Ideas”;
Ransdell, “The Relevance of Peircean Semiotic to Computational Intelligence Augmentation”;
Skagestad, “Thinking with Machines”; Skagestad, “The Mind’s Machines”; Sowa, Knowledge
Representation; Sowa, Conceptual Structures; Sowa, “Future Directions for Semantic Systems”; Tanaka-
Ishii, Semiotics of Programming.
7 For background, see: Brownlee, Samuel F. B. Morse’s “Gallery of the Louvre” and the Art of Invention;
Silverman, Lightning Man; Morse, Samuel F.B. Morse; Prime, The Life of Samuel F. B. Morse, LL. D.
9 Ibid., 253.
11 For primary historical background: Morse, Samuel F.B. Morse; Library of Congress, “Samuel F. B.
Morse Papers at the Library of Congress, 1793-1919 | Collections | Library of Congress”; Prime,
The Life of Samuel F. B. Morse, LL. D.; Mabee, The American Leonardo; Silverman, Lightning Man;
Hochfelder, The Telegraph in America, 1832-1920.
12 For background, see: Beauchamp, History of Telegraphy; Huurdeman, The Worldwide History of
Telecommunications; Hochfelder, The Telegraph in America, 1832-1920; Sterling and Shiers, History of
Telecommunications Technology; Standage, The Victorian Internet.
13 The accounts of the various designs for telegraphic devices in the prior note reveal that other
inventors simply take a corresponding encoded signal system for granted or focus on engineering
and marketing motives. Other inventors do not discuss the ideas motivating Morse’s broader
sense of code and representational media.
34
14 See Brent, Charles Sanders Peirce (Enlarged Edition), Revised and Enlarged Edition and the
introductions to both volumes of the Essential Peirce.
16 For historical background on Peirce and computing, see Ketner and Stewart, “The Early History
of Computer Design”; Burks, “Review”; Burks, “Peirce and the Year of the Computer”; Ketner,
“Peirce and Turing”; Burks and Burks, The First Electronic Computer, 333–54.
17 Brandt, “Toward a Cognitive Semiotics”; Daddesio, On Minds and Symbols; Dror and Harnad,
Cognition Distributed; Perlovsky, “Symbols: Integrated Cognition and Language”; Rieger, “Semiotic
Cognitive Information Processing: Learning to Understand Discourse, A Systemic Model of
Meaning Constitution”; Sowa, “Cognitive Architectures For Conceptual Structures”; Terenzi,
“Semiosis in Cognitive Systems”; Zlatev, “Cognitive Semiotics.”
18 References to Peirce’s works are in the conventional citation format for the published editions
(see Bibliography).
19 See Gardner, Logic Machines and Diagrams, 104–13; Burks and Burks, The First Electronic Computer,
340–47; Buck and Hunka, “W. Stanley Jevons, Allan Marquand, and the Origins of Digital
Computing”; Marquand, “A Machine for Producing Syllogistic Variations.”
20 There are many relevant texts still in manuscripts of Peirce’s unpublished papers: For example,
MS 1361 (c.1902): telegraph cypher code in binary strings, and MS 425a (1902), “Reasoning by
Machinery.”
21 See Shannon, “A Symbolic Analysis of Relay and Switching Circuits,” December 1938; Zuse,
The Computer - My Life.
22 Shannon, “A Symbolic Analysis of Relay and Switching Circuits,” December 1938, 174. This is
the published version of his thesis which was written in 1936, submitted in 1937, and approved for
his degree in 1940. Vannevar Bush was one of his thesis directors. See: Shannon, “A Symbolic
Analysis of Relay and Switching Circuits,” 1940.
23 For essential background, see Liszka, A General Introduction to the Semiotic of Charles Sanders Peirce;
Short, Peirce’s Theory of Signs; Misak, The Cambridge Companion to Peirce.
35
24 See MS 283, “The Basis of Pragmaticism,” ed. in part, EP 2.371-97; MS 339, The Logic
Notebooks: “A sign is a species of medium of communication. The object, O, determines the sign,
S, and S determines the Interpretant sign, I, to being determined by O through S.” (Jan. 30, 1906,
f.271r.)
25 Many studies over the past 20 years confirm this description from different theoretical
perspectives: see Nadin, Goguen, Sowa, de Souza, O’Neill, Cantwell-Smith.
26 For a good overview of research on this class of symbolic actions, see McNeill, Language and
Gesture; West, Deictic Imaginings. The Latin etymology of the word index (mark, sign, pointer,
indication) also includes the index finger, and thus reveals the deep association of pointing and
indicating in embodied signs for spatial-temporal attention markers in a parallel modality with
spoken and visual representations used indexically.
28 This is a long MS with variant drafts. The quoted passage corresponds to selections from the
MS published in CP 4.447.
29 Recent studies that apply Peircean semiotics to computing architectures include: Tanaka-Ishii,
Semiotics of Programming; Skagestad, “The Mind’s Machines”; Rocchi, “How Semiotics Can Improve
Our Knowledge on Computing”; Rocchi, Logic of Analog and Digital Machines; Rocchi, “What Is
Information.”
31 Many of the major expositions of HCI and GUI design principles are implicitly, if not explicitly,
based on managing semiotic structures and actions; see Kazmierczak, “Design as Meaning
Making”; Moggridge, Designing Interactions; Norman, “Cognitive Artifacts”; Dourish, Where the
Action Is; Suchman, Human-Machine Reconfigurations; Goguen, “An Introduction to Algebraic
Semiotics, with Application to User Interface Design”; Nadin, “Information and Semiotic
Processes: The Semiotics of Computation.”
32 See Agre and Rosenschein, Computational Theories of Interaction and Agency; Goldin, Smolka, and
Wegner, Interactive Computation; de Souza, The Semiotic Engineering of Human-Computer Interaction;
Wegner, “Why Interaction Is More Powerful Than Algorithms”; Nadin, “One Cannot Not
Interact.”
33 Kay and Goldberg, “Personal Dynamic Media”; See Kay, “The Dynabook - Past, Present, and
Future”; the continuing relevance of Kay’s metamedia concepts are explored in Manovich, Software
Takes Command.
36
34 Thacker et al., “Alto: A Personal Computer,” 556.
36 At the data level, this is operationalized in dozens of transcoding schemes for preserving the
interpretability of material structures while changing their total characteristics (compression,
format conversion, and device or client properties).
References
Agre, Philip E., and Stanley J. Rosenschein, eds. Computational Theories of Interaction and Agency.
Cambridge, MA: The MIT Press, 1996.
Andersen, Peter Bogh. “A Semiotic Approach to Programming.” In The Computer as Medium, edited
by Peter Bogh Andersen, Berit Holmqvist, and Jens F. Jensen, 16–67. Cambridge:
Cambridge University Press, 2007.
Andersen, Peter Bøgh. A Theory of Computer Semiotics: Semiotic Approaches to Construction and
Assessment of Computer Systems. Cambridge; New York, NY: Cambridge University Press,
1997.
Beauchamp, Ken. History of Telegraphy. London: The Institution of Engineering and Technology,
2001.
Brandt, Per Åge. “Toward a Cognitive Semiotics.” Recherches En Communication 19, no. 19 (January
3, 2003): 21–34.
Brent, Joseph. Charles Sanders Peirce: A Life. Revised ed. Bloomington, IN: Indiana University Press,
1998.
Brier, Soren. Cybersemiotics: Why Information Is Not Enough. Toronto, Canada: University of
Toronto Press, 2008.
Brownlee, Peter John, ed. Samuel F. B. Morse’s “Gallery of the Louvre” and the Art of Invention. New
Haven: Yale University Press and Terra Foundation, 2014.
Buck, G.H., and S.M. Hunka. “W. Stanley Jevons, Allan Marquand, and the Origins of Digital
Computing.” IEEE Annals of the History of Computing 21, no. 4 (October 1999): 21–27.
doi:10.1109/85.801529.
Burks, Alice R., and Arthur W. Burks. The First Electronic Computer: The Atanasoff Story. Ann
Arbor: University of Michigan Press, 1988.
Burks, Arthur W. “Peirce and the Year of the Computer.” Peirce Project Newsletter 2, no. 2 (Winter
-1996 1995). http://www.iupui.edu/~peirce/news/2_2/2.2.htm.
———. “Review: Charles S. Peirce, the New Elements of Mathematics, Ed. Carolyn Eisele.” Bulletin
of the American Mathematical Society 84, no. 5 (September 1978): 913–18.
Daddesio, Thomas C. On Minds and Symbols: The Relevance of Cognitive Science for Semiotics. Berlin
and New York: Mouton De Gruyter, 1994.
Dourish, Paul. Where the Action Is: The Foundations of Embodied Interaction. Cambridge, MA: MIT
Press, 2001.
Dror, Itiel E., and Stevan Robert Harnad, eds. Cognition Distributed: How Cognitive Technology
Extends Our Minds. John Benjamins Publishing, 2008.
Dyson, George. Turing’s Cathedral: The Origins of the Digital Universe. New York, NY: Pantheon,
2012.
Gardner, Martin. Logic Machines and Diagrams. McGraw-Hill, 1958.
Goguen, Joseph. “An Introduction to Algebraic Semiotics, with Application to User Interface
Design.” In Computation for Metaphors, Analogy, and Agents, edited by Chrystopher L.
Nehaniv, 242–91. Lecture Notes in Computer Science 1562. Springer Berlin Heidelberg,
1999. http://link.springer.com/chapter/10.1007/3-540-48834-0_15.
37
Goguen, Joseph A., and D. Fox Harrell. “Information Visualization and Semiotic Morphisms.” In
Multidisciplinary Approaches to Visual Representations and Interpretations, Volume 2,
edited by Grant Malcolm, 83–98. Amsterdam; London: Elsevier Science, 2005.
http://cseweb.ucsd.edu/~goguen/papers/sm/vzln.html.
Goldin, Dina, Scott A. Smolka, and Peter Wegner, eds. Interactive Computation: The New Paradigm.
Berlin; New York: Springer, 2006.
Gomes, A., R. Gudwin, and J. Queiroz. “On a Computational Model of the Peircean Semiosis.” In
International Conference on Integration of Knowledge Intensive Multi-Agent Systems, 2003,
703–8. IEEE, 2003. doi:10.1109/KIMAS.2003.1245124.
Gomes, Antônio, Ricardo Gudwin, and João Queiroz. “Towards Meaning Processes in Computers
from Peircean Semiotics.” SEED 3, no. 2 (November 2003): 69–79.
Hochfelder, David. The Telegraph in America, 1832-1920. Baltimore: Johns Hopkins University
Press, 2012.
Huurdeman, Anton A. The Worldwide History of Telecommunications. New York: Wiley-IEEE Press,
2003.
Kay, Alan. “The Dynabook - Past, Present, and Future.” In Proceedings of the ACM Conference on The
History of Personal Workstations, 85–. HPW ’86. New York, NY, USA: ACM, 1986.
doi:10.1145/12178.2533809.
Kay, Alan, and Adele Goldberg. “Personal Dynamic Media.” Computer 10, no. 3 (March 1977): 31–
41. doi:10.1109/C-M.1977.217672.
Kazmierczak, Elzbieta T. “Design as Meaning Making: From Making Things to the Design of
Thinking.” Design Issues 19, no. 2 (March 2003): 45–59.
doi:10.1162/074793603765201406.
Ketner, K. L. “Peirce and Turing: Comparisons and Conjectures.” Semiotica 68, no. 1/2 (February
1988): 33–61.
Ketner, Kenneth Laine, and Arthur F. Stewart. “The Early History of Computer Design: Charles
Sanders Peirce and Marquand’s Logical Machines.” The Princeton University Library
Chronicle 45 / 3 (Spring 1984): 187–225.
Kockelman, Paul, and Anya Bernstein. “Semiotic Technologies, Temporal Reckoning, and the
Portability of Meaning. or: Modern Modes of Temporality – Just How Abstract Are They?”
Anthropological Theory 12, no. 3 (September 1, 2012): 320–48.
doi:10.1177/1463499612463308.
Library of Congress. “Samuel F. B. Morse Papers at the Library of Congress, 1793-1919 |
Collections | Library of Congress.” Webpage. Accessed January 12, 2015.
http://www.loc.gov/collection/samuel-morse-papers/about-this-collection/.
Liszka, James. A General Introduction to the Semiotic of Charles Sanders Peirce. Bloomington:
Indiana University Press, 1996.
Mabee, Carleton. The American Leonardo: A Life of Samuel F. B. Morse. Revised. Fleischmanns, NY:
Purple Mountain Press, 2000.
Manovich, Lev. Software Takes Command: Extending the Language of New Media. London; New
York: Bloomsbury Academic, 2013.
Marquand, Allan. “A Machine for Producing Syllogistic Variations.” In Studies in Logic by Members
of the Johns Hopkins University, Ed. C. S. Peirce, edited by C. S. Peirce, 12–16. Boston: Little,
Brown, and Co., 1883.
McNeill, David, ed. Language and Gesture. Cambridge; New York: Cambridge University Press,
2000.
Meystel, A.M. “Intelligent Systems: A Semiotic Perspective.” Proceedings of the 1996 IEEE
International Symposium on Intelligent Control, 1996, Intelligent Control, September 1996,
61–67. doi:10.1109/ISIC.1996.556178.
Misak, Cheryl, ed. The Cambridge Companion to Peirce. Cambridge, UK: Cambridge University
Press, 2004.
Moggridge, Bill. Designing Interactions. Cambridge, MA: The MIT Press, 2007.
Morse, Samuel Finley Breese. Samuel F. B. Morse: His Letters and Journals. Edited by Edward Lind
Morse. Vol. 2 vols. Boston: Houghton Mifflin Co., 1914.
38
Nadin, Mihai. “Information and Semiotic Processes: The Semiotics of Computation.” Cybernetics &
Human Knowing 18, no. 1–2 (January 1, 2011): 153–75.
———. “One Cannot Not Interact.” Knowledge-Based Systems, Semiotic Approaches to User
Interface Design, 14, no. 8 (December 1, 2001): 437–40. doi:10.1016/S0950-
7051(01)00138-1.
———. “Semiotic Machine.” The Public Journal of Semiotics 1, no. 1 (January 1, 2007): 57–75.
Norman, Donald A. “Cognitive Artifacts.” In Designing Interaction, edited by John M. Carroll, 17–38.
New York, NY: Cambridge University Press, 1991.
http://dl.acm.org/citation.cfm?id=120352.120354.
Nöth, Winfried. “Representation in Semiotics and in Computer Science.” Semiotica 115, no. 3/4
(August 1997): 203–13.
———. “Semiotic Machines.” SEED 3, no. 3 (December 2003).
http://www.library.utoronto.ca/see/SEED/Vol3-3/Winfried.htm.
Perlovsky, Leonid I. “Symbols: Integrated Cognition and Language.” In Semiotics and Intelligent
Systems Development, edited by Ricardo Gudwin and João Queiroz, 121–51. Hershey, PA:
IGI Global, 2006. http://www.igi-global.com/chapter/semiotics-intelligent-systems-
development/28939.
Priestley, Mark. A Science of Operations: Machines, Logic and the Invention of Programming. New
York; London: Springer, 2011.
Prime, Samuel Irenæus. The Life of Samuel F. B. Morse, LL. D.: Inventor of the Electro-Magnetic
Recording Telegraph. New York: D. Appleton and Company, 1875.
Queiroz, João, Claus Emmeche, and Charbel Niño El-Hani. “A Peircean Approach to ‘Information’
and Its Relationship with Bateson’s and Jablonka’s Ideas.” The American Journal of
Semiotics 24, no. 1–3 (2008): 75–94, 206–7.
Ransdell, Joseph. “The Relevance of Peircean Semiotic to Computational Intelligence
Augmentation.” SEED 3, no. 3 (2003): 5–36.
Rieger, Burghard B. “Semiotic Cognitive Information Processing: Learning to Understand
Discourse, A Systemic Model of Meaning Constitution.” In Adaptivity and Learning: An
Interdisciplinary Debate, edited by Reimer Kühn, Randolf Menzel, Wolfram Menzel, Ulrich
Ratsch, Michael M. Richter, and Ion-Olimpiu Stamatescu, 347–403. New York: Springer,
2003.
Rocchi, Paolo. “How Semiotics Can Improve Our Knowledge on Computing.” Sino-US English
Teaching 10, no. 12 (2013): 914–919.
———. Logic of Analog and Digital Machines. Hauppauge, New York: Nova Science Publishers,
2013.
———. “What Is Information: Beyond the Jungle of Information Theories.” The Computer Journal
55, no. 7 (July 1, 2012): 856–60. doi:10.1093/comjnl/bxs079.
Shannon, C.E. “A Symbolic Analysis of Relay and Switching Circuits.” Transactions of the American
Institute of Electrical Engineers 57, no. 12 (December 1938): 713–23. doi:10.1109/T-
AIEE.1938.5057767.
Shannon, Claude E. “A Symbolic Analysis of Relay and Switching Circuits.” MS Thesis, submitted
1937, MIT, 1940. http://dspace.mit.edu/handle/1721.1/11173.
Short, T. L. Peirce’s Theory of Signs. Cambridge, UK: Cambridge University Press, 2009.
Silverman, Kenneth. Lightning Man: The Accursed Life of Samuel F. B. Morse. New York: Knopf,
2003.
Skagestad, Peter. “The Mind’s Machines: The Turing Machine, the Memex, and the Personal
Computer.” Semiotica 111, no. 3/4 (October 1996): 217–43.
———. “Thinking with Machines: Intelligence Augmentation, Evolutionary Epistemology, and
Semiotic.” Journal of Social and Evolutionary Systems 16, no. 2 (1993): 157–80.
doi:10.1016/1061-7361(93)90026-N.
Souza, Clarisse Sieckenius de. “Semiotic Approaches to User Interface Design.” Knowledge-Based
Systems, Semiotic Approaches to User Interface Design, 14, no. 8 (December 1, 2001):
415–18. doi:10.1016/S0950-7051(01)00133-2.
———. The Semiotic Engineering of Human-Computer Interaction. Cambridge, MA: The MIT Press,
2005.
39
Sowa, John F. “Cognitive Architectures For Conceptual Structures.” In Conceptual Structures for
Discovering Knowledge: 19th International Conference on Conceptual Structures, ICCS 2011,
Derby, UK, July 25-29, 2011, edited by Simon Andrews, Simon Polovina, Richard Hill, and
Babak Akhgar, 35–49. New York: Springer, 2011.
———. Conceptual Structures: Information Processing in Mind and Machine. Reading, MA: Addison-
Wesley, 1983.
———. “Future Directions for Semantic Systems.” In Intelligence-Based Systems Engineering,
edited by Andreas Tolk and Lakhmi C. Jain, 23–48. Berlin; Heidelberg: Springer, 2011.
———. Knowledge Representation: Logical, Philosophical, and Computational Foundations. Pacific
Grove: Brooks / Cole , Thomson Learning, 1999.
Standage, Tom. The Victorian Internet: The Remarkable Story of the Telegraph and the Nineteenth
Century’s On-Line Pioneers. New York, NY: Walker & Company, 2007.
Sterling, Christopher H., and George Shiers. History of Telecommunications Technology. Lanham,
MD: Scarecrow Press, 2000.
Suchman, Lucy. Human-Machine Reconfigurations: Plans and Situated Actions. 2nd ed. Cambridge;
New York, NY: Cambridge University Press, 2006.
Tanaka-Ishii, Kumiko. Semiotics of Programming. New York: Cambridge University Press, 2010.
Tanimoto, Steven L. An Interdisciplinary Introduction to Image Processing: Pixels, Numbers, and
Programs. Cambridge, MA: The MIT Press, 2012.
Terenzi, Graziano. “Semiosis in Cognitive Systems.” Semiotica 171, no. 1–4 (September 2008):
131–62. doi:10.1515/SEMI.2008.071.
Thacker, C. P., E. M. McCreight, B. W. Lampson, R. F. Sproull, and D. R. Boggs. “Alto: A Personal
Computer.” In Computer Structures: Principles and Examples, edited by Daniel P.
Siewiorek, C. Gordon Bell, and Allen Newell, 2nd ed., 549–72. New York: Mcgraw-Hill,
1981.
Wegner, Peter. “Why Interaction Is More Powerful Than Algorithms.” Communications of the ACM
40, no. 5 (May 1, 1997): 80–91. doi:10.1145/253769.253801.
West, Donna E. Deictic Imaginings: Semiosis at Work and at Play. New York: Springer, 2013.
Zlatev, Jordan. “Cognitive Semiotics: An Emerging Field for the Transdisciplinary Study of
Meaning.” The Public Journal of Semiotics 4, no. 1 (October 1, 2012): 2–24.
Zuse, Konrad. The Computer - My Life. Translated by P. McKenna and J. A. Ross. Berlin; New York:
Springer, 1993.
40