“To the analyst, such as me, interfaces are where the fun lies. Interfaces between people, people and machines, machines and machines, people and organizations. Anytime one system or set of activities abuts another, there must be an interface. Interfaces are where problems arise, where miscommunications and conflicting assumptions collide. Mismatched anything: schedules, communication protocols, cultures, conventions, impedances, coding schemes, nomenclature, procedures. it is a designer’s heaven and the practitioners hell. And it is where I prefer to be.”
Donald Norman

It’s a couple hundred years before the advent of GPS, Sextons and the Compass. You’re in the middle of the ocean, and your not lost. This is most likely because you have become literate to seamarks, the environmental cues and signals of open waters. Schools of fish, celestial alignment, cloud formations, and flocks of birds all tell of ones position and probable future conditions. It is also likely that you did not discover all of these cues on your own. The understanding of these signals and signs is a skill that has been passed down, preserved and cherished for generations. A complex mix of curiosity, mortality, and passion combined with the perceived value of unknowns too seductive to abandon forced these first literacies to emerge.

But are these signs really signs, or are they cues? In terms of the present-day way finding of concepts, could it be that our equivalent forms are more like hyperlinks than definitive words? Does a cloud, icon, or object contain meaning, or trigger a learned response?

When asked to write a paper for DeSForM: Design & Semantics of Form and Movement, I was intrigued by the purpose of the conference. Its edict is to facilitate a dialogue between schools of design, human-computer interaction, interface design, and tangible computing. The resultant discussion would determine how information will be communicated through a dynamic use of form and movement by objects and spaces.

How might object and spatial behavior communicate meaning? What dilemmas can we foresee at the outset of this exploration and how might we approach them? The question posed by the organizers of DeSForM was twofold. How might we set forth to design this system of dynamic symbols in a systematic and scientific manner, as well as what are some of the best practices, precedents and knowledge in this area?

One only needs to look at conferences such as DesFoRM and blogs such as WeMakeMoneyNotArt to see existing advances in university labs, art and design colleges, unorthodox startups, basements and garages all working towards solutions and applications in this area. The examples are exhaustive, there is opportunity in harnessing this narrative of edge competencies these diverse and driven collections of individuals are creating. In identifying the connections, links and overlaps between their work, there is much value in acknowledging, aggregating, framing their discoveries and explorations. Navigating these concepts and contributions to this language will be much like navigating open waters. Through the sketches, prototypes, products and papers we will individually and collectively negotiate their value and use, their successes or failures.

Meanings associated with forms and how they behave is a human production both malleable and undefined. They are unpredictable and differ from person to person and from one situation to another. As a cultural construct, form has the potential to both consecrate meaning as well as confound it. New definitions of form semantics can be revealed through the examination of the intersection of human behavior and emerging technological discourse in the present social climes. A language of form and movement cannot be designed, but a platform in which this language can emerge through negotiation can be. Like everyday-lexographers existing within day-to-day psychogeographies, we will need to continually negotiate the duration of relevance within, and the morphology of, this language. There is much work that can be appropriated form other disciplines that understand to some degree how languages evolve. It is my opinion that we should address the “what” and “why” of the exploration before we jump into “how”, or we should identify and adjust our “how” as new understanding and knowledge presents itself.

Julian Bleecker of Near Future Labs recently stated in his Manufacturing paper for Share 2008 “These are distinct kinds of digital objects that mix physical space, digital technology and design …. The weak signals suggest kinds of design-art-technology that are growing tired of the screen.” When objects shape shift and change their properties to communicate something they become screen, and take on more cinematic qualities. The question here is who then has the authority to program these channels, and how is this new form of content created? How will these “characters” interface and enhance individual and collective narratives in everyday living?

I want to propose a disruptive thought in regard to form semantics that, in essence, form has no meaning. Form is an invitation, a window to possible meanings. I would go further to say that meaning resides, and is latent within us. In the relationships we perceive and cultivate in our minds and through what we negotiate with others.

This is the paradox that makes the stable attribution of meaning to anything so difficult, whether it is form, color or motion. People see more, they see differently, and thus, they challenge existing limitations. What means something in one place may mean something else in another. Meaning is emergent, and does not reside within any thing. Forms are however participants in the management of meaning. They are present and actively involved in the production, preservation, recall and engagement with meaning. We subvert meaning by designing it into form. Sole prefixed meanings stagnate possibility and curb the creation of one’s own meanings through the discovery of what is and what can be. By considering the relationship of form while engaging in a critical and open-minded conversation about people as producers and vessels of meaning, we can form a deeper understanding that will inform the development of new species of forms as interfaces embedded in, and inseparable from, a dynamic “ecology of meaning”.

The outcomes of conversation about the inherency of meaning in form could potentially overturn how we produce, interact with, and communicate our relationships with form. This shift in thinking may enrich product, service and environment interactions and make them much more intimately engaged in people’s everyday lives. As people continually contribute to them through use and “misuse”. Critical to success is expanding the voices authoring product functionality, behavioral language and social roles beyond a small privileged priesthood of creators.

If it is meaning we are aiming to communicate then what then is the nature of meaning, how is it produced, perceived, transferred? Is there an ecology of meaning? If so what are its properties and dynamics? And how then can we design in, from, and through this process? How might forms that explicitly enable the making of meaning, the flow and transformation and triggering of meaning? How might these forms differ from static representational forms? What we are really engaging in is creating better interfaces that reflect, and are inseparable from, the human process of producing meaning, personal schemas, and cultural dialects of understanding. The objective is that through the examination and inquiry of the cycling, transformative and intimately personal nature of meaning, we will emphasize in our objects, spaces, and interfaces the affordances for personal expression, annotation and modification rather than fixed semantic constraints.

As forms become increasingly computational, they inherit a league of sensory, expressive and communicative capabilities associated with digital artifacts. New possibilities then arise to more intimately engage with the dynamism of our ecology of meaning as opposed to merely notating them. A central question that this exploration intends to initiate is how do we approach, enable, and mobilize a spectrum of subjectivities and cultures over time in the creation and evolution a robust system of representation. A flexible system of representation, recognition, and memory that is in step with changing that occurs within systems of meaning.

If this frontier is a mix of human perception, linguistics, design, business, computer science and dance, then how do we create a common vocabulary for the type of interdisciplinary understanding and collaboration needed to move forward? Creating strategic value for organizations, institutions and individuals lies in cracking this representational dilemma, and in creating the interfaces that operate not just for many people, but also because of them. One motivation for the creation of such a language is driven by national innovation policies for global competitiveness built upon organizational delivery of relevance and sustained value. It would make sense then to build a lasting and dynamic infrastructure and manufacturing process aligned in nature with the way this language will naturally unfold.

Causes, values, motivations and concentrated actions become destinations as people and places become increasingly findable. How do situated technologies augment the lifecycles of these elements?

Through locative and social technologies what was once qualitative speculation now has the ability to shift into the domain of the measurable and recordable, generating a new species of data around the relationships between individuals, groups and their environment. Organizations such as Microsofts spinoff company Inrix “tracks the behavior of 750,000 vehicles, cell phone users, and others with digital devices to determine how Americans would react to different situations.”   vividly illustrates the growing interest and unique opportunity to harness human geospatial data. Experimentation with this form of information makes practices like Suprise Modelling thinkable.

Intrix, being such a novel geographic information retrieval company is a weak signal pointing toward our data traces becoming a new form of communication that creates a social metric, which diversifies the voices authoring geographical data. As urban computing is further embedded in our daily interactions new potentials for informing decisions and revealing choices can be realized. Through our data traces we will be able to co-author custom, space, policy and our material culture in new ways. In the background of this emerging context personal behavior will become personal advocacy. Our implicitly authored data may be used to cast votes, leave gestural and locative commentary. It may become a new voice, and new way to speak.

New dialogues of power occur at the intersection of human behavior, emerging technology, and inherited infrastructures and processes. Locative technologies point toward real-time visualization that offers a new individual and collective vantage, which will undoubtedly inform urban “logistics of perception” deeply effecting social dynamics and personal action. How might near-future “geo-strategic” identity building, expression, and management demand a “re-socialization” between individuals and their environment? How will people adapt?

Locative technologies, situational awareness and mobile haptics are a form of contextual and behavioral stenography. They offer a fertile ground to deploy location and behavior based “passive or explicit multi-player citizen games”. These future games (or “systems”) could also act as a base for dynamically negotiating rules, goals, conditions, and social accountability. We discuss current weak signals in our environment, such as serious games like World Without Oil, and the BCN Formula Game that contain elements that can be recombined and resituated as a robust infrastructure for effective democracy populated with informed citizens engaging in civil participation through new media.

“Social learning/training” based on the affordances of locative technologies and a “re-socialization” around the concepts of city and citizen could be proposed. Consider the environment itself as an agent - that is, the city would be, communicable and mutable rather than only a background. It is possible to reinvent the social space of the street as a much-needed infrastructure of expression and presence to facilitate and realize influential social relations. These emergent systems can be designed as educational frameworks for interacting and understanding converging literacies in the new environment. A new literacy and curricula are a vital to better equip people for emerging contexts that will require augmented political and social mobilization.

The need for a new educational framework is evident. However, the curriculum and implementation is up for debate.

INTRODUCTION

How is it that we truly know the world?

What is the true nature of things?

Science tells us that the true nature of things lies beneath the surface. If we look upon a chair and ask, what is that, the common answer would be, but of course - a chair. And who would deny this? It is after all what we have learned.

A chair is not a chair however. Beyond the surface of the chair are a whole series of phenomena that are invisible to the “naked” eye. The chair is actually made up of atoms, tiny particles that are the building blocks for everything. A desk, a chair, even humans are made up of atoms. And atoms themselves are made up of even smaller subatomic particles.

Science and the scientific pursuit have proven this to us through instruments that measure, and through experimentation. The scanning tunneling microscope used for observing atoms, for example, presents us with a reality that is objective and rational. Or so it seems.

But is this really how we know the world?

Knowing seems to be indicative of a deeper visceral engagement with the world. One that is mediated through the senses and understood only after repeated sensory impressions.

Experience is the mother of all invention.

Reality is not objective and rational. Rather it is a subjective experience dependant on the one doing the perceiving. And it is precisely this dependency that has given birth not only to the apparatus for sensing, but also to that which the senses pick up. There is a certain mutuality or reciprocity to the phenomenal world in the way that acts of expression and perception form mutually co-dependant relationships. If the bee cannot smell the pollen what good is it to the pollen producing plant? Thus, our perceptual ways of knowing are tethered to the world itself, intimately weaved, and not separate from it.

Science and technology however have distanced themselves from these corporeal ways of knowing preferring instead a rational and objective outlook. And has as a result of these extensions created a gap between nature and us. Between the mind, and the body.

Take for example, the term “naked eye” as it is used above. As if the eye that is naked is insufficient without the prosthesis of scientific technology. The assumption itself that the eye is naked to begin with is skewed. These assumptions, that have in essence made their way into a common language suggests a deeply engrained prejudice that displaces the senses from their natural ability to pick up what is given, and instead suggests that the true nature of things somehow lie beneath all appearances. This way of thinking has not only displaced the senses from their inherent right to know, but has distanced us from what I would call proper ways of thinking about the world and our place in it, by privileging a rational techno-scientific outlook over one grounded in the bodies corporeal ways of knowing.

I would like to close that gap and reclaim the rightful place of the Sensus Communis as an intelligent and rational faculty that can, and should inform our forages into technological and scientific pursuits.

Over the past several years I have been deeply engaged with a body of work that advocates for the inclusion of the human senses into a common practice. One that states that the senses are the foundational support for everything we experience and for everything that we create as a result of those experiences. As such, the senses are something we should be paying more attention to, and giving them their due credit.

In consideration of the many facets of multi-sensorial intelligence, this paper will briefly touch on three aspects I think are important in bringing to bear the relevance and significance of a return to the Sensus Communis.

FOOTNOTE: “Sensus Communis”

Aristotle postulated a central faculty capable of uniting all the senses, synthesizing what each sense picked up to form a coherent whole in the mind. He called this the “Sensus Communis”.

THE SENSES ARE PRIMARY MEDIA
The first will explore the senses as primary media in how the whole world is mediated through the senses. And look at the senses as natural communication systems as a way to understand the mechanisms that place and register meaning through experience.

ADVOCACY – A CRITICAL AND REFLEXIVE AWARENESS
In the face of the technologization of human and social capability this trajectory gives voice to the dimensions of analogue human ability. Arguing for the often-neglected abilities of the human senses as a way to challenge the technologization of perception working towards the inclusion of the analogue senses into the dominant discourse.

THE SENSES AND SOCIETY
The second focuses on the senses and society, viewing every object as the subjects of perception, as things that mediate our interactions and contribute to the construction and diffusion of a techno-scientific outlook. An outlook and a way of thinking about the world that has affected the very way we see the world and thus, how we perceive it, unquestionably having significant impacts into our social and cultural production.

THE SENSES ARE PRIMARY MEDIA

Our first and best way of engaging with the world is through the senses.

We see, hear, taste, feel, smell and move about in the world – all visceral experiences.

The primary mechanism through which our reality is grounded in the world is through our senses. We know the world through the senses.

When we touch the world, that world equally pushes back. And thus that thing is registered through the sense of touch as a thing, but as a thing that is sensed through my hands as a cup; its weight, the heat of its contents, whether is it full or empty, and not what is just thought to be a cup, or a wall for that matter. Although haptic feedback in definition is limited to that of touch, but extends as a metaphor for lack of a word, and translates into the domains of the other senses.

What is the haptic equivalent for smell, for sight, sound for movement, etc? We push and the world pushes back. With every step I take the ground pushes back equally on my foot, this is a true registering of the world. I do not need a formula or a theorem such as gravity to understand this. I know through repeated sensory experience that it is so. No theorem, explanation, or description can beat that.

The world is out there, whether we are here or not, whether we act out in it or not. Things happen in it all the time.

If the senses mediate the world and the world mediates the senses then when we look out, the world looks back at us, we touch and get touched back, we call out and the world calls back.

Each of our senses provides us with a different way interpreting and comprehending the world. A tree looks a certain way, feels a certain way, sounds a certain way, and each sense is finely tuned to pick up all these things. It is the picture built up by the senses together, the synthesizing act of the Sensus Communis that registers the tree, or rather the experience of the tree, as it is felt, seen, heard.

Can I describe a tree and have you comprehend it if you have never seen, heard or touched a tree ever? Probably not.

As children we learn from our exploratory forages and inquisitive natures through our experiences – through our sensorial experiences. These are grounded in reality by way that is not abstract, it is real, it is felt, it is tasted, it is heard, and thus the senses as primary media are the go-link to the phenomenal world and in turn inform all our ideas thereafter.

Piaget wrote of infants, that the world was essentially a thing to be sucked, indicative of the primacy and intimacy of the senses in discovering and knowing the world, in this case through taste; the mouth and the sense of taste conferrable to the act of living.

Taking the Sensus Communis as the experiential grounding of the world into a reality that we can understand and grasp at deep visceral levels, proposing that the world is not made up of things or objects, but rather of experiences, considering the nature of experience as more real than things. For things only have meaning once we know what they are. And how we know what they are is deeply entrenched in our interactions and experiences with them.

The senses are primary media that are intelligent so to speak, they don’t lie and we are tied to the world, part of it, not separate from it.
As a young adolescent teen I had the unpleasant experience of being told not to trust my own common sense. The setting was a mathematics class where I had told the teacher that the method by which I had come to my conclusions was as a result of my own common sense. The teacher was unimpressed and stated very flatly that my answer was not good enough – that my answer was not really an answer, and that I must use the proper theorem to prove it. What better proof is there than a common sense I thought. My attempts to stand up and argue that my common sense answer made more sense than that of the theorem did little to impress my teacher, in the end earning me only partial marks. Needless to say, that I was upset and depressed about having to forfeit my deeply sensed intuition to some theorem.

What I have realized since then is that what at the time to me was just common sense was actually superior to that of the theorem, simply because the theorem was an abstract representation of the original insight that gave rise to it in the first place.

What I had, and what my teacher missed was that my common sense had a real and tangible quality to it, a reality that had as its basis the phenomenal world, a reality placed and registered through experience.

Instead of capitalizing on these insights I was steered away from the opportunity to expand the horizons of perception, and perceptual ways of knowing.

I was being asked to deny what I felt viscerally, intuitively through the body so to speak and accept an abstracted reality, one step removed from what I could see, hear, touch, taste, and move through.

These are our first ways of knowing the world – the senses inform our world – they are the primary medium through which we touch taste feel the world. They inform our ideas of the world in a visceral manner; they explain the world through deep mechanisms. The rest are ideas about our experiences, one step removed from reality so to speak.

(rough draft 1 - 2 sections to go)

Original Doc: A New Dialogue - Original

A New Dialogue Redux

Introduction
The central concerns of the following text focuses on addressing the cultural meanings attributable to form, and initiate a discursive dialogue with respect to the inherency of meanings which potentially reside within forms themselves.

The former instantiation of this text was written for a conference DeSForM: Design & Semantics of Form and Movement in the Netherlands which purpose is to develop a platform of products that communicate information through a dynamic language of form and movement. The question posed by the organizers was how might we set forth to design this system of symbols in a systematic and scientific manner. The meaning of form is a human production, as it is both malleable and undefined. As a cultural construct, it has the potential to consecrate meaning as well as confound it. New definitions of form semantics can be revealed through the examination of the intersection of human behavior and emerging technological discourse in the present social climes, a language of form and movement cannot be designed, but a platform in which this language can emerge through negotiation can be.

Form Has No Meaning
The capability and meaning of any form can be defined by the limits of people’s ability to imagine what it can be physically or represent spiritually or intangibly. In essence, form has no meaning; it is an invitation, a window to possible relationships which produce a myriad of meanings. Meaning resides, and is latent within us, in the relationships we perceive and cultivate in our minds and through what we negotiate with others.

What can we learn from these characteristics of forms and how can we, as designers, challenge the need for designing explicit meanings?

How can we create a new dialogue between objects and people to harness the emergent properties of meaning within the human experience?

Time Matters. Location matters. Context matters.
Form triggers multiple meanings. A gun locked safely at a hunting lodge has entirely different denotative and connotative associations than the same gun in the hands of a man in a dark alley. The sign itself, the gun and its function, within the context of alternative environments, is pollinated with probable outcomes that are associated with location and situation. Context influences interpretation of the purpose of that form, shaping the message it conveys. It is through the process of negotiating meaning between social actors, place and location, that form evokes multiple meanings, and likewise meanings can inspire multiple forms. Physical properties in differing contexts can trigger landscapes of ulterior meanings around and between people determined by their personally and culturally informed associations. Context again alters the meanings that forms and motions may trigger, influencing their social role at a given time. The introduction of new information and the mutation of old information alters associated meanings because of what is accumulated, paired and lost during the reallocations and migrations of forms in multiple contexts.

How can we enable forms to embrace this continual process of the production of meaning? Is it our objects that adapt, or do we?

Can a form change its mode of expression to fit those who are present?

How can a single meaning be maintained through multiple modes of expression in multiple contexts?

How can these unfolding dialects, migrations and morphologies of the compositionality of meaning over time be documented?

Will context be the dominant hypermedia form? Will it be the request, the curator, the aggregator and synthesizer in relation to ones identity and state, emotional, situational and physical context?

Meaning and Functionality

Functionality affords meaning through the use of an object. Meaning is cultivated through experience, not by what the thing is supposed to be. Through the process of doing, understanding and participating with a given form, a sense of meaning is developed. Emerging forms that possess processing and communicative capabilities add a new layer to form. It makes the form variable and responsive. Forms were formerly limited by their physical capabilities, and to the imagination of those who encounter them. However, once they are enabled, and networked, they will have a new breadth of roles, social relationships and possible meanings that are physically bound to place, yet extend and include that which is beyond place borders.

How can these new roles and relationships be felt, visualized and animated through form and motion?

Things enable behaviors, and behaviors find themselves eventually as forms. All of the forms of our built environment are signifiers of behaviors that bind beneficial material and immaterial relationships.

There are levels of meaning that satisfy multiple levels of self, such as our immediate physical, mental and spiritual needs. These levels of meaning can be realized and attained through interfacing with a solitary form. This can be observed in the specialized context of jail, as prisoners draw on a restrictive palette of materials and forms, shaping a wide selection of objects that perform various functions and hold widely different values [Angelo] Through the reshaping of one form, prisoners can satisfy different levels of human needs. The toothbrush can be multiple things at once. At a given time it can be a shiv , a tool for cleaning teeth, or a carved piece of art.

It is our motivations that curate the relationships we engage in to trigger revelations of what things can be, and what meanings those relationships bring. We perceive context and conditions that we are participants within, and we forge our own meanings; in turn we can contribute this meaning in a shared dialogue to negotiate normative ones.

What can we learn from this perceptive ability of people and the potentiality of form?

How can objects be sensitive to our goals? Can they be literate to our actions? How can we better equip them to help us achieve?

Will a language of form and movement inevitably produce a Universal Homogenous dialect, or alternatively will a heterogeneous explosion of dialects evolve?

Can contextual mapping and meaning mapping over time be achieved to form a living document capturing a language unfolding in real time?

Form Collapses Possibility
Forms alone do not collapse possibility, but the essentialist framing of forms that associate a sole meaning to a form neuter ones perception to imagine what the form can do or be. Forms will be a platform of dialogue: they will be simultaneously the substrate of a question or goal, the expression of a possible answer, and a passport to exploration of possibility.

Kevin Kelly of Wired has been developing a blog “Street Use”[Kelly] about how objects are repurposed to fit specific needs of people. The blog exhibits fryer baskets that are made into antennae, and regular pick-up trucks that are re-appropriated as highly reinforced armed assault vehicles in Iraq. These interventions are a testament to not only possibility inherent in forms, but also to the multiple potentials people see in them as a way to facilitate their goals. The idea of forms having multiple meanings can be substantiated by the fact that meaning is emergent. This is why we cannot design semantics in a systematic and scientific way, for they are individually variable, as well as culturally and socially formed. The failure of Esperanto and the success of Creole and Pidgin languages [Diamond] illustrate this vividly.

Have we considered how this inherent human characteristic of emergence can be harnessed, socially and individually through forms that are literate of human intent?

“Real Time Graz”[Ratti] is a project where GPS mapping of cell phone trajectories is used to illustrate the nomadic portrait of people in the city; this information describes in a rather low resolution how people use space. These traces of flow are then used to inform the emerging shape of the city to complement how space is actually used. “The BCN Formula Game” [Hubers] has taken this further by developing software that generates building proposals for Barcelona in real time through modeling software that creates structures in response to flows of people, traffic and commercial activities.

How can our traces influence the language of form and movement to best-fit context and particular actors in real time?

In general, the most valid current definitions of forms, which are the ones most widely and currently understood, are the ones that are commonly applied day to day. How can we allow for these definitions to evolve and be continually negotiated? The Association of Robotic Architecture & the Bureau of Responsive Architecture work together to develop responsive architectures that adapt to changing functions as well as environmental conditions. These architectures have fluid form/function relationships. On an object level Dynamic Textures has developed the idea of a coffee cup made of a thermally sensitive polymer that sprouts spikes on the exterior if the content of the cup exceeds a safe temperature. [Tsu] The future may depend on building upon existing senses and semiotics as they have evolved over time. We can devise new modes of communication through form based on our native capabilities of recognition, understanding and sharing.

It will take time to learn to “read” newly emerging responsive architectures and objects as innately as we respond to natural events. Only after we have interfaced with them on an ongoing basis will we be able to effortlessly understand them. For example, being able to understand the state of the internal social dynamics of a building by observing its shape from down the street, or recognizing the state of the contents of a container by its visual attitude from across the room at a glance. It will take time and communication between people to form a consensus of a shared meaning associated with the expression of animate forms; however, we also need to allow for change, and polysemous meanings in this process.

Can our signs become deeper participants and documenters of this process?

We are all producers of meaning; form triggers these meanings or the production of new ones, but holds no inherent meaning itself. Objects or produced forms are codified, commoditized beliefs, intent and behaviors. However, there is a big difference between what an object can do, and what an object is meant to do. Moreover, there is a difference between what an object is supposed to mean and the meaning that people attribute to it. Meaning is myth. The desire to understand, unfold, know and create unique perspectives has been overshadowed by the traditional way of perceiving forms as hyperlinks to a normative known. By diverting philosophical meaning and understanding to that which is static and consumable we destroy whole landscapes of unique expansions on the ‘meaning’ of anything.

Disruptions such as Geocaching, context aware gaming and location based technologies alter the production and perception of meanings in a space as well as the forms within. Context aware games like Pacmanhattan [Bloomberg] transform the streets of New York into a dual virtual physical Pacman playspace. In Japan the game Mogi [Tester] requires players to discover hidden virtual items; these experiential properties transform the perception and the meaning of what that space is. Shifting meaning can be achieved by something as simple as a laptop in a park: the park then becomes a studio. A personal phone transforms whatever space it is in into a bank, a mailbox, a warzone, a voting booth or a gamespace. Meaning is brought to places and objects by people as we encounter them, and the properties of these spaces and objects afford the kind of meaning that may be manufactured.

Platform For Emergence – Users Sketching Experiences
By providing the conditions and tools within objects themselves we can create an open platform for naturally unfolding semantics. If we are to create a language of form and movement, then its core elements should embrace the emergent nature of meaning.

Embracing emergence requires an object to have a recognition and memory of what happens around them, building a history or narrative of change. In addition to expressive capabilities and the ability to build on the past, these forms should have the plasticity to allow transformation in all of its capabilities to adapt to people in the present. A form will be able to observe the world around it, but human participation with it will continually re-define what it is, to re-contextualize the purpose of its form, expanding the experiences it may elicit. These forms should be aware of this process of emergent meaning if they are to be deeper participants of social practice. The forms that comprise our objects and spaces are igniting a new dialogue when they explicitly become a form of media and interface. Building a living framework of consideration, by engaging in an inquiry around the proprieties and impact of such a platform of dialogue will be helpful in determining which capabilities are most necessary.

As we create enabled and communicative forms that comprise our objects and spaces, we will require an equal consideration of both the digital and the physical aspects. The new method should consider relationships that are happening between things and people in spaces. The attention and sensitivity we prescribe to physical representation and expression of associated meaning has to be directly proportional to that which we devote to the sensing, processing, understanding, storing and sharing in the ecology of meaning surrounding it. This language will enable a new spectrum of authoring and expression of meaning. The following are characteristics of expressive forms that should be considered:

• Awareness and Memory
• Expression and Literacy
• Transformation: an element that is pervasive in all the others

Awareness and Memory
We leave traces of ourselves behind as every moment passes. Streams of gestures, wafts of pheromones, a wake of heat, a dynamic gait emanating a chorus of continuous audio. This species of media is the most valuable; it is the unfolding map of ones self. When these individual maps are read on a larger scale it is the Map of Us, it is the combined interactions and behaviors of communities of people that illustrate underpinning motivations, intents, and shared meanings. Objects, equipped with the culmination of several maturing and emerging technologies, including sensing technologies, have the potential to seek the meanings of our traces, and observe their composition and morphology over time. This raises the opportunity for objects to be cartographers of experience, to be moment hunters and stenographers, seekers of the zeitgeist of place.

Our immaterial traces are codified in an array of manifestations. Objects can be seen as traces of beneficial behaviors. Architecture and civic spaces are traces of social formations. As you read this sentence you are encountering traces of thoughts, codified in type. The sensory perception capability of things is directly proportional to its ability to read not only the physical but also the invisible relationships that sculpt our environment. Most of today’s objects and spaces are blind, deaf and mute, however, scores of new capabilities are migrating to the shores and guts of products, and are being woven into the very fabric of architecture. It’s not about the objects or spaces themselves, but of them as images, products of what happens in-between. They will increasingly become a form of socio-cultural barometers that aid in the flows in ways that they have never before.

The shared memory of this network will be as beneficial and as epic as the impact of the World Wide Web. It will be a living documentation of our memetic propagation, and our behavior in the physical environment, and its resulting effect on the shaping of the environment itself.

How can we enable a co-evolution and interplay between our personal traces, and objects and spaces?

How will our “pasts” converse with, and enhance, our present experience?

How will our present physical, situational and emotional context contribute to what is displaying?

Expression And Literacy

Blogs such as Information Aesthetics and We Make Money Not Art [Debatty] successfully showcase emerging practices and developments in object expression including and form and movement showcased from sources as varied as basements, art galleries, and university laboratories. The projects are tagged and tracked to reveal how memes catch on and how methods evolve.

What if the projects, the Objects themselves circulate and share these developments?

What can we learn from heraldry, patina and natural expression, such as a ripening fruit, as a pre-existing historical or natural form of communication?

Communication between objects will, in the beginning, work in chorus to combine sensor capabilities, processing power, storage, expressions, behaviors, and share memories, to understand and render responses to meaning. Our meanings will be derived from our behaviors in context, read by objects, which become a responsive porous vessel, a new form of media. They are becoming dynamic content.

This language of form and movement has been around forever. Throughout the course of a day we may employ about a thousand words, however, we express 30,000 facial expressions, and 5,000 hand and bodily gestures [Danesi]. Most of our communication happens through our visual interpretation of form and movement. We can “read” clouds, water and the landscapes of rock and of flesh. Undoubtedly, if we were to propose a semantic framework for form and movement, we should employ the wealth of experience in our literacy of displayed behavior.

Expressive capabilities are already migrating from screens to the surfaces and movements of objects for public use. Such is the case with Ambient Devices [2], a Massachusetts based startup, pioneered by graduates of the MIT media lab. Their products enable data from the internet to be translated into personally programmed color and movements in forms such as The Orb, which can display stock information, weather or any other information from the Ambient Information Network. The Inflating USB key [Komissarov] alters its size in relation to how much data is stored on it. Email erosion [Ham; Muilenberg] is a more art centric piece that aims to display relationships that are usually invisible. EmailErosion is a project that uses a water-soluble block as a display; incoming emails cause flows of water to alter the form of the block in proportion to the flows of email. Alternatively, the broadband cord project [Weiser; Seely Brown] displays data packet flows in physical space through servos that shake the cord in accordance to Internet activity. These projects are examples of how this language is alive and its evolution underway.

Due to the emergent properties of meaning we cannot design semantics in a systematic and scientific way, for they are individually variable, culturally and socially formed. Instead, we should provide the conditions and tools that create a platform for semantics to unfold naturally, because people already shape form and produce meaning around them continually. If an enhanced dialogue with the environment is to evolve through a language of form and movement, then it will be sensitive and responsive to context and identities of people in the transformative process of meaning making.

How can we begin to bring into motion a platform to allow the public to assist in the creation and evolution of this language?

Enabling characteristics that thrive off of the morphology of meaning, rather than being threatened by it, turns a very difficult problem into an opportunity. This approach includes and complements the ongoing development of forms that communicate. The future of design is not in form or movement, color or texture, but in the temporal curation of relationships within changing landscape of social practice.

Tim O’Reilly said that Web2.0 applications change with use, the more people use them the better they get. This isn’t merely accounting for the threshold of viability that is expressed in how social software thrives off of social activity. People additionally change the software through using it. This happens because web behavior is recorded and responded to by developers engaged in a perpetual beta, and users themselves contribute to API’s and SDK’s adding value and functionality to the scripts.

Products and spaces too are in perpetual beta, however the climate of behavior that surrounds then is currently not recordable outside of labs and basement studios. The Nintendo Wii is a weak signal that points toward such products due to its ability to identify and respond to motion, that however is only one dimension. A stronger signal is Bungie Studios, Halo creator and Microsoft affiliated game laboratory which simultaneously observes many human dimensions to improve the user experience associated with gameplay. It is foreseeable that products will have the capability to house “Bungies” in their shells. Users will one day sketch future experiences, products, services and spaces through their mere interaction with them. A future Google result for Sketching User Experiences may return – Did you mean: Users sketching experiences?

PREAMBLE:

Once upon a time not long ago – when people whore tag jammers and rendered life on the low – when laws were stern and justice stood – and kids were misbehaving like they aught to should – There lived a little toy who was mislead – by another little toy and this is what he said: Me and you kid we’re gonna craft some fame – posting up drama while tagging his name. They hit some spots – fame came with ease – but one couldn’t stop it’s like he has a disease…

-Adapted from Children’s Story By Slick Rick 

The  scenario to follow is a “day in the life of” story about a Graffiti Writer overcoming and subverting what we might call “writers block”: the combined chilling effects of materials prohibition, pervasive surveillance technologies, inaccessible mediums, shrinking spaces for expression (aka habitat destruction) and the inattention of fragmented audiences lost in an over mediated fog of advertising. How does the artist overcome and adapt?

Along with the help of a growing creative insurgency made up of home grown hactivists, moonshiners, code breakers, trackers and map makers; Graffiti Writers do their best to bypass the barriers standing in between them and their pursuits of mediated presence, dialogue and fame. What is the future Graffiti  experience? Where does it take place?

Le Confiture (The Jam) takes place “once upon a time” several decades after the disappearance of graffiti, as we know it today. Technological progress, privatization and public paranoia has forced Graffiti Culture out of its more familiar and tangible surroundings- causing its migration towards freshly augmented realities and virtual worlds that present themselves as blank canvases (Habitats) to express on and speak from. Within this new context, Graffiti lives on.

Unfortunately, many of these augmented realities and virtual spaces apply and enforce the same laws, codes of conduct, and logics that squeezed Graffiti from its original habitats. Within this new paradigm, security, cost of production, bandwidth, connection speeds, traffic volume and membership agreements frequently determine the artist’s potential for self-actualization. Which often finds it-self half rendered and lagging. Further adaptation is required. How does one make their mark within this plural context?

As method to this madness, graffiti artists rely more heavily on soft-tools and semi-autonomous agents, also know as “toy soldiers”. These agents are designed and programmed to help the artist achieve and maintain a plural-all city- digital graffiti presence, now necessary within the complex mixture of overlapping worlds. These agents are, all at once, collaborators, students, and works of art; always evolving in style, sketching themselves out, questioning, collecting, learning from, and commenting on, the landscapes and experiences that surround them. Graffiti writers are their “imaginations”, temporary masters, and mentors guiding their actions, teaching and training them towards achieving mutually beneficial outcomes. What is the nature of this collaborative relationship? What is gained and what is lost? What is the role of imagination in a world of automation?

Within this highly mediated theatre known as Dataspace, the Graffiti Writer and his agents are just one in a billion actors fighting for space, bandwidth and attention on a massively multiplayer mixed reality stage; Performing for an always on and lurking audience inhabited by people places and things. Graffiti is the anomaly always acting up to be seen. What does this mean?

Graffiti has  evolved; writers embrace performance, playing more seriously with time-context-medium, and most importantly message. Graffiti shouts out at you; knowing it will be seen, heard, captured, broken apart, re-assembled and transmediated back into- and across other worlds, contexts- spaces and times by the all seeing, all appropriating audience.   

LE CONFITURE is a scenario-based exploration of expression, identity, learning and collaboration within a futuristic and intentionally dystopic - ubiquitous computing landscape otherwise known as Dataspace. The scenario introduces and projects existing political, socio-cultural, technological, and economic signals into a panoptic world inhabited by yet another generation of Graffiti artists and their culture jamming  hacktivist agents. The narrative explores their motives, logics, values and experiences as they struggle to make their mark within a feudal and over saturated attention economy controlled by pervasive and chilling DRM infrastructures, advanced materials prohibition, and networked surveillance. Furthermore, the scenario explores the context of multiple networked identities, master-mentor and actor-author-audience dynamics, and how new forms of digital media literacy might emerge within a transmediated (Plural) landscape where even the subtlest forms of temporary presence and participation can guarantee new and unimagined forms of permanence, fame and legacy.

 Keywords

Graffiti 2.OH!, Remix Culture, Prohibition, Digital Rights, Collaboration, Ubiquitous Computing, Dataspace, Transmediation New-Media, Expression, Virtual Identity, Surveillance, Telepresence, Trespass, Plurality, Asymmetric, Literacy, Attention Economy, Feudalism, Augmented Reality, Artificial Intelligence, Pattern Recognition, Mapping, RFID, 

On the slushiest day of the year, Rafael Fajardo, a young game artist and media professor walks in our Toronto office with a bright green webkins frog. What proceeded was an informal discussion about serious games.

The University of Denver (DU), where Fajardo teaches, has staked a claim in doing “good” for humanity. The digital media studies department prepares graduates “with significant experience in three areas of investigation and aptitude: design, technical, and critical. Trained to establish and foster dialogue across these areas, DMS graduates are prepared to address the entangled, technology-fueled challenges shaping contemporary experience.”

Humane Games
Rafael’s personal interests lie in a concept he calls “Humane Games;” games that are educational through play and through their making. “Socially conscience games”, which serve to articulate and address a variety of perspectives on various social, political and cultural issues. He is also underway working on Humane Games as a concept that has medical and health value. These are games that have physiological advantages, aiding therapeutic practice and patient pain aversion. Humane games are games that are good for people. It is here where the design of games is considered as a component in forming a discipline around the development of games. Game development is described on the DU School of Engineering & Computer Science Game Development Programs website as:

“the academic field focused on the creation of electronic games. The field includes interactive graphics/animation programming, fundamental computer science, game design, studio art, electronic art, narrative storytelling, and critical game studies. The field is concerned with both entertainment and serious games.”

This marks a shift in games as they move from a weaponized nature rooted in a heritage of military simulation and wargaming, to a more holistic and diverse social and cultural expression. An evolution “from sword to plowshare”; from tactical to practical; from kill to chill. In our conversation it was discussed how game developers, by in large, create games for people such as themselves; they are “audience-creators” generally male, young and Anglo. Thus, the industry having been flooded with a spectrum of competitive, action and strategy driven games has come to a level of saturation, and is showing signs of waning in popularity. Rafael in his P4 initiative looks to enable the diversification of voices creating games, and tools to assist in making them mark entry points to relieve saturation and restore integrity to the medium.

Design is all about relationships. Games deal with contexts, decisions and outcomes; they are in effect a medium of fluid and dynamic relationships. Rafael asks in his work, “What are the relationships we wish to curate when designing games?” Could they be a form of editorial media for social and cultural critique and art? To find out he takes the complexity of serious issues and simmers them to an interactive “reduction” that one can engage and embody. Digital Games are a relatively young media, and the concept of games as social critic, action, or as a literary object is even more new. Games like Pax Warrior and Darfur is Dying are cause driven artifacts, suitable and viral pickets in an age of internet ubiquity. Games are also a medium of attention, which is all he asks us to pay.

“Can Games Handle Heavy Content?”
In finding the answer(s) to this question, which has defined his research and practice, he sets out to hear stories and enables people to express these stories in forms they never thought they could. The first project is Crosser*, a story about “Carlos Moreno, a clean cut hardworking man looking for a good future.” The game was born south of the US/Mexico border, from IBM “abandonware”, consisting of rejected computers and material, whatever his team could get there hands on. Of interest is the building of this game; where the waste of the “developed world” is transformed by the “underdeveloped world” at a critical junction, the border that separates and binds these two worlds. The team was a juxtaposition of Denver academics and underrepresented youth living near the Mexico-USA border. In Crosser, Rafael has included the process and context of creation as a critical object.

The objective is to cross the US/Mexico border in the face of a blitzkrieg of obstacles. The glitchy lag timed play milieu of a pastiche techno-creation process itself becomes a social, political and cultural comment. Positioning the crosser, an (il)legal alien as the protagonist or hero also adds to the work’s commentary, reversing the zeitgeist belief of crossers being criminal. Rafael, later communicated his discomfort in the one sidedness of the work, and set out to balance the scale.

La Migra was an addition to, and an alternate component of, Crosser. They are in fact two levels, or dimensions of the same game. LaMigra created a dialogical tension between the works, being a game composed of the perspective of the border police. The purpose of the works is to highlight the relationship and realizations that occur at the intersection of, in the tension between the two games, in the dual nature of the content and its emphasis. Further the graphical treatment, the GUI is sampled from arguably the best games of all time, Frogger and Space Invaders. The appropriation of this form of representation highlights that even lo-fi graphics can communicate heavy issues, as well as, emphasizing through contrast the light nature of purely entertainment driven industrially produced games.

It is intended and anticipated, that through varied game authoring tools and a diverse chorus of creation, the spectrum of stories being told and potentially the nature of conversations about the past present and future that occur within games and play media will expand.

Initial thoughts: 

Relational Representation
I attended a Digital Holography and 3d Visualization pilot course this year. It was a joint collaboration between the Ontario College of Art and Design and the University of Toronto. There were two professors, one from the optics engineering department and one from Holography department. They taught the principles of light, as well as the possibilities and affordances of holography as a medium of expression. The students were already split into Artists and Scientists. And this distinction was strengthened through the assignment of alternate deliverables, different materials and expectations. The course was supposed to be a ground in which to synthesize these camps, to learn from each other. However we spoke different languages, had different intents, and different ideas about what was salient.

If you’re an engineer or computer science student at the DU you may find yourself in a life drawing class in the art department, similar to the situation above. In his talk, Rafael emphasized the variety of skills it takes to create games. “You have to be good at creating visual experiences, creating experiential experiences as well as coding.” Rarely do you find polymaths that are fluent in all of these skills, for each one of the above fans out into a series of sub skills and capabilities that take years to develop, and their all taught separately. Not always in the same school.

Game Development as a discipline calls for the convergence of many academic fields, however may serve to do much more than “establish and foster dialogue across these areas” for the purpose of developing games. Areas such Art, Design, Literary Arts, Critical theory, Engineering and Computer Science are all part of the knowledge mosaic that makes for a good game. After experiencing first hand the dilemma of disciplinary difference, I wonder if games have potential to synthesize these alternative thinking models in a space that absorbs, samples and synthesizes jargon, tables, graphs, that describe dynamics and relationships. Games as a medium could serve as an arena to develop and evolve a new system of symbolic representation that can build bridges between alternative systems of disciplinary and cultural symbolic representation. Can games present Opportunities for semantic conflict resolution?

Will there be Regional and individual game representational dialects? And will future games allow for the plasticity to change in relation to peoples representational needs while retaining the deep structures of meaning intended by the game developers?

Action Games?
During the question period at the end of Fajardo evening presentation a fellow from the audience asked, “Where is the action, the real world resolution, in the end it’s just a game?” This question highlights only the infancy of games as critique, not their inability to contribute meaningfully to political or social change. This is an area Rafael expresses he had intended his games to eventually address. But for now its one step at a time, it’s about small victories.

This question did however point toward a perceived limit of games that may be obsolesced in years to come. As technologies become increasingly “human literate” tracking and recording the meaning of position, movement, and various transactions of people in depth I wonder if there is an intrinsic promise in haptic interfaces and locative media. Games have inhabited all forms of media dead or alive. One of the first things we do with new media is play with it, or kill with it, but technology by its very nature is neutral. It is shaped by our desires, our needs and our wants.

Locative technologies could easily record our trajectories through places and our actions in them, in a sense they and others, such as situational awareness and haptics may become a form of contextual and behavioral stenography. They will be a fertile ground in which to deploy location/behavior based passive or explicit multiplayer citizen games. Our data could be easily correlated with goal sets, point systems and envisioned victory conditions. These future games could also be grounds in which to dynamically negotiate rules goals and conditions, social responsibility and accountability and may quantify more readily that which was formerly unrecordable and qualitative. “You can be dead serious in your aims but playful in getting there“

Game Borne Products?
Game borne products are not about merchandising to support game narratives but emergent products born from human interaction and experiences in game. They begin as data, intangible in game behavior or custom which has the potential to be expressed as matter. Games, like 3d CAD and simulation software in the past could be appropriated to negotiate and author future products and services culturally before we chose to appoint energy and resources to their material production. Could we develop Experience and play driven micro production on a mass scale?

Engineering creations are certified mathematically through simulation and modeling, surgeons and pilots are certified through in game hours and examination. Can game-born products and services creations be validated experientially, and be translated into future real world products. Could in game prototyping, social evaluation or “massively authored value creation” be a staple component of near future manufacturing processes? As these interactive media technologies, desktop and just-in-time manufacturing practices mature, the speed at which we may move from concept to percept, or conversation to product shortens.

Fajardo’s next project is critical toys, think webkins meets Jane Jacobs meets Paulo Fiere. But each toy he has created thus far hail from his in-game dialogues, they sample from and are instantiations of the representational models he and his teams negotiated in the making of them.

There is no print button in games.

Yet.

Scenario: Saw-sees

My name is John Hammond and I can do almost anything you can imagine with wood. Ive spent the last thirty years building cabinets in my spare time, and mine are as good as any. Ivory inlays for days.

I’m considered a master.

I’m Xavier Borden, I’m an industrial poet and musician. I make kinetic sculptures that have a dual purpose as instruments. I transform, reform and perform, interact or die.

I’m considered an eccentric.

My friends call me Sady, I take consumer electronics, tools and vintage computers and build violent robots that’ll eat your kitsch installations.

I’m considered an artist

Call me John, Xavier or Sady, I’m a playlist of mastery, but my real name is DeWalt im a model 568X skill saw. Nice to meet you. My tricks, my trades are recallable at any time. Just ask my traces, they’ll shed light on an imagination that lived. The residual imprints of hands, paths, and purposes is my personality.

I’m considered innovation

—

Significance:

I began to toy with the idea that objects can house avatars. There are many ways in which people can express themselves and their identity beyond pixel and volume. As objects become more literate of our behaviors they may one day house our gestural traces.

How would one “play an environment” like a conductor of a remote symphony in real-time? - Embodying an instrument, a section, or whole orchestra at any given moment, an expressive variance similar to that of emphasis in speech. My son should be able to bring a play-list of these “contacts” - live or residual- with him everywhere, for use in a wide variety of contexts. Making any wood shop his fathers’ wood shop. The right contextual links and personal inquiry could prompt, or summon residual knowledge, or “live” presence of an individual or mix of individuals which that a given situation requires - providing - in context, information rich, experiential-mash-up-swarms, with read write capability.

I wonder how something like this could change educational models, could El P be in my sequencer? When I buy a paintbrush, could you throw in Rembrandt too? It brings a whole other meaning to buying a pair of “Jordan’s.”

Objects will perform unimagined means and modalities of representation, vessels of personal and group communication. It is foreseeable that things with communications will enable the expression of our various forms of presence. It is likely that social networks will evolve within single and multiple objects. Merely using something is synonymous to pressing record.

These recordings will be entries contributing to the history of an object, a history that can be searched, recalled, annotated and resituated. Object that teach are nothing new. New is how object may provide a platform to direct the hands in a spectrum of meaningful ways, based on other hands. Like a scene from Ghost. Gesture is technique, and technique can be recorded, codified and expressed. Objects may come to teach, translating motions, motivations and intents into tutorials and curricula. There are stories to tell of the roles they’ve played in the lives of people. If only these walls could talk.