Proception

Artbrain.org is the web site of biologist/artist/theorist Warren Neidich, winner of this year’s Villem Flusser award at Transmediale. Flagging it here for future reference. Looks like a fine place for roots and berries.

Note also his artist’s site, from which I pulled a quote (my italics): “In these pictures a deliberate attempt was made to renegotiate the basic laws governing what i am calling “proception.” Perception refers to the general rules through which sense impressions are built into more complex formations in the brain and become part of memory and contemplation. Proception is the way that those impressions become projected upon the natural world and sometimes when they become part of the language of art produce an archive of the changing conditions of the mind and brain. Proception is the reversed perception and incorporates the projection of changing internal states.” (text from How Many Triangles, 2005. Site architecture doesn’t make it easy to get a direct link).

Interoception

Some provocative ideas about our interior sensory perception or interoception are posted in Mapping the Neural Pathways of Feelings at the Dana Foundation. Interoception appears to have its own neural pathways, very ancient ones in evolutionary terms. It may also be implicated in decision-making.  The “somatic marker” hypothesis of Martin Paulus suggests that we map emotions conducive to evaluation of choices onto physical sensations, and that people with impaired interoception may have have difficulty making decisions. “Gut feelings” may be as trustworthy as logic, if logic maps its values onto gut feelings. Other research suggests that our elusive sense of self may be allied with interception: perhaps we are not just a epiphenomenon of language after all.

The Dana Foundation offers a heady mix of information on brain research, immunology, and arts education.

The Guggenheim Museum presents a symposium, The Universe Resounds: Kandinsky, Synesthesia and Art, Tuesday, January 12. Looks like a very interesting intersection of neuroscientists,  visual artists, art historians, and musicians.

I’m making my essay Synesthetic Art: An Imaginary Number? available to Nina Wenhart’s course “Prehystories of New Media” at the School of the Art Institute of Chicago, where I’m stepping in to talk about algorithmics and intermedia. I hope the ideas in this essay will provoke at least a little conversation. If they interest a few students in the poetics of synesthesia in Baudelaire and the Symbolist poets, that would also be a good outcome. As with Wagnerian Gesamtkunstwerk, their ideas have had a far-reaching influence in the many historical attempts to achieve the fusion of disparate media and sensory modalities, even though that influence is rarely explicit, and the ideas, struck off from their historical matrix, have undergone many odd mutations (psychedelic light shows in the 60s, for one).

The title may be somewhat unfortunate, in that as mathematical entities imaginary numbers are every bit as real (i.e., functional) as “real” numbers. I hoped to suggest–though perhaps only math geeks will catch my drift–that synesthetic art, like imaginary numbers, occupies a contested site in the history of ideas, but has long ago proven its utility.

Synesthetic Art: An Imaginary Number?, was published in Leonardo, v32-5, pp. 399-404, 1999. The version linked to this page is licensed under a Creative Commons License. It does not contain the illustrations of the published essay, and has a few textual variations, but otherwise is substantially the same.

Abstract

Works that fuse the senses are often referred to as “synesthetic art.” Computers, because they offer the possibility of controlling and synchronizing different media and implementing highly abstract compositional structures across media, seem an ideal tool for synesthetic art. This essay argues that a structural approach to such an artform is inadequate, and that it must be grounded its potential symbolic functions. Starting from a brief examination of synesthesia as a neurological phenomenon and a sketch of the origins and influence of Baudelaire’s poetics of synesthesia, this essay suggests points of departure for a poetics of multisensory composition.

A PDF of the final draft can be found here.

Alan Merriam, in the Anthropology of Music, devotes a chapter to “Synesthesia and Intersense Modalities,” where he offers a number of terms to express various types of intermodal and intermedial experiences.

He briefly discusses synesthesia, defining it along familiar lines such as “…the experience of an associated sensation when another sense is stimulated” (Carl Seashore). This is the classical neurological synesthesia.

He also discusses a neurorological phenomenon akin to synesthesia where stimulus in one sense affects the acuity of perception in another. This he refers to as intersense stimulation. It appears to be a common physiological phenomenon that has been statistically evaluated. How do smells affect your hearing? How does light affect your sense of taste?

Intersense Transfer involves the perception of sensory qualities in different modalities as similar. “Rough” and “smooth” sounds and surfaces, for example. It’s partly a cultural construct, but probably not entirely: language serves multiple senses, but language evolves with experience.

Intersense Modalities are culturally constructed associations of sensory modalities. For example: association of colors with cardinal directions in many cultures. Artistic systems (Scriabin, Rimsky) fall into this category–and I would suggest that the question of whether Rimsky or Scriabin or Rimbaud were synesthetes is largely beside the point. They constructed systems that served their artistic purpose in a cultural moment when such correspondences were part of avant-garde practices. In Rimbaud’s case, there is the Symbolist tradition starting with Baudelaire. Rimsky and Scriabin were in contact with the Theosophists and any number of theories of unification of the arts. These influences made it possible to develop a cross-modal artistic language that would have symbolic potential.

Sensory Multimodality is a cognitive principal that asserts that the senses evolve as an integrated source of information (consider the recognition of food). This principal is evoked in the cultural critiques accompanying some multi-modal art. It is suggested that the senses become separated by education, routine, exposure to media. Multimodal art then restores an “original unity of the senses.” (Actually, count me among those who think this point of view has some considerable weight).

We might also consider how gesture functions within sensory integration. Studies of kinesthesis (the sense of the body’s position, movement, and gesture, also referred to as proprioception) apparently reveal that kinesthetic sensory qualities of a gesture are reproduced in our imagination when we perceive the sound or the mark that the gesture creates, without the gesture actually being present.

I think we could agree that “gesture” is a cross-modal category: intensity over time, no matter the modality by which we experience it. Gesture can be informal (spontaneous) or formal (measured and systematized). Gesture can be reversed in time, or inverted in intensity–then we have counterpoint. Because it can be mapped onto different sensory modalities, gesture supports correspondences and mappings from one sense to another: loud:soft :: bright:dim, and many others.

In light of the above categories, I would assert that when artists combine artforms intended for different senses, they are not necessarily doing so in reference to synesthesia. Correspondences are constructed between events in different sensory modalities because we have a wide range of experience in which the senses interoperate. Even the specific category of “synesthetic art” may spring from a cultural tradition that originates in other cross-modal experiences and later becomes identified with synesthesia because of the symbolic weight accorded to multisensorial experiences. I think this is clearly the case with the Baudelairean tradition of “correspondances.” I can think of very little synesthetic art that is just intended to simulate a synesthetic state–even when that is its avowed purpose, it nearly always freights the work with other communications, with symbolic meaning that recreated experience alone does not carry.

In any case, the idea of synesthesia has been more useful to the arts (in the form of poetic fusion, metaphor, and cross-modal structure) than the actual phenomenon. By this, I mean in no way to suggest that synesthesia is not a fascinating phenomenon from which we can learn a great deal, only that it is one province of a wide realm of intermedial and intermodal experiences and cultural representations. I think it is particularly important to place artistic endeavors within this wider and more richly varied context. If we want to explore multimodal art, we might as well draw on the entire context, both in our creative and our analytical approaches, and not attempt to stretch the bounds of “synesthesia” too far, when there are other descriptive terms.

When mapping from one sensory channel to another operates in real-time with a relatively high temporal or spatial resolution, we can construct an interface by providing a point of interaction–e.g., mouse movement translates into cursor movement, pressure on a pad translates to amplitude control of a sound, etc. When the mapping is slower or not interactive but provides an “objective” rendering of information, where scalar values are conserved across the mapping, we a probably looking at a visualization–e.g., a signal is rendered by an FFT as a histogram of frequencies, weather data are mapped into audible frequencies, etc. The visualization may be delivered as an instrument, such as an oscilloscope or sonar display. If the mappings are at least partly symbolic, or involve subjective renderings (in which we include aesthetic choices), we are probably dealing with intermedia art. Intermedia patterns potentially have something to offer to all three disciplines.

Cognitive Informatics

I do not as yet know much about this field and the WikiPedia entry on it is rather slim. Much of the work in the field seems to originate with one person, Yingxu Wang at the University of Calgary. Those disclaimers apart, it seems worthy of interest on two accounts. First, it suggests that Information is a category co-equal with Matter and Energy, a point of view very much of our time when material objects appear charged with informational patterns (i.e., a state of virtuality, as N. Katherine Hayles defines it). Second, it proposes a model for computation apparently influenced by theories of embodied or distributed consciousness, and thus for constructing a new generation of computational devices (entities?) that interact with the world.

It also apparently offers a mathematical notation for modeling its theoretical discourse. To be sure, cybernetics and general semantics have made similar claims about modeling consciousness, with variable success. Wang’s articles are found in the IEEE Proceedings, and should be available online through Project MUSE and other journal databases to institutional subscribers (check your university library).

I have put the materials on this site under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 Unported License. This seems the best way of both sharing and protecting the text. The icon at the top of the sidebar and the footer text both indicate the licensing.

Novelists and psychologists share an interest in the way we think, argues Charles Fernyhough, but writers must do more to keep up with science.

Steadily, in vivid colour, the brain gives up its secrets. Hardly a week goes by without the news that another elusive human quality – the capacity to understand sarcasm, to give one recent example, or to judge another person’s trustworthiness – has been traced to a side-alley of neuroanatomy. Away from the fMRI scanner, psychologists and cognitive scientists – those who study the software that runs on the brain’s hardware – have made great progress in the modelling of human thought processes. Although the big problems of consciousness and free will show little sign of yielding to scientific analysis just yet, at least some of the mysteries of mind and brain are close to being accounted for by objective, testable theories. Link

Charles Fernyhough
Saturday October 15, 2005
The Guardian

I upgraded the blog software to WordPress 2.3.2, an upgrade that included a security patch and so seemed rather necessary. In the process, I had to change the mechanism for loading TinyMCE, the HTML visual editor. Previously, I was able to load the editor in a compressed (zipped) version. That seems to be broken, so I am loading the uncompressed version. If you have any problems with editing, clear your cache, quit your browser, launch the browser again, and try editing again. If it doesn’t work, contact me. Versions prior to 2.3.1 had the same problem. I’m not sure what is going on.

Older Posts »