Alan Merriam, in the Anthropology of Music, devotes a chapter to “Synesthesia and Intersense Modalities,” where he offers a number of terms to express various types of intermodal and intermedial experiences.
He briefly discusses synesthesia, defining it along familiar lines such as “…the experience of an associated sensation when another sense is stimulated” (Carl Seashore). This is the classical neurological synesthesia.
He also discusses a neurorological phenomenon akin to synesthesia where stimulus in one sense affects the acuity of perception in another. This he refers to as intersense stimulation. It appears to be a common physiological phenomenon that has been statistically evaluated. How do smells affect your hearing? How does light affect your sense of taste?
Intersense Transfer involves the perception of sensory qualities in different modalities as similar. “Rough” and “smooth” sounds and surfaces, for example. It’s partly a cultural construct, but probably not entirely: language serves multiple senses, but language evolves with experience.
Intersense Modalities are culturally constructed associations of sensory modalities. For example: association of colors with cardinal directions in many cultures. Artistic systems (Scriabin, Rimsky) fall into this category–and I would suggest that the question of whether Rimsky or Scriabin or Rimbaud were synesthetes is largely beside the point. They constructed systems that served their artistic purpose in a cultural moment when such correspondences were part of avant-garde practices. In Rimbaud’s case, there is the Symbolist tradition starting with Baudelaire. Rimsky and Scriabin were in contact with the Theosophists and any number of theories of unification of the arts. These influences made it possible to develop a cross-modal artistic language that would have symbolic potential.
Sensory Multimodality is a cognitive principal that asserts that the senses evolve as an integrated source of information (consider the recognition of food). This principal is evoked in the cultural critiques accompanying some multi-modal art. It is suggested that the senses become separated by education, routine, exposure to media. Multimodal art then restores an “original unity of the senses.” (Actually, count me among those who think this point of view has some considerable weight).
We might also consider how gesture functions within sensory integration. Studies of kinesthesis (the sense of the body’s position, movement, and gesture, also referred to as proprioception) apparently reveal that kinesthetic sensory qualities of a gesture are reproduced in our imagination when we perceive the sound or the mark that the gesture creates, without the gesture actually being present.
I think we could agree that “gesture” is a cross-modal category: intensity over time, no matter the modality by which we experience it. Gesture can be informal (spontaneous) or formal (measured and systematized). Gesture can be reversed in time, or inverted in intensity–then we have counterpoint. Because it can be mapped onto different sensory modalities, gesture supports correspondences and mappings from one sense to another: loud:soft :: bright:dim, and many others.
In light of the above categories, I would assert that when artists combine artforms intended for different senses, they are not necessarily doing so in reference to synesthesia. Correspondences are constructed between events in different sensory modalities because we have a wide range of experience in which the senses interoperate. Even the specific category of “synesthetic art” may spring from a cultural tradition that originates in other cross-modal experiences and later becomes identified with synesthesia because of the symbolic weight accorded to multisensorial experiences. I think this is clearly the case with the Baudelairean tradition of “correspondances.” I can think of very little synesthetic art that is just intended to simulate a synesthetic state–even when that is its avowed purpose, it nearly always freights the work with other communications, with symbolic meaning that recreated experience alone does not carry.
In any case, the idea of synesthesia has been more useful to the arts (in the form of poetic fusion, metaphor, and cross-modal structure) than the actual phenomenon. By this, I mean in no way to suggest that synesthesia is not a fascinating phenomenon from which we can learn a great deal, only that it is one province of a wide realm of intermedial and intermodal experiences and cultural representations. I think it is particularly important to place artistic endeavors within this wider and more richly varied context. If we want to explore multimodal art, we might as well draw on the entire context, both in our creative and our analytical approaches, and not attempt to stretch the bounds of “synesthesia” too far, when there are other descriptive terms.