new essay published—what does a photograph sound like? digital image sonification as synesthetic audiovisual digital humanities, digital humanities quarterly 15, 1 (2021).
Abstract
Computers have the capacity to transpose the pixels, shapes, and other features of visual material into sound. This act of data correlation between the visual and the audial produces a new artifact, a sonic composition created from the visual source. The new artifact, however, correlates precisely to data in the original, thus allowing for fresh ways of perceiving its form, content, and context. Seeming to distort the visual object into an aural one paradoxically allows an observer to observe the visual evidence anew, with more accuracy. A kind of generative, synesthetic criticism becomes possible by cutting across typical boundaries between the visual and the audio, the optic and the aural. Listening to as well as looking at visual artifacts by way of digital transpositions of data enables better close readings, more compelling interpretations, and deeper contextual understandings. Building on my earlier scholarship into image glitching, remixing, and sonification, this essay investigates a photograph of Joan Baez performing at the Greek Amphitheater in Berkeley, California, during the early 1960s. The image comes from my project on the Berkeley Folk Music Festival and the history of the folk music revival on the West Coast of the United States. Here, the use of digital image sonification becomes particularly intriguing. While we cannot magically recover the music being made in the photograph, we can more closely attend to the ghosts of sound within the silent snapshot. Digital image sonification does not recover the music itself, but it does help to amplify issues of gender, power, embodiment, spectacle, performance, hierarchy, and performance in my perceptions of Baez making music in the photograph. Using the ear as well as the eye to scan the image for its multiple levels of meaning leads to unsuspected perceptions, which then support more revealing analysis. In digital image sonification, a cyborgian dance of data, signal, image, sound, history, and human perception emerges, activating visual materials for renewed scrutiny. In doing so, this mode of AudioVisual DH activates the scholarly imagination in promising new ways.
To read the full essay: Michael J. Kramer, “What Does A Photograph Sound Like? Digital Image Sonification As Synesthetic AudioVisual Digital Humanities,” Digital Humanities Quarterly 15, 1 (2021).