MUSIC & MOVEMENT
Update: Joe Hanson from the PBS site It's Okay to be Smart has a short video highlighting this research:
also has a nice review of this study with figures, videos, and sound files here.
A SIMPLE QUESTION
Why is happy music bouncy and sad music heavy?
If you read the lyrics to "Close to Me" by The Cure, you'd be forgiven for thinking it would put you in a melancholy, introspective kind of mood.
"I've waited hours for this.
I've made myself so sick.
I wish i'd stayed asleep today…"
But give the song a listen. The tinkling ivories and snappy beat are just plain happy. Music, beyond lyrics, sets a tone. It can be up tempo and light or slow and plodding and these qualities telegraph emotion the same way that a person walking can look happy or sad depending on their gait (bouncy step vs downcast trudge).
We propose that movement and music convey similar types of information such as emotion. Our project seeks to discover the underlying structure of emotions that allows them to be represented similarly by movement and music.
To illustrate, click on the pictures below to see animations that communicate emotion without faces, voices, or limbs. Each animation is paired with a clip of music that appears to "fit" the dynamics of the ball.
Of course this doesn't prove that music and movement have the same structure, just that they can express similar emotions. To demonstrate an isomorphic relationship between music and movement we had to develop a special computer program.
Beau Sievers wrote a computer program that enables a person to create music or bouncing ball animations by manipulating a single set of five dynamic features:
Rate, Regularity of rate, Direction (up/down), Size (big/small), and Smoothness.
The sliders operate in real time. By manipulating the slider bars, a person can alter the music notes being played or the animation being shown. For example, manipulating the RATE slider such that it makes the notes play faster also makes the ball bounce faster (see a random manipulation of the sliders here to demonstrate the real-time yoking of music and animation).
A crossmodal mapping of the same five features in music and movement (animation) provides a stringent, quantifiable test for whether music and movement express similar concepts by using the same dynamic features in the same ways. For example, does angry music have the same dynamics as angry movement? This would help explain why music "moves" us -- why humans everywhere dance to music and why music is so important for creating group cohesion (synchrony), emotional engagement, and ritual.
We asked Dartmouth college students to use the slider program to create five different emotional expressions (angry, happy, sad, peaceful, and scared). We found that they placed the slider bars in a particular, distinct configuration for each emotion. Most importantly this configuration was the same regardless of whether they were trying to express that emotion in music or animation. To find out whether these crossmodal emotional expressions are universal, we took our program to a remote tribal village in Cambodia. Villagers had never used computers before or been exposed to Western music and yet created the five emotional expressions in the same way as our Dartmouth college students. Thus, these emotional expressions are crosscultural and crossmodal. Perhaps, most importantly this study provides evidence for the fundamental link between music and movement -- they share a dynamic structure.
For the paper, click here.