Paper accepted in Psychonomic Bulletin & Review!
Curious about how beat gestures can affect the words people hear in Spanish? Read it in: “From ‘I dance’ to ‘she danced’ with a flick of the hands: Audiovisual stress perception in Spanish”
Published!
This week, our paper “From ‘I dance’ to ‘she danced’ with a flick of the hands: Audiovisual stress perception in Spanish” was published in Psychonomic Bulletin & Review, authored by Patrick Louis Rohrer, Ronny Bujok, Lieke van Maastricht, and Hans Rutger Bosker. The full-text and all data are publicly available at the links provided at the bottom of this post.
What’s it about?
This study explored how hand gestures can change the way people hear words in Spanish. Spanish has words that change meaning based on which syllable is stressed. For example, bailo means “I dance” while bailó means “she danced”, making stress patterns very important for Spanish listeners. In addition to stress, we also gesture when we speak. For instance, non-referential “beat” gestures represent simple hand movements that go up and down, marking stressed syllables in speech.
What we did was to show videos of a speaker saying these types of words while using beat gestures either on the first or second syllable. Crucially, many of these words were acoustically manipulated to have varying degrees of ambiguity — that is, some words sounded more bailo-like, or bailó-like, or were completely ambiguous. Participants watched these videos and were asked to say which version of the word they thought they heard.
The results showed that when the beat gesture matched a certain syllable, listeners were more likely to think that syllable was stressed. So if the hand moved with the first syllable, people often heard bailo, but if it moved with the second, they heard bailó. This effect was even stronger when the audio was unclear or ambiguous. Interestingly, a person’s working memory (an individual characteristic of how well they could remember sounds or visuals) didn’t seem to change how much they were influenced by the gestures.
Why is this important?
This research is important because it shows that we don’t just use our ears to understand spoken language — we also use our eyes. Simple gestures that don’t carry meaning on their own can still guide how we hear and interpret words.
The findings help us understand how spoken language works in real-life settings, where gestures often come with speech. It also has implications for language learning, teaching, and even speech technology, showing that visual cues are a powerful part of communication — even when they’re just small, rhythmic hand movements.
Full reference
The full citation, open access PDF, and all data are publicly available from the links below:
(2025). From “I dance” to “she danced” with a flick of the hands: Audiovisual stress perception in Spanish. Psychonomic Bulletin & Review, doi:10.3758/s13423-025-02683-9.