When eyes beat lips

speaker gaze affects audiovisual integration in the McGurk illusion

verfasst von
Basil Wahn, Laura Schmitz, Alan Kingstone, Anne Böckler-Raettig
Abstract

Eye contact is a dynamic social signal that captures attention and plays a critical role in human communication. In particular, direct gaze often accompanies communicative acts in an ostensive function: a speaker directs her gaze towards the addressee to highlight the fact that this message is being intentionally communicated to her. The addressee, in turn, integrates the speaker’s auditory and visual speech signals (i.e., her vocal sounds and lip movements) into a unitary percept. It is an open question whether the speaker’s gaze affects how the addressee integrates the speaker’s multisensory speech signals. We investigated this question using the classic McGurk illusion, an illusory percept created by presenting mismatching auditory (vocal sounds) and visual information (speaker’s lip movements). Specifically, we manipulated whether the speaker (a) moved his eyelids up/down (i.e., open/closed his eyes) prior to speaking or did not show any eye motion, and (b) spoke with open or closed eyes. When the speaker’s eyes moved (i.e., opened or closed) before an utterance, and when the speaker spoke with closed eyes, the McGurk illusion was weakened (i.e., addressees reported significantly fewer illusory percepts). In line with previous research, this suggests that motion (opening or closing), as well as the closed state of the speaker’s eyes, captured addressees’ attention, thereby reducing the influence of the speaker’s lip movements on the addressees’ audiovisual integration process. Our findings reaffirm the power of speaker gaze to guide attention, showing that its dynamics can modulate low-level processes such as the integration of multisensory speech signals.

Organisationseinheit(en)
Institut für Psychologie
Institut für Sportwissenschaft
Externe Organisation(en)
University of British Columbia
Typ
Artikel
Journal
Psychological research
Band
86
Seiten
1930-1943
Anzahl der Seiten
14
ISSN
0340-0727
Publikationsdatum
09.2022
Publikationsstatus
Veröffentlicht
Peer-reviewed
Ja
ASJC Scopus Sachgebiete
Experimentelle und kognitive Psychologie, Pädagogische und Entwicklungspsychologie, Geisteswissenschaftliche Fächer (sonstige)
Elektronische Version(en)
https://doi.org/10.1007/s00426-021-01618-y (Zugang: Offen)