The MobEyeLab Freiburg uses mobile eye tracking devices to study the manifold functions of eye gaze in human interaction. On this page, we introduce the different projects that are currently conducted by the members of our lab.
Gaze and turn-taking
The turn-taking-model, as introduced by Sacks, Jefferson and Schegloff in 1974, mentions non-verbal cues that regulate turn-taking, i.e. the organization of who gets to speak when in interaction, only in passing. It is only in recent years that we have witnessed an increasing interest in the role of the non-verbal resources for turn-taking. Researchers from our lab are contributing to this discussion by studying the role of eye gaze for turn allocation in triadic interactions.
Most notably, Peter Auer and Clarissa Weiß share an interest in the role of gaze as a next-speaker selection technique in triadic interactions. Peter Auer has recently published a paper (2018) on gaze patterns that speakers use for next-speaker selection and address, respectively. Clarissa Weiß recently completed her PhD project which zooms in on the implications of selection by gaze for the resolution of various potentially problematic turn-taking issues (e.g. how gaze selected speakers use gaze to pass on an unwanted turn, cf. Weiß 2018; cf. Weiß 2019).
Furthermore, Elisabeth Zima is interested in the role of gesture and gaze for turn management in joint triadic storytelling activities (see Zima 2018).
We investigate turn transition cues from a comprehensive, multimodal perspective including gaze but also prosody and gesture in a DFG funded project co-directed by Christoph Rühlemann, Peter Auer, Stefan Gries and Marion Schulte (in collaboration with Mercator fellow Judith Holler) to start later in 2022:
Der Beitrag multimodaler Abschlussignale zur zeitlich präzisen Turn-Übergabe
Gaze as an overlap management device
One of the essential characteristics of (face-to-face) conversation is speakers' tendency to avoid overlap and gaps between turns as much as possible. Nonetheless, in multi-party interaction it is not uncommon for two (or even more) speakers to start turns simultaneously, leading to a situation of overlap, which needs to be resolved in order to return to the core principle of "one speaker at a time". In a recent paper, Elisabeth Zima, Clarissa Weiß and our affiliated lab member Geert Brône present gaze patterns that play a role in the resolution of overlap that is caused by simultaneous co-starts (Zima et al. 2018).
Gaze and recipient feedback
The relationship between speaker and recipient gaze and verbal as well as non-verbal feedback is investigated in an ongoing PhD project by Johanna Hantsch. It has long been acknowledged that listeners are not passive recipients but actively shape the course of an ongoing conversation. Previous research on gaze and its impact on the occurrence of listener feedback has focused on the role of speaker gaze for eliciting feedback (e.g. Bavelas et al. 2002, Stivers & Rossano 2010). Little attention, however, has been paid to how recipients use gaze together with verbal and non-verbal feedback to actively participate in the ongoing talk. This project therefore aims for a finer-grained picture of the reciprocal dynamics of speaker and listener gaze and the interactional functions of different types of recipient behavior.
Gaze and recipient feedback in storytelling activities
In this project, we focus on the interplay of gaze and feedback in storytelling activities (dyadic, triadic with one narrator and two recipients, and joint storytelling of two co-narrators for a third participant). In 2002, Bavelas, Coates and Johnson have put forward an account of how gaze is used to elicit listener feedback in dyadic storytelling activities between unacquainted participants. Lab member Elisabeth Zima investigates whether the hypothesis that feedback is associated with so-called gaze windows, i.e. short phases of mutual gaze between narrators and recipients, holds also if eye gaze is measured directly, with mobile eye tracking glasses. She also investigates whether Bavelas et al.’s findings can be extended to triadic storytelling activities.
Gaze and storytelling
Christoph Rühlemann's DFG-funded project uses eye tracking to examine gaze behavior in storytelling in conversation based on an XML corpus, the Storytelling Interaction Corpus (STICO). Specifically the project uses eye-tracking data to address the following research questions:
- How is the suspension of ordinary turn-taking co-achieved by teller and recipient?
- How is gaze withdrawal correlated with the movement from Orientation to Climax?
- How is the recipient-to-be-spokesperson selected?
- How is the teller's gaze employed as a texturing device?
More on this project can be found under
and in Rühlemann/Matt/Ptak (2019).
Gaze patterns of joint storytelling activities
Telling a story together, based on a sared experience, is a compex task that involves constant coordination between co-tellers. Not only do need to negotiate who gets to tell which aspect of the story at which moment during teh activuty (turn-taking) but they also have to monitor engagement and understand of the recipient. Gaze playes a major role in borth the turn-taking process as well as the recipient's inclusion and her signalling of engagement and understanding (her multimodal backchannel behavior).
Gaze while talking and walking
When two or more persons walk together they often also talk - about copresent things (situated speech) or non-visible ones (displaced speech). Gaze has multiple functions in such a setting: as in immobile conversations, it serves regulatory functions (for instance, turn-taking), but in addition, co-walkers have to organize their joint movement and need to monitor their movements, they have to be attentive to the (changing) physical environment in which they move, and they use gaze and language to monitor and make noticeable to each other whatever they may find worth mentioning in the environment. Using eye-tracking and a synchronized quadrocopter we investigate the role of gaze and other bodily resources in this complex constellation.
We co-operate with the Nationalpark Schwarzwald (www.nationalpark-schwarzwald.de) to investigate how walkers experience nature and talk ab out it. This is also the dissertation project of Barbara Laner (see Laner 2022 for first results).
Looking, Noticing and Talking: How walkers experience the Black Forest National Park
Gaze, gestures and talk
Gestures are ubiquitous in interaction, but some of them are embedded into specific gaze patternings - among them, prominently, gestures linked to deictic elements of language. We are interested in the question of when and why speakers and/or recipients look at the speaker's gestures. A first study of gaze, gesturing and the German manner deictic so 'like' can bei found in Ningelgen & Auer (2017). Pfeiffer & Weiß (2022) investigate the use of gaze and gestures in enactments during story-tellings.