Data annotation for parallel investigation of prosody and gesture at turn boundaries: categorization of form and function

Talk abstract:
In this talk we present work-in-progress on our project Identifying
Intersections between Prosody, Gesture, and Conversation (Deutsche
Forschungsgemeinschaft). One of the challenges of a parallel
investigation of prosody and gesture is the development of an
appropriate annotation scheme which allows for the identification of
formal and/or functional relationships between these two modalities, as
well as ideally being applicable across languages with different
typological features. We will discuss our proposed annotation scheme,
providing examples from talk-in-interaction in Swedish and German, in
addition to identifying problematic cases and considering the best means
of resolving them. Our goal for the annotation scheme is to be able to
investigate the interplay of variation in the speech signal in parallel
with hand movements at locations where speaker transition becomes
relevant. In particular, we hope to determine whether there are trading
relationships between modalities, or alternatively, whether they make
mutual contributions to guiding the flow of the interaction.

Bios:
Meg Zellers is Junior Professor for Phonetics and Phonology at the
University of Kiel, Germany. Her current research projects include an
investigation of pre-turn-final prosodic features in German, and
prosodic and interactive analysis of the Bantu language Ruuli.

Jan Gorisch is a Researcher at the Archive for Spoken German at the
Leibniz-Institute for the German Language, Mannheim, Germany, developing
technology for corpus curation and dissemination. He is also currently
involved in the project Segmentation of Oral Corpora.

Note: Joint work with David House (KTH, Sweden) and Benno Peters
(University of Kiel, Germany)