Communication & Multimodaliy

Friday 14:00- 14:45



„Establishing Conversational Pedagogical Agents as Credible Knowledge Providers: the Case of Synthesized Italian English“

Sven Albrecht, Rewa Tamboli, Stefan Taubert, Felicia Meusel, Maximilian Eibl, Günter Daniel Rey and Josef Schmied

Abstract— English is the dominant language in academia, with twice as many non-native speakers as native speakers. Previous studies have shown that if a teacher speaks in a non-native variety that is similar to that of the learners, it positively impacts information processing and retention. Therefore, a pedagogical agent speaking a non-native variety of English could improve the learning outcomes of university students speaking the same non-native variety. In a pre-study in Italy, we evaluated our hypothesis that increased perceived credibility of an agent speaking non-native English improves the learning outcomes of undergraduates. Contrary to our expectations, preliminary results indicate that the standard American variety of English is perceived as more credible than the Italian variety of English. However, participants learned equally well with both varieties.

Keywords. credibility, linguistics, phonetics, evaluation, TTS, speech synthesis



„Modelling Intentional Complexity in Hybrid Interaction Scenarios Beyond Explicit and Implicit Communication“

Martin Siefkes, Ellen Fricke, Jana Bressem and Akira Charoensit

Abstract—The present contribution investigates the role of intentionality in human-machine interactions. Building on previous research on intentionality in human-human interactions, it is shown that different “levels of intending” (including complex nested intentions and beliefs) need to be taken into account; dichotomies such as “implicit vs. explicit” are insufficient to capture the necessary distinctions. Adequately guessing the degree of intending of interaction behavior, from basic actions up to and including communication processes, requires taking into account all relevant aspects of the behavior (speech, gesture, gaze, and further aspects of movement and/or body behavior). Furthermore, an adequate “model of intentionality” is needed in order to infer the degree of intending of an interaction based on various types of cues. It will be argued that Embodied Digital Technologies (EDTs) with the capabilities necessary to adequately infer and represent intentions and beliefs in their mental models of themselves and of interactants may be able to achieve improved situational awareness and a more human-like interaction quality.


Keywords: intentionality, human-robot interaction, communication, multimodality, gestures, complexity, speech act theory



„Designing Computer-mediated Communication with Affective Technology to Increase Feedback Acceptance“

Katharina Jahn, Oliver Rehren, Bastian Kordyaka, Sebastian Jansen, Peter Ohler and Günter Daniel Rey

Abstract—As text-based computer-mediated communication (CMC) supported by affective technology becomes increasingly common in our daily life, new opportunities for the communication of critical information, such as negative feedback, arise. Research on affective technology has already shown that the acceptance of negative feedback can be improved by using emoticons under specific conditions. However, in which way emotion recognizing affective technology can increase the acceptance of negative feedback automatically is still unclear. We hypothesized that automatically reported stress and a low stress level increases feedback acceptance and its predictors. Additionally, we hypothesized an interaction effect that could attenuate the negative effect of high stress when the stress level is automatically detected. Using a messenger that reports the feedback provider’s stress level to the feedback recipient, we investigate how the automaticity of stress detection and the displayed stress level can increase negative feedback acceptance and its predictors. We conducted a 2 (stress level: low vs. high) x 2 (automaticity: automatically detected vs. self-reported) + 1 (control group) between-subjects laboratory experiment, resulting in five experimental groups. Our results show that whereas an automatically detected stress level increases perspective taking, seeing a low stress level increases perceived good intention.


Keywords: affective technology, feedback acceptance, negative feedback, emoticons