Abstract accepted for presentation at Georgetown University Roundtable on Languages and Linguistics (GURT): Discourse Analysis and Technology: Multimodal Discourse Analysis, Washington DC March 7-9, 2002.
What interactive gestures may reveal about the structure of dialogue
According to Clark's (1996) theory of grounding, conversation consists of two parallel tracks of action. On the one hand, interlocutors communicate topical content and on the other hand, interlocutors communicate information about the process of communication itself. Clark's hypothesis is that interlocutors are concerned with making sure that their contributions are mutually understood or grounded. This idea is predicated on the notion that conversation is a joint action and requires the coordination of individual actions by conversational participants.
In his seminal work, "Hand and Mind", David McNeill (1992) made a strong argument that gestures are not only a part of language, but that they must be represented in conversational discourse structure. In fact, McNeill claimed that gesture not only often reflects discourse structure in conversation, but that accompanying speech may not. Though many of McNeill's observations about the function and role of gesture in discourse were based on spoken monologue, Janet Bavelas et al. (1992) specifically conducted studies comparing monologue and dialogue discourse. They categorically distinguish two broad groups of spontaneous gesture: topic and interactive gestures. Topic gestures are believed to depict semantic information relevant to the topic of discourse while interactive gestures are believed to serve a "social" function of coordinating information exchange between dialogue participants. Interactive gestures are considered to play a role in the social grounding process but do not relate directly to the informational content of dialogue.
In this presentation, I look at interactive gestures from a different perspective than that of Bavelas et. al. (1992). I am concerned with the role of interactive gestures in conveying information. I show that interactive gestures play a role in both pragmatic and semantic processes in dialogue. On the one hand, interactive gestures participate in the grounding process as Bavelas et al (1992, 1995) suggest. On the other hand, at least some interactive gestures simultaneously contribute to a speaker's intended meaning. These gestures index information in the common ground and may also be associated with the creation of new discourse topics. I conclude that a linguistic analysis of utterances containing interactive gestures lend support to David McNeill's claim; namely, that gestural cues give insight into the structure of discourse. I argue that this kind of re-evaluative analysis can help in the development of predictive models of behavior for the purpose of generating more naturalistic behaviors by embodied conversational agents. Furthermore, follow-up research may provide fruitful support for psycholinguistic theories advocating that gesture has access to a linguistic model of discourse.