TUJOTA - Turn Taking at the Joint Action Table

Contact: Rosari Naveena Selvan

In this collaborative project between Prof. Florentin Wörgötter (head of Computational Neuroscience department at the University of Göttingen) and our lab, we aim to investigate the cognitive architecture of action perception. Our particular focus of interest is the neurocognitive basis of a behavior or function called ‘turn taking’ that has been first described in conversations. Here, taking turns means that while the listener still decodes what s/he hears, s/he already prepares the own upcoming utterance so that the average transition time between the two conversational partners is no more than 200 msec. Since the planning of an utterance itself takes considerably longer, it becomes obvious that turn taking entails several coincident anticipatory processes: predicting the approximate content of the so far unspoken, predicting the timing of the other’s current utterance and hence the most probable point in time where a reply is suitable, and preparing the own utterance. Turn taking is also evident in joint action but has been rarely investigated. In the current project, we measure brain responses, hand movements, and eye movements of an action observer getting reading for turn taking. Using computer vision at the Göttingen lab, we will assess cues that an observer derives from an observed object manipulation performed by an actor/actress and measure the observer’s eye movements as well as the point in time where s/he starts turn taking. Using these data points to model entropy and surprisal in an fMRI study in Münster, we will examine the brain activity in action observers presented with videos from the same actions. Our findings may contribute to the development of robots that can engage in joint action with human beings.