TY  - THES
T1  - The role of visual scenes in spoken language comprehension : evidence from eye-tracking
A1  - Knoeferle,Pia Stefanie
Y1  - 2005/04/05
N2  - The present thesis has investigated comprehension in situations in which language was related to the immediate scene. While current psycholinguistic theories have mainly been developed on the basis of insights from comprehension during reading of isolated sentences, we provide a description of how utterances are processed that relate to the immediate environment. The present research thus adds to existing theories of comprehension, making the first step towards a theory that can account for comprehension both when language is and when it is not about the immediate scene. We have suggested that, under the influence of Fodorian modularity, theoretical as well as experimental psycholinguistic research on on-line sentence comprehension has predominantly been concerned with comprehension during reading and with the effects of stored linguistic and world knowledge on comprehension. This was due to the assumption that core comprehension processes such as the structuring of an utterance could not be influenced on-line by ongoing non-linguistic processes such as visual perception. As a result, relatively little is known about the influence of scene perception on comprehension. One important study, however, by Tanenhaus et al. (1995) has demonstrated the rapid incremental influence of a visual referential context on the on-line structuring of an utterance. To extend our knowledge on how comprehension proceeds when language is related to the scene, we investigated two key issues. First, we explored which types of information in scenes have the potential to affect on-line comprehension processes such as structural disambiguation and incremental thematic role assignment. Second, we examined the nature of the interplay between visual perception and on-line utterance comprehension. Our findings show that perceived event relations between entities in the immediate visual context allow the rapid recovery of mental representations such as thematic roles in the on-line comprehension of spoken sentences, and actively influence incremental resolution of initial structural and role ambiguity in the linguistic input. The influence of scene information on comprehension was determined by a closely temporally coordinated interplay between visual perception and comprehension, in which both scene-driven and utterance-driven processes can take the lead. Furthermore, when the verb in the utterance did not uniquely identify either a depicted or another, stereotypical agent in the scene people had a clear preference for relying on role relations available from the depicted action performed by an agent in the scene rather than on their stereotypical thematic role knowledge. Our findings can be motivated by insights from language acquisition. During acquisition, children explore their environment and acquire new information from entities and events in the immediate scene. When they hear an adult describe that a girl is kicking a ball, and they do not know the verb "kick", perception of the event may allow them to acquire the conceptual meaning of the kicking-action, and to further infer that the entity doing the kicking is called "girl". Crucially, such perceptual exploration of the immediate scene for the acquisition of new concepts is utterance-mediated. The sentence directs attention to entities or events that then in turn can inform language acquisition. Taken together, these findings suggest that the important role of the coordinated interplay between visual perception and comprehension in language acquisition has fundamentally shaped the mechanisms of language comprehension.
KW  - Hörverstehen
KW  - Kontextsensitive Sprache
KW  - Situativer Kontext
KW  - Ereignis
KW  - Ambiguität
KW  - Disambiguierung
CY  - Saarbrücken
PB  - Universitäts- und Landesbibliothek
AD  - Postfach 151141, 66041 Saarbrücken
UR  - http://scidok.sulb.uni-saarland.de/volltexte/2005/438
ER  -