Language, Modality, and the Brain
The general objective of our research is to study the biological foundations of human language. Signed languages provide a powerful tool for investigating the nature of human language and language processing, the relation between cognition and language, and the neural organization for language. For perception, signed languages depend upon high-level vision and motion processing systems, and for production, they require the integration of motor systems involving the hands and face. These facts raise many questions: What impact does this different biological base have for grammatical systems? For the acquisition of language? How does it affect non-linguistic cognitive structures and processing? Are the same or different neural systems involved? Our laboratory investigates these issues by studying Deaf and hearing ASL signers.
Linguistic Studies of American Sign Language
Because the vast majority of what we know about human language comes from the study of spoken languages, increased knowledge of signed languages help us to determine whether linguistic properties are shaped by the auditory nature of speech perception or the modality of transmission (i.e. the vocal tract vs. the hands). Our research has focused a unique aspect on sign language expression: the linguistic use of physical space. "Classifier" predicates are universal to signed languages, and in these constructions, handshape specifies object type, and the position of the hands in signing space schematically represents the spatial relations between objects. In contrast, spoken languages tend to express spatial information through adpositions, adverbs, and the lexical semantics of different verbs. Our research program investigates the ramifications of these distinct linguistic properties for a) the nature of human linguistic systems and b) the relation between language and spatial cognition.
Psycholinguistic Studies
One goal of our pyscholinguistic research is to examine what aspects of language processing and production may be universal and what aspects are affected by the particular characteristics of audition vs. vision or by the specific constraints on gestural vs. vocal articulation. Our laboratory has adapted and developed several psycholinguistic paradigms which permit exploration of real-time processing of a visual-spatial language. Our results indicate that lexical access and word recognition processes are similar for signed and spoken languages, but language modality effects are observed in the speed of sign identification. Our studies also show that signed and spoken languages use the same processing, mechanisms in resolving and interpreting pronouns, even though ASL uses spatial locations to convey co-reference. Finally, our language production studies indicate that signers exhibit "tip-of-the-fingers" effects, slips of the hand, and produce gestures in conjunction with signing. Our results primarily reveal similarities between the mechanisms involved in comprehending and producing both spoken and signed languages.
We are also conducting some of the first experiments using head-mounted eye tracking technology to study eye behaviors during the perception and production of ASL. We are investigating how the social and conversational functions of eye gaze interact with the requirements of eye gaze for sign perception. Another set of studies investigates eye movements during sign production. These studies address theoretical claims regarding the grammatical functions of eye gaze in ASL and identify how signers co-ordinate their eye movements with the linguistic structure of signed sentences. In addition, we are comparing the eye behaviors of native deaf signers with adult late learners (hearing and deaf) during sign perception and while signing. The use of eye gaze to mark linguistic structure is unique to signed languages, and the results of these studies provide insight into how language modality affects the nature of grammatical encoding and what eye gaze behaviors must be learned in order to perceive and produce ASL efficiently and effectively.
Language and Cognition
Our research has revealed that the habitual use of a visual-spatial language such as ASL has an impact on nonlingustic aspects of visual-spatial cognition. The effects appear to be due to linguistic experience, rather than to deafness, because hearing people who learned ASL as their first language from their Deaf parents exhibit the same patterns of performance as Deaf ASL signers. Sign language experience appears to enhance or alter performance within certain cognitive domains, while leaving other domains unaffected, as outlined below:
Summary of the Impact of Sign Language Use on Visuospatial Cognition
 |
||||||||||
 |
Affected Domains | Unaffected Domains | ||||||||
|
||||||||||
|
||||||||||
|
||||||||||
|
||||||||||
Because signed languages rely on visual-spatial cognitive processes, they provide a unique tool for investigating the relation between linguistic and non-linguistic domains of cognition.
Sign Language and the Brain
One over-arching finding that emerges from our studies of both neurologically intact and brain-injured signers is that the left cerebral hemisphere is critical for sign language processes, as it is for spoken language.
Several structures within the left hemisphere that have been shown to be involved in spoken language comprehension and production are also recruited for sign language. These results demonstrate the extent and limits of neural plasticity in the developing brain. Further, they show that left hemisphere specialization for language does not arise from the particular demands of auditory speech perception.
Our current studies focus on uncovering what neural regions are engaged in processing constructions, linguistic facial expressions, and the use of space within discourse.
