Functional imaging during speech production

Abstract

Physiological studies of speech production have demonstrated that even simple articulation involves a range of specialized motor and cognitive processes and the neural mechanisms responsible for speech reflect this complexity. Recently, a number of functional imaging techniques have contributed to our knowledge of the neuroanatomical and neurophysiological correlates of speech production. These new imaging approaches have the advantage of permitting study of large numbers of normal and disordered subjects but they bring with them a host of new methodological concerns. One of the challenges for understanding language production is the recording of articulation itself. The problems associated with measuring the vocal tract and measuring the neural activity during overt speech are reviewed. It is argued that advances in understanding fundamental questions such as what are the planning units of speech, what is the role of feedback during speech and what is the influence of learning, await the development of better methods for assessing task performance.

Introduction

The production of spoken words is the end product of a complex network of linguistic and cognitive processes. Thoughts and intentions are remarkably transformed into a sequence of movements and sounds in the order of hundreds of milliseconds. One of the great achievements of psycholinguistic research in the past 30 years has been the preliminary system identification of this language output system. Distinct phases of planning and control in this process have been identified through the study of errors (e.g., Fromkin, 1973) and studies of the timing of production (Levelt, 1989) and formal models of language production have been proposed to account for these data.

Levelt's model is representative of these frameworks and it is summarized in Fig. 1. As can be seen in this figure, the progression from concept to articulation is thought to involve separate processes that impart syntactic, morphological, phonological and phonetic organization. At the end of this flow of information processing lies the final and often forgotten stage of the language sequence, articulation itself. Most psycholinguistic models of this kind give minimal attention to the final stage of actual speech production. Like a last minute addition to a guest list, articulation does not feature in early phases of planning. In fact, in many models of language production, articulation is portrayed as a passive and modular final filter for language production. For a variety of technical reasons, this de-emphasis of articulation has been mirrored in functional imaging studies of language production. Many neural imaging studies have used a range of “silent” tasks in which words are not even spoken.¹ This is unfortunate because it de-emphasizes the medium in which the evolution of language presumably took place and it makes an unwarranted “pure insertion” assumption about speech motor control (Jennings, McIntosh, Kapur, Tulving, & Houle, 1997).

There are many reasons for giving the articulation component of language production more attention in functional imaging studies. Foremost amongst these is the fact that the speech motor system is not a passive channel for the transmission of linguistic signals from prior planning stages. Rather, it transforms those signals in a number of ways. At the most peripheral level, the nonlinear mechanics of tissue and muscle, the inertial forces of the moving articulators and the complexities of force generation in muscles each contribute to the form of articulation and thus to the final form of the speech output. In order to accommodate this degree of nonlinearity, the nervous system is thought to have internal models of the vocal tract and the acoustic consequences of articulation (Guenther, 1994; Hirayama, Vatikiotis-Bateson, & Kawato, 1994; Jones & Munhall, 2000; Jordan, 1990, Jordan, 1996; Kawato, Furukawa, & Suzuki, 1987). An internal model is a representation of the forces and kinematics of movements as well as the feedback that results from those movements (see Miall & Wolpert, 1996; Kawato, 1999, for discussions of internal models in movement control). In this depiction of the act of speaking, articulation involves significant information processing in order to manage the motor system in the vocal tract during phonetic sequencing. Timing, force, and trajectory control of a large number of articulators have to be programmed in rapid succession. This computation engages a neurally distributed planning and control system.

The amount of information that must be conveyed and therefore programmed by the motor system is remarkable. Talkers modify their speech style and precision in real time to fit social and environmental context and to respond to conversational demands. Mood, intent, attention, as well as the conceptual and emotional meaning of a message are all transmitted in parallel with phonetic information by subtle differences in the way in which words are spoken. These subtle differences are produced by systematic modifications to the movement timing and trajectories of the oral articulators. For example, talkers can voluntarily modify the clarity with which they speak (Gagne, Masterson, Munhall, Bilida, & Querengesser, 1995) and speech is perceptually distinct if it is read, recited, voluntarily produced, attentionally engaged, etc. (e.g., Remez, Lipton, & Fellowes, 1996). In addition, there is variability in articulation from a number of other sources. Speaking rate, lexical and emphatic stress and syllabification can vary from repetition to repetition.

How this motor planning is accomplished is not well understood but an interaction between levels of planning is indicated. For example, Saltzman, Lofqvist, and Mitra (2000) have used a phase resetting paradigm to study the relationship between central “clock” mechanisms and the timing of peripheral motor events in speech. When randomly timed mechanical perturbations are applied to the lower lip during repetitive syllable production, the patterns of timing adjustments are consistent with a model in which central timing is modulated by the peripheral motor system (Gracco & Abbs, 1989). Kawato (1997) has suggested that this arrangement is preferred for computational reasons as well. During trajectory planning, the solution space can be reduced more effectively if constraints (e.g., smoothness) are specified at other planning levels (Kawato, 1997). To do this however, the trajectory planning (along with other information processing problems in motor control such as coordinate transformations and motor command generation) must be computed simultaneously rather than sequentially.

Understanding articulation, then, is not simply a matter of understanding how the jaw, for example, moves for a canonical phoneme target. Rather, such an understanding must include a description of coordination in the context of the range of performance conditions occurring during communication. Trajectory formation for the articulators would thus involve interaction with many levels of the language system as well as involving complex motor planning in order to achieve these goals in the presence of the biomechanical and physiological complexity of the vocal tract (Munhall, Kawato, & Vatikiotis-Bateson, 2000).

The nonadditivity of cognitive processing and type of response has recently been demonstrated in a semantic judgment task (Jennings et al., 1997). PET activation patterns² varied for the same semantic task depending on the response mode (mouse click, overt speech, silent thought). Jennings et al. suggest that the way in which subjects organized their responses induced activation in different areas of the brain for the semantic processing. This kind of interaction is consistent with a system with many reciprocal connections and feedback projections such as the language system.

Factors such as these argue strongly for careful study of speech production in functional imaging studies. From a practical point of view, the manner in which words are spoken defines part of the “task” in any language production study and the presence of real articulation permits more precise task monitoring in imaging studies. Response timing, accuracy and task compliance can only be measured when there actually is a response! There is good evidence that this should be a concern in language production studies. Untrained talkers are not very precise at controlling many speech variables in laboratory studies of language output. For example, speaking rate, in spite of its popularity as a manipulation in speech production studies, is notoriously variable (Miller & Baer, 1983). Rate of articulation has also been reported to influence regional blood flow in PET experiments (e.g., Paus, Perry, Zatorre, Worsley, & Evans, 1996).

Indefrey and Levelt (2000) have recently argued that a range of poorly understood “lead-in” processes used in neuroimaging studies of word production (e.g., picture naming, generating words from beginning letter, word reading, repetition) have to be more carefully considered since they influence neuroimaging results. In a similar vein, details of articulation define a range of “lead-out” processes that will influence imaging data. Variability in speech obviously can result from different neural control strategies at many levels of the production system and this will be reflected in different neural activation patterns.

In addition to these methodological reasons for studying the functional representation of overt speech there are more theoretical reasons as well. Speech can be viewed as a complex linguistic process (Liberman & Whalen, 2000), and thus the neural representation of speech gestures is seen as an inextricable part of the language system. In Liberman's view, the phonetic gestures are the “common currency” of two-way communication and thus are the primitives of human language.

More pragmatically, there is a set of classic problems in the understanding of articulation that can profitably be addressed using functional imaging. These include (a) specifying the fundamental units of speech coordination, understanding the implementation of these units in a range of contexts that influence precision and timing, (b) understanding how central representation of units interact with feedback in real time and (c) understanding how these units are acquired and modified by learning. For each of these problems imaging could aid in both the neural mapping of the behavior and potentially the specification of the processing components of the behavior (Kosslyn, 1999). In the final section of the paper I will return to these problems.

To date, functional imaging has played two distinct roles in the study of speaking. First, a form of functional structural imaging has long been important for the precise description of the actual behavior of talking. Secondly, functional neural imaging has recently contributed to our understanding of the cognitive and linguistic processes responsible for articulation and for mapping these putative processes onto the neural architecture. In this paper I will give an overview of these two roles played by imaging, particularly magnetic resonance imaging (MRI), and comment on how functional imaging can help resolve enduring problems in the field. I will begin by commenting on the state of functional structural and functional neural imaging using MR.