Skip to main content
SpringerLink
Log in

Multisensory Functional Magnetic Resonance Imaging

  • Chapter
  • First Online:
Multisensory Object Perception in the Primate Brain

Abstract

Since its invention almost two decades ago functional magnetic resonance imaging (fMRI) has become the prime research methodology in human neuroscience. Its capabilities continue to evolve based on combined improvements of scanner hardware, experimental designs, and data analysis tools. Within the rapidly growing field of multisensory research the use of noninvasive neuroimaging techniques in general and fMRI in particular is also of increasing relevance. For several years, discussion in the multisensory fMRI community has mainly focused on principles of and statistical criteria for multisensory integration. The recent availability of more sophisticated experimental designs and increasingly sensitive (multivariate) analysis tools allows multisensory researchers to (noninvasively) differentiate between regional and neuronal convergence and to reveal the connectional basis of human multisensory integration.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Beauchamp MS (2005) Statistical criteria in FMRI studies of multisensory integration. Neuroinformatics 3:93–113

    Article  PubMed  Google Scholar 

  • Beauchamp MS, Laconte S, Yasar N (2009) Distributed representation of single touches in somatosensory and visual cortex. Hum Brain Mapp 30:3163–3171

    Article  PubMed  Google Scholar 

  • Beauchamp MS, Ro T (2008) Neural substrates of sound-touch synesthesia after a thalamic lesion. J Neurosci 28:13696–13702

    Article  PubMed  CAS  Google Scholar 

  • Calvert GA (2001) Crossmodal processing in the human brain: insights from functional neuroimaging studies. Cereb Cortex 11:1110–1123

    Article  PubMed  CAS  Google Scholar 

  • Calvert GA, Hansen PC, Iversen SD, Brammer MJ (2001) Detection of audio-visual integration sites in humans by application of electrophysiological criteria to the BOLD effect. Neuroimage 14:427–438

    Article  PubMed  CAS  Google Scholar 

  • Cappe C, Morel A, Barone P, Rouiller EM (2009) The thalamocortical projection systems in primate: an anatomical support for multisensory and sensorimotor interplay. Cerebral Cortex, Jan 15 [Epub ahead of print]

    Google Scholar 

  • David O, Cosmelli D, Friston KJ (2004) Evaluation of different measures of functional connectivity using a neural mass model. Neuroimage 21:659–673

    Article  PubMed  Google Scholar 

  • Deshpande G, Hu X, Stilla R, Sathian K (2008) Effective connectivity during haptic perception: a study using Granger causality analysis of functional magnetic resonance imaging data. Neuroimage 40:1807–1814

    Article  PubMed  Google Scholar 

  • Doehrmann O, Naumer MJ (2008) Semantics and the multisensory brain: how meaning modulates processes of audio-visual integration. Brain Res 1242:136–150

    Article  PubMed  CAS  Google Scholar 

  • Doehrmann O, Weigelt S, Altmann CF, Kaiser J, Naumer MJ (2010) Audio-visual fMRI adaptation reveals multisensory integration effects in object-related sensory cortices J Neurosci 30:3370–3379

    Google Scholar 

  • Driver J, Noesselt T (2008) Multisensory interplay reveals crossmodal influences on ‘sensory-specific’ brain regions, neural responses, and judgments. Neuron 57:11–23

    Article  PubMed  CAS  Google Scholar 

  • Eckert MA, Kamdar NV, Chang CE, Beckmann CF, Greicius MD, Menon V (2008) A cross-modal system linking primary auditory and visual cortices: evidence from intrinsic fMRI connectivity analysis. Hum Brain Mapp 29:848–857

    Article  PubMed  Google Scholar 

  • Etzel JA, Gazzola V, Keysers C (2008) Testing simulation theory with cross-modal multivariate classification of fMRI data. PLoS ONE 3:e3690

    Article  PubMed  Google Scholar 

  • Fischl B, Sereno MI, Tootel RBH, Dale AM (1999) High-resolution intersubject averaging and a coordinate system for the cortical surface. Hum Brain Mapp 8:272–284

    Article  PubMed  CAS  Google Scholar 

  • Friston KJ, Buechel C, Fink GR, Morris J, Rolls E, Dolan RJ (1997) Psychophysiological and modulatory interactions in neuroimaging. Neuroimage 6:218–229

    Article  PubMed  CAS  Google Scholar 

  • Friston KJ, Harrison L, Penny W (2003) Dynamic causal modelling. Neuroimage 19:1273–1302

    Article  PubMed  CAS  Google Scholar 

  • Goebel R, Esposito F, Formisano E (2006) Analysis of Functional Image Analysis Contest (FIAC) data with BrainVoyager QX: from single-subject to cortically aligned group general linear model analysis and self-organizing group independent component analysis. Hum Brain Mapp 27:392–402

    Article  PubMed  Google Scholar 

  • Goebel R, van Atteveldt N (2009) Multisensory functional magnetic resonance imaging: a future perspective. Exp Brain Res 198:153–164

    Article  PubMed  Google Scholar 

  • Grill-Spector K, Malach R (2001) fMR-adaptation: a tool for studying the functional properties of human cortical neurons. Acta Psychol (Amst) 107:293–321

    Article  CAS  Google Scholar 

  • Haller S, Wetzel SG, Radue EW, Bilecen D (2006) Mapping continuous neuronal activation without an ON–OFF paradigm: initial results of BOLD ceiling fMRI. Eur J Neurosci 24:2672–2678

    Article  PubMed  Google Scholar 

  • Hasson U, Skipper JI, Nusbaum HC, Small SL (2007) Abstract coding of audiovisual speech: beyond sensory representation. Neuron 56:1116–1126

    Article  PubMed  CAS  Google Scholar 

  • Haynes JD, Rees G (2006) Decoding mental states from brain activity in humans. Nat Rev Neurosci 7:523–534

    Article  PubMed  CAS  Google Scholar 

  • Haxby JV, Gobbini MI, Furey ML, Ishai A, Schouten JL, Pietrini P (2001) Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science 293:2425–2430

    Article  PubMed  CAS  Google Scholar 

  • Hein G, Doehrmann O, Muller NG, Kaiser J, Muckli L, Naumer MJ (2007) Object familiarity and semantic congruency modulate responses in cortical audiovisual integration areas. J Neurosci 27:7881–7887

    Article  PubMed  CAS  Google Scholar 

  • Hocking J, Price CJ (2008) The role of the posterior superior temporal sulcus in audiovisual processing. Cereb Cortex 18:2439–2449

    Article  PubMed  Google Scholar 

  • James TW, Humphrey GK, Gati JS, Servos P, Menon RS, Goodale MA (2002) Haptic study of three-dimensional objects activates extrastriate visual areas. Neuropsychologia 40:1706–1714

    Article  PubMed  Google Scholar 

  • Kim S, James TW (2009) Enhanced effectiveness in visuo-haptic object-selective brain regions with increasing stimulus salience. Hum Brain Mapp, Oct 14 [Epub ahead of print]

    Google Scholar 

  • Kriegeskorte N, Mur M, Bandettini P (2008a) Representational similarity analysis - connecting the branches of systems neuroscience. Front Syst Neurosci 2:4

    Article  PubMed  Google Scholar 

  • Kriegeskorte N, Mur M, Ruff DA, Kiani R, Bodurka J, Esteky H, Tanaka K, Bandettini PA (2008b) Matching categorical object representations in inferior temporal cortex of man and monkey. Neuron 60:1126–1141

    Article  PubMed  CAS  Google Scholar 

  • Lacey S, Tal N, Amedi A, Sathian K (2009) A putative model of multisensory object representation. Brain Topogr 21:269–274

    Article  PubMed  Google Scholar 

  • Laurienti PJ, Perrault TJ, Stanford TR, Wallace MT, Stein BE (2005) On the use of superadditivity as a metric for characterizing multisensory integration in functional neuroimaging studies. Exp Brain Res 166:289–297

    Article  PubMed  Google Scholar 

  • Logothetis NK (2008) What we can do and what we cannot do with fMRI. Nature 453:869–878

    Article  PubMed  CAS  Google Scholar 

  • Meienbrock A, Naumer MJ, Doehrmann O, Singer W, Muckli L (2007) Retinotopic effects during spatial audio-visual integration. Neuropsychologia 45:531–539

    Article  PubMed  CAS  Google Scholar 

  • Mur M, Bandettini PA, Kriegeskorte N (2009) Revealing representational content with pattern-information fMRI--an introductory guide. Soc Cogn Affect Neurosci 4:101–109

    Article  PubMed  Google Scholar 

  • Naumer MJ, Ratz L, Yalachkov Y, Polony A, Doehrmann O, Müller NG, Kaiser J, Hein G (2010) Visuo-haptic convergence in a cortico-cerebellar network. Eur J Neurosci (in press)

    Google Scholar 

  • Naumer MJ, van den Bosch JJF (2009) Touching sounds: thalamo-cortical plasticity and the neural basis of multisensory integration. J Neurophysiol 102:7–8

    Article  PubMed  Google Scholar 

  • Naumer MJ, van den Bosch JJF, Wibral M, Kohler A, Singer W, Kaiser J, van de Ven V, Muckli L (2009b) Audio-visual integration in the human brain: data-driven detection and independent validation.

    Google Scholar 

  • Niogi SN, McCandliss BD (2006) Left lateralized white matter microstructure accounts for individual differences in reading ability and disability. Neuropsychologia 44:2178–2188

    Article  PubMed  Google Scholar 

  • Noesselt T, Rieger JW, Schoenfeld MA, Kanowski M, Hinrichs H, Heinze HJ, Driver J (2007) Audiovisual temporal correspondence modulates human multisensory superior temporal sulcus plus primary sensory cortices. J Neurosci 27:11431–11441

    Article  PubMed  CAS  Google Scholar 

  • Noppeney U Characterization of multisensory integration with fMRI - experimental design, statistical analysis and interpretation. In: Wallace M, Murray M (eds) Frontiers in the neural bases of multisensory processes. Taylor and Francis Group, London (in press)

    Google Scholar 

  • Noppeney U, Josephs O, Hocking J, Price CJ, Friston KJ (2008) The Effect of Prior Visual Information on Recognition of Speech and Sounds. Cereb Cortex 18:598–609

    Article  PubMed  Google Scholar 

  • Pietrini P, Furey ML, Ricciardi E, Gobbini MI, Wu W-HC, Cohen L, Guazzelli M, Haxby JV (2004) Beyond sensory images: objectbased representation in the human ventral pathway. Proc Natl Acad Sci USA 101:5658–5663

    Article  PubMed  CAS  Google Scholar 

  • Polony A, Ratz L, Doehrmann O, Kaiser J, Naumer MJ (2007) Audio-tactile integration of meaningful objects in the human brain. In: Annual Meeting of the International Multisensory Research Forum, Sydney, Australia

    Google Scholar 

  • Roebroeck A, Formisano E, Goebel R (2005) Mapping directed influence over the brain using Granger causality and fMRI. Neuroimage 25:230–242

    Article  PubMed  Google Scholar 

  • Roebroeck A, Formisano E, Goebel R (2009a) The identification of interacting networks in the brain using fMRI: Model selection, causality and deconvolution. Neuroimage, Sep 25 [Epub ahead of print]

    Google Scholar 

  • Roebroeck A, Formisano E, Goebel R (2009b) Reply to Friston and David After comments on: The identification of interacting networks in the brain using fMRI: Model selection, causality and deconvolution. Neuroimage, Oct 31 [Epub ahead of print]

    Google Scholar 

  • Rouw R, Scholte HS (2007) Increased structural connectivity in grapheme-color synesthesia. Nat Neurosci 10:792–797

    Article  PubMed  CAS  Google Scholar 

  • Stein BE, Meredith MA (1993) The merging of the senses. Cambridge, Massachussetts: MIT Press

    Google Scholar 

  • Stein BE, Stanford TR (2008) Multisensory integration: current issues from the perspective of the single neuron. Nat Rev Neurosci 9:255–266

    Article  PubMed  CAS  Google Scholar 

  • Stevenson RA, Geoghegan ML, James TW (2007) Superadditive BOLD activation in superior temporal sulcus with threshold nonspeech objects. Exp Brain Res 179:85–95

    Article  PubMed  Google Scholar 

  • Stevenson RA, James TW (2009) Audiovisual integration in human superior temporal sulcus: Inverse effectiveness and the neural processing of speech and object recognition. Neuroimage 44:1210–1223

    Article  PubMed  Google Scholar 

  • Stevenson RA, Kim S, James TW (2009) An additive-factors design to disambiguate neuronal and areal convergence: measuring multisensory interactions between audio, visual, and haptic sensory streams using fMRI. Exp Brain Res 198:183–194

    Article  PubMed  Google Scholar 

  • Tal N, Amedi A (2009) Multisensory visual-tactile object related network in humans: insights gained using a novel crossmodal adaptation approach. Exp Brain Res 198:165–182

    Article  PubMed  Google Scholar 

  • van Atteveldt N, Blau V, Blomert L, Goebel R (2008) fMR-adaptation reveals multisensory integration in human superior temporal cortex. In: Annual Meeting of the International Multisensory Research Forum, Hamburg, Germany

    Google Scholar 

  • van Atteveldt NM, Formisano E, Blomert L, Goebel R (2007) The effect of temporal asynchrony on the multisensory integration of letters and speech sounds. Cereb Cortex 17:962–974

    Article  PubMed  Google Scholar 

  • van Atteveldt N, Roebroeck A, Goebel R (2009) Interaction of speech and script in human auditory cortex: Insights from neuroimaging and effective connectivity. Hear Res 258:152–164

    Article  PubMed  Google Scholar 

  • van Essen DC, Drury HA, Joshi S, Miller MI (1998) Functional and structural mapping of human cerebral cortex: solutions are in the surfaces. Proc Natl Acad Sci U S A 95:788–95

    Article  PubMed  Google Scholar 

  • von Kriegstein K, Giraud AL (2006) Implicit multisensory associations influence voice recognition. PLoS Biol 4:e326

    Article  Google Scholar 

  • von Kriegstein K, Kleinschmidt A, Sterzer P, Giraud AL (2005) Interaction of face and voice areas during speaker recognition. J Cogn Neurosci 17:367–376

    Article  Google Scholar 

  • Weigelt S, Muckli L, Kohler A (2008) Functional magnetic resonance adaptation in visual neuroscience. Rev Neurosci 19:363–380

    PubMed  Google Scholar 

  • Werner S, Noppeney U (2009) Superadditive responses in superior temporal sulcus predict audiovisual benefits in object categorization. Nov 18 [Epub ahead of print]

    Google Scholar 

Download references

Acknowledgments

This work was supported by the German Ministry of Education and Research (BMBF) and Frankfurt Medical School (Intramural Young Investigator Program to M.J.N.). We are grateful to Sarah Weigelt for helpful suggestions and Christoph Bledowski and Yavor Yalachkov for their helpful comments to an earlier version of this chapter.

Author information

Authors and Affiliations

  1. Institute of Medical Psychology, Faculty of Medicine, Goethe University, Heinrich-Hoffmann-Str. 10, 60528, Frankfurt am Main, Germany

    Marcus J. Naumer, Jasper J. F. van den Bosch, Andrea Polony & Jochen Kaiser

Authors
  1. Marcus J. Naumer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jasper J. F. van den Bosch
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andrea Polony
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jochen Kaiser
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marcus J. Naumer .

Editor information

Editors and Affiliations

  1. Inst. Medizinische Psychologie (IMP), Universität Frankfurt, Heinrich-Hoffmann-Str. 10, Frankfurt, 60528, Germany

    Jochen Kaiser

  2. Abt. Neurophysiologie, MPI für Hirnforschung, Deutschordenstr. 46, Frankfurt/Main, 60528, Germany

    Marcus Johannes Naumer

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer Science + Business Media, LLC

About this chapter

Cite this chapter

Naumer, M.J., van den Bosch, J.J.F., Polony, A., Kaiser, J. (2010). Multisensory Functional Magnetic Resonance Imaging. In: Kaiser, J., Naumer, M. (eds) Multisensory Object Perception in the Primate Brain. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-5615-6_6

Download citation

Publish with us

Policies and ethics

Search

Navigation