Abstract
In this study we compared tactile and visual feedbacks for the motor imagery-based brain–computer interface (BCI) in five healthy subjects. A vertical green bar from the center of the fixing cross to the edge of the screen was used as visual feedback. Vibration motors that were placed on the forearms of the right and the left hands and on the back of the subject’s neck were used as tactile feedback. A vibration signal was used to confirm the correct classification of the EEG patterns of the motor imagery of right and left hand movements and the rest task. The accuracy of recognition in the classification of the three states (right hand movement, left hand movement, and rest) in the BCI without feedback exceeded the random level (33% for the three states) for all the subjects and was rather high (67.8% ± 13.4% (mean ± standard deviation)). Including the visual and tactile feedback in the BCI did not significantly change the mean accuracy of recognition of mental states for all the subjects (70.5% ± 14.8% for the visual feedback and 65.9% ± 12.4% for the tactile feedback). The analysis of the dynamics of the movement imagery skill in BCI users with the tactile and visual feedback showed no significant differences between these types of feedback. Thus, it has been found that the tactile feedback can be used in the motor imagery-based BCI instead of the commonly used visual feedback, which greatly expands the possibilities of the practical application of the BCI.
Similar content being viewed by others
References
Wolpaw, J.R., Birbaumer, N., Mcfarland, D.J., et al., Brain–computer interfaces for communication and control, Clin. Neurophysiol., 2002, vol. 113, no. 6, p. 767.
Kaplan, A.Ya., Zhigul’skaya, D.D., and Kir’yanov, D.A., The possible control of individual fingers of the phantom of the human hand in the contour of the brain-computer interface on the P300 wave, Vestn. Ross. Gos. Med. Univ., 2016, no. 2, p. 26.
Birbaumer, N. and Cohen, L.G., Brain–computer interfaces: communication and restoration of movement in paralysis, J. Physiol., 2007, vol. 579, no. 3, p. 621.
Frolov, A.A., Húsek, D., Biryukova, E.V., et al., Principles of motor recovery in post-stroke patients using hand exoskeleton controlled by the brain-computer interface based on motor imagery, Neural Network World, 2017, vol. 107, p. 137.
Kotov, S.V., Turbina, L.G., Bobrov, P.D., et al., Rehabilitation of stroke patients with a bioengineered “brain–computer interface with exoskeleton” system, Neurosci. Behav. Physiol., 2016, vol. 46, no. 5, p. 518.
Kaplan, A., Vasilyev, A., Liburkina, S., and Yakovlev, L., Poor BCI performers still could benefit from motor imagery training, Proc. 10th Int. Conf. “AC Foundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience,” Toronto, ON, Canada, July 17–22, 2016, New York: Springer-Verlag, 2016, part 1, p. 46.
Vasilyev, A., Liburkina, S., Yakovlev, L., et al., Assessing motor imagery in brain-computer interface training: psychological and neurophysiological correlates, Neuropsychologia, 2017, vol. 97, p. 56.
Mokienko, O.A., Chervyakov, A.V., Kulikova, S.N., et al., Increased motor cortex excitability during motor imagery in brain-computer interface trained subjects, Front. Comput. Neurosci., 2013, vol. 7, p. 168. doi 10.3389/fncom.2013.00168
Mulder, T., Motor imagery and action observation: cognitive tools for rehabilitation, J. Neural Transm., 2007, vol. 114, no. 10, p. 1265.
De Vries, S., Tepper, M., Feenstra, W., et al., Motor imagery ability in stroke patients: the relationship between implicit and explicit motor imagery measures, Front. Hum. Neurosci., 2013, vol. 7, p. 790.
Simmons, L., Sharma, N., Baron, J.C., and Pomeroy, V.M., Motor imagery to enhance recovery after subcortical stroke: who might benefit, daily dose, and potential effects, Neurorehabil. Neural Repair, 2008, vol. 22, no. 5, p. 458.
Vuckovic, A. and Osuagwu, B.A., Using a motor imagery questionnaire to estimate the performance of a brain–computer interface based on object oriented motor imagery, Clin. Neurophysiol., 2013, vol. 124, no. 8, p. 1586.
Alimardani, M., Nishio, S., and Ishiguro, H., The importance of visual feedback design in BCIs; from embodiment to motor imagery learning, PloS One, 2016, vol. 11, no. 9, p. e0161945.
Sollfrank, T., Ramsay, A., Perdikis, S., et al., The effect of multimodal and enriched feedback on SMR-BCI performance, Clin. Neurophysiol., 2016, vol. 127, no. 1, p. 490.
Kaufmann, T., Holz, E.M., and Kübler, A., Comparison of tactile, auditory, and visual modality for brain–computer interface use: a case study with a patient in the locked-in state, Front. Neurosci., 2013, vol. 7, p. 129.
Kauhanen, L., Palomäki, T., Jylänki, P., et al., Haptic feedback compared with visual feedback for BCI, Proc. 3rd Int. Brain-Computer Interface Workshop & Training Course 2006, Graz, 2006.
Angulo-Sherman, I.N. and Gutierrez, D., Effect of different feedback modalities in the performance of braincomputer interfaces, 24th Int. Conf. on Electronics, Communications and Computers (CONIELECOMP 2014), México: Univ. Am. Puebla, 2014, p. 14.
Godlove, J.M., Whaite, E.O., and Batista, A.P., Comparing temporal aspects of visual, tactile, and microstimulation feedback for motor control, J. Neural Eng., 2014, vol. 11, no. 4, p. 046025.
Halder, S., Rea, M., Andreoni, R., et al., An auditory oddball brain–computer interface for binary choices, Clin. Neurophysiol., 2010, vol. 121, no. 4, p. 516.
Schreuder, M., Rost, T., and Tangermann, M., Listen, you are writing! Speeding up online spelling with a dynamic auditory BCI, Front. Neurosci., 2011, vol. 5, p. 112.
Hinterberger, T., Neumann, N., Pham, M., et al., A multimodal brain-based feedback and communication system, Exp. Brain Res., 2004, vol. 154, no. 4, p. 521.
Pham, M., Hinterberger, T., Neumann, N., et al., An auditory brain-computer interface based on the selfregulation of slow cortical potentials, Neurorehabil. Neural Repair, 2005, vol. 19, no. 3, p. 206.
Nijboer, F., Furdea, A., Gunst, I., et al., An auditory brain–computer interface (BCI), J. Neurosci. Methods, 2008, vol. 167, no. 1, p. 43.
Furdea, A., Halder, S., Krusienski, D.J., et al., An auditory oddball (P300) spelling system for brain computer interfaces, Psychophysiology, 2009, vol. 46, no. 3, p. 617.
Schreuder, M., Blankertz, B., and Tangermann, M., A new auditory multi-class brain-computer interface paradigm: spatial hearing as an informative cue, PloS One, 2010, vol. 5, no. 4, p. e9813.
Changa, M., Morib, K., Makinoa, S., and Rutkowsk, T.M., Spatial auditory two-step input Japanese syllabary brain-computer interface speller, Procedia Technol., 2014, vol. 18, p. 25.
Leeb, R., Gwak, K., Kim, D.-S., and del Millán, J.R., Freeing the visual channel by exploiting vibrotactile BCI feedback, 35th Annual Int. Conf. of the IEEE Engineering in Medicine and Biology Society (EMBC), Piscataway, NJ: IEEE Eng. Med. Biol. Soc., 2013, p. 3093.
Cincotti, F., Kauhanen, L., Aloise, F., et al., Vibrotactile feedback for brain-computer interface operation, Comput. Intell. Neurosci., 2007, vol. 2007, art. ID 48937. doi 10.1155/2007/48937
Gwak, K., Leeb, R., del Millán, J.R., and Kim, D.-S., Quantification and reduction of visual load during BCI operation, IEEE Int. Conf. on Systems, Man, and Cybernetics, San Diego, 2014, p. 2795.
Jeunet, C., Vi, C., Spelmezan, D., et al., Continuous tactile feedback for motor-imagery based brain-computer interaction in a multitasking context, Int. Conf. on Human-Computer Interaction, INTERACT 2015, New York: Springer-Verlag, 2015, p. 488.
Gomez-Rodriguez, M., Peters, J., Hill, J., et al., Closing the sensorimotor loop: haptic feedback facilitates decoding of motor imagery, J. Neural Eng., 2011, vol. 8, no. 3, p. 036005.
Vourvopoulos, A., Ferreira, A., and Badia, S.B., Neu-Row: an immersive VRenvironment for motor-imagery training with the use of brain-computer interfaces and vibrotactile feedback, Proc. 3rd Int. Conf. on Physiological Computing Systems (PhyCS 2016), Lisbon, 2016, p. 43. doi 10.5220/0005939400430053
Oldfield, R.C., The assessment and analysis of handedness: the Edinburgh inventory, Neuropsychologia, 1971, vol. 9, no. 1, p. 97.
Doskin, V.A., Lavrent’eva, N.A., Miroshnikova, M.P., and Sharai, V.B., Test of differential self-evaluation of a functional state, Vopr. Psikhol., 1973, vol. 6, p. 141.
Koles, Z.J., Lazar, M.S., and Zhou, S.Z., Spatial patterns underlying population differences in the background EEG, Brain Topogr., 1990, vol. 2, no. 4, p. 275.
Trefethen, L.N. and Bau, III D., Numerical Linear Algebra, Philadelphia: Soc. Ind. Appl. Math., 1997. ISBN 978-0-89871-361-9.
http://martinos.org/mne/stable/generated/mne.decoding. CSP.html.
Hjorth, B., An on-line transformation of EEG scalp potentials into orthogonal source derivations, Electroencephalogr. Clin. Neurophysiol., 1975, vol. 39, no. 5, p. 526.
R.C. Team, R: A language and environment for statistical computing, R Foundation for Statistical Computing, Vienna, 2016. http://www.R-project.org.
Lawrence, M.A., ez: Easy analysis and visualization of factorial experiments, R Package version 4.4-0, 2016. https://CRAN.R-project.org/package=ez.
Thurlings, M.E., van Erp, J.B.F., Brouwer, A.M., et al., Control-display mapping in brain–computer interfaces, Ergonomics, 2012, vol. 55, no. 5, p. 564.
Author information
Authors and Affiliations
Corresponding author
Additional information
Original Russian Text © M.V. Lukoyanov, S.Yu. Gordleeva, A.S. Pimashkin, N.A. Grigor’ev, A.V. Savosenkov, A. Motailo, V.B. Kazantsev, A.Ya. Kaplan, 2018, published in Fiziologiya Cheloveka, 2018, Vol. 44, No. 3, pp. 53–61.
Rights and permissions
About this article
Cite this article
Lukoyanov, M.V., Gordleeva, S.Y., Pimashkin, A.S. et al. The Efficiency of the Brain-Computer Interfaces Based on Motor Imagery with Tactile and Visual Feedback. Hum Physiol 44, 280–288 (2018). https://doi.org/10.1134/S0362119718030088
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1134/S0362119718030088