Active triggering control of pneumatic rehabilitation gloves based on surface electromyography sensors

Construction of pneumatic rehabilitation glove trigger control system based on sEMG

The pneumatic rehabilitation glove trigger control system based on sEMG consists of one pneumatic gloves, an air pump, a Stm32f103 microprocessor equipped with an ARM chip, two electric relays, a Myoware sEMG sensor, two-position three-way solenoid valves and a host computer as shown in Fig. 1. The pneumatic rehabilitation gloves can well wrap the patients’ fingers, palms and hand back. Air pump provides power for pneumatic gloves. sEMG sensors are used to collect patient’s sEMG signals. The Stm32f103 microprocessor equipped with an ARM chip is used to process the original sEMG signals collected by sEMG sensors. It is also used as the driver of air pump and transmits the processed sEMG signals to the host computer. The host computer is developed with QT software (Cross-platform software development framework for the development of apps and devices, developed by QT Group) as the development environment. It judges the movement trend of the hand by analyzing the collected sEMG signals. According to the movement trend of the hand, it also sends related instructions to the air pump driver. Then the air pump driver controls pneumatic rehabilitation gloves to flex and extend. The above hardware platform can be divided into an acquisition layer, a decision-making layer, a driving layer and an execution layer as shown in Fig. 1. The RS232-USB (RS232 to USB) serial port is adopted between the acquisition layer and the decision layer, the decision-making layer and the drive layer. The high and low level control of the IO port pins is used between the drive layer and the execution layer. The host computer uses the QSerialPort component (Function pack of QT) to receive the sEMG signals through the RS232-USB serial port, and stores the received sEMG data in an Excel table to facilitate the subsequent static data processing.

Processing and selection of optimal eigenvalues of sEMG signals

Acquisition and processing of the sEMG Signals

In order to facilitate the collection of sEMG signals, the muscle group on the forearm is selected as the collection object. The muscle groups of the forearm mainly include palmar longus, flexor carpi radialis, brachioradialis, teres pronatorus, extensor carpi radialis longus, extensor digitorum and flexor digitorum superficialis. The flexor carpi radialis is a flexor wrist muscle located on the inner side of the forearm. It starts from the medial epicondyle of the humerus and the olecranon, and ends at the proximal end of the second metacarpal bone. The flexor superficialis is mainly responsible for flexing the metacarpophalangeal joint and proximal interphalangeal joint of the 2nd to 5th fingers. The extensor digitorum can extend the metacarpophalangeal joint of the four fingers. The original sEMG signals are collected by dual-channel sEMG sensors. Each sEMG sensor has two detection electrodes and one reference electrode. The detection electrode is attached to the central part of the muscle belly of the target muscle, and the reference electrode is attached to the muscle not participating in the test exercise. The processed sEMG signal amplitude varies from 0 to 3.3V and the original sEMG signal acquisition and processing process is shown in Fig. 2.

Three healthy volunteers were recruited in this experiment with the informed consents of all volunteers and the Ethical Approval (No. [2020]LLSP(12), Ethics Committee of Faculty of Mechanical Engineering & Mechanics, Ningbo University). Volunteer 1: Male, weight 64 kg, height 175 cm, 24 years old; Volunteer 2: Male, weight 73 kg, height 177 cm, 26 years old; Volunteer 3: Male, weight 75 kg, height 180 cm, 20 years old. Using sEMG sensors and Stm32f103 microprocessor, the original sEMG signals are digitally filtered, amplified, rectified and smoothed (Lyu et al., 2020; Shi et al., 2020). After repeated experiments and comparing the amplitudes of the sEMG signals of different muscle groups collected during the same hand action, the extensor digitorum and flexor digitorum superficialis are finally selected as the muscle groups for sEMG signal collection. Volunteer 1 uses dual-channel sEMG sensors to collect the actual sEMG signals during the flexion and extension movement of his hand, as shown in Fig. 3. The total signal collection duration is about 90 s, of which the sEMG signal curves do not fluctuate much in the first 3 s, as the volunteer is in a state of inactivity. During the movement of the subject’s hand, the corresponding to the hand sEMG signal curves have changed, and the waveform in the figure appears to be convex. By observing the sEMG signals of the two channels, it can be seen that the signals of the two channels fluctuate synchronously when the subject hand is moving, but there are certain differences in the waveforms of each channel.

Selection of optimal eigenvalues of the sEMG signals

Figure 4 shows the obtained eigenvalues of sEMG sensor’s channel 1. It is necessary to use the law of statistics to find the accurate physical quantities that best represent the essence of the surface EMG signal, that is, the extracting eigenvalues of sEMG signals. The original sEMG signal after amplification, rectification and rectification integration loses a lot of frequency domain characteristics of the original signal. By directly analyzing and processing the sEMG signal in the time domain, it will be intuitive and accurate. In the time domain, the sEMG signal can be approximated as a Gaussian distribution. At present, the most commonly used time domain eigenvalues of the signal are the root mean square value (RMS), peak value (PV), mean value (MAV), wavelength average (WAV), form factor (FF) and Willison amplitude (WAMP) (Liu & Cheng, 2018). The number of eigenvalues selected is positively correlated with the accuracy of the information representation contained in the sEMG signals, but too many eigenvalues will affect the speed of the computer to make decisions, which is manifested in the deterioration of the follow ability of the pneumatic gloves to the patient’s intention. On the contrary, if the selected number of eigenvalues of the sEMG signal is too small, the pneumatic rehabilitation glove control system cannot accurately recognize the patient’s movement intention. x_i represents the amplitude of the signal, and n represents the extracted step size. First, N (N = 30) groups of sEMG signals are extracted to form sEMG samples with empirical steps n = 100, n = 150, n = 200 in the continuously collected sEMG signals respectively as W1, W2, and W3. And then the above-mentioned 6 eigenvalues with each segment length as the unit to form an eigenvalue sample E_i (6 ×N) is calculated, where i = 1, 2, 3 corresponds to the sEMG samples W1, W2, W3, respectively.

The patient’s hand movement trend will be expressed as fluctuations in sEMG signals. The eigenvalues of the signals reflect the nature of the signals over a period of time, so the fluctuation of the sEMG will also be specifically reflected in the fluctuation of sEMG eigen-values. According to prior knowledge, it can be known that the greater the degree of dispersion of eigenvalues, the more conducive the neural network to the recognition of the movement trend based on eigenvalues. Based on the six eigenvalues, three eigenvalues with a large degree of dispersion will be selected as the parameters of the next action classification, participating in the training and testing of the neural network for intention recognition. Since a single dispersion index is not sufficient to fully characterize the degree of dispersion of the signals, 4 dispersion indicators will be used to process the 6 eigenvalues that have been obtained, namely range (R), interquartile range (Q), and variance (V) and fourth-order center distance (K).

Range is the difference between the maximum and minimum values between data. The greater the range, the greater the degree of dispersion, namely: (1)${\displaystyle R=max\left(s_i\right)-min\left(s_i\right).}$ The interquartile range represents the range of the middle half of the data. The larger the interval, the greater the degree of dispersion. Arrange a set of data in ascending order. The number in the x% position is represented by P_x. The lower quartile and upper quartile are P₈ and P₂₃ respectively, namely: (2)${\displaystyle Q=P_{23}-P_8.}$ Variance describes the degree of dispersion of data mathematical expectation, that is, the greater the variance, the greater the degree of dispersion, namely: (3)${\displaystyle V=\frac{1}{N}\sum _{i=1}^N{\left(s_i-\frac{1}{N}\sum _{i=1}^N{s}_i\right)}^2.}$ The fourth-order center distance is a cumulative numerical statistics reflecting the distribution characteristics of random variables. The larger the fourth-order center distance, the smaller the degree of dispersion, namely: (4)${\displaystyle K=\frac{\frac{1}{N}\sum _{i=1}^N{\left(\left|s_i\right|-\frac{1}{N}\sum _{i=1}^N{s}_i\right)}^4}{{\left(\frac{1}{N}\sum _{i=1}^N{s}_i^2\right)}^2}.}$ In Eqs. (1) and (4), S_i represents the data amplitude and N represents the data length. The process of determining the optimal eigenvalue is shown in Fig. 5.

By observing the sorting results of the data dispersion degree in Table 1, three eigenvalues with the largest dispersion degree are selected, which are WAMP, PV and RMS. For further verification, the dispersion index of E₂ and E₃ are calculated by the same method, and a comprehensive ranking is performed according to the magnitude of the dispersion index, as shown in Tables 2 and 3.

Research on hand movement trend recognition based on BP neural network

Using the collected sEMG signals to achieve the purpose of identifying the patient’s finger movement trend is the main problem in the design of the pattern recognition classifier. The back propagation (BP) neural network model was chosen to construct the motion recognition classifier, as the BP neural network model has good self-learning, nonlinear mapping and adaptation, generalization and fault tolerance (Wang et al., 2020a; Wang et al., 2020b). It could be an ideal movement trend pattern recognition tool.

Construction of BP neural network classifier

BP neural network is an adaptive nonlinear dynamic system composed of a large number of interconnected neurons. It can learn and store the mapping relationship of multiple input–output modes without describing specific mathematical equations in advance. The quality of neural network classifiers is closely related to the number of neural network layers, the number of nodes in each layer, the transfer function of the hidden layer, and the learning algorithm. The training algorithm flow chart of constructing BP neural network under QT software development environment is shown in Fig. 6.

The number of BP neural layers is selected as 3 layers, namely, the input layer (I), the hidden layer (H) and the output layer (O). This is because Robert Hecht-Nielson proved that a three-layer neural network can complete the mapping of any n-dimensional input and m-dimensional output, so in order to simplify the calculation, a three-layer network is adapted (Hecht-Nielsen, 1992).

Hidden layer transfer function: (5)${\displaystyle y_i=\frac{\left(x_i-MinValue+A\right)}{MaxValue-MinValue+A}}$ Transfer function of the output layer: (6)${\displaystyle y_k=x_k\times \left(MaxValue-MinValue+A\right)-A+MinValue}$ Logsig activation function is used: (7)${\displaystyle y_j=\frac{1}{1+e^{-x_i}}}$ Levenberg-Marquart (L-M) learning algorithm is used: (8)${\displaystyle \Delta \omega ={\left(J^{T}J+\mu I\right)}^{-1}g{J}^{T}e}$ In Eqs. (5) and (6), MinValue is the minimum value of the input layer value; MaxValue is the maximum value of the input layer value; constant A reprevents the denominator from being zero; x_i represents the eigenvalue extracted from the sEMG signals; y_i represents the normalized feature value of the input layer; x_k represents the output value of the hidden layer, and y_k represents the final output value of the output layer. In Eq. (7), x_i represents the sum of the product of the output value and the weight of each neuron in the previous network, y_j represents the output of the j neuron in the current layer network. In Eq. (8), J represents the Jacobian matrix of the derivative of weights from network error, e represents the error vector, µisan adaptive constant, which is greater than 0. The input layer is a 6 ×1 vector composed of the optimal eigenvalues of the 2-channel sEMG signals, so the number of nodes in the input layer is 6, set the number of nodes in the output layer to 1, and use the output result of the output layer to determine the triggered action. The action code is built as in Table 4.

The number of hidden layer nodes is determined by the following empirical formula (Sheela & Deepa, 2013): (9)${\displaystyle n_1=\sqrt{n+m}+a}$

where, n is the number of input nodes; m is the number of output nodes; n₁ is the number of hidden nodes; a is a constant between 1 and 10.

The number of hidden nodes gradually increases, and the training error of the neural network is observed during this process. As the number of hidden layer nodes increases, the training error gradually decreases, but after a certain number of nodes, the test error will fluctuate greatly. Therefore, considering the trend of training and test error changes, the number of hidden layer nodes is finally determined to be 12.

Training and testing of BP neural network

In order to realize the mapping function of the input matrix and the output matrix, the BP neural network needs to be trained. The feedback mechanism of BP neural network includes two parts. One is that the BP neural network produces prediction results. The other is to compare the prediction results with sample results, and then correct the neuron error until the error meets the specified requirements or reaches the specified number of training sessions. 160 sets of data are used as training samples to train the BP neural network as shown in Table 5. Each set of data contains the input and target output of the BP neural network. The input is the optimal eigenvalues of the sEMG signals collected by the two channels of the sEMG sensors, and the output is the code value of the corresponding action.

Before training the BP neural network, the training samples need to be randomly divided into two types at a ratio of 3:1, as training samples and test samples separately. After the BP neural network uses the training sample to complete each iteration, it is judged whether the average error value meets the accuracy requirements (e < 0.01). If the accuracy requirements are met, the training is completed. Otherwise, the prediction results are compared with the sample target results, and then start neural Meta-feedback learning, repeat the above steps until reaching the specified number of training times or meet the accuracy requirements to complete the training.

Considering that BP neural network is prone to over training and lack of generalization ability, the training samples input into the neural network training algorithm are divided into three kinds of samples: train samples, validation samples and test samples. In each epoch of training, the errors between the results of three samples and the target results are tested. When the error of validation samples does not decrease in six successive epochs, the training of BP neural network is stopped to prevent over fitting, which is caused by overtraining of BP neural network. It can be seen from Fig. 7 that the total number of epochs of BP neural network is 116. After 110 epoch of BP neural network, the error of train samples, the error of test samples and the error of validation samples no longer have a downward trend, or their downward trend is not obvious. The best validation performance is 6.293e⁻⁶. Therefore, the training of BP neural network is finished at the 116th epoch. The threshold w is set 0.98, and the trained BP neural network is used to classify and recognize patient actions, the recognition result is shown in Fig. 8. Common classification performance measures are Precision (PRE), Recall (REC), and the harmonized average of the two (F₁).

According to Table 6, the calculation formula of Pre, Rec and F₁: (10)${\displaystyle Pre=\frac{TP}{TP\mathit{+}FP}}$ (11)${\displaystyle Rec=\frac{TP}{TP\mathit{+}FN}}$ (12)${\displaystyle F_1=2\frac{PR\cdot REC}{PR+REC}}$

From Eqs. (10)–(12), Pre = 1, Rec = 0.818, F₁ = 0.8998.

Active trigger control strategy for pneumatic gloves

The software processing algorithm of the control system mainly includes a two-channel optimal eigenvalue amplitude calculation and a BP neural network action recognition calculation. Among them, the same optimal eigenvalue is selected for different patients, and the eigenvalue amplitude calculation formula is unique. However, due to differences between individuals, the weights and thresholds of the nodes in the BP neural network model corresponding to different patients are not the same, so the BP neural network model library needs to be established in the actual application process. Different patients call their corresponding BP neural network models during training. When a patient conducts active training based on sEMG signals for the first time, he needs to collect sEMG signals under the guidance of a physician, and complete the training of the BP neural network, and store the required neural network in the BP neural network model library. The corresponding database will be called during a training session. The algorithm flow of active trigger control strategy for pneumatic rehabilitation gloves based on sEMG signals is shown in Fig. 9.