Multi-modal synergistic quantitative analysis and rehabilitation assessment of lower limbs for exoskeleton_Journal of Biomedical Engineering

Authors：

ZHONG Xu ^1,2,3 , ZHANG Bi ^1,2 , LI Jiwei ^1,2 , ZHANG Liang ⁴ , YUAN Xiangnan ⁵ , ZHANG Peng ³ ,  ZHAO Xingang ^1,2

1. State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, P. R. China;
2. Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang 110016, P. R. China;
3. Medical Engineering Department, Affiliated Hospital of Yangzhou University, Yangzhou, Jiangsu 225003, P. R. China;
4. Department of Rehabilitation, Liaoning Provincial People's Hospital, Shenyang 110067, P. R. China;
5. Shengjing Hospital of China Medical University, Shenyang 110004, P. R. China;

Corresponding?author：

ZHAO Xingang, Email: zhaoxingang@sia.cn

Keywords：

Rehabilitation assessment; Muscle synergy; Modal fusion; Machine learning; Stroke

DOI：

10.7507/1001-5515.202212028

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

In response to the problem that the traditional lower limb rehabilitation scale assessment method is time-consuming and difficult to use in exoskeleton rehabilitation training, this paper proposes a quantitative assessment method for lower limb walking ability based on lower limb exoskeleton robot training with multimodal synergistic information fusion. The method significantly improves the efficiency and reliability of the rehabilitation assessment process by introducing quantitative synergistic indicators fusing electrophysiological and kinematic level information. First, electromyographic and kinematic data of the lower extremity were collected from subjects trained to walk wearing an exoskeleton. Then, based on muscle synergy theory, a synergistic quantification algorithm was used to construct synergistic index features of electromyography and kinematics. Finally, the electrophysiological and kinematic level information was fused to build a modal feature fusion model and output the lower limb motor function score. The experimental results showed that the correlation coefficients of the constructed synergistic features of electromyography and kinematics with the clinical scale were 0.799 and 0.825, respectively. The results of the fused synergistic features in the K-nearest neighbor (KNN) model yielded higher correlation coefficients (r = 0.921, P < 0.01). This method can modify the rehabilitation training mode of the exoskeleton robot according to the assessment results, which provides a basis for the synchronized assessment-training mode of “human in the loop” and provides a potential method for remote rehabilitation training and assessment of the lower extremity.

Citation： ZHONG Xu, ZHANG Bi, LI Jiwei, ZHANG Liang, YUAN Xiangnan, ZHANG Peng, ZHAO Xingang. Multi-modal synergistic quantitative analysis and rehabilitation assessment of lower limbs for exoskeleton. Journal of Biomedical Engineering, 2023, 40(5): 953-964. doi: 10.7507/1001-5515.202212028 Copy

1.	Zhou M, Wang H, Zeng X, et al. Mortality, morbidity, and risk factors in China and its provinces, 1990-2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet, 2019, 394(10204): 1145-1158.
2.	Hatem S M, Saussez G, Della Faille M, et al. Rehabilitation of motor function after stroke: a multiple systematic review focused on techniques to stimulate upper extremity recovery. Front Hum Neurosci, 2016, 10: 442.
3.	中華醫學會神經病學分會神經康復學組, 中華醫學會神經病學分會腦血管病學組, 衛生部腦卒中篩查與防治工程委員會辦公室, 等. 中國腦卒中康復治療指南(2011完全版). 中國康復理論與實踐, 2012, 18(4): 301-318.
4.	國家衛生健康委員會. 2021中國衛生健康統計年鑒. 北京: 中國協和醫科大學出版社, 2021.
5.	Esquenazi A, Talaty M, Packel A, et al. The ReWalk powered exoskeleton to restore ambulatory function to individuals with thoracic-level motor-complete spinal cord injury. American Journal of Physical Medicine Rehabilitation, 2012, 91(11): 911-921.
6.	Nilsson A, Vreede K S, H?glund V, et al. Gait training early after stroke with a new exoskeleton–the hybrid assistive limb: a study of safety and feasibility. J Neuroeng Rehabil, 2014, 11: 92.
7.	Nam K Y, Kim H J, Kwon B S, et al. Robot-assisted gait training (Lokomat) improves walking function and activity in people with spinal cord injury: a systematic review. J Neuroeng Rehabil, 2017, 14(1): 24.
8.	劉京運. 從馬拉松到冬奧會,大艾外骨骼機器人為殘疾人創造更多可能. 機器人產業, 2022, 2022(4): 35-39.
9.	程洪, 黃瑞, 邱靜, 等. 康復機器人及其臨床應用綜述. 機器人, 2021, 43(5): 606-619.
10.	王天. 一種下肢康復訓練用外骨骼機器人. 浙江省: CN215193457U, 2021-12-17.
11.	丁逸葦, 涂利娟, 劉怡希, 等. 可穿戴式下肢外骨骼康復機器人研究進展. 機器人, 2022, 44(5): 522-532.
12.	Zhang J, Fiers P, Witte K A, et al. Human-in-the-loop optimization of exoskeleton assistance during walking. Science, 2017, 356(6344): 1280-1284.
13.	Gordon D F, Mcgreavy C, Christou A, et al. Human-in-the-loop optimization of exoskeleton assistance via online simulation of metabolic cost. IEEE Trans Rob, 2022, 38(3): 1410-1429.
14.	Brunnstrom S. Motor testing procedures in hemiplegia: based on sequential recovery stages. Physical Therapy, 1966, 46(4): 357-375.
15.	Sanford J, Moreland J, Swanson L R, et al. Reliability of the Fugl-Meyer assessment for testing motor performance in patients following stroke. Physical therapy, 1993, 73(7): 447-454.
16.	Carr J H, Shepherd R B, Nordholm L, et al. Investigation of a new motor assessment scale for stroke patients. Physical therapy, 1985, 65(2): 175-180.
17.	Kinoshita S, Abo M, Okamoto T. Effectiveness of ICF-based multidisciplinary rehabilitation approach with serial assessment and discussion using the ICF rehabilitation set in a convalescent rehabilitation ward. Int J Rehabil Res, 2020, 43(3): 255-260.
18.	姜榮榮, 陳艷, 潘翠環. 腦卒中后上肢和手運動功能康復評定的研究進展. 中國康復理論與實踐, 2015, 21(10): 1173-1177.
19.	Gao F, Wang L, Lin T. Intelligent wearable rehabilitation robot control system based on mobile communication network. Comput Commun, 2020, 153: 286-293.
20.	Bai J, Song A. Development of a novel home based multi-scene upper limb rehabilitation training and evaluation system for post-stroke patients. IEEE Access, 2019, 7: 9667-9677.
21.	Cerfoglio S, Ferraris C, Vismara L, et al. Kinect-based assessment of lower limbs during gait in post-stroke hemiplegic patients: a narrative review. Sensors, 2022, 22(13): 4910.
22.	Qian C, Li W, Jia T, et al. Quantitative assessment of motor function by an end-effector upper limb rehabilitation robot based on admittance control. Appl Sci, 2021, 11(15): 6854.
23.	Meziani Y, Morère Y, Hadj-Abdelkader A, et al. Towards adaptive and finer rehabilitation assessment: a learning framework for kinematic evaluation of upper limb rehabilitation on an Armeo Spring exoskeleton. Control Engineering Practice, 2021, 111: 104804.
24.	Liparulo L, Zhang Z, Panella M, et al. A novel fuzzy approach for automatic Brunnstrom stage classification using surface electromyography. Med Biol Eng Comput, 2017, 55(8): 1367-1378.
25.	Yu L, Xiong D, Guo L, et al. A remote quantitative Fugl-Meyer assessment framework for stroke patients based on wearable sensor networks. Comput Methods Programs Biomed, 2016, 128: 100-110.
26.	王躍, 郁磊, 傅建明, 等. 基于極限學習機的腦卒中上肢康復 Brunnstrom 遠程智能評定系統. 生物醫學工程學雜志, 2014, 31(2): 251-256.
27.	Ye F, Yang B, Nam C, et al. A data-driven investigation on surface electromyography based clinical assessment in chronic stroke. Front Neurorobot, 2021, 15: 648855.
28.	李素姣, 吳坤, 孟巧玲, 等. 腦卒中患者上肢功能智能評估系統研究進展. 生物醫學工程學雜志, 2022, 39(3): 620-626.
29.	Zhang Z, Fang Q, Gu X. Objective assessment of upper-limb mobility for poststroke rehabilitation. IEEE Trans Biomed Eng, 2015, 63(4): 859-868.
30.	Rahman S, Sarker S, Haque A K M N, et al. AI-driven stroke rehabilitation systems and assessment: a systematic review. IEEE Trans Neural Syst Rehabil Eng, 2023, 31: 192-207.
31.	Woodward R B, Shefelbine S J, Vaidyanathan R. Pervasive monitoring of motion and muscle activation: Inertial and mechanomyography fusion. IEEE/ASME Trans Mechatron, 2017, 22(5): 2022-2033.
32.	Zhang X, Yue Z, Wang J. Robotics in lower-limb rehabilitation after stroke. Behav Neurol, 2017, 2017: 3731802.
33.	束一銘, 錢競光, 戎科, 等. 偏癱患者步態特征的動力學仿真分析. 醫用生物力學, 2017, 32(6): 535-540.
34.	婁智, 姚博, 楊基海. 基于表面肌電信號的小兒腦癱步態活動段檢測研究. 生物醫學工程學雜志, 2017, 34(3): 342-349.
35.	陳萬鑫, 張弼, 張慶超等. 面向康復外骨骼的串聯彈性關節設計與控制. 機器人, 2023, 45(5): 554-567.
36.	Hermens H J, Freriks B, Disselhorst-Klug C, et al. Development of recommendations for SEMG sensors and sensor placement procedures. J Electromyogr Kinesiol, 2000, 10(5): 361-374.
37.	陳玲玲, 張存, 宋曉偉, 等. 面向外骨骼機器人的肌肉功能網絡構建及分析. 生物醫學工程學雜志, 2019, 36(4): 565-572.
38.	Wen L, Xu J, Li D, et al. Continuous estimation of upper limb joint angle from sEMG based on multiple decomposition feature and BiLSTM network. Biomed Signal Process Control, 2023, 80: 104303.
39.	Sartori M, Llyod D G, Farina D. Neural data-driven musculoskeletal modeling for personalized neurorehabilitation technologies. IEEE Trans Biomed Eng, 2016, 63(5): 879-893.
40.	Zhang L, Li Z, Hu Y, et al. Ankle joint torque estimation using an EMG-driven neuromusculoskeletal model and an artificial neural network model. IEEE Trans Autom Sci Eng, 2021, 18(2): 564-573.
41.	Yang Y, Chen L, Pang J, et al. Validation of a spatiotemporal gait model using inertial measurement units for early-stage Parkinson’s disease detection during turns. IEEE Trans Biomed Eng, 2022, 69(12): 3591-3699.
42.	Ison M, Artemiadis P. The role of muscle synergies in myoelectric control: trends and challenges for simultaneous multifunction control. J Neural Eng, 2014, 11(5): 051001.
43.	Hong Y N G, Ballekere A N, Fregly B J, et al. Are muscle synergies useful for stroke rehabilitation?. Current Opinion in Biomedical Engineering, 2021, 19: 100315.
44.	Ivanenko Y P, Poppele R E, Lacquaniti F. Five basic muscle activation patterns account for muscle activity during human locomotion. J Physiol, 2004, 556: 267-282.
45.	Frère J, Hug F. Between-subject variability of muscle synergies during a complex motor skill. Front Comput Neurosci, 2012, 6: 99.
46.	Ryait H S, Arora A, Agarwal R. Interpretations of wrist/grip operations from SEMG signals at different locations on arm. IEEE Trans Biomed Circuits Syst, 2010, 4(2): 101-111.
47.	Yang J, Zhang D, Frangi A F, et al. Two-dimensional PCA: a new approach to appearance-based face representation and recognition. IEEE Trans Pattern Anal Mach Intell, 2004, 26(1): 131-137.
48.	Hao W. Classification of sport actions using principal component analysis and random forest based on three-dimensional data. Displays, 2022, 72: 102135.
49.	Krüger B, V?gele A, Willig T, et al. Efficient unsupervised temporal segmentation of motion data. IEEE Trans Multimedia, 2016, 19(4): 797-812.
50.	Li C, Zheng S Q, Prabhakaran B. Segmentation and recognition of motion streams by similarity search. ACM Trans Multimedia Comput Commun, 2007, 3(3): 16.

1. Zhou M, Wang H, Zeng X, et al. Mortality, morbidity, and risk factors in China and its provinces, 1990-2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet, 2019, 394(10204): 1145-1158.
2. Hatem S M, Saussez G, Della Faille M, et al. Rehabilitation of motor function after stroke: a multiple systematic review focused on techniques to stimulate upper extremity recovery. Front Hum Neurosci, 2016, 10: 442.
3. 中華醫學會神經病學分會神經康復學組, 中華醫學會神經病學分會腦血管病學組, 衛生部腦卒中篩查與防治工程委員會辦公室, 等. 中國腦卒中康復治療指南(2011完全版). 中國康復理論與實踐, 2012, 18(4): 301-318.
4. 國家衛生健康委員會. 2021中國衛生健康統計年鑒. 北京: 中國協和醫科大學出版社, 2021.
5. Esquenazi A, Talaty M, Packel A, et al. The ReWalk powered exoskeleton to restore ambulatory function to individuals with thoracic-level motor-complete spinal cord injury. American Journal of Physical Medicine Rehabilitation, 2012, 91(11): 911-921.
6. Nilsson A, Vreede K S, H?glund V, et al. Gait training early after stroke with a new exoskeleton–the hybrid assistive limb: a study of safety and feasibility. J Neuroeng Rehabil, 2014, 11: 92.
7. Nam K Y, Kim H J, Kwon B S, et al. Robot-assisted gait training (Lokomat) improves walking function and activity in people with spinal cord injury: a systematic review. J Neuroeng Rehabil, 2017, 14(1): 24.
8. 劉京運. 從馬拉松到冬奧會,大艾外骨骼機器人為殘疾人創造更多可能. 機器人產業, 2022, 2022(4): 35-39.
9. 程洪, 黃瑞, 邱靜, 等. 康復機器人及其臨床應用綜述. 機器人, 2021, 43(5): 606-619.
10. 王天. 一種下肢康復訓練用外骨骼機器人. 浙江省: CN215193457U, 2021-12-17.
11. 丁逸葦, 涂利娟, 劉怡希, 等. 可穿戴式下肢外骨骼康復機器人研究進展. 機器人, 2022, 44(5): 522-532.
12. Zhang J, Fiers P, Witte K A, et al. Human-in-the-loop optimization of exoskeleton assistance during walking. Science, 2017, 356(6344): 1280-1284.
13. Gordon D F, Mcgreavy C, Christou A, et al. Human-in-the-loop optimization of exoskeleton assistance via online simulation of metabolic cost. IEEE Trans Rob, 2022, 38(3): 1410-1429.
14. Brunnstrom S. Motor testing procedures in hemiplegia: based on sequential recovery stages. Physical Therapy, 1966, 46(4): 357-375.
15. Sanford J, Moreland J, Swanson L R, et al. Reliability of the Fugl-Meyer assessment for testing motor performance in patients following stroke. Physical therapy, 1993, 73(7): 447-454.
16. Carr J H, Shepherd R B, Nordholm L, et al. Investigation of a new motor assessment scale for stroke patients. Physical therapy, 1985, 65(2): 175-180.
17. Kinoshita S, Abo M, Okamoto T. Effectiveness of ICF-based multidisciplinary rehabilitation approach with serial assessment and discussion using the ICF rehabilitation set in a convalescent rehabilitation ward. Int J Rehabil Res, 2020, 43(3): 255-260.
18. 姜榮榮, 陳艷, 潘翠環. 腦卒中后上肢和手運動功能康復評定的研究進展. 中國康復理論與實踐, 2015, 21(10): 1173-1177.
19. Gao F, Wang L, Lin T. Intelligent wearable rehabilitation robot control system based on mobile communication network. Comput Commun, 2020, 153: 286-293.
20. Bai J, Song A. Development of a novel home based multi-scene upper limb rehabilitation training and evaluation system for post-stroke patients. IEEE Access, 2019, 7: 9667-9677.
21. Cerfoglio S, Ferraris C, Vismara L, et al. Kinect-based assessment of lower limbs during gait in post-stroke hemiplegic patients: a narrative review. Sensors, 2022, 22(13): 4910.
22. Qian C, Li W, Jia T, et al. Quantitative assessment of motor function by an end-effector upper limb rehabilitation robot based on admittance control. Appl Sci, 2021, 11(15): 6854.
23. Meziani Y, Morère Y, Hadj-Abdelkader A, et al. Towards adaptive and finer rehabilitation assessment: a learning framework for kinematic evaluation of upper limb rehabilitation on an Armeo Spring exoskeleton. Control Engineering Practice, 2021, 111: 104804.
24. Liparulo L, Zhang Z, Panella M, et al. A novel fuzzy approach for automatic Brunnstrom stage classification using surface electromyography. Med Biol Eng Comput, 2017, 55(8): 1367-1378.
25. Yu L, Xiong D, Guo L, et al. A remote quantitative Fugl-Meyer assessment framework for stroke patients based on wearable sensor networks. Comput Methods Programs Biomed, 2016, 128: 100-110.
26. 王躍, 郁磊, 傅建明, 等. 基于極限學習機的腦卒中上肢康復 Brunnstrom 遠程智能評定系統. 生物醫學工程學雜志, 2014, 31(2): 251-256.
27. Ye F, Yang B, Nam C, et al. A data-driven investigation on surface electromyography based clinical assessment in chronic stroke. Front Neurorobot, 2021, 15: 648855.
28. 李素姣, 吳坤, 孟巧玲, 等. 腦卒中患者上肢功能智能評估系統研究進展. 生物醫學工程學雜志, 2022, 39(3): 620-626.
29. Zhang Z, Fang Q, Gu X. Objective assessment of upper-limb mobility for poststroke rehabilitation. IEEE Trans Biomed Eng, 2015, 63(4): 859-868.
30. Rahman S, Sarker S, Haque A K M N, et al. AI-driven stroke rehabilitation systems and assessment: a systematic review. IEEE Trans Neural Syst Rehabil Eng, 2023, 31: 192-207.
31. Woodward R B, Shefelbine S J, Vaidyanathan R. Pervasive monitoring of motion and muscle activation: Inertial and mechanomyography fusion. IEEE/ASME Trans Mechatron, 2017, 22(5): 2022-2033.
32. Zhang X, Yue Z, Wang J. Robotics in lower-limb rehabilitation after stroke. Behav Neurol, 2017, 2017: 3731802.
33. 束一銘, 錢競光, 戎科, 等. 偏癱患者步態特征的動力學仿真分析. 醫用生物力學, 2017, 32(6): 535-540.
34. 婁智, 姚博, 楊基海. 基于表面肌電信號的小兒腦癱步態活動段檢測研究. 生物醫學工程學雜志, 2017, 34(3): 342-349.
35. 陳萬鑫, 張弼, 張慶超等. 面向康復外骨骼的串聯彈性關節設計與控制. 機器人, 2023, 45(5): 554-567.
36. Hermens H J, Freriks B, Disselhorst-Klug C, et al. Development of recommendations for SEMG sensors and sensor placement procedures. J Electromyogr Kinesiol, 2000, 10(5): 361-374.
37. 陳玲玲, 張存, 宋曉偉, 等. 面向外骨骼機器人的肌肉功能網絡構建及分析. 生物醫學工程學雜志, 2019, 36(4): 565-572.
38. Wen L, Xu J, Li D, et al. Continuous estimation of upper limb joint angle from sEMG based on multiple decomposition feature and BiLSTM network. Biomed Signal Process Control, 2023, 80: 104303.
39. Sartori M, Llyod D G, Farina D. Neural data-driven musculoskeletal modeling for personalized neurorehabilitation technologies. IEEE Trans Biomed Eng, 2016, 63(5): 879-893.
40. Zhang L, Li Z, Hu Y, et al. Ankle joint torque estimation using an EMG-driven neuromusculoskeletal model and an artificial neural network model. IEEE Trans Autom Sci Eng, 2021, 18(2): 564-573.
41. Yang Y, Chen L, Pang J, et al. Validation of a spatiotemporal gait model using inertial measurement units for early-stage Parkinson’s disease detection during turns. IEEE Trans Biomed Eng, 2022, 69(12): 3591-3699.
42. Ison M, Artemiadis P. The role of muscle synergies in myoelectric control: trends and challenges for simultaneous multifunction control. J Neural Eng, 2014, 11(5): 051001.
43. Hong Y N G, Ballekere A N, Fregly B J, et al. Are muscle synergies useful for stroke rehabilitation?. Current Opinion in Biomedical Engineering, 2021, 19: 100315.
44. Ivanenko Y P, Poppele R E, Lacquaniti F. Five basic muscle activation patterns account for muscle activity during human locomotion. J Physiol, 2004, 556: 267-282.
45. Frère J, Hug F. Between-subject variability of muscle synergies during a complex motor skill. Front Comput Neurosci, 2012, 6: 99.
46. Ryait H S, Arora A, Agarwal R. Interpretations of wrist/grip operations from SEMG signals at different locations on arm. IEEE Trans Biomed Circuits Syst, 2010, 4(2): 101-111.
47. Yang J, Zhang D, Frangi A F, et al. Two-dimensional PCA: a new approach to appearance-based face representation and recognition. IEEE Trans Pattern Anal Mach Intell, 2004, 26(1): 131-137.
48. Hao W. Classification of sport actions using principal component analysis and random forest based on three-dimensional data. Displays, 2022, 72: 102135.
49. Krüger B, V?gele A, Willig T, et al. Efficient unsupervised temporal segmentation of motion data. IEEE Trans Multimedia, 2016, 19(4): 797-812.
50. Li C, Zheng S Q, Prabhakaran B. Segmentation and recognition of motion streams by similarity search. ACM Trans Multimedia Comput Commun, 2007, 3(3): 16.

Journal of Biomedical Engineering

Multi-modal synergistic quantitative analysis and rehabilitation assessment of lower limbs for exoskeleton

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content