A Video-Based Augmented Reality System for Human-in-the-Loop Muscle Strength Assessment of Juvenile Dermatomyositis

Kanglei Zhou, Ruizhi Cai, Yue Ma, Qingqing Tan, Xinning Wang, Jianguo Li, Hubert P. H. Shum, Frederick W. B. Li, Song Jin and Xiaohui Liang
IEEE Transactions on Visualization and Computer Graphics (TVCG) - Proceedings of the 2023 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 2023

 Impact Factor: 5.2 Top 25% Journal in Computer Science, Software Engineering

A Video-Based Augmented Reality System for Human-in-the-Loop Muscle Strength Assessment of Juvenile Dermatomyositis

Abstract

As the most common idiopathic inflammatory myopathy in children, juvenile dermatomyositis (JDM) is characterized by skin rashes and muscle weakness. The childhood myositis assessment scale (CMAS) is commonly used to measure the degree of muscle involvement for diagnosis or rehabilitation monitoring. On the one hand, human diagnosis is not scalable and may be subject to personal bias. On the other hand, automatic action quality assessment (AQA) algorithms cannot guarantee 100% accuracy, making them not suitable for biomedical applications. As a solution, we propose a video-based augmented reality system for human-in-the-loop muscle strength assessment of children with JDM. We first propose an AQA algorithm for muscle strength assessment of JDM using contrastive regression trained by a JDM dataset. Our core insight is to visualize the AQA results as a virtual character facilitated by a 3D animation dataset, so that users can compare the real-world patient and the virtual character to understand and verify the AQA results. To allow effective comparisons, we propose a video-based augmented reality system. Given a feed, we adapt computer vision algorithms for scene understanding, evaluate the optimal way of augmenting the virtual character into the scene, and highlight important parts for effective human verification. The experimental results confirm the effectiveness of our AQA algorithm, and the results of the user study demonstrate that humans can more accurately and quickly assess the muscle strength of children using our system.

Downloads

YouTube

Citations

BibTeX

@article{zhou23video,
 author={Zhou, Kanglei and Cai, Ruizhi and Ma, Yue and Tan, Qingqing and Wang, Xinning and Li, Jianguo and Shum, Hubert P. H. and Li, Frederick W. B. and Jin, Song and Liang, Xiaohui},
 journal={IEEE Transactions on Visualization and Computer Graphics},
 title={A Video-Based Augmented Reality System for Human-in-the-Loop Muscle Strength Assessment of Juvenile Dermatomyositis},
 year={2023},
 volume={29},
 number={5},
 pages={2456--2466},
 numpages={11},
 doi={10.1109/TVCG.2023.3247092},
 issn={1941-0506},
 publisher={IEEE},
 location={Shanghai, China},
}

RIS

TY  - JOUR
AU  - Zhou, Kanglei
AU  - Cai, Ruizhi
AU  - Ma, Yue
AU  - Tan, Qingqing
AU  - Wang, Xinning
AU  - Li, Jianguo
AU  - Shum, Hubert P. H.
AU  - Li, Frederick W. B.
AU  - Jin, Song
AU  - Liang, Xiaohui
T2  - IEEE Transactions on Visualization and Computer Graphics
TI  - A Video-Based Augmented Reality System for Human-in-the-Loop Muscle Strength Assessment of Juvenile Dermatomyositis
PY  - 2023
VL  - 29
IS  - 5
SP  - 2456
EP  - 2466
DO  - 10.1109/TVCG.2023.3247092
SN  - 1941-0506
PB  - IEEE
ER  - 

Plain Text

Kanglei Zhou, Ruizhi Cai, Yue Ma, Qingqing Tan, Xinning Wang, Jianguo Li, Hubert P. H. Shum, Frederick W. B. Li, Song Jin and Xiaohui Liang, "A Video-Based Augmented Reality System for Human-in-the-Loop Muscle Strength Assessment of Juvenile Dermatomyositis," IEEE Transactions on Visualization and Computer Graphics, vol. 29, no. 5, pp. 2456-2466, Shanghai, China, IEEE, 2023.

Supporting Grants

Similar Research

Kanglei Zhou, Yue Ma, Hubert P. H. Shum and Xiaohui Liang, "Hierarchical Graph Convolutional Networks for Action Quality Assessment", IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), 2023
Haozheng Zhang, Hubert P. H. Shum and Edmond S. L. Ho, "Cerebral Palsy Prediction with Frequency Attention Informed Graph Convolutional Networks", Proceedings of the 2022 International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2022
Manli Zhu, Qianhui Men, Edmond S. L. Ho, Howard Leung and Hubert P. H. Shum, "A Two-Stream Convolutional Network for Musculoskeletal and Neurological Disorders Prediction", Journal of Medical Systems (JMS), 2022
Haozheng Zhang, Edmond S. L. Ho and Hubert P. H. Shum, "CP-AGCN: Pytorch-Based Attention Informed Graph Convolutional Network for Identifying Infants at Risk of Cerebral Palsy", Software Impacts (SIMPAC), 2022

 

 

Last updated on 14 April 2024
RSS Feed