Shape-aware cross-modal domain adaptive segmentation model_Journal of Biomedical Engineering

Authors：

LIU Yusi ^1,2,3 , QI Liangce ^1,2,3 , DIAO Zhaoheng ^1,2,3 , FENG Guanyuan ^1,2,3 , LI Yuqin ^1,2,3 ,  JIANG Zhengang ^1,2,3

1. School of Computer Science and Technology, Changchun University of Science and Technology, Changchun 130022, P. R. China;
2. Key Laboratory of Medical Imaging Intelligence Technology of Jilin Province, Changchun University of Science and Technology, Changchun 130022, P. R. China;
3. Jilin Cross-regional Science and Technology Innovation Center of Medical Intelligent Technology and precision Diagnosis and Treatment Equipment, Changchun University of Science and Technology, Changchun 130022, P. R. China;

Corresponding?author：

JIANG Zhengang, Email: jiangzhengang@cust.edu.cn

Keywords：

Domain adaptation; Cross modal; Semantic segmentation; Self-adaptation; Shape perception

DOI：

10.7507/1001-5515.202506045

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Cross-modal unsupervised domain adaptation (UDA) aims to transfer segmentation models trained on a labeled source modality to an unlabeled target modality. However, existing methods often fail to fully exploit shape priors and intermediate feature representations, resulting in limited generalization ability of the model in cross-modal transfer tasks. To address this challenge, we propose a segmentation model based on shape-aware adaptive weighting (SAWS) that enhance the model's ability to perceive the target area and capture global and local information. Specifically, we design a multi-angle strip-shaped shape perception (MSSP) module that captures shape features from multiple orientations through an angular pooling strategy, improving structural modeling under cross-modal settings. In addition, an adaptive weighted hierarchical contrastive (AWHC) loss is introduced to fully leverage intermediate features and enhance segmentation accuracy for small target structures. The proposed method is evaluated on the multi-modality whole heart segmentation (MMWHS) dataset. Experimental results demonstrate that SAWS achieves superior performance in cross-modal cardiac segmentation tasks, with a Dice score (Dice) of 70.1% and an average symmetric surface distance (ASSD) of 4.0 for the computed tomography (CT)→magnetic resonance imaging (MRI) task, and a Dice of 83.8% and ASSD of 3.7 for the MRI→CT task, outperforming existing state-of-the-art methods. Overall, this study proposes a cross-modal medical image segmentation method with shape-aware, which effectively improves the structure-aware ability and generalization performance of the UDA model.

1.	王新穎, 王婉秋, 王慧. 基于改進的L-BFGS稀疏降噪自編碼網絡的MRI腦圖像識別方法. 圖學學報, 2019, 40(2): 261-266.
2.	王晴, 趙新堯, 劉心月, 等. Stroke-p2pHD: 腦梗CT到DWI圖像的跨模態生成模型. 生物醫學工程學雜志, 2025, 42(2): 255-262.
3.	Li Y, El Habib Daho M, Conze P H, et al. A review of deep learning-based information fusion techniques for multimodal medical image classification. Computers in Biology and Medicine, 2024, 177: 108635.
4.	Zhou X, Sun Y, Deng M, et al. Robust semi-supervised multimodal medical image segmentation via cross modality collaboration//International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), Morocco: Springer Nature Switzerland, 2024: 57-67.
5.	張桂梅, 陶輝, 魯飛飛, 等. 基于雙源判別器的域自適應城市場景語義分割. 圖學學報, 2023, 44(5): 907-917.
6.	Chen C, Dou Q, Chen H, et al. Synergistic image and feature adaptation: towards cross-modality domain adaptation for medical image segmentation//Proceedings of the AAAI conference on artificial intelligence (AAAI), Hawaii: AAAI, 2019, 33(1): 865-872.
7.	Wu F, Zhuang X. Unsupervised domain adaptation with variational approximation for cardiac segmentation. IEEE Transactions on Medical Imaging, 2021, 40(12): 3555-3567.
8.	Cui H, Yuwen C, Jiang L, et al. Bidirectional cross-modality unsupervised domain adaptation using generative adversarial networks for cardiac image segmentation. Computers in Biology and Medicine, 2021, 136: 104726.
9.	Kumari S, Singh P. Deep learning for unsupervised domain adaptation in medical imaging: recent advancements and future perspectives. Computers in Biology and Medicine, 2024, 170: 107912.
10.	呼偉, 徐巧枝, 葛湘巍, 等. 醫學圖像分割的無監督域適應研究綜述. 計算機工程與應用, 2024, 60(6): 10-26.
11.	Tomar D, Lortkipanidze M, Vray G, et al. Self-attentive spatial adaptive normalization for cross-modality domain adaptation. IEEE Transactions on Medical Imaging, 2021, 40(10): 2926-2938.
12.	Chen C, Dou Q, Chen H, et al. Unsupervised bidirectional cross-modality adaptation via deeply synergistic image and feature alignment for medical image segmentation. IEEE Transactions on Medical Imaging, 2020, 39(7): 2494-2505.
13.	Zou D, Zhu Q, Yan P. Unsupervised domain adaptation with dual-scheme fusion network for medical image segmentation//Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence (IJCAI), Yokohama: IJCA, 2020: 3291-3298.
14.	Pei C, Wu F, Huang L, et al. Disentangle domain features for cross-modality cardiac image segmentation. Medical Image Analysis, 2021, 71: 102078.
15.	Ji W, Chung A C S. Unsupervised domain adaptation for medical image segmentation using transformer with meta-attention. IEEE Transactions on Medical Imaging, 2023, 43(2): 820-831.
16.	Zhang Y, Wang Y, Xu L, et al. ST-GAN: A swin transformer-based generative adversarial network for unsupervised domain adaptation of cross-modality cardiac segmentation. IEEE Journal of Biomedical and Health Informatics, 2024, 28(2): 893-904.
17.	Guo M, Luo Z, Yang X. Transformer-enhanced cross-modal learning for robust biomedical image segmentation//2024 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Lisbo: IEEE, 2024: 4537-4544.
18.	Ge Y, Chen Z M, Zhang G, et al. Unsupervised domain adaptation via style adaptation and boundary enhancement for medical semantic segmentation. Neurocomputing, 2023, 550: 126469.
19.	Ding Y, Mu D, Zhang J, et al. A cascaded framework with cross-modality transfer learning for whole heart segmentation. Pattern Recognition, 2024, 147: 110088.
20.	Liu Z, Chen Z M, Chen H, et al. Style adaptation for avoiding semantic inconsistency in unsupervised domain adaptation medical image segmentation. Biomedical Signal Processing and Control, 2025, 105: 107573.
21.	Liu Z, Zhu Z, Zheng S, et al. Margin preserving self-paced contrastive learning towards domain adaptation for medical image segmentation. IEEE Journal of Biomedical and Health Informatics, 2022, 26(2): 638-647.
22.	Lin H, Schiffers F, López-Tapia S, et al. DRL-STNet: unsupervised domain adaptation for cross-modality medical image segmentation via disentangled representation learning//International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), Morocco: Springer Nature Switzerland, 2024: 178-194.
23.	Li S, Zhang C, He X. Shape-aware semi-supervised 3D semantic segmentation for medical images//International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), Peru: Springer International Publishing, 2020: 552-561.
24.	Huang Q, Zhou Y, Tao L. Dual-term loss function for shape-aware medical image segmentation//2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), Washington: IEEE, 2021: 1798-1802.
25.	Liu J, Desrosiers C, Zhou Y. Semi-supervised medical image segmentation using cross-model pseudo-supervision with shape awareness and local context constraints//International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), Singapore: Springer Nature Switzerland, 2022: 140-150.
26.	Zhou G Q, Zhang W B, Shi Z Q, et al. DSANet: dual-branch shape-aware network for echocardiography segmentation in apical views. IEEE Journal of Biomedical and Health Informatics, 2023, 27(10): 4804-4815.
27.	Zhou L, Jiang Y, Li W, et al. Shape-scale co-awareness network for 3D brain tumor segmentation. IEEE Transactions on Medical Imaging, 2024, 43(7): 2495-2508.
28.	Goodfellow I J, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets//Proceedings of the 27th International Conference on Neural Information Processing Systems (ICONIP), Montreal: MIT Press, 2014: 2672-2680.
29.	Chen L C, Papandreou G, Kokkinos I, et al. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4): 834-848.
30.	Zhuang X, Shen J. Multi-scale patch and multi-modality atlases for whole heart segmentation of MRI. Medical Image Analysis, 2016, 31: 77-87.
31.	Bottou L. Large-scale machine learning with stochastic gradient descent//19th International Conference on Computational StatisticsParis France (COMPSTAT), Heidelberg: Physica-Verlag HD, 2010: 177-186.
32.	Isola P, Zhu J Y, Zhou T, et al. Image-to-image translation with conditional adversarial networks//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Hawaii: IEEE, 2017: 1125-1134.
33.	Tsai Y H, Hung W C, Schulter S, et al. Learning to adapt structured output space for semantic segmentation//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Utah: IEEE, 2018: 7472-7481.
34.	Vu T H, Jain H, Bucher M, et al. Advent: adversarial entropy minimization for domain adaptation in semantic segmentation//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), California: IEEE, 2019: 2517-2526.
35.	Eijpe A, Corbetta V, Chupetlovska K, et al. Enhancing cross-modal medical image segmentation through compositionality//International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), Morocco: Springer Nature Switzerland, 2024: 43-53.

1. 王新穎, 王婉秋, 王慧. 基于改進的L-BFGS稀疏降噪自編碼網絡的MRI腦圖像識別方法. 圖學學報, 2019, 40(2): 261-266.
2. 王晴, 趙新堯, 劉心月, 等. Stroke-p2pHD: 腦梗CT到DWI圖像的跨模態生成模型. 生物醫學工程學雜志, 2025, 42(2): 255-262.
3. Li Y, El Habib Daho M, Conze P H, et al. A review of deep learning-based information fusion techniques for multimodal medical image classification. Computers in Biology and Medicine, 2024, 177: 108635.
4. Zhou X, Sun Y, Deng M, et al. Robust semi-supervised multimodal medical image segmentation via cross modality collaboration//International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), Morocco: Springer Nature Switzerland, 2024: 57-67.
5. 張桂梅, 陶輝, 魯飛飛, 等. 基于雙源判別器的域自適應城市場景語義分割. 圖學學報, 2023, 44(5): 907-917.
6. Chen C, Dou Q, Chen H, et al. Synergistic image and feature adaptation: towards cross-modality domain adaptation for medical image segmentation//Proceedings of the AAAI conference on artificial intelligence (AAAI), Hawaii: AAAI, 2019, 33(1): 865-872.
7. Wu F, Zhuang X. Unsupervised domain adaptation with variational approximation for cardiac segmentation. IEEE Transactions on Medical Imaging, 2021, 40(12): 3555-3567.
8. Cui H, Yuwen C, Jiang L, et al. Bidirectional cross-modality unsupervised domain adaptation using generative adversarial networks for cardiac image segmentation. Computers in Biology and Medicine, 2021, 136: 104726.
9. Kumari S, Singh P. Deep learning for unsupervised domain adaptation in medical imaging: recent advancements and future perspectives. Computers in Biology and Medicine, 2024, 170: 107912.
10. 呼偉, 徐巧枝, 葛湘巍, 等. 醫學圖像分割的無監督域適應研究綜述. 計算機工程與應用, 2024, 60(6): 10-26.
11. Tomar D, Lortkipanidze M, Vray G, et al. Self-attentive spatial adaptive normalization for cross-modality domain adaptation. IEEE Transactions on Medical Imaging, 2021, 40(10): 2926-2938.
12. Chen C, Dou Q, Chen H, et al. Unsupervised bidirectional cross-modality adaptation via deeply synergistic image and feature alignment for medical image segmentation. IEEE Transactions on Medical Imaging, 2020, 39(7): 2494-2505.
13. Zou D, Zhu Q, Yan P. Unsupervised domain adaptation with dual-scheme fusion network for medical image segmentation//Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence (IJCAI), Yokohama: IJCA, 2020: 3291-3298.
14. Pei C, Wu F, Huang L, et al. Disentangle domain features for cross-modality cardiac image segmentation. Medical Image Analysis, 2021, 71: 102078.
15. Ji W, Chung A C S. Unsupervised domain adaptation for medical image segmentation using transformer with meta-attention. IEEE Transactions on Medical Imaging, 2023, 43(2): 820-831.
16. Zhang Y, Wang Y, Xu L, et al. ST-GAN: A swin transformer-based generative adversarial network for unsupervised domain adaptation of cross-modality cardiac segmentation. IEEE Journal of Biomedical and Health Informatics, 2024, 28(2): 893-904.
17. Guo M, Luo Z, Yang X. Transformer-enhanced cross-modal learning for robust biomedical image segmentation//2024 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Lisbo: IEEE, 2024: 4537-4544.
18. Ge Y, Chen Z M, Zhang G, et al. Unsupervised domain adaptation via style adaptation and boundary enhancement for medical semantic segmentation. Neurocomputing, 2023, 550: 126469.
19. Ding Y, Mu D, Zhang J, et al. A cascaded framework with cross-modality transfer learning for whole heart segmentation. Pattern Recognition, 2024, 147: 110088.
20. Liu Z, Chen Z M, Chen H, et al. Style adaptation for avoiding semantic inconsistency in unsupervised domain adaptation medical image segmentation. Biomedical Signal Processing and Control, 2025, 105: 107573.
21. Liu Z, Zhu Z, Zheng S, et al. Margin preserving self-paced contrastive learning towards domain adaptation for medical image segmentation. IEEE Journal of Biomedical and Health Informatics, 2022, 26(2): 638-647.
22. Lin H, Schiffers F, López-Tapia S, et al. DRL-STNet: unsupervised domain adaptation for cross-modality medical image segmentation via disentangled representation learning//International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), Morocco: Springer Nature Switzerland, 2024: 178-194.
23. Li S, Zhang C, He X. Shape-aware semi-supervised 3D semantic segmentation for medical images//International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), Peru: Springer International Publishing, 2020: 552-561.
24. Huang Q, Zhou Y, Tao L. Dual-term loss function for shape-aware medical image segmentation//2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), Washington: IEEE, 2021: 1798-1802.
25. Liu J, Desrosiers C, Zhou Y. Semi-supervised medical image segmentation using cross-model pseudo-supervision with shape awareness and local context constraints//International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), Singapore: Springer Nature Switzerland, 2022: 140-150.
26. Zhou G Q, Zhang W B, Shi Z Q, et al. DSANet: dual-branch shape-aware network for echocardiography segmentation in apical views. IEEE Journal of Biomedical and Health Informatics, 2023, 27(10): 4804-4815.
27. Zhou L, Jiang Y, Li W, et al. Shape-scale co-awareness network for 3D brain tumor segmentation. IEEE Transactions on Medical Imaging, 2024, 43(7): 2495-2508.
28. Goodfellow I J, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets//Proceedings of the 27th International Conference on Neural Information Processing Systems (ICONIP), Montreal: MIT Press, 2014: 2672-2680.
29. Chen L C, Papandreou G, Kokkinos I, et al. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4): 834-848.
30. Zhuang X, Shen J. Multi-scale patch and multi-modality atlases for whole heart segmentation of MRI. Medical Image Analysis, 2016, 31: 77-87.
31. Bottou L. Large-scale machine learning with stochastic gradient descent//19th International Conference on Computational StatisticsParis France (COMPSTAT), Heidelberg: Physica-Verlag HD, 2010: 177-186.
32. Isola P, Zhu J Y, Zhou T, et al. Image-to-image translation with conditional adversarial networks//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Hawaii: IEEE, 2017: 1125-1134.
33. Tsai Y H, Hung W C, Schulter S, et al. Learning to adapt structured output space for semantic segmentation//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Utah: IEEE, 2018: 7472-7481.
34. Vu T H, Jain H, Bucher M, et al. Advent: adversarial entropy minimization for domain adaptation in semantic segmentation//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), California: IEEE, 2019: 2517-2526.
35. Eijpe A, Corbetta V, Chupetlovska K, et al. Enhancing cross-modal medical image segmentation through compositionality//International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), Morocco: Springer Nature Switzerland, 2024: 43-53.

Journal of Biomedical Engineering

Latest ArticlesShape-aware cross-modal domain adaptive segmentation model

Abstract Full text Figures/Tables Video References Cited by

Format

Content