Evidence certainty grading of network meta-analysis: method update and case application_Chinese Journal of Evidence-Based Medicine

Authors：

HUANG Jiajie ¹ , LAI Honghao ^2,3 , LIU Jianing ¹ , ZHAO Weilong ^2,3 , SUN Mingyao ⁴ , YE Ziying ^2,3 , LI Ying ^2,3 , PAN Bei ^5,6 , TIAN Jinhui ^5,6 ,  LI Mengting ⁷ ,  GE Long ^2,3,5

1. College of Nursing, Gansu University of Chinese Medicine, Lanzhou 730000, P. R. China;
2. Evidence-Based Social Science Research Center, School of Public Health, Lanzhou University, Lanzhou 730000, P. R. China;
3. Center of Health Management and Health Development, School of Public Health, Lanzhou University, Lanzhou 730000, P. R. China;
4. Evidence Based Nursing Centre, School of Nursing, Lanzhou University, Lanzhou 730000, P. R. China;
5. Key Laboratory of Evidence Based Medicine of Gansu Province, Lanzhou 730000, P. R. China;
6. Evidence Based Medicine Center, School of Basic Medical Sciences, Lanzhou University, Lanzhou 730000, P. R. China;
7. College of Social Science and Health Management, Gansu University of Chinese Medicine, Lanzhou 730000, P. R. China;

Corresponding?author：

LI Mengting, Email: limengting1997@163.com; GE Long, Email: gelong2009@163.com

Keywords：

Network meta-analysis; GRADE; Credibility of evidence

DOI：

10.7507/1672-2531.202310039

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Network meta-analysis (NMA) is a method that can compare and rank the effects of different interventions, which plays an important role in evidence translation and evidence-based decision-making. In 2014, the GRADE working group first introduced the GRADE method for NMA evidence certainty grading. Since then, its method system has been gradually supplemented and improved. In recent years, the GRADE working group has further improved the methods for evaluating intransitivity and imprecision in NMA, and has made recommendations for the presentation and interpretation of NMA results, forming a complete methodological chain of NMA evidence certainty grading and result interpretation consisting of 6 steps. Our team updated the method system of GRADE applied in NMA with specific cases to provide references for relevant researchers.

Citation： HUANG Jiajie, LAI Honghao, LIU Jianing, ZHAO Weilong, SUN Mingyao, YE Ziying, LI Ying, PAN Bei, TIAN Jinhui, LI Mengting, GE Long. Evidence certainty grading of network meta-analysis: method update and case application. Chinese Journal of Evidence-Based Medicine, 2024, 24(10): 1231-1240. doi: 10.7507/1672-2531.202310039 Copy

1.	Lu G, Ades AE. Combination of direct and indirect evidence in mixed treatment comparisons. Stat Med, 2004, 23(20): 3105-3124.
2.	田金徽, 李倫, 趙曄, 等. 網狀Meta分析現狀. 中國藥物評價, 2014, 31(3): 129-133.
3.	Lee A. The development of network meta-analysis. J R Soc Med, 2022, 115(8): 313-321.
4.	Efthimiou O, Debray TP, van Valkenhoef G, et al. GetReal in network meta-analysis: a review of the methodology. Res Synth Methods, 2016, 7(3): 236-263.
5.	Puhan MA, Schünemann HJ, Murad MH, et al. A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis. BMJ, 2014, 349: g5630.
6.	Brignardello-Petersen R, Bonner A, Alexander PE, et al. Advances in the GRADE approach to rate the certainty in estimates from a network meta-analysis. J Clin Epidemiol, 2018, 93: 36-44.
7.	Brignardello-Petersen R, Mustafa RA, Siemieniuk RAC, et al. GRADE approach to rate the certainty from a network meta-analysis: addressing incoherence. J Clin Epidemiol, 2019, 108: 77-85.
8.	Brignardello-Petersen R, Florez ID, Izcovich A, et al. GRADE approach to drawing conclusions from a network meta-analysis using a minimally contextualised framework. BMJ, 2020, 371: m3900.
9.	Brignardello-Petersen R, Izcovich A, Rochwerg B, et al. GRADE approach to drawing conclusions from a network meta-analysis using a partially contextualised framework. BMJ, 2020, 371: m3907.
10.	Brignardello-Petersen R, Tomlinson G, Florez I, et al. Grading of recommendations assessment, development, and evaluation concept article 5: addressing intransitivity in a network meta-analysis. J Clin Epidemiol, 2023, 160: 151-159.
11.	Brignardello-Petersen R, Murad MH, Walter SD, et al. GRADE approach to rate the certainty from a network meta-analysis: avoiding spurious judgments of imprecision in sparse networks. J Clin Epidemiol, 2019, 105: 60-67.
12.	Wang Q, Xiao Y, Guo T, et al. The use of GRADE approach in Cochrane reviews of TCM was insufficient: a cross-sectional survey. J Clin Epidemiol, 2022, 142: 1-9.
13.	Xun Y, Zhou Q, Yang N, et al. Clinical epidemiology in China series. Paper 2: promoting GRADE at the national level: The experience from China. J Clin Epidemiol, 2021, 140: 172-177.
14.	Gao Y, Ge L, Ma X, et al. Improvement needed in the network geometry and inconsistency of Cochrane network meta-analyses: a cross-sectional survey. J Clin Epidemiol, 2019, 113: 214-227.
15.	Brignardello-Petersen R, Guyatt GH, Mustafa RA, et al. GRADE guidelines 33: addressing imprecision in a network meta-analysis. J Clin Epidemiol, 2021, 139: 49-56.
16.	Izcovich A, Chu DK, Mustafa RA, et al. A guide and pragmatic considerations for applying GRADE to network meta-analysis. BMJ, 2023, 381: e074495.
17.	王琪, 王建成, 潘蓓, 等. 網狀Meta分析證據質量分級: GRADE方法學前沿與進展. 中國循證醫學雜志, 2020, 20(8): 979-985.
18.	栗夢婷, 朱鴻飛, 后亮瑛, 等. 網狀Meta分析結論形成的方法: 最小背景化框架. 中國循證醫學雜志, 2021, 21(9): 1102-1109.
19.	朱鴻飛, 栗夢婷, 后亮瑛, 等. 網狀Meta分析結論形成的方法: 部分背景化框架. 中國循證醫學雜志, 2021, 21(9): 1110-1116.
20.	王琪, 王韻華, 賴鴻皓, 等. 網狀Meta分析證據質量分級: CINeMA在線應用程序簡介. 中國循證醫學雜志, 2020, 20(9): 1111-1116.
21.	Pan B, Ge L, Lai H, et al. The comparative effectiveness and safety of insomnia drugs: a systematic review and network meta-analysis of 153 randomized trials. Drugs, 2023, 83(7): 587-619.
22.	李倫, 田金徽, 姚亮, 等. 網狀Meta分析的統計學基礎、假設和證據質量評估. 循證醫學, 2015, 15(3): 180-183.
23.	劉鳳琪, 石舒原, 楊智榮, 等. 網絡Meta分析研究進展系列(七): 多水平網狀Meta回歸. 中國循證心血管醫學雜志, 2020, 12(12): 1409-1412.
24.	Schandelmaier S, Chang Y, Devasenapathy N, et al. A systematic survey identified 36 criteria for assessing effect modification claims in randomized trials or meta-analyses. J Clin Epidemiol, 2019, 113: 159-167.
25.	Schandelmaier S, Briel M, Varadhan R, et al. Development of the instrument to assess the credibility of effect modification analyses (ICEMAN) in randomized controlled trials and meta-analyses. CMAJ, 2020, 192(32): E901-E906.
26.	李戈, 楊智榮, 王巍巍, 等. 《GRADE在網狀Meta分析不精確性的處理》文獻解讀. 中國循證兒科雜志, 2021, 16(6): 487-489.
27.	Guyatt GH, Oxman AD, Kunz R, et al. GRADE guidelines 6. Rating the quality of evidence--imprecision. J Clin Epidemiol, 2011, 64(12): 1283-1293.
28.	Schünemann HJ. Interpreting GRADE's levels of certainty or quality of the evidence: GRADE for statisticians, considering review information size or less emphasis on imprecision. J Clin Epidemiol, 2016, 75: 6-15.
29.	GRADE Working Group. A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis. BMJ, 2015, 350: h3326.
30.	Papakonstantinou T, Nikolakopoulou A, Rücker G, et al. Estimating the contribution of studies in network meta-analysis: paths, flows and streams. F1000Res, 2018, 7: 610.
31.	Brignardello-Petersen R, Guyatt G H. β-Blockers in heart failure--are all created equal? Pol Arch Med Wewn, 2013, 123(5): 204-205.
32.	Schünemann HJ, Wiercioch W, Brozek J, et al. GRADE Evidence to Decision (EtD) frameworks for adoption, adaptation, and de novo development of trustworthy recommendations: GRADE-ADOLOPMENT. J Clin Epidemiol, 2017, 81: 101-110.
33.	黃嘉杰, 賴鴻皓, 孫銘謠, 等. 實施性研究綜合框架(CFIR)更新解讀. 中國全科醫學, 2023, 26(31): 3863-3871, 3876.
34.	賴鴻皓, 孫銘謠, 黃嘉杰, 等. 中西醫結合臨床實踐指南實施框架構建的思考. 協和醫學雜志, 2023, 14(1): 31-38.
35.	葛龍, 秦鈺, 宋忠陽, 等. 新型冠狀病毒感染恢復期常見癥中西醫結合管理循證實踐指南. 蘭州大學學報(醫學版), 2023, 49(5): 28-40,46.
36.	田晨, 孫銘謠, 李穎, 等. 快速動態循證要覽制訂方法與流程. 中國循證醫學雜志, 2023, 23(4): 450-456.
37.	孫銘謠, 田晨, 李穎, 等. 藥物治療新型冠狀病毒感染快速動態循證要覽. 中國循證醫學雜志, 2023, 23(7): 745-754.
38.	田晨, 楊秋玉, 孫銘謠, 等. 快速動態循證指南制訂方法. 協和醫學雜志, 2024, 15(1): 183-191.
39.	孫銘謠, 曾茵茹, 賴鴻皓, 等. 動態指南發展現狀與挑戰. 中國循證醫學雜志, 2023, 23(8): 970-977.
40.	Institute of Social and Preventive Medicine. CINeMA: confidence in network meta-analysis. 2017.
41.	田晨, 晏毅龍, 王勇, 等. 證據智能合成與分級: AutoMeta平臺開發與驗證. 中國循證醫學雜志, 2024, 24(4): 459-465.
42.	賴鴻皓, 王浙, 李瀅, 等. 中醫藥多元證據整合方法學MERGE框架. 協和醫學雜志, 2024, 15(1): 172-182.
43.	陳昊, 王艷, 陳耀龍, 等. 從GRADE系統方法學看中醫藥臨床證據體系的構建. 中華中醫藥雜志, 2017, 32(1): 25-28.
44.	張薇, 李小娟, 鄧宏勇. 中醫臨床證據分級和推薦體系發展現狀. 中國中醫藥信息雜志, 2020, 27(5): 133-136.
45.	賴鴻皓, 田晨, 朱鴻飛, 等. 如何整合隨機與非隨機干預研究. 中國循證醫學雜志, 2022, 22(7): 836-841.
46.	Cuello-Garcia CA, Santesso N, Morgan RL, et al. GRADE guidance 24 optimizing the integration of randomized and non-randomized studies of interventions in evidence syntheses and health guidelines. J Clin Epidemiol, 2022, 142: 200-208.

1. Lu G, Ades AE. Combination of direct and indirect evidence in mixed treatment comparisons. Stat Med, 2004, 23(20): 3105-3124.
2. 田金徽, 李倫, 趙曄, 等. 網狀Meta分析現狀. 中國藥物評價, 2014, 31(3): 129-133.
3. Lee A. The development of network meta-analysis. J R Soc Med, 2022, 115(8): 313-321.
4. Efthimiou O, Debray TP, van Valkenhoef G, et al. GetReal in network meta-analysis: a review of the methodology. Res Synth Methods, 2016, 7(3): 236-263.
5. Puhan MA, Schünemann HJ, Murad MH, et al. A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis. BMJ, 2014, 349: g5630.
6. Brignardello-Petersen R, Bonner A, Alexander PE, et al. Advances in the GRADE approach to rate the certainty in estimates from a network meta-analysis. J Clin Epidemiol, 2018, 93: 36-44.
7. Brignardello-Petersen R, Mustafa RA, Siemieniuk RAC, et al. GRADE approach to rate the certainty from a network meta-analysis: addressing incoherence. J Clin Epidemiol, 2019, 108: 77-85.
8. Brignardello-Petersen R, Florez ID, Izcovich A, et al. GRADE approach to drawing conclusions from a network meta-analysis using a minimally contextualised framework. BMJ, 2020, 371: m3900.
9. Brignardello-Petersen R, Izcovich A, Rochwerg B, et al. GRADE approach to drawing conclusions from a network meta-analysis using a partially contextualised framework. BMJ, 2020, 371: m3907.
10. Brignardello-Petersen R, Tomlinson G, Florez I, et al. Grading of recommendations assessment, development, and evaluation concept article 5: addressing intransitivity in a network meta-analysis. J Clin Epidemiol, 2023, 160: 151-159.
11. Brignardello-Petersen R, Murad MH, Walter SD, et al. GRADE approach to rate the certainty from a network meta-analysis: avoiding spurious judgments of imprecision in sparse networks. J Clin Epidemiol, 2019, 105: 60-67.
12. Wang Q, Xiao Y, Guo T, et al. The use of GRADE approach in Cochrane reviews of TCM was insufficient: a cross-sectional survey. J Clin Epidemiol, 2022, 142: 1-9.
13. Xun Y, Zhou Q, Yang N, et al. Clinical epidemiology in China series. Paper 2: promoting GRADE at the national level: The experience from China. J Clin Epidemiol, 2021, 140: 172-177.
14. Gao Y, Ge L, Ma X, et al. Improvement needed in the network geometry and inconsistency of Cochrane network meta-analyses: a cross-sectional survey. J Clin Epidemiol, 2019, 113: 214-227.
15. Brignardello-Petersen R, Guyatt GH, Mustafa RA, et al. GRADE guidelines 33: addressing imprecision in a network meta-analysis. J Clin Epidemiol, 2021, 139: 49-56.
16. Izcovich A, Chu DK, Mustafa RA, et al. A guide and pragmatic considerations for applying GRADE to network meta-analysis. BMJ, 2023, 381: e074495.
17. 王琪, 王建成, 潘蓓, 等. 網狀Meta分析證據質量分級: GRADE方法學前沿與進展. 中國循證醫學雜志, 2020, 20(8): 979-985.
18. 栗夢婷, 朱鴻飛, 后亮瑛, 等. 網狀Meta分析結論形成的方法: 最小背景化框架. 中國循證醫學雜志, 2021, 21(9): 1102-1109.
19. 朱鴻飛, 栗夢婷, 后亮瑛, 等. 網狀Meta分析結論形成的方法: 部分背景化框架. 中國循證醫學雜志, 2021, 21(9): 1110-1116.
20. 王琪, 王韻華, 賴鴻皓, 等. 網狀Meta分析證據質量分級: CINeMA在線應用程序簡介. 中國循證醫學雜志, 2020, 20(9): 1111-1116.
21. Pan B, Ge L, Lai H, et al. The comparative effectiveness and safety of insomnia drugs: a systematic review and network meta-analysis of 153 randomized trials. Drugs, 2023, 83(7): 587-619.
22. 李倫, 田金徽, 姚亮, 等. 網狀Meta分析的統計學基礎、假設和證據質量評估. 循證醫學, 2015, 15(3): 180-183.
23. 劉鳳琪, 石舒原, 楊智榮, 等. 網絡Meta分析研究進展系列(七): 多水平網狀Meta回歸. 中國循證心血管醫學雜志, 2020, 12(12): 1409-1412.
24. Schandelmaier S, Chang Y, Devasenapathy N, et al. A systematic survey identified 36 criteria for assessing effect modification claims in randomized trials or meta-analyses. J Clin Epidemiol, 2019, 113: 159-167.
25. Schandelmaier S, Briel M, Varadhan R, et al. Development of the instrument to assess the credibility of effect modification analyses (ICEMAN) in randomized controlled trials and meta-analyses. CMAJ, 2020, 192(32): E901-E906.
26. 李戈, 楊智榮, 王巍巍, 等. 《GRADE在網狀Meta分析不精確性的處理》文獻解讀. 中國循證兒科雜志, 2021, 16(6): 487-489.
27. Guyatt GH, Oxman AD, Kunz R, et al. GRADE guidelines 6. Rating the quality of evidence--imprecision. J Clin Epidemiol, 2011, 64(12): 1283-1293.
28. Schünemann HJ. Interpreting GRADE's levels of certainty or quality of the evidence: GRADE for statisticians, considering review information size or less emphasis on imprecision. J Clin Epidemiol, 2016, 75: 6-15.
29. GRADE Working Group. A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis. BMJ, 2015, 350: h3326.
30. Papakonstantinou T, Nikolakopoulou A, Rücker G, et al. Estimating the contribution of studies in network meta-analysis: paths, flows and streams. F1000Res, 2018, 7: 610.
31. Brignardello-Petersen R, Guyatt G H. β-Blockers in heart failure--are all created equal? Pol Arch Med Wewn, 2013, 123(5): 204-205.
32. Schünemann HJ, Wiercioch W, Brozek J, et al. GRADE Evidence to Decision (EtD) frameworks for adoption, adaptation, and de novo development of trustworthy recommendations: GRADE-ADOLOPMENT. J Clin Epidemiol, 2017, 81: 101-110.
33. 黃嘉杰, 賴鴻皓, 孫銘謠, 等. 實施性研究綜合框架(CFIR)更新解讀. 中國全科醫學, 2023, 26(31): 3863-3871, 3876.
34. 賴鴻皓, 孫銘謠, 黃嘉杰, 等. 中西醫結合臨床實踐指南實施框架構建的思考. 協和醫學雜志, 2023, 14(1): 31-38.
35. 葛龍, 秦鈺, 宋忠陽, 等. 新型冠狀病毒感染恢復期常見癥中西醫結合管理循證實踐指南. 蘭州大學學報(醫學版), 2023, 49(5): 28-40,46.
36. 田晨, 孫銘謠, 李穎, 等. 快速動態循證要覽制訂方法與流程. 中國循證醫學雜志, 2023, 23(4): 450-456.
37. 孫銘謠, 田晨, 李穎, 等. 藥物治療新型冠狀病毒感染快速動態循證要覽. 中國循證醫學雜志, 2023, 23(7): 745-754.
38. 田晨, 楊秋玉, 孫銘謠, 等. 快速動態循證指南制訂方法. 協和醫學雜志, 2024, 15(1): 183-191.
39. 孫銘謠, 曾茵茹, 賴鴻皓, 等. 動態指南發展現狀與挑戰. 中國循證醫學雜志, 2023, 23(8): 970-977.
40. Institute of Social and Preventive Medicine. CINeMA: confidence in network meta-analysis. 2017.
41. 田晨, 晏毅龍, 王勇, 等. 證據智能合成與分級: AutoMeta平臺開發與驗證. 中國循證醫學雜志, 2024, 24(4): 459-465.
42. 賴鴻皓, 王浙, 李瀅, 等. 中醫藥多元證據整合方法學MERGE框架. 協和醫學雜志, 2024, 15(1): 172-182.
43. 陳昊, 王艷, 陳耀龍, 等. 從GRADE系統方法學看中醫藥臨床證據體系的構建. 中華中醫藥雜志, 2017, 32(1): 25-28.
44. 張薇, 李小娟, 鄧宏勇. 中醫臨床證據分級和推薦體系發展現狀. 中國中醫藥信息雜志, 2020, 27(5): 133-136.
45. 賴鴻皓, 田晨, 朱鴻飛, 等. 如何整合隨機與非隨機干預研究. 中國循證醫學雜志, 2022, 22(7): 836-841.
46. Cuello-Garcia CA, Santesso N, Morgan RL, et al. GRADE guidance 24 optimizing the integration of randomized and non-randomized studies of interventions in evidence syntheses and health guidelines. J Clin Epidemiol, 2022, 142: 200-208.

Previous Article
Principles of latent variable mixture modeling and its value in clinical research applications

Chinese Journal of Evidence-Based Medicine

Evidence certainty grading of network meta-analysis: method update and case application

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Format

Content