Variability evaluation and empirical study in the importance rating of clinical questions and outcomes in the development of guidelines_Chinese Journal of Evidence-Based Medicine

Authors：

GAO Yicheng ^1,2,3 , YU Zijin ⁴ , WANG Yaqi ^1,2,3 , FANG Rui ⁵ , WANG Cheng ⁵ , LI Yuanyuan ⁵ , DENG Yingjie ⁵ , XIANG Wenyuan ⁵ ,  FEI Yutong ^1,2,3

1. Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing 100029, P. R. China;
2. Institute for Excellence in Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing 100029, P. R. China;
3. Beijing GRADE Center, Beijing University of Chinese Medicine, Beijing 100029, P. R. China;
4. School of Acupuncture-Moxibustion and Tuina, Beijing University of Chinese Medicine, Beijing 100029, P. R. China;
5. Affiliated Hospital of Traditional Chinese Medicine, Xinjiang Medical University, Urumqi 830000, P. R. China;

Corresponding?author：

FEI Yutong, Email: yutong_fei@163.com

Keywords：

Clinical question; Variability; Outcome; Methodology; Clinical practice guideline

DOI：

10.7507/1672-2531.202402106

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Objective The application of the coefficient of variation (CV) in the development of clinical practice guidelines is limited to evaluating the consistency of the consensus panel in clinical questions rating, and the application of variability was limited. This study presents the application and results of variability evaluation in the development of guidelines. Methods We conducted a large-scale clinical survey through questionnaire survey, and conducted two rounds of questionnaire survey and face-to-face consensus meeting for the consensus group. Means and CV were calculated for clinical questions and outcome importance ratings. We performed the summary and analysis by SPSS and Microsoft Excel. Results A total of 356 clinical survey questionnaires and two rounds survey in consensus panel were collected. We found that in the clinical survey and the first-round of the consensus panel, the CV was greater than 25% for all clinical questions regardless of the overall importance score. In the consensus panel, the results of the second-round were greatly changed. On the one hand, compared with the first-round, the CV of almost all clinical questions was smaller in the second-round, and the CV of high-priority clinical questions was less than 25%, while the clinical questions with a CV greater than 25% were of low-priority. In view of the CV of outcome importance, the clinical survey was similar to the results of the first-round of consensus panel. The CV of very important outcomes was less than 30%. In the second-round of consensus panel, the variability of very important outcomes was less than 20%. The higher the importance level of the outcome was, the smaller the CV was. Conclusion The study of variability evaluation has practical methodological value, which can assist clinical questions and outcomes priority selection, and help to fully consider the influence of different factors and values, and develop high-quality guidelines.

Citation： GAO Yicheng, YU Zijin, WANG Yaqi, FANG Rui, WANG Cheng, LI Yuanyuan, DENG Yingjie, XIANG Wenyuan, FEI Yutong. Variability evaluation and empirical study in the importance rating of clinical questions and outcomes in the development of guidelines. Chinese Journal of Evidence-Based Medicine, 2024, 24(10): 1198-1204. doi: 10.7507/1672-2531.202402106 Copy

1.	Institute of Medicine. Clinical practice guidelines we can trust. Washington DC: The National Academies Press, 2011, 16-25.
2.	Guyatt GH, Oxman AD, Kunz R, et al. GRADE guidelines: 2. Framing the question and deciding on important outcomes. J Clin Epidemiol, 2011, 64(4): 395-400.
3.	El-Harakeh A, Morsi RZ, Fadlallah R, et al. Prioritization approaches in the development of health practice guidelines: a systematic review. BMC Health Serv Res, 2019, 19(1): 692.
4.	Wiercioch W, Nieuwlaat R, Dahm P, et al. Development and application of health outcome descriptors facilitated decision-making in the production of practice guidelines. J Clin Epidemiol, 2021, 138: 115-127.
5.	Wiercioch W, Nieuwlaat R, Zhang Y, et al. New methods facilitated the process of prioritizing questions and health outcomes in guideline development. J Clin Epidemiol, 2022, 143: 91-104.
6.	Zaror C, Deana NF, Espinoza-Espinoza G, et al. Questions and health outcomes prioritization for the development of a COVID-19 dental clinical practice guideline: a case study. J Eval Clin Pract, 2022, 28(3): 404-410.
7.	Murphy MK, Black NA, Lamping DL, et al. Consensus development methods, and their use in clinical guideline development. Health Tech Assess, 1998, 2(3): 1-88.
8.	Chapple M, Murphy R. The nominal group technique: extending the evaluation of students' teaching and learning experiences. Assess Eval High Edu, 1996, 21(2): 147-160.
9.	高一城, 夏如玉, 柴倩云, 等. 組建患者小組: 將患者價值觀與偏好整合融入指南制訂過程的建議. 協和醫學雜志, 2023, (6): 1-13.
10.	肖晶旻, 謝倩文, 劉少南, 等. 循證中醫藥實踐中患者價值觀及偏好研究的概括性評價. 中醫雜志, 2019, 60(24): 2089-2094.
11.	李元, 韓學杰, 王麗穎. 中醫臨床實踐指南共識專家遴選方法. 北京中醫藥, 2021, 40(9): 1002-1004.
12.	周殷華, 程瑜. 國際干細胞醫療市場治理與醫生的職業價值觀. 廣西民族大學學報(哲學社會科學版), 2017, 39(1): 20-26.
13.	王偉, 陳軍. 生命價值觀視角下醫生對患者的態度調查. 中國醫學倫理學, 2016, 29(6): 963-965.
14.	Xing D, Wang Q, Yang Z, et al. Evidence-based guidelines for intra-articular injection in knee osteoarthritis: formulating and evaluating research questions. Int J Rheum Dis, 2018, 21(8): 1533-1542.
15.	高一城, 劉美君, 蘇澄元, 等. 變異性量化應用在指南臨床問題構建中的方法學建議. 中國循證醫學雜志, 2023, 23(11): 1329-1333.
16.	《中成藥治療優勢病種臨床應用指南》標準化項目組. 中成藥治療更年期綜合征臨床應用指南(2020年). 中國中西醫結合雜志, 2021, 41(4): 418-426.
17.	楊超, 杜世豪, 王舢澤, 等. 基于改良德爾菲法構建《女性尿失禁針灸臨床實踐指南》臨床問題. 中國針灸, 2022, 42(8): 927-931.
18.	Sackett DL, Rosenberg WM, Gray JA, et al. Evidence based medicine: what it is and what it isn't. BMJ, 1996, 312(7023): 71-72.
19.	高一城, 夏如玉, 王雅琪, 等. 循證臨床實踐指南中臨床問題結構完整性及表述規范性的建議. 中國循證醫學雜志, 2023, 23(1): 120-124.
20.	Chakraborty S, Brijnath B, Dermentzis J, et al. Defining key questions for clinical practice guidelines: a novel approach for developing clinically relevant questions. Health Res Policy Syst, 2020, 18(1): 113.
21.	陳薇, 郭宇博, 方賽男, 等. 中醫藥臨床實踐指南中臨床問題的確定. 中國中西醫結合雜志, 2020, 40(2): 220-223.
22.	李慧, 陳耀龍, 韋當, 等. 中醫(中西醫結合)臨床實踐指南制修訂方法—臨床問題的構建. 中華中醫藥雜志, 2016, 31(6): 2202-2205.
23.	Sanabria AJ, Pardo-Hernandez H, Ballesteros M, et al. The UpPriority tool was developed to guide the prioritization of clinical guideline questions for updating. J Clin Epidemiol, 2020, 126: 80-92.
24.	Speckman RA, Friedly JL. Asking structured, answerable clinical questions using the population, intervention/comparator, outcome (pico) framework. PM R, 2019, 11(5): 548-553.

1. Institute of Medicine. Clinical practice guidelines we can trust. Washington DC: The National Academies Press, 2011, 16-25.
2. Guyatt GH, Oxman AD, Kunz R, et al. GRADE guidelines: 2. Framing the question and deciding on important outcomes. J Clin Epidemiol, 2011, 64(4): 395-400.
3. El-Harakeh A, Morsi RZ, Fadlallah R, et al. Prioritization approaches in the development of health practice guidelines: a systematic review. BMC Health Serv Res, 2019, 19(1): 692.
4. Wiercioch W, Nieuwlaat R, Dahm P, et al. Development and application of health outcome descriptors facilitated decision-making in the production of practice guidelines. J Clin Epidemiol, 2021, 138: 115-127.
5. Wiercioch W, Nieuwlaat R, Zhang Y, et al. New methods facilitated the process of prioritizing questions and health outcomes in guideline development. J Clin Epidemiol, 2022, 143: 91-104.
6. Zaror C, Deana NF, Espinoza-Espinoza G, et al. Questions and health outcomes prioritization for the development of a COVID-19 dental clinical practice guideline: a case study. J Eval Clin Pract, 2022, 28(3): 404-410.
7. Murphy MK, Black NA, Lamping DL, et al. Consensus development methods, and their use in clinical guideline development. Health Tech Assess, 1998, 2(3): 1-88.
8. Chapple M, Murphy R. The nominal group technique: extending the evaluation of students' teaching and learning experiences. Assess Eval High Edu, 1996, 21(2): 147-160.
9. 高一城, 夏如玉, 柴倩云, 等. 組建患者小組: 將患者價值觀與偏好整合融入指南制訂過程的建議. 協和醫學雜志, 2023, (6): 1-13.
10. 肖晶旻, 謝倩文, 劉少南, 等. 循證中醫藥實踐中患者價值觀及偏好研究的概括性評價. 中醫雜志, 2019, 60(24): 2089-2094.
11. 李元, 韓學杰, 王麗穎. 中醫臨床實踐指南共識專家遴選方法. 北京中醫藥, 2021, 40(9): 1002-1004.
12. 周殷華, 程瑜. 國際干細胞醫療市場治理與醫生的職業價值觀. 廣西民族大學學報(哲學社會科學版), 2017, 39(1): 20-26.
13. 王偉, 陳軍. 生命價值觀視角下醫生對患者的態度調查. 中國醫學倫理學, 2016, 29(6): 963-965.
14. Xing D, Wang Q, Yang Z, et al. Evidence-based guidelines for intra-articular injection in knee osteoarthritis: formulating and evaluating research questions. Int J Rheum Dis, 2018, 21(8): 1533-1542.
15. 高一城, 劉美君, 蘇澄元, 等. 變異性量化應用在指南臨床問題構建中的方法學建議. 中國循證醫學雜志, 2023, 23(11): 1329-1333.
16. 《中成藥治療優勢病種臨床應用指南》標準化項目組. 中成藥治療更年期綜合征臨床應用指南(2020年). 中國中西醫結合雜志, 2021, 41(4): 418-426.
17. 楊超, 杜世豪, 王舢澤, 等. 基于改良德爾菲法構建《女性尿失禁針灸臨床實踐指南》臨床問題. 中國針灸, 2022, 42(8): 927-931.
18. Sackett DL, Rosenberg WM, Gray JA, et al. Evidence based medicine: what it is and what it isn't. BMJ, 1996, 312(7023): 71-72.
19. 高一城, 夏如玉, 王雅琪, 等. 循證臨床實踐指南中臨床問題結構完整性及表述規范性的建議. 中國循證醫學雜志, 2023, 23(1): 120-124.
20. Chakraborty S, Brijnath B, Dermentzis J, et al. Defining key questions for clinical practice guidelines: a novel approach for developing clinically relevant questions. Health Res Policy Syst, 2020, 18(1): 113.
21. 陳薇, 郭宇博, 方賽男, 等. 中醫藥臨床實踐指南中臨床問題的確定. 中國中西醫結合雜志, 2020, 40(2): 220-223.
22. 李慧, 陳耀龍, 韋當, 等. 中醫(中西醫結合)臨床實踐指南制修訂方法—臨床問題的構建. 中華中醫藥雜志, 2016, 31(6): 2202-2205.
23. Sanabria AJ, Pardo-Hernandez H, Ballesteros M, et al. The UpPriority tool was developed to guide the prioritization of clinical guideline questions for updating. J Clin Epidemiol, 2020, 126: 80-92.
24. Speckman RA, Friedly JL. Asking structured, answerable clinical questions using the population, intervention/comparator, outcome (pico) framework. PM R, 2019, 11(5): 548-553.

Chinese Journal of Evidence-Based Medicine

Variability evaluation and empirical study in the importance rating of clinical questions and outcomes in the development of guidelines

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content