Assessment tool ArRoWS for the quality of real world observational studies: an interpretation_Chinese Journal of Evidence-Based Medicine

Authors：

CAO Xue ¹ , MENG Xiangran ¹ , WANG Xin ¹ , SUN Minglin ¹ , DENG Hui ² , AI Yanke ¹ ,  LIU Jia ¹

1. Institute of Basic Research in Clinical Medicine, China Academy of Chinese Medical Sciences, Beijing 100700, P. R. China;
2. College of Acupuncture and Moxibustion, Shanxi University of Chinese Medicine, Xianyang 712046, P. R. China;

Corresponding?author：

LIU Jia, Email: marie_liujia@sina.cn

Keywords：

Observational studies; Real world evidence; Appraisal tool; ArRoWS

DOI：

10.7507/1672-2531.202206043

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Assessment of Real World Observational Studies (ArRoWS) is a tool developed by the Leicester Real World Evidence (LRWE) Unit of the Diabetes Research Centre of the University of Leicester in the United Kingdom to assess the quality of real world evidence research, and has been reported to have good practicability. ArRoWS can be used to quickly and specifically assess the quality of real world evidence research that uses electronic health record information. The tool contains 16 items, nine of which are common items, and seven of which are related to specific research designs. The current study introduces the development background, development process, assessment items, assessment criteria, and application methods of ArRoWS and other related aspects, to provide references for real world researchers in China.

Citation： CAO Xue, MENG Xiangran, WANG Xin, SUN Minglin, DENG Hui, AI Yanke, LIU Jia. Assessment tool ArRoWS for the quality of real world observational studies: an interpretation. Chinese Journal of Evidence-Based Medicine, 2023, 23(2): 227-232. doi: 10.7507/1672-2531.202206043 Copy

1.	孫鑫, 譚婧, 唐立, 等. 重新認識真實世界研究. 中國循證醫學雜志, 2017, 17(2): 126-130.
2.	國家藥品監督管理局. 用于產生真實世界證據的真實世界數據指導原則(試行). Available at: https://www.cde.org.cn/main/news/viewInfoCommon/2a1c437ed54e7b838a7e86f4ac21c539.
3.	王雯, 譚婧, 任燕, 等. 重新認識真實世界數據研究: 更新與展望. 中國循證醫學雜志, 2020, 20(11): 1241-1246.
4.	孫鑫, 譚婧, 王雯, 等. 建立真實世界數據與研究技術規范, 促進中國真實世界證據的生產與使用. 中國循證醫學雜志, 2019, 19(7): 755-762.
5.	王雪, 烏蘭其其格, 楊悅. 美國應用真實世界證據輔助醫藥產品監管決策的研究. 中國藥事, 2020, 34(9): 1057-1062.
6.	王麗茹, 劉東輝, 邢杰, 等. 基于真實世界數據的結直腸癌預后特征識別. 國際遺傳學雜志, 2021, 44(6): 401-408.
7.	孟若谷, 卓琳, 喬瑞, 等. 真實世界證據在上市后藥品安全性監測與評價中的應用價值. 中國藥物警戒, 2021, 18(7): 624-627.
8.	任靜, 王永樂, 劉亭亭, 等. 雙抗治療輕型缺血性卒中或短暫性腦缺血發作——從臨床指南到真實世界研究. 中國全科醫學, 2021, 24(21): 2696-2700, 2706.
9.	明丹丹, 李軍, 許璇, 等. 真實世界研究的方法及其在臨床研究中的應用. 中國藥房, 2018, 29(15): 2138-2143.
10.	中華中醫藥學會《中醫藥真實世界研究技術規范》制訂組. 中醫藥真實世界研究技術規范——證據質量評價與報告. 中醫雜志, 2022, 63(3): 293-300.
11.	Ma LL, Wang YY, Yang ZH, et al. Methodological quality (risk of bias) assessment tools for primary and secondary medical studies: what are they and which is better. Mil Med Res, 2020, 7(1): 7.
12.	Zeng X, Zhang Y, Kwong JS, et al. The methodological quality assessment tools for preclinical and clinical studies, systematic review and meta-analysis, and clinical practice guideline: a systematic review. J Evid Based Med, 2015, 8(1): 2-10.
13.	Wells G A, Shea B, O'Connell D. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. Available at: https://www.ohri.ca/programs/clinical_epidemiology/oxford.asp.
14.	Critical Appraisal Skills Programme. CASP (case control study checklist). Available at: https://casp-uk.net/casp-tools-checklists/.
15.	廖星, 章軼立, 謝雁鳴. 真實世界研究標準: RECORD清單和GRACE清單的解讀. 中國中藥雜志, 2015, 40(24): 4734-4738.
16.	Coles B, Tyrer F, Hussein H, et al. Development, content validation, and reliability of the assessment of real-world observational studies (ArRoWS) critical appraisal tool. Ann Epidemiol, 2021, 55: 57-63.
17.	Coles B, Tyrer F, Khunti K. Improving quality in cardiometabolic medicine real-world evidence. Prim Care Diabetes, 2019, 13(2): 188.
18.	Sanderson S, Tatt ID, Higgins JP. Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography. Int J Epidemiol, 2007, 36(3): 666-676.
19.	陳耀龍, 李幼平, 杜亮, 等. 醫學研究中證據分級和推薦強度的演進. 中國循證醫學雜志, 2008, 8(2): 127-133.
20.	Waldram A, McKerr C, Gobin M, et al. Control selection methods in recent case-control studies conducted as part of infectious disease outbreaks. Eur J Epidemiol, 2015, 30(6): 465-471.
21.	Rose S, Laan MJ. Why match? Investigating matched case-control study designs with causal effect estimation. Int J Biostat, 2009, 5(1): Article 1.
22.	Pocock S, Kleinbaum D, Kupper L, et al. Epidemiologic research: principles and quantitative methods. Biometrics, 1983, 39(3): 822.
23.	Breslow NE, Day NE. Statistical methods in cancer research. Volume I - The analysis of case-control studies. IARC Sci Publ, 1980, (32): 5-338.
24.	Pearce N. Analysis of matched case-control studies. BMJ, 2016, 352: i969.
25.	Hsu CW, Tseng WT, Wang LJ, et al. Comparative effectiveness of antidepressants on geriatric depression: real-world evidence from a population-based study. J Affect Disord, 2022, 296: 609-615.
26.	Schechter M, Melzer-Cohen C, Rozenberg A, et al. Cardiorenal outcomes with sodium/glucose cotransporter-2 inhibitors in patients with type 2 diabetes and low kidney risk: real world evidence. Cardiovasc Diabetol, 2021, 20(1): 169.
27.	艾飛玲, 胡葵茹, 石鈺霖, 等. 基于紐卡斯爾-渥太華量表對中國吸煙隊列研究文獻的質量評價. 中華疾病控制雜志, 2021, 25(6): 722-729.
28.	張蒙, 張雪梅, 楊璐, 等. 我國老年譫妄危險因素的隊列研究文獻的質量評價. 護理研究, 2016, 30(33): 4206-4207.
29.	楊璐, 楊佳, 張澤梅, 等. 我國老年譫妄預后隊列研究的文獻質量評價. 護理研究, 2016, (1): 80-82.
30.	何瓊, 黃淵秀, 康文婧, 等. 2001-2010年我國傷害預防病例對照研究及隊列研究文獻質量評價. 中華疾病控制雜志, 2014, 18(10): 913-916.
31.	王梅, 王建華, 張抗, 等. 中醫藥療效評價隊列研究的方法學質量評價. 中醫雜志, 2016, 57(16): 1379-1383.
32.	張學成, 劉巖, 張晶晶, 等. 急性心力衰竭患者30天再住院率: 基于真實世界研究與隨機對照試驗比較的Meta分析. 中國循證醫學雜志, 2021, 21(10): 1203-1210.
33.	曾憲濤, 劉慧, 陳曦, 等. Meta分析系列之四: 觀察性研究的質量評價工具. 中國循證心血管醫學雜志, 2012, 04(4): 297-299.
34.	中華中醫藥學會. 中醫藥真實世界研究技術規范證據質量評價與報告(試行). Available at: https://www.cacm.org.cn/2021/06/30/13874/.
35.	Khambholja K, Gehani M. Use of structured template and reporting tool for real-world evidence template for critical appraisal of the quality of reporting of real-world evidence studies: a systematic review. Value Health, 2022. DOI: 10.1016/j.jval.2022.09.003.

1. 孫鑫, 譚婧, 唐立, 等. 重新認識真實世界研究. 中國循證醫學雜志, 2017, 17(2): 126-130.
2. 國家藥品監督管理局. 用于產生真實世界證據的真實世界數據指導原則(試行). Available at: https://www.cde.org.cn/main/news/viewInfoCommon/2a1c437ed54e7b838a7e86f4ac21c539.
3. 王雯, 譚婧, 任燕, 等. 重新認識真實世界數據研究: 更新與展望. 中國循證醫學雜志, 2020, 20(11): 1241-1246.
4. 孫鑫, 譚婧, 王雯, 等. 建立真實世界數據與研究技術規范, 促進中國真實世界證據的生產與使用. 中國循證醫學雜志, 2019, 19(7): 755-762.
5. 王雪, 烏蘭其其格, 楊悅. 美國應用真實世界證據輔助醫藥產品監管決策的研究. 中國藥事, 2020, 34(9): 1057-1062.
6. 王麗茹, 劉東輝, 邢杰, 等. 基于真實世界數據的結直腸癌預后特征識別. 國際遺傳學雜志, 2021, 44(6): 401-408.
7. 孟若谷, 卓琳, 喬瑞, 等. 真實世界證據在上市后藥品安全性監測與評價中的應用價值. 中國藥物警戒, 2021, 18(7): 624-627.
8. 任靜, 王永樂, 劉亭亭, 等. 雙抗治療輕型缺血性卒中或短暫性腦缺血發作——從臨床指南到真實世界研究. 中國全科醫學, 2021, 24(21): 2696-2700, 2706.
9. 明丹丹, 李軍, 許璇, 等. 真實世界研究的方法及其在臨床研究中的應用. 中國藥房, 2018, 29(15): 2138-2143.
10. 中華中醫藥學會《中醫藥真實世界研究技術規范》制訂組. 中醫藥真實世界研究技術規范——證據質量評價與報告. 中醫雜志, 2022, 63(3): 293-300.
11. Ma LL, Wang YY, Yang ZH, et al. Methodological quality (risk of bias) assessment tools for primary and secondary medical studies: what are they and which is better. Mil Med Res, 2020, 7(1): 7.
12. Zeng X, Zhang Y, Kwong JS, et al. The methodological quality assessment tools for preclinical and clinical studies, systematic review and meta-analysis, and clinical practice guideline: a systematic review. J Evid Based Med, 2015, 8(1): 2-10.
13. Wells G A, Shea B, O'Connell D. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. Available at: https://www.ohri.ca/programs/clinical_epidemiology/oxford.asp.
14. Critical Appraisal Skills Programme. CASP (case control study checklist). Available at: https://casp-uk.net/casp-tools-checklists/.
15. 廖星, 章軼立, 謝雁鳴. 真實世界研究標準: RECORD清單和GRACE清單的解讀. 中國中藥雜志, 2015, 40(24): 4734-4738.
16. Coles B, Tyrer F, Hussein H, et al. Development, content validation, and reliability of the assessment of real-world observational studies (ArRoWS) critical appraisal tool. Ann Epidemiol, 2021, 55: 57-63.
17. Coles B, Tyrer F, Khunti K. Improving quality in cardiometabolic medicine real-world evidence. Prim Care Diabetes, 2019, 13(2): 188.
18. Sanderson S, Tatt ID, Higgins JP. Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography. Int J Epidemiol, 2007, 36(3): 666-676.
19. 陳耀龍, 李幼平, 杜亮, 等. 醫學研究中證據分級和推薦強度的演進. 中國循證醫學雜志, 2008, 8(2): 127-133.
20. Waldram A, McKerr C, Gobin M, et al. Control selection methods in recent case-control studies conducted as part of infectious disease outbreaks. Eur J Epidemiol, 2015, 30(6): 465-471.
21. Rose S, Laan MJ. Why match? Investigating matched case-control study designs with causal effect estimation. Int J Biostat, 2009, 5(1): Article 1.
22. Pocock S, Kleinbaum D, Kupper L, et al. Epidemiologic research: principles and quantitative methods. Biometrics, 1983, 39(3): 822.
23. Breslow NE, Day NE. Statistical methods in cancer research. Volume I - The analysis of case-control studies. IARC Sci Publ, 1980, (32): 5-338.
24. Pearce N. Analysis of matched case-control studies. BMJ, 2016, 352: i969.
25. Hsu CW, Tseng WT, Wang LJ, et al. Comparative effectiveness of antidepressants on geriatric depression: real-world evidence from a population-based study. J Affect Disord, 2022, 296: 609-615.
26. Schechter M, Melzer-Cohen C, Rozenberg A, et al. Cardiorenal outcomes with sodium/glucose cotransporter-2 inhibitors in patients with type 2 diabetes and low kidney risk: real world evidence. Cardiovasc Diabetol, 2021, 20(1): 169.
27. 艾飛玲, 胡葵茹, 石鈺霖, 等. 基于紐卡斯爾-渥太華量表對中國吸煙隊列研究文獻的質量評價. 中華疾病控制雜志, 2021, 25(6): 722-729.
28. 張蒙, 張雪梅, 楊璐, 等. 我國老年譫妄危險因素的隊列研究文獻的質量評價. 護理研究, 2016, 30(33): 4206-4207.
29. 楊璐, 楊佳, 張澤梅, 等. 我國老年譫妄預后隊列研究的文獻質量評價. 護理研究, 2016, (1): 80-82.
30. 何瓊, 黃淵秀, 康文婧, 等. 2001-2010年我國傷害預防病例對照研究及隊列研究文獻質量評價. 中華疾病控制雜志, 2014, 18(10): 913-916.
31. 王梅, 王建華, 張抗, 等. 中醫藥療效評價隊列研究的方法學質量評價. 中醫雜志, 2016, 57(16): 1379-1383.
32. 張學成, 劉巖, 張晶晶, 等. 急性心力衰竭患者30天再住院率: 基于真實世界研究與隨機對照試驗比較的Meta分析. 中國循證醫學雜志, 2021, 21(10): 1203-1210.
33. 曾憲濤, 劉慧, 陳曦, 等. Meta分析系列之四: 觀察性研究的質量評價工具. 中國循證心血管醫學雜志, 2012, 04(4): 297-299.
34. 中華中醫藥學會. 中醫藥真實世界研究技術規范證據質量評價與報告(試行). Available at: https://www.cacm.org.cn/2021/06/30/13874/.
35. Khambholja K, Gehani M. Use of structured template and reporting tool for real-world evidence template for critical appraisal of the quality of reporting of real-world evidence studies: a systematic review. Value Health, 2022. DOI: 10.1016/j.jval.2022.09.003.

Previous Article
Patient version of guidelines: current practice and research gaps
Next Article
Interpretation of standards for conducting and reporting Delphi studies (CREDES) in healthcare research

Chinese Journal of Evidence-Based Medicine

Assessment tool ArRoWS for the quality of real world observational studies: an interpretation

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content