Reliability of Objective Structured Clinical Examination: A Meta-Analysis_Chinese Journal of Evidence-Based Medicine

Authors：

 WANGHai-ping ¹ , XIANGLiang-cheng ¹ , ZHANGWei ¹ , YUANJia-ying ¹ , LIMei ² ,  FENGPing ^1,2

1. Department of Statistics, West China School of Public Health, Sichuan University, Chengdu 610041, China;
2. National Institution of Clinical Trial of Drug, West China Hospital, Sichuan University, Chengdu 610041, China;

Corresponding?author：

FENGPing, Email: pfyq@yahoo.com

Keywords：

Objective structured clinical examination; Reliability; Coefficient of Cronbach's α; Meta-analysis

DOI：

10.7507/1672-2531.20140145

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Objective To conduct meta-analysis with the reliability of objective structured clinical examination (OSCE), so as to the reliability of OSCE used in the tests for evaluating clinical capacities of medical students. Methods Articles about evaluating clinical capacities of medical students using OSCE and using comprehensive coefficient of Cronbach's α to measure the reliability of OSCE were electronically searched in PubMed, ScienceDirect, CNKI, WanFang data and VIP from Jan. 1998 to May. 2013. Two reviewers screened literature according to the inclusion and exclusion criteria, extracted data, and assessed the methodological quality of the included studies. Then meta-analysis was performed using SSPS 17.0 software. Results Thirty-four studies involving 53 coefficients of Cronbach's α were included, of which, there were 18 articles written in English involving 28 coefficients of Cronbach's α and 16 articles written in Chinese involving 25 coefficients of Cronbach's α. The results of meta-analysis showed that:the total coefficient of Cronbach's α of OSCE was 0.700 (95%CI 0.660 to 0.737). The coefficient of Cronbach's α of internationally-published literature was 0.745 (95%CI 0.696 to 0.790) and that of nationally-published literature was 0.648 (95%CI 0.584 to 0.705), with a significant difference between two groups. Conclusion Currently, the reliability of internationally/nationally-used OSCE is 0.7, which has relatively better reliability when used in the tests for evaluating clinical capacities of medical students at home and aboard. However, the reliability of OSCE at abroad is fairly better than that at home.

Citation： WANGHai-ping, XIANGLiang-cheng, ZHANGWei, YUANJia-ying, LIMei, FENGPing. Reliability of Objective Structured Clinical Examination: A Meta-Analysis. Chinese Journal of Evidence-Based Medicine, 2014, 14(7): 878-883. doi: 10.7507/1672-2531.20140145 Copy

1.	Harden RM, Stevenson M, Downie WW, et al. Assessment of clinical competence using objective structured examination. BMJ, 1975, 22(1):447-451.
2.	Sidhu RS, Grober ED, Musselman LJ, et al. Assessing competency in surgery:Where to begin. Surgery, 2004, 135(1):6-20.
3.	Wanzel KR, Ward M, Reznick RK, et al. Teaching the surgical craft:From selection to certification. Curr Probl Surg, 2002, 39(6):573-659.
4.	曾五一, 黃炳藝.調查問卷的可信度和有效度分析.統計與信息論壇, 2005, 20(6):11-15.
5.	Schultz JH, Nikendei C, Weyrich P, et al. Quality assurance of assessments using the example of the OSCE examination format:experiences of the Medical School of Heidelberg University. Z Evid Fortbild Qual Gesundhwes, 2008, 102(10):668-672.
6.	鄒揚, 繆青, 蘆開芳, 等.本科和長學制畢業考試中客觀結構化臨床考試的應用.上海交通大學學報(醫學版), 2008, 28(z1):71-75.
7.	Corrie K, Wiles M, Flack J, et al. A novel objective structured clinical examination for the assessment of transfusion practice in anaesthesia. Clin Teach, 2011, 8(2):97-100.
8.	Rodriguez MC, Maeda Y. Meta-analysis of coefficient alpha. Psychol Methods, 2006, 11(3):306-322.
9.	Eberhard L, Hassel A, B?umer A, et al. Analysis of quality and feasibility of an objective structured clinical examination (OSCE) in preclinical dental education. Eur J Dent Educ, 2011, 15(3):172-178.
10.	Yang YY, LEE FY, Hsu HC, et al. A core competence-based objective structured clinical examination (OSCE) in evaluation of clinical performance of postgraduate year-1 (PGY-1) residents. J Chin Med Assoc, 2011, 74(5):198-204.
11.	Pandya JS, Bhagwat SM, Kini SL, et al. Evaluation of clinical skills for first-year surgical residents using orientation programme and objective structured clinical evaluation as a tool of assessment. J Postgrad Med, 2010, 56(4):297-300.
12.	Yu TC, Wheeler BR, Hill AG. Effectiveness of standardized clerkship teaching across multiple sites. J Surg Res, 2011, 168(1):e17-e23.
13.	Shehmar M, Cruikshank M, Finn C, et al. A validity study of the national UK colposcopy objective structured clinical examination:is it a test fit for purpose? BJOG, 2009, 116(13):1796-1999.
14.	Heckmann JG, Knossalla F, Gollwitzer S, et al. OSCE in the neurology clerkship. Experiences at the neurological department of the university hospital Erlangen. Fortschr Neurol Psychiatr, 2009, 77(1):32-37.
15.	Huang CC, Chan CY, Wu CL, et al. Assessment of Clinical Competence of Medical Students Using the Objective Structured Clinical Examination:First 2 Years' Experience in Taipei Veterans General Hospital. J Chin Med Assoc, 2010, 73(11):589-595.
16.	Pedregal GM, Molina FE, Prados Castillejo JA, et al. Evaluation of clinical competence of family medicine trainer. Aten Primaria, 2004, 34(2):68-72.
17.	Wessel J, Williams R, Finch E, et al. Reliability and validity of an objective structured clinical examination for physical therapy students. J Allied Health, 2003, 32(4):266-269.
18.	Rothman Al, Cusimano M. A comparison of physician examiners', standardized patients', and communication experts' ratings of international medical graduates' English proficiency. Acad Med, 2000, 75(12):1206-1211.
19.	Schwartz RW, Witzke DB, Donnelly MB, et al. Assessing residents' clinical performance:cumulative results of a four-year study with the Objective Structured Clinical Examination. Surgery, 1998, 124(2):307-312.
20.	Sidhu RS, Mcllroy JH, Regehr G, et al. Using a Comprehensive Examination to Assess Multiple Competencies in Surgical Residents:Does the Oral Examination Still Have a Role? J Am Coll Surg, 2001(5):754-758.
21.	Bosse HM, Schultz JH, Nickel M, et al. The effect of using standardized patients or peer role play on ratings of undergraduate communication training:A randomized controlled trial. Patient Educ Couns, 2012, 87(3):300-306.
22.	Yang YY, LEE FY, Hsu HC, et al. Assessment of first-year post-graduate residents:Usefulness of multiple tools. J Chin Med Assoc, 2011, 74(12):531-538.
23.	MacRae H, Regehr G, Leadbetter W, et al. A comprehensive examination for senior surgical residents. Am J Surg, 2000, 179(3):190-193.
24.	Zabar S, Hanley K, Stevens DL, et al. Can interactive skills-based seminars with standardized patients enhance clinicians' prevention skills? Measuring the impact of a CME program. Patient Educ Couns, 2010, 80(2):248-252.
25.	Ault G, Rezinck R, MacRae H, et al. Exporting a technical skills evaluation technology to other sites. Am J Surg, 2001, 182(3):254-256.
26.	陳英, 陳淳, 曾凡, 等. OSCE在本科護生臨床實習評價中的應用研究.護理學雜志, 2011, 26(15):5-7.
27.	陳瑤, 厲英, 周玲君, 等.簡化的客觀結構化臨床考核在《急救護理學》技能考核中的應用.解放軍護理雜志, 2011, 28(5B):67-70.
28.	鄭克勤, 孟慶勇, 劉瓊玲, 等.護理學專業本科畢業技能考試改革的探索與實踐.廣東醫學院學報, 2012, 30(5):580-583.
29.	安力彬, 李文濤, 張萍.客觀結構化臨床考試在護理本科生畢業考核中的應用研究.中國高等醫學, 2009, (3):16-17.
30.	黃燁, 王斌全, 商臨萍.客觀結構化臨床考試在護理本科生臨床綜合能力評價中的應用研究, 護理研究, 2008, 22(3):567-569.
31.	沈娟, 崔焱, 林征, 等.客觀結構化臨床考試在護理本科畢業生臨床能力評價中的應用研究.護理進修雜志, 2009, 24(5):395-397.
32.	路中, 戚麗, 李貴新, 等.內科標準化病人在評價實習醫師臨床技能中的應用.西北醫學教育, 2009, 17(2):308-310.
33.	劉敏, 李穎.應用客觀結構化臨床考試評價護理本科畢業生的臨床能力.中華護理教育, 2011, 8(8):348-349.
34.	吳建軍, 劉革新, 陳三妹, 等.客觀結構化臨床考試在內科護理學中的應用.齊魯護理雜志, 2010, 16(17):109-111.
35.	吳末生, 殷朝陽, 彭紅, 等.從多站考試的成績分析探討臨床能力評價的有效途徑.中國高等醫學教育, 2000, (2):6-8.
36.	于曉松, 孫寶志, 時瑾, 等.對醫學生臨床綜合能力評價的初步研究.中國高等醫學教育, 2000, (2):9-10.
37.	張艷, 于蘭貞, 馬國平, 等.客觀結構化臨床考試在ICU專科護生臨床能力評價中的應用研究.護理研究, 2012, 26(3):656-659.
38.	朱濱海, 占伊揚, 孫寧生, 等.客觀結構化臨床考試在江蘇省住院醫師培訓階段考核中的應用與分析.中國醫院, 2012, 116(12):56-59.
39.	孫瑋.國家臨床醫師資格考試理論考試的實證效度研究.沈陽:中國醫科大學, 2010.
40.	Hedges LV, Vevea JL. Fixed-and random-effects models in meta-analysis. Psych Meth, 1998, 3(4):486-504.
41.	羅海波, 胡靖, 王安莉.淺析如何提高OSCE質量.中國中醫藥現代遠程教育, 2007, 5(8):58-59.
42.	許克祥, 齊建強, 劉赟.中醫院校開展客觀結構化臨床考試的思考與探索.黑龍江醫藥科學, 2012, 35(1):63-63.
43.	魯威, 張劍戈, 繆青, 等.客觀結構化考試的信度及主成分分析.上海交通大學學報(醫學版), 2008, 28(12):1577-1580.
44.	Hofer M, Kamper L, Sadio M, et al. Evaluation of an OSCE assessment tool for abdominal ultrasound courses. Ultraschall Med, 2011, 32(2):184-190.
45.	Sloan DA, Donnelly MB, Schwartz RW, et al. The Objective Structured Clinical Examination:The New Gold Standard for Evaluating Postgraduate Clinical Performance. Ann Surg, 1995, 222(6):735-742.
46.	高鐳, 李曉松, 萬學紅, 等.客觀結構化臨床考試的多元概化模型研究.中國高等醫學教育, 2004, 3(1):46-49.

1. Harden RM, Stevenson M, Downie WW, et al. Assessment of clinical competence using objective structured examination. BMJ, 1975, 22(1):447-451.
2. Sidhu RS, Grober ED, Musselman LJ, et al. Assessing competency in surgery:Where to begin. Surgery, 2004, 135(1):6-20.
3. Wanzel KR, Ward M, Reznick RK, et al. Teaching the surgical craft:From selection to certification. Curr Probl Surg, 2002, 39(6):573-659.
4. 曾五一, 黃炳藝.調查問卷的可信度和有效度分析.統計與信息論壇, 2005, 20(6):11-15.
5. Schultz JH, Nikendei C, Weyrich P, et al. Quality assurance of assessments using the example of the OSCE examination format:experiences of the Medical School of Heidelberg University. Z Evid Fortbild Qual Gesundhwes, 2008, 102(10):668-672.
6. 鄒揚, 繆青, 蘆開芳, 等.本科和長學制畢業考試中客觀結構化臨床考試的應用.上海交通大學學報(醫學版), 2008, 28(z1):71-75.
7. Corrie K, Wiles M, Flack J, et al. A novel objective structured clinical examination for the assessment of transfusion practice in anaesthesia. Clin Teach, 2011, 8(2):97-100.
8. Rodriguez MC, Maeda Y. Meta-analysis of coefficient alpha. Psychol Methods, 2006, 11(3):306-322.
9. Eberhard L, Hassel A, B?umer A, et al. Analysis of quality and feasibility of an objective structured clinical examination (OSCE) in preclinical dental education. Eur J Dent Educ, 2011, 15(3):172-178.
10. Yang YY, LEE FY, Hsu HC, et al. A core competence-based objective structured clinical examination (OSCE) in evaluation of clinical performance of postgraduate year-1 (PGY-1) residents. J Chin Med Assoc, 2011, 74(5):198-204.
11. Pandya JS, Bhagwat SM, Kini SL, et al. Evaluation of clinical skills for first-year surgical residents using orientation programme and objective structured clinical evaluation as a tool of assessment. J Postgrad Med, 2010, 56(4):297-300.
12. Yu TC, Wheeler BR, Hill AG. Effectiveness of standardized clerkship teaching across multiple sites. J Surg Res, 2011, 168(1):e17-e23.
13. Shehmar M, Cruikshank M, Finn C, et al. A validity study of the national UK colposcopy objective structured clinical examination:is it a test fit for purpose? BJOG, 2009, 116(13):1796-1999.
14. Heckmann JG, Knossalla F, Gollwitzer S, et al. OSCE in the neurology clerkship. Experiences at the neurological department of the university hospital Erlangen. Fortschr Neurol Psychiatr, 2009, 77(1):32-37.
15. Huang CC, Chan CY, Wu CL, et al. Assessment of Clinical Competence of Medical Students Using the Objective Structured Clinical Examination:First 2 Years' Experience in Taipei Veterans General Hospital. J Chin Med Assoc, 2010, 73(11):589-595.
16. Pedregal GM, Molina FE, Prados Castillejo JA, et al. Evaluation of clinical competence of family medicine trainer. Aten Primaria, 2004, 34(2):68-72.
17. Wessel J, Williams R, Finch E, et al. Reliability and validity of an objective structured clinical examination for physical therapy students. J Allied Health, 2003, 32(4):266-269.
18. Rothman Al, Cusimano M. A comparison of physician examiners', standardized patients', and communication experts' ratings of international medical graduates' English proficiency. Acad Med, 2000, 75(12):1206-1211.
19. Schwartz RW, Witzke DB, Donnelly MB, et al. Assessing residents' clinical performance:cumulative results of a four-year study with the Objective Structured Clinical Examination. Surgery, 1998, 124(2):307-312.
20. Sidhu RS, Mcllroy JH, Regehr G, et al. Using a Comprehensive Examination to Assess Multiple Competencies in Surgical Residents:Does the Oral Examination Still Have a Role? J Am Coll Surg, 2001(5):754-758.
21. Bosse HM, Schultz JH, Nickel M, et al. The effect of using standardized patients or peer role play on ratings of undergraduate communication training:A randomized controlled trial. Patient Educ Couns, 2012, 87(3):300-306.
22. Yang YY, LEE FY, Hsu HC, et al. Assessment of first-year post-graduate residents:Usefulness of multiple tools. J Chin Med Assoc, 2011, 74(12):531-538.
23. MacRae H, Regehr G, Leadbetter W, et al. A comprehensive examination for senior surgical residents. Am J Surg, 2000, 179(3):190-193.
24. Zabar S, Hanley K, Stevens DL, et al. Can interactive skills-based seminars with standardized patients enhance clinicians' prevention skills? Measuring the impact of a CME program. Patient Educ Couns, 2010, 80(2):248-252.
25. Ault G, Rezinck R, MacRae H, et al. Exporting a technical skills evaluation technology to other sites. Am J Surg, 2001, 182(3):254-256.
26. 陳英, 陳淳, 曾凡, 等. OSCE在本科護生臨床實習評價中的應用研究.護理學雜志, 2011, 26(15):5-7.
27. 陳瑤, 厲英, 周玲君, 等.簡化的客觀結構化臨床考核在《急救護理學》技能考核中的應用.解放軍護理雜志, 2011, 28(5B):67-70.
28. 鄭克勤, 孟慶勇, 劉瓊玲, 等.護理學專業本科畢業技能考試改革的探索與實踐.廣東醫學院學報, 2012, 30(5):580-583.
29. 安力彬, 李文濤, 張萍.客觀結構化臨床考試在護理本科生畢業考核中的應用研究.中國高等醫學, 2009, (3):16-17.
30. 黃燁, 王斌全, 商臨萍.客觀結構化臨床考試在護理本科生臨床綜合能力評價中的應用研究, 護理研究, 2008, 22(3):567-569.
31. 沈娟, 崔焱, 林征, 等.客觀結構化臨床考試在護理本科畢業生臨床能力評價中的應用研究.護理進修雜志, 2009, 24(5):395-397.
32. 路中, 戚麗, 李貴新, 等.內科標準化病人在評價實習醫師臨床技能中的應用.西北醫學教育, 2009, 17(2):308-310.
33. 劉敏, 李穎.應用客觀結構化臨床考試評價護理本科畢業生的臨床能力.中華護理教育, 2011, 8(8):348-349.
34. 吳建軍, 劉革新, 陳三妹, 等.客觀結構化臨床考試在內科護理學中的應用.齊魯護理雜志, 2010, 16(17):109-111.
35. 吳末生, 殷朝陽, 彭紅, 等.從多站考試的成績分析探討臨床能力評價的有效途徑.中國高等醫學教育, 2000, (2):6-8.
36. 于曉松, 孫寶志, 時瑾, 等.對醫學生臨床綜合能力評價的初步研究.中國高等醫學教育, 2000, (2):9-10.
37. 張艷, 于蘭貞, 馬國平, 等.客觀結構化臨床考試在ICU專科護生臨床能力評價中的應用研究.護理研究, 2012, 26(3):656-659.
38. 朱濱海, 占伊揚, 孫寧生, 等.客觀結構化臨床考試在江蘇省住院醫師培訓階段考核中的應用與分析.中國醫院, 2012, 116(12):56-59.
39. 孫瑋.國家臨床醫師資格考試理論考試的實證效度研究.沈陽:中國醫科大學, 2010.
40. Hedges LV, Vevea JL. Fixed-and random-effects models in meta-analysis. Psych Meth, 1998, 3(4):486-504.
41. 羅海波, 胡靖, 王安莉.淺析如何提高OSCE質量.中國中醫藥現代遠程教育, 2007, 5(8):58-59.
42. 許克祥, 齊建強, 劉赟.中醫院校開展客觀結構化臨床考試的思考與探索.黑龍江醫藥科學, 2012, 35(1):63-63.
43. 魯威, 張劍戈, 繆青, 等.客觀結構化考試的信度及主成分分析.上海交通大學學報(醫學版), 2008, 28(12):1577-1580.
44. Hofer M, Kamper L, Sadio M, et al. Evaluation of an OSCE assessment tool for abdominal ultrasound courses. Ultraschall Med, 2011, 32(2):184-190.
45. Sloan DA, Donnelly MB, Schwartz RW, et al. The Objective Structured Clinical Examination:The New Gold Standard for Evaluating Postgraduate Clinical Performance. Ann Surg, 1995, 222(6):735-742.
46. 高鐳, 李曉松, 萬學紅, 等.客觀結構化臨床考試的多元概化模型研究.中國高等醫學教育, 2004, 3(1):46-49.

Previous Article
Distribution of Aminoglycoside-resistant Genes in Pseudomonas aeruginosa in China: A Literature Study
Next Article
Differentiation and Handling of Homogeneity in Network Meta-analysis

Chinese Journal of Evidence-Based Medicine

Reliability of Objective Structured Clinical Examination: A Meta-Analysis

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content