Objective
To evaluate the methodological and reporting quality of systematic reviews/meta-analyses related to the efficacy and safety of corticosteroid-assisted treatment for severe pneumonia.
Methods
PubMed, Embase, Cochrane Library, China National Knowledge Infrastructure, WanFang Data and VIP databases were searched by computer, and the systematic reviews/meta-analyses of corticosteroid hormone as an auxiliary means for the treatment of severe pneumonia which were published from establishment of the databases to October 25th, 2018 were searched. A Measurement Tool to Assess Systematic Review-2 (AMSTAR-2) was used to assess the methodological quality of the included studies, and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) was used to evaluate the quality of literature reports.
Results
A total of 16 systematic reviews/meta-analyses were included, all of which were non-Cochrane systematic reviews. In terms of methodological quality assessed by AMSTAR-2, there was no plan in all studies; only one study explained the reasons for inclusion in the study type; eight studies did not describe the dose and follow-up time of the intervention/control measures in detail; three studies did not indicate the evaluation tools and did not describe the risk bias; six studies did not explicitly examine publication bias. In terms of reporting quality assessed by PRISMA, all studies had no pre-registered study protocol or registration number; thirteen studies did not describe the specific amount of articles retrieved from each database; three studies did not present their retrieval strategies or excluded reasons in detail; no funding sources were identified in included studies; eight studies reported both whether the study was funded and whether there was a conflict of interest.
Conclusions
At present, there are many systematic review/meta-analysis studies on the efficacy and safety of corticosteroid-assisted treatment for severe pneumonia, and the overall quality of the study has been gradually improved. However, the common problems in the study are relatively prominent. The follow-up period and dose of intervention in the study of severe pneumonia are different, so the baseline is difficult to be unified. Suggestions: strengthening the training of researchers, standardize the research process, and report articles in strict accordance with the PRISMA statement; subgroup analysis being conducted according to the dose and duration of the hormone.
ObjectiveUsing SYRCLE tool (the SYstematic Review Centre for Laboratory animal Experimentation) to evaluate the risk of bias of animal studies in stroke field published in Chinese journals, identify problems of these studies in design, implementation and measurement, in order to provide references for improving the quality of animal studies in China.
MethodsWe searched databases including CBM, VIP, CNKI and WanFang Data from inception to December 31st, 2014. Two reviewers independently screened literature, extracted data, and assessed the risk of bias of included animal studies using SYRCLE tool developed by the National Centre for the Replacement, Refinement and Reduction of Animals in Research.
ResultsA total of 582 studies were included. The assessment results showed that the number of reported items with "Low Risk" in SYRCLE, which have 22 items, reported in >50% of the 4 items and in <30% the 16 items in articles of animals experiments. More than 99% of the studies fulfilled the 3 items and more than 17% of the articles to meet the 10 items while less than 1% of the documents met the 17 items. The quality of studies increased excepted the period of 2010 to 2014. The methodological quality of animal experiments presented a trend of increasing and no significant differences were found in CSCD indexed or not.
ConclusionThe methodological quality of animal experiments of stroke is poor in China in terms of the selection bias, implementation bias, measurement bias, withdraw bias and reporting bias.
ObjectiveTo provide references for scientific selection of different tools/guidelines by comprehensively collecting international and national tools/guidelines for assessing reporting quality and methodological quality of animal experiments, comparing them in development foundation, application scope, and aims.
MethodsPubMed, EMbase, CNKI, VIP and WanFang Data were searched up to July 2014, to collect tools/guidelines for reporting quality and methodological quality of primary animal experiments. We extracted data from included guidelines/tools, including the number of items, development foundation, disease models, application scope, and assessment focus. Then descriptive analysis was conducted.
ResultsA total of 32 studies were finally included, of which, 6 were for reporting quality and 26 for methodological quality. The item number of the included tools/guidelines ranged from 2 to 54. Seven tools/guidelines applied score system to assess methodological quality. Fifteen tools/guidelines were designed for specific disease models. Nineteen tools/guidelines were suitable for assessing preclinical drug studies, and 4 were designed to assess environmental toxicology research.
ConclusionAlthough many tools for assessing methodological quality of animal experiments have been published so far, SYRCLE's risk of bias tool is the only one that is used to assess internal validity of animal experiments at present. Besides, although the ARRIVE guidelines and GSPC are not official mandatory reporting criteria at present, they are acknowledged by many researchers as efficient reference checklists and writing guidelines for writing and publishing animal experiments. We recommend the application of SYRCLE's risk of bias tool, ARRIVE guidelines and GSPC, in order to efficiently improve research design, implementation, reporting, differentiation, and evaluation of animal experiments, promote the development of animal experiments, and to promote full application and translation of scientific achievements.
ObjectivesTo evaluate the methodology quality and report quality of the published systematic reviews/meta-analyses (SRs/MAs) of pediatric tuina domestically and abroad.MethodsCBM, VIP, CNKI, WanFang Data, PubMed, EMbase, and The Cochrane Library were electronically searched to collect published pediatric tuina SRs/MAs from inception to December 10th, 2018. The SRs/MAs which includes scale evaluation used AMSTAR2 and the PRISMA report quality evaluation tool to systematically review methodology, adopts Excel to carry out data collation and statistical analysis. ResultsA total of 18 studies (14 in Chinese and 4 in English) on the SRs/MAs of pediatric tuina were finally included. In terms of methodological quality, 6 studies were of low quality and 12 studies were of very low quality. All studies did not explain the reasons for adopting a particular research design type, and few of them explained the pre-plan, exclusion list, reasons and funding. In terms of report quality, 7 studies were relatively complete, 10 studies had certain defects and one study had serious defects. The existing problems were program and registration, comprehensive retrieval, information sources, financial support and so on. ConclusionsSRs/MAs of pediatric tuina have different degrees of issues in terms of methodological quality and report quality which still require further improvement and continuous strengthening.
ObjectivesTo assess the methodological quality and reporting quality of meta-analysis published on The Chinese Journal of Nursing.MethodsCNKI and WanFang Data databases were electronically searched to collect meta-analysis which published on The Chinese Journal of Nursing from inception to December 2017. Two reviewers independently screened literature, extracted data and assessed the methodological quality and the reporting quality by AMSTAR scale and PRISMA statement. Statistical analysis was then performed by using SPSS 19.0 software.ResultsA total of 53 meta-analyses were included, which involved 7 disease systems and sub-health status. The mean score of the methodological assessment by AMSTAR was 7.75±1.32, including 9 high-quality papers (17.0%), 41 middle-quality papers (77.4%), and 3 low-quality papers (5.6%). The mean score of the reporting quality assessment by PRISMA was 22.5±3.08, including 39 relatively complete papers (73.6%), 11 papers with certain defects (20.8%), and 3 papers with serious defects (5.6%).ConclusionsThe methodological and reporting quality of meta-analysis published on The Chinese Journal of Nursing deserves further improvement.
ObjectiveTo evaluate the current status and trend of methodological quality of multi-center randomized controlled trials (RCTs) of stroke treatments in Chinese Mainland.MethodsMulti-center RCTs of stroke treatments conducted in Chinese Mainland published in Chinese or English language from January 2000 to December 2019 were retrieved from seven databases including PubMed, Cochrane Central Registry of Controlled Trials, Embase, China Biology Medicine, China National Knowledge Infrastructure, Chinese Science and Technique Journals Database, and Wanfang Database. The basic information was collected. Methodological items were referred to the Cochrane Collaboration’s tool for assessing risk of bias. The definitions of Wade were used to assess the outcome measure.ResultsA total of 90 multi-center RCTs were included, of which 39 were published from 2000 to 2009, and 51 were published from 2010 to 2019. The total number of trials published from 2010 to 2019 was 1.31 times of that published from 2000 to 2009. The research subjects were ischemic stroke patients in 58.9% (53/90) of the RCTs, intracerebral hemorrhage patients in 14.4% (13/90) of the RCTs, and ischemic stroke patients as well as hemorrhagic stroke patients in 26.7% (24/90) of the RCTs. There were 55.6% (50/90) drug trials, and 44.4% (40/90) non-drug trials. There were statistically significant differences in the loss of visit report (P=0.005), primary and secondary outcome indicators report (P=0.027), and adverse reaction report (P=0.007) between the two periods; there was no statistically significant difference in reported adequate randomized methods (P=0.341), allocation concealment (P=0.611), blindness (P=0.551), used intentionality analysis (P=0.573), or follow-up time (P=0.061) between the two periods.ConclusionIn the past 20 years in Chinese Mainland, the quality of stroke treatment RCTs improves slowly, and more attention should be paid to develop the RCTs of true randomization, blinding, and better patient outcome measures.
ObjectiveTo evaluate the methodological quality of animal experiments published in high impact journals, in order to provide references for improving the quality of animal experiments.MethodsCSCD and Web of Science databases were electronically searched to collect intervening primordial animal experiments from 2014 to August, 2016. Four reviewers independently screened literatures, extracted data and assessed the methodological quality of included studies by using SYRCLE tool.ResultsA total of 1 999 animal experiments were included. The cited frequency of more than 90% studies were ≤5 times, and of which 52.53% studies were zero. The results of SYRCLE evaluation showed that 54.55% of sub items rated as "low risk" were less than 30%. And 84.62% of them were less than 10%.ConclusionThere are defeet in methodological quality of animal experiments either domestic or abroad. The problems of domestic researches in implementation bias, measurement bias and loss of access bias are particularly obvious. The coincidence rates of "low risk" are much lower than those of abroad studies. Therefore, we suggest that it is necessary to take specific measures to popularize SYRCLE tool to effectively guide the development of animal experiments and improve the design and implementation of animal experiments.
Objective To systematically review the methodological quality of research on clinical prediction models of traditional Chinese medicine. Methods The PubMed, Embase, Web of Science, CNKI, WanFang Data, VIP and SinoMed databases were electronically searched to collect literature related to the research on clinical prediction models of traditional Chinese medicine from inception to March 31, 2023. Two reviewers independently screened literature, extracted data and assessed the risk of bias of the included studies based on prediction model risk of bias assessment tool (PROBAST). Results A total of 113 studies on clinical prediction models of traditional Chinese medicine (79 diagnostic model studies and 34 prognostic model studies) were included. Among them, 111 (98.2%) studies were rated at high risk of bias, while 1 (0.9%) study was rated at low risk of bias and risk of bias of 1 (0.9%) study was unclear. The analysis domain was rated with the highest proportion of high risk of bias, followed by the participants domain. Due to the widespread lack of reporting of specific study information, risk of bias of a large number of studies was unclear in both predictors and outcome domain. Conclusion Most existing researches on clinical prediction models of traditional Chinese medicine show poor methodological quality and are at high risk of bias. Factors contributing to risk of bias include non-prospective data source, outcome definitions that include predictors, inadequate modeling sample size, inappropriate feature selection, inaccurate performance evaluation, and incorrect internal validation methods. Comprehensive methodological improvements on design, conduct, evaluation, and validation of modeling, as well as reporting of all key information of the models are urgently needed for future modeling studies, aiming to facilitate their translational application in medical practice.
ObjectiveTo evaluate the methodological quality of clinical practice guidelines and expert consensus of chronic heart failure domestically and abroad.MethodsPubMed, EMbase, SinoMed, CNKI, WanFang Data, and VIP databases, and related websites were searched to collect guidelines and expert consensus on chronic heart failure published from January 1st, 2011 to December 31st, 2020. Four reviewers evaluated the methodological quality of the guidelines and expert consensus with the AGREE Ⅱ tool after the consistency evaluation training.ResultsA total of 17 studies were included (consisting of 11 English and 6 Chinese studies). The recommended levels were B level (recommend after modification) for 10 studies and C level (not recommended) for 7 studies. The AGREE Ⅱ standardized mean scores for various fields were 69.61% (scope and purpose), 34.20% (stakeholder involvement), 33.13% (rigor of development), 84.53% (clarity and presentation), 42.40% (applicability), and 37.09% (editorial independence). The methodological quality of English guidelines was generally high (level B for 10 and level C for 1), while all scores of Chinese guidelines or consensus in the 6 fields were mostly lower than the average (level C for 6).ConclusionsThe guidelines for the diagnosis and treatment of chronic heart failure requires further improvement in terms of stakeholder involvement and rigor of development. It should develop standards and methods to improve the quality for Chinese guidelines and expert consensus to better serve clinical practice.
This paper summarizes the methodological quality assessment tools of artificial intelligence-based diagnostic test accuracy studies, and introduces QUADAS-AI and modified QUADAS-2. Moreover, this paper summarizes reporting guidelines of these studies as well, and then introduces specific reporting standards in AI-centred research, and checklist for AI in dental research.
