Specific guidelines for assessing and improving the methodological quality of economic evaluations of newborn screening

PubMed Central

2012-01-01

Background Economic evaluation of newborn screening poses specific methodological challenges. Amongst others, these challenges refer to the use of quality adjusted life years (QALYs) in newborns, and which costs and outcomes need to be considered in a full evaluation of newborn screening programmes. Because of the increasing scale and scope of such programmes, a better understanding of the methods of high-quality economic evaluations may be crucial for both producers/authors and consumers/reviewers of newborn screening-related economic evaluations. The aim of this study was therefore to develop specific guidelines designed to assess and improve the methodological quality of economic evaluations in newborn screening. Methods To develop the guidelines, existing guidelines for assessing the quality of economic evaluations were identified through a literature search, and were reviewed and consolidated using a deductive iterative approach. In a subsequent test phase, these guidelines were applied to various economic evaluations which acted as case studies. Results The guidelines for assessing and improving the methodological quality of economic evaluations in newborn screening are organized into 11 categories: “bibliographic details”, “study question and design”, “modelling”, “health outcomes”, “costs”, “discounting”, “presentation of results”, “sensitivity analyses”, “discussion”, “conclusions”, and “commentary”. Conclusions The application of the guidelines highlights important issues regarding newborn screening-related economic evaluations, and underscores the need for such issues to be afforded greater consideration in future economic evaluations. The variety in methodological quality detected by this study reveals the need for specific guidelines on the appropriate methods for conducting sound economic evaluations in newborn screening. PMID:22947299

Systematic review adherence to methodological or reporting quality.

PubMed

Pussegoda, Kusala; Turner, Lucy; Garritty, Chantelle; Mayhew, Alain; Skidmore, Becky; Stevens, Adrienne; Boutron, Isabelle; Sarkis-Onofre, Rafael; Bjerre, Lise M; Hróbjartsson, Asbjørn; Altman, Douglas G; Moher, David

2017-07-19

Guidelines for assessing methodological and reporting quality of systematic reviews (SRs) were developed to contribute to implementing evidence-based health care and the reduction of research waste. As SRs assessing a cohort of SRs is becoming more prevalent in the literature and with the increased uptake of SR evidence for decision-making, methodological quality and standard of reporting of SRs is of interest. The objective of this study is to evaluate SR adherence to the Quality of Reporting of Meta-analyses (QUOROM) and PRISMA reporting guidelines and the A Measurement Tool to Assess Systematic Reviews (AMSTAR) and Overview Quality Assessment Questionnaire (OQAQ) quality assessment tools as evaluated in methodological overviews. The Cochrane Library, MEDLINE®, and EMBASE® databases were searched from January 1990 to October 2014. Title and abstract screening and full-text screening were conducted independently by two reviewers. Reports assessing the quality or reporting of a cohort of SRs of interventions using PRISMA, QUOROM, OQAQ, or AMSTAR were included. All results are reported as frequencies and percentages of reports and SRs respectively. Of the 20,765 independent records retrieved from electronic searching, 1189 reports were reviewed for eligibility at full text, of which 56 reports (5371 SRs in total) evaluating the PRISMA, QUOROM, AMSTAR, and/or OQAQ tools were included. Notable items include the following: of the SRs using PRISMA, over 85% (1532/1741) provided a rationale for the review and less than 6% (102/1741) provided protocol information. For reports using QUOROM, only 9% (40/449) of SRs provided a trial flow diagram. However, 90% (402/449) described the explicit clinical problem and review rationale in the introduction section. Of reports using AMSTAR, 30% (534/1794) used duplicate study selection and data extraction. Conversely, 80% (1439/1794) of SRs provided study characteristics of included studies. In terms of OQAQ, 37% (499/1367) of the

Methodological quality and implications for practice of systematic Cochrane reviews in pediatric oral health: a critical assessment

PubMed Central

2014-01-01

Background To ensure evidence-based decision-making in pediatric oral health, Cochrane systematic reviews that address topics pertinent to this field are necessary. We aimed to identify all systematic reviews of paediatric dentistry and oral health by the Cochrane Oral Health Group (COHG), summarize their characteristics and assess their methodological quality. Our second objective was to assess implications for practice in the review conclusions and provide an overview of clinical implications about the usefulness of paediatric oral health interventions in practice. Methods We conducted a methodological survey including all paediatric dentistry reviews from the COHG. We extracted data on characteristics of included reviews, then assessed the methodological quality using a validated 11-item quality assessment tool (AMSTAR). Finally, we coded each review to indicate whether its authors concluded that an intervention should be implemented in practice, was not supported or was refuted by the evidence, or should be used only in research (inconclusive evidence). Results We selected 37 reviews; most concerned the prevention of caries. The methodological quality was high, except for the assessment of reporting bias. In 7 reviews (19%), the research showed that benefits outweighed harms; in 1, the experimental intervention was found ineffective; and in 29 (78%), evidence was insufficient to assess benefits and harms. In the 7 reviews, topical fluoride treatments (with toothpaste, gel or varnish) were found effective for permanent and deciduous teeth in children and adolescents, and sealants for occlusal tooth surfaces of permanent molars. Conclusions Cochrane reviews of paediatric dentistry were of high quality. They provided strong evidence that topical fluoride treatments and sealants are effective for children and adolescents and thus should be implemented in practice. However, a substantial number of reviews yielded inconclusive evidence. PMID:24716532

Methodological issues in the quantitative assessment of quality of life.

PubMed

Panagiotakos, Demosthenes B; Yfantopoulos, John N

2011-10-01

The term quality of life can be identified in Aristotle's classical writings of 330 BC. In his Nichomachian ethics he recognises the multiple relationships between happiness, well-being, "eudemonia" and quality of life. Historically the concept of quality of life has undergone various interpretations. It involves personal experience, perceptions and beliefs, attitudes concerning philosophical, cultural, spiritual, psychological, political, and financial aspects of everyday living. Quality of life has been extensively used both as an outcome and an explanatory factor in relation to human health, in various clinical trials, epidemiologic studies and health interview surveys. Because of the variations in the definition of quality of life, both in theory and in practice, there are also a wide range of procedures that are used to assess quality of life. In this paper several methodological issues regarding the tools used to evaluate quality of life is discussed. In summary, the use of components consisted of large number of classes, as well as the use of specific weights for each scale component, and the low-to-moderate inter-correlation level between the components, is evident from simulated and empirical studies.

Evaluation of AMSTAR to assess the methodological quality of systematic reviews in overviews of reviews of healthcare interventions.

PubMed

Pollock, Michelle; Fernandes, Ricardo M; Hartling, Lisa

2017-03-23

Overviews of reviews (overviews) compile information from multiple systematic reviews (SRs) to provide a single synthesis of relevant evidence for decision-making. It is recommended that authors assess and report the methodological quality of SRs in overviews-for example, using A MeaSurement Tool to Assess systematic Reviews (AMSTAR). Currently, there is variation in whether and how overview authors assess and report SR quality, and limited guidance is available. Our objectives were to: examine methodological considerations involved in using AMSTAR to assess the quality of Cochrane and non-Cochrane SRs in overviews of healthcare interventions; identify challenges (and develop potential decision rules) when using AMSTAR in overviews; and examine the potential impact of considering methodological quality when making inclusion decisions in overviews. We selected seven overviews of healthcare interventions and included all SRs meeting each overview's inclusion criteria. For each SR, two reviewers independently conducted AMSTAR assessments with consensus and discussed challenges encountered. We also examined the correlation between AMSTAR assessments and SR results/conclusions. Ninety-five SRs were included (30 Cochrane, 65 non-Cochrane). Mean AMSTAR assessments (9.6/11 vs. 5.5/11; p < 0.001) and inter-rater reliability (AC1 statistic: 0.84 vs. 0.69; "almost perfect" vs. "substantial" using the Landis & Koch criteria) were higher for Cochrane compared to non-Cochrane SRs. Four challenges were identified when applying AMSTAR in overviews: the scope of the SRs and overviews often differed; SRs examining similar topics sometimes made different methodological decisions; reporting of non-Cochrane SRs was sometimes poor; and some non-Cochrane SRs included other SRs as well as primary studies. Decision rules were developed to address each challenge. We found no evidence that AMSTAR assessments were correlated with SR results/conclusions. Results indicate that the AMSTAR

Methodological quality of systematic reviews on influenza vaccination.

PubMed

Remschmidt, Cornelius; Wichmann, Ole; Harder, Thomas

2014-03-26

There is a growing body of evidence on the risks and benefits of influenza vaccination in various target groups. Systematic reviews are of particular importance for policy decisions. However, their methodological quality can vary considerably. To investigate the methodological quality of systematic reviews on influenza vaccination (efficacy, effectiveness, safety) and to identify influencing factors. A systematic literature search on systematic reviews on influenza vaccination was performed, using MEDLINE, EMBASE and three additional databases (1990-2013). Review characteristics were extracted and the methodological quality of the reviews was evaluated using the assessment of multiple systematic reviews (AMSTAR) tool. U-test, Kruskal-Wallis test, chi-square test, and multivariable linear regression analysis were used to assess the influence of review characteristics on AMSTAR-score. Fourty-six systematic reviews fulfilled the inclusion criteria. Average methodological quality was high (median AMSTAR-score: 8), but variability was large (AMSTAR range: 0-11). Quality did not differ significantly according to vaccination target group. Cochrane reviews had higher methodological quality than non-Cochrane reviews (p=0.001). Detailed analysis showed that this was due to better study selection and data extraction, inclusion of unpublished studies, and better reporting of study characteristics (all p<0.05). In the adjusted analysis, no other factor, including industry sponsorship or journal impact factor had an influence on AMSTAR score. Systematic reviews on influenza vaccination showed large differences regarding their methodological quality. Reviews conducted by the Cochrane collaboration were of higher quality than others. When using systematic reviews to guide the development of vaccination recommendations, the methodological quality of a review in addition to its content should be considered. Copyright © 2014 Elsevier Ltd. All rights reserved.

Use of methodological tools for assessing the quality of studies in periodontology and implant dentistry: a systematic review.

PubMed

Faggion, Clovis M; Huda, Fahd; Wasiak, Jason

2014-06-01

To evaluate the methodological approaches used to assess the quality of studies included in systematic reviews (SRs) in periodontology and implant dentistry. Two electronic databases (PubMed and Cochrane Database of Systematic Reviews) were searched independently to identify SRs examining interventions published through 2 September 2013. The reference lists of included SRs and records of 10 specialty dental journals were searched manually. Methodological approaches were assessed using seven criteria based on the Cochrane Handbook for Systematic Reviews of Interventions. Temporal trends in methodological quality were also explored. Of the 159 SRs with meta-analyses included in the analysis, 44 (28%) reported the use of domain-based tools, 15 (9%) reported the use of checklists and 7 (4%) reported the use of scales. Forty-two (26%) SRs reported use of more than one tool. Criteria were met heterogeneously; authors of 15 (9%) publications incorporated the quality of evidence of primary studies into SRs, whereas 69% of SRs reported methodological approaches in the Materials/Methods section. Reporting of four criteria was significantly better in recent (2010-2013) than in previous publications. The analysis identified several methodological limitations of approaches used to assess evidence in studies included in SRs in periodontology and implant dentistry. © 2014 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd.

Volume and methodological quality of randomized controlled trials in laparoscopic surgery: assessment over a 10-year period.

PubMed

Antoniou, Stavros A; Andreou, Alexandros; Antoniou, George A; Koch, Oliver O; Köhler, Gernot; Luketina, Ruzica-R; Bertsias, Antonios; Pointner, Rudolph; Granderath, Frank-Alexander

2015-11-01

Measures have been taken to improve methodological quality of randomized controlled trials (RCTs). This review systematically assessed the trends in volume and methodological quality of RCTs on minimally invasive surgery within a 10-year period. RCTs on minimally invasive surgery were searched in the 10 most cited general surgical journals and the 5 most cited journals of laparoscopic interest for the years 2002 and 2012. Bibliometric and methodological quality components were abstracted using the Scottish Intercollegiate Guidelines Network. The pooled number of RCTs from low-contribution regions demonstrated an increasing proportion of the total published RCTs, compensating for a concomitant decrease of the respective contributions from Europe and North America. International collaborations were more frequent in 2012. Acceptable or high quality RCTs accounted for 37.9% and 54.4% of RCTs published in 2002 and 2012, respectively. Components of external validity were poorly reported. Both the volume and the reporting quality of laparoscopic RCTs have increased from 2002 to 2012, but there seems to be ample room for improvement of methodological quality. Copyright © 2015 Elsevier Inc. All rights reserved.

Methodological quality of meta-analyses of single-case experimental studies.

PubMed

Jamshidi, Laleh; Heyvaert, Mieke; Declercq, Lies; Fernández-Castilla, Belén; Ferron, John M; Moeyaert, Mariola; Beretvas, S Natasha; Onghena, Patrick; Van den Noortgate, Wim

2017-12-28

Methodological rigor is a fundamental factor in the validity and credibility of the results of a meta-analysis. Following an increasing interest in single-case experimental design (SCED) meta-analyses, the current study investigates the methodological quality of SCED meta-analyses. We assessed the methodological quality of 178 SCED meta-analyses published between 1985 and 2015 through the modified Revised-Assessment of Multiple Systematic Reviews (R-AMSTAR) checklist. The main finding of the current review is that the methodological quality of the SCED meta-analyses has increased over time, but is still low according to the R-AMSTAR checklist. A remarkable percentage of the studies (93.80% of the included SCED meta-analyses) did not even reach the midpoint score (22, on a scale of 0-44). The mean and median methodological quality scores were 15.57 and 16, respectively. Relatively high scores were observed for "providing the characteristics of the included studies" and "doing comprehensive literature search". The key areas of deficiency were "reporting an assessment of the likelihood of publication bias" and "using the methods appropriately to combine the findings of studies". Although the results of the current review reveal that the methodological quality of the SCED meta-analyses has increased over time, still more efforts are needed to improve their methodological quality. Copyright © 2017 Elsevier Ltd. All rights reserved.

Methodological Quality Assessment of Meta-Analyses and Systematic Reviews of Probiotics in Inflammatory Bowel Disease and Pouchitis.

PubMed

Dong, Jinpei; Teng, Guigen; Wei, Tiantong; Gao, Wen; Wang, Huahong

2016-01-01

Probiotics are widely used for the induction and maintenance of remission in inflammatory bowel disease (IBD) and pouchitis. There are a large number of meta-analyses (MAs)/ systematic reviews (SRs) on this subject, the methodological quality of which has not been evaluated. This study aimed to evaluate the methodological quality of and summarize the evidence obtained from MAs/SRs of probiotic treatments for IBD and pouchitis patients. The PubMed, EMBASE, Cochrane Library and China National Knowledge Infrastructure (CNKI) databases were searched to identify Chinese and English language MAs/SRs of the use of probiotics for IBD and pouchitis. The Assessment of Multiple Systematic Reviews (AMSTAR) scale was used to assess the methodological quality of the studies. A total of 36 MAs/SRs were evaluated. The AMSTAR scores of the included studies ranged from 1 to 10, and the average score was 5.81. According to the Canadian Agency for Drugs and Technologies in Health, 4 articles were classified as high quality, 24 articles were classified as moderate quality, and 8 articles were classified as low quality. Most of the MAs/SRs suggested that probiotics had potential benefits for patients with ulcerative colitis (UC), but failed to show effectiveness in the induction and maintenance of remission in Crohn's disease (CD). The probiotic preparation VSL#3 may play a beneficial role in pouchitis. The overall methodological quality of the current MAs/SRs in the field of probiotics for IBD and pouchitis was found to be low to moderate. More MAs/SRs of high quality are required to support using probiotics to treat IBD and pouchitis.

METHODOLOGICAL QUALITY OF ECONOMIC EVALUATIONS ALONGSIDE TRIALS OF KNEE PHYSIOTHERAPY.

PubMed

García-Pérez, Lidia; Linertová, Renata; Arvelo-Martín, Alejandro; Guerra-Marrero, Carolina; Martínez-Alberto, Carlos Enrique; Cuéllar-Pompa, Leticia; Escobar, Antonio; Serrano-Aguilar, Pedro

2017-01-01

The methodological quality of an economic evaluation performed alongside a clinical trial can be underestimated if the paper does not report key methodological features. This study discusses methodological assessment issues on the example of a systematic review on cost-effectiveness of physiotherapy for knee osteoarthritis. Six economic evaluation studies included in the systematic review and related clinical trials were assessed using the 10-question check-list by Drummond and the Physiotherapy Evidence Database (PEDro) scale. All economic evaluations were performed alongside a clinical trial but the studied interventions were too heterogeneous to be synthesized. Methodological quality of the economic evaluations reported in the papers was not free of drawbacks, and in some cases, it improved when information from the related clinical trial was taken into account. Economic evaluation papers dedicate little space to methodological features of related clinical trials; therefore, the methodological quality can be underestimated if evaluated separately from the trials. Future economic evaluations should follow more strictly the recommendations about methodology and the authors should pay special attention to the quality of reporting.

Methodological quality of guidelines in gastroenterology.

PubMed

Malheiro, Rui; de Monteiro-Soares, Matilde; Hassan, Cesare; Dinis-Ribeiro, Mário

2014-06-01

Clinical guidelines are a common feature in modern endoscopy practice and they are being produced faster than ever. However, their methodological quality is rarely assessed. This study evaluated the methodological quality of current clinical guidelines in the field of gastroenterology, with an emphasis on endoscopy. Practice guidelines published by the American College of Gastroenterology (ACG), American Gastroenterological Association (AGA), American Society for Gastrointestinal Endoscopy (ASGE), European Society of Gastrointestinal Endoscopy (ESGE), British Society of Gastroenterology (BSG), National Institute for Health and Care Excellence (NICE), and the Scottish Intercollegiate Guidelines Network (SIGN) were searched between September and October 2012 and evaluated using the AGREE II (Appraisal of Guidelines for Research and Evaluation) instrument (23 items, scores 1 - 7 for each item; higher scores mean better quality). A total of 100 guidelines were assessed. The mean number of items scoring 6 or 7 per guideline was 9.2 (out of 23 items). Overall, 99 % of guidelines failed to include the target population in the development process, and 96 % did not report facilitators and barriers to guideline application. In addition, 86 % did not include advice or tools, and 94 % did not present monitoring or auditing criteria. The global methodological quality of clinical guidelines in the field of gastroenterology is poor, particularly regarding involvement of the target population in the development of guidelines and in the provision of clear suggestions to practitioners. © Georg Thieme Verlag KG Stuttgart · New York.

Methodological quality of diagnostic accuracy studies on non-invasive coronary CT angiography: influence of QUADAS (Quality Assessment of Diagnostic Accuracy Studies included in systematic reviews) items on sensitivity and specificity.

PubMed

Schueler, Sabine; Walther, Stefan; Schuetz, Georg M; Schlattmann, Peter; Dewey, Marc

2013-06-01

To evaluate the methodological quality of diagnostic accuracy studies on coronary computed tomography (CT) angiography using the QUADAS (Quality Assessment of Diagnostic Accuracy Studies included in systematic reviews) tool. Each QUADAS item was individually defined to adapt it to the special requirements of studies on coronary CT angiography. Two independent investigators analysed 118 studies using 12 QUADAS items. Meta-regression and pooled analyses were performed to identify possible effects of methodological quality items on estimates of diagnostic accuracy. The overall methodological quality of coronary CT studies was merely moderate. They fulfilled a median of 7.5 out of 12 items. Only 9 of the 118 studies fulfilled more than 75 % of possible QUADAS items. One QUADAS item ("Uninterpretable Results") showed a significant influence (P = 0.02) on estimates of diagnostic accuracy with "no fulfilment" increasing specificity from 86 to 90 %. Furthermore, pooled analysis revealed that each QUADAS item that is not fulfilled has the potential to change estimates of diagnostic accuracy. The methodological quality of studies investigating the diagnostic accuracy of non-invasive coronary CT is only moderate and was found to affect the sensitivity and specificity. An improvement is highly desirable because good methodology is crucial for adequately assessing imaging technologies. • Good methodological quality is a basic requirement in diagnostic accuracy studies. • Most coronary CT angiography studies have only been of moderate design quality. • Weak methodological quality will affect the sensitivity and specificity. • No improvement in methodological quality was observed over time. • Authors should consider the QUADAS checklist when undertaking accuracy studies.

The methodological quality of systematic reviews of animal studies in dentistry.

PubMed

Faggion, C M; Listl, S; Giannakopoulos, N N

2012-05-01

Systematic reviews and meta-analyses of animal studies are important for improving estimates of the effects of treatment and for guiding future clinical studies on humans. The purpose of this systematic review was to assess the methodological quality of systematic reviews and meta-analyses of animal studies in dentistry through using a validated checklist. A literature search was conducted independently and in duplicate in the PubMed and LILACS databases. References in selected systematic reviews were assessed to identify other studies not captured by the electronic searches. The methodological quality of studies was assessed independently and in duplicate by using the AMSTAR checklist; the quality was scored as low, moderate, or high. The reviewers were calibrated before the assessment and agreement between them was assessed using Cohen's Kappa statistic. Of 444 studies retrieved, 54 systematic reviews were selected after full-text assessment. Agreement between the reviewers was regarded as excellent. Only two studies were scored as high quality; 17 and 35 studies were scored as medium and low quality, respectively. There is room for improvement of the methodological quality of systematic reviews of animal studies in dentistry. Checklists, such as AMSTAR, can guide researchers in planning and executing systematic reviews and meta-analyses. For determining the need for additional investigations in animals and in order to provide good data for potential application in human, such reviews should be based on animal experiments performed according to sound methodological principles. Copyright © 2011 Elsevier Ltd. All rights reserved.

Methodological quality and reporting of systematic reviews in hand and wrist pathology.

PubMed

Wasiak, J; Shen, A Y; Ware, R; O'Donohoe, T J; Faggion, C M

2017-10-01

The objective of this study was to assess methodological and reporting quality of systematic reviews in hand and wrist pathology. MEDLINE, EMBASE and Cochrane Library were searched from inception to November 2016 for relevant studies. Reporting quality was evaluated using Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and methodological quality using a measurement tool to assess systematic reviews, the Assessment of Multiple Systematic Reviews (AMSTAR). Descriptive statistics and linear regression were used to identify features associated with improved methodological quality. A total of 91 studies were included in the analysis. Most reviews inadequately reported PRISMA items regarding study protocol, search strategy and bias and AMSTAR items regarding protocol, publication bias and funding. Systematic reviews published in a plastics journal, or which included more authors, were associated with higher AMSTAR scores. A large proportion of systematic reviews within hand and wrist pathology literature score poorly with validated methodological assessment tools, which may affect the reliability of their conclusions. I.

The landscape of systematic reviews in urology (1998 to 2015): an assessment of methodological quality.

PubMed

Han, Julia L; Gandhi, Shreyas; Bockoven, Crystal G; Narayan, Vikram M; Dahm, Philipp

2017-04-01

To assess the quality of published systematic reviews in the urology literature (an extension of our previously reported work), as high-quality systematic reviews play a paramount role in informing evidence-based clinical practice. Our focus was on systematic reviews in the urology literature that incorporated questions of prevention and therapy. To identify such reviews published during a 36-month period (2013-2015), we systematically searched PubMed and hand-searched the table of contents of four major urology journals. Two reviewers independently assessed the methodological quality of those reviews, using the 11-point 'Assessment of Multiple Systematic Reviews' (AMSTAR) instrument. We performed protocol-driven analyses of the data from our present study's 36-month period alone, as well as in aggregate with the data from our previously reported work's study periods (2009-2012 and 1998-2008). In our literature search of the 36-month period (2013-2015), we initially identified 490 possibly relevant reviews, of which 125 met our inclusion criteria. The most common topic of reviews for the 2013-2015 period was oncology (51.2%; n = 64), followed by voiding dysfunction (21.6%; n = 27). The mean [standard deviation (SD)] AMSTAR score in the 2013-2015 period (n = 125) was 4.8 (2.4); 2009-2012 (n = 113), 5.4 (2.3); and 1998-2008 (n = 57), 4.8 (2.0) (P = 0.127). In the 2013-2015 period, the mean (SD) AMSTAR score for the BJU International (n = 25) was 5.6 (2.9); for The Journal of Urology (n = 20), 5.1 (2.6); for European Urology (n = 60), 4.5 (2.2); and for Urology (n = 20), 4.4 (2.2) (P = 0.106). The number of systematic reviews published in the urology literature has exponentially increased, year by year, but their methodological quality has stagnated. To enhance the validity and impact of systematic reviews, all authors and editors must apply established methodological standards. © 2016 The Authors BJU International © 2016 BJU International Published by John Wiley

Does updating improve the methodological and reporting quality of systematic reviews?

PubMed

Shea, Beverley; Boers, Maarten; Grimshaw, Jeremy M; Hamel, Candyce; Bouter, Lex M

2006-06-13

Systematic reviews (SRs) must be of high quality. The purpose of our research was to compare the methodological and reporting quality of original versus updated Cochrane SRs to determine whether updating had improved these two quality dimensions. We identified updated Cochrane SRs published in issue 4, 2002 of the Cochrane Library. We assessed the updated and original versions of the SRs using two instruments: the 10 item enhanced Overview Quality Assessment Questionnaire (OQAQ), and an 18-item reporting quality checklist and flow chart based upon the Quality of Reporting of Meta-analyses (QUOROM) statement. At least two reviewers extracted data and assessed quality. We calculated the percentage (with a 95% confidence interval) of 'yes' answers to each question. We calculated mean differences in percentage, 95% confidence intervals and p-values for each of the individual items and the overall methodological quality score of the updated and pre-updated versions using OQAQ. We assessed 53 SRs. There was no significant improvement in the global quality score of the OQAQ (mean difference 0.11 (-0.28; 0.70 p = 0.52)). Updated reviews showed a significant improvement of 18.9 (7.2; 30.6 p < .01) on the OQAQ item assessing whether the conclusions drawn by the author(s) were supported by the data and/or analysis presented in the SR. The QUOROM statement showed that the quality of reporting of Cochrane reviews improved in some areas with updating. Improvements were seen on the items relating to data sources reported in the abstract, with a significant difference of 17.0 (9.8; 28.7 p = 0.01), review methods, reported in the abstract 35 (24.1; 49.1 p = 0.00), searching methods 18.9 (9.7; 31.6 p = 0.01), and data abstraction 18.9 (11.7; 30.9 p = 0.00). The overall quality of Cochrane SRs is fair-to-good. Although reporting quality improved on certain individual items there was no overall improvement seen with updating and methodological quality remained unchanged. Further

Indirect Comparisons: A Review of Reporting and Methodological Quality

PubMed Central

Donegan, Sarah; Williamson, Paula; Gamble, Carrol; Tudur-Smith, Catrin

2010-01-01

Background The indirect comparison of two interventions can be valuable in many situations. However, the quality of an indirect comparison will depend on several factors including the chosen methodology and validity of underlying assumptions. Published indirect comparisons are increasingly more common in the medical literature, but as yet, there are no published recommendations of how they should be reported. Our aim is to systematically review the quality of published indirect comparisons to add to existing empirical data suggesting that improvements can be made when reporting and applying indirect comparisons. Methodology/Findings Reviews applying statistical methods to indirectly compare the clinical effectiveness of two interventions using randomised controlled trials were eligible. We searched (1966–2008) Database of Abstracts and Reviews of Effects, The Cochrane library, and Medline. Full review publications were assessed for eligibility. Specific criteria to assess quality were developed and applied. Forty-three reviews were included. Adequate methodology was used to calculate the indirect comparison in 41 reviews. Nineteen reviews assessed the similarity assumption using sensitivity analysis, subgroup analysis, or meta-regression. Eleven reviews compared trial-level characteristics. Twenty-four reviews assessed statistical homogeneity. Twelve reviews investigated causes of heterogeneity. Seventeen reviews included direct and indirect evidence for the same comparison; six reviews assessed consistency. One review combined both evidence types. Twenty-five reviews urged caution in interpretation of results, and 24 reviews indicated when results were from indirect evidence by stating this term with the result. Conclusions This review shows that the underlying assumptions are not routinely explored or reported when undertaking indirect comparisons. We recommend, therefore, that the quality of indirect comparisons should be improved, in particular, by assessing

Vending machine assessment methodology. A systematic review.

PubMed

Matthews, Melissa A; Horacek, Tanya M

2015-07-01

The nutritional quality of food and beverage products sold in vending machines has been implicated as a contributing factor to the development of an obesogenic food environment. How comprehensive, reliable, and valid are the current assessment tools for vending machines to support or refute these claims? A systematic review was conducted to summarize, compare, and evaluate the current methodologies and available tools for vending machine assessment. A total of 24 relevant research studies published between 1981 and 2013 met inclusion criteria for this review. The methodological variables reviewed in this study include assessment tool type, study location, machine accessibility, product availability, healthfulness criteria, portion size, price, product promotion, and quality of scientific practice. There were wide variations in the depth of the assessment methodologies and product healthfulness criteria utilized among the reviewed studies. Of the reviewed studies, 39% evaluated machine accessibility, 91% evaluated product availability, 96% established healthfulness criteria, 70% evaluated portion size, 48% evaluated price, 52% evaluated product promotion, and 22% evaluated the quality of scientific practice. Of all reviewed articles, 87% reached conclusions that provided insight into the healthfulness of vended products and/or vending environment. Product healthfulness criteria and complexity for snack and beverage products was also found to be variable between the reviewed studies. These findings make it difficult to compare results between studies. A universal, valid, and reliable vending machine assessment tool that is comprehensive yet user-friendly is recommended. Copyright © 2015 Elsevier Ltd. All rights reserved.

Panning for the gold in health research: incorporating studies' methodological quality in meta-analysis.

PubMed

Johnson, Blair T; Low, Robert E; MacDonald, Hayley V

2015-01-01

Systematic reviews now routinely assess methodological quality to gauge the validity of the included studies and of the synthesis as a whole. Although trends from higher quality studies should be clearer, it is uncertain how often meta-analyses incorporate methodological quality in models of study results either as predictors, or, more interestingly, in interactions with theoretical moderators. We survey 200 meta-analyses in three health promotion domains to examine when and how meta-analyses incorporate methodological quality. Although methodological quality assessments commonly appear in contemporary meta-analyses (usually as scales), they are rarely incorporated in analyses, and still more rarely analysed in interaction with theoretical determinants of the success of health promotions. The few meta-analyses (2.5%) that did include such an interaction analysis showed that moderator results remained significant in higher quality studies or were present only among higher quality studies. We describe how to model quality interactively with theoretically derived moderators and discuss strengths and weaknesses of this approach and in relation to current meta-analytic practice. In large literatures exhibiting heterogeneous effects, meta-analyses can incorporate methodological quality and generate conclusions that enable greater confidence not only about the substantive phenomenon but also about the role that methodological quality itself plays.

Systematic Reviews and Meta-Analyses of Home Telemonitoring Interventions for Patients With Chronic Diseases: A Critical Assessment of Their Methodological Quality

PubMed Central

2013-01-01

Background Systematic reviews and meta-analyses of home telemonitoring interventions for patients with chronic diseases have increased over the past decade and become increasingly important to a wide range of clinicians, policy makers, and other health care stakeholders. While a few criticisms about their methodological rigor and synthesis approaches have recently appeared, no formal appraisal of their quality has been conducted yet. Objective The primary aim of this critical review was to evaluate the methodology, quality, and reporting characteristics of prior reviews that have investigated the effects of home telemonitoring interventions in the context of chronic diseases. Methods Ovid MEDLINE, the Database of Abstract of Reviews of Effects (DARE), and Health Technology Assessment Database (HTA) of the Cochrane Library were electronically searched to find relevant systematic reviews, published between January 1966 and December 2012. Potential reviews were screened and assessed for inclusion independently by three reviewers. Data pertaining to the methods used were extracted from each included review and examined for accuracy by two reviewers. A validated quality assessment instrument, R-AMSTAR, was used as a framework to guide the assessment process. Results Twenty-four reviews, nine of which were meta-analyses, were identified from more than 200 citations. The bibliographic search revealed that the number of published reviews has increased substantially over the years in this area and although most reviews focus on studying the effects of home telemonitoring on patients with congestive heart failure, researcher interest has extended to other chronic diseases as well, such as diabetes, hypertension, chronic obstructive pulmonary disease, and asthma. Nevertheless, an important number of these reviews appear to lack optimal scientific rigor due to intrinsic methodological issues. Also, the overall quality of reviews does not appear to have improved over time

Quality assessment of urban environment

NASA Astrophysics Data System (ADS)

Ovsiannikova, T. Y.; Nikolaenko, M. N.

2015-01-01

This paper is dedicated to the research applicability of quality management problems of construction products. It is offered to expand quality management borders in construction, transferring its principles to urban systems as economic systems of higher level, which qualitative characteristics are substantially defined by quality of construction product. Buildings and structures form spatial-material basis of cities and the most important component of life sphere - urban environment. Authors justify the need for the assessment of urban environment quality as an important factor of social welfare and life quality in urban areas. The authors suggest definition of a term "urban environment". The methodology of quality assessment of urban environment is based on integrated approach which includes the system analysis of all factors and application of both quantitative methods of assessment (calculation of particular and integrated indicators) and qualitative methods (expert estimates and surveys). The authors propose the system of indicators, characterizing quality of the urban environment. This indicators fall into four classes. The authors show the methodology of their definition. The paper presents results of quality assessment of urban environment for several Siberian regions and comparative analysis of these results.

Methodological quality and descriptive characteristics of prosthodontic-related systematic reviews.

PubMed

Aziz, T; Compton, S; Nassar, U; Matthews, D; Ansari, K; Flores-Mir, C

2013-04-01

Ideally, healthcare systematic reviews (SRs) should be beneficial to practicing professionals in making evidence-based clinical decisions. However, the conclusions drawn from SRs are directly related to the quality of the SR and of the included studies. The aim was to investigate the methodological quality and key descriptive characteristics of SRs published in prosthodontics. Methodological quality was analysed using the Assessment of Multiple Reviews (AMSTAR) tool. Several electronic resources (MEDLINE, EMBASE, Web of Science and American Dental Association's Evidence-based Dentistry website) were searched. In total 106 SRs were located. Key descriptive characteristics and methodological quality features were gathered and assessed, and descriptive and inferential statistical testing performed. Most SRs in this sample originated from the European continent followed by North America. Two to five authors conducted most SRs; the majority was affiliated with academic institutions and had prior experience publishing SRs. The majority of SRs were published in specialty dentistry journals, with implant or implant-related topics, the primary topics of interest for most. According to AMSTAR, most quality aspects were adequately fulfilled by less than half of the reviews. Publication bias and grey literature searches were the most poorly adhered components. Overall, the methodological quality of the prosthodontic-related systematic was deemed limited. Future recommendations would include authors to have prior training in conducting SRs and for journals to include a universal checklist that should be adhered to address all key characteristics of an unbiased SR process. © 2013 Blackwell Publishing Ltd.

Tracer methodology: an appropriate tool for assessing compliance with accreditation standards?

PubMed

Bouchard, Chantal; Jean, Olivier

2017-10-01

Tracer methodology has been used by Accreditation Canada since 2008 to collect evidence on the quality and safety of care and services, and to assess compliance with accreditation standards. Given the importance of this methodology in the accreditation program, the objective of this study is to assess the quality of the methodology and identify its strengths and weaknesses. A mixed quantitative and qualitative approach was adopted to evaluate consistency, appropriateness, effectiveness and stakeholder synergy in applying the methodology. An online questionnaire was sent to 468 Accreditation Canada surveyors. According to surveyors' perceptions, tracer methodology is an effective tool for collecting useful, credible and reliable information to assess compliance with Qmentum program standards and priority processes. The results show good coherence between methodology components (appropriateness of the priority processes evaluated, activities to evaluate a tracer, etc.). The main weaknesses are the time constraints faced by surveyors and management's lack of cooperation during the evaluation of tracers. The inadequate amount of time allowed for the methodology to be applied properly raises questions about the quality of the information obtained. This study paves the way for a future, more in-depth exploration of the identified weaknesses to help the accreditation organization make more targeted improvements to the methodology. Copyright © 2016 John Wiley & Sons, Ltd. Copyright © 2016 John Wiley & Sons, Ltd.

Development and Field Test of an Audit Tool and Tracer Methodology for Clinician Assessment of Quality in End-of-Life Care.

PubMed

Bookbinder, Marilyn; Hugodot, Amandine; Freeman, Katherine; Homel, Peter; Santiago, Elisabeth; Riggs, Alexa; Gavin, Maggie; Chu, Alice; Brady, Ellen; Lesage, Pauline; Portenoy, Russell K

2018-02-01

Quality improvement in end-of-life care generally acquires data from charts or caregivers. "Tracer" methodology, which assesses real-time information from multiple sources, may provide complementary information. The objective of this study was to develop a valid brief audit tool that can guide assessment and rate care when used in a clinician tracer to evaluate the quality of care for the dying patient. To identify items for a brief audit tool, 248 items were created to evaluate overall quality, quality in specific content areas (e.g., symptom management), and specific practices. Collected into three instruments, these items were used to interview professional caregivers and evaluate the charts of hospitalized patients who died. Evidence that this information could be validly captured using a small number of items was obtained through factor analyses, canonical correlations, and group comparisons. A nurse manager field tested tracer methodology using candidate items to evaluate the care provided to other patients who died. The survey of 145 deaths provided chart data and data from 445 interviews (26 physicians, 108 nurses, 18 social workers, and nine chaplains). The analyses yielded evidence of construct validity for a small number of items, demonstrating significant correlations between these items and content areas identified as latent variables in factor analyses. Criterion validity was suggested by significant differences in the ratings on these items between the palliative care unit and other units. The field test evaluated 127 deaths, demonstrated the feasibility of tracer methodology, and informed reworking of the candidate items into the 14-item Tracer EoLC v1. The Tracer EoLC v1 can be used with tracer methodology to guide the assessment and rate the quality of end-of-life care. Copyright © 2017 American Academy of Hospice and Palliative Medicine. Published by Elsevier Inc. All rights reserved.

A new approach to subjectively assess quality of plenoptic content

NASA Astrophysics Data System (ADS)

Viola, Irene; Řeřábek, Martin; Ebrahimi, Touradj

2016-09-01

Plenoptic content is becoming increasingly popular thanks to the availability of acquisition and display devices. Thanks to image-based rendering techniques, a plenoptic content can be rendered in real time in an interactive manner allowing virtual navigation through the captured scenes. This way of content consumption enables new experiences, and therefore introduces several challenges in terms of plenoptic data processing, transmission and consequently visual quality evaluation. In this paper, we propose a new methodology to subjectively assess the visual quality of plenoptic content. We also introduce a prototype software to perform subjective quality assessment according to the proposed methodology. The proposed methodology is further applied to assess the visual quality of a light field compression algorithm. Results show that this methodology can be successfully used to assess the visual quality of plenoptic content.

The methodological quality assessment tools for preclinical and clinical studies, systematic review and meta-analysis, and clinical practice guideline: a systematic review.

PubMed

Zeng, Xiantao; Zhang, Yonggang; Kwong, Joey S W; Zhang, Chao; Li, Sheng; Sun, Feng; Niu, Yuming; Du, Liang

2015-02-01

To systematically review the methodological assessment tools for pre-clinical and clinical studies, systematic review and meta-analysis, and clinical practice guideline. We searched PubMed, the Cochrane Handbook for Systematic Reviews of Interventions, Joanna Briggs Institute (JBI) Reviewers Manual, Centre for Reviews and Dissemination, Critical Appraisal Skills Programme (CASP), Scottish Intercollegiate Guidelines Network (SIGN), and the National Institute for Clinical Excellence (NICE) up to May 20th, 2014. Two authors selected studies and extracted data; quantitative analysis was performed to summarize the characteristics of included tools. We included a total of 21 assessment tools for analysis. A number of tools were developed by academic organizations, and some were developed by only a small group of researchers. The JBI developed the highest number of methodological assessment tools, with CASP coming second. Tools for assessing the methodological quality of randomized controlled studies were most abundant. The Cochrane Collaboration's tool for assessing risk of bias is the best available tool for assessing RCTs. For cohort and case-control studies, we recommend the use of the Newcastle-Ottawa Scale. The Methodological Index for Non-Randomized Studies (MINORS) is an excellent tool for assessing non-randomized interventional studies, and the Agency for Healthcare Research and Quality (ARHQ) methodology checklist is applicable for cross-sectional studies. For diagnostic accuracy test studies, the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tool is recommended; the SYstematic Review Centre for Laboratory animal Experimentation (SYRCLE) risk of bias tool is available for assessing animal studies; Assessment of Multiple Systematic Reviews (AMSTAR) is a measurement tool for systematic reviews/meta-analyses; an 18-item tool has been developed for appraising case series studies, and the Appraisal of Guidelines, Research and Evaluation (AGREE

Assessing the Methodological Quality of Glaucoma Clinical Practice Guidelines and Their Recommendations on Microinvasive Glaucoma Surgery: A Systematic Review.

PubMed

Michaelov, Evan; Armstrong, James J; Nguyen, Mary; Instrum, Bridget; Lam, Tracey; Denstedt, James; Hutnik, Cindy M L

2018-02-01

Clinical practice guidelines (CPG) are regarded by many as critical communications providing guidance within specific medical fields. Over a decade ago, the first microinvasive glaucoma surgical (MIGS) procedures were introduced. Since then, a number of these novel intraocular pressure controlling surgical options have been approved worldwide. Governing bodies and health care administration often utilize CPGs when considering funding for newer technologies. This highlights the importance of well-written, accurate, and up-to-date CPGs in the rapidly evolving field of MIGS. If CPGs are unable to fill this role, their use in treatment decision-making is doing a disservice to patients, who will be denied currently available and potentially superior care. To determine the overall value of a CPG, the methodological quality with which it was developed, in addition to the current relevance and appropriateness of its recommendations, should be evaluated. The objective of the present study was to assess the methodological quality of currently available international glaucoma CPGs, as well as their coverage of MIGS as a surrogate marker of relevance and appropriateness to policy-makers and ophthalmologists alike. To identify potentially relevant CPGs, a predefined search strategy was used to search the following databases: Medline, EMBASE, BIOSIS, and Web of Science. All CPGs related to adult glaucoma and published in English were included. CPG methodological quality was assessed by 3 individuals using the Appraisal of Guidelines for Research and Evaluation II (AGREE II) tool. Studies were then assessed for coverage of MIGS devices and procedures. Search strategy and subsequent screening identified 11 CPGs for analysis. Eight were of high quality according to the AGREE II criteria. Three included basic information on MIGS, but none provided specific recommendations regarding their indications or which patient populations would benefit most. Many international glaucoma CPGs

The methodological and reporting quality of systematic reviews from China and the USA are similar.

PubMed

Tian, Jinhui; Zhang, Jun; Ge, Long; Yang, Kehu; Song, Fujian

2017-05-01

To compare the methodological and reporting quality of systematic reviews by authors from China and those from the United States (USA). From systematic reviews of randomized trials published in 2014 in English, we randomly selected 100 from China and 100 from the USA. The methodological quality was assessed using the Assessing the Methodological Quality of Systematic Reviews (AMSTAR) tool, and reporting quality assessed using the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) tool. Compared with systematic reviews from the USA, those from China were more likely to be a meta-analysis, published in low-impact journals, and a non-Cochrane review. The mean summary Assessing the Methodological Quality of Systematic Reviews score was 6.7 (95% confidence interval: 6.5, 7.0) for reviews from China and 6.6 (6.1, 7.1) for reviews from the USA, and the mean summary Preferred Reporting Items for Systematic Reviews and Meta-analyses score was 21.2 (20.7, 21.6) for reviews from China and 20.6 (19.9, 21.3) for reviews from the USA. The differences in summary quality scores between China and the USA were statistically nonsignificant after adjusting for multiple review factors. The overall methodological and reporting quality of systematic reviews by authors from China are similar to those from the USA, although the quality of systematic reviews from both countries could be further improved. Copyright © 2017 The Author(s). Published by Elsevier Inc. All rights reserved.

Most systematic reviews of high methodological quality on psoriasis interventions are classified as high risk of bias using ROBIS tool.

PubMed

Gómez-García, Francisco; Ruano, Juan; Gay-Mimbrera, Jesus; Aguilar-Luque, Macarena; Sanz-Cabanillas, Juan Luis; Alcalde-Mellado, Patricia; Maestre-López, Beatriz; Carmona-Fernández, Pedro Jesús; González-Padilla, Marcelino; García-Nieto, Antonio Vélez; Isla-Tejera, Beatriz

2017-12-01

No gold standard exists to assess methodological quality of systematic reviews (SRs). Although Assessing the Methodological Quality of Systematic Reviews (AMSTAR) is widely accepted for analyzing quality, the ROBIS instrument has recently been developed. This study aimed to compare the capacity of both instruments to capture the quality of SRs concerning psoriasis interventions. Systematic literature searches were undertaken on relevant databases. For each review, methodological quality and bias risk were evaluated using the AMSTAR and ROBIS tools. Descriptive and principal component analyses were conducted to describe similarities and discrepancies between both assessment tools. We classified 139 intervention SRs as displaying high/moderate/low methodological quality and as high/low risk of bias. A high risk of bias was detected for most SRs classified as displaying high or moderate methodological quality by AMSTAR. When comparing ROBIS result profiles, responses to domain 4 signaling questions showed the greatest differences between bias risk assessments, whereas domain 2 items showed the least. When considering SRs published about psoriasis, methodological quality remains suboptimal, and the risk of bias is elevated, even for SRs exhibiting high methodological quality. Furthermore, the AMSTAR and ROBIS tools may be considered as complementary when conducting quality assessment of SRs. Copyright © 2017 Elsevier Inc. All rights reserved.

Quality and methodological challenges in Internet-based mental health trials.

PubMed

Ye, Xibiao; Bapuji, Sunita Bayyavarapu; Winters, Shannon; Metge, Colleen; Raynard, Mellissa

2014-08-01

To review the quality of Internet-based mental health intervention studies and their methodological challenges. We searched multiple literature databases to identify relevant studies according to the Population, Interventions, Comparators, Outcomes, and Study Design framework. Two reviewers independently assessed selection bias, allocation bias, confounding bias, blinding, data collection methods, and withdrawals/dropouts, using the Quality Assessment Tool for Quantitative Studies. We rated each component as strong, moderate, or weak and assigned a global rating (strong, moderate, or weak) to each study. We discussed methodological issues related to the study quality. Of 122 studies included, 31 (25%), 44 (36%), and 47 (39%) were rated strong, moderate, and weak, respectively. Only five studies were rated strong for all of the six quality components (three of them were published by the same group). Lack of blinding, selection bias, and low adherence were the top three challenges in Internet-based mental health intervention studies. The overall quality of Internet-based mental health intervention needs to improve. In particular, studies need to improve sample selection, intervention allocation, and blinding.

Reporting and methodological quality of meta-analyses in urological literature.

PubMed

Xia, Leilei; Xu, Jing; Guzzo, Thomas J

2017-01-01

To assess the overall quality of published urological meta-analyses and identify predictive factors for high quality. We systematically searched PubMed to identify meta-analyses published from January 1st, 2011 to December 31st, 2015 in 10 predetermined major paper-based urology journals. The characteristics of the included meta-analyses were collected, and their reporting and methodological qualities were assessed by the PRISMA checklist (27 items) and AMSTAR tool (11 items), respectively. Descriptive statistics were used for individual items as a measure of overall compliance, and PRISMA and AMSTAR scores were calculated as the sum of adequately reported domains. Logistic regression was used to identify predictive factors for high qualities. A total of 183 meta-analyses were included. The mean PRISMA and AMSTAR scores were 22.74 ± 2.04 and 7.57 ± 1.41, respectively. PRISMA item 5, protocol and registration, items 15 and 22, risk of bias across studies, items 16 and 23, additional analysis had less than 50% adherence. AMSTAR item 1, " a priori " design, item 5, list of studies and item 10, publication bias had less than 50% adherence. Logistic regression analyses showed that funding support and " a priori " design were associated with superior reporting quality, following PRISMA guideline and " a priori " design were associated with superior methodological quality. Reporting and methodological qualities of recently published meta-analyses in major paper-based urology journals are generally good. Further improvement could potentially be achieved by strictly adhering to PRISMA guideline and having " a priori " protocol.

Comparison of methodological quality rating of systematic reviews on neuropathic pain using AMSTAR and R-AMSTAR.

PubMed

Dosenovic, Svjetlana; Jelicic Kadic, Antonia; Vucic, Katarina; Markovina, Nikolina; Pieper, Dawid; Puljak, Livia

2018-05-08

Systematic reviews (SRs) in the field of neuropathic pain (NeuP) are increasingly important for decision-making. However, methodological flaws in SRs can reduce the validity of conclusions. Hence, it is important to assess the methodological quality of NeuP SRs critically. Additionally, it remains unclear which assessment tool should be used. We studied the methodological quality of SRs published in the field of NeuP and compared two assessment tools. We systematically searched 5 electronic databases to identify SRs of randomized controlled trials of interventions for NeuP available up to March 2015. Two independent reviewers assessed the methodological quality of the studies using the Assessment of Multiple Systematic Reviews (AMSTAR) and the revised AMSTAR (R-AMSTAR) tools. The scores were converted to percentiles and ranked into 4 grades to allow comparison between the two checklists. Gwet's AC1 coefficient was used for interrater reliability assessment. The 97 included SRs had a wide range of methodological quality scores (AMSTAR median (IQR): 6 (5-8) vs. R-AMSTAR median (IQR): 30 (26-35)). The overall agreement score between the 2 raters was 0.62 (95% CI 0.39-0.86) for AMSTAR and 0.62 (95% CI 0.53-0.70) for R-AMSTAR. The 31 Cochrane systematic reviews (CSRs) were consistently ranked higher than the 66 non-Cochrane systematic reviews (NCSRs). The analysis of individual domains showed the best compliance in a comprehensive literature search (item 3) on both checklists. The results for the domain that was the least compliant differed: conflict of interest (item 11) was the item most poorly reported on AMSTAR vs. publication bias assessment (item 10) on R-AMSTAR. A high positive correlation between the total AMSTAR and R-AMSTAR scores for all SRs, as well as for CSRs and NCSRs, was observed. The methodological quality of analyzed SRs in the field of NeuP was not optimal, and CSRs had a higher quality than NCSRs. Both AMSTAR and R-AMSTAR tools produced comparable

The Development of a Checklist to Enhance Methodological Quality in Intervention Programs.

PubMed

Chacón-Moscoso, Salvador; Sanduvete-Chaves, Susana; Sánchez-Martín, Milagrosa

2016-01-01

The methodological quality of primary studies is an important issue when performing meta-analyses or systematic reviews. Nevertheless, there are no clear criteria for how methodological quality should be analyzed. Controversies emerge when considering the various theoretical and empirical definitions, especially in relation to three interrelated problems: the lack of representativeness, utility, and feasibility. In this article, we (a) systematize and summarize the available literature about methodological quality in primary studies; (b) propose a specific, parsimonious, 12-items checklist to empirically define the methodological quality of primary studies based on a content validity study; and (c) present an inter-coder reliability study for the resulting 12-items. This paper provides a precise and rigorous description of the development of this checklist, highlighting the clearly specified criteria for the inclusion of items and a substantial inter-coder agreement in the different items. Rather than simply proposing another checklist, however, it then argues that the list constitutes an assessment tool with respect to the representativeness, utility, and feasibility of the most frequent methodological quality items in the literature, one that provides practitioners and researchers with clear criteria for choosing items that may be adequate to their needs. We propose individual methodological features as indicators of quality, arguing that these need to be taken into account when designing, implementing, or evaluating an intervention program. This enhances methodological quality of intervention programs and fosters the cumulative knowledge based on meta-analyses of these interventions. Future development of the checklist is discussed.

The exchangeability of self-reports and administrative health care resource use measurements: assessement of the methodological reporting quality.

PubMed

Noben, Cindy Yvonne; de Rijk, Angelique; Nijhuis, Frans; Kottner, Jan; Evers, Silvia

2016-06-01

To assess the exchangeability of self-reported and administrative health care resource use measurements for cost estimation. In a systematic review (NHS EED and MEDLINE), reviewers evaluate, in duplicate, the methodological reporting quality of studies comparing the validation evidence of instruments measuring health care resource use. The appraisal tool Methodological Reporting Quality (MeRQ) is developed by merging aspects form the Guidelines for Reporting Reliability and Agreement Studies and the Standards for Reporting Diagnostic Accuracy. Out of 173 studies, 35 full-text articles are assessed for eligibility. Sixteen articles are included in this study. In seven articles, more than 75% of the reporting criteria assessed by MERQ are considered "good." Most studies score at least "fair" on most of the reporting quality criteria. In the end, six studies score "good" on the minimal criteria for reporting. Varying levels of agreement among the different data sources are found, with correlations ranging from 0.14 up to 0.93 and with occurrences of both random and systematic errors. The validation evidence of the small number of studies with adequate MeRQ cautiously supports the exchangeability of both the self-reported and administrative resource use measurement methods. Copyright © 2016 Elsevier Inc. All rights reserved.

Comparison of tools for assessing the methodological quality of primary and secondary studies in health technology assessment reports in Germany.

PubMed

Dreier, Maren; Borutta, Birgit; Stahmeyer, Jona; Krauth, Christian; Walter, Ulla

2010-06-14

HEALTH CARE POLICY BACKGROUND: Findings from scientific studies form the basis for evidence-based health policy decisions. Quality assessments to evaluate the credibility of study results are an essential part of health technology assessment reports and systematic reviews. Quality assessment tools (QAT) for assessing the study quality examine to what extent study results are systematically distorted by confounding or bias (internal validity). The tools can be divided into checklists, scales and component ratings. What QAT are available to assess the quality of interventional studies or studies in the field of health economics, how do they differ from each other and what conclusions can be drawn from these results for quality assessments? A systematic search of relevant databases from 1988 onwards is done, supplemented by screening of the references, of the HTA reports of the German Agency for Health Technology Assessment (DAHTA) and an internet search. The selection of relevant literature, the data extraction and the quality assessment are carried out by two independent reviewers. The substantive elements of the QAT are extracted using a modified criteria list consisting of items and domains specific to randomized trials, observational studies, diagnostic studies, systematic reviews and health economic studies. Based on the number of covered items and domains, more and less comprehensive QAT are distinguished. In order to exchange experiences regarding problems in the practical application of tools, a workshop is hosted. A total of eight systematic methodological reviews is identified as well as 147 QAT: 15 for systematic reviews, 80 for randomized trials, 30 for observational studies, 17 for diagnostic studies and 22 for health economic studies. The tools vary considerably with regard to the content, the performance and quality of operationalisation. Some tools do not only include the items of internal validity but also the items of quality of reporting and

Reporting and methodological quality of meta-analyses in urological literature

PubMed Central

Xu, Jing

2017-01-01

Purpose To assess the overall quality of published urological meta-analyses and identify predictive factors for high quality. Materials and Methods We systematically searched PubMed to identify meta-analyses published from January 1st, 2011 to December 31st, 2015 in 10 predetermined major paper-based urology journals. The characteristics of the included meta-analyses were collected, and their reporting and methodological qualities were assessed by the PRISMA checklist (27 items) and AMSTAR tool (11 items), respectively. Descriptive statistics were used for individual items as a measure of overall compliance, and PRISMA and AMSTAR scores were calculated as the sum of adequately reported domains. Logistic regression was used to identify predictive factors for high qualities. Results A total of 183 meta-analyses were included. The mean PRISMA and AMSTAR scores were 22.74 ± 2.04 and 7.57 ± 1.41, respectively. PRISMA item 5, protocol and registration, items 15 and 22, risk of bias across studies, items 16 and 23, additional analysis had less than 50% adherence. AMSTAR item 1, “a priori” design, item 5, list of studies and item 10, publication bias had less than 50% adherence. Logistic regression analyses showed that funding support and “a priori” design were associated with superior reporting quality, following PRISMA guideline and “a priori” design were associated with superior methodological quality. Conclusions Reporting and methodological qualities of recently published meta-analyses in major paper-based urology journals are generally good. Further improvement could potentially be achieved by strictly adhering to PRISMA guideline and having “a priori” protocol. PMID:28439452

Development of Management Methodology for Engineering Production Quality

NASA Astrophysics Data System (ADS)

Gorlenko, O.; Miroshnikov, V.; Borbatc, N.

2016-04-01

The authors of the paper propose four directions of the methodology developing the quality management of engineering products that implement the requirements of new international standard ISO 9001:2015: the analysis of arrangement context taking into account stakeholders, the use of risk management, management of in-house knowledge, assessment of the enterprise activity according to the criteria of effectiveness

Towards Quality Assessment in an EFL Programme

ERIC Educational Resources Information Center

Ali, Holi Ibrahim Holi; Al Ajmi, Ahmed Ali Saleh

2013-01-01

Assessment is central in education and the teaching-learning process. This study attempts to explore the perspectives and views about quality assessment among teachers of English as a Foreign Language (EFL), and to find ways of promoting quality assessment. Quantitative methodology was used to collect data. To answer the study questions, a…

The Development of a Checklist to Enhance Methodological Quality in Intervention Programs

PubMed Central

Chacón-Moscoso, Salvador; Sanduvete-Chaves, Susana; Sánchez-Martín, Milagrosa

2016-01-01

The methodological quality of primary studies is an important issue when performing meta-analyses or systematic reviews. Nevertheless, there are no clear criteria for how methodological quality should be analyzed. Controversies emerge when considering the various theoretical and empirical definitions, especially in relation to three interrelated problems: the lack of representativeness, utility, and feasibility. In this article, we (a) systematize and summarize the available literature about methodological quality in primary studies; (b) propose a specific, parsimonious, 12-items checklist to empirically define the methodological quality of primary studies based on a content validity study; and (c) present an inter-coder reliability study for the resulting 12-items. This paper provides a precise and rigorous description of the development of this checklist, highlighting the clearly specified criteria for the inclusion of items and a substantial inter-coder agreement in the different items. Rather than simply proposing another checklist, however, it then argues that the list constitutes an assessment tool with respect to the representativeness, utility, and feasibility of the most frequent methodological quality items in the literature, one that provides practitioners and researchers with clear criteria for choosing items that may be adequate to their needs. We propose individual methodological features as indicators of quality, arguing that these need to be taken into account when designing, implementing, or evaluating an intervention program. This enhances methodological quality of intervention programs and fosters the cumulative knowledge based on meta-analyses of these interventions. Future development of the checklist is discussed. PMID:27917143

Abstract analysis method facilitates filtering low-methodological quality and high-bias risk systematic reviews on psoriasis interventions.

PubMed

Gómez-García, Francisco; Ruano, Juan; Aguilar-Luque, Macarena; Alcalde-Mellado, Patricia; Gay-Mimbrera, Jesús; Hernández-Romero, José Luis; Sanz-Cabanillas, Juan Luis; Maestre-López, Beatriz; González-Padilla, Marcelino; Carmona-Fernández, Pedro J; García-Nieto, Antonio Vélez; Isla-Tejera, Beatriz

2017-12-29

Article summaries' information and structure may influence researchers/clinicians' decisions to conduct deeper full-text analyses. Specifically, abstracts of systematic reviews (SRs) and meta-analyses (MA) should provide structured summaries for quick assessment. This study explored a method for determining the methodological quality and bias risk of full-text reviews using abstract information alone. Systematic literature searches for SRs and/or MA about psoriasis were undertaken on MEDLINE, EMBASE, and Cochrane database. For each review, quality, abstract-reporting completeness, full-text methodological quality, and bias risk were evaluated using Preferred Reporting Items for Systematic Reviews and Meta-analyses for abstracts (PRISMA-A), Assessing the Methodological Quality of Systematic Reviews (AMSTAR), and ROBIS tools, respectively. Article-, author-, and journal-derived metadata were systematically extracted from eligible studies using a piloted template, and explanatory variables concerning abstract-reporting quality were assessed using univariate and multivariate-regression models. Two classification models concerning SRs' methodological quality and bias risk were developed based on per-item and total PRISMA-A scores and decision-tree algorithms. This work was supported, in part, by project ICI1400136 (JR). No funding was received from any pharmaceutical company. This study analysed 139 SRs on psoriasis interventions. On average, they featured 56.7% of PRISMA-A items. The mean total PRISMA-A score was significantly higher for high-methodological-quality SRs than for moderate- and low-methodological-quality reviews. SRs with low-bias risk showed higher total PRISMA-A values than reviews with high-bias risk. In the final model, only 'authors per review > 6' (OR: 1.098; 95%CI: 1.012-1.194), 'academic source of funding' (OR: 3.630; 95%CI: 1.788-7.542), and 'PRISMA-endorsed journal' (OR: 4.370; 95%CI: 1.785-10.98) predicted PRISMA-A variability. Reviews with a

Cumulative impact assessment: Application of a methodology

DOE Office of Scientific and Technical Information (OSTI.GOV)

Witmer, G.W.; Bain, M.B.; Irving, J.S.

We expanded upon the Federal Energy Regulatory Commission's (FERC) Cluster Impact Assessment Procedure (CIAP) to provide a practical methodology for assessing potential cumulative impacts from multiple hydroelectric projects within a river basin. The objectives in designing the methodology were to allow the evaluation of a large number of combinations of proposed projects and to minimize constraints on the use of ecological knowledge for planning and regulating hydroelectric development at the river basin level. Interactive workshops and evaluative matrices were used to identify preferred development scenarios in the Snohomish (Washington) and Salmon (Idaho) River Basins. Although the methodology achieved its basicmore » objectives, some difficulties were encountered. These revolved around issues of (1) data quality and quantity, (2) alternatives analysis, (3) determination of project interactions, (4) determination of cumulative impact thresholds, and (5) the use of evaluative techniques to express degrees of impact. 8 refs., 1 fig., 2 tabs.« less

Clinical practice guidelines and consensus statements in oncology--an assessment of their methodological quality.

PubMed

Jacobs, Carmel; Graham, Ian D; Makarski, Julie; Chassé, Michaël; Fergusson, Dean; Hutton, Brian; Clemons, Mark

2014-01-01

Consensus statements and clinical practice guidelines are widely available for enhancing the care of cancer patients. Despite subtle differences in their definition and purpose, these terms are often used interchangeably. We systematically assessed the methodological quality of consensus statements and clinical practice guidelines published in three commonly read, geographically diverse, cancer-specific journals. Methods Consensus statements and clinical practice guidelines published between January 2005 and September 2013 in Current Oncology, European Journal of Cancer and Journal of Clinical Oncology were evaluated. Each publication was assessed using the Appraisal of Guidelines for Research and Evaluation II (AGREE II) rigour of development and editorial independence domains. For assessment of transparency of document development, 7 additional items were taken from the Institute of Medicine's standards for practice guidelines and the Journal of Clinical Oncology guidelines for authors of guidance documents. Consensus statements and clinical practice guidelines published between January 2005 and September 2013 in Current Oncology, European Journal of Cancer and Journal of Clinical Oncology were evaluated. Each publication was assessed using the Appraisal of Guidelines for Research and Evaluation II (AGREE II) rigour of development and editorial independence domains. For assessment of transparency of document development, 7 additional items were taken from the Institute of Medicine's standards for practice guidelines and the Journal of Clinical Oncology guidelines for authors of guidance documents. Thirty-four consensus statements and 67 clinical practice guidelines were evaluated. The rigour of development score for consensus statements over the three journals was 32% lower than that of clinical practice guidelines. The editorial independence score was 15% lower for consensus statements than clinical practice guidelines. One journal scored consistently lower than

Author-paper affiliation network architecture influences the methodological quality of systematic reviews and meta-analyses of psoriasis.

PubMed

Sanz-Cabanillas, Juan Luis; Ruano, Juan; Gomez-Garcia, Francisco; Alcalde-Mellado, Patricia; Gay-Mimbrera, Jesus; Aguilar-Luque, Macarena; Maestre-Lopez, Beatriz; Gonzalez-Padilla, Marcelino; Carmona-Fernandez, Pedro J; Velez Garcia-Nieto, Antonio; Isla-Tejera, Beatriz

2017-01-01

Moderate-to-severe psoriasis is associated with significant comorbidity, an impaired quality of life, and increased medical costs, including those associated with treatments. Systematic reviews (SRs) and meta-analyses (MAs) of randomized clinical trials are considered two of the best approaches to the summarization of high-quality evidence. However, methodological bias can reduce the validity of conclusions from these types of studies and subsequently impair the quality of decision making. As co-authorship is among the most well-documented forms of research collaboration, the present study aimed to explore whether authors' collaboration methods might influence the methodological quality of SRs and MAs of psoriasis. Methodological quality was assessed by two raters who extracted information from full articles. After calculating total and per-item Assessment of Multiple Systematic Reviews (AMSTAR) scores, reviews were classified as low (0-4), medium (5-8), or high (9-11) quality. Article metadata and journal-related bibliometric indices were also obtained. A total of 741 authors from 520 different institutions and 32 countries published 220 reviews that were classified as high (17.2%), moderate (55%), or low (27.7%) methodological quality. The high methodological quality subnetwork was larger but had a lower connection density than the low and moderate methodological quality subnetworks; specifically, the former contained relatively fewer nodes (authors and reviews), reviews by authors, and collaborators per author. Furthermore, the high methodological quality subnetwork was highly compartmentalized, with several modules representing few poorly interconnected communities. In conclusion, structural differences in author-paper affiliation network may influence the methodological quality of SRs and MAs on psoriasis. As the author-paper affiliation network structure affects study quality in this research field, authors who maintain an appropriate balance between

Author-paper affiliation network architecture influences the methodological quality of systematic reviews and meta-analyses of psoriasis

PubMed Central

Gomez-Garcia, Francisco; Alcalde-Mellado, Patricia; Gay-Mimbrera, Jesus; Aguilar-Luque, Macarena; Maestre-Lopez, Beatriz; Gonzalez-Padilla, Marcelino; Carmona-Fernandez, Pedro J.; Velez Garcia-Nieto, Antonio; Isla-Tejera, Beatriz

2017-01-01

Moderate-to-severe psoriasis is associated with significant comorbidity, an impaired quality of life, and increased medical costs, including those associated with treatments. Systematic reviews (SRs) and meta-analyses (MAs) of randomized clinical trials are considered two of the best approaches to the summarization of high-quality evidence. However, methodological bias can reduce the validity of conclusions from these types of studies and subsequently impair the quality of decision making. As co-authorship is among the most well-documented forms of research collaboration, the present study aimed to explore whether authors’ collaboration methods might influence the methodological quality of SRs and MAs of psoriasis. Methodological quality was assessed by two raters who extracted information from full articles. After calculating total and per-item Assessment of Multiple Systematic Reviews (AMSTAR) scores, reviews were classified as low (0-4), medium (5-8), or high (9-11) quality. Article metadata and journal-related bibliometric indices were also obtained. A total of 741 authors from 520 different institutions and 32 countries published 220 reviews that were classified as high (17.2%), moderate (55%), or low (27.7%) methodological quality. The high methodological quality subnetwork was larger but had a lower connection density than the low and moderate methodological quality subnetworks; specifically, the former contained relatively fewer nodes (authors and reviews), reviews by authors, and collaborators per author. Furthermore, the high methodological quality subnetwork was highly compartmentalized, with several modules representing few poorly interconnected communities. In conclusion, structural differences in author-paper affiliation network may influence the methodological quality of SRs and MAs on psoriasis. As the author-paper affiliation network structure affects study quality in this research field, authors who maintain an appropriate balance between

[Methodological quality and reporting quality evaluation of randomized controlled trials published in China Journal of Chinese Materia Medica].

PubMed

Yu, Dan-Dan; Xie, Yan-Ming; Liao, Xing; Zhi, Ying-Jie; Jiang, Jun-Jie; Chen, Wei

2018-02-01

To evaluate the methodological quality and reporting quality of randomized controlled trials(RCTs) published in China Journal of Chinese Materia Medica, we searched CNKI and China Journal of Chinese Materia webpage to collect RCTs since the establishment of the magazine. The Cochrane risk of bias assessment tool was used to evaluate the methodological quality of RCTs. The CONSORT 2010 list was adopted as reporting quality evaluating tool. Finally, 184 RCTs were included and evaluated methodologically, of which 97 RCTs were evaluated with reporting quality. For the methodological evaluating, 62 trials(33.70%) reported the random sequence generation; 9(4.89%) trials reported the allocation concealment; 25(13.59%) trials adopted the method of blinding; 30(16.30%) trials reported the number of patients withdrawing, dropping out and those lost to follow-up;2 trials （1.09%） reported trial registration and none of the trial reported the trial protocol; only 8(4.35%) trials reported the sample size estimation in details. For reporting quality appraising, 3 reporting items of 25 items were evaluated with high-quality,including: abstract, participants qualified criteria, and statistical methods; 4 reporting items with medium-quality, including purpose, intervention, random sequence method, and data collection of sites and locations; 9 items with low-quality reporting items including title, backgrounds, random sequence types, allocation concealment, blindness, recruitment of subjects, baseline data, harms, and funding;the rest of items were of extremely low quality(the compliance rate of reporting item<10%). On the whole, the methodological and reporting quality of RCTs published in the magazine are generally low. Further improvement in both methodological and reporting quality for RCTs of traditional Chinese medicine are warranted. It is recommended that the international standards and procedures for RCT design should be strictly followed to conduct high-quality trials

Using Quality Tools and Methodologies to Improve a Hospital's Quality Position.

PubMed

Branco, Daniel; Wicks, Angela M; Visich, John K

2017-01-01

The authors identify the quality tools and methodologies most frequently used by quality-positioned hospitals versus nonquality hospitals. Northeastern U.S. hospitals in both groups received a brief, 12-question survey. The authors found that 93.75% of the quality hospitals and 81.25% of the nonquality hospitals used some form of process improvement methodologies. However, there were significant differences between the groups regarding the impact of quality improvement initiatives on patients. The findings indicate that in quality hospitals the use of quality improvement initiatives had a significantly greater positive impact on patient satisfaction and patient outcomes when compared to nonquality hospitals.

Association of journal quality indicators with methodological quality of clinical research articles.

PubMed

Lee, Kirby P; Schotland, Marieka; Bacchetti, Peter; Bero, Lisa A

2002-06-05

The ability to identify scientific journals that publish high-quality research would help clinicians, scientists, and health-policy analysts to select the most up-to-date medical literature to review. To assess whether journal characteristics of (1) peer-review status, (2) citation rate, (3) impact factor, (4) circulation, (5) manuscript acceptance rate, (6) MEDLINE indexing, and (7) Brandon/Hill Library List indexing are predictors of methodological quality of research articles, we conducted a cross-sectional study of 243 original research articles involving human subjects published in general internal medical journals. The mean (SD) quality score of the 243 articles was 1.37 (0.22). All journals reported a peer-review process and were indexed on MEDLINE. In models that controlled for article type (randomized controlled trial [RCT] or non-RCT), journal citation rate was the most statistically significant predictor (0.051 increase per doubling; 95% confidence interval [CI], 0.037-0.065; P<.001). In separate analyses by article type, acceptance rate was the strongest predictor for RCT quality (-0.113 per doubling; 95% CI, -0.148 to -0.078; P<.001), while journal citation rate was the most predictive factor for non-RCT quality (0.051 per doubling; 95% CI, 0.044-0.059; P<.001). High citation rates, impact factors, and circulation rates, and low manuscript acceptance rates and indexing on Brandon/Hill Library List appear to be predictive of higher methodological quality scores for journal articles.

Development of a Valid and Reliable Knee Articular Cartilage Condition-Specific Study Methodological Quality Score.

PubMed

Harris, Joshua D; Erickson, Brandon J; Cvetanovich, Gregory L; Abrams, Geoffrey D; McCormick, Frank M; Gupta, Anil K; Verma, Nikhil N; Bach, Bernard R; Cole, Brian J

2014-02-01

Condition-specific questionnaires are important components in evaluation of outcomes of surgical interventions. No condition-specific study methodological quality questionnaire exists for evaluation of outcomes of articular cartilage surgery in the knee. To develop a reliable and valid knee articular cartilage-specific study methodological quality questionnaire. Cross-sectional study. A stepwise, a priori-designed framework was created for development of a novel questionnaire. Relevant items to the topic were identified and extracted from a recent systematic review of 194 investigations of knee articular cartilage surgery. In addition, relevant items from existing generic study methodological quality questionnaires were identified. Items for a preliminary questionnaire were generated. Redundant and irrelevant items were eliminated, and acceptable items modified. The instrument was pretested and items weighed. The instrument, the MARK score (Methodological quality of ARticular cartilage studies of the Knee), was tested for validity (criterion validity) and reliability (inter- and intraobserver). A 19-item, 3-domain MARK score was developed. The 100-point scale score demonstrated face validity (focus group of 8 orthopaedic surgeons) and criterion validity (strong correlation to Cochrane Quality Assessment score and Modified Coleman Methodology Score). Interobserver reliability for the overall score was good (intraclass correlation coefficient [ICC], 0.842), and for all individual items of the MARK score, acceptable to perfect (ICC, 0.70-1.000). Intraobserver reliability ICC assessed over a 3-week interval was strong for 2 reviewers (≥0.90). The MARK score is a valid and reliable knee articular cartilage condition-specific study methodological quality instrument. This condition-specific questionnaire may be used to evaluate the quality of studies reporting outcomes of articular cartilage surgery in the knee.

A flood tide of systematic reviews on endodontic posts: methodological assessment using of R-AMSTAR.

PubMed

Schmitter, M; Sterzenbach, G; Faggion, C M; Krastl, G

2013-06-01

To help the dental practitioner solve a specific clinical problem, systematic reviews (SRs) are seen as the best guide. In addition to the unmanageable quantity of SRs, however, one should be aware of their variable quality. The present review describes the methodological quality of SRs on postendodontic restorations to work out the value of these reviews for the dental practitioner. SRs were searched in April 2012, independently and in triplicate. Post survival was used as measure of outcome. The methodological quality of included SRs was assessed with the Revised Assessment of Multiple Systematic Reviews (R-AMSTAR) checklist. Kappa statistics were used to assess reviewer agreement. Three hundred sixty-three papers were retrieved from the initial search. Ten SRs were included. One SR achieved a high R-AMSTAR score, whereas the other nine SRs achieved scores that indicate a substantial lack of methodological quality. Especially the items "grey literature," "combination of findings," "likelihood of publication bias," and conflict of interest" showed low R-AMSTAR scores. The three reviews with the highest R-AMSTAR scores tended to conclude that fewer failures occurred when using nonmetal posts. The reviewer agreement was excellent (kappa ranged from 0.79 to 0.85) in the R-AMSTAR classification. The approach presented revealed a lack of SRs with high methodological quality. Thus, no decisive conclusion can be drawn with respect to this topic. It appears that there is a trend for the superiority of fiber-reinforced posts. SRs must be of high methodological quality. This can be achieved by taking into consideration the results of this review. Improved methodological quality would make SRs more supportive for the general practitioner.

Methodological quality of behavioural weight loss studies: a systematic review

PubMed Central

Lemon, S. C.; Wang, M. L.; Haughton, C. F.; Estabrook, D. P.; Frisard, C. F.; Pagoto, S. L.

2018-01-01

Summary This systematic review assessed the methodological quality of behavioural weight loss intervention studies conducted among adults and associations between quality and statistically significant weight loss outcome, strength of intervention effectiveness and sample size. Searches for trials published between January, 2009 and December, 2014 were conducted using PUBMED, MEDLINE and PSYCINFO and identified ninety studies. Methodological quality indicators included study design, anthropometric measurement approach, sample size calculations, intent-to-treat (ITT) analysis, loss to follow-up rate, missing data strategy, sampling strategy, report of treatment receipt and report of intervention fidelity (mean = 6.3). Indicators most commonly utilized included randomized design (100%), objectively measured anthropometrics (96.7%), ITT analysis (86.7%) and reporting treatment adherence (76.7%). Most studies (62.2%) had a follow-up rate >75% and reported a loss to follow-up analytic strategy or minimal missing data (69.9%). Describing intervention fidelity (34.4%) and sampling from a known population (41.1%) were least common. Methodological quality was not associated with reporting a statistically significant result, effect size or sample size. This review found the published literature of behavioural weight loss trials to be of high quality for specific indicators, including study design and measurement. Identified for improvement include utilization of more rigorous statistical approaches to loss to follow up and better fidelity reporting. PMID:27071775

Organ Donation European Quality System: ODEQUS project methodology.

PubMed

Manyalich, M; Guasch, X; Gomez, M P; Páez, G; Teixeira, L

2013-01-01

Differences in the number of organ donors among hospitals cannot be explained only by the number of intensive care unit beds used or neurologic patients treated. The figures obtained are influenced by the organizational structure of the donation process and how efficient it is. The Organ Donation European Quality System (ODEQUS) is a 3-year project (from October 2010 to September 2013) co-financed by the European Agency for Health and Consumers (EAHC20091108) which aims to define a methodology to evaluate organ procurement performance at the hospital level. ODEQUS's specific objectives are to identify quality criteria and to develop quality indicators in three types of organ donation (after brain death, after cardiac death, and living donation). Those tools will be useful for hospitals' self-assessment as well as for developing an international auditing model. A consortium has been established involving 14 associated partners from Austria, Croatia, France, Germany, Italy, Poland, Portugal, Romania, Spain, Sweden, and the United Kingdom, as well as five collaborating partners from Greece, Hungary, Malta, Slovenia, and Turkey. The project has been established in three steps: 1) Design of a survey about the use of quality tools in a wide sample of European hospitals; 2) Development of quality criteria and quality indicators by the project experts. The main fields considered have been organizational structures, clinical procedures, and outcomes; and 3) Elaboration of an evaluation system to test the quality indicators in 11 European hospitals. Two types of training have been designed and performed: one concerns the development of quality criteria and quality indicators, whereas another is focused on how to use evaluation tools. Following this methodology, the project has so far identified 131 quality criteria and developed 31 quality indicators. Currently, the quality indicators are being tested in 11 selected hospitals. Copyright © 2013 Elsevier Inc. All rights

Security Quality Requirements Engineering (SQUARE) Methodology

DTIC Science & Technology

2005-11-01

such as Joint Application Development and the Accelerated Requirements Method [Wood 89, Hubbard 99] • Soft Systems Methodology [Checkland 89...investigated were misuse cases [Jacobson 92], Soft Systems Methodology (SSM) [Checkland 89], Quality Function Deployment (QFD) [QFD 05], Con- trolled...html (2005). [Checkland 89] Checkland, Peter. Soft Systems Methodology . Rational Analysis for a Problematic World. New York, NY: John Wiley & Sons

Assessment of maternal health care quality: conceptual and methodologic issues.

PubMed

Lane, D S; Kelman, H R

1975-10-01

Past efforts in assessment of the quality of maternity care have been analyzed in order to develop an evaluation framework that will have utility and applicability beyond a specific program, population, or health discipline. Presently available evaluation approaches have focused attention on either "high risk" populations or upon women experiencing a complicated pregnancy or delivery. Quality has been defined as the extent to which normative or empirically derived standards of obstetrical care have been applied. An alternative approach is suggested which conceives of the pregnancy as a normal physiological event but with the potentiality of either causing or exacerbating social or health problems. Maternity care quality is viewed as the application of those necessary health and health-related services that are required to safeguard the health of the mother and offspring, minimize the noxious consequences of pre-existing or concurrent health hazards or conditions, and upgrade the health and social functioning of those women who require it. Additionally, the system of services should be functionally organized to optimize care. Indicators of quality are suggested which incorporate structural, process, and outcome variables, and which link medical and consumer criteria in a comprehensive community level approach to quality assessment.

Environmental probabilistic quantitative assessment methodologies

USGS Publications Warehouse

Crovelli, R.A.

1995-01-01

In this paper, four petroleum resource assessment methodologies are presented as possible pollution assessment methodologies, even though petroleum as a resource is desirable, whereas pollution is undesirable. A methodology is defined in this paper to consist of a probability model and a probabilistic method, where the method is used to solve the model. The following four basic types of probability models are considered: 1) direct assessment, 2) accumulation size, 3) volumetric yield, and 4) reservoir engineering. Three of the four petroleum resource assessment methodologies were written as microcomputer systems, viz. TRIAGG for direct assessment, APRAS for accumulation size, and FASPU for reservoir engineering. A fourth microcomputer system termed PROBDIST supports the three assessment systems. The three assessment systems have different probability models but the same type of probabilistic method. The type of advantages of the analytic method are in computational speed and flexibility, making it ideal for a microcomputer. -from Author

Randomized clinical trials in dentistry: Risks of bias, risks of random errors, reporting quality, and methodologic quality over the years 1955-2013.

PubMed

Saltaji, Humam; Armijo-Olivo, Susan; Cummings, Greta G; Amin, Maryam; Flores-Mir, Carlos

2017-01-01

To examine the risks of bias, risks of random errors, reporting quality, and methodological quality of randomized clinical trials of oral health interventions and the development of these aspects over time. We included 540 randomized clinical trials from 64 selected systematic reviews. We extracted, in duplicate, details from each of the selected randomized clinical trials with respect to publication and trial characteristics, reporting and methodologic characteristics, and Cochrane risk of bias domains. We analyzed data using logistic regression and Chi-square statistics. Sequence generation was assessed to be inadequate (at unclear or high risk of bias) in 68% (n = 367) of the trials, while allocation concealment was inadequate in the majority of trials (n = 464; 85.9%). Blinding of participants and blinding of the outcome assessment were judged to be inadequate in 28.5% (n = 154) and 40.5% (n = 219) of the trials, respectively. A sample size calculation before the initiation of the study was not performed/reported in 79.1% (n = 427) of the trials, while the sample size was assessed as adequate in only 17.6% (n = 95) of the trials. Two thirds of the trials were not described as double blinded (n = 358; 66.3%), while the method of blinding was appropriate in 53% (n = 286) of the trials. We identified a significant decrease over time (1955-2013) in the proportion of trials assessed as having inadequately addressed methodological quality items (P < 0.05) in 30 out of the 40 quality criteria, or as being inadequate (at high or unclear risk of bias) in five domains of the Cochrane risk of bias tool: sequence generation, allocation concealment, incomplete outcome data, other sources of bias, and overall risk of bias. The risks of bias, risks of random errors, reporting quality, and methodological quality of randomized clinical trials of oral health interventions have improved over time; however, further efforts that contribute to the development of more stringent

Methodologic issues in assessing the quality of life of cancer patients.

PubMed

Aaronson, N K

1991-02-01

Although quality of life assessments have been employed successfully in descriptive and evaluative studies in oncology, their use in cancer clinical trials has, to date, been limited. A range of issues have impeded the conduct of clinical trial-based quality of life investigations. These include: the absence of theoretical models to guide the development of quality of life measures; over-reliance on ad hoc approaches to quality of life assessment; and insufficient attention to the practical constraints operating in clinical research settings. Of primary importance is the need to develop multidimensional quality of life instruments that are brief and psychometrically robust. It is suggested that future work on instrument development focus on refining currently available generic or cancer-specific measures, and on developing new diagnostic-specific questionnaire modules. This psychometric work should be guided by appropriate theoretical models of the relationship among health-related quality of life domains. Although it is widely accepted that the patient represents the most appropriate source of quality of life data, it is suggested that efforts also be directed toward improving the validity and reliability of physician-generated assessments of patients' performance status and of treatment toxicities, and toward determining the feasibility of employing family members as proxy raters of the psychologic and social health status of patients who are unwilling or unable to provide such information. Additional attention should be paid to the many logistical problems that arise in clinical trial-based quality of life investigations. In particular, research designs and data collection procedures should be selected that minimize patient, medical staff, and institutional burden.

Methodologic quality and relevance of references in pharmaceutical advertisements in a Canadian medical journal.

PubMed

Lexchin, J; Holbrook, A

1994-07-01

To evaluate the methodologic quality and relevance of references in pharmaceutical advertisements in the Canadian Medical Association Journal (CMAJ). Analytic study. All 114 references cited in the first 22 distinct pharmaceutical advertisements in volume 146 of CMAJ. Mean methodologic quality score (modified from the 6-point scale used to assess articles in the American College of Physicians' Journal Club) and mean relevance score (based on a new 5-point scale) for all references in each advertisement. Twenty of the 22 companies responded, sending 78 (90%) of the 87 references requested. The mean methodologic quality score was 58% (95% confidence limits [CL] 51% and 65%) and the mean relevance score 76% (95% CL 72% and 80%). The two mean scores were statistically lower than the acceptable score of 80% (p < 0.05), and the methodologic quality score was outside the preset clinically significant difference of 15%. The poor rating for methodologic quality was primarily because of the citation of references to low-quality review articles and "other" sources (i.e., other than reports of clinical trials). Half of the advertisements had a methodologic quality score of less than 65%, but only five had a relevance score of less than 65%. Although the relevance of most of the references was within minimal acceptable limits, the methodologic quality was often unacceptable. Because advertisements are an important part of pharmaceutical marketing and education, we suggest that companies develop written standards for their advertisements and monitor their advertisements for adherence to these standards. We also suggest that the Pharmaceutical Advertising Advisory Board develop more stringent guidelines for advertising and that it enforce these guidelines in a consistent, rigorous fashion.

WE-B-BRC-01: Current Methodologies in Risk Assessment

DOE Office of Scientific and Technical Information (OSTI.GOV)

Rath, F.

Prospective quality management techniques, long used by engineering and industry, have become a growing aspect of efforts to improve quality management and safety in healthcare. These techniques are of particular interest to medical physics as scope and complexity of clinical practice continue to grow, thus making the prescriptive methods we have used harder to apply and potentially less effective for our interconnected and highly complex healthcare enterprise, especially in imaging and radiation oncology. An essential part of most prospective methods is the need to assess the various risks associated with problems, failures, errors, and design flaws in our systems. Wemore » therefore begin with an overview of risk assessment methodologies used in healthcare and industry and discuss their strengths and weaknesses. The rationale for use of process mapping, failure modes and effects analysis (FMEA) and fault tree analysis (FTA) by TG-100 will be described, as well as suggestions for the way forward. This is followed by discussion of radiation oncology specific risk assessment strategies and issues, including the TG-100 effort to evaluate IMRT and other ways to think about risk in the context of radiotherapy. Incident learning systems, local as well as the ASTRO/AAPM ROILS system, can also be useful in the risk assessment process. Finally, risk in the context of medical imaging will be discussed. Radiation (and other) safety considerations, as well as lack of quality and certainty all contribute to the potential risks associated with suboptimal imaging. The goal of this session is to summarize a wide variety of risk analysis methods and issues to give the medical physicist access to tools which can better define risks (and their importance) which we work to mitigate with both prescriptive and prospective risk-based quality management methods. Learning Objectives: Description of risk assessment methodologies used in healthcare and industry Discussion of radiation

Quality in the Basic Grant Delivery System: Volume 3, Methodology.

ERIC Educational Resources Information Center

Advanced Technology, Inc., McLean, VA.

The research methodology of a study to assess 1980-1981 award accuracy of the Basic Educational Opportunity Grants (BEOG), or Pell grants, is described. The study is the first stage of a three-stage quality control project. During the spring of 1981 a nationally representative sample of 305 public, private, and proprietary institutions was…

Quality Assessment in the Blog Space

ERIC Educational Resources Information Center

Schaal, Markus; Fidan, Guven; Muller, Roland M.; Dagli, Orhan

2010-01-01

Purpose: The purpose of this paper is the presentation of a new method for blog quality assessment. The method uses the temporal sequence of link creation events between blogs as an implicit source for the collective tacit knowledge of blog authors about blog quality. Design/methodology/approach: The blog data are processed by the novel method for…

Comparison of methodological quality of positive versus negative comparative studies published in Indian medical journals: a systematic review

PubMed Central

Charan, Jaykaran; Chaudhari, Mayur; Jackson, Ryan; Mhaskar, Rahul; Reljic, Tea; Kumar, Ambuj

2015-01-01

Objectives Published negative studies should have the same rigour of methodological quality as studies with positive findings. However, the methodological quality of negative versus positive studies is not known. The objective was to assess the reported methodological quality of positive versus negative studies published in Indian medical journals. Design A systematic review (SR) was performed of all comparative studies published in Indian medical journals with a clinical science focus and impact factor >1 between 2011 and 2013. The methodological quality of randomised controlled trials (RCTs) was assessed using the Cochrane risk of bias tool, and the Newcastle-Ottawa scale for observational studies. The results were considered positive if the primary outcome was statistically significant and negative otherwise. When the primary outcome was not specified, we used data on the first outcome reported in the history followed by the results section. Differences in various methodological quality domains between positive versus negative studies were assessed by Fisher's exact test. Results Seven journals with 259 comparative studies were included in this SR. 24% (63/259) were RCTs, 24% (63/259) cohort studies, and 49% (128/259) case–control studies. 53% (137/259) of studies explicitly reported the primary outcome. Five studies did not report sufficient data to enable us to determine if results were positive or negative. Statistical significance was determined by p value in 78.3% (199/254), CI in 2.8% (7/254), both p value and CI in 11.8% (30/254), and only descriptive in 6.3% (16/254) of studies. The overall methodological quality was poor and no statistically significant differences between reporting of methodological quality were detected between studies with positive versus negative findings. Conclusions There was no difference in the reported methodological quality of positive versus negative studies. However, the uneven reporting of positive versus negative studies

Comparison of methodological quality of positive versus negative comparative studies published in Indian medical journals: a systematic review.

PubMed

Charan, Jaykaran; Chaudhari, Mayur; Jackson, Ryan; Mhaskar, Rahul; Reljic, Tea; Kumar, Ambuj

2015-06-24

Published negative studies should have the same rigour of methodological quality as studies with positive findings. However, the methodological quality of negative versus positive studies is not known. The objective was to assess the reported methodological quality of positive versus negative studies published in Indian medical journals. A systematic review (SR) was performed of all comparative studies published in Indian medical journals with a clinical science focus and impact factor >1 between 2011 and 2013. The methodological quality of randomised controlled trials (RCTs) was assessed using the Cochrane risk of bias tool, and the Newcastle-Ottawa scale for observational studies. The results were considered positive if the primary outcome was statistically significant and negative otherwise. When the primary outcome was not specified, we used data on the first outcome reported in the history followed by the results section. Differences in various methodological quality domains between positive versus negative studies were assessed by Fisher's exact test. Seven journals with 259 comparative studies were included in this SR. 24% (63/259) were RCTs, 24% (63/259) cohort studies, and 49% (128/259) case-control studies. 53% (137/259) of studies explicitly reported the primary outcome. Five studies did not report sufficient data to enable us to determine if results were positive or negative. Statistical significance was determined by p value in 78.3% (199/254), CI in 2.8% (7/254), both p value and CI in 11.8% (30/254), and only descriptive in 6.3% (16/254) of studies. The overall methodological quality was poor and no statistically significant differences between reporting of methodological quality were detected between studies with positive versus negative findings. There was no difference in the reported methodological quality of positive versus negative studies. However, the uneven reporting of positive versus negative studies (72% vs 28%) indicates a publication

A systematic and transparent approach for assessing the methodological quality of intervention effectiveness research: the Study Design and Implementation Assessment Device (Study DIAD).

PubMed

Valentine, Jeffrey C; Cooper, Harris

2008-06-01

Assessments of studies meant to evaluate the effectiveness of interventions, programs, and policies can serve an important role in the interpretation of research results. However, evidence suggests that available quality assessment tools have poor measurement characteristics and can lead to opposing conclusions when applied to the same body of studies. These tools tend to (a) be insufficiently operational, (b) rely on arbitrary post-hoc decision rules, and (c) result in a single number to represent a multidimensional construct. In response to these limitations, a multilevel and hierarchical instrument was developed in consultation with a wide range of methodological and statistical experts. The instrument focuses on the operational details of studies and results in a profile of scores instead of a single score to represent study quality. A pilot test suggested that satisfactory between-judge agreement can be obtained using well-trained raters working in naturalistic conditions. Limitations of the instrument are discussed, but these are inherent in making decisions about study quality given incomplete reporting and in the absence of strong, contextually based information about the effects of design flaws on study outcomes. (PsycINFO Database Record (c) 2008 APA, all rights reserved).

Systematic Review of the Application of Lean and Six Sigma Quality Improvement Methodologies in Radiology.

PubMed

Amaratunga, Thelina; Dobranowski, Julian

2016-09-01

Preventable yet clinically significant rates of medical error remain systemic, while health care spending is at a historic high. Industry-based quality improvement (QI) methodologies show potential for utility in health care and radiology because they use an empirical approach to reduce variability and improve workflow. The aim of this review was to systematically assess the literature with regard to the use and efficacy of Lean and Six Sigma (the most popular of the industrial QI methodologies) within radiology. MEDLINE, the Allied & Complementary Medicine Database, Embase Classic + Embase, Health and Psychosocial Instruments, and the Ovid HealthStar database, alongside the Cochrane Library databases, were searched on June 2015. Empirical studies in peer-reviewed journals were included if they assessed the use of Lean, Six Sigma, or Lean Six Sigma with regard to their ability to improve a variety of quality metrics in a radiology-centered clinical setting. Of the 278 articles returned, 23 studies were suitable for inclusion. Of these, 10 assessed Six Sigma, 7 assessed Lean, and 6 assessed Lean Six Sigma. The diverse range of measured outcomes can be organized into 7 common aims: cost savings, reducing appointment wait time, reducing in-department wait time, increasing patient volume, reducing cycle time, reducing defects, and increasing staff and patient safety and satisfaction. All of the included studies demonstrated improvements across a variety of outcomes. However, there were high rates of systematic bias and imprecision as per the Grading of Recommendations Assessment, Development and Evaluation guidelines. Lean and Six Sigma QI methodologies have the potential to reduce error and costs and improve quality within radiology. However, there is a pressing need to conduct high-quality studies in order to realize the true potential of these QI methodologies in health care and radiology. Recommendations on how to improve the quality of the literature are proposed

Randomized clinical trials in dentistry: Risks of bias, risks of random errors, reporting quality, and methodologic quality over the years 1955–2013

PubMed Central

Armijo-Olivo, Susan; Cummings, Greta G.; Amin, Maryam; Flores-Mir, Carlos

2017-01-01

Objectives To examine the risks of bias, risks of random errors, reporting quality, and methodological quality of randomized clinical trials of oral health interventions and the development of these aspects over time. Methods We included 540 randomized clinical trials from 64 selected systematic reviews. We extracted, in duplicate, details from each of the selected randomized clinical trials with respect to publication and trial characteristics, reporting and methodologic characteristics, and Cochrane risk of bias domains. We analyzed data using logistic regression and Chi-square statistics. Results Sequence generation was assessed to be inadequate (at unclear or high risk of bias) in 68% (n = 367) of the trials, while allocation concealment was inadequate in the majority of trials (n = 464; 85.9%). Blinding of participants and blinding of the outcome assessment were judged to be inadequate in 28.5% (n = 154) and 40.5% (n = 219) of the trials, respectively. A sample size calculation before the initiation of the study was not performed/reported in 79.1% (n = 427) of the trials, while the sample size was assessed as adequate in only 17.6% (n = 95) of the trials. Two thirds of the trials were not described as double blinded (n = 358; 66.3%), while the method of blinding was appropriate in 53% (n = 286) of the trials. We identified a significant decrease over time (1955–2013) in the proportion of trials assessed as having inadequately addressed methodological quality items (P < 0.05) in 30 out of the 40 quality criteria, or as being inadequate (at high or unclear risk of bias) in five domains of the Cochrane risk of bias tool: sequence generation, allocation concealment, incomplete outcome data, other sources of bias, and overall risk of bias. Conclusions The risks of bias, risks of random errors, reporting quality, and methodological quality of randomized clinical trials of oral health interventions have improved over time; however, further efforts that contribute

Methodologic quality and relevance of references in pharmaceutical advertisements in a Canadian medical journal.

PubMed Central

Lexchin, J; Holbrook, A

1994-01-01

OBJECTIVE: To evaluate the methodologic quality and relevance of references in pharmaceutical advertisements in the Canadian Medical Association Journal (CMAJ). DESIGN: Analytic study. DATA SOURCE: All 114 references cited in the first 22 distinct pharmaceutical advertisements in volume 146 of CMAJ. MAIN OUTCOME MEASURES: Mean methodologic quality score (modified from the 6-point scale used to assess articles in the American College of Physicians' Journal Club) and mean relevance score (based on a new 5-point scale) for all references in each advertisement. MAIN RESULTS: Twenty of the 22 companies responded, sending 78 (90%) of the 87 references requested. The mean methodologic quality score was 58% (95% confidence limits [CL] 51% and 65%) and the mean relevance score 76% (95% CL 72% and 80%). The two mean scores were statistically lower than the acceptable score of 80% (p < 0.05), and the methodologic quality score was outside the preset clinically significant difference of 15%. The poor rating for methodologic quality was primarily because of the citation of references to low-quality review articles and "other" sources (i.e., other than reports of clinical trials). Half of the advertisements had a methodologic quality score of less than 65%, but only five had a relevance score of less than 65%. CONCLUSIONS: Although the relevance of most of the references was within minimal acceptable limits, the methodologic quality was often unacceptable. Because advertisements are an important part of pharmaceutical marketing and education, we suggest that companies develop written standards for their advertisements and monitor their advertisements for adherence to these standards. We also suggest that the Pharmaceutical Advertising Advisory Board develop more stringent guidelines for advertising and that it enforce these guidelines in a consistent, rigorous fashion. PMID:8004560

Information security system quality assessment through the intelligent tools

NASA Astrophysics Data System (ADS)

Trapeznikov, E. V.

2018-04-01

The technology development has shown the automated system information security comprehensive analysis necessity. The subject area analysis indicates the study relevance. The research objective is to develop the information security system quality assessment methodology based on the intelligent tools. The basis of the methodology is the information security assessment model in the information system through the neural network. The paper presents the security assessment model, its algorithm. The methodology practical implementation results in the form of the software flow diagram are represented. The practical significance of the model being developed is noted in conclusions.

Systematic review of communication partner training in aphasia: methodological quality.

PubMed

Cherney, Leora R; Simmons-Mackie, Nina; Raymer, Anastasia; Armstrong, Elizabeth; Holland, Audrey

2013-10-01

Twenty-three studies identified from a previous systematic review examining the effects of communication partner training on persons with aphasia and their communication partners were evaluated for methodological quality. Two reviewers rated the studies on defined methodological quality criteria relevant to each study design. There were 11 group studies, seven single-subject participant design studies, and five qualitative studies. Quality scores were derived for each study. The mean inter-rater reliability of scores for each study design ranged from 85-93%, with Cohen's Kappa indicating substantial agreement between raters. Methodological quality of research on communication partner training in aphasia was highly varied. Overall, group studies employed the least rigorous methodology as compared to single subject and qualitative research. Only two of 11 group studies complied with more than half of the quality criteria. No group studies reported therapist blinding and only one group study reported participant blinding. Across all types of studies, the criterion of treatment fidelity was most commonly omitted. Failure to explicitly report certain methodological quality criteria may account for low ratings. Using methodological rating scales specific to the type of study design may help improve the methodological quality of aphasia treatment studies, including those on communication partner training.

Methodological and Reporting Quality of Comparative Studies Evaluating Health-Related Quality of Life of Colorectal Cancer Patients and Controls: A Systematic Review.

PubMed

Wong, Carlos K H; Guo, Vivian Y W; Chen, Jing; Lam, Cindy L K

2016-11-01

Health-related quality of life is an important outcome measure in patients with colorectal cancer. Comparison with normative data has been increasingly undertaken to assess the additional impact of colorectal cancer on health-related quality of life. This review aimed to critically appraise the methodological details and reporting characteristics of comparative studies evaluating differences in health-related quality of life between patients and controls. A systematic search of English-language literature published between January 1985 and May 2014 was conducted through a database search of PubMed, Web of Science, Embase, and Medline. Comparative studies reporting health-related quality-of-life outcomes among patients who have colorectal cancer and controls were selected. Methodological and reporting quality per comparison study was evaluated based on a 11-item methodological checklist proposed by Efficace in 2003 and a set of criteria predetermined by reviewers. Thirty-one comparative studies involving >10,000 patients and >10,000 controls were included. Twenty-three studies (74.2%) originated from European countries, with the largest number from the Netherlands (n = 6). Twenty-eight studies (90.3%) compared the health-related quality of life of patients with normative data published elsewhere, whereas the remaining studies recruited a group of patients who had colorectal cancer and a group of control patients within the same studies. The European Organisation for Research and Treatment of Cancer Quality-of-Life Questionnaire Core 30 was the most extensively used instrument (n = 16; 51.6%). Eight studies (25.8%) were classified as "probably robust" for clinical decision making according to the Efficace standard methodological checklist. Our further quality assessment revealed the lack of score differences reported (61.3%), contemporary comparisons (36.7%), statistical significance tested (38.7%), and matching of control group (58.1%), possibly leading to

Epidemiological characteristics and methodological quality of meta-analyses on diabetes mellitus treatment: a systematic review.

PubMed

Wu, Xin Yin; Lam, Victor C K; Yu, Yue Feng; Ho, Robin S T; Feng, Ye; Wong, Charlene H L; Yip, Benjamin H K; Tsoi, Kelvin K F; Wong, Samuel Y S; Chung, Vincent C H

2016-11-01

Well-conducted meta-analyses (MAs) are considered as one of the best sources of clinical evidence for treatment decision. MA with methodological flaws may introduce bias and mislead evidence users. The aim of this study is to investigate the characteristics and methodological quality of MAs on diabetes mellitus (DM) treatments. Systematic review. Cochrane Database of Systematic Review and Database of Abstract of Reviews of Effects were searched for relevant MAs. Assessing methodological quality of systematic reviews (AMSTAR) tool was used to evaluate the methodological quality of included MAs. Logistic regression analysis was used to identify association between characteristics of MA and AMSTAR results. A total of 252 MAs including 4999 primary studies and 13,577,025 patients were included. Over half of the MAs (65.1%) only included type 2 DM patients and 160 MAs (63.5%) focused on pharmacological treatments. About 89.7% MAs performed comprehensive literature search and 89.3% provided characteristics of included studies. Included MAs generally had poor performance on the remaining AMSTAR items, especially in assessing publication bias (39.3%), providing lists of studies (19.0%) and declaring source of support comprehensively (7.5%). Only 62.7% MAs mentioned about harm of interventions. MAs with corresponding author from Asia performed less well in providing MA protocol than those from Europe. Methodological quality of MA on DM treatments was unsatisfactory. There is considerable room for improvement, especially in assessing publication bias, providing lists of studies and declaring source of support comprehensively. Also, there is an urgent need for MA authors to report treatment harm comprehensively. © 2016 European Society of Endocrinology.

Are normative sonographic values of kidney size in children valid and reliable? A systematic review of the methodological quality of ultrasound studies using the Anatomical Quality Assessment (AQUA) tool.

PubMed

Chhapola, Viswas; Tiwari, Soumya; Deepthi, Bobbity; Henry, Brandon Michael; Brar, Rekha; Kanwal, Sandeep Kumar

2018-06-01

A plethora of research is available on ultrasonographic kidney size standards. We performed a systematic review of methodological quality of ultrasound studies aimed at developing normative renal parameters in healthy children, by evaluating the risk of bias (ROB) using the 'Anatomical Quality Assessment (AQUA)' tool. We searched Medline, Scopus, CINAHL, and Google Scholar on June 04 2018, and observational studies measuring kidney size by ultrasonography in healthy children (0-18 years) were included. The ROB of each study was evaluated in five domains using a 20 item coding scheme based on AQUA tool framework. Fifty-four studies were included. Domain 1 (subject characteristics) had a high ROB in 63% of studies due to the unclear description of age, sex, and ethnicity. The performance in Domain 2 (study design) was the best with 85% of studies having a prospective design. Methodological characterization (Domain 3) was poor across the studies (< 10% compliance), with suboptimal performance in the description of patient positioning, operator experience, and assessment of intra/inter-observer reliability. About three-fourth of the studies had a low ROB in Domain 4 (descriptive anatomy). Domain 5 (reporting of results) had a high ROB in approximately half of the studies, the majority reporting results in the form of central tendency measures. Significant deficiencies and heterogeneity were observed in the methodological quality of USG studies performed to-date for measurement of kidney size in children. We hereby provide a framework for the conducting such studies in future. PROSPERO (CRD42017071601).

Methodological quality of systematic reviews on treatments for depression: a cross-sectional study.

PubMed

Chung, V C H; Wu, X Y; Feng, Y; Ho, R S T; Wong, S Y S; Threapleton, D

2017-05-02

Depression is one of the most common mental disorders and identifying effective treatment strategies is crucial for the control of depression. Well-conducted systematic reviews (SRs) and meta-analyses can provide the best evidence for supporting treatment decision-making. Nevertheless, the trustworthiness of conclusions can be limited by lack of methodological rigour. This study aims to assess the methodological quality of a representative sample of SRs on depression treatments. A cross-sectional study on the bibliographical and methodological characteristics of SRs published on depression treatments trials was conducted. Two electronic databases (the Cochrane Database of Systematic Reviews and the Database of Abstracts of Reviews of Effects) were searched for potential SRs. SRs with at least one meta-analysis on the effects of depression treatments were considered eligible. The methodological quality of included SRs was assessed using the validated AMSTAR (Assessing the Methodological Quality of Systematic Reviews) tool. The associations between bibliographical characteristics and scoring on AMSTAR items were analysed using logistic regression analysis. A total of 358 SRs were included and appraised. Over half of included SRs (n = 195) focused on non-pharmacological treatments and harms were reported in 45.5% (n = 163) of all studies. Studies varied in methods and reporting practices: only 112 (31.3%) took the risk of bias among primary studies into account when formulating conclusions; 245 (68.4%) did not fully declare conflict of interests; 93 (26.0%) reported an 'a priori' design and 104 (29.1%) provided lists of both included and excluded studies. Results from regression analyses showed: more recent publications were more likely to report 'a priori' designs [adjusted odds ratio (AOR) 1.31, 95% confidence interval (CI) 1.09-1.57], to describe study characteristics fully (AOR 1.16, 95% CI 1.06-1.28), and to assess presence of publication bias (AOR 1.13, 95% CI 1

Methodological quality of systematic reviews in subfertility: a comparison of Cochrane and non-Cochrane systematic reviews in assisted reproductive technologies.

PubMed

Windsor, B; Popovich, I; Jordan, V; Showell, M; Shea, B; Farquhar, C

2012-12-01

Are there differences in the methodological quality of Cochrane systematic reviews (CRs) and non-Cochrane systematic reviews (NCRs) of assisted reproductive technologies? CRs on assisted reproduction are of higher methodological quality than similar reviews published in other journals. The quality of systematic reviews varies. This was a cross-sectional study of 30 CR and 30 NCR systematic reviews that were randomly selected from the eligible reviews identified from a literature search for the years 2007-2011. We extracted data on the reporting and methodological characteristics of the included systematic reviews. We assessed the methodological quality of the reviews using the 11-domain Measurement Tool to Assess the Methodological Quality of Systematic Reviews (AMSTAR) tool and subsequently compared CR and NCR systematic reviews. The AMSTAR quality assessment found that CRs were superior to NCRs. For 10 of 11 AMSTAR domains, the requirements were met in >50% of CRs, but only 4 of 11 domains showed requirements being met in >50% of NCRs. The strengths of CRs are the a priori study design, comprehensive literature search, explicit lists of included and excluded studies and assessments of internal validity. Significant failings in the CRs were found in duplicate study selection and data extraction (67% meeting requirements), assessment for publication bias (53% meeting requirements) and reporting of conflicts of interest (47% meeting requirements). NCRs were more likely to contain methodological weaknesses as the majority of the domains showed <40% of reviews meeting requirements, e.g. a priori study design (17%), duplicate study selection and data extraction (17%), assessment of study quality (27%), study quality in the formulation of conclusions (23%) and reporting of conflict of interests (10%). The AMSTAR assessment can only judge what is reported by authors. Although two of the five authors are involved in the production of CRs, the risk of bias was reduced by

Methodological and Reporting Quality of Systematic Reviews and Meta-analyses in Endodontics.

PubMed

Nagendrababu, Venkateshbabu; Pulikkotil, Shaju Jacob; Sultan, Omer Sheriff; Jayaraman, Jayakumar; Peters, Ove A

2018-06-01

The aim of this systematic review (SR) was to evaluate the quality of SRs and meta-analyses (MAs) in endodontics. A comprehensive literature search was conducted to identify relevant articles in the electronic databases from January 2000 to June 2017. Two reviewers independently assessed the articles for eligibility and data extraction. SRs and MAs on interventional studies with a minimum of 2 therapeutic strategies in endodontics were included in this SR. Methodologic and reporting quality were assessed using A Measurement Tool to Assess Systematic Reviews (AMSTAR) and Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA), respectively. The interobserver reliability was calculated using the Cohen kappa statistic. Statistical analysis with the level of significance at P < .05 was performed using Kruskal-Wallis tests and simple linear regression analysis. A total of 30 articles were selected for the current SR. Using AMSTAR, the item related to the scientific quality of studies used in conclusion was adhered by less than 40% of studies. Using PRISMA, 3 items were reported by less than 40% of studies, which were on objectives, protocol registration, and funding. No association was evident comparing the number of authors and country with quality. Statistical significance was observed when quality was compared among journals, with studies published as Cochrane reviews superior to those published in other journals. AMSTAR and PRISMA scores were significantly related. SRs in endodontics showed variability in both methodologic and reporting quality. Copyright © 2018 American Association of Endodontists. Published by Elsevier Inc. All rights reserved.

[A Methodological Quality Assessment of South Korean Nursing Research using Structural Equation Modeling in South Korea].

PubMed

Kim, Jung-Hee; Shin, Sujin; Park, Jin-Hwa

2015-04-01

The purpose of this study was to evaluate the methodological quality of nursing studies using structural equation modeling in Korea. Databases of KISS, DBPIA, and National Assembly Library up to March 2014 were searched using the MeSH terms 'nursing', 'structure', 'model'. A total of 152 studies were screened. After removal of duplicates and non-relevant titles, 61 papers were read in full. Of the sixty-one articles retrieved, 14 studies were published between 1992 and 2000, 27, between 2001 and 2010, and 20, between 2011 and March 2014. The methodological quality of the review examined varied considerably. The findings of this study suggest that more rigorous research is necessary to address theoretical identification, two indicator rule, distribution of sample, treatment of missing values, mediator effect, discriminant validity, convergent validity, post hoc model modification, equivalent models issues, and alternative models issues should be undergone. Further research with robust consistent methodological study designs from model identification to model respecification is needed to improve the validity of the research.

Proactive telephone counseling for smoking cessation: meta-analyses by recruitment channel and methodological quality.

PubMed

Tzelepis, Flora; Paul, Christine L; Walsh, Raoul A; McElduff, Patrick; Knight, Jenny

2011-06-22

Systematic reviews demonstrated that proactive telephone counseling increases smoking cessation rates. However, these reviews did not differentiate studies by recruitment channel, did not adequately assess methodological quality, and combined different measures of abstinence. Twenty-four randomized controlled trials published before December 31, 2008, included seven of active recruitment, 16 of passive recruitment, and one of mixed recruitment. We rated methodological quality on selection bias, study design, confounders, blinding, data collection methods, withdrawals, and dropouts, according to the Quality Assessment Tool for Quantitative Studies. We conducted random effects meta-analysis to pool the results according to abstinence type and follow-up time for studies overall and segregated by recruitment channel, and methodological quality. The level of statistical heterogeneity was quantified by I(2). All statistical tests were two-sided. Methodological quality ratings indicated two strong, 10 moderate, and 12 weak studies. Overall, compared with self-help materials or no intervention control groups, proactive telephone counseling had a statistically significantly greater effect on point prevalence abstinence (nonsmoking at follow-up or abstinent for at least 24 hours, 7 days before follow-up) at 6-9 months (relative risk [RR] = 1.26, 95% confidence interval [CI] = 1.11 to 1.43, P < .001, I(2) = 21.4%) but not at 12-15 months after recruitment. This pattern also emerged when studies were segregated by recruitment channel (active, passive) or methodological quality (strong/moderate, weak). Overall, the positive effect on prolonged/continuous abstinence (abstinent for 3 months or longer before follow-up) was also statistically significantly greater at 6-9 months (RR = 1.58, CI = 1.26 to 1.98, P < .001, I(2) = 49.1%) and 12-18 months after recruitment (RR = 1.40, CI = 1.23 to 1.60, P < .001, I(2) = 18.5%). With the exception of point prevalence abstinence in the long

Poor methodological quality and reporting standards of systematic reviews in burn care management.

PubMed

Wasiak, Jason; Tyack, Zephanie; Ware, Robert; Goodwin, Nicholas; Faggion, Clovis M

2017-10-01

The methodological and reporting quality of burn-specific systematic reviews has not been established. The aim of this study was to evaluate the methodological quality of systematic reviews in burn care management. Computerised searches were performed in Ovid MEDLINE, Ovid EMBASE and The Cochrane Library through to February 2016 for systematic reviews relevant to burn care using medical subject and free-text terms such as 'burn', 'systematic review' or 'meta-analysis'. Additional studies were identified by hand-searching five discipline-specific journals. Two authors independently screened papers, extracted and evaluated methodological quality using the 11-item A Measurement Tool to Assess Systematic Reviews (AMSTAR) tool and reporting quality using the 27-item Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist. Characteristics of systematic reviews associated with methodological and reporting quality were identified. Descriptive statistics and linear regression identified features associated with improved methodological quality. A total of 60 systematic reviews met the inclusion criteria. Six of the 11 AMSTAR items reporting on 'a priori' design, duplicate study selection, grey literature, included/excluded studies, publication bias and conflict of interest were reported in less than 50% of the systematic reviews. Of the 27 items listed for PRISMA, 13 items reporting on introduction, methods, results and the discussion were addressed in less than 50% of systematic reviews. Multivariable analyses showed that systematic reviews associated with higher methodological or reporting quality incorporated a meta-analysis (AMSTAR regression coefficient 2.1; 95% CI: 1.1, 3.1; PRISMA regression coefficient 6·3; 95% CI: 3·8, 8·7) were published in the Cochrane library (AMSTAR regression coefficient 2·9; 95% CI: 1·6, 4·2; PRISMA regression coefficient 6·1; 95% CI: 3·1, 9·2) and included a randomised control trial (AMSTAR regression

Methodological quality of randomised controlled trials in burns care. A systematic review.

PubMed

Danilla, Stefan; Wasiak, Jason; Searle, Susana; Arriagada, Cristian; Pedreros, Cesar; Cleland, Heather; Spinks, Anneliese

2009-11-01

To evaluate the methodological quality of published randomised controlled trials (RCTs) in burn care treatment and management. Using a predetermined search strategy we searched Ovid MEDLINE (1950 to January 2008) database to identify all English RCTs related to burn care. Full text studies identified were reviewed for key demographic and methodological characteristics. Methodological trial quality was assessed using the Jadad scale. A total of 257 studies involving 14,535 patients met the inclusion criteria. The median Jadad score was 2 (out of a best possible score of 5). Information was given in the introduction and discussion sections of most RCTs, although insufficient detail was provided on randomisation, allocation concealment, and blinding. The number of RCTs increased between 1950 and 2008 (Spearman's rho=0.6129, P<0.001), although the reporting quality did not improve over the same time period (P=0.1896) and was better in RCTs with larger sample sizes (median Jadad score, 4 vs. 2 points, P<0.0001). Methodological quality did not correlate with journal impact factor (P=0.2371). The reporting standards of RCTs are highly variable and less than optimal in most cases. The advent of evidence-based medicine heralds a new approach to burns care and systematic steps are needed to improve the quality of RCTs in this field. Identifying and reviewing the existing number of RCTs not only highlights the need for burn clinicians to conduct more trials, but may also encourage burn health clinicians to consider the importance of conducting trials that follow appropriate, evidence-based standards.

Epidemiology Characteristics, Methodological Assessment and Reporting of Statistical Analysis of Network Meta-Analyses in the Field of Cancer

PubMed Central

Ge, Long; Tian, Jin-hui; Li, Xiu-xia; Song, Fujian; Li, Lun; Zhang, Jun; Li, Ge; Pei, Gai-qin; Qiu, Xia; Yang, Ke-hu

2016-01-01

Because of the methodological complexity of network meta-analyses (NMAs), NMAs may be more vulnerable to methodological risks than conventional pair-wise meta-analysis. Our study aims to investigate epidemiology characteristics, conduction of literature search, methodological quality and reporting of statistical analysis process in the field of cancer based on PRISMA extension statement and modified AMSTAR checklist. We identified and included 102 NMAs in the field of cancer. 61 NMAs were conducted using a Bayesian framework. Of them, more than half of NMAs did not report assessment of convergence (60.66%). Inconsistency was assessed in 27.87% of NMAs. Assessment of heterogeneity in traditional meta-analyses was more common (42.62%) than in NMAs (6.56%). Most of NMAs did not report assessment of similarity (86.89%) and did not used GRADE tool to assess quality of evidence (95.08%). 43 NMAs were adjusted indirect comparisons, the methods used were described in 53.49% NMAs. Only 4.65% NMAs described the details of handling of multi group trials and 6.98% described the methods of similarity assessment. The median total AMSTAR-score was 8.00 (IQR: 6.00–8.25). Methodological quality and reporting of statistical analysis did not substantially differ by selected general characteristics. Overall, the quality of NMAs in the field of cancer was generally acceptable. PMID:27848997

Rapid evidence assessment: increasing the transparency of an emerging methodology.

PubMed

Varker, Tracey; Forbes, David; Dell, Lisa; Weston, Adele; Merlin, Tracy; Hodson, Stephanie; O'Donnell, Meaghan

2015-12-01

Within the field of evidence-based practice, policy makers, health care professionals and consumers require timely reviews to inform decisions on efficacious health care and treatments. Rapid evidence assessment (REA), also known as rapid review, has emerged in recent years as a literature review methodology that fulfils this need. It highlights what is known in a clinical area to the target audience in a relatively short time frame. This article discusses the lack of transparency and limited critical appraisal that can occur in REA, and goes on to propose general principles for conducting a REA. The approach that we describe is consistent with the principles underlying systematic review methodology, but also makes allowances for the rapid delivery of information as required while utilizing explicit and reproducible methods at each stage. Our method for conducting REA includes: developing an explicit research question in consultation with the end-users; clear definition of the components of the research question; development of a thorough and reproducible search strategy; development of explicit evidence selection criteria; and quality assessments and transparent decisions about the level of information to be obtained from each study. In addition, the REA may also include an assessment of the quality of the total body of evidence. Transparent reporting of REA methodologies will provide greater clarity to end-users about how the information is obtained and about the trade-offs that are made between speed and rigour. © 2015 John Wiley & Sons, Ltd.

The methodological quality of systematic reviews published in high-impact nursing journals: a review of the literature.

PubMed

Pölkki, Tarja; Kanste, Outi; Kääriäinen, Maria; Elo, Satu; Kyngäs, Helvi

2014-02-01

To analyse systematic review articles published in the top 10 nursing journals to determine the quality of the methods employed within them. Systematic review is defined as a scientific research method that synthesises high-quality scientific knowledge on a given topic. The number of such reviews in nursing science has increased dramatically during recent years, but their methodological quality has not previously been assessed. A review of the literature using a narrative approach. Ranked impact factor scores for nursing journals were obtained from the Journal Citation Report database of the Institute of Scientific Information (ISI Web of Knowledge). All issues from the years 2009 and 2010 of the top 10 ranked journals were included. CINAHL and MEDLINE databases were searched to locate studies using the search terms 'systematic review' and 'systematic literature review'. A total of 39 eligible studies were identified. Their methodological quality was evaluated through the specific criteria of quality assessment, description of synthesis and strengths and weaknesses reported in the included studies. Most of the eligible systematic reviews included several different designs or types of quantitative study. The majority included a quality assessment, and a total of 17 different criteria were identified. The method of synthesis was mentioned in about half of the reviews, the most common being narrative synthesis. The weaknesses of reviews were discussed, while strengths were rarely highlighted. The methodological quality of the systematic reviews examined varied considerably, although they were all published in nursing journals with a high-impact factor. Despite the fact that systematic reviews are considered the most robust source of research evidence, they vary in methodological quality. This point is important to consider in clinical practice when applying the results to patient care. © 2013 Blackwell Publishing Ltd.

Methodological quality of randomized trials published in the Journal of the American Podiatric Medical Association, 1999-2013.

PubMed

Landorf, Karl B; Menz, Hylton B; Armstrong, David G; Herbert, Robert D

2015-07-01

Randomized trials must be of high methodological quality to yield credible, actionable findings. The main aim of this project was to evaluate whether there has been an improvement in the methodological quality of randomized trials published in the Journal of the American Podiatric Medical Association (JAPMA). Randomized trials published in JAPMA during a 15-year period (January 1999 to December 2013) were evaluated. The methodological quality of randomized trials was evaluated using the PEDro scale (scores range from 0 to 10, with 0 being lowest quality). Linear regression was used to assess changes in methodological quality over time. A total of 1,143 articles were published in JAPMA between January 1999 and December 2013. Of these, 44 articles were reports of randomized trials. Although the number of randomized trials published each year increased, there was only minimal improvement in their methodological quality (mean rate of improvement = 0.01 points per year). The methodological quality of the trials studied was typically moderate, with a mean ± SD PEDro score of 5.1 ± 1.5. Although there were a few high-quality randomized trials published in the journal, most (84.1%) scored between 3 and 6. Although there has been an increase in the number of randomized trials published in JAPMA, there is substantial opportunity for improvement in the methodological quality of trials published in the journal. Researchers seeking to publish reports of randomized trials should seek to meet current best-practice standards in the conduct and reporting of their trials.

[Systematic Review of the Methodology Quality in Lung Cancer Screening Guidelines].

PubMed

Li, Jiang; Su, Kai; Li, Fang; Tang, Wei; Huang, Yao; Wang, Le; Huang, Huiyao; Shi, Jufang; Dai, Min

2016-10-20

Lung cancer is the most common malignancy and screening can decrease the mortality. High quality screening guideline is necessary and important for effective work. Our study is to review and evaluate the basic characteristics and methodology quality of the current global lung cancer screening guidelines so as to provide useful information for domestic study in the future. Electronic searches were done in English and Chinese databases including PubMed, the Cochrane Library, Web of Science, Embase, CNKI, CBM, Wanfang, and some cancer official websites. Articles were screened according to the predefined inclusion and exclusion criteria by two researchers. The quality of guidelines was assessed by AGREE II. At last, a total of 11 guidelines with methodology were included. The guidelines were issued mainly by USA (81%). Canada and China developed one, respectively. As for quality, the average score in the "Scale and objective" of all guidelines was 80, the average score in the "Participants" was 52, the average score in the "rigorism" was 50, the average score in the "clarity" was 76, the average score in the "application" was 43 and the average score in the "independence" was 59. The highest average score was found in 2013 and 2015. Canada guideline had higher quality in six domains. 7 guidelines were evaluated as A level. The number of clinical guidelines showed an increasing trend. Most guidelines were issued by developed countries with heavy burden. Multi-country contribution to one guideline was another trend. Evidence-based methodology was accepted globally in the guideline development.

Methodological quality of systematic reviews analyzing the use of laser therapy in restorative dentistry.

PubMed

Salmos, Janaina; Gerbi, Marleny E M M; Braz, Rodivan; Andrade, Emanuel S S; Vasconcelos, Belmiro C E; Bessa-Nogueira, Ricardo V

2010-01-01

The purpose of this study was to identify systematic reviews (SRs) that compared laser with other dental restorative procedures and to evaluate their methodological quality. A search strategy was developed and implemented for MEDLINE, the Cochrane Library, LILACS, and the Brazilian Dentistry Bibliography (1966- 2007). Inclusion criteria were: the article had to be an SR (+/- meta-analysis); primary focus was the use of laser in restorative dentistry; published in English, Spanish, Portuguese, Italian, German. Two investigators independently selected and evaluated the SRs. The overview quality assessment questionnaire (OQAQ) was used to evaluate methodological quality, and the results were averaged. There were 145 references identified, of which seven were SRs that met the inclusion criteria (kappa=0.81). Of the SRs, 71.4% appraised lasers in dental caries diagnosis. The mean overall OQAQ score was 4.4 [95% confidence interval (CI) 2.4- 6.5]. Of the SRs, 57.1% had major flaws, scoring < or = 4. SR methodological quality is low; therefore, clinicians should critically appraise them prior to considering their recommendations to guide patient care.

Methodological quality of randomized controlled trials of spinal manipulation and mobilization in tension-type headache, migraine, and cervicogenic headache.

PubMed

Fernández-de-las-Peñas, César; Alonso-Blanco, Cristina; San-Roman, Jesús; Miangolarra-Page, Juan C

2006-03-01

Literature review of quality of clinical trials. To determine the methodological quality of published randomized controlled trials that used spinal manipulation and/or mobilization to treat patients with tension-type headache (TTH), cervicogenic headache (CeH), and migraine (M) in the last decade. TTH, CeH, and M are the most prevalent types of headaches seen in adults. Individuals who have headaches frequently use physical therapy, manual therapy, or chiropractic care. Randomized controlled trials are considered an optimal method with which to assess the efficacy of any intervention. Computerized literature searches were performed in MEDLINE, EMBASE, COCHRANE, AMED, MANTIS, CINHAL, and PEDro databases. Randomized controlled trials in which spinal manipulation and/or mobilization had been used for TTH, CeH, and M published in a peer-reviewed journal as full text, and with at least 1 clinically relevant outcome measure (ie, headache intensity, duration, or frequency) were reviewed. The methodological quality of the studies was assessed independently by 2 reviewers using a set of predefined criteria. Only 8 studies met all the inclusion criteria. One clinical trial evaluated spinal manipulation and mobilization together, and the remaining 7 assessed spinal manipulative therapy. No controlled trials analyzing exclusively the effects of spinal mobilization were found. Methodological scores ranged from 35 to 56 points out of a theoretical maximum of 100 points, indicating an overall poor methodology of the studies. Only 2 studies obtained a high-quality score (greater than 50 points). No significant differences in quality scores were found based on the type of headache investigated. Methodological quality was not associated with the year of publication (before 2000, or later) nor with the results (positive, neutral, negative) reported in the studies. The most common flaws were a small sample size, the absence of a placebo control group, lack of blinded patients, and no

78 FR 15023 - Office of Health Assessment and Translation Webinar on the Assessment of Data Quality in Animal...

Federal Register 2010, 2011, 2012, 2013, 2014

2013-03-08

... and Translation Webinar on the Assessment of Data Quality in Animal Studies; Notice of Public Webinar...- based meeting on the assessment of data quality in animal studies. The Office of Health Assessment and... meetings with a focus on methodological issues related to OHAT implementing systematic review. The first...

Quality Assessment of TPB-Based Questionnaires: A Systematic Review

PubMed Central

Oluka, Obiageli Crystal; Nie, Shaofa; Sun, Yi

2014-01-01

Objective This review is aimed at assessing the quality of questionnaires and their development process based on the theory of planned behavior (TPB) change model. Methods A systematic literature search for studies with the primary aim of TPB-based questionnaire development was conducted in relevant databases between 2002 and 2012 using selected search terms. Ten of 1,034 screened abstracts met the inclusion criteria and were assessed for methodological quality using two different appraisal tools: one for the overall methodological quality of each study and the other developed for the appraisal of the questionnaire content and development process. Both appraisal tools consisted of items regarding the likelihood of bias in each study and were eventually combined to give the overall quality score for each included study. Results 8 of the 10 included studies showed low risk of bias in the overall quality assessment of each study, while 9 of the studies were of high quality based on the quality appraisal of questionnaire content and development process. Conclusion Quality appraisal of the questionnaires in the 10 reviewed studies was successfully conducted, highlighting the top problem areas (including: sample size estimation; inclusion of direct and indirect measures; and inclusion of questions on demographics) in the development of TPB-based questionnaires and the need for researchers to provide a more detailed account of their development process. PMID:24722323

Methodological Quality of Consensus Guidelines in Implant Dentistry.

PubMed

Faggion, Clovis Mariano; Apaza, Karol; Ariza-Fritas, Tania; Málaga, Lilian; Giannakopoulos, Nikolaos Nikitas; Alarcón, Marco Antonio

2017-01-01

Consensus guidelines are useful to improve clinical decision making. Therefore, the methodological evaluation of these guidelines is of paramount importance. Low quality information may guide to inadequate or harmful clinical decisions. To evaluate the methodological quality of consensus guidelines published in implant dentistry using a validated methodological instrument. The six implant dentistry journals with impact factors were scrutinised for consensus guidelines related to implant dentistry. Two assessors independently selected consensus guidelines, and four assessors independently evaluated their methodological quality using the Appraisal of Guidelines for Research & Evaluation (AGREE) II instrument. Disagreements in the selection and evaluation of guidelines were resolved by consensus. First, the consensus guidelines were analysed alone. Then, systematic reviews conducted to support the guidelines were included in the analysis. Non-parametric statistics for dependent variables (Wilcoxon signed rank test) was used to compare both groups. Of 258 initially retrieved articles, 27 consensus guidelines were selected. Median scores in four domains (applicability, rigour of development, stakeholder involvement, and editorial independence), expressed as percentages of maximum possible domain scores, were below 50% (median, 26%, 30.70%, 41.70%, and 41.70%, respectively). The consensus guidelines and consensus guidelines + systematic reviews data sets could be compared for 19 guidelines, and the results showed significant improvements in all domain scores (p < 0.05). Methodological improvement of consensus guidelines published in major implant dentistry journals is needed. The findings of the present study may help researchers to better develop consensus guidelines in implant dentistry, which will improve the quality and trust of information needed to make proper clinical decisions.

Methodologic European external quality assurance for DNA sequencing: the EQUALseq program.

PubMed

Ahmad-Nejad, Parviz; Dorn-Beineke, Alexandra; Pfeiffer, Ulrike; Brade, Joachim; Geilenkeuser, Wolf-Jochen; Ramsden, Simon; Pazzagli, Mario; Neumaier, Michael

2006-04-01

DNA sequencing is a key technique in molecular diagnostics, but to date no comprehensive methodologic external quality assessment (EQA) programs have been instituted. Between 2003 and 2005, the European Union funded, as specific support actions, the EQUAL initiative to develop methodologic EQA schemes for genotyping (EQUALqual), quantitative PCR (EQUALquant), and sequencing (EQUALseq). Here we report on the results of the EQUALseq program. The participating laboratories received a 4-sample set comprising 2 DNA plasmids, a PCR product, and a finished sequencing reaction to be analyzed. Data and information from detailed questionnaires were uploaded online and evaluated by use of a scoring system for technical skills and proficiency of data interpretation. Sixty laboratories from 21 European countries registered, and 43 participants (72%) returned data and samples. Capillary electrophoresis was the predominant platform (n = 39; 91%). The median contiguous correct sequence stretch was 527 nucleotides with considerable variation in quality of both primary data and data evaluation. The association between laboratory performance and the number of sequencing assays/year was statistically significant (P <0.05). Interestingly, more than 30% of participants neither added comments to their data nor made efforts to identify the gene sequences or mutational positions. Considerable variations exist even in a highly standardized methodology such as DNA sequencing. Methodologic EQAs are appropriate tools to uncover strengths and weaknesses in both technique and proficiency, and our results emphasize the need for mandatory EQAs. The results of EQUALseq should help improve the overall quality of molecular genetics findings obtained by DNA sequencing.

Approaches and Methodological As Well As Interpretative Issues in Quality of Life Research in Malaysia

PubMed Central

Hasanah, C. I.

2003-01-01

Quality of life measures are designed to enable patients’ perspectives on the impact of health and healthcare interventions on their lives to be assessed and taken into account in clinical decision-making and research. This paper discusses some approaches, methodological as well as interpretative issues of health related quality of life research. PMID:23386798

Environmental risk assessment of water quality in harbor areas: a new methodology applied to European ports.

PubMed

Gómez, Aina G; Ondiviela, Bárbara; Puente, Araceli; Juanes, José A

2015-05-15

This work presents a standard and unified procedure for assessment of environmental risks at the contaminant source level in port aquatic systems. Using this method, port managers and local authorities will be able to hierarchically classify environmental hazards and proceed with the most suitable management actions. This procedure combines rigorously selected parameters and indicators to estimate the environmental risk of each contaminant source based on its probability, consequences and vulnerability. The spatio-temporal variability of multiple stressors (agents) and receptors (endpoints) is taken into account to provide accurate estimations for application of precisely defined measures. The developed methodology is tested on a wide range of different scenarios via application in six European ports. The validation process confirms its usefulness, versatility and adaptability as a management tool for port water quality in Europe and worldwide. Copyright © 2015 Elsevier Ltd. All rights reserved.

Epidemiology characteristics, reporting characteristics, and methodological quality of systematic reviews and meta-analyses on traditional Chinese medicine nursing interventions published in Chinese journals.

PubMed

Yang, Min; Jiang, Li; Wang, Aihong; Xu, Guihua

2017-02-01

To evaluate the epidemiological characteristics, reporting characteristics, and methodological quality of systematic reviews in the traditional Chinese medicine nursing field published in Chinese journals. The number of systematic reviews in the traditional Chinese medicine nursing field has increased, but their epidemiology, quality, and reporting characteristics have not been assessed completely. We generated an overview of reviews using a narrative approach. Four Chinese databases were searched for systematic reviews from inception to December 2015. The Preferred Reporting Items of Systematic Reviews and Meta-analyses and the Assessment of Multiple Systematic Reviews checklists were adopted to evaluate reporting and methodological quality, respectively. A total of 73 eligible systematic reviews, published from 2005 to 2015, were included. The deficiencies in reporting characteristics mainly lay in the lack of structured abstract or protocol, incomplete reporting of search strategies, study selection, and risk of bias. The deficiencies in methodological quality were reflected in the lack of a priori design and conflict of interest, incomplete literature searches, and assessment of publication bias. The quality of the evaluated reviews was unsatisfactory; attention should be paid to the improvement of reporting and methodological quality in the conduct of systematic reviews. © 2016 John Wiley & Sons Australia, Ltd.

Methodological Quality of Consensus Guidelines in Implant Dentistry

PubMed Central

Faggion, Clovis Mariano; Apaza, Karol; Ariza-Fritas, Tania; Málaga, Lilian; Giannakopoulos, Nikolaos Nikitas; Alarcón, Marco Antonio

2017-01-01

Background Consensus guidelines are useful to improve clinical decision making. Therefore, the methodological evaluation of these guidelines is of paramount importance. Low quality information may guide to inadequate or harmful clinical decisions. Objective To evaluate the methodological quality of consensus guidelines published in implant dentistry using a validated methodological instrument. Methods The six implant dentistry journals with impact factors were scrutinised for consensus guidelines related to implant dentistry. Two assessors independently selected consensus guidelines, and four assessors independently evaluated their methodological quality using the Appraisal of Guidelines for Research & Evaluation (AGREE) II instrument. Disagreements in the selection and evaluation of guidelines were resolved by consensus. First, the consensus guidelines were analysed alone. Then, systematic reviews conducted to support the guidelines were included in the analysis. Non-parametric statistics for dependent variables (Wilcoxon signed rank test) was used to compare both groups. Results Of 258 initially retrieved articles, 27 consensus guidelines were selected. Median scores in four domains (applicability, rigour of development, stakeholder involvement, and editorial independence), expressed as percentages of maximum possible domain scores, were below 50% (median, 26%, 30.70%, 41.70%, and 41.70%, respectively). The consensus guidelines and consensus guidelines + systematic reviews data sets could be compared for 19 guidelines, and the results showed significant improvements in all domain scores (p < 0.05). Conclusions Methodological improvement of consensus guidelines published in major implant dentistry journals is needed. The findings of the present study may help researchers to better develop consensus guidelines in implant dentistry, which will improve the quality and trust of information needed to make proper clinical decisions. PMID:28107405

Clinical trials in palliative care: a systematic review of their methodological characteristics and of the quality of their reporting.

PubMed

Bouça-Machado, Raquel; Rosário, Madalena; Alarcão, Joana; Correia-Guedes, Leonor; Abreu, Daisy; Ferreira, Joaquim J

2017-01-25

Over the past decades there has been a significant increase in the number of published clinical trials in palliative care. However, empirical evidence suggests that there are methodological problems in the design and conduct of studies, which raises questions about the validity and generalisability of the results and of the strength of the available evidence. We sought to evaluate the methodological characteristics and assess the quality of reporting of clinical trials in palliative care. We performed a systematic review of published clinical trials assessing therapeutic interventions in palliative care. Trials were identified using MEDLINE (from its inception to February 2015). We assessed methodological characteristics and describe the quality of reporting using the Cochrane Risk of Bias tool. We retrieved 107 studies. The most common medical field studied was oncology, and 43.9% of trials evaluated pharmacological interventions. Symptom control and physical dimensions (e.g. intervention on pain, breathlessness, nausea) were the palliative care-specific issues most studied. We found under-reporting of key information in particular on random sequence generation, allocation concealment, and blinding. While the number of clinical trials in palliative care has increased over time, methodological quality remains suboptimal. This compromises the quality of studies. Therefore, a greater effort is needed to enable the appropriate performance of future studies and increase the robustness of evidence-based medicine in this important field.

The methodological quality of nurse-sensitive indicators in Dutch hospitals: A descriptive exploratory research study.

PubMed

Kieft, R A M M; Stalpers, D; Jansen, A P M; Francke, A L; Delnoij, D M J

2018-06-01

Nurse-sensitive indicators (NSIs) are increasingly being developed and used to establish quality of nursing care in Western countries. The objective was to gain insights into the methodological quality of mandatory NSIs in Dutch hospitals, including indicators for pain, wound care, malnutrition and delirium. A descriptive exploratory design was used, starting with desk research into publicly available documents and reports describing the development of the NSIs included in this study. We used the validated Appraisal of Indicators through Research and Evaluation (AIRE) instrument to evaluate the methodological quality. Although the purpose and relevance of each individual NSI have been described, no detailed information about the criteria for selecting these topics is available. It is not clear which specific stakeholders participated and how their input was used. We found no information about the process of collecting and compiling scientific evidence. It is unclear whether and to what extent the usability of NSIs has been tested. The methodological quality of NSIs used in Dutch hospitals is less than optimal in various ways and it is therefore questionable if the indicators are accurate enough to identify changes or improve nursing practice. Our study also provides an example of how the methodological quality of NSIs can be assessed systematically, which is relevant considering the increasing use of NSIs in various countries. Copyright © 2018 Elsevier B.V. All rights reserved.

Methodological Quality of National Guidelines for Pediatric Inpatient Conditions

PubMed Central

Hester, Gabrielle; Nelson, Katherine; Mahant, Sanjay; Eresuma, Emily; Keren, Ron; Srivastava, Rajendu

2014-01-01

Background Guidelines help inform standardization of care for quality improvement (QI). The Pediatric Research in Inpatient Settings (PRIS) network published a prioritization list of inpatient conditions with high prevalence, cost, and variation in resource utilization across children’s hospitals. The methodological quality of guidelines for priority conditions is unknown. Objective To rate the methodological quality of national guidelines for 20 priority pediatric inpatient conditions. Design We searched sources including PubMed for national guidelines published 2002–2012. Guidelines specific to one organism, test or treatment, or institution were excluded. Guidelines were rated by two raters using a validated tool (AGREE II) with an overall rating on a 7-point scale (7–highest). Inter-rater reliability was measured with a weighted kappa coefficient. Results 17 guidelines met inclusion criteria for 13 conditions, 7 conditions yielded no relevant national guidelines. The highest methodological quality guidelines were for asthma, tonsillectomy, and bronchiolitis (mean overall rating 7, 6.5 and 6.5 respectively); the lowest were for sickle cell disease (2 guidelines) and dental caries (mean overall rating 4, 3.5, and 3 respectively). The overall weighted kappa was 0.83 (95% confidence interval 0.78–0.87). Conclusions We identified a group of moderate to high methodological quality national guidelines for priority pediatric inpatient conditions. Hospitals should consider these guidelines to inform QI initiatives. PMID:24677729

Q methodology, risk training and quality management.

PubMed

McKeown, M; Hinks, M; Stowell-Smith, M; Mercer, D; Forster, J

1999-01-01

The results of a Q methodological study of professional understandings of the notion of risk in mental health services within the UK are discussed in relation to the relevance for staff training and quality assurance. The study attempted to access the diversity of understandings of risk issues amongst a multi-professional group of staff (n = 60) attending inter-agency risk training workshops in 1998. Q methodology is presented as both an appropriate means for such inquiry and as a novel experiential technique for training purposes. A tentative argument is advanced that the qualitative accounts generated by Q research could assist in systematic reviews of quality, complementing the singularly quantitative approaches typically represented in the audit process.

Using Q Methodology in Quality Improvement Projects.

PubMed

Tiernon, Paige; Hensel, Desiree; Roy-Ehri, Leah

Q methodology consists of a philosophical framework and procedures to identify subjective viewpoints that may not be well understood, but its use in nursing is still quite limited. We describe how Q methodology can be used in quality improvement projects to better understand local viewpoints that act as facilitators or barriers to the implementation of evidence-based practice. We describe the use of Q methodology to identify nurses' attitudes about the provision of skin-to-skin care after cesarean birth. Copyright © 2017 AWHONN, the Association of Women's Health, Obstetric and Neonatal Nurses. Published by Elsevier Inc. All rights reserved.

Methodological and reporting quality of systematic reviews on tuberculosis.

PubMed

Nicolau, I; Ling, D; Tian, L; Lienhardt, C; Pai, M

2013-09-01

Systematic reviews are used to inform tuberculosis (TB) guidelines. However, there are no data on whether TB systematic reviews are conducted well and reported transparently. We searched four databases for reviews published between 2005 and 2010. Methodological quality was evaluated using AMSTAR and quality of reporting was assessed using PRISMA. Of 152 articles, 137 (90%) met the inclusion criteria. Only 3 of 11 AMSTAR quality items were met in most reviews: appropriate methods to combine findings (67%), comprehensive literature search (72%) and presentation of characteristics of included studies (90%). The other eight items were met in 4-53% of the reviews. Only 4% of the reviews disclosed conflicts of interest. The majority of the PRISMA items were reported in more than 60-76% of the reviews. Only nine items were reported in less than 55% of the reviews, the lowest being the full-search strategy (30%), risk of bias across studies in the Methods (27%) and Results (21%) sections, and indication of a review protocol (15%). Systematic reviews in our survey were well reported but generally of moderate to low quality. Better training, use of reporting guidelines and registration of systematic reviews could improve the quality of TB reviews.

Quality of reporting of surveys in critical care journals: a methodologic review.

PubMed

Duffett, Mark; Burns, Karen E; Adhikari, Neill K; Arnold, Donald M; Lauzier, François; Kho, Michelle E; Meade, Maureen O; Hayani, Omar; Koo, Karen; Choong, Karen; Lamontagne, François; Zhou, Qi; Cook, Deborah J

2012-02-01

Adequate reporting is needed to judge methodologic quality and assess the risk of bias of surveys. The objective of this study is to describe the methodology and quality of reporting of surveys published in five critical care journals. All issues (1996-2009) of the American Journal of Respiratory and Critical Care Medicine, Critical Care, Critical Care Medicine, Intensive Care Medicine, and Pediatric Critical Care Medicine. Two reviewers hand-searched all issues in duplicate. We included publications of self-administered questionnaires of health professionals and excluded surveys that were part of a multi-method study or measured the effect of an intervention. Data were abstracted in duplicate. We included 151 surveys. The frequency of survey publication increased at an average rate of 0.38 surveys per 1000 citations per year from 1996-2009 (p for trend = 0.001). The median number of respondents and reported response rates were 217 (interquartile range 90 to 402) and 63.3% (interquartile range 45.0% to 81.0%), respectively. Surveys originated predominantly from North America (United States [40.4%] and Canada [18.5%]). Surveys most frequently examined stated practice (78.8%), attitudes or opinions (60.3%), and less frequently knowledge (9.9%). The frequency of reporting on the survey design and methods were: 1) instrument development: domains (59.1%), item generation (33.1%), item reduction (12.6%); 2) instrument testing: pretesting or pilot testing (36.2%) and assessments of clarity (25.2%) or clinical sensibility (15.7%); and 3) clinimetric properties: qualitative or quantitative description of at least one of face, content, construct validity, intra- or inter-rater reliability, or consistency (28.5%). The reporting of five key elements of survey design and conduct did not significantly change over time. Surveys, primarily conducted in North America and focused on self-reported practice, are increasingly published in highly cited critical care journals. More

[The methodological assessment and qualitative evaluation of psychometric performance tests based on the example of modern tests that assess reading and spelling skills].

PubMed

Galuschka, Katharina; Rothe, Josefine; Schulte-Körne, Gerd

2015-09-01

This article looks at a means of objectively evaluating the quality of psychometric tests. This approach enables users to evaluate psychometric tests based on their methodological characteristics, in order to decide which instrument should be used. Reading and spelling assessment tools serve as examples. The paper also provides a review of German psychometric tests for the assessment of reading and spelling skills. This method facilitates the identification of psychometric tests.of high methodological quality which can be used for the assessment of reading and spelling skills. Reading performance should ideally be assessed with the following instruments: ELFE 1-6, LGVT 6-12, LESEN 6-7, LESEN 8-9, or WLLP-R. The tests to be used for the evaluation of spelling skills are DERET 1-2+, DERET 3-4+, WRT 1+, WRT 2+, WRT 3+, WRT 4+ or HSP 1-10.

Systematic review of the methodological and reporting quality of case series in surgery.

PubMed

Agha, R A; Fowler, A J; Lee, S-Y; Gundogan, B; Whitehurst, K; Sagoo, H K; Jeong, K J L; Altman, D G; Orgill, D P

2016-09-01

Case series are an important and common study type. No guideline exists for reporting case series and there is evidence of key data being missed from such reports. The first step in the process of developing a methodologically sound reporting guideline is a systematic review of literature relevant to the reporting deficiencies of case series. A systematic review of methodological and reporting quality in surgical case series was performed. The electronic search strategy was developed by an information specialist and included MEDLINE, Embase, Cochrane Methods Register, Science Citation Index and Conference Proceedings Citation index, from the start of indexing to 5 November 2014. Independent screening, eligibility assessments and data extraction were performed. Included articles were then analysed for five areas of deficiency: failure to use standardized definitions, missing or selective data (including the omission of whole cases or important variables), transparency or incomplete reporting, whether alternative study designs were considered, and other issues. Database searching identified 2205 records. Through the process of screening and eligibility assessments, 92 articles met inclusion criteria. Frequencies of methodological and reporting issues identified were: failure to use standardized definitions (57 per cent), missing or selective data (66 per cent), transparency or incomplete reporting (70 per cent), whether alternative study designs were considered (11 per cent) and other issues (52 per cent). The methodological and reporting quality of surgical case series needs improvement. The data indicate that evidence-based guidelines for the conduct and reporting of case series may be useful. © 2016 BJS Society Ltd Published by John Wiley & Sons Ltd.

Toward a Methodology for Conducting Social Impact Assessments Using Quality of Social Life Indicators.

ERIC Educational Resources Information Center

Olsen, Marvin E.; Merwin, Donna J.

Broadly conceived, social impacts refer to all changes in the structure and functioning of patterned social ordering that occur in conjunction with an environmental, technological, or social innovation or alteration. Departing from the usual cost-benefit analysis approach, a new methodology proposes conducting social impact assessment grounded in…

USGS Methodology for Assessing Continuous Petroleum Resources

USGS Publications Warehouse

Charpentier, Ronald R.; Cook, Troy A.

2011-01-01

The U.S. Geological Survey (USGS) has developed a new quantitative methodology for assessing resources in continuous (unconventional) petroleum deposits. Continuous petroleum resources include shale gas, coalbed gas, and other oil and gas deposits in low-permeability ("tight") reservoirs. The methodology is based on an approach combining geologic understanding with well productivities. The methodology is probabilistic, with both input and output variables as probability distributions, and uses Monte Carlo simulation to calculate the estimates. The new methodology is an improvement of previous USGS methodologies in that it better accommodates the uncertainties in undrilled or minimally drilled deposits that must be assessed using analogs. The publication is a collection of PowerPoint slides with accompanying comments.

Relationships between abstract features and methodological quality explained variations of social media activity derived from systematic reviews about psoriasis interventions.

PubMed

Ruano, J; Aguilar-Luque, M; Isla-Tejera, B; Alcalde-Mellado, P; Gay-Mimbrera, J; Hernandez-Romero, José Luis; Sanz-Cabanillas, J L; Maestre-López, B; González-Padilla, M; Carmona-Fernández, P J; Gómez-García, F; García-Nieto, A Vélez

2018-05-24

The aim of this study was to describe the relationship among abstract structure, readability, and completeness, and how these features may influence social media activity and bibliometric results, considering systematic reviews (SRs) about interventions in psoriasis classified by methodological quality. Systematic literature searches about psoriasis interventions were undertaken on relevant databases. For each review, methodological quality was evaluated using the Assessing the Methodological Quality of Systematic Reviews (AMSTAR) tool. Abstract extension, structure, readability, and quality and completeness of reporting were analyzed. Social media activity, which consider Twitter and Facebook mention counts, as well as Mendeley readers and Google scholar citations were obtained for each article. Analyses were conducted to describe any potential influence of abstract characteristics on review's social media diffusion. We classified 139 intervention SRs as displaying high/moderate/low methodological quality. We observed that abstract readability of SRs has been maintained high for last 20 years, although there are some differences based on their methodological quality. Free-format abstracts were most sensitive to the increase of text readability as compared with more structured abstracts (IMRAD or 8-headings), yielding opposite effects on their quality and completeness depending on the methodological quality: a worsening in low quality reviews and an improvement in those of high-quality. Both readability indices and PRISMA for Abstract total scores showed an inverse relationship with social media activity and bibliometric results in high methodological quality reviews but not in those of lower quality. Our results suggest that increasing abstract readability must be specially considered when writing free-format summaries of high-quality reviews, because this fact correlates with an improvement of their completeness and quality, and this may help to achieve broader

Retinal Image Quality Assessment for Spaceflight-Induced Vision Impairment Study

NASA Technical Reports Server (NTRS)

Vu, Amanda Cadao; Raghunandan, Sneha; Vyas, Ruchi; Radhakrishnan, Krishnan; Taibbi, Giovanni; Vizzeri, Gianmarco; Grant, Maria; Chalam, Kakarla; Parsons-Wingerter, Patricia

2015-01-01

Long-term exposure to space microgravity poses significant risks for visual impairment. Evidence suggests such vision changes are linked to cephalad fluid shifts, prompting a need to directly quantify microgravity-induced retinal vascular changes. The quality of retinal images used for such vascular remodeling analysis, however, is dependent on imaging methodology. For our exploratory study, we hypothesized that retinal images captured using fluorescein imaging methodologies would be of higher quality in comparison to images captured without fluorescein. A semi-automated image quality assessment was developed using Vessel Generation Analysis (VESGEN) software and MATLAB® image analysis toolboxes. An analysis of ten images found that the fluorescein imaging modality provided a 36% increase in overall image quality (two-tailed p=0.089) in comparison to nonfluorescein imaging techniques.

The Statistical point of view of Quality: the Lean Six Sigma methodology

PubMed Central

Viti, Andrea; Terzi, Alberto

2015-01-01

Six Sigma and Lean are two quality improvement methodologies. The Lean Six Sigma methodology is applicable to repetitive procedures. Therefore, the use of this methodology in the health-care arena has focused mainly on areas of business operations, throughput, and case management and has focused on efficiency outcomes. After the revision of methodology, the paper presents a brief clinical example of the use of Lean Six Sigma as a quality improvement method in the reduction of the complications during and after lobectomies. Using Lean Six Sigma methodology, the multidisciplinary teams could identify multiple modifiable points across the surgical process. These process improvements could be applied to different surgical specialties and could result in a measurement, from statistical point of view, of the surgical quality. PMID:25973253

The Statistical point of view of Quality: the Lean Six Sigma methodology.

PubMed

Bertolaccini, Luca; Viti, Andrea; Terzi, Alberto

2015-04-01

Six Sigma and Lean are two quality improvement methodologies. The Lean Six Sigma methodology is applicable to repetitive procedures. Therefore, the use of this methodology in the health-care arena has focused mainly on areas of business operations, throughput, and case management and has focused on efficiency outcomes. After the revision of methodology, the paper presents a brief clinical example of the use of Lean Six Sigma as a quality improvement method in the reduction of the complications during and after lobectomies. Using Lean Six Sigma methodology, the multidisciplinary teams could identify multiple modifiable points across the surgical process. These process improvements could be applied to different surgical specialties and could result in a measurement, from statistical point of view, of the surgical quality.

[Assessment of the methodological quality of theses submitted to the Faculty of Medicine Fez].

PubMed

Boly, A; Tachfouti, N; Zohoungbogbo, I S S; Achhab, Y El; Nejjari, C

2014-06-09

A thesis in medicine is a scientific work which allows a medical student to acquire a Doctor of Medicine degree. It is therefore recommended that theses presented by students fulfill essential methodological criteria in order to obtain scientifically credible results and recommendations. The aim of this study was to assess the methodology of thesis presented to the Faculty of Medicine in Fez in 2008. We developed an evaluation table containing questions on the different sections of the IMRAD structure on which these theses were based and we estimated the proportion of theses that conformed to each criterion. There were 160 theses on various specialties presented in 2008. The majority of the theses (79.3%) were case series. Research questions were clearly expressed in 62.0% but the primary objectives were pertinent in only 52.0%. Our study shows that there were important deficiencies in the methodological rigor of the theses and very little representation of the theses in publications.

Methodological quality is underrated in systematic reviews and meta-analyses in health psychology.

PubMed

Oliveras, Isabel; Losilla, Josep-Maria; Vives, Jaume

2017-06-01

In this paper, we compile and describe the main approaches proposed in the literature to include methodological quality (MQ) or risk of bias (RoB) into research synthesis. We also meta-review how the RoB of observational primary studies is being assessed and to what extent the results are incorporated in the conclusions of research synthesis. Electronic databases were searched for systematic reviews or meta-analyses related to health and clinical psychology. A random sample of 90 reviews published between January 2010 and May 2016 was examined. A total of 46 reviews (51%) performed a formal assessment of the RoB of primary studies. Only 17 reviews (19%) linked the outcomes of quality assessment with the results of the review. According to the previous literature, our results corroborate the lack of guidance to incorporate the RoB assessment in the results of systematic reviews and meta-analyses. Our recommendation is to appraise MQ according to domains of RoB to rate the degree of credibility of the results of a research synthesis, as well as subgroup analysis or meta-regression as analytical methods to incorporate the quality assessment. Copyright © 2017 Elsevier Inc. All rights reserved.

What Is the Methodologic Quality of Human Therapy Studies in ISI Surgical Publications?

PubMed Central

Manterola, Carlos; Pineda, Viviana; Vial, Manuel; Losada, Héctor

2006-01-01

Objective: To determine the methodologic quality of therapy articles about humans published in ISI surgical journals, and to explore the association between methodologic quality, origin, and subject matter. Summary Background Data: It is supposed that ISI journals contain the best methodologic articles. Methods: This is a bibliometric study. All journals listed in the 2002 ISI under the subject heading of “Surgery” were included. A simple randomized sampling was conducted for selected journals (Annals of Surgery, The American Surgeon, Archives of Surgery, British Journal of Surgery, European Journal of Surgery, Journal of the American College of Surgeons, Surgery, and World Journal of Surgery). Published articles related to therapy on humans of the selected journals were reviewed and analyzed. All kinds of clinical designs were considered, excluding editorials, review articles, letters to the editor, and experimental studies. The variables considered were: place of origin, design, and the methodologic quality of articles, which was determined by applying a valid and reliable scale. The review was performed interchangeably and independently by 2 research teams. Descriptive and analytical statistics were used. Statistical significance was defined as P values less than 1%. Results: A total of 653 articles were studied. Studies came predominantly from the United States and Europe (43.6% and 36.8%, respectively). The subject areas most frequently found were digestive and hepatobiliopancreatic surgery (29.1% and 24.5%, respectively). Average and median methodologic quality scores of the entire series were 11.6 ± 4.9 points and 11 points, respectively. The association between methodologic quality and journals was determined. Also, the association between methodologic quality and origin was observed, but no association with subject area was verified. Conclusions: The methodologic quality of therapy articles published in the journals analyzed is low; however

Q-SEA – a tool for quality assessment of ethics analyses conducted as part of health technology assessments

PubMed Central

Scott, Anna Mae; Hofmann, Björn; Gutiérrez-Ibarluzea, Iñaki; Bakke Lysdahl, Kristin; Sandman, Lars; Bombard, Yvonne

2017-01-01

Introduction: Assessment of ethics issues is an important part of health technology assessments (HTA). However, in terms of existence of quality assessment tools, ethics for HTA is methodologically underdeveloped in comparison to other areas of HTA, such as clinical or cost effectiveness. Objective: To methodologically advance ethics for HTA by: (1) proposing and elaborating Q-SEA, the first instrument for quality assessment of ethics analyses, and (2) applying Q-SEA to a sample systematic review of ethics for HTA, in order to illustrate and facilitate its use. Methods: To develop a list of items for the Q-SEA instrument, we systematically reviewed the literature on methodology in ethics for HTA, reviewed HTA organizations’ websites, and solicited views from 32 experts in the field of ethics for HTA at two 2-day workshops. We subsequently refined Q-SEA through its application to an ethics analysis conducted for HTA. Results: Q-SEA instrument consists of two domains – the process domain and the output domain. The process domain consists of 5 elements: research question, literature search, inclusion/exclusion criteria, perspective, and ethics framework. The output domain consists of 5 elements: completeness, bias, implications, conceptual clarification, and conflicting values. Conclusion: Q-SEA is the first instrument for quality assessment of ethics analyses in HTA. Further refinements to the instrument to enhance its usability continue. PMID:28326147

Health economic assessment: a methodological primer.

PubMed

Simoens, Steven

2009-12-01

This review article aims to provide an introduction to the methodology of health economic assessment of a health technology. Attention is paid to defining the fundamental concepts and terms that are relevant to health economic assessments. The article describes the methodology underlying a cost study (identification, measurement and valuation of resource use, calculation of costs), an economic evaluation (type of economic evaluation, the cost-effectiveness plane, trial- and model-based economic evaluation, discounting, sensitivity analysis, incremental analysis), and a budget impact analysis. Key references are provided for those readers who wish a more advanced understanding of health economic assessments.

Health Economic Assessment: A Methodological Primer

PubMed Central

Simoens, Steven

2009-01-01

This review article aims to provide an introduction to the methodology of health economic assessment of a health technology. Attention is paid to defining the fundamental concepts and terms that are relevant to health economic assessments. The article describes the methodology underlying a cost study (identification, measurement and valuation of resource use, calculation of costs), an economic evaluation (type of economic evaluation, the cost-effectiveness plane, trial- and model-based economic evaluation, discounting, sensitivity analysis, incremental analysis), and a budget impact analysis. Key references are provided for those readers who wish a more advanced understanding of health economic assessments. PMID:20049237

Assessing quality of reports on randomized clinical trials in nursing journals.

PubMed

Parent, Nicole; Hanley, James A

2009-01-01

Several surveys have presented the quality of reports on randomized clinical trials (RCTs) published in general and specialty medical journals. The aim of these surveys was to raise scientific consciousness on methodological aspects pertaining to internal and external validity. These reviews have suggested that the methodological quality could be improved. We conducted a survey of reports on RCTs published in nursing journals to assess their methodological quality. The features we considered included sample size, flow of participants, assessment of baseline comparability, randomization, blinding, and statistical analysis. We collected data from all reports of RCTs published between January 1994 and December 1997 in Applied Nursing Research, Heart & Lung and Nursing Research. We hand-searched the journals and included all 54 articles in which authors reported that individuals have been randomly allocated to distinct groups. We collected data using a condensed form of the Consolidated Standards of Reporting Trials (CONSORT) statement for structured reporting of RCTs (Begg et al., 1996). Sample size calculations were included in only 22% of the reports. Only 48% of the reports provided information about the type of randomization, and a mere 22% described blinding strategies. Comparisons of baseline characteristics using hypothesis tests were abusively produced in more than 76% of the reports. Excessive use and unstructured reports of significance testing were common (59%), and all reports failed to provide magnitude of treatment differences with confidence intervals. Better methodological quality in reports of RCTs will contribute to increase the standards of nursing research.

A quality evaluation methodology of health web-pages for non-professionals.

PubMed

Currò, Vincenzo; Buonuomo, Paola Sabrina; Onesimo, Roberta; de Rose, Paola; Vituzzi, Andrea; di Tanna, Gian Luca; D'Atri, Alessandro

2004-06-01

The proposal of an evaluation methodology for determining the quality of healthcare web sites for the dissemination of medical information to non-professionals. Three (macro) factors are considered for the quality evaluation: medical contents, accountability of the authors, and usability of the web site. Starting from two results in the literature the problem of whether or not to introduce a weighting function has been investigated. This methodology has been validated on a specialized information content, i.e., sore throats, due to the large interest such a topic enjoys with target users. The World Wide Web was accessed using a meta-search system merging several search engines. A statistical analysis was made to compare the proposed methodology with the obtained ranks of the sample web pages. The statistical analysis confirms that the variables examined (per item and sub factor) show substantially similar ranks and are capable of contributing to the evaluation of the main quality macro factors. A comparison between the aggregation functions in the proposed methodology (non-weighted averages) and the weighting functions, derived from the literature, allowed us to verify the suitability of the method. The proposed methodology suggests a simple approach which can quickly award an overall quality score for medical web sites oriented to non-professionals.

Rating the methodological quality in systematic reviews of studies on measurement properties: a scoring system for the COSMIN checklist.

PubMed

Terwee, Caroline B; Mokkink, Lidwine B; Knol, Dirk L; Ostelo, Raymond W J G; Bouter, Lex M; de Vet, Henrica C W

2012-05-01

The COSMIN checklist is a standardized tool for assessing the methodological quality of studies on measurement properties. It contains 9 boxes, each dealing with one measurement property, with 5-18 items per box about design aspects and statistical methods. Our aim was to develop a scoring system for the COSMIN checklist to calculate quality scores per measurement property when using the checklist in systematic reviews of measurement properties. The scoring system was developed based on discussions among experts and testing of the scoring system on 46 articles from a systematic review. Four response options were defined for each COSMIN item (excellent, good, fair, and poor). A quality score per measurement property is obtained by taking the lowest rating of any item in a box ("worst score counts"). Specific criteria for excellent, good, fair, and poor quality for each COSMIN item are described. In defining the criteria, the "worst score counts" algorithm was taken into consideration. This means that only fatal flaws were defined as poor quality. The scores of the 46 articles show how the scoring system can be used to provide an overview of the methodological quality of studies included in a systematic review of measurement properties. Based on experience in testing this scoring system on 46 articles, the COSMIN checklist with the proposed scoring system seems to be a useful tool for assessing the methodological quality of studies included in systematic reviews of measurement properties.

Prediction of ground water quality index to assess suitability for drinking purposes using fuzzy rule-based approach

NASA Astrophysics Data System (ADS)

Gorai, A. K.; Hasni, S. A.; Iqbal, Jawed

2016-11-01

Groundwater is the most important natural resource for drinking water to many people around the world, especially in rural areas where the supply of treated water is not available. Drinking water resources cannot be optimally used and sustained unless the quality of water is properly assessed. To this end, an attempt has been made to develop a suitable methodology for the assessment of drinking water quality on the basis of 11 physico-chemical parameters. The present study aims to select the fuzzy aggregation approach for estimation of the water quality index of a sample to check the suitability for drinking purposes. Based on expert's opinion and author's judgement, 11 water quality (pollutant) variables (Alkalinity, Dissolved Solids (DS), Hardness, pH, Ca, Mg, Fe, Fluoride, As, Sulphate, Nitrates) are selected for the quality assessment. The output results of proposed methodology are compared with the output obtained from widely used deterministic method (weighted arithmetic mean aggregation) for the suitability of the developed methodology.

Systematic review and quality assessment of cost-effectiveness analysis of pharmaceutical therapies for advanced colorectal cancer.

PubMed

Leung, Henry W C; Chan, Agnes L F; Leung, Matthew S H; Lu, Chin-Li

2013-04-01

To systematically review and assess the quality of cost-effectiveness analyses (CEAs) of pharmaceutical therapies for metastatic colorectal cancer (mCRC). The MEDLINE, EMBASE, Cochrane, and EconLit databases were searched for the Medical Subject Headings or text key words quality-adjusted, QALY, life-year gained (LYG), and cost-effectiveness (January 1, 1999-December 31, 2009). Original CEAs of mCRC pharmacotherapy published in English were included. CEAs that measured health effects in units other than quality-adjusted life years or LYG and letters to the editor, case reports, posters, and editorials were excluded. Each article was independently assessed by 2 trained reviewers according to a quality checklist created by the Panel on Cost-Effectiveness in Health and Medicine. Twenty-four CEA studies pertaining to pharmaceutical therapies for mCRC were identified. All studies showed a wide variation in methodologic approaches, which resulted in a different range of incremental cost-effectiveness ratios reported for each regimen. We found common methodologic flaws in a significant number of CEA studies, including lack of clear description for critique of data quality; lack of method for adjusting costs for inflation and methods for obtaining expert judgment; no results of model validation; wide differences in the types of perspective, time horizon, study design, cost categories, and effect outcomes; and no quality assessment of data (cost and effectiveness) for the interventions evaluation. This study has shown a wide variation in the methodology and quality of cost-effectiveness analysis for mCRC. Improving quality and harmonization of CEA for cancer treatment is needed. Further study is suggested to assess the quality of CEA methodology outside the mCRC disease state.

Rating the methodological quality of single-subject designs and n-of-1 trials: introducing the Single-Case Experimental Design (SCED) Scale.

PubMed

Tate, Robyn L; McDonald, Skye; Perdices, Michael; Togher, Leanne; Schultz, Regina; Savage, Sharon

2008-08-01

Rating scales that assess methodological quality of clinical trials provide a means to critically appraise the literature. Scales are currently available to rate randomised and non-randomised controlled trials, but there are none that assess single-subject designs. The Single-Case Experimental Design (SCED) Scale was developed for this purpose and evaluated for reliability. Six clinical researchers who were trained and experienced in rating methodological quality of clinical trials developed the scale and participated in reliability studies. The SCED Scale is an 11-item rating scale for single-subject designs, of which 10 items are used to assess methodological quality and use of statistical analysis. The scale was developed and refined over a 3-year period. Content validity was addressed by identifying items to reduce the main sources of bias in single-case methodology as stipulated by authorities in the field, which were empirically tested against 85 published reports. Inter-rater reliability was assessed using a random sample of 20/312 single-subject reports archived in the Psychological Database of Brain Impairment Treatment Efficacy (PsycBITE). Inter-rater reliability for the total score was excellent, both for individual raters (overall ICC = 0.84; 95% confidence interval 0.73-0.92) and for consensus ratings between pairs of raters (overall ICC = 0.88; 95% confidence interval 0.78-0.95). Item reliability was fair to excellent for consensus ratings between pairs of raters (range k = 0.48 to 1.00). The results were replicated with two independent novice raters who were trained in the use of the scale (ICC = 0.88, 95% confidence interval 0.73-0.95). The SCED Scale thus provides a brief and valid evaluation of methodological quality of single-subject designs, with the total score demonstrating excellent inter-rater reliability using both individual and consensus ratings. Items from the scale can also be used as a checklist in the design, reporting and critical

Methodologic quality of meta-analyses and systematic reviews on the Mediterranean diet and cardiovascular disease outcomes: a review.

PubMed

Huedo-Medina, Tania B; Garcia, Marissa; Bihuniak, Jessica D; Kenny, Anne; Kerstetter, Jane

2016-03-01

Several systematic reviews/meta-analyses published within the past 10 y have examined the associations of Mediterranean-style diets (MedSDs) on cardiovascular disease (CVD) risk. However, these reviews have not been evaluated for satisfying contemporary methodologic quality standards. This study evaluated the quality of recent systematic reviews/meta-analyses on MedSD and CVD risk outcomes by using an established methodologic quality scale. The relation between review quality and impact per publication value of the journal in which the article had been published was also evaluated. To assess compliance with current standards, we applied a modified version of the Assessment of Multiple Systematic Reviews (AMSTARMedSD) quality scale to systematic reviews/meta-analyses retrieved from electronic databases that had met our selection criteria: 1) used systematic or meta-analytic procedures to review the literature, 2) examined MedSD trials, and 3) had MedSD interventions independently or combined with other interventions. Reviews completely satisfied from 8% to 75% of the AMSTARMedSD items (mean ± SD: 31.2% ± 19.4%), with those published in higher-impact journals having greater quality scores. At a minimum, 60% of the 24 reviews did not disclose full search details or apply appropriate statistical methods to combine study findings. Only 5 of the reviews included participant or study characteristics in their analyses, and none evaluated MedSD diet characteristics. These data suggest that current meta-analyses/systematic reviews evaluating the effect of MedSD on CVD risk do not fully comply with contemporary methodologic quality standards. As a result, there are more research questions to answer to enhance our understanding of how MedSD affects CVD risk or how these effects may be modified by the participant or MedSD characteristics. To clarify the associations between MedSD and CVD risk, future meta-analyses and systematic reviews should not only follow methodologic

ASSESSING BIOACCUMULATION FOR DERIVING NATIONAL HUMAN HEALTH WATER QUALITY CRITERIA

EPA Science Inventory

The United States Environmental Protection Agency is revising its methodology for deriving national ambient water quality criteria (AWQC) to protect human health. A component of this guidance involves assessing the potential for chemical bioaccumulation in commonly consumed fish ...

Assessment Methodology for Process Validation Lifecycle Stage 3A.

PubMed

Sayeed-Desta, Naheed; Pazhayattil, Ajay Babu; Collins, Jordan; Chen, Shu; Ingram, Marzena; Spes, Jana

2017-07-01

The paper introduces evaluation methodologies and associated statistical approaches for process validation lifecycle Stage 3A. The assessment tools proposed can be applied to newly developed and launched small molecule as well as bio-pharma products, where substantial process and product knowledge has been gathered. The following elements may be included in Stage 3A: number of 3A batch determination; evaluation of critical material attributes, critical process parameters, critical quality attributes; in vivo in vitro correlation; estimation of inherent process variability (IPV) and PaCS index; process capability and quality dashboard (PCQd); and enhanced control strategy. US FDA guidance on Process Validation: General Principles and Practices, January 2011 encourages applying previous credible experience with suitably similar products and processes. A complete Stage 3A evaluation is a valuable resource for product development and future risk mitigation of similar products and processes. Elements of 3A assessment were developed to address industry and regulatory guidance requirements. The conclusions made provide sufficient information to make a scientific and risk-based decision on product robustness.

Objective assessment of MPEG-2 video quality

NASA Astrophysics Data System (ADS)

Gastaldo, Paolo; Zunino, Rodolfo; Rovetta, Stefano

2002-07-01

The increasing use of video compression standards in broadcasting television systems has required, in recent years, the development of video quality measurements that take into account artifacts specifically caused by digital compression techniques. In this paper we present a methodology for the objective quality assessment of MPEG video streams by using circular back-propagation feedforward neural networks. Mapping neural networks can render nonlinear relationships between objective features and subjective judgments, thus avoiding any simplifying assumption on the complexity of the model. The neural network processes an instantaneous set of input values, and yields an associated estimate of perceived quality. Therefore, the neural-network approach turns objective quality assessment into adaptive modeling of subjective perception. The objective features used for the estimate are chosen according to the assessed relevance to perceived quality and are continuously extracted in real time from compressed video streams. The overall system mimics perception but does not require any analytical model of the underlying physical phenomenon. The capability to process compressed video streams represents an important advantage over existing approaches, like avoiding the stream-decoding process greatly enhances real-time performance. Experimental results confirm that the system provides satisfactory, continuous-time approximations for actual scoring curves concerning real test videos.

A critical appraisal of the methodology and quality of evidence of systematic reviews and meta-analyses of traditional Chinese medical nursing interventions: a systematic review of reviews

PubMed Central

Jin, Ying-Hui; Wang, Guo-Hao; Sun, Yi-Rong; Li, Qi; Zhao, Chen; Li, Ge; Si, Jin-Hua; Li, Yan; Lu, Cui; Shang, Hong-Cai

2016-01-01

Objective To assess the methodology and quality of evidence of systematic reviews and meta-analyses of traditional Chinese medical nursing (TCMN) interventions in Chinese journals. These interventions include acupressure, massage, Tai Chi, Qi Gong, electroacupuncture and use of Chinese herbal medicines—for example, in enemas, foot massage and compressing the umbilicus. Design A systematic literature search for systematic reviews and meta-analyses of TCMN interventions was performed. Review characteristics were extracted. The methodological quality and the quality of the evidence were evaluated using the Assessment of Multiple Systematic Reviews (AMSTAR) and Grading of Recommendations Assessment, Development and Evaluation (GRADE) approaches. Result We included 20 systematic reviews and meta-analyses, and a total of 11 TCMN interventions were assessed in the 20 reviews. The compliance with AMSTAR checklist items ranged from 4.5 to 8 and systematic reviews/meta-analyses were, on average, of medium methodological quality. The quality of the evidence we assessed ranged from very low to moderate; no high-quality evidence was found. The top two causes for downrating confidence in effect estimates among the 31 bodies of evidence assessed were the risk of bias and inconsistency. Conclusions There is room for improvement in the methodological quality of systematic reviews/meta-analyses of TCMN interventions published in Chinese journals. Greater efforts should be devoted to ensuring a more comprehensive search strategy, clearer specification of the interventions of interest in the eligibility criteria and identification of meaningful outcomes for clinicians and patients (consumers). The overall quality of evidence among reviews remains suboptimal, which raise concerns about their roles in influencing clinical practice. Thus, the conclusions in reviews we assessed must be treated with caution and their roles in influencing clinical practice should be limited. A critical

Proposal and application of a regional methodology of comparative risk assessment for potentially contaminated sites.

PubMed

Marzocchini, Manrico; Tatàno, Fabio; Moretti, Michela Simona; Antinori, Caterina; Orilisi, Stefano

2018-06-05

A possible approach for determining soil and groundwater quality criteria for contaminated sites is the comparative risk assessment. Originating from but not limited to Italian interest in a decentralised (regional) implementation of comparative risk assessment, this paper first addresses the proposal of an original methodology called CORIAN REG-M , which was created with initial attention to the context of potentially contaminated sites in the Marche Region (Central Italy). To deepen the technical-scientific knowledge and applicability of the comparative risk assessment, the following characteristics of the CORIAN REG-M methodology appear to be relevant: the simplified but logical assumption of three categories of factors (source and transfer/transport of potential contamination, and impacted receptors) within each exposure pathway; the adaptation to quality and quantity of data that are available or derivable at the given scale of concern; the attention to a reliable but unsophisticated modelling; the achievement of a conceptual linkage to the absolute risk assessment approach; and the potential for easy updating and/or refining of the methodology. Further, the application of the CORIAN REG-M methodology to some case-study sites located in the Marche Region indicated the following: a positive correlation can be expected between air and direct contact pathway scores, as well as between individual pathway scores and the overall site scores based on a root-mean-square algorithm; the exposure pathway, which presents the highest variability of scores, tends to be dominant at sites with the highest computed overall site scores; and the adoption of a root-mean-square algorithm can be expected to emphasise the overall site scoring.

Couple Attachment and Relationship Duration in Psychotherapy Patients: Exploring a New Methodology of Assessment

ERIC Educational Resources Information Center

Sochos, Antigonos

2014-01-01

The couple relationship is an essential source of support for individuals undergoing psychological treatment and the aim of this study was to apply a new methodology in assessing the quality of such support. A theoretically informed thematic analysis of interview transcripts was conducted, triangulated by quantitative data. Twenty-one brief…

Characteristics and Methodological Quality of Meta-Analyses on Hypertension Treatments-A Cross-Sectional Study.

PubMed

Wu, Xin Yin; Du, Xin Jian; Ho, Robin S T; Lee, Clarence C Y; Yip, Benjamin H K; Wong, Martin C S; Wong, Samuel Y S; Chung, Vincent C H

2017-02-01

Methodological quality of meta-analyses on hypertension treatments can affect treatment decision-making. The authors conducted a cross-sectional study to investigate the methodological quality of meta-analyses on hypertension treatments. One hundred and fifty-eight meta-analyses were identified. Overall, methodological quality was unsatisfactory in the following aspects: comprehensive reporting of financial support (1.9%), provision of included and excluded lists of studies (22.8%), inclusion of grey literature (27.2%), and inclusion of protocols (32.9%). The 126 non-Cochrane meta-analyses had poor performance on almost all the methodological items. Non-Cochrane meta-analyses focused on nonpharmacologic treatments were more likely to consider scientific quality of included studies when making conclusions. The 32 Cochrane meta-analyses generally had good methodological quality except for comprehensive reporting of the sources of support. These results highlight the need for cautious interpretation of these meta-analyses, especially among physicians and policy makers when guidelines are formulated. Future meta-analyses should pay attention to improving these methodological aspects. ©2016 Wiley Periodicals, Inc.

Reporting characteristics of meta-analyses in orthodontics: methodological assessment and statistical recommendations.

PubMed

Papageorgiou, Spyridon N; Papadopoulos, Moschos A; Athanasiou, Athanasios E

2014-02-01

Ideally meta-analyses (MAs) should consolidate the characteristics of orthodontic research in order to produce an evidence-based answer. However severe flaws are frequently observed in most of them. The aim of this study was to evaluate the statistical methods, the methodology, and the quality characteristics of orthodontic MAs and to assess their reporting quality during the last years. Electronic databases were searched for MAs (with or without a proper systematic review) in the field of orthodontics, indexed up to 2011. The AMSTAR tool was used for quality assessment of the included articles. Data were analyzed with Student's t-test, one-way ANOVA, and generalized linear modelling. Risk ratios with 95% confidence intervals were calculated to represent changes during the years in reporting of key items associated with quality. A total of 80 MAs with 1086 primary studies were included in this evaluation. Using the AMSTAR tool, 25 (27.3%) of the MAs were found to be of low quality, 37 (46.3%) of medium quality, and 18 (22.5%) of high quality. Specific characteristics like explicit protocol definition, extensive searches, and quality assessment of included trials were associated with a higher AMSTAR score. Model selection and dealing with heterogeneity or publication bias were often problematic in the identified reviews. The number of published orthodontic MAs is constantly increasing, while their overall quality is considered to range from low to medium. Although the number of MAs of medium and high level seems lately to rise, several other aspects need improvement to increase their overall quality.

Reporting and Methodological Quality of Randomised Controlled Trials in Vascular and Endovascular Surgery.

PubMed

Hajibandeh, S; Hajibandeh, S; Antoniou, G A; Green, P A; Maden, M; Torella, F

2015-11-01

Randomised controlled trials (RCTs) are subject to bias if they lack methodological quality. Moreover, optimal and transparent reporting of RCT findings aids their critical appraisal and interpretation. The aim of this study was to ascertain whether the methodological and reporting quality of RCTs in vascular and endovascular surgery is improving. The most recent 75 and oldest 75 RCTs published in leading journals over a 10-year period (2003-2012) were identified. The reporting quality and methodological quality data of the old and new RCTs were extracted and compared. The former was analysed using the Consolidated Standards of Reporting Trials (CONSORT) statement, the latter with the Scottish Intercollegiate Guidelines Network (SIGN) checklist. Reporting quality measured by CONSORT was better in the new studies than in the old studies (0.68 [95% CI, 0.66-0.7] vs. 0.60 [95% CI, 0.58-0.62], p < .001); however, both new and old studies had similar methodological quality measured by SIGN (0.9 [IQR 0.1] vs. .09 [IQR: 0.2], p = .787). Unlike clinical items, the methodological items of the CONSORT statement were not well reported in old and new RCTs. More trials in the new group were endovascular related (33.33% vs. 17.33%, p = .038) and industry sponsored (28% vs. 6.67%, p = .001). Despite some progress, there remains room for improvement in the reporting quality of RCTs in vascular and endovascular surgery. The methodological quality of recent RCTs is similar to that of trials performed >10 years ago. Copyright © 2015 European Society for Vascular Surgery. Published by Elsevier Ltd. All rights reserved.

Methodological quality evaluation of systematic reviews or meta-analyses on ERCC1 in non-small cell lung cancer: a systematic review.

PubMed

Tao, Huan; Zhang, Yueyuan; Li, Qian; Chen, Jin

2017-11-01

To assess the methodological quality of systematic reviews (SRs) or meta-analysis concerning the predictive value of ERCC1 in platinum chemotherapy of non-small cell lung cancer. We searched the PubMed, EMbase, Cochrane library, international prospective register of systematic reviews, Chinese BioMedical Literature Database, China National Knowledge Infrastructure, Wan Fang and VIP database for SRs or meta-analysis. The methodological quality of included literatures was evaluated by risk of bias in systematic review (ROBIS) scale. Nineteen eligible SRs/meta-analysis were included. The most frequently searched databases were EMbase (74%), PubMed, Medline and CNKI. Fifteen SRs did additional retrieval manually, but none of them retrieved the registration platform. 47% described the two-reviewers model in the screening for eligible original articles, and seven SRs described the two reviewers to extract data. In methodological quality assessment, inter-rater reliability Kappa was 0.87 between two reviewers. Research question were well related to all SRs in phase 1 and the eligibility criteria was suitable for each SR, and rated as 'low' risk bias. But the 'high' risk bias existed in all the SRs regarding methods used to identify and/or select studies, and data collection and study appraisal. More than two-third of SRs or meta-analysis were finished with high risk of bias in the synthesis, findings and the final phase. The study demonstrated poor methodological quality of SRs/meta-analysis assessing the predictive value of ERCC1 in chemotherapy among the NSCLC patients, especially the high performance bias. Registration or publishing the protocol is recommended in future research.

[Method for the quality assessment of data collection processes in epidemiological studies].

PubMed

Schöne, G; Damerow, S; Hölling, H; Houben, R; Gabrys, L

2017-10-01

For a quantitative evaluation of primary data collection processes in epidemiological surveys based on accompaniments and observations (in the field), there is no description of test criteria and methodologies in relevant literature and thus no known application in practice. Therefore, methods need to be developed and existing procedures adapted. The aim was to identify quality-relevant developments within quality dimensions by means of inspection points (quality indicators) during the process of data collection. As a result we seek to implement and establish a methodology for the assessment of overall survey quality supplementary to standardized data analyses. Monitors detect deviations from standard primary data collection during site visits by applying standardized checklists. Quantitative results - overall and for each dimension - are obtained by numerical calculation of quality indicators. Score results are categorized and color coded. This visual prioritization indicates necessity for intervention. The results obtained give clues regarding the current quality of data collection. This allows for the identification of such sections where interventions for quality improvement are needed. In addition, process quality development can be shown over time on an intercomparable basis. This methodology for the evaluation of data collection quality can identify deviations from norms, focalize quality analyses and help trace causes for significant deviations.

Methodological quality of systematic reviews referenced in clinical practice guidelines for the treatment of opioid use disorder.

PubMed

Ross, Andrew; Rankin, Justin; Beaman, Jason; Murray, Kelly; Sinnett, Philip; Riddle, Ross; Haskins, Jordan; Vassar, Matt

2017-01-01

With efforts to combat opioid use disorder, there is an increased interest in clinical practice guidelines (CPGs) for opioid use disorder treatments. No literature exists examining the quality of systematic reviews used in opioid use disorder CPGs. This study aims to describe the methodological quality and reporting clarity of systematic reviews (SRs) used to create CPGs for opioid use disorder. From June to July 2016 guideline clearinghouses and medical literature databases were searched for relevant CPGs used in the treatment of opioid use disorder. Included CPGs must have been recognized by a national organization. SRs from the reference section of each CPG was scored by using AMSTAR (a measurement tool to assess the methodological quality of systematic reviews) tool and PRISMA (preferred reporting items for systematic reviews and meta-analyses) checklist. Seventeen CPGs from 2006-2016 were included in the review. From these, 57 unique SRs were extracted. SRS comprised 0.28% to 17.92% of all references found in the CPGs. All SRs obtained moderate or high methodological quality score on the AMSTAR tool. All reviews met at least 70% of PRISMA criteria. In PRISMA, underperforming areas included accurate title labeling, protocol registration, and risk of bias. Underperforming areas in AMSTAR included conflicts of interest, funding, and publication bias. A positive correlation was found between AMSTAR and PRISMA scores (r = .79). Although the SRs in the CPGs were of good quality, there are still areas for improvement. Systematic reviewers should consult PRISMA and AMSTAR when conducting and reporting reviews. It is important for CPG developers to consider methodological quality as a factor when developing CPG recommendations, recognizing that the quality of systematic reviews underpinning guidelines does not necessarily correspond to the quality of the guideline itself.

78 FR 63972 - Notice of Proposed Methodology for the 2014 Delaware River and Bay Water Quality Assessment Report

Federal Register 2010, 2011, 2012, 2013, 2014

2013-10-25

... Water Quality Assessment Report AGENCY: Delaware River Basin Commission. ACTION: Notice. SUMMARY: Notice....us , with ``Water Quality Assessment 2014'' as the subject line; via fax to 609-883-9522; via U.S. Mail to DRBC, Attn: Water Quality Assessment 2014, P.O. Box 7360, West Trenton, NJ 08628-0360; via...

Feedback Effects of Teaching Quality Assessment: Macro and Micro Evidence

ERIC Educational Resources Information Center

Bianchini, Stefano

2014-01-01

This study investigates the feedback effects of teaching quality assessment. Previous literature looked separately at the evolution of individual and aggregate scores to understand whether instructors and university performance depends on its past evaluation. I propose a new quantitative-based methodology, combining statistical distributions and…

Analysis of Alternatives for Risk Assessment Methodologies and Tools

DOE Office of Scientific and Technical Information (OSTI.GOV)

Nachtigal, Noel M.; Fruetel, Julia A.; Gleason, Nathaniel J.

The purpose of this document is to provide a basic overview and understanding of risk assessment methodologies and tools from the literature and to assess the suitability of these methodologies and tools for cyber risk assessment. Sandia National Laboratories (SNL) performed this review in support of risk modeling activities performed for the Stakeholder Engagement and Cyber Infrastructure Resilience (SECIR) division of the Department of Homeland Security (DHS) Office of Cybersecurity and Communications (CS&C). The set of methodologies and tools covered in this document is not intended to be exhaustive; instead, it focuses on those that are commonly used in themore » risk assessment community. The classification of methodologies and tools was performed by a group of analysts with experience in risk analysis and cybersecurity, and the resulting analysis of alternatives has been tailored to address the needs of a cyber risk assessment.« less

Patient-perceived hospital service quality: an empirical assessment.

PubMed

Pai, Yogesh P; Chary, Satyanarayana T; Pai, Rashmi Yogesh

2018-02-12

Purpose The purpose of this paper is to appraise Pai and Chary's (2016) conceptual framework for measuring patient-perceived hospital service quality (HSQ). Design/methodology/approach A structured questionnaire was used to obtain data from teaching, public and corporate hospital patients. Several tests were conducted to assess the instrument's reliability and validity. Pai and Chary's (2016) nine dimensions for measuring HSQ were examined in this paper. Findings The tests confirm that Pai and Chary's (2016) conceptual framework is reliable and valid. The study also establishes that the nine dimensions measure HSQ. Practical implications The framework empowers managers to assess service quality in any hospital settings, corporate, public and teaching, using an approach that is superior to the existing HSQ scales. Originality/value This paper helps researchers and practitioners to assess HSQ from patient perspectives in any hospital setting.

Systematic reviews and meta-analyses on psoriasis: role of funding sources, conflict of interest and bibliometric indices as predictors of methodological quality.

PubMed

Gómez-García, F; Ruano, J; Aguilar-Luque, M; Gay-Mimbrera, J; Maestre-Lopez, B; Sanz-Cabanillas, J L; Carmona-Fernández, P J; González-Padilla, M; Vélez García-Nieto, A; Isla-Tejera, B

2017-06-01

The quality of systematic reviews and meta-analyses on psoriasis, a chronic inflammatory skin disease that severely impairs quality of life and is associated with high costs, remains unknown. To assess the methodological quality of systematic reviews published on psoriasis. After a comprehensive search in MEDLINE, Embase and the Cochrane Database (PROSPERO: CDR42016041611), the quality of studies was assessed by two raters using the Assessment of Multiple Systematic Reviews (AMSTAR) tool. Article metadata and journal-related bibliometric indices were also obtained. Systematic reviews were classified as low (0-4), moderate (5-8) or high (9-11) quality. A prediction model for methodological quality was fitted using principal component and multivariate ordinal logistic regression analyses. We classified 220 studies as high (17·2%), moderate (55·0%) or low (27·8%) quality. Lower compliance rates were found for AMSTAR question (Q)5 (list of studies provided, 11·4%), Q10 (publication bias assessed, 27·7%), Q4 (status of publication included, 39·5%) and Q1 (a priori design provided, 40·9%). Factors such as meta-analysis inclusion [odds ratio (OR) 6·22; 95% confidence interval (CI) 2·78-14·86], funding by academic institutions (OR 2·90, 95% CI 1·11-7·89), Article Influence score (OR 2·14, 95% CI 1·05-6·67), 5-year impact factor (OR 1·34, 95% CI 1·02-1·40) and article page count (OR 1·08, 95% CI 1·02-1·15) significantly predicted higher quality. A high number of authors with a conflict of interest (OR 0·90, 95% CI 0·82-0·99) was significantly associated with lower quality. The methodological quality of systematic reviews published about psoriasis remains suboptimal. The type of funding sources and author conflicts may compromise study quality, increasing the risk of bias. © 2017 British Association of Dermatologists.

Methodological review: quality of randomized controlled trials in health literacy.

PubMed

Brainard, Julii; Wilsher, Stephanie Howard; Salter, Charlotte; Loke, Yoon Kong

2016-07-11

The growing move towards patient-centred care has led to substantial research into improving the health literacy skills of patients and members of the public. Hence, there is a pressing need to assess the methodology used in contemporary randomized controlled trials (RCTs) of interventions directed at health literacy, in particular the quality (risk of bias), and the types of outcomes reported. We conducted a systematic database search for RCTs involving interventions directed at health literacy in adults, published from 2009 to 2014. The Cochrane Risk of Bias tool was used to assess quality of RCT implementation. We also checked the sample size calculation for primary outcomes. Reported evidence of efficacy (statistical significance) was extracted for intervention outcomes in any of three domains of effect: knowledge, behaviour, health status. Demographics of intervention participants were also extracted, including socioeconomic status. We found areas of methodological strength (good randomization and allocation concealment), but areas of weakness regarding blinding of participants, people delivering the intervention and outcomes assessors. Substantial attrition (losses by monitoring time point) was seen in a third of RCTs, potentially leading to insufficient power to obtain precise estimates of intervention effect on primary outcomes. Most RCTs showed that the health literacy interventions had some beneficial effect on knowledge outcomes, but this was typically for less than 3 months after intervention end. There were far fewer reports of significant improvements in substantive patient-oriented outcomes, such as beneficial effects on behavioural change or health (clinical) status. Most RCTs featured participants from vulnerable populations. Our evaluation shows that health literacy trial design, conduct and reporting could be considerably improved, particularly by reducing attrition and obtaining longer follow-up. More meaningful RCTs would also result if health

How to Select a Questionnaire with a Good Methodological Quality?

PubMed

Paiva, Saul Martins; Perazzo, Matheus de França; Ortiz, Fernanda Ruffo; Pordeus, Isabela Almeida; Martins-Júnior, Paulo Antônio

2018-01-01

In the last decades, several instruments have been used to evaluate the impact of oral health problems on the oral health-related quality of life (OHRQoL) of individuals. However, some instruments lack thorough methodological validation or present conceptual differences that hinder comparisons with instruments. Thus, it can be difficult to clinicians and researchers to select a questionnaire that accurately reflect what are really meaningful to individuals. This short communication aimed to discuss the importance of use an appropriate checklist to select an instrument with a good methodological quality. The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist was developed to provide tools for evidence-based instrument selection. The COSMIN checklist comprises ten boxes that evaluate whether a study meets the standard for good methodological quality and two additional boxes to meet studies that use the Item Response Theory method and general requirements for results generalization, resulting in four steps to be followed. In this way, it is required at least some expertise in psychometrics or clinimetrics to a wide-ranging use of this checklist. The COSMIN applications include its use to ensure the standardization of cross-cultural adaptations and safer comparisons between measurement studies and evaluation of methodological quality of systematic reviews of measurement properties. Also, it can be used by students when training about measurement properties and by editors and reviewers when revising manuscripts on this topic. The popularization of COSMIN checklist is therefore necessary to improve the selection and evaluation of health measurement instruments.

Evolution of the methodological quality of controlled clinical trials for myofascial trigger point treatments for the period 1978-2015: A systematic review.

PubMed

Stoop, Rahel; Clijsen, Ron; Leoni, Diego; Soldini, Emiliano; Castellini, Greta; Redaelli, Valentina; Barbero, Marco

2017-08-01

The methodological quality of controlled clinical trials (CCTs) of physiotherapeutic treatment modalities for myofascial trigger points (MTrP) has not been investigated yet. To detect the methodological quality of CCTs for physiotherapy treatments of MTrPs and demonstrating the possible increase over time. Systematic review. A systematic search was conducted in two databases, Physiotherapy Evidence Database (PEDro) and Medicine Medical Literature Analysis and Retrieval System online (MEDLINE), using the same keywords and selection procedure corresponding to pre-defined inclusion criteria. The methodological quality, assessed by the 11-item PEDro scale, served as outcome measure. The CCTs had to compare at least two interventions, where one intervention had to lay within the scope of physiotherapy. Participants had to be diagnosed with myofascial pain syndrome or trigger points (active or latent). A total of n = 230 studies was analysed. The cervico-thoracic region was the most frequently treated body part (n = 143). Electrophysical agent applications was the most frequent intervention. The average methodological quality reached 5.5 on the PEDro scale. A total of n = 6 studies scored the value of 9. The average PEDro score increased by 0.7 points per decade between 1978 and 2015. The average PEDro score of CCTs for MTrP treatments does not reach the cut-off of 6 proposed for moderate to high methodological quality. Nevertheless, a promising trend towards an increase of the average methodological quality of CCTs for MTrPs was recorded. More high-quality CCT studies with thorough research procedures are recommended to enhance methodological quality. Copyright © 2017 The Author(s). Published by Elsevier Ltd.. All rights reserved.

Methodological Quality of Randomized Clinical Trials of Respiratory Physiotherapy in Coronary Artery Bypass Grafting Patients in the Intensive Care Unit: a Systematic Review

PubMed Central

Lorscheitter, Jaqueline; Stein, Cinara; Plentz, Rodrigo Della Méa

2017-01-01

Objective To assess methodological quality of the randomized controlled trials of physiotherapy in patients undergoing coronary artery bypass grafting in the intensive care unit. Methods The studies published until May 2015, in MEDLINE, Cochrane and PEDro were included. The primary outcome extracted was proper filling of the Cochrane Collaboration's tool's items and the secondary was suitability to the requirements of the CONSORT Statement and its extension. Results From 807 studies identified, 39 were included. Most at CONSORT items showed a better adequacy after the statement's publication. Studies with positive outcomes presented better methodological quality. Conclusion The methodological quality of the studies has been improving over the years. However, many aspects can still be better designed. PMID:28977205

Space Transportation Operations: Assessment of Methodologies and Models

NASA Technical Reports Server (NTRS)

Joglekar, Prafulla

2001-01-01

The systems design process for future space transportation involves understanding multiple variables and their effect on lifecycle metrics. Variables such as technology readiness or potential environmental impact are qualitative, while variables such as reliability, operations costs or flight rates are quantitative. In deciding what new design concepts to fund, NASA needs a methodology that would assess the sum total of all relevant qualitative and quantitative lifecycle metrics resulting from each proposed concept. The objective of this research was to review the state of operations assessment methodologies and models used to evaluate proposed space transportation systems and to develop recommendations for improving them. It was found that, compared to the models available from other sources, the operations assessment methodology recently developed at Kennedy Space Center has the potential to produce a decision support tool that will serve as the industry standard. Towards that goal, a number of areas of improvement in the Kennedy Space Center's methodology are identified.

Space Transportation Operations: Assessment of Methodologies and Models

NASA Technical Reports Server (NTRS)

Joglekar, Prafulla

2002-01-01

The systems design process for future space transportation involves understanding multiple variables and their effect on lifecycle metrics. Variables such as technology readiness or potential environmental impact are qualitative, while variables such as reliability, operations costs or flight rates are quantitative. In deciding what new design concepts to fund, NASA needs a methodology that would assess the sum total of all relevant qualitative and quantitative lifecycle metrics resulting from each proposed concept. The objective of this research was to review the state of operations assessment methodologies and models used to evaluate proposed space transportation systems and to develop recommendations for improving them. It was found that, compared to the models available from other sources, the operations assessment methodology recently developed at Kennedy Space Center has the potential to produce a decision support tool that will serve as the industry standard. Towards that goal, a number of areas of improvement in the Kennedy Space Center's methodology are identified.

International drug price comparisons: quality assessment.

PubMed

Machado, Márcio; O'Brodovich, Ryan; Krahn, Murray; Einarson, Thomas R

2011-01-01

To quantitatively summarize results (i.e., prices and affordability) reported from international drug price comparison studies and assess their methodological quality. A systematic search of the most relevant databases-Medline, Embase, International Pharmaceutical Abstracts (IPA), and Scopus, from their inception to May 2009-was conducted to identify original research comparing international drug prices. International drug price information was extracted and recorded from accepted papers. Affordability was reported as drug prices adjusted for income. Study quality was assessed using six criteria: use of similar countries, use of a representative sample of drugs, selection of specific types of prices, identification of drug packaging, different weights on price indices, and the type of currency conversion used. Of the 1 828 studies identified, 21 were included. Only one study adequately addressed all quality issues. A large variation in study quality was observed due to the many methods used to conduct the drug price comparisons, such as different indices, economic parameters, price types, basket of drugs, and more. Thus, the quality of published studies was considered poor. Results varied across studies, but generally, higher income countries had higher drug prices. However, after adjusting drug prices for affordability, higher income countries had more affordable prices than lower income countries. Differences between drug prices and affordability in different countries were found. Low income countries reported less affordability of drugs, leaving room for potential problems with drug access, and consequently, a negative impact on health. The quality of the literature on this topic needs improvement.

A critical appraisal of the methodology and quality of evidence of systematic reviews and meta-analyses of traditional Chinese medical nursing interventions: a systematic review of reviews.

PubMed

Jin, Ying-Hui; Wang, Guo-Hao; Sun, Yi-Rong; Li, Qi; Zhao, Chen; Li, Ge; Si, Jin-Hua; Li, Yan; Lu, Cui; Shang, Hong-Cai

2016-11-14

To assess the methodology and quality of evidence of systematic reviews and meta-analyses of traditional Chinese medical nursing (TCMN) interventions in Chinese journals. These interventions include acupressure, massage, Tai Chi, Qi Gong, electroacupuncture and use of Chinese herbal medicines-for example, in enemas, foot massage and compressing the umbilicus. A systematic literature search for systematic reviews and meta-analyses of TCMN interventions was performed. Review characteristics were extracted. The methodological quality and the quality of the evidence were evaluated using the Assessment of Multiple Systematic Reviews (AMSTAR) and Grading of Recommendations Assessment, Development and Evaluation (GRADE) approaches. We included 20 systematic reviews and meta-analyses, and a total of 11 TCMN interventions were assessed in the 20 reviews. The compliance with AMSTAR checklist items ranged from 4.5 to 8 and systematic reviews/meta-analyses were, on average, of medium methodological quality. The quality of the evidence we assessed ranged from very low to moderate; no high-quality evidence was found. The top two causes for downrating confidence in effect estimates among the 31 bodies of evidence assessed were the risk of bias and inconsistency. There is room for improvement in the methodological quality of systematic reviews/meta-analyses of TCMN interventions published in Chinese journals. Greater efforts should be devoted to ensuring a more comprehensive search strategy, clearer specification of the interventions of interest in the eligibility criteria and identification of meaningful outcomes for clinicians and patients (consumers). The overall quality of evidence among reviews remains suboptimal, which raise concerns about their roles in influencing clinical practice. Thus, the conclusions in reviews we assessed must be treated with caution and their roles in influencing clinical practice should be limited. A critical appraisal of systematic reviews

Systematic review of the methodological quality of clinical guideline development for the management of chronic disease in Europe.

PubMed

Knai, Cécile; Brusamento, Serena; Legido-Quigley, Helena; Saliba, Vanessa; Panteli, Dimitra; Turk, Eva; Car, Josip; McKee, Martin; Busse, Reinhard

2012-10-01

The use of evidence-based clinical guidelines is an essential component of chronic disease management. However, there is well-documented concern about variability in the quality of clinical guidelines, with evidence of persisting methodological shortcomings. The most widely accepted approach to assessing the quality of guidelines is the Appraisal of Guidelines for Research and Evaluation (AGREE) instrument. We have conducted a systematic review of the methodological quality (as assessed by AGREE) of clinical guidelines developed in Europe for the management of chronic diseases published since 2000. The systematic review was undertaken in accordance with the Cochrane methodology. The inclusion criteria were that studies should have appraised European clinical guidelines for certain selected chronic disorders using the AGREE instrument. We searched five databases (Cab Abstracts, EMBASE, MEDLINE, Trip and EPPI). Nine studies reported in 10 papers, analysing a total of 28 European guidelines from eight countries as well as pan-European, were included. There was considerable variation in the quality of clinical guidelines across the AGREE domains. The least well addressed domains were 'editorial independence' (with a mean domain score of 41%), 'applicability' (44%), 'stakeholder involvement' (55%), and 'rigour of development' (64%), while 'clarity of presentation' (80%) and 'scope and purpose' (84%) were less problematic. This review indicates that there is considerable scope for improvement in the methods used to develop clinical guidelines for the prevention, management and treatment of chronic diseases in Europe. Given the importance of decision support strategies such as clinical guidelines in chronic disease management, improvement measures should include the explicit and transparent involvement of key stakeholders (especially scientific experts, guideline users and methodological specialists) and consideration of the implications for guideline implementation and

Four years of early benefit assessment of new drugs in Germany: a qualitative study on methodological requirements for quality of life data.

PubMed

Blome, Christine; Augustin, Matthias; Metin, Hidayet; Lohrberg, David

2017-03-01

Since 2011, an early benefit assessment (EBA) of new drugs constricts free price setting in Germany. According to the Pharmaceutical Market Restructuring Act (AMNOG), pharmaceutical companies are obliged to demonstrate added benefit of new drugs over comparative treatment. Benefit is usually evaluated by the Institute for Quality and Efficiency in Health Care (IQWiG). The final appraisal is made by the Federal Joint Committee, Germany's highest-ranking decision body in the health sector, triggering drug prize negotiations between companies and statutory health insurance funds. One of four evaluation criteria is quality of life (QoL). QoL outcomes have, however, only rarely been pivotal in EBAs. This study determined methodological requirements for QoL measurement and data presentation in the EBA. In a qualitative content analysis, documents of all EBAs completed by 2014 were searched for the term QoL. Relevant passages of all EBAs of 2011-2013 were independently extracted and reduced to key content by two researchers. Recurring patterns were identified and verified through comparison with EBAs of 2014. We identified a range of requirements regarding QoL assessment, analysis, presentation, and interpretation, which go beyond official regulations. Disease-specific questionnaires are preferred and have to be validated according to certain standards and in the respective patient group. Effects must exceed the minimal important difference, which in turn must be validated in compliance with specific requirements. Often, instruments were not accepted as QoL measures, sometimes inconsistently across EBAs. Another frequent reason for non-acceptance of QoL data was that more than 30 % of randomized patients could not be analyzed due to missing data. Non-compliance with methodological requirements for QoL evidence impairs chances for positive benefit evaluation.

Assessing primary care data quality.

PubMed

Lim, Yvonne Mei Fong; Yusof, Maryati; Sivasampu, Sheamini

2018-04-16

Purpose The purpose of this paper is to assess National Medical Care Survey data quality. Design/methodology/approach Data completeness and representativeness were computed for all observations while other data quality measures were assessed using a 10 per cent sample from the National Medical Care Survey database; i.e., 12,569 primary care records from 189 public and private practices were included in the analysis. Findings Data field completion ranged from 69 to 100 per cent. Error rates for data transfer from paper to web-based application varied between 0.5 and 6.1 per cent. Error rates arising from diagnosis and clinical process coding were higher than medication coding. Data fields that involved free text entry were more prone to errors than those involving selection from menus. The authors found that completeness, accuracy, coding reliability and representativeness were generally good, while data timeliness needs to be improved. Research limitations/implications Only data entered into a web-based application were examined. Data omissions and errors in the original questionnaires were not covered. Practical implications Results from this study provided informative and practicable approaches to improve primary health care data completeness and accuracy especially in developing nations where resources are limited. Originality/value Primary care data quality studies in developing nations are limited. Understanding errors and missing data enables researchers and health service administrators to prevent quality-related problems in primary care data.

A Comparison of Methodological Approaches for Evaluating the Quality of Basic Skills Programs. AIR Forum 1982 Paper.

ERIC Educational Resources Information Center

Dumont, Richard G.; And Others

A multiple-indicator and multiple-design approach for evaluating the quality of basic skills programs in reading, writing, and mathematics that has been implemented within a statewide system of higher education in the southeastern United States was assessed. Several of the evaluation methodologies are compared in order to determine whether results…

Methodology for assessing laser-based equipment

NASA Astrophysics Data System (ADS)

Pelegrina-Bonilla, Gabriel; Hermsdorf, Jörg; Thombansen, Ulrich; Abels, Peter; Kaierle, Stefan; Neumann, Jörg

2017-10-01

Methodologies for the assessment of technology's maturity are widely used in industry and research. Probably the best known are technology readiness levels (TRLs), initially pioneered by the National Aeronautics and Space Administration (NASA). At the beginning, only descriptively defined TRLs existed, but over time, automated assessment techniques in the form of questionnaires emerged in order to determine TRLs. Originally TRLs targeted equipment for space applications, but the demands on industrial relevant equipment are partly different in terms of, for example, overall costs, product quantities, or the presence of competitors. Therefore, we present a commonly valid assessment methodology with the aim of assessing laser-based equipment for industrial use, in general. The assessment is carried out with the help of a questionnaire, which allows for a user-friendly and easy accessible way to monitor the progress from the lab-proven state to the application-ready product throughout the complete development period. The assessment result is presented in a multidimensional metric in order to reveal the current specific strengths and weaknesses of the equipment development process, which can be used to direct the remaining development process of the equipment in the right direction.

Assessment of the methodological and ethical quality of clinical trials published in family medicine journals

PubMed

Castaño-García, Alberto; Guillén-Grima, Francisco; León-Sanz, Pilar

2018-01-01

To evaluate some methodological and ethical quality variables of clinical trials (CTs) published in 10 family medicine journals. Quality descriptive study of 10 family medicine journals including CTs in humans published since 2010 to 2013. We obtained 141 CT and 2447 were excluded. CTs parallels controlled in 92.9% (95% confidence interval [95% CI]: 92.0-93.9). Masked randomization in 72.3% (95% CI: 71.7-73.1), decentralized in 51.8% (95% CI: 51.4-52.4) and using as control an active treatment in 82.2% (95% CI: 81.5-83.1). Wrote informed consent in 48.9% (95% CI: 48.5-49.5) and it was not withdrawn in 56.0% of cases (95% CI: 55.5-56.7). Approval by clinical research ethics committee (CREC) in 134, and there was no conflict of interest in 117 CTs. Average κ was 0.96 (95% CI: 0.93-0.99). We observe an increase in some quality variables like masked randomization (19.6%) and approval by CREC (75%) post CONSORT, in CTs published in 10 family medicine journals (2010-2013). Copyright: © 2018 SecretarÍa de Salud

Methodological review of the quality of reach out and read: does it "work"?

PubMed

Yeager Pelatti, Christina; Pentimonti, Jill M; Justice, Laura M

2014-04-01

A considerable percentage of American children and adults fail to learn adequate literacy skills and read below a third grade level. Shared book reading is perhaps the single most important activity to prepare young children for success in reading. The primary objective of this manuscript was to critically review the methodological quality of Read Out and Read (ROR), a clinically based literacy program/intervention that teaches parents strategies to incorporate while sharing books with children as a method of preventing reading difficulties and academic struggles. A PubMed search was conducted. Articles that met three criteria were considered. First, the study must be clinically based and include parent contact with a pediatrician. Second, parental counseling ("anticipatory guidance") about the importance of parent-child book reading must be included. Third, only experimental or quasi-experimental studies were included; no additional criteria were used. Published articles from any year and peer-reviewed journal were considered. Study quality was determined using a modified version of the Downs and Black (1998) checklist assessing four categories: (1) Reporting, (2) External Validity, (3) Internal Validity-Bias, and (4) Internal Validity-Confounding. We were also interested in whether quality differed based on study design, children's age, sample size, and study outcome. Eleven studies met the inclusion criteria. The overall quality of evidence was variable across all studies; Reporting and External Validity categories were relatively strong while methodological concerns were found in the area of internal validity. Quality scores differed on the four study characteristics. Implications related to clinical practice and future studies are discussed.

Methodological quality and scientific impact of quantitative nursing education research over 18 months.

PubMed

Yucha, Carolyn B; Schneider, Barbara St Pierre; Smyer, Tish; Kowalski, Susan; Stowers, Eva

2011-01-01

The methodological quality of nursing education research has not been rigorously studied. The purpose of this study was to evaluate the methodological quality and scientific impact of nursing education research reports. The methodological quality of 133 quantitative nursing education research articles published between July 2006 and December 2007 was evaluated using the Medical Education Research Study Quality Instrument (MERSQI).The mean (+/- SD) MERSQI score was 9.8 +/- 2.2. It correlated (p < .05) with several scientific impact indicators: citation counts from Scopus (r = .223), Google Scholar (r = .224), and journal impact factor (r = .216); it was not associated with Web of Science citation count, funding, or h Index. The similarities between this study's MERSQI ratings for nursing literature and those reported for the medical literature, coupled with the association with citation counts, suggest that the MERSQI is an appropriate instrument to evaluate the quality of nursing education research.

Adapting needs assessment methodologies to build integrated health pathways for people in the criminal justice system.

PubMed

de Viggiani, N

2012-09-01

Criminal justice health services should be underpinned with good public health evidence about the population's health needs. Health needs assessment methodologies can provide valuable intelligence for commissioners to evaluate the quality of services and innovate according to need. However, health needs assessment can be limited if it takes a conventional epidemiological approach, focussing on individuals' healthcare needs in criminal justice settings. Techniques used to measure health and social need could be more widely applied and appropriately employed in the planning of health and social care services, especially if the intention is to be effective in reducing social exclusion and tackling health inequalities. Assessment tools are available that capture individual, social and environmental risk factors and determinants predisposing people to health and criminogenic risks. Good evidence gathering can mean that public health practitioners not only improve health, reduce inequalities and tackle social exclusion, but contribute to reducing re-offending. This paper suggests a new approach to assessment that integrates the full range of assessment methodologies available to practitioners. An integrated approach may be the way to enhance and enrich the public health function in providing evidence to improve the quality of local public services. Copyright © 2012 The Royal Society for Public Health. Published by Elsevier Ltd. All rights reserved.

Methodological Quality of Systematic Reviews Published in the Urological Literature from 1998 to 2012.

PubMed

Corbyons, Katherine; Han, Julia; Neuberger, Molly M; Dahm, Philipp

2015-11-01

Systematic reviews synthesize the current best evidence to address a clinical question. Given the growing emphasis on evidence-based clinical practice, systematic reviews are being increasingly sought after and published. We previously reported limitations in the methodological quality of 57 individual systematic reviews published from 1998 to 2008. We provide an update to our previous study, adding systematic reviews published from 2009 to 2012. We systematically searched PubMed® and hand searched the table of contents of 4 major urological journals to identify systematic reviews related to questions of prevention and therapy. Two independent reviewers with prior formal evidence-based medicine training assessed the methodological quality using the validated 11-point AMSTAR (A Measurement Tool to Assess Systematic Reviews) instrument. We performed predefined statistical hypothesis testing for differences by publication period (1998 to 2008 vs 2009 to 2012) and journal of publication. We performed statistical testing using SPSS®, version 23.0 with a 2-sided α of 0.05 using the Student t-test, ANOVA and the chi-square test. A total of 113 systematic reviews published from 2009 to 2012 met study inclusion criteria. The most common topics were oncology (44 reviews or 38.9%), voiding dysfunction (26 or 23.0%) and stones/endourology (13 or 11.5%). The largest contributor was European Urology (46 reviews or 40.7%), followed by BJU International (31 or 27.4%) and The Journal of Urology® (22 or 19.5%). The mean ± SD AMSTAR score for the 2009 to 2012 period was 5.3 ± 2.3 compared to 4.8 ± 2.0 for 1998 to 2008 with a mean difference of 0.5 (95% CI 0.2 to 1.2, p = 0.133). While the number of systematic reviews published in the urological literature has increased substantially, the methodological quality of these studies remains suboptimal. Systematic review authors and editors should make every effort to adhere to well established methodological standards to enhance

Training effectiveness assessment: Methodological problems and issues

NASA Technical Reports Server (NTRS)

Cross, Kenneth D.

1992-01-01

The U.S. military uses a large number of simulators to train and sustain the flying skills of helicopter pilots. Despite the enormous resources required to purchase, maintain, and use those simulators, little effort has been expended in assessing their training effectiveness. One reason for this is the lack of an evaluation methodology that yields comprehensive and valid data at a practical cost. Some of these methodological problems and issues that arise in assessing simulator training effectiveness, as well as problems with the classical transfer-of-learning paradigm were discussed.

Improving the Quality of Patient Care Utilizing Tracer Methodology

DTIC Science & Technology

2011-01-25

Sharing Knowledge: Achieving Breakthrough Performance 2011 Military Health System Conference Improving the Quality of Patient Care Utilizing Tracer...Methodology 25 Jan 2011 Leslie Atkins, RN 1 Military Health System Conference T e Quadruple Aim Working Together, Achieving Success USA MEDDAC...25 JAN 2011 2. REPORT TYPE 3. DATES COVERED 00-00-2011 to 00-00-2011 4. TITLE AND SUBTITLE Improving the Quality of Patient Care Utilizing

Appraising the methodological quality of the clinical practice guideline for diabetes mellitus using the AGREE II instrument: a methodological evaluation.

PubMed

Radwan, Mahmoud; Akbari Sari, Ali; Rashidian, Arash; Takian, Amirhossein; Abou-Dagga, Sanaa; Elsous, Aymen

2017-02-01

To evaluate the methodological quality of the Palestinian Clinical Practice Guideline for Diabetes Mellitus using the Translated Arabic Version of the AGREE II. Methodological evaluation. A cross-cultural adaptation framework was followed to translate and develop a standardised Translated Arabic Version of the AGREE II. Palestinian Primary Healthcare Centres. Sixteen appraisers independently evaluated the Clinical Practice Guideline for Diabetes Mellitus using the Translated Arabic Version of the AGREE II. Methodological quality of diabetic guideline. The Translated Arabic Version of the AGREE II showed an acceptable reliability and validity. Internal consistency ranged between 0.67 and 0.88 (Cronbach's α). Intra-class coefficient among appraisers ranged between 0.56 and 0.88. The quality of this guideline is low. Both domains 'Scope and Purpose' and 'Clarity of Presentation' had the highest quality scores (66.7% and 61.5%, respectively), whereas the scores for 'Applicability', 'Stakeholder Involvement', 'Rigour of Development' and 'Editorial Independence' were the lowest (27%, 35%, 36.5%, and 40%, respectively). The findings suggest that the quality of this Clinical Practice Guideline is disappointingly low. To improve the quality of current and future guidelines, the AGREE II instrument is extremely recommended to be incorporated as a gold standard for developing, evaluating or updating the Palestinian Clinical Practice Guidelines. Future guidelines can be improved by setting specific strategies to overcome implementation barriers with respect to economic considerations, engaging of all relevant end-users and patients, ensuring a rigorous methodology for searching, selecting and synthesising the evidences and recommendations, and addressing potential conflict of interests within the development group.

Risk Assessment Methodology for Hazardous Waste Management (1998)

EPA Pesticide Factsheets

A methodology is described for systematically assessing and comparing the risks to human health and the environment of hazardous waste management alternatives. The methodology selects and links appropriate models and techniques for performing the process.

A framework for assessing Health Economic Evaluation (HEE) quality appraisal instruments.

PubMed

Langer, Astrid

2012-08-16

Health economic evaluations support the health care decision-making process by providing information on costs and consequences of health interventions. The quality of such studies is assessed by health economic evaluation (HEE) quality appraisal instruments. At present, there is no instrument for measuring and improving the quality of such HEE quality appraisal instruments. Therefore, the objectives of this study are to establish a framework for assessing the quality of HEE quality appraisal instruments to support and improve their quality, and to apply this framework to those HEE quality appraisal instruments which have been subject to more scrutiny than others, in order to test the framework and to demonstrate the shortcomings of existing HEE quality appraisal instruments. To develop the quality assessment framework for HEE quality appraisal instruments, the experiences of using appraisal tools for clinical guidelines are used. Based on a deductive iterative process, clinical guideline appraisal instruments identified through literature search are reviewed, consolidated, and adapted to produce the final quality assessment framework for HEE quality appraisal instruments. The final quality assessment framework for HEE quality appraisal instruments consists of 36 items organized within 7 dimensions, each of which captures a specific domain of quality. Applying the quality assessment framework to four existing HEE quality appraisal instruments, it is found that these four quality appraisal instruments are of variable quality. The framework described in this study should be regarded as a starting point for appraising the quality of HEE quality appraisal instruments. This framework can be used by HEE quality appraisal instrument producers to support and improve the quality and acceptance of existing and future HEE quality appraisal instruments. By applying this framework, users of HEE quality appraisal instruments can become aware of methodological deficiencies

Forty years of quality-of-life research in congenital heart disease: Temporal trends in conceptual and methodological rigor.

PubMed

Bratt, Ewa-Lena; Moons, Philip

2015-09-15

The first study on quality of life (QoL) in patients with congenital heart disease was published 40 years ago. Since then, the number of QoL articles on these patients has grown exponentially. We conducted a systematic literature review of all empirical studies on QoL in patients with congenital heart disease published since 1974, with the aim of determining the range of conceptual and methodological rigor of studies and identifying temporal trends in these parameters. PubMed, Embase, and Cinahl were searched for empirical studies addressing QoL in children, adolescents, or adults with congenital heart disease, published between January 1, 1974, and December 31, 2014. We applied 10 review criteria that were previously developed by Gill and Feinstein in 1994 and further refined by Moons et al. in 2004. Overall, 234 articles were reviewed. We found slight but non-significant temporal improvements in conceptual and methodological rigor and in use of assessment methods. This indicates a trend toward a more professional and exacting approach in QoL assessments. However, the majority of articles still had substantial conceptual and methodological deficits. Furthermore, we observed that citation of the publications of Gill and Feinstein and Moons et al. in published QoL research is associated with higher quality scores, suggesting that these articles have a positive impact on conceptual and methodological caliber. Despite 40 years of QoL research in this field, this review shows that major weaknesses in methodological rigor remain highly prevalent, which may make QoL studies inconclusive. Copyright © 2015 Elsevier Ireland Ltd. All rights reserved.

Social cognition interventions for people with schizophrenia: a systematic review focussing on methodological quality and intervention modality.

PubMed

Grant, Nina; Lawrence, Megan; Preti, Antonio; Wykes, Til; Cella, Matteo

2017-08-01

People with a diagnosis of schizophrenia have significant social and functional difficulties. Social cognition was found to influences these outcomes and in recent years interventions targeting this domain were developed. This paper reviews the existing literature on social cognition interventions for people with a diagnosis of schizophrenia focussing on: i) comparing focussed (i.e. targeting only one social cognitive domain) and global interventions and ii) studies methodological quality. Systematic search was conducted on PubMed and PsycInfo. Studies were included if they were randomised control trials, participants had a diagnosis of schizophrenia or schizoaffective disorder, and the intervention targeted at least one out of four social cognition domains (i.e. theory of mind, affect recognition, social perception and attribution bias). All papers were assessed for methodological quality. Information on the intervention, control condition, study methodology and the main findings from each study were extracted and critically summarised. Data from 32 studies fulfilled the inclusion criteria, considering a total of 1440 participants. Taking part in social cognition interventions produced significant improvements in theory of mind and affect recognition compared to both passive and active control conditions. Results were less clear for social perception and attributional bias. Focussed and global interventions had similar results on outcomes. Overall study methodological quality was modest. There was very limited evidence showing that social cognitive intervention result in functional outcome improvement. The evidence considered suggests that social cognition interventions may be a valuable approach for people with a diagnosis of schizophrenia. However, evidence quality is limited by measure heterogeneity, modest study methodology and short follow-up periods. The findings point to a number of recommendations for future research, including measurement standardisation

Methodological and Statistical Quality in Research Evaluating Nutritional Attitudes in Sports.

PubMed

Kouvelioti, Rozalia; Vagenas, George

2015-12-01

The assessment of dietary attitudes and behaviors provides information of interest to sports nutritionists. Although there has been little analysis of the quality of research undertaken in this field, there is evidence of a number of flaws and methodological concerns in some of the studies in the available literature. This review undertook a systematic assessment of the attributes of research assessing the nutritional knowledge and attitudes of athletes and coaches. Sixty questionnaire-based studies were identified by a search of official databases using specific key terms with subsequent analysis by certain inclusion-exclusion criteria. These studies were then analyzed using 33 research quality criteria related to the methods, questionnaires, and statistics used. We found that many studies did not provide information on critical issues such as research hypotheses (92%), the gaining of ethics approval (50%) or informed consent (35%), or acknowledgment of limitations in the implementation of studies or interpretation of data (72%). Many of the samples were nonprobabilistic (85%) and rather small (42%). Many questionnaires were of unknown origin (30%), validity (72%), and reliability (70%) and resulted in low (≤ 60%) response rates (38%). Pilot testing was not undertaken in 67% of the studies. Few studies dealt with sample size (2%), power (3%), assumptions (7%), confidence intervals (3%), or effect sizes (3%). Improving some of these problems and deficits may enhance future research in this field.

Association between bibliometric parameters, reporting and methodological quality of randomised controlled trials in vascular and endovascular surgery.

PubMed

Hajibandeh, Shahab; Hajibandeh, Shahin; Antoniou, George A; Green, Patrick A; Maden, Michelle; Torella, Francesco

2017-04-01

Purpose We aimed to investigate association between bibliometric parameters, reporting and methodological quality of vascular and endovascular surgery randomised controlled trials. Methods The most recent 75 and oldest 75 randomised controlled trials published in leading journals over a 10-year period were identified. The reporting quality was analysed using the CONSORT statement, and methodological quality with the Intercollegiate Guidelines Network checklist. We used exploratory univariate and multivariable linear regression analysis to investigate associations. Findings Bibliometric parameters such as type of journal, study design reported in title, number of pages; external funding, industry sponsoring and number of citations are associated with reporting quality. Moreover, parameters such as type of journal, subject area and study design reported in title are associated with methodological quality. Conclusions The bibliometric parameters of randomised controlled trials may be independent predictors for their reporting and methodological quality. Moreover, the reporting quality of randomised controlled trials is associated with their methodological quality and vice versa.

A novel integrated assessment methodology of urban water reuse.

PubMed

Listowski, A; Ngo, H H; Guo, W S; Vigneswaran, S

2011-01-01

Wastewater is no longer considered a waste product and water reuse needs to play a stronger part in securing urban water supply. Although treatment technologies for water reclamation have significantly improved the question that deserves further analysis is, how selection of a particular wastewater treatment technology relates to performance and sustainability? The proposed assessment model integrates; (i) technology, characterised by selected quantity and quality performance parameters; (ii) productivity, efficiency and reliability criteria; (iii) quantitative performance indicators; (iv) development of evaluation model. The challenges related to hierarchy and selections of performance indicators have been resolved through the case study analysis. The goal of this study is to validate a new assessment methodology in relation to performance of the microfiltration (MF) technology, a key element of the treatment process. Specific performance data and measurements were obtained at specific Control and Data Acquisition Points (CP) to satisfy the input-output inventory in relation to water resources, products, material flows, energy requirements, chemicals use, etc. Performance assessment process contains analysis and necessary linking across important parametric functions leading to reliable outcomes and results.

Using Plant Clinic Registers to Assess the Quality of Diagnoses and Advice Given to Farmers: A Case Study from Uganda

ERIC Educational Resources Information Center

Danielsen, Solveig; Boa, Eric; Mafabi, Moses; Mutebi, Emmanuel; Reeder, Robert; Kabeere, Flavia; Karyeija, Robert

2013-01-01

Purpose: This study developed a framework for quality assessment of diagnoses and advice given at plant clinics. Design/methodology/approach: Clinic registers from five plant clinics in Uganda (2006-2010) were used to develop quality assessment protocols for diagnoses and advice given by plant doctors. Assessment of quality of diagnoses was based…

Assessing the quality of the volume-outcome relationship in uro-oncology.

PubMed

Mayer, Erik K; Purkayastha, Sanjay; Athanasiou, Thanos; Darzi, Ara; Vale, Justin A

2009-02-01

To assess systematically the quality of evidence for the volume-outcome relationship in uro-oncology, and thus facilitate the formulating of health policy within this speciality, as 'Implementation of Improving Outcome Guidance' has led to centralization of uro-oncology based on published studies that have supported a 'higher volume-better outcome' relationship, but improved awareness of methodological drawbacks in health service research has questioned the strength of this proposed volume-outcome relationship. We systematically searched previous relevant reports and extracted all articles from 1980 onwards assessing the volume-outcome relationship for cystectomy, prostatectomy and nephrectomy at the institution and/or surgeon level. Studies were assessed for their methodological quality using a previously validated rating system. Where possible, meta-analytical methods were used to calculate overall differences in outcome measures between low and high volume healthcare providers. In all, 22 studies were included in the final analysis; 19 of these were published in the last 5 years. Only four studies appropriately explored the effect of both the institution and surgeon volume on outcome measures. Mortality and length of stay were the most frequently measured outcomes. The median total quality scores within each of the operation types were 8.5, 9 and 8 for cystectomy, prostatectomy and nephrectomy, respectively (possible maximum score 18). Random-effects modelling showed a higher risk of mortality in low-volume institutions than in higher-volume institutions for both cystectomy and nephrectomy (odds ratio 1.88, 95% confidence interval 1.54-2.29, and 1.28, 1.10-1.49, respectively). The methodological quality of volume-outcome research as applied to cystectomy, prostatectomy and nephrectomy is only modest at best. Accepting several limitations, pooled analysis confirms a higher-volume, lower-mortality relationship for cystectomy and nephrectomy. Future research should

The quality of instruments to assess the process of shared decision making: A systematic review.

PubMed

Gärtner, Fania R; Bomhof-Roordink, Hanna; Smith, Ian P; Scholl, Isabelle; Stiggelbout, Anne M; Pieterse, Arwen H

2018-01-01

To inventory instruments assessing the process of shared decision making and appraise their measurement quality, taking into account the methodological quality of their validation studies. In a systematic review we searched seven databases (PubMed, Embase, Emcare, Cochrane, PsycINFO, Web of Science, Academic Search Premier) for studies investigating instruments measuring the process of shared decision making. Per identified instrument, we assessed the level of evidence separately for 10 measurement properties following a three-step procedure: 1) appraisal of the methodological quality using the COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN) checklist, 2) appraisal of the psychometric quality of the measurement property using three possible quality scores, 3) best-evidence synthesis based on the number of studies, their methodological and psychometrical quality, and the direction and consistency of the results. The study protocol was registered at PROSPERO: CRD42015023397. We included 51 articles describing the development and/or evaluation of 40 shared decision-making process instruments: 16 patient questionnaires, 4 provider questionnaires, 18 coding schemes and 2 instruments measuring multiple perspectives. There is an overall lack of evidence for their measurement quality, either because validation is missing or methods are poor. The best-evidence synthesis indicated positive results for a major part of instruments for content validity (50%) and structural validity (53%) if these were evaluated, but negative results for a major part of instruments when inter-rater reliability (47%) and hypotheses testing (59%) were evaluated. Due to the lack of evidence on measurement quality, the choice for the most appropriate instrument can best be based on the instrument's content and characteristics such as the perspective that they assess. We recommend refinement and validation of existing instruments, and the use of COSMIN

The quality of instruments to assess the process of shared decision making: A systematic review

PubMed Central

Bomhof-Roordink, Hanna; Smith, Ian P.; Scholl, Isabelle; Stiggelbout, Anne M.; Pieterse, Arwen H.

2018-01-01

Objective To inventory instruments assessing the process of shared decision making and appraise their measurement quality, taking into account the methodological quality of their validation studies. Methods In a systematic review we searched seven databases (PubMed, Embase, Emcare, Cochrane, PsycINFO, Web of Science, Academic Search Premier) for studies investigating instruments measuring the process of shared decision making. Per identified instrument, we assessed the level of evidence separately for 10 measurement properties following a three-step procedure: 1) appraisal of the methodological quality using the COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN) checklist, 2) appraisal of the psychometric quality of the measurement property using three possible quality scores, 3) best-evidence synthesis based on the number of studies, their methodological and psychometrical quality, and the direction and consistency of the results. The study protocol was registered at PROSPERO: CRD42015023397. Results We included 51 articles describing the development and/or evaluation of 40 shared decision-making process instruments: 16 patient questionnaires, 4 provider questionnaires, 18 coding schemes and 2 instruments measuring multiple perspectives. There is an overall lack of evidence for their measurement quality, either because validation is missing or methods are poor. The best-evidence synthesis indicated positive results for a major part of instruments for content validity (50%) and structural validity (53%) if these were evaluated, but negative results for a major part of instruments when inter-rater reliability (47%) and hypotheses testing (59%) were evaluated. Conclusions Due to the lack of evidence on measurement quality, the choice for the most appropriate instrument can best be based on the instrument’s content and characteristics such as the perspective that they assess. We recommend refinement and validation of

Using Risk Assessment Methodologies to Meet Management Objectives

NASA Technical Reports Server (NTRS)

DeMott, D. L.

2015-01-01

Current decision making involves numerous possible combinations of technology elements, safety and health issues, operational aspects and process considerations to satisfy program goals. Identifying potential risk considerations as part of the management decision making process provides additional tools to make more informed management decision. Adapting and using risk assessment methodologies can generate new perspectives on various risk and safety concerns that are not immediately apparent. Safety and operational risks can be identified and final decisions can balance these considerations with cost and schedule risks. Additional assessments can also show likelihood of event occurrence and event consequence to provide a more informed basis for decision making, as well as cost effective mitigation strategies. Methodologies available to perform Risk Assessments range from qualitative identification of risk potential, to detailed assessments where quantitative probabilities are calculated. Methodology used should be based on factors that include: 1) type of industry and industry standards, 2) tasks, tools, and environment 3) type and availability of data and 4) industry views and requirements regarding risk & reliability. Risk Assessments are a tool for decision makers to understand potential consequences and be in a position to reduce, mitigate or eliminate costly mistakes or catastrophic failures.

HOW GOOD ARE MY DATA? INFORMATION QUALITY ASSESSMENT METHODOLOGY

EPA Science Inventory

Quality assurance techniques used in software development and hardware maintenance/reliability help to ensure that data in a computerized information management system are maintained well. However, information workers may not know the quality of the data resident in their inf...

Mobile and Web 2.0 interventions for weight management: an overview of review evidence and its methodological quality

PubMed Central

Smith, Jane R.; Samaha, Laya; Abraham, Charles

2016-01-01

Abstract Background : The use of Internet and related technologies for promoting weight management (WM), physical activity (PA), or dietary-related behaviours has been examined in many articles and systematic reviews. This overview aims to summarize and assess the quality of the review evidence specifically focusing on mobile and Web 2.0 technologies, which are the most utilized, currently available technologies. Methods: Following a registered protocol (CRD42014010323), we searched 16 databases for articles published in English until 31 December 2014 discussing the use of either mobile or Web 2.0 technologies to promote WM or related behaviors, i.e. diet and physical activity (PA). Two reviewers independently selected reviews and assessed their methodological quality using the AMSTAR checklist. Citation matrices were used to determine the overlap among reviews. Results: Forty-four eligible reviews were identified, 39 of which evaluated the effects of interventions using mobile or Web 2.0 technologies. Methodological quality was generally low with only 7 reviews (16%) meeting the highest standards. Suggestive evidence exists for positive effects of mobile technologies on weight-related outcomes and, to a lesser extent, PA. Evidence is inconclusive regarding Web 2.0 technologies. Conclusions : Reviews on mobile and Web 2.0 interventions for WM and related behaviors suggest that these technologies can, under certain circumstances, be effective, but conclusions are limited by poor review quality based on a heterogeneous evidence base. PMID:27335330

Appraisal of the methodological quality and summary of the findings of systematic reviews on the relationship between SSRIs and suicidality.

PubMed

Li, Wei; Li, Wei; Wan, Yumei; Ren, Juanjuan; Li, Ting; Li, Chunbo

2014-10-01

Several systematic reviews have been published about the relationship of the use of selective serotonin reuptake inhibitors (SSRIs) and risk of suicidal ideation or behavior but there has been no formal assessment of the quality of these reports. Assess the methodological quality of systematic reviews about the relationship of SSRI use and suicidal ideation and behavior; and provide overall conclusions based on this assessment. Systematic reviews of RCTs that compared SSRIs to placebo and used suicidal ideation or behavior as a key outcome variable were identified by searching Pubmed, Embase, The Cochrane Library, EBSCO, PsycINFO, Chinese National Knowledge Infrastructure, Chongqing VIP database for Chinese Technical Periodicals, WANFANG DATA, and the Chinese Biological Medical Literature Database. The methodological quality of included reviews was independently assessed by two expert raters using the 11-item Assessment of Multiple Systematic Reviews (AMSTAR) scale. Twelve systematic reviews and meta-analyses were identified. The inter-rater reliability of the overall AMSTAR quality score was excellent (ICC=0.86) but the inter-rater reliability of 5 of the 11 AMSTAR items was poor (Kappa <0.60). Based on the AMSTAR total score, there was one high-quality review, eight moderate-quality reviews, and three low-quality reviews. The high-quality review and three of the moderate-quality reviews reported a significantly increased risk of suicidal ideation or behavior in the SSRI group compared to the placebo group. Three of the four reviews limited to children and adolescents found a significantly increased risk of suicidal ideation or behavior with SSRI use which was most evident in teenagers taking paroxetine and in teenagers with depressive disorders. The available evidence suggests that adolescents may experience an increase in suicidal ideation and behavior with SSRI use, particularly those who have a depressive disorder and those treated with paroxetine. However

A systematic literature review of open source software quality assessment models.

PubMed

Adewumi, Adewole; Misra, Sanjay; Omoregbe, Nicholas; Crawford, Broderick; Soto, Ricardo

2016-01-01

Many open source software (OSS) quality assessment models are proposed and available in the literature. However, there is little or no adoption of these models in practice. In order to guide the formulation of newer models so they can be acceptable by practitioners, there is need for clear discrimination of the existing models based on their specific properties. Based on this, the aim of this study is to perform a systematic literature review to investigate the properties of the existing OSS quality assessment models by classifying them with respect to their quality characteristics, the methodology they use for assessment, and their domain of application so as to guide the formulation and development of newer models. Searches in IEEE Xplore, ACM, Science Direct, Springer and Google Search is performed so as to retrieve all relevant primary studies in this regard. Journal and conference papers between the year 2003 and 2015 were considered since the first known OSS quality model emerged in 2003. A total of 19 OSS quality assessment model papers were selected. To select these models we have developed assessment criteria to evaluate the quality of the existing studies. Quality assessment models are classified into five categories based on the quality characteristics they possess namely: single-attribute, rounded category, community-only attribute, non-community attribute as well as the non-quality in use models. Our study reflects that software selection based on hierarchical structures is found to be the most popular selection method in the existing OSS quality assessment models. Furthermore, we found that majority (47%) of the existing models do not specify any domain of application. In conclusion, our study will be a valuable contribution to the community and helps the quality assessment model developers in formulating newer models and also to the practitioners (software evaluators) in selecting suitable OSS in the midst of alternatives.

A framework for assessing Health Economic Evaluation (HEE) quality appraisal instruments

PubMed Central

2012-01-01

Background Health economic evaluations support the health care decision-making process by providing information on costs and consequences of health interventions. The quality of such studies is assessed by health economic evaluation (HEE) quality appraisal instruments. At present, there is no instrument for measuring and improving the quality of such HEE quality appraisal instruments. Therefore, the objectives of this study are to establish a framework for assessing the quality of HEE quality appraisal instruments to support and improve their quality, and to apply this framework to those HEE quality appraisal instruments which have been subject to more scrutiny than others, in order to test the framework and to demonstrate the shortcomings of existing HEE quality appraisal instruments. Methods To develop the quality assessment framework for HEE quality appraisal instruments, the experiences of using appraisal tools for clinical guidelines are used. Based on a deductive iterative process, clinical guideline appraisal instruments identified through literature search are reviewed, consolidated, and adapted to produce the final quality assessment framework for HEE quality appraisal instruments. Results The final quality assessment framework for HEE quality appraisal instruments consists of 36 items organized within 7 dimensions, each of which captures a specific domain of quality. Applying the quality assessment framework to four existing HEE quality appraisal instruments, it is found that these four quality appraisal instruments are of variable quality. Conclusions The framework described in this study should be regarded as a starting point for appraising the quality of HEE quality appraisal instruments. This framework can be used by HEE quality appraisal instrument producers to support and improve the quality and acceptance of existing and future HEE quality appraisal instruments. By applying this framework, users of HEE quality appraisal instruments can become aware

Guidance on the Technology Performance Level (TPL) Assessment Methodology

DOE Office of Scientific and Technical Information (OSTI.GOV)

Weber, Jochem; Roberts, Jesse D.; Babarit, Aurelien

This document presents the revised Technology Performance Level (TPL) assessment methodology. There are three parts to this revised methodology 1) the Stakeholder Needs and Assessment Guidance (this document), 2) the Technical Submission form, 3) the TPL scoring spreadsheet. The TPL assessment is designed to give a technology neutral or agnostic assessment of any wave energy converter technology. The focus of the TPL is on the performance of the technology in meeting the customer’s needs. The original TPL is described in [1, 2] and those references also detail the critical differences in the nature of the TPL when compared to themore » more widely used technology readiness level (TRL). (Wave energy TRL is described in [3]). The revised TPL is particularly intended to be useful to investors and also to assist technology developers to conduct comprehensive assessments in a way that is meaningful and attractive to investors. The revised TPL assessment methodology has been derived through a structured Systems Engineering approach. This was a formal process which involved analyzing customer and stakeholder needs through the discipline of Systems Engineering. The results of the process confirmed the high level of completeness of the original methodology presented in [1] (as used in the Wave Energy Prize judging) and now add a significantly increased level of detail in the assessment and an improved more investment focused structure. The revised TPL also incorporates the feedback of the Wave Energy Prize judges.« less

The relationship between return on investment and quality of study methodology in workplace health promotion programs.

PubMed

Baxter, Siyan; Sanderson, Kristy; Venn, Alison J; Blizzard, C Leigh; Palmer, Andrew J

2014-01-01

To determine the relationship between return on investment (ROI) and quality of study methodology in workplace health promotion programs. Data were obtained through a systematic literature search of National Health Service Economic Evaluation Database (NHS EED), Database of Abstracts of Reviews of Effects (DARE), Health Technology Database (HTA), Cost Effectiveness Analysis (CEA) Registry, EconLit, PubMed, Embase, Wiley, and Scopus. Included were articles written in English or German reporting cost(s) and benefit(s) and single or multicomponent health promotion programs on working adults. Return-to-work and workplace injury prevention studies were excluded. Methodological quality was graded using British Medical Journal Economic Evaluation Working Party checklist. Economic outcomes were presented as ROI. ROI was calculated as ROI = (benefits - costs of program)/costs of program. Results were weighted by study size and combined using meta-analysis techniques. Sensitivity analysis was performed using two additional methodological quality checklists. The influences of quality score and important study characteristics on ROI were explored. Fifty-one studies (61 intervention arms) published between 1984 and 2012 included 261,901 participants and 122,242 controls from nine industry types across 12 countries. Methodological quality scores were highly correlated between checklists (r = .84-.93). Methodological quality improved over time. Overall weighted ROI [mean ± standard deviation (confidence interval)] was 1.38 ± 1.97 (1.38-1.39), which indicated a 138% return on investment. When accounting for methodological quality, an inverse relationship to ROI was found. High-quality studies (n = 18) had a smaller mean ROI, 0.26 ± 1.74 (.23-.30), compared to moderate (n = 16) 0.90 ± 1.25 (.90-.91) and low-quality (n = 27) 2.32 ± 2.14 (2.30-2.33) studies. Randomized control trials (RCTs) (n = 12) exhibited negative ROI, -0.22 ± 2.41(-.27 to -.16). Financial returns become

[Methodological quality of an article on the treatment of gastric cancer adopted as protocol by some Chilean hospitals].

PubMed

Manterola, Carlos; Torres, Rodrigo; Burgos, Luis; Vial, Manuel; Pineda, Viviana

2006-07-01

Surgery is a curative treatment for gastric cancer (GC). As relapse is frequent, adjuvant therapies such as postoperative chemo radiotherapy have been tried. In Chile, some hospitals adopted Macdonald's study as a protocol for the treatment of GC. To determine methodological quality and internal and external validity of the Macdonald study. Three instruments were applied that assess methodological quality. A critical appraisal was done and the internal and external validity of the methodological quality was analyzed with two scales: MINCIR (Methodology and Research in Surgery), valid for therapy studies and CONSORT (Consolidated Standards of Reporting Trials), valid for randomized controlled trials (RCT). Guides and scales were applied by 5 researchers with training in clinical epidemiology. The reader's guide verified that the Macdonald study was not directed to answer a clearly defined question. There was random assignment, but the method used is not described and the patients were not considered until the end of the study (36% of the group with surgery plus chemo radiotherapy did not complete treatment). MINCIR scale confirmed a multicentric RCT, not blinded, with an unclear randomized sequence, erroneous sample size estimation, vague objectives and no exclusion criteria. CONSORT system proved the lack of working hypothesis and specific objectives as well as an absence of exclusion criteria and identification of the primary variable, an imprecise estimation of sample size, ambiguities in the randomization process, no blinding, an absence of statistical adjustment and the omission of a subgroup analysis. The instruments applied demonstrated methodological shortcomings that compromise the internal and external validity of the.

Recommendations for benefit-risk assessment methodologies and visual representations.

PubMed

Hughes, Diana; Waddingham, Ed; Mt-Isa, Shahrul; Goginsky, Alesia; Chan, Edmond; Downey, Gerald F; Hallgreen, Christine E; Hockley, Kimberley S; Juhaeri, Juhaeri; Lieftucht, Alfons; Metcalf, Marilyn A; Noel, Rebecca A; Phillips, Lawrence D; Ashby, Deborah; Micaleff, Alain

2016-03-01

The purpose of this study is to draw on the practical experience from the PROTECT BR case studies and make recommendations regarding the application of a number of methodologies and visual representations for benefit-risk assessment. Eight case studies based on the benefit-risk balance of real medicines were used to test various methodologies that had been identified from the literature as having potential applications in benefit-risk assessment. Recommendations were drawn up based on the results of the case studies. A general pathway through the case studies was evident, with various classes of methodologies having roles to play at different stages. Descriptive and quantitative frameworks were widely used throughout to structure problems, with other methods such as metrics, estimation techniques and elicitation techniques providing ways to incorporate technical or numerical data from various sources. Similarly, tree diagrams and effects tables were universally adopted, with other visualisations available to suit specific methodologies or tasks as required. Every assessment was found to follow five broad stages: (i) Planning, (ii) Evidence gathering and data preparation, (iii) Analysis, (iv) Exploration and (v) Conclusion and dissemination. Adopting formal, structured approaches to benefit-risk assessment was feasible in real-world problems and facilitated clear, transparent decision-making. Prior to this work, no extensive practical application and appraisal of methodologies had been conducted using real-world case examples, leaving users with limited knowledge of their usefulness in the real world. The practical guidance provided here takes us one step closer to a harmonised approach to benefit-risk assessment from multiple perspectives. Copyright © 2016 John Wiley & Sons, Ltd.

Scoping review of pediatric tonsillectomy quality of life assessment instruments.

PubMed

Kao, Stephen Shih-Teng; Peters, Micah D J; Dharmawardana, Nuwan; Stew, Benjamin; Ooi, Eng Hooi

2017-10-01

Sleep-disordered breathing or recurrent tonsillitis have detrimental effects on the child's physical health and quality of life. Tonsillectomy is commonly performed to treat these common conditions and improve the child's quality of life. This scoping review aims to present a comprehensive and descriptive analysis of quality of life questionnaires as a resource for clinicians and researchers when deciding which tool to use when assessing the quality of life effects after tonsillectomy. A comprehensive search strategy was undertaken across MEDLINE (PubMed), CINAHL, Embase, and Cochrane CENTRAL. Quality of life questionnaires utilized in studies investigating pediatric patients undergoing tonsillectomy for chronic tonsillitis or sleep-disordered breathing were included. Methodological quality and data extraction were conducted as per Joanna Briggs Institute methodology. Ten questionnaires were identified, consisting of six generic and four disease-specific instruments. The Pediatric Quality of Life Inventory was the most commonly utilized generic questionnaire. The Obstructive Sleep Apnea-18 was the most commonly utilized disease-specific questionnaire. This review identified a range of generic and disease-specific quality of life questionnaires utilized in pediatric patients who have undergone tonsillectomy with or without adenoidectomy for sleep-disordered breathing or chronic tonsillitis. Important aspects of each questionnaire have been summarized to aid researchers and clinicians in choosing the appropriate questionnaire when evaluating the quality of life effects of tonsillectomy. NA Laryngoscope, 127:2399-2406, 2017. © 2017 The American Laryngological, Rhinological and Otological Society, Inc.

Using quality assessment tools to critically appraise ageing research: a guide for clinicians.

PubMed

Harrison, Jennifer Kirsty; Reid, James; Quinn, Terry J; Shenkin, Susan Deborah

2017-05-01

Evidence based medicine tells us that we should not accept published research at face value. Even research from established teams published in the highest impact journals can have methodological flaws, biases and limited generalisability. The critical appraisal of research studies can seem daunting, but tools are available to make the process easier for the non-specialist. Understanding the language and process of quality assessment is essential when considering or conducting research, and is also valuable for all clinicians who use published research to inform their clinical practice.We present a review written specifically for the practising geriatrician. This considers how quality is defined in relation to the methodological conduct and reporting of research. Having established why quality assessment is important, we present and critique tools which are available to standardise quality assessment. We consider five study designs: RCTs, non-randomised studies, observational studies, systematic reviews and diagnostic test accuracy studies. Quality assessment for each of these study designs is illustrated with an example of published cognitive research. The practical applications of the tools are highlighted, with guidance on their strengths and limitations. We signpost educational resources and offer specific advice for use of these tools.We hope that all geriatricians become comfortable with critical appraisal of published research and that use of the tools described in this review - along with awareness of their strengths and limitations - become a part of teaching, journal clubs and practice. © The Author 2016. Published by Oxford University Press on behalf of the British Geriatrics Society.

A risk assessment methodology for critical transportation infrastructure.

DOT National Transportation Integrated Search

2002-01-01

Infrastructure protection typifies a problem of risk assessment and management in a large-scale system. This study offers a methodological framework to identify, prioritize, assess, and manage risks. It includes the following major considerations: (1...

[Reevaluation of the methodological quality in meta-analyses of accelerated rehabilitation on recovery after surgery for colorectal cancer].

PubMed

Ding, S N; Pan, H Y; Zhang, J G

2017-03-14

Objective: To evaluate the methodological quality and impacts on outcomes for systematic reviews (SRs) of accelerated rehabilitation versus traditional control for colorectal surgery. Methods: We comprehensively searched six databases and additional websites to collect SRs, or meta-analysis from inception to July 2016. The Overview Quality Assessment Questionnaire (OQAQ) was applied for quality assessment of the included studies, the tools recommended by the Cochrane Collaboration was applied for quality assessment for RCT and CCT and the Newcastle-Ottawa Scale (NOS) was applied to assess observational study. The relative ratios (RRs) and 95% confidence intervals (CIs) were integrated using Review Manager 5.3 software. Results: Fourteen meta-analyses were included in total. The mean OQAQ score was 3.8 with 95% CI 3.2 to 4.3. Only three meta-analyses were assessed as good quality. Two studies misused statistical models. A total of 42 primary studies referenced by meta-analyses were included, of which, 25 RCTs were levelled grade B and 1 CCT was levelled grade C. An estimated mean NOS score of 16 observation studies was 6.75 (totally scored 9 with 95% CI 6.4 to 7.1), of which, 10 studies scored ≥7 were high quality, 6 studies scored 6 were moderate quality. Conclusions: Currently, the overall quality of meta-analyses about comparing the effects and safety between accelerated rehabilitation and traditional control for colorectal surgery is fairly poor and the evidence level is lower. Health providers should apply the evidence with caution in clinical practice.

Methodological Quality of Systematic Reviews in Subfertility: A Comparison of Two Different Approaches

PubMed Central

Popovich, Ivor; Windsor, Bethany; Jordan, Vanessa; Showell, Marian; Shea, Bev; Farquhar, Cynthia M.

2012-01-01

Background Systematic reviews are used widely to guide health care decisions. Several tools have been created to assess systematic review quality. The measurement tool for assessing the methodological quality of systematic reviews known as the AMSTAR tool applies a yes/no score to eleven relevant domains of review methodology. This tool has been reworked so that each domain is scored based on a four point scale, producing R-AMSTAR. Methods and Findings We aimed to compare the AMSTAR and R-AMSTAR tools in assessing systematic reviews in the field of assisted reproduction for subfertility. All published systematic reviews on assisted reproductive technology, with the latest search for studies taking place from 2007–2011, were considered. Reviews that contained no included studies or considered diagnostic outcomes were excluded. Thirty each of Cochrane and non-Cochrane reviews were randomly selected from a search of relevant databases. Both tools were then applied to all sixty reviews. The results were converted to percentage scores and all reviews graded and ranked based on this. AMSTAR produced a much wider variation in percentage scores and achieved higher inter-rater reliability than R-AMSTAR according to kappa statistics. The average rating for Cochrane reviews was consistent between the two tools (88.3% for R-AMSTAR versus 83.6% for AMSTAR) but inconsistent for non-Cochrane reviews (63.9% R-AMSTAR vs. 38.5% AMSTAR). In comparing the rankings generated between the two tools Cochrane reviews changed an average of 4.2 places, compared to 2.9 for non-Cochrane. Conclusion R-AMSTAR provided greater guidance in the assessment of domains and produced quantitative results. However, there were many problems with the construction of its criteria and AMSTAR was much easier to apply consistently. We recommend that AMSTAR incorporates the findings of this study and produces additional guidance for its application in order to improve its reliability and usefulness. PMID

Quality assessment of systematic reviews on alveolar socket preservation.

PubMed

Moraschini, V; Barboza, E Dos S P

2016-09-01

The aim of this overview was to evaluate and compare the quality of systematic reviews, with or without meta-analysis, that have evaluated studies on techniques or biomaterials used for the preservation of alveolar sockets post tooth extraction in humans. An electronic search was conducted without date restrictions using the Medline/PubMed, Cochrane Library, and Web of Science databases up to April 2015. Eligibility criteria included systematic reviews, with or without meta-analysis, focused on the preservation of post-extraction alveolar sockets in humans. Two independent authors assessed the quality of the included reviews using AMSTAR and the checklist proposed by Glenny et al. in 2003. After the selection process, 12 systematic reviews were included. None of these reviews obtained the maximum score using the quality assessment tools implemented, and the results of the analyses were highly variable. A significant statistical correlation was observed between the scores of the two checklists. A wide structural and methodological variability was observed between the systematic reviews published on the preservation of alveolar sockets post tooth extraction. None of the reviews evaluated obtained the maximum score using the two quality assessment tools implemented. Copyright © 2016 International Association of Oral and Maxillofacial Surgeons. Published by Elsevier Ltd. All rights reserved.

Sugar-Sweetened Beverages and Obesity Risk in Children and Adolescents: A Systematic Analysis on How Methodological Quality May Influence Conclusions.

PubMed

Bucher Della Torre, Sophie; Keller, Amélie; Laure Depeyre, Jocelyne; Kruseman, Maaike

2016-04-01

In the context of a worldwide high prevalence of childhood obesity, the role of sugar-sweetened beverage (SSB) consumption as a cause of excess weight gain remains controversial. Conflicting results may be due to methodological issues in original studies and in reviews. The aim of this review was to systematically analyze the methodology of studies investigating the influence of SSB consumption on risk of obesity and obesity among children and adolescents, and the studies' ability to answer this research question. A systematic review of cohort and experimental studies published until December 2013 in peer-reviewed journals was performed on Medline, CINAHL, Web of Knowledge, and ClinicalTrials.gov. Studies investigating the influence of SSB consumption on risk of obesity and obesity among children and adolescents were included, and methodological quality to answer this question was assessed independently by two investigators using the Academy of Nutrition and Dietetics Quality Criteria Checklist. Among the 32 identified studies, nine had positive quality ratings and 23 studies had at least one major methodological issue. Main methodological issues included SSB definition and inadequate measurement of exposure. Studies with positive quality ratings found an association between SSB consumption and risk of obesity or obesity (n=5) (ie, when SSB consumption increased so did obesity) or mixed results (n=4). Studies with a neutral quality rating found a positive association (n=7), mixed results (n=9), or no association (n=7). The present review shows that the majority of studies with strong methodology indicated a positive association between SSB consumption and risk of obesity or obesity, especially among overweight children. In addition, study findings highlight the need for the careful and precise measurement of the consumption of SSBs and of important confounders. Copyright © 2016 Academy of Nutrition and Dietetics. Published by Elsevier Inc. All rights reserved.

Design guidelines for an umbilical cord blood stem cell therapy quality assessment model

NASA Astrophysics Data System (ADS)

Januszewski, Witold S.; Michałek, Krzysztof; Yagensky, Oleksandr; Wardzińska, Marta

The paper enlists the pivotal guidelines for producing an empirical umbilical cord blood stem cell therapy quality assessment model. The methodology adapted was single equation linear model with domain knowledge derived from MEDAFAR classification. The resulting model is ready for therapeutical application.

Evaluative methodology for comprehensive water quality management planning

DOE Office of Scientific and Technical Information (OSTI.GOV)

Dyer, H. L.

Computer-based evaluative methodologies have been developed to provide for the analysis of coupled phenomena associated with natural resource comprehensive planning requirements. Provisions for planner/computer interaction have been included. Each of the simulation models developed is described in terms of its coded procedures. An application of the models for water quality management planning is presented; and the data requirements for each of the models are noted.

The methodological quality of health economic evaluations for the management of hip fractures: A systematic review of the literature.

PubMed

Sabharwal, Sanjeeve; Carter, Alexander; Darzi, Lord Ara; Reilly, Peter; Gupte, Chinmay M

2015-06-01

Approximately 76,000 people a year sustain a hip fracture in the UK and the estimated cost to the NHS is £1.4 billion a year. Health economic evaluations (HEEs) are one of the methods employed by decision makers to deliver healthcare policy supported by clinical and economic evidence. The objective of this study was to (1) identify and characterize HEEs for the management of patients with hip fractures, and (2) examine their methodological quality. A literature search was performed in MEDLINE, EMBASE and the NHS Economic Evaluation Database. Studies that met the specified definition for a HEE and evaluated hip fracture management were included. Methodological quality was assessed using the Consensus on Health Economic Criteria (CHEC). Twenty-seven publications met the inclusion criteria of this study and were included in our descriptive and methodological analysis. Domains of methodology that performed poorly included use of an appropriate time horizon (66.7% of studies), incremental analysis of costs and outcomes (63%), future discounting (44.4%), sensitivity analysis (40.7%), declaration of conflicts of interest (37%) and discussion of ethical considerations (29.6%). HEEs for patients with hip fractures are increasing in publication in recent years. Most of these studies fail to adopt a societal perspective and key aspects of their methodology are poor. The development of future HEEs in this field must adhere to established principles of methodology, so that better quality research can be used to inform health policy on the management of patients with a hip fracture. Copyright © 2014 Royal College of Surgeons of Edinburgh (Scottish charity number SC005317) and Royal College of Surgeons in Ireland. Published by Elsevier Ltd. All rights reserved.

Is Article Methodological Quality Associated With Conflicts of Interest?: An Analysis of the Plastic Surgery Literature.

PubMed

Cho, Brian H; Lopez, Joseph; Means, Jessica; Lopez, Sandra; Milton, Jacqueline; Tufaro, Anthony P; May, James W; Dorafshar, Amir H

2017-12-01

Conflicts of interest (COI) are an emerging area of discussion within the field of plastic surgery. Recently, several reports have found that research studies that disclose COI are associated with publication of positive outcomes. We hypothesize that this association is driven by higher-quality studies receiving industry funding. This study aimed to investigate the association between industry support and study methodological quality. We reviewed all entries in Plastic and Reconstructive Surgery, Annals of Plastic Surgery, and Journal of Plastic, Reconstructive, and Aesthetic Surgery within a 1-year period encompassing 2013. All clinical research articles were analyzed. Studies were evaluated blindly for methodology quality based on a validated scoring system. An ordinal logistic regression model was used to examine the association between methodology score and COI. A total of 1474 articles were reviewed, of which 483 met our inclusion criteria. These articles underwent methodological quality scoring. Conflicts of interest were reported in 28 (5.8%) of these articles. After adjusting for article characteristics in the ordinal logistic regression analysis, there was no significant association between articles with COI and higher methodological scores (P = 0.7636). Plastic surgery studies that disclose COI are not associated with higher methodological quality when compared with studies that do not disclose COI. These findings suggest that although the presence of COI is associated with positive findings, the association is not shown to be driven by higher-quality studies.

Methodological challenges in assessing the environmental status of a marine ecosystem: case study of the Baltic Sea.

PubMed

Ojaveer, Henn; Eero, Margit

2011-04-29

Assessments of the environmental status of marine ecosystems are increasingly needed to inform management decisions and regulate human pressures to meet the objectives of environmental policies. This paper addresses some generic methodological challenges and related uncertainties involved in marine ecosystem assessment, using the central Baltic Sea as a case study. The objectives of good environmental status of the Baltic Sea are largely focusing on biodiversity, eutrophication and hazardous substances. In this paper, we conduct comparative evaluations of the status of these three segments, by applying different methodological approaches. Our analyses indicate that the assessment results are sensitive to a selection of indicators for ecological quality objectives that are affected by a broad spectrum of human activities and natural processes (biodiversity), less so for objectives that are influenced by a relatively narrow array of drivers (eutrophications, hazardous substances). The choice of indicator aggregation rule appeared to be of essential importance for assessment results for all three segments, whereas the hierarchical structure of indicators had only a minor influence. Trend-based assessment was shown to be a useful supplement to reference-based evaluation, being independent of the problems related to defining reference values and indicator aggregation methodologies. Results of this study will help in setting priorities for future efforts to improve environmental assessments in the Baltic Sea and elsewhere, and to ensure the transparency of the assessment procedure.

Increasing accuracy in the assessment of motion sickness: A construct methodology

NASA Technical Reports Server (NTRS)

Stout, Cynthia S.; Cowings, Patricia S.

1993-01-01

The purpose is to introduce a new methodology that should improve the accuracy of the assessment of motion sickness. This construct methodology utilizes both subjective reports of motion sickness and objective measures of physiological correlates to assess motion sickness. Current techniques and methods used in the framework of a construct methodology are inadequate. Current assessment techniques for diagnosing motion sickness and space motion sickness are reviewed, and attention is called to the problems with the current methods. Further, principles of psychophysiology that when applied will probably resolve some of these problems are described in detail.

Evaluation of the methodological quality of studies of the performance of diagnostic tests for bovine tuberculosis using QUADAS.

PubMed

Downs, Sara H; More, Simon J; Goodchild, Anthony V; Whelan, Adam O; Abernethy, Darrell A; Broughan, Jennifer M; Cameron, Angus; Cook, Alasdair J; Ricardo de la Rua-Domenech, R; Greiner, Matthias; Gunn, Jane; Nuñez-Garcia, Javier; Rhodes, Shelley; Rolfe, Simon; Sharp, Michael; Upton, Paul; Watson, Eamon; Welsh, Michael; Woolliams, John A; Clifton-Hadley, Richard S; Parry, Jessica E

2018-05-01

There has been little assessment of the methodological quality of studies measuring the performance (sensitivity and/or specificity) of diagnostic tests for animal diseases. In a systematic review, 190 studies of tests for bovine tuberculosis (bTB) in cattle (published 1934-2009) were assessed by at least one of 18 reviewers using the QUADAS (Quality Assessment of Diagnostic Accuracy Studies) checklist adapted for animal disease tests. VETQUADAS (VQ) included items measuring clarity in reporting (n = 3), internal validity (n = 9) and external validity (n = 2). A similar pattern for compliance was observed in studies of different diagnostic test types. Compliance significantly improved with year of publication for all items measuring clarity in reporting and external validity but only improved in four of the nine items measuring internal validity (p < 0.05). 107 references, of which 83 had performance data eligible for inclusion in a meta-analysis were reviewed by two reviewers. In these references, agreement between reviewers' responses was 71% for compliance, 32% for unsure and 29% for non-compliance. Mean compliance with reporting items was 2, 5.2 for internal validity and 1.5 for external validity. The index test result was described in sufficient detail in 80.1% of studies and was interpreted without knowledge of the reference standard test result in only 33.1%. Loss to follow-up was adequately explained in only 31.1% of studies. The prevalence of deficiencies observed may be due to inadequate reporting but may also reflect lack of attention to methodological issues that could bias the results of diagnostic test performance estimates. QUADAS was a useful tool for assessing and comparing the quality of studies measuring the performance of diagnostic tests but might be improved further by including explicit assessment of population sampling strategy. Crown Copyright © 2017. Published by Elsevier B.V. All rights reserved.

Biosecurity Risk Assessment Methodology (BioRAM) v. 2.0

DOE Office of Scientific and Technical Information (OSTI.GOV)

CASKEY, SUSAN; GAUDIOSO, JENNIFER; SALERNO, REYNOLDS

Sandia National Laboratories International Biological Threat Reduction Dept (SNL/IBTR) has an ongoing mission to enhance biosecurity assessment methodologies, tools, and guise. These will aid labs seeking to implement biosecurity as advocated in the recently released WHO's Biorisk Management: Lab Biosecurity Guidance. BioRAM 2.0 is the software tool developed initially using the SNL LDRD process and designed to complement the "Laboratory Biosecurity Risk Handbook" written by Ren Salerno and Jennifer Gaudioso defining biosecurity risk assessment methodologies.

Water Quality Assessment using Satellite Remote Sensing

NASA Astrophysics Data System (ADS)

Haque, Saad Ul

2016-07-01

The two main global issues related to water are its declining quality and quantity. Population growth, industrialization, increase in agriculture land and urbanization are the main causes upon which the inland water bodies are confronted with the increasing water demand. The quality of surface water has also been degraded in many countries over the past few decades due to the inputs of nutrients and sediments especially in the lakes and reservoirs. Since water is essential for not only meeting the human needs but also to maintain natural ecosystem health and integrity, there are efforts worldwide to assess and restore quality of surface waters. Remote sensing techniques provide a tool for continuous water quality information in order to identify and minimize sources of pollutants that are harmful for human and aquatic life. The proposed methodology is focused on assessing quality of water at selected lakes in Pakistan (Sindh); namely, HUBDAM, KEENJHAR LAKE, HALEEJI and HADEERO. These lakes are drinking water sources for several major cities of Pakistan including Karachi. Satellite imagery of Landsat 7 (ETM+) is used to identify the variation in water quality of these lakes in terms of their optical properties. All bands of Landsat 7 (ETM+) image are analyzed to select only those that may be correlated with some water quality parameters (e.g. suspended solids, chlorophyll a). The Optimum Index Factor (OIF) developed by Chavez et al. (1982) is used for selection of the optimum combination of bands. The OIF is calculated by dividing the sum of standard deviations of any three bands with the sum of their respective correlation coefficients (absolute values). It is assumed that the band with the higher standard deviation contains the higher amount of 'information' than other bands. Therefore, OIF values are ranked and three bands with the highest OIF are selected for the visual interpretation. A color composite image is created using these three bands. The water quality

Guidance on Data Quality Assessment for Life Cycle Inventory ...

EPA Pesticide Factsheets

Data quality within Life Cycle Assessment (LCA) is a significant issue for the future support and development of LCA as a decision support tool and its wider adoption within industry. In response to current data quality standards such as the ISO 14000 series, various entities within the LCA community have developed different methodologies to address and communicate the data quality of Life Cycle Inventory (LCI) data. Despite advances in this field, the LCA community is still plagued by the lack of reproducible data quality results and documentation. To address these issues, US EPA has created this guidance in order to further support reproducible life cycle inventory data quality results and to inform users of the proper application of the US EPA supported data quality system. The work for this report was begun in December 2014 and completed as of April 2016.The updated data quality system includes a novel approach to the pedigree matrix by addressing data quality at the flow and the process level. Flow level indicators address source reliability, temporal correlation, geographic correlation, technological correlation and data sampling methods. The process level indicators address the level of review the unit process has undergone and its completeness. This guidance is designed to be updatable as part of the LCA Research Center’s continuing commitment to data quality advancements. Life cycle assessment is increasingly being used as a tool to identify areas of

Impact Evaluation of Quality Assurance in Higher Education: Methodology and Causal Designs

ERIC Educational Resources Information Center

Leiber, Theodor; Stensaker, Bjørn; Harvey, Lee

2015-01-01

In this paper, the theoretical perspectives and general methodological elements of impact evaluation of quality assurance in higher education institutions are discussed, which should be a cornerstone of quality development in higher education and contribute to improving the knowledge about the effectiveness (or ineffectiveness) of quality…

Assessment of air quality in Haora River basin using fuzzy multiple-attribute decision making techniques.

PubMed

Singh, Ajit Pratap; Chakrabarti, Sumanta; Kumar, Sumit; Singh, Anjaney

2017-08-01

This paper deals with assessment of air quality in Haora River basin using two techniques. Initially, air quality indices were evaluated using a modified EPA method. The indices were also evaluated using a fuzzy comprehensive assessment (FCA) method. The results obtained from the fuzzy comprehensive assessment method were compared to that obtained from the modified EPA method. To illustrate the applicability of the methodology proposed herein, a case study has been presented. Air samples have been collected at 10 sampling sites located along Haora River. Six important air pollutants, namely, carbon monoxide, sulfur dioxide, nitrogen dioxide, suspended particulate matter (SPM), PM 10 , and lead, were monitored continuously, and air quality maps were generated on the GIS platform. Comparison of the methodologies has clearly highlighted superiority and robustness of the fuzzy comprehensive assessment method in determining air quality indices under study. It has effectively addressed the inherent uncertainties involved in the evaluation, modeling, and interpretation of sampling data, which was beyond the scope of the traditional weighted approaches employed otherwise. The FCA method is robust and prepares a credible platform of air quality evaluation and identification, in face of the uncertainties that remain eclipsed in the traditional approaches like the modified EPA method. The insights gained through the present study are believed to be of pivotal significance in guiding the development and implementation of effective environmental remedial action plans in the study area.

A Model of Risk Analysis in Analytical Methodology for Biopharmaceutical Quality Control.

PubMed

Andrade, Cleyton Lage; Herrera, Miguel Angel De La O; Lemes, Elezer Monte Blanco

2018-01-01

One key quality control parameter for biopharmaceutical products is the analysis of residual cellular DNA. To determine small amounts of DNA (around 100 pg) that may be in a biologically derived drug substance, an analytical method should be sensitive, robust, reliable, and accurate. In principle, three techniques have the ability to measure residual cellular DNA: radioactive dot-blot, a type of hybridization; threshold analysis; and quantitative polymerase chain reaction. Quality risk management is a systematic process for evaluating, controlling, and reporting of risks that may affects method capabilities and supports a scientific and practical approach to decision making. This paper evaluates, by quality risk management, an alternative approach to assessing the performance risks associated with quality control methods used with biopharmaceuticals, using the tool hazard analysis and critical control points. This tool provides the possibility to find the steps in an analytical procedure with higher impact on method performance. By applying these principles to DNA analysis methods, we conclude that the radioactive dot-blot assay has the largest number of critical control points, followed by quantitative polymerase chain reaction, and threshold analysis. From the analysis of hazards (i.e., points of method failure) and the associated method procedure critical control points, we conclude that the analytical methodology with the lowest risk for performance failure for residual cellular DNA testing is quantitative polymerase chain reaction. LAY ABSTRACT: In order to mitigate the risk of adverse events by residual cellular DNA that is not completely cleared from downstream production processes, regulatory agencies have required the industry to guarantee a very low level of DNA in biologically derived pharmaceutical products. The technique historically used was radioactive blot hybridization. However, the technique is a challenging method to implement in a quality

A methodology model for quality management in a general hospital.

PubMed

Stern, Z; Naveh, E

1997-01-01

A reappraisal is made of the relevance of industrial modes of quality management to the issues of medical care. Analysis of the nature of medical care, which differentiates it from the supplier-client relationships of industry, presents the main intrinsic characteristics, which create problems in application of the industrial quality management approaches to medical care. Several examples are the complexity of the relationship between the medical action and the result obtained, the client's nonacceptance of economic profitability as a value in his medical care, and customer satisfaction biased by variable standards of knowledge. The real problems unique to hospitals are addressed, and a methodology model for their quality management is offered. Included is a sample of indicator vectors, measurements of quality care, cost of medical care, quality of service, and human resources. These are based on the trilogy of planning quality, quality control, and improving quality. The conclusions confirm the inadequacy of industrial quality management approaches for medical institutions and recommend investment in formulation of appropriate concepts.

Service quality assessment of workers compensation health care delivery programs in New York using SERVQUAL.

PubMed

Arunasalam, Mark; Paulson, Albert; Wallace, William

2003-01-01

Preferred provider organizations (PPOs) provide healthcare services to an expanding proportion of the U.S. population. This paper presents a programmatic assessment of service quality in the workers' compensation environment using two different models: the PPO program model and the fee-for-service (FFS) payor model. The methodology used here will augment currently available research in workers' compensation, which has been lacking in measuring service quality determinants and assessing programmatic success/failure of managed care type programs. Results indicated that the SERVQUAL tool provided a reliable and valid clinical quality assessment tool that ascertained that PPO marketers should focus on promoting physician outreach (to show empathy) and accessibility (to show reliability) for injured workers.

Patient-reported outcomes in randomised controlled trials of prostate cancer: methodological quality and impact on clinical decision making.

PubMed

Efficace, Fabio; Feuerstein, Michael; Fayers, Peter; Cafaro, Valentina; Eastham, James; Pusic, Andrea; Blazeby, Jane

2014-09-01

Patient-reported outcomes (PRO) data from randomised controlled trials (RCTs) are increasingly used to inform patient-centred care as well as clinical and health policy decisions. The main objective of this study was to investigate the methodological quality of PRO assessment in RCTs of prostate cancer (PCa) and to estimate the likely impact of these studies on clinical decision making. A systematic literature search of studies was undertaken on main electronic databases to retrieve articles published between January 2004 and March 2012. RCTs were evaluated on a predetermined extraction form, including (1) basic trial demographics and clinical and PRO characteristics; (2) level of PRO reporting based on the recently published recommendations by the International Society for Quality of Life Research; and (3) bias, assessed using the Cochrane Risk of Bias tool. Studies were systematically analysed to evaluate their relevance for supporting clinical decision making. Sixty-five RCTs enrolling a total of 22 071 patients were evaluated, with 31 (48%) in patients with nonmetastatic disease. When a PRO difference between treatments was found, it related in most cases to symptoms only (n=29, 58%). Although the extent of missing data was generally documented (72% of RCTs), few reported details on statistical handling of this data (18%) and reasons for dropout (35%). Improvements in key methodological aspects over time were found. Thirteen (20%) RCTs were judged as likely to be robust in informing clinical decision making. Higher-quality PRO studies were generally associated with those RCTs that had higher internal validity. Including PRO in RCTs of PCa patients is critical for better evaluating the treatment effectiveness of new therapeutic approaches. Marked improvements in PRO quality reporting over time were found, and it is estimated that at least one-fifth of PRO RCTs have provided sufficient details to allow health policy makers and physicians to make critical

ASSESSMENT OF TOXICANT-INDUCED ALTERATIONS IN OVARIAN STEROIDOGENESIS: A METHODOLOGICAL OVERVIEW

EPA Science Inventory

RTD-03-035

Assessment of Toxicant-induced Alterations in Ovarian Steroidogenesis:
A Methodological Overview

Jerome M. Goldman, Susan C. Laws and Ralph L. Cooper

Abstract

A variety of methodological approaches have been used for the assessment of tox...

Assessment methodology for computer-based instructional simulations.

PubMed

Koenig, Alan; Iseli, Markus; Wainess, Richard; Lee, John J

2013-10-01

Computer-based instructional simulations are becoming more and more ubiquitous, particularly in military and medical domains. As the technology that drives these simulations grows ever more sophisticated, the underlying pedagogical models for how instruction, assessment, and feedback are implemented within these systems must evolve accordingly. In this article, we review some of the existing educational approaches to medical simulations, and present pedagogical methodologies that have been used in the design and development of games and simulations at the University of California, Los Angeles, Center for Research on Evaluation, Standards, and Student Testing. In particular, we present a methodology for how automated assessments of computer-based simulations can be implemented using ontologies and Bayesian networks, and discuss their advantages and design considerations for pedagogical use. Reprint & Copyright © 2013 Association of Military Surgeons of the U.S.

Assessing species saturation: conceptual and methodological challenges.

PubMed

Olivares, Ingrid; Karger, Dirk N; Kessler, Michael

2018-05-07

Is there a maximum number of species that can coexist? Intuitively, we assume an upper limit to the number of species in a given assemblage, or that a lineage can produce, but defining and testing this limit has proven problematic. Herein, we first outline seven general challenges of studies on species saturation, most of which are independent of the actual method used to assess saturation. Among these are the challenge of defining saturation conceptually and operationally, the importance of setting an appropriate referential system, and the need to discriminate among patterns, processes and mechanisms. Second, we list and discuss the methodological approaches that have been used to study species saturation. These approaches vary in time and spatial scales, and in the variables and assumptions needed to assess saturation. We argue that assessing species saturation is possible, but that many studies conducted to date have conceptual and methodological flaws that prevent us from currently attaining a good idea of the occurrence of species saturation. © 2018 Cambridge Philosophical Society.

[Methodology for the comprehensive evaluation of the quality of performance of activities of medical and social experts].

PubMed

Moskalenko, V F; Gorban', Ie M; Marunich, V V; Ipatov, A V; Sergiieni, O V

2001-01-01

The paper scientifically substantiates methodology, approaches, criteria, and control indices for assessment of activities of establishments of medical-and-social performance. Most indices for efficiency and certain indices for week points in the work of establishments of the service depend on interaction thereof with curative- and prophylactic institutions; the best results with the problem of prevention of disability and rehabilitation of invalids are supposed to be achieved through collaborative efforts. Other criteria and intermediate indices having an effect on the quality of activities reflect the resource- and trained personnel supplies of establishments of the service, amount of work, organizational measures designed to raise the quality of medical-and-social expert performance.

Indirect Lightning Safety Assessment Methodology

DOE Office of Scientific and Technical Information (OSTI.GOV)

Ong, M M; Perkins, M P; Brown, C G

2009-04-24

Lightning is a safety hazard for high-explosives (HE) and their detonators. In the However, the current flowing from the strike point through the rebar of the building The methodology for estimating the risk from indirect lighting effects will be presented. It has two parts: a method to determine the likelihood of a detonation given a lightning strike, and an approach for estimating the likelihood of a strike. The results of these two parts produce an overall probability of a detonation. The probability calculations are complex for five reasons: (1) lightning strikes are stochastic and relatively rare, (2) the quality ofmore » the Faraday cage varies from one facility to the next, (3) RF coupling is inherently a complex subject, (4) performance data for abnormally stressed detonators is scarce, and (5) the arc plasma physics is not well understood. Therefore, a rigorous mathematical analysis would be too complex. Instead, our methodology takes a more practical approach combining rigorous mathematical calculations where possible with empirical data when necessary. Where there is uncertainty, we compensate with conservative approximations. The goal is to determine a conservative estimate of the odds of a detonation. In Section 2, the methodology will be explained. This report will discuss topics at a high-level. The reasons for selecting an approach will be justified. For those interested in technical details, references will be provided. In Section 3, a simple hypothetical example will be given to reinforce the concepts. While the methodology will touch on all the items shown in Figure 1, the focus of this report is the indirect effect, i.e., determining the odds of a detonation from given EM fields. Professor Martin Uman from the University of Florida has been characterizing and defining extreme lightning strikes. Using Professor Uman's research, Dr. Kimball Merewether at Sandia National Laboratory in Albuquerque calculated the EM fields inside a Faraday

A methodology to analyze the quality of health information on the internet: the example of diabetic neuropathy.

PubMed

Chumber, Sundeep; Huber, Jörg; Ghezzi, Pietro

2015-02-01

The purpose of this work was to evaluate the criteria used to assess the quality of information on diabetic neuropathy on the Internet. Different search engines (Google, Yahoo, Bing, and Ask) and 1 governmental health website (MedlinePlus) were studied. The websites returned (200 for each search engine) were then classified according to their affiliation (eg, commercial, professional, patient groups). A scoring system was devised from the literature to assess quality of information. Websites were also analyzed using the 2 most widely used instruments for assessing the quality of health information, the Journal of the American Medical Association (JAMA) scoring system and the Health On the Net Foundation (HON) certification. Professional websites or health portals scored better according to most criteria. Google and MedlinePlus returned results scoring significantly higher than other engines in some of the criteria. The use of different instruments gave different results and indicates that the JAMA score and the HON certification may not be sufficient ones. This methodology could be used to evaluate the reliability and trustworthiness of information on the Internet on different topics to identify topic areas or websites where the available information is not appropriate. © 2014 The Author(s).

Quality of life assessment in children: a review of conceptual and methodological issues in multidimensional health status measures.

PubMed Central

Pal, D K

1996-01-01

STUDY OBJECTIVE: To clarify concepts and methodological problems in existing multidimensional health status measures for children. DESIGN: Thematic review of instruments found by computerised and manual searches, 1979-95. SUBJECTS: Nine health status instruments. MAIN RESULTS: Many instruments did not satisfy criteria of being child centered or family focussed; few had sufficient psychometric properties for research or clinical use; underlying conceptual assumptions were rarely explicit. CONCLUSIONS: Quality of life measures should be viewed cautiously. Interdisciplinary discussion is required, as well as discussion with children and parents, to establish constructs that are truly useful. PMID:8882220

Evaluation of Quality of Lower Limb Arthroplasty Observational Studies Using the Assessment of Quality in Lower Limb Arthroplasty (AQUILA) Checklist.

PubMed

Cowan, James B; Mlynarek, Ryan A; Nelissen, Rob G H H; Pijls, Bart G C W; Gagnier, Joel J

2015-09-01

This study used the assessment of quality in lower limb arthroplasty (AQUILA) checklist to assess the quality of lower limb arthroplasty observational studies. Among 132 studies the mean reporting quality score was 5.4 (SD=1.2) out of 8 possible points. Most studies adequately reported reasons for revisions (98%) and prosthesis brand and fixation (95%) in sufficient detail. Only 3% of studies adequately reported the number of patients unwilling to participate, 15% stated a clear primary research question or hypothesis, 11% reported a worst-case analysis or competing risk analysis for endpoints, and 42% reported more than 5% of patients were lost to follow-up. There is significant room for improvement in the reporting and methodology of lower limb arthroplasty observational studies. Level III. Copyright © 2015 Elsevier Inc. All rights reserved.

Ambient air quality programmes for health impact assessment in the WHO European region.

PubMed

Mücke, H G

2000-06-01

An important aim of air quality assessment is to provide information about population exposure and health impact assessment. Numerous epidemiological studies have already shown that exposure to excessive levels of ambient air pollutants are associated with either acute or chronic health effects. Until recently, the adequacy of monitoring population exposure in relation to quantitative assessment of health effects of air pollution was rarely considered in ambient air monitoring strategies. This made the formulation of health-related recommendations to risk management difficult and weakens preventive and other measures to reduce adverse health effects of air pollution. To improve local and national capacities for health impact assessment, the European Centre for Environment and Health of the World Health Organization has prepared methodology guidelines concerning selected aspects of air monitoring. The WHO Collaborating Centre for Air Quality Management and Air Pollution Control support efforts in line with international programmes on quality assurance and control for Europe.

Hydrogen Hazards Assessment Protocol (HHAP): Approach and Methodology

NASA Technical Reports Server (NTRS)

Woods, Stephen

2009-01-01

This viewgraph presentation reviews the approach and methodology to develop a assessment protocol for hydrogen hazards. Included in the presentation are the reasons to perform hazards assessment, the types of hazard assessments that exist, an analysis of hydrogen hazards, specific information about the Hydrogen Hazards Assessment Protocol (HHAP). The assessment is specifically tailored for hydrogen behavior. The end product of the assesment is a compilation of hazard, mitigations and associated factors to facilitate decision making and achieve the best practice.

Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research

PubMed Central

Weng, Chunhua

2013-01-01

Objective To review the methods and dimensions of data quality assessment in the context of electronic health record (EHR) data reuse for research. Materials and methods A review of the clinical research literature discussing data quality assessment methodology for EHR data was performed. Using an iterative process, the aspects of data quality being measured were abstracted and categorized, as well as the methods of assessment used. Results Five dimensions of data quality were identified, which are completeness, correctness, concordance, plausibility, and currency, and seven broad categories of data quality assessment methods: comparison with gold standards, data element agreement, data source agreement, distribution comparison, validity checks, log review, and element presence. Discussion Examination of the methods by which clinical researchers have investigated the quality and suitability of EHR data for research shows that there are fundamental features of data quality, which may be difficult to measure, as well as proxy dimensions. Researchers interested in the reuse of EHR data for clinical research are recommended to consider the adoption of a consistent taxonomy of EHR data quality, to remain aware of the task-dependence of data quality, to integrate work on data quality assessment from other fields, and to adopt systematic, empirically driven, statistically based methods of data quality assessment. Conclusion There is currently little consistency or potential generalizability in the methods used to assess EHR data quality. If the reuse of EHR data for clinical research is to become accepted, researchers should adopt validated, systematic methods of EHR data quality assessment. PMID:22733976

[Risk on bias assessment: (6) A Revised Tool for the Quality Assessment on Diagnostic Accuracy Studies (QUADAS-2)].

PubMed

Qu, Y J; Yang, Z R; Sun, F; Zhan, S Y

2018-04-10

This paper introduced the Revised Tool for the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2), including the development and comparison with the original QUADAS, and illustrated the application of QUADAS-2 in a published paper related to the study on diagnostic accuracy which was included in systematic review and Meta-analysis. QUADAS-2 presented considerable improvement over the original tool. Confused items that included in QUADAS had disappeared and the quality assessment of the original study replaced by the rating of risk on bias and applicability. This was implemented through the description on the four main domains with minimal overlapping and answering the signal questions in each domain. The risk of bias and applicability with 'high','low' or 'unclear' was in line with the risk of bias assessment of intervention studies in Cochrane, so to replace the total score of quality assessment in QUADAS. Meanwhile, QUADAS-2 was also applicable to assess the diagnostic accuracy studies in which follow-up without prognosis was involved in golden standard. It was useful to assess the overall methodological quality of the study despite more time consuming than the original QUADAS. However, QUADAS-2 needs to be modified to apply in comparative studies on diagnostic accuracy and we hope the users would follow the updates and give their feedbacks on line.

Response surface methodology for predicting quality characteristics of beef patties added with flaxseed and tomato paste.

PubMed

Valenzuela Melendres, M; Camou, J P; Torrentera Olivera, N G; Alvarez Almora, E; González Mendoza, D; Avendaño Reyes, L; González Ríos, H

2014-05-01

Response surface methodology was used to study the effect of flaxseed flour (FS) and tomato paste (TP) addition, from 0 to 10% and 0 to 20% respectively, on beef patty quality characteristics. The assessed quality characteristics were color (L, a, and b), pH and texture profile analysis (TPA). Also, sensory analysis was performed for the assessment of color, juiciness, firmness, and general acceptance. FS addition reduced L and a values and decreased weight loss of cooked products (P<0.05). An opposite effect was observed when TP was added (P<0.05). All TPA parameters decreased when percentages of FS and TP were increased in the formulation of beef patties. Furthermore, FS and TP addition adversely affected the sensory characteristics of the cooked product (P<0.05); nevertheless, all sensory characteristics evaluated had an acceptable score (>5.6). Thus FS and TP are ingredients that can be used in beef patty preparation. Copyright © 2014 Elsevier Ltd. All rights reserved.

Performance-based methodology for assessing seismic vulnerability and capacity of buildings

NASA Astrophysics Data System (ADS)

Shibin, Lin; Lili, Xie; Maosheng, Gong; Ming, Li

2010-06-01

This paper presents a performance-based methodology for the assessment of seismic vulnerability and capacity of buildings. The vulnerability assessment methodology is based on the HAZUS methodology and the improved capacitydemand-diagram method. The spectral displacement ( S d ) of performance points on a capacity curve is used to estimate the damage level of a building. The relationship between S d and peak ground acceleration (PGA) is established, and then a new vulnerability function is expressed in terms of PGA. Furthermore, the expected value of the seismic capacity index (SCev) is provided to estimate the seismic capacity of buildings based on the probability distribution of damage levels and the corresponding seismic capacity index. The results indicate that the proposed vulnerability methodology is able to assess seismic damage of a large number of building stock directly and quickly following an earthquake. The SCev provides an effective index to measure the seismic capacity of buildings and illustrate the relationship between the seismic capacity of buildings and seismic action. The estimated result is compared with damage surveys of the cities of Dujiangyan and Jiangyou in the M8.0 Wenchuan earthquake, revealing that the methodology is acceptable for seismic risk assessment and decision making. The primary reasons for discrepancies between the estimated results and the damage surveys are discussed.

Back schools in Brazil: a review of the intervention methodology, assessment tools, and results.

PubMed

Noll, Matias; Vieira, Adriane; Darski, Caroline; Candotti, Cláudia Tarragô

2014-01-01

The Back School is characterized as an educational program aimed at individuals who have chronic pain in spine. In Brazil, a growing number of research studies on the effects of such programs have been observed in the last decades. Thus, the purpose of this systematic review was to identify studies on Back Schools carried out in Brazil with a population of adults and elderlies, and to compare their intervention methodologies, data collection tools, and results. A search for scientific articles and Master's and doctoral theses in several databases was conducted. The keywords used in that search were Postural Program; Postural School; Back School; Back Care Education; Back Education, as well as the respective terms in Portuguese. Eighteen studies were found, being fourteen quantitative and four qualitative studies.The studies showed many differences between them, in both the interventions proposed and the methodological design. However, the use of similar tools for assessing pain intensity, functional capacity, and quality of life between the quantitative studies allowed a partial comparison of the efficacy of those programs.The studies included in this systematic review demonstrated, in general, the immediate efficacy in reducing pain and improving functional capacity and quality of life after the Back School. Results, however, are still conflicting regarding the efficacy in the medium and long terms. Thus, it becomes necessary to conduct further studies that include follow-up assessments in the medium and long terms in order to obtain more accurate conclusions about the efficacy of Back Schools.

Quality of Reporting of Bioequivalence Trials Comparing Generic to Brand Name Drugs: A Methodological Systematic Review

PubMed Central

van der Meersch, Amélie; Dechartres, Agnès; Ravaud, Philippe

2011-01-01

Background Generic drugs are used by millions of patients for economic reasons, so their evaluation must be highly transparent. Objective To assess the quality of reporting of bioequivalence trials comparing generic to brand-name drugs. Methodology/Principal Findings PubMed was searched for reports of bioequivalence trials comparing generic to brand-name drugs between January 2005 and December 2008. Articles were included if the aim of the study was to assess the bioequivalency of generic and brand-name drugs. We excluded case studies, pharmaco-economic evaluations, and validation dosage assays of drugs. We evaluated whether important information about funding, methodology, location of trials, and participants were reported. We also assessed whether the criteria required by the Food and Drug Administration (FDA) and the European Medicine Agency (EMA) to conclude bioequivalence were reported and that the conclusions were in agreement with the results. We identified 134 potentially relevant articles but eliminated 55 because the brand-name or generic drug status of the reference drug was unknown. Thus, we evaluated 79 articles. The funding source and location of the trial were reported in 41% and 56% of articles, respectively. The type of statistical analysis was reported in 94% of articles, but the methods to generate the randomization sequence and to conceal allocation were reported in only 15% and 5%, respectively. In total, 65 articles of single-dose trials (89%) concluded bioequivalence. Of these, 20 (31%) did not report the 3 criteria within the limits required by the FDA and 11 (17%) did not report the 2 criteria within the limits required by the EMA. Conclusions/Significance Important information to judge the validity and relevance of results are frequently missing in published reports of trials assessing generic drugs. The quality of reporting of such trials is in need of improvement. PMID:21858184

Quality and rigor of the concept mapping methodology: a pooled study analysis.

PubMed

Rosas, Scott R; Kane, Mary

2012-05-01

The use of concept mapping in research and evaluation has expanded dramatically over the past 20 years. Researchers in academic, organizational, and community-based settings have applied concept mapping successfully without the benefit of systematic analyses across studies to identify the features of a methodologically sound study. Quantitative characteristics and estimates of quality and rigor that may guide for future studies are lacking. To address this gap, we conducted a pooled analysis of 69 concept mapping studies to describe characteristics across study phases, generate specific indicators of validity and reliability, and examine the relationship between select study characteristics and quality indicators. Individual study characteristics and estimates were pooled and quantitatively summarized, describing the distribution, variation and parameters for each. In addition, variation in the concept mapping data collection in relation to characteristics and estimates was examined. Overall, results suggest concept mapping yields strong internal representational validity and very strong sorting and rating reliability estimates. Validity and reliability were consistently high despite variation in participation and task completion percentages across data collection modes. The implications of these findings as a practical reference to assess the quality and rigor for future concept mapping studies are discussed. Copyright © 2011 Elsevier Ltd. All rights reserved.

[What is the methodological quality of articles on therapeutic procedures published in Cirugía Española?].

PubMed

Manterola, Carlos; Busquets, Juli; Pascual, Marta; Grande, Luis

2006-02-01

The aim of this study was to determine the methodological quality of articles on therapeutic procedures published in Cirugía Española and to study its association with the publication year, center, and subject-matter. A bibliometric study that included all articles on therapeutic procedures published in Cirugía Española between 2001 and 2004 was performed. All kinds of clinical designs were considered, excluding editorials, review articles, letters to editor, and experimental studies. The variables analyzed were: year of publication, center, design, and methodological quality. Methodological quality was determined by a valid and reliable scale. Descriptive statistics (calculation of means, standard deviation and medians) and analytical statistics (Pearson's chi2, nonparametric, ANOVA and Bonferroni tests) were used. A total of 244 articles were studied (197 case series [81%], 28 cohort studies [12%], 17 clinical trials [7%], 1 cross sectional study and 1 case-control study [0.8%]). The studies were performed mainly in Catalonia and Murcia (22% and 16%, respectively). The most frequent subject areas were soft tissue and hepatobiliopancreatic surgery (23% and 19%, respectively). The mean and median of the methodological quality score calculated for the entire series was 10.2 +/- 3.9 points and 9.5 points, respectively. Methodological quality significantly increased by publication year (p < 0.001). An association between methodological quality and subject area was observed but no association was detected with the center performing the study. The methodological quality of articles on therapeutic procedures published in Cirugía Española between 2001 and 2004 is low. However, a statistically significant trend toward improvement was observed.

Methodological Issues in Curriculum-Based Reading Assessment.

ERIC Educational Resources Information Center

Fuchs, Lynn S.; And Others

1984-01-01

Three studies involving elementary students examined methodological issues in curriculum-based reading assessment. Results indicated that (1) whereas sample duration did not affect concurrent validity, increasing duration reduced performance instability and increased performance slopes and (2) domain size was related inversely to performance slope…

Safety assessment methodology in management of spent sealed sources.

PubMed

Mahmoud, Narmine Salah

2005-02-14

Environmental hazards can be caused from radioactive waste after their disposal. It was therefore important that safety assessment methodologies be developed and established to study and estimate the possible hazards, and institute certain safety methodologies that lead and prevent the evolution of these hazards. Spent sealed sources are specific type of radioactive waste. According to IAEA definition, spent sealed sources are unused sources because of activity decay, damage, misuse, loss, or theft. Accidental exposure of humans from spent sealed sources can occur at the moment they become spent and before their disposal. Because of that reason, safety assessment methodologies were tailored to suit the management of spent sealed sources. To provide understanding and confidence of this study, validation analysis was undertaken by considering the scenario of an accident that occurred in Egypt, June 2000 (the Meet-Halfa accident from an iridium-192 source). The text of this work includes consideration related to the safety assessment approaches of spent sealed sources which constitutes assessment context, processes leading an active source to be spent, accident scenarios, mathematical models for dose calculations, and radiological consequences and regulatory criteria. The text also includes a validation study, which was carried out by evaluating a theoretical scenario compared to the real scenario of Meet-Halfa accident depending on the clinical assessment of affected individuals.

Quality reporting of carotid intima-media thickness methodology; Current state of the science in the field of spinal cord injury.

PubMed

Hoskin, Jordan D; Miyatani, Masae; Craven, B Catharine

2017-03-30

Carotid intima-media thickness (cIMT) may be used increasingly as a cardiovascular disease (CVD) screening tool in individuals with spinal cord injury (SCI) as other routine invasive diagnostic tests are often unfeasible. However, variation in cIMT acquisition and analysis methods is an issue in the current published literature. The growth of the field is dependent on cIMT quality acquisition and analysis to ensure accurate reporting of CVD risk. The purpose of this study is to evaluate the quality of the reported methodology used to collect cIMT values in SCI. Data from 12 studies, which measured cIMT in individuals with SCI, were identified from the Medline, Embase and CINAHL databases. The quality of the reported methodologies was scored based on adherence to cIMT methodological guidelines abstracted from two consensus papers. Five studies were scored as 'moderate quality' in methodological reporting, having specified 9 to 11 of 15 quality reporting criterion. The remaining seven studies were scored as 'low quality', having reported less than 9 of 15 quality reporting criterion. No study had methodological reporting that was scored as 'high quality'. The overall reporting of quality methodology was poor in the published SCI literature. A greater adherence to current methodological guidelines is needed to advance the field of cIMT in SCI. Further research is necessary to refine cIMT acquisition and analysis guidelines to aid authors designing research and journals in screening manuscripts for publication.

Development and application of a safety assessment methodology for waste disposals

DOE Office of Scientific and Technical Information (OSTI.GOV)

Little, R.H.; Torres, C.; Schaller, K.H.

1996-12-31

As part of a European Commission funded research programme, QuantiSci (formerly the Environmental Division of Intera Information Technologies) and Instituto de Medio Ambiente of the Centro de Investigaciones Energeticas Medioambientales y Tecnologicas (IMA/CIEMAT) have developed and applied a comprehensive, yet practicable, assessment methodology for post-disposal safety assessment of land-based disposal facilities. This Safety Assessment Comparison (SACO) Methodology employs a systematic approach to the collection, evaluation and use of waste and disposal system data. It can be used to assess engineered barrier performance, the attenuating properties of host geological formations, and the long term impacts of a facility on the environmentmore » and human health, as well as allowing the comparison of different disposal options for radioactive, mixed and non-radioactive wastes. This paper describes the development of the methodology and illustrates its use.« less

Methodological quality of economic evaluations of new pharmaceuticals in The Netherlands.

PubMed

Hoomans, Ties; Severens, Johan L; van der Roer, Nicole; Delwel, Gepke O

2012-03-01

In the Netherlands, decisions about the reimbursement of new pharmaceuticals are based on cost effectiveness, as well as therapeutic value and budget impact. Since 1 January 2005, drug manufacturers are formally required to substantiate the cost effectiveness of drugs that have therapeutic added value in comparison with existing ones through pharmacoeconomic evaluations. Dutch guidelines for pharmacoeconomic research provide methods guidance, ensuring consistency in both the evidence and the decision-making process about drug reimbursement. This study reviewed the methodological quality of all 21 formally required pharmacoeconomic evaluations of new pharmaceuticals between 1 January 2005 and 1 October 2008, and verified whether these evaluations complied with pharmacoeconomic guidelines. Data on the quality of the pharmacoeconomic evaluations were extracted from the pharmacoeconomic reports published by the Dutch Health Care Insurance Board (CVZ). The Board's newsletters provided information on the advice to, and reimbursement decisions made by, the Dutch Minister of Health. All data extraction was carried out by two independent reviewers, and descriptive analyses were conducted. The methodological quality was sound in only 8 of the 21 pharmacoeconomic evaluations. In most cases, the perspective of analysis, the comparator drugs, and the reporting of both total and incremental costs and effects were correct. However, drug indication, form (i.e. cost utility/cost effectiveness) and time horizon of the evaluations were frequently flawed. Moreover, the costs and effects of the pharmaceuticals were not always analysed correctly, and modelling studies were often non-transparent. Twelve drugs were reimbursed, and nine were not. The compliance with pharmacoeconomic guidelines in economic evaluations of new pharmaceuticals can be improved. This would improve the methodological quality of the pharmacoeconomic evaluations and ensure consistency in the evidence and the

[Needs assessment to improve the applicability and methodological quality of a German S3 guideline].

PubMed

Burckhardt, Marion; Hoffmann, Cristina; Nink-Grebe, Brigitte; Sänger, Sylvia

2018-04-01

Clinical practice guidelines can change the practice in healthcare only if their recommendations are implemented in a comprehensive way. The German S3 guideline "Local Therapy of Chronic Wounds in Patients with Peripheral Vascular Disease, Chronic Venous Insufficiency, and Diabetes" will be updated in 2017. The emphasis here is on the guideline's validity, user-friendliness and implementation into practice. Therefore, the aim was to identify the improvements required in regard to the guideline's methods and content presentation. The methodological approach used was the critical appraisal of the guideline according to established quality criteria and an additional stakeholder survey. Both were conducted between August and November 2016. The guideline and its related documents were reviewed independently by two researchers according to the criteria of the "Appraisal of Guidelines for Research and Evaluation" (AGREE-II). Published reviews and peer reviews by external experts and organisations were also taken into account. For the stakeholder survey, a questionnaire with open questions was distributed by e-mail and via the Internet to health professionals and organisations involved in the care of patients with leg ulcers in Germany. The questions were aimed at amendments and new topics based on the stakeholders' experience in inpatient and outpatient care. In addition, the survey focused on gathering suggestions to improve the applicability of the guideline. Suggested new topics and amendments were summarised thematically. The stakeholders' suggestions to improve the applicability, the results of the critical appraisal and the relevant aspects of the external reviews were then summarised according to the AGREE-II domains and presented in a cause and effect diagram. 17 questionnaires (out of 864 sent out by e-mail) were returned. Due to high practice relevance, the stakeholders suggested an expansion of the inclusion criteria to patients with infected wounds and

Development and Validation of a Combined Methodology for Assessing the Total Quality Control of Herbal Medicinal Products – Application to Oleuropein Preparations

PubMed Central

Lemonakis, Nikolaos; Gikas, Evagelos; Halabalaki, Maria; Skaltsounis, Alexios-Leandros

2013-01-01

Oleuropein (OE) is a secoiridoid glycoside, which occurs mostly in the Oleaceae family presenting several pharmacological properties, including antioxidant, cardio-protective, anti-atherogenic effects etc. Based on these findings OE is commercially available, as Herbal Medicinal Product (HMP), claimed for its antioxidant effects. As there are general provisions of the medicine regulating bodies e.g. European Medicines Agency, the quality of the HMP’s must always be demonstrated. Therefore, a novel LC-MS methodology was developed and validated for the simultaneous quantification of OE and its main degradation product, hydroxytyrosol (HT), for the relevant OE claimed HMP’s. The internal standard (IS) methodology was employed and separation of OE, HT and IS was achieved on a C18 Fused Core column with 3.1 min overall run time employing the SIM method for the analytical signal acquisition. The method was validated according to the International Conference on Harmonisation requirements and the results show adequate linearity (r2 > 0.99) over a wide concentration range [0.1–15 μg/mL (n=12)] and a LLOQ value of 0.1 μg/mL, for both OE and HT. Furthermore, as it would be beneficial to control the quality taking into account all the substances of the OE claimed HMP’s; a metabolomics-like approach has been developed and applied for the total quality control of the different preparations employing UHPLC-HRMS-multivariate analysis (MVA). Four OE-claimed commercial HMP’s have been randomly selected and MVA similarity-based measurements were performed. The results showed that the examined samples could also be differentiated as evidenced according to their scores plot. Batch to batch reproducibility between the samples of the same brand has also been determined and found to be acceptable. Overall, the developed combined methodology has been found to be an efficient tool for the monitoring of the HMP’s total quality. Only one OE HMP has been found to be consistent to

Lunar Surface Habitat Configuration Assessment: Methodology and Observations

NASA Technical Reports Server (NTRS)

Carpenter, Amanda

2008-01-01

The Lunar Habitat Configuration Assessment evaluated the major habitat approaches that were conceptually developed during the Lunar Architecture Team II Study. The objective of the configuration assessment was to identify desired features, operational considerations, and risks to derive habitat requirements. This assessment only considered operations pertaining to the lunar surface and did not consider all habitat conceptual designs developed. To examine multiple architectures, the Habitation Focus Element Team defined several adequate concepts which warranted the need for a method to assess the various configurations. The fundamental requirement designed into each concept included the functional and operational capability to support a crew of four on a six-month lunar surface mission; however, other conceptual aspects were diverse in comparison. The methodology utilized for this assessment consisted of defining figure of merits, providing relevant information, and establishing a scoring system. In summary, the assessment considered the geometric configuration of each concept to determine the complexity of unloading, handling, mobility, leveling, aligning, mating to other elements, and the accessibility to the lunar surface. In theory, the assessment was designed to derive habitat requirements, potential technology development needs and identify risks associated with living and working on the lunar surface. Although the results were more subjective opposed to objective, the assessment provided insightful observations for further assessments and trade studies of lunar surface habitats. This overall methodology and resulting observations will be describe in detail and illustrative examples will be discussed.

Air quality assessment of benzo(a)pyrene from asphalt plant operation.

PubMed

Gibson, Nigel; Stewart, Robert; Rankin, Erika

2012-01-01

A study has been carried out to assess the contribution of Polycyclic Aromatic Hydrocarbons (PAHs) from asphalt plant operation, utilising Benzo(a)pyrene (BaP) as a marker for PAHs, to the background air concentration around asphalt plants in the UK. The purpose behind this assessment was to determine whether the use of published BaP emission factors based on the US Environmental Protection Agency (EPA) methodology is appropriate in the context of the UK, especially as the EPA methodology does not give BaP emission factors for all activities. The study also aimed to improve the overall understanding of BaP emissions from asphalt plants in the UK, and determine whether site location and operation is likely to influence the contribution of PAHs to ambient air quality. In order to establish whether the use of US EPA emissions factors is appropriate, the study has compared the BaP emissions measured and calculated emissions rates from two UK sites with those estimated using US EPA emission factors. A dispersion modelling exercise was carried out to show the BaP contribution to ambient air around each site. This study showed that, as the US EPA methodology does not provide factors for all emission sources on asphalt plants, their use may give rise to over- or under-estimations, particularly where sources of BaP are temperature dependent. However, the contribution of both the estimated and measured BaP concentrations to environmental concentration were low, averaging about 0.05 ng m(-3) at the boundary of the sites, which is well below the UK BaP assessment threshold of 0.25 ng m(-3). Therefore, BaP concentrations, and hence PAH concentrations, from similar asphalt plant operations are unlikely to contribute negatively to ambient air quality.

A Probabilistic Assessment Methodology for the Evaluation of Geologic Carbon Dioxide Storage

USGS Publications Warehouse

Brennan, Sean T.; Burruss, Robert A.; Merrill, Matthew D.; Freeman, P.A.; Ruppert, Leslie F.

2010-01-01

In 2007, the Energy Independence and Security Act (Public Law 110-140) authorized the U.S. Geological Survey (USGS) to conduct a national assessment of potential geologic storage resources for carbon dioxide (CO2) in cooperation with the U.S. Environmental Protection Agency and the U.S. Department of Energy. The first year of that activity was specified for development of a methodology to estimate storage potential that could be applied uniformly to geologic formations across the United States. After its release, the methodology was to receive public comment and external expert review. An initial methodology was developed and published in March 2009 (Burruss and others, 2009), and public comments were received. The report was then sent to a panel of experts for external review. The external review report was received by the USGS in December 2009. This report is in response to those external comments and reviews and describes how the previous assessment methodology (Burruss and others, 2009) was revised. The resource that is assessed is the technically accessible storage resource, which is defined as the mass of CO2 that can be stored in the pore volume of a storage formation. The methodology that is presented in this report is intended to be used for assessments at scales ranging from regional to subbasinal in which storage assessment units are defined on the basis of common geologic and hydrologic characteristics. The methodology does not apply to site-specific evaluation of storage resources or capacity.

Improving the Quality of Experience Journals: Training Educational Psychology Students in Basic Qualitative Methodology

ERIC Educational Resources Information Center

Reynolds-Keefer, Laura

2010-01-01

This study evaluates the impact of teaching basic qualitative methodology to preservice teachers enrolled in an educational psychology course in the quality of observation journals. Preservice teachers enrolled in an educational psychology course requiring 45 hr of field experience were given qualitative methodological training as a part of the…

Review of quality assessment tools for the evaluation of pharmacoepidemiological safety studies

PubMed Central

Neyarapally, George A; Hammad, Tarek A; Pinheiro, Simone P; Iyasu, Solomon

2012-01-01

Objectives Pharmacoepidemiological studies are an important hypothesis-testing tool in the evaluation of postmarketing drug safety. Despite the potential to produce robust value-added data, interpretation of findings can be hindered due to well-recognised methodological limitations of these studies. Therefore, assessment of their quality is essential to evaluating their credibility. The objective of this review was to evaluate the suitability and relevance of available tools for the assessment of pharmacoepidemiological safety studies. Design We created an a priori assessment framework consisting of reporting elements (REs) and quality assessment attributes (QAAs). A comprehensive literature search identified distinct assessment tools and the prespecified elements and attributes were evaluated. Primary and secondary outcome measures The primary outcome measure was the percentage representation of each domain, RE and QAA for the quality assessment tools. Results A total of 61 tools were reviewed. Most tools were not designed to evaluate pharmacoepidemiological safety studies. More than 50% of the reviewed tools considered REs under the research aims, analytical approach, outcome definition and ascertainment, study population and exposure definition and ascertainment domains. REs under the discussion and interpretation, results and study team domains were considered in less than 40% of the tools. Except for the data source domain, quality attributes were considered in less than 50% of the tools. Conclusions Many tools failed to include critical assessment elements relevant to observational pharmacoepidemiological safety studies and did not distinguish between REs and QAAs. Further, there is a lack of considerations on the relative weights of different domains and elements. The development of a quality assessment tool would facilitate consistent, objective and evidence-based assessments of pharmacoepidemiological safety studies. PMID:23015600

Assessment of analytical quality in Nordic clinical chemistry laboratories using data from contemporary national programs.

PubMed

Aronsson, T; Bjørnstad, P; Leskinen, E; Uldall, A; de Verdier, C H

1984-01-01

The aim of this investigation was primarily to assess analytical quality expressed as between-laboratory, within-laboratory, and total imprecision, not in order to detect laboratories with poor performance, but in the positive sense to provide data for improving critical steps in analytical methodology. The aim was also to establish the present state of the art in comparison with earlier investigations to see if improvement in analytical quality could be observed.

Applying GRADE-CERQual to qualitative evidence synthesis findings-paper 3: how to assess methodological limitations.

PubMed

Munthe-Kaas, Heather; Bohren, Meghan A; Glenton, Claire; Lewin, Simon; Noyes, Jane; Tunçalp, Özge; Booth, Andrew; Garside, Ruth; Colvin, Christopher J; Wainwright, Megan; Rashidian, Arash; Flottorp, Signe; Carlsen, Benedicte

2018-01-25

The GRADE-CERQual (Confidence in Evidence from Reviews of Qualitative research) approach has been developed by the GRADE (Grading of Recommendations Assessment, Development and Evaluation) Working Group. The approach has been developed to support the use of findings from qualitative evidence syntheses in decision-making, including guideline development and policy formulation. CERQual includes four components for assessing how much confidence to place in findings from reviews of qualitative research (also referred to as qualitative evidence syntheses): (1) methodological limitations, (2) coherence, (3) adequacy of data and (4) relevance. This paper is part of a series providing guidance on how to apply CERQual and focuses on CERQual's methodological limitations component. We developed the methodological limitations component by searching the literature for definitions, gathering feedback from relevant research communities and developing consensus through project group meetings. We tested the CERQual methodological limitations component within several qualitative evidence syntheses before agreeing on the current definition and principles for application. When applying CERQual, we define methodological limitations as the extent to which there are concerns about the design or conduct of the primary studies that contributed evidence to an individual review finding. In this paper, we describe the methodological limitations component and its rationale and offer guidance on how to assess methodological limitations of a review finding as part of the CERQual approach. This guidance outlines the information required to assess methodological limitations component, the steps that need to be taken to assess methodological limitations of data contributing to a review finding and examples of methodological limitation assessments. This paper provides guidance for review authors and others on undertaking an assessment of methodological limitations in the context of the CERQual

77 FR 70451 - Proposed Collection; Comment Request; Methodological Studies for the Population Assessment of...

Federal Register 2010, 2011, 2012, 2013, 2014

2012-11-26

... Request; Methodological Studies for the Population Assessment of Tobacco and Health (PATH) Study SUMMARY... Collection: Title: Methodological Studies for Population Assessment of Tobacco and Health (PATH) Study. Type... methodological studies to improve the PATH study instrumentation and data collection procedures. These...

An evaluation of contaminated estuarine sites using sediment quality guidelines and ecological assessment methodologies.

PubMed

Fulton, M; Key, P; Wirth, E; Leight, A K; Daugomah, J; Bearden, D; Sivertsen, S; Scott, G

2006-10-01

Toxic contaminants may enter estuarine ecosystems through a variety of pathways. When sediment contaminant levels become sufficiently high, they may impact resident biota. One approach to predict sediment-associated toxicity in estuarine ecosystems involves the use of sediment quality guidelines (ERMs, ERLs) and site-specific contaminant chemistry while a second approach utilizes site-specific ecological sampling to assess impacts at the population or community level. The goal of this study was to utilize an integrated approach including chemical contaminant analysis, sediment quality guidelines and grass shrimp population monitoring to evaluate the impact of contaminants from industrial sources. Three impacted sites and one reference site were selected for study. Grass shrimp populations were sampled using a push-netting approach. Sediment samples were collected at each site and analyzed for metals, polycyclic aromatic hydrocarbons (PAHs), polychlorinated biphenyls (PCBs) and pesticides. Contaminant levels were then compared to sediment quality guidelines. In general, grass shrimp population densities at the sites decreased as the ERM quotients increased. Grass shrimp densities were significantly reduced at the impacted site that had an ERM exceedance for chromium and the highest Mean ERM quotient. Regression analysis indicated that sediment chromium concentrations were negatively correlated with grass shrimp density. Grass shrimp size was reduced at two sites with intermediate levels of contamination. These findings support the use of both sediment quality guidelines and site-specific population monitoring to evaluate the impacts of sediment-associated contaminants in estuarine systems.

A study of undue pain and surfing: using hierarchical criteria to assess website quality.

PubMed

Lorence, Daniel; Abraham, Joanna

2008-09-01

In studies of web-based consumer health information, scant attention has been paid to the selective development of differential methodologies for website quality evaluation, or to selective grouping and analysis of specific ;domains of uncertainty' in healthcare. Our objective is to introduce a more refined model for website evaluation, and illustrate its application using assessment of websites within an area of ongoing medical uncertainty, back pain. In this exploratory technology assessment, we suggest a model for assessing these ;domains of uncertainty' within healthcare, using qualitative assessment of websites and hierarchical concepts. Using such a hierarchy of quality criteria, we review medical information provided by the most frequently accessed websites related to back pain. Websites are evaluated using standardized criteria, with results rated from the viewpoint of the consumer. Results show that standardization of quality rating across subjective content, and between commercial and niche search results, can provide a consumer-friendly dimension to health information.

[Assessing the balance of quality indicator sets of external quality assurance according to SGB V section 136].

PubMed

Doebler, Klaus; Geraedts, Max

2017-12-20

The value and usefulness of the results of indicator-based performance measurement in healthcare for different purposes do not only depend on the methodological quality of the individual indicators but also on the composition of the indicator sets. So far, the balance of the currently used indicator sets of the German mandatory national performance measurement system for hospitals has not been systematically analyzed. Due to the lack of a methodological gold standard for the assessment of balance and orientation of indicator sets we adapted the OECD concept of quality dimensions and defined four categories: 1) "Achieving primary goals of treatment", 2) "Avoiding adverse events", 3) "Indication" and 4) "Patient-centeredness". We defined rules for the assignment to the categories and analyzed the distribution of the 239 indicators from 29 medical areas in relation to these categories. 63 indicators (26.4 %) were assigned to the category "Achieving primary goals of treatment", 153 (64.0 %) to the category "Avoiding adverse events", 18 (7.5 %) to the category "Indication", one indicator (0.4 %) to the category "Patient-centeredness". Four indicators (1.7 %) addressed documentation quality. 12 of the 29 indicator sets only covered one OECD quality dimension by at least one indicator. The current indicator sets seem to be unbalanced with a strong focus on the category "Avoiding adverse events". As regards the goal of monitoring the compliance with minimal safety standards and performing improvement interventions, the direction of the indicator sets seems to be appropriate. With respect to other goals, such as for example the identification of "excellence", further development efforts are required. One relevant reason for the dominant focus on the category "Avoiding adverse events" seems to be that data sources for a follow-up and for the inclusion of the patient perspective have not been available until recently. There is a strong demand for the consequent use of

Comparison of methodologic quality and study/report characteristics between quantitative clinical nursing and nursing education research articles.

PubMed

Schneider, Barbara St Pierre; Nicholas, Jennifer; Kurrus, Jeffrey E

2013-01-01

To compare the methodologic quality and study/report characteristics between quantitative clinical nursing and nursing education research articles. The methodologic quality of quantitative nursing education research needs to advance to a higher level. Clinical research can provide guidance for nursing education to reach this level. One hundred quantitative clinical research articles from-high impact journals published in 2007 and 37 education research articles from high impact journals published in 2006 to 2007 were chosen for analysis. Clinical articles had significantly higher quality scores than education articles in three domains: number of institutions studied, type of data, and outcomes. The findings indicate three ways in which nursing education researchers can strengthen the methodologic quality of their quantitative research. With this approach, greater funding may be secured for advancing the science of nursing education.

Electronic symptom reporting between patient and provider for improved health care service quality: a systematic review of randomized controlled trials. part 2: methodological quality and effects.

PubMed

Johansen, Monika Alise; Berntsen, Gro K Rosvold; Schuster, Tibor; Henriksen, Eva; Horsch, Alexander

2012-10-03

We conducted in two parts a systematic review of randomized controlled trials (RCTs) on electronic symptom reporting between patients and providers to improve health care service quality. Part 1 reviewed the typology of patient groups, health service innovations, and research targets. Four innovation categories were identified: consultation support, monitoring with clinician support, self-management with clinician support, and therapy. To assess the methodological quality of the RCTs, and summarize effects and benefits from the methodologically best studies. We searched Medline, EMBASE, PsycINFO, Cochrane Central Register of Controlled Trials, and IEEE Xplore for original studies presented in English-language articles between 1990 and November 2011. Risk of bias and feasibility were judged according to the Cochrane recommendation, and theoretical evidence and preclinical testing were evaluated according to the Framework for Design and Evaluation of Complex Interventions to Improve Health. Three authors assessed the risk of bias and two authors extracted the effect data independently. Disagreement regarding bias assessment, extraction, and interpretation of results were resolved by consensus discussions. Of 642 records identified, we included 32 articles representing 29 studies. No articles fulfilled all quality requirements. All interventions were feasible to implement in a real-life setting, and theoretical evidence was provided for almost all studies. However, preclinical testing was reported in only a third of the articles. We judged three-quarters of the articles to have low risk for random sequence allocation and approximately half of the articles to have low risk for the following biases: allocation concealment, incomplete outcome data, and selective reporting. Slightly more than one fifth of the articles were judged as low risk for blinding of outcome assessment. Only 1 article had low risk of bias for blinding of participants and personnel. We excluded 12

Electronic Symptom Reporting Between Patient and Provider for Improved Health Care Service Quality: A Systematic Review of Randomized Controlled Trials. Part 2: Methodological Quality and Effects

PubMed Central

Berntsen, Gro K Rosvold; Schuster, Tibor; Henriksen, Eva; Horsch, Alexander

2012-01-01

Background We conducted in two parts a systematic review of randomized controlled trials (RCTs) on electronic symptom reporting between patients and providers to improve health care service quality. Part 1 reviewed the typology of patient groups, health service innovations, and research targets. Four innovation categories were identified: consultation support, monitoring with clinician support, self-management with clinician support, and therapy. Objective To assess the methodological quality of the RCTs, and summarize effects and benefits from the methodologically best studies. Methods We searched Medline, EMBASE, PsycINFO, Cochrane Central Register of Controlled Trials, and IEEE Xplore for original studies presented in English-language articles between 1990 and November 2011. Risk of bias and feasibility were judged according to the Cochrane recommendation, and theoretical evidence and preclinical testing were evaluated according to the Framework for Design and Evaluation of Complex Interventions to Improve Health. Three authors assessed the risk of bias and two authors extracted the effect data independently. Disagreement regarding bias assessment, extraction, and interpretation of results were resolved by consensus discussions. Results Of 642 records identified, we included 32 articles representing 29 studies. No articles fulfilled all quality requirements. All interventions were feasible to implement in a real-life setting, and theoretical evidence was provided for almost all studies. However, preclinical testing was reported in only a third of the articles. We judged three-quarters of the articles to have low risk for random sequence allocation and approximately half of the articles to have low risk for the following biases: allocation concealment, incomplete outcome data, and selective reporting. Slightly more than one fifth of the articles were judged as low risk for blinding of outcome assessment. Only 1 article had low risk of bias for blinding of

Architecture, Design, and System; Performance Assessment and Development Methodology for Computer-Based Systems. Volume 1. Methodology Description, Discussion, and Assessment,

DTIC Science & Technology

1983-12-30

AD-Ri46 57? ARCHITECTURE DESIGN AND SYSTEM; PERFORMANCE ASSESSMENT i/i AND DEVELOPMENT ME..(U) NAVAL SURFACE WEAPONS CENTER SILYER SPRING MD J...AD-A 146 577 NSIWC TR 83-324 ARCHITECTURE , DESIGN , AND SYSTEM; PERFORMANCE ASSESSMENT AND DEVELOPMENT METHODOLOGY...REPORT NUMBER 12. GOVT ACCESSION NO.3. RECIPIENT’S CATALOG NUMBER NSWC TR 83-324 10- 1 1 51’ 4. ?ITLE (and subtitle) ARCHITECTURE , DESIGN , AND SYSTEM; S

A systematic review of instruments that assess the implementation of hospital quality management systems.

PubMed

Groene, Oliver; Botje, Daan; Suñol, Rosa; Lopez, Maria Andrée; Wagner, Cordula

2013-10-01

Health-care providers invest substantial resources to establish and implement hospital quality management systems. Nevertheless, few tools are available to assess implementation efforts and their effect on quality and safety outcomes. This review aims to (i) identify instruments to assess the implementation of hospital quality management systems, (ii) describe their measurement properties and (iii) assess the effects of quality management on quality improvement and quality of care outcomes. We performed a systematic literature search from 1990 to 2011 in PubMed, CINAHL, EMBASE, Cochrane Library and Web of Science. In addition, we used snowball strategies, screened the reference lists of eligible papers, reviewed grey literature and contacted experts in the field. and data extraction Two reviewers screened eligible papers based on pre-defined inclusion and exclusion criteria and all authors extracted data. Eligible papers are described in terms of general characteristics (settings, type and level of respondents, mode of data collection), methodological properties (sampling strategy, item derivation, conceptualization of quality management, assessment of reliability and validity, scoring) and application/implementation (accounting for context, organizational adaptations, sensitivity to change, deployment and effect size). Eighteen papers were deemed eligible for inclusion. While some common domains emerged in measurement conceptualization, substantial differences in scope persist. The instruments' measurement properties were insufficiently described and only few instruments assessed links between the implementation of quality management systems (QMS) and improvement strategies or outcomes. There is currently no well-established measure to assess the implementation and effectiveness of quality management systems. Future research should address this gap.

A systematic review of studies evaluating Australian indigenous community development projects: the extent of community participation, their methodological quality and their outcomes.

PubMed

Snijder, Mieke; Shakeshaft, Anthony; Wagemakers, Annemarie; Stephens, Anne; Calabria, Bianca

2015-11-21

Community development is a health promotion approach identified as having great potential to improve Indigenous health, because of its potential for extensive community participation. There has been no systematic examination of the extent of community participation in community development projects and little analysis of their effectiveness. This systematic review aims to identify the extent of community participation in community development projects implemented in Australian Indigenous communities, critically appraise the qualitative and quantitative methods used in their evaluation, and summarise their outcomes. Ten electronic peer-reviewed databases and two electronic grey literature databases were searched for relevant studies published between 1990 and 2015. The level of community participation and the methodological quality of the qualitative and quantitative components of the studies were assessed against standardised criteria. Thirty one evaluation studies of community development projects were identified. Community participation varied between different phases of project development, generally high during project implementation, but low during the evaluation phase. For the majority of studies, methodological quality was low and the methods were poorly described. Although positive qualitative or quantitative outcomes were reported in all studies, only two studies reported statistically significant outcomes. Partnerships between researchers, community members and service providers have great potential to improve methodological quality and community participation when research skills and community knowledge are integrated to design, implement and evaluate community development projects. The methodological quality of studies evaluating Australian Indigenous community development projects is currently too weak to confidently determine the cost-effectiveness of community development projects in improving the health and wellbeing of Indigenous Australians

Health-Related Quality of Life in Non–Small-Cell Lung Cancer: An Update of a Systematic Review on Methodologic Issues in Randomized Controlled Trials

PubMed Central

Claassens, Lily; van Meerbeeck, Jan; Coens, Corneel; Quinten, Chantal; Ghislain, Irina; Sloan, Elizabeth K.; Wang, Xin Shelly; Velikova, Galina; Bottomley, Andrew

2011-01-01

Purpose This study is an update of a systematic review of health-related quality-of-life (HRQOL) methodology reporting in non–small-cell lung cancer (NSCLC) randomized controlled trials (RCTs). The objective was to evaluate HRQOL methodology reporting over the last decade and its benefit for clinical decision making. Methods A MEDLINE systematic literature review was performed. Eligible RCTs implemented patient-reported HRQOL assessments and regular oncology treatments for newly diagnosed adult patients with NSCLC. Included studies were published in English from August 2002 to July 2010. Two independent reviewers evaluated all included RCTs. Results Fifty-three RCTs were assessed. Of the 53 RCTs, 81% reported that there was no significant difference in overall survival (OS). However, 50% of RCTs that were unable to find OS differences reported a significant difference in HRQOL scores. The quality of HRQOL reporting has improved; both reporting of clinically significant differences and statistical testing of HRQOL have improved. A European Organisation for Research and Treatment of Cancer HRQOL questionnaire was used in 57% of the studies. However, reporting of HRQOL hypotheses and rationales for choosing HRQOL instruments were significantly less than before 2002 (P < .05). Conclusion The number of NSCLC RCTs incorporating HRQOL assessments has considerably increased. HRQOL continues to demonstrate its importance in RCTs, especially in those studies in which no OS difference is found. Despite the improved quality of HRQOL methodology reporting, certain aspects remain underrepresented. Our findings suggest need for an international standardization of HRQOL reporting similar to the CONSORT guidelines for clinical findings. PMID:21464420

Quality indicators for the assessment and management of pain in the emergency department: a systematic review.

PubMed

Stang, Antonia Schirmer; Hartling, Lisa; Fera, Cassandra; Johnson, David; Ali, Samina

2014-01-01

Evidence indicates that pain is undertreated in the emergency department (ED). The first step in improving the pain experience for ED patients is to accurately and systematically assess the actual care being provided. Identifying gaps in the assessment and treatment of pain and improving patient outcomes requires relevant, evidence-based performance measures. To systematically review the literature and identify quality indicators specific to the assessment and management of pain in the ED. Four major bibliographical databases were searched from January 1980 to December 2010, and relevant journals and conference proceedings were manually searched. Original research that described the development or collection of data on one or more quality indicators relevant to the assessment or management of pain in the ED was included. The search identified 18,078 citations. Twenty-three articles were included: 15 observational (cohort) studies; three before-after studies; three audits; one quality indicator development study; and one survey. Methodological quality was moderate, with weaknesses in the reporting of study design and methodology. Twenty unique indicators were identified, with the majority (16 of 20) measuring care processes. Overall, 91% (21 of 23) of the studies reported indicators for the assessment or management of presenting pain, as opposed to procedural pain. Three of the studies included children; however, none of the indicators were developed specifically for a pediatric population. Gaps in the existing literature include a lack of measures reflecting procedural pain, patient outcomes and the pediatric population. Future efforts should focus on developing indicators specific to these key areas.

Journal impact factor and methodological quality of surgical randomized controlled trials: an empirical study.

PubMed

Ahmed Ali, Usama; Reiber, Beata M M; Ten Hove, Joren R; van der Sluis, Pieter C; Gooszen, Hein G; Boermeester, Marja A; Besselink, Marc G

2017-11-01

The journal impact factor (IF) is often used as a surrogate marker for methodological quality. The objective of this study is to evaluate the relation between the journal IF and methodological quality of surgical randomized controlled trials (RCTs). Surgical RCTs published in PubMed in 1999 and 2009 were identified. According to IF, RCTs were divided into groups of low (<2), median (2-3) and high IF (>3), as well as into top-10 vs all other journals. Methodological quality characteristics and factors concerning funding, ethical approval and statistical significance of outcomes were extracted and compared between the IF groups. Additionally, a multivariate regression was performed. The median IF was 2.2 (IQR 2.37). The percentage of 'low-risk of bias' RCTs was 13% for top-10 journals vs 4% for other journals in 1999 (P < 0.02), and 30 vs 12% in 2009 (P < 0.02). Similar results were observed for high vs low IF groups. The presence of sample-size calculation, adequate generation of allocation and intention-to-treat analysis were independently associated with publication in higher IF journals; as were multicentre trials and multiple authors. Publication of RCTs in high IF journals is associated with moderate improvement in methodological quality compared to RCTs published in lower IF journals. RCTs with adequate sample-size calculation, generation of allocation or intention-to-treat analysis were associated with publication in a high IF journal. On the other hand, reporting a statistically significant outcome and being industry funded were not independently associated with publication in a higher IF journal.

Identification of sensitive indicators to assess the interrelationship between soil quality, management practices and human health

NASA Astrophysics Data System (ADS)

Zornoza, R.; Acosta, J. A.; Bastida, F.; Domínguez, S. G.; Toledo, D. M.; Faz, A.

2014-09-01

Soil quality (SQ) assessment has been a challenging issue since soils present high variability in properties and functions. This paper aims to increase understanding of SQ through review of SQ assessments in different scenarios providing evidence about the interrelationship between SQ, land use and human health. There is a general consensus that there is a need to develop methods to assess and monitor SQ for assuring sustainable land use with no prejudicial effects on human health. This review points out the importance of adopting indicators of different nature (physical, chemical and biological) to achieve a holistic image of SQ. Most authors use single indicators to assess SQ and its relationship with land uses, being the most used indicators soil organic carbon and pH. The use of nitrogen and nutrients content has resulted sensitive for agricultural and forest systems, together with physical properties such as texture, bulk density, available water and aggregate stability. These physical indicators have also been widely used to assess SQ after land use changes. The use of biological indicators is less generalized, being microbial biomass and enzyme activities the most selected indicators. Although most authors assess SQ using independent indicators, it is preferable to combine some of them into models to create a soil quality index (SQI), since it provides integrated information about soil processes and functioning. The majority of revised articles used the same methodology to establish a SQI, based on scoring and weighting of different soil indicators, selected by multivariate analyses. The use of multiple linear regressions has been successfully used under forest land use. Urban soil quality has been poorly assessed, with lack of adoption of SQIs. In addition, SQ assessments were human health indicators or exposure pathways are incorporated are practically inexistent. Thus, new efforts should be carried out to establish new methodologies not only to assess

Enhancing the Assessment of Verbal Aggression through Observational Methodology

ERIC Educational Resources Information Center

van Dulmen, Manfred H. M.; Mata, Andrea D.; Klipfel, Katherine M.

2012-01-01

The assessment of verbal aggression in adolescent and young adult dating relationships has largely relied on self-report methodology. We investigated whether information on verbal aggression derived from an observational assessment would enhance the prediction of romantic relationship satisfaction and dissolution in a sample of young adult dating…

Assessment of methodologies for analysis of the dungeness B accidental aircraft crash risk.

DOE Office of Scientific and Technical Information (OSTI.GOV)

LaChance, Jeffrey L.; Hansen, Clifford W.

2010-09-01

The Health and Safety Executive (HSE) has requested Sandia National Laboratories (SNL) to review the aircraft crash methodology for nuclear facilities that are being used in the United Kingdom (UK). The scope of the work included a review of one method utilized in the UK for assessing the potential for accidental airplane crashes into nuclear facilities (Task 1) and a comparison of the UK methodology against similar International Atomic Energy Agency (IAEA), United States (US) Department of Energy (DOE), and the US Nuclear Regulatory Commission (NRC) methods (Task 2). Based on the conclusions from Tasks 1 and 2, an additionalmore » Task 3 would provide an assessment of a site-specific crash frequency for the Dungeness B facility using one of the other methodologies. This report documents the results of Task 2. The comparison of the different methods was performed for the three primary contributors to aircraft crash risk at the Dungeness B site: airfield related crashes, crashes below airways, and background crashes. The methods and data specified in each methodology were compared for each of these risk contributors, differences in the methodologies were identified, and the importance of these differences was qualitatively and quantitatively assessed. The bases for each of the methods and the data used were considered in this assessment process. A comparison of the treatment of the consequences of the aircraft crashes was not included in this assessment because the frequency of crashes into critical structures is currently low based on the existing Dungeness B assessment. Although the comparison found substantial differences between the UK and the three alternative methodologies (IAEA, NRC, and DOE) this assessment concludes that use of any of these alternative methodologies would not change the conclusions reached for the Dungeness B site. Performance of Task 3 is thus not recommended.« less

Application of process mining to assess the data quality of routinely collected time-based performance data sourced from electronic health records by validating process conformance.

PubMed

Perimal-Lewis, Lua; Teubner, David; Hakendorf, Paul; Horwood, Chris

2016-12-01

Effective and accurate use of routinely collected health data to produce Key Performance Indicator reporting is dependent on the underlying data quality. In this research, Process Mining methodology and tools were leveraged to assess the data quality of time-based Emergency Department data sourced from electronic health records. This research was done working closely with the domain experts to validate the process models. The hospital patient journey model was used to assess flow abnormalities which resulted from incorrect timestamp data used in time-based performance metrics. The research demonstrated process mining as a feasible methodology to assess data quality of time-based hospital performance metrics. The insight gained from this research enabled appropriate corrective actions to be put in place to address the data quality issues. © The Author(s) 2015.

Methodology for assessing the effectiveness of access management techniques : executive summary.

DOT National Transportation Integrated Search

1998-09-14

A methodology for assessing the effectiveness of access management techniques on suburban arterial highways is developed. The methodology is described as a seven-step process as follows: (1) establish the purpose of the analysis (2) establish the mea...

A Systematic and Transparent Approach for Assessing the Methodological Quality of Intervention Effectiveness Research: The Study Design and Implementation Assessment Device (Study DIAD)

ERIC Educational Resources Information Center

Valentine, Jeffrey C.; Cooper, Harris

2008-01-01

Assessments of studies meant to evaluate the effectiveness of interventions, programs, and policies can serve an important role in the interpretation of research results. However, evidence suggests that available quality assessment tools have poor measurement characteristics and can lead to opposing conclusions when applied to the same body of…

Designing a Method with Physician Participation to Assess and Improve Quality of Healthcare in Otolaryngology.

PubMed

Arce, José M; Martín Cleary, Catalina; Cenjor, Carlos; Ramos, Ángel; Ortiz, Alberto

2017-04-01

Patient choice of healthcare centers to be treated for specific diseases is compromised by the low accessibility of understandable information. Physicians are rarely involved in healthcare quality assessment, despite their potentially valuable input. The purpose of this study was to develop a methodology for evaluating the quality of care that specifically incorporates advice from medical specialists and provides accessible information for patients in search of high-quality healthcare. A pilot Delphi study was conducted among 28 Spanish otolaryngology experts, seeking their opinion on the quality-of-care indicators and on their ability to recommend the most suitable department for the treatment of specific otolaryngologic diseases. The average acceptance rate was 91.9% for quality-of-care indicator and 96.5% for the resources needed for improving the quality of care. Furthermore, 93% experts reported that patients frequently ask for physician advice on which center provides better care for a specific disease, 92.6% experts believe they could recommend the best centers for specific otolaryngologic diseases, and 80% experts agreed that expert opinion on the quality of care offered by different centers would be a valuable addition to quality-of-care assessment. The incorporation of physician advice into healthcare quality assessment may improve the usefulness of healthcare quality indicators for patients. Assessment tools incorporating physician advice should be developed and validated.

Methodological Validation of Quality of Life Questionnaire for Coal Mining Groups-Indian Scenario

ERIC Educational Resources Information Center

Sen, Sayanti; Sen, Goutam; Tewary, B. K.

2012-01-01

Maslow's hierarchy-of-needs theory has been used to predict development of Quality of Life (QOL) in countries over time. In this paper an attempt has been taken to derive a methodological validation of quality of life questionnaire which have been prepared for the study area. The objective of the study is to standardize a questionnaire tool to…

AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews.

PubMed

Shea, Beverley J; Hamel, Candyce; Wells, George A; Bouter, Lex M; Kristjansson, Elizabeth; Grimshaw, Jeremy; Henry, David A; Boers, Maarten

2009-10-01

Our purpose was to measure the agreement, reliability, construct validity, and feasibility of a measurement tool to assess systematic reviews (AMSTAR). We randomly selected 30 systematic reviews from a database. Each was assessed by two reviewers using: (1) the enhanced quality assessment questionnaire (Overview of Quality Assessment Questionnaire [OQAQ]); (2) Sacks' instrument; and (3) our newly developed measurement tool (AMSTAR). We report on reliability (interobserver kappas of the 11 AMSTAR items), intraclass correlation coefficients (ICCs) of the sum scores, construct validity (ICCs of the sum scores of AMSTAR compared with those of other instruments), and completion times. The interrater agreement of the individual items of AMSTAR was substantial with a mean kappa of 0.70 (95% confidence interval [CI]: 0.57, 0.83) (range: 0.38-1.0). Kappas recorded for the other instruments were 0.63 (95% CI: 0.38, 0.78) for enhanced OQAQ and 0.40 (95% CI: 0.29, 0.50) for the Sacks' instrument. The ICC of the total score for AMSTAR was 0.84 (95% CI: 0.65, 0.92) compared with 0.91 (95% CI: 0.82, 0.96) for OQAQ and 0.86 (95% CI: 0.71, 0.94) for the Sacks' instrument. AMSTAR proved easy to apply, each review taking about 15 minutes to complete. AMSTAR has good agreement, reliability, construct validity, and feasibility. These findings need confirmation by a broader range of assessors and a more diverse range of reviews.

MASQOT: a method for cDNA microarray spot quality control

PubMed Central

Bylesjö, Max; Eriksson, Daniel; Sjödin, Andreas; Sjöström, Michael; Jansson, Stefan; Antti, Henrik; Trygg, Johan

2005-01-01

Background cDNA microarray technology has emerged as a major player in the parallel detection of biomolecules, but still suffers from fundamental technical problems. Identifying and removing unreliable data is crucial to prevent the risk of receiving illusive analysis results. Visual assessment of spot quality is still a common procedure, despite the time-consuming work of manually inspecting spots in the range of hundreds of thousands or more. Results A novel methodology for cDNA microarray spot quality control is outlined. Multivariate discriminant analysis was used to assess spot quality based on existing and novel descriptors. The presented methodology displays high reproducibility and was found superior in identifying unreliable data compared to other evaluated methodologies. Conclusion The proposed methodology for cDNA microarray spot quality control generates non-discrete values of spot quality which can be utilized as weights in subsequent analysis procedures as well as to discard spots of undesired quality using the suggested threshold values. The MASQOT approach provides a consistent assessment of spot quality and can be considered an alternative to the labor-intensive manual quality assessment process. PMID:16223442

Methodological quality in randomised controlled trials of transcutaneous electric nerve stimulation for pain: low fidelity may explain negative findings.

PubMed

Bennett, Michael I; Hughes, Nicola; Johnson, Mark I

2011-06-01

The benefits of transcutaneous electrical nerve stimulation (TENS) for pain relief have not been reliably established, as most systematic reviews find poor methodological quality in many studies. The paradox within the evidence base for TENS is that despite identified sources of bias that may lead to an overestimation of treatment effects, no benefits for TENS can be clearly demonstrated. Conventional assessments of quality assume a single direction of bias, and little work has been undertaken examining other directions of bias. Our hypothesis was that low fidelity in studies (bias leading to an underestimation of treatment effects) may account for inconclusive findings. We included 38 studies from 3 recently published Cochrane systematic reviews that examined TENS for acute, chronic, and cancer pain. We extracted data relating to treatment allocation, application of TENS and to the assessment of outcomes. We quantified these data and judged this against standardised assessment criteria using a "traffic light" approach based on the number of studies reaching the standard. We identified significant sources of potential bias in both directions in relation to study design and implementation fidelity that have not been quantified previously. Suboptimal dosing of TENS and inappropriate outcome assessment were particularly prevalent weaknesses indicating low fidelity. We propose criteria for judging directions of bias in future studies of TENS that may be adapted to assess other trials in which implementation fidelity is important, such as other nonpharmacological interventions for pain. Poor implementation fidelity was identified as a significant source of bias in systematic reviews of TENS studies and might explain lack of consistent treatment effects of TENS in pain. Here, criteria for assessing methodology are proposed for use in designing future clinical trials of TENS. Copyright © 2010 International Association for the Study of Pain. Published by Elsevier B.V. All

Methodology of quality improvement projects for the Texas Medicare population.

PubMed

Pendergrass, P W; Abel, R L; Bing, M; Vaughn, R; McCauley, C

1998-07-01

The Texas Medical Foundation, the quality improvement organization for the state of Texas, develops local quality improvement projects for the Medicare population. These projects are developed as part of the Health Care Quality Improvement Program undertaken by the Health Care Financing Administration. The goal of a local quality improvement project is to collaborate with providers to identify and reduce the incidence of unintentional variations in the delivery of care that negatively impact outcomes. Two factors are critical to the success of a quality improvement project. First, as opposed to peer review that is based on implicit criteria, quality improvement must be based on explicit criteria. These criteria represent key steps in the delivery of care that have been shown to improve outcomes for a specific disease. Second, quality improvement must be performed in partnership with the health care community. As such, the health care community must play an integral role in the design and evaluation of a quality improvement project and in the design and implementation of the resulting quality improvement plan. Specifically, this article provides a historical perspective for the transition from peer review to quality improvement. It discusses key steps used in developing and implementing local quality improvement projects including topic selection, quality indicator development, collaborator recruitment, and measurement of performance/improvement. Two Texas Medical Foundation projects are described to highlight the current methodology and to illustrate the impact of quality improvement projects.

Assessment of emergency general surgery care based on formally developed quality indicators.

PubMed

Ingraham, Angela; Nathens, Avery; Peitzman, Andrew; Bode, Allison; Dorlac, Gina; Dorlac, Warren; Miller, Preston; Sadeghi, Mahsa; Wasserman, Deena D; Bilimoria, Karl

2017-08-01

Emergency general surgery outcomes vary widely across the United States. The utilization of quality indicators can reduce variation and assist providers in administering care aligned with established recommendations. Previous quality indicators have not focused on emergency general surgery patients. We identified indicators of high-quality emergency general surgery care and assessed patient- and hospital-level compliance with these indicators. We utilized a modified Delphi technique (RAND Appropriateness Methodology) to develop quality indicators. Through 2 rankings, an expert panel ranked potential quality indicators for validity. We then examined historic compliance with select quality indicators after 4 nonelective procedures (cholecystectomy, appendectomy, colectomy, small bowel resection) at 4 academic centers. Of 25 indicators rated as valid, 13 addressed patient-level quality and 12 addressed hospital-level quality. Adherence with 18 indicators was assessed. Compliance with performing a cholecystectomy for acute cholecystitis within 72 hours of symptom onset ranged from 45% to 76%. Compliance with surgery start times within 3 hours from the decision to operate for uncontained perforated viscus ranged from 20% to 100%. Compliance with exploration of patients with small bowel obstructions with ischemia/impending perforation within 3 hours of the decision to operate was 0% to 88%. For 3 quality indicators (auditing 30-day unplanned readmissions/operations for patients previously managed nonoperatively, monitoring time to source control for intra-abdominal infections, and having protocols for bypass/transfer), none of the hospitals were compliant. Developing indicators for providers to assess their performance provides a foundation for specific initiatives. Adherence to quality indicators may improve the quality of emergency general surgery care provided for which current outcomes are potentially modifiable. Copyright © 2017 Elsevier Inc. All rights reserved.

Comparability of fish-based ecological quality assessments for geographically distinct Iberian regions.

PubMed

Segurado, P; Caiola, N; Pont, D; Oliveira, J M; Delaigue, O; Ferreira, M T

2014-04-01

In this work we compare two Iberian and a pan-European fish-based methods to assess ecological quality in rivers: the Fish-based Index of Biotic Integrity for Portuguese Wadeable Streams (F-IBIP), the Mediterranean Index of Biotic Integrity (IBIMED) and the pan-European Fish Index (EFI+). The results presented herein were developed in the context of the 2nd phase of the Intercalibration Exercise (IC), as required by the Water Frame Directive (WFD). The IC is aimed at ensuring comparability of the quality boundaries among the different WFD assessment methods developed by the Member States for each biological quality element. Although the two national assessment methods were developed for very distinct regions of Iberia (Western and Eastern Iberian Peninsula) they share the same methodological background: both are type-specific and guild-based multimetric indices. EFI+ is a multimetric guild-based model, but it is site-specific and uses a predictive modelling approach. The three indices were computed for all sites included in the Iberian Intercalibration database to allow the direct comparison, by means of linear regressions, of the resulting three quality values per site. The quality boundary harmonization between the two Iberian methods was only possible through an indirect comparison between the two indices, using EFI+ as a common metric. The three indices were also shown to be responsive to a common set of human induced pressures. This study highlights the need to develop general assessment methods adapted to wide geographical ranges with high species turnover to help intercalibrating assessment methods tailored for geographically more restricted regions. © 2013.

Methodology for Air Quality Forecast Downscaling from Regional- to Street-Scale

NASA Astrophysics Data System (ADS)

Baklanov, Alexander; Nuterman, Roman; Mahura, Alexander; Amstrup, Bjarne; Hansen Saas, Bent; Havskov Sørensen, Jens; Lorenzen, Thomas; Weismann, Jakob

2010-05-01

The most serious air pollution events occur in cities where there is a combination of high population density and air pollution, e.g. from vehicles. The pollutants can lead to serious human health problems, including asthma, irritation of the lungs, bronchitis, pneumonia, decreased resistance to respiratory infections, and premature death. In particular air pollution is associated with increase in cardiovascular disease and lung cancer. In 2000 WHO estimated that between 2.5 % and 11 % of total annual deaths are caused by exposure to air pollution. However, European-scale air quality models are not suited for local forecasts, as their grid-cell is typically of the order of 5 to 10km and they generally lack detailed representation of urban effects. Two suites are used in the framework of the EC FP7 project MACC (Monitoring of Atmosphere Composition and Climate) to demonstrate how downscaling from the European MACC ensemble to local-scale air quality forecast will be carried out: one will illustrate capabilities for the city of Copenhagen (Denmark); the second will focus on the city of Bucharest (Romania). This work is devoted to the first suite, where methodological aspects of downscaling from regional (European/ Denmark) to urban scale (Copenhagen), and from the urban down to street scale. The first results of downscaling according to the proposed methodology are presented. The potential for downscaling of European air quality forecasts by operating urban and street-level forecast models is evaluated. This will bring a strong support for continuous improvement of the regional forecast modelling systems for air quality in Europe, and underline clear perspectives for the future regional air quality core and downstream services for end-users. At the end of the MACC project, requirements on "how-to-do" downscaling of European air-quality forecasts to the city and street levels with different approaches will be formulated.

Guiding principles of USGS methodology for assessment of undiscovered conventional oil and gas resources

USGS Publications Warehouse

Charpentier, R.R.; Klett, T.R.

2005-01-01

During the last 30 years, the methodology for assessment of undiscovered conventional oil and gas resources used by the Geological Survey has undergone considerable change. This evolution has been based on five major principles. First, the U.S. Geological Survey has responsibility for a wide range of U.S. and world assessments and requires a robust methodology suitable for immaturely explored as well as maturely explored areas. Second, the assessments should be based on as comprehensive a set of geological and exploration history data as possible. Third, the perils of methods that solely use statistical methods without geological analysis are recognized. Fourth, the methodology and course of the assessment should be documented as transparently as possible, within the limits imposed by the inevitable use of subjective judgement. Fifth, the multiple uses of the assessments require a continuing effort to provide the documentation in such ways as to increase utility to the many types of users. Undiscovered conventional oil and gas resources are those recoverable volumes in undiscovered, discrete, conventional structural or stratigraphic traps. The USGS 2000 methodology for these resources is based on a framework of assessing numbers and sizes of undiscovered oil and gas accumulations and the associated risks. The input is standardized on a form termed the Seventh Approximation Data Form for Conventional Assessment Units. Volumes of resource are then calculated using a Monte Carlo program named Emc2, but an alternative analytic (non-Monte Carlo) program named ASSESS also can be used. The resource assessment methodology continues to change. Accumulation-size distributions are being examined to determine how sensitive the results are to size-distribution assumptions. The resource assessment output is changing to provide better applicability for economic analysis. The separate methodology for assessing continuous (unconventional) resources also has been evolving. Further

A methodology for space-time classification of groundwater quality.

PubMed

Passarella, G; Caputo, M C

2006-04-01

Safeguarding groundwater from civil, agricultural and industrial contamination is matter of great interest in water resource management. During recent years, much legislation has been produced stating the importance of groundwater as a source for drinking water supplies, underlining its vulnerability and defining the required quality standards. Thus, schematic tools, able to characterise the quality and quantity of groundwater systems, are of very great interest in any territorial planning and/or water resource management activity. This paper proposes a groundwater quality classification method which has been applied to a real aquifer, starting from several studies published by the Italian National Hydrogeologic Catastrophe Defence Group (GNDCI). The methodology is based on the concentration values of several parameters used as indexes of the natural hydro-chemical water condition and of potential man-induced modifications of groundwater quality. The resulting maps, although representative of the quality, do not include any information on its evolution in time. In this paper, this "stationary" classification method has been improved by crossing the quality classes with three indexes of temporal behaviour during recent years. It was then applied to data from monitoring campaigns, performed in spring and autumn, from 1990 to 1996, in the plain of Modena aquifer (central Italy). The results are reported in the form of space-time classification table and maps.

Quality of systematic reviews in pediatric oncology--a systematic review.

PubMed

Lundh, Andreas; Knijnenburg, Sebastiaan L; Jørgensen, Anders W; van Dalen, Elvira C; Kremer, Leontien C M

2009-12-01

To ensure evidence-based decision making in pediatric oncology systematic reviews are necessary. The objective of our study was to evaluate the methodological quality of all currently existing systematic reviews in pediatric oncology. We identified eligible systematic reviews through a systematic search of the literature. Data on clinical and methodological characteristics of the included systematic reviews were extracted. The methodological quality of the included systematic reviews was assessed using the overview quality assessment questionnaire, a validated 10-item quality assessment tool. We compared the methodological quality of systematic reviews published in regular journals with that of Cochrane systematic reviews. We included 117 systematic reviews, 99 systematic reviews published in regular journals and 18 Cochrane systematic reviews. The average methodological quality of systematic reviews was low for all ten items, but the quality of Cochrane systematic reviews was significantly higher than systematic reviews published in regular journals. On a 1-7 scale, the median overall quality score for all systematic reviews was 2 (range 1-7), with a score of 1 (range 1-7) for systematic reviews in regular journals compared to 6 (range 3-7) in Cochrane systematic reviews (p<0.001). Most systematic reviews in the field of pediatric oncology seem to have serious methodological flaws leading to a high risk of bias. While Cochrane systematic reviews were of higher methodological quality than systematic reviews in regular journals, some of them also had methodological problems. Therefore, the methodology of each individual systematic review should be scrutinized before accepting its results.

Non-pharmacological sleep interventions for youth with chronic health conditions: a critical review of the methodological quality of the evidence.

PubMed

Brown, Cary A; Kuo, Melissa; Phillips, Leah; Berry, Robyn; Tan, Maria

2013-07-01

Restorative sleep is clearly linked with well-being in youth with chronic health conditions. This review addresses the methodological quality of non-pharmacological sleep intervention (NPSI) research for youth with chronic health conditions. The Guidelines for Critical Review (GCR) and the Effective Public Health Practice Project Quality Assessment Tool (EPHPP) were used in the review. The search yielded 31 behavioural and 10 non-behavioural NPSI for review. Most studies had less than 10 participants. Autism spectrum disorders, attention deficit/hyperactivity disorders, down syndrome, intellectual disabilities, and visual impairments were the conditions that most studies focused upon. The global EPHPP scores indicated most reviewed studies were of weak quality. Only 7 studies were rated as moderate, none were strong. Studies rated as weak quality frequently had recruitment issues; non-blinded participants/parents and/or researchers; and used outcome measures without sound psychometric properties. Little conclusive evidence exists for NPSIs in this population. However, NPSIs are widely used and these preliminary studies demonstrate promising outcomes. There have not been any published reports of negative outcomes that would preclude application of the different NPSIs on a case-by-case basis guided by clinical judgement. These findings support the need for more rigorous, applied research. • Methodological Quality of Sleep Research • Disordered sleep (DS) in youth with chronic health conditions is pervasive and is important to rehabilitation therapists because DS contributes to significant functional problems across psychological, physical and emotional domains. • Rehabilitation therapists and other healthcare providers receive little education about disordered sleep and are largely unaware of the range of assessment and non-pharmacological intervention strategies that exist. An evidence-based website of pediatric sleep resources can be found at http

A methodology for post-mainshock probabilistic assessment of building collapse risk

USGS Publications Warehouse

Luco, N.; Gerstenberger, M.C.; Uma, S.R.; Ryu, H.; Liel, A.B.; Raghunandan, M.

2011-01-01

This paper presents a methodology for post-earthquake probabilistic risk (of damage) assessment that we propose in order to develop a computational tool for automatic or semi-automatic assessment. The methodology utilizes the same so-called risk integral which can be used for pre-earthquake probabilistic assessment. The risk integral couples (i) ground motion hazard information for the location of a structure of interest with (ii) knowledge of the fragility of the structure with respect to potential ground motion intensities. In the proposed post-mainshock methodology, the ground motion hazard component of the risk integral is adapted to account for aftershocks which are deliberately excluded from typical pre-earthquake hazard assessments and which decrease in frequency with the time elapsed since the mainshock. Correspondingly, the structural fragility component is adapted to account for any damage caused by the mainshock, as well as any uncertainty in the extent of this damage. The result of the adapted risk integral is a fully-probabilistic quantification of post-mainshock seismic risk that can inform emergency response mobilization, inspection prioritization, and re-occupancy decisions.

The PHM-Ethics methodology: interdisciplinary technology assessment of personal health monitoring.

PubMed

Schmidt, Silke; Verweij, Marcel

2013-01-01

The contribution briefly introduces the PHM Ethics project and the PHM methodology. Within the PHM-Ethics project, a set of tools and modules had been developed that may assist in the evaluation and assessment of new technologies for personal health monitoring, referred to as "PHM methodology" or "PHM toolbox". An overview on this interdisciplinary methodology and its comprising modules is provided, areas of application and intended target groups are indicated.

[Methodological quality of articles on therapeutic procedures published in Cirugía Española. Evaluation of the period 2005-2008].

PubMed

Manterola, Carlos; Grande, Luís

2010-04-01

To determine methodological quality of therapy articles published in Cirugía Española and to study its association with the publication year, the centre of origin and subjects. A literature study which included all therapy articles published between 2005 and 2008. All kinds of clinical designs were considered, excluding editorials, review articles, letters to editor and experimental studies. Variables analysed included: year of publication, centre of origin, design, and methodological quality of articles. A valid and reliable scale was applied to determine methodological quality. A total of 243 articles [206 series of cases (84.8%), 27 cohort studies (11.1%), 9 clinical trials (3.7%) and 1 case control study (0.4%)] were found. Studies came preferentially from Catalonia and Valencia (22.3% and 12.3% respectively). Thematic areas most frequently found were hepato-bilio-pancreatic and colorectal surgery (20.0% and 16.6%, respectively). Average and median of the methodological quality score calculated for the entire series were 9.5+/-4.3 points and 8 points, respectively. Association between methodological quality and geographical area (p=0.0101), subject area (p=0.0267), and university origin (p=0.0369) was found. A significant increase of methodological quality by publication year was observed (p=0.0004). Methodological quality of therapy articles published in Cirugía Española between 2005 and 2008 is low; but an increase tendency with statistical significance was observed.

A Methodology for Performing Effects-Based Assessments

DTIC Science & Technology

2006-03-01

can then be used to make assessments (Smith, 2002:384). 2.8 Cause-and-Effect Diagrams Developed by Dr. Kaoru Ishikawa in the field of Quality Control...John Wiley & Sons, Inc., 2004. Ishikawa , Kaoru . Guide to Quality Control. White Plains NY: Asian Productivity Organization, 1989...various causes affecting product quality by sorting out and relating the causes. Cause-and-effect diagrams are sometimes called Ishikawa diagrams, or

A new framework for interactive quality assessment with application to light field coding

NASA Astrophysics Data System (ADS)

Viola, Irene; Ebrahimi, Touradj

2017-09-01

In recent years, light field has experienced a surge of popularity, mainly due to the recent advances in acquisition and rendering technologies that have made it more accessible to the public. Thanks to image-based rendering techniques, light field contents can be rendered in real time on common 2D screens, allowing virtual navigation through the captured scenes in an interactive fashion. However, this richer representation of the scene poses the problem of reliable quality assessments for light field contents. In particular, while subjective methodologies that enable interaction have already been proposed, no work has been done on assessing how users interact with light field contents. In this paper, we propose a new framework to subjectively assess the quality of light field contents in an interactive manner and simultaneously track users behaviour. The framework is successfully used to perform subjective assessment of two coding solutions. Moreover, statistical analysis performed on the results shows interesting correlation between subjective scores and average interaction time.

78 FR 12335 - Submission for OMB review; Comment Request: Methodological Studies for the Population Assessment...

Federal Register 2010, 2011, 2012, 2013, 2014

2013-02-22

...; Comment Request: Methodological Studies for the Population Assessment of Tobacco and Health (PATH) Study... approval from OMB for methodological studies to improve the PATH study instrumentation and data collection procedures. These methodological studies will support ongoing assessment and refinement of the PATH study's...

Development of a methodological tool for the assessment of the hydromorphological conditions of lakes in Europe

NASA Astrophysics Data System (ADS)

Gay, Aurore; Argillier, Christine; Reynaud, Nathalie; Nicolas, Delphine; Baudoin, Jean-Marc

2017-04-01

The assessment of the ecological status of surface waters considering the biological, physico-chemical and hydromorphological conditions is requested by the European Water Framework Directive (WFD). If research efforts have particularly concentrated on rivers, lakes have yet received less attention. Nevertheless, due to their function of receptacles of inland waters, the habitats they provide to an important biodiversity and the numerous services they support (water supply, recreational activities, hydroelectricity), assessing the ecological quality of lakes becomes crucial for their protection. Still, this task remains challenging, especially considering the hydromorphological compartments. Indeed, while promising tools already exist to assess the lake biological and physico-chemical status, our comprehension of the impact of hydromophological impairments on the global ecosystem functioning remains poor and existing tools to assess such impacts often focus only on morphological aspects and in a qualitative rather than quantitative way. In this context, our study aims at providing stakeholders with a methodology to assess quantitatively the hydrological and morphological quality of lakes in Europe. The developed methodology, LAKe HYdromorphological Conditions tool (LAKHYC tool) is based on our current knowledge of the functioning of lakes and pre-existing works (e.g., Rowan et al., 2012; Rinaldi et al., 2013). The LAKHYC tool integrates the six parameters requested by the WFD, each one being assessed by at least three descriptors that are calculated as Ecological Quality Ratios, i.e. as the deviation from a reference condition. The originality of the present method lies in the fact that specific reference conditions are defined for each descriptor. In this way, we avoid using a predetermined set of lakes considered as not impacted by human activities and which often corresponds to natural lakes in specific areas (e.g., mountains) and do not represent the diversity

Objective Assessment of Image Quality VI: Imaging in Radiation Therapy

PubMed Central

Barrett, Harrison H.; Kupinski, Matthew A.; Müeller, Stefan; Halpern, Howard J.; Morris, John C.; Dwyer, Roisin

2015-01-01

Earlier work on Objective Assessment of Image Quality (OAIQ) focused largely on estimation or classification tasks in which the desired outcome of imaging is accurate diagnosis. This paper develops a general framework for assessing imaging quality on the basis of therapeutic outcomes rather than diagnostic performance. By analogy to Receiver Operating Characteristic (ROC) curves and their variants as used in diagnostic OAIQ, the method proposed here utilizes the Therapy Operating Characteristic or TOC curves, which are plots of the probability of tumor control vs. the probability of normal-tissue complications as the overall dose level of a radiotherapy treatment is varied. The proposed figure of merit is the area under the TOC curve, denoted AUTOC. This paper reviews an earlier exposition of the theory of TOC and AUTOC, which was specific to the assessment of image-segmentation algorithms, and extends it to other applications of imaging in external-beam radiation treatment as well as in treatment with internal radioactive sources. For each application, a methodology for computing the TOC is presented. A key difference between ROC and TOC is that the latter can be defined for a single patient rather than a population of patients. PMID:24200954

A new methodology for assessing health policy and systems research and analysis capacity in African universities.

PubMed

Lê, Gillian; Mirzoev, Tolib; Orgill, Marsha; Erasmus, Ermin; Lehmann, Uta; Okeyo, Stephen; Goudge, Jane; Maluka, Stephen; Uzochukwu, Benjamin; Aikins, Moses; de Savigny, Don; Tomson, Goran; Gilson, Lucy

2014-10-08

The importance of health policy and systems research and analysis (HPSR+A) has been increasingly recognised, but it is still unclear how most effectively to strengthen the capacity of the different organisations involved in this field. Universities are particularly crucial but the expansive literature on capacity development has little to offer the unique needs of HPSR+A activity within universities, and often overlooks the pivotal contribution of capacity assessments to capacity strengthening. The Consortium for Health Policy and Systems Analysis in Africa 2011-2015 designed and implemented a new framework for capacity assessment for HPSR+A within universities. The methodology is reported in detail. Our reflections on developing and conducting the assessment generated four lessons for colleagues in the field. Notably, there are currently no published capacity assessment methodologies for HPSR+A that focus solely on universities - we report a first for the field to initiate the dialogue and exchange of experiences with others. Second, in HPSR+A, the unit of assessment can be a challenge, because HPSR+A groups within universities tend to overlap between academic departments and are embedded in different networks. Third, capacity assessment experience can itself be capacity strengthening, even when taking into account that doing such assessments require capacity. From our experience, we propose that future systematic assessments of HPSR+A capacity need to focus on both capacity assets and needs and assess capacity at individual, organisational, and systems levels, whilst taking into account the networked nature of HPSR+A activity. A genuine partnership process between evaluators and those participating in an assessment can improve the quality of assessment and uptake of results in capacity strengthening.

The Northeast Stream Quality Assessment

USGS Publications Warehouse

Van Metre, Peter C.; Riva-Murray, Karen; Coles, James F.

2016-04-22

In 2016, the U.S. Geological Survey (USGS) National Water-Quality Assessment (NAWQA) is assessing stream quality in the northeastern United States. The goal of the Northeast Stream Quality Assessment (NESQA) is to assess the quality of streams in the region by characterizing multiple water-quality factors that are stressors to aquatic life and evaluating the relation between these stressors and biological communities. The focus of NESQA in 2016 will be on the effects of urbanization and agriculture on stream quality in all or parts of eight states: Connecticut, Massachusetts, New Hampshire, New Jersey, New York, Pennsylvania, Rhode Island, and Vermont.Findings will provide the public and policymakers with information about the most critical factors affecting stream quality, thus providing insights about possible approaches to protect the health of streams in the region. The NESQA study will be the fourth regional study conducted as part of NAWQA and will be of similar design and scope to the first three, in the Midwest in 2013, the Southeast in 2014, and the Pacific Northwest in 2015 (http://txpub.usgs.gov/RSQA/).

The cost-effectiveness of exercise-based cardiac rehabilitation: a systematic review of the characteristics and methodological quality of published literature.

PubMed

Edwards, Katherine; Jones, Natasha; Newton, Julia; Foster, Charlie; Judge, Andrew; Jackson, Kate; Arden, Nigel K; Pinedo-Villanueva, Rafael

2017-10-19

This descriptive review aimed to assess the characteristics and methodological quality of economic evaluations of cardiac rehabilitation (CR) programs according to updated economic guidelines for healthcare interventions. Recommendations will be made to inform future research addressing the impact of a physical exercise component on cost-effectiveness. Electronic databases were searched for economic evaluations of exercise-based CR programs published in English between 2000 and 2014. The Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement was used to review the methodological quality of included economic evaluations. Fifteen economic evaluations met the review inclusion criteria. Assessed study characteristics exhibited wide variability, particularly in their economic perspective, time horizon, setting, comparators and included costs, with significant heterogeneity in exercise dose across interventions. Ten evaluations were based on randomised controlled trials (RCTs) spanning 6-24 months but often with weak or inconclusive results; two were modelling studies; and the final three utilised longer time horizons of 3.5-5 years from which findings suggest that long-term exercise-based CR results in lower costs, reduced hospitalisations and a longer cumulative patient lifetime. None of the 15 articles met all the CHEERS quality criteria, with the majority either fully or partially meeting a selection of the assessed variables. Evidence exists supporting the cost-effectiveness of exercise-based CR for cardiovascular disease patients. However, variability in CR program delivery and weak consistency between study perspective and design limits study comparability and therefore the accumulation of evidence in support of a particular exercise regime. The generalisability of study findings was limited due to the exclusion of patients with comorbidities as would typically be found in a real-world setting. The use of longer time-horizons would be

A parsimonious dynamic model for river water quality assessment.

PubMed

Mannina, Giorgio; Viviani, Gaspare

2010-01-01

Water quality modelling is of crucial importance for the assessment of physical, chemical, and biological changes in water bodies. Mathematical approaches to water modelling have become more prevalent over recent years. Different model types ranging from detailed physical models to simplified conceptual models are available. Actually, a possible middle ground between detailed and simplified models may be parsimonious models that represent the simplest approach that fits the application. The appropriate modelling approach depends on the research goal as well as on data available for correct model application. When there is inadequate data, it is mandatory to focus on a simple river water quality model rather than detailed ones. The study presents a parsimonious river water quality model to evaluate the propagation of pollutants in natural rivers. The model is made up of two sub-models: a quantity one and a quality one. The model employs a river schematisation that considers different stretches according to the geometric characteristics and to the gradient of the river bed. Each stretch is represented with a conceptual model of a series of linear channels and reservoirs. The channels determine the delay in the pollution wave and the reservoirs cause its dispersion. To assess the river water quality, the model employs four state variables: DO, BOD, NH(4), and NO. The model was applied to the Savena River (Italy), which is the focus of a European-financed project in which quantity and quality data were gathered. A sensitivity analysis of the model output to the model input or parameters was done based on the Generalised Likelihood Uncertainty Estimation methodology. The results demonstrate the suitability of such a model as a tool for river water quality management.

An integrated science-based methodology to assess potential ...

EPA Pesticide Factsheets

There is an urgent need for broad and integrated studies that address the risks of engineered nanomaterials (ENMs) along the different endpoints of the society, environment, and economy (SEE) complex adaptive system. This article presents an integrated science-based methodology to assess the potential risks of engineered nanomaterials. To achieve the study objective, two major tasks are accomplished, knowledge synthesis and algorithmic computational methodology. The knowledge synthesis task is designed to capture “what is known” and to outline the gaps in knowledge from ENMs risk perspective. The algorithmic computational methodology is geared toward the provision of decisions and an understanding of the risks of ENMs along different endpoints for the constituents of the SEE complex adaptive system. The approach presented herein allows for addressing the formidable task of assessing the implications and risks of exposure to ENMs, with the long term goal to build a decision-support system to guide key stakeholders in the SEE system towards building sustainable ENMs and nano-enabled products. The following specific aims are formulated to achieve the study objective: (1) to propose a system of systems (SoS) architecture that builds a network management among the different entities in the large SEE system to track the flow of ENMs emission, fate and transport from the source to the receptor; (2) to establish a staged approach for knowledge synthesis methodo

Concepts of Quality in Student Assessment.

ERIC Educational Resources Information Center

Harlen, Wynne

This paper gives an overview of the methods of moderation, or quality assurance and quality control, as they may be more widely known, that are used to enhance the quality of student assessment. The discussion is based on the educational systems of the United Kingdom but is applicable to assessment in other countries. Quality in assessment is seen…

A methodology for the assessment of manned flight simulator fidelity

NASA Technical Reports Server (NTRS)

Hess, Ronald A.; Malsbury, Terry N.

1989-01-01

A relatively simple analytical methodology for assessing the fidelity of manned flight simulators for specific vehicles and tasks is offered. The methodology is based upon an application of a structural model of the human pilot, including motion cue effects. In particular, predicted pilot/vehicle dynamic characteristics are obtained with and without simulator limitations. A procedure for selecting model parameters can be implemented, given a probable pilot control strategy. In analyzing a pair of piloting tasks for which flight and simulation data are available, the methodology correctly predicted the existence of simulator fidelity problems. The methodology permitted the analytical evaluation of a change in simulator characteristics and indicated that a major source of the fidelity problems was a visual time delay in the simulation.

Identification of sensitive indicators to assess the interrelationship between soil quality, management practices and human health

NASA Astrophysics Data System (ADS)

Zornoza, R.; Acosta, J. A.; Bastida, F.; Domínguez, S. G.; Toledo, D. M.; Faz, A.

2015-02-01

Soil quality (SQ) assessment has long been a challenging issue, since soils present high variability in properties and functions. This paper aims to increase the understanding of SQ through the review of SQ assessments in different scenarios providing evidence about the interrelationship between SQ, land use and human health. There is a general consensus that there is a need to develop methods to assess and monitor SQ for assuring sustainable land use with no prejudicial effects on human health. This review points out the importance of adopting indicators of different nature (physical, chemical and biological) to achieve a holistic image of SQ. Most authors use single indicators to assess SQ and its relationship with land uses - soil organic carbon and pH being the most used indicators. The use of nitrogen and nutrient content has resulted sensitive for agricultural and forest systems, together with physical properties such as texture, bulk density, available water and aggregate stability. These physical indicators have also been widely used to assess SQ after land use changes. The use of biological indicators is less generalized, with microbial biomass and enzyme activities being the most selected indicators. Although most authors assess SQ using independent indicators, it is preferable to combine some of them into models to create a soil quality index (SQI), since it provides integrated information about soil processes and functioning. The majority of revised articles used the same methodology to establish an SQI, based on scoring and weighting of different soil indicators, selected by means of multivariate analyses. The use of multiple linear regressions has been successfully used for forest land use. Urban soil quality has been poorly assessed, with a lack of adoption of SQIs. In addition, SQ assessments where human health indicators or exposure pathways are incorporated are practically inexistent. Thus, further efforts should be carried out to establish new

The reporting characteristics and methodological quality of Cochrane reviews about health policy research.

PubMed

Xiu-xia, Li; Ya, Zheng; Yao-long, Chen; Ke-hu, Yang; Zong-jiu, Zhang

2015-04-01

The systematic review has increasingly become a popular tool for researching health policy. However, due to the complexity and diversity in the health policy research, it has also encountered more challenges. We set out the Cochrane reviews on health policy research as a representative to provide the first examination of epidemiological and descriptive characteristics as well as the compliance of methodological quality with the AMSTAR. 99 reviews were included by inclusion criteria, 73% of which were Implementation Strategies, 15% were Financial Arrangements and 12% were Governance Arrangements; involved Public Health (34%), Theoretical Exploration (18%), Hospital Management (17%), Medical Insurance (12%), Pharmaceutical Policy (9%), Community Health (7%) and Rural Health (2%). Only 39% conducted meta-analysis, and 49% reported being updates, and none was rated low methodological quality. Our research reveals that the quantity and quality of the evidence should be improved, especially Financial Arrangements and Governance Arrangements involved Rural Health, Health Care Reform and Health Equity, etc. And the reliability of AMSTAR needs to be tested in larger range in this field. Copyright © 2014 Elsevier Ireland Ltd. All rights reserved.

Systematic review of the application of quality improvement methodologies from the manufacturing industry to surgical healthcare.

PubMed

Nicolay, C R; Purkayastha, S; Greenhalgh, A; Benn, J; Chaturvedi, S; Phillips, N; Darzi, A

2012-03-01

The demand for the highest-quality patient care coupled with pressure on funding has led to the increasing use of quality improvement (QI) methodologies from the manufacturing industry. The aim of this systematic review was to identify and evaluate the application and effectiveness of these QI methodologies to the field of surgery. MEDLINE, the Cochrane Database, Allied and Complementary Medicine Database, British Nursing Index, Cumulative Index to Nursing and Allied Health Literature, Embase, Health Business(™) Elite, the Health Management Information Consortium and PsycINFO(®) were searched according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement. Empirical studies were included that implemented a described QI methodology to surgical care and analysed a named outcome statistically. Some 34 of 1595 articles identified met the inclusion criteria after consensus from two independent investigators. Nine studies described continuous quality improvement (CQI), five Six Sigma, five total quality management (TQM), five plan-do-study-act (PDSA) or plan-do-check-act (PDCA) cycles, five statistical process control (SPC) or statistical quality control (SQC), four Lean and one Lean Six Sigma; 20 of the studies were undertaken in the USA. The most common aims were to reduce complications or improve outcomes (11), to reduce infection (7), and to reduce theatre delays (7). There was one randomized controlled trial. QI methodologies from industry can have significant effects on improving surgical care, from reducing infection rates to increasing operating room efficiency. The evidence is generally of suboptimal quality, and rigorous randomized multicentre studies are needed to bring evidence-based management into the same league as evidence-based medicine. Copyright © 2011 British Journal of Surgery Society Ltd. Published by John Wiley & Sons, Ltd.

A Methodology for the Development of a Reliability Database for an Advanced Reactor Probabilistic Risk Assessment

DOE Office of Scientific and Technical Information (OSTI.GOV)

Grabaskas, Dave; Brunett, Acacia J.; Bucknor, Matthew

GE Hitachi Nuclear Energy (GEH) and Argonne National Laboratory are currently engaged in a joint effort to modernize and develop probabilistic risk assessment (PRA) techniques for advanced non-light water reactors. At a high level the primary outcome of this project will be the development of next-generation PRA methodologies that will enable risk-informed prioritization of safety- and reliability-focused research and development, while also identifying gaps that may be resolved through additional research. A subset of this effort is the development of a reliability database (RDB) methodology to determine applicable reliability data for inclusion in the quantification of the PRA. The RDBmore » method developed during this project seeks to satisfy the requirements of the Data Analysis element of the ASME/ANS Non-LWR PRA standard. The RDB methodology utilizes a relevancy test to examine reliability data and determine whether it is appropriate to include as part of the reliability database for the PRA. The relevancy test compares three component properties to establish the level of similarity to components examined as part of the PRA. These properties include the component function, the component failure modes, and the environment/boundary conditions of the component. The relevancy test is used to gauge the quality of data found in a variety of sources, such as advanced reactor-specific databases, non-advanced reactor nuclear databases, and non-nuclear databases. The RDB also establishes the integration of expert judgment or separate reliability analysis with past reliability data. This paper provides details on the RDB methodology, and includes an example application of the RDB methodology for determining the reliability of the intermediate heat exchanger of a sodium fast reactor. The example explores a variety of reliability data sources, and assesses their applicability for the PRA of interest through the use of the relevancy test.« less

Development of a Probabilistic Assessment Methodology for Evaluation of Carbon Dioxide Storage

USGS Publications Warehouse

Burruss, Robert A.; Brennan, Sean T.; Freeman, P.A.; Merrill, Matthew D.; Ruppert, Leslie F.; Becker, Mark F.; Herkelrath, William N.; Kharaka, Yousif K.; Neuzil, Christopher E.; Swanson, Sharon M.; Cook, Troy A.; Klett, Timothy R.; Nelson, Philip H.; Schenk, Christopher J.

2009-01-01

This report describes a probabilistic assessment methodology developed by the U.S. Geological Survey (USGS) for evaluation of the resource potential for storage of carbon dioxide (CO2) in the subsurface of the United States as authorized by the Energy Independence and Security Act (Public Law 110-140, 2007). The methodology is based on USGS assessment methodologies for oil and gas resources created and refined over the last 30 years. The resource that is evaluated is the volume of pore space in the subsurface in the depth range of 3,000 to 13,000 feet that can be described within a geologically defined storage assessment unit consisting of a storage formation and an enclosing seal formation. Storage assessment units are divided into physical traps (PTs), which in most cases are oil and gas reservoirs, and the surrounding saline formation (SF), which encompasses the remainder of the storage formation. The storage resource is determined separately for these two types of storage. Monte Carlo simulation methods are used to calculate a distribution of the potential storage size for individual PTs and the SF. To estimate the aggregate storage resource of all PTs, a second Monte Carlo simulation step is used to sample the size and number of PTs. The probability of successful storage for individual PTs or the entire SF, defined in this methodology by the likelihood that the amount of CO2 stored will be greater than a prescribed minimum, is based on an estimate of the probability of containment using present-day geologic knowledge. The report concludes with a brief discussion of needed research data that could be used to refine assessment methodologies for CO2 sequestration.

Quality Assurance and Its Impact from Higher Education Institutions' Perspectives: Methodological Approaches, Experiences and Expectations

ERIC Educational Resources Information Center

Bejan, Stelian Andrei; Janatuinen, Tero; Jurvelin, Jouni; Klöpping, Susanne; Malinen, Heikki; Minke, Bernhard; Vacareanu, Radu

2015-01-01

This paper reports on methodological approaches, experiences and expectations referring to impact analysis of quality assurance from the perspective of three higher education institutions (students, teaching staff, quality managers) from Germany, Finland and Romania. The presentations of the three sample institutions focus on discussing the core…

A quality assessment of randomized controlled trial reports in endodontics.

PubMed

Lucena, C; Souza, E M; Voinea, G C; Pulgar, R; Valderrama, M J; De-Deus, G

2017-03-01

To assess the quality of the randomized clinical trial (RCT) reports published in Endodontics between 1997 and 2012. Retrieval of RCTs in Endodontics was based on a search of the Thomson Reuters Web of Science (WoS) database (March 2013). Quality evaluation was performed using a checklist based on the Jadad criteria, CONSORT (Consolidated Standards of Reporting Trials) statement and SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials). Descriptive statistics were used for frequency distribution of data. Student's t-test and Welch test were used to identify the influence of certain trial characteristics upon report quality (α = 0.05). A total of 89 RCTs were evaluated, and several methodological flaws were found: only 45% had random sequence generation at low risk of bias, 75% did not provide information on allocation concealment, and 19% were nonblinded designs. Regarding statistics, only 55% of the RCTs performed adequate sample size estimations, only 16% presented confidence intervals, and 25% did not provide the exact P-value. Also, 2% of the articles used no statistical tests, and in 87% of the RCTs, the information provided was insufficient to determine whether the statistical methodology applied was appropriate or not. Significantly higher scores were observed for multicentre trials (P = 0.023), RCTs signed by more than 5 authors (P = 0.03), articles belonging to journals ranked above the JCR median (P = 0.03), and articles complying with the CONSORT guidelines (P = 0.000). The quality of RCT reports in key areas for internal validity of the study was poor. Several measures, such as compliance with the CONSORT guidelines, are important in order to raise the quality of RCTs in Endodontics. © 2016 International Endodontic Journal. Published by John Wiley & Sons Ltd.

Quality indicators for hip fracture care, a systematic review.

PubMed

Voeten, S C; Krijnen, P; Voeten, D M; Hegeman, J H; Wouters, M W J M; Schipper, I B

2018-05-17

Quality indicators are used to measure quality of care and enable benchmarking. An overview of all existing hip fracture quality indicators is lacking. The primary aim was to identify quality indicators for hip fracture care reported in literature, hip fracture audits, and guidelines. The secondary aim was to compose a set of methodologically sound quality indicators for the evaluation of hip fracture care in clinical practice. A literature search according to the PRISMA guidelines and an internet search were performed to identify hip fracture quality indicators. The indicators were subdivided into process, structure, and outcome indicators. The methodological quality of the indicators was judged using the Appraisal of Indicators through Research and Evaluation (AIRE) instrument. For structure and process indicators, the construct validity was assessed. Sixteen publications, nine audits and five guidelines were included. In total, 97 unique quality indicators were found: 9 structure, 63 process, and 25 outcome indicators. Since detailed methodological information about the indicators was lacking, the AIRE instrument could not be applied. Seven indicators correlated with an outcome measure. A set of nine quality indicators was extracted from the literature, audits, and guidelines. Many quality indicators are described and used. Not all of them correlate with outcomes of care and have been assessed methodologically. As methodological evidence is lacking, we recommend the extracted set of nine indicators to be used as the starting point for further clinical research. Future research should focus on assessing the clinimetric properties of the existing quality indicators.

The California stream quality assessment

USGS Publications Warehouse

Van Metre, Peter C.; Egler, Amanda L.; May, Jason T.

2017-03-06

In 2017, the U.S. Geological Survey (USGS) National Water-Quality Assessment (NAWQA) project is assessing stream quality in coastal California, United States. The USGS California Stream Quality Assessment (CSQA) will sample streams over most of the Central California Foothills and Coastal Mountains ecoregion (modified from Griffith and others, 2016), where rapid urban growth and intensive agriculture in the larger river valleys are raising concerns that stream health is being degraded. Findings will provide the public and policy-makers with information regarding which human and natural factors are the most critical in affecting stream quality and, thus, provide insights about possible approaches to protect the health of streams in the region.

Sexual health education interventions for young people: a methodological review.

PubMed Central

Oakley, A.; Fullerton, D.; Holland, J.; Arnold, S.; France-Dawson, M.; Kelley, P.; McGrellis, S.

1995-01-01

OBJECTIVES--To locate reports of sexual health education interventions for young people, assess the methodological quality of evaluations, identify the subgroup with a methodologically sound design, and assess the evidence with respect to the effectiveness of different approaches to promoting young people's sexual health. DESIGN--Survey of reports in English by means of electronic databases and hand searches for relevant studies conducted in the developed world since 1982. Papers were reviewed for eight methodological qualities. The evidence on effectiveness generated by studies meeting four core criteria was assessed. Judgments on effectiveness by reviewers and authors were compared. PAPERS--270 papers reporting sexual health interventions. MAIN OUTCOME MEASURE--The methodological quality of evaluations. RESULTS--73 reports of evaluations of sexual health interventions examining the effectiveness of these interventions in changing knowledge, attitudes, or behavioural outcomes were identified, of which 65 were separate outcome evaluations. Of these studies, 45 (69%) lacked random control groups, 44 (68%) failed to present preintervention and 38 (59%) postintervention data, and 26 (40%) omitted to discuss the relevance of loss of data caused by drop outs. Only 12 (18%) of the 65 outcome evaluations were judged to be methodologically sound. Academic reviewers were more likely than authors to judge studies as unclear because of design faults. Only two of the sound evaluations recorded interventions which were effective in showing an impact on young people's sexual behaviour. CONCLUSIONS--The design of evaluations in sexual health intervention needs to be improved so that reliable evidence of the effectiveness of different approaches to promoting young people's sexual health may be generated. PMID:7833754

Systematic reviews identify important methodological flaws in stroke rehabilitation therapy primary studies: review of reviews.

PubMed

Santaguida, Pasqualina; Oremus, Mark; Walker, Kathryn; Wishart, Laurie R; Siegel, Karen Lohmann; Raina, Parminder

2012-04-01

A "review of reviews" was undertaken to assess methodological issues in studies evaluating nondrug rehabilitation interventions in stroke patients. MEDLINE, CINAHL, PsycINFO, and the Cochrane Database of Systematic Reviews were searched from January 2000 to January 2008 within the stroke rehabilitation setting. Electronic searches were supplemented by reviews of reference lists and citations identified by experts. Eligible studies were systematic reviews; excluded citations were narrative reviews or reviews of reviews. Review characteristics and criteria for assessing methodological quality of primary studies within them were extracted. The search yielded 949 English-language citations. We included a final set of 38 systematic reviews. Cochrane reviews, which have a standardized methodology, were generally of higher methodological quality than non-Cochrane reviews. Most systematic reviews used standardized quality assessment criteria for primary studies, but not all were comprehensive. Reviews showed that primary studies had problems with randomization, allocation concealment, and blinding. Baseline comparability, adverse events, and co-intervention or contamination were not consistently assessed. Blinding of patients and providers was often not feasible and was not evaluated as a source of bias. The eligible systematic reviews identified important methodological flaws in the evaluated primary studies, suggesting the need for improvement of research methods and reporting. Copyright © 2012 Elsevier Inc. All rights reserved.

Assessment of Teaching Quality: Survey of University Graduates

ERIC Educational Resources Information Center

Stankeviciene, Jurate

2007-01-01

Purpose: The purpose was to investigate higher education quality because there are links between the main society and university graduates and the university. Methodology: This research sought aspects of the teaching quality of the Faculty that could be improved. The spheres were: improvement in qualifications of the teachers; finding ways to…

Association between funding source, methodological quality and research outcomes in randomized controlled trials of synbiotics, probiotics and prebiotics added to infant formula: A Systematic Review

PubMed Central

2013-01-01

Background There is little or no information available on the impact of funding by the food industry on trial outcomes and methodological quality of synbiotics, probiotics and prebiotics research in infants. The objective of this study was to compare the methodological quality, outcomes of food industry sponsored trials versus non industry sponsored trials, with regards to supplementation of synbiotics, probiotics and prebiotics in infant formula. Methods A comprehensive search was conducted to identify published and unpublished randomized clinical trials (RCTs). Cochrane methodology was used to assess the risk of bias of included RCTs in the following domains: 1) sequence generation; 2) allocation concealment; 3) blinding; 4) incomplete outcome data; 5) selective outcome reporting; and 6) other bias. Clinical outcomes and authors’ conclusions were reported in frequencies and percentages. The association between source of funding, risk of bias, clinical outcomes and conclusions were assessed using Pearson’s Chi-square test and the Fisher’s exact test. A p-value < 0.05 was statistically significant. Results Sixty seven completed and 3 on-going RCTs were included. Forty (59.7%) were funded by food industry, 11 (16.4%) by non-industry entities and 16 (23.9%) did not specify source of funding. Several risk of bias domains, especially sequence generation, allocation concealment and blinding, were not adequately reported. There was no significant association between the source of funding and sequence generation, allocation concealment, blinding and selective reporting, majority of reported clinical outcomes or authors’ conclusions. On the other hand, source of funding was significantly associated with the domains of incomplete outcome data, free of other bias domains as well as reported antibiotic use and conclusions on weight gain. Conclusion In RCTs on infants fed infant formula containing probiotics, prebiotics or synbiotics, the source of funding did not

A Practical Risk Assessment Methodology for Safety-Critical Train Control Systems

DOT National Transportation Integrated Search

2009-07-01

This project proposes a Practical Risk Assessment Methodology (PRAM) for analyzing railroad accident data and assessing the risk and benefit of safety-critical train control systems. This report documents in simple steps the algorithms and data input...

The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study.

PubMed

Mokkink, Lidwine B; Terwee, Caroline B; Patrick, Donald L; Alonso, Jordi; Stratford, Paul W; Knol, Dirk L; Bouter, Lex M; de Vet, Henrica C W

2010-05-01

Aim of the COSMIN study (COnsensus-based Standards for the selection of health status Measurement INstruments) was to develop a consensus-based checklist to evaluate the methodological quality of studies on measurement properties. We present the COSMIN checklist and the agreement of the panel on the items of the checklist. A four-round Delphi study was performed with international experts (psychologists, epidemiologists, statisticians and clinicians). Of the 91 invited experts, 57 agreed to participate (63%). Panel members were asked to rate their (dis)agreement with each proposal on a five-point scale. Consensus was considered to be reached when at least 67% of the panel members indicated 'agree' or 'strongly agree'. Consensus was reached on the inclusion of the following measurement properties: internal consistency, reliability, measurement error, content validity (including face validity), construct validity (including structural validity, hypotheses testing and cross-cultural validity), criterion validity, responsiveness, and interpretability. The latter was not considered a measurement property. The panel also reached consensus on how these properties should be assessed. The resulting COSMIN checklist could be useful when selecting a measurement instrument, peer-reviewing a manuscript, designing or reporting a study on measurement properties, or for educational purposes.

Epidemiology, methodological and reporting characteristics of systematic reviews of nursing interventions published in China.

PubMed

Shi, Chunhu; Zhu, Lin; Wang, Xue; Qin, Chunxia; Xu, Qi; Tian, Jinhui

2014-12-01

The importance of systematic reviews (SRs) of nursing interventions' impact on practice makes their methodological quality and reporting characteristics especially important as it directly influence their utility for clinicians, patients and policy makers.The study aims to assess the methodological quality and reporting characteristics of SRs of nursing interventions in Chinese nursing journals. Three Chinese databases were searched for SRs of nursing interventions from inception to October 2011. The assessment of multiple systematic reviews (AMSTAR) and Preferred Reporting Items for Systematic Reviews and Meta Analyses (PRISMA) statements were used to assess methodological quality and reporting characteristics. Seventy-four SRs were included. The proportion of SRs complying with AMSTAR checklist items ranged from 0% to 82.4%. No SRs reported an 'a priori' design or conflict of interest. Only four items were found to be reported in more than 50% of the SRs: a list of included and excluded studies, the scientific quality of included studies, the appropriate use of methods to combine findings, and formulating conclusions appropriately. The majority of SRs of nursing interventions in China had major methodological and reporting flaws that limited their value to guide decisions. Chinese authors and journals should adopt and keep up with the AMSTAR and PRISMA statements to improve the quality of SRs in this field. © 2014 Wiley Publishing Asia Pty Ltd.

Quality of Life Assessment.

ERIC Educational Resources Information Center

Gore, Peter H.

Explaining the use of overlay methodology in the assessment of rural social service this paper describes a technique of visual juxtaposition wherein information is matched with geographic location. To ascertain whether senior citizen centers are located in areas of client concentration, for example, this model superimposes the location of senior…

Translating Oral Health-Related Quality of Life Measures: Are There Alternative Methodologies?

ERIC Educational Resources Information Center

Brondani, Mario; He, Sarah

2013-01-01

Translating existing sociodental indicators to another language involves a rigorous methodology, which can be costly. Free-of-charge online translator tools are available, but have not been evaluated in the context of research involving quality of life measures. To explore the value of using online translator tools to develop oral health-related…

Anti-tobacco mass media and socially disadvantaged groups: a systematic and methodological review.

PubMed

Guillaumier, Ashleigh; Bonevski, Billie; Paul, Chris

2012-07-01

Only a limited amount of research has been conducted to explore whether there are socioeconomic status differences in responses to mass media. However, the methodological quality of this evidence has not been assessed, limiting confidence in conclusions that can be drawn regarding study outcomes. A systematic review of the effectiveness of anti-tobacco mass media campaigns with socially disadvantaged groups was conducted, and the methodological quality of included studies was assessed. Medline, The Cochrane Library, PsycInfo, Embase and Web of Science were searched using MeSH and keywords for quantitative studies conducted in Western countries prior to March 2012. A methodological quality assessment and narrative analysis of included studies was undertaken. Seventeen relevant studies (reported in 18 papers) were identified; however, weak study designs and selection bias were common characteristics, limiting strong conclusions about effectiveness. Using predominantly non-cessation related outcome measures reviewed papers indicated mixed results for mass media tobacco control campaign effectiveness among various social groups. Most studies assessed mass media impact on low socioeconomic status groups rather than highly socially disadvantaged groups. Methodological rigour of evaluations in this field must be improved to aid understanding regarding the effectiveness of mass media campaigns in driving cessation among disadvantaged groups. The results of this review indicate a gap in methodologically rigorous research into the effectiveness of mass media campaigns among socially disadvantaged groups, particularly the highly disadvantaged. © 2012 Australasian Professional Society on Alcohol and other Drugs.

Back-and-Forth Methodology for Objective Voice Quality Assessment: From/to Expert Knowledge to/from Automatic Classification of Dysphonia

NASA Astrophysics Data System (ADS)

Fredouille, Corinne; Pouchoulin, Gilles; Ghio, Alain; Revis, Joana; Bonastre, Jean-François; Giovanni, Antoine

2009-12-01

This paper addresses voice disorder assessment. It proposes an original back-and-forth methodology involving an automatic classification system as well as knowledge of the human experts (machine learning experts, phoneticians, and pathologists). The goal of this methodology is to bring a better understanding of acoustic phenomena related to dysphonia. The automatic system was validated on a dysphonic corpus (80 female voices), rated according to the GRBAS perceptual scale by an expert jury. Firstly, focused on the frequency domain, the classification system showed the interest of 0-3000 Hz frequency band for the classification task based on the GRBAS scale. Later, an automatic phonemic analysis underlined the significance of consonants and more surprisingly of unvoiced consonants for the same classification task. Submitted to the human experts, these observations led to a manual analysis of unvoiced plosives, which highlighted a lengthening of VOT according to the dysphonia severity validated by a preliminary statistical analysis.

DEVELOPMENT OF RISK ASSESSMENT METHODOLOGY FOR MUNICIPAL SLUDGE INCINERATION

EPA Science Inventory

This is one of a series of reports that present methodologies for assessing the potential risks to humans or other organisms from the disposal or reuse of municipal sludge. he sludge management practices addressed by this series include land application practices, distribution an...

DEVELOPMENT OF RISK ASSESSMENT METHODOLOGY FOR MUNICIPAL SLUDGE LANDFILLING

EPA Science Inventory

This is one of a series of reports that present methodologies for assessing the potential risks to humans or other organisms from the disposal or reuse of municipal sludge. he sludge management practices addressed by this series include land application practices, distribution an...

A novel methodology for interpreting air quality measurements from urban streets using CFD modelling

NASA Astrophysics Data System (ADS)

Solazzo, Efisio; Vardoulakis, Sotiris; Cai, Xiaoming

2011-09-01

In this study, a novel computational fluid dynamics (CFD) based methodology has been developed to interpret long-term averaged measurements of pollutant concentrations collected at roadside locations. The methodology is applied to the analysis of pollutant dispersion in Stratford Road (SR), a busy street canyon in Birmingham (UK), where a one-year sampling campaign was carried out between August 2005 and July 2006. Firstly, a number of dispersion scenarios are defined by combining sets of synoptic wind velocity and direction. Assuming neutral atmospheric stability, CFD simulations are conducted for all the scenarios, by applying the standard k-ɛ turbulence model, with the aim of creating a database of normalised pollutant concentrations at specific locations within the street. Modelled concentration for all wind scenarios were compared with hourly observed NO x data. In order to compare with long-term averaged measurements, a weighted average of the CFD-calculated concentration fields was derived, with the weighting coefficients being proportional to the frequency of each scenario observed during the examined period (either monthly or annually). In summary the methodology consists of (i) identifying the main dispersion scenarios for the street based on wind speed and directions data, (ii) creating a database of CFD-calculated concentration fields for the identified dispersion scenarios, and (iii) combining the CFD results based on the frequency of occurrence of each dispersion scenario during the examined period. The methodology has been applied to calculate monthly and annually averaged benzene concentration at several locations within the street canyon so that a direct comparison with observations could be made. The results of this study indicate that, within the simplifying assumption of non-buoyant flow, CFD modelling can aid understanding of long-term air quality measurements, and help assessing the representativeness of monitoring locations for population

The health-nutrition dimension: a methodological approach to assess the nutritional sustainability of typical agro-food products and the Mediterranean diet.

PubMed

Azzini, Elena; Maiani, Giuseppe; Turrini, Aida; Intorre, Federica; Lo Feudo, Gabriella; Capone, Roberto; Bottalico, Francesco; El Bilali, Hamid; Polito, Angela

2018-08-01

The aim of this paper is to provide a methodological approach to evaluate the nutritional sustainability of typical agro-food products, representing Mediterranean eating habits and included in the Mediterranean food pyramid. For each group of foods, suitable and easily measurable indicators were identified. Two macro-indicators were used to assess the nutritional sustainability of each product. The first macro-indicator, called 'business distinctiveness', takes into account the application of different regulations and standards regarding quality, safety and traceability as well as the origin of raw materials. The second macro-indicator, called 'nutritional quality', assesses product nutritional quality taking into account the contents of key compounds including micronutrients and bioactive phytochemicals. For each indicator a 0-10 scoring system was set up, with scores from 0 (unsustainable) to 10 (very sustainable), with 5 as a sustainability benchmark value. The benchmark value is the value from which a product can be considered sustainable. A simple formula was developed to produce a sustainability index. The proposed sustainability index could be considered a useful tool to describe both the qualitative and quantitative value of micronutrients and bioactive phytochemical present in foodstuffs. This methodological approach can also be applied beyond the Mediterranean, to food products in other world regions. © 2018 Society of Chemical Industry. © 2018 Society of Chemical Industry.

Clinical audit, a valuable tool to improve quality of care: General methodology and applications in nephrology.

PubMed

Esposito, Pasquale; Dal Canton, Antonio

2014-11-06

Evaluation and improvement of quality of care provided to the patients are of crucial importance in the daily clinical practice and in the health policy planning and financing. Different tools have been developed, including incident analysis, health technology assessment and clinical audit. The clinical audit consist of measuring a clinical outcome or a process, against well-defined standards set on the principles of evidence-based medicine in order to identify the changes needed to improve the quality of care. In particular, patients suffering from chronic renal diseases, present many problems that have been set as topics for clinical audit projects, such as hypertension, anaemia and mineral metabolism management. Although the results of these studies have been encouraging, demonstrating the effectiveness of audit, overall the present evidence is not clearly in favour of clinical audit. These findings call attention to the need to further studies to validate this methodology in different operating scenarios. This review examines the principle of clinical audit, focusing on experiences performed in nephrology settings.

Multi-scale landslide hazard assessment: Advances in global and regional methodologies

NASA Astrophysics Data System (ADS)

Kirschbaum, Dalia; Peters-Lidard, Christa; Adler, Robert; Hong, Yang

2010-05-01

The increasing availability of remotely sensed surface data and precipitation provides a unique opportunity to explore how smaller-scale landslide susceptibility and hazard assessment methodologies may be applicable at larger spatial scales. This research first considers an emerging satellite-based global algorithm framework, which evaluates how the landslide susceptibility and satellite derived rainfall estimates can forecast potential landslide conditions. An analysis of this algorithm using a newly developed global landslide inventory catalog suggests that forecasting errors are geographically variable due to improper weighting of surface observables, resolution of the current susceptibility map, and limitations in the availability of landslide inventory data. These methodological and data limitation issues can be more thoroughly assessed at the regional level, where available higher resolution landslide inventories can be applied to empirically derive relationships between surface variables and landslide occurrence. The regional empirical model shows improvement over the global framework in advancing near real-time landslide forecasting efforts; however, there are many uncertainties and assumptions surrounding such a methodology that decreases the functionality and utility of this system. This research seeks to improve upon this initial concept by exploring the potential opportunities and methodological structure needed to advance larger-scale landslide hazard forecasting and make it more of an operational reality. Sensitivity analysis of the surface and rainfall parameters in the preliminary algorithm indicates that surface data resolution and the interdependency of variables must be more appropriately quantified at local and regional scales. Additionally, integrating available surface parameters must be approached in a more theoretical, physically-based manner to better represent the physical processes underlying slope instability and landslide initiation

Designing trials for pressure ulcer risk assessment research: methodological challenges.

PubMed

Balzer, K; Köpke, S; Lühmann, D; Haastert, B; Kottner, J; Meyer, G

2013-08-01

For decades various pressure ulcer risk assessment scales (PURAS) have been developed and implemented into nursing practice despite uncertainty whether use of these tools helps to prevent pressure ulcers. According to current methodological standards, randomised controlled trials (RCTs) are required to conclusively determine the clinical efficacy and safety of this risk assessment strategy. In these trials, PURAS-aided risk assessment has to be compared to nurses' clinical judgment alone in terms of its impact on pressure ulcer incidence and adverse outcomes. However, RCTs evaluating diagnostic procedures are prone to specific risks of bias and threats to the statistical power which may challenge their validity and feasibility. This discussion paper critically reflects on the rigour and feasibility of experimental research needed to substantiate the clinical efficacy of PURAS-aided risk assessment. Based on reflections of the methodological literature, a critical appraisal of available trials on this subject and an analysis of a protocol developed for a methodologically robust cluster-RCT, this paper arrives at the following conclusions: First, available trials do not provide reliable estimates of the impact of PURAS-aided risk assessment on pressure ulcer incidence compared to nurses' clinical judgement alone due to serious risks of bias and insufficient sample size. Second, it seems infeasible to assess this impact by means of rigorous experimental studies since sample size would become extremely high if likely threats to validity and power are properly taken into account. Third, means of evidence linkages seem to currently be the most promising approaches for evaluating the clinical efficacy and safety of PURAS-aided risk assessment. With this kind of secondary research, the downstream effect of use of PURAS on pressure ulcer incidence could be modelled by combining best available evidence for single parts of this pathway. However, to yield reliable modelling

Systematic Review and Quality Appraisal of Cost-Effectiveness Analyses of Pharmacologic Maintenance Treatment for Chronic Obstructive Pulmonary Disease: Methodological Considerations and Recommendations.

PubMed

van der Schans, Simon; Goossens, Lucas M A; Boland, Melinde R S; Kocks, Janwillem W H; Postma, Maarten J; van Boven, Job F M; Rutten-van Mölken, Maureen P M H

2017-01-01

Worldwide, chronic obstructive pulmonary disease (COPD) is a highly prevalent chronic lung disease with considerable clinical and socioeconomic impact. Pharmacologic maintenance drugs (such as bronchodilators and inhaled corticosteroids) play an important role in the treatment of COPD. The cost effectiveness of these treatments has been frequently assessed, but studies to date have largely neglected the impact of treatment sequence and the exact stage of disease in which the drugs are used in real life. We aimed to systematically review recently published articles that reported the cost effectiveness of COPD maintenance treatments, with a focus on key findings, quality and methodological issues. We performed a systematic literature search in Embase, PubMed, the UK NHS Economic Evaluation Database (NHS-EED) and EURONHEED (European Network of Health Economics Evaluation Databases) and included all relevant articles published between 2011 and 2015 in either Dutch, English or German. Main study characteristics, methods and outcomes were extracted and critically assessed. The Quality of Health Economic Studies (QHES) instrument was used as basis for quality assessment, but additional items were also addressed. The search identified 18 recent pharmacoeconomic analyses of COPD maintenance treatments. Papers reported the cost effectiveness of long-acting muscarinic antagonist (LAMA) monotherapy (n = 6), phosphodiesterase (PDE)-4 inhibitors (n = 4), long-acting beta agonist/inhaled corticosteroid (LABA/ICS) combinations (n = 4), LABA monotherapy (n = 2) and LABA/LAMA combinations (n = 2). All but two studies were funded by the manufacturer, and all studies indicated favourable cost effectiveness; however, the number of quality-adjusted life-years (QALYs) gained was small. Less than half of the studies reported a COPD-specific outcome in addition to a generic outcome (mostly QALYs). Exacerbation and mortality rates were found to be the main drivers of cost

Quality Assessment for Placement Centres: A Case Study of the University of Barcelona's Faculty of Education

ERIC Educational Resources Information Center

Freixa Niella, Montse; Vilà Baños, Ruth; Rubio Hurtado, M. José

2015-01-01

The aim of this research was to identify the factors that could be used for quality assessments of the placement centres used by the University of Barcelona's Faculty of Education. To achieve this, a multiple case study method (bachelor's degrees in Education, Social Education and Social Work) was used, which was based on a survey methodology. A…

40 CFR Appendix C to Part 132 - Great Lakes Water Quality Initiative Methodologies for Development of Human Health Criteria and...

Code of Federal Regulations, 2011 CFR

2011-07-01

... LAKES SYSTEM Pt. 132, App. C Appendix C to Part 132—Great Lakes Water Quality Initiative Methodologies...; or consuming fish from the water, and water-related recreation activities using the Methodologies for... 40 Protection of Environment 22 2011-07-01 2011-07-01 false Great Lakes Water Quality Initiative...

40 CFR Appendix C to Part 132 - Great Lakes Water Quality Initiative Methodologies for Development of Human Health Criteria and...

Code of Federal Regulations, 2010 CFR

2010-07-01

... LAKES SYSTEM Pt. 132, App. C Appendix C to Part 132—Great Lakes Water Quality Initiative Methodologies...; or consuming fish from the water, and water-related recreation activities using the Methodologies for... 40 Protection of Environment 21 2010-07-01 2010-07-01 false Great Lakes Water Quality Initiative...

[Integrated Quality Management System (IQMS): a model for improving the quality of reproductive health care in rural Kenya].

PubMed

Herrler, Claudia; Bramesfeld, Anke; Brodowski, Marc; Prytherch, Helen; Marx, Irmgard; Nafula, Maureen; Richter-Aairijoki, Heide; Musyoka, Lucy; Marx, Michael; Szecsenyi, Joachim

2015-01-01

To develop a model aiming to improve the quality of services for reproductive health care in rural Kenya and designed to measure the quality of reproductive health services in such a way that allows these services to identify measures for improving their performance. The Integrated Quality Management System (IQMS) was developed on the basis of a pre-existing and validated model for quality promotion, namely the European Practice Assessment (EPA). The methodology for quality assessment and feedback of assessment results to the service teams was adopted from the EPA model. Quality assessment methodology included data assessment through staff, patient surveys and service visitation. Quality is assessed by indicators, and so indicators had to be developed that were appropriate for assessing reproductive health care in rural Kenya. A search of the Kenyan and international literature was conducted to identify potential indicators. These were then rated for their relevance and clarity by a panel of Kenyan experts. 260 indicators were rated as relevant and assigned to 29 quality dimensions and 5 domains. The implementation of IQMS in ten facilities showed that IQMS is a feasible model for assessing the quality of reproductive health services in rural Kenya. IQMS enables these services to identify quality improvement targets and necessary improvement measures. Both strengths and limitations of IQMS will be discussed. Copyright © 2015. Published by Elsevier GmbH.

Higher Education Quality Assessment Model: Towards Achieving Educational Quality Standard

ERIC Educational Resources Information Center

Noaman, Amin Y.; Ragab, Abdul Hamid M.; Madbouly, Ayman I.; Khedra, Ahmed M.; Fayoumi, Ayman G.

2017-01-01

This paper presents a developed higher education quality assessment model (HEQAM) that can be applied for enhancement of university services. This is because there is no universal unified quality standard model that can be used to assess the quality criteria of higher education institutes. The analytical hierarchy process is used to identify the…

QUALITY OF HEALTH TECHNOLOGY ASSESSMENT REPORTS PREPARED FOR THE MEDICAL SERVICES ADVISORY COMMITTEE.

PubMed

Hua, Martin; Boonstra, Tristan; Kelly, Patrick J; Wilson, Andrew; Craig, Jonathan C; Webster, Angela C

2016-01-01

The Medical Services Advisory Committee (MSAC) makes recommendations to the Australian Government for funding health technologies under the Medicare Benefits Schedule (MBS). Differences in public, clinical, commercial, and political opinions on health expenditure emphasize the importance of defensible funding decisions. We aimed to evaluate the quality of health technology assessment (HTA) reports over time and among health technologies assessed for MSAC. A cohort study was performed of HTA reports prepared for MSAC between 1998 and 2013. We measured the quality of HTA reports using reporting guidelines proposed by the European Collaboration for Assessment of Health Interventions. Individual component scores across eleven domains were calculated, and summed for an overall aggregate score. We used linear regression to investigate any change in quality over time and among the types of technologies assessed. We included 110 HTA reports. The safety (80 percent), effectiveness (84 percent), economic (74 percent), and organizational (99 percent) domains were better reported than the psychological, social, and ethical considerations (34 percent). The basic (75 percent), methodological (62 percent), background (82 percent), contextual (46 percent), status quo (54 percent), and technical information (66 percent) that framed each assessment were inconsistently reported. On average, overall quality scores increased by 2 percent (p < 0.001) per year, from approximately 60 percent to 80 percent over the 15-year period, with no significant difference among surgical, diagnostic or other nonpharmaceutical health technologies (p = 0.22). HTA reports prepared for MSAC are a key tool in allocating scarce health resources. The overall quality of these reports has improved, but the reporting of specific domains and subthemes therein could be better addressed.

Can formal collaborative methodologies improve quality in primary health care in New Zealand? Insights from the EQUIPPED Auckland Collaborative.

PubMed

Palmer, Celia; Bycroft, Janine; Healey, Kate; Field, Adrian; Ghafel, Mazin

2012-12-01

Auckland District Health Board was one of four District Health Boards to trial the Breakthrough Series (BTS) methodology to improve the management of long-term conditions in New Zealand, with support from the Ministry of Health. To improve clinical outcomes, facilitate planned care and promote quality improvement within participating practices in Auckland. Implementation of the Collaborative followed the improvement model / Institute for Healthcare Improvement methodology. Three topic areas were selected: system redesign, cardio-vascular disease/diabetes, and self-management support. An expert advisory group and the Improvement Foundation Australia helped guide project development and implementation. Primary Health Organisation facilitators were trained in the methodology and 15 practice teams participated in the three learning workshops and action periods over 12 months. An independent evaluation study using both quantitative and qualitative methods was conducted. Improvements were recorded in cardiovascular disease risk assessment, practice-level systems of care, self-management systems and follow-up and coordination for patients. Qualitative research found improvements in coordination and teamwork, knowledge of practice populations and understanding of managing long-term conditions. The Collaborative process delivered some real improvements in the systems of care for people with long-term conditions and a change in culture among participating practices. The findings suggest that by strengthening facilitation processes, improving access to comprehensive population audit tools and lengthening the time frame, the process has the potential to make significant improvements in practice. Other organisations should consider this approach when investigating quality improvement programmes.

A ranking index for quality assessment of forensic DNA profiles forensic DNA profiles

PubMed Central

2010-01-01

Background Assessment of DNA profile quality is vital in forensic DNA analysis, both in order to determine the evidentiary value of DNA results and to compare the performance of different DNA analysis protocols. Generally the quality assessment is performed through manual examination of the DNA profiles based on empirical knowledge, or by comparing the intensities (allelic peak heights) of the capillary electrophoresis electropherograms. Results We recently developed a ranking index for unbiased and quantitative quality assessment of forensic DNA profiles, the forensic DNA profile index (FI) (Hedman et al. Improved forensic DNA analysis through the use of alternative DNA polymerases and statistical modeling of DNA profiles, Biotechniques 47 (2009) 951-958). FI uses electropherogram data to combine the intensities of the allelic peaks with the balances within and between loci, using Principal Components Analysis. Here we present the construction of FI. We explain the mathematical and statistical methodologies used and present details about the applied data reduction method. Thereby we show how to adapt the ranking index for any Short Tandem Repeat-based forensic DNA typing system through validation against a manual grading scale and calibration against a specific set of DNA profiles. Conclusions The developed tool provides unbiased quality assessment of forensic DNA profiles. It can be applied for any DNA profiling system based on Short Tandem Repeat markers. Apart from crime related DNA analysis, FI can therefore be used as a quality tool in paternal or familial testing as well as in disaster victim identification. PMID:21062433

40 CFR Appendix A to Part 132 - Great Lakes Water Quality Initiative Methodologies for Development of Aquatic Life Criteria and...

Code of Federal Regulations, 2010 CFR

2010-07-01

... LAKES SYSTEM Pt. 132, App. A Appendix A to Part 132—Great Lakes Water Quality Initiative Methodologies... 40 Protection of Environment 21 2010-07-01 2010-07-01 false Great Lakes Water Quality Initiative Methodologies for Development of Aquatic Life Criteria and Values A Appendix A to Part 132 Protection of...

40 CFR Appendix A to Part 132 - Great Lakes Water Quality Initiative Methodologies for Development of Aquatic Life Criteria and...

Code of Federal Regulations, 2011 CFR

2011-07-01

... LAKES SYSTEM Pt. 132, App. A Appendix A to Part 132—Great Lakes Water Quality Initiative Methodologies... 40 Protection of Environment 22 2011-07-01 2011-07-01 false Great Lakes Water Quality Initiative Methodologies for Development of Aquatic Life Criteria and Values A Appendix A to Part 132 Protection of...

Assessing importance and satisfaction judgments of intermodal work commuters with electronic survey methodology.

DOT National Transportation Integrated Search

2013-09-01

Recent advances in multivariate methodology provide an opportunity to further the assessment of service offerings in public transportation for work commuting. We offer methodologies that are alternative to direct rating scale and have advantages in t...

HEALTH TECHNOLOGY ASSESSMENT OF PUBLIC HEALTH INTERVENTIONS: A SYNTHESIS OF METHODOLOGICAL GUIDANCE.

PubMed

Mathes, Tim; Antoine, Sunya-Lee; Prengel, Peggy; Bühn, Stefanie; Polus, Stephanie; Pieper, Dawid

2017-01-01

The evaluation of public health interventions poses some challenges. As a consequence, health technology assessment (HTA) methods for public health interventions (PHI) have to be adapted. This study aimed to summarize the available guidance on methods for HTA of PHI. We systematically searched for methodological guidance on HTA of PHIs. Our focus was on research synthesis methods to evaluate effectiveness. Relevant information was synthesized narratively in a standardized way. Only four guidance documents were identified specifically for HTAs of PHI. The approaches used for HTAs of PHIs are broader and more flexible than those for medical interventions. For this reason, there is a tendency to identify the intervention components and context factors that influence the effectiveness and transferability of an intervention rather than to assess its effectiveness in general. The details in the guidance vary without justification. Unjustified heterogeneity between the different guidance approaches is most pronounced for quality assessment, assessment of applicability, and methods to integrate qualitative and quantitative evidence. Descriptions for the assessment of integrity, heterogeneity, sustainability, context factors, and applicability are often vague. The heterogeneity in approaches indicates that there is currently no consensus on methods to deal with the challenges of the PHI evaluations. A possible explanation for this may be that the methods are not sufficiently developed, and advantages and disadvantages of a certain method in relation to the research question (e.g., broad/focused) have not yet been sufficiently evaluated.