Combining Review Text Content and Reviewer-Item Rating Matrix to Predict Review Rating.

PubMed

Wang, Bingkun; Huang, Yongfeng; Li, Xing

2016-01-01

E-commerce develops rapidly. Learning and taking good advantage of the myriad reviews from online customers has become crucial to the success in this game, which calls for increasingly more accuracy in sentiment classification of these reviews. Therefore the finer-grained review rating prediction is preferred over the rough binary sentiment classification. There are mainly two types of method in current review rating prediction. One includes methods based on review text content which focus almost exclusively on textual content and seldom relate to those reviewers and items remarked in other relevant reviews. The other one contains methods based on collaborative filtering which extract information from previous records in the reviewer-item rating matrix, however, ignoring review textual content. Here we proposed a framework for review rating prediction which shows the effective combination of the two. Then we further proposed three specific methods under this framework. Experiments on two movie review datasets demonstrate that our review rating prediction framework has better performance than those previous methods.

2008 annual merit review

DOE Office of Scientific and Technical Information (OSTI.GOV)

None, None

2009-01-18

The 2008 DOE Vehicle Technologies Program Annual Merit Review was held February 25-28, 2008 in Bethesda, Maryland. The review encompassed all of the work done by the Vehicle Technologies Program: a total of 280 individual activities were reviewed, by a total of just over 100 reviewers. A total of 1,908 individual review responses were received for the technical reviews, and an additional 29 individual review responses were received for the plenary session review.

Systematic reviews identify important methodological flaws in stroke rehabilitation therapy primary studies: review of reviews.

PubMed

Santaguida, Pasqualina; Oremus, Mark; Walker, Kathryn; Wishart, Laurie R; Siegel, Karen Lohmann; Raina, Parminder

2012-04-01

A "review of reviews" was undertaken to assess methodological issues in studies evaluating nondrug rehabilitation interventions in stroke patients. MEDLINE, CINAHL, PsycINFO, and the Cochrane Database of Systematic Reviews were searched from January 2000 to January 2008 within the stroke rehabilitation setting. Electronic searches were supplemented by reviews of reference lists and citations identified by experts. Eligible studies were systematic reviews; excluded citations were narrative reviews or reviews of reviews. Review characteristics and criteria for assessing methodological quality of primary studies within them were extracted. The search yielded 949 English-language citations. We included a final set of 38 systematic reviews. Cochrane reviews, which have a standardized methodology, were generally of higher methodological quality than non-Cochrane reviews. Most systematic reviews used standardized quality assessment criteria for primary studies, but not all were comprehensive. Reviews showed that primary studies had problems with randomization, allocation concealment, and blinding. Baseline comparability, adverse events, and co-intervention or contamination were not consistently assessed. Blinding of patients and providers was often not feasible and was not evaluated as a source of bias. The eligible systematic reviews identified important methodological flaws in the evaluated primary studies, suggesting the need for improvement of research methods and reporting. Copyright © 2012 Elsevier Inc. All rights reserved.

Making literature reviews more reliable through application of lessons from systematic reviews.

PubMed

Haddaway, N R; Woodcock, P; Macura, B; Collins, A

2015-12-01

Review articles can provide valuable summaries of the ever-increasing volume of primary research in conservation biology. Where findings may influence important resource-allocation decisions in policy or practice, there is a need for a high degree of reliability when reviewing evidence. However, traditional literature reviews are susceptible to a number of biases during the identification, selection, and synthesis of included studies (e.g., publication bias, selection bias, and vote counting). Systematic reviews, pioneered in medicine and translated into conservation in 2006, address these issues through a strict methodology that aims to maximize transparency, objectivity, and repeatability. Systematic reviews will always be the gold standard for reliable synthesis of evidence. However, traditional literature reviews remain popular and will continue to be valuable where systematic reviews are not feasible. Where traditional reviews are used, lessons can be taken from systematic reviews and applied to traditional reviews in order to increase their reliability. Certain key aspects of systematic review methods that can be used in a context-specific manner in traditional reviews include focusing on mitigating bias; increasing transparency, consistency, and objectivity, and critically appraising the evidence and avoiding vote counting. In situations where conducting a full systematic review is not feasible, the proposed approach to reviewing evidence in a more systematic way can substantially improve the reliability of review findings, providing a time- and resource-efficient means of maximizing the value of traditional reviews. These methods are aimed particularly at those conducting literature reviews where systematic review is not feasible, for example, for graduate students, single reviewers, or small organizations. © 2015 Society for Conservation Biology.

TMA certification review state-of-the-practice review report : 2016

DOT National Transportation Integrated Search

2017-02-01

The 2016 TMA Certification Review State-of-the-Practice Review Report documents the Federal Highway Administrations (FHWA) review of Division Office oversight processes relating to TMA Certification Reviews. This is the sixth review of Division Of...

Impact of interventions to improve the quality of peer review of biomedical journals: a systematic review and meta-analysis.

PubMed

Bruce, Rachel; Chauvin, Anthony; Trinquart, Ludovic; Ravaud, Philippe; Boutron, Isabelle

2016-06-10

The peer review process is a cornerstone of biomedical research. We aimed to evaluate the impact of interventions to improve the quality of peer review for biomedical publications. We performed a systematic review and meta-analysis. We searched CENTRAL, MEDLINE (PubMed), Embase, Cochrane Database of Systematic Reviews, and WHO ICTRP databases, for all randomized controlled trials (RCTs) evaluating the impact of interventions to improve the quality of peer review for biomedical publications. We selected 22 reports of randomized controlled trials, for 25 comparisons evaluating training interventions (n = 5), the addition of a statistical peer reviewer (n = 2), use of a checklist (n = 2), open peer review (i.e., peer reviewers informed that their identity would be revealed; n = 7), blinded peer review (i.e., peer reviewers blinded to author names and affiliation; n = 6) and other interventions to increase the speed of the peer review process (n = 3). Results from only seven RCTs were published since 2004. As compared with the standard peer review process, training did not improve the quality of the peer review report and use of a checklist did not improve the quality of the final manuscript. Adding a statistical peer review improved the quality of the final manuscript (standardized mean difference (SMD), 0.58; 95 % CI, 0.19 to 0.98). Open peer review improved the quality of the peer review report (SMD, 0.14; 95 % CI, 0.05 to 0.24), did not affect the time peer reviewers spent on the peer review (mean difference, 0.18; 95 % CI, -0.06 to 0.43), and decreased the rate of rejection (odds ratio, 0.56; 95 % CI, 0.33 to 0.94). Blinded peer review did not affect the quality of the peer review report or rejection rate. Interventions to increase the speed of the peer review process were too heterogeneous to allow for pooling the results. Despite the essential role of peer review, only a few interventions have been assessed in randomized controlled trials. Evidence-based peer review needs to be developed in biomedical journals.

A Guide to Writing a Qualitative Systematic Review Protocol to Enhance Evidence-Based Practice in Nursing and Health Care.

PubMed

Butler, Ashleigh; Hall, Helen; Copnell, Beverley

2016-06-01

The qualitative systematic review is a rapidly developing area of nursing research. In order to present trustworthy, high-quality recommendations, such reviews should be based on a review protocol to minimize bias and enhance transparency and reproducibility. Although there are a number of resources available to guide researchers in developing a quantitative review protocol, very few resources exist for qualitative reviews. To guide researchers through the process of developing a qualitative systematic review protocol, using an example review question. The key elements required in a systematic review protocol are discussed, with a focus on application to qualitative reviews: Development of a research question; formulation of key search terms and strategies; designing a multistage review process; critical appraisal of qualitative literature; development of data extraction techniques; and data synthesis. The paper highlights important considerations during the protocol development process, and uses a previously developed review question as a working example. This paper will assist novice researchers in developing a qualitative systematic review protocol. By providing a worked example of a protocol, the paper encourages the development of review protocols, enhancing the trustworthiness and value of the completed qualitative systematic review findings. Qualitative systematic reviews should be based on well planned, peer reviewed protocols to enhance the trustworthiness of results and thus their usefulness in clinical practice. Protocols should outline, in detail, the processes which will be used to undertake the review, including key search terms, inclusion and exclusion criteria, and the methods used for critical appraisal, data extraction and data analysis to facilitate transparency of the review process. Additionally, journals should encourage and support the publication of review protocols, and should require reference to a protocol prior to publication of the review results. © 2016 Sigma Theta Tau International.

Investing in updating: how do conclusions change when Cochrane systematic reviews are updated?

PubMed Central

French, Simon D; McDonald, Steve; McKenzie, Joanne E; Green, Sally E

2005-01-01

Background Cochrane systematic reviews aim to provide readers with the most up-to-date evidence on the effects of healthcare interventions. The policy of updating Cochrane reviews every two years consumes valuable time and resources and may not be appropriate for all reviews. The objective of this study was to examine the effect of updating Cochrane systematic reviews over a four year period. Methods This descriptive study examined all completed systematic reviews in the Cochrane Database of Systematic Reviews (CDSR) Issue 2, 1998. The latest version of each of these reviews was then identified in CDSR Issue 2, 2002 and changes in the review were described. For reviews that were updated within this time period and had additional studies, we determined whether their conclusion had changed and if there were factors that were predictive of this change. Results A total of 377 complete reviews were published in CDSR Issue 2, 1998. In Issue 2, 2002, 14 of these reviews were withdrawn and one was split, leaving 362 reviews to examine for the purpose of this study. Of these reviews, 254 (70%) were updated. Of these updated reviews, 23 (9%) had a change in conclusion. Both an increase in precision and a change in statistical significance of the primary outcome were predictive of a change in conclusion of the review. Conclusion The concerns around a lack of updating for some reviews may not be justified considering the small proportion of updated reviews that resulted in a changed conclusion. A priority-setting approach to the updating of Cochrane systematic reviews may be more appropriate than a time-based approach. Updating all reviews as frequently as every two years may not be necessary, however some reviews may need to be updated more often than every two years. PMID:16225692

Integration of existing systematic reviews into new reviews: identification of guidance needs

PubMed Central

2014-01-01

Background An exponential increase in the number of systematic reviews published, and constrained resources for new reviews, means that there is an urgent need for guidance on explicitly and transparently integrating existing reviews into new systematic reviews. The objectives of this paper are: 1) to identify areas where existing guidance may be adopted or adapted, and 2) to suggest areas for future guidance development. Methods We searched documents and websites from healthcare focused systematic review organizations to identify and, where available, to summarize relevant guidance on the use of existing systematic reviews. We conducted informational interviews with members of Evidence-based Practice Centers (EPCs) to gather experiences in integrating existing systematic reviews, including common issues and challenges, as well as potential solutions. Results There was consensus among systematic review organizations and the EPCs about some aspects of incorporating existing systematic reviews into new reviews. Current guidance may be used in assessing the relevance of prior reviews and in scanning references of prior reviews to identify studies for a new review. However, areas of challenge remain. Areas in need of guidance include how to synthesize, grade the strength of, and present bodies of evidence composed of primary studies and existing systematic reviews. For instance, empiric evidence is needed regarding how to quality check data abstraction and when and how to use study-level risk of bias assessments from prior reviews. Conclusions There remain areas of uncertainty for how to integrate existing systematic reviews into new reviews. Methods research and consensus processes among systematic review organizations are needed to develop guidance to address these challenges. PMID:24956937

Study protocol for a scoping review on rehabilitation scoping reviews.

PubMed

Colquhoun, Heather L; Jesus, Tiago S; O'Brien, Kelly K; Tricco, Andrea C; Chui, Adora; Zarin, Wasifa; Lillie, Erin; Hitzig, Sander L; Straus, Sharon

2017-09-01

Scoping reviews are increasingly popular in rehabilitation. However, significant variability in scoping review conduct and reporting currently exists, limiting potential for the methodology to advance rehabilitation research, practice and policy. Our aim is to conduct a scoping review of rehabilitation scoping reviews in order to examine the current volume, yearly distribution, proportion, scope and methodological practices involved in the conduct of scoping reviews in rehabilitation. Key areas of methodological improvement will be described. Methods and analysis: We will undertake the review using the Arksey and O'Malley scoping review methodology. Our search will involve two phases. The first will combine a previously conducted scoping review of scoping reviews (not distinct to rehabilitation, with data current to July 2014) together with a rehabilitation keyword search in PubMed. Articles found in the first phase search will undergo a full text review. The second phase will include an update of the previously conducted scoping review of scoping reviews (July 2014 to current). This update will include the search of nine electronic databases, followed by title and abstract screening as well as a full text review. All screening and extraction will be performed independently by two authors. Articles will be included if they are scoping reviews within the field of rehabilitation. A consultation exercise with key targets will inform plans to improve rehabilitation scoping reviews. Ethics and dissemination: Ethics will be required for the consultation phase of our scoping review. Dissemination will include peer-reviewed publication and conferences in rehabilitation-specific contexts.

28 CFR 34.109 - Qualifications of peer reviewers.

Code of Federal Regulations, 2010 CFR

2010-07-01

... Section 34.109 Judicial Administration DEPARTMENT OF JUSTICE OJJDP COMPETITION AND PEER REVIEW PROCEDURES Peer Review § 34.109 Qualifications of peer reviewers. The general reviewer qualification criteria to...). Additional details concerning peer reviewer qualifications are provided in the OJJDP “Peer Review Guideline”. ...

40 CFR 155.42 - Registration review cases.

Code of Federal Regulations, 2012 CFR

2012-07-01

... 40 Protection of Environment 25 2012-07-01 2012-07-01 false Registration review cases. 155.42... REGISTRATION STANDARDS AND REGISTRATION REVIEW Registration Review Procedures § 155.42 Registration review cases. (a) Establishing registration review cases. A registration review case will be composed of one or...

40 CFR 155.42 - Registration review cases.

Code of Federal Regulations, 2013 CFR

2013-07-01

... 40 Protection of Environment 25 2013-07-01 2013-07-01 false Registration review cases. 155.42... REGISTRATION STANDARDS AND REGISTRATION REVIEW Registration Review Procedures § 155.42 Registration review cases. (a) Establishing registration review cases. A registration review case will be composed of one or...

40 CFR 155.42 - Registration review cases.

Code of Federal Regulations, 2011 CFR

2011-07-01

... 40 Protection of Environment 24 2011-07-01 2011-07-01 false Registration review cases. 155.42... REGISTRATION STANDARDS AND REGISTRATION REVIEW Registration Review Procedures § 155.42 Registration review cases. (a) Establishing registration review cases. A registration review case will be composed of one or...

40 CFR 155.42 - Registration review cases.

Code of Federal Regulations, 2014 CFR

2014-07-01

... 40 Protection of Environment 24 2014-07-01 2014-07-01 false Registration review cases. 155.42... REGISTRATION STANDARDS AND REGISTRATION REVIEW Registration Review Procedures § 155.42 Registration review cases. (a) Establishing registration review cases. A registration review case will be composed of one or...

40 CFR 155.42 - Registration review cases.

Code of Federal Regulations, 2010 CFR

2010-07-01

... 40 Protection of Environment 23 2010-07-01 2010-07-01 false Registration review cases. 155.42... REGISTRATION STANDARDS AND REGISTRATION REVIEW Registration Review Procedures § 155.42 Registration review cases. (a) Establishing registration review cases. A registration review case will be composed of one or...

Nitrogen Oxides (NOx) Primary NAAQS REVIEW: Integrated ...

EPA Pesticide Factsheets

The NOx Integrated Review Plan is the first document generated as part of the National Ambient Air Quality Standards (NAAQS) review process. The Plan presents background information, the schedule for the review, the process to be used in conducting the review, and the key policy-relevant science issues that will guide the review. The integrated review plan also discusses the frameworks for the various assessment documents to be prepared by the EPA as part of the review, including an Integrated Science Assessment (ISA), and as warranted, a Risk/Exposure Assessment (REA), and a Policy Assessment (PA). The primary purpose of the NOx Integrated Review Plan is to highlight the key policy-relevant issues to be considered in the Review of the NO2 primary NAAQS. A draft of the integrated review plan will be the subject of an advisory review with the Clean Air Scientific Advisory Committee (CASAC) and made available to the public for review and comment.

NASA Product Peer Review Process

NASA Technical Reports Server (NTRS)

Jenks, Ken

2009-01-01

This viewgraph presentation describes NASA's product peer review process. The contents include: 1) Inspection/Peer Review at NASA; 2) Reasons for product peer reviews; 3) Different types of peer reviews; and 4) NASA requirements for peer reviews. This presentation also includes a demonstration of an actual product peer review.

IRIS Toxicological Review of Inorganic Arsenic (Cancer) (Interagency Science Consultation Draft)

EPA Science Inventory

On February 19, 2010, the draft IRIS Toxicological Review of Inorganic Arsenic (Cancer) external review draft document and the charge to external peer reviewers were released for public review and comment. The draft document and the charge to external peer reviewers were reviewed...

Converting systematic reviews to Cochrane format: a cross-sectional survey of Australian authors of systematic reviews

PubMed Central

Piehl, Janet H; Green, Sally; McDonald, Steve

2003-01-01

Background Despite the growing reputation and subject coverage of the Cochrane Database of Systematic Reviews, many systematic reviews continue to be published solely in paper-based health care journals. This study was designed to determine why authors choose to publish their systematic reviews outside of the Cochrane Collaboration and if they might be interested in converting their reviews to Cochrane format for publication in the Cochrane Database of Systematic Reviews. Methods Cross-sectional survey of Australian primary authors of systematic reviews not published on the Cochrane Database of Systematic Reviews identified from the Database of Abstracts of Reviews of Effectiveness. Results We identified 88 systematic reviews from the Database of Abstracts of Reviews of Effectiveness with an Australian as the primary author. We surveyed 52 authors for whom valid contact information was available. The response rate was 88 per cent (46/52). Ten authors replied without completing the survey, leaving 36 valid surveys for analysis. The most frequently cited reasons for not undertaking a Cochrane review were: lack of time (78%), the need to undergo specific Cochrane training (46%), unwillingness to update reviews (36%), difficulties with the Cochrane process (26%) and the review topic already registered with the Cochrane Collaboration (21%). (Percentages based on completed responses to individual questions.) Nearly half the respondents would consider converting their review to Cochrane format. Dedicated time emerged as the most important factor in facilitating the potential conversion process. Other factors included navigating the Cochrane system, assistance with updating and financial support. Eighty-six per cent were willing to have their review converted to Cochrane format by another author. Conclusion Time required to complete a Cochrane review and the need for specific training are the primary reasons why some authors publish systematic reviews outside of the Cochrane Collaboration. Encouragingly, almost half of the authors would consider converting their review to Cochrane format. Based on the current number of reviews in the Database of Abstracts of Reviews of Effectiveness, this could result in more than 700 additional Cochrane reviews. Ways of supporting these authors and how to provide dedicated time to convert systematic reviews needs further consideration. PMID:12533194

Does knowledge brokering improve the quality of rapid review proposals? A before and after study.

PubMed

Moore, Gabriel; Redman, Sally; D'Este, Catherine; Makkar, Steve; Turner, Tari

2017-01-28

Rapid reviews are increasingly being used to help policy makers access research in short time frames. A clear articulation of the review's purpose, questions, scope, methods and reporting format is thought to improve the quality and generalisability of review findings. The aim of the study is to explore the effectiveness of knowledge brokering in improving the perceived clarity of rapid review proposals from the perspective of potential reviewers. To conduct the study, we drew on the Evidence Check program, where policy makers draft a review proposal (a pre knowledge brokering proposal) and have a 1-hour session with a knowledge broker, who re-drafts the proposal based on the discussion (a post knowledge brokering proposal). We asked 30 reviewers who had previously undertaken Evidence Check reviews to examine the quality of 60 pre and 60 post knowledge brokering proposals. Reviewers were blind to whether the review proposals they received were pre or post knowledge brokering. Using a six-point Likert scale, reviewers scored six questions examining clarity of information about the review's purpose, questions, scope, method and format and reviewers' confidence that they could meet policy makers' needs. Each reviewer was allocated two pre and two post knowledge brokering proposals, randomly ordered, from the 60 reviews, ensuring no reviewer received a pre and post knowledge brokering proposal from the same review. The results showed that knowledge brokering significantly improved the scores for all six questions addressing the perceived clarity of the review proposal and confidence in meeting policy makers' needs; with average changes of 0.68 to 1.23 from pre to post across the six domains. This study found that knowledge brokering increased the perceived clarity of information provided in Evidence Check rapid review proposals and the confidence of reviewers that they could meet policy makers' needs. Further research is needed to identify how the knowledge brokering process achieves these improvements and to test the applicability of the findings in other rapid review programs.

A scoping review of rapid review methods.

PubMed

Tricco, Andrea C; Antony, Jesmin; Zarin, Wasifa; Strifler, Lisa; Ghassemi, Marco; Ivory, John; Perrier, Laure; Hutton, Brian; Moher, David; Straus, Sharon E

2015-09-16

Rapid reviews are a form of knowledge synthesis in which components of the systematic review process are simplified or omitted to produce information in a timely manner. Although numerous centers are conducting rapid reviews internationally, few studies have examined the methodological characteristics of rapid reviews. We aimed to examine articles, books, and reports that evaluated, compared, used or described rapid reviews or methods through a scoping review. MEDLINE, EMBASE, the Cochrane Library, internet websites of rapid review producers, and reference lists were searched to identify articles for inclusion. Two reviewers independently screened literature search results and abstracted data from included studies. Descriptive analysis was conducted. We included 100 articles plus one companion report that were published between 1997 and 2013. The studies were categorized as 84 application papers, seven development papers, six impact papers, and four comparison papers (one was included in two categories). The rapid reviews were conducted between 1 and 12 months, predominantly in Europe (58 %) and North America (20 %). The included studies failed to report 6 % to 73 % of the specific systematic review steps examined. Fifty unique rapid review methods were identified; 16 methods occurred more than once. Streamlined methods that were used in the 82 rapid reviews included limiting the literature search to published literature (24 %) or one database (2 %), limiting inclusion criteria by date (68 %) or language (49 %), having one person screen and another verify or screen excluded studies (6 %), having one person abstract data and another verify (23 %), not conducting risk of bias/quality appraisal (7 %) or having only one reviewer conduct the quality appraisal (7 %), and presenting results as a narrative summary (78 %). Four case studies were identified that compared the results of rapid reviews to systematic reviews. Three studies found that the conclusions between rapid reviews and systematic reviews were congruent. Numerous rapid review approaches were identified and few were used consistently in the literature. Poor quality of reporting was observed. A prospective study comparing the results from rapid reviews to those obtained through systematic reviews is warranted.

Life Review: Implementation, Theory, Research, and Therapy

ERIC Educational Resources Information Center

Haber, David

2006-01-01

A selective literature review of publications on life review generated ideas on implementation, theory, research, and therapy. The review begins by differentiating life review from reminiscence, and summarizing ways to conduct a life review. A dozen theories that have been influenced by the life review technique are presented, with a focus placed…

Evaluating alternative systems of peer review: a large-scale agent-based modelling approach to scientific publication.

PubMed

Kovanis, Michail; Trinquart, Ludovic; Ravaud, Philippe; Porcher, Raphaël

2017-01-01

The debate on whether the peer-review system is in crisis has been heated recently. A variety of alternative systems have been proposed to improve the system and make it sustainable. However, we lack sufficient evidence and data related to these issues. Here we used a previously developed agent-based model of the scientific publication and peer-review system calibrated with empirical data to compare the efficiency of five alternative peer-review systems with the conventional system. We modelled two systems of immediate publication, with and without online reviews (crowdsourcing), a system with only one round of reviews and revisions allowed (re-review opt-out) and two review-sharing systems in which rejected manuscripts are resubmitted along with their past reviews to any other journal (portable) or to only those of the same publisher but of lower impact factor (cascade). The review-sharing systems outperformed or matched the performance of the conventional one in all peer-review efficiency, reviewer effort and scientific dissemination metrics we used. The systems especially showed a large decrease in total time of the peer-review process and total time devoted by reviewers to complete all reports in a year. The two systems with immediate publication released more scientific information than the conventional one but provided almost no other benefit. Re-review opt-out decreased the time reviewers devoted to peer review but had lower performance on screening papers that should not be published and relative increase in intrinsic quality of papers due to peer review than the conventional system. Sensitivity analyses showed consistent findings to those from our main simulations. We recommend prioritizing a system of review-sharing to create a sustainable scientific publication and peer-review system.

Cochrane Systematic Reviews of Chinese Herbal Medicines: An Overview

PubMed Central

Hu, Jing; Zhang, Junhua; Zhao, Wei; Zhang, Yongling; Zhang, Li; Shang, Hongcai

2011-01-01

Objectives Our study had two objectives: a) to systematically identify all existing systematic reviews of Chinese herbal medicines (CHM) published in Cochrane Library; b) to assess the methodological quality of included reviews. Methodology/Principal Findings We performed a systematic search of the Cochrane Database of Systematic Reviews (CDSR, Issue 5, 2010) to identify all reviews of CHM. A total of fifty-eight reviews were eligible for our study. Twenty-one of the included reviews had at least one Traditional Chinese Medicine (TCM) practitioner as its co-author. 7 reviews didn't include any primary study, the remaining reviews (n = 51) included a median of 9 studies and 936 participants. 50% of reviews were last assessed as up-to-date prior to 2008. The questions addressed by 39 reviews were broad in scope, in which 9 reviews combined studies with different herbal medicines. For OQAQ, the mean of overall quality score (item 10) was 5.05 (95% CI; 4.58-5.52). All reviews assessed the methodological quality of primary studies, 16% of included primary studies used adequate sequence generation and 7% used adequate allocation concealment. Of the 51 nonempty reviews, 23 reviews were reported as being inconclusive, while 27 concluded that there might be benefit of CHM, which was limited by the poor quality or inadequate quantity of included studies. 58 reviews reported searching a median of seven electronic databases, while 10 reviews did not search any Chinese database. Conclusions Now CDSR has included large numbers of CHM reviews, our study identified some areas which could be improved, such as almost half of included reviews did not have the participation of TCM practitioners and were not up-to-date according to Cochrane criteria, some reviews pooled the results of different herbal medicines and ignored the searching of Chinese databases. PMID:22174870

Tardive Dyskinesia

MedlinePlus

... Supplements Peer Review Process Review Committees Application Support Library Clinical Research Next Steps Pre-Funding: After Review Terms of ... Supplements Peer Review Process Review Committees Application Support Library Clinical Research Next Steps Pre-Funding: After Review Terms of ...

Hemicrania Continua

MedlinePlus

... Supplements Peer Review Process Review Committees Application Support Library Clinical Research Next Steps Pre-Funding: After Review Terms of ... Supplements Peer Review Process Review Committees Application Support Library Clinical Research Next Steps Pre-Funding: After Review Terms of ...

Benign Essential Blepharospasm

MedlinePlus

... Supplements Peer Review Process Review Committees Application Support Library Clinical Research Next Steps Pre-Funding: After Review Terms of ... Supplements Peer Review Process Review Committees Application Support Library Clinical Research Next Steps Pre-Funding: After Review Terms of ...

Why Participate in Peer Review as a Journal Manuscript Reviewer: What's in It for You?

PubMed

Pytynia, Kristen B

2017-06-01

The peer review process for scientific journals relies on the efforts of volunteer reviewers. Reviewers are selected due to their expertise in their fields. With so many demands on professional time, the benefits of participating in peer review may not be obvious. However, reviewers benefit by exposure to the latest developments in their fields, facilitating their keeping up-to-date with the latest publications. Tenure committees look favorably on participation in peer review, and invitations to review underscore that the reviewer is a respected subject matter expert. Contacts made during the peer review process can lead to long-lasting collaboration. Continuing medical education credit can be obtained through various mechanisms. Overall, participating in peer review is an important part of career development and should be viewed as a critical component of advancement.

The privilege and burden of peer review.

PubMed

Blume, Arthur W; Schmaling, Karen B

2016-01-01

Peer review is a core value and method of quality control in psychological research, academic psychology, and other disciplines, but little is known about the peer-reviewing behavior of ethnic minority reviewers in particular. The purpose of this study was to examine the self-identified ethnicity of those invited to peer review articles for 76 journals that utilized the American Psychological Association's Journals Back Office (JBO) system from 2003 to 2012. It was hypothesized that a modest increase in the ratio of requests for reviews from self-identified ethnic minority reviewers would be observed over time, that self-identified ethnic minority reviewers would be less likely to refuse a review request than those who do not self-identify as an ethnic minority, and finally that increases in reviewer burden would be evident in significant increases in declines to requests by all reviewers. Reviewer requests and responses were examined among the 76 journals that used the JBO system over a 10-year period. Using hierarchical linear models, the percentage of review invitations extended to ethnic minorities was found to significantly increase over time: Initially, an estimated 8.34% of review requests were made to ethnic minority reviewers, and that percentage increased an average of 0.41% per year. Ethnic minority reviewers were significantly less likely to refuse a review request than ethnic majority reviewers. Results are discussed in terms of perceived pressure to demonstrate scholastic impact and the disproportionate service burden often borne by ethnic minority psychologists. (c) 2016 APA, all rights reserved).

Effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial.

PubMed

van Rooyen, Susan; Delamothe, Tony; Evans, Stephen J W

2010-11-16

To see whether telling peer reviewers that their signed reviews of original research papers might be posted on the BMJ's website would affect the quality of their reviews. Randomised controlled trial. A large international general medical journal based in the United Kingdom. 541 authors, 471 peer reviewers, and 12 editors. Consecutive eligible papers were randomised either to have the reviewer's signed report made available on the BMJ's website alongside the published paper (intervention group) or to have the report made available only to the author-the BMJ's normal procedure (control group). The intervention was the act of revealing to reviewers-after they had agreed to review but before they undertook their review-that their signed report might appear on the website. The main outcome measure was the quality of the reviews, as independently rated on a scale of 1 to 5 using a validated instrument by two editors and the corresponding author. Authors and editors were blind to the intervention group. Authors rated review quality before the fate of their paper had been decided. Additional outcomes were the time taken to complete the review and the reviewer's recommendation regarding publication. 558 manuscripts were randomised, and 471 manuscripts remained after exclusions. Of the 1039 reviewers approached to take part in the study, 568 (55%) declined. Two editors' evaluations of the quality of the peer review were obtained for all 471 manuscripts, with the corresponding author's evaluation obtained for 453. There was no significant difference in review quality between the intervention and control groups (mean difference for editors 0.04, 95% CI -0.09 to 0.17; for authors 0.06, 95% CI -0.09 to 0.20). Any possible difference in favour of the control group was well below the level regarded as editorially significant. Reviewers in the intervention group took significantly longer to review (mean difference 25 minutes, 95% CI 3.0 to 47.0 minutes). Telling peer reviewers that their signed reviews might be available in the public domain on the BMJ's website had no important effect on review quality. Although the possibility of posting reviews online was associated with a high refusal rate among potential peer reviewers and an increase in the amount of time taken to write a review, we believe that the ethical arguments in favour of open peer review more than outweigh these disadvantages.

Summarizing systematic reviews: methodological development, conduct and reporting of an umbrella review approach.

PubMed

Aromataris, Edoardo; Fernandez, Ritin; Godfrey, Christina M; Holly, Cheryl; Khalil, Hanan; Tungpunkom, Patraporn

2015-09-01

With the increase in the number of systematic reviews available, a logical next step to provide decision makers in healthcare with the evidence they require has been the conduct of reviews of existing systematic reviews. Syntheses of existing systematic reviews are referred to by many different names, one of which is an umbrella review. An umbrella review allows the findings of reviews relevant to a review question to be compared and contrasted. An umbrella review's most characteristic feature is that this type of evidence synthesis only considers for inclusion the highest level of evidence, namely other systematic reviews and meta-analyses. A methodology working group was formed by the Joanna Briggs Institute to develop methodological guidance for the conduct of an umbrella review, including diverse types of evidence, both quantitative and qualitative. The aim of this study is to describe the development and guidance for the conduct of an umbrella review. Discussion and testing of the elements of methods for the conduct of an umbrella review were held over a 6-month period by members of a methodology working group. The working group comprised six participants who corresponded via teleconference, e-mail and face-to-face meeting during this development period. In October 2013, the methodology was presented in a workshop at the Joanna Briggs Institute Convention. Workshop participants, review authors and methodologists provided further testing, critique and feedback on the proposed methodology. This study describes the methodology and methods developed for the conduct of an umbrella review that includes published systematic reviews and meta-analyses as the analytical unit of the review. Details are provided regarding the essential elements of an umbrella review, including presentation of the review question in a Population, Intervention, Comparator, Outcome format, nuances of the inclusion criteria and search strategy. A critical appraisal tool with 10 questions to help assess risk of bias in systematic reviews and meta-analyses was also developed and tested. Relevant details to extract from included reviews and how to best present the findings of both quantitative and qualitative systematic reviews in a reader friendly format are provided. Umbrella reviews provide a ready means for decision makers in healthcare to gain a clear understanding of a broad topic area. The umbrella review methodology described here is the first to consider reviews that report other than quantitative evidence derived from randomized controlled trials. The methodology includes an easy to use and informative summary of evidence table to readily provide decision makers with the available, highest level of evidence relevant to the question posed.

Optimal strategies to consider when peer reviewing a systematic review and meta-analysis.

PubMed

Moher, David

2015-11-02

Systematic reviews are popular. A recent estimate indicates that 11 new systematic reviews are published daily. Nevertheless, evidence indicates that the quality of reporting of systematic reviews is not optimal. One likely reason is that the authors' reports have received inadequate peer review. There are now many different types of systematic reviews and peer reviewing them can be enhanced by using a reporting guideline to supplement whatever template the journal editors have asked you, as a peer reviewer, to use. Additionally, keeping up with the current literature, whether as a content expert or being aware of advances in systematic review methods is likely be make for a more comprehensive and effective peer review. Providing a brief summary of what the systematic review has reported is an important first step in the peer review process (and not performed frequently enough). At its core, it provides the authors with some sense of what the peer reviewer believes was performed (Methods) and found (Results). Importantly, it also provides clarity regarding any potential problems in the methods, including statistical approaches for meta-analysis, results, and interpretation of the systematic review, for which the peer reviewer can seek explanations from the authors; these clarifications are best presented as questions to the authors.

E-commerce Review System to Detect False Reviews.

PubMed

Kolhar, Manjur

2017-08-15

E-commerce sites have been doing profitable business since their induction in high-speed and secured networks. Moreover, they continue to influence consumers through various methods. One of the most effective methods is the e-commerce review rating system, in which consumers provide review ratings for the products used. However, almost all e-commerce review rating systems are unable to provide cumulative review ratings. Furthermore, review ratings are influenced by positive and negative malicious feedback ratings, collectively called false reviews. In this paper, we proposed an e-commerce review system framework developed using the cumulative sum method to detect and remove malicious review ratings.

Peer review and the publication process.

PubMed

Ali, Parveen Azam; Watson, Roger

2016-10-01

To provide an overview of the peer review process, its various types, selection of peer reviewers, the purpose and significance of the peer review with regard to the assessment and management of quality of publications in academic journals. Discussion paper. This paper draws on information gained from literature on the peer review process and the authors' knowledge and experience of contributing as peer reviewers and editors in the field of health care, including nursing. There are various types of peer review: single blind; double blind; open; and post-publication review. The role of the reviewers in reviewing manuscripts and their contribution to the scientific and academic community remains important.

40 CFR 725.170 - EPA review of the MCAN.

Code of Federal Regulations, 2014 CFR

2014-07-01

... 40 Protection of Environment 31 2014-07-01 2014-07-01 false EPA review of the MCAN. 725.170... Notification Requirements § 725.170 EPA review of the MCAN. General procedures for review of all submissions... to EPA review of MCANs submitted under this subpart: (a) Length of the review period. The MCAN review...

40 CFR 725.170 - EPA review of the MCAN.

Code of Federal Regulations, 2010 CFR

2010-07-01

... 40 Protection of Environment 30 2010-07-01 2010-07-01 false EPA review of the MCAN. 725.170... Notification Requirements § 725.170 EPA review of the MCAN. General procedures for review of all submissions... to EPA review of MCANs submitted under this subpart: (a) Length of the review period. The MCAN review...

15 CFR 752.14 - System reviews.

Code of Federal Regulations, 2014 CFR

2014-01-01

... 15 Commerce and Foreign Trade 2 2014-01-01 2014-01-01 false System reviews. 752.14 Section 752.14... LICENSE § 752.14 System reviews. (a) Post-license system reviews. BIS may conduct system reviews of the... consignees in advance of a system review. The review will involve interviews with company officials, the...

NOAA Office of Exploration and Research > Home

Science.gov Websites

2014 Funding Opportunities Contact Us Program Review Review Panel Final Report Review Team Documents Review Review Panel Final Report Review Team Documents Presentations Supporting Documents Guiding the NOAA Science Advisory Board, a thirteen-member Review Panel conducted the first-ever formal

42 CFR 457.1140 - Program specific review process: Core elements of review.

Code of Federal Regulations, 2011 CFR

2011-10-01

... 42 Public Health 4 2011-10-01 2011-10-01 false Program specific review process: Core elements of review. 457.1140 Section 457.1140 Public Health CENTERS FOR MEDICARE & MEDICAID SERVICES, DEPARTMENT OF... review process: Core elements of review. In adopting the procedures for review of matters described in...

42 CFR 493.575 - Removal of deeming authority or CLIA exemption and final determination review.

Code of Federal Regulations, 2010 CFR

2010-10-01

... review findings, irrespective of the rate of disparity, indicate widespread or systematic problems in the... final determination review. 493.575 Section 493.575 Public Health CENTERS FOR MEDICARE & MEDICAID... determination review. (a) CMS review. CMS conducts a review of the following: (1) A deeming authority review of...

42 CFR 493.575 - Removal of deeming authority or CLIA exemption and final determination review.

Code of Federal Regulations, 2011 CFR

2011-10-01

... review findings, irrespective of the rate of disparity, indicate widespread or systematic problems in the... final determination review. 493.575 Section 493.575 Public Health CENTERS FOR MEDICARE & MEDICAID... determination review. (a) CMS review. CMS conducts a review of the following: (1) A deeming authority review of...

42 CFR 8.25 - Informal review and the reviewing official's response.

Code of Federal Regulations, 2011 CFR

2011-10-01

... 42 Public Health 1 2011-10-01 2011-10-01 false Informal review and the reviewing official's response. 8.25 Section 8.25 Public Health PUBLIC HEALTH SERVICE, DEPARTMENT OF HEALTH AND HUMAN SERVICES... Accreditation Body § 8.25 Informal review and the reviewing official's response. (a) Request for review. Within...

42 CFR 8.25 - Informal review and the reviewing official's response.

Code of Federal Regulations, 2010 CFR

2010-10-01

... 42 Public Health 1 2010-10-01 2010-10-01 false Informal review and the reviewing official's response. 8.25 Section 8.25 Public Health PUBLIC HEALTH SERVICE, DEPARTMENT OF HEALTH AND HUMAN SERVICES... Accreditation Body § 8.25 Informal review and the reviewing official's response. (a) Request for review. Within...

77 FR 73060 - Standard Review Plan for Review of Fuel Cycle Facility License Applications

Federal Register 2010, 2011, 2012, 2013, 2014

2012-12-07

... NUCLEAR REGULATORY COMMISSION [NRC-2012-0220] Standard Review Plan for Review of Fuel Cycle... 1, ``Standard Review Plan (SRP) for the Review of a License Application for a Fuel Cycle Facility... for a fuel cycle facility (NUREG-1520) provides NRC staff guidance for reviewing and evaluating the...

42 CFR 457.1140 - Program specific review process: Core elements of review.

Code of Federal Regulations, 2010 CFR

2010-10-01

... 42 Public Health 4 2010-10-01 2010-10-01 false Program specific review process: Core elements of review. 457.1140 Section 457.1140 Public Health CENTERS FOR MEDICARE & MEDICAID SERVICES, DEPARTMENT OF... review process: Core elements of review. In adopting the procedures for review of matters described in...

What are the best methodologies for rapid reviews of the research evidence for evidence-informed decision making in health policy and practice: a rapid review.

PubMed

Haby, Michelle M; Chapman, Evelina; Clark, Rachel; Barreto, Jorge; Reveiz, Ludovic; Lavis, John N

2016-11-25

Rapid reviews have the potential to overcome a key barrier to the use of research evidence in decision making, namely that of the lack of timely and relevant research. This rapid review of systematic reviews and primary studies sought to answer the question: What are the best methodologies to enable a rapid review of research evidence for evidence-informed decision making in health policy and practice? This rapid review utilised systematic review methods and was conducted according to a pre-defined protocol including clear inclusion criteria (PROSPERO registration: CRD42015015998). A comprehensive search strategy was used, including published and grey literature, written in English, French, Portuguese or Spanish, from 2004 onwards. Eleven databases and two websites were searched. Two review authors independently applied the eligibility criteria. Data extraction was done by one reviewer and checked by a second. The methodological quality of included studies was assessed independently by two reviewers. A narrative summary of the results is presented. Five systematic reviews and one randomised controlled trial (RCT) that investigated methodologies for rapid reviews met the inclusion criteria. None of the systematic reviews were of sufficient quality to allow firm conclusions to be made. Thus, the findings need to be treated with caution. There is no agreed definition of rapid reviews in the literature and no agreed methodology for conducting rapid reviews. While a wide range of 'shortcuts' are used to make rapid reviews faster than a full systematic review, the included studies found little empirical evidence of their impact on the conclusions of either rapid or systematic reviews. There is some evidence from the included RCT (that had a low risk of bias) that rapid reviews may improve clarity and accessibility of research evidence for decision makers. Greater care needs to be taken in improving the transparency of the methods used in rapid review products. There is no evidence available to suggest that rapid reviews should not be done or that they are misleading in any way. We offer an improved definition of rapid reviews to guide future research as well as clearer guidance for policy and practice.

Subspecialty Influence on Scientific Peer Review for an Obstetrics and Gynecology Journal With a High Impact Factor.

PubMed

Parikh, Laura I; Benner, Rebecca S; Riggs, Thomas W; Hazen, Nicholas; Chescheir, Nancy C

2017-02-01

To evaluate whether quality of peer review and reviewer recommendation differ based on reviewer subspecialty in obstetrics and gynecology and to determine the role of experience on reviewer recommendation. We performed a retrospective cohort study of reviews submitted to Obstetrics & Gynecology between January 2010 and December 2014. Subspecialties were determined based on classification terms selected by each reviewer and included all major obstetrics and gynecology subspecialties, general obstetrics and gynecology, and nonobstetrics and gynecology categories. Review quality (graded on a 5-point Likert scale by the journal's editors) and reviewer recommendation of "reject" were compared across subspecialties using χ, analysis of variance, and multivariate logistic regression. There were 20,027 reviews from 1,889 individual reviewers. Reviewers with family planning subspecialty provided higher-quality peer reviews compared with reviewers with gynecology only, reproductive endocrinology and infertility, gynecologic oncology, and general obstetrics and gynecology specialties (3.61±0.75 compared with 3.44±0.78, 3.42±0.72, 3.35±0.75, and 3.32±0.81, respectively, P<.05). Reviewers with gynecology-only subspecialty recommended rejection more often compared with reviewers with a nonobstetrics and gynecology subspecialty (57.7% compared with 38.7%, P<.05). Editorial Board members recommended rejection more often than new reviewers (68.0% compared with 41.5%, P<.05). Increased adjusted odds of manuscript rejection recommendation were associated with reproductive endocrinology, female pelvic medicine and reconstructive surgery, and gynecology-only reviewer subspecialty (adjusted odds ratio [OR] 1.23 [1.07-1.41], 1.21 [1.05-1.39], and 1.11 [1.02-1.20]). Manuscript rejection recommendation rate was also increased for reviewers who had completed the highest quintile of peer reviews (greater than 195) compared with the lowest quintile (one to seven) (adjusted OR 2.85 [2.60-3.12]). Peer review quality differs based on obstetrics and gynecology subspecialty. Obstetrics and gynecology subspecialty and reviewer experience have implications for manuscript rejection recommendation. Reviewer assignment is pivotal to maintaining a rigorous manuscript selection process.

NOx SOx Secondary NAAQS: Integrated Review Plan ...

EPA Pesticide Factsheets

The NOx SOx Secondary NAAQS Integrated Review Plan is the first document generated as part of the National Ambient Air Quality Standards (NAAQS) review process. The Plan presents background information, the schedule for the review, the process to be used in conducting the review, and the key policy-relevant science issues that will guide the review. The integrated review plan also discusses the frameworks for the various assessment documents to be prepared by the EPA as part of the review, including an Integrated Science Assessment (ISA), and as warranted, a Risk/Exposure Assessment (REA), and a Policy Assessment (PA). The primary purpose of the NOx SOx Secondary NAAQS Integrated Review Plan is to highlight the key policy-relevant issues to be considered in the Review of the Oxides of Nitrogen and Oxides of Sulfur Secondary NAAQS. A draft of the integrated review plan will be the subject of an advisory review with the Clean Air Scientific Advisory Committee (CASAC) and made available to the public for review and comment.

What's in a name? The challenge of describing interventions in systematic reviews: analysis of a random sample of reviews of non-pharmacological stroke interventions

PubMed Central

Hoffmann, Tammy C; Walker, Marion F; Langhorne, Peter; Eames, Sally; Thomas, Emma; Glasziou, Paul

2015-01-01

Objective To assess, in a sample of systematic reviews of non-pharmacological interventions, the completeness of intervention reporting, identify the most frequently missing elements, and assess review authors’ use of and beliefs about providing intervention information. Design Analysis of a random sample of systematic reviews of non-pharmacological stroke interventions; online survey of review authors. Data sources and study selection The Cochrane Library and PubMed were searched for potentially eligible systematic reviews and a random sample of these assessed for eligibility until 60 (30 Cochrane, 30 non-Cochrane) eligible reviews were identified. Data collection In each review, the completeness of the intervention description in each eligible trial (n=568) was assessed by 2 independent raters using the Template for Intervention Description and Replication (TIDieR) checklist. All review authors (n=46) were invited to complete a survey. Results Most reviews were missing intervention information for the majority of items. The most incompletely described items were: modifications, fidelity, materials, procedure and tailoring (missing from all interventions in 97%, 90%, 88%, 83% and 83% of reviews, respectively). Items that scored better, but were still incomplete for the majority of reviews, were: ‘when and how much’ (in 31% of reviews, adequate for all trials; in 57% of reviews, adequate for some trials); intervention mode (in 22% of reviews, adequate for all trials; in 38%, adequate for some trials); and location (in 19% of reviews, adequate for all trials). Of the 33 (71%) authors who responded, 58% reported having further intervention information but not including it, and 70% tried to obtain information. Conclusions Most focus on intervention reporting has been directed at trials. Poor intervention reporting in stroke systematic reviews is prevalent, compounded by poor trial reporting. Without adequate intervention descriptions, the conduct, usability and interpretation of reviews are restricted and therefore, require action by trialists, systematic reviewers, peer reviewers and editors. PMID:26576811

Association between prospective registration and overall reporting and methodological quality of systematic reviews: a meta-epidemiological study.

PubMed

Ge, Long; Tian, Jin-Hui; Li, Ya-Nan; Pan, Jia-Xue; Li, Ge; Wei, Dang; Xing, Xin; Pan, Bei; Chen, Yao-Long; Song, Fu-Jian; Yang, Ke-Hu

2018-01-01

The aim of this study was to investigate the differences in main characteristics, reporting and methodological quality between prospectively registered and nonregistered systematic reviews. PubMed was searched to identify systematic reviews of randomized controlled trials published in 2015 in English. After title and abstract screening, potentially relevant reviews were divided into three groups: registered non-Cochrane reviews, Cochrane reviews, and nonregistered reviews. For each group, random number tables were generated in Microsoft Excel, and the first 50 eligible studies from each group were randomly selected. Data of interest from systematic reviews were extracted. Regression analyses were conducted to explore the association between total Revised Assessment of Multiple Systematic Review (R-AMSTAR) or Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) scores and the selected characteristics of systematic reviews. The conducting and reporting of literature search in registered reviews were superior to nonregistered reviews. Differences in 9 of the 11 R-AMSTAR items were statistically significant between registered and nonregistered reviews. The total R-AMSTAR score of registered reviews was higher than nonregistered reviews [mean difference (MD) = 4.82, 95% confidence interval (CI): 3.70, 5.94]. Sensitivity analysis by excluding the registration-related item presented similar result (MD = 4.34, 95% CI: 3.28, 5.40). Total PRISMA scores of registered reviews were significantly higher than nonregistered reviews (all reviews: MD = 1.47, 95% CI: 0.64-2.30; non-Cochrane reviews: MD = 1.49, 95% CI: 0.56-2.42). However, the difference in the total PRISMA score was no longer statistically significant after excluding the item related to registration (item 5). Regression analyses showed similar results. Prospective registration may at least indirectly improve the overall methodological quality of systematic reviews, although its impact on the overall reporting quality was not significant. Copyright © 2017 Elsevier Inc. All rights reserved.

Quality of reviews on sugar-sweetened beverages and health outcomes: a systematic review123

PubMed Central

Weed, Douglas L; Mink, Pamela J

2011-01-01

Background: Medical and public health decisions are informed by reviews, which makes the quality of reviews an important scientific concern. Objective: We systematically assessed the quality of published reviews on sugar-sweetened beverages (SSBs) and health, which is a controversial topic that is important to public health. Design: We performed a search of PubMed and Cochrane databases and a hand search of reference lists. Studies that were selected were published reviews and meta-analyses (June 2001 to June 2011) of epidemiologic studies of the relation between SSBs and obesity, type 2 diabetes, metabolic syndrome, and coronary heart disease. A standardized data-abstraction form was used. Review quality was assessed by using the validated instrument AMSTAR (assessment of multiple systematic reviews), which is a one-page tool with 11 questions. Results: Seventeen reviews met our inclusion and exclusion criteria: obesity or weight (16 reviews), diabetes (3 reviews), metabolic syndrome (3 reviews), and coronary heart disease (2 reviews). Authors frequently used a strictly narrative review (7 of 17 reviews). Only 6 of 17 reviews reported quantitative data in a table format. Overall, reviews of SSBs and health outcomes received moderately low–quality scores by the AMSTAR [mean: 4.4 points; median: 4 points; range: 1–8.5 points (out of a possible score of 11 points)]. AMSTAR scores were not related to the conclusions of authors (8 reviews reported an association with a mean AMSTAR score of 4.1 points; 9 reviews with equivocal conclusions scored 4.7 points; P value = 0.84). Less than one-third of published reviews reported a comprehensive literature search, listed included and excluded studies, or used duplicate study selection and data abstraction. Conclusion: The comprehensive reporting of epidemiologic evidence and use of systematic methodologies to interpret evidence were underused in published reviews on SSBs and health. PMID:21918218

Sulfur Dioxide (SO2) Primary NAAQS Review: Integrated ...

EPA Pesticide Factsheets

The SO2 Integrated Review Plan is the first document generated as part of the National Ambient Air Quality Standards (NAAQS) review process. The Plan presents background information, the schedule for the review, the process to be used in conducting the review, and the key policy-relevant science issues that will guide the review. The integrated review plan also discusses the frameworks for the various assessment documents to be prepared by the EPA as part of the review, including an Integrated Science Assessment (ISA), and as warranted, a Risk/Exposure Assessment (REA), and a Policy Assessment (PA). The primary purpose of the SO2 Integrated Review Plan is to highlight the key policy-relevant issues to be considered in the Review of the SO2 primary NAAQS. A draft of the integrated review plan will be the subject of an advisory with the Clean Air Scientific Advisory Committee (CASAC) and made available to the public for review and comment.

Comparison of Onsite Versus Online Chart Reviews as Part of the American College of Radiation Oncology Accreditation Program.

PubMed

Hepel, Jaroslaw T; Heron, Dwight E; Mundt, Arno J; Yashar, Catheryn; Feigenberg, Steven; Koltis, Gordon; Regine, William F; Prasad, Dheerendra; Patel, Shilpen; Sharma, Navesh; Hebert, Mary; Wallis, Norman; Kuettel, Michael

2017-05-01

Accreditation based on peer review of professional standards of care is essential in ensuring quality and safety in administration of radiation therapy. Traditionally, medical chart reviews have been performed by a physical onsite visit. The American College of Radiation Oncology Accreditation Program has remodeled its process whereby electronic charts are reviewed remotely. Twenty-eight radiation oncology practices undergoing accreditation had three charts per practice undergo both onsite and online review. Onsite review was performed by a single reviewer for each practice. Online review consisted of one or more disease site-specific reviewers for each practice. Onsite and online reviews were blinded and scored on a 100-point scale on the basis of 20 categories. A score of less than 75 was failing, and a score of 75 to 79 was marginal. Any failed charts underwent rereview by a disease site team leader. Eighty-four charts underwent both onsite and online review. The mean scores were 86.0 and 86.9 points for charts reviewed onsite and online, respectively. Comparison of onsite and online reviews revealed no statistical difference in chart scores ( P = .43). Of charts reviewed, 21% had a marginal (n = 8) or failing (n = 10) score. There was no difference in failing charts ( P = .48) or combined marginal and failing charts ( P = .13) comparing onsite and online reviews. The American College of Radiation Oncology accreditation process of online chart review results in comparable review scores and rate of failing scores compared with traditional on-site review. However, the modern online process holds less potential for bias by using multiple reviewers per practice and allows for greater oversight via disease site team leader rereview.

Evaluating the care of general medicine inpatients: how good is implicit review?

PubMed

Hayward, R A; McMahon, L F; Bernard, A M

1993-04-01

Peer review often consists of implicit evaluations by physician reviewers of the quality and appropriateness of care. This study evaluated the ability of implicit review to measure reliably various aspects of care on a general medicine inpatient service. Retrospective review of patients' charts, using structured implicit review, of a stratified random sample of consecutive admissions to a general medicine ward. A university teaching hospital. Twelve internists were trained in structured implicit review and reviewed 675 patient admissions (with 20% duplicate reviews for a total of 846 reviews). Although inter-rater reliabilities for assessments of overall quality of care and preventable deaths (kappa = 0.5) were adequate for aggregate comparisons (for example, comparing mean ratings on two hospital wards), they were inadequate for reliable evaluations of single patients using one or two reviewers. Reviewers' agreement about most focused quality problems (for example, timeliness of diagnostic evaluation and clinical readiness at time of discharge) and about the appropriateness of hospital ancillary resource use was poor (kappa < or = 0.2). For most focused implicit measures, bias due to specific reviewers who were systematically more harsh or lenient (particularly for evaluation of resource-use appropriateness) accounted for much of the variation in reviewers' assessments, but this was not a substantial problem for the measure of overall quality. Reviewers rarely reported being unable to evaluate the quality of care because of deficiencies in documentation in the patient's chart. For assessment of overall quality and preventable deaths of general medicine inpatients, implicit review by peers had moderate degrees of reliability, but for most other specific aspects of care, physician reviewers could not agree. Implicit review was particularly unreliable at evaluating the appropriateness of hospital resource use and the patient's readiness for discharge, two areas where this type of review is often used.

Cooperation between referees and authors increases peer review accuracy.

PubMed

Leek, Jeffrey T; Taub, Margaret A; Pineda, Fernando J

2011-01-01

Peer review is fundamentally a cooperative process between scientists in a community who agree to review each other's work in an unbiased fashion. Peer review is the foundation for decisions concerning publication in journals, awarding of grants, and academic promotion. Here we perform a laboratory study of open and closed peer review based on an online game. We show that when reviewer behavior was made public under open review, reviewers were rewarded for refereeing and formed significantly more cooperative interactions (13% increase in cooperation, P = 0.018). We also show that referees and authors who participated in cooperative interactions had an 11% higher reviewing accuracy rate (P = 0.016). Our results suggest that increasing cooperation in the peer review process can lead to a decreased risk of reviewing errors.

Systematising "System": One Reviewer's Analysis of the Review Process

ERIC Educational Resources Information Center

Coniam, David

2011-01-01

This paper describes one reviewer's experience of reviewing for the journal "System" over an eight-year period, 2003-2011. The paper reports on the reviews produced by the single reviewer, which have been compiled into a specific purpose--an "occluded"--corpus (Swales, 1996) of 122 reviews, comprising 93,000 words. The paper first describes the…

10 CFR 13.5 - Review by the reviewing official.

Code of Federal Regulations, 2013 CFR

2013-01-01

... 10 Energy 1 2013-01-01 2013-01-01 false Review by the reviewing official. 13.5 Section 13.5 Energy NUCLEAR REGULATORY COMMISSION PROGRAM FRAUD CIVIL REMEDIES § 13.5 Review by the reviewing official. (a) If, based on the report of the investigating official under § 13.4(b), the reviewing official determines...

10 CFR 13.5 - Review by the reviewing official.

Code of Federal Regulations, 2010 CFR

2010-01-01

... 10 Energy 1 2010-01-01 2010-01-01 false Review by the reviewing official. 13.5 Section 13.5 Energy NUCLEAR REGULATORY COMMISSION PROGRAM FRAUD CIVIL REMEDIES § 13.5 Review by the reviewing official. (a) If, based on the report of the investigating official under § 13.4(b), the reviewing official determines...

NOAA Office of Exploration and Research > Education > Materials

Science.gov Websites

2014 Funding Opportunities Contact Us Program Review Review Panel Final Report Review Team Documents Review Review Panel Final Report Review Team Documents Presentations Supporting Documents Guiding

Assessment of the NASA Flight Assurance Review Program

NASA Technical Reports Server (NTRS)

Holmes, J.; Pruitt, G.

1983-01-01

The NASA flight assurance review program to develop minimum standard guidelines for flight assurance reviews was assessed. Documents from NASA centers and NASA headquarters to determine current design review practices and procedures were evaluated. Six reviews were identified for the recommended minimum. The practices and procedures used at the different centers to incorporate the most effective ones into the minimum standard review guidelines were analyzed and guidelines for procedures, personnel and responsibilies, review items/data checklist, and feedback and closeout were defined. The six recommended reviews and the minimum standards guidelines developed for flight assurance reviews are presented. Observations and conclusions for further improving the NASA review and quality assurance process are outlined.

Appraisal of systematic reviews on the management of peri-implant diseases with two methodological tools.

PubMed

Faggion, Clovis Mariano; Monje, Alberto; Wasiak, Jason

2018-06-01

This study aimed to evaluate and compare the performance of two methodological instruments to appraise systematic reviews and to identify potential disagreements of systematic review authors regarding risk of bias (RoB) evaluation of randomized controlled trials (RCTs) included in systematic reviews on peri-implant diseases. We searched Medline, Web of Science, Cochrane Library, PubMed Central, and Google Scholar for systematic reviews on peri-implant diseases published before July 11, 2017. Two authors independently evaluated the RoB and methodological quality of the systematic reviews by applying the Risk of Bias in Systematic Reviews (ROBIS) tool and Assessing the Methodological Quality of Systematic Reviews (AMSTAR) checklist, respectively. We assessed the RoB scores of the same RCTs published in different systematic reviews. Of the 32 systematic reviews identified, 23 reviews addressed the clinical topic of peri-implantitis. A high RoB was detected for most systematic reviews (n=25) using ROBIS, whilst five systematic reviews displayed low methodological quality by AMSTAR. Almost 30% of the RoB comparisons (for the same RCTs) had different RoB ratings across systematic reviews. The ROBIS tool appears to provide more conservative results than AMSTAR checklist. Considerable disagreement was found among systematic review authors rating the same RCT included in different systematic reviews. © 2018 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd.

How to Write a Scholarly Book Review for Publication in a Peer-Reviewed Journal

PubMed Central

Lee, Alexander D.; Green, Bart N.; Johnson, Claire D.; Nyquist, Julie

2010-01-01

Purpose: To describe and discuss the processes used to write scholarly book reviews for publication in peer-reviewed journals and to provide a recommended strategy and book appraisal worksheet to use when conducting book reviews. Methods: A literature search of MEDLINE, EMBASE, CINAHL, and the Index to Chiropractic Literature was conducted in June 2009 using a combination of controlled vocabulary and truncated text words to capture articles relevant to writing scholarly book reviews for publication in peer-reviewed journals. Results: The initial search identified 839 citations. Following the removal of duplicates and the application of selection criteria, a total of 78 articles were included in this review including narrative commentaries (n = 26), editorials or journal announcements (n = 25), original research (n = 18), and journal correspondence pieces (n = 9). Discussion: Recommendations for planning and writing an objective and quality book review are presented based on the evidence gleaned from the articles reviewed and from the authors' experiences. A worksheet for conducting a book review is provided. Conclusions: The scholarly book review serves many purposes and has the potential to be an influential literary form. The process of publishing a successful scholarly book review requires the reviewer to appreciate the book review publication process and to be aware of the skills and strategies involved in writing a successful review. PMID:20480015

NOAA Office of Exploration and Research > Science > Overview

Science.gov Websites

2014 Funding Opportunities Contact Us Program Review Review Panel Final Report Review Team Documents Opportunities Contact Us Program Review Review Panel Final Report Review Team Documents Presentations Supporting

The integrative review: updated methodology.

PubMed

Whittemore, Robin; Knafl, Kathleen

2005-12-01

The aim of this paper is to distinguish the integrative review method from other review methods and to propose methodological strategies specific to the integrative review method to enhance the rigour of the process. Recent evidence-based practice initiatives have increased the need for and the production of all types of reviews of the literature (integrative reviews, systematic reviews, meta-analyses, and qualitative reviews). The integrative review method is the only approach that allows for the combination of diverse methodologies (for example, experimental and non-experimental research), and has the potential to play a greater role in evidence-based practice for nursing. With respect to the integrative review method, strategies to enhance data collection and extraction have been developed; however, methods of analysis, synthesis, and conclusion drawing remain poorly formulated. A modified framework for research reviews is presented to address issues specific to the integrative review method. Issues related to specifying the review purpose, searching the literature, evaluating data from primary sources, analysing data, and presenting the results are discussed. Data analysis methods of qualitative research are proposed as strategies that enhance the rigour of combining diverse methodologies as well as empirical and theoretical sources in an integrative review. An updated integrative review method has the potential to allow for diverse primary research methods to become a greater part of evidence-based practice initiatives.

Insights into the impact of online physician reviews on patients' decision making: randomized experiment.

PubMed

Grabner-Kräuter, Sonja; Waiguny, Martin K J

2015-04-09

Physician-rating websites combine public reporting with social networking and offer an attractive means by which users can provide feedback on their physician and obtain information about other patients' satisfaction and experiences. However, research on how users evaluate information on these portals is still scarce and only little knowledge is available about the potential influence of physician reviews on a patient's choice. Starting from the perspective of prospective patients, this paper sets out to explore how certain characteristics of physician reviews affect the evaluation of the review and users' attitudes toward the rated physician. We propose a model that relates review style and review number to constructs of review acceptance and check it with a Web-based experiment. We employed a randomized 2x2 between-subject, factorial experiment manipulating the style of a physician review (factual vs emotional) and the number of reviews for a certain physician (low vs high) to test our hypotheses. A total of 168 participants were presented with a Web-based questionnaire containing a short description of a dentist search scenario and the manipulated reviews for a fictitious dental physician. To investigate the proposed hypotheses, we carried out moderated regression analyses and a moderated mediation analysis using the PROCESS macro 2.11 for SPSS version 22. Our analyses indicated that a higher number of reviews resulted in a more positive attitude toward the rated physician. The results of the regression model for attitude toward the physician suggest a positive main effect of the number of reviews (mean [low] 3.73, standard error [SE] 0.13, mean [high] 4.15, SE 0.13). We also observed an interaction effect with the style of the review—if the physician received only a few reviews, fact-oriented reviews (mean 4.09, SE 0.19) induced a more favorable attitude toward the physician compared to emotional reviews (mean 3.44, SE 0.19), but there was no such effect when the physician received many reviews. Furthermore, we found that review style also affected the perceived expertise of the reviewer. Fact-oriented reviews (mean 3.90, SE 0.13) lead to a higher perception of reviewer expertise compared to emotional reviews (mean 3.19, SE 0.13). However, this did not transfer to the attitude toward the physician. A similar effect of review style and number on the perceived credibility of the review was observed. While no differences between emotional and factual style were found if the physician received many reviews, a low number of reviews received lead to a significant difference in the perceived credibility, indicating that emotional reviews were rated less positively (mean 3.52, SE 0.18) compared to fact-oriented reviews (mean 4.15, SE 0.17). Our analyses also showed that perceived credibility of the review fully mediated the observed interaction effect on attitude toward the physician. Physician-rating websites are an interesting new source of information about the quality of health care from the patient's perspective. This paper makes a unique contribution to an understudied area of research by providing some insights into how people evaluate online reviews of individual doctors. Information attributes, such as review style and review number, have an impact on the evaluation of the review and on the patient's attitude toward the rated doctor. Further research is necessary to improve our understanding of the influence of such rating sites on the patient's choice of a physician.

Insights Into the Impact of Online Physician Reviews on Patients’ Decision Making: Randomized Experiment

PubMed Central

Waiguny, Martin KJ

2015-01-01

Background Physician-rating websites combine public reporting with social networking and offer an attractive means by which users can provide feedback on their physician and obtain information about other patients’ satisfaction and experiences. However, research on how users evaluate information on these portals is still scarce and only little knowledge is available about the potential influence of physician reviews on a patient’s choice. Objective Starting from the perspective of prospective patients, this paper sets out to explore how certain characteristics of physician reviews affect the evaluation of the review and users’ attitudes toward the rated physician. We propose a model that relates review style and review number to constructs of review acceptance and check it with a Web-based experiment. Methods We employed a randomized 2x2 between-subject, factorial experiment manipulating the style of a physician review (factual vs emotional) and the number of reviews for a certain physician (low vs high) to test our hypotheses. A total of 168 participants were presented with a Web-based questionnaire containing a short description of a dentist search scenario and the manipulated reviews for a fictitious dental physician. To investigate the proposed hypotheses, we carried out moderated regression analyses and a moderated mediation analysis using the PROCESS macro 2.11 for SPSS version 22. Results Our analyses indicated that a higher number of reviews resulted in a more positive attitude toward the rated physician. The results of the regression model for attitude toward the physician suggest a positive main effect of the number of reviews (mean [low] 3.73, standard error [SE] 0.13, mean [high] 4.15, SE 0.13). We also observed an interaction effect with the style of the review—if the physician received only a few reviews, fact-oriented reviews (mean 4.09, SE 0.19) induced a more favorable attitude toward the physician compared to emotional reviews (mean 3.44, SE 0.19), but there was no such effect when the physician received many reviews. Furthermore, we found that review style also affected the perceived expertise of the reviewer. Fact-oriented reviews (mean 3.90, SE 0.13) lead to a higher perception of reviewer expertise compared to emotional reviews (mean 3.19, SE 0.13). However, this did not transfer to the attitude toward the physician. A similar effect of review style and number on the perceived credibility of the review was observed. While no differences between emotional and factual style were found if the physician received many reviews, a low number of reviews received lead to a significant difference in the perceived credibility, indicating that emotional reviews were rated less positively (mean 3.52, SE 0.18) compared to fact-oriented reviews (mean 4.15, SE 0.17). Our analyses also showed that perceived credibility of the review fully mediated the observed interaction effect on attitude toward the physician. Conclusions Physician-rating websites are an interesting new source of information about the quality of health care from the patient’s perspective. This paper makes a unique contribution to an understudied area of research by providing some insights into how people evaluate online reviews of individual doctors. Information attributes, such as review style and review number, have an impact on the evaluation of the review and on the patient’s attitude toward the rated doctor. Further research is necessary to improve our understanding of the influence of such rating sites on the patient's choice of a physician. PMID:25862516

The quality of systematic reviews about interventions for refractive error can be improved: a review of systematic reviews.

PubMed

Mayo-Wilson, Evan; Ng, Sueko Matsumura; Chuck, Roy S; Li, Tianjing

2017-09-05

Systematic reviews should inform American Academy of Ophthalmology (AAO) Preferred Practice Pattern® (PPP) guidelines. The quality of systematic reviews related to the forthcoming Preferred Practice Pattern® guideline (PPP) Refractive Errors & Refractive Surgery is unknown. We sought to identify reliable systematic reviews to assist the AAO Refractive Errors & Refractive Surgery PPP. Systematic reviews were eligible if they evaluated the effectiveness or safety of interventions included in the 2012 PPP Refractive Errors & Refractive Surgery. To identify potentially eligible systematic reviews, we searched the Cochrane Eyes and Vision United States Satellite database of systematic reviews. Two authors identified eligible reviews and abstracted information about the characteristics and quality of the reviews independently using the Systematic Review Data Repository. We classified systematic reviews as "reliable" when they (1) defined criteria for the selection of studies, (2) conducted comprehensive literature searches for eligible studies, (3) assessed the methodological quality (risk of bias) of the included studies, (4) used appropriate methods for meta-analyses (which we assessed only when meta-analyses were reported), (5) presented conclusions that were supported by the evidence provided in the review. We identified 124 systematic reviews related to refractive error; 39 met our eligibility criteria, of which we classified 11 to be reliable. Systematic reviews classified as unreliable did not define the criteria for selecting studies (5; 13%), did not assess methodological rigor (10; 26%), did not conduct comprehensive searches (17; 44%), or used inappropriate quantitative methods (3; 8%). The 11 reliable reviews were published between 2002 and 2016. They included 0 to 23 studies (median = 9) and analyzed 0 to 4696 participants (median = 666). Seven reliable reviews (64%) assessed surgical interventions. Most systematic reviews of interventions for refractive error are low methodological quality. Following widely accepted guidance, such as Cochrane or Institute of Medicine standards for conducting systematic reviews, would contribute to improved patient care and inform future research.

Conflicts of interest and spin in reviews of psychological therapies: a systematic review

PubMed Central

Lieb, Klaus; von der Osten-Sacken, Jan; Stoffers-Winterling, Jutta; Reiss, Neele; Barth, Jürgen

2016-01-01

Objective To explore conflicts of interest (COI) and their reporting in systematic reviews of psychological therapies, and to evaluate spin in the conclusions of the reviews. Methods MEDLINE and PsycINFO databases were searched for systematic reviews published between 2010 and 2013 that assessed effects of psychological therapies for anxiety, depressive or personality disorders, and included at least one randomised controlled trial. Required COI disclosure by journal, disclosed COI by review authors, and the inclusion of own primary studies by review authors were extracted. Researcher allegiance, that is, that researchers concluded favourably about the interventions they have studied, as well as spin, that is, differences between results and conclusions of the reviews, were rated by 2 independent raters. Results 936 references were retrieved, 95 reviews fulfilled eligibility criteria. 59 compared psychological therapies with other forms of psychological therapies, and 36 psychological therapies with pharmacological interventions. Financial, non-financial, and personal COI were disclosed in 22, 4 and 1 review, respectively. 2 of 86 own primary studies of review authors included in 34 reviews were disclosed by review authors. In 15 of the reviews, authors showed an allegiance effect to the evaluated psychological therapy that was never disclosed. Spin in review conclusions was found in 27 of 95 reviews. Reviews with a conclusion in favour of psychological therapies (vs pharmacological interventions) were at high risk for a spin in conclusions (OR=8.31 (1.41 to 49.05)). Spin was related in trend to the inclusion of own primary studies in the systematic review (OR=2.08 (CI 0.83 to 5.18) p=0.11) and researcher allegiance (OR=2.63 (0.84 to 8.16) p=0.16). Conclusions Non-financial COI, especially the inclusion of own primary studies into reviews and researcher allegiance, are frequently seen in systematic reviews of psychological therapies and need more transparency and better management. PMID:27118287

Economics methods in Cochrane systematic reviews of health promotion and public health related interventions.

PubMed

Shemilt, Ian; Mugford, Miranda; Drummond, Michael; Eisenstein, Eric; Mallender, Jacqueline; McDaid, David; Vale, Luke; Walker, Damian

2006-11-15

Provision of evidence on costs alongside evidence on the effects of interventions can enhance the relevance of systematic reviews to decision-making. However, patterns of use of economics methods alongside systematic review remain unclear. Reviews of evidence on the effects of interventions are published by both the Cochrane and Campbell Collaborations. Although it is not a requirement that Cochrane or Campbell Reviews should consider economic aspects of interventions, many do. This study aims to explore and describe approaches to incorporating economics methods in a selection of Cochrane systematic reviews in the area of health promotion and public health, to help inform development of methodological guidance on economics for reviewers. The Cochrane Database of Systematic Reviews was searched using a search strategy for potential economic evaluation studies. We included current Cochrane reviews and review protocols retrieved using the search that are also identified as relevant to health promotion or public health topics. A reviewer extracted data which describe the economics components of included reviews. Extracted data were summarised in tables and analysed qualitatively. Twenty-one completed Cochrane reviews and seven review protocols met inclusion criteria. None incorporate formal economic evaluation methods. Ten completed reviews explicitly aim to incorporate economics studies and data. There is a lack of transparent reporting of methods underpinning the incorporation of economics studies and data. Some reviews are likely to exclude useful economics studies and data due to a failure to incorporate search strategies tailored to the retrieval of such data or use of key specialist databases, and application of inclusion criteria designed for effectiveness studies. There is a need for consistency and transparency in the reporting and conduct of the economics components of Cochrane reviews, as well as regular dialogue between Cochrane reviewers and economists to develop increased capacity for economic analyses alongside such reviews. Use of applicable economics methods in Cochrane reviews can help provide the international context within which economics data can be interpreted and assessed as a preliminary to full economic evaluation.

Peer reviewing e-learning: opportunities, challenges, and solutions.

PubMed

Ruiz, Jorge G; Candler, Chris; Teasdale, Thomas A

2007-05-01

Peer review is the foundation of academic publication and a necessary step in the scrutiny of any scholarly work. Simply defined, peer review is the attentive, unbiased assessment of any scholarly work that is submitted for formal scrutiny. Although medical school faculty increasingly use technology in clinical teaching, e-learning materials are often not subjected to a rigorous peer review process. The authors contrast peer review of e-learning materials with that of print materials, describe peer review issues regarding e-learning materials, propose approaches to address the challenges of peer review of e-learning materials, and outline directions for refinement of the e-learning peer review process. At its core, the peer review of e-learning materials should not differ substantially from that of traditional manuscripts. However, e-learning introduces new demands that impel reviewers to consider aspects that are unique to educational technology, including pedagogy, format, usability, navigation, interactivity, delivery, ease of updating, distribution, and access. Four approaches are offered to ease the burden and improve the quality of e-learning peer review: develop peer review training, embrace multidisciplinary peer review, develop guidelines, and provide incentives and compensation. The authors conclude with suggestions about peer review research.

IRIS Toxicological Review of Methanol (Noncancer) (Interagency Science Discussion Draft)

EPA Science Inventory

On May 3, 2013, the Toxicological Review of Methanol (noncancer) (Revised External Review Draft) was posted for public review and comment. Subsequently, the draft Toxicological Review, Appendices, and draft IRIS Summary were reviewed internally by EPA and by other federal agenci...

Letting the daylight in: Reviewing the reviewers and other ways to maximize transparency in science

PubMed Central

Wicherts, Jelte M.; Kievit, Rogier A.; Bakker, Marjan; Borsboom, Denny

2012-01-01

With the emergence of online publishing, opportunities to maximize transparency of scientific research have grown considerably. However, these possibilities are still only marginally used. We argue for the implementation of (1) peer-reviewed peer review, (2) transparent editorial hierarchies, and (3) online data publication. First, peer-reviewed peer review entails a community-wide review system in which reviews are published online and rated by peers. This ensures accountability of reviewers, thereby increasing academic quality of reviews. Second, reviewers who write many highly regarded reviews may move to higher editorial positions. Third, online publication of data ensures the possibility of independent verification of inferential claims in published papers. This counters statistical errors and overly positive reporting of statistical results. We illustrate the benefits of these strategies by discussing an example in which the classical publication system has gone awry, namely controversial IQ research. We argue that this case would have likely been avoided using more transparent publication practices. We argue that the proposed system leads to better reviews, meritocratic editorial hierarchies, and a higher degree of replicability of statistical analyses. PMID:22536180

Producing Cochrane systematic reviews-a qualitative study of current approaches and opportunities for innovation and improvement.

PubMed

Turner, Tari; Green, Sally; Tovey, David; McDonald, Steve; Soares-Weiser, Karla; Pestridge, Charlotte; Elliott, Julian

2017-08-01

Producing high-quality, relevant systematic reviews and keeping them up to date is challenging. Cochrane is a leading provider of systematic reviews in health. For Cochrane to continue to contribute to improvements in heath, Cochrane Reviews must be rigorous, reliable and up to date. We aimed to explore existing models of Cochrane Review production and emerging opportunities to improve the efficiency and sustainability of these processes. To inform discussions about how to best achieve this, we conducted 26 interviews and an online survey with 106 respondents. Respondents highlighted the importance and challenge of creating reliable, timely systematic reviews. They described the challenges and opportunities presented by current production models, and they shared what they are doing to improve review production. They particularly highlighted significant challenges with increasing complexity of review methods; difficulty keeping authors on board and on track; and the length of time required to complete the process. Strong themes emerged about the roles of authors and Review Groups, the central actors in the review production process. The results suggest that improvements to Cochrane's systematic review production models could come from improving clarity of roles and expectations, ensuring continuity and consistency of input, enabling active management of the review process, centralising some review production steps; breaking reviews into smaller "chunks", and improving approaches to building capacity of and sharing information between authors and Review Groups. Respondents noted the important role new technologies have to play in enabling these improvements. The findings of this study will inform the development of new Cochrane Review production models and may provide valuable data for other systematic review producers as they consider how best to produce rigorous, reliable, up-to-date reviews.

How long is too long in contemporary peer review? Perspectives from authors publishing in conservation biology journals.

PubMed

Nguyen, Vivian M; Haddaway, Neal R; Gutowsky, Lee F G; Wilson, Alexander D M; Gallagher, Austin J; Donaldson, Michael R; Hammerschlag, Neil; Cooke, Steven J

2015-01-01

Delays in peer reviewed publication may have consequences for both assessment of scientific prowess in academics as well as communication of important information to the knowledge receptor community. We present an analysis on the perspectives of authors publishing in conservation biology journals regarding their opinions on the importance of speed in peer-review as well as how to improve review times. Authors were invited to take part in an online questionnaire, of which the data was subjected to both qualitative (open coding, categorizing) and quantitative analyses (generalized linear models). We received 637 responses to a total of 6,547 e-mail invitations sent. Peer-review speed was generally perceived as slow, with authors experiencing a typical turnaround time of 14 weeks while their perceived optimal review time is six weeks. Male and younger respondents seem to have higher expectations of review speed than females and older respondents. Majority of participants attributed lengthy review times to the 'stress' on the peer-review system (i.e., reviewer and editor fatigue), while editor persistence and journal prestige were believed to speed up the review process. Negative consequences of lengthy review times appear to be greater for early career researchers and can also have impact on author morale (e.g. motivation or frustration). Competition among colleagues were also of concern to respondents. Incentivizing peer review was among the top suggested alterations to the system along with training graduate students in peer review, increased editorial persistence, and changes to the norms of peer-review such as opening the peer-review process to the public. It is clear that authors surveyed in this study view the peer-review system as under stress and we encourage scientists and publishers to push the envelope for new peer review models.

What is open peer review? A systematic review.

PubMed

Ross-Hellauer, Tony

2017-01-01

Background : "Open peer review" (OPR), despite being a major pillar of Open Science, has neither a standardized definition nor an agreed schema of its features and implementations. The literature reflects this, with numerous overlapping and contradictory definitions. While for some the term refers to peer review where the identities of both author and reviewer are disclosed to each other, for others it signifies systems where reviewer reports are published alongside articles. For others it signifies both of these conditions, and for yet others it describes systems where not only "invited experts" are able to comment. For still others, it includes a variety of combinations of these and other novel methods. Methods : Recognising the absence of a consensus view on what open peer review is, this article undertakes a systematic review of definitions of "open peer review" or "open review", to create a corpus of 122 definitions. These definitions are systematically analysed to build a coherent typology of the various innovations in peer review signified by the term, and hence provide the precise technical definition currently lacking. Results : This quantifiable data yields rich information on the range and extent of differing definitions over time and by broad subject area. Quantifying definitions in this way allows us to accurately portray exactly how ambiguously the phrase "open peer review" has been used thus far, for the literature offers 22 distinct configurations of seven traits, effectively meaning that there are 22 different definitions of OPR in the literature reviewed. Conclusions : I propose a pragmatic definition of open peer review as an umbrella term for a number of overlapping ways that peer review models can be adapted in line with the aims of Open Science, including making reviewer and author identities open, publishing review reports and enabling greater participation in the peer review process.

Twelve recommendations for integrating existing systematic reviews into new reviews: EPC guidance.

PubMed

Robinson, Karen A; Chou, Roger; Berkman, Nancy D; Newberry, Sydne J; Fu, Rongwei; Hartling, Lisa; Dryden, Donna; Butler, Mary; Foisy, Michelle; Anderson, Johanna; Motu'apuaka, Makalapua; Relevo, Rose; Guise, Jeanne-Marie; Chang, Stephanie

2016-02-01

As time and cost constraints in the conduct of systematic reviews increase, the need to consider the use of existing systematic reviews also increases. We developed guidance on the integration of systematic reviews into new reviews. A workgroup of methodologists from Evidence-based Practice Centers developed consensus-based recommendations. Discussions were informed by a literature scan and by interviews with organizations that conduct systematic reviews. Twelve recommendations were developed addressing selecting reviews, assessing risk of bias, qualitative and quantitative synthesis, and summarizing and assessing body of evidence. We provide preliminary guidance for an efficient and unbiased approach to integrating existing systematic reviews with primary studies in a new review. Copyright © 2016 Elsevier Inc. All rights reserved.

17 CFR 201.400 - Interlocutory review.

Code of Federal Regulations, 2010 CFR

2010-04-01

... 17 Commodity and Securities Exchanges 2 2010-04-01 2010-04-01 false Interlocutory review. 201.400... Rules of Practice Appeal to the Commission and Commission Review § 201.400 Interlocutory review. (a... review. Petitions by parties for interlocutory review are disfavored, and the Commission ordinarily will...

IRIS Toxicological Review of Trimethylbenzenes (Interagency Science Consultation Draft)

EPA Science Inventory

On June 26, 2012, the draft Toxicological Review of Trimethylbenzenes and the draft charge to external peer reviewers were released for external peer review and public comment. The Toxicological Review and charge were reviewed internally by EPA and by other federal agencies and W...

IRIS Toxicological Review of Trichloroethylene (Interagency Science Consultation Draft)

EPA Science Inventory

On November 3, 2009, the Toxicological Review of Trichloroethylene and the charge to external peer reviewers were released for external peer review and public comment. The Toxicological Review and charge were reviewed internally by EPA and by other federal agencies and White Hous...

IRIS Toxicological Review of Formaldehyde (Interagency Science Consultation Draft)

EPA Science Inventory

On June 2, 2010, the Toxicological Review of Formaldehyde and the charge to external peer reviewers were released for external peer review and public comment. The Toxicological Review and charge were reviewed internally by EPA and by other federal agencies and White House Offices...

IRIS Toxicological Review of Biphenyl (Interagency Science Consultation Draft)

EPA Science Inventory

On September 30, 2011, the draft Toxicological Review of Biphenyl and the charge to external peer reviewers were released for external peer review and public comment. The Toxicological Review and charge were reviewed internally by EPA and by other federal agencies and White House...

40 CFR 155.57 - Registration review decision.

Code of Federal Regulations, 2010 CFR

2010-07-01

... 40 Protection of Environment 23 2010-07-01 2010-07-01 false Registration review decision. 155.57... REGISTRATION STANDARDS AND REGISTRATION REVIEW Registration Review Procedures § 155.57 Registration review decision. A registration review decision is the Agency's determination whether a pesticide meets, or does...

78 FR 17673 - Federal Acquisition Regulation; Submission for OMB Review; Contractors' Purchasing Systems Reviews

Federal Register 2010, 2011, 2012, 2013, 2014

2013-03-22

...; Submission for OMB Review; Contractors' Purchasing Systems Reviews AGENCY: Department of Defense (DOD... approved information collection requirement concerning contractors' purchasing systems reviews. A notice... Collection 9000- 0132, Contractors' Purchasing Systems Reviews, by any of the following methods: Regulations...

IRIS Toxicological Review of Acrylonitrile (Interagency Science Consultation Draft)

EPA Science Inventory

On June 30, 2011, the draft Toxicological Review of Acrylonitrile and the charge to external peer reviewers were released for external peer review and public comment. The Toxicological Review and charge were reviewed internally by EPA and by other federal agencies and White House...

Integer Linear Programming for Constrained Multi-Aspect Committee Review Assignment

PubMed Central

Karimzadehgan, Maryam; Zhai, ChengXiang

2011-01-01

Automatic review assignment can significantly improve the productivity of many people such as conference organizers, journal editors and grant administrators. A general setup of the review assignment problem involves assigning a set of reviewers on a committee to a set of documents to be reviewed under the constraint of review quota so that the reviewers assigned to a document can collectively cover multiple topic aspects of the document. No previous work has addressed such a setup of committee review assignments while also considering matching multiple aspects of topics and expertise. In this paper, we tackle the problem of committee review assignment with multi-aspect expertise matching by casting it as an integer linear programming problem. The proposed algorithm can naturally accommodate any probabilistic or deterministic method for modeling multiple aspects to automate committee review assignments. Evaluation using a multi-aspect review assignment test set constructed using ACM SIGIR publications shows that the proposed algorithm is effective and efficient for committee review assignments based on multi-aspect expertise matching. PMID:22711970

ACC/AHA Special Report: Clinical Practice Guideline Implementation Strategies: A Summary of Systematic Reviews by the NHLBI Implementation Science Work Group: A Report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines.

PubMed

Chan, Wiley V; Pearson, Thomas A; Bennett, Glen C; Cushman, William C; Gaziano, Thomas A; Gorman, Paul N; Handler, Joel; Krumholz, Harlan M; Kushner, Robert F; MacKenzie, Thomas D; Sacco, Ralph L; Smith, Sidney C; Stevens, Victor J; Wells, Barbara L

2017-02-28

In 2008, the National Heart, Lung, and Blood Institute convened an Implementation Science Work Group to assess evidence-based strategies for effectively implementing clinical practice guidelines. This was part of a larger effort to update existing clinical practice guidelines on cholesterol, blood pressure, and overweight/obesity. Review evidence from the published implementation science literature and identify effective or promising strategies to enhance the adoption and implementation of clinical practice guidelines. This systematic review was conducted on 4 critical questions, each focusing on the adoption and effectiveness of 4 intervention strategies: (1) reminders, (2) educational outreach visits, (3) audit and feedback, and (4) provider incentives. A scoping review of the Rx for Change database of systematic reviews was used to identify promising guideline implementation interventions aimed at providers. Inclusion and exclusion criteria were developed a priori for each question, and the published literature was initially searched up to 2012, and then updated with a supplemental search to 2015. Two independent reviewers screened the returned citations to identify relevant reviews and rated the quality of each included review. Audit and feedback and educational outreach visits were generally effective in improving both process of care (15 of 21 reviews and 12 of 13 reviews, respectively) and clinical outcomes (7 of 12 reviews and 3 of 5 reviews, respectively). Provider incentives showed mixed effectiveness for improving both process of care (3 of 4 reviews) and clinical outcomes (3 reviews equally distributed between generally effective, mixed, and generally ineffective). Reminders showed mixed effectiveness for improving process of care outcomes (27 reviews with 11 mixed and 3 generally ineffective results) and were generally ineffective for clinical outcomes (18 reviews with 6 mixed and 9 generally ineffective results). Educational outreach visits (2 of 2 reviews), reminders (3 of 4 reviews), and provider incentives (1 of 1 review) were generally effective for cost reduction. Educational outreach visits (1 of 1 review) and provider incentives (1 of 1 review) were also generally effective for cost-effectiveness outcomes. Barriers to clinician adoption or adherence to guidelines included time constraints (8 reviews/overviews); limited staffing resources (2 overviews); timing (5 reviews/overviews); clinician skepticism (5 reviews/overviews); clinician knowledge of guidelines (4 reviews/overviews); and higher age of the clinician (1 overview). Facilitating factors included guideline characteristics such as format, resources, and end-user involvement (6 reviews/overviews); involving stakeholders (5 reviews/overviews); leadership support (5 reviews/overviews); scope of implementation (5 reviews/overviews); organizational culture such as multidisciplinary teams and low-baseline adherence (9 reviews/overviews); and electronic guidelines systems (3 reviews). The strategies of audit and feedback and educational outreach visits were generally effective in improving both process of care and clinical outcomes. Reminders and provider incentives showed mixed effectiveness, or were generally ineffective. No general conclusion could be reached about cost effectiveness, because of limitations in the evidence. Important gaps exist in the evidence on effectiveness of implementation interventions, especially regarding clinical outcomes, cost effectiveness and contextual issues affecting successful implementation. Copyright © 2017 American College of Cardiology Foundation and American Heart Association, Inc. Published by Elsevier Inc. All rights reserved.

ACC/AHA Special Report: Clinical Practice Guideline Implementation Strategies: A Summary of Systematic Reviews by the NHLBI Implementation Science Work Group: A Report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines.

PubMed

Chan, Wiley V; Pearson, Thomas A; Bennett, Glen C; Cushman, William C; Gaziano, Thomas A; Gorman, Paul N; Handler, Joel; Krumholz, Harlan M; Kushner, Robert F; MacKenzie, Thomas D; Sacco, Ralph L; Smith, Sidney C; Stevens, Victor J; Wells, Barbara L; Castillo, Graciela; Heil, Susan K R; Stephens, Jennifer; Vann, Julie C Jacobson

2017-02-28

In 2008, the National Heart, Lung, and Blood Institute convened an Implementation Science Work Group to assess evidence-based strategies for effectively implementing clinical practice guidelines. This was part of a larger effort to update existing clinical practice guidelines on cholesterol, blood pressure, and overweight/obesity. Review evidence from the published implementation science literature and identify effective or promising strategies to enhance the adoption and implementation of clinical practice guidelines. This systematic review was conducted on 4 critical questions, each focusing on the adoption and effectiveness of 4 intervention strategies: (1) reminders, (2) educational outreach visits, (3) audit and feedback, and (4) provider incentives. A scoping review of the Rx for Change database of systematic reviews was used to identify promising guideline implementation interventions aimed at providers. Inclusion and exclusion criteria were developed a priori for each question, and the published literature was initially searched up to 2012, and then updated with a supplemental search to 2015. Two independent reviewers screened the returned citations to identify relevant reviews and rated the quality of each included review. Audit and feedback and educational outreach visits were generally effective in improving both process of care (15 of 21 reviews and 12 of 13 reviews, respectively) and clinical outcomes (7 of 12 reviews and 3 of 5 reviews, respectively). Provider incentives showed mixed effectiveness for improving both process of care (3 of 4 reviews) and clinical outcomes (3 reviews equally distributed between generally effective, mixed, and generally ineffective). Reminders showed mixed effectiveness for improving process of care outcomes (27 reviews with 11 mixed and 3 generally ineffective results) and were generally ineffective for clinical outcomes (18 reviews with 6 mixed and 9 generally ineffective results). Educational outreach visits (2 of 2 reviews), reminders (3 of 4 reviews), and provider incentives (1 of 1 review) were generally effective for cost reduction. Educational outreach visits (1 of 1 review) and provider incentives (1 of 1 review) were also generally effective for cost-effectiveness outcomes. Barriers to clinician adoption or adherence to guidelines included time constraints (8 reviews/overviews); limited staffing resources (2 overviews); timing (5 reviews/overviews); clinician skepticism (5 reviews/overviews); clinician knowledge of guidelines (4 reviews/overviews); and higher age of the clinician (1 overview). Facilitating factors included guideline characteristics such as format, resources, and end-user involvement (6 reviews/overviews); involving stakeholders (5 reviews/overviews); leadership support (5 reviews/overviews); scope of implementation (5 reviews/overviews); organizational culture such as multidisciplinary teams and low-baseline adherence (9 reviews/overviews); and electronic guidelines systems (3 reviews). The strategies of audit and feedback and educational outreach visits were generally effective in improving both process of care and clinical outcomes. Reminders and provider incentives showed mixed effectiveness, or were generally ineffective. No general conclusion could be reached about cost effectiveness, because of limitations in the evidence. Important gaps exist in the evidence on effectiveness of implementation interventions, especially regarding clinical outcomes, cost effectiveness and contextual issues affecting successful implementation. © 2017 by the American College of Cardiology Foundation and the American Heart Association, Inc.

A systematic scoping review of the evidence for consumer involvement in organisations undertaking systematic reviews: focus on Cochrane.

PubMed

Morley, Richard F; Norman, Gill; Golder, Su; Griffith, Polly

2016-01-01

Cochrane is the largest international producer of systematic reviews of clinical trial evidence. We looked for published evidence that reports where consumers (patients and the public) have been involved in Cochrane systematic reviews, and also in reviews published by other organisations.We found 36 studies that reported about consumer involvement either in individual systematic reviews, or in other organisations. The studies showed that consumers were involved in reviews in a range of different ways: coordinating and producing reviews, making reviews more accessible, and spreading the results of reviews ("knowledge transfer"). The most common role was commenting on reviews ("peer reviewing"). Consumers also had other general roles, for example in educating people about evidence or helping other consumers. There were some interesting examples of new ways of involving consumers. The studies showed that most consumers came from rich and English speaking countries. There was little evidence about how consumer involvement had changed the reviews ("impact"). The studies found that consumer involvement needed to be properly supported.In future we believe that more research should be done to understand what kind of consumer involvement has the best impact; that more review authors should report how consumers have been involved; and that consumers who help with reviews should come from more varied backgrounds. Background Cochrane is the largest international producer of systematic reviews, and is committed to consumer involvement in the production and dissemination of its reviews. The review aims to systematically scope the evidence base for consumer involvement in organisations which commission, undertake or support systematic reviews; with an emphasis on Cochrane. Methods In June 2015 we searched six databases and other sources for studies of consumer involvement in organisations which commission, undertake or support systematic reviews, or in individual systematic review processes. All types of reports and evaluations were eligible. Included studies were combined in a narrative synthesis structured by the level of evaluation and the type of involvement. Results We identified 36 relevant studies. Eleven of these were evaluations at the level of a whole organisation; seven of these studied consumer involvement in Cochrane. Ten studies examined individual Cochrane review groups. Twelve studies reported on individual reviews; only two of these were Cochrane reviews. Finally, three studies were themselves syntheses of other studies. The included studies reported varying levels of consumer involvement across a wide range of activities related to review design and conduct. These included activities such as priority setting and outcome definition as well as review-specific roles such as acting as peer referees and producing plain language summaries. The level of satisfaction and awareness of impact was generally higher in studies focused on individual Cochrane review groups than in the organisation-wide studies. Conclusions There was evidence of highly variable levels and types of consumer involvement within and beyond Cochrane, but limited evidence for what makes the most effective methods and levels of involving consumers in review production. Where evidence of impact was found at the level of individual reviews and review groups it underlined the need for properly resourced and supported processes in order to derive the greatest benefit from the lived experiences of consumers who are willing to be involved. Where reviews do involve consumers, their contribution to the final result could be more clearly identified. More rigorous evaluations are needed to determine the best approach to achieving this. Trial registration Not applicable.

36 CFR 218.6 - Reviewing officer.

Code of Federal Regulations, 2010 CFR

2010-07-01

... ADMINISTRATIVE REVIEW PROCESSES Predecisional Administrative Review Process for Hazardous Fuel Reduction Projects... the administrative review processes of other Federal agencies, for authorized hazardous fuel reduction... administrative review. ...

Engineering Review Information System

NASA Technical Reports Server (NTRS)

Grems, III, Edward G. (Inventor); Henze, James E. (Inventor); Bixby, Jonathan A. (Inventor); Roberts, Mark (Inventor); Mann, Thomas (Inventor)

2015-01-01

A disciplinal engineering review computer information system and method by defining a database of disciplinal engineering review process entities for an enterprise engineering program, opening a computer supported engineering item based upon the defined disciplinal engineering review process entities, managing a review of the opened engineering item according to the defined disciplinal engineering review process entities, and closing the opened engineering item according to the opened engineering item review.

Considering methodological options for reviews of theory: illustrated by a review of theories linking income and health.

PubMed

Campbell, Mhairi; Egan, Matt; Lorenc, Theo; Bond, Lyndal; Popham, Frank; Fenton, Candida; Benzeval, Michaela

2014-10-13

Review of theory is an area of growing methodological advancement. Theoretical reviews are particularly useful where the literature is complex, multi-discipline, or contested. It has been suggested that adopting methods from systematic reviews may help address these challenges. However, the methodological approaches to reviews of theory, including the degree to which systematic review methods can be incorporated, have received little discussion in the literature. We recently employed systematic review methods in a review of theories about the causal relationship between income and health. This article discusses some of the methodological issues we considered in developing the review and offers lessons learnt from our experiences. It examines the stages of a systematic review in relation to how they could be adapted for a review of theory. The issues arising and the approaches taken in the review of theories in income and health are considered, drawing on the approaches of other reviews of theory. Different approaches to searching were required, including electronic and manual searches, and electronic citation tracking to follow the development of theories. Determining inclusion criteria was an iterative process to ensure that inclusion criteria were specific enough to make the review practical and focused, but not so narrow that key literature was excluded. Involving subject specialists was valuable in the literature searches to ensure principal papers were identified and during the inductive approaches used in synthesis of theories to provide detailed understanding of how theories related to another. Reviews of theory are likely to involve iterations and inductive processes throughout, and some of the concepts and techniques that have been developed for qualitative evidence synthesis can be usefully translated to theoretical reviews of this kind. It may be useful at the outset of a review of theory to consider whether the key aim of the review is to scope out theories relating to a particular issue; to conduct in-depth analysis of key theoretical works with the aim of developing new, overarching theories and interpretations; or to combine both these processes in the review. This can help decide the most appropriate methodological approach to take at particular stages of the review.

Effectiveness of Pilates exercise in treating people with chronic low back pain: a systematic review of systematic reviews

PubMed Central

2013-01-01

Background Systematic reviews provide clinical practice recommendations that are based on evaluation of primary evidence. When systematic reviews with the same aims have different conclusions, it is difficult to ascertain which review reported the most credible and robust findings. Methods This study examined five systematic reviews that have investigated the effectiveness of Pilates exercise in people with chronic low back pain. A four-stage process was used to interpret findings of the reviews. This process included comparison of research questions, included primary studies, and the level and quality of evidence of systematic reviews. Two independent reviewers assessed the level of evidence and the methodological quality of systematic reviews, using the National Health and Medical Research Council hierarchy of evidence, and the Revised Assessment of Multiple Systematic Reviews respectively. Any disagreements were resolved by a third researcher. Results A high level of consensus was achieved between the reviewers. Conflicting findings were reported by the five systematic reviews regarding the effectiveness of Pilates in reducing pain and disability in people with chronic low back pain. Authors of the systematic reviews included primary studies that did not match their questions in relation to treatment or population characteristics. A total of ten primary studies were identified across five systematic reviews. Only two of the primary studies were included in all of the reviews due to different inclusion criteria relating to publication date and status, definition of Pilates, and methodological quality. The level of evidence of reviews was low due to the methodological design of the primary studies. The methodological quality of reviews varied. Those which conducted a meta-analysis obtained higher scores. Conclusion There is inconclusive evidence that Pilates is effective in reducing pain and disability in people with chronic low back pain. This is due to the small number and poor methodological quality of primary studies. The Revised Assessment of Multiple Systematic Reviews provides a useful method of appraising the methodological quality of systematic reviews. Individual item scores, however, should be examined in addition to total scores, so that significant methodological flaws of systematic reviews are not missed, and results are interpreted appropriately. (348 words) PMID:23331384

Effectiveness of Pilates exercise in treating people with chronic low back pain: a systematic review of systematic reviews.

PubMed

Wells, Cherie; Kolt, Gregory S; Marshall, Paul; Hill, Bridget; Bialocerkowski, Andrea

2013-01-19

Systematic reviews provide clinical practice recommendations that are based on evaluation of primary evidence. When systematic reviews with the same aims have different conclusions, it is difficult to ascertain which review reported the most credible and robust findings. This study examined five systematic reviews that have investigated the effectiveness of Pilates exercise in people with chronic low back pain. A four-stage process was used to interpret findings of the reviews. This process included comparison of research questions, included primary studies, and the level and quality of evidence of systematic reviews. Two independent reviewers assessed the level of evidence and the methodological quality of systematic reviews, using the National Health and Medical Research Council hierarchy of evidence, and the Revised Assessment of Multiple Systematic Reviews respectively. Any disagreements were resolved by a third researcher. A high level of consensus was achieved between the reviewers. Conflicting findings were reported by the five systematic reviews regarding the effectiveness of Pilates in reducing pain and disability in people with chronic low back pain. Authors of the systematic reviews included primary studies that did not match their questions in relation to treatment or population characteristics. A total of ten primary studies were identified across five systematic reviews. Only two of the primary studies were included in all of the reviews due to different inclusion criteria relating to publication date and status, definition of Pilates, and methodological quality. The level of evidence of reviews was low due to the methodological design of the primary studies. The methodological quality of reviews varied. Those which conducted a meta-analysis obtained higher scores. There is inconclusive evidence that Pilates is effective in reducing pain and disability in people with chronic low back pain. This is due to the small number and poor methodological quality of primary studies. The Revised Assessment of Multiple Systematic Reviews provides a useful method of appraising the methodological quality of systematic reviews. Individual item scores, however, should be examined in addition to total scores, so that significant methodological flaws of systematic reviews are not missed, and results are interpreted appropriately. (348 words).

The methodological and reporting quality of systematic reviews from China and the USA are similar.

PubMed

Tian, Jinhui; Zhang, Jun; Ge, Long; Yang, Kehu; Song, Fujian

2017-05-01

To compare the methodological and reporting quality of systematic reviews by authors from China and those from the United States (USA). From systematic reviews of randomized trials published in 2014 in English, we randomly selected 100 from China and 100 from the USA. The methodological quality was assessed using the Assessing the Methodological Quality of Systematic Reviews (AMSTAR) tool, and reporting quality assessed using the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) tool. Compared with systematic reviews from the USA, those from China were more likely to be a meta-analysis, published in low-impact journals, and a non-Cochrane review. The mean summary Assessing the Methodological Quality of Systematic Reviews score was 6.7 (95% confidence interval: 6.5, 7.0) for reviews from China and 6.6 (6.1, 7.1) for reviews from the USA, and the mean summary Preferred Reporting Items for Systematic Reviews and Meta-analyses score was 21.2 (20.7, 21.6) for reviews from China and 20.6 (19.9, 21.3) for reviews from the USA. The differences in summary quality scores between China and the USA were statistically nonsignificant after adjusting for multiple review factors. The overall methodological and reporting quality of systematic reviews by authors from China are similar to those from the USA, although the quality of systematic reviews from both countries could be further improved. Copyright © 2017 The Author(s). Published by Elsevier Inc. All rights reserved.

Alcohol and drug treatment outcome studies: new methodological review (2005-2010) and comparison with past reviews.

PubMed

Robinson, Sean M; Sobell, Linda Carter; Sobell, Mark B; Arcidiacono, Steven; Tzall, David

2014-01-01

Several methodological reviews of alcohol treatment outcome studies and one review of drug studies have been published over the past 40 years. Although past reviews demonstrated methodological improvements in alcohol studies, they also found continued deficiencies. The current review allows for an updated evaluation of the methodological rigor of alcohol and drug studies and, by utilizing inclusion criteria similar to previous reviews, it allows for a comparative review over time. In addition, this is the first review that compares the methodology of alcohol and drug treatment outcome studies published during the same time period. The methodology for 25 alcohol and 11 drug treatment outcome studies published from 2005 through 2010 that met the review's inclusion criteria was evaluated. The majority of variables evaluated were used in prior reviews. The current review found that more alcohol and drug treatment outcome studies are now using continuous substance use measures and assessing problem severity. Although there have been methodological improvements over time, the current reviews differed little from their most recent past counterpart. Despite this finding, some areas, particularly the continued low reporting of demographic data, needs strengthening. Improvement in the methodological rigor of alcohol and drug treatment outcome studies has occurred over time. The current review found few differences between alcohol and drug study methodologies as well as few differences between the current review and the most recent past alcohol and drug reviews. © 2013 Elsevier Ltd. All rights reserved.

Teleconference versus face-to-face scientific peer review of grant application: effects on review outcomes.

PubMed

Gallo, Stephen A; Carpenter, Afton S; Glisson, Scott R

2013-01-01

Teleconferencing as a setting for scientific peer review is an attractive option for funding agencies, given the substantial environmental and cost savings. Despite this, there is a paucity of published data validating teleconference-based peer review compared to the face-to-face process. Our aim was to conduct a retrospective analysis of scientific peer review data to investigate whether review setting has an effect on review process and outcome measures. We analyzed reviewer scoring data from a research program that had recently modified the review setting from face-to-face to a teleconference format with minimal changes to the overall review procedures. This analysis included approximately 1600 applications over a 4-year period: two years of face-to-face panel meetings compared to two years of teleconference meetings. The average overall scientific merit scores, score distribution, standard deviations and reviewer inter-rater reliability statistics were measured, as well as reviewer demographics and length of time discussing applications. The data indicate that few differences are evident between face-to-face and teleconference settings with regard to average overall scientific merit score, scoring distribution, standard deviation, reviewer demographics or inter-rater reliability. However, some difference was found in the discussion time. These findings suggest that most review outcome measures are unaffected by review setting, which would support the trend of using teleconference reviews rather than face-to-face meetings. However, further studies are needed to assess any correlations among discussion time, application funding and the productivity of funded research projects.

Current Status of Biomedical Book Reviewing: Part III. Duplication Patterns in Biomedical Book Reviewing

PubMed Central

Chen, Ching-chih

1974-01-01

This is the third part of a comprehensive, quantitative study of biomedical book reviewing. The data base of the total project was built from statistics of 3,347 reviews of 2,067 biomedical books appearing in all 1970 issues of fifty-four reviewing journals. This part of the study explores the duplication patterns in book reviewing among these media. It is found that 35.17% (727 books) of the 2,067 titles were reviewed more than once in 1970, these titles accounting for 2,007 of the total of 3,347 reviews. For the most part, reviews of the most frequently reviewed titles appeared in such journals as British Medical Journal, Annals of Internal Medicine, Lancet, Journal of the American Medical Association, and New England Journal of Medicine. These five journals covered 93.53% of the 727 books reviewed more than once in 1970. PMID:4471577

IRIS Toxicological Review of Libby Amphibole Asbestos ...

EPA Pesticide Factsheets

On August 25, 2011, the draft Toxicological Review of Libby Amphibole Asbestos and the charge to external peer reviewers were released for external peer review and public comment. The Toxicological Review and charge were reviewed internally by EPA and by other federal agencies and White House Offices before public release. In the new IRIS process (May 2009), introduced by the EPA Administrator, all written comments on IRIS assessments submitted by other federal agencies and White House Offices will be made publicly available. Accordingly, interagency comments and the interagency science consultation draft of the IRIS Toxicological Review of Libby Amphibole Asbestos and the charge to external peer reviewers are posted on this site. The project supports OSWER and Region 8 site clean-up and related risk management initiatives at this Superfund site. As with all IRIS assessments, the draft document will undergo agency review, OMB/interagency review and independent external peer review before being included on the IRIS data base.

77 FR 8266 - Center for Scientific Review; Notice of Closed Meetings

Federal Register 2010, 2011, 2012, 2013, 2014

2012-02-14

..., (Telephone Conference Call). Contact Person: Michael M. Sveda, Ph.D., Scientific Review Officer, Center for..., MD, Ph.D., Scientific Review Officer, Center for Scientific Review, National Institutes of Health...). Contact Person: Richard Panniers, Ph.D., Scientific Review Officer, Center for Scientific Review, National...

42 CFR 423.1976 - Judicial review.

Code of Federal Regulations, 2010 CFR

2010-10-01

... 42 Public Health 3 2010-10-01 2010-10-01 false Judicial review. 423.1976 Section 423.1976 Public...) MEDICARE PROGRAM VOLUNTARY MEDICARE PRESCRIPTION DRUG BENEFIT Reopening, ALJ Hearings, MAC review, and Judicial Review § 423.1976 Judicial review. (a) Review of ALJ's decision. The enrollee may request judicial...

36 CFR 251.100 - Discretionary review.

Code of Federal Regulations, 2010 CFR

2010-07-01

... 36 Parks, Forests, and Public Property 2 2010-07-01 2010-07-01 false Discretionary review. 251.100... review. (a) Petitions or requests for discretionary review shall not, in and of themselves, give rise to a decision to exercise discretionary review. In electing to exercise discretion, a Reviewing Officer...

IRIS Toxicological Review of Dichloromethane (Methylene Chloride) (Interagency Science Consultation Draft)

EPA Science Inventory

On March 31, 2010, the draft IRIS Toxicological Review of Dichloromethane (Methylene Chloride) external review draft document and the charge to external peer reviewers were released for public review and comment. The draft document and the charge to external peer reviewers were r...

IRIS TOXICOLOGICAL REVIEW OF HEXAVALENT CHROMIUM (INTERAGENCY SCIENCE CONSULTATION DRAFT)

EPA Science Inventory

On Septemeber 30, 2010, the draft Toxicological Review of Hexavalent Chromium and the charge to external peer reviewers were released for external peer review and public comment. The Toxicological Review and charge were reviewed internally by EPA and by other federal agenc...

IRIS Toxicological Review of Libby Amphibole Asbestos (Interagency Science Consultation Draft)

EPA Science Inventory

On August 25, 2011, the draft Toxicological Review of Libby Amphibole Asbestos and the charge to external peer reviewers were released for external peer review and public comment. The Toxicological Review and charge were reviewed internally by EPA and by other federal agencies an...

IRIS Toxicological Review of Urea (Interagency Science Consultation Draft)

EPA Science Inventory

On September 28, 2010, the Toxicological Review of Urea and the charge to external peer reviewers were released for external peer review and public comment. The Toxicological Review and charge were reviewed internally by EPA and by other federal agencies and White House Of...

28 CFR 34.108 - Selection of reviewers.

Code of Federal Regulations, 2010 CFR

2010-07-01

....108 Judicial Administration DEPARTMENT OF JUSTICE OJJDP COMPETITION AND PEER REVIEW PROCEDURES Peer Review § 34.108 Selection of reviewers. The Program Manager, through the Director of the OJJDP program... by the Administrator. The selection process for peer reviewers is detailed in the OJJDP “Peer Review...

IRIS Toxicological Review of Ammonia (Interagency Science Consultation Draft)

EPA Science Inventory

On June 1, 2012, the draft Toxicological Review of Ammonia and the draft charge to external peer reviewers were released for external peer review and public comment. The Toxicological Review and charge were reviewed internally by EPA and by other federal agencies and White House ...

IRIS Toxicological Review of 1,4-Dioxane (Interagency Science Consultation Draft)

EPA Science Inventory

On September 9, 2011, the Toxicological Review of 1,4-Dioxane (with inhalation update) (External Review Draft) was posted for external peer review and public comment. The Toxicological Review and charge were reviewed internally by EPA and by other federal agencies and White House...

75 FR 73117 - New Agency Information Collection Activity Under OMB Review: Pipeline Corporate Security Review

Federal Register 2010, 2011, 2012, 2013, 2014

2010-11-29

... Collection Activity Under OMB Review: Pipeline Corporate Security Review AGENCY: Transportation Security.... Information Collection Requirement Title: Pipeline Corporate Security Review (PCSR). Type of Request: New collection. OMB Control Number: Not yet assigned. Form(s): Pipeline Corporate Security Review (PCSR) Protocol...

32 CFR 724.303 - Functions: President, Naval Discharge Review Board.

Code of Federal Regulations, 2010 CFR

2010-07-01

... 32 National Defense 5 2010-07-01 2010-07-01 false Functions: President, Naval Discharge Review... PERSONNEL NAVAL DISCHARGE REVIEW BOARD Director, Secretary of the Navy Council of Review Boards and President Naval Discharge Review Board; Responsibilities in Support of the Naval Discharge Review Board...

32 CFR 724.303 - Functions: President, Naval Discharge Review Board.

Code of Federal Regulations, 2011 CFR

2011-07-01

... 32 National Defense 5 2011-07-01 2011-07-01 false Functions: President, Naval Discharge Review... PERSONNEL NAVAL DISCHARGE REVIEW BOARD Director, Secretary of the Navy Council of Review Boards and President Naval Discharge Review Board; Responsibilities in Support of the Naval Discharge Review Board...

47 CFR 54.723 - Standard of review.

Code of Federal Regulations, 2011 CFR

2011-10-01

... Competition Bureau shall conduct de novo review of request for review of decisions issue by the Administrator. (b) The Federal Communications Commission shall conduct de novo review of requests for review of... the Commission shall not conduct de novo review of decisions issued by the Wireline Competition Bureau...

47 CFR 54.723 - Standard of review.

Code of Federal Regulations, 2010 CFR

2010-10-01

... Competition Bureau shall conduct de novo review of request for review of decisions issue by the Administrator. (b) The Federal Communications Commission shall conduct de novo review of requests for review of... the Commission shall not conduct de novo review of decisions issued by the Wireline Competition Bureau...

7 CFR 275.8 - Review coverage.

Code of Federal Regulations, 2014 CFR

2014-01-01

... 7 Agriculture 4 2014-01-01 2014-01-01 false Review coverage. 275.8 Section 275.8 Agriculture Regulations of the Department of Agriculture (Continued) FOOD AND NUTRITION SERVICE, DEPARTMENT OF AGRICULTURE...) Reviews § 275.8 Review coverage. (a) During each review period, State agencies shall review the national...

7 CFR 275.8 - Review coverage.

Code of Federal Regulations, 2012 CFR

2012-01-01

... 7 Agriculture 4 2012-01-01 2012-01-01 false Review coverage. 275.8 Section 275.8 Agriculture Regulations of the Department of Agriculture (Continued) FOOD AND NUTRITION SERVICE, DEPARTMENT OF AGRICULTURE...) Reviews § 275.8 Review coverage. (a) During each review period, State agencies shall review the national...

7 CFR 275.8 - Review coverage.

Code of Federal Regulations, 2011 CFR

2011-01-01

... 7 Agriculture 4 2011-01-01 2011-01-01 false Review coverage. 275.8 Section 275.8 Agriculture Regulations of the Department of Agriculture (Continued) FOOD AND NUTRITION SERVICE, DEPARTMENT OF AGRICULTURE...) Reviews § 275.8 Review coverage. (a) During each review period, State agencies shall review the national...

7 CFR 275.8 - Review coverage.

Code of Federal Regulations, 2013 CFR

2013-01-01

... 7 Agriculture 4 2013-01-01 2013-01-01 false Review coverage. 275.8 Section 275.8 Agriculture Regulations of the Department of Agriculture (Continued) FOOD AND NUTRITION SERVICE, DEPARTMENT OF AGRICULTURE...) Reviews § 275.8 Review coverage. (a) During each review period, State agencies shall review the national...

IRIS Toxicological Review of n-Butanol (Interagency Science Consultation Draft)

EPA Science Inventory

On September 8, 2011, the Toxicological Review of n-Butanol (External Review Draft) was released for external peer review and public comment. The Toxicological Review and charge were reviewed internally by EPA and by other federal agencies and White House Offices before public re...

IRIS Toxicological Review of Vanadium Pentoxide (Interagency Science Consultation Draft)

EPA Science Inventory

On September 30, 2011, the draft Toxicological Review of Vanadium Pentoxide and the charge to external peer reviewers were released for external peer review and public comment. The Toxicological Review and charge were reviewed internally by EPA and by other federal agencies and W...

Relationship between Research Outcomes and Risk of Bias, Study Sponsorship, and Author Financial Conflicts of Interest in Reviews of the Effects of Artificially Sweetened Beverages on Weight Outcomes: A Systematic Review of Reviews.

PubMed

Mandrioli, Daniele; Kearns, Cristin E; Bero, Lisa A

2016-01-01

Artificially sweetened beverage consumption has steadily increased in the last 40 years. Several reviews examining the effects of artificially sweetened beverages on weight outcomes have discrepancies in their results and conclusions. To determine whether risk of bias, results, and conclusions of reviews of effects of artificially sweetened beverage consumption on weight outcomes differ depending on review sponsorship and authors' financial conflicts of interest. We performed a systematic review of reviews of the effects of artificially sweetened beverages on weight. Two assessors independently screened articles for inclusion, extracted data, and assessed risks of bias. We compared risk of bias, results and conclusions of reviews by different industry sponsors, authors' financial conflict of interest and journal sponsor. We also report the concordance between review results and conclusions. Artificial sweetener industry sponsored reviews were more likely to have favorable results (3/4) than non-industry sponsored reviews (1/23), RR: 17.25 (95% CI: 2.34 to 127.29), as well as favorable conclusions (4/4 vs. 15/23), RR: 1.52 (95% CI: 1.14 to 2.06). All reviews funded by competitor industries reported unfavorable conclusions (4/4). In 42% of the reviews (13/31), authors' financial conflicts of interest were not disclosed. Reviews performed by authors that had a financial conflict of interest with the food industry (disclosed in the article or not) were more likely to have favorable conclusions (18/22) than reviews performed by authors without conflicts of interest (4/9), RR: 7.36 (95% CI: 1.15 to 47.22). Risk of bias was similar and high in most of the reviews. Review sponsorship and authors' financial conflicts of interest introduced bias affecting the outcomes of reviews of artificially sweetened beverage effects on weight that could not be explained by other sources of bias.

Nature and reporting characteristics of UK health technology assessment systematic reviews.

PubMed

Carroll, Christopher; Kaltenthaler, Eva

2018-05-08

A recent study by Page et al. (PLoS Med. 2016;13(5):e1002028) claimed that increasing numbers of reviews are being published and many are poorly-conducted and reported. The aim of the present study was to assess how well reporting standards of systematic reviews produced in a Health Technology Assessment (HTA) context compare with reporting in Cochrane and other 'non-Cochrane' systematic reviews from the same years (2004 and 2014), as reported by Page et al. (PLoS Med. 2016;13(5):e1002028). All relevant UK HTA programme systematic reviews published in 2004 and 2014 were identified. After piloting of the form, two reviewers each extracted relevant data on conduct and reporting from these reviews. These data were compared with data for Cochrane and "non-Cochrane" systematic reviews, as published by Page et al. (PLoS Med. 2016;13(5):e1002028). All data were tabulated and summarized. There were 30 UK HTA programme systematic reviews and 300 other systematic reviews, including Cochrane reviews (n = 45). The percentage of HTA reviews with required elements of conduct and reporting was frequently very similar to Cochrane and much higher than all other systematic reviews, e.g. availability of protocols (90, 98 and 16% respectively); the specification of study design criteria (100, 100, 79%); the reporting of outcomes (100, 100, 78%), quality assessment (100, 100, 70%); the searching of trial registries for unpublished data (70, 62, 19%); reporting of reasons for excluding studies (91, 91 and 70%) and reporting of authors' conflicts of interests (100, 100, 87%). HTA reviews only compared less favourably with Cochrane and other reviews in assessments of publication bias. UK HTA systematic reviews are often produced within a specific policy-making context. This context has implications for timelines, tools and resources. However, UK HTA systematic reviews still tend to present standards of conduct and reporting equivalent to "gold standard" Cochrane reviews and superior to systematic reviews more generally.

Variability of Reviewers' Comments in the Peer Review Process for Orthopaedic Research.

PubMed

Iantorno, Stephanie E; Andras, Lindsay M; Skaggs, David L

2016-07-01

Retrospective analysis of peer review comments. To assess the likelihood that comments provided by peer reviewers of one orthopaedic journal would be similar to comments of reviewers from the same journal and a second journal. The consistency of the peer review process in orthopedic research has not been objectively examined. Nine separate clinical papers related to spinal deformity were submitted for publication in major peer-reviewed journals and initially rejected. The exact same manuscripts were then submitted to different journals. All papers were returned with comments from two to three reviewers from each journal. Reviews were divided into distinct conceptual criticisms that were regarded as separate comments. Comments were compared between reviewers of the same journal and to comments from reviewers of the second journal. When comparing comments from reviewers of the same journal, an average of 11% of comments were repeated (range 0% [0/12] to 23% [3/13]). On average, 20% of comments from the first journal were repeated by a reviewer at the second journal (range 10% [1/10] to 33% [6/18]). If a comment was made by two or more reviewers from the first journal, it had a higher likelihood (43% [6/14]) of being repeated by a reviewer from the second journal. When an identical manuscript is submitted to a second journal after being rejected, 80% of peer review comments from the first journal are not repeated by reviewers from the second journal. One may question if addressing every peer review comment in a rejected manuscript prior to resubmission is an efficient use of resources. Comments that appear twice or more in the first journal review are more likely to reappear and may warrant special attention from the researcher. Level IV. Copyright © 2016 Scoliosis Research Society. Published by Elsevier Inc. All rights reserved.

Relationship between Research Outcomes and Risk of Bias, Study Sponsorship, and Author Financial Conflicts of Interest in Reviews of the Effects of Artificially Sweetened Beverages on Weight Outcomes: A Systematic Review of Reviews

PubMed Central

Kearns, Cristin E; Bero, Lisa A.

2016-01-01

Background Artificially sweetened beverage consumption has steadily increased in the last 40 years. Several reviews examining the effects of artificially sweetened beverages on weight outcomes have discrepancies in their results and conclusions. Objectives To determine whether risk of bias, results, and conclusions of reviews of effects of artificially sweetened beverage consumption on weight outcomes differ depending on review sponsorship and authors’ financial conflicts of interest. Methods We performed a systematic review of reviews of the effects of artificially sweetened beverages on weight. Two assessors independently screened articles for inclusion, extracted data, and assessed risks of bias. We compared risk of bias, results and conclusions of reviews by different industry sponsors, authors’ financial conflict of interest and journal sponsor. We also report the concordance between review results and conclusions. Results Artificial sweetener industry sponsored reviews were more likely to have favorable results (3/4) than non-industry sponsored reviews (1/23), RR: 17.25 (95% CI: 2.34 to 127.29), as well as favorable conclusions (4/4 vs. 15/23), RR: 1.52 (95% CI: 1.14 to 2.06). All reviews funded by competitor industries reported unfavorable conclusions (4/4). In 42% of the reviews (13/31), authors’ financial conflicts of interest were not disclosed. Reviews performed by authors that had a financial conflict of interest with the food industry (disclosed in the article or not) were more likely to have favorable conclusions (18/22) than reviews performed by authors without conflicts of interest (4/9), RR: 7.36 (95% CI: 1.15 to 47.22). Risk of bias was similar and high in most of the reviews. Conclusions Review sponsorship and authors’ financial conflicts of interest introduced bias affecting the outcomes of reviews of artificially sweetened beverage effects on weight that could not be explained by other sources of bias. PMID:27606602

The Relationship of Previous Training and Experience of Journal Peer Reviewers to Subsequent Review Quality

PubMed Central

Callaham, Michael L; Tercier, John

2007-01-01

Background Peer review is considered crucial to the selection and publication of quality science, but very little is known about the previous experiences and training that might identify high-quality peer reviewers. The reviewer selection processes of most journals, and thus the qualifications of their reviewers, are ill defined. More objective selection of peer reviewers might improve the journal peer review process and thus the quality of published science. Methods and Findings 306 experienced reviewers (71% of all those associated with a specialty journal) completed a survey of past training and experiences postulated to improve peer review skills. Reviewers performed 2,856 reviews of 1,484 separate manuscripts during a four-year study period, all prospectively rated on a standardized quality scale by editors. Multivariable analysis revealed that most variables, including academic rank, formal training in critical appraisal or statistics, or status as principal investigator of a grant, failed to predict performance of higher-quality reviews. The only significant predictors of quality were working in a university-operated hospital versus other teaching environment and relative youth (under ten years of experience after finishing training). Being on an editorial board and doing formal grant (study section) review were each predictors for only one of our two comparisons. However, the predictive power of all variables was weak. Conclusions Our study confirms that there are no easily identifiable types of formal training or experience that predict reviewer performance. Skill in scientific peer review may be as ill defined and hard to impart as is “common sense.” Without a better understanding of those skills, it seems unlikely journals and editors will be successful in systematically improving their selection of reviewers. This inability to predict performance makes it imperative that all but the smallest journals implement routine review ratings systems to routinely monitor the quality of their reviews (and thus the quality of the science they publish). PMID:17411314

Evidence from the Cochrane Collaboration for Traditional Chinese Medicine Therapies

PubMed Central

Wieland, Susan; Kimbrough, Elizabeth; Cheng, Ker; Berman, Brian M.

2009-01-01

Abstract Background The Cochrane Collaboration, an international not-for-profit organization that prepares and maintains systematic reviews of randomized trials of health care therapies, has produced reviews summarizing much of the evidence on Traditional Chinese Medicine (TCM). Our objective was to review the evidence base according to Cochrane systematic reviews. Methods In order to detect reviews focusing on TCM, we searched the titles and abstracts of all reviews in Issue 4, 2008 of the Cochrane Database of Systematic Reviews. For each review, we extracted data on the number of trials included and the total number of participants. We provided an indication of the strength of the review findings by assessing the reviewers' abstract conclusions statement. We supplemented our assessment of the abstract conclusions statements with a listing of the comparisons and outcomes showing statistically significant meta-analyses results. Results We identified 70 Cochrane systematic reviews of TCM, primarily acupuncture (n = 26) and Chinese herbal medicine (n = 42), and 1 each of moxibustion and t'ai chi. Nineteen (19) of 26 acupuncture reviews and 22/42 herbal medicine reviews concluded that there was not enough good quality trial evidence to make any conclusion about the efficacy of the evaluated treatment, while the remaining 7 acupuncture and 20 herbal medicine reviews and each of the moxibustion and t'ai chi reviews indicated a suggestion of benefit, which was qualified by a caveat about the poor quality and quantity of studies. Most reviews included many distinct interventions, controls, outcomes, and populations, and a large number of different comparisons were made, each with a distinct forest plot. Conclusions Most Cochrane systematic reviews of TCM are inconclusive, due specifically to the poor methodology and heterogeneity of the studies reviewed. Some systematic reviews provide preliminary evidence of Chinese medicine's benefits to certain patient populations, underscoring the importance and appropriateness of further research. These preliminary findings should be considered tentative and need to be confirmed with rigorous randomized controlled trials. PMID:19757977

Use of simulation-based learning in undergraduate nurse education: An umbrella systematic review.

PubMed

Cant, Robyn P; Cooper, Simon J

2017-02-01

To conduct a systematic review to appraise and review evidence on the impact of simulation-based education for undergraduate/pre-licensure nursing students, using existing reviews of literature. An umbrella review (review of reviews). Cumulative Index of Nursing and Allied Health Literature (CINAHLPlus), PubMed, and Google Scholar. Reviews of literature conducted between 2010 and 2015 regarding simulation-based education for pre-licensure nursing students. The Joanna Briggs Institute methodology for conduct of an umbrella review was used to inform the review process. Twenty-five systematic reviews of literature were included, of which 14 were recent (2013-2015). Most described the level of evidence of component studies as a mix of experimental and quasi-experimental designs. The reviews measured around 14 different main outcome variables, thus limiting the number of primary studies that each individual review could pool to appraise. Many reviews agreed on the key learning outcome of knowledge acquisition, although no overall quantitative effect was derived. Three of four high-quality reviews found that simulation supported psychomotor development; a fourth found too few high quality studies to make a statistical comparison. Simulation statistically improved self-efficacy in pretest-posttest studies, and in experimental designs self-efficacy was superior to that of other teaching methods; lower level research designs limiting further comparison. The reviews commonly reported strong student satisfaction with simulation education and some reported improved confidence and/or critical thinking. This umbrella review took a global view of 25 reviews of simulation research in nursing education, comprising over 700 primary studies. To discern overall outcomes across reviews, statistical comparison of quantitative results (effect size) must be the key comparator. Simulation-based education contributes to students' learning in a number of ways when integrated into pre-licensure nursing curricula. Overall, use of a constellation of instruments and a lack of high quality study designs mean that there are still some gaps in evidence of effects that need to be addressed. Copyright © 2016 Elsevier Ltd. All rights reserved.

40 CFR 791.60 - Review.

Code of Federal Regulations, 2010 CFR

2010-07-01

... 40 Protection of Environment 31 2010-07-01 2010-07-01 true Review. 791.60 Section 791.60...) DATA REIMBURSEMENT Review § 791.60 Review. (a) The hearing officer's proposed order shall become the... Agency review or the Administrator of his own initiative decides to review the proposed order. (b) The...

Performing Systematic Literature Reviews with Novices: An Iterative Approach

ERIC Educational Resources Information Center

Lavallée, Mathieu; Robillard, Pierre-N.; Mirsalari, Reza

2014-01-01

Reviewers performing systematic literature reviews require understanding of the review process and of the knowledge domain. This paper presents an iterative approach for conducting systematic literature reviews that addresses the problems faced by reviewers who are novices in one or both levels of understanding. This approach is derived from…

IRIS Toxicological Review of cis- & trans-1,2-Dichloroethylene (Interagency Science Consultation Draft)

EPA Science Inventory

On September 24, 2009, the Toxicological Review of cis- and trans-1,2-dichloroethylene and the charge to external peer reviewers were released for external peer review and public comment. The Toxicological Review and charge were reviewed internally by EPA and by other federal ag...

76 FR 76403 - Agency Information Collection Activities; Submission to OMB for Review and Approval; Comment...

Federal Register 2010, 2011, 2012, 2013, 2014

2011-12-07

... Activities; Submission to OMB for Review and Approval; Comment Request; Pre-Manufacture Review Reporting and... review and approval: Pre-Manufacture Review Reporting and Exemption Requirements for New Chemical....regulations.gov . Title: Pre-Manufacture Review Reporting and Exemption Requirements for New Chemical...

5 CFR 1203.12 - Granting or denying the request for regulation review.

Code of Federal Regulations, 2010 CFR

2010-01-01

... ORGANIZATION AND PROCEDURES PROCEDURES FOR REVIEW OF RULES AND REGULATIONS OF THE OFFICE OF PERSONNEL MANAGEMENT Procedures for Review § 1203.12 Granting or denying the request for regulation review. (a) The..., review a regulation before its effective date. (b) After considering the request for regulation review...

Division of Legislative Audit

Science.gov Websites

Reviews - Performance Review of the Department of Corrections, 2014 - Performance Review of the Department of Health and Social Services, 2015 - A Performance Review of the Department of Education and Early Statewide Audits Sunset Audits Performance Reviews Performance Review of the Department of Corrections, 2014

28 CFR 34.106 - Number of peer reviewers.

Code of Federal Regulations, 2010 CFR

2010-07-01

... 28 Judicial Administration 1 2010-07-01 2010-07-01 false Number of peer reviewers. 34.106 Section 34.106 Judicial Administration DEPARTMENT OF JUSTICE OJJDP COMPETITION AND PEER REVIEW PROCEDURES Peer Review § 34.106 Number of peer reviewers. The number of peer reviewers will vary by program (as...

42 CFR 412.44 - Medical review requirements: Admissions and quality review.

Code of Federal Regulations, 2014 CFR

2014-10-01

... 42 Public Health 2 2014-10-01 2014-10-01 false Medical review requirements: Admissions and quality review. 412.44 Section 412.44 Public Health CENTERS FOR MEDICARE & MEDICAID SERVICES, DEPARTMENT OF... Capital-Related Costs § 412.44 Medical review requirements: Admissions and quality review. Beginning on...

IRIS Toxicological Review of Benzo[a]pyrene (Interagency Science Consultation Draft)

EPA Science Inventory

On August 21, 2013, the draft Toxicological Review of Benzo[a]pyrene and the draft charge to external peer reviewers were released for public review and comment. The Toxicological Review and charge were reviewed internally by EPA and by other federal agencies and White House Offi...

IRIS Toxicological Review of 1,4-Dioxane (with Inhalation Update) (Interagency Science Discussion Draft)

EPA Science Inventory

On September 9, 2011, the Toxicological Review of 1,4-Dioxane (inhalation) (External Review Draft) was posted for external peer review and public comment. Subsequently, the Toxicological Review and IRIS Summary were reviewed internally by EPA and by other federal agencies and Exe...

Sulfur Dioxide (SO2) Primary NAAQS Review: Integrated Review Plan - Advisory with CASAC

EPA Science Inventory

The SO2 Integrated Review Plan is the first document generated as part of the National Ambient Air Quality Standards (NAAQS) review process. The Plan presents background information, the schedule for the review, the process to be used in conducting the review, and the key policy-...

32 CFR 724.203 - Broad objectives of naval discharge review.

Code of Federal Regulations, 2013 CFR

2013-07-01

... 32 National Defense 5 2013-07-01 2013-07-01 false Broad objectives of naval discharge review. 724... PERSONNEL NAVAL DISCHARGE REVIEW BOARD Authority/Policy for Departmental Discharge Review § 724.203 Broad objectives of naval discharge review. Naval discharge review shall have as its broad objectives: (a) The...

32 CFR 724.203 - Broad objectives of naval discharge review.

Code of Federal Regulations, 2012 CFR

2012-07-01

... 32 National Defense 5 2012-07-01 2012-07-01 false Broad objectives of naval discharge review. 724... PERSONNEL NAVAL DISCHARGE REVIEW BOARD Authority/Policy for Departmental Discharge Review § 724.203 Broad objectives of naval discharge review. Naval discharge review shall have as its broad objectives: (a) The...

32 CFR 724.203 - Broad objectives of naval discharge review.

Code of Federal Regulations, 2014 CFR

2014-07-01

... 32 National Defense 5 2014-07-01 2014-07-01 false Broad objectives of naval discharge review. 724... PERSONNEL NAVAL DISCHARGE REVIEW BOARD Authority/Policy for Departmental Discharge Review § 724.203 Broad objectives of naval discharge review. Naval discharge review shall have as its broad objectives: (a) The...

32 CFR 724.203 - Broad objectives of naval discharge review.

Code of Federal Regulations, 2010 CFR

2010-07-01

... 32 National Defense 5 2010-07-01 2010-07-01 false Broad objectives of naval discharge review. 724... PERSONNEL NAVAL DISCHARGE REVIEW BOARD Authority/Policy for Departmental Discharge Review § 724.203 Broad objectives of naval discharge review. Naval discharge review shall have as its broad objectives: (a) The...

32 CFR 724.203 - Broad objectives of naval discharge review.

Code of Federal Regulations, 2011 CFR

2011-07-01

... 32 National Defense 5 2011-07-01 2011-07-01 false Broad objectives of naval discharge review. 724... PERSONNEL NAVAL DISCHARGE REVIEW BOARD Authority/Policy for Departmental Discharge Review § 724.203 Broad objectives of naval discharge review. Naval discharge review shall have as its broad objectives: (a) The...

77 FR 48755 - Office Patent Trial Practice Guide

Federal Register 2010, 2011, 2012, 2013, 2014

2012-08-14

... Trial and Appeal Board (Board) including inter partes review, post-grant review, the transitional... proceedings. DATES: Effective Date: This practice guide applies to inter partes review, post-grant review, and... proceedings to be conducted by the Board including: (1) Inter partes review (IPR); (2) post-grant review (PGR...

28 CFR 34.106 - Number of peer reviewers.

Code of Federal Regulations, 2013 CFR

2013-07-01

... 28 Judicial Administration 1 2013-07-01 2013-07-01 false Number of peer reviewers. 34.106 Section 34.106 Judicial Administration DEPARTMENT OF JUSTICE OJJDP COMPETITION AND PEER REVIEW PROCEDURES Peer Review § 34.106 Number of peer reviewers. The number of peer reviewers will vary by program (as...

An Evaluation of Four Book Review Journals.

ERIC Educational Resources Information Center

Ream, Daniel

1979-01-01

Evaluates "Booklist,""Choice,""Library Journal," and "the New York Times Book Review" by measuring and comparing (1) total number of adult books reviewed, (2) total number of juvenile and young adult books reviewed, (3) percentage of randomly selected Notable Books reviewed, and (4) how quickly those Notable Books were reviewed. (Author)

7 CFR 3400.14 - Proposal review.

Code of Federal Regulations, 2011 CFR

2011-01-01

... RESEARCH GRANTS PROGRAM Scientific Peer Review of Research Grant Applications § 3400.14 Proposal review. (a... review will be made for responsiveness to the request for proposals (e.g., relationship of application to... peer review group, and ad hoc reviewers, as required. Written comments will be solicited from ad hoc...

20 CFR 405.410 - Selecting claims for Decision Review Board review.

Code of Federal Regulations, 2010 CFR

2010-04-01

... 20 Employees' Benefits 2 2010-04-01 2010-04-01 false Selecting claims for Decision Review Board review. 405.410 Section 405.410 Employees' Benefits SOCIAL SECURITY ADMINISTRATION ADMINISTRATIVE REVIEW... will not review claims based on the identity of the administrative law judge who decided the claim. (b...

45 CFR 91.41 - Compliance reviews.

Code of Federal Regulations, 2011 CFR

2011-10-01

... 45 Public Welfare 1 2011-10-01 2011-10-01 false Compliance reviews. 91.41 Section 91.41 Public..., Conciliation, and Enforcement Procedures § 91.41 Compliance reviews. (a) HHS may conduct compliance reviews and... the Act and these regulations has occurred. (b) If a compliance review or pre-award review indicates a...

34 CFR 110.30 - Compliance reviews.

Code of Federal Regulations, 2010 CFR

2010-07-01

... 34 Education 1 2010-07-01 2010-07-01 false Compliance reviews. 110.30 Section 110.30 Education..., Conciliation, and Enforcement Procedures § 110.30 Compliance reviews. (a) ED may conduct compliance reviews... these regulations occurred. (b) If a compliance review or pre-award review indicates a violation of the...

45 CFR 1156.14 - Compliance reviews.

Code of Federal Regulations, 2010 CFR

2010-10-01

... 45 Public Welfare 3 2010-10-01 2010-10-01 false Compliance reviews. 1156.14 Section 1156.14 Public..., and Enforcement Procedures § 1156.14 Compliance reviews. The Endowment may conduct compliance reviews... recipient. In the event a compliance review or pre-award review indicates a violation of the regulations in...

Rewarding peer reviewers: maintaining the integrity of science communication.

PubMed

Gasparyan, Armen Yuri; Gerasimov, Alexey N; Voronov, Alexander A; Kitas, George D

2015-04-01

This article overviews currently available options for rewarding peer reviewers. Rewards and incentives may help maintain the quality and integrity of scholarly publications. Publishers around the world implemented a variety of financial and nonfinancial mechanisms for incentivizing their best reviewers. None of these is proved effective on its own. A strategy of combined rewards and credits for the reviewers1 creative contributions seems a workable solution. Opening access to reviews and assigning publication credits to the best reviews is one of the latest achievements of digitization. Reviews, posted on academic networking platforms, such as Publons, add to the transparency of the whole system of peer review. Reviewer credits, properly counted and displayed on individual digital profiles, help distinguish the best contributors, invite them to review and offer responsible editorial posts.

Open peer review at four STEM journals: an observational overview.

PubMed

Ford, Emily

2015-01-01

Open peer review, peer review where authors' and reviewers' identities are disclosed to one another, is a growing trend in scholarly publishing. Through observation of four journals in STEM disciplines, PLOS One, Atmospheric Chemistry & Physics, PeerJ, and F1000Research, an observational overview is conducted. The overview relies on defined characteristics of open peer review. Results show that despite differing open peer review implementations, each journal retains editorial involvement in scholarly publishing. Further, the analysis shows that only one of these implementations is fully transparent in its peer review and decision making process. Finally, the overview contends that journals should clearly outline peer review and editorial processes in order to allow for open peer review to be better understood and adopted by authors, reviewers, editors, and readers of science communications.

Rewarding Peer Reviewers: Maintaining the Integrity of Science Communication

PubMed Central

2015-01-01

This article overviews currently available options for rewarding peer reviewers. Rewards and incentives may help maintain the quality and integrity of scholarly publications. Publishers around the world implemented a variety of financial and nonfinancial mechanisms for incentivizing their best reviewers. None of these is proved effective on its own. A strategy of combined rewards and credits for the reviewers1 creative contributions seems a workable solution. Opening access to reviews and assigning publication credits to the best reviews is one of the latest achievements of digitization. Reviews, posted on academic networking platforms, such as Publons, add to the transparency of the whole system of peer review. Reviewer credits, properly counted and displayed on individual digital profiles, help distinguish the best contributors, invite them to review and offer responsible editorial posts. PMID:25829801

How Long Is Too Long in Contemporary Peer Review? Perspectives from Authors Publishing in Conservation Biology Journals

PubMed Central

Nguyen, Vivian M.; Haddaway, Neal R.; Gutowsky, Lee F. G.; Wilson, Alexander D. M.; Gallagher, Austin J.; Donaldson, Michael R.; Hammerschlag, Neil; Cooke, Steven J.

2015-01-01

Delays in peer reviewed publication may have consequences for both assessment of scientific prowess in academia as well as communication of important information to the knowledge receptor community. We present an analysis on the perspectives of authors publishing in conservation biology journals regarding their opinions on the importance of speed in peer-review as well as how to improve review times. Authors were invited to take part in an online questionnaire, of which the data was subjected to both qualitative (open coding, categorizing) and quantitative analyses (generalized linear models). We received 637 responses to 6,547 e-mail invitations sent. Peer-review speed was generally perceived as slow, with authors experiencing a typical turnaround time of 14 weeks while their perceived optimal review time was six weeks. Male and younger respondents seem to have higher expectations of review speed than females and older respondents. The majority of participants attributed lengthy review times to reviewer and editor fatigue, while editor persistence and journal prestige were believed to speed up the review process. Negative consequences of lengthy review times were perceived to be greater for early career researchers and to have impact on author morale (e.g. motivation or frustration). Competition among colleagues was also of concern to respondents. Incentivizing peer-review was among the top suggested alterations to the system along with training graduate students in peer-review, increased editorial persistence, and changes to the norms of peer-review such as opening the peer-review process to the public. It is clear that authors surveyed in this study viewed the peer-review system as under stress and we encourage scientists and publishers to push the envelope for new peer-review models. PMID:26267491

ROBIS: A new tool to assess risk of bias in systematic reviews was developed.

PubMed

Whiting, Penny; Savović, Jelena; Higgins, Julian P T; Caldwell, Deborah M; Reeves, Barnaby C; Shea, Beverley; Davies, Philippa; Kleijnen, Jos; Churchill, Rachel

2016-01-01

To develop ROBIS, a new tool for assessing the risk of bias in systematic reviews (rather than in primary studies). We used four-stage approach to develop ROBIS: define the scope, review the evidence base, hold a face-to-face meeting, and refine the tool through piloting. ROBIS is currently aimed at four broad categories of reviews mainly within health care settings: interventions, diagnosis, prognosis, and etiology. The target audience of ROBIS is primarily guideline developers, authors of overviews of systematic reviews ("reviews of reviews"), and review authors who might want to assess or avoid risk of bias in their reviews. The tool is completed in three phases: (1) assess relevance (optional), (2) identify concerns with the review process, and (3) judge risk of bias. Phase 2 covers four domains through which bias may be introduced into a systematic review: study eligibility criteria; identification and selection of studies; data collection and study appraisal; and synthesis and findings. Phase 3 assesses the overall risk of bias in the interpretation of review findings and whether this considered limitations identified in any of the phase 2 domains. Signaling questions are included to help judge concerns with the review process (phase 2) and the overall risk of bias in the review (phase 3); these questions flag aspects of review design related to the potential for bias and aim to help assessors judge risk of bias in the review process, results, and conclusions. ROBIS is the first rigorously developed tool designed specifically to assess the risk of bias in systematic reviews. Copyright © 2016 The Authors. Published by Elsevier Inc. All rights reserved.

Comparison of self-citation by peer reviewers in a journal with single-blind peer review versus a journal with open peer review.

PubMed

Levis, Alexander W; Leentjens, Albert F G; Levenson, James L; Lumley, Mark A; Thombs, Brett D

2015-12-01

Some peer reviewers may inappropriately, or coercively request that authors include references to the reviewers' own work. The objective of this study was to evaluate whether, compared to reviews for a journal with single-blind peer review, reviews for a journal with open peer review included (1) fewer self-citations; (2) a lower proportion of self-citations without a rationale; and (3) a lower ratio of proportions of citations without a rationale in self-citations versus citations to others' work. Peer reviews for published manuscripts submitted in 2012 to a single-blind peer review journal, the Journal of Psychosomatic Research, were previously evaluated (Thombs et al., 2015). These were compared to publically available peer reviews of manuscripts published in 2012 in an open review journal, BMC Psychiatry. Two investigators independently extracted data for both journals. There were no significant differences between journals in the proportion of all reviewer citations that were self-citations (Journal of Psychosomatic Research: 71/225, 32%; BMC Psychiatry: 90/315, 29%; p=.50), or in the proportion of self-citations without a rationale (Journal of Psychosomatic Research: 15/71, 21%; BMC Psychiatry: 12/90, 13%; p=.21). There was no significant difference between journals in the proportion of self-citations versus citations to others' work without a rationale (p=.31). Blind and open peer review methodologies have distinct advantages and disadvantages. The present study found that, in reasonably similar journals that use single-blind and open review, there were no substantive differences in the pattern of peer reviewer self-citations. Copyright © 2015 Elsevier Inc. All rights reserved.

Quality of systematic reviews: an example of studies comparing artificial disc replacement with fusion in the cervical spine.

PubMed

Tashani, Osama A; El-Tumi, Hanan; Aneiba, Khaled

2015-01-01

Cervical artificial disc replacement (C-ADR) is now an alternative to anterior cervical discectomy and fusion (ACDF). Many studies have evaluated the efficacy of C-ADR compared with ACDF. This led to a series of systematic reviews and meta-analyses to evaluate the evidence of the superiority of one intervention against the other. The aim of the study presented here was to evaluate the quality of these reviews and meta-analyses. Medline via Ovid, Embase, and Cochrane Library were searched using the keywords: (total disk replacement, prosthesis, implantation, discectomy, and arthroplasty) AND (cervical vertebrae, cervical spine, and spine) AND (systematic reviews, reviews, and meta-analysis). Screening and data extraction were conducted by two reviewers independently. Two reviewers then assessed the quality of the selected reviews and meta-analysis using 11-item AMSTAR score which is a validated measurement tool to assess the methodological quality of systematic reviews. Screening of full reports of 46 relevant abstracts resulted in the selection of 15 systematic reviews and/or meta-analyses as eligible for this study. The two reviewers' inter-rater agreement level was high as indicated by kappa of >0.72. The AMSTAR score of the reviews ranged from 3 to 11. Only one study (a Cochrane review) scored 100% (AMSTAR 11). Five studies scored below (AMSTAR 5) indicating low-quality reviews. The most significant drawbacks of reviews of a score below 5 were not using an extensive search strategy, failure to use the scientific quality of the included studies appropriately in formulating a conclusion, not assessing publication bias, and not reporting the excluded studies. With a significant exception of a Cochrane review, the methodological quality of systematic reviews evaluating the evidence of C-ADR versus ACDF has to be improved.

The use of life review to enhance spiritual well-being in patients with terminal illnesses: An integrative review.

PubMed

Kwan, Cecilia W M; Ng, Marques S N; Chan, Carmen W H

2017-12-01

To conduct an integrative review of the current literature on using life review as an intervention to address the spiritual need of patients with terminal illnesses. Palliative care highlights the holistic approach of care including the spiritual aspect. Life review has been used in palliative nursing intending to enhance patients' emotional and spiritual well-being, and quality of life. However, there is a lack of publications that provide a structured overview on life review programmes and their effectiveness. Integrative review. The Whittemore and Knafl integrative review method was used. Five major online databases were included in our literature search. The keywords used were "life review" and "palliative care, terminal care, terminally ill, death & dying, hospice, spiritual wellbeing, spirituality". Seven primary papers were identified, critically appraised and synthesised in the final review. There are limited clinical studies on life review programmes for patients with terminal illness. The research design of these studies is too widely varied for meta-analysis. Here, we identified two major programmes of life review as an intervention to address the spiritual well-being of patients with terminal illness. However, repeated studies on the effectiveness of these two programmes are lacking. The shorter programme of life review is more likely to be applicable and effective for terminal patients. Further research in this area is required to provide strong evidence on the effectiveness and applicability of life review in patients receiving palliative care. This review adds weight to the need of a better understanding on the use of life review in addressing the spiritual needs of patients with terminal illness. Such understanding would provide evidence for the use of life review as an alternative approach in palliative care delivery. © 2017 John Wiley & Sons Ltd.

Dissemination Bias in Systematic Reviews of Animal Research: A Systematic Review

PubMed Central

Mueller, Katharina F.; Briel, Matthias; Strech, Daniel; Meerpohl, Joerg J.; Lang, Britta; Motschall, Edith; Gloy, Viktoria; Lamontagne, Francois; Bassler, Dirk

2014-01-01

Background Systematic reviews of preclinical studies, in vivo animal experiments in particular, can influence clinical research and thus even clinical care. Dissemination bias, selective dissemination of positive or significant results, is one of the major threats to validity in systematic reviews also in the realm of animal studies. We conducted a systematic review to determine the number of published systematic reviews of animal studies until present, to investigate their methodological features especially with respect to assessment of dissemination bias, and to investigate the citation of preclinical systematic reviews on clinical research. Methods Eligible studies for this systematic review constitute systematic reviews that summarize in vivo animal experiments whose results could be interpreted as applicable to clinical care. We systematically searched Ovid Medline, Embase, ToxNet, and ScienceDirect from 1st January 2009 to 9th January 2013 for eligible systematic reviews without language restrictions. Furthermore we included articles from two previous systematic reviews by Peters et al. and Korevaar et al. Results The literature search and screening process resulted in 512 included full text articles. We found an increasing number of published preclinical systematic reviews over time. The methodological quality of preclinical systematic reviews was low. The majority of preclinical systematic reviews did not assess methodological quality of the included studies (71%), nor did they assess heterogeneity (81%) or dissemination bias (87%). Statistics quantifying the importance of clinical research citing systematic reviews of animal studies showed that clinical studies referred to the preclinical research mainly to justify their study or a future study (76%). Discussion Preclinical systematic reviews may have an influence on clinical research but their methodological quality frequently remains low. Therefore, systematic reviews of animal research should be critically appraised before translating them to a clinical context. PMID:25541734

Integrated care programmes for adults with chronic conditions: a meta-review.

PubMed

Martínez-González, Nahara Anani; Berchtold, Peter; Ullman, Klara; Busato, André; Egger, Matthias

2014-10-01

To review systematic reviews and meta-analyses of integrated care programmes in chronically ill patients, with a focus on methodological quality, elements of integration assessed and effects reported. Meta-review of systematic reviews and meta-analyses identified in Medline (1946-March 2012), Embase (1980-March 2012), CINHAL (1981-March 2012) and the Cochrane Library of Systematic Reviews (issue 1, 2012). Methodological quality assessed by the 11-item Assessment of Multiple Systematic Reviews (AMSTAR) checklist; elements of integration assessed using a published list of 10 key principles of integration; effects on patient-centred outcomes, process quality, use of healthcare and costs. Twenty-seven systematic reviews were identified; conditions included chronic heart failure (CHF; 12 reviews), diabetes mellitus (DM; seven reviews), chronic obstructive pulmonary disease (COPD; seven reviews) and asthma (five reviews). The median number of AMSTAR checklist items met was five: few reviewers searched for unpublished literature or described the primary studies and interventions in detail. Most reviews covered comprehensive services across the care continuum or standardization of care through inter-professional teams, but organizational culture, governance structure or financial management were rarely assessed. A majority of reviews found beneficial effects of integration, including reduced hospital admissions and re-admissions (in CHF and DM), improved adherence to treatment guidelines (DM, COPD and asthma) or quality of life (DM). Few reviews showed reductions in costs. Systematic reviews of integrated care programmes were of mixed quality, assessed only some components of integration of care, and showed consistent benefits for some outcomes but not others. © The Author 2014. Published by Oxford University Press in association with the International Society for Quality in Health Care.

A scoping review of scoping reviews: advancing the approach and enhancing the consistency.

PubMed

Pham, Mai T; Rajić, Andrijana; Greig, Judy D; Sargeant, Jan M; Papadopoulos, Andrew; McEwen, Scott A

2014-12-01

The scoping review has become an increasingly popular approach for synthesizing research evidence. It is a relatively new approach for which a universal study definition or definitive procedure has not been established. The purpose of this scoping review was to provide an overview of scoping reviews in the literature. A scoping review was conducted using the Arksey and O'Malley framework. A search was conducted in four bibliographic databases and the gray literature to identify scoping review studies. Review selection and characterization were performed by two independent reviewers using pretested forms. The search identified 344 scoping reviews published from 1999 to October 2012. The reviews varied in terms of purpose, methodology, and detail of reporting. Nearly three-quarter of reviews (74.1%) addressed a health topic. Study completion times varied from 2 weeks to 20 months, and 51% utilized a published methodological framework. Quality assessment of included studies was infrequently performed (22.38%). Scoping reviews are a relatively new but increasingly common approach for mapping broad topics. Because of variability in their conduct, there is a need for their methodological standardization to ensure the utility and strength of evidence. © 2014 The Authors. Research Synthesis Methods published by John Wiley & Sons, Ltd.

Methodological and reporting quality of systematic reviews on tuberculosis.

PubMed

Nicolau, I; Ling, D; Tian, L; Lienhardt, C; Pai, M

2013-09-01

Systematic reviews are used to inform tuberculosis (TB) guidelines. However, there are no data on whether TB systematic reviews are conducted well and reported transparently. We searched four databases for reviews published between 2005 and 2010. Methodological quality was evaluated using AMSTAR and quality of reporting was assessed using PRISMA. Of 152 articles, 137 (90%) met the inclusion criteria. Only 3 of 11 AMSTAR quality items were met in most reviews: appropriate methods to combine findings (67%), comprehensive literature search (72%) and presentation of characteristics of included studies (90%). The other eight items were met in 4-53% of the reviews. Only 4% of the reviews disclosed conflicts of interest. The majority of the PRISMA items were reported in more than 60-76% of the reviews. Only nine items were reported in less than 55% of the reviews, the lowest being the full-search strategy (30%), risk of bias across studies in the Methods (27%) and Results (21%) sections, and indication of a review protocol (15%). Systematic reviews in our survey were well reported but generally of moderate to low quality. Better training, use of reporting guidelines and registration of systematic reviews could improve the quality of TB reviews.

Diversifying customer review rankings.

PubMed

Krestel, Ralf; Dokoohaki, Nima

2015-06-01

E-commerce Web sites owe much of their popularity to consumer reviews accompanying product descriptions. On-line customers spend hours and hours going through heaps of textual reviews to decide which products to buy. At the same time, each popular product has thousands of user-generated reviews, making it impossible for a buyer to read everything. Current approaches to display reviews to users or recommend an individual review for a product are based on the recency or helpfulness of each review. In this paper, we present a framework to rank product reviews by optimizing the coverage of the ranking with respect to sentiment or aspects, or by summarizing all reviews with the top-K reviews in the ranking. To accomplish this, we make use of the assigned star rating for a product as an indicator for a review's sentiment polarity and compare bag-of-words (language model) with topic models (latent Dirichlet allocation) as a mean to represent aspects. Our evaluation on manually annotated review data from a commercial review Web site demonstrates the effectiveness of our approach, outperforming plain recency ranking by 30% and obtaining best results by combining language and topic model representations. Copyright © 2015 Elsevier Ltd. All rights reserved.

Scoping meta-review: introducing a new methodology.

PubMed

Sarrami-Foroushani, Pooria; Travaglia, Joanne; Debono, Deborah; Clay-Williams, Robyn; Braithwaite, Jeffrey

2015-02-01

For researchers, policymakers, and practitioners facing a new field, undertaking a systematic review can typically present a challenge due to the enormous number of relevant papers. A scoping review is a method suggested for addressing this dilemma; however, scoping reviews present their own challenges. This paper introduces the "scoping meta-review" (SMR) for expanding current methodologies and is based on our experiences in mapping the field of consumer engagement in healthcare. During this process, we developed the novel SMR method. An SMR combines aspects of a scoping review and a meta-review to establish an evidence-based map of a field. Similar to a scoping review, an SMR offers a practical and flexible methodology. However, unlike in a traditional scoping review, only systematic reviews are included. Stages of the SMR include: undertaking a preliminary nonsystematic review; building a search strategy; interrogating academic literature databases; classifying and excluding studies based on titles and abstracts; saving the refined database of references; revising the search strategy; selecting and reviewing the full text papers; and thematically analyzing the selected texts and writing the report. The main benefit of an SMR is to map a new field based on high-level evidence provided by systematic reviews. © 2014 Wiley Periodicals, Inc.

Economics methods in Cochrane systematic reviews of health promotion and public health related interventions

PubMed Central

Shemilt, Ian; Mugford, Miranda; Drummond, Michael; Eisenstein, Eric; Mallender, Jacqueline; McDaid, David; Vale, Luke; Walker, Damian

2006-01-01

Background Provision of evidence on costs alongside evidence on the effects of interventions can enhance the relevance of systematic reviews to decision-making. However, patterns of use of economics methods alongside systematic review remain unclear. Reviews of evidence on the effects of interventions are published by both the Cochrane and Campbell Collaborations. Although it is not a requirement that Cochrane or Campbell Reviews should consider economic aspects of interventions, many do. This study aims to explore and describe approaches to incorporating economics methods in a selection of Cochrane systematic reviews in the area of health promotion and public health, to help inform development of methodological guidance on economics for reviewers. Methods The Cochrane Database of Systematic Reviews was searched using a search strategy for potential economic evaluation studies. We included current Cochrane reviews and review protocols retrieved using the search that are also identified as relevant to health promotion or public health topics. A reviewer extracted data which describe the economics components of included reviews. Extracted data were summarised in tables and analysed qualitatively. Results Twenty-one completed Cochrane reviews and seven review protocols met inclusion criteria. None incorporate formal economic evaluation methods. Ten completed reviews explicitly aim to incorporate economics studies and data. There is a lack of transparent reporting of methods underpinning the incorporation of economics studies and data. Some reviews are likely to exclude useful economics studies and data due to a failure to incorporate search strategies tailored to the retrieval of such data or use of key specialist databases, and application of inclusion criteria designed for effectiveness studies. Conclusion There is a need for consistency and transparency in the reporting and conduct of the economics components of Cochrane reviews, as well as regular dialogue between Cochrane reviewers and economists to develop increased capacity for economic analyses alongside such reviews. Use of applicable economics methods in Cochrane reviews can help provide the international context within which economics data can be interpreted and assessed as a preliminary to full economic evaluation. PMID:17107612

Do peer review models affect clinicians' trust in journals? A survey of junior doctors.

PubMed

Patel, Jigisha; Pierce, Mary; Boughton, Stephanie L; Baldeweg, Stephanie E

2017-01-01

The aim of this survey was to determine the level of awareness and understanding of peer review and peer review models amongst junior hospital doctors and whether this influences clinical decision-making. A 30-question online anonymous survey was developed aimed at determining awareness of peer review models and the purpose of peer review, perceived trustworthiness of different peer review models and the role of peer review in clinical decision-making. It was sent to 800 trainee doctors in medical specialties on the University College London Partners trainee database. The response rate was (178/800) 22%. Most respondents were specialist registrars. Checking that research is conducted correctly (152/178, 85%) and the data interpreted correctly (148/178, 83%) were viewed as the most important purposes of peer review. Most respondents were aware of open (133/178, 75%), double-blind (125/178, 70%) and single-blind peer review (121/178, 68%). 101/178 (57%) had heard of collaborative, 87/178 (49%) of post publication and 29/178 (16%) of decoupled peer review. Of those who were aware of double-blind, single-blind open and collaborative peer review, 85 (68%), 82 (68%), 74 (56%) and 24 (24%), respectively, understood how they worked. The NEJM , Lancet and The BMJ were deemed to have most trustworthy peer review, 137/178 (77%), 129/178 (72%) and 115/178 (65%), respectively. That peer review had taken place was important for a journal content to be used for clinical decision-making 152/178 (85%), but the ability to see peer review reports was not as important 22/178 (12%). Most felt there was a need for peer review training and that this should be at the specialist registrar stage of training. Junior hospital doctors view peer review to be important as a means of quality control, but do not value the ability to scrutinize peer review themselves. The unquestioning acceptance of peer review as final validation in the field of medicine emphasises not only the responsibility held by medical journals to ensure peer review is done well but also the need to raise awareness amongst the medical community of the limitations of the current peer review process.

Conflicts of interest and spin in reviews of psychological therapies: a systematic review.

PubMed

Lieb, Klaus; von der Osten-Sacken, Jan; Stoffers-Winterling, Jutta; Reiss, Neele; Barth, Jürgen

2016-04-26

To explore conflicts of interest (COI) and their reporting in systematic reviews of psychological therapies, and to evaluate spin in the conclusions of the reviews. MEDLINE and PsycINFO databases were searched for systematic reviews published between 2010 and 2013 that assessed effects of psychological therapies for anxiety, depressive or personality disorders, and included at least one randomised controlled trial. Required COI disclosure by journal, disclosed COI by review authors, and the inclusion of own primary studies by review authors were extracted. Researcher allegiance, that is, that researchers concluded favourably about the interventions they have studied, as well as spin, that is, differences between results and conclusions of the reviews, were rated by 2 independent raters. 936 references were retrieved, 95 reviews fulfilled eligibility criteria. 59 compared psychological therapies with other forms of psychological therapies, and 36 psychological therapies with pharmacological interventions. Financial, non-financial, and personal COI were disclosed in 22, 4 and 1 review, respectively. 2 of 86 own primary studies of review authors included in 34 reviews were disclosed by review authors. In 15 of the reviews, authors showed an allegiance effect to the evaluated psychological therapy that was never disclosed. Spin in review conclusions was found in 27 of 95 reviews. Reviews with a conclusion in favour of psychological therapies (vs pharmacological interventions) were at high risk for a spin in conclusions (OR=8.31 (1.41 to 49.05)). Spin was related in trend to the inclusion of own primary studies in the systematic review (OR=2.08 (CI 0.83 to 5.18) p=0.11) and researcher allegiance (OR=2.63 (0.84 to 8.16) p=0.16). Non-financial COI, especially the inclusion of own primary studies into reviews and researcher allegiance, are frequently seen in systematic reviews of psychological therapies and need more transparency and better management. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/

Different contributions of internal reviewers and external experts to labelling decisions on therapeutic indications in new drug reviews in Japan.

PubMed

Yokota, M; Kusama, M; Matsuki, N; Ono, S

2013-12-01

External experts play an important role in shaping regulatory decisions in the new drug review process in the United States, Europe and Japan. No rigorous study has been performed addressing how and to what extent external experts, in contrast to internal reviewers in the agency, influence the regulatory decisions during new drug reviews. We examined their contributions in Japanese regulatory reviews in contrast to the internal reviewers, focusing on the labelling decision on therapeutic indications. With the data set of 219 new molecular entities (NMEs) approved in Japan from 2000 to 2009, we observed how proposed indications in labelling were modified in a stepwise manner during the review process and conducted multinomial logistic analysis to examine the possible mechanism behind. We found that interim assessment of indications by the internal reviewers was modified substantially by the influence of the external experts in about 20% of the 219 NMEs. Our analysis suggested that internal reviewers provided their opinion mainly based on strict review discipline, whereas external experts added flexibility and reality to their reviews. Our analysis revealed different evaluations between internal reviewers and external experts during regulatory discussions in new drug reviews and how the external panel contributes to changing internal decisions. This study provides a new and quantitative approach to better label setting by emphasizing the contributions of each stakeholder in new drug reviews, which would improve the efficiency, quality and transparency of new drug reviews to enhance public health. © 2013 John Wiley & Sons Ltd.

Pharmacological and surgical treatment of nonreproductive outcomes in polycystic ovary syndrome: An overview of systematic reviews.

PubMed

Tay, Chau T; Joham, Anju E; Hiam, Danielle S; Gadalla, Moustafa A; Pundir, Jyotsna; Thangaratinam, Shakila; Teede, Helena J; Moran, Lisa J

2018-05-30

Polycystic ovary syndrome (PCOS) affects up to 13% women and is associated with significant complications. The quality of evidence supporting the recommendations on treatment of nonreproductive outcomes in PCOS is unknown. To summarize and appraise the methodological quality of systematic reviews and meta-analyses evaluating pharmacological and surgical treatments for nonreproductive outcomes in PCOS. A literature search from MEDLINE, EMBASE, CINAHL PLUS and PROSPERO was performed from inception until 15th of September 2017. Article selection, data extraction and quality appraisal of included reviews were performed in duplicate. A narrative synthesis of the findings was conducted. This overview included 31 reviews. The quality was low for 7 (23%), moderate for sixteen (52%) and high for 8 reviews (26%). Two reviews assessed psychological outcomes. Metformin improved anthropometric (7 of 10 reviews), metabolic (4 of 14 reviews) and endocrine outcomes (3 of twelve reviews). Thiazolidinediones improved metabolic (2 of 5 reviews) and endocrine outcomes (one of 5 reviews) but worsened weight gain (5 of 5 reviews). Combined oral contraceptive pill (COCP) improved clinical hyperandrogenism (2 of 2 reviews). Statins improved lipid profile (3 of 3 reviews) and testosterone level (2 of 3 reviews). There was no conclusive evidence from included systematic reviews regarding the use of other interventions. There is reliable evidence regarding the use of metformin for anthropometric outcomes and COCPs for hyperandrogenism in women with PCOS but not for other interventions. There is significant gap in knowledge regarding the management of psychological outcomes in women with PCOS which needs further evaluation. © 2018 John Wiley & Sons Ltd.

Teleconference versus Face-to-Face Scientific Peer Review of Grant Application: Effects on Review Outcomes

PubMed Central

Gallo, Stephen A.; Carpenter, Afton S.; Glisson, Scott R.

2013-01-01

Teleconferencing as a setting for scientific peer review is an attractive option for funding agencies, given the substantial environmental and cost savings. Despite this, there is a paucity of published data validating teleconference-based peer review compared to the face-to-face process. Our aim was to conduct a retrospective analysis of scientific peer review data to investigate whether review setting has an effect on review process and outcome measures. We analyzed reviewer scoring data from a research program that had recently modified the review setting from face-to-face to a teleconference format with minimal changes to the overall review procedures. This analysis included approximately 1600 applications over a 4-year period: two years of face-to-face panel meetings compared to two years of teleconference meetings. The average overall scientific merit scores, score distribution, standard deviations and reviewer inter-rater reliability statistics were measured, as well as reviewer demographics and length of time discussing applications. The data indicate that few differences are evident between face-to-face and teleconference settings with regard to average overall scientific merit score, scoring distribution, standard deviation, reviewer demographics or inter-rater reliability. However, some difference was found in the discussion time. These findings suggest that most review outcome measures are unaffected by review setting, which would support the trend of using teleconference reviews rather than face-to-face meetings. However, further studies are needed to assess any correlations among discussion time, application funding and the productivity of funded research projects. PMID:23951223

A low proportion of systematic reviews in physical therapy are registered: a survey of 150 published systematic reviews.

PubMed

Oliveira, Crystian B; Elkins, Mark R; Lemes, Ítalo Ribeiro; de Oliveira Silva, Danilo; Briani, Ronaldo V; Monteiro, Henrique Luiz; Azevedo, Fábio Mícolis de; Pinto, Rafael Zambelli

Systematic reviews provide the best evidence about the effectiveness of healthcare interventions. Although systematic reviews are conducted with explicit and transparent methods, discrepancies might occur between the protocol and the publication. To estimate the proportion of systematic reviews of physical therapy interventions that are registered, the methodological quality of (un)registered systematic reviews and the prevalence of outcome reporting bias in registered systematic reviews. A random sample of 150 systematic reviews published in 2015 indexed on the PEDro database. We included systematic reviews written in English, Italian, Portuguese and Spanish. A checklist for assessing the methodological quality of systematic reviews tool was used. Relative risk was calculated to explore the association between meta-analysis results and the changes in the outcomes. Twenty-nine (19%) systematic reviews were registered. Funding and publication in a journal with an impact factor higher than 5.0 were associated with registration. Registered systematic reviews demonstrated significantly higher methodological quality (median=8) than unregistered systematic reviews (median=5). Nine (31%) registered systematic reviews demonstrated discrepancies between protocol and publication with no evidence that such discrepancies were applied to favor the statistical significance of the intervention (RR=1.16; 95% CI: 0.63-2.12). A low proportion of systematic reviews in the physical therapy field are registered. The registered systematic reviews showed high methodological quality without evidence of outcome reporting bias. Further strategies should be implemented to encourage registration. Copyright © 2017 Associação Brasileira de Pesquisa e Pós-Graduação em Fisioterapia. Publicado por Elsevier Editora Ltda. All rights reserved.

High-Stakes Systematic Reviews: A Case Study From the Field of Teen Pregnancy Prevention.

PubMed

Goesling, Brian; Oberlander, Sarah; Trivits, Lisa

2016-08-19

Systematic reviews help policy makers and practitioners make sense of research findings in a particular program, policy, or practice area by synthesizing evidence across multiple studies. However, the link between review findings and practical decision-making is rarely one-to-one. Policy makers and practitioners may use systematic review findings to help guide their decisions, but they may also rely on other information sources or personal judgment. To describe a recent effort by the U.S. federal government to narrow the gap between review findings and practical decision-making. The Teen Pregnancy Prevention (TPP) Evidence Review was launched by the U.S. Department of Health and Human Services (HHS) in 2009 as a systematic review of the TPP literature. HHS has used the review findings to determine eligibility for federal funding for TPP programs, marking one of the first attempts to directly link systematic review findings with federal funding decisions. The high stakes attached to the review findings required special considerations in designing and conducting the review. To provide a sound basis for federal funding decisions, the review had to meet accepted methodological standards. However, the review team also had to account for practical constraints of the funding legislation and needs of the federal agencies responsible for administering the grant programs. The review team also had to develop a transparent process for both releasing the review findings and updating them over time. Prospective review authors and sponsors must recognize both the strengths and limitations of this approach before applying it in other areas. © The Author(s) 2016.

Measurement of patient safety: a systematic review of the reliability and validity of adverse event detection with record review

PubMed Central

Hanskamp-Sebregts, Mirelle; Zegers, Marieke; Vincent, Charles; van Gurp, Petra J; de Vet, Henrica C W; Wollersheim, Hub

2016-01-01

Objectives Record review is the most used method to quantify patient safety. We systematically reviewed the reliability and validity of adverse event detection with record review. Design A systematic review of the literature. Methods We searched PubMed, EMBASE, CINAHL, PsycINFO and the Cochrane Library and from their inception through February 2015. We included all studies that aimed to describe the reliability and/or validity of record review. Two reviewers conducted data extraction. We pooled κ values (κ) and analysed the differences in subgroups according to number of reviewers, reviewer experience and training level, adjusted for the prevalence of adverse events. Results In 25 studies, the psychometric data of the Global Trigger Tool (GTT) and the Harvard Medical Practice Study (HMPS) were reported and 24 studies were included for statistical pooling. The inter-rater reliability of the GTT and HMPS showed a pooled κ of 0.65 and 0.55, respectively. The inter-rater agreement was statistically significantly higher when the group of reviewers within a study consisted of a maximum five reviewers. We found no studies reporting on the validity of the GTT and HMPS. Conclusions The reliability of record review is moderate to substantial and improved when a small group of reviewers carried out record review. The validity of the record review method has never been evaluated, while clinical data registries, autopsy or direct observations of patient care are potential reference methods that can be used to test concurrent validity. PMID:27550650

Systematic reviews and meta-analyses on treatment of asthma: critical evaluation

PubMed Central

Jadad, Alejandro R; Moher, Michael; Browman, George P; Booker, Lynda; Sigouin, Christopher; Fuentes, Mario; Stevens, Robert

2000-01-01

Objective To evaluate the clinical, methodological, and reporting aspects of systematic reviews and meta-analyses on the treatment of asthma and to compare those published by the Cochrane Collaboration with those published in paper based journals. Design Analysis of studies identified from Medline, CINAHL, HealthSTAR, EMBASE, Cochrane Library, personal collections, and reference lists. Studies Articles describing a systematic review or a meta-analysis of the treatment of asthma that were published as a full report, in any language or format, in a peer reviewed journal or the Cochrane Library. Main outcome measures General characteristics of studies reviewed and methodological characteristics (sources of articles; language restrictions; format, design, and publication status of studies included; type of data synthesis; and methodological quality). Results 50 systematic reviews and meta-analyses were included. More than half were published in the past two years. Twelve reviews were published in the Cochrane Library and 38 were published in 22 peer reviewed journals. Forced expiratory volume in one second was the most frequently used outcome, but few reviews evaluated the effect of treatment on costs or patient preferences. Forty reviews were judged to have serious or extensive flaws. All six reviews associated with industry were in this group. Seven of the 10 most rigorous reviews were published in the Cochrane Library. Conclusions Most reviews published in peer reviewed journals or funded by industry have serious methodological flaws that limit their value to guide decisions. Cochrane reviews are more rigorous and better reported than those published in peer reviewed journals. PMID:10688558

Effect of using reporting guidelines during peer review on quality of final manuscripts submitted to a biomedical journal: masked randomised trial

PubMed Central

Cortés, J; Ribera, J M; Cardellach, F; Selva-O’Callaghan, A; Kostov, B; García, L; Cirugeda, L; Altman, D G; González, J A; Sànchez, J A; Miras, F; Urrutia, A; Fonollosa, V; Rey-Joly, C; Vilardell, M

2011-01-01

Objective To investigate the effect of an additional review based on reporting guidelines such as STROBE and CONSORT on quality of manuscripts. Design Masked randomised trial. Population Original research manuscripts submitted to the Medicina Clínica journal from May 2008 to April 2009 and considered suitable for publication. Intervention Control group: conventional peer reviews alone. Intervention group: conventional review plus an additional review looking for missing items from reporting guidelines. Outcomes Manuscript quality, assessed with a 5 point Likert scale (primary: overall quality; secondary: average quality of specific items in paper). Main analysis compared groups as allocated, after adjustment for baseline factors (analysis of covariance); sensitivity analysis compared groups as reviewed. Adherence to reviewer suggestions assessed with Likert scale. Results Of 126 consecutive papers receiving conventional review, 34 were not suitable for publication. The remaining 92 papers were allocated to receive conventional reviews alone (n=41) or additional reviews (n=51). Four papers assigned to the conventional review group deviated from protocol; they received an additional review based on reporting guidelines. We saw an improvement in manuscript quality in favour of the additional review group (comparison as allocated, 0.25, 95% confidence interval –0.05 to 0.54; as reviewed, 0.33, 0.03 to 0.63). More papers with additional reviews than with conventional reviews alone improved from baseline (22 (43%) v eight (20%), difference 23.6% (3.2% to 44.0%), number needed to treat 4.2 (from 2.3 to 31.2), relative risk 2.21 (1.10 to 4.44)). Authors in the additional review group adhered more to suggestions from conventional reviews than to those from additional reviews (average increase 0.43 Likert points (0.19 to 0.67)). Conclusions Additional reviews based on reporting guidelines improve manuscript quality, although the observed effect was smaller than hypothesised and not definitively demonstrated. Authors adhere more to suggestions from conventional reviews than to those from additional reviews, showing difficulties in adhering to high methodological standards at the latest research phases. To boost paper quality and impact, authors should be aware of future requirements of reporting guidelines at the very beginning of their study. Trial registration and protocol Although registries do not include trials of peer review, the protocol design was submitted to sponsored research projects (Instituto de Salud Carlos III, PI081903). PMID:22108262

Interventions for Age-Related Macular Degeneration: Are Practice Guidelines Based on Systematic Reviews?

PubMed Central

Lindsley, Kristina; Li, Tianjing; Ssemanda, Elizabeth; Virgili, Gianni; Dickersin, Kay

2016-01-01

Topic Are existing systematic reviews of interventions for age-related macular degeneration incorporated into clinical practice guidelines? Clinical relevance High-quality systematic reviews should be used to underpin evidence-based clinical practice guidelines and clinical care. We have examined the reliability of systematic reviews of interventions for age-related macular degeneration (AMD) and described the main findings of reliable reviews in relation to clinical practice guidelines. Methods Eligible publications are systematic reviews of the effectiveness of treatment interventions for AMD. We searched a database of systematic reviews in eyes and vision and employed no language or date restrictions; the database is up-to-date as of May 6, 2014. Two authors independently screened records for eligibility and abstracted and assessed the characteristics and methods of each review. We classified reviews as “reliable” when they reported eligibility criteria, comprehensive searches, appraisal of methodological quality of included studies, appropriate statistical methods for meta-analysis, and conclusions based on results. We mapped treatment recommendations from the American Academy of Ophthalmology Preferred Practice Patterns (AAO PPP) for AMD to the identified systematic reviews and assessed whether any reliable systematic review was cited or could have been cited to support each treatment recommendation. Results Of 1,570 systematic reviews in our database, 47 met our inclusion criteria. Most of the systematic reviews targeted neovascular AMD and investigated anti-vascular endothelial growth factor (anti-VEGF) interventions, dietary supplements or photodynamic therapy. We classified over two-thirds (33/47) of the reports as reliable. The quality of reporting varied, with criteria for reliable reporting met more often for Cochrane reviews and for reviews whose authors disclosed conflicts of interest. Although most systematic reviews were reliable, anti-VEGF agents and photodynamic therapy were the only interventions identified as effective by reliable reviews. Of 35 treatment recommendations extracted from the AAO PPP, 15 could have been supported with reliable systematic reviews; however, only one recommendation had an accompanying intervention systematic review citation, which we assessed as a reliable systematic review. No reliable systematic review was identified for 20 treatment recommendations, highlighting areas of evidence gaps. Conclusions For AMD, reliable systematic reviews exist for many treatment recommendations in the AAO PPP and should be used to support these recommendations. We also identified areas where no high-level evidence exists. Mapping clinical practice guidelines to existing systematic reviews is one way to highlight areas where evidence generation or evidence synthesis is either available or needed. PMID:26804762

2012 DOE Vehicle Technologies Program Annual Merit Review

DOE Office of Scientific and Technical Information (OSTI.GOV)

None

The 2012 DOE Hydrogen Program and Vehicle Technologies Program Annual Merit Review and Peer Evaluation Meeting was held May 14-18, 2012 in Crystal City, Virginia. The review encompassed all of the work done by the Hydrogen Program and the Vehicle Technologies Program: a total of 309 individual activities were reviewed for Vehicle Technologies, by a total of 189 reviewers. A total of 1,473 individual review responses were received for the technical reviews.

Some opinions on the review process of research papers destined for publication.

PubMed

Roohi, Ehsan; Mahian, Omid

2015-06-01

The current paper discusses the peer review process in journals that publish research papers purveying new science and understandings (scientific journals). Different aspects of peer review including the selection of reviewers, the review process and the decision policy of editor are discussed in details. Here, the pros and cons of different conventional methods of review processes are mentioned. Finally, a suggestion is presented for the review process of scientific papers.

14 CFR 302.32 - Petitions for discretionary review of initial or recommended decisions; review proceedings.

Code of Federal Regulations, 2010 CFR

2010-01-01

... 14 Aeronautics and Space 4 2010-01-01 2010-01-01 false Petitions for discretionary review of... Petitions for discretionary review of initial or recommended decisions; review proceedings. (a) Petitions... petition for discretionary review by the DOT decisionmaker of an initial decision or recommended decision...

42 CFR 426.425 - LCD review.

Code of Federal Regulations, 2010 CFR

2010-10-01

... 42 Public Health 3 2010-10-01 2010-10-01 false LCD review. 426.425 Section 426.425 Public Health... PROGRAM REVIEW OF NATIONAL COVERAGE DETERMINATIONS AND LOCAL COVERAGE DETERMINATIONS Review of an LCD § 426.425 LCD review. (a) Opportunity for the aggrieved party, after his or her review of the LCD record...

42 CFR 426.525 - NCD review.

Code of Federal Regulations, 2010 CFR

2010-10-01

... 42 Public Health 3 2010-10-01 2010-10-01 false NCD review. 426.525 Section 426.525 Public Health... PROGRAM REVIEW OF NATIONAL COVERAGE DETERMINATIONS AND LOCAL COVERAGE DETERMINATIONS Review of an NCD § 426.525 NCD review. (a) Opportunity for the aggrieved party after his or her review of the NCD record...

78 FR 47016 - Submission for Review: Request for External Review

Federal Register 2010, 2011, 2012, 2013, 2014

2013-08-02

... OFFICE OF PERSONNEL MANAGEMENT Submission for Review: Request for External Review AGENCY: U.S... External Review. As required by the Paperwork Reduction Act of 1995, (Pub. L. 104-13, 44 U.S.C. chapter 35... the Multi-State Plan Program (MSPP) on March 11, 2013, 78 FR 15560, which outlined an external review...

14 CFR 415.23 - Policy review.

Code of Federal Regulations, 2010 CFR

2010-01-01

... 14 Aeronautics and Space 4 2010-01-01 2010-01-01 false Policy review. 415.23 Section 415.23... TRANSPORTATION LICENSING LAUNCH LICENSE Policy Review and Approval § 415.23 Policy review. (a) The FAA reviews a... raised during a policy review that would impede issuance of a policy approval. The applicant may respond...

14 CFR 415.57 - Payload review.

Code of Federal Regulations, 2010 CFR

2010-01-01

... 14 Aeronautics and Space 4 2010-01-01 2010-01-01 false Payload review. 415.57 Section 415.57... TRANSPORTATION LICENSING LAUNCH LICENSE Payload Review and Determination § 415.57 Payload review. (a) Timing. A payload review may be conducted as part of a license application review or may be requested by a payload...

40 CFR 725.270 - EPA review of the TERA.

Code of Federal Regulations, 2010 CFR

2010-07-01

... 40 Protection of Environment 30 2010-07-01 2010-07-01 false EPA review of the TERA. 725.270... Development Activities § 725.270 EPA review of the TERA. General procedures for review of all submissions... to EPA review of applications submitted under this subpart: (a) Length of the review period. (1) The...

6 CFR 27.405 - Review and preemption of State laws and regulations.

Code of Federal Regulations, 2010 CFR

2010-01-01

... 6 Domestic Security 1 2010-01-01 2010-01-01 false Review and preemption of State laws and... CHEMICAL FACILITY ANTI-TERRORISM STANDARDS Other § 27.405 Review and preemption of State laws and...) Review and opinion—(1) Review. The Department may review State laws, administrative actions, or opinions...

NOAA Office of Exploration and Research > Data Access > Publications

Science.gov Websites

2014 Funding Opportunities Contact Us Program Review Review Panel Final Report Review Team Documents Affiliate Locations Strategic Plan 2014 Funding Opportunities Contact Us Program Review Review Panel Final Report Review Team Documents Presentations Supporting Documents Guiding Documents Exploration Overview

Re-Viewing Peer Review

ERIC Educational Resources Information Center

Flynn, Elizabeth A.

2011-01-01

In this article, the author revisits her essay, "Students as Readers of Their Classmates' Writing," by providing a review of the literature on peer review over the past three decades and comments on patterns she sees in waves of peer review research and theorizing. She describes her subsequent experience with peer review in her own classes, and…

20 CFR 802.301 - Scope of review.

Code of Federal Regulations, 2011 CFR

2011-04-01

... 20 Employees' Benefits 3 2011-04-01 2011-04-01 false Scope of review. 802.301 Section 802.301 Employees' Benefits BENEFITS REVIEW BOARD, DEPARTMENT OF LABOR RULES OF PRACTICE AND PROCEDURE Procedure for Review Action by the Board § 802.301 Scope of review. (a) The Benefits Review Board is not empowered to...

20 CFR 802.301 - Scope of review.

Code of Federal Regulations, 2012 CFR

2012-04-01

... 20 Employees' Benefits 4 2012-04-01 2012-04-01 false Scope of review. 802.301 Section 802.301 Employees' Benefits BENEFITS REVIEW BOARD, DEPARTMENT OF LABOR RULES OF PRACTICE AND PROCEDURE Procedure for Review Action by the Board § 802.301 Scope of review. (a) The Benefits Review Board is not empowered to...

20 CFR 802.301 - Scope of review.

Code of Federal Regulations, 2013 CFR

2013-04-01

... 20 Employees' Benefits 4 2013-04-01 2013-04-01 false Scope of review. 802.301 Section 802.301 Employees' Benefits BENEFITS REVIEW BOARD, DEPARTMENT OF LABOR RULES OF PRACTICE AND PROCEDURE Procedure for Review Action by the Board § 802.301 Scope of review. (a) The Benefits Review Board is not empowered to...

20 CFR 802.301 - Scope of review.

Code of Federal Regulations, 2014 CFR

2014-04-01

... 20 Employees' Benefits 4 2014-04-01 2014-04-01 false Scope of review. 802.301 Section 802.301 Employees' Benefits BENEFITS REVIEW BOARD, DEPARTMENT OF LABOR RULES OF PRACTICE AND PROCEDURE Procedure for Review Action by the Board § 802.301 Scope of review. (a) The Benefits Review Board is not empowered to...

20 CFR 802.301 - Scope of review.

Code of Federal Regulations, 2010 CFR

2010-04-01

... 20 Employees' Benefits 3 2010-04-01 2010-04-01 false Scope of review. 802.301 Section 802.301 Employees' Benefits BENEFITS REVIEW BOARD, DEPARTMENT OF LABOR RULES OF PRACTICE AND PROCEDURE Procedure for Review Action by the Board § 802.301 Scope of review. (a) The Benefits Review Board is not empowered to...

45 CFR 79.5 - Review by the reviewing official.

Code of Federal Regulations, 2010 CFR

2010-10-01

... 45 Public Welfare 1 2010-10-01 2010-10-01 false Review by the reviewing official. 79.5 Section 79.5 Public Welfare DEPARTMENT OF HEALTH AND HUMAN SERVICES GENERAL ADMINISTRATION PROGRAM FRAUD CIVIL REMEDIES § 79.5 Review by the reviewing official. (a) If, based on the report of the investigating official...

6 CFR 13.5 - Review by the Reviewing Official.

Code of Federal Regulations, 2010 CFR

2010-01-01

... 6 Domestic Security 1 2010-01-01 2010-01-01 false Review by the Reviewing Official. 13.5 Section 13.5 Domestic Security DEPARTMENT OF HOMELAND SECURITY, OFFICE OF THE SECRETARY PROGRAM FRAUD CIVIL REMEDIES § 13.5 Review by the Reviewing Official. (a) If, based on the report of the Investigating Official...

Test Reviewing at the Buros Center for Testing

ERIC Educational Resources Information Center

Carlson, Janet F.; Geisinger, Kurt F.

2012-01-01

The test review process used by the Buros Center for Testing is described as a series of 11 steps: (1) identifying tests to be reviewed, (2) obtaining tests and preparing test descriptions, (3) determining whether tests meet review criteria, (4) identifying appropriate reviewers, (5) selecting reviewers, (6) sending instructions and materials to…

Design for Review - Applying Lessons Learned to Improve the FPGA Review Process

NASA Technical Reports Server (NTRS)

Figueiredo, Marco A.; Li, Kenneth E.

2014-01-01

Flight Field Programmable Gate Array (FPGA) designs are required to be independently reviewed. This paper provides recommendations to Flight FPGA designers to properly prepare their designs for review in order to facilitate the review process, and reduce the impact of the review time in the overall project schedule.

Assessment of the Risk of Bias in Rehabilitation Reviews

ERIC Educational Resources Information Center

Farmer, Sybil E.; Wood, Duncan; Swain, Ian D.; Pandyan, Anand D.

2012-01-01

Systematic reviews are used to inform practice, and develop guidelines and protocols. A questionnaire to quantify the risk of bias in systematic reviews, the review paper assessment (RPA) tool, was developed and tested. A search of electronic databases provided a data set of review articles that were then independently reviewed by two assessors…

Academic Community and Post-Tenure Review.

ERIC Educational Resources Information Center

Tierney, William G.

1997-01-01

Discusses the need for post-tenure faculty review to root out "dead wood" faculty and increase faculty accountability, focusing on the time frame for such reviews, who gets reviewed, and the intensity and ramifications of the review. Also notes criticisms of post-tenure reviews and the need to build community through self-regulation.…

The art and science of knowledge synthesis.

PubMed

Tricco, Andrea C; Tetzlaff, Jennifer; Moher, David

2011-01-01

To review methods for completing knowledge synthesis. We discuss how to complete a broad range of knowledge syntheses. Our article is intended as an introductory guide. Many groups worldwide conduct knowledge syntheses, and some methods are applicable to most reviews. However, variations of these methods are apparent for different types of reviews, such as realist reviews and mixed-model reviews. Review validity is dependent on the validity of the included primary studies and the review process itself. Steps should be taken to avoid bias in the conduct of knowledge synthesis. Transparency in reporting will help readers assess review validity and applicability, increasing its utility. Given the magnitude of the literature, the increasing demands on knowledge syntheses teams, and the diversity of approaches, continuing efforts will be important to increase the efficiency, validity, and applicability of systematic reviews. Future research should focus on increasing the uptake of knowledge synthesis, how best to update reviews, the comparability between different types of reviews (eg, rapid vs. comprehensive reviews), and how to prioritize knowledge synthesis topics. Copyright © 2011 Elsevier Inc. All rights reserved.

Clinical peer review in the United States: history, legal development and subsequent abuse.

PubMed

Vyas, Dinesh; Hozain, Ahmed E

2014-06-07

The Joint Commission on Accreditation requires hospitals to conduct peer review to retain accreditation. Despite the intended purpose of improving quality medical care, the peer review process has suffered several setbacks throughout its tenure. In the 1980s, abuse of peer review for personal economic interest led to a highly publicized multimillion-dollar verdict by the United States Supreme Court against the perpetrating physicians and hospital. The verdict led to decreased physician participation for fear of possible litigation. Believing that peer review was critical to quality medical care, Congress subsequently enacted the Health Care Quality Improvement Act (HCQIA) granting comprehensive legal immunity for peer reviewers to increase participation. While serving its intended goal, HCQIA has also granted peer reviewers significant immunity likely emboldening abuses resulting in Sham Peer Reviews. While legal reform of HCQIA is necessary to reduce sham peer reviews, further measures including the need for standardization of the peer review process alongside external organizational monitoring are critical to improving peer review and reducing the prevalence of sham peer reviews.

Clinical peer review in the United States: History, legal development and subsequent abuse

PubMed Central

Vyas, Dinesh; Hozain, Ahmed E

2014-01-01

The Joint Commission on Accreditation requires hospitals to conduct peer review to retain accreditation. Despite the intended purpose of improving quality medical care, the peer review process has suffered several setbacks throughout its tenure. In the 1980s, abuse of peer review for personal economic interest led to a highly publicized multimillion-dollar verdict by the United States Supreme Court against the perpetrating physicians and hospital. The verdict led to decreased physician participation for fear of possible litigation. Believing that peer review was critical to quality medical care, Congress subsequently enacted the Health Care Quality Improvement Act (HCQIA) granting comprehensive legal immunity for peer reviewers to increase participation. While serving its intended goal, HCQIA has also granted peer reviewers significant immunity likely emboldening abuses resulting in Sham Peer Reviews. While legal reform of HCQIA is necessary to reduce sham peer reviews, further measures including the need for standardization of the peer review process alongside external organizational monitoring are critical to improving peer review and reducing the prevalence of sham peer reviews. PMID:24914357

Engineering Technical Review Planning Briefing

NASA Technical Reports Server (NTRS)

Gardner, Terrie

2012-01-01

The general topics covered in the engineering technical planning briefing are 1) overviews of NASA, Marshall Space Flight Center (MSFC), and Engineering, 2) the NASA Systems Engineering(SE) Engine and its implementation , 3) the NASA Project Life Cycle, 4) MSFC Technical Management Branch Services in relation to the SE Engine and the Project Life Cycle , 5) Technical Reviews, 6) NASA Human Factor Design Guidance , and 7) the MSFC Human Factors Team. The engineering technical review portion of the presentation is the primary focus of the overall presentation and will address the definition of a design review, execution guidance, the essential stages of a technical review, and the overall review planning life cycle. Examples of a technical review plan content, review approaches, review schedules, and the review process will be provided and discussed. The human factors portion of the presentation will focus on the NASA guidance for human factors. Human factors definition, categories, design guidance, and human factor specialist roles will be addressed. In addition, the NASA Systems Engineering Engine description, definition, and application will be reviewed as background leading into the NASA Project Life Cycle Overview and technical review planning discussion.

The ethics of peer review in bioethics

PubMed Central

Wendler, David; Miller, Franklin

2014-01-01

A good deal has been written on the ethics of peer review, especially in the scientific and medical literatures. In contrast, we are unaware of any articles on the ethics of peer review in bioethics. Recognising this gap, we evaluate the extant proposals regarding ethical standards for peer review in general and consider how they apply to bioethics. We argue that scholars have an obligation to perform peer review based on the extent to which they personally benefit from the peer review process. We also argue, contrary to existing proposals and guidelines, that it can be appropriate for peer reviewers to benefit in their own scholarship from the manuscripts they review. With respect to bioethics in particular, we endorse double-blind review and suggest several ways in which the peer review process might be improved. PMID:24131903

IRIS Toxicological Review of Dichloromethane (Methylene ...

EPA Pesticide Factsheets

On March 31, 2010, the draft IRIS Toxicological Review of Dichloromethane (Methylene Chloride) external review draft document and the charge to external peer reviewers were released for public review and comment. The draft document and the charge to external peer reviewers were reviewed internally by EPA and by other federal agencies and White House Offices before public release. In the new IRIS process, introduced by the EPA Administrator, all written comments on IRIS assessments submitted by other federal agencies and White House Offices will be made publicly available. Accordingly, interagency comments and the interagency science consultation draft of the Toxicological Review of Dichloromethane (Methylene Chloride) and the charge to external peer reviewers are posted on this site. The draft Toxicological Review of Dichloromethane provides scientific support and rationale for the hazard and dose-response assessment pertaining to chronic exposure to dichloromethane.

Neuroradiology Using Secure Mobile Device Review.

PubMed

Randhawa, Privia A; Morrish, William; Lysack, John T; Hu, William; Goyal, Mayank; Hill, Michael D

2016-04-05

Image review on computer-based workstations has made film-based review outdated. Despite advances in technology, the lack of portability of digital workstations creates an inherent disadvantage. As such, we sought to determine if the quality of image review on a handheld device is adequate for routine clinical use. Six CT/CTA cases and six MR/MRA cases were independently reviewed by three neuroradiologists in varying environments: high and low ambient light using a handheld device and on a traditional imaging workstation in ideal conditions. On first review (using a handheld device in high ambient light), a preliminary diagnosis for each case was made. Upon changes in review conditions, neuroradiologists were asked if any additional features were seen that changed their initial diagnoses. Reviewers were also asked to comment on overall clinical quality and if the handheld display was of acceptable quality for image review. After the initial CT review in high ambient light, additional findings were reported in 2 of 18 instances on subsequent reviews. Similarly, additional findings were identified in 4 of 18 instances after the initial MR review in high ambient lighting. Only one of these six additional findings contributed to the diagnosis made on the initial preliminary review. Use of a handheld device for image review is of adequate diagnostic quality based on image contrast, sharpness of structures, visible artefacts and overall display quality. Although reviewers were comfortable with using this technology, a handheld device with a larger screen may be diagnostically superior.

Expanding Group Peer Review: A Proposal for Medical Education Scholarship.

PubMed

Dumenco, Luba; Engle, Deborah L; Goodell, Kristen; Nagler, Alisa; Ovitsh, Robin K; Whicker, Shari A

2017-02-01

After participating in a group peer-review exercise at a workshop presented by Academic Medicine and MedEdPORTAL editors at the 2015 Association of American Medical Colleges Medical Education Meeting, the authors realized that the way their work group reviewed a manuscript was very different from the way by which they each would have reviewed the paper as an individual. Further, the group peer-review process yielded more robust feedback for the manuscript's authors than did the traditional individual peer-review process. This realization motivated the authors to reconvene and collaborate to write this Commentary to share their experience and propose the expanded use of group peer review in medical education scholarship.The authors consider the benefits of a peer-review process for reviewers, including learning how to improve their own manuscripts. They suggest that the benefits of a team review model may be similar to those of teamwork and team-based learning in medicine and medical education. They call for research to investigate this, to provide evidence to support group review, and to determine whether specific paper types would benefit most from team review (e.g., particularly complex manuscripts, those receiving widely disparate initial individual reviews). In addition, the authors propose ways in which a team-based approach to peer review could be expanded by journals and institutions. They believe that exploring the use of group peer review potentially could create a new methodology for skill development in research and scholarly writing and could enhance the quality of medical education scholarship.

The Influence of Peer Reviewer Expertise on the Evaluation of Research Funding Applications

PubMed Central

Sullivan, Joanne H.; Glisson, Scott R.

2016-01-01

Although the scientific peer review process is crucial to distributing research investments, little has been reported about the decision-making processes used by reviewers. One key attribute likely to be important for decision-making is reviewer expertise. Recent data from an experimental blinded review utilizing a direct measure of expertise has found that closer intellectual distances between applicant and reviewer lead to harsher evaluations, possibly suggesting that information is differentially sampled across subject-matter expertise levels and across information type (e.g. strengths or weaknesses). However, social and professional networks have been suggested to play a role in reviewer scoring. In an effort to test whether this result can be replicated in a real-world unblinded study utilizing self-assessed reviewer expertise, we conducted a retrospective multi-level regression analysis of 1,450 individual unblinded evaluations of 725 biomedical research funding applications by 1,044 reviewers. Despite the large variability in the scoring data, the results are largely confirmatory of work from blinded reviews, by which a linear relationship between reviewer expertise and their evaluations was observed—reviewers with higher levels of self-assessed expertise tended to be harsher in their evaluations. However, we also found that reviewer and applicant seniority could influence this relationship, suggesting social networks could have subtle influences on reviewer scoring. Overall, these results highlight the need to explore how reviewers utilize their expertise to gather and weight information from the application in making their evaluations. PMID:27768760

Methological quality of systematic reviews and meta-analyses on acupuncture for stroke: A review of review.

PubMed

Chen, Xin-Lin; Mo, Chuan-Wei; Lu, Li-Ya; Gao, Ri-Yang; Xu, Qian; Wu, Min-Feng; Zhou, Qian-Yi; Hu, Yue; Zhou, Xuan; Li, Xian-Tao

2017-11-01

To assess the methodological quality of systematic reviews and meta-analyses regarding acupuncture intervention for stroke and the primary studies within them. Two researchers searched PubMed, Cumulative index to Nursing and Allied Health Literature, Embase, ISI Web of Knowledge, Cochrane, Allied and Complementary Medicine, Ovid Medline, Chinese Biomedical Literature Database, China National Knowledge Infrastructure, Wanfang and Traditional Chinese Medical Database to identify systematic reviews and meta-analyses about acupuncture for stroke published from the inception to December 2016. Review characteristics and the criteria for assessing the primary studies within reviews were extracted. The methodological quality of the reviews was assessed using adapted Oxman and Guyatt Scale. The methodological quality of primary studies was also assessed. Thirty-two eligible reviews were identified, 15 in English and 17 in Chinese. The English reviews were scored higher than the Chinese reviews (P=0.025), especially in criteria for avoiding bias and the scope of search. All reviews used the quality criteria to evaluate the methodological quality of primary studies, but some criteria were not comprehensive. The primary studies, in particular the Chinese reviews, had problems with randomization, allocation concealment, blinding, dropouts and withdrawals, intent-to-treat analysis and adverse events. Important methodological flaws were found in Chinese systematic reviews and primary studies. It was necessary to improve the methodological quality and reporting quality of both the systematic reviews published in China and primary studies on acupuncture for stroke.

Reviewing Manuscripts for Biomedical Journals

PubMed Central

Garmel, Gus M

2010-01-01

Writing for publication is a complex task. For many professionals, producing a well-executed manuscript conveying one's research, ideas, or educational wisdom is challenging. Authors have varying emotions related to the process of writing for scientific publication. Although not studied, a relationship between an author's enjoyment of the writing process and the product's outcome is highly likely. As with any skill, practice generally results in improvements. Literature focused on preparing manuscripts for publication and the art of reviewing submissions exists. Most journals guard their reviewers' anonymity with respect to the manuscript review process. This is meant to protect them from direct or indirect author demands, which may occur during the review process or in the future. It is generally accepted that author identities are masked in the peer-review process. However, the concept of anonymity for reviewers has been debated recently; many editors consider it problematic that reviewers are not held accountable to the public for their decisions. The review process is often arduous and underappreciated, one reason why biomedical journals acknowledge editors and frequently recognize reviewers who donate their time and expertise in the name of science. This article describes essential elements of a submitted manuscript, with the hopes of improving scientific writing. It also discusses the review process within the biomedical literature, the importance of reviewers to the scientific process, responsibilities of reviewers, and qualities of a good review and reviewer. In addition, it includes useful insights to individuals who read and interpret the medical literature. PMID:20740129

Cluster Randomised Trials in Cochrane Reviews: Evaluation of Methodological and Reporting Practice.

PubMed

Richardson, Marty; Garner, Paul; Donegan, Sarah

2016-01-01

Systematic reviews can include cluster-randomised controlled trials (C-RCTs), which require different analysis compared with standard individual-randomised controlled trials. However, it is not known whether review authors follow the methodological and reporting guidance when including these trials. The aim of this study was to assess the methodological and reporting practice of Cochrane reviews that included C-RCTs against criteria developed from existing guidance. Criteria were developed, based on methodological literature and personal experience supervising review production and quality. Criteria were grouped into four themes: identifying, reporting, assessing risk of bias, and analysing C-RCTs. The Cochrane Database of Systematic Reviews was searched (2nd December 2013), and the 50 most recent reviews that included C-RCTs were retrieved. Each review was then assessed using the criteria. The 50 reviews we identified were published by 26 Cochrane Review Groups between June 2013 and November 2013. For identifying C-RCTs, only 56% identified that C-RCTs were eligible for inclusion in the review in the eligibility criteria. For reporting C-RCTs, only eight (24%) of the 33 reviews reported the method of cluster adjustment for their included C-RCTs. For assessing risk of bias, only one review assessed all five C-RCT-specific risk-of-bias criteria. For analysing C-RCTs, of the 27 reviews that presented unadjusted data, only nine (33%) provided a warning that confidence intervals may be artificially narrow. Of the 34 reviews that reported data from unadjusted C-RCTs, only 13 (38%) excluded the unadjusted results from the meta-analyses. The methodological and reporting practices in Cochrane reviews incorporating C-RCTs could be greatly improved, particularly with regard to analyses. Criteria developed as part of the current study could be used by review authors or editors to identify errors and improve the quality of published systematic reviews incorporating C-RCTs.

Academic Primer Series: Key Papers About Peer Review.

PubMed

Yarris, Lalena M; Gottlieb, Michael; Scott, Kevin; Sampson, Christopher; Rose, Emily; Chan, Teresa M; Ilgen, Jonathan

2017-06-01

Peer review, a cornerstone of academia, promotes rigor and relevance in scientific publishing. As educators are encouraged to adopt a more scholarly approach to medical education, peer review is becoming increasingly important. Junior educators both receive the reviews of their peers, and are also asked to participate as reviewers themselves. As such, it is imperative for junior clinician educators to be well-versed in the art of peer reviewing their colleagues' work. In this article, our goal was to identify and summarize key papers that may be helpful for faculty members interested in learning more about the peer-review process and how to improve their reviewing skills. The online discussions of the 2016-17 Academic Life in Emergency Medicine (ALiEM) Faculty Incubator program included a robust discussion about peer review, which highlighted a number of papers on that topic. We sought to augment this list with further suggestions by guest experts and by an open call on Twitter for other important papers. Via this process, we created a list of 24 total papers on the topic of peer review. After gathering these papers, our authorship group engaged in a consensus-building process incorporating Delphi methods to identify the papers that best described peer review, and also highlighted important tips for new reviewers. We found and reviewed 24 papers. In our results section, we present our authorship group's top five most highly rated papers on the topic of peer review. We also summarize these papers with respect to their relevance to junior faculty members and to faculty developers. We present five key papers on peer review that can be used for faculty development for novice writers and reviewers. These papers represent a mix of foundational and explanatory papers that may provide some basis from which junior faculty members might build upon as they both undergo the peer-review process and act as reviewers in turn.

Academic Primer Series: Key Papers About Peer Review

PubMed Central

Yarris, Lalena M.; Gottlieb, Michael; Scott, Kevin; Sampson, Christopher; Rose, Emily; Chan, Teresa M.; Ilgen, Jonathan

2017-01-01

Introduction Peer review, a cornerstone of academia, promotes rigor and relevance in scientific publishing. As educators are encouraged to adopt a more scholarly approach to medical education, peer review is becoming increasingly important. Junior educators both receive the reviews of their peers, and are also asked to participate as reviewers themselves. As such, it is imperative for junior clinician educators to be well-versed in the art of peer reviewing their colleagues’ work. In this article, our goal was to identify and summarize key papers that may be helpful for faculty members interested in learning more about the peer-review process and how to improve their reviewing skills. Methods The online discussions of the 2016–17 Academic Life in Emergency Medicine (ALiEM) Faculty Incubator program included a robust discussion about peer review, which highlighted a number of papers on that topic. We sought to augment this list with further suggestions by guest experts and by an open call on Twitter for other important papers. Via this process, we created a list of 24 total papers on the topic of peer review. After gathering these papers, our authorship group engaged in a consensus-building process incorporating Delphi methods to identify the papers that best described peer review, and also highlighted important tips for new reviewers. Results We found and reviewed 24 papers. In our results section, we present our authorship group’s top five most highly rated papers on the topic of peer review. We also summarize these papers with respect to their relevance to junior faculty members and to faculty developers. Conclusion We present five key papers on peer review that can be used for faculty development for novice writers and reviewers. These papers represent a mix of foundational and explanatory papers that may provide some basis from which junior faculty members might build upon as they both undergo the peer-review process and act as reviewers in turn. PMID:28611894

An overview and methodological assessment of systematic reviews and meta-analyses of enhanced recovery programmes in colorectal surgery

PubMed Central

Chambers, Duncan; Paton, Fiona; Wilson, Paul; Eastwood, Alison; Craig, Dawn; Fox, Dave; Jayne, David; McGinnes, Erika

2014-01-01

Objectives To identify and critically assess the extent to which systematic reviews of enhanced recovery programmes for patients undergoing colorectal surgery differ in their methodology and reported estimates of effect. Design Review of published systematic reviews. We searched the Cochrane Database of Systematic Reviews, the Database of Abstracts of Reviews of Effects (DARE) and Health Technology Assessment (HTA) Database from 1990 to March 2013. Systematic reviews of enhanced recovery programmes for patients undergoing colorectal surgery were eligible for inclusion. Primary and secondary outcome measures The primary outcome was length of hospital stay. We assessed changes in pooled estimates of treatment effect over time and how these might have been influenced by decisions taken by researchers as well as by the availability of new trials. The quality of systematic reviews was assessed using the Centre for Reviews and Dissemination (CRD) DARE critical appraisal process. Results 10 systematic reviews were included. Systematic reviews of randomised controlled trials have consistently shown a reduction in length of hospital stay with enhanced recovery compared with traditional care. The estimated effect tended to increase from 2006 to 2010 as more trials were published but has not altered significantly in the most recent review, despite the inclusion of several unique trials. The best estimate appears to be an average reduction of around 2.5 days in primary postoperative length of stay. Differences between reviews reflected differences in interpretation of inclusion criteria, searching and analytical methods or software. Conclusions Systematic reviews of enhanced recovery programmes show a high level of research waste, with multiple reviews covering identical or very similar groups of trials. Where multiple reviews exist on a topic, interpretation may require careful attention to apparently minor differences between reviews. Researchers can help readers by acknowledging existing reviews and through clear reporting of key decisions, especially on inclusion/exclusion and on statistical pooling. PMID:24879828

Indirect Comparisons: A Review of Reporting and Methodological Quality

PubMed Central

Donegan, Sarah; Williamson, Paula; Gamble, Carrol; Tudur-Smith, Catrin

2010-01-01

Background The indirect comparison of two interventions can be valuable in many situations. However, the quality of an indirect comparison will depend on several factors including the chosen methodology and validity of underlying assumptions. Published indirect comparisons are increasingly more common in the medical literature, but as yet, there are no published recommendations of how they should be reported. Our aim is to systematically review the quality of published indirect comparisons to add to existing empirical data suggesting that improvements can be made when reporting and applying indirect comparisons. Methodology/Findings Reviews applying statistical methods to indirectly compare the clinical effectiveness of two interventions using randomised controlled trials were eligible. We searched (1966–2008) Database of Abstracts and Reviews of Effects, The Cochrane library, and Medline. Full review publications were assessed for eligibility. Specific criteria to assess quality were developed and applied. Forty-three reviews were included. Adequate methodology was used to calculate the indirect comparison in 41 reviews. Nineteen reviews assessed the similarity assumption using sensitivity analysis, subgroup analysis, or meta-regression. Eleven reviews compared trial-level characteristics. Twenty-four reviews assessed statistical homogeneity. Twelve reviews investigated causes of heterogeneity. Seventeen reviews included direct and indirect evidence for the same comparison; six reviews assessed consistency. One review combined both evidence types. Twenty-five reviews urged caution in interpretation of results, and 24 reviews indicated when results were from indirect evidence by stating this term with the result. Conclusions This review shows that the underlying assumptions are not routinely explored or reported when undertaking indirect comparisons. We recommend, therefore, that the quality of indirect comparisons should be improved, in particular, by assessing assumptions and reporting the assessment methods applied. We propose that the quality criteria applied in this article may provide a basis to help review authors carry out indirect comparisons and to aid appropriate interpretation. PMID:21085712

Considering methodological options for reviews of theory: illustrated by a review of theories linking income and health

PubMed Central

2014-01-01

Background Review of theory is an area of growing methodological advancement. Theoretical reviews are particularly useful where the literature is complex, multi-discipline, or contested. It has been suggested that adopting methods from systematic reviews may help address these challenges. However, the methodological approaches to reviews of theory, including the degree to which systematic review methods can be incorporated, have received little discussion in the literature. We recently employed systematic review methods in a review of theories about the causal relationship between income and health. Methods This article discusses some of the methodological issues we considered in developing the review and offers lessons learnt from our experiences. It examines the stages of a systematic review in relation to how they could be adapted for a review of theory. The issues arising and the approaches taken in the review of theories in income and health are considered, drawing on the approaches of other reviews of theory. Results Different approaches to searching were required, including electronic and manual searches, and electronic citation tracking to follow the development of theories. Determining inclusion criteria was an iterative process to ensure that inclusion criteria were specific enough to make the review practical and focused, but not so narrow that key literature was excluded. Involving subject specialists was valuable in the literature searches to ensure principal papers were identified and during the inductive approaches used in synthesis of theories to provide detailed understanding of how theories related to another. Reviews of theory are likely to involve iterations and inductive processes throughout, and some of the concepts and techniques that have been developed for qualitative evidence synthesis can be usefully translated to theoretical reviews of this kind. Conclusions It may be useful at the outset of a review of theory to consider whether the key aim of the review is to scope out theories relating to a particular issue; to conduct in-depth analysis of key theoretical works with the aim of developing new, overarching theories and interpretations; or to combine both these processes in the review. This can help decide the most appropriate methodological approach to take at particular stages of the review. PMID:25312937

Evaluation of risk of bias assessment of trials in systematic reviews of oral health interventions, 1991-2014: A methodology study.

PubMed

Saltaji, Humam; Ospina, Maria B; Armijo-Olivo, Susan; Agarwal, Shruti; Cummings, Greta G; Amin, Maryam; Flores-Mir, Carlos

2016-09-01

The authors aimed to describe how often and by what means investigators assessed the risk of bias of clinical trials in systematic reviews of oral health interventions and to identify factors associated with risk of bias assessments. The authors selected therapeutic oral health systematic reviews published from 1991 through 2014. They extracted data related to the tools used for risk of bias assessment of primary studies and data related to other review characteristics. They descriptively analyzed the data and used multivariate logistic regression. The authors identified 1,114 oral health systematic reviews (130 Cochrane reviews and 984 non-Cochrane reviews). The investigators of the primary studies assessed risk of bias in 61.4% of the reviews, and the risk of bias assessments occurred more often in Cochrane reviews than in non-Cochrane reviews (100% versus 56.3%; P < .001) and in reviews published after the dissemination of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement (odds ratio [OR], 1.55; 95% confidence interval [CI], 1.17-2.06). Compared with the investigators of reviews of public oral health interventions, investigators of reviews of oral surgery were less likely to assess risk of bias (OR, 0.41; 95% CI, 0.25-0.67). Furthermore, the investigators of systematic reviews published in dental journals were less likely to assess risk of bias of individual trials (OR, 0.28; 95% CI, 0.19-0.41) compared with the investigators of reviews published in nondental journals. The investigators of primary studies did not undertake risk of bias assessment in a considerable portion of non-Cochrane oral health systematic reviews. The investigators of reviews published in dental journals were less likely to assess risk of bias than the investigators of reviews published in nondental journals. The results of this study provide evidence of the need for improving the conduct and reporting of oral health systematic reviews with respect to risk of bias assessment. Clinicians should determine to what extent the findings of a systematic review are valid on the basis of whether the investigators assessed and considered risk of bias during the interpretation of findings. Copyright © 2016 American Dental Association. Published by Elsevier Inc. All rights reserved.

Interventions for Age-Related Macular Degeneration: Are Practice Guidelines Based on Systematic Reviews?

PubMed

Lindsley, Kristina; Li, Tianjing; Ssemanda, Elizabeth; Virgili, Gianni; Dickersin, Kay

2016-04-01

Are existing systematic reviews of interventions for age-related macular degeneration incorporated into clinical practice guidelines? High-quality systematic reviews should be used to underpin evidence-based clinical practice guidelines and clinical care. We examined the reliability of systematic reviews of interventions for age-related macular degeneration (AMD) and described the main findings of reliable reviews in relation to clinical practice guidelines. Eligible publications were systematic reviews of the effectiveness of treatment interventions for AMD. We searched a database of systematic reviews in eyes and vision without language or date restrictions; the database was up to date as of May 6, 2014. Two authors independently screened records for eligibility and abstracted and assessed the characteristics and methods of each review. We classified reviews as reliable when they reported eligibility criteria, comprehensive searches, methodologic quality of included studies, appropriate statistical methods for meta-analysis, and conclusions based on results. We mapped treatment recommendations from the American Academy of Ophthalmology (AAO) Preferred Practice Patterns (PPPs) for AMD to systematic reviews and citations of reliable systematic reviews to support each treatment recommendation. Of 1570 systematic reviews in our database, 47 met inclusion criteria; most targeted neovascular AMD and investigated anti-vascular endothelial growth factor (VEGF) interventions, dietary supplements, or photodynamic therapy. We classified 33 (70%) reviews as reliable. The quality of reporting varied, with criteria for reliable reporting met more often by Cochrane reviews and reviews whose authors disclosed conflicts of interest. Anti-VEGF agents and photodynamic therapy were the only interventions identified as effective by reliable reviews. Of 35 treatment recommendations extracted from the PPPs, 15 could have been supported with reliable systematic reviews; however, only 1 recommendation cited a reliable intervention systematic review. No reliable systematic review was identified for 20 treatment recommendations, highlighting areas of evidence gaps. For AMD, reliable systematic reviews exist for many treatment recommendations in the AAO PPPs and should be cited to support these recommendations. We also identified areas where no high-level evidence exists. Mapping clinical practice guidelines to existing systematic reviews is one way to highlight areas where evidence generation or evidence synthesis is either available or needed. Copyright © 2016 American Academy of Ophthalmology. Published by Elsevier Inc. All rights reserved.

Be an NIH Reviewer: Contribute to Multidisciplinary Research.

PubMed

Jenkins, Melinda L

2018-03-01

One of the best ways to contribute to multidisciplinary research and to improve your own knowledge of the review process at the National Institutes of Health (NIH) is to serve as a peer reviewer for research, traineeship, and small business innovation research proposals. Proactive targeted outreach to Scientific Review Officers (SROs) at NIH will increase your chances to become a reviewer. Reviewers with nursing expertise are especially welcome as multidisciplinary research is becoming more prevalent. Steps to identify a likely study section, contact the correct SRO, and review responsibly are described in this article, written by an experienced NIH review officer.

Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement.

PubMed

Moher, David; Shamseer, Larissa; Clarke, Mike; Ghersi, Davina; Liberati, Alessandro; Petticrew, Mark; Shekelle, Paul; Stewart, Lesley A

2015-01-01

Systematic reviews should build on a protocol that describes the rationale, hypothesis, and planned methods of the review; few reviews report whether a protocol exists. Detailed, well-described protocols can facilitate the understanding and appraisal of the review methods, as well as the detection of modifications to methods and selective reporting in completed reviews. We describe the development of a reporting guideline, the Preferred Reporting Items for Systematic reviews and Meta-Analyses for Protocols 2015 (PRISMA-P 2015). PRISMA-P consists of a 17-item checklist intended to facilitate the preparation and reporting of a robust protocol for the systematic review. Funders and those commissioning reviews might consider mandating the use of the checklist to facilitate the submission of relevant protocol information in funding applications. Similarly, peer reviewers and editors can use the guidance to gauge the completeness and transparency of a systematic review protocol submitted for publication in a journal or other medium.

On the evolving open peer review culture for chemical information science.

PubMed

Walters, W Patrick; Bajorath, Jürgen

2015-01-01

Compared to the traditional anonymous peer review process, open post-publication peer review provides additional opportunities -and challenges- for reviewers to judge scientific studies. In this editorial, we comment on the open peer review culture and provide some guidance for reviewers of manuscripts submitted to the Chemical Information Science channel of F1000Research.

How to Become a More Effective Reviewer

ERIC Educational Resources Information Center

Snyder, Kate E.

2018-01-01

Effective peer review is a critical element of the scientific process, but it is one that can be daunting for new reviewers. In this article, the author shares guidelines she has learned on how to review, based on her personal process and the literature on reviewing, in the hopes that new and experienced reviewers alike can use these…

The Law Review Approach: What the Humanities Can Learn

ERIC Educational Resources Information Center

Mendenhall, Allen

2013-01-01

Readers of this journal probably know how the peer review process works in the humanities disciplines and at various journals. Therefore the author explains how the law review process generally works and then what the humanities can learn and borrow from the law review process. He ends by advocating for a hybrid law review/peer review approach to…

IRIS Toxicological Review of Hexahydro-1,3,5-Trinitro-1,3,5-Triazine (RDX) (Interagency Science Consultation Draft)

EPA Science Inventory

On March 10, 2016, the public comment draft Toxicological Review of Hexahydro-1,3,5-trinitro-1,3,5-triazine and the draft charge to external peer reviewers were released for public review and comment. The Toxicological Review and charge were reviewed internally by EPA and by othe...

Meta-Review: Systematic Assessment of Program Review

ERIC Educational Resources Information Center

Harlan, Brian

2012-01-01

Over 20 years ago, Robert J. Barak and Barbara E. Breier suggested incorporating a regular assessment of the entire program review system into the review schedule in order to ensure that the system itself is as efficient and effective as the programs under review. Barak and Breier's seminal book on the goals and processes of program review has…

19 CFR 174.23 - Further review of protests.

Code of Federal Regulations, 2010 CFR

2010-04-01

... 19 Customs Duties 2 2010-04-01 2010-04-01 false Further review of protests. 174.23 Section 174.23... TREASURY (CONTINUED) PROTESTS Review and Disposition of Protests § 174.23 Further review of protests. A protesting party may seek further review of a protest in lieu of review by the port director by filing, on...

34 CFR 5.85 - Decisions on review.

Code of Federal Regulations, 2010 CFR

2010-07-01

.... L. 90-23 (Eff. until 7-14-10) Administrative Review § 5.85 Decisions on review. (a) Decisions on review shall be in writing within 20 working days from receipt of the request for review. Extension of... 34 Education 1 2010-07-01 2010-07-01 false Decisions on review. 5.85 Section 5.85 Education Office...

14 CFR 385.33 - Review by the staff.

Code of Federal Regulations, 2013 CFR

2013-01-01

... 14 Aeronautics and Space 4 2013-01-01 2013-01-01 false Review by the staff. 385.33 Section 385.33...) ORGANIZATION STAFF ASSIGNMENTS AND REVIEW OF ACTION UNDER ASSIGNMENTS Procedure on Review of Staff Action § 385.33 Review by the staff. Where a petition for review is duly filed, the staff member may, upon...

14 CFR 385.33 - Review by the staff.

Code of Federal Regulations, 2010 CFR

2010-01-01

... 14 Aeronautics and Space 4 2010-01-01 2010-01-01 false Review by the staff. 385.33 Section 385.33...) ORGANIZATION STAFF ASSIGNMENTS AND REVIEW OF ACTION UNDER ASSIGNMENTS Procedure on Review of Staff Action § 385.33 Review by the staff. Where a petition for review is duly filed, the staff member may, upon...

14 CFR 385.33 - Review by the staff.

Code of Federal Regulations, 2012 CFR

2012-01-01

... 14 Aeronautics and Space 4 2012-01-01 2012-01-01 false Review by the staff. 385.33 Section 385.33...) ORGANIZATION STAFF ASSIGNMENTS AND REVIEW OF ACTION UNDER ASSIGNMENTS Procedure on Review of Staff Action § 385.33 Review by the staff. Where a petition for review is duly filed, the staff member may, upon...

14 CFR 385.33 - Review by the staff.

Code of Federal Regulations, 2011 CFR

2011-01-01

... 14 Aeronautics and Space 4 2011-01-01 2011-01-01 false Review by the staff. 385.33 Section 385.33...) ORGANIZATION STAFF ASSIGNMENTS AND REVIEW OF ACTION UNDER ASSIGNMENTS Procedure on Review of Staff Action § 385.33 Review by the staff. Where a petition for review is duly filed, the staff member may, upon...

14 CFR 385.33 - Review by the staff.

Code of Federal Regulations, 2014 CFR

2014-01-01

... 14 Aeronautics and Space 4 2014-01-01 2014-01-01 false Review by the staff. 385.33 Section 385.33...) ORGANIZATION STAFF ASSIGNMENTS AND REVIEW OF ACTION UNDER ASSIGNMENTS Procedure on Review of Staff Action § 385.33 Review by the staff. Where a petition for review is duly filed, the staff member may, upon...

15 CFR 752.14 - System reviews.

Code of Federal Regulations, 2010 CFR

2010-01-01

...) Other reviews. BIS may require an SCL holder or consignee to submit to its office a list of all sales... 15 Commerce and Foreign Trade 2 2010-01-01 2010-01-01 false System reviews. 752.14 Section 752.14... LICENSE § 752.14 System reviews. (a) Post-license system reviews. BIS may conduct system reviews of the...

The golden rule of reviewing

USGS Publications Warehouse

McPeek, Mark A.; DeAngelis, Donald L.; Shaw, Ruth G.; Moore, Allen J.; Rausher, Mark D.; Strong, Donald R.; Ellison, Aaron M.; Barrett, Louise; Rieseberg, Loren; Breed, Michael D.; Sullivan, Jack; Osenberg, Craig W.; Holyoak, Marcel; Elgar, Mark A.

2009-01-01

A major bottleneck in the time required to publish a scientific or scholarly paper is the speed with which reviews by peers are returned to journals. Peer review is a reciprocal altruistic system in which each individual may perform every task—editors, reviewers, and authors—at different times. Journals have no way to coerce reviewers to return their critiques faster. To greatly shorten the time to publication, all actors in this altruistic network should abide by the Golden Rule of Reviewing: review for others as you would have others review for you. Say yes to reviewing whenever your duties and schedule allow; provide a thorough, fair, and constructive critique of the work; and do it at your first opportunity regardless of the deadline.

A systematic review of the quality of conduct and reporting of systematic reviews and meta-analyses in paediatric surgery

PubMed Central

Gudlaugsdottir, Katrin; Andrews, James

2017-01-01

Objective Our objective was to evaluate quality of conduct and reporting of published systematic reviews and meta-analyses in paediatric surgery. We also aimed to identify characteristics predictive of review quality. Background Systematic reviews summarise evidence by combining sources, but are potentially prone to bias. To counter this, the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) was published to aid in reporting. Similarly, the Assessing the Methodological Quality of Systematic Reviews (AMSTAR) measurement tool was designed to appraise methodology. The paediatric surgical literature has seen an increasing number of reviews over the past decade, but quality has not been evaluated. Methods Adhering to PRISMA guidelines, we performed a systematic review with a priori design to identify systematic reviews and meta-analyses of interventions in paediatric surgery. From 01/2010 to 06/2016, we searched: MEDLINE, EMBASE, Cochrane, Centre for Reviews and Dissemination, Web of Science, Google Scholar, reference lists and journals. Two reviewers independently selected studies and extracted data. We assessed conduct and reporting using AMSTAR and PRISMA. Scores were calculated as the sum of reported items. We also extracted author, journal and article characteristics, and used them in exploratory analysis to determine which variables predict quality. Results 112 articles fulfilled eligibility criteria (53 systematic reviews; 59 meta-analyses). Overall, 68% AMSTAR and 56.8% PRISMA items were reported adequately. Poorest scores were identified with regards a priori design, inclusion of structured summaries, including the grey literature, citing excluded articles and evaluating bias. 13 reviews were pre-registered and 6 in PRISMA-endorsing journals. The following predicted quality in univariate analysis:, word count, Cochrane review, journal h-index, impact factor, journal endorses PRISMA, PRISMA adherence suggested in author guidance, article mentions PRISMA, review includes comparison of interventions and review registration. The latter three variables were significant in multivariate regression. Conclusions There are gaps in the conduct and reporting of systematic reviews in paediatric surgery. More endorsement by journals of the PRISMA guideline may improve review quality, and the dissemination of reliable evidence to paediatric clinicians. PMID:28384296

Towards core outcome set (COS) development: a follow-up descriptive survey of outcomes in Cochrane reviews.

PubMed

Wuytack, Francesca; Smith, Valerie; Clarke, Mike; Williamson, Paula; Gargon, Elizabeth

2015-05-19

A core outcome set (COS) can address problems of outcome heterogeneity and outcome reporting bias in trials and systematic reviews, including Cochrane reviews, helping to reduce waste. One of the aims of the international Core Outcome Measures in Effectiveness Trials (COMET) Initiative is to link the development and use of COS with the outcomes specified and reported in Cochrane reviews, including the outcomes listed in the summary of findings (SoF) tables. As part of this work, an earlier exploratory survey of the outcomes of newly published 2007 and 2011 Cochrane reviews was performed. This survey examined the use of COS, the variety of specified outcomes, and outcome reporting in Cochrane reviews by Cochrane Review Group (CRG). To examine changes over time and to explore outcomes that were repeatedly specified over time in Cochrane reviews by CRG, we conducted a follow-up survey of outcomes in 2013 Cochrane reviews. A descriptive survey of outcomes in Cochrane reviews that were first published in 2013. Outcomes specified in the methods sections and reported in the results section of the Cochrane reviews were examined by CRG. We also explored the uptake of SoF tables, the number of outcomes included in these, and the quality of the evidence for the outcomes. Across the 50 CRGs, 375 Cochrane reviews that included at least one study specified a total of 3142 outcomes. Of these outcomes, 32 % (1008) were not reported in the results section of these reviews. For 23 % (233) of these non-reported outcomes, we did not find any reason in the text of the review for this non-report. Fifty-seven percent (216/375) of reviews included a SoF table. The proportion of specified outcomes that were reported in Cochrane reviews had increased in 2013 (68 %) compared to 2007 (61 %) and 2011 (65 %). Importantly, 2013 Cochrane reviews that did not report specified outcomes were twice as likely to provide an explanation for why the outcome was not reported. There has been an increased uptake of SoF tables in Cochrane reviews. Outcomes that were repeatedly specified in Cochrane reviews by CRG in 2007, 2011, and 2013 may assist COS development.

Salutogenically focused outcomes in systematic reviews of intrapartum interventions: a systematic review of systematic reviews.

PubMed

Smith, Valerie; Daly, Deirdre; Lundgren, Ingela; Eri, Tine; Benstoem, Carina; Devane, Declan

2014-04-01

research on intrapartum interventions in maternity care has focused traditionally on the identification of risk factors' and on the reduction of adverse outcomes with less attention given to the measurement of factors that contribute to well-being and positive health outcomes. We conducted a systematic review of reviews to determine the type and number of salutogenically-focused reported outcomes in current maternity care intrapartum intervention-based research. For the conduct of this review, we interpreted salutogenic outcomes as those relating to optimum and/or positive maternal and neonatal health and well-being. to identify salutogenically-focused outcomes reported in systematic reviews of randomised trials of intrapartum interventions. we searched Issue 9 (September) 2011 of the Cochrane Database of Systematic Reviews for all reviews of intrapartum interventions published by the Cochrane Pregnancy and Childbirth Group using the group filter "hm-preg". Systematic reviews of randomised trials of intrapartum interventions were eligible for inclusion. We excluded protocols for systematic reviews and systematic reviews that had been withdrawn. Outcome data were extracted independently from each included review by at least two review authors. Unique lists of salutogenically and non-salutogenically focused outcomes were established. 16 salutogenically-focused outcome categories were identified in 102 included reviews. Maternal satisfaction and breast feeding were reported most frequently. 49 non-salutogenically-focused outcome categories were identified in the 102 included reviews. Measures of neonatal morbidity were reported most frequently. there is an absence of salutogenically-focused outcomes reported in intrapartum intervention-based research. We recommend the development of a core outcome data set of salutogenically-focused outcomes for intrapartum research. © 2013 Published by Elsevier Ltd.

ROBIS: A new tool to assess risk of bias in systematic reviews was developed

PubMed Central

Whiting, Penny; Savović, Jelena; Higgins, Julian P.T.; Caldwell, Deborah M.; Reeves, Barnaby C.; Shea, Beverley; Davies, Philippa; Kleijnen, Jos; Churchill, Rachel

2016-01-01

Objective To develop ROBIS, a new tool for assessing the risk of bias in systematic reviews (rather than in primary studies). Study Design and Setting We used four-stage approach to develop ROBIS: define the scope, review the evidence base, hold a face-to-face meeting, and refine the tool through piloting. Results ROBIS is currently aimed at four broad categories of reviews mainly within health care settings: interventions, diagnosis, prognosis, and etiology. The target audience of ROBIS is primarily guideline developers, authors of overviews of systematic reviews (“reviews of reviews”), and review authors who might want to assess or avoid risk of bias in their reviews. The tool is completed in three phases: (1) assess relevance (optional), (2) identify concerns with the review process, and (3) judge risk of bias. Phase 2 covers four domains through which bias may be introduced into a systematic review: study eligibility criteria; identification and selection of studies; data collection and study appraisal; and synthesis and findings. Phase 3 assesses the overall risk of bias in the interpretation of review findings and whether this considered limitations identified in any of the phase 2 domains. Signaling questions are included to help judge concerns with the review process (phase 2) and the overall risk of bias in the review (phase 3); these questions flag aspects of review design related to the potential for bias and aim to help assessors judge risk of bias in the review process, results, and conclusions. Conclusions ROBIS is the first rigorously developed tool designed specifically to assess the risk of bias in systematic reviews. PMID:26092286

Compliance of systematic reviews in veterinary journals with Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) literature search reporting guidelines.

PubMed

Toews, Lorraine C

2017-07-01

Complete, accurate reporting of systematic reviews facilitates assessment of how well reviews have been conducted. The primary objective of this study was to examine compliance of systematic reviews in veterinary journals with Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines for literature search reporting and to examine the completeness, bias, and reproducibility of the searches in these reviews from what was reported. The second objective was to examine reporting of the credentials and contributions of those involved in the search process. A sample of systematic reviews or meta-analyses published in veterinary journals between 2011 and 2015 was obtained by searching PubMed. Reporting in the full text of each review was checked against certain PRISMA checklist items. Over one-third of reviews (37%) did not search the CAB Abstracts database, and 9% of reviews searched only 1 database. Over two-thirds of reviews (65%) did not report any search for grey literature or stated that they excluded grey literature. The majority of reviews (95%) did not report a reproducible search strategy. Most reviews had significant deficiencies in reporting the search process that raise questions about how these searches were conducted and ultimately cast serious doubts on the validity and reliability of reviews based on a potentially biased and incomplete body of literature. These deficiencies also highlight the need for veterinary journal editors and publishers to be more rigorous in requiring adherence to PRISMA guidelines and to encourage veterinary researchers to include librarians or information specialists on systematic review teams to improve the quality and reporting of searches.

An international survey and modified Delphi approach revealed numerous rapid review methods.

PubMed

Tricco, Andrea C; Zarin, Wasifa; Antony, Jesmin; Hutton, Brian; Moher, David; Sherifali, Diana; Straus, Sharon E

2016-02-01

To solicit experiences with and perceptions of rapid reviews from stakeholders, including researchers, policy makers, industry, journal editors, and health care providers. An international survey of rapid review producers and modified Delphi. Forty rapid review producers responded on our survey (63% response rate). Eighty-eight rapid reviews with 31 different names were reported. Rapid review commissioning organizations were predominantly government (78%) and health care (58%) organizations. Several rapid review approaches were identified, including updating the literature search of previous reviews (92%); limiting the search strategy by date of publication (88%); and having only one reviewer screen (85%), abstract data (84%), and assess the quality of studies (86%). The modified Delphi included input from 113 stakeholders on the rapid review approaches from the survey. Approach 1 (search limited by date and language; study selection by one reviewer only, and data abstraction and quality appraisal conducted by one reviewer and one verifier) was ranked the most feasible (72%, 81/113 responses), with the lowest perceived risk of bias (12%, 12/103); it also ranked second in timeliness (37%, 38/102) and fifth in comprehensiveness (5%, 5/100). Rapid reviews have many names and approaches, and some methods might be more desirable than others. Copyright © 2016 Elsevier Inc. All rights reserved.

A scoping review of scoping reviews: advancing the approach and enhancing the consistency

PubMed Central

Pham, Mai T; Rajić, Andrijana; Greig, Judy D; Sargeant, Jan M; Papadopoulos, Andrew; McEwen, Scott A

2014-01-01

Background The scoping review has become an increasingly popular approach for synthesizing research evidence. It is a relatively new approach for which a universal study definition or definitive procedure has not been established. The purpose of this scoping review was to provide an overview of scoping reviews in the literature. Methods A scoping review was conducted using the Arksey and O'Malley framework. A search was conducted in four bibliographic databases and the gray literature to identify scoping review studies. Review selection and characterization were performed by two independent reviewers using pretested forms. Results The search identified 344 scoping reviews published from 1999 to October 2012. The reviews varied in terms of purpose, methodology, and detail of reporting. Nearly three-quarter of reviews (74.1%) addressed a health topic. Study completion times varied from 2 weeks to 20 months, and 51% utilized a published methodological framework. Quality assessment of included studies was infrequently performed (22.38%). Conclusions Scoping reviews are a relatively new but increasingly common approach for mapping broad topics. Because of variability in their conduct, there is a need for their methodological standardization to ensure the utility and strength of evidence. © 2014 The Authors. Research Synthesis Methods published by John Wiley & Sons, Ltd. PMID:26052958

Manual search approaches used by systematic reviewers in dermatology.

PubMed

Vassar, Matt; Atakpo, Paul; Kash, Melissa J

2016-10-01

Manual searches are supplemental approaches to database searches to identify additional primary studies for systematic reviews. The authors argue that these manual approaches, in particular hand-searching and perusing reference lists, are often considered the same yet lead to different outcomes. We conducted a PubMed search for systematic reviews in the top 10 dermatology journals (January 2006-January 2016). After screening, the final sample comprised 292 reviews. Statements related to manual searches were extracted from each review and categorized by the primary and secondary authors. Each statement was categorized as either "Search of Reference List," "Hand Search," "Both," or "Unclear." Of the 292 systematic reviews included in our sample, 143 reviews (48.97%) did not report a hand-search or scan of reference lists. One-hundred thirty-six reviews (46.58%) reported searches of reference lists, while 4 reviews (1.37%) reported systematic hand-searches. Three reviews (1.03%) reported use of both hand-searches and scanning reference lists. Six reviews (2.05%) were classified as unclear due to vague wording. Authors of systematic reviews published in dermatology journals in our study sample scanned reference lists more frequently than they conducted hand-searches, possibly contributing to biased search outcomes. We encourage systematic reviewers to routinely practice hand-searching in order to minimize bias.

Systematic reviews addressing microsurgical head and neck reconstruction.

PubMed

Momeni, Arash; Jacobson, Joshua Y; Lee, Gordon K

2015-01-01

Systematic reviews frequently form the basis for clinical decision making and guideline development. Yet, the quality of systematic reviews has been variable, thus raising concerns about the validity of their conclusions. In the current study, a quality analysis of systematic reviews was performed, addressing microsurgical head and neck reconstruction. A PubMed search was performed to identify all systematic reviews published up to and including December 2012 in 12 surgical journals. Two authors independently reviewed the literature and extracted data from the included reviews. Discrepancies were resolved by consensus. Quality assessment was performed using AMSTAR. The initial search retrieved 1020 articles. After screening titles and abstracts, 987 articles were excluded. Full-text review of the remaining 33 articles resulted in further exclusion of 18 articles, leaving 15 systematic reviews for final analysis. A marked increase in the number of published systematic reviews over time was noted (P = 0.07). The median AMSTAR score was 5, thus reflecting a "fair" quality. No evidence for improvement in methodological quality over time was noted. The trend to publish more systematic reviews in microsurgical head and neck reconstruction is encouraging. However, efforts are indicated to improve the methodological quality of systematic reviews. Familiarity with criteria of methodological quality is critical to ensure future improvements in the quality of systematic reviews conducted in microsurgery.

Editors’ Perspectives on Enhancing Manuscript Quality and Editorial Decisions Through Peer Review and Reviewer Development

PubMed Central

Janke, Kristin K.; Traynor, Andrew P.

2017-01-01

Objectives. To identify peer reviewer and peer review characteristics that enhance manuscript quality and editorial decisions, and to identify valuable elements of peer reviewer training programs. Methods. A three-school, 15-year review of pharmacy practice and pharmacy administration faculty’s publications was conducted to identify high-publication volume journals for inclusion. Editors-in-chief identified all editors managing manuscripts for participation. A three-round modified Delphi process was used. Rounds advanced from open-ended questions regarding actions and attributes of good reviewers to consensus-seeking and clarifying questions related to quality, importance, value, and priority. Results. Nineteen editors representing eight pharmacy journals participated. Three characteristics of reviews were rated required or helpful in enhancing manuscript quality by all respondents: includes a critical analysis of the manuscript (88% required, 12% helpful), includes feedback that contains both strengths and areas of improvement (53% required, 47% helpful), and speaks to the manuscript’s utility in the literature (41% required, 59% helpful). Hands-on experience with review activities (88%) and exposure to good and bad reviews (88%) were identified as very valuable to peer reviewer development. Conclusion. Reviewers, individuals involved in faculty development, and journals should work to assist new reviewers in defining focused areas of expertise, building knowledge in these areas, and developing critical analysis skills. PMID:28630514

Consistency and accuracy of indexing systematic review articles and meta-analyses in medline.

PubMed

Wilczynski, Nancy L; Haynes, R Brian

2009-09-01

Systematic review articles support the advance of science and translation of research evidence into healthcare practice. Inaccurate retrieval from medline could limit access to reviews. To determine the quality of indexing systematic reviews and meta-analyses in medline. The Clinical Hedges Database, containing the results of a hand search of 161 journals, was used to test medline indexing terms for their ability to retrieve systematic reviews that met predefined methodologic criteria (labelled as 'pass' review articles) and reviews that reported a meta-analysis. The Clinical Hedges Database contained 49 028 articles; 753 were 'pass' review articles (552 with a meta-analysis). In total 758 review articles (independent of whether they passed) reported a meta-analysis. The search strategy that retrieved the highest number of 'pass' systematic reviews achieved a sensitivity of 97.1%. The publication type 'meta analysis' had a false positive rate of 5.6% (95% CI 3.9 to 7.6), and false negative rate of 0.31% (95% CI 0.26 to 0.36) for retrieving systematic reviews that reported a meta-analysis. Inaccuracies in indexing systematic reviews and meta-analyses in medline can be partly overcome by a 5-term search strategy. Introducing a publication type for systematic reviews of the literature could improve retrieval performance.

Impartial judgment by the "gatekeepers" of science: fallibility and accountability in the peer review process.

PubMed

Hojat, Mohammadreza; Gonnella, Joseph S; Caelleigh, Addeane S

2003-01-01

High publication demands and the low acceptance rate of peer review journals place the journal editors and their reviewers in a powerful position. Journal reviewers have a vital role not only in influencing the journal editor's publication decisions, but also in the very nature and direction of scientific research. Because of their influence in peer review outcomes, journal reviewers are aptly described as the "gatekeepers of science." In this article we describe several pitfalls that can impede reviewers' impartial judgement. These include such issues as confirmatory bias, the negative results bias (the file drawer problem), the Matthew effect, the Doctor Fox effect, and gender, race, theoretical orientation, and "political correctness." We argue that procedures currently used by many professional journals, such as blind or masked review, may not completely alleviate the effects of these pitfalls. Instead, we suggest that increasing reviewers' awareness of the pitfalls, accountability, and vigilance can improve fairness in the peer review process. The ultimate responsibilities belong to the journal editors who are confronted with the difficult task of satisfying journal readers, contributors, reviewers, and owners. We recommend that the journal editors conduct periodic internal and external evaluations of their journals' peer review process and outcomes, with participation of reviewers, contributors, readers and owners.

77 FR 60146 - SES Performance Review Board

Federal Register 2010, 2011, 2012, 2013, 2014

2012-10-02

... NATIONAL TRANSPORTATION SAFETY BOARD SES Performance Review Board AGENCY: National Transportation... National Transportation Safety Board, Performance Review Board (PRB). FOR FURTHER INFORMATION CONTACT... Performance Review Boards. The board reviews and evaluates the initial appraisal of a senior executive's...

76 FR 70169 - SES Performance Review Board

Federal Register 2010, 2011, 2012, 2013, 2014

2011-11-10

... NATIONAL TRANSPORTATION SAFETY BOARD SES Performance Review Board AGENCY: National Transportation... National Transportation Safety Board, Performance Review Board (PRB). FOR FURTHER INFORMATION CONTACT... Performance Review Boards. The board reviews and evaluates the initial appraisal of a senior executive's...

IRIS Toxicological Review of Benzo[a]pyrene (Interagency ...

EPA Pesticide Factsheets

On August 21, 2013, the draft Toxicological Review of Benzo[a]pyrene and the draft charge to external peer reviewers were released for public review and comment. The Toxicological Review and charge were reviewed internally by EPA and by other federal agencies and White House Offices before public release. Consistent with the May 2009 IRIS assessment development process, all written comments on IRIS assessments submitted by other federal agencies and White House Offices are made publicly available. Accordingly, interagency comments and the interagency science consultation materials provided to other agencies, including interagency review drafts of the IRIS Toxicological Review of Benzo[a]pyrene and the charge to external peer reviewers, are posted on this site. EPA is undertaking an update of the Integrated Risk Information System (IRIS) health assessment for benzo[a]pyrene (BaP). The outcome of this project is an updated Toxicological Review and IRIS Summary for BaP that will be entered into the IRIS database.

Conclusiveness of the Cochrane reviews in gynaecological cancer: A systematic analysis.

PubMed

Yin, Shande; Chuai, Yunhai; Wang, Aiming; Zhang, Lanmei

2015-06-01

To assess the conclusiveness of Cochrane reviews in the field of gynaecological cancer. The Cochrane Library was searched for reviews regarding gynaecological cancer published between 1 January 2000 and 1 November 2014. Data were extracted from each paper and the conclusiveness of each review was assessed. The study included 66 reviews, 41 (62.1%) of which were conclusive. Of these, 58 included randomized controlled trials (RCTs), 37 (63.8%) of which were conclusive. Conclusive reviews of RCTs included significantly more patients than inconclusive reviews, but there was no difference in the number of included studies. Of the eight reviews of nonrandomized studies, four (50.0%) were conclusive. The majority of reviews recognized the need for additional studies. In the field of gynaecological cancer, reviews are more likely to be conclusive when they include RCTs, as well as large numbers of patients. © The Author(s) 2015 Reprints and permissions: sagepub.co.uk/journalsPermissions.nav.

IRIS Toxicological Review of Inorganic Arsenic (Cancer) ...

EPA Pesticide Factsheets

On February 19, 2010, the draft IRIS Toxicological Review of Inorganic Arsenic (Cancer) external review draft document and the charge to external peer reviewers were released for public review and comment. The draft document and the charge to external peer reviewers were reviewed internally by EPA and by other federal agencies and White House Offices before public release. In the new IRIS process, introduced by the EPA Administrator, all written comments on IRIS assessments submitted by other federal agencies and White House Offices will be made publicly available. Accordingly, interagency comments and the interagency science consultation draft of the Toxicological Review of Inorganic Arsenic and the charge to external peer reviewers are posted on this site. This draft IRIS health assessment addresses only cancer human health effects that may result from chronic exposure to this chemical. An assessment of noncancer health effects of inorganic arsenic will be released for external peer review and public comment at a later date.

External Peer Review Report on the Defense Contract Management Agency Office of Independent Assessment Internal Review Team

DTIC Science & Technology

2015-11-02

External Peer Review Report on the Defense Contract Management Agency Office of Independent Assessment Internal Review Team N O V E M B E R 2 , 2 0 1 5... MANAGEMENT AGENCY SUBJECT: External Peer Review Report on the Defense Contract Management Agency Office of Independent Assessment Internal Review Team...Report No. DODIG-2016-007) Attached is the External Peer Review Report on the Defense Contract Management Agency Office of Independent Assessment

Same family, different species: methodological conduct and quality varies according to purpose for five types of knowledge synthesis.

PubMed

Tricco, Andrea C; Zarin, Wasifa; Ghassemi, Marco; Nincic, Vera; Lillie, Erin; Page, Matthew J; Shamseer, Larissa; Antony, Jesmin; Rios, Patricia; Hwee, Jeremiah; Veroniki, Areti Angeliki; Moher, David; Hartling, Lisa; Pham, Ba'; Straus, Sharon E

2018-04-01

The aim of the study was to characterize methodological conduct, reporting, and quality of five knowledge synthesis (KS) approaches. Retrospective analysis of a convenience sample of five published databases of KS approaches: overview of reviews (n = 74), scoping reviews (n = 494), rapid reviews (n = 84), systematic reviews (n = 300), and network meta-analyses (NMAs; n = 456). Data in the five published databases were abstracted by two reviewers independently, any missing data for this retrospective analysis were abstracted by one experienced reviewer. Methods were appraised using the A MeaSurement Tool to Assess systematic Reviews (AMSTAR) tool. Descriptive analysis was performed. Reporting the use of a protocol ranged from 4% for rapid reviews to 32% for systematic reviews. The use of two reviewers for citation and full-text screening ranged from 20% for scoping reviews to 60% for NMAs. Data abstraction was performed in duplicate for 11% of rapid reviews and 54% of NMAs, and for risk of bias appraisal, this ranged from 6% for scoping reviews to 41% for NMAs. NMAs had the highest median percentage of maximum obtainable AMSTAR score (64%; Q1-Q3:45-73%), while scoping reviews had the lowest (25%; Q1-Q3:13-38%). NMAs consistently scored the highest on the AMSTAR tool likely because the purpose is to estimate treatment effects statistically. Scoping reviews scored the lowest (even after adjusting the score for not relevant items) likely because the purpose is to characterize the literature. Copyright © 2017 Elsevier Inc. All rights reserved.

Developing and Optimising the Use of Logic Models in Systematic Reviews: Exploring Practice and Good Practice in the Use of Programme Theory in Reviews.

PubMed

Kneale, Dylan; Thomas, James; Harris, Katherine

2015-01-01

Logic models are becoming an increasingly common feature of systematic reviews, as is the use of programme theory more generally in systematic reviewing. Logic models offer a framework to help reviewers to 'think' conceptually at various points during the review, and can be a useful tool in defining study inclusion and exclusion criteria, guiding the search strategy, identifying relevant outcomes, identifying mediating and moderating factors, and communicating review findings. In this paper we critique the use of logic models in systematic reviews and protocols drawn from two databases representing reviews of health interventions and international development interventions. Programme theory featured only in a minority of the reviews and protocols included. Despite drawing from different disciplinary traditions, reviews and protocols from both sources shared several limitations in their use of logic models and theories of change, and these were used almost unanimously to solely depict pictorially the way in which the intervention worked. Logic models and theories of change were consequently rarely used to communicate the findings of the review. Logic models have the potential to be an aid integral throughout the systematic reviewing process. The absence of good practice around their use and development may be one reason for the apparent limited utility of logic models in many existing systematic reviews. These concerns are addressed in the second half of this paper, where we offer a set of principles in the use of logic models and an example of how we constructed a logic model for a review of school-based asthma interventions.

Happy and Unhappy Patients: A Quantitative Analysis of Online Plastic Surgeon Reviews for Breast Augmentation.

PubMed

Dorfman, Robert G; Purnell, Chad; Qiu, Cecil; Ellis, Marco F; Basu, C Bob; Kim, John Y S

2018-05-01

Online reviews have become modern versions of the word-of-mouth recommendation, and prospective patients are increasingly consulting them before making decisions about their surgical care. The authors' objectives were to (1) identify trends in the use of online reviews, and (2) important reasons for patient satisfaction and dissatisfaction with aesthetic surgery. The authors selected breast augmentation as the primary procedure of interest. Reviews of the top 10 to 20 most reviewed plastic surgeons in each of six large metropolitan areas were obtained from Google, Yelp, and RealSelf. Reviews were assessed for predefined dimensions of satisfaction and dissatisfaction. A total of 1077 breast augmentation reviews were obtained. Ratings were distributed bimodally, with peaks at five stars and one star. The majority of reviews were positive (87.5 percent). Relative popularity of Google versus Yelp varied across geographic regions, and average rating varied by platform. Between 2011 and 2016, the number of online reviews for breast augmentation grew at an average rate of 42.6 percent per year. Aesthetic outcome was the most commonly cited dimension (69.8 percent of reviews), whereas cost was mentioned in only 7.8 percent of reviews. A substantial minority of negative Yelp (37 percent) and Google (9.4 percent) reviews were written by patients who did not actually undergo surgery. Free-text analysis of heterogeneous reviews (containing positive and negative attributes) classified dimensions as critical, redeemable, or protective. As the influence of online review platforms continues to grow, understanding drivers of positive and negative reviews may help surgeons improve patient satisfaction.

75 FR 69706 - SES Performance Review Board

Federal Register 2010, 2011, 2012, 2013, 2014

2010-11-15

... NATIONAL TRANSPORTATION SAFETY BOARD SES Performance Review Board AGENCY: National Transportation... National Transportation Safety Board Performance Review Board (PRB). FOR FURTHER INFORMATION CONTACT: Emily... Review Boards. The board reviews and evaluates the initial appraisal of a senior executive's performance...

What Works Clearinghouse Study Review Guide Instructions for Reviewing Single-Case Designs Studies

ERIC Educational Resources Information Center

What Works Clearinghouse, 2016

2016-01-01

This document provides step-by-step instructions on how to complete the Study Review Guide (SRG, Version S3, V2) for single-case designs (SCDs). Reviewers will complete an SRG for every What Works Clearinghouse (WWC) review. A completed SRG should be a reviewer's independent assessment of the study, relative to the criteria specified in the review…

6th NCEP/NWS Ensemble User Workshop

Science.gov Websites

5th Ensemble Users Workshop pdf<> Discussion Hogsett/Bright pdf Session 2: NCEP Centers Review Chair: Jun Du Israel Jirak NCEP SPC's Review pdf David Bright NCEP AWC's Review pdf Wallace Hogsett NCEP WPC's Review pdf Joseph Sienkiewicz NCEP OPC's Review pdf Dave Unger NCEP CPC's Review pdf<>

Synthesising Evidence on the Impacts of Programmes and Policies in Education, Crime and Justice, and Social Welfare: Practical Recommendations Based on 14 Test-Bed Reviews

ERIC Educational Resources Information Center

Cottingham, Phoebe; Maynard, Rebecca; Stagner, Matthew

2004-01-01

Review teams tested the systematic review procedures and principles developed under the Campbell Collaboration Fourteen review teams selected topics for intervention reviews in social policy, education, and criminal justice. Review protocols gave criteria for the extensive research literature search. Randomised Controlled Trials were selected.…

76 FR 52638 - Fisheries of the Gulf of Mexico and South Atlantic; Southeast Data, Assessment, and Review (SEDAR...

Federal Register 2010, 2011, 2012, 2013, 2014

2011-08-23

... specific dates and times. ADDRESSES: The Review Workshop will be held at Florida Wildlife Research... Review Workshop Schedule November 1-4, 2011; SEDAR 27 Review Workshop, November 1, 2011: 9 a.m.-8 p.m... the Gulf of Mexico and South Atlantic; Southeast Data, Assessment, and Review (SEDAR); Review Workshop...

77 FR 48611 - Rules of Practice for Trials Before the Patent Trial and Appeal Board and Judicial Review of...

Federal Register 2010, 2011, 2012, 2013, 2014

2012-08-14

... consolidated set of rules relating to Board trial practice for inter partes review, post-grant review, the... rules relating to Board trial practice for inter partes review, post- grant review, the transitional...), to provide a consolidated set of rules relating to Board trial practice for inter partes review, post...

EDITOR IN CHIEF'S ANNOUNCEMENT: New Review article type New Review article type

NASA Astrophysics Data System (ADS)

2010-09-01

This issue sees the publication of our first `Brief Review', a new kind of review article that we are introducing to complement our existing, very popular Topical Review programme. While a Topical Review is a broad overview article providing comprehensive coverage of progress in an area, a Brief Review is designed to be a shorter, `snapshot' of a field that is expanding or developing rapidly. Written by experts in the field and commissioned by members of our Editorial Board, we hope you find Brief Reviews to be a useful source of information and perspective. Clifford M Will Editor in Chief Classical and Quantum Gravity

Eligibility criteria in systematic reviews published in prominent medical journals: a methodological review.

PubMed

McCrae, Niall; Purssell, Edward

2015-12-01

Clear and logical eligibility criteria are fundamental to the design and conduct of a systematic review. This methodological review examined the quality of reporting and application of eligibility criteria in systematic reviews published in three leading medical journals. All systematic reviews in the BMJ, JAMA and The Lancet in the years 2013 and 2014 were extracted. These were assessed using a refined version of a checklist previously designed by the authors. A total of 113 papers were eligible, of which 65 were in BMJ, 17 in The Lancet and 31 in JAMA. Although a generally high level of reporting was found, eligibility criteria were often problematic. In 67% of papers, eligibility was specified after the search sources or terms. Unjustified time restrictions were used in 21% of reviews, and unpublished or unspecified data in 27%. Inconsistency between journals was apparent in the requirements for systematic reviews. The quality of reviews in these leading medical journals was high; however, there were issues that reduce the clarity and replicability of the review process. As well as providing a useful checklist, this methodological review informs the continued development of standards for systematic reviews. © 2015 John Wiley & Sons, Ltd.

A Study of Technical Engineering Peer Reviews at NASA

NASA Technical Reports Server (NTRS)

Chao, Lawrence P.; Tumer, Irem Y.; Bell, David G.

2003-01-01

This report describes the state of practices of design reviews at NASA and research into what can be done to improve peer review practices. There are many types of reviews at NASA: required and not, formalized and informal, programmatic and technical. Standing project formal reviews such as the Preliminary Design Review and Critical Design Review are a required part of every project and mission development. However, the technical, engineering peer reviews that support teams' work on such projects are informal, some times ad hoc, and inconsistent across the organization. The goal of this work is to identify best practices and lessons learned from NASA's experience, supported by academic research and methodologies to ultimately improve the process. This research has determined that the organization, composition, scope, and approach of the reviews impact their success. Failure Modes and Effects Analysis (FMEA) can identify key areas of concern before or in the reviews. Product definition tools like the Project Priority Matrix, engineering-focused Customer Value Chain Analysis (CVCA), and project or system-based Quality Function Deployment (QFD) help prioritize resources in reviews. The use of information technology and structured design methodologies can strengthen the engineering peer review process to help NASA work towards error-proofing the design process.

Time to consider sharing data extracted from trials included in systematic reviews.

PubMed

Wolfenden, Luke; Grimshaw, Jeremy; Williams, Christopher M; Yoong, Sze Lin

2016-11-03

While the debate regarding shared clinical trial data has shifted from whether such data should be shared to how this is best achieved, the sharing of data collected as part of systematic reviews has received little attention. In this commentary, we discuss the potential benefits of coordinated efforts to share data collected as part of systematic reviews. There are a number of potential benefits of systematic review data sharing. Shared information and data obtained as part of the systematic review process may reduce unnecessary duplication, reduce demand on trialist to service repeated requests from reviewers for data, and improve the quality and efficiency of future reviews. Sharing also facilitates research to improve clinical trial and systematic review methods and supports additional analyses to address secondary research questions. While concerns regarding appropriate use of data, costs, or the academic return for original review authors may impede more open access to information extracted as part of systematic reviews, many of these issues are being addressed, and infrastructure to enable greater access to such information is being developed. Embracing systems to enable more open access to systematic review data has considerable potential to maximise the benefits of research investment in undertaking systematic reviews.

Evaluation of drug reviews.

PubMed

Hendrickson, N M; Amerson, A B

1986-10-01

Drug reviews appearing in Clinical Pharmacy, Drug Intelligence and Clinical Pharmacy (DICP), Drugs, and Pharmacotherapy from January 1982 through December 1984 were evaluated for number, duplication among journals, timeliness, scope, and format. The design of this study was primarily quantitative rather than qualitative. Pharmacotherapy published the most reviews (49), followed by Drugs (43), Clinical Pharmacy (37), and DICP (29). Drugs and Pharmacotherapy published the largest number of unique reviews (agents not reviewed by the other journals during the study period), while Pharmacotherapy and Clinical Pharmacy published the most reviews on newly marketed drugs. Reviews of four drugs (acyclovir, moxalactam, ranitidine, and trazodone) were compared in terms of major sections, terminology and format, bibliography, use of tables and figures, scope of evaluative comments, and review process. Reviews in Drugs consistently contained the most references and tables and provided the most detail. Information was most accessible in Drugs, followed by Pharmacotherapy. Drugs used the largest panel of reviewers. All of the journals provided evaluative comments, although the scope varied. Continuing-education credit is available for review articles in Clinical Pharmacy and DICP. In selecting one or more of these journals, individuals or institutions should compare their needs with regard to the timeliness, scope, and format of the review articles in each journal.

The value of 100% retrospective peer review in a forensic pathology practice.

PubMed

Obenson, Ken; Wright, Claire M

2013-11-01

Peer review in forensic pathology practice has become an important cornerstone of continuous quality improvement. Although there are several components to an effective and transparent peer review process, one of the most essential is the review of completed reports. The autopsy report may be reviewed prospectively (report reviewed before sign out) or retrospectively (report reviewed after sign out). Prospective reviews are more likely to be performed on criminal or criminally suspicious cases, pediatric and SIDS deaths and high profile cases. Retrospective reviews on the other hand are performed on a proportion of all other signed-out routine medico-legal cases. The actual percentage varies by jurisdiction since there are no agreed minimum standards. Manpower and workload factors appear to be critical to determining what percentage of cases are reviewed retrospectively. The objective of this report is to present a mechanism by which a 100% retrospective review policy has been implemented, how it integrates with quality management protocols, the outcomes of the reviews and what challenges remain to improve compliance with key quality indicators especially turn around time (TAT) statistics. Copyright © 2013 Elsevier Ltd and Faculty of Forensic and Legal Medicine. All rights reserved.

A scoping review protocol on the roles and tasks of peer reviewers in the manuscript review process in biomedical journals.

PubMed

Glonti, Ketevan; Cauchi, Daniel; Cobo, Erik; Boutron, Isabelle; Moher, David; Hren, Darko

2017-10-22

The primary functions of peer reviewers are poorly defined. Thus far no body of literature has systematically identified the roles and tasks of peer reviewers of biomedical journals. A clear establishment of these can lead to improvements in the peer review process. The purpose of this scoping review is to determine what is known on the roles and tasks of peer reviewers. We will use the methodological framework first proposed by Arksey and O'Malley and subsequently adapted by Levac et al and the Joanna Briggs Institute. The scoping review will include all study designs, as well as editorials, commentaries and grey literature. The following eight electronic databases will be searched (from inception to May 2017): Cochrane Library, Cumulative Index to Nursing and Allied Health Literature, Educational Resources Information Center, EMBASE, MEDLINE, PsycINFO, Scopus and Web of Science. Two reviewers will use inclusion and exclusion criteria based on the 'Population-Concept-Context' framework to independently screen titles and abstracts of articles considered for inclusion. Full-text screening of relevant eligible articles will also be carried out by two reviewers. The search strategy for grey literature will include searching in websites of existing networks, biomedical journal publishers and organisations that offer resources for peer reviewers. In addition we will review journal guidelines to peer reviewers on how to perform the manuscript review. Journals will be selected using the 2016 journal impact factor. We will identify and assess the top five, middle five and lowest-ranking five journals across all medical specialties. This scoping review will undertake a secondary analysis of data already collected and does not require ethical approval. The results will be disseminated through journals and conferences targeting stakeholders involved in peer review in biomedical research. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2017. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

How do authors of systematic reviews deal with research malpractice and misconduct in original studies? A cross-sectional analysis of systematic reviews and survey of their authors

PubMed Central

Elia, Nadia; von Elm, Erik; Chatagner, Alexandra; Pöpping, Daniel M; Tramèr, Martin R

2016-01-01

Objectives To study whether systematic reviewers apply procedures to counter-balance some common forms of research malpractice such as not publishing completed research, duplicate publications, or selective reporting of outcomes, and to see whether they identify and report misconduct. Design Cross-sectional analysis of systematic reviews and survey of their authors. Participants 118 systematic reviews published in four journals (Ann Int Med, BMJ, JAMA, Lancet), and the Cochrane Library, in 2013. Main outcomes and measures Number (%) of reviews that applied procedures to reduce the impact of: (1) publication bias (through searching of unpublished trials), (2) selective outcome reporting (by contacting the authors of the original studies), (3) duplicate publications, (4) sponsors’ and (5) authors’ conflicts of interest, on the conclusions of the review, and (6) looked for ethical approval of the studies. Number (%) of reviewers who suspected misconduct are reported. The procedures applied were compared across journals. Results 80 (68%) reviewers confirmed their data. 59 (50%) reviews applied three or more procedures; 11 (9%) applied none. Unpublished trials were searched in 79 (66%) reviews. Authors of original studies were contacted in 73 (62%). Duplicate publications were searched in 81 (69%). 27 reviews (23%) reported sponsors of the included studies; 6 (5%) analysed their impact on the conclusions of the review. Five reviews (4%) looked at conflicts of interest of study authors; none of them analysed their impact. Three reviews (2.5%) looked at ethical approval of the studies. Seven reviews (6%) suspected misconduct; only 2 (2%) reported it explicitly. Procedures applied differed across the journals. Conclusions Only half of the systematic reviews applied three or more of the six procedures examined. Sponsors, conflicts of interest of authors and ethical approval remain overlooked. Research misconduct is sometimes identified, but rarely reported. Guidance on when, and how, to report suspected misconduct is needed. PMID:26936908

Interventions for postoperative pain in children: An overview of systematic reviews.

PubMed

Boric, Krste; Dosenovic, Svjetlana; Jelicic Kadic, Antonia; Batinic, Marijan; Cavar, Marija; Urlic, Marjan; Markovina, Nikolina; Puljak, Livia

2017-09-01

The aim of this study was to conduct an overview of systematic reviews that summarizes the results about efficacy and safety from randomized controlled trials involving the various strategies used for postoperative pain management in children. We searched the Cochrane Database of Systematic Reviews, CINAHL, Database of Reviews of Effect, Embase, MEDLINE, and PsycINFO from the earliest date to January 24, 2016. This overview included 45 systematic reviews that evaluated interventions for postoperative pain in children. Out of 45 systematic reviews that investigated various interventions for postoperative pain in children, 19 systematic reviews (42%) presented conclusive evidence of efficacy. Positive conclusive evidence was reported in 18 systematic reviews (40%) for the efficacy of diclofenac, ketamine, caudal analgesia, dexmedetomidine, music therapy, corticosteroid, epidural analgesia, paracetamol, and/or nonsteroidal anti-inflammatory drugs and transversus abdominis plane block. Only one systematic review reported conclusive evidence of equal efficacy that involved a comparison of dexmedetomidine vs morphine and fentanyl. Safety of interventions was reported as conclusive in 14 systematic reviews (31%), with positive conclusive evidence for dexmedetomidine, corticosteroid, epidural analgesia, transversus abdominis plane block, and clonidine. Seven systematic reviews reported equal conclusive safety for epidural infusion, diclofenac intravenous vs ketamine added to opioid analgesia, bupivacaine, ketamine, paracetamol, and dexmedetomidine vs intravenous infusions of various opioid analgesics, oral suspension and suppository of diclofenac, only opioid, normal saline, no treatment, placebo, and midazolam. Negative conclusive statement for safety was reported in one systematic review for caudal analgesia vs noncaudal regional analgesia. More than half of systematic reviews included in this overview were rated as having medium methodological quality. Of 45 included systematic reviews, 10 were Cochrane reviews and they had higher methodological quality than non-Cochrane reviews. As evidence concerning efficacy and safety is inconclusive for most of the analyzed interventions, our review points out the need for more rigorous trials concerning pain management in children. © 2017 John Wiley & Sons Ltd.

Reducing NPR 7120.5D to Practice: Transitioning from Design Reviews to the SIR Hardware Review

NASA Technical Reports Server (NTRS)

Taylor, Randall

2011-01-01

The Gravity Recovery And Interior Laboratory (GRAIL) mission was the first Jet Propulsion Laboratory (JPL) project initiated under NASA's revised rules for space flight project management, NPR 7120.5D, "NASA Space Flight Program and Project Management Requirements." NASA selected GRAIL through a competitive Announcement of Opportunity process and funded its Phase B Preliminary Design effort. The team's first major milestone was a JPL institutional milestone, the Project Mission System Review (PMSR), which proved an excellent tune-up for the end-of-Phase-B NASA life-cycle review, the Preliminary Design Review (PDR). Building on JPL experience on the Prometheus and Juno projects, the team successfully organized for and conducted these reviews on an aggressive schedule. For the Project Critical Design Review (CDR), lessons learned from the PDR and updated Standing Review Board (SRB) practices from the Agency were factored into the review preparation effort. Additionally, the review was held at the Principal Investigator's institution, the Massachusetts Institute of Technology, rather than at the project management center (JPL), which necessitated additional cross-country coordination steps. The PMSR, PDR, and CDR were design reviews and largely paper-oriented. For the System Integration Review (SIR), the project needed to transition to a hardware review and deal with paper in a very different manner. While many of the practices employed for the design reviews were modified and retained (e.g., review preparation team, gate products management, pre-reviews, SRB coordination), the review agenda, presentation style, and slide templates were significantly changed. A key success factor concerned the handling of project open paper, which was succinctly and effectively communicated to the SRB in presentations.This paper provides a brief overview of the GRAIL mission and its project management challenges, provides a detailed description of project SIR preparation and execution activities, including positive and negative lessons learned and identifies recommendations for future NASA (and non- NASA) project teams.

Time required for institutional review board review at one Veterans Affairs medical center.

PubMed

Hall, Daniel E; Hanusa, Barbara H; Stone, Roslyn A; Ling, Bruce S; Arnold, Robert M

2015-02-01

Despite growing concern that institutional review boards (IRBs) impose burdensome delays on research, little is known about the time required for IRB review across different types of research. To measure the overall and incremental process times for IRB review as a process of quality improvement. After developing a detailed process flowchart of the IRB review process, 2 analysts abstracted temporal data from the records pertaining to all 103 protocols newly submitted to the IRB at a large urban Veterans Affairs medical center from June 1, 2009, through May 31, 2011. Disagreements were reviewed with the principal investigator to reach consensus. We then compared the review times across review types using analysis of variance and post hoc Scheffé tests after achieving normally distributed data through logarithmic transformation. Calendar days from initial submission to final approval of research protocols. Initial IRB review took 2 to 4 months, with expedited and exempt reviews requiring less time (median [range], 85 [23-631] and 82 [16-437] days, respectively) than full board reviews (median [range], 131 [64-296] days; P = .008). The median time required for credentialing of investigators was 1 day (range, 0-74 days), and review by the research and development committee took a median of 15 days (range, 0-184 days). There were no significant differences in credentialing or research and development times across review types (exempt, expedited, or full board). Of the extreme delays in IRB review, 80.0% were due to investigators' slow responses to requested changes. There were no systematic delays attributable to the information security officer, privacy officer, or IRB chair. Measuring and analyzing review times is a critical first step in establishing a culture and process of continuous quality improvement among IRBs that govern research programs. The review times observed at this IRB are substantially longer than the 60-day target recommended by expert panels. The method described here could be applied to other IRBs to begin identifying and improving inefficiencies.

IRIS Toxicological Review of Trichloroacetic Acid (TCA) ...

EPA Pesticide Factsheets

On September 24, 2009, the Toxicological Review of Trichloroacetic Acid (TCA) and the charge to external peer reviewers were released for external peer review and public comment. The Toxicological Review and charge were reviewed internally by EPA and by other federal agencies and White House Offices before public release. In the new IRIS process, introduced by the EPA Administrator, all written comments on IRIS assessments submitted by other federal agencies and White House Offices will be made publicly available. Accordingly, interagency comments and the interagency science consultation draft of the IRIS Toxicological Review of Trichloroacetic Acid and the charge to external peer reviewers are posted on this site. The draft Toxicological Review of Trichloroacetic Acid provides scientific support and rationale for the hazard identification and dose-response assessment pertaining to chronic exposure to trichloroacetic acid.

IRIS Toxicological Review of Formaldehyde (Interagency ...

EPA Pesticide Factsheets

On June 2, 2010, the Toxicological Review of Formaldehyde and the charge to external peer reviewers were released for external peer review and public comment. The Toxicological Review and charge were reviewed internally by EPA and by other federal agencies and White House Offices before public release. In the new IRIS process, introduced by the EPA Administrator, all written comments on IRIS assessments submitted by other federal agencies and White House Offices will be made publicly available. Accordingly, interagency comments and the interagency science consultation draft of the IRIS Toxicological Review of Formaldehyde and the charge to external peer reviewers are posted on this site. The draft Toxicological Review of Formaldehyde-Inhalation Assessment provides scientific support and rationale for the hazard and dose-response assessment pertaining to chronic inhalation exposure to formaldehyde.

IRIS Toxicological Review of Urea (Interagency Science ...

EPA Pesticide Factsheets

On September 28, 2010, the Toxicological Review of Urea and the charge to external peer reviewers were released for external peer review and public comment. The Toxicological Review and charge were reviewed internally by EPA and by other federal agencies and White House Offices before public release. In the new IRIS process, introduced by the EPA Administrator, all written comments on IRIS assessments submitted by other federal agencies and White House Offices will be made publicly available. Accordingly, interagency comments and the interagency science consultation draft of the IRIS Toxicological Review of Urea and the charge to external peer reviewers are posted on this site. The draft Toxicological Review of Urea provides scientific support and rationale for the hazard and dose-response assessment pertaining to chronic exposure to Urea.

The Global Burden of Journal Peer Review in the Biomedical Literature: Strong Imbalance in the Collective Enterprise.

PubMed

Kovanis, Michail; Porcher, Raphaël; Ravaud, Philippe; Trinquart, Ludovic

2016-01-01

The growth in scientific production may threaten the capacity for the scientific community to handle the ever-increasing demand for peer review of scientific publications. There is little evidence regarding the sustainability of the peer-review system and how the scientific community copes with the burden it poses. We used mathematical modeling to estimate the overall quantitative annual demand for peer review and the supply in biomedical research. The modeling was informed by empirical data from various sources in the biomedical domain, including all articles indexed at MEDLINE. We found that for 2015, across a range of scenarios, the supply exceeded by 15% to 249% the demand for reviewers and reviews. However, 20% of the researchers performed 69% to 94% of the reviews. Among researchers actually contributing to peer review, 70% dedicated 1% or less of their research work-time to peer review while 5% dedicated 13% or more of it. An estimated 63.4 million hours were devoted to peer review in 2015, among which 18.9 million hours were provided by the top 5% contributing reviewers. Our results support that the system is sustainable in terms of volume but emphasizes a considerable imbalance in the distribution of the peer-review effort across the scientific community. Finally, various individual interactions between authors, editors and reviewers may reduce to some extent the number of reviewers who are available to editors at any point.

The Global Burden of Journal Peer Review in the Biomedical Literature: Strong Imbalance in the Collective Enterprise

PubMed Central

Porcher, Raphaël; Ravaud, Philippe; Trinquart, Ludovic

2016-01-01

The growth in scientific production may threaten the capacity for the scientific community to handle the ever-increasing demand for peer review of scientific publications. There is little evidence regarding the sustainability of the peer-review system and how the scientific community copes with the burden it poses. We used mathematical modeling to estimate the overall quantitative annual demand for peer review and the supply in biomedical research. The modeling was informed by empirical data from various sources in the biomedical domain, including all articles indexed at MEDLINE. We found that for 2015, across a range of scenarios, the supply exceeded by 15% to 249% the demand for reviewers and reviews. However, 20% of the researchers performed 69% to 94% of the reviews. Among researchers actually contributing to peer review, 70% dedicated 1% or less of their research work-time to peer review while 5% dedicated 13% or more of it. An estimated 63.4 million hours were devoted to peer review in 2015, among which 18.9 million hours were provided by the top 5% contributing reviewers. Our results support that the system is sustainable in terms of volume but emphasizes a considerable imbalance in the distribution of the peer-review effort across the scientific community. Finally, various individual interactions between authors, editors and reviewers may reduce to some extent the number of reviewers who are available to editors at any point. PMID:27832157

75 FR 34519 - SES Performance Review Board

Federal Register 2010, 2011, 2012, 2013, 2014

2010-06-17

... TRADE AND DEVELOPMENT AGENCY SES Performance Review Board AGENCY: U.S. Trade and Development... and Development Agency's Performance Review Board. FOR FURTHER INFORMATION CONTACT: Carolyn Hum... more SES Performance Review Boards. The Board shall review and evaluate the initial appraisal of a...

76 FR 80729 - Intergovernmental Review

Federal Register 2010, 2011, 2012, 2013, 2014

2011-12-27

... Rural Utilities Service 7 CFR Part 4274 Intergovernmental Review AGENCIES: Rural Housing Service, Rural... reference to RD Instruction 1940-J, Intergovernmental Review.'' This action is necessary since the affected... offices have current guidance on intergovernmental review of proposed projects being reviewed pursuant to...

32 CFR 2001.16 - Fundamental classification guidance review.

Code of Federal Regulations, 2010 CFR

2010-07-01

... 32 National Defense 6 2010-07-01 2010-07-01 false Fundamental classification guidance review. 2001... INFORMATION Classification § 2001.16 Fundamental classification guidance review. (a) Performance of fundamental classification guidance reviews. An initial fundamental classification guidance review shall be...

76 FR 77777 - North American Free Trade Agreement (NAFTA), Article 1904; Binational Panel Reviews: Notice of...

Federal Register 2010, 2011, 2012, 2013, 2014

2011-12-14

... (NAFTA), Article 1904; Binational Panel Reviews: Notice of Completion of Panel Review AGENCY: NAFTA... Rule 71(3) of the Rules of Procedure for Article 1904 Binational Panel Review, ``A panel review is...

Peer review versus editorial review and their role in innovative science.

PubMed

Steinhauser, Georg; Adlassnig, Wolfram; Risch, Jesaka Ahau; Anderlini, Serena; Arguriou, Petros; Armendariz, Aaron Zolen; Bains, William; Baker, Clark; Barnes, Martin; Barnett, Jonathan; Baumgartner, Michael; Baumgartner, Thomas; Bendall, Charles A; Bender, Yvonne S; Bichler, Max; Biermann, Teresa; Bini, Ronaldo; Blanco, Eduardo; Bleau, John; Brink, Anthony; Brown, Darin; Burghuber, Christopher; Calne, Roy; Carter, Brian; Castaño, Cesar; Celec, Peter; Celis, Maria Eugenia; Clarke, Nicky; Cockrell, David; Collins, David; Coogan, Brian; Craig, Jennifer; Crilly, Cal; Crowe, David; Csoka, Antonei B; Darwich, Chaza; Del Kebos, Topiciprin; Derinaldi, Michele; Dlamini, Bongani; Drewa, Tomasz; Dwyer, Michael; Eder, Fabienne; de Palma, Raúl Ehrichs; Esmay, Dean; Rött, Catherine Evans; Exley, Christopher; Falkov, Robin; Farber, Celia Ingrid; Fearn, William; Felsmann, Sophie; Flensmark, Jarl; Fletcher, Andrew K; Foster, Michaela; Fountoulakis, Kostas N; Fouratt, Jim; Blanca, Jesus Garcia; Sotelo, Manuel Garrido; Gittler, Florian; Gittler, Georg; Gomez, Juan; Gomez, Juan F; Polar, Maria Grazia Gonzales; Gonzalez, Jossina; Gösselsberger, Christoph; Habermacher, Lynn; Hajek, Michael; Hakala, Faith; Haliburton, Mary-Sue; Hankins, John Robert; Hart, Jason; Hasslberger, Sepp; Hennessey, Donalyn; Herrmann, Andrea; Hersee, Mike; Howard, Connie; Humphries, Suzanne; Isharc, Laeeth; Ivanovski, Petar; Jenuth, Stephen; Jerndal, Jens; Johnson, Christine; Keleta, Yonas; Kenny, Anna; Kidd, Billie; Kohle, Fritz; Kolahi, Jafar; Koller-Peroutka, Marianne; Kostova, Lyubov; Kumar, Arunachalam; Kurosawa, Alejandro; Lance, Tony; Lechermann, Michael; Lendl, Bernhard; Leuchters, Michael; Lewis, Evan; Lieb, Edward; Lloyd, Gloria; Losek, Angelika; Lu, Yao; Maestracci, Saadia; Mangan, Dennis; Mares, Alberto W; Barnett, Juan Mazar; McClain, Valerie; McNair, John Sydney; Michael, Terry; Miller, Lloyd; Monzani, Partizia; Moran, Belen; Morris, Mike; Mößmer, Georg; Mountain, Johny; Phuthe, Onnie Mary Moyo; Muñoz, Marcos; Nakken, Sheri; Wambui, Anne Nduta; Neunteufl, Bettina; Nikolić, Dimitrije; Oberoi, Devesh V; Obmode, Gregory; Ogar, Laura; Ohara, Jo; Rybine, Naion Olej; Owen, Bryan; Owen, Kim Wilson; Parikh, Rakesh; Pearce, Nicholas J G; Pemmer, Bernhard; Piper, Chris; Prince, Ian; Reid, Terence; Rindermann, Heiner; Risch, Stefan; Robbins, Josh; Roberts, Seth; Romero, Ajeandro; Rothe, Michael Thaddäus; Ruiz, Sergio; Sacher, Juliane; Sackl, Wolfgang; Salletmaier, Markus; Sanand, Jairaj; Sauerzopf, Clemens; Schwarzgruber, Thomas; Scott, David; Seegers, Laura; Seppi, David; Shields, Kyle; Siller-Matula, Jolanta; Singh, Beldeu; Sithole, Sibusio; Six, Florian; Skoyles, John R; Slofstra, Jildou; Sole, Daphne Anne; Sommer, Werner F; Sonko, Mels; Starr-Casanova, Chrislie J; Steakley, Marjorie Elizabeth; Steinhauser, Wolfgang; Steinhoff, Konstantin; Sterba, Johannes H; Steppan, Martin; Stindl, Reinhard; Stokely, Joe; Stokely, Karri; St-Pierre, Gilles; Stratford, James; Streli, Christina; Stryg, Carl; Sullivan, Mike; Summhammer, Johann; Tadesse, Amhayes; Tavares, David; Thompson, Laura; Tomlinson, Alison; Tozer, Jack; Trevisanato, Siro I; Trimmel, Michaela; Turner, Nicole; Vahur, Paul; van der Byl, Jennie; van der Maas, Tine; Varela, Leo; Vega, Carlos A; Vermaak, Shiloh; Villasenor, Alex; Vogel, Matt; von Wintzigerode, Georg; Wagner, Christoph; Weinberger, Manuel; Weinberger, Peter; Wilson, Nick; Wolfe, Jennifer Finocchio; Woodley, Michael A; Young, Ian; Zuraw, Glenn; Zwiren, Nicole

2012-10-01

Peer review is a widely accepted instrument for raising the quality of science. Peer review limits the enormous unstructured influx of information and the sheer amount of dubious data, which in its absence would plunge science into chaos. In particular, peer review offers the benefit of eliminating papers that suffer from poor craftsmanship or methodological shortcomings, especially in the experimental sciences. However, we believe that peer review is not always appropriate for the evaluation of controversial hypothetical science. We argue that the process of peer review can be prone to bias towards ideas that affirm the prior convictions of reviewers and against innovation and radical new ideas. Innovative hypotheses are thus highly vulnerable to being "filtered out" or made to accord with conventional wisdom by the peer review process. Consequently, having introduced peer review, the Elsevier journal Medical Hypotheses may be unable to continue its tradition as a radical journal allowing discussion of improbable or unconventional ideas. Hence we conclude by asking the publisher to consider re-introducing the system of editorial review to Medical Hypotheses.

Frequency of data extraction errors and methods to increase data extraction quality: a methodological review.

PubMed

Mathes, Tim; Klaßen, Pauline; Pieper, Dawid

2017-11-28

Our objective was to assess the frequency of data extraction errors and its potential impact on results in systematic reviews. Furthermore, we evaluated the effect of different extraction methods, reviewer characteristics and reviewer training on error rates and results. We performed a systematic review of methodological literature in PubMed, Cochrane methodological registry, and by manual searches (12/2016). Studies were selected by two reviewers independently. Data were extracted in standardized tables by one reviewer and verified by a second. The analysis included six studies; four studies on extraction error frequency, one study comparing different reviewer extraction methods and two studies comparing different reviewer characteristics. We did not find a study on reviewer training. There was a high rate of extraction errors (up to 50%). Errors often had an influence on effect estimates. Different data extraction methods and reviewer characteristics had moderate effect on extraction error rates and effect estimates. The evidence base for established standards of data extraction seems weak despite the high prevalence of extraction errors. More comparative studies are needed to get deeper insights into the influence of different extraction methods.

Potential value of systematic reviews of qualitative evidence in informing user-centered health and social care: findings from a descriptive overview.

PubMed

Dalton, Jane; Booth, Andrew; Noyes, Jane; Sowden, Amanda J

2017-08-01

Systematic reviews of quantitative evidence are well established in health and social care. Systematic reviews of qualitative evidence are increasingly available, but volume, topics covered, methods used, and reporting quality are largely unknown. We provide a descriptive overview of systematic reviews of qualitative evidence assessing health and social care interventions included on the Database of Abstracts of Reviews of Effects (DARE). We searched DARE for reviews published between January 1, 2009, and December 31, 2014. We extracted data on review content and methods, summarized narratively, and explored patterns over time. We identified 145 systematic reviews conducted worldwide (64 in the UK). Interventions varied but largely covered treatment or service delivery in community and hospital settings. There were no discernible patterns over time. Critical appraisal of primary studies was conducted routinely. Most reviews were poorly reported. Potential exists to use systematic reviews of qualitative evidence when driving forward user-centered health and social care. We identify where more research is needed and propose ways to improve review methodology and reporting. Copyright © 2017 Elsevier Inc. All rights reserved.

Peer review in forensic science.

PubMed

Ballantyne, Kaye N; Edmond, Gary; Found, Bryan

2017-08-01

Peer review features prominently in the forensic sciences. Drawing on recent research and studies, this article examines different types of peer review, specifically: editorial peer review; peer review by the scientific community; technical and administrative review; and verification (and replication). The article reviews the different meanings of these quite disparate activities and their utility in relation to enhancing performance and reducing error. It explains how forensic practitioners should approach and use peer review, as well as how it should be described in expert reports and oral testimony. While peer review has considerable potential, and is a key component of modern quality management systems, its actual value in most forensic science settings has yet to be determined. In consequence, forensic practitioners should reflect on why they use specific review procedures and endeavour to make their actual practices and their potential value transparent to consumers; whether investigators, lawyers, jurors or judges. Claims that review increases the validity of a scientific technique or accuracy of opinions within a particular case should be avoided until empirical evidence is available to support such assertions. Copyright © 2017 Elsevier B.V. All rights reserved.

Can editors save peer review from peer reviewers?

PubMed

D'Andrea, Rafael; O'Dwyer, James P

2017-01-01

Peer review is the gold standard for scientific communication, but its ability to guarantee the quality of published research remains difficult to verify. Recent modeling studies suggest that peer review is sensitive to reviewer misbehavior, and it has been claimed that referees who sabotage work they perceive as competition may severely undermine the quality of publications. Here we examine which aspects of suboptimal reviewing practices most strongly impact quality, and test different mitigating strategies that editors may employ to counter them. We find that the biggest hazard to the quality of published literature is not selfish rejection of high-quality manuscripts but indifferent acceptance of low-quality ones. Bypassing or blacklisting bad reviewers and consulting additional reviewers to settle disagreements can reduce but not eliminate the impact. The other editorial strategies we tested do not significantly improve quality, but pairing manuscripts to reviewers unlikely to selfishly reject them and allowing revision of rejected manuscripts minimize rejection of above-average manuscripts. In its current form, peer review offers few incentives for impartial reviewing efforts. Editors can help, but structural changes are more likely to have a stronger impact.

Training practitioners in preparing systematic reviews: a cross-sectional survey of participants in the Australasian Cochrane Centre training program.

PubMed

Piehl, Janet H; Green, Sally; Silagy, Chris

2002-06-02

Although systematic reviews of health care interventions are an invaluable tool for health care providers and researchers, many potential authors never publish reviews. This study attempts to determine why some people with interest in performing systematic reviews do not subsequently publish a review; and what steps could possibly increase review completion. Cross-sectional survey by email and facsimile of the 179 participants in Australasian Cochrane Centre training events between 1998 and 2000. Ninety-two participants responded to the survey (51 percent). Response rate of deliverable surveys was 82 percent (92/112). The remainder of the participants had invalid or no contact information on file. More than 75 percent of respondents felt that the current workshops met their needs for training. The most critical barriers to completion of a Cochrane review were: lack of time (80 percent), lack of financial support (36 percent), methodological problems (23 percent) and problems with group dynamics (10 percent). Strategies to protect reviewer time and increase the efficiency of the review process may increase the numbers of trained reviewers completing a systematic review.

Studying the Study Section: How Group Decision Making in Person and via Videoconferencing Affects the Grant Peer Review Process. WCER Working Paper No. 2015-6

ERIC Educational Resources Information Center

Pier, Elizabeth L.; Raclaw, Joshua; Nathan, Mitchell J.; Kaatz, Anna; Carnes, Molly; Ford, Cecilia E.

2015-01-01

Grant peer review is a foundational component of scientific research. In the context of grant review meetings, the review process is a collaborative, socially mediated, locally constructed decision-making task. The current study examines how collaborative discussion affects reviewers' scores of grant proposals, how different review panels score…

Mission Drift in Qualitative Research, or Moving Toward a Systematic Review of Qualitative Studies, Moving Back to a More Systematic Narrative Review

ERIC Educational Resources Information Center

Jones, Kip

2004-01-01

The paper argues that the systematic review of qualitative research is best served by reliance upon qualitative methods themselves. A case is made for strengthening the narrative literature review and using narrative itself as a method of review. A technique is proposed that builds upon recent developments in qualitative systematic review by the…

75 FR 55307 - Honey From the People's Republic of China: Preliminary Intent To Rescind New Shipper Reviews

Federal Register 2010, 2011, 2012, 2013, 2014

2010-09-10

... Shipper Review, 75 FR 38980 (July 7, 2010). Expansion of the POR When the sale of the subject merchandise... review is based on a single sale, exclusion of that sale as non-bona fide necessarily must end the review... Shipper Review of Honey from the People's Republic of China: Bona Fide Analysis of the Sale Under Review...

What determines the effects and costs of breast cancer screening? A protocol of a systematic review of reviews.

PubMed

Mandrik, O; Ekwunife, O I; Zielonke, N; Meheus, F; Severens, J L; Lhachimi, S K; Murillo, R

2017-06-28

Multiple reviews demonstrated high variability in effectiveness and cost-effectiveness outcomes among studies on breast cancer screening (BCS) programmes. No study to our knowledge has summarized the current evidence on determinants of effectiveness and cost-effectiveness of the most used BCS approaches or tried to explain differences in conclusions of systematic reviews on this topic. Based on published reviews, this systematic review aims to assess the degree of variability of determinants for (a) effectiveness and (b) cost-effectiveness of BCS programmes using mammography, clinical breast examination, breast self-examination, ultrasonography, or their combinations among the general population. We will perform a comprehensive systematic literature search in Cochrane, Scopus, Embase, and Medline (via Pubmed). The search will be supplemented with hand searching of references of the included reviews, with hand searching in the specialized journals, and by contacting prominent experts in the field. Additional search for grey literature will be conducted on the websites of international cancer associations and networks. Two trained research assistants will screen titles and abstracts of publications independently, with at least random 10% of all abstracts being also screened by the principal researcher. The full texts of the systematic reviews will then be screened independently by two authors, and disagreements will be solved by consensus. The included reviews will be grouped by publication year, outcomes, designs of original studies, and quality. Additionally, for reviews published since 2011, transparency in reporting will be assessed using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist for the review on determinants of effectiveness and a modified PRISMA checklist for the review on determinants for cost-effectiveness. The study will apply the Assessing the Methodological Quality of Systematic Reviews checklist to assess the methodological quality of systematic reviews. We will report the data extracted from the systematic reviews in a systematic format. Meta-meta-analysis of extracted data will be conducted when feasible. This systematic review of reviews will examine the degree of variability in the effectiveness and cost-effectiveness of BCS programmes. PROSPERO CRD42016050764 and CRD42016050765.

Systematic reviews addressing identified health policy priorities in Eastern Mediterranean countries: a situational analysis.

PubMed

El-Jardali, Fadi; Akl, Elie A; Karroum, Lama Bou; Kdouh, Ola; Akik, Chaza; Fadlallah, Racha; Hammoud, Rawan

2014-08-20

Systematic reviews can offer policymakers and stakeholders concise, transparent, and relevant evidence pertaining to pressing policy priorities to help inform the decision-making process. The production and the use of systematic reviews are specifically limited in the Eastern Mediterranean region. The extent to which published systematic reviews address policy priorities in the region is still unknown. This situational analysis exercise aims at assessing the extent to which published systematic reviews address policy priorities identified by policymakers and stakeholders in Eastern Mediterranean region countries. It also provides an overview about the state of systematic review production in the region and identifies knowledge gaps. We conducted a systematic search of the Health System Evidence database to identify published systematic reviews on policy-relevant priorities pertaining to the following themes: human resources for health, health financing, the role of the non-state sector, and access to medicine. Priorities were identified from two priority-setting exercises conducted in the region. We described the distribution of these systematic reviews across themes, sub-themes, authors' affiliations, and countries where included primary studies were conducted. Out of the 1,045 systematic reviews identified in Health System Evidence on selected themes, a total of 200 systematic reviews (19.1%) addressed the priorities from the Eastern Mediterranean region. The theme with the largest number of systematic reviews included was human resources for health (115) followed by health financing (33), access to medicine (27), and role of the non-state sector (25). Authors based in the region produced only three systematic reviews addressing regional priorities (1.5%). Furthermore, no systematic review focused on the Eastern Mediterranean region. Primary studies from the region had limited contribution to systematic reviews; 17 systematic reviews (8.5%) included primary studies conducted in the region. There are still gaps in the production of systematic reviews addressing policymakers' and stakeholders' priorities in the Eastern Mediterranean region. Efforts should be directed towards better aligning systematic review production with policy needs and priorities. Study findings can inform the agendas of researchers, research institutions, and international funding agencies of priority areas where systematic reviews are required.

Systematic reviews addressing identified health policy priorities in Eastern Mediterranean countries: a situational analysis

PubMed Central

2014-01-01

Background Systematic reviews can offer policymakers and stakeholders concise, transparent, and relevant evidence pertaining to pressing policy priorities to help inform the decision-making process. The production and the use of systematic reviews are specifically limited in the Eastern Mediterranean region. The extent to which published systematic reviews address policy priorities in the region is still unknown. This situational analysis exercise aims at assessing the extent to which published systematic reviews address policy priorities identified by policymakers and stakeholders in Eastern Mediterranean region countries. It also provides an overview about the state of systematic review production in the region and identifies knowledge gaps. Methods We conducted a systematic search of the Health System Evidence database to identify published systematic reviews on policy-relevant priorities pertaining to the following themes: human resources for health, health financing, the role of the non-state sector, and access to medicine. Priorities were identified from two priority-setting exercises conducted in the region. We described the distribution of these systematic reviews across themes, sub-themes, authors’ affiliations, and countries where included primary studies were conducted. Results Out of the 1,045 systematic reviews identified in Health System Evidence on selected themes, a total of 200 systematic reviews (19.1%) addressed the priorities from the Eastern Mediterranean region. The theme with the largest number of systematic reviews included was human resources for health (115) followed by health financing (33), access to medicine (27), and role of the non-state sector (25). Authors based in the region produced only three systematic reviews addressing regional priorities (1.5%). Furthermore, no systematic review focused on the Eastern Mediterranean region. Primary studies from the region had limited contribution to systematic reviews; 17 systematic reviews (8.5%) included primary studies conducted in the region. Conclusions There are still gaps in the production of systematic reviews addressing policymakers’ and stakeholders’ priorities in the Eastern Mediterranean region. Efforts should be directed towards better aligning systematic review production with policy needs and priorities. Study findings can inform the agendas of researchers, research institutions, and international funding agencies of priority areas where systematic reviews are required. PMID:25139256

Promoting Children's Health with Digital Games: A Review of Reviews.

PubMed

Parisod, Heidi; Pakarinen, Anni; Kauhanen, Lotta; Aromaa, Minna; Leppänen, Ville; Liukkonen, Tapani N; Smed, Jouni; Salanterä, Sanna

2014-06-01

Effective, evidence-based, and interesting methods are needed for children's health promotion. Digital games can be such a method, but there is need for a summary of the evidence on the effectiveness of digital games in promoting children's health. The aim of this review of reviews was to evaluate the quality of systematic reviews, to summarize the evidence in systematic reviews and reviews related to the effectiveness of digital games in children's health promotion, and to identify gaps in knowledge. A systematic literature search was conducted in May-August 2013 from relevant databases, and 1178 references were found. In total, 15 systematic reviews and reviews met the inclusion criteria. Most of the systematic reviews were found to be medium quality on the AMSTAR checklist. Most commonly, systematic reviews and reviews evaluated active videogames. According to the results, evidence of the highest level and quality seems to support an increase in physical activity to light to moderate levels and energy expenditure, especially when playing active videogames that require both upper and lower body movements. In addition, sedentary games were shown to have potential in children's health education, especially in supporting changes in asthma- and diabetes-related behavior and in dietary habits. However, there are still several gaps in the knowledge. There is a need for further high-quality systematic reviews and research in the field of health games.

77 FR 66824 - Performance Review Board Membership

Federal Register 2010, 2011, 2012, 2013, 2014

2012-11-07

... DEPARTMENT OF DEFENSE Department of the Army Performance Review Board Membership AGENCY... Performance Review Board for the Department of the Army. DATES: Effective Date: November 15, 2012. FOR FURTHER... performance review boards. The boards shall review and evaluate the initial appraisal of senior executives...

28 CFR 71.42 - Judicial review.

Code of Federal Regulations, 2010 CFR

2010-07-01

... 28 Judicial Administration 2 2010-07-01 2010-07-01 false Judicial review. 71.42 Section 71.42....42 Judicial review. Section 3805 of title 31, United States Code, authorizes judicial review by an... assessments under this part and specifies the procedures for such review. ...

76 FR 33181 - Regulatory Review Schedule

Federal Register 2010, 2011, 2012, 2013, 2014

2011-06-08

... DEPARTMENT OF THE INTERIOR National Indian Gaming Commission 25 CFR Chapter III Regulatory Review... conducting a comprehensive review of all its regulations and requesting public comment on the process for conducting the regulatory review. On April 4, 2011, after holding eight consultation meetings and reviewing...

75 FR 70915 - Performance Review Board Membership

Federal Register 2010, 2011, 2012, 2013, 2014

2010-11-19

... DEPARTMENT OF DEFENSE Department of the Army Performance Review Board Membership AGENCY... Performance Review Board for the Department of the Army. DATES: Effective Date: November 16, 2010. FOR FURTHER... performance review boards. The boards shall review and evaluate the initial appraisal of senior executives...

76 FR 26967 - Regulatory Review Schedule

Federal Register 2010, 2011, 2012, 2013, 2014

2011-05-10

... DEPARTMENT OF THE INTERIOR National Indian Gaming Commission 25 CFR Chapter III Regulatory Review... review of its regulations and requesting public comment on the process for conducting the regulatory review. On April 4, 2011, after holding eight consultation meetings and reviewing all comments, NIGC...

75 FR 56999 - Senior Executive Service Performance Review Board

Federal Register 2010, 2011, 2012, 2013, 2014

2010-09-17

... DEFENSE NUCLEAR FACILITIES SAFETY BOARD Senior Executive Service Performance Review Board AGENCY... the Defense Nuclear Facilities Safety Board (DNFSB) Senior Executive Service (SES) Performance Review... performance review boards. The PRB shall review and evaluate the initial summary rating of the senior...

77 FR 62211 - Senior Executive Services (SES) Performance Review Board: Update

Federal Register 2010, 2011, 2012, 2013, 2014

2012-10-12

... AGENCY FOR INTERNATIONAL DEVELOPMENT Senior Executive Services (SES) Performance Review Board... International Development, Office of Inspector General's Senior Executive Service Performance Review Board... Performance Review Boards. The board shall review and evaluate the initial appraisal of each USAID OIG senior...

7 CFR 3400.14 - Proposal review.

Code of Federal Regulations, 2010 CFR

2010-01-01

... SERVICE, DEPARTMENT OF AGRICULTURE SPECIAL RESEARCH GRANTS PROGRAM Scientific Peer Review of Research... employees of the Department, the respective peer review group, and ad hoc reviewers, as required. Written...-depth discussions will be provided by peer review group members prior to recommending applications for...

76 FR 70428 - Performance Review Board Membership

Federal Register 2010, 2011, 2012, 2013, 2014

2011-11-14

... DEPARTMENT OF DEFENSE Department of the Army Performance Review Board Membership AGENCY... Performance Review Board for the Department of the Army. DATES: Effective Date: November 14, 2011. FOR FURTHER... performance review boards. The boards shall review and evaluate the initial appraisal of senior executives...

78 FR 63460 - Performance Review Board Membership

Federal Register 2010, 2011, 2012, 2013, 2014

2013-10-24

... DEPARTMENT OF DEFENSE Department of the Army Performance Review Board Membership AGENCY... Performance Review Board for the Department of the Army. DATES: Effective Date: November 21, 2013. FOR FURTHER... performance review boards. The boards shall review and evaluate the initial appraisal of senior executives...

Developing and Optimising the Use of Logic Models in Systematic Reviews: Exploring Practice and Good Practice in the Use of Programme Theory in Reviews

PubMed Central

Kneale, Dylan; Thomas, James; Harris, Katherine

2015-01-01

Background Logic models are becoming an increasingly common feature of systematic reviews, as is the use of programme theory more generally in systematic reviewing. Logic models offer a framework to help reviewers to ‘think’ conceptually at various points during the review, and can be a useful tool in defining study inclusion and exclusion criteria, guiding the search strategy, identifying relevant outcomes, identifying mediating and moderating factors, and communicating review findings. Methods and Findings In this paper we critique the use of logic models in systematic reviews and protocols drawn from two databases representing reviews of health interventions and international development interventions. Programme theory featured only in a minority of the reviews and protocols included. Despite drawing from different disciplinary traditions, reviews and protocols from both sources shared several limitations in their use of logic models and theories of change, and these were used almost unanimously to solely depict pictorially the way in which the intervention worked. Logic models and theories of change were consequently rarely used to communicate the findings of the review. Conclusions Logic models have the potential to be an aid integral throughout the systematic reviewing process. The absence of good practice around their use and development may be one reason for the apparent limited utility of logic models in many existing systematic reviews. These concerns are addressed in the second half of this paper, where we offer a set of principles in the use of logic models and an example of how we constructed a logic model for a review of school-based asthma interventions. PMID:26575182

Methodological quality of systematic reviews on influenza vaccination.

PubMed

Remschmidt, Cornelius; Wichmann, Ole; Harder, Thomas

2014-03-26

There is a growing body of evidence on the risks and benefits of influenza vaccination in various target groups. Systematic reviews are of particular importance for policy decisions. However, their methodological quality can vary considerably. To investigate the methodological quality of systematic reviews on influenza vaccination (efficacy, effectiveness, safety) and to identify influencing factors. A systematic literature search on systematic reviews on influenza vaccination was performed, using MEDLINE, EMBASE and three additional databases (1990-2013). Review characteristics were extracted and the methodological quality of the reviews was evaluated using the assessment of multiple systematic reviews (AMSTAR) tool. U-test, Kruskal-Wallis test, chi-square test, and multivariable linear regression analysis were used to assess the influence of review characteristics on AMSTAR-score. Fourty-six systematic reviews fulfilled the inclusion criteria. Average methodological quality was high (median AMSTAR-score: 8), but variability was large (AMSTAR range: 0-11). Quality did not differ significantly according to vaccination target group. Cochrane reviews had higher methodological quality than non-Cochrane reviews (p=0.001). Detailed analysis showed that this was due to better study selection and data extraction, inclusion of unpublished studies, and better reporting of study characteristics (all p<0.05). In the adjusted analysis, no other factor, including industry sponsorship or journal impact factor had an influence on AMSTAR score. Systematic reviews on influenza vaccination showed large differences regarding their methodological quality. Reviews conducted by the Cochrane collaboration were of higher quality than others. When using systematic reviews to guide the development of vaccination recommendations, the methodological quality of a review in addition to its content should be considered. Copyright © 2014 Elsevier Ltd. All rights reserved.

Communication Partner Training in Aphasia: An Updated Systematic Review.

PubMed

Simmons-Mackie, Nina; Raymer, Anastasia; Cherney, Leora R

2016-12-01

To update a previous systematic review describing the effect of communication partner training on individuals with aphasia and their communication partners, with clinical questions addressing effects of partner training on language, communication activity/participation, psychosocial adjustment, and quality of life. Twelve electronic databases were searched using 23 search terms. References from relevant articles were hand searched. Three reviewers independently reviewed abstracts, excluding those that failed to meet inclusion criteria. Thirty-two full text articles were reviewed by 2 independent reviewers. Articles not meeting inclusion criteria were eliminated, resulting in a corpus of 25 articles for full review. For the 25 articles, 1 reviewer extracted descriptive data regarding participants, intervention, outcome measures, and results. A second reviewer verified the accuracy of the extracted data. The 3-member review team classified studies using the American Academy of Neurology levels of evidence. Two independent reviewers evaluated each article using design-specific tools to assess research quality. All 25 of the current review articles reported positive changes from partner training. Therefore, to date, 56 studies across 2 systematic reviews have reported positive outcomes from communication partner training in aphasia. The results of the current review are consistent with the previous review and necessitate no change to the earlier recommendations, suggesting that communication partner training should be conducted to improve partner skill in facilitating the communication of people with chronic aphasia. Additional high-quality research is needed to strengthen the original 2010 recommendations and expand recommendations to individuals with acute aphasia. High-quality clinical trials are also needed to demonstrate implementation of communication partner training in complex environments (eg, health care). Copyright © 2016 American Congress of Rehabilitation Medicine. Published by Elsevier Inc. All rights reserved.

The Effects of Resident Peer- and Self-Chart Review on Outpatient Laboratory Result Follow-up.

PubMed

Hale, Andrew J; Nall, Ryan W; Mukamal, Kenneth J; Libman, Howard; Smith, C Christopher; Sternberg, Scot B; Kim, Hans S; Kriegel, Gila

2016-05-01

Performing and teaching appropriate follow-up of outpatient laboratory results (LRs) is a challenge. The authors tested peer-review among residents as a potentially valuable intervention. Investigators assigned residents to perform self-review (n = 27), peer-review (n = 21), or self- + peer-review (n = 30) of outpatient charts. They also compared residence performance with that of historical controls (n = 20). In September 2012, residents examined 10 LRs from April 2012 onward. A second review in November 2012 ascertained whether performing chart review improved residents' practice behaviors. Initially, the least-square (LS) mean number of LRs without documentation of follow-up per resident in the self-, peer-, and self- + peer-review group was, respectively, 0.5 (SD 1.0), 1.0 (SD 1.7), and 0.9 (SD 1.3), and post intervention, this was 1.0 (SD 0.2), 0.3 (SD 0.2), and 0.6 (SD 0.2) (self- versus peer-review P = .03). Initially the LS mean follow-up time per resident in the self-, peer-, and self- + peer-review group was, respectively, 4.2 (SD 1.2), 6.9 (SD 1.4), and 5.9 (SD 1.2) days, and after the intervention, LS mean time was 5.0 (SD 0.5), 2.5 (SD 0.6), and 3.9 (SD 0.5) days (self- versus peer-review P < .01). Self-review was not associated with significant improvements in practice. In this comparison of self- and peer-review, only residents who performed peer-review demonstrated significant improvements in their documentation practices. These findings support the use of resident peer-review in improving LR follow-up, and potentially, in other, broader resident quality improvement initiatives.

Peer review of health research funding proposals: A systematic map and systematic review of innovations for effectiveness and efficiency.

PubMed

Shepherd, Jonathan; Frampton, Geoff K; Pickett, Karen; Wyatt, Jeremy C

2018-01-01

To investigate methods and processes for timely, efficient and good quality peer review of research funding proposals in health. A two-stage evidence synthesis: (1) a systematic map to describe the key characteristics of the evidence base, followed by (2) a systematic review of the studies stakeholders prioritised as relevant from the map on the effectiveness and efficiency of peer review 'innovations'. Standard processes included literature searching, duplicate inclusion criteria screening, study keyword coding, data extraction, critical appraisal and study synthesis. A total of 83 studies from 15 countries were included in the systematic map. The evidence base is diverse, investigating many aspects of the systems for, and processes of, peer review. The systematic review included eight studies from Australia, Canada, and the USA, evaluating a broad range of peer review innovations. These studies showed that simplifying the process by shortening proposal forms, using smaller reviewer panels, or expediting processes can speed up the review process and reduce costs, but this might come at the expense of peer review quality, a key aspect that has not been assessed. Virtual peer review using videoconferencing or teleconferencing appears promising for reducing costs by avoiding the need for reviewers to travel, but again any consequences for quality have not been adequately assessed. There is increasing international research activity into the peer review of health research funding. The studies reviewed had methodological limitations and variable generalisability to research funders. Given these limitations it is not currently possible to recommend immediate implementation of these innovations. However, many appear promising based on existing evidence, and could be adapted as necessary by funders and evaluated. Where feasible, experimental evaluation, including randomised controlled trials, should be conducted, evaluating impact on effectiveness, efficiency and quality.

Systematic reviews of surgical procedures in children: quantity, coverage and quality.

PubMed

McGee, Richard G; Craig, Jonathan C; Rogerson, Thomas E; Webster, Angela C

2013-04-01

Systematic reviews have the potential to map those areas where children are under-represented in surgical research. We aimed to describe and evaluate the quantity, coverage and the quality of conduct and reporting of systematic reviews of surgical procedures in children. We searched four biomedical databases, a systematic review register, reference lists and conducted hand searching to identify relevant reviews. Two reviewers worked independently to critically appraise included studies and abstract data. We assessed reporting quality using the preferred reporting items for systematic reviews and meta-analysis statement and methodological quality using the Assessment of Multiple SysTemAtic Reviews tool. Fifteen systematic reviews were identified, representing 0.01% of all paediatric surgical citations in MEDLINE and Embase. Thirteen of the reviews were Cochrane reviews, and most reviews (12/15) addressed subspecialty interests such as otorhinolaryngology. The median number of included trials per systematic review was four (interquartile range 1 to 9.5), the median number of primary outcomes was 5.5 (interquartile range 3.5 to 7.5). In general, reporting and methodological quality was good although there were several omissions, particularly around completeness of reporting of statistical methods used, and utilisation of quality assessments in analyses. Outcomes were often not clearly defined and descriptions of procedures lacked sufficient detail to determine the similarities and differences among surgical procedures within the contributing trials. Systematic reviews of surgical procedures in children are rarely published. To improve the evidence base and guide research agendas, more systematic reviews should be conducted, using standard guidelines for conduct and reporting. © 2013 The Authors. Journal of Paediatrics and Child Health © 2013 Paediatrics and Child Health Division (Royal Australasian College of Physicians).

Effects of life review on mental health and well-being among cancer patients: A systematic review.

PubMed

Zhang, Xiaoling; Xiao, Huimin; Chen, Ying

2017-09-01

Cancer patients often experience psychological distress. Life review has increasingly been used to enhance their mental health and well-being. However, no systematic review has synthesized the evidence, and its effects remain unclear. To examine and synthesize the best available evidence on the effects of life review on mental health and well-being among cancer patients. Systematic review of randomized controlled trials and clinical controlled trials. Twelve electronic databases were searched for published studies reported in English or Chinese, from inception to September 2016. Other supplementary sources, such as related websites, professional books, reference lists, and author contacts were also used for published or unpublished studies. A comprehensive literature search was conducted to identify eligible randomized controlled trials or clinical controlled trials about the effects of life review on cancer patients. Study selection, quality assessment, and data extraction were independently performed by two reviewers. The results were synthesized without meta-analysis in this review. Fifteen studies (899 participants) were identified; of that total, nine studies were rated as strong in quality, while six studies were of moderate quality. In addition to structured life review interviews, other elements such as memory prompts and a legacy product were integrated into life review programs. A majority of studies indicated that life review programs benefited cancer patients by reducing depression and anxiety, as well as improving their sense of hope, self-esteem and quality of life. Life review can improve mental health and well-being among cancer patients. This suggests that life review can be integrated into typical cancer treatment to enhance patients' mental health and well-being. More research with rigorous design is necessary to further explore the effects of life review. Copyright © 2017 Elsevier Ltd. All rights reserved.

From the closest observers of patient care: a thematic analysis of online narrative reviews of hospitals.

PubMed

Bardach, Naomi S; Lyndon, Audrey; Asteria-Peñaloza, Renée; Goldman, L Elizabeth; Lin, Grace A; Dudley, R Adams

2016-11-01

Patient-centred care has become a priority in many countries. It is unknown whether current tools capture aspects of care patients and their surrogates consider important. We investigated whether online narrative reviews from patients and surrogates reflect domains in the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) and we described additional potential domains. We used thematic analysis to assess online narrative reviews for reference to HCAHPS domains and salient non-HCAHPS domains and compared results by reviewer type (patient vs surrogate). We identified hospitals for review from the American Hospital Association database using a stratified random sampling approach. This approach ensured inclusion of reviews of a diverse set of hospitals. We searched online in February 2013 for narrative reviews from any source for each hospital. We included up to two narrative reviews for each hospital. Outpatient or emergency department reviews, reviews from self-identified hospital employees, or reviews of <10 words. 50.0% (n=122) of reviews (N=244) were from patients and 38.1% (n=93) from friends or family members. Only 57.0% (n=139) of reviews mentioned any HCAHPS domain. Additional salient domains were: Financing, including unexpected out-of-pocket costs and difficult interactions with billing departments; system-centred care; and perceptions of safety. These domains were mentioned in 51.2% (n=125) of reviews. Friends and family members commented on perceptions of safety more frequently than patients. A substantial proportion of consumer reviews do not mention HCAHPS domains. Surrogates appear to observe care differently than patients, particularly around safety. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/.

Simulation Modelling in Healthcare: An Umbrella Review of Systematic Literature Reviews.

PubMed

Salleh, Syed; Thokala, Praveen; Brennan, Alan; Hughes, Ruby; Booth, Andrew

2017-09-01

Numerous studies examine simulation modelling in healthcare. These studies present a bewildering array of simulation techniques and applications, making it challenging to characterise the literature. The aim of this paper is to provide an overview of the level of activity of simulation modelling in healthcare and the key themes. We performed an umbrella review of systematic literature reviews of simulation modelling in healthcare. Searches were conducted of academic databases (JSTOR, Scopus, PubMed, IEEE, SAGE, ACM, Wiley Online Library, ScienceDirect) and grey literature sources, enhanced by citation searches. The articles were included if they performed a systematic review of simulation modelling techniques in healthcare. After quality assessment of all included articles, data were extracted on numbers of studies included in each review, types of applications, techniques used for simulation modelling, data sources and simulation software. The search strategy yielded a total of 117 potential articles. Following sifting, 37 heterogeneous reviews were included. Most reviews achieved moderate quality rating on a modified AMSTAR (A Measurement Tool used to Assess systematic Reviews) checklist. All the review articles described the types of applications used for simulation modelling; 15 reviews described techniques used for simulation modelling; three reviews described data sources used for simulation modelling; and six reviews described software used for simulation modelling. The remaining reviews either did not report or did not provide enough detail for the data to be extracted. Simulation modelling techniques have been used for a wide range of applications in healthcare, with a variety of software tools and data sources. The number of reviews published in recent years suggest an increased interest in simulation modelling in healthcare.

Evaluation of the antipsychotic medication review process at four long-term facilities in Alberta.

PubMed

Birney, Arden; Charland, Paola; Cole, Mollie; Aslam Arain, Mubashir

2016-01-01

The goal of this evaluation was to understand how four long-term care (LTC) facilities in Alberta have implemented medication reviews for the Appropriate Use of Antipsychotics (AUA) initiative. We aimed to determine how interprofessional (IP) collaboration was incorporated in the antipsychotic medication reviews and how the reviews had been sustained. Four LTC facilities in Alberta participated in this evaluation. We conducted semistructured interviews with 18 facility staff and observed one antipsychotic medication review at each facility. We analyzed data according to the following key components that we identified as relevant to the antipsychotic medication reviews: the structure of the reviews, IP interactions between the staff members, and strategies for sustaining the reviews. The duration of antipsychotic medication reviews ranged from 1 to 1.5 hours. The number of professions in attendance ranged from 3 to 9; a pharmacist led the review at two sites, while a registered nurse led the review at one site and a nurse practitioner at the remaining site. The number of residents discussed during the review ranged from 6 to 20. The process at some facilities was highly IP, demonstrating each of the six IP competencies. Other facilities conducted the review in a less IP manner due to challenges of physician involvement and staff workload, particularly of health care aides. Facilities that had an nurse practitioner on site were more efficient with the process of implementing recommendations resulting from the medication reviews. The LTC facilities were successful in implementing the medication review process and the process seemed to be sustainable. A few challenges were observed in the implementation process at two facilities. IP practice moved forward the goals of the AUA initiative to reduce the inappropriate use of antipsychotics.

Oral Chinese proprietary medicine for angina pectoris: an overview of systematic reviews/meta-analyses.

PubMed

Luo, Jing; Xu, Hao; Yang, Guoyan; Qiu, Yu; Liu, Jianping; Chen, Keji

2014-08-01

Oral Chinese proprietary medicine (CPM) is commonly used to treat angina pectoris, and many relevant systematic reviews/meta-analyses are available. However, these reviews have not been systematically summarized and evaluated. We conducted an overview of these reviews, and explored their methodological and reporting quality to inform both practice and further research. We included systematic reviews/meta-analyses on oral CPM in treating angina until March 2013 by searching PubMed, Embase, the Cochrane Library and four Chinese databases. We extracted data according to a pre-designed form, and assessed the methodological and reporting characteristics of the reviews in terms of AMSTAR and PRISMA respectively. Most of the data analyses were descriptive. 36 systematic reviews/meta-analyses involving over 82,105 participants with angina reviewing 13 kinds of oral CPM were included. The main outcomes assessed in the reviews were surrogate outcomes (34/36, 94.4%), adverse events (31/36, 86.1%), and symptoms (30/36, 83.3%). Six reviews (6/36, 16.7%) drew definitely positive conclusions, while the others suggested potential benefits in the symptoms, electrocardiogram, and adverse events. The overall methodological and reporting quality of the reviews was limited, with many serious flaws such as the lack of review protocol and incomprehensive literature searches. Though many systematic reviews/meta-analyses on oral CPM for angina suggested potential benefits or definitely positive effects, stakeholders should interpret the findings of these reviews with caution, considering the overall limited methodological and reporting quality. We recommend further studies should be appropriately conducted and systematic reviews reported according to PRISMA standard. Copyright © 2014 Elsevier Ltd. All rights reserved.

Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation.

PubMed

Shamseer, Larissa; Moher, David; Clarke, Mike; Ghersi, Davina; Liberati, Alessandro; Petticrew, Mark; Shekelle, Paul; Stewart, Lesley A

2015-01-02

Protocols of systematic reviews and meta-analyses allow for planning and documentation of review methods, act as a guard against arbitrary decision making during review conduct, enable readers to assess for the presence of selective reporting against completed reviews, and, when made publicly available, reduce duplication of efforts and potentially prompt collaboration. Evidence documenting the existence of selective reporting and excessive duplication of reviews on the same or similar topics is accumulating and many calls have been made in support of the documentation and public availability of review protocols. Several efforts have emerged in recent years to rectify these problems, including development of an international register for prospective reviews (PROSPERO) and launch of the first open access journal dedicated to the exclusive publication of systematic review products, including protocols (BioMed Central's Systematic Reviews). Furthering these efforts and building on the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) guidelines, an international group of experts has created a guideline to improve the transparency, accuracy, completeness, and frequency of documented systematic review and meta-analysis protocols--PRISMA-P (for protocols) 2015. The PRISMA-P checklist contains 17 items considered to be essential and minimum components of a systematic review or meta-analysis protocol.This PRISMA-P 2015 Explanation and Elaboration paper provides readers with a full understanding of and evidence about the necessity of each item as well as a model example from an existing published protocol. This paper should be read together with the PRISMA-P 2015 statement. Systematic review authors and assessors are strongly encouraged to make use of PRISMA-P when drafting and appraising review protocols. © BMJ Publishing Group Ltd 2014.

NASA Standing Review Board Handbook: Revision B

NASA Technical Reports Server (NTRS)

Amer, Tahani R.; Ortiz, James N.; Calloway, Michelle A.; Greathouse, Richard M.; Polen, Charles A.; Baker, Ronald L.; Borchardt, Heidemarie E.; Chromik, Christopher C.; Moran, Erin; Paraska, Michael W.;

The inclusion of reports of randomised trials published in languages other than English in systematic reviews.

PubMed

Moher, D; Pham, B; Lawson, M L; Klassen, T P

2003-01-01

To assemble a large dataset of language restricted and language inclusive systematic reviews, including both conventional medicinal (CM) and complementary and alternative medicine (CAM) interventions. To then assess the quality of these reports by considering and comparing different types of systematic reviews and their associated RCTs; CM and CAM interventions; the effect of language restrictions compared with language inclusions, and whether these results are influenced by other issues, including statistical heterogeneity and publication bias, in the systematic review process. MEDLINE, EMBASE, the Cochrane Database of Systematic Reviews and the Centralised Information Service for Complementary Medicine. Three types of systematic reviews were included: language restricted; language inclusive/English language (EL) reviews that searched RCTs in languages other than English (LOE) but did not find any and, hence, could not include any, in the quantitative data synthesis; and systematic reviews that searched for RCTs in LOE and included them in the quantitative data synthesis. Fisher's exact test was applied to compare the three different types of systematic reviews with respect to their reporting characteristics and the systematic review quality assessment tool. The odds ratio of LOE trials versus EL trials was computed for each review and this information was pooled across the reviews to examine the influence that language of publication and type of intervention (CM, CAM) have on the estimates of intervention effect. Several sensitivity analyses were performed. The LOE RCTs were predominantly in French and German. Language inclusive/LOE systematic reviews were of the highest quality compared with the other types of reviews. The CAM reviews were of higher quality compared with the CM reviews. There were only minor differences in the quality of reports of EL RCTs compared with the eight other languages considered. However, there are inconsistent differences in the quality of LOE reports depending on the intervention type. The results, and those reported previously, suggest that excluding reports of RCTs in LOE from the analytical part of a systematic review is reasonable. Because the present research and previous efforts have not included every type of CM RCT and the resulting possibility of the uncertainty as to when bias will be present by excluding LOE, it is always prudent to perform a comprehensive search for all evidence. This result only applies to reviews investigating the benefits of CM interventions. This does not imply that systematic reviewers should neglect reports in LOE. We recommend that systematic reviewers search for reports regardless of the language. There may be merit in including them in some aspects of the review process although this decision is likely to depend on several factors, including fiscal and other resources being available. Language restrictions significantly shift the estimates of an intervention's effectiveness when the intervention is CAM. Here, excluding trials reported in LOE, compared with their inclusion, resulted in a reduced intervention effect. The present results do not appear to be influenced by statistical heterogeneity and publication bias. With the exception of CAM systematic reviews, the quality of recently published systematic reviews is less than optimal. Language inclusive/LOE systematic reviews appear to be a marker for a better quality systematic review. Language restrictions do not appear to bias the estimates of a conventional intervention's effectiveness. However, there is substantial bias in the results of a CAM systematic review if LOE reports are excluded from it.

IRIS Toxicological Review of 1,4-Dioxane (Interagency ...

EPA Pesticide Factsheets

On September 9, 2011, the Toxicological Review of 1,4-Dioxane (with inhalation update) (External Review Draft) was posted for external peer review and public comment. The Toxicological Review and charge were reviewed internally by EPA and by other federal agencies and White House Offices before public release. In the new IRIS process, introduced by the EPA Administrator, all written comments on IRIS assessments submitted by other federal agencies and White House Offices will be made publicly available. Accordingly, interagency comments with EPA's response and the interagency science consultation draft of the IRIS Toxicological Review of 1,4-dioxane and the charge to external peer reviewers are posted on this site. The Toxicological Review of 1,4-dioxane provides scientific support and rationale for the hazard and dose-response assessment pertaining to chronic exposure to 1,4-dioxane.

Surveys of current status in biomedical science grant review: funding organisations' and grant reviewers' perspectives

PubMed Central

2010-01-01

Background The objectives of this research were (a) to describe the current status of grant review for biomedical projects and programmes from the perspectives of international funding organisations and grant reviewers, and (b) to explore funders' interest in developing uniform requirements for grant review aimed at making the processes and practices of grant review more consistent, transparent, and user friendly. Methods A survey to a convenience sample of 57 international public and private organisations that give grants for biomedical research was conducted. Nine participating organisations then emailed a random sample of their external reviewers an invitation to participate in a second electronic survey. Results A total of 28 of 57 (49%) organisations in 19 countries responded. Organisations reported these problems as frequent or very frequent: declined review requests (16), late reports (10), administrative burden (7), difficulty finding new reviewers (4), and reviewers not following guidelines (4). The administrative burden of the process was reported to have increased over the past 5 years. In all, 17 organisations supported the idea of uniform requirements for conducting grant review and for formatting grant proposals. A total of 258/418 (62%) reviewers responded from 22 countries. Of those, 48% (123/258) said their institutions encouraged grant review, yet only 7% (17/258) were given protected time and 74% (192/258) received no academic recognition for this. Reviewers rated these factors as extremely or very important in deciding to review proposals: 51% (131/258) desire to support external fairness, 47% (120/258) professional duty, 46% (118/258) relevance of the proposal's topic, 43% (110/258) wanting to keep up to date, 40% (104/258) desire to avoid suppression of innovation. Only 16% (42/258) reported that guidance from funders was very clear. In all, 85% (220/258) had not been trained in grant review and 64% (166/258) wanted this. Conclusions Funders reported a growing workload of biomedical proposals that is getting harder to peer review. Just under half of grant reviewers take part for the good of science and professional development, but many report lack of academic and practical support and clear guidance. Around two-thirds of funders supported the development of uniform requirements for the format and peer review of proposals to help ease the current situation. PMID:20961441

Pedagogical Merit Review of Animal Use for Education in Canada

PubMed Central

Griffin, Gilly

2016-01-01

There are two components to the review of animal based protocols in Canada: review for the merit of the study itself, and review of the ethical acceptability of the work. Despite the perceived importance for the quality assurance these reviews provide; there are few studies of the peer-based merit review system for animal-based protocols for research and education. Institutional animal care committees (ACC)s generally rely on the external peer review of scientific merit for animal-based research. In contrast, peer review for animal based teaching/training is dependent on the review of pedagogical merit carried out by the ACC itself or another committee within the institution. The objective of this study was to evaluate the views of ACC members about current practices and policies as well as alternate policies for the review of animal based teaching/training. We conducted a national web-based survey of ACC members with both quantitative and qualitative response options. Responses from 167 ACC members indicated broad concerns about administrative burden despite strong support for both the current and alternate policies. Participants’ comments focused mostly on the merit review process (54%) relative to the efficiency (21%), impact (13%), and other (12%) aspects of evaluation. Approximately half (49%) of the comments were classified into emergent themes that focused on some type of burden: burden from additional pedagogical merit review (16%), a limited need for the review (12%), and a lack of resources (expertise 11%; people/money 10%). Participants indicated that the current system for pedagogical merit review is effective (60%); but most also indicated that there was at least some challenge (86%) with the current peer review process. There was broad support for additional guidance on the justification, criteria, types of animal use, and objectives of pedagogical merit review. Participants also supported the ethical review and application of the Three Rs in the review process. A clear priority from participants in the survey was updating guidance to better facilitate the merit review process of animal-based protocols for education. Balancing the need for improved guidance with the reality of limited resources at local institutions will be essential to do this successfully; a familiar dilemma to both scientists and policy makers alike. PMID:27352243

Pedagogical Merit Review of Animal Use for Education in Canada.

PubMed

Avey, Marc T; Griffin, Gilly

2016-01-01

There are two components to the review of animal based protocols in Canada: review for the merit of the study itself, and review of the ethical acceptability of the work. Despite the perceived importance for the quality assurance these reviews provide; there are few studies of the peer-based merit review system for animal-based protocols for research and education. Institutional animal care committees (ACC)s generally rely on the external peer review of scientific merit for animal-based research. In contrast, peer review for animal based teaching/training is dependent on the review of pedagogical merit carried out by the ACC itself or another committee within the institution. The objective of this study was to evaluate the views of ACC members about current practices and policies as well as alternate policies for the review of animal based teaching/training. We conducted a national web-based survey of ACC members with both quantitative and qualitative response options. Responses from 167 ACC members indicated broad concerns about administrative burden despite strong support for both the current and alternate policies. Participants' comments focused mostly on the merit review process (54%) relative to the efficiency (21%), impact (13%), and other (12%) aspects of evaluation. Approximately half (49%) of the comments were classified into emergent themes that focused on some type of burden: burden from additional pedagogical merit review (16%), a limited need for the review (12%), and a lack of resources (expertise 11%; people/money 10%). Participants indicated that the current system for pedagogical merit review is effective (60%); but most also indicated that there was at least some challenge (86%) with the current peer review process. There was broad support for additional guidance on the justification, criteria, types of animal use, and objectives of pedagogical merit review. Participants also supported the ethical review and application of the Three Rs in the review process. A clear priority from participants in the survey was updating guidance to better facilitate the merit review process of animal-based protocols for education. Balancing the need for improved guidance with the reality of limited resources at local institutions will be essential to do this successfully; a familiar dilemma to both scientists and policy makers alike.

77 FR 53206 - National Institute of Allergy and Infectious Diseases; Notice of Closed Meetings

Federal Register 2010, 2011, 2012, 2013, 2014

2012-08-31

...). Contact Person: Maryam Feili-Hariri, Ph.D., Scientific Review Officer, Immunology Review Branch... Feili-Hariri, Ph.D., Scientific Review Officer, Immunology Review Branch, Scientific Review Program....gov . (Catalogue of Federal Domestic Assistance Program Nos. 93.855, Allergy, Immunology, and...

48 CFR 219.7106 - Performance reviews.

Code of Federal Regulations, 2010 CFR

2010-10-01

... 48 Federal Acquisition Regulations System 3 2010-10-01 2010-10-01 false Performance reviews. 219... Performance reviews. The Defense Contract Management Agency will conduct annual performance reviews of all... reviews should be a major factor in determinations of amounts of reimbursement, if any, that the mentor...

14 CFR 415.57 - Payload review.

Code of Federal Regulations, 2011 CFR

2011-01-01

... Aeronautics and Space COMMERCIAL SPACE TRANSPORTATION, FEDERAL AVIATION ADMINISTRATION, DEPARTMENT OF TRANSPORTATION LICENSING LAUNCH LICENSE Payload Review and Determination § 415.57 Payload review. (a) Timing. A payload review may be conducted as part of a license application review or may be requested by a payload...

75 FR 35075 - Center for Scientific Review; Notice of Closed Meetings

Federal Register 2010, 2011, 2012, 2013, 2014

2010-06-21

... DEPARTMENT OF HEALTH AND HUMAN SERVICES National Institutes of Health Center for Scientific Review... unwarranted invasion of personal privacy. Name of Committee: Center for Scientific Review Special Emphasis..., PhD, Scientific Review Officer, Center for Scientific Review, National Institutes of Health, 6701...

49 CFR 552.6 - Technical review.

Code of Federal Regulations, 2010 CFR

2010-10-01

... § 552.6 Technical review. The appropriate Associate Administrator conducts a technical review of the petition. The technical review may consist of an analysis of the material submitted, together with... 49 Transportation 6 2010-10-01 2010-10-01 false Technical review. 552.6 Section 552.6...

IRIS Toxicological Review of Mirex (2003 External Review Draft)

EPA Science Inventory

EPA is providing for public information a draft Toxicological Review, draft IRIS Summary, and charge to external peer reviewers for EPA's Health Assessment of Mirex. These documents are provided for public viewing during an external scientific peer review period. While EPA is not...

75 FR 14214 - Senior Executive Service Performance Review Board Membership

Federal Register 2010, 2011, 2012, 2013, 2014

2010-03-24

... OCCUPATIONAL SAFETY AND HEALTH REVIEW COMMISSION Senior Executive Service Performance Review Board Membership AGENCY: Occupational Safety and Health Review Commission. ACTION: Annual notice. SUMMARY: Notice... established a Senior Executive Service PRB. The PRB reviews and evaluates the initial appraisal of a senior...

78 FR 5516 - Senior Executive Service Performance Review Board Membership

Federal Register 2010, 2011, 2012, 2013, 2014

2013-01-25

... OCCUPATIONAL SAFETY AND HEALTH REVIEW COMMISSION Senior Executive Service Performance Review Board Membership AGENCY: Occupational Safety and Health Review Commission. ACTION: Annual notice. SUMMARY: Notice... established a Senior Executive Service PRB. The PRB reviews and evaluates the initial appraisal of a senior...

76 FR 39926 - Senior Executive Service Performance Review Board Membership

Federal Register 2010, 2011, 2012, 2013, 2014

2011-07-07

... OCCUPATIONAL SAFETY AND HEALTH REVIEW COMMISSION Senior Executive Service Performance Review Board Membership AGENCY: Occupational Safety and Health Review Commission. ACTION: Annual notice. SUMMARY: Notice... established a Senior Executive Service PRB. The PRB reviews and evaluates the initial appraisal of a senior...

75 FR 6729 - Senior Executive Service (SES) Performance Review Board; Members

Federal Register 2010, 2011, 2012, 2013, 2014

2010-02-10

... NATIONAL ARCHIVES AND RECORDS ADMINISTRATION Senior Executive Service (SES) Performance Review... Performance Review Boards. The Board shall review the initial appraisal of a senior executive's performance by... senior executive performance. The members of the Performance Review Board for the National Archives and...

76 FR 81998 - Senior Executive Service Performance Review Board Membership

Federal Register 2010, 2011, 2012, 2013, 2014

2011-12-29

... OCCUPATIONAL SAFETY AND HEALTH REVIEW COMMISSION Senior Executive Service Performance Review Board Membership AGENCY: Occupational Safety and Health Review Commission. ACTION: Annual notice. SUMMARY: Notice... established a Senior Executive Service PRB. The PRB reviews and evaluates the initial appraisal of a senior...

47 CFR 8.17 - Review.

Code of Federal Regulations, 2012 CFR

2012-10-01

... 47 Telecommunication 1 2012-10-01 2012-10-01 false Review. 8.17 Section 8.17 Telecommunication FEDERAL COMMUNICATIONS COMMISSION GENERAL PRESERVING THE OPEN INTERNET § 8.17 Review. (a) Interlocutory review. (1) Except as provided below, no party may seek review of interlocutory rulings until a decision...

47 CFR 8.17 - Review.

Code of Federal Regulations, 2013 CFR

2013-10-01

... 47 Telecommunication 1 2013-10-01 2013-10-01 false Review. 8.17 Section 8.17 Telecommunication FEDERAL COMMUNICATIONS COMMISSION GENERAL PRESERVING THE OPEN INTERNET § 8.17 Review. (a) Interlocutory review. (1) Except as provided below, no party may seek review of interlocutory rulings until a decision...

47 CFR 8.17 - Review.

Code of Federal Regulations, 2014 CFR

2014-10-01

... 47 Telecommunication 1 2014-10-01 2014-10-01 false Review. 8.17 Section 8.17 Telecommunication FEDERAL COMMUNICATIONS COMMISSION GENERAL PRESERVING THE OPEN INTERNET § 8.17 Review. (a) Interlocutory review. (1) Except as provided below, no party may seek review of interlocutory rulings until a decision...

40 CFR 155.50 - Initiate a pesticide's registration review.

Code of Federal Regulations, 2013 CFR

2013-07-01

... 40 Protection of Environment 25 2013-07-01 2013-07-01 false Initiate a pesticide's registration...) PESTICIDE PROGRAMS REGISTRATION STANDARDS AND REGISTRATION REVIEW Registration Review Procedures § 155.50 Initiate a pesticide's registration review. The Agency will initiate a pesticide's registration review by...

40 CFR 155.50 - Initiate a pesticide's registration review.

Code of Federal Regulations, 2014 CFR

2014-07-01

... 40 Protection of Environment 24 2014-07-01 2014-07-01 false Initiate a pesticide's registration...) PESTICIDE PROGRAMS REGISTRATION STANDARDS AND REGISTRATION REVIEW Registration Review Procedures § 155.50 Initiate a pesticide's registration review. The Agency will initiate a pesticide's registration review by...

40 CFR 155.50 - Initiate a pesticide's registration review.

Code of Federal Regulations, 2012 CFR

2012-07-01

... 40 Protection of Environment 25 2012-07-01 2012-07-01 false Initiate a pesticide's registration...) PESTICIDE PROGRAMS REGISTRATION STANDARDS AND REGISTRATION REVIEW Registration Review Procedures § 155.50 Initiate a pesticide's registration review. The Agency will initiate a pesticide's registration review by...

40 CFR 155.50 - Initiate a pesticide's registration review.

Code of Federal Regulations, 2011 CFR

2011-07-01

... 40 Protection of Environment 24 2011-07-01 2011-07-01 false Initiate a pesticide's registration...) PESTICIDE PROGRAMS REGISTRATION STANDARDS AND REGISTRATION REVIEW Registration Review Procedures § 155.50 Initiate a pesticide's registration review. The Agency will initiate a pesticide's registration review by...

77 FR 59935 - Center for Scientific Review Notice of Closed Meetings

Federal Register 2010, 2011, 2012, 2013, 2014

2012-10-01

... DEPARTMENT OF HEALTH AND HUMAN SERVICES National Institutes of Health Center for Scientific Review....m. Agenda: To review and evaluate grant applications. Place: National Institutes of Health, 6701... Review Officer, Center for Scientific, Review National Institutes of Health, 6701 Rockledge Drive, Room...

42 CFR 476.160 - General quality of care review procedures.

Code of Federal Regulations, 2013 CFR

2013-10-01

... 42 Public Health 4 2013-10-01 2013-10-01 false General quality of care review procedures. 476.160... SERVICES (CONTINUED) QUALITY IMPROVEMENT ORGANIZATIONS UTILIZATION AND QUALITY CONTROL REVIEW Review Responsibilities of Utilization and Quality Control Quality Improvement Organizations (QIOs) Qio Review Functions...

Caveat Lector: Reviewing Popular Social Science.

ERIC Educational Resources Information Center

Hixson, Vivian Scott

1981-01-01

Discusses problems with reviews and criticisms of popular social science books: the quality and background of reviewers, the difficulty of distinguishing between fact and opinion, and the scarcity of competent reviewers. Analyzes reviews of Robert Ardrey's "African Genesis" and "The Territorial Imperative," Konrad Lorenz's "On Aggression," and…

49 CFR 552.6 - Technical review.

Code of Federal Regulations, 2011 CFR

2011-10-01

... § 552.6 Technical review. The appropriate Associate Administrator conducts a technical review of the petition. The technical review may consist of an analysis of the material submitted, together with... 49 Transportation 6 2011-10-01 2011-10-01 false Technical review. 552.6 Section 552.6...

47 CFR 8.17 - Review.

Code of Federal Regulations, 2011 CFR

2011-10-01

... 47 Telecommunication 1 2011-10-01 2011-10-01 false Review. 8.17 Section 8.17 Telecommunication FEDERAL COMMUNICATIONS COMMISSION GENERAL PRESERVING THE OPEN INTERNET § 8.17 Review. (a) Interlocutory review. (1) Except as provided below, no party may seek review of interlocutory rulings until a decision...

24 CFR 50.36 - Updating of environmental reviews.

Code of Federal Regulations, 2011 CFR

2011-04-01

... 24 Housing and Urban Development 1 2011-04-01 2011-04-01 false Updating of environmental reviews... Urban Development PROTECTION AND ENHANCEMENT OF ENVIRONMENTAL QUALITY Environmental Assessments and Related Reviews § 50.36 Updating of environmental reviews. The environmental review must be re-evaluated...

24 CFR 50.36 - Updating of environmental reviews.

Code of Federal Regulations, 2010 CFR

2010-04-01

... 24 Housing and Urban Development 1 2010-04-01 2010-04-01 false Updating of environmental reviews... Urban Development PROTECTION AND ENHANCEMENT OF ENVIRONMENTAL QUALITY Environmental Assessments and Related Reviews § 50.36 Updating of environmental reviews. The environmental review must be re-evaluated...

78 FR 26057 - Extension of Agency Information Collection Activity Under OMB Review: Pipeline Corporate Security...

Federal Register 2010, 2011, 2012, 2013, 2014

2013-05-03

... Information Collection Activity Under OMB Review: Pipeline Corporate Security Review AGENCY: Transportation.... Information Collection Requirement Title: Pipeline Corporate Security Review (PCSR). Type of Request... current industry security practices through its Pipeline Corporate Security Review (PCSR) program. The...

24 CFR 92.550 - Performance reviews.

Code of Federal Regulations, 2011 CFR

2011-04-01

... 24 Housing and Urban Development 1 2011-04-01 2011-04-01 false Performance reviews. 92.550 Section... Development HOME INVESTMENT PARTNERSHIPS PROGRAM Performance Reviews and Sanctions § 92.550 Performance reviews. (a) General. HUD will review the performance of each participating jurisdiction in carrying out...

24 CFR 92.550 - Performance reviews.

Code of Federal Regulations, 2013 CFR

2013-04-01

... 24 Housing and Urban Development 1 2013-04-01 2013-04-01 false Performance reviews. 92.550 Section... Development HOME INVESTMENT PARTNERSHIPS PROGRAM Performance Reviews and Sanctions § 92.550 Performance reviews. (a) General. HUD will review the performance of each participating jurisdiction in carrying out...

24 CFR 92.550 - Performance reviews.

Code of Federal Regulations, 2012 CFR

2012-04-01

... 24 Housing and Urban Development 1 2012-04-01 2012-04-01 false Performance reviews. 92.550 Section... Development HOME INVESTMENT PARTNERSHIPS PROGRAM Performance Reviews and Sanctions § 92.550 Performance reviews. (a) General. HUD will review the performance of each participating jurisdiction in carrying out...

24 CFR 92.550 - Performance reviews.

Code of Federal Regulations, 2010 CFR

2010-04-01

... 24 Housing and Urban Development 1 2010-04-01 2010-04-01 false Performance reviews. 92.550 Section... Development HOME INVESTMENT PARTNERSHIPS PROGRAM Performance Reviews and Sanctions § 92.550 Performance reviews. (a) General. HUD will review the performance of each participating jurisdiction in carrying out...

24 CFR 92.550 - Performance reviews.

Code of Federal Regulations, 2014 CFR

2014-04-01

... 24 Housing and Urban Development 1 2014-04-01 2014-04-01 false Performance reviews. 92.550 Section... Development HOME INVESTMENT PARTNERSHIPS PROGRAM Performance Reviews and Sanctions § 92.550 Performance reviews. (a) General. HUD will review the performance of each participating jurisdiction in carrying out...

78 FR 9064 - Center for Scientific Review; Notice of Closed Meetings

Federal Register 2010, 2011, 2012, 2013, 2014

2013-02-07

... DEPARTMENT OF HEALTH AND HUMAN SERVICES National Institutes of Health Center for Scientific Review... unwarranted invasion of personal privacy. Name of Committee: Center for Scientific Review Special Emphasis..., Ph.D., Scientific Review Officer, Center for Scientific Review, National Institutes of Health, 6701...

76 FR 35454 - Center for Scientific Review; Notice of Closed Meetings

Federal Register 2010, 2011, 2012, 2013, 2014

2011-06-17

... DEPARTMENT OF HEALTH AND HUMAN SERVICES National Institutes of Health Center for Scientific Review... personal privacy. Name of Committee: Center for Scientific Review Special Emphasis Panel, PAR10-074...D, Scientific Review Officer, Center for Scientific Review, National Institutes of Health, 6701...

77 FR 33476 - Center for Scientific Review; Notice of Closed Meetings

Federal Register 2010, 2011, 2012, 2013, 2014

2012-06-06

... DEPARTMENT OF HEALTH AND HUMAN SERVICES National Institutes of Health Center for Scientific Review... personal privacy. Name of Committee: Center for Scientific Review Special Emphasis Panel; Small Business... Person: John Firrell, Ph.D., Scientific Review Officer, Center for Scientific Review, National Institutes...

78 FR 66026 - Center for Scientific Review; Notice of Closed Meetings

Federal Register 2010, 2011, 2012, 2013, 2014

2013-11-04

... DEPARTMENT OF HEALTH AND HUMAN SERVICES National Institutes of Health Center for Scientific Review... personal privacy. Name of Committee: Center for Scientific Review Special Emphasis Panel; Member Conflict...: Julius Cinque, MS, Scientific Review Officer, Center for Scientific Review, National Institutes of Health...

40 CFR 155.56 - Interim registration review decision.

Code of Federal Regulations, 2010 CFR

2010-07-01

... 40 Protection of Environment 23 2010-07-01 2010-07-01 false Interim registration review decision... PROGRAMS REGISTRATION STANDARDS AND REGISTRATION REVIEW Registration Review Procedures § 155.56 Interim registration review decision. The Agency may issue, when it determines it to be appropriate, an interim...

NOx SOx Secondary NAAQS: Integrated Review Plan - CASAC Advisory

EPA Science Inventory

The NOx SOx Secondary NAAQS Integrated Review Plan is the first document generated as part of the National Ambient Air Quality Standards (NAAQS) review process. The Plan presents background information, the schedule for the review, the process to be used in conducting the review,...

76 FR 59066 - Notice of Regulatory Review Plan

Federal Register 2010, 2011, 2012, 2013, 2014

2011-09-23

... [No. 2011-N-10] Notice of Regulatory Review Plan AGENCY: Federal Housing Finance Agency. ACTION... of and requesting comments on the FHFA interim regulatory review plan for review of existing... comments on all aspects of the interim regulatory review plan, including legal and policy considerations...

20 CFR 405.501 - Judicial review.

Code of Federal Regulations, 2010 CFR

2010-04-01

... 20 Employees' Benefits 2 2010-04-01 2010-04-01 false Judicial review. 405.501 Section 405.501 Employees' Benefits SOCIAL SECURITY ADMINISTRATION ADMINISTRATIVE REVIEW PROCESS FOR ADJUDICATING INITIAL DISABILITY CLAIMS Judicial Review § 405.501 Judicial review. You may file an action in a Federal district...

Review Typology: The Basic Types of Reviews for Synthesizing Evidence for the Purpose of Knowledge Translation.

PubMed

Samnani, Sunil Sadruddin; Vaska, Marcus; Ahmed, Salim; Turin, Tanvir C

2017-10-01

With advances in medical practice and fields of research, reviews occupy a key position for summarizing existing knowledge. Due to the differences and overlap in terminology, the full potential for reviews may be lost due to confusion of indistinct approaches. The main objective of this study was to provide a descriptive outline of each of the common review types with their characteristics and examples in a health care system. Ascoping search was conducted using the keywords associated with the literature review typology. The SALSA(Search, Appraisal, Synthesis and Analysis) analytical framework was used to identify and distinguish each type of review. Nine common types of reviews and associated methodologies were evaluated against the already established SALSA framework. Their description, strengths and weaknesses are presented. The results provided a basic idea of different types of reviews based on the intended level of knowledge synthesis by which researchers can identify the appropriate type of review based on their intended audience.

Redefining the Practice of Peer Review Through Intelligent Automation Part 2: Data-Driven Peer Review Selection and Assignment.

PubMed

Reiner, Bruce I

2017-12-01

In conventional radiology peer review practice, a small number of exams (routinely 5% of the total volume) is randomly selected, which may significantly underestimate the true error rate within a given radiology practice. An alternative and preferable approach would be to create a data-driven model which mathematically quantifies a peer review risk score for each individual exam and uses this data to identify high risk exams and readers, and selectively target these exams for peer review. An analogous model can also be created to assist in the assignment of these peer review cases in keeping with specific priorities of the service provider. An additional option to enhance the peer review process would be to assign the peer review cases in a truly blinded fashion. In addition to eliminating traditional peer review bias, this approach has the potential to better define exam-specific standard of care, particularly when multiple readers participate in the peer review process.

What Do Ethical Guidelines for Epidemiology Say About an Ethics Review? A Qualitative Systematic Review.

PubMed

Piasecki, Jan; Waligora, Marcin; Dranseika, Vilius

2017-06-01

Epidemiological research is subject to an ethics review. The aim of this qualitative review is to compare existing ethical guidelines in English for epidemiological research and public health practice in regard to the scope and matter of an ethics review. Authors systematically searched PubMed, Google Scholar and Google Search for ethical guidelines. Qualitative analysis (constant comparative method) was applied to categorize important aspects of the an ethics review process. Eight ethical guidelines in English for epidemiological research were retrieved. Five main categories that are relevant to the review of epidemiological research by Institutional Review Boards/Research Ethics Committees were distinguished. Within the scope of main categories, fifty-nine subcategories were analyzed. There are important differences between the guidelines in terms of the scope and matter of an ethics review. Not all guidelines encompass all identified ethically important issues, and some do not define precisely the scope and matter of an ethics review, leaving much to the ethics of the individual researchers and the discretion of IRBs/RECs.

The ethics of peer review in bioethics.

PubMed

Wendler, David; Miller, Franklin

2014-10-01

A good deal has been written on the ethics of peer review, especially in the scientific and medical literatures. In contrast, we are unaware of any articles on the ethics of peer review in bioethics. Recognising this gap, we evaluate the extant proposals regarding ethical standards for peer review in general and consider how they apply to bioethics. We argue that scholars have an obligation to perform peer review based on the extent to which they personally benefit from the peer review process. We also argue, contrary to existing proposals and guidelines, that it can be appropriate for peer reviewers to benefit in their own scholarship from the manuscripts they review. With respect to bioethics in particular, we endorse double-blind review and suggest several ways in which the peer review process might be improved. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

Bridging the Otolaryngology Peer Review Knowledge Gap: A Call for a Residency Development Program.

PubMed

Schmalbach, Cecelia E

2016-07-01

Current otolaryngology literature and future scientific direction rely heavily on a rigorous peer review process. Just as manuscripts warrant thoughtful review with constructive feedback to the authors, the same can be said for critiques written by novice peer reviewers. Formal scientific peer review training programs are lacking. Recognizing this knowledge gap, Otolaryngology-Head and Neck Surgery is excited to offer its new Resident Reviewer Development Program. All otolaryngology residents who are postgraduate year 2 and above and in excellent academic standing are eligible to participate in this mentored program, during which they will conduct 6 manuscript reviews under the direction of a seasoned reviewer in his or her subspecialty area of interest. By completing reviews alongside a mentor, participants gain the required skills to master the peer review process-a first step that often leads to journal editorial board and associate editor invitations. © American Academy of Otolaryngology—Head and Neck Surgery Foundation 2016.

IRIS Toxicological Review of Hexahydro-1,3,5-Trinitro-1,3,5 ...

EPA Pesticide Factsheets

On March 10, 2016, the public comment draft Toxicological Review of Hexahydro-1,3,5-trinitro-1,3,5-triazine and the draft charge to external peer reviewers were released for public review and comment. The Toxicological Review and charge were reviewed internally by EPA and by other federal agencies and White House Offices before public release. Consistent with the May 2009 IRIS assessment development process, all written comments on IRIS assessments submitted by other federal agencies and White House Offices are made publicly available. Accordingly, interagency comments and the interagency science consultation materials provided to other agencies, including interagency review drafts of the IRIS Toxicological Review of Hexahydro-1,3,5-trinitro-1,3,5-triazine and the charge to external peer reviewers, are posted on this site. EPA is undertaking an update of the Integrated Risk Information System (IRIS) health assessment for RDX. The outcome of this project is an updated Toxicological Review and IRIS Summary for RDX that will be entered into the IRIS database.

Apollo experience report the command and service module milestone review process

NASA Technical Reports Server (NTRS)

Brendle, H. L.; York, J. A.

1974-01-01

The sequence of the command and service module milestone review process is given, and the Customer Acceptance Readiness Review and Flight Readiness Review plans are presented. Contents of the System Summary Acceptance Documents for the two formal spacecraft reviews are detailed, and supplemental data required for presentation to the review boards are listed. Typical forms, correspondence, supporting documentation, and minutes of a board meeting are included.

The retrospective chart review: important methodological considerations.

PubMed

Vassar, Matt; Holzmann, Matthew

2013-01-01

In this paper, we review and discuss ten common methodological mistakes found in retrospective chart reviews. The retrospective chart review is a widely applicable research methodology that can be used by healthcare disciplines as a means to direct subsequent prospective investigations. In many cases in this review, we have also provided suggestions or accessible resources that researchers can apply as a "best practices" guide when planning, conducting, or reviewing this investigative method.

Grant Peer Review: Improving Inter-Rater Reliability with Training.

PubMed

Sattler, David N; McKnight, Patrick E; Naney, Linda; Mathis, Randy

2015-01-01

This study developed and evaluated a brief training program for grant reviewers that aimed to increase inter-rater reliability, rating scale knowledge, and effort to read the grant review criteria. Enhancing reviewer training may improve the reliability and accuracy of research grant proposal scoring and funding recommendations. Seventy-five Public Health professors from U.S. research universities watched the training video we produced and assigned scores to the National Institutes of Health scoring criteria proposal summary descriptions. For both novice and experienced reviewers, the training video increased scoring accuracy (the percentage of scores that reflect the true rating scale values), inter-rater reliability, and the amount of time reading the review criteria compared to the no video condition. The increase in reliability for experienced reviewers is notable because it is commonly assumed that reviewers--especially those with experience--have good understanding of the grant review rating scale. The findings suggest that both experienced and novice reviewers who had not received the type of training developed in this study may not have appropriate understanding of the definitions and meaning for each value of the rating scale and that experienced reviewers may overestimate their knowledge of the rating scale. The results underscore the benefits of and need for specialized peer reviewer training.

Group consensus peer review in radiation oncology: commitment to quality.

PubMed

Duggar, W Neil; Bhandari, Rahul; Yang, Chunli Claus; Vijayakumar, Srinivasan

2018-03-27

Peer review, especially prospective peer review, has been supported by professional organizations as an important element in optimal Radiation Oncology practice based on its demonstration of efficacy at detecting and preventing errors prior to patient treatment. Implementation of peer review is not without barriers, but solutions do exist to mitigate or eliminate some of those barriers. Peer review practice at our institution involves three key elements: new patient conference, treatment planning conference, and chart rounds. The treatment planning conference is an adaptation of the group consensus peer review model from radiology which utilizes a group of peers reviewing each treatment plan prior to implementation. The peer group in radiation oncology includes Radiation Oncologists, Physician Residents, Medical Physicists, Dosimetrists, and Therapists. Thus, technical and clinical aspects of each plan are evaluated simultaneously. Though peer review is held in high regard in Radiation Oncology, many barriers commonly exist preventing optimal implementation such as time intensiveness, repetition, and distraction from clinic time with patients. Through the use of automated review tools and commitment by individuals and administration in regards to staffing, scheduling, and responsibilities, these barriers have been mitigated to implement this Group Consensus Peer Review model into a Radiation Oncology Clinic. A Group Consensus Peer Review model has been implemented with strategies to address common barriers to effective and efficient peer review.

Characteristics of private-sector managed care for mental health and substance abuse treatment.

PubMed

Garnick, D W; Hendricks, A M; Dulski, J D; Thorpe, K E; Horgan, C

1994-12-01

This study examined diversity during the late 1980s in managed care programs for mental health, alcohol abuse, and drug abuse to identify ways in which research can generate more meaningful data on the effectiveness of utilization review programs. Telephone interviews were conducted with representatives of utilization review programs for employee health insurance plans in 31 firms that employed 2.1 million people in 1990. Questions addressed qualifications of personnel, clinical criteria to authorize care, integration with employee assistance plans, penalties for not complying with utilization review procedures, outpatient review, and carve out of mental health and substance abuse review. Large variations in utilization review programs were found. Programs employed a range of review personnel and used a variety of clinical criteria to authorize care. More than two-thirds did not carve out mental health and substance abuse review from medical-surgical review. Some firms' employee assistance plans were integrated with utilization review programs, while others remained unintegrated. Penalties for not following program procedures varied widely, as did review of outpatient services. Because of trends toward even more diversity in utilization review programs in the 1990s, research that identifies the specific features of managed care programs that hold most promise for controlling costs while maintaining quality of care will increasingly be needed.

Testing the Rebound Peer Review Concept

PubMed Central

Choi, Augustine M.K.

2013-01-01

Abstract This invited editorial addresses the rescue of the article by Skrzypek et al. “Interplay between heme oxygenase-1 and miR-378 affects non-small cell lung carcinoma growth, vascularization, and metastasis.” The work was rejected by the standard peer review system and subsequently rescued by the Rebound Peer Review (RPR) mechanism offered by Antioxidants and Redox Signaling (Antioxid Redox Signal 16: 293–296, 2012). The reviewers who openly rescued the article were James F. George, Justin C. Mason, Mahin D. Maines, and Yasufumi Sato. The initial article was a de novo resubmission of a previously rejected article, which was then reviewed by six reviewers. The reviewers raised substantial scientific concerns, including questions pertaining to the specificity of the findings, quality of the presentation, and other technical concerns; the editor returned a decision of reject. The authors voluntarily chose to exercise the option to rescue the article utilizing the RPR system, where the authors found qualified reviewers who were willing to advocate for acceptance with scientific reasoning. The open reviewers felt that the scientific and technical concerns raised by the reviewers were outweighed by the strengths and novelty of the findings to justify acceptance. The RPR, in this case, was a “success” in that it rescued a rejected article. Despite this assessment, we question the necessity of open peer review as a means to overturn a peer review decision, with concerns for the larger-than-usual peer review process, and the voluntary relinquishing of editorial privilege and disclosure of reviewer identity. Antioxid. Redox Signal. 19, 639–643. PMID:23725371

How to write a systematic review.

PubMed

Harris, Joshua D; Quatman, Carmen E; Manring, M M; Siston, Robert A; Flanigan, David C

2014-11-01

The role of evidence-based medicine in sports medicine and orthopaedic surgery is rapidly growing. Systematic reviews and meta-analyses are also proliferating in the medical literature. To provide the outline necessary for a practitioner to properly understand and/or conduct a systematic review for publication in a sports medicine journal. Review. The steps of a successful systematic review include the following: identification of an unanswered answerable question; explicit definitions of the investigation's participant(s), intervention(s), comparison(s), and outcome(s); utilization of PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) guidelines and PROSPERO registration; thorough systematic data extraction; and appropriate grading of the evidence and strength of the recommendations. An outline to understand and conduct a systematic review is provided, and the difference between meta-analyses and systematic reviews is described. The steps necessary to perform a systematic review are fully explained, including the study purpose, search methodology, data extraction, reporting of results, identification of bias, and reporting of the study's main findings. Systematic reviews or meta-analyses critically appraise and formally synthesize the best existing evidence to provide a statement of conclusion that answers specific clinical questions. Readers and reviewers, however, must recognize that the quality and strength of recommendations in a review are only as strong as the quality of studies that it analyzes. Thus, great care must be used in the interpretation of bias and extrapolation of the review's findings to translation to clinical practice. Without advanced education on the topic, the reader may follow the steps discussed herein to perform a systematic review. © 2013 The Author(s).

Review of reviews on exposures to synthetic organic chemicals and children's neurodevelopment: Methodological and interpretation challenges.

PubMed

LaKind, Judy S; Anthony, Laura G; Goodman, Michael

2017-01-01

Environmental epidemiology data are becoming increasingly important in public health decision making, which commonly incorporates a systematic review of multiple studies. This review addresses two fundamental questions: What is the quality of available reviews on associations between exposure to synthetic organic chemicals and neurodevelopmental outcomes? What is the value (e.g., quality and consistency) of the underlying literature? Published reviews on associations between synthetic organic environmental chemical exposures and neurodevelopmental outcomes in children were systematically evaluated. Seventy-four relevant reviews were identified, and these were evaluated with respect to four methodological characteristics: (1) systematic inclusion/exclusion criteria and reproducible methods for search and retrieval of studies; (2) structured evaluation of underlying data quality; (3) systematic assessment of consistency across specific exposure-outcome associations; and (4) evaluation of reporting/publication bias. None of the 74 reviews fully met the criteria for all four methodological characteristics. Only four reviews met two criteria, and six reviews fulfilled only one criterion. Perhaps more importantly, the higher quality reviews were not able to meet all of the criteria owing to the shortcomings of underlying studies, which lacked comparability in terms of specific research question of interest, overall design, exposure assessment, outcome ascertainment, and analytic methods. Thus, even the most thoughtful and rigorous review may be of limited value if the underlying literature includes investigations that address different hypotheses and are beset by methodological inconsistencies and limitations. Issues identified in this review of reviews illustrate considerable challenges that are facing assessments of epidemiological evidence.

Factors contributing to chronic ankle instability: a protocol for a systematic review of systematic reviews.

PubMed

Thompson, Cassandra; Schabrun, Siobhan; Romero, Rick; Bialocerkowski, Andrea; Marshall, Paul

2016-06-07

Ankle sprains are a significant clinical problem. Researchers have identified a multitude of factors contributing to the presence of recurrent ankle sprains including deficits in balance, postural control, kinematics, muscle activity, strength, range of motion, ligament laxity and bone/joint characteristics. Unfortunately, the literature examining the presence of these factors in chronic ankle instability (CAI) is conflicting. As a result, researchers have attempted to integrate this evidence using systematic reviews to reach conclusions; however, readers are now faced with an increasing number of systematic review findings that are also conflicting. The overall aim of this review is to critically appraise the methodological quality of previous systematic reviews and pool this evidence to identify contributing factors to CAI. A systematic review will be conducted on systematic reviews that investigate the presence of various deficits identified in CAI. Databases will be searched using pre-determined search terms. Reviews will then be assessed for inclusion based on the set eligibility criteria. Two independent reviewers will assess the articles for inclusion before evaluating the methodological quality and presence of bias of the included studies; any disagreements will be resolved by discussion between reviewers to reach consensus or by a third reviewer. Data concerning the specific research question, search strategy, inclusion/exclusion criteria, population, method and outcomes will be extracted. Findings will be analysed with respect to the methodological quality of the included reviews. It is expected that this review will clarify the cause of contradicting findings in the literature and facilitate future research directions. PROSPERO CRD42016032592 .

Methodological quality of systematic reviews addressing femoroacetabular impingement.

PubMed

Kowalczuk, Marcin; Adamich, John; Simunovic, Nicole; Farrokhyar, Forough; Ayeni, Olufemi R

2015-09-01

As the body of literature on femoroacetabular impingement (FAI) continues to grow, clinicians turn to systematic reviews to remain current with the best available evidence. The quality of systematic reviews in the FAI literature is currently unknown. The goal of this study was to assess the quality of the reporting of systematic reviews addressing FAI over the last 11 years (2003-2014) and to identify the specific methodological shortcomings and strengths. A search of the electronic databases, MEDLINE, EMBASE and PubMed, was performed to identify relevant systematic reviews. Methodological quality was assessed by two reviewers using the revised assessment of multiple systematic reviews (R-AMSTAR) scoring tool. An intraclass correlation coefficient (ICC) with 95 % confidence intervals (CI) was used to determine agreement between reviewers on R-AMSTAR quality scores. A total of 22 systematic reviews were assessed for methodological quality. The mean consensus R-AMSTAR score across all studies was 26.7 out of 40.0, indicating fair methodological quality. An ICC of 0.931, 95 % CI 0.843-0.971 indicated excellent agreement between reviewers during the scoring process. The systematic reviews addressing FAI are generally of fair methodological quality. Use of tools such as the R-AMSTAR score or PRISMA guidelines while designing future systematic reviews can assist in eliminating methodological shortcomings identified in this review. These shortcomings need to be kept in mind by clinicians when applying the current literature to their patient populations and making treatment decisions. Systematic reviews of highest methodological quality should be used by clinicians when possible to answer clinical questions.

A narrative meta-review of a series of systematic and meta-analytic reviews on the intervention outcome for children with developmental co-ordination disorder.

PubMed

Miyahara, M; Lagisz, M; Nakagawa, S; Henderson, S E

2017-09-01

Systematic reviews and meta-analyses are considered to be the 'gold standards' for synthesizing research evidence in particular areas of enquiry. However, such reviews are only useful if they themselves are conducted to a sufficiently high standard. The aim of this study was to conduct a narrative meta-review of existing analyses of the effectiveness of interventions designed for children with developmental co-ordination disorder (DCD). A narrative meta-review of systematic and meta-analytic reviews aimed at evaluating the effectiveness of intervention for children with DCD was conducted on studies published between 1950 and 2014. We identified suitable reviews, using a modification of the Population, Intervention, Comparison, Outcome (PICO) system and evaluated their methodological quality using the Assessment of Multiple Systematic Reviews (AMSTAR). In addition, the consistency of the quality of evidence and classification of intervention approaches was assessed independently by two assessors. The literature search yielded a total of four appropriate reviews published in the selected time span. The Assessment of Multiple Systematic Reviews percentage quality scores assigned to each review ranged from 0% (low quality) to 55% (medium quality). Evaluation of the quality of evidence and classification of intervention approaches yielded a discrepancy rate of 25%. All reviews concluded that some kind of intervention was better than none at all. Although the quality of the reviews progressively improved over the years, the shortcomings identified need to be addressed before concrete evidence regarding the best approach to intervention for children with DCD can be specified. © 2016 John Wiley & Sons Ltd.

Effective implementation of research into practice: an overview of systematic reviews of the health literature.

PubMed

Boaz, Annette; Baeza, Juan; Fraser, Alec

2011-06-22

The gap between research findings and clinical practice is well documented and a range of interventions has been developed to increase the implementation of research into clinical practice. A review of systematic reviews of the effectiveness of interventions designed to increase the use of research in clinical practice. A search for relevant systematic reviews was conducted of Medline and the Cochrane Database of Reviews 1998-2009. 13 systematic reviews containing 313 primary studies were included. Four strategy types are identified: audit and feedback; computerised decision support; opinion leaders; and multifaceted interventions. Nine of the reviews reported on multifaceted interventions. This review highlights the small effects of single interventions such as audit and feedback, computerised decision support and opinion leaders. Systematic reviews of multifaceted interventions claim an improvement in effectiveness over single interventions, with effect sizes ranging from small to moderate. This review found that a number of published systematic reviews fail to state whether the recommended practice change is based on the best available research evidence. This overview of systematic reviews updates the body of knowledge relating to the effectiveness of key mechanisms for improving clinical practice and service development. Multifaceted interventions are more likely to improve practice than single interventions such as audit and feedback. This review identified a small literature focusing explicitly on getting research evidence into clinical practice. It emphasizes the importance of ensuring that primary studies and systematic reviews are precise about the extent to which the reported interventions focus on changing practice based on research evidence (as opposed to other information codified in guidelines and education materials).

Engaging patients and stakeholders in research proposal review: the patient-centered outcomes research institute.

PubMed

Fleurence, Rachael L; Forsythe, Laura P; Lauer, Michael; Rotter, Jason; Ioannidis, John P A; Beal, Anne; Frank, Lori; Selby, Joseph V

2014-07-15

The inaugural round of merit review for the Patient-Centered Outcomes Research Institute (PCORI) in November 2012 included patients and other stakeholders, as well as scientists. This article examines relationships among scores of the 3 reviewer types, changes in scoring after in-person discussion, and the effect of inclusion of patient and stakeholder reviewers on the review process. In the first phase, 363 scientists scored 480 applications. In the second phase, 59 scientists, 21 patients, and 31 stakeholders provided a "prediscussion" score and a final "postdiscussion" score after an in-person meeting for applications. Bland-Altman plots were used to characterize levels of agreement among and within reviewer types before and after discussion. Before discussion, there was little agreement among average scores given by the 4 lead scientific reviewers and patient and stakeholder reviewers. After discussion, the 4 primary reviewers showed mild convergence in their scores, and the 21-member panel came to a much stronger agreement. Of the 25 awards with the best (and lowest) scores after phase 2, only 13 had ranked in the top 25 after the phase 1 review by scientists. Five percent of the 480 proposals submitted were funded. The authors conclude that patient and stakeholder reviewers brought different perspectives to the review process but that in-person discussion led to closer agreement among reviewer types. It is not yet known whether these conclusions are generalizable to future rounds of peer review. Future work would benefit from additional data collection for evaluation purposes and from long-term evaluation of the effect on the funded research.

Cochrane Rapid Reviews Methods Group to play a leading role in guiding the production of informed high-quality, timely research evidence syntheses.

PubMed

Garritty, Chantelle; Stevens, Adrienne; Gartlehner, Gerald; King, Valerie; Kamel, Chris

2016-10-28

Policymakers and healthcare stakeholders are increasingly seeking evidence to inform the policymaking process, and often use existing or commissioned systematic reviews to inform decisions. However, the methodologies that make systematic reviews authoritative take time, typically 1 to 2 years to complete. Outside the traditional SR timeline, "rapid reviews" have emerged as an efficient tool to get evidence to decision-makers more quickly. However, the use of rapid reviews does present challenges. To date, there has been limited published empirical information about this approach to compiling evidence. Thus, it remains a poorly understood and ill-defined set of diverse methodologies with various labels. In recent years, the need to further explore rapid review methods, characteristics, and their use has been recognized by a growing network of healthcare researchers, policymakers, and organizations, several with ties to Cochrane, which is recognized as representing an international gold standard for high-quality, systematic reviews. In this commentary, we introduce the newly established Cochrane Rapid Reviews Methods Group developed to play a leading role in guiding the production of rapid reviews given they are increasingly employed as a research synthesis tool to support timely evidence-informed decision-making. We discuss how the group was formed and outline the group's structure and remit. We also discuss the need to establish a more robust evidence base for rapid reviews in the published literature, and the importance of promoting registration of rapid review protocols in an effort to promote efficiency and transparency in research. As with standard systematic reviews, the core principles of evidence-based synthesis should apply to rapid reviews in order to minimize bias to the extent possible. The Cochrane Rapid Reviews Methods Group will serve to establish a network of rapid review stakeholders and provide a forum for discussion and training. By facilitating exchange, the group will strive to conduct research to advance the methods of rapid reviews.

Medical journal peer review: process and bias.

PubMed

Manchikanti, Laxmaiah; Kaye, Alan D; Boswell, Mark V; Hirsch, Joshua A

2015-01-01

Scientific peer review is pivotal in health care research in that it facilitates the evaluation of findings for competence, significance, and originality by qualified experts. While the origins of peer review can be traced to the societies of the eighteenth century, it became an institutionalized part of the scholarly process in the latter half of the twentieth century. This was a response to the growth of research and greater subject specialization. With the current increase in the number of specialty journals, the peer review process continues to evolve to meet the needs of patients, clinicians, and policy makers. The peer review process itself faces challenges. Unblinded peer review might suffer from positive or negative bias towards certain authors, specialties, and institutions. Peer review can also suffer when editors and/or reviewers might be unable to understand the contents of the submitted manuscript. This can result in an inability to detect major flaws, or revelations of major flaws after acceptance of publication by the editors. Other concerns include potentially long delays in publication and challenges uncovering plagiarism, duplication, corruption and scientific misconduct. Conversely, a multitude of these challenges have led to claims of scientific misconduct and an erosion of faith. These challenges have invited criticism of the peer review process itself. However, despite its imperfections, the peer review process enjoys widespread support in the scientific community. Peer review bias is one of the major focuses of today's scientific assessment of the literature. Various types of peer review bias include content-based bias, confirmation bias, bias due to conservatism, bias against interdisciplinary research, publication bias, and the bias of conflicts of interest. Consequently, peer review would benefit from various changes and improvements with appropriate training of reviewers to provide quality reviews to maintain the quality and integrity of research without bias. Thus, an appropriate, transparent peer review is not only ideal, but necessary for the future to facilitate scientific progress.

Medical students review of formative OSCE scores, checklists, and videos improves with student-faculty debriefing meetings

PubMed Central

Bernard, Aaron W.; Ceccolini, Gabbriel; Feinn, Richard; Rockfeld, Jennifer; Rosenberg, Ilene; Thomas, Listy; Cassese, Todd

2017-01-01

ABSTRACT Background: Performance feedback is considered essential to clinical skills development. Formative objective structured clinical exams (F-OSCEs) often include immediate feedback by standardized patients. Students can also be provided access to performance metrics including scores, checklists, and video recordings after the F-OSCE to supplement this feedback. How often students choose to review this data and how review impacts future performance has not been documented. Objective: We suspect student review of F-OSCE performance data is variable. We hypothesize that students who review this data have better performance on subsequent F-OSCEs compared to those who do not. We also suspect that frequency of data review can be improved with faculty involvement in the form of student-faculty debriefing meetings. Design: Simulation recording software tracks and time stamps student review of performance data. We investigated a cohort of first- and second-year medical students from the 2015-16 academic year. Basic descriptive statistics were used to characterize frequency of data review and a linear mixed-model analysis was used to determine relationships between data review and future F-OSCE performance. Results: Students reviewed scores (64%), checklists (42%), and videos (28%) in decreasing frequency. Frequency of review of all metric and modalities improved when student-faculty debriefing meetings were conducted (p<.001). Among 92 first-year students, checklist review was associated with an improved performance on subsequent F-OSCEs (p = 0.038) by 1.07 percentage points on a scale of 0-100. Among 86 second year students, no review modality was associated with improved performance on subsequent F-OSCEs. Conclusion: Medical students review F-OSCE checklists and video recordings less than 50% of the time when not prompted. Student-faculty debriefing meetings increased student data reviews. First-year student’s review of checklists on F-OSCEs was associated with increases in performance on subsequent F-OSCEs, however this outcome was not observed among second-year students. PMID:28521646

Using systematic reviews to inform NIHR HTA trial planning and design: a retrospective cohort.

PubMed

Bhurke, Sheetal; Cook, Andrew; Tallant, Anna; Young, Amanda; Williams, Elaine; Raftery, James

2015-12-29

Chalmers and Glasziou's paper published in 2014 recommends research funding bodies should mandate that proposals for additional primary research are built on systematic reviews of existing evidence showing what is already known. Jones et al. identified 11 (23%) of 48 trials funded during 2006-8 by the National Institute for Health Research Health Technology Assessment (NIHR HTA) Programme did not reference a systematic review. This study did not explore the reasons for trials not referencing a systematic review or consider trials using any other evidence in the absence of a systematic review. Referencing a systematic review may not be possible in certain circumstances, for instance if the systematic review does not address the question being proposed in the trial. The current study extended Jones' study by exploring the reasons for why trials did not reference a systematic review and included a more recent cohort of trials funded in 2013 to determine if there were any changes in the referencing or use of systematic reviews. Two cohorts of NIHR HTA randomised controlled trials were included. Cohort I included the same trials as Jones et al. (with the exception of one trial which was discontinued). Cohort II included NIHR HTA trials funded in 2013. Data extraction was undertaken independently by two reviewers using full applications and trial protocols. Descriptive statistics was used and no formal statistical analyses were conducted. Five (11%) trials of the 47 funded during 2006-2008 did not reference a systematic review. These 5 trials had warranted reasons for not referencing systematic reviews. All trials from Cohort II referenced a systematic review. A quarter of all those trials with a preceding systematic review used a different primary outcome than those stated in the reviews. The NIHR requires that proposals for new primary research are justified by existing evidence and the findings of this study confirm the adherence to this requirement with a high rate of applications using systematic reviews.

A survey of radiation treatment planning peer-review activities in a provincial radiation oncology programme: current practice and future directions

PubMed Central

Brundage, Michael; Foxcroft, Sophie; McGowan, Tom; Gutierrez, Eric; Sharpe, Michael; Warde, Padraig

2013-01-01

Objectives To describe current patterns of practice of radiation oncology peer review within a provincial cancer system, identifying barriers and facilitators to its use with the ultimate aim of process improvement. Design A survey of radiation oncology programmes at provincial cancer centres. Setting All cancer centres within the province of Ontario, Canada (n=14). These are community-based outpatient facilities overseen by Cancer Care Ontario, the provincial cancer agency. Participants A delegate from each radiation oncology programme filled out a single survey based on input from their multidisciplinary team. Outcome measures Rated importance of peer review; current utilisation; format of the peer-review process; organisation and timing; case attributes; outcomes of the peer-review process and perceived barriers and facilitators to expanding peer-review processes. Results 14 (100%) centres responded. All rated the importance of peer review as at least 8/10 (10=extremely important). Detection of medical error and improvement of planning processes were the highest rated perceived benefits of peer review (each median 9/10). Six centres (43%) reviewed at least 50% of curative cases; four of these centres (29%) conducted peer review in more than 80% of cases treated with curative intent. Fewer than 20% of cases treated with palliative intent were reviewed in most centres. Five centres (36%) reported usually conducting peer review prior to the initiation of treatment. Five centres (36%) recorded the outcomes of peer review on the medical record. Thirteen centres (93%) planned to expand peer-review activities; a critical mass of radiation oncologists was the most important limiting factor (median 6/10). Conclusions Radiation oncology peer-review practices can vary even within a cancer system with provincial oversight. The application of guidelines and standards for peer-review processes, and monitoring of implementation and outcomes, will require effective knowledge translation activities. PMID:23903814

Medical students review of formative OSCE scores, checklists, and videos improves with student-faculty debriefing meetings.

PubMed

Bernard, Aaron W; Ceccolini, Gabbriel; Feinn, Richard; Rockfeld, Jennifer; Rosenberg, Ilene; Thomas, Listy; Cassese, Todd

2017-01-01

Performance feedback is considered essential to clinical skills development. Formative objective structured clinical exams (F-OSCEs) often include immediate feedback by standardized patients. Students can also be provided access to performance metrics including scores, checklists, and video recordings after the F-OSCE to supplement this feedback. How often students choose to review this data and how review impacts future performance has not been documented. We suspect student review of F-OSCE performance data is variable. We hypothesize that students who review this data have better performance on subsequent F-OSCEs compared to those who do not. We also suspect that frequency of data review can be improved with faculty involvement in the form of student-faculty debriefing meetings. Simulation recording software tracks and time stamps student review of performance data. We investigated a cohort of first- and second-year medical students from the 2015-16 academic year. Basic descriptive statistics were used to characterize frequency of data review and a linear mixed-model analysis was used to determine relationships between data review and future F-OSCE performance. Students reviewed scores (64%), checklists (42%), and videos (28%) in decreasing frequency. Frequency of review of all metric and modalities improved when student-faculty debriefing meetings were conducted (p<.001). Among 92 first-year students, checklist review was associated with an improved performance on subsequent F-OSCEs (p = 0.038) by 1.07 percentage points on a scale of 0-100. Among 86 second year students, no review modality was associated with improved performance on subsequent F-OSCEs. Medical students review F-OSCE checklists and video recordings less than 50% of the time when not prompted. Student-faculty debriefing meetings increased student data reviews. First-year student's review of checklists on F-OSCEs was associated with increases in performance on subsequent F-OSCEs, however this outcome was not observed among second-year students.

Peer Review of Grant Applications: Criteria Used and Qualitative Study of Reviewer Practices

PubMed Central

Abdoul, Hendy; Perrey, Christophe; Amiel, Philippe; Tubach, Florence; Gottot, Serge; Durand-Zaleski, Isabelle; Alberti, Corinne

2012-01-01

Background Peer review of grant applications has been criticized as lacking reliability. Studies showing poor agreement among reviewers supported this possibility but usually focused on reviewers’ scores and failed to investigate reasons for disagreement. Here, our goal was to determine how reviewers rate applications, by investigating reviewer practices and grant assessment criteria. Methods and Findings We first collected and analyzed a convenience sample of French and international calls for proposals and assessment guidelines, from which we created an overall typology of assessment criteria comprising nine domains relevance to the call for proposals, usefulness, originality, innovativeness, methodology, feasibility, funding, ethical aspects, and writing of the grant application. We then performed a qualitative study of reviewer practices, particularly regarding the use of assessment criteria, among reviewers of the French Academic Hospital Research Grant Agencies (Programmes Hospitaliers de Recherche Clinique, PHRCs). Semi-structured interviews and observation sessions were conducted. Both the time spent assessing each grant application and the assessment methods varied across reviewers. The assessment criteria recommended by the PHRCs were listed by all reviewers as frequently evaluated and useful. However, use of the PHRC criteria was subjective and varied across reviewers. Some reviewers gave the same weight to each assessment criterion, whereas others considered originality to be the most important criterion (12/34), followed by methodology (10/34) and feasibility (4/34). Conceivably, this variability might adversely affect the reliability of the review process, and studies evaluating this hypothesis would be of interest. Conclusions Variability across reviewers may result in mistrust among grant applicants about the review process. Consequently, ensuring transparency is of the utmost importance. Consistency in the review process could also be improved by providing common definitions for each assessment criterion and uniform requirements for grant application submissions. Further research is needed to assess the feasibility and acceptability of these measures. PMID:23029386

What's new in atopic eczema? An analysis of systematic reviews published in 2007 and 2008. Part 2. Disease prevention and treatment.

PubMed

Williams, H C; Grindlay, D J C

2010-04-01

This review summarizes clinically important findings from systematic reviews indexed in bibliographical databases between August 2007 and August 2008 that dealt with disease prevention (six reviews) and treatment of atopic eczema (seven reviews). Regarding disease prevention, two independent systematic reviews found some clinical trial evidence that ingestion of probiotics by mothers during pregnancy might reduce the incidence of subsequent eczema. Another review failed to find any clear benefit of prebiotics in eczema prevention. Although furry pets are often cited as causing allergic disease, a systematic review of observational studies found no evidence that exposure to cats or dogs at birth increases eczema risk. One very large review of studies of breastfeeding found some evidence of a protective effect on eczema risk, although all the studies were limited by their observational nature. A German group has attempted an overview of eczema prevention studies with a view to informing national guidelines. In terms of eczema treatment, two systematic reviews have confirmed the efficacy of topical tacrolimus ointment. Another review of 31 trials confirms the efficacy of topical pimecrolimus, although many of those trials were vehicle controlled, which limits their clinical utility. A review of 23 studies of desensitization therapy for allergic diseases found some evidence of benefit for eczema, which needs to be explored further. Despite the popularity of antistaphylococcal therapies for eczema, a Cochrane Review of 21 trials failed to show any clear benefit for any of the therapies for infected or clinically noninfected eczema. Another Cochrane Review dealt with dietary exclusions for people with eczema and found little evidence to support any dietary exclusion, apart from avoidance of eggs in infants with suspected egg allergy supported by evidence of sensitization. A review of 13 studies of probiotics for treating established eczema did not show convincing evidence of a clinically worthwhile benefit, an observation that has been substantiated in a subsequent Cochrane Review.

IRIS Toxicological Review of Benzo[a]pyrene (External Review Draft)

EPA Science Inventory

The IRIS Toxicological Review of Benzo[a]pyrene was released for external peer review in September 2014. The EPA’s Science Advisory Board’s (SAB) Chemical Assessment Advisory Committee (CAAC) conducted a peer review of the scientific basis supporting the benzo[a]pyrene assessmen...

78 FR 69093 - Performance Review Board Members

Federal Register 2010, 2011, 2012, 2013, 2014

2013-11-18

... Review Board Members AGENCY: Centers for Disease Control and Prevention (CDC), Department of Health and... Performance Review Board Members who are reviewing performance for Fiscal Year 2013. FOR FURTHER INFORMATION... Performance Review Board Members be published in the Federal Register. The following persons will serve on the...

Advancing Kinesiology through Improved Peer Review

ERIC Educational Resources Information Center

Knudson, Duane V.; Morrow, James R., Jr.; Thomas, Jerry R.

2014-01-01

Peer review of scholarship is essential to journal quality, evidence, knowledge advancement, and application of that knowledge in any field. This commentary summarizes recent literature on issues related to peer-review quality and current review practice in kinesiology and provides recommendations to improve peer review in kinesiology journals. We…

7 CFR 622.32 - Reviews and approvals.

Code of Federal Regulations, 2012 CFR

2012-01-01

..., DEPARTMENT OF AGRICULTURE WATER RESOURCES WATERSHED PROJECTS Planning § 622.32 Reviews and approvals. (a) The watershed plan-environmental impact statement (or assessment) will be subject to internal technical reviews..., and a final review as stated in NRCS's National Watersheds Manual. (b) After thorough review by NRCS...

7 CFR 622.32 - Reviews and approvals.

Code of Federal Regulations, 2010 CFR

2010-01-01

..., DEPARTMENT OF AGRICULTURE WATER RESOURCES WATERSHED PROJECTS Planning § 622.32 Reviews and approvals. (a) The watershed plan-environmental impact statement (or assessment) will be subject to internal technical reviews..., and a final review as stated in NRCS's National Watersheds Manual. (b) After thorough review by NRCS...

7 CFR 622.32 - Reviews and approvals.

Code of Federal Regulations, 2013 CFR

2013-01-01

..., DEPARTMENT OF AGRICULTURE WATER RESOURCES WATERSHED PROJECTS Planning § 622.32 Reviews and approvals. (a) The watershed plan-environmental impact statement (or assessment) will be subject to internal technical reviews..., and a final review as stated in NRCS's National Watersheds Manual. (b) After thorough review by NRCS...

7 CFR 622.32 - Reviews and approvals.

Code of Federal Regulations, 2014 CFR

2014-01-01

..., DEPARTMENT OF AGRICULTURE WATER RESOURCES WATERSHED PROJECTS Planning § 622.32 Reviews and approvals. (a) The watershed plan-environmental impact statement (or assessment) will be subject to internal technical reviews..., and a final review as stated in NRCS's National Watersheds Manual. (b) After thorough review by NRCS...

7 CFR 622.32 - Reviews and approvals.

Code of Federal Regulations, 2011 CFR

2011-01-01

..., DEPARTMENT OF AGRICULTURE WATER RESOURCES WATERSHED PROJECTS Planning § 622.32 Reviews and approvals. (a) The watershed plan-environmental impact statement (or assessment) will be subject to internal technical reviews..., and a final review as stated in NRCS's National Watersheds Manual. (b) After thorough review by NRCS...

76 FR 32221 - Center for Scientific Review; Notice of Closed Meetings

Federal Register 2010, 2011, 2012, 2013, 2014

2011-06-03

... DEPARTMENT OF HEALTH AND HUMAN SERVICES National Institutes of Health Center for Scientific Review... personal privacy. Name of Committee: Center for Scientific Review Special Emphasis Panel, Fellowship: Cell... Person: Ross D Shonat, PH.D, Scientific Review Officer, Center for Scientific Review, National Institutes...

8 CFR 1299.2 - Specific immigration review forms.

Code of Federal Regulations, 2010 CFR

2010-01-01

... 8 Aliens and Nationality 1 2010-01-01 2010-01-01 false Specific immigration review forms. 1299.2 Section 1299.2 Aliens and Nationality EXECUTIVE OFFICE FOR IMMIGRATION REVIEW, DEPARTMENT OF JUSTICE NATIONALITY REGULATIONS IMMIGRATION REVIEW FORMS § 1299.2 Specific immigration review forms. The Director of...

78 FR 77125 - Senior Executive Service Performance Review Board; Membership

Federal Register 2010, 2011, 2012, 2013, 2014

2013-12-20

... ENVIRONMENTAL PROTECTION AGENCY [FRL-9904-20-OARM] Senior Executive Service Performance Review... review boards. This board shall review and evaluate the initial appraisal of a senior executive's... performance of the senior executive. Members of the 2013 EPA Performance Review Board are: Benita Best-Wong...

77 FR 65685 - Senior Executive Service Performance Review Board; Membership

Federal Register 2010, 2011, 2012, 2013, 2014

2012-10-30

... ENVIRONMENTAL PROTECTION AGENCY [FRL-9747-4] Senior Executive Service Performance Review Board... performance review boards. This board shall review and evaluate the initial appraisal of a senior executive's... performance of the senior executive. Members of the 2012 EPA Performance Review Board are: Benita Best-Wong...

40 CFR 155.53 - Conduct of a pesticide's registration review.

Code of Federal Regulations, 2012 CFR

2012-07-01

... 40 Protection of Environment 25 2012-07-01 2012-07-01 false Conduct of a pesticide's registration...) PESTICIDE PROGRAMS REGISTRATION STANDARDS AND REGISTRATION REVIEW Registration Review Procedures § 155.53 Conduct of a pesticide's registration review. The Agency will review data and information described in...

40 CFR 155.53 - Conduct of a pesticide's registration review.

Code of Federal Regulations, 2011 CFR

2011-07-01

... 40 Protection of Environment 24 2011-07-01 2011-07-01 false Conduct of a pesticide's registration...) PESTICIDE PROGRAMS REGISTRATION STANDARDS AND REGISTRATION REVIEW Registration Review Procedures § 155.53 Conduct of a pesticide's registration review. The Agency will review data and information described in...

40 CFR 155.53 - Conduct of a pesticide's registration review.

Code of Federal Regulations, 2014 CFR

2014-07-01

... 40 Protection of Environment 24 2014-07-01 2014-07-01 false Conduct of a pesticide's registration...) PESTICIDE PROGRAMS REGISTRATION STANDARDS AND REGISTRATION REVIEW Registration Review Procedures § 155.53 Conduct of a pesticide's registration review. The Agency will review data and information described in...

40 CFR 155.53 - Conduct of a pesticide's registration review.

Code of Federal Regulations, 2013 CFR

2013-07-01

... 40 Protection of Environment 25 2013-07-01 2013-07-01 false Conduct of a pesticide's registration...) PESTICIDE PROGRAMS REGISTRATION STANDARDS AND REGISTRATION REVIEW Registration Review Procedures § 155.53 Conduct of a pesticide's registration review. The Agency will review data and information described in...

24 CFR 200.224 - Multifamily Participation Review Committee and Participation Control Officer.

Code of Federal Regulations, 2012 CFR

2012-04-01

... 24 Housing and Urban Development 2 2012-04-01 2012-04-01 false Multifamily Participation Review... § 200.224 Multifamily Participation Review Committee and Participation Control Officer. The membership and authority of the Multifamily Participation Review Committee (hereinafter referred to as the Review...

24 CFR 200.224 - Multifamily Participation Review Committee and Participation Control Officer.

Code of Federal Regulations, 2014 CFR

2014-04-01

... 24 Housing and Urban Development 2 2014-04-01 2014-04-01 false Multifamily Participation Review... § 200.224 Multifamily Participation Review Committee and Participation Control Officer. The membership and authority of the Multifamily Participation Review Committee (hereinafter referred to as the Review...

24 CFR 200.224 - Multifamily Participation Review Committee and Participation Control Officer.

Code of Federal Regulations, 2011 CFR

2011-04-01

... 24 Housing and Urban Development 2 2011-04-01 2011-04-01 false Multifamily Participation Review... § 200.224 Multifamily Participation Review Committee and Participation Control Officer. The membership and authority of the Multifamily Participation Review Committee (hereinafter referred to as the Review...

24 CFR 200.224 - Multifamily Participation Review Committee and Participation Control Officer.

Code of Federal Regulations, 2010 CFR

2010-04-01

... 24 Housing and Urban Development 2 2010-04-01 2010-04-01 false Multifamily Participation Review... § 200.224 Multifamily Participation Review Committee and Participation Control Officer. The membership and authority of the Multifamily Participation Review Committee (hereinafter referred to as the Review...

42 CFR 476.160 - General quality of care review procedures.

Code of Federal Regulations, 2014 CFR

2014-10-01

... 42 Public Health 4 2014-10-01 2014-10-01 false General quality of care review procedures. 476.160... SERVICES (CONTINUED) QUALITY IMPROVEMENT ORGANIZATIONS QUALITY IMPROVEMENT ORGANIZATION REVIEW Review Responsibilities of Quality Improvement Organizations (QIOs) Qio Review Functions § 476.160 General quality of care...

14 CFR 431.57 - Information requirements for payload reentry review.

Code of Federal Regulations, 2010 CFR

2010-01-01

... reentry review. 431.57 Section 431.57 Aeronautics and Space COMMERCIAL SPACE TRANSPORTATION, FEDERAL... VEHICLE (RLV) Payload Reentry Review and Determination § 431.57 Information requirements for payload reentry review. A person requesting reentry review of a particular payload or payload class must identify...

20 CFR 405.220 - Decision by the Federal reviewing official.

Code of Federal Regulations, 2011 CFR

2011-04-01

... PROCESS FOR ADJUDICATING INITIAL DISABILITY CLAIMS Review of Initial Determinations by a Federal Reviewing... his or her decision, the Federal reviewing official may consult with a medical, psychological, or... evidence, the Federal reviewing official will consult with a medical or psychological expert through the...

75 FR 21012 - Extension of Agency Information Collection Activity Under OMB Review: Highway Corporate Security...

Federal Register 2010, 2011, 2012, 2013, 2014

2010-04-22

... Information Collection Activity Under OMB Review: Highway Corporate Security Review (CSR) AGENCY... in the highway and motor carrier industry by way of its Highway Corporate Security Review Program... comments. Information Collection Requirement Title: Corporate Security Review (CSR). Type of Request...

24 CFR 511.80 - Performance review.

Code of Federal Regulations, 2014 CFR

2014-04-01

... 24 Housing and Urban Development 3 2014-04-01 2013-04-01 true Performance review. 511.80 Section... DEVELOPMENT SLUM CLEARANCE AND URBAN RENEWAL RENTAL REHABILITATION GRANT PROGRAM Grantee Performance: Review, Reporting and Corrective or Remedial Actions § 511.80 Performance review. (a) General. HUD will review the...

24 CFR 214.307 - Performance review.

Code of Federal Regulations, 2014 CFR

2014-04-01

... 24 Housing and Urban Development 2 2014-04-01 2014-04-01 false Performance review. 214.307 Section... COUNSELING PROGRAM Program Administration § 214.307 Performance review. (a) HUD may conduct periodic on-site or desk performance reviews of all participating agencies. (b) The performance review will consist of...

24 CFR 511.80 - Performance review.

Code of Federal Regulations, 2010 CFR

2010-04-01

... 24 Housing and Urban Development 3 2010-04-01 2010-04-01 false Performance review. 511.80 Section... DEVELOPMENT SLUM CLEARANCE AND URBAN RENEWAL RENTAL REHABILITATON GRANT PROGRAM Grantee Performance: Review, Reporting and Corrective or Remedial Actions § 511.80 Performance review. (a) General. HUD will review the...

24 CFR 511.80 - Performance review.

Code of Federal Regulations, 2012 CFR

2012-04-01

... 24 Housing and Urban Development 3 2012-04-01 2012-04-01 false Performance review. 511.80 Section... DEVELOPMENT SLUM CLEARANCE AND URBAN RENEWAL RENTAL REHABILITATION GRANT PROGRAM Grantee Performance: Review, Reporting and Corrective or Remedial Actions § 511.80 Performance review. (a) General. HUD will review the...

24 CFR 214.307 - Performance review.

Code of Federal Regulations, 2010 CFR

2010-04-01

... 24 Housing and Urban Development 2 2010-04-01 2010-04-01 false Performance review. 214.307 Section... COUNSELING PROGRAM Program Administration § 214.307 Performance review. (a) HUD may conduct periodic on-site or desk performance reviews of all participating agencies. (b) The performance review will consist of...

24 CFR 511.80 - Performance review.

Code of Federal Regulations, 2011 CFR

2011-04-01

... 24 Housing and Urban Development 3 2011-04-01 2010-04-01 true Performance review. 511.80 Section... DEVELOPMENT SLUM CLEARANCE AND URBAN RENEWAL RENTAL REHABILITATON GRANT PROGRAM Grantee Performance: Review, Reporting and Corrective or Remedial Actions § 511.80 Performance review. (a) General. HUD will review the...

24 CFR 214.307 - Performance review.

Code of Federal Regulations, 2012 CFR

2012-04-01

... 24 Housing and Urban Development 2 2012-04-01 2012-04-01 false Performance review. 214.307 Section... COUNSELING PROGRAM Program Administration § 214.307 Performance review. (a) HUD may conduct periodic on-site or desk performance reviews of all participating agencies. (b) The performance review will consist of...

24 CFR 511.80 - Performance review.

Code of Federal Regulations, 2013 CFR

2013-04-01

... 24 Housing and Urban Development 3 2013-04-01 2013-04-01 false Performance review. 511.80 Section... DEVELOPMENT SLUM CLEARANCE AND URBAN RENEWAL RENTAL REHABILITATION GRANT PROGRAM Grantee Performance: Review, Reporting and Corrective or Remedial Actions § 511.80 Performance review. (a) General. HUD will review the...

24 CFR 954.600 - Performance reviews.

Code of Federal Regulations, 2010 CFR

2010-04-01

... 24 Housing and Urban Development 4 2010-04-01 2010-04-01 false Performance reviews. 954.600... DEVELOPMENT INDIAN HOME PROGRAM Performance Reviews and Sanctions § 954.600 Performance reviews. (a) General. HUD will review the performance of each grantee in carrying out its responsibilities under this part...

24 CFR 954.600 - Performance reviews.

Code of Federal Regulations, 2011 CFR

2011-04-01

... 24 Housing and Urban Development 4 2011-04-01 2011-04-01 false Performance reviews. 954.600... DEVELOPMENT INDIAN HOME PROGRAM Performance Reviews and Sanctions § 954.600 Performance reviews. (a) General. HUD will review the performance of each grantee in carrying out its responsibilities under this part...

78 FR 13363 - Center for Scientific Review; Notice of Closed Meetings

Federal Register 2010, 2011, 2012, 2013, 2014

2013-02-27

... DEPARTMENT OF HEALTH AND HUMAN SERVICES National Institutes of Health Center for Scientific Review... personal privacy. Name of Committee: Center for Scientific Review Special Emphasis Panel; Small Business...). Contact Person: Bukhtiar H Shah, DVM, Ph.D., Scientific Review Officer, Center for Scientific Review...

77 FR 71429 - Center For Scientific Review; Notice of Closed Meetings

Federal Register 2010, 2011, 2012, 2013, 2014

2012-11-30

... DEPARTMENT OF HEALTH AND HUMAN SERVICES National Institutes of Health Center For Scientific Review... personal privacy. Name of Committee: Center for Scientific Review Special Emphasis Panel Cancer Therapy... Conference Call). Contact Person: Lilia Topol, Ph.D., Scientific Review Officer, Center for Scientific Review...

40 CFR 152.110 - Time for agency review.

Code of Federal Regulations, 2010 CFR

2010-07-01

... 40 Protection of Environment 23 2010-07-01 2010-07-01 false Time for agency review. 152.110... PESTICIDE REGISTRATION AND CLASSIFICATION PROCEDURES Agency Review of Applications § 152.110 Time for agency review. The Agency will complete its review of applications as expeditiously as possible. Applications...

40 CFR 152.108 - Review of labeling.

Code of Federal Regulations, 2010 CFR

2010-07-01

... 40 Protection of Environment 23 2010-07-01 2010-07-01 false Review of labeling. 152.108 Section... PESTICIDE REGISTRATION AND CLASSIFICATION PROCEDURES Agency Review of Applications § 152.108 Review of labeling. The Agency will review all draft labeling submitted with the application. If an applicant for...

Management Reviewing Literature: An Evaluation of Selected Characteristics.

ERIC Educational Resources Information Center

Rehman, Sajjad ur

1987-01-01

Reports results of a study which compared the treatment of selected characteristics of the reviewing literature of management in professional and trade journals. The characteristics examined included lag time, review length, descriptive or analytic nature of reviews, positive or negative evaluations, and affiliation of the reviewer. (CLB)

7 CFR 3400.14 - Proposal review.

Code of Federal Regulations, 2012 CFR

2012-01-01

... RESEARCH GRANTS PROGRAM Scientific Peer Review of Research Grant Applications § 3400.14 Proposal review. (a... review will be made for responsiveness to the request for proposals (e.g., relationship of application to... carefully reviewed by the Director, qualified officers or employees of the Department, the respective peer...

7 CFR 3400.14 - Proposal review.

Code of Federal Regulations, 2014 CFR

2014-01-01

... RESEARCH GRANTS PROGRAM Scientific Peer Review of Research Grant Applications § 3400.14 Proposal review. (a... review will be made for responsiveness to the request for proposals (e.g., relationship of application to... carefully reviewed by the Director, qualified officers or employees of the Department, the respective peer...

7 CFR 3400.14 - Proposal review.

Code of Federal Regulations, 2013 CFR

2013-01-01

... RESEARCH GRANTS PROGRAM Scientific Peer Review of Research Grant Applications § 3400.14 Proposal review. (a... review will be made for responsiveness to the request for proposals (e.g., relationship of application to... carefully reviewed by the Director, qualified officers or employees of the Department, the respective peer...

Writing a book review.

PubMed

Bryson, David; Hudson, Robert

2017-04-01

Book reviews are a good way to get started with writing for a journal and this Learning and CPD activity takes you through the process of understanding the aims of book review, undertaking practice pieces through to reviewing a book and advice on the dos and don'ts of book reviewing.

43 CFR 17.330 - Compliance reviews.

Code of Federal Regulations, 2011 CFR

2011-10-01

..., Conciliation, and Enforcement Procedures § 17.330 Compliance reviews. (a) DOI may conduct compliance reviews... and correct violations of the Act and these regulations. DOI may conduct these reviews even in the... review indicates a violation of the Act or these regulations, DOI will attempt to secure voluntary...

'Are you siding with a personality or the grant proposal?': observations on how peer review panels function.

PubMed

Coveney, John; Herbert, Danielle L; Hill, Kathy; Mow, Karen E; Graves, Nicholas; Barnett, Adrian

2017-01-01

In Australia, the peer review process for competitive funding is usually conducted by a peer review group in conjunction with prior assessment from external assessors. This process is quite mysterious to those outside it. The purpose of this research was to throw light on grant review panels (sometimes called the 'black box') through an examination of the impact of panel procedures, panel composition and panel dynamics on the decision-making in the grant review process. A further purpose was to compare experience of a simplified review process with more conventional processes used in assessing grant proposals in Australia. This project was one aspect of a larger study into the costs and benefits of a simplified peer review process. The Queensland University of Technology (QUT)-simplified process was compared with the National Health and Medical Research Council's (NHMRC) more complex process. Grant review panellists involved in both processes were interviewed about their experience of the decision-making process that assesses the excellence of an application. All interviews were recorded and transcribed. Each transcription was de-identified and returned to the respondent for review. Final transcripts were read repeatedly and coded, and similar codes were amalgamated into categories that were used to build themes. Final themes were shared with the research team for feedback. Two major themes arose from the research: (1) assessing grant proposals and (2) factors influencing the fairness, integrity and objectivity of review. Issues such as the quality of writing in a grant proposal, comparison of the two review methods, the purpose and use of the rebuttal, assessing the financial value of funded projects, the importance of the experience of the panel membership and the role of track record and the impact of group dynamics on the review process were all discussed. The research also examined the influence of research culture on decision-making in grant review panels. One of the aims of this study was to compare a simplified review process with more conventional processes. Generally, participants were supportive of the simplified process. Transparency in the grant review process will result in better appreciation of the outcome. Despite the provision of clear guidelines for peer review, reviewing processes are likely to be subjective to the extent that different reviewers apply different rules. The peer review process will come under more scrutiny as funding for research becomes even more competitive. There is justification for further research on the process, especially of a kind that taps more deeply into the 'black box' of peer review.

42 CFR 8.25 - Informal review and the reviewing official's response.

Code of Federal Regulations, 2013 CFR

2013-10-01

... 42 Public Health 1 2013-10-01 2013-10-01 false Informal review and the reviewing official's response. 8.25 Section 8.25 Public Health PUBLIC HEALTH SERVICE, DEPARTMENT OF HEALTH AND HUMAN SERVICES GENERAL PROVISIONS CERTIFICATION OF OPIOID TREATMENT PROGRAMS Procedures for Review of Suspension or...

42 CFR 8.25 - Informal review and the reviewing official's response.

Code of Federal Regulations, 2012 CFR

2012-10-01

... 42 Public Health 1 2012-10-01 2012-10-01 false Informal review and the reviewing official's response. 8.25 Section 8.25 Public Health PUBLIC HEALTH SERVICE, DEPARTMENT OF HEALTH AND HUMAN SERVICES GENERAL PROVISIONS CERTIFICATION OF OPIOID TREATMENT PROGRAMS Procedures for Review of Suspension or...

42 CFR 8.25 - Informal review and the reviewing official's response.

Code of Federal Regulations, 2014 CFR

2014-10-01

... 42 Public Health 1 2014-10-01 2014-10-01 false Informal review and the reviewing official's response. 8.25 Section 8.25 Public Health PUBLIC HEALTH SERVICE, DEPARTMENT OF HEALTH AND HUMAN SERVICES GENERAL PROVISIONS CERTIFICATION OF OPIOID TREATMENT PROGRAMS Procedures for Review of Suspension or...

75 FR 32956 - Center for Scientific Review; Notice of Closed Meetings

Federal Register 2010, 2011, 2012, 2013, 2014

2010-06-10

... Center, 5701 Marinelli Road, Bethesda, MD 20852. Contact Person: Joanna M. Pyper, PhD, Scientific Review...: Yi-Hsin Liu, PhD, Scientific Review Officer, Center for Scientific Review, National Institutes of... . Name of Committee: Center for Scientific Review Special Emphasis Panel, RFA-DA-10-017: Seek, Test, and...

49 CFR 385.501 - Roadability review.

Code of Federal Regulations, 2010 CFR

2010-10-01

... 49 Transportation 5 2010-10-01 2010-10-01 false Roadability review. 385.501 Section 385.501... Intermodal Equipment Providers § 385.501 Roadability review. (a) FMCSA will perform roadability reviews of... the roadability review using the criteria in appendix A to this part as they relate to compliance with...

43 CFR 10.16 - Review committee.

Code of Federal Regulations, 2010 CFR

2010-10-01

... 43 Public Lands: Interior 1 2010-10-01 2010-10-01 false Review committee. 10.16 Section 10.16... REPATRIATION REGULATIONS General § 10.16 Review committee. (a) General. The Review Committee will advise... recommendation, finding, report, or other action of the Review Committee is advisory only and not binding on any...

42 CFR 419.50 - Annual review.

Code of Federal Regulations, 2010 CFR

2010-10-01

... 42 Public Health 3 2010-10-01 2010-10-01 false Annual review. 419.50 Section 419.50 Public Health... review. (a) General rule. Not less often than annually, CMS reviews and updates groups, relative payment... selection of representatives of providers to review (and advise CMS concerning) the clinical integrity of...

14 CFR 1262.309 - Judicial review.

Code of Federal Regulations, 2010 CFR

2010-01-01

... 14 Aeronautics and Space 5 2010-01-01 2010-01-01 false Judicial review. 1262.309 Section 1262.309... PROCEEDINGS Procedures for Considering Applications § 1262.309 Judicial review. Judicial review of final... the determination to the court of the United States having jurisdiction to review the merits of the...

42 CFR 423.2136 - Judicial review.

Code of Federal Regulations, 2010 CFR

2010-10-01

... 42 Public Health 3 2010-10-01 2010-10-01 false Judicial review. 423.2136 Section 423.2136 Public...) MEDICARE PROGRAM VOLUNTARY MEDICARE PRESCRIPTION DRUG BENEFIT Reopening, ALJ Hearings, MAC review, and Judicial Review § 423.2136 Judicial review. (a) General rule. To the extent authorized by sections 1876(c...

14 CFR 413.15 - Review period.

Code of Federal Regulations, 2010 CFR

2010-01-01

... 14 Aeronautics and Space 4 2010-01-01 2010-01-01 false Review period. 413.15 Section 413.15... TRANSPORTATION LICENSING LICENSE APPLICATION PROCEDURES § 413.15 Review period. (a) Review period duration. Unless otherwise specified in this chapter, the FAA reviews and makes a decision on an application within...

78 FR 52116 - Request for Public Comment on a Review Level Alternative Dispute Resolution Program

Federal Register 2010, 2011, 2012, 2013, 2014

2013-08-22

... OCCUPATIONAL SAFETY AND HEALTH REVIEW COMMISSION 29 CFR Part 2200 Request for Public Comment on a Review Level Alternative Dispute Resolution Program AGENCY: Occupational Safety and Health Review Commission. ACTION: Request for comment. SUMMARY: The Occupational Safety and Health Review Commission...

29 CFR 525.18 - Review.

Code of Federal Regulations, 2010 CFR

2010-07-01

... 29 Labor 3 2010-07-01 2010-07-01 false Review. 525.18 Section 525.18 Labor Regulations Relating to... DISABILITIES UNDER SPECIAL CERTIFICATES § 525.18 Review. Any person aggrieved by any action of the... allow, file with the Administrator a petition for review. Such review, if granted, shall be made by the...

42 CFR 422.612 - Judicial review.

Code of Federal Regulations, 2010 CFR

2010-10-01

... 42 Public Health 3 2010-10-01 2010-10-01 false Judicial review. 422.612 Section 422.612 Public... Judicial review. (a) Review of ALJ's decision. Any party, including the MA organization, may request judicial review (upon notifying the other parties) of an ALJ's decision if— (1) The Board denied the party...

49 CFR 1016.309 - Agency review.

Code of Federal Regulations, 2010 CFR

2010-10-01

... 49 Transportation 8 2010-10-01 2010-10-01 false Agency review. 1016.309 Section 1016.309... Agency review. In the event the adjudicative officer is not the entire Board, the applicant or agency counsel may seek review of the initial decision on the fee application, or the Board may review the...

14 CFR 1252.400 - Compliance reviews.

Code of Federal Regulations, 2010 CFR

2010-01-01

... 14 Aeronautics and Space 5 2010-01-01 2010-01-01 false Compliance reviews. 1252.400 Section 1252..., and Enforcement Procedures § 1252.400 Compliance reviews. (a) NASA may conduct compliance reviews and pre-award reviews of recipients or use other similar procedures that will permit it to investigate and...

43 CFR 17.330 - Compliance reviews.

Code of Federal Regulations, 2010 CFR

2010-10-01

... 43 Public Lands: Interior 1 2010-10-01 2010-10-01 false Compliance reviews. 17.330 Section 17.330..., Conciliation, and Enforcement Procedures § 17.330 Compliance reviews. (a) DOI may conduct compliance reviews and pre-award reviews of recipients or use other similar procedures that will permit it to investigate...

Reviewing Reviews of Research in Educational Leadership: An Empirical Assessment

ERIC Educational Resources Information Center

Hallinger, Philip

2014-01-01

Purpose: Reviews of research play a critical but underappreciated role in knowledge production and accumulation. Yet, until relatively recently, limited attention has been given to the "methodology" of conducting reviews of research. This observation also applies in educational leadership and management where reviews of research have…

40 CFR 179.125 - Judicial review.

Code of Federal Regulations, 2010 CFR

2010-07-01

... 40 Protection of Environment 23 2010-07-01 2010-07-01 false Judicial review. 179.125 Section 179... EVIDENTIARY PUBLIC HEARING Judicial Review § 179.125 Judicial review. (a) The Administrator's final decision... judicial review within the period ending on the 60th day after the date of the publication of the order...

75 FR 60125 - SES Performance Review Board

Federal Register 2010, 2011, 2012, 2013, 2014

2010-09-29

... FEDERAL TRADE COMMISSION SES Performance Review Board AGENCY: Federal Trade Commission. ACTION: Notice. SUMMARY: Notice is hereby given of the appointment of members to the FTC Performance Review Board... the Performance Review Board (PRB) membership is required by 5 U.S.C. 4314(c)(4). The PRB reviews and...

47 CFR 1.1528 - Commission review.

Code of Federal Regulations, 2010 CFR

2010-10-01

... 47 Telecommunication 1 2010-10-01 2010-10-01 false Commission review. 1.1528 Section 1.1528... Commission review. Either the applicant or Bureau counsel may seek Commission review of the initial decision on the application, or the Commission may decide to review the decision on its own initiative, in...

20 CFR 422.605 - Request for review.

Code of Federal Regulations, 2014 CFR

2014-04-01

... 20 Employees' Benefits 2 2014-04-01 2014-04-01 false Request for review. 422.605 Section 422.605 Employees' Benefits SOCIAL SECURITY ADMINISTRATION ORGANIZATION AND PROCEDURES Administrative Review Process Under the Coal Industry Retiree Health Benefit Act of 1992 § 422.605 Request for review. We will review...

20 CFR 422.605 - Request for review.

Code of Federal Regulations, 2012 CFR

2012-04-01

... 20 Employees' Benefits 2 2012-04-01 2012-04-01 false Request for review. 422.605 Section 422.605 Employees' Benefits SOCIAL SECURITY ADMINISTRATION ORGANIZATION AND PROCEDURES Administrative Review Process Under the Coal Industry Retiree Health Benefit Act of 1992 § 422.605 Request for review. We will review...

20 CFR 422.605 - Request for review.

Code of Federal Regulations, 2011 CFR

2011-04-01

... 20 Employees' Benefits 2 2011-04-01 2011-04-01 false Request for review. 422.605 Section 422.605 Employees' Benefits SOCIAL SECURITY ADMINISTRATION ORGANIZATION AND PROCEDURES Administrative Review Process Under the Coal Industry Retiree Health Benefit Act of 1992 § 422.605 Request for review. We will review...

IRIS Toxicological Review of Methanol (External Review Draft; December 2009)

EPA Science Inventory

NOTE: The IRIS Toxicological Review of Methanol (external review draft; December 2009) was released for public comment and external peer review in January 2010 (Federal Register Not...

10 CFR 1045.43 - Systematic review for declassification.

Code of Federal Regulations, 2014 CFR

2014-01-01

... ensure that FRD documents, are periodically and systematically reviewed for declassification. The focus... declassification upon review. (b) Agencies with RD or FRD document holdings shall cooperate with the Director of Classification (and with the DoD for FRD) to ensure the systematic review of RD and FRD documents. (c) Review of...

10 CFR 1045.43 - Systematic review for declassification.

Code of Federal Regulations, 2013 CFR

2013-01-01

... ensure that FRD documents, are periodically and systematically reviewed for declassification. The focus... declassification upon review. (b) Agencies with RD or FRD document holdings shall cooperate with the Director of Classification (and with the DoD for FRD) to ensure the systematic review of RD and FRD documents. (c) Review of...

10 CFR 1045.43 - Systematic review for declassification.

Code of Federal Regulations, 2012 CFR

2012-01-01

... ensure that FRD documents, are periodically and systematically reviewed for declassification. The focus... declassification upon review. (b) Agencies with RD or FRD document holdings shall cooperate with the Director of Classification (and with the DoD for FRD) to ensure the systematic review of RD and FRD documents. (c) Review of...

A Review of Global Learning & Observations to Benefit the Environment (GLOBE)

DTIC Science & Technology

2010-04-01

review (Yoder et al., 2008) of the GPO performance, NASA continued the cooperative agreement with UCAR. Another external review ( Bybee et al., 2008) of...Submitted by the Program Office External Review Committee. Bybee , Rodger W. (Chair), 2008. NASA External Review of a GLOBE Proposal, The Globe

24 CFR 2002.25 - Administrative review.

Code of Federal Regulations, 2010 CFR

2010-04-01

... INFORMATION TO THE PUBLIC § 2002.25 Administrative review. (a) Review is available only from a written denial... should clearly indicate that the subject is a Freedom of Information Act request for review. (c) Review... counsel for the Office of Inspector General. (d) The time of receipt for processing of a request for...

28 CFR 115.86 - Sexual abuse incident reviews.

Code of Federal Regulations, 2013 CFR

2013-07-01

... 28 Judicial Administration 2 2013-07-01 2013-07-01 false Sexual abuse incident reviews. 115.86... NATIONAL STANDARDS Standards for Adult Prisons and Jails Data Collection and Review § 115.86 Sexual abuse incident reviews. (a) The facility shall conduct a sexual abuse incident review at the conclusion of every...

28 CFR 115.386 - Sexual abuse incident reviews.

Code of Federal Regulations, 2012 CFR

2012-07-01

... 28 Judicial Administration 2 2012-07-01 2012-07-01 false Sexual abuse incident reviews. 115.386... NATIONAL STANDARDS Standards for Juvenile Facilities Data Collection and Review § 115.386 Sexual abuse incident reviews. (a) The facility shall conduct a sexual abuse incident review at the conclusion of every...

28 CFR 115.86 - Sexual abuse incident reviews.

Code of Federal Regulations, 2014 CFR

2014-07-01

... 28 Judicial Administration 2 2014-07-01 2014-07-01 false Sexual abuse incident reviews. 115.86... NATIONAL STANDARDS Standards for Adult Prisons and Jails Data Collection and Review § 115.86 Sexual abuse incident reviews. (a) The facility shall conduct a sexual abuse incident review at the conclusion of every...

28 CFR 115.386 - Sexual abuse incident reviews.

Code of Federal Regulations, 2013 CFR

2013-07-01

... 28 Judicial Administration 2 2013-07-01 2013-07-01 false Sexual abuse incident reviews. 115.386... NATIONAL STANDARDS Standards for Juvenile Facilities Data Collection and Review § 115.386 Sexual abuse incident reviews. (a) The facility shall conduct a sexual abuse incident review at the conclusion of every...

28 CFR 115.86 - Sexual abuse incident reviews.

Code of Federal Regulations, 2012 CFR

2012-07-01

... 28 Judicial Administration 2 2012-07-01 2012-07-01 false Sexual abuse incident reviews. 115.86... NATIONAL STANDARDS Standards for Adult Prisons and Jails Data Collection and Review § 115.86 Sexual abuse incident reviews. (a) The facility shall conduct a sexual abuse incident review at the conclusion of every...

28 CFR 115.386 - Sexual abuse incident reviews.

Code of Federal Regulations, 2014 CFR

2014-07-01

... 28 Judicial Administration 2 2014-07-01 2014-07-01 false Sexual abuse incident reviews. 115.386... NATIONAL STANDARDS Standards for Juvenile Facilities Data Collection and Review § 115.386 Sexual abuse incident reviews. (a) The facility shall conduct a sexual abuse incident review at the conclusion of every...

WASTE ANALYSIS PLAN REVIEW ADVISOR - AN INTELLIGENT DATABASE TO ASSIST RCRA PERMIT REVIEWERS

EPA Science Inventory

The Waste Analysis Plan Review Advisor (WAPRA) system assists in the review of the Waste Analysis Plan Section of RCRA Part B facility permit applications. Specifically, this program automates two functions of the waste analysis plan review. First, the system checks all wastes wh...

77 FR 26046 - Agency Information Collection Activities; Submission for OMB Review; Comment Request; Affordable...

Federal Register 2010, 2011, 2012, 2013, 2014

2012-05-02

... for OMB Review; Comment Request; Affordable Care Act Internal Claims and Appeals and External Review...) titled, ``Affordable Care Act Internal Claims and Appeals and External Review Procedures for Non... provisions of the Affordable Care Act pertaining to internal claims and appeals, and the external review...

7 CFR 3406.14 - Proposal review-teaching.

Code of Federal Regulations, 2010 CFR

2010-01-01

... 7 Agriculture 15 2010-01-01 2010-01-01 false Proposal review-teaching. 3406.14 Section 3406.14... Review and Evaluation of a Teaching Proposal § 3406.14 Proposal review—teaching. The proposal evaluation process includes both internal staff review and merit evaluation by peer review panels comprised of...

20 CFR 422.605 - Request for review.

Code of Federal Regulations, 2010 CFR

2010-04-01

... 20 Employees' Benefits 2 2010-04-01 2010-04-01 false Request for review. 422.605 Section 422.605 Employees' Benefits SOCIAL SECURITY ADMINISTRATION ORGANIZATION AND PROCEDURES Administrative Review Process Under the Coal Industry Retiree Health Benefit Act of 1992 § 422.605 Request for review. We will review...

32 CFR 206.4 - Proposal development and review.

Code of Federal Regulations, 2013 CFR

2013-07-01

... 32 National Defense 2 2013-07-01 2013-07-01 false Proposal development and review. 206.4 Section... Proposal development and review. The purpose of this section is to explain the NSEP review process. [Note: A number of important approaches to proposal development and review have been adapted from...

32 CFR 206.4 - Proposal development and review.

Code of Federal Regulations, 2014 CFR

2014-07-01

... 32 National Defense 2 2014-07-01 2014-07-01 false Proposal development and review. 206.4 Section... Proposal development and review. The purpose of this section is to explain the NSEP review process. [Note: A number of important approaches to proposal development and review have been adapted from...

32 CFR 206.4 - Proposal development and review.

Code of Federal Regulations, 2012 CFR

2012-07-01

... 32 National Defense 2 2012-07-01 2012-07-01 false Proposal development and review. 206.4 Section... Proposal development and review. The purpose of this section is to explain the NSEP review process. [Note: A number of important approaches to proposal development and review have been adapted from...

29 CFR 2200.92 - Review by the Commission.

Code of Federal Regulations, 2010 CFR

2010-07-01

... Regulations Relating to Labor (Continued) OCCUPATIONAL SAFETY AND HEALTH REVIEW COMMISSION RULES OF PROCEDURE Posthearing Procedures § 2200.92 Review by the Commission. (a) Jurisdiction of the Commission; issues on... Commission to review the entire case. The issues to be decided on review are within the discretion of the...

28 CFR 115.186 - Sexual abuse incident reviews.

Code of Federal Regulations, 2014 CFR

2014-07-01

... 28 Judicial Administration 2 2014-07-01 2014-07-01 false Sexual abuse incident reviews. 115.186... NATIONAL STANDARDS Standards for Lockups Data Collection and Review § 115.186 Sexual abuse incident reviews. (a) The lockup shall conduct a sexual abuse incident review at the conclusion of every sexual abuse...

28 CFR 115.186 - Sexual abuse incident reviews.

Code of Federal Regulations, 2012 CFR

2012-07-01

... 28 Judicial Administration 2 2012-07-01 2012-07-01 false Sexual abuse incident reviews. 115.186... NATIONAL STANDARDS Standards for Lockups Data Collection and Review § 115.186 Sexual abuse incident reviews. (a) The lockup shall conduct a sexual abuse incident review at the conclusion of every sexual abuse...

28 CFR 115.186 - Sexual abuse incident reviews.

Code of Federal Regulations, 2013 CFR

2013-07-01

... 28 Judicial Administration 2 2013-07-01 2013-07-01 false Sexual abuse incident reviews. 115.186... NATIONAL STANDARDS Standards for Lockups Data Collection and Review § 115.186 Sexual abuse incident reviews. (a) The lockup shall conduct a sexual abuse incident review at the conclusion of every sexual abuse...