Design Development Test and Evaluation (DDT and E) Considerations for Safe and Reliable Human Rated Spacecraft Systems

NASA Technical Reports Server (NTRS)

Miller, James; Leggett, Jay; Kramer-White, Julie

2008-01-01

A team directed by the NASA Engineering and Safety Center (NESC) collected methodologies for how best to develop safe and reliable human rated systems and how to identify the drivers that provide the basis for assessing safety and reliability. The team also identified techniques, methodologies, and best practices to assure that NASA can develop safe and reliable human rated systems. The results are drawn from a wide variety of resources, from experts involved with the space program since its inception to the best-practices espoused in contemporary engineering doctrine. This report focuses on safety and reliability considerations and does not duplicate or update any existing references. Neither does it intend to replace existing standards and policy.

Rating the methodological quality of single-subject designs and n-of-1 trials: introducing the Single-Case Experimental Design (SCED) Scale.

PubMed

Tate, Robyn L; McDonald, Skye; Perdices, Michael; Togher, Leanne; Schultz, Regina; Savage, Sharon

2008-08-01

Rating scales that assess methodological quality of clinical trials provide a means to critically appraise the literature. Scales are currently available to rate randomised and non-randomised controlled trials, but there are none that assess single-subject designs. The Single-Case Experimental Design (SCED) Scale was developed for this purpose and evaluated for reliability. Six clinical researchers who were trained and experienced in rating methodological quality of clinical trials developed the scale and participated in reliability studies. The SCED Scale is an 11-item rating scale for single-subject designs, of which 10 items are used to assess methodological quality and use of statistical analysis. The scale was developed and refined over a 3-year period. Content validity was addressed by identifying items to reduce the main sources of bias in single-case methodology as stipulated by authorities in the field, which were empirically tested against 85 published reports. Inter-rater reliability was assessed using a random sample of 20/312 single-subject reports archived in the Psychological Database of Brain Impairment Treatment Efficacy (PsycBITE). Inter-rater reliability for the total score was excellent, both for individual raters (overall ICC = 0.84; 95% confidence interval 0.73-0.92) and for consensus ratings between pairs of raters (overall ICC = 0.88; 95% confidence interval 0.78-0.95). Item reliability was fair to excellent for consensus ratings between pairs of raters (range k = 0.48 to 1.00). The results were replicated with two independent novice raters who were trained in the use of the scale (ICC = 0.88, 95% confidence interval 0.73-0.95). The SCED Scale thus provides a brief and valid evaluation of methodological quality of single-subject designs, with the total score demonstrating excellent inter-rater reliability using both individual and consensus ratings. Items from the scale can also be used as a checklist in the design, reporting and critical appraisal of single-subject designs, thereby assisting to improve standards of single-case methodology.

Measurement properties of existing clinical assessment methods evaluating scapular positioning and function. A systematic review.

PubMed

Larsen, Camilla Marie; Juul-Kristensen, Birgit; Lund, Hans; Søgaard, Karen

2014-10-01

The aims were to compile a schematic overview of clinical scapular assessment methods and critically appraise the methodological quality of the involved studies. A systematic, computer-assisted literature search using Medline, CINAHL, SportDiscus and EMBASE was performed from inception to October 2013. Reference lists in articles were also screened for publications. From 50 articles, 54 method names were identified and categorized into three groups: (1) Static positioning assessment (n = 19); (2) Semi-dynamic (n = 13); and (3) Dynamic functional assessment (n = 22). Fifteen studies were excluded for evaluation due to no/few clinimetric results, leaving 35 studies for evaluation. Graded according to the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN checklist), the methodological quality in the reliability and validity domains was "fair" (57%) to "poor" (43%), with only one study rated as "good". The reliability domain was most often investigated. Few of the assessment methods in the included studies that had "fair" or "good" measurement property ratings demonstrated acceptable results for both reliability and validity. We found a substantially larger number of clinical scapular assessment methods than previously reported. Using the COSMIN checklist the methodological quality of the included measurement properties in the reliability and validity domains were in general "fair" to "poor". None were examined for all three domains: (1) reliability; (2) validity; and (3) responsiveness. Observational evaluation systems and assessment of scapular upward rotation seem suitably evidence-based for clinical use. Future studies should test and improve the clinimetric properties, and especially diagnostic accuracy and responsiveness, to increase utility for clinical practice.

Reliability and precision of pellet-group counts for estimating landscape-level deer density

Treesearch

David S. deCalesta

2013-01-01

This study provides hitherto unavailable methodology for reliably and precisely estimating deer density within forested landscapes, enabling quantitative rather than qualitative deer management. Reliability and precision of the deer pellet-group technique were evaluated in 1 small and 2 large forested landscapes. Density estimates, adjusted to reflect deer harvest and...

A study on the real-time reliability of on-board equipment of train control system

NASA Astrophysics Data System (ADS)

Zhang, Yong; Li, Shiwei

2018-05-01

Real-time reliability evaluation is conducive to establishing a condition based maintenance system for the purpose of guaranteeing continuous train operation. According to the inherent characteristics of the on-board equipment, the connotation of reliability evaluation of on-board equipment is defined and the evaluation index of real-time reliability is provided in this paper. From the perspective of methodology and practical application, the real-time reliability of the on-board equipment is discussed in detail, and the method of evaluating the realtime reliability of on-board equipment at component level based on Hidden Markov Model (HMM) is proposed. In this method the performance degradation data is used directly to realize the accurate perception of the hidden state transition process of on-board equipment, which can achieve a better description of the real-time reliability of the equipment.

A Review on VSC-HVDC Reliability Modeling and Evaluation Techniques

NASA Astrophysics Data System (ADS)

Shen, L.; Tang, Q.; Li, T.; Wang, Y.; Song, F.

2017-05-01

With the fast development of power electronics, voltage-source converter (VSC) HVDC technology presents cost-effective ways for bulk power transmission. An increasing number of VSC-HVDC projects has been installed worldwide. Their reliability affects the profitability of the system and therefore has a major impact on the potential investors. In this paper, an overview of the recent advances in the area of reliability evaluation for VSC-HVDC systems is provided. Taken into account the latest multi-level converter topology, the VSC-HVDC system is categorized into several sub-systems and the reliability data for the key components is discussed based on sources with academic and industrial backgrounds. The development of reliability evaluation methodologies is reviewed and the issues surrounding the different computation approaches are briefly analysed. A general VSC-HVDC reliability evaluation procedure is illustrated in this paper.

The Application of a Residual Risk Evaluation Technique Used for Expendable Launch Vehicles

NASA Technical Reports Server (NTRS)

Latimer, John A.

2009-01-01

This presentation provides a Residual Risk Evaluation Technique (RRET) developed by Kennedy Space Center (KSC) Safety and Mission Assurance (S&MA) Launch Services Division. This technique is one of many procedures used by S&MA at KSC to evaluate residual risks for each Expendable Launch Vehicle (ELV) mission. RRET is a straight forward technique that incorporates the proven methodology of risk management, fault tree analysis, and reliability prediction. RRET derives a system reliability impact indicator from the system baseline reliability and the system residual risk reliability values. The system reliability impact indicator provides a quantitative measure of the reduction in the system baseline reliability due to the identified residual risks associated with the designated ELV mission. An example is discussed to provide insight into the application of RRET.

Evaluation of Explosive Strength for Young and Adult Athletes

ERIC Educational Resources Information Center

Viitasalo, Jukka T.

1988-01-01

The reliability of new electrical measurements of vertical jumping height and of throwing velocity was tested. These results were compared to traditional measurement techniques. The new method was found to give reliable results from children to adults. Methodology is discussed. (Author/JL)

Evaluation of the HARDMAN comparability methodology for manpower, personnel and training

NASA Technical Reports Server (NTRS)

Zimmerman, W.; Butler, R.; Gray, V.; Rosenberg, L.

1984-01-01

The methodology evaluation and recommendation are part of an effort to improve Hardware versus Manpower (HARDMAN) methodology for projecting manpower, personnel, and training (MPT) to support new acquisition. Several different validity tests are employed to evaluate the methodology. The methodology conforms fairly well with both the MPT user needs and other accepted manpower modeling techniques. Audits of three completed HARDMAN applications reveal only a small number of potential problem areas compared to the total number of issues investigated. The reliability study results conform well with the problem areas uncovered through the audits. The results of the accuracy studies suggest that the manpower life-cycle cost component is only marginally sensitive to changes in other related cost variables. Even with some minor problems, the methodology seem sound and has good near term utility to the Army. Recommendations are provided to firm up the problem areas revealed through the evaluation.

On-time reliability impacts of ATIS. Volume III, Implications for ATIS investment strategies

DOT National Transportation Integrated Search

2003-05-01

The effect of ATIS accuracy and extent of ATIS roadway instrumentation on the on-time reliability benefits to routine users of ATIS are evaluated through the application of Heuristic On-line Web-linked Arrival Time Estimation (HOWLATE) methodology. T...

76 FR 3604 - Information Collection; Qualified Products List for Engine Driven Pumps

Federal Register 2010, 2011, 2012, 2013, 2014

2011-01-20

... levels. 2. Reliability and endurance requirements. These requirements include a 100-hour endurance test... evaluated to meet specific requirements related to safety, effectiveness, efficiency, and reliability of the... of the collection of information, including the validity of the methodology and assumptions used; (3...

Reliability of specific physical examination tests for the diagnosis of shoulder pathologies: a systematic review and meta-analysis.

PubMed

Lange, Toni; Matthijs, Omer; Jain, Nitin B; Schmitt, Jochen; Lützner, Jörg; Kopkow, Christian

2017-03-01

Shoulder pain in the general population is common and to identify the aetiology of shoulder pain, history, motion and muscle testing, and physical examination tests are usually performed. The aim of this systematic review was to summarise and evaluate intrarater and inter-rater reliability of physical examination tests in the diagnosis of shoulder pathologies. A comprehensive systematic literature search was conducted using MEDLINE, EMBASE, Allied and Complementary Medicine Database (AMED) and Physiotherapy Evidence Database (PEDro) through 20 March 2015. Methodological quality was assessed using the Quality Appraisal of Reliability Studies (QAREL) tool by 2 independent reviewers. The search strategy revealed 3259 articles, of which 18 finally met the inclusion criteria. These studies evaluated the reliability of 62 test and test variations used for the specific physical examination tests for the diagnosis of shoulder pathologies. Methodological quality ranged from 2 to 7 positive criteria of the 11 items of the QAREL tool. This review identified a lack of high-quality studies evaluating inter-rater as well as intrarater reliability of specific physical examination tests for the diagnosis of shoulder pathologies. In addition, reliability measures differed between included studies hindering proper cross-study comparisons. PROSPERO CRD42014009018. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/.

Methodological Issues in Measuring the Development of Character

ERIC Educational Resources Information Center

Card, Noel A.

2017-01-01

In this article I provide an overview of the methodological issues involved in measuring constructs relevant to character development and education. I begin with a nontechnical overview of the 3 fundamental psychometric properties of measurement: reliability, validity, and equivalence. Developing and evaluating measures to ensure evidence of all 3…

The reliability of the Glasgow Coma Scale: a systematic review.

PubMed

Reith, Florence C M; Van den Brande, Ruben; Synnot, Anneliese; Gruen, Russell; Maas, Andrew I R

2016-01-01

The Glasgow Coma Scale (GCS) provides a structured method for assessment of the level of consciousness. Its derived sum score is applied in research and adopted in intensive care unit scoring systems. Controversy exists on the reliability of the GCS. The aim of this systematic review was to summarize evidence on the reliability of the GCS. A literature search was undertaken in MEDLINE, EMBASE and CINAHL. Observational studies that assessed the reliability of the GCS, expressed by a statistical measure, were included. Methodological quality was evaluated with the consensus-based standards for the selection of health measurement instruments checklist and its influence on results considered. Reliability estimates were synthesized narratively. We identified 52 relevant studies that showed significant heterogeneity in the type of reliability estimates used, patients studied, setting and characteristics of observers. Methodological quality was good (n = 7), fair (n = 18) or poor (n = 27). In good quality studies, kappa values were ≥0.6 in 85%, and all intraclass correlation coefficients indicated excellent reliability. Poor quality studies showed lower reliability estimates. Reliability for the GCS components was higher than for the sum score. Factors that may influence reliability include education and training, the level of consciousness and type of stimuli used. Only 13% of studies were of good quality and inconsistency in reported reliability estimates was found. Although the reliability was adequate in good quality studies, further improvement is desirable. From a methodological perspective, the quality of reliability studies needs to be improved. From a clinical perspective, a renewed focus on training/education and standardization of assessment is required.

Reliability and availability evaluation of Wireless Sensor Networks for industrial applications.

PubMed

Silva, Ivanovitch; Guedes, Luiz Affonso; Portugal, Paulo; Vasques, Francisco

2012-01-01

Wireless Sensor Networks (WSN) currently represent the best candidate to be adopted as the communication solution for the last mile connection in process control and monitoring applications in industrial environments. Most of these applications have stringent dependability (reliability and availability) requirements, as a system failure may result in economic losses, put people in danger or lead to environmental damages. Among the different type of faults that can lead to a system failure, permanent faults on network devices have a major impact. They can hamper communications over long periods of time and consequently disturb, or even disable, control algorithms. The lack of a structured approach enabling the evaluation of permanent faults, prevents system designers to optimize decisions that minimize these occurrences. In this work we propose a methodology based on an automatic generation of a fault tree to evaluate the reliability and availability of Wireless Sensor Networks, when permanent faults occur on network devices. The proposal supports any topology, different levels of redundancy, network reconfigurations, criticality of devices and arbitrary failure conditions. The proposed methodology is particularly suitable for the design and validation of Wireless Sensor Networks when trying to optimize its reliability and availability requirements.

Reliability and Availability Evaluation of Wireless Sensor Networks for Industrial Applications

PubMed Central

Silva, Ivanovitch; Guedes, Luiz Affonso; Portugal, Paulo; Vasques, Francisco

2012-01-01

Wireless Sensor Networks (WSN) currently represent the best candidate to be adopted as the communication solution for the last mile connection in process control and monitoring applications in industrial environments. Most of these applications have stringent dependability (reliability and availability) requirements, as a system failure may result in economic losses, put people in danger or lead to environmental damages. Among the different type of faults that can lead to a system failure, permanent faults on network devices have a major impact. They can hamper communications over long periods of time and consequently disturb, or even disable, control algorithms. The lack of a structured approach enabling the evaluation of permanent faults, prevents system designers to optimize decisions that minimize these occurrences. In this work we propose a methodology based on an automatic generation of a fault tree to evaluate the reliability and availability of Wireless Sensor Networks, when permanent faults occur on network devices. The proposal supports any topology, different levels of redundancy, network reconfigurations, criticality of devices and arbitrary failure conditions. The proposed methodology is particularly suitable for the design and validation of Wireless Sensor Networks when trying to optimize its reliability and availability requirements. PMID:22368497

The PEDro scale had acceptably high convergent validity, construct validity, and interrater reliability in evaluating methodological quality of pharmaceutical trials.

PubMed

Yamato, Tie Parma; Maher, Chris; Koes, Bart; Moseley, Anne

2017-06-01

The Physiotherapy Evidence Database (PEDro) scale has been widely used to investigate methodological quality in physiotherapy randomized controlled trials; however, its validity has not been tested for pharmaceutical trials. The aim of this study was to investigate the validity and interrater reliability of the PEDro scale for pharmaceutical trials. The reliability was also examined for the Cochrane Back and Neck (CBN) Group risk of bias tool. This is a secondary analysis of data from a previous study. We considered randomized placebo controlled trials evaluating any pain medication for chronic spinal pain or osteoarthritis. Convergent validity was evaluated by correlating the PEDro score with the summary score of the CBN risk of bias tool. The construct validity was tested using a linear regression analysis to determine the degree to which the total PEDro score is associated with treatment effect sizes, journal impact factor, and the summary score for the CBN risk of bias tool. The interrater reliability was estimated using the Prevalence and Bias Adjusted Kappa coefficient and 95% confidence interval (CI) for the PEDro scale and CBN risk of bias tool. Fifty-three trials were included, with 91 treatment effect sizes included in the analyses. The correlation between PEDro scale and CBN risk of bias tool was 0.83 (95% CI 0.76-0.88) after adjusting for reliability, indicating strong convergence. The PEDro score was inversely associated with effect sizes, significantly associated with the summary score for the CBN risk of bias tool, and not associated with the journal impact factor. The interrater reliability for each item of the PEDro scale and CBN risk of bias tool was at least substantial for most items (>0.60). The intraclass correlation coefficient for the PEDro score was 0.80 (95% CI 0.68-0.88), and for the CBN, risk of bias tool was 0.81 (95% CI 0.69-0.88). There was evidence for the convergent and construct validity for the PEDro scale when used to evaluate methodological quality of pharmacological trials. Both risk of bias tools have acceptably high interrater reliability. Copyright © 2017 Elsevier Inc. All rights reserved.

Vending machine assessment methodology. A systematic review.

PubMed

Matthews, Melissa A; Horacek, Tanya M

2015-07-01

The nutritional quality of food and beverage products sold in vending machines has been implicated as a contributing factor to the development of an obesogenic food environment. How comprehensive, reliable, and valid are the current assessment tools for vending machines to support or refute these claims? A systematic review was conducted to summarize, compare, and evaluate the current methodologies and available tools for vending machine assessment. A total of 24 relevant research studies published between 1981 and 2013 met inclusion criteria for this review. The methodological variables reviewed in this study include assessment tool type, study location, machine accessibility, product availability, healthfulness criteria, portion size, price, product promotion, and quality of scientific practice. There were wide variations in the depth of the assessment methodologies and product healthfulness criteria utilized among the reviewed studies. Of the reviewed studies, 39% evaluated machine accessibility, 91% evaluated product availability, 96% established healthfulness criteria, 70% evaluated portion size, 48% evaluated price, 52% evaluated product promotion, and 22% evaluated the quality of scientific practice. Of all reviewed articles, 87% reached conclusions that provided insight into the healthfulness of vended products and/or vending environment. Product healthfulness criteria and complexity for snack and beverage products was also found to be variable between the reviewed studies. These findings make it difficult to compare results between studies. A universal, valid, and reliable vending machine assessment tool that is comprehensive yet user-friendly is recommended. Copyright © 2015 Elsevier Ltd. All rights reserved.

Evaluating Federal Social Programs: An Uncertain Act.

ERIC Educational Resources Information Center

Levitan, Sar A.; Wurzburg, Gregory K.

This study of the federal government's evaluation of social programs indicates that it is virtually impossible to establish a bias-free, valid, and reliable system of inquiry to determine the effects of social programs. Divided into five chapters, the document examines the aspirations and limitations of evaluations, methodology, evaluation in the…

Reliability based design optimization: Formulations and methodologies

NASA Astrophysics Data System (ADS)

Agarwal, Harish

Modern products ranging from simple components to complex systems should be designed to be optimal and reliable. The challenge of modern engineering is to ensure that manufacturing costs are reduced and design cycle times are minimized while achieving requirements for performance and reliability. If the market for the product is competitive, improved quality and reliability can generate very strong competitive advantages. Simulation based design plays an important role in designing almost any kind of automotive, aerospace, and consumer products under these competitive conditions. Single discipline simulations used for analysis are being coupled together to create complex coupled simulation tools. This investigation focuses on the development of efficient and robust methodologies for reliability based design optimization in a simulation based design environment. Original contributions of this research are the development of a novel efficient and robust unilevel methodology for reliability based design optimization, the development of an innovative decoupled reliability based design optimization methodology, the application of homotopy techniques in unilevel reliability based design optimization methodology, and the development of a new framework for reliability based design optimization under epistemic uncertainty. The unilevel methodology for reliability based design optimization is shown to be mathematically equivalent to the traditional nested formulation. Numerical test problems show that the unilevel methodology can reduce computational cost by at least 50% as compared to the nested approach. The decoupled reliability based design optimization methodology is an approximate technique to obtain consistent reliable designs at lesser computational expense. Test problems show that the methodology is computationally efficient compared to the nested approach. A framework for performing reliability based design optimization under epistemic uncertainty is also developed. A trust region managed sequential approximate optimization methodology is employed for this purpose. Results from numerical test studies indicate that the methodology can be used for performing design optimization under severe uncertainty.

An Evaluation Methodology for Natural Language Processing Systems

DTIC Science & Technology

1992-12-01

8217DT".3 "_Griffiss Air Force Base U , . d J _!::! • •, .>:--------. ..._ _. ...........• E v¢ .................. ......... Av,:iabihty Codes Avv L...each " trial " or evaluation item. This approach to assessing reliability has some similarity to the reliability study per- formed by Hix and Schulman for...the clinic . Criteria: Demonstrated understanding that the object or entity expressed by the first noun benefits from the object expressed by the second

Ensuring reliability in expansion schemes.

PubMed

Kamal-Uddin, Abu Sayed; Williams, Donald Leigh

2005-01-01

Existing electricity power supplies must serve, or be adapted to serve, the expansion of hospital buildings. With the existing power supply assets of many hospitals being up to 20 years old, assessing the security and reliability of the power system must be given appropriate priority to avoid unplanned outages due to overloads and equipment failures. It is imperative that adequate contingency is planned for essential and non-essential electricity circuits. This article describes the methodology undertaken, and the subsequent recommendations that were made, when evaluating the security and reliability of electricity power supplies to a number of major London hospitals. The methodology described aligns with the latest issue of NHS Estates HTM 2011 'Primary Electrical Infrastructure Emergency Electrical Services Design Guidance' (to which ERA Technology has contributed).

Reliability in perceptual analysis of voice quality.

PubMed

Bele, Irene Velsvik

2005-12-01

This study focuses on speaking voice quality in male teachers (n = 35) and male actors (n = 36), who represent untrained and trained voice users, because we wanted to investigate normal and supranormal voices. In this study, both substantial and methodologic aspects were considered. It includes a method for perceptual voice evaluation, and a basic issue was rater reliability. A listening group of 10 listeners, 7 experienced speech-language therapists, and 3 speech-language therapist students evaluated the voices by 15 vocal characteristics using VA scales. Two sets of voice signals were investigated: text reading (2 loudness levels) and sustained vowel (3 levels). The results indicated a high interrater reliability for most perceptual characteristics. Connected speech was evaluated more reliably, especially at the normal level, but both types of voice signals were evaluated reliably, although the reliability for connected speech was somewhat higher than for vowels. Experienced listeners tended to be more consistent in their ratings than did the student raters. Some vocal characteristics achieved acceptable reliability even with a smaller panel of listeners. The perceptual characteristics grouped in 4 factors reflected perceptual dimensions.

Review of evaluation on ecological carrying capacity: The progress and trend of methodology

NASA Astrophysics Data System (ADS)

Wang, S. F.; Xu, Y.; Liu, T. J.; Ye, J. M.; Pan, B. L.; Chu, C.; Peng, Z. L.

2018-02-01

The ecological carrying capacity (ECC) has been regarded as an important reference to indicate the level of regional sustainable development since the very beginning of twenty-first century. By a brief review of the main progress in ECC evaluation methodologies in recent five years, this paper systematically discusses the features and differences of these methods and expounds the current states and future development trend of ECC methodology. The result shows that further exploration in terms of the dynamic, comprehensive and intelligent assessment technologies needs to be provided in order to form a unified and scientific ECC methodology system and to produce a reliable basis for environmental-economic decision-makings.

How to assess and compare inter-rater reliability, agreement and correlation of ratings: an exemplary analysis of mother-father and parent-teacher expressive vocabulary rating pairs

PubMed Central

Stolarova, Margarita; Wolf, Corinna; Rinker, Tanja; Brielmann, Aenne

2014-01-01

This report has two main purposes. First, we combine well-known analytical approaches to conduct a comprehensive assessment of agreement and correlation of rating-pairs and to dis-entangle these often confused concepts, providing a best-practice example on concrete data and a tutorial for future reference. Second, we explore whether a screening questionnaire developed for use with parents can be reliably employed with daycare teachers when assessing early expressive vocabulary. A total of 53 vocabulary rating pairs (34 parent–teacher and 19 mother–father pairs) collected for two-year-old children (12 bilingual) are evaluated. First, inter-rater reliability both within and across subgroups is assessed using the intra-class correlation coefficient (ICC). Next, based on this analysis of reliability and on the test-retest reliability of the employed tool, inter-rater agreement is analyzed, magnitude and direction of rating differences are considered. Finally, Pearson correlation coefficients of standardized vocabulary scores are calculated and compared across subgroups. The results underline the necessity to distinguish between reliability measures, agreement and correlation. They also demonstrate the impact of the employed reliability on agreement evaluations. This study provides evidence that parent–teacher ratings of children's early vocabulary can achieve agreement and correlation comparable to those of mother–father ratings on the assessed vocabulary scale. Bilingualism of the evaluated child decreased the likelihood of raters' agreement. We conclude that future reports of agreement, correlation and reliability of ratings will benefit from better definition of terms and stricter methodological approaches. The methodological tutorial provided here holds the potential to increase comparability across empirical reports and can help improve research practices and knowledge transfer to educational and therapeutic settings. PMID:24994985

Fracture mechanics methodology: Evaluation of structural components integrity

NASA Astrophysics Data System (ADS)

Sih, G. C.; de Oliveira Faria, L.

1984-09-01

The application of fracture mechanics to structural-design problems is discussed in lectures presented in the AGARD Fracture Mechanics Methodology course held in Lisbon, Portugal, in June 1981. The emphasis is on aeronautical design, and chapters are included on fatigue-life prediction for metals and composites, the fracture mechanics of engineering structural components, failure mechanics and damage evaluation of structural components, flaw-acceptance methods, and reliability in probabilistic design. Graphs, diagrams, drawings, and photographs are provided.

Reliability-Productivity Curve, a Tool for Adaptation Measures Identification

NASA Astrophysics Data System (ADS)

Chávez-Jiménez, A.; Granados, A.; Garrote, L. M.

2015-12-01

Due to climate change effects, water scarcity problems would intensify in several regions. These problems are going to impact negatively in the water low-priority demands, since these will be reduced in favor of those with high-priority. An example would be the reduction of agriculture water resources in favor of the urban ones. Then, it is important the evaluation of adaptation measures for a better water resources management. An important tool to face this challenge is the economic valuation of the water demands' impact within a water resources system. In agriculture this valuation is usually performed through the water productivity evaluation. The water productivity evaluation requires detailed information regarding the different crops like the applied technology, the agricultural supplies management, the water availability, etc. This is a restriction for an evaluation at basin scale due to the difficulty of gathers this level of detailed information. Besides, only the water availability is taken into account, but not the period when the water is distributed (i.e. water resources reliability). Water resources reliability is one of the most important variables in water resources management. This research proposes a methodology to determine the agriculture water productivity, using as variables the crops information, the crops price, the water resources availability, and the water resources reliability, at a basin scale. This methodology would allow identifying general water resources adaptation measures, providing the basis for further detailed studies in critical regions.

Life Cycle Analysis of a SpaceCube Printed Circuit Board Assembly Using Physics of Failure Methodologies

NASA Technical Reports Server (NTRS)

Sood, Bhanu; Evans, John; Daniluk, Kelly; Sturgis, Jason; Davis, Milton; Petrick, David

2017-01-01

In this reliability life cycle evaluation of the SpaceCube 2.0 processor card, a partially populated version of the card is being evaluated to determine its durability with respect to typical GSFC mission loads.

Lifetime Reliability Prediction of Ceramic Structures Under Transient Thermomechanical Loads

NASA Technical Reports Server (NTRS)

Nemeth, Noel N.; Jadaan, Osama J.; Gyekenyesi, John P.

2005-01-01

An analytical methodology is developed to predict the probability of survival (reliability) of ceramic components subjected to harsh thermomechanical loads that can vary with time (transient reliability analysis). This capability enables more accurate prediction of ceramic component integrity against fracture in situations such as turbine startup and shutdown, operational vibrations, atmospheric reentry, or other rapid heating or cooling situations (thermal shock). The transient reliability analysis methodology developed herein incorporates the following features: fast-fracture transient analysis (reliability analysis without slow crack growth, SCG); transient analysis with SCG (reliability analysis with time-dependent damage due to SCG); a computationally efficient algorithm to compute the reliability for components subjected to repeated transient loading (block loading); cyclic fatigue modeling using a combined SCG and Walker fatigue law; proof testing for transient loads; and Weibull and fatigue parameters that are allowed to vary with temperature or time. Component-to-component variation in strength (stochastic strength response) is accounted for with the Weibull distribution, and either the principle of independent action or the Batdorf theory is used to predict the effect of multiaxial stresses on reliability. The reliability analysis can be performed either as a function of the component surface (for surface-distributed flaws) or component volume (for volume-distributed flaws). The transient reliability analysis capability has been added to the NASA CARES/ Life (Ceramic Analysis and Reliability Evaluation of Structures/Life) code. CARES/Life was also updated to interface with commercially available finite element analysis software, such as ANSYS, when used to model the effects of transient load histories. Examples are provided to demonstrate the features of the methodology as implemented in the CARES/Life program.

Advanced reliability modeling of fault-tolerant computer-based systems

NASA Technical Reports Server (NTRS)

Bavuso, S. J.

1982-01-01

Two methodologies for the reliability assessment of fault tolerant digital computer based systems are discussed. The computer-aided reliability estimation 3 (CARE 3) and gate logic software simulation (GLOSS) are assessment technologies that were developed to mitigate a serious weakness in the design and evaluation process of ultrareliable digital systems. The weak link is based on the unavailability of a sufficiently powerful modeling technique for comparing the stochastic attributes of one system against others. Some of the more interesting attributes are reliability, system survival, safety, and mission success.

Protocol for Reliability Assessment of Structural Health Monitoring Systems Incorporating Model-assisted Probability of Detection (MAPOD) Approach

DTIC Science & Technology

2011-09-01

a quality evaluation with limited data, a model -based assessment must be...that affect system performance, a multistage approach to system validation, a modeling and experimental methodology for efficiently addressing a ...affect system performance, a multistage approach to system validation, a modeling and experimental methodology for efficiently addressing a wide range

Reliability-Based Stability Analysis of Rock Slopes Using Numerical Analysis and Response Surface Method

NASA Astrophysics Data System (ADS)

Dadashzadeh, N.; Duzgun, H. S. B.; Yesiloglu-Gultekin, N.

2017-08-01

While advanced numerical techniques in slope stability analysis are successfully used in deterministic studies, they have so far found limited use in probabilistic analyses due to their high computation cost. The first-order reliability method (FORM) is one of the most efficient probabilistic techniques to perform probabilistic stability analysis by considering the associated uncertainties in the analysis parameters. However, it is not possible to directly use FORM in numerical slope stability evaluations as it requires definition of a limit state performance function. In this study, an integrated methodology for probabilistic numerical modeling of rock slope stability is proposed. The methodology is based on response surface method, where FORM is used to develop an explicit performance function from the results of numerical simulations. The implementation of the proposed methodology is performed by considering a large potential rock wedge in Sumela Monastery, Turkey. The accuracy of the developed performance function to truly represent the limit state surface is evaluated by monitoring the slope behavior. The calculated probability of failure is compared with Monte Carlo simulation (MCS) method. The proposed methodology is found to be 72% more efficient than MCS, while the accuracy is decreased with an error of 24%.

Tracer methodology: an appropriate tool for assessing compliance with accreditation standards?

PubMed

Bouchard, Chantal; Jean, Olivier

2017-10-01

Tracer methodology has been used by Accreditation Canada since 2008 to collect evidence on the quality and safety of care and services, and to assess compliance with accreditation standards. Given the importance of this methodology in the accreditation program, the objective of this study is to assess the quality of the methodology and identify its strengths and weaknesses. A mixed quantitative and qualitative approach was adopted to evaluate consistency, appropriateness, effectiveness and stakeholder synergy in applying the methodology. An online questionnaire was sent to 468 Accreditation Canada surveyors. According to surveyors' perceptions, tracer methodology is an effective tool for collecting useful, credible and reliable information to assess compliance with Qmentum program standards and priority processes. The results show good coherence between methodology components (appropriateness of the priority processes evaluated, activities to evaluate a tracer, etc.). The main weaknesses are the time constraints faced by surveyors and management's lack of cooperation during the evaluation of tracers. The inadequate amount of time allowed for the methodology to be applied properly raises questions about the quality of the information obtained. This study paves the way for a future, more in-depth exploration of the identified weaknesses to help the accreditation organization make more targeted improvements to the methodology. Copyright © 2016 John Wiley & Sons, Ltd. Copyright © 2016 John Wiley & Sons, Ltd.

Probability techniques for reliability analysis of composite materials

NASA Technical Reports Server (NTRS)

Wetherhold, Robert C.; Ucci, Anthony M.

1994-01-01

Traditional design approaches for composite materials have employed deterministic criteria for failure analysis. New approaches are required to predict the reliability of composite structures since strengths and stresses may be random variables. This report will examine and compare methods used to evaluate the reliability of composite laminae. The two types of methods that will be evaluated are fast probability integration (FPI) methods and Monte Carlo methods. In these methods, reliability is formulated as the probability that an explicit function of random variables is less than a given constant. Using failure criteria developed for composite materials, a function of design variables can be generated which defines a 'failure surface' in probability space. A number of methods are available to evaluate the integration over the probability space bounded by this surface; this integration delivers the required reliability. The methods which will be evaluated are: the first order, second moment FPI methods; second order, second moment FPI methods; the simple Monte Carlo; and an advanced Monte Carlo technique which utilizes importance sampling. The methods are compared for accuracy, efficiency, and for the conservativism of the reliability estimation. The methodology involved in determining the sensitivity of the reliability estimate to the design variables (strength distributions) and importance factors is also presented.

76 FR 45804 - Agency Information Collection Request; 60-Day Public Comment Request

Federal Register 2010, 2011, 2012, 2013, 2014

2011-08-01

... an algorithm that enables reliable prediction of a certain event. A responder could submit the correct algorithm, but without the methodology, the evaluation process could not be adequately performed...

78 FR 13381 - Agency Information Collection Activities; Proposed Collection; Comments Requested: Reinstatement...

Federal Register 2010, 2011, 2012, 2013, 2014

2013-02-27

... of information, including the validity of the methodology and assumptions used; --Evaluate whether...- 52. In addition, 70 respondents of these respondents will be used for reliability testing averaging 1...

78 FR 70577 - Agency Information Collection Activities; Proposed Collection, Comments Requested, New Collection...

Federal Register 2010, 2011, 2012, 2013, 2014

2013-11-26

... Program the ability to conduct pretests which evaluate the validity and reliability of information... the proposed collection of information, including the validity of the methodology and assumptions used...

Are consumer surveys valuable as a service improvement tool in health services? A critical appraisal.

PubMed

Patwardhan, Anjali; Patwardhan, Prakash

2009-01-01

In the recent climate of consumerism and consumer focused care, health and social care needs to be more responsive than ever before. Consumer needs and preferences can be elicited with accepted validity and reliability only by strict methodological control, customerisation of the questionnaire and skilled interpretation. To construct, conduct, interpret and implement improved service provision, requires a trained work force and infrastructure. This article aims to appraise various aspects of consumer surveys and to assess their value as effective service improvement tools. The customer is the sole reason organisations exist. Consumer surveys are used worldwide as service and quality of care improvement tools by all types of service providers including health service providers. The article critically appraises the value of consumer surveys as service improvement tools in health services tool and its future applications. No one type of survey is the best or ideal. The key is the selection of the correct survey methodology, unique and customised for the particular type/aspect of care being evaluated. The method used should reflect the importance of the information required. Methodological rigor is essential for the effectiveness of consumer surveys as service improvement tools. Unfortunately so far there is no universal consensus on superiority of one particular methodology over another or any benefit of one specific methodology in a given situation. More training and some dedicated resource allocation is required to develop consumer surveys. More research is needed to develop specific survey methodology and evaluation techniques for improved validity and reliability of the surveys as service improvement tools. Measurement of consumer preferences/priorities, evaluation of services and key performance scores, is not easy. Consumer surveys seem impressive tools as they provide the customer a voice for change or modification. However, from a scientific point-of-view their credibility in service improvement in terms of reproducibility, reliability and validity, has remained debatable. This artcile is a critical appraisal of the value of consumer surveys as a service improvement tool in health services--a lesson which needs to be learnt.

Psychometric Evaluation of the D-Catch, an Instrument to Measure the Accuracy of Nursing Documentation.

PubMed

D'Agostino, Fabio; Barbaranelli, Claudio; Paans, Wolter; Belsito, Romina; Juarez Vela, Raul; Alvaro, Rosaria; Vellone, Ercole

2017-07-01

To evaluate the psychometric properties of the D-Catch instrument. A cross-sectional methodological study. Validity and reliability were estimated with confirmatory factor analysis (CFA) and internal consistency and inter-rater reliability, respectively. A sample of 250 nursing documentations was selected. CFA showed the adequacy of a 1-factor model (chronologically descriptive accuracy) with an outlier item (nursing diagnosis accuracy). Internal consistency and inter-rater reliability were adequate. The D-Catch is a valid and reliable instrument for measuring the accuracy of nursing documentation. Caution is needed when measuring diagnostic accuracy since only one item measures this dimension. The D-Catch can be used as an indicator of the accuracy of nursing documentation and the quality of nursing care. © 2015 NANDA International, Inc.

CMOS Active Pixel Sensor Technology and Reliability Characterization Methodology

NASA Technical Reports Server (NTRS)

Chen, Yuan; Guertin, Steven M.; Pain, Bedabrata; Kayaii, Sammy

2006-01-01

This paper describes the technology, design features and reliability characterization methodology of a CMOS Active Pixel Sensor. Both overall chip reliability and pixel reliability are projected for the imagers.

Comment on Hall et al. (2017), "How to Choose Between Measures of Tinnitus Loudness for Clinical Research? A Report on the Reliability and Validity of an Investigator-Administered Test and a Patient-Reported Measure Using Baseline Data Collected in a Phase IIa Drug Trial".

PubMed

Sabour, Siamak

2018-03-08

The purpose of this letter, in response to Hall, Mehta, and Fackrell (2017), is to provide important knowledge about methodology and statistical issues in assessing the reliability and validity of an audiologist-administered tinnitus loudness matching test and a patient-reported tinnitus loudness rating. The author uses reference textbooks and published articles regarding scientific assessment of the validity and reliability of a clinical test to discuss the statistical test and the methodological approach in assessing validity and reliability in clinical research. Depending on the type of the variable (qualitative or quantitative), well-known statistical tests can be applied to assess reliability and validity. The qualitative variables of sensitivity, specificity, positive predictive value, negative predictive value, false positive and false negative rates, likelihood ratio positive and likelihood ratio negative, as well as odds ratio (i.e., ratio of true to false results), are the most appropriate estimates to evaluate validity of a test compared to a gold standard. In the case of quantitative variables, depending on distribution of the variable, Pearson r or Spearman rho can be applied. Diagnostic accuracy (validity) and diagnostic precision (reliability or agreement) are two completely different methodological issues. Depending on the type of the variable (qualitative or quantitative), well-known statistical tests can be applied to assess validity.

Predicting the Reliability of Brittle Material Structures Subjected to Transient Proof Test and Service Loading

NASA Astrophysics Data System (ADS)

Nemeth, Noel N.; Jadaan, Osama M.; Palfi, Tamas; Baker, Eric H.

Brittle materials today are being used, or considered, for a wide variety of high tech applications that operate in harsh environments, including static and rotating turbine parts, thermal protection systems, dental prosthetics, fuel cells, oxygen transport membranes, radomes, and MEMS. Designing brittle material components to sustain repeated load without fracturing while using the minimum amount of material requires the use of a probabilistic design methodology. The NASA CARES/Life 1 (Ceramic Analysis and Reliability Evaluation of Structure/Life) code provides a general-purpose analysis tool that predicts the probability of failure of a ceramic component as a function of its time in service. This capability includes predicting the time-dependent failure probability of ceramic components against catastrophic rupture when subjected to transient thermomechanical loads (including cyclic loads). The developed methodology allows for changes in material response that can occur with temperature or time (i.e. changing fatigue and Weibull parameters with temperature or time). For this article an overview of the transient reliability methodology and how this methodology is extended to account for proof testing is described. The CARES/Life code has been modified to have the ability to interface with commercially available finite element analysis (FEA) codes executed for transient load histories. Examples are provided to demonstrate the features of the methodology as implemented in the CARES/Life program.

Establishing Inter- and Intrarater Reliability for High-Stakes Testing Using Simulation.

PubMed

Kardong-Edgren, Suzan; Oermann, Marilyn H; Rizzolo, Mary Anne; Odom-Maryon, Tamara

This article reports one method to develop a standardized training method to establish the inter- and intrarater reliability of a group of raters for high-stakes testing. Simulation is used increasingly for high-stakes testing, but without research into the development of inter- and intrarater reliability for raters. Eleven raters were trained using a standardized methodology. Raters scored 28 student videos over a six-week period. Raters then rescored all videos over a two-day period to establish both intra- and interrater reliability. One rater demonstrated poor intrarater reliability; a second rater failed all students. Kappa statistics improved from the moderate to substantial agreement range with the exclusion of the two outlier raters' scores. There may be faculty who, for different reasons, should not be included in high-stakes testing evaluations. All faculty are content experts, but not all are expert evaluators.

Assessment and Evaluation.

ERIC Educational Resources Information Center

Bachman, Lyle F.

1989-01-01

Applied linguistics and psychometrics have influenced language testing, providing additional tools for investigating factors affecting language test performance and assuring measurement reliability. An examination is presented of language testing, including the theoretical issues involved, the methodological advances, language test development,…

Extending the Implicit Association Test (IAT): Assessing Consumer Attitudes Based on Multi-Dimensional Implicit Associations

PubMed Central

Gattol, Valentin; Sääksjärvi, Maria; Carbon, Claus-Christian

2011-01-01

Background The authors present a procedural extension of the popular Implicit Association Test (IAT; [1]) that allows for indirect measurement of attitudes on multiple dimensions (e.g., safe–unsafe; young–old; innovative–conventional, etc.) rather than on a single evaluative dimension only (e.g., good–bad). Methodology/Principal Findings In two within-subjects studies, attitudes toward three automobile brands were measured on six attribute dimensions. Emphasis was placed on evaluating the methodological appropriateness of the new procedure, providing strong evidence for its reliability, validity, and sensitivity. Conclusions/Significance This new procedure yields detailed information on the multifaceted nature of brand associations that can add up to a more abstract overall attitude. Just as the IAT, its multi-dimensional extension/application (dubbed md-IAT) is suited for reliably measuring attitudes consumers may not be consciously aware of, able to express, or willing to share with the researcher [2], [3]. PMID:21246037

Improving Student Evaluation of Teaching: Determining Multiple Perspectives within a Course for Future Math Educators

ERIC Educational Resources Information Center

Ramlo, Susan

2017-01-01

Instructors in higher education are very familiar with the Likert scale Students' Evaluation of Teaching (SET) used to evaluate teaching. Researchers have raised concerns about biases affecting the results of SET surveys, as well as their validity and reliability and use in high-stakes decision making. Here, we demonstrate that Q methodology,…

On Information Retrieval (IR) Systems: Revisiting Their Development, Evaluation Methodologies, and Assumptions (SIGs LAN, ED).

ERIC Educational Resources Information Center

Stirling, Keith

2000-01-01

Describes a session on information retrieval systems that planned to discuss relevance measures with Web-based information retrieval; retrieval system performance and evaluation; probabilistic independence of index terms; vector-based models; metalanguages and digital objects; how users assess the reliability, timeliness and bias of information;…

Are validated outcome measures used in distal radial fractures truly valid?

PubMed Central

Nienhuis, R. W.; Bhandari, M.; Goslings, J. C.; Poolman, R. W.; Scholtes, V. A. B.

2016-01-01

Objectives Patient-reported outcome measures (PROMs) are often used to evaluate the outcome of treatment in patients with distal radial fractures. Which PROM to select is often based on assessment of measurement properties, such as validity and reliability. Measurement properties are assessed in clinimetric studies, and results are often reviewed without considering the methodological quality of these studies. Our aim was to systematically review the methodological quality of clinimetric studies that evaluated measurement properties of PROMs used in patients with distal radial fractures, and to make recommendations for the selection of PROMs based on the level of evidence of each individual measurement property. Methods A systematic literature search was performed in PubMed, EMbase, CINAHL and PsycINFO databases to identify relevant clinimetric studies. Two reviewers independently assessed the methodological quality of the studies on measurement properties, using the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist. Level of evidence (strong / moderate / limited / lacking) for each measurement property per PROM was determined by combining the methodological quality and the results of the different clinimetric studies. Results In all, 19 out of 1508 identified unique studies were included, in which 12 PROMs were rated. The Patient-rated wrist evaluation (PRWE) and the Disabilities of Arm, Shoulder and Hand questionnaire (DASH) were evaluated on most measurement properties. The evidence for the PRWE is moderate that its reliability, validity (content and hypothesis testing), and responsiveness are good. The evidence is limited that its internal consistency and cross-cultural validity are good, and its measurement error is acceptable. There is no evidence for its structural and criterion validity. The evidence for the DASH is moderate that its responsiveness is good. The evidence is limited that its reliability and the validity on hypothesis testing are good. There is no evidence for the other measurement properties. Conclusion According to this systematic review, there is, at best, moderate evidence that the responsiveness of the PRWE and DASH are good, as are the reliability and validity of the PRWE. We recommend these PROMs in clinical studies in patients with distal radial fractures; however, more clinimetric studies of higher methodological quality are needed to adequately determine the other measurement properties. Cite this article: Dr Y. V. Kleinlugtenbelt. Are validated outcome measures used in distal radial fractures truly valid?: A critical assessment using the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist. Bone Joint Res 2016;5:153–161. DOI: 10.1302/2046-3758.54.2000462. PMID:27132246

Evaluating information skills training in health libraries: a systematic review.

PubMed

Brettle, Alison

2007-12-01

Systematic reviews have shown that there is limited evidence to demonstrate that the information literacy training health librarians provide is effective in improving clinicians' information skills or has an impact on patient care. Studies lack measures which demonstrate validity and reliability in evaluating the impact of training. To determine what measures have been used; the extent to which they are valid and reliable; to provide guidance for health librarians who wish to evaluate the impact of their information skills training. Systematic review methodology involved searching seven databases, and personal files. Studies were included if they were about information skills training, used an objective measure to assess outcomes, and occurred in a health setting. Fifty-four studies were included in the review. Most outcome measures used in the studies were not tested for the key criteria of validity and reliability. Three tested for validity and reliability are described in more detail. Selecting an appropriate measure to evaluate the impact of training is a key factor in carrying out any evaluation. This systematic review provides guidance to health librarians by highlighting measures used in various circumstances, and those that demonstrate validity and reliability.

Instruments for Assessing Risk of Bias and Other Methodological Criteria of Published Animal Studies: A Systematic Review

PubMed Central

Krauth, David; Woodruff, Tracey J.

2013-01-01

Background: Results from animal toxicology studies are critical to evaluating the potential harm from exposure to environmental chemicals or the safety of drugs prior to human testing. However, there is significant debate about how to evaluate the methodology and potential biases of the animal studies. There is no agreed-upon approach, and a systematic evaluation of current best practices is lacking. Objective: We performed a systematic review to identify and evaluate instruments for assessing the risk of bias and/or other methodological criteria of animal studies. Method: We searched Medline (January 1966–November 2011) to identify all relevant articles. We extracted data on risk of bias criteria (e.g., randomization, blinding, allocation concealment) and other study design features included in each assessment instrument. Discussion: Thirty distinct instruments were identified, with the total number of assessed risk of bias, methodological, and/or reporting criteria ranging from 2 to 25. The most common criteria assessed were randomization (25/30, 83%), investigator blinding (23/30, 77%), and sample size calculation (18/30, 60%). In general, authors failed to empirically justify why these or other criteria were included. Nearly all (28/30, 93%) of the instruments have not been rigorously tested for validity or reliability. Conclusion: Our review highlights a number of risk of bias assessment criteria that have been empirically tested for animal research, including randomization, concealment of allocation, blinding, and accounting for all animals. In addition, there is a need for empirically testing additional methodological criteria and assessing the validity and reliability of a standard risk of bias assessment instrument. Citation: Krauth D, Woodruff TJ, Bero L. 2013. Instruments for assessing risk of bias and other methodological criteria of published animal studies: a systematic review. Environ Health Perspect 121:985–992 (2013); http://dx.doi.org/10.1289/ehp.1206389 PMID:23771496

Field reliability of competency and sanity opinions: A systematic review and meta-analysis.

PubMed

Guarnera, Lucy A; Murrie, Daniel C

2017-06-01

We know surprisingly little about the interrater reliability of forensic psychological opinions, even though courts and other authorities have long called for known error rates for scientific procedures admitted as courtroom testimony. This is particularly true for opinions produced during routine practice in the field, even for some of the most common types of forensic evaluations-evaluations of adjudicative competency and legal sanity. To address this gap, we used meta-analytic procedures and study space methodology to systematically review studies that examined the interrater reliability-particularly the field reliability-of competency and sanity opinions. Of 59 identified studies, 9 addressed the field reliability of competency opinions and 8 addressed the field reliability of sanity opinions. These studies presented a wide range of reliability estimates; pairwise percentage agreements ranged from 57% to 100% and kappas ranged from .28 to 1.0. Meta-analytic combinations of reliability estimates obtained by independent evaluators returned estimates of κ = .49 (95% CI: .40-.58) for competency opinions and κ = .41 (95% CI: .29-.53) for sanity opinions. This wide range of reliability estimates underscores the extent to which different evaluation contexts tend to produce different reliability rates. Unfortunately, our study space analysis illustrates that available field reliability studies typically provide little information about contextual variables crucial to understanding their findings. Given these concerns, we offer suggestions for improving research on the field reliability of competency and sanity opinions, as well as suggestions for improving reliability rates themselves. (PsycINFO Database Record (c) 2017 APA, all rights reserved).

Methodological variation in economic evaluations conducted in low- and middle-income countries: information for reference case development.

PubMed

Santatiwongchai, Benjarin; Chantarastapornchit, Varit; Wilkinson, Thomas; Thiboonboon, Kittiphong; Rattanavipapong, Waranya; Walker, Damian G; Chalkidou, Kalipso; Teerawattananon, Yot

2015-01-01

Information generated from economic evaluation is increasingly being used to inform health resource allocation decisions globally, including in low- and middle- income countries. However, a crucial consideration for users of the information at a policy level, e.g. funding agencies, is whether the studies are comparable, provide sufficient detail to inform policy decision making, and incorporate inputs from data sources that are reliable and relevant to the context. This review was conducted to inform a methodological standardisation workstream at the Bill and Melinda Gates Foundation (BMGF) and assesses BMGF-funded cost-per-DALY economic evaluations in four programme areas (malaria, tuberculosis, HIV/AIDS and vaccines) in terms of variation in methodology, use of evidence, and quality of reporting. The findings suggest that there is room for improvement in the three areas of assessment, and support the case for the introduction of a standardised methodology or reference case by the BMGF. The findings are also instructive for all institutions that fund economic evaluations in LMICs and who have a desire to improve the ability of economic evaluations to inform resource allocation decisions.

Interpretive Reliability of Six Computer-Based Test Interpretation Programs for the Minnesota Multiphasic Personality Inventory-2.

PubMed

Deskovitz, Mark A; Weed, Nathan C; McLaughlan, Joseph K; Williams, John E

2016-04-01

The reliability of six Minnesota Multiphasic Personality Inventory-Second edition (MMPI-2) computer-based test interpretation (CBTI) programs was evaluated across a set of 20 commonly appearing MMPI-2 profile codetypes in clinical settings. Evaluation of CBTI reliability comprised examination of (a) interrater reliability, the degree to which raters arrive at similar inferences based on the same CBTI profile and (b) interprogram reliability, the level of agreement across different CBTI systems. Profile inferences drawn by four raters were operationalized using q-sort methodology. Results revealed no significant differences overall with regard to interrater and interprogram reliability. Some specific CBTI/profile combinations (e.g., the CBTI by Automated Assessment Associates on a within normal limits profile) and specific profiles (e.g., the 4/9 profile displayed greater interprogram reliability than the 2/4 profile) were interpreted with variable consensus (α range = .21-.95). In practice, users should consider that certain MMPI-2 profiles are interpreted more or less consensually and that some CBTIs show variable reliability depending on the profile. © The Author(s) 2015.

Patient-reported outcome instruments that evaluate adherence behaviours in adults with asthma: A systematic review of measurement properties.

PubMed

Gagné, Myriam; Boulet, Louis-Philippe; Pérez, Norma; Moisan, Jocelyne

2018-04-30

To systematically identify the measurement properties of patient-reported outcome instruments (PROs) that evaluate adherence to inhaled maintenance medication in adults with asthma. We conducted a systematic review of six databases. Two reviewers independently included studies on the measurement properties of PROs that evaluated adherence in asthmatic participants aged ≥18 years. Based on the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN), the reviewers (1) extracted data on internal consistency, reliability, measurement error, content validity, structural validity, hypotheses testing, cross-cultural validity, criterion validity, and responsiveness; (2) assessed the methodological quality of the included studies; (3) assessed the quality of the measurement properties (positive or negative); and (4) summarised the level of evidence (limited, moderate, or strong). We screened 6,068 records and included 15 studies (14 PROs). No studies evaluated measurement error or responsiveness. Based on methodological and measurement property quality assessments, we found limited positive evidence of: (a) internal consistency of the Adherence Questionnaire, Refined Medication Adherence Reason Scale (MAR-Scale), Medication Adherence Report Scale for Asthma (MARS-A), and Test of the Adherence to Inhalers (TAI); (b) reliability of the TAI; and (c) structural validity of the Adherence Questionnaire, MAR-Scale, MARS-A, and TAI. We also found limited negative evidence of: (d) hypotheses testing of Adherence Questionnaire; (e) reliability of the MARS-A; and (f) criterion validity of the MARS-A and TAI. Our results highlighted the need to conduct further high-quality studies that will positively evaluate the reliability, validity, and responsiveness of the available PROs. This article is protected by copyright. All rights reserved.

Evaluation of fault-tolerant parallel-processor architectures over long space missions

NASA Technical Reports Server (NTRS)

Johnson, Sally C.

1989-01-01

The impact of a five year space mission environment on fault-tolerant parallel processor architectures is examined. The target application is a Strategic Defense Initiative (SDI) satellite requiring 256 parallel processors to provide the computation throughput. The reliability requirements are that the system still be operational after five years with .99 probability and that the probability of system failure during one-half hour of full operation be less than 10(-7). The fault tolerance features an architecture must possess to meet these reliability requirements are presented, many potential architectures are briefly evaluated, and one candidate architecture, the Charles Stark Draper Laboratory's Fault-Tolerant Parallel Processor (FTPP) is evaluated in detail. A methodology for designing a preliminary system configuration to meet the reliability and performance requirements of the mission is then presented and demonstrated by designing an FTPP configuration.

A Methodological Evaluation of an Environmental Education Survey: Is There a Technological Advantage

ERIC Educational Resources Information Center

Sharp, Ryan L.; Bradley, Michael J.; Maples, James N.

2017-01-01

Environmental education represents a conceivable way to counter the effects of youth's lack of exposure to the natural environment. However, the effectiveness of these programs is often not evaluated, and when they are, the methods for doing so are not consistent. Without proper and reliable methods of data collection, the results may be…

Effective Faculty Evaluation at the Teaching-Centered University: Building a Fair and Authentic Portfolio of Faculty Work

ERIC Educational Resources Information Center

Lakin, Amy L.

2016-01-01

Purpose: The purpose of this paper is to determine the most fair, authentic, and reliable elements to include in a portfolio of faculty work, specifically at teaching-centered institutions. Design/methodology/approach: This paper examines and evaluates relevant literature pertaining to faculty portfolios of work and recommends portfolio formats…

Appraising the quality of medical education research methods: the Medical Education Research Study Quality Instrument and the Newcastle-Ottawa Scale-Education.

PubMed

Cook, David A; Reed, Darcy A

2015-08-01

The Medical Education Research Study Quality Instrument (MERSQI) and the Newcastle-Ottawa Scale-Education (NOS-E) were developed to appraise methodological quality in medical education research. The study objective was to evaluate the interrater reliability, normative scores, and between-instrument correlation for these two instruments. In 2014, the authors searched PubMed and Google for articles using the MERSQI or NOS-E. They obtained or extracted data for interrater reliability-using the intraclass correlation coefficient (ICC)-and normative scores. They calculated between-scale correlation using Spearman rho. Each instrument contains items concerning sampling, controlling for confounders, and integrity of outcomes. Interrater reliability for overall scores ranged from 0.68 to 0.95. Interrater reliability was "substantial" or better (ICC > 0.60) for nearly all domain-specific items on both instruments. Most instances of low interrater reliability were associated with restriction of range, and raw agreement was usually good. Across 26 studies evaluating published research, the median overall MERSQI score was 11.3 (range 8.9-15.1, of possible 18). Across six studies, the median overall NOS-E score was 3.22 (range 2.08-3.82, of possible 6). Overall MERSQI and NOS-E scores correlated reasonably well (rho 0.49-0.72). The MERSQI and NOS-E are useful, reliable, complementary tools for appraising methodological quality of medical education research. Interpretation and use of their scores should focus on item-specific codes rather than overall scores. Normative scores should be used for relative rather than absolute judgments because different research questions require different study designs.

Screening for oropharyngeal dysphagia in older adults: A systematic review of self-reported questionnaires.

PubMed

Magalhães Junior, Hipólito V; Pernambuco, Leandro de Araújo; Lima, Kenio C; Ferreira, Maria Angela F

2018-04-03

Oropharyngeal dysphagia is a swallowing disorder with signs and symptoms which may be present in older adults, but they are rarely noticed as a health concern by older people. The earliest possible identification of this clinical condition is needed by self-reported population-based screening questionnaire, which are valid and reliable for preventing risks to nutritional status, increased morbidity and mortality. The aim of this systematic review was to identify self-reported screening questionnaires for oropharyngeal dysphagia in older adults to evaluate their methodological quality for population-based studies. An extensive search of electronic databases (PubMed (MEDLINE), Ovid MEDLINE(R), Scopus, Cochrane Library, CINAHL, Web of Science (WOS), PsycINFO (APA), Lilacs and Scielo) was conducted in the period from April to May 2017 using previously established search strategies by the two evaluators. The methodological quality and the psychometric properties of the included studies were evaluated by the COSMIN (Consensus based Standards for the selection of health Measurement Instruments) checklist and the quality criteria of Terwee and colleagues, respectively. The analysed information was extracted from three articles which had conducted studies on the prevalence of oropharyngeal dysphagia by self-reported screening questionnaires, showing poor methodological quality and flaws in the methodological description to demonstrate its psychometric properties. This study did not find any self-reported screening questionnaires for oropharyngeal dysphagia with suitable methodological quality and appropriate evidence in its psychometric properties for elders. Therefore, the self-reported questionnaires within the diagnostic proposal require greater details in its process for obtaining valid and reliable evidence. © 2018 John Wiley & Sons A/S and The Gerodontology Association. Published by John Wiley & Sons Ltd.

The reliability of physical examination tests for the clinical assessment of scapular dyskinesis in subjects with shoulder complaints: A systematic review.

PubMed

Lange, Toni; Struyf, Filip; Schmitt, Jochen; Lützner, Jörg; Kopkow, Christian

2017-07-01

Systematic review. The aim of this systematic review was to summarize and evaluate intra- and interrater reliability research of physical examination tests used for the assessment of scapular dyskinesis. Scapular dyskinesis, defined as alteration of normal scapular kinematics, is described as a non-specific response to different shoulder pathologies. A systematic literature search was conducted in MEDLINE, EMBASE, AMED and PEDro until March 20th, 2015. Methodological quality was assessed with the Quality Appraisal of Reliability Studies (QAREL) by two independent reviewers. The search strategy revealed 3259 articles, of which 15 met the inclusion criteria. These studies evaluated the reliability of 41 test and test variations used for the assessment of scapular dyskinesis. This review identified a lack of high-quality studies evaluating intra- as well as interrater reliability of tests used for the assessment of scapular dyskinesis. In addition, reliability measures differed between included studies hindering proper cross-study comparisons. The effect of manual correction of the scapula on shoulder symptoms was evaluated in only one study, which is striking, since symptom alteration tests are used in routine care to guide further treatment. Thus, there is a strong need for further research in this area. Diagnosis, level 3a. Copyright © 2016. Published by Elsevier Ltd.

Integrated HTA-FMEA/FMECA methodology for the evaluation of robotic system in urology and general surgery.

PubMed

Frosini, Francesco; Miniati, Roberto; Grillone, Saverio; Dori, Fabrizio; Gentili, Guido Biffi; Belardinelli, Andrea

2016-11-14

The following study proposes and tests an integrated methodology involving Health Technology Assessment (HTA) and Failure Modes, Effects and Criticality Analysis (FMECA) for the assessment of specific aspects related to robotic surgery involving safety, process and technology. The integrated methodology consists of the application of specific techniques coming from the HTA joined to the aid of the most typical models from reliability engineering such as FMEA/FMECA. The study has also included in-site data collection and interviews to medical personnel. The total number of robotic procedures included in the analysis was 44: 28 for urology and 16 for general surgery. The main outcomes refer to the comparative evaluation between robotic, laparoscopic and open surgery. Risk analysis and mitigation interventions come from FMECA application. The small sample size available for the study represents an important bias, especially for the clinical outcomes reliability. Despite this, the study seems to confirm the better trend for robotics' surgical times with comparison to the open technique as well as confirming the robotics' clinical benefits in urology. More complex situation is observed for general surgery, where robotics' clinical benefits directly measured are the lowest blood transfusion rate.

Analysis and Evaluation of Processes and Equipment in Tasks 2 and 4 of the Low-cost Solar Array Project

NASA Technical Reports Server (NTRS)

Wolf, M.

1979-01-01

To facilitate the task of objectively comparing competing process options, a methodology was needed for the quantitative evaluation of their relative cost effectiveness. Such a methodology was developed and is described, together with three examples for its application. The criterion for the evaluation is the cost of the energy produced by the system. The method permits the evaluation of competing design options for subsystems, based on the differences in cost and efficiency of the subsystems, assuming comparable reliability and service life, or of competing manufacturing process options for such subsystems, which include solar cells or modules. This process option analysis is based on differences in cost, yield, and conversion efficiency contribution of the process steps considered.

Traumatic brain injury: methodological approaches to estimate health and economic outcomes.

PubMed

Lu, Juan; Roe, Cecilie; Aas, Eline; Lapane, Kate L; Niemeier, Janet; Arango-Lasprilla, Juan Carlos; Andelic, Nada

2013-12-01

The effort to standardize the methodology and adherence to recommended principles for all economic evaluations has been emphasized in medical literature. The objective of this review is to examine whether economic evaluations in traumatic brain injury (TBI) research have been compliant with existing guidelines. Medline search was performed between January 1, 1995 and August 11, 2012. All original TBI-related full economic evaluations were included in the study. Two authors independently rated each study's methodology and data presentation to determine compliance to the 10 methodological principles recommended by Blackmore et al. Descriptive analysis was used to summarize the data. Inter-rater reliability was assessed with Kappa statistics. A total of 28 studies met the inclusion criteria. Eighteen of these studies described cost-effectiveness, seven cost-benefit, and three cost-utility analyses. The results showed a rapid growth in the number of published articles on the economic impact of TBI since 2000 and an improvement in their methodological quality. However, overall compliance with recommended methodological principles of TBI-related economic evaluation has been deficient. On average, about six of the 10 criteria were followed in these publications, and only two articles met all 10 criteria. These findings call for an increased awareness of the methodological standards that should be followed by investigators both in performance of economic evaluation and in reviews of evaluation reports prior to publication. The results also suggest that all economic evaluations should be made by following the guidelines within a conceptual framework, in order to facilitate evidence-based practices in the field of TBI.

Design Optimization Method for Composite Components Based on Moment Reliability-Sensitivity Criteria

NASA Astrophysics Data System (ADS)

Sun, Zhigang; Wang, Changxi; Niu, Xuming; Song, Yingdong

2017-08-01

In this paper, a Reliability-Sensitivity Based Design Optimization (RSBDO) methodology for the design of the ceramic matrix composites (CMCs) components has been proposed. A practical and efficient method for reliability analysis and sensitivity analysis of complex components with arbitrary distribution parameters are investigated by using the perturbation method, the respond surface method, the Edgeworth series and the sensitivity analysis approach. The RSBDO methodology is then established by incorporating sensitivity calculation model into RBDO methodology. Finally, the proposed RSBDO methodology is applied to the design of the CMCs components. By comparing with Monte Carlo simulation, the numerical results demonstrate that the proposed methodology provides an accurate, convergent and computationally efficient method for reliability-analysis based finite element modeling engineering practice.

COMPONENTS IDENTIFIED IN ENERGY-RELATED WASTES AND EFFLUENTS

EPA Science Inventory

A state-of-the-art review of the characterization of solid wastes and aqueous effluents generated by energy-related processes was conducted. The reliability of these data was evaluated according to preselected criteria or sample source, sampling and analytical methodology, and da...

Reliability Issues and Solutions in Flexible Electronics Under Mechanical Fatigue

NASA Astrophysics Data System (ADS)

Yi, Seol-Min; Choi, In-Suk; Kim, Byoung-Joon; Joo, Young-Chang

2018-07-01

Flexible devices are of significant interest due to their potential expansion of the application of smart devices into various fields, such as energy harvesting, biological applications and consumer electronics. Due to the mechanically dynamic operations of flexible electronics, their mechanical reliability must be thoroughly investigated to understand their failure mechanisms and lifetimes. Reliability issue caused by bending fatigue, one of the typical operational limitations of flexible electronics, has been studied using various test methodologies; however, electromechanical evaluations which are essential to assess the reliability of electronic devices for flexible applications had not been investigated because the testing method was not established. By employing the in situ bending fatigue test, we has studied the failure mechanism for various conditions and parameters, such as bending strain, fatigue area, film thickness, and lateral dimensions. Moreover, various methods for improving the bending reliability have been developed based on the failure mechanism. Nanostructures such as holes, pores, wires and composites of nanoparticles and nanotubes have been suggested for better reliability. Flexible devices were also investigated to find the potential failures initiated by complex structures under bending fatigue strain. In this review, the recent advances in test methodology, mechanism studies, and practical applications are introduced. Additionally, perspectives including the future advance to stretchable electronics are discussed based on the current achievements in research.

Reliability Issues and Solutions in Flexible Electronics Under Mechanical Fatigue

NASA Astrophysics Data System (ADS)

Yi, Seol-Min; Choi, In-Suk; Kim, Byoung-Joon; Joo, Young-Chang

2018-03-01

Flexible devices are of significant interest due to their potential expansion of the application of smart devices into various fields, such as energy harvesting, biological applications and consumer electronics. Due to the mechanically dynamic operations of flexible electronics, their mechanical reliability must be thoroughly investigated to understand their failure mechanisms and lifetimes. Reliability issue caused by bending fatigue, one of the typical operational limitations of flexible electronics, has been studied using various test methodologies; however, electromechanical evaluations which are essential to assess the reliability of electronic devices for flexible applications had not been investigated because the testing method was not established. By employing the in situ bending fatigue test, we has studied the failure mechanism for various conditions and parameters, such as bending strain, fatigue area, film thickness, and lateral dimensions. Moreover, various methods for improving the bending reliability have been developed based on the failure mechanism. Nanostructures such as holes, pores, wires and composites of nanoparticles and nanotubes have been suggested for better reliability. Flexible devices were also investigated to find the potential failures initiated by complex structures under bending fatigue strain. In this review, the recent advances in test methodology, mechanism studies, and practical applications are introduced. Additionally, perspectives including the future advance to stretchable electronics are discussed based on the current achievements in research.

Moving from theory to practice: A participatory social network mapping approach to address unmet need for family planning in Benin.

PubMed

Igras, Susan; Diakité, Mariam; Lundgren, Rebecka

2017-07-01

In West Africa, social factors influence whether couples with unmet need for family planning act on birth-spacing desires. Tékponon Jikuagou is testing a social network-based intervention to reduce social barriers by diffusing new ideas. Individuals and groups judged socially influential by their communities provide entrée to networks. A participatory social network mapping methodology was designed to identify these diffusion actors. Analysis of monitoring data, in-depth interviews, and evaluation reports assessed the methodology's acceptability to communities and staff and whether it produced valid, reliable data to identify influential individuals and groups who diffuse new ideas through their networks. Results indicated the methodology's acceptability. Communities were actively and equitably engaged. Staff appreciated its ability to yield timely, actionable information. The mapping methodology also provided valid and reliable information by enabling communities to identify highly connected and influential network actors. Consistent with social network theory, this methodology resulted in the selection of informal groups and individuals in both informal and formal positions. In-depth interview data suggest these actors were diffusing new ideas, further confirming their influence/connectivity. The participatory methodology generated insider knowledge of who has social influence, challenging commonly held assumptions. Collecting and displaying information fostered staff and community learning, laying groundwork for social change.

Comparison of Methodologies of Activation Barrier Measurements for Reactions with Deactivation

DOE Office of Scientific and Technical Information (OSTI.GOV)

Xie, Zhenhua; Yan, Binhang; Zhang, Li

In this work, methodologies of activation barrier measurements for reactions with deactivation were theoretically analyzed. Reforming of ethane with CO 2 was introduced as an example for reactions with deactivation to experimentally evaluate these methodologies. Both the theoretical and experimental results showed that due to catalyst deactivation, the conventional method would inevitably lead to a much lower activation barrier, compared to the intrinsic value, even though heat and mass transport limitations were excluded. In this work, an optimal method was identified in order to provide a reliable and efficient activation barrier measurement for reactions with deactivation.

Comparison of Methodologies of Activation Barrier Measurements for Reactions with Deactivation

DOE PAGES

Xie, Zhenhua; Yan, Binhang; Zhang, Li; ...

2017-01-25

In this work, methodologies of activation barrier measurements for reactions with deactivation were theoretically analyzed. Reforming of ethane with CO 2 was introduced as an example for reactions with deactivation to experimentally evaluate these methodologies. Both the theoretical and experimental results showed that due to catalyst deactivation, the conventional method would inevitably lead to a much lower activation barrier, compared to the intrinsic value, even though heat and mass transport limitations were excluded. In this work, an optimal method was identified in order to provide a reliable and efficient activation barrier measurement for reactions with deactivation.

Probabilistic design of fibre concrete structures

NASA Astrophysics Data System (ADS)

Pukl, R.; Novák, D.; Sajdlová, T.; Lehký, D.; Červenka, J.; Červenka, V.

2017-09-01

Advanced computer simulation is recently well-established methodology for evaluation of resistance of concrete engineering structures. The nonlinear finite element analysis enables to realistically predict structural damage, peak load, failure, post-peak response, development of cracks in concrete, yielding of reinforcement, concrete crushing or shear failure. The nonlinear material models can cover various types of concrete and reinforced concrete: ordinary concrete, plain or reinforced, without or with prestressing, fibre concrete, (ultra) high performance concrete, lightweight concrete, etc. Advanced material models taking into account fibre concrete properties such as shape of tensile softening branch, high toughness and ductility are described in the paper. Since the variability of the fibre concrete material properties is rather high, the probabilistic analysis seems to be the most appropriate format for structural design and evaluation of structural performance, reliability and safety. The presented combination of the nonlinear analysis with advanced probabilistic methods allows evaluation of structural safety characterized by failure probability or by reliability index respectively. Authors offer a methodology and computer tools for realistic safety assessment of concrete structures; the utilized approach is based on randomization of the nonlinear finite element analysis of the structural model. Uncertainty of the material properties or their randomness obtained from material tests are accounted in the random distribution. Furthermore, degradation of the reinforced concrete materials such as carbonation of concrete, corrosion of reinforcement, etc. can be accounted in order to analyze life-cycle structural performance and to enable prediction of the structural reliability and safety in time development. The results can serve as a rational basis for design of fibre concrete engineering structures based on advanced nonlinear computer analysis. The presented methodology is illustrated on results from two probabilistic studies with different types of concrete structures related to practical applications and made from various materials (with the parameters obtained from real material tests).

Use of Model-Based Design Methods for Enhancing Resiliency Analysis of Unmanned Aerial Vehicles

NASA Astrophysics Data System (ADS)

Knox, Lenora A.

The most common traditional non-functional requirement analysis is reliability. With systems becoming more complex, networked, and adaptive to environmental uncertainties, system resiliency has recently become the non-functional requirement analysis of choice. Analysis of system resiliency has challenges; which include, defining resilience for domain areas, identifying resilience metrics, determining resilience modeling strategies, and understanding how to best integrate the concepts of risk and reliability into resiliency. Formal methods that integrate all of these concepts do not currently exist in specific domain areas. Leveraging RAMSoS, a model-based reliability analysis methodology for Systems of Systems (SoS), we propose an extension that accounts for resiliency analysis through evaluation of mission performance, risk, and cost using multi-criteria decision-making (MCDM) modeling and design trade study variability modeling evaluation techniques. This proposed methodology, coined RAMSoS-RESIL, is applied to a case study in the multi-agent unmanned aerial vehicle (UAV) domain to investigate the potential benefits of a mission architecture where functionality to complete a mission is disseminated across multiple UAVs (distributed) opposed to being contained in a single UAV (monolithic). The case study based research demonstrates proof of concept for the proposed model-based technique and provides sufficient preliminary evidence to conclude which architectural design (distributed vs. monolithic) is most resilient based on insight into mission resilience performance, risk, and cost in addition to the traditional analysis of reliability.

Reliability and Validity of the Turkish Version of the Job Performance Scale Instrument.

PubMed

Harmanci Seren, Arzu Kader; Tuna, Rujnan; Eskin Bacaksiz, Feride

2018-02-01

Objective measurement of the job performance of nursing staff using valid and reliable instruments is important in the evaluation of healthcare quality. A current, valid, and reliable instrument that specifically measures the performance of nurses is required for this purpose. The aim of this study was to determine the validity and reliability of the Turkish version of the Job Performance Instrument. This study used a methodological design and a sample of 240 nurses working at different units in four hospitals in Istanbul, Turkey. A descriptive data form, the Job Performance Scale, and the Employee Performance Scale were used to collect data. Data were analyzed using IBM SPSS Statistics Version 21.0 and LISREL Version 8.51. On the basis of the data analysis, the instrument was revised. Some items were deleted, and subscales were combined. The Turkish version of the Job Performance Instrument was determined to be valid and reliable to measure the performance of nurses. The instrument is suitable for evaluating current nursing roles.

Software reliability studies

NASA Technical Reports Server (NTRS)

Hoppa, Mary Ann; Wilson, Larry W.

1994-01-01

There are many software reliability models which try to predict future performance of software based on data generated by the debugging process. Our research has shown that by improving the quality of the data one can greatly improve the predictions. We are working on methodologies which control some of the randomness inherent in the standard data generation processes in order to improve the accuracy of predictions. Our contribution is twofold in that we describe an experimental methodology using a data structure called the debugging graph and apply this methodology to assess the robustness of existing models. The debugging graph is used to analyze the effects of various fault recovery orders on the predictive accuracy of several well-known software reliability algorithms. We found that, along a particular debugging path in the graph, the predictive performance of different models can vary greatly. Similarly, just because a model 'fits' a given path's data well does not guarantee that the model would perform well on a different path. Further we observed bug interactions and noted their potential effects on the predictive process. We saw that not only do different faults fail at different rates, but that those rates can be affected by the particular debugging stage at which the rates are evaluated. Based on our experiment, we conjecture that the accuracy of a reliability prediction is affected by the fault recovery order as well as by fault interaction.

Methodological Variation in Economic Evaluations Conducted in Low- and Middle-Income Countries: Information for Reference Case Development

PubMed Central

2015-01-01

Information generated from economic evaluation is increasingly being used to inform health resource allocation decisions globally, including in low- and middle- income countries. However, a crucial consideration for users of the information at a policy level, e.g. funding agencies, is whether the studies are comparable, provide sufficient detail to inform policy decision making, and incorporate inputs from data sources that are reliable and relevant to the context. This review was conducted to inform a methodological standardisation workstream at the Bill and Melinda Gates Foundation (BMGF) and assesses BMGF-funded cost-per-DALY economic evaluations in four programme areas (malaria, tuberculosis, HIV/AIDS and vaccines) in terms of variation in methodology, use of evidence, and quality of reporting. The findings suggest that there is room for improvement in the three areas of assessment, and support the case for the introduction of a standardised methodology or reference case by the BMGF. The findings are also instructive for all institutions that fund economic evaluations in LMICs and who have a desire to improve the ability of economic evaluations to inform resource allocation decisions. PMID:25950443

DRUG EVALUATION AND DECISION MAKING IN CATALONIA: DEVELOPMENT AND VALIDATION OF A METHODOLOGICAL FRAMEWORK BASED ON MULTI-CRITERIA DECISION ANALYSIS (MCDA) FOR ORPHAN DRUGS.

PubMed

Gilabert-Perramon, Antoni; Torrent-Farnell, Josep; Catalan, Arancha; Prat, Alba; Fontanet, Manel; Puig-Peiró, Ruth; Merino-Montero, Sandra; Khoury, Hanane; Goetghebeur, Mireille M; Badia, Xavier

2017-01-01

The aim of this study was to adapt and assess the value of a Multi-Criteria Decision Analysis (MCDA) framework (EVIDEM) for the evaluation of Orphan drugs in Catalonia (Catalan Health Service). The standard evaluation and decision-making procedures of CatSalut were compared with the EVIDEM methodology and contents. The EVIDEM framework was adapted to the Catalan context, focusing on the evaluation of Orphan drugs (PASFTAC program), during a Workshop with sixteen PASFTAC members. The criteria weighting was done using two different techniques (nonhierarchical and hierarchical). Reliability was assessed by re-test. The EVIDEM framework and methodology was found useful and feasible for Orphan drugs evaluation and decision making in Catalonia. All the criteria considered for the development of the CatSalut Technical Reports and decision making were considered in the framework. Nevertheless, the framework could improve the reporting of some of these criteria (i.e., "unmet needs" or "nonmedical costs"). Some Contextual criteria were removed (i.e., "Mandate and scope of healthcare system", "Environmental impact") or adapted ("population priorities and access") for CatSalut purposes. Independently of the weighting technique considered, the most important evaluation criteria identified for orphan drugs were: "disease severity", "unmet needs" and "comparative effectiveness", while the "size of the population" had the lowest relevance for decision making. Test-retest analysis showed weight consistency among techniques, supporting reliability overtime. MCDA (EVIDEM framework) could be a useful tool to complement the current evaluation methods of CatSalut, contributing to standardization and pragmatism, providing a method to tackle ethical dilemmas and facilitating discussions related to decision making.

Methodology for Physics and Engineering of Reliable Products

NASA Technical Reports Server (NTRS)

Cornford, Steven L.; Gibbel, Mark

1996-01-01

Physics of failure approaches have gained wide spread acceptance within the electronic reliability community. These methodologies involve identifying root cause failure mechanisms, developing associated models, and utilizing these models to inprove time to market, lower development and build costs and higher reliability. The methodology outlined herein sets forth a process, based on integration of both physics and engineering principles, for achieving the same goals.

Factors Influencing the Reliability of the Glasgow Coma Scale: A Systematic Review.

PubMed

Reith, Florence Cm; Synnot, Anneliese; van den Brande, Ruben; Gruen, Russell L; Maas, Andrew Ir

2017-06-01

The Glasgow Coma Scale (GCS) characterizes patients with diminished consciousness. In a recent systematic review, we found overall adequate reliability across different clinical settings, but reliability estimates varied considerably between studies, and methodological quality of studies was overall poor. Identifying and understanding factors that can affect its reliability is important, in order to promote high standards for clinical use of the GCS. The aim of this systematic review was to identify factors that influence reliability and to provide an evidence base for promoting consistent and reliable application of the GCS. A comprehensive literature search was undertaken in MEDLINE, EMBASE, and CINAHL from 1974 to July 2016. Studies assessing the reliability of the GCS in adults or describing any factor that influences reliability were included. Two reviewers independently screened citations, selected full texts, and undertook data extraction and critical appraisal. Methodological quality of studies was evaluated with the consensus-based standards for the selection of health measurement instruments checklist. Data were synthesized narratively and presented in tables. Forty-one studies were included for analysis. Factors identified that may influence reliability are education and training, the level of consciousness, and type of stimuli used. Conflicting results were found for experience of the observer, the pathology causing the reduced consciousness, and intubation/sedation. No clear influence was found for the professional background of observers. Reliability of the GCS is influenced by multiple factors and as such is context dependent. This review points to the potential for improvement from training and education and standardization of assessment methods, for which recommendations are presented. Copyright © 2017 by the Congress of Neurological Surgeons.

Reliability of Radioisotope Stirling Convertor Linear Alternator

NASA Technical Reports Server (NTRS)

Shah, Ashwin; Korovaichuk, Igor; Geng, Steven M.; Schreiber, Jeffrey G.

2006-01-01

Onboard radioisotope power systems being developed and planned for NASA s deep-space missions would require reliable design lifetimes of up to 14 years. Critical components and materials of Stirling convertors have been undergoing extensive testing and evaluation in support of a reliable performance for the specified life span. Of significant importance to the successful development of the Stirling convertor is the design of a lightweight and highly efficient linear alternator. Alternator performance could vary due to small deviations in the permanent magnet properties, operating temperature, and component geometries. Durability prediction and reliability of the alternator may be affected by these deviations from nominal design conditions. Therefore, it is important to evaluate the effect of these uncertainties in predicting the reliability of the linear alternator performance. This paper presents a study in which a reliability-based methodology is used to assess alternator performance. The response surface characterizing the induced open-circuit voltage performance is constructed using 3-D finite element magnetic analysis. Fast probability integration method is used to determine the probability of the desired performance and its sensitivity to the alternator design parameters.

Disturbance characteristics of half-selected cells in a cross-point resistive switching memory array

NASA Astrophysics Data System (ADS)

Chen, Zhe; Li, Haitong; Chen, Hong-Yu; Chen, Bing; Liu, Rui; Huang, Peng; Zhang, Feifei; Jiang, Zizhen; Ye, Hongfei; Gao, Bin; Liu, Lifeng; Liu, Xiaoyan; Kang, Jinfeng; Wong, H.-S. Philip; Yu, Shimeng

2016-05-01

Disturbance characteristics of cross-point resistive random access memory (RRAM) arrays are comprehensively studied in this paper. An analytical model is developed to quantify the number of pulses (#Pulse) the cell can bear before disturbance occurs under various sub-switching voltage stresses based on physical understanding. An evaluation methodology is proposed to assess the disturb behavior of half-selected (HS) cells in cross-point RRAM arrays by combining the analytical model and SPICE simulation. The characteristics of cross-point RRAM arrays such as energy consumption, reliable operating cycles and total error bits are evaluated by the methodology. A possible solution to mitigate disturbance is proposed.

Development Of Methodologies Using PhabrOmeter For Fabric Drape Evaluation

NASA Astrophysics Data System (ADS)

Lin, Chengwei

Evaluation of fabric drape is important for textile industry as it reveals the aesthetic and functionality of the cloth and apparel. Although many fabric drape measuring methods have been developed for several decades, they are falling behind the need for fast product development by the industry. To meet the requirement of industries, it is necessary to develop an effective and reliable method to evaluate fabric drape. The purpose of the present study is to determine if PhabrOmeter can be applied to fabric drape evaluation. PhabrOmeter is a fabric sensory performance evaluating instrument which is developed to provide fast and reliable quality testing results. This study was sought to determine the relationship between fabric drape and other fabric attributes. In addition, a series of conventional methods including AATCC standards, ASTM standards and ISO standards were used to characterize the fabric samples. All the data were compared and analyzed with linear correlation method. The results indicate that PhabrOmeter is reliable and effective instrument for fabric drape evaluation. Besides, some effects including fabric structure, testing directions were considered to examine their impact on fabric drape.

Enhancement of Text Representations Using Related Document Titles.

ERIC Educational Resources Information Center

Salton, G.; Zhang, Y.

1986-01-01

Briefly reviews various methodologies for constructing enhanced document representations, discusses their general lack of usefulness, and describes a method of document indexing which uses title words taken from bibliographically related items. Evaluation of this process indicates that it is not sufficiently reliable to warrant incorporation into…

Sexual health education interventions for young people: a methodological review.

PubMed Central

Oakley, A.; Fullerton, D.; Holland, J.; Arnold, S.; France-Dawson, M.; Kelley, P.; McGrellis, S.

1995-01-01

OBJECTIVES--To locate reports of sexual health education interventions for young people, assess the methodological quality of evaluations, identify the subgroup with a methodologically sound design, and assess the evidence with respect to the effectiveness of different approaches to promoting young people's sexual health. DESIGN--Survey of reports in English by means of electronic databases and hand searches for relevant studies conducted in the developed world since 1982. Papers were reviewed for eight methodological qualities. The evidence on effectiveness generated by studies meeting four core criteria was assessed. Judgments on effectiveness by reviewers and authors were compared. PAPERS--270 papers reporting sexual health interventions. MAIN OUTCOME MEASURE--The methodological quality of evaluations. RESULTS--73 reports of evaluations of sexual health interventions examining the effectiveness of these interventions in changing knowledge, attitudes, or behavioural outcomes were identified, of which 65 were separate outcome evaluations. Of these studies, 45 (69%) lacked random control groups, 44 (68%) failed to present preintervention and 38 (59%) postintervention data, and 26 (40%) omitted to discuss the relevance of loss of data caused by drop outs. Only 12 (18%) of the 65 outcome evaluations were judged to be methodologically sound. Academic reviewers were more likely than authors to judge studies as unclear because of design faults. Only two of the sound evaluations recorded interventions which were effective in showing an impact on young people's sexual behaviour. CONCLUSIONS--The design of evaluations in sexual health intervention needs to be improved so that reliable evidence of the effectiveness of different approaches to promoting young people's sexual health may be generated. PMID:7833754

Reliability modelling and analysis of thermal MEMS

NASA Astrophysics Data System (ADS)

Muratet, Sylvaine; Lavu, Srikanth; Fourniols, Jean-Yves; Bell, George; Desmulliez, Marc P. Y.

2006-04-01

This paper presents a MEMS reliability study methodology based on the novel concept of 'virtual prototyping'. This methodology can be used for the development of reliable sensors or actuators and also to characterize their behaviour in specific use conditions and applications. The methodology is demonstrated on the U-shaped micro electro thermal actuator used as test vehicle. To demonstrate this approach, a 'virtual prototype' has been developed with the modeling tools MatLab and VHDL-AMS. A best practice FMEA (Failure Mode and Effect Analysis) is applied on the thermal MEMS to investigate and assess the failure mechanisms. Reliability study is performed by injecting the identified defaults into the 'virtual prototype'. The reliability characterization methodology predicts the evolution of the behavior of these MEMS as a function of the number of cycles of operation and specific operational conditions.

Probabilistic structural analysis methods of hot engine structures

NASA Technical Reports Server (NTRS)

Chamis, C. C.; Hopkins, D. A.

1989-01-01

Development of probabilistic structural analysis methods for hot engine structures at Lewis Research Center is presented. Three elements of the research program are: (1) composite load spectra methodology; (2) probabilistic structural analysis methodology; and (3) probabilistic structural analysis application. Recent progress includes: (1) quantification of the effects of uncertainties for several variables on high pressure fuel turbopump (HPFT) turbine blade temperature, pressure, and torque of the space shuttle main engine (SSME); (2) the evaluation of the cumulative distribution function for various structural response variables based on assumed uncertainties in primitive structural variables; and (3) evaluation of the failure probability. Collectively, the results demonstrate that the structural durability of hot engine structural components can be effectively evaluated in a formal probabilistic/reliability framework.

Supervisor/Peer Involvement in Evaluation Transfer of Training Process and Results Reliability: A Research in an Italian Public Body

ERIC Educational Resources Information Center

Capaldo, Guido; Depolo, Marco; Rippa, Pierluigi; Schiattone, Domenico

2017-01-01

Purpose: The aim of this paper is to present a study performed in conjunction with a branch of the Italian Public Italian Administration, the ISSP (Istituto Superiore di Studi Penitenziari--the Higher Institute of Penitentiary Studies). The study aimed to develop a Transfer of Training (ToT) evaluation methodology that would be both scientifically…

Development of Reliable and Validated Tools to Evaluate Technical Resuscitation Skills in a Pediatric Simulation Setting: Resuscitation and Emergency Simulation Checklist for Assessment in Pediatrics.

PubMed

Faudeux, Camille; Tran, Antoine; Dupont, Audrey; Desmontils, Jonathan; Montaudié, Isabelle; Bréaud, Jean; Braun, Marc; Fournier, Jean-Paul; Bérard, Etienne; Berlengi, Noémie; Schweitzer, Cyril; Haas, Hervé; Caci, Hervé; Gatin, Amélie; Giovannini-Chami, Lisa

2017-09-01

To develop a reliable and validated tool to evaluate technical resuscitation skills in a pediatric simulation setting. Four Resuscitation and Emergency Simulation Checklist for Assessment in Pediatrics (RESCAPE) evaluation tools were created, following international guidelines: intraosseous needle insertion, bag mask ventilation, endotracheal intubation, and cardiac massage. We applied a modified Delphi methodology evaluation to binary rating items. Reliability was assessed comparing the ratings of 2 observers (1 in real time and 1 after a video-recorded review). The tools were assessed for content, construct, and criterion validity, and for sensitivity to change. Inter-rater reliability, evaluated with Cohen kappa coefficients, was perfect or near-perfect (>0.8) for 92.5% of items and each Cronbach alpha coefficient was ≥0.91. Principal component analyses showed that all 4 tools were unidimensional. Significant increases in median scores with increasing levels of medical expertise were demonstrated for RESCAPE-intraosseous needle insertion (P = .0002), RESCAPE-bag mask ventilation (P = .0002), RESCAPE-endotracheal intubation (P = .0001), and RESCAPE-cardiac massage (P = .0037). Significantly increased median scores over time were also demonstrated during a simulation-based educational program. RESCAPE tools are reliable and validated tools for the evaluation of technical resuscitation skills in pediatric settings during simulation-based educational programs. They might also be used for medical practice performance evaluations. Copyright © 2017 Elsevier Inc. All rights reserved.

Assessing the reliability and validity of anti-tobacco attitudes/beliefs in the context of a campaign strategy.

PubMed

Arheart, Kristopher L; Sly, David F; Trapido, Edward J; Rodriguez, Richard D; Ellestad, Amy J

2004-11-01

To identify multi-item attitude/belief scales associated with the theoretical foundations of an anti-tobacco counter-marketing campaign and assess their reliability and validity. The data analyzed are from two state-wide, random, cross-sectional telephone surveys [n(S1)=1,079, n(S2)=1,150]. Items forming attitude/belief scales are identified using factor analysis. Reliability is assessed with Chronbach's alpha. Relationships among scales are explored using Pearson correlation. Validity is assessed by testing associations derived from the Centers for Disease Control and Prevention's (CDC) logic model for tobacco control program development and evaluation linking media exposure to attitudes/beliefs, and attitudes/beliefs to smoking-related behaviors. Adjusted odds ratios are employed for these analyses. Three factors emerged: traditional attitudes/beliefs about tobacco and tobacco use, tobacco industry manipulation and anti-tobacco empowerment. Reliability coefficients are in the range of 0.70 and vary little between age groups. The factors are correlated with one-another as hypothesized. Associations between media exposure and the attitude/belief scales and between these scales and behaviors are consistent with the CDC logic model. Using reliable, valid multi-item scales is theoretically and methodologically more sound than employing single-item measures of attitudes/beliefs. Methodological, theoretical and practical implications are discussed.

Systematic Review of Childhood Sedentary Behavior Questionnaires: What do We Know and What is Next?

PubMed

Hidding, Lisan M; Altenburg, Teatske M; Mokkink, Lidwine B; Terwee, Caroline B; Chinapaw, Mai J M

2017-04-01

Accurate measurement of child sedentary behavior is necessary for monitoring trends, examining health effects, and evaluating the effectiveness of interventions. We therefore aimed to summarize studies examining the measurement properties of self-report or proxy-report sedentary behavior questionnaires for children and adolescents under the age of 18 years. Additionally, we provided an overview of the characteristics of the evaluated questionnaires. We performed systematic literature searches in the EMBASE, PubMed, and SPORTDiscus electronic databases. Studies had to report on at least one measurement property of a questionnaire assessing sedentary behavior. Questionnaire data were extracted using a standardized checklist, i.e. the Quality Assessment of Physical Activity Questionnaire (QAPAQ) checklist, and the methodological quality of the included studies was rated using a standardized tool, i.e. the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist. Forty-six studies on 46 questionnaires met our inclusion criteria, of which 33 examined test-retest reliability, nine examined measurement error, two examined internal consistency, 22 examined construct validity, eight examined content validity, and two examined structural validity. The majority of the included studies were of fair or poor methodological quality. Of the studies with at least a fair methodological quality, six scored positive on test-retest reliability, and two scored positive on construct validity. None of the questionnaires included in this review were considered as both valid and reliable. High-quality studies on the most promising questionnaires are required, with more attention to the content validity of the questionnaires. PROSPERO registration number: CRD42016035963.

Uncertainty evaluation of EnPIs in industrial applications as a key factor in setting improvement actions

NASA Astrophysics Data System (ADS)

D'Emilia, G.; Di Gasbarro, D.; Gaspari, A.; Natale, E.

2015-11-01

A methodology is proposed assuming high-level Energy Performance Indicators (EnPIs) uncertainty as quantitative indicator of the evolution of an Energy Management System (EMS). Motivations leading to the selection of the EnPIs, uncertainty evaluation techniques and criteria supporting decision-making are discussed, in order to plan and pursue reliable measures for energy performance improvement. In this paper, problems, priorities, operative possibilities and reachable improvement limits are examined, starting from the measurement uncertainty assessment. Two different industrial cases are analysed with reference to the following aspects: absence/presence of energy management policy and action plans; responsibility level for the energy issues; employees’ training and motivation in respect of the energy problems; absence/presence of adequate infrastructures for monitoring and sharing of energy information; level of standardization and integration of methods and procedures linked to the energy activities; economic and financial resources for the improvement of energy efficiency. A critic and comparative analysis of the obtained results is realized. The methodology, experimentally validated, allows developing useful considerations for effective, realistic and economically feasible improvement plans, depending on the specific situation. Recursive application of the methodology allows getting reliable and resolved assessment of the EMS status, also in dynamic industrial contexts.

Superior model for fault tolerance computation in designing nano-sized circuit systems

DOE Office of Scientific and Technical Information (OSTI.GOV)

Singh, N. S. S., E-mail: narinderjit@petronas.com.my; Muthuvalu, M. S., E-mail: msmuthuvalu@gmail.com; Asirvadam, V. S., E-mail: vijanth-sagayan@petronas.com.my

2014-10-24

As CMOS technology scales nano-metrically, reliability turns out to be a decisive subject in the design methodology of nano-sized circuit systems. As a result, several computational approaches have been developed to compute and evaluate reliability of desired nano-electronic circuits. The process of computing reliability becomes very troublesome and time consuming as the computational complexity build ups with the desired circuit size. Therefore, being able to measure reliability instantly and superiorly is fast becoming necessary in designing modern logic integrated circuits. For this purpose, the paper firstly looks into the development of an automated reliability evaluation tool based on the generalizationmore » of Probabilistic Gate Model (PGM) and Boolean Difference-based Error Calculator (BDEC) models. The Matlab-based tool allows users to significantly speed-up the task of reliability analysis for very large number of nano-electronic circuits. Secondly, by using the developed automated tool, the paper explores into a comparative study involving reliability computation and evaluation by PGM and, BDEC models for different implementations of same functionality circuits. Based on the reliability analysis, BDEC gives exact and transparent reliability measures, but as the complexity of the same functionality circuits with respect to gate error increases, reliability measure by BDEC tends to be lower than the reliability measure by PGM. The lesser reliability measure by BDEC is well explained in this paper using distribution of different signal input patterns overtime for same functionality circuits. Simulation results conclude that the reliability measure by BDEC depends not only on faulty gates but it also depends on circuit topology, probability of input signals being one or zero and also probability of error on signal lines.« less

Predicting the onset and persistence of episodes of depression in primary health care. The predictD-Spain study: Methodology

PubMed Central

Bellón, Juan Ángel; Moreno-Küstner, Berta; Torres-González, Francisco; Montón-Franco, Carmen; GildeGómez-Barragán, María Josefa; Sánchez-Celaya, Marta; Díaz-Barreiros, Miguel Ángel; Vicens, Catalina; de Dios Luna, Juan; Cervilla, Jorge A; Gutierrez, Blanca; Martínez-Cañavate, María Teresa; Oliván-Blázquez, Bárbara; Vázquez-Medrano, Ana; Sánchez-Artiaga, María Soledad; March, Sebastia; Motrico, Emma; Ruiz-García, Victor Manuel; Brangier-Wainberg, Paulette Renée; del Mar Muñoz-García, María; Nazareth, Irwin; King, Michael

2008-01-01

Background The effects of putative risk factors on the onset and/or persistence of depression remain unclear. We aim to develop comprehensive models to predict the onset and persistence of episodes of depression in primary care. Here we explain the general methodology of the predictD-Spain study and evaluate the reliability of the questionnaires used. Methods This is a prospective cohort study. A systematic random sample of general practice attendees aged 18 to 75 has been recruited in seven Spanish provinces. Depression is being measured with the CIDI at baseline, and at 6, 12, 24 and 36 months. A set of individual, environmental, genetic, professional and organizational risk factors are to be assessed at each follow-up point. In a separate reliability study, a proportional random sample of 401 participants completed the test-retest (251 researcher-administered and 150 self-administered) between October 2005 and February 2006. We have also checked 118,398 items for data entry from a random sample of 480 patients stratified by province. Results All items and questionnaires had good test-retest reliability for both methods of administration, except for the use of recreational drugs over the previous six months. Cronbach's alphas were good and their factorial analyses coherent for the three scales evaluated (social support from family and friends, dissatisfaction with paid work, and dissatisfaction with unpaid work). There were 191 (0.16%) data entry errors. Conclusion The items and questionnaires were reliable and data quality control was excellent. When we eventually obtain our risk index for the onset and persistence of depression, we will be able to determine the individual risk of each patient evaluated in primary health care. PMID:18657275

A Methodology for the Development of a Reliability Database for an Advanced Reactor Probabilistic Risk Assessment

DOE Office of Scientific and Technical Information (OSTI.GOV)

Grabaskas, Dave; Brunett, Acacia J.; Bucknor, Matthew

GE Hitachi Nuclear Energy (GEH) and Argonne National Laboratory are currently engaged in a joint effort to modernize and develop probabilistic risk assessment (PRA) techniques for advanced non-light water reactors. At a high level the primary outcome of this project will be the development of next-generation PRA methodologies that will enable risk-informed prioritization of safety- and reliability-focused research and development, while also identifying gaps that may be resolved through additional research. A subset of this effort is the development of a reliability database (RDB) methodology to determine applicable reliability data for inclusion in the quantification of the PRA. The RDBmore » method developed during this project seeks to satisfy the requirements of the Data Analysis element of the ASME/ANS Non-LWR PRA standard. The RDB methodology utilizes a relevancy test to examine reliability data and determine whether it is appropriate to include as part of the reliability database for the PRA. The relevancy test compares three component properties to establish the level of similarity to components examined as part of the PRA. These properties include the component function, the component failure modes, and the environment/boundary conditions of the component. The relevancy test is used to gauge the quality of data found in a variety of sources, such as advanced reactor-specific databases, non-advanced reactor nuclear databases, and non-nuclear databases. The RDB also establishes the integration of expert judgment or separate reliability analysis with past reliability data. This paper provides details on the RDB methodology, and includes an example application of the RDB methodology for determining the reliability of the intermediate heat exchanger of a sodium fast reactor. The example explores a variety of reliability data sources, and assesses their applicability for the PRA of interest through the use of the relevancy test.« less

A Comparison of Two Methods of Determining Interrater Reliability

ERIC Educational Resources Information Center

Fleming, Judith A.; Taylor, Janeen McCracken; Carran, Deborah

2004-01-01

This article offers an alternative methodology for practitioners and researchers to use in establishing interrater reliability for testing purposes. The majority of studies on interrater reliability use a traditional methodology where by two raters are compared using a Pearson product-moment correlation. This traditional method of estimating…

A methodology for producing reliable software, volume 1

NASA Technical Reports Server (NTRS)

Stucki, L. G.; Moranda, P. B.; Foshee, G.; Kirchoff, M.; Omre, R.

1976-01-01

An investigation into the areas having an impact on producing reliable software including automated verification tools, software modeling, testing techniques, structured programming, and management techniques is presented. This final report contains the results of this investigation, analysis of each technique, and the definition of a methodology for producing reliable software.

Reliability of Soft Tissue Model Based Implant Surgical Guides; A Methodological Mistake.

PubMed

Sabour, Siamak; Dastjerdi, Elahe Vahid

2012-08-20

Abstract We were interested to read the paper by Maney P and colleagues published in the July 2012 issue of J Oral Implantol. The authors aimed to assess the reliability of soft tissue model based implant surgical guides reported that the accuracy was evaluated using software. 1 I found the manuscript title of Maney P, et al. incorrect and misleading. Moreover, they reported twenty-two sites (46.81%) were considered accurate (13 of 24 maxillary and 9 of 23 mandibular sites). As the authors point out in their conclusion, Soft tissue models do not always provide sufficient accuracy for implant surgical guide fabrication.Reliability (precision) and validity (accuracy) are two different methodological issues in researches. Sensitivity, specificity, PPV, NPV, likelihood ratio positive (true positive/false negative) and likelihood ratio negative (false positive/ true negative) as well as odds ratio (true results\\false results - preferably more than 50) are among the tests to evaluate the validity (accuracy) of a single test compared to a gold standard.2-4 It is not clear that the reported twenty-two sites (46.81%) which were considered accurate related to which of the above mentioned estimates for validity analysis. Reliability (repeatability or reproducibility) is being assessed by different statistical tests such as Pearson r, least square and paired t.test which all of them are among common mistakes in reliability analysis 5. Briefly, for quantitative variable Intra Class Correlation Coefficient (ICC) and for qualitative variables weighted kappa should be used with caution because kappa has its own limitation too. Regarding reliability or agreement, it is good to know that for computing kappa value, just concordant cells are being considered, whereas discordant cells should also be taking into account in order to reach a correct estimation of agreement (Weighted kappa).2-4 As a take home message, for reliability and validity analysis, appropriate tests should be applied.

Development and Application of a Novel Rasch-Based Methodology for Evaluating Multi-Tiered Assessment Instruments: Validation and Utilization of an Undergraduate Diagnostic Test of the Water Cycle

ERIC Educational Resources Information Center

Romine, William L.; Schaffer, Dane L.; Barrow, Lloyd

2015-01-01

We describe the development and validation of a three-tiered diagnostic test of the water cycle (DTWC) and use it to evaluate the impact of prior learning experiences on undergraduates' misconceptions. While most approaches to instrument validation take a positivist perspective using singular criteria such as reliability and fit with a measurement…

Reliability analysis and fault-tolerant system development for a redundant strapdown inertial measurement unit. [inertial platforms

NASA Technical Reports Server (NTRS)

Motyka, P.

1983-01-01

A methodology is developed and applied for quantitatively analyzing the reliability of a dual, fail-operational redundant strapdown inertial measurement unit (RSDIMU). A Markov evaluation model is defined in terms of the operational states of the RSDIMU to predict system reliability. A 27 state model is defined based upon a candidate redundancy management system which can detect and isolate a spectrum of failure magnitudes. The results of parametric studies are presented which show the effect on reliability of the gyro failure rate, both the gyro and accelerometer failure rates together, false alarms, probability of failure detection, probability of failure isolation, and probability of damage effects and mission time. A technique is developed and evaluated for generating dynamic thresholds for detecting and isolating failures of the dual, separated IMU. Special emphasis is given to the detection of multiple, nonconcurrent failures. Digital simulation time histories are presented which show the thresholds obtained and their effectiveness in detecting and isolating sensor failures.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Agalgaonkar, Yashodhan P.; Hammerstrom, Donald J.

The Pacific Northwest Smart Grid Demonstration (PNWSGD) was a smart grid technology performance evaluation project that included multiple U.S. states and cooperation from multiple electric utilities in the northwest region. One of the local objectives for the project was to achieve improved distribution system reliability. Toward this end, some PNWSGD utilities automated their distribution systems, including the application of fault detection, isolation, and restoration and advanced metering infrastructure. In light of this investment, a major challenge was to establish a correlation between implementation of these smart grid technologies and actual improvements of distribution system reliability. This paper proposes using Welch’smore » t-test to objectively determine and quantify whether distribution system reliability is improving over time. The proposed methodology is generic, and it can be implemented by any utility after calculation of the standard reliability indices. The effectiveness of the proposed hypothesis testing approach is demonstrated through comprehensive practical results. It is believed that wider adoption of the proposed approach can help utilities to evaluate a realistic long-term performance of smart grid technologies.« less

[Methodology for clinical research in Orthodontics, the assets of the beOrtho website].

PubMed

Ruiz, Martial; Thibult, François

2014-06-01

The rules applying to the "evidence-based" methodology strongly influenced the clinical research in orthodontics. However, the implementation of clinical studies requires rigour, important statistical and methodological knowledge, as well as a reliable environment in order to compile and store the data obtained from research. We developed the project "beOrtho.com" (based on orthodontic evidence) in order to fill up the gap between our desire to drive clinical research and the necessity of methodological rigour in the exploitation of its results. BeOrtho website was created to answer the issue of sample recruitment, data compilation and storage, while providing help for the methodological design of clinical studies. It allows the development and monitoring of clinical studies, as well as the creation of databases. On the other hand, we designed an evaluation grid for clinical studies which helps developing systematic reviews. In order to illustrate our point, we tested a research protocol evaluating the interest of the mandibular advancement in the framework of Class II treatment. © EDP Sciences, SFODF, 2014.

Advancing Usability Evaluation through Human Reliability Analysis

DOE Office of Scientific and Technical Information (OSTI.GOV)

Ronald L. Boring; David I. Gertman

2005-07-01

This paper introduces a novel augmentation to the current heuristic usability evaluation methodology. The SPAR-H human reliability analysis method was developed for categorizing human performance in nuclear power plants. Despite the specialized use of SPAR-H for safety critical scenarios, the method also holds promise for use in commercial off-the-shelf software usability evaluations. The SPAR-H method shares task analysis underpinnings with human-computer interaction, and it can be easily adapted to incorporate usability heuristics as performance shaping factors. By assigning probabilistic modifiers to heuristics, it is possible to arrive at the usability error probability (UEP). This UEP is not a literal probabilitymore » of error but nonetheless provides a quantitative basis to heuristic evaluation. When combined with a consequence matrix for usability errors, this method affords ready prioritization of usability issues.« less

Interrater reliability levels of multiple clinical examiners in the evaluation of a schizophrenic patient: quality of life, level of functioning, and neuropsychological symptomatology.

PubMed

Cicchetti, D V; Rosenheck, R; Showalter, D; Charney, D; Cramer, J

1999-05-01

Sir Ronald Fisher used a single-subject design to derive the concepts of appropriate research design, randomization, sensitivity, and tests of statistical significance. The seminal work of Broca demonstrated that valid and generalizable findings can and have emerged from studies of a single patient in neuropsychology. In order to assess the reliability and/or validity of any clinical phenomena that derive from single subject research, it becomes necessary to apply appropriate biostatistical methodology. The authors develop just such an approach and apply it successfully to the evaluation of the functioning, quality of life, and neuropsychological symptomatology of a single schizophrenic patient.

[Intraclass reliability of the Alberta Infant Motor Scale in the Brazilian version].

PubMed

Silva, Larissa Paiva; Maia, Polyana Candeia; Lopes, Márcia Maria Coelho Oliveira; Cardoso, Maria Vera Lúcia Moreira Leitão

2013-10-01

This study had as its objective to analyze the intraclass reliability of the Alberta Infant Motor Scale (AIMS), in the Brazilian version, in preterm and term infants. It was a methodological study, conducted from November 2009 to April 2010, with 50 children receiving care in two public institutions in Fortaleza, Ceará, Brazil. Children were grouped according to gestational age as preterm and term, and evaluated by three evaluators in the communication laboratory of a public institution or at home. The intraclass correlation indices for the categories prone, supine, sitting and standing ranged from 0.553 to 0.952; most remained above 0.800, except for the standing category of the third evaluator, in which the index was 0.553. As for the total score and percentile, rates ranged from 0.843 to 0.954. The scale proved to be a reliable instrument for assessing gross motor performance of Brazilian children, particularly in Ceará, regardless of gestational age at birth.

Evaluating Random Error in Clinician-Administered Surveys: Theoretical Considerations and Clinical Applications of Interobserver Reliability and Agreement.

PubMed

Bennett, Rebecca J; Taljaard, Dunay S; Olaithe, Michelle; Brennan-Jones, Chris; Eikelboom, Robert H

2017-09-18

The purpose of this study is to raise awareness of interobserver concordance and the differences between interobserver reliability and agreement when evaluating the responsiveness of a clinician-administered survey and, specifically, to demonstrate the clinical implications of data types (nominal/categorical, ordinal, interval, or ratio) and statistical index selection (for example, Cohen's kappa, Krippendorff's alpha, or interclass correlation). In this prospective cohort study, 3 clinical audiologists, who were masked to each other's scores, administered the Practical Hearing Aid Skills Test-Revised to 18 adult owners of hearing aids. Interobserver concordance was examined using a range of reliability and agreement statistical indices. The importance of selecting statistical measures of concordance was demonstrated with a worked example, wherein the level of interobserver concordance achieved varied from "no agreement" to "almost perfect agreement" depending on data types and statistical index selected. This study demonstrates that the methodology used to evaluate survey score concordance can influence the statistical results obtained and thus affect clinical interpretations.

Reliability of Real-time Ultrasound Imaging for the Assessment of Trunk Stabilizer Muscles: A Systematic Review of the Literature.

PubMed

Taghipour, Morteza; Mohseni-Bandpei, Mohammad Ali; Behtash, Hamid; Abdollahi, Iraj; Rajabzadeh, Fatemeh; Pourahmadi, Mohammad Reza; Emami, Mahnaz

2018-04-24

Rehabilitative ultrasound (US) imaging is one of the popular methods for investigating muscle morphologic characteristics and dimensions in recent years. The reliability of this method has been investigated in different studies. As studies have been performed with different designs and quality, reported values of rehabilitative US have a wide range. The objective of this study was to systematically review the literature conducted on the reliability of rehabilitative US imaging for the assessment of deep abdominal and lumbar trunk muscle dimensions. The PubMed/MEDLINE, Scopus, Google Scholar, Science Direct, Embase, Physiotherapy Evidence, Ovid, and CINAHL databases were searched to identify original research articles conducted on the reliability of rehabilitative US imaging published from June 2007 to August 2017. The articles were qualitatively assessed; reliability data were extracted; and the methodological quality was evaluated by 2 independent reviewers. Of the 26 included studies, 16 were considered of high methodological quality. Except for 2 studies, all high-quality studies reported intraclass correlation coefficients (ICCs) for intra-rater reliability of 0.70 or greater. Also, ICCs reported for inter-rater reliability in high-quality studies were generally greater than 0.70. Among low-quality studies, reported ICCs ranged from 0.26 to 0.99 and 0.68 to 0.97 for intra- and inter-rater reliability, respectively. Also, the reported standard error of measurement and minimal detectable change for rehabilitative US were generally in an acceptable range. Generally, the results of the reviewed studies indicate that rehabilitative US imaging has good levels of both inter- and intra-rater reliability. © 2018 by the American Institute of Ultrasound in Medicine.

Assessing the reliability of ecotoxicological studies: An overview of current needs and approaches.

PubMed

Moermond, Caroline; Beasley, Amy; Breton, Roger; Junghans, Marion; Laskowski, Ryszard; Solomon, Keith; Zahner, Holly

2017-07-01

In general, reliable studies are well designed and well performed, and enough details on study design and performance are reported to assess the study. For hazard and risk assessment in various legal frameworks, many different types of ecotoxicity studies need to be evaluated for reliability. These studies vary in study design, methodology, quality, and level of detail reported (e.g., reviews, peer-reviewed research papers, or industry-sponsored studies documented under Good Laboratory Practice [GLP] guidelines). Regulators have the responsibility to make sound and verifiable decisions and should evaluate each study for reliability in accordance with scientific principles regardless of whether they were conducted in accordance with GLP and/or standardized methods. Thus, a systematic and transparent approach is needed to evaluate studies for reliability. In this paper, 8 different methods for reliability assessment were compared using a number of attributes: categorical versus numerical scoring methods, use of exclusion and critical criteria, weighting of criteria, whether methods are tested with case studies, domain of applicability, bias toward GLP studies, incorporation of standard guidelines in the evaluation method, number of criteria used, type of criteria considered, and availability of guidance material. Finally, some considerations are given on how to choose a suitable method for assessing reliability of ecotoxicity studies. Integr Environ Assess Manag 2017;13:640-651. © 2016 The Authors. Integrated Environmental Assessment and Management published by Wiley Periodicals, Inc. on behalf of Society of Environmental Toxicology & Chemistry (SETAC). © 2016 The Authors. Integrated Environmental Assessment and Management published by Wiley Periodicals, Inc. on behalf of Society of Environmental Toxicology & Chemistry (SETAC).

Appraising the methodological quality of the clinical practice guideline for diabetes mellitus using the AGREE II instrument: a methodological evaluation.

PubMed

Radwan, Mahmoud; Akbari Sari, Ali; Rashidian, Arash; Takian, Amirhossein; Abou-Dagga, Sanaa; Elsous, Aymen

2017-02-01

To evaluate the methodological quality of the Palestinian Clinical Practice Guideline for Diabetes Mellitus using the Translated Arabic Version of the AGREE II. Methodological evaluation. A cross-cultural adaptation framework was followed to translate and develop a standardised Translated Arabic Version of the AGREE II. Palestinian Primary Healthcare Centres. Sixteen appraisers independently evaluated the Clinical Practice Guideline for Diabetes Mellitus using the Translated Arabic Version of the AGREE II. Methodological quality of diabetic guideline. The Translated Arabic Version of the AGREE II showed an acceptable reliability and validity. Internal consistency ranged between 0.67 and 0.88 (Cronbach's α). Intra-class coefficient among appraisers ranged between 0.56 and 0.88. The quality of this guideline is low. Both domains 'Scope and Purpose' and 'Clarity of Presentation' had the highest quality scores (66.7% and 61.5%, respectively), whereas the scores for 'Applicability', 'Stakeholder Involvement', 'Rigour of Development' and 'Editorial Independence' were the lowest (27%, 35%, 36.5%, and 40%, respectively). The findings suggest that the quality of this Clinical Practice Guideline is disappointingly low. To improve the quality of current and future guidelines, the AGREE II instrument is extremely recommended to be incorporated as a gold standard for developing, evaluating or updating the Palestinian Clinical Practice Guidelines. Future guidelines can be improved by setting specific strategies to overcome implementation barriers with respect to economic considerations, engaging of all relevant end-users and patients, ensuring a rigorous methodology for searching, selecting and synthesising the evidences and recommendations, and addressing potential conflict of interests within the development group.

Reliability of Instruments Measuring At-Risk and Problem Gambling Among Young Individuals: A Systematic Review Covering Years 2009-2015.

PubMed

Edgren, Robert; Castrén, Sari; Mäkelä, Marjukka; Pörtfors, Pia; Alho, Hannu; Salonen, Anne H

2016-06-01

This review aims to clarify which instruments measuring at-risk and problem gambling (ARPG) among youth are reliable and valid in light of reported estimates of internal consistency, classification accuracy, and psychometric properties. A systematic search was conducted in PubMed, Medline, and PsycInfo covering the years 2009-2015. In total, 50 original research articles fulfilled the inclusion criteria: target age under 29 years, using an instrument designed for youth, and reporting a reliability estimate. Articles were evaluated with the revised Quality Assessment of Diagnostic Accuracy Studies tool. Reliability estimates were reported for five ARPG instruments. Most studies (66%) evaluated the South Oaks Gambling Screen Revised for Adolescents. The Gambling Addictive Behavior Scale for Adolescents was the only novel instrument. In general, the evaluation of instrument reliability was superficial. Despite its rare use, the Canadian Adolescent Gambling Inventory (CAGI) had a strong theoretical and methodological base. The Gambling Addictive Behavior Scale for Adolescents and the CAGI were the only instruments originally developed for youth. All studies, except the CAGI study, were population based. ARPG instruments for youth have not been rigorously evaluated yet. Further research is needed especially concerning instruments designed for clinical use. Copyright © 2016 The Society for Adolescent Health and Medicine. Published by Elsevier Inc. All rights reserved.

An Independent Evaluation of the FMEA/CIL Hazard Analysis Alternative Study

NASA Technical Reports Server (NTRS)

Ray, Paul S.

1996-01-01

The present instruments of safety and reliability risk control for a majority of the National Aeronautics and Space Administration (NASA) programs/projects consist of Failure Mode and Effects Analysis (FMEA), Hazard Analysis (HA), Critical Items List (CIL), and Hazard Report (HR). This extensive analytical approach was introduced in the early 1970's and was implemented for the Space Shuttle Program by NHB 5300.4 (1D-2. Since the Challenger accident in 1986, the process has been expanded considerably and resulted in introduction of similar and/or duplicated activities in the safety/reliability risk analysis. A study initiated in 1995, to search for an alternative to the current FMEA/CIL Hazard Analysis methodology generated a proposed method on April 30, 1996. The objective of this Summer Faculty Study was to participate in and conduct an independent evaluation of the proposed alternative to simplify the present safety and reliability risk control procedure.

Single-case research design in pediatric psychology: considerations regarding data analysis.

PubMed

Cohen, Lindsey L; Feinstein, Amanda; Masuda, Akihiko; Vowles, Kevin E

2014-03-01

Single-case research allows for an examination of behavior and can demonstrate the functional relation between intervention and outcome in pediatric psychology. This review highlights key assumptions, methodological and design considerations, and options for data analysis. Single-case methodology and guidelines are reviewed with an in-depth focus on visual and statistical analyses. Guidelines allow for the careful evaluation of design quality and visual analysis. A number of statistical techniques have been introduced to supplement visual analysis, but to date, there is no consensus on their recommended use in single-case research design. Single-case methodology is invaluable for advancing pediatric psychology science and practice, and guidelines have been introduced to enhance the consistency, validity, and reliability of these studies. Experts generally agree that visual inspection is the optimal method of analysis in single-case design; however, statistical approaches are becoming increasingly evaluated and used to augment data interpretation.

Comparison of sampling methodologies for nutrient monitoring in streams: uncertainties, costs and implications for mitigation

NASA Astrophysics Data System (ADS)

Audet, J.; Martinsen, L.; Hasler, B.; de Jonge, H.; Karydi, E.; Ovesen, N. B.; Kronvang, B.

2014-07-01

Eutrophication of aquatic ecosystems caused by excess concentrations of nitrogen and phosphorus may have harmful consequences for biodiversity and poses a health risk to humans via the water supplies. Reduction of nitrogen and phosphorus losses to aquatic ecosystems involves implementation of costly measures, and reliable monitoring methods are therefore essential to select appropriate mitigation strategies and to evaluate their effects. Here, we compare the performances and costs of three methodologies for the monitoring of nutrients in rivers: grab sampling, time-proportional sampling and passive sampling using flow proportional samplers. Assuming time-proportional sampling to be the best estimate of the "true" nutrient load, our results showed that the risk of obtaining wrong total nutrient load estimates by passive samplers is high despite similar costs as the time-proportional sampling. Our conclusion is that for passive samplers to provide a reliable monitoring alternative, further development is needed. Grab sampling was the cheapest of the three methods and was more precise and accurate than passive sampling. We conclude that although monitoring employing time-proportional sampling is costly, its reliability precludes unnecessarily high implementation expenses.

Comparison of sampling methodologies for nutrient monitoring in streams: uncertainties, costs and implications for mitigation

NASA Astrophysics Data System (ADS)

Audet, J.; Martinsen, L.; Hasler, B.; de Jonge, H.; Karydi, E.; Ovesen, N. B.; Kronvang, B.

2014-11-01

Eutrophication of aquatic ecosystems caused by excess concentrations of nitrogen and phosphorus may have harmful consequences for biodiversity and poses a health risk to humans via water supplies. Reduction of nitrogen and phosphorus losses to aquatic ecosystems involves implementation of costly measures, and reliable monitoring methods are therefore essential to select appropriate mitigation strategies and to evaluate their effects. Here, we compare the performances and costs of three methodologies for the monitoring of nutrients in rivers: grab sampling; time-proportional sampling; and passive sampling using flow-proportional samplers. Assuming hourly time-proportional sampling to be the best estimate of the "true" nutrient load, our results showed that the risk of obtaining wrong total nutrient load estimates by passive samplers is high despite similar costs as the time-proportional sampling. Our conclusion is that for passive samplers to provide a reliable monitoring alternative, further development is needed. Grab sampling was the cheapest of the three methods and was more precise and accurate than passive sampling. We conclude that although monitoring employing time-proportional sampling is costly, its reliability precludes unnecessarily high implementation expenses.

A Guide to the Application of Probability Risk Assessment Methodology and Hazard Risk Frequency Criteria as a Hazard Control for the Use of the Mobile Servicing System on the International Space Station

NASA Astrophysics Data System (ADS)

D'silva, Oneil; Kerrison, Roger

2013-09-01

A key feature for the increased utilization of space robotics is to automate Extra-Vehicular manned space activities and thus significantly reduce the potential for catastrophic hazards while simultaneously minimizing the overall costs associated with manned space. The principal scope of the paper is to evaluate the use of industry standard accepted Probability risk/safety assessment (PRA/PSA) methodologies and Hazard Risk frequency Criteria as a hazard control. This paper illustrates the applicability of combining the selected Probability risk assessment methodology and hazard risk frequency criteria, in order to apply the necessary safety controls that allow for the increased use of the Mobile Servicing system (MSS) robotic system on the International Space Station. This document will consider factors such as component failure rate reliability, software reliability, and periods of operation and dormancy, fault tree analyses and their effects on the probability risk assessments. The paper concludes with suggestions for the incorporation of existing industry Risk/Safety plans to create an applicable safety process for future activities/programs

Development of Testing Methodologies for the Mechanical Properties of MEMS

NASA Technical Reports Server (NTRS)

Ekwaro-Osire, Stephen

2003-01-01

This effort is to investigate and design testing strategies to determine the mechanical properties of MicroElectroMechanical Systems (MEMS) as well as investigate the development of a MEMS Probabilistic Design Methodology (PDM). One item of potential interest is the design of a test for the Weibull size effect in pressure membranes. The Weibull size effect is a consequence of a stochastic strength response predicted from the Weibull distribution. Confirming that MEMS strength is controlled by the Weibull distribution will enable the development of a probabilistic design methodology for MEMS - similar to the GRC developed CARES/Life program for bulk ceramics. However, the primary area of investigation will most likely be analysis and modeling of material interfaces for strength as well as developing a strategy to handle stress singularities at sharp corners, filets, and material interfaces. This will be a continuation of the previous years work. The ultimate objective of this effort is to further develop and verify the ability of the Ceramics Analysis and Reliability Evaluation of Structures Life (CARES/Life) code to predict the time-dependent reliability of MEMS structures subjected to multiple transient loads.

Systematic review of studies on measurement properties of instruments for adults published in the American Journal of Occupational Therapy, 2009-2013.

PubMed

Yuen, Hon K; Austin, Sarah L

2014-01-01

We describe the methodological quality of recent studies on instrument development and testing published in the American Journal of Occupational Therapy (AJOT). We conducted a systematic review using the COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN) checklist to appraise 48 articles on measurement properties of assessments for adults published in AJOT between 2009 and 2013. Most studies had adequate methodological quality in design and statistical analysis. Common methodological limitations included that methods used to examine internal consistency were not consistently linked to the theoretical constructs underpinning assessments; participants in some test-retest reliability studies were not stable during the interim period; and in several studies of reliability and convergent validity, sample sizes were inadequate. AJOT's dissemination of psychometric research evidence has made important contributions to moving the profession toward the American Occupational Therapy Association's Centennial Vision. This study's results provide a benchmark by which to evaluate future accomplishments. Copyright © 2014 by the American Occupational Therapy Association, Inc.

Adaptation of the ToxRTool to Assess the Reliability of Toxicology Studies Conducted with Genetically Modified Crops and Implications for Future Safety Testing.

PubMed

Koch, Michael S; DeSesso, John M; Williams, Amy Lavin; Michalek, Suzanne; Hammond, Bruce

2016-01-01

To determine the reliability of food safety studies carried out in rodents with genetically modified (GM) crops, a Food Safety Study Reliability Tool (FSSRTool) was adapted from the European Centre for the Validation of Alternative Methods' (ECVAM) ToxRTool. Reliability was defined as the inherent quality of the study with regard to use of standardized testing methodology, full documentation of experimental procedures and results, and the plausibility of the findings. Codex guidelines for GM crop safety evaluations indicate toxicology studies are not needed when comparability of the GM crop to its conventional counterpart has been demonstrated. This guidance notwithstanding, animal feeding studies have routinely been conducted with GM crops, but their conclusions on safety are not always consistent. To accurately evaluate potential risks from GM crops, risk assessors need clearly interpretable results from reliable studies. The development of the FSSRTool, which provides the user with a means of assessing the reliability of a toxicology study to inform risk assessment, is discussed. Its application to the body of literature on GM crop food safety studies demonstrates that reliable studies report no toxicologically relevant differences between rodents fed GM crops or their non-GM comparators.

Performance and Reliability Analysis of Water Distribution Systems under Cascading Failures and the Identification of Crucial Pipes

PubMed Central

Shuang, Qing; Zhang, Mingyuan; Yuan, Yongbo

2014-01-01

As a mean of supplying water, Water distribution system (WDS) is one of the most important complex infrastructures. The stability and reliability are critical for urban activities. WDSs can be characterized by networks of multiple nodes (e.g. reservoirs and junctions) and interconnected by physical links (e.g. pipes). Instead of analyzing highest failure rate or highest betweenness, reliability of WDS is evaluated by introducing hydraulic analysis and cascading failures (conductive failure pattern) from complex network. The crucial pipes are identified eventually. The proposed methodology is illustrated by an example. The results show that the demand multiplier has a great influence on the peak of reliability and the persistent time of the cascading failures in its propagation in WDS. The time period when the system has the highest reliability is when the demand multiplier is less than 1. There is a threshold of tolerance parameter exists. When the tolerance parameter is less than the threshold, the time period with the highest system reliability does not meet minimum value of demand multiplier. The results indicate that the system reliability should be evaluated with the properties of WDS and the characteristics of cascading failures, so as to improve its ability of resisting disasters. PMID:24551102

Exploratory Factor Analysis and Multisection Validation by Correlation of Student Self-Reported Attitudes of Teacher Effectiveness with Student Achievement as Measured by Achievement Tests for Middle School Students

ERIC Educational Resources Information Center

Weber, David M.

2013-01-01

This study investigated the use of a student evaluation of teaching survey designed by a suburban school district. Several statistical methodologies were used to evaluate the validity and reliability of the instrument. One hundred sections of grades 6-8 reading and mathematics courses were used to examine the research question: Is the Student…

Using experts feedback in clinical case resolution and arbitration as accuracy diagnosis methodology.

PubMed

Rodríguez-González, Alejandro; Torres-Niño, Javier; Valencia-Garcia, Rafael; Mayer, Miguel A; Alor-Hernandez, Giner

2013-09-01

This paper proposes a new methodology for assessing the efficiency of medical diagnostic systems and clinical decision support systems by using the feedback/opinions of medical experts. The methodology behind this work is based on a comparison between the expert feedback that has helped solve different clinical cases and the expert system that has evaluated these same cases. Once the results are returned, an arbitration process is carried out in order to ensure the correctness of the results provided by both methods. Once this process has been completed, the results are analyzed using Precision, Recall, Accuracy, Specificity and Matthews Correlation Coefficient (MCC) (PRAS-M) metrics. When the methodology is applied, the results obtained from a real diagnostic system allow researchers to establish the accuracy of the system based on objective facts. The methodology returns enough information to analyze the system's behavior for each disease in the knowledge base or across the entire knowledge base. It also returns data on the efficiency of the different assessors involved in the evaluation process, analyzing their behavior in the diagnostic process. The proposed work facilitates the evaluation of medical diagnostic systems, having a reliable process based on objective facts. The methodology presented in this research makes it possible to identify the main characteristics that define a medical diagnostic system and their values, allowing for system improvement. A good example of the results provided by the application of the methodology is shown in this paper. A diagnosis system was evaluated by means of this methodology, yielding positive results (statistically significant) when comparing the system with the assessors that participated in the evaluation process of the system through metrics such as recall (+27.54%) and MCC (+32.19%). These results demonstrate the real applicability of the methodology used. Copyright © 2013 Elsevier Ltd. All rights reserved.

The Research Diagnostic Criteria for Temporomandibular Disorders. I: overview and methodology for assessment of validity.

PubMed

Schiffman, Eric L; Truelove, Edmond L; Ohrbach, Richard; Anderson, Gary C; John, Mike T; List, Thomas; Look, John O

2010-01-01

The purpose of the Research Diagnostic Criteria for Temporomandibular Disorders (RDC/TMD) Validation Project was to assess the diagnostic validity of this examination protocol. The aim of this article is to provide an overview of the project's methodology, descriptive statistics, and data for the study participant sample. This article also details the development of reliable methods to establish the reference standards for assessing criterion validity of the Axis I RDC/TMD diagnoses. The Axis I reference standards were based on the consensus of two criterion examiners independently performing a comprehensive history, clinical examination, and evaluation of imaging. Intersite reliability was assessed annually for criterion examiners and radiologists. Criterion examination reliability was also assessed within study sites. Study participant demographics were comparable to those of participants in previous studies using the RDC/TMD. Diagnostic agreement of the criterion examiners with each other and with the consensus-based reference standards was excellent with all kappas > or = 0.81, except for osteoarthrosis (moderate agreement, k = 0.53). Intrasite criterion examiner agreement with reference standards was excellent (k > or = 0.95). Intersite reliability of the radiologists for detecting computed tomography-disclosed osteoarthrosis and magnetic resonance imaging-disclosed disc displacement was good to excellent (k = 0.71 and 0.84, respectively). The Validation Project study population was appropriate for assessing the reliability and validity of the RDC/TMD Axis I and II. The reference standards used to assess the validity of Axis I TMD were based on reliable and clinically credible methods.

The reliability and validity of cervical auscultation in the diagnosis of dysphagia: a systematic review.

PubMed

Lagarde, Marloes L J; Kamalski, Digna M A; van den Engel-Hoek, Lenie

2016-02-01

To systematically review the available evidence for the reliability and validity of cervical auscultation in diagnosing the several aspects of dysphagia in adults and children suffering from dysphagia. Medline (PubMed), Embase and the Cochrane Library databases. The systematic review was carried out applying the steps of the PRISMA-statement. The methodological quality of the included studies were evaluated using the Dutch 'Cochrane checklist for diagnostic accuracy studies'. A total of 90 articles were identified through the search strategy, and after applying the inclusion and exclusion criteria, six articles were included in this review. In the six studies, 197 patients were assessed with cervical auscultation. Two of the six articles were considered to be of 'good' quality and three studies were of 'moderate' quality. One article was excluded because of a 'poor' methodological quality. Sensitivity ranges from 23%-94% and specificity ranges from 50%-74%. Inter-rater reliability was 'poor' or 'fair' in all studies. The intra-rater reliability shows a wide variance among speech language therapists. In this systematic review, conflicting evidence is found for the validity of cervical auscultation. The reliability of cervical auscultation is insufficient when used as a stand-alone tool in the diagnosis of dysphagia in adults. There is no available evidence for the validity and reliability of cervical auscultation in children. Cervical auscultation should not be used as a stand-alone instrument to diagnose dysphagia. © The Author(s) 2015.

Overcoming the Challenges of Unstructured Data in Multisite, Electronic Medical Record-based Abstraction.

PubMed

Polnaszek, Brock; Gilmore-Bykovskyi, Andrea; Hovanes, Melissa; Roiland, Rachel; Ferguson, Patrick; Brown, Roger; Kind, Amy J H

2016-10-01

Unstructured data encountered during retrospective electronic medical record (EMR) abstraction has routinely been identified as challenging to reliably abstract, as these data are often recorded as free text, without limitations to format or structure. There is increased interest in reliably abstracting this type of data given its prominent role in care coordination and communication, yet limited methodological guidance exists. As standard abstraction approaches resulted in substandard data reliability for unstructured data elements collected as part of a multisite, retrospective EMR study of hospital discharge communication quality, our goal was to develop, apply and examine the utility of a phase-based approach to reliably abstract unstructured data. This approach is examined using the specific example of discharge communication for warfarin management. We adopted a "fit-for-use" framework to guide the development and evaluation of abstraction methods using a 4-step, phase-based approach including (1) team building; (2) identification of challenges; (3) adaptation of abstraction methods; and (4) systematic data quality monitoring. Unstructured data elements were the focus of this study, including elements communicating steps in warfarin management (eg, warfarin initiation) and medical follow-up (eg, timeframe for follow-up). After implementation of the phase-based approach, interrater reliability for all unstructured data elements demonstrated κ's of ≥0.89-an average increase of +0.25 for each unstructured data element. As compared with standard abstraction methodologies, this phase-based approach was more time intensive, but did markedly increase abstraction reliability for unstructured data elements within multisite EMR documentation.

A Chip and Pixel Qualification Methodology on Imaging Sensors

NASA Technical Reports Server (NTRS)

Chen, Yuan; Guertin, Steven M.; Petkov, Mihail; Nguyen, Duc N.; Novak, Frank

2004-01-01

This paper presents a qualification methodology on imaging sensors. In addition to overall chip reliability characterization based on sensor s overall figure of merit, such as Dark Rate, Linearity, Dark Current Non-Uniformity, Fixed Pattern Noise and Photon Response Non-Uniformity, a simulation technique is proposed and used to project pixel reliability. The projected pixel reliability is directly related to imaging quality and provides additional sensor reliability information and performance control.

Advanced Reactor PSA Methodologies for System Reliability Analysis and Source Term Assessment

DOE Office of Scientific and Technical Information (OSTI.GOV)

Grabaskas, D.; Brunett, A.; Passerini, S.

Beginning in 2015, a project was initiated to update and modernize the probabilistic safety assessment (PSA) of the GE-Hitachi PRISM sodium fast reactor. This project is a collaboration between GE-Hitachi and Argonne National Laboratory (Argonne), and funded in part by the U.S. Department of Energy. Specifically, the role of Argonne is to assess the reliability of passive safety systems, complete a mechanistic source term calculation, and provide component reliability estimates. The assessment of passive system reliability focused on the performance of the Reactor Vessel Auxiliary Cooling System (RVACS) and the inherent reactivity feedback mechanisms of the metal fuel core. Themore » mechanistic source term assessment attempted to provide a sequence specific source term evaluation to quantify offsite consequences. Lastly, the reliability assessment focused on components specific to the sodium fast reactor, including electromagnetic pumps, intermediate heat exchangers, the steam generator, and sodium valves and piping.« less

Space Transportation Operations: Assessment of Methodologies and Models

NASA Technical Reports Server (NTRS)

Joglekar, Prafulla

2001-01-01

The systems design process for future space transportation involves understanding multiple variables and their effect on lifecycle metrics. Variables such as technology readiness or potential environmental impact are qualitative, while variables such as reliability, operations costs or flight rates are quantitative. In deciding what new design concepts to fund, NASA needs a methodology that would assess the sum total of all relevant qualitative and quantitative lifecycle metrics resulting from each proposed concept. The objective of this research was to review the state of operations assessment methodologies and models used to evaluate proposed space transportation systems and to develop recommendations for improving them. It was found that, compared to the models available from other sources, the operations assessment methodology recently developed at Kennedy Space Center has the potential to produce a decision support tool that will serve as the industry standard. Towards that goal, a number of areas of improvement in the Kennedy Space Center's methodology are identified.

Space Transportation Operations: Assessment of Methodologies and Models

NASA Technical Reports Server (NTRS)

Joglekar, Prafulla

2002-01-01

The systems design process for future space transportation involves understanding multiple variables and their effect on lifecycle metrics. Variables such as technology readiness or potential environmental impact are qualitative, while variables such as reliability, operations costs or flight rates are quantitative. In deciding what new design concepts to fund, NASA needs a methodology that would assess the sum total of all relevant qualitative and quantitative lifecycle metrics resulting from each proposed concept. The objective of this research was to review the state of operations assessment methodologies and models used to evaluate proposed space transportation systems and to develop recommendations for improving them. It was found that, compared to the models available from other sources, the operations assessment methodology recently developed at Kennedy Space Center has the potential to produce a decision support tool that will serve as the industry standard. Towards that goal, a number of areas of improvement in the Kennedy Space Center's methodology are identified.

Generalized approach for identification and evaluation of technology-insertion options for military avionics systems

NASA Astrophysics Data System (ADS)

Harkness, Linda L.; Sjoberg, Eric S.

1996-06-01

The Georgia Tech Research Institute, sponsored by the Warner Robins Air Logistics Center, has developed an approach for efficiently postulating and evaluating methods for extending the life of radars and other avionics systems. The technique identified specific assemblies for potential replacement and evaluates the system level impact, including performance, reliability and life-cycle cost of each action. The initial impetus for this research was the increasing obsolescence of integrated circuits contained in the AN/APG-63 system. The operational life of military electronics is typically in excess of twenty years, which encompasses several generations of IC technology. GTRI has developed a systems approach to inserting modern technology components into older systems based upon identification of those functions which limit the system's performance or reliability and which are cost drivers. The presentation will discuss the above methodology and a technique for evaluating and ranking the different potential system upgrade options.

Drug utilization review: mechanisms to improve its effectiveness and broaden its scope. The U.S. Pharmacopeia Drug Utilization Review Advisory Panel.

PubMed

2000-01-01

To address important problems and needed changes in online and retrospective drug utilization review (DUR) programs. Emphasis is placed on reliability of DUR criteria and the shift of traditional retrospective DUR programs toward disease management and health care outcomes. Published literature evaluating the role of online and retrospective DUR programs. Particular attention was given to studies assessing DUR criteria reliability and new interventions with retrospective DUR programs. A literature review was conducted along with an expert summary from the U.S. Pharmacopeia Drug Utilization Review Advisory Panel. Studies have revealed variations in DUR criteria that could be affecting clinical practice and patient care. Appropriate formal methodologies and use of consistent procedures in developing online prospective DUR programs and systems could help resolve these problems. Traditional retrospective DUR is also shifting to incorporate disease management and methodologies from health outcomes and pharmacoeconomics studies. Refinements are needed to improve the reliability and validity of online DUR criteria and to minimize false positive messages. Databases created as a result of DUR efforts have been used in new and innovative ways to incorporate health outcomes data and disease management interventions. Additional outcomes data, combined with quality assurance efforts, should increase the utility of DUR/disease management efforts in evaluating health systems while improving the effectiveness and efficiency of pharmacists' health care interventions.

Measurement of General and Specific Approaches to Physical Activity Parenting: A Systematic Review

PubMed Central

McDonald, Samantha; Cohen, Alysia

2013-01-01

Abstract Background Parents play a significant role in shaping youth physical activity (PA). However, interventions targeting PA parenting have been ineffective. Methodological inconsistencies related to the measurement of parental influences may be a contributing factor. The purpose of this article is to review the extant peer-reviewed literature related to the measurement of general and specific parental influences on youth PA. Methods A systematic review of studies measuring constructs of PA parenting was conducted. Computerized searches were completed using PubMed, MEDLINE, Academic Search Premier, SPORTDiscus, and PsycINFO. Reference lists of the identified articles were manually reviewed as well as the authors' personal collections. Articles were selected on the basis of strict inclusion criteria and details regarding the measurement protocols were extracted. A total of 117 articles met the inclusionary criteria. Methodological articles that evaluated the validity and reliability of PA parenting measures (n=10) were reviewed separately from parental influence articles (n=107). Results A significant percentage of studies used measures with indeterminate validity and reliability. A significant percentage of articles did not provide sample items, describe the response format, or report the possible range of scores. No studies were located that evaluated sensitivity to change. Conclusion The reporting of measurement properties and the use of valid and reliable measurement scales need to be improved considerably. PMID:23944923

Evaluation of validity and reliability of a methodology for measuring human postural attitude and its relation to temporomandibular joint disorders

PubMed Central

Fernández, Ramón Fuentes; Carter, Pablo; Muñoz, Sergio; Silva, Héctor; Venegas, Gonzalo Hernán Oporto; Cantin, Mario; Ottone, Nicolás Ernesto

2016-01-01

INTRODUCTION Temporomandibular joint disorders (TMJDs) are caused by several factors such as anatomical, neuromuscular and psychological alterations. A relationship has been established between TMJDs and postural alterations, a type of anatomical alteration. An anterior position of the head requires hyperactivity of the posterior neck region and shoulder muscles to prevent the head from falling forward. This compensatory muscular function may cause fatigue, discomfort and trigger point activation. To our knowledge, a method for assessing human postural attitude in more than one plane has not been reported. Thus, the aim of this study was to design a methodology to measure the external human postural attitude in frontal and sagittal planes, with proper validity and reliability analyses. METHODS The variable postures of 78 subjects (36 men, 42 women; age 18–24 years) were evaluated. The postural attitudes of the subjects were measured in the frontal and sagittal planes, using an acromiopelvimeter, grid panel and Fox plane. RESULTS The method we designed for measuring postural attitudes had adequate reliability and validity, both qualitatively and quantitatively, based on Cohen’s Kappa coefficient (> 0.87) and Pearson’s correlation coefficient (r = 0.824, > 80%). CONCLUSION This method exhibits adequate metrical properties and can therefore be used in further research on the association of human body posture with skeletal types and TMJDs. PMID:26768173

Evaluation of validity and reliability of a methodology for measuring human postural attitude and its relation to temporomandibular joint disorders.

PubMed

Fuentes Fernández, Ramón; Carter, Pablo; Muñoz, Sergio; Silva, Héctor; Oporto Venegas, Gonzalo Hernán; Cantin, Mario; Ottone, Nicolás Ernesto

2016-04-01

Temporomandibular joint disorders (TMJDs) are caused by several factors such as anatomical, neuromuscular and psychological alterations. A relationship has been established between TMJDs and postural alterations, a type of anatomical alteration. An anterior position of the head requires hyperactivity of the posterior neck region and shoulder muscles to prevent the head from falling forward. This compensatory muscular function may cause fatigue, discomfort and trigger point activation. To our knowledge, a method for assessing human postural attitude in more than one plane has not been reported. Thus, the aim of this study was to design a methodology to measure the external human postural attitude in frontal and sagittal planes, with proper validity and reliability analyses. The variable postures of 78 subjects (36 men, 42 women; age 18-24 years) were evaluated. The postural attitudes of the subjects were measured in the frontal and sagittal planes, using an acromiopelvimeter, grid panel and Fox plane. The method we designed for measuring postural attitudes had adequate reliability and validity, both qualitatively and quantitatively, based on Cohen's Kappa coefficient (> 0.87) and Pearson's correlation coefficient (r = 0.824, > 80%). This method exhibits adequate metrical properties and can therefore be used in further research on the association of human body posture with skeletal types and TMJDs. Copyright © Singapore Medical Association.

Implementing and Evaluating a National Certification Technical Skills Examination: The Colorectal Objective Structured Assessment of Technical Skill.

PubMed

de Montbrun, Sandra; Roberts, Patricia L; Satterthwaite, Lisa; MacRae, Helen

2016-07-01

To implement the Colorectal Objective Structured Assessment of Technical skill (COSATS) into American Board of Colon and Rectal Surgery (ABCRS) certification and build evidence of validity for the interpretation of the scores of this high stakes assessment tool. Currently, technical skill assessment is not a formal component of board certification. With the technical demands of surgical specialties, documenting competence in technical skill at the time of certification with a valid tool is ideal. In September 2014, the COSATS was a mandatory component of ABCRS certification. Seventy candidates took the examination, with their performance evaluated by expert colorectal surgeons using a task-specific checklist, global rating scale, and overall performance scale. Passing scores were set and compared using 2 standard setting methodologies, using a compensatory and conjunctive model. Inter-rater reliability and the reliability of the pass/fail decision were calculated using Cronbach alpha and Subkoviak methodology, respectively. Overall COSATS scores and pass/fail status were compared with results on the ABCRS oral examination. The pass rate ranged from 85.7% to 90%. Inter-rater reliability (0.85) and reliability of the pass/fail decision (0.87 and 0.84) were high. A low positive correlation (r= 0.25) was seen between the COSATS and oral examination. All individuals who failed the COSATS passed the ABCRS oral examination. COSATS is the first technical skill examination used in national surgical board certification. This study suggests that the current certification process may be failing to identify individuals who have demonstrated technical deficiencies on this standardized assessment tool.

Are normative sonographic values of kidney size in children valid and reliable? A systematic review of the methodological quality of ultrasound studies using the Anatomical Quality Assessment (AQUA) tool.

PubMed

Chhapola, Viswas; Tiwari, Soumya; Deepthi, Bobbity; Henry, Brandon Michael; Brar, Rekha; Kanwal, Sandeep Kumar

2018-06-01

A plethora of research is available on ultrasonographic kidney size standards. We performed a systematic review of methodological quality of ultrasound studies aimed at developing normative renal parameters in healthy children, by evaluating the risk of bias (ROB) using the 'Anatomical Quality Assessment (AQUA)' tool. We searched Medline, Scopus, CINAHL, and Google Scholar on June 04 2018, and observational studies measuring kidney size by ultrasonography in healthy children (0-18 years) were included. The ROB of each study was evaluated in five domains using a 20 item coding scheme based on AQUA tool framework. Fifty-four studies were included. Domain 1 (subject characteristics) had a high ROB in 63% of studies due to the unclear description of age, sex, and ethnicity. The performance in Domain 2 (study design) was the best with 85% of studies having a prospective design. Methodological characterization (Domain 3) was poor across the studies (< 10% compliance), with suboptimal performance in the description of patient positioning, operator experience, and assessment of intra/inter-observer reliability. About three-fourth of the studies had a low ROB in Domain 4 (descriptive anatomy). Domain 5 (reporting of results) had a high ROB in approximately half of the studies, the majority reporting results in the form of central tendency measures. Significant deficiencies and heterogeneity were observed in the methodological quality of USG studies performed to-date for measurement of kidney size in children. We hereby provide a framework for the conducting such studies in future. PROSPERO (CRD42017071601).

Measurement of the Inter-Rater Reliability Rate Is Mandatory for Improving the Quality of a Medical Database: Experience with the Paulista Lung Cancer Registry.

PubMed

Lauricella, Leticia L; Costa, Priscila B; Salati, Michele; Pego-Fernandes, Paulo M; Terra, Ricardo M

2018-06-01

Database quality measurement should be considered a mandatory step to ensure an adequate level of confidence in data used for research and quality improvement. Several metrics have been described in the literature, but no standardized approach has been established. We aimed to describe a methodological approach applied to measure the quality and inter-rater reliability of a regional multicentric thoracic surgical database (Paulista Lung Cancer Registry). Data from the first 3 years of the Paulista Lung Cancer Registry underwent an audit process with 3 metrics: completeness, consistency, and inter-rater reliability. The first 2 methods were applied to the whole data set, and the last method was calculated using 100 cases randomized for direct auditing. Inter-rater reliability was evaluated using percentage of agreement between the data collector and auditor and through calculation of Cohen's κ and intraclass correlation. The overall completeness per section ranged from 0.88 to 1.00, and the overall consistency was 0.96. Inter-rater reliability showed many variables with high disagreement (>10%). For numerical variables, intraclass correlation was a better metric than inter-rater reliability. Cohen's κ showed that most variables had moderate to substantial agreement. The methodological approach applied to the Paulista Lung Cancer Registry showed that completeness and consistency metrics did not sufficiently reflect the real quality status of a database. The inter-rater reliability associated with κ and intraclass correlation was a better quality metric than completeness and consistency metrics because it could determine the reliability of specific variables used in research or benchmark reports. This report can be a paradigm for future studies of data quality measurement. Copyright © 2018 American College of Surgeons. Published by Elsevier Inc. All rights reserved.

Knowledge-based system verification and validation

NASA Technical Reports Server (NTRS)

Johnson, Sally C.

1990-01-01

The objective of this task is to develop and evaluate a methodology for verification and validation (V&V) of knowledge-based systems (KBS) for space station applications with high reliability requirements. The approach consists of three interrelated tasks. The first task is to evaluate the effectiveness of various validation methods for space station applications. The second task is to recommend requirements for KBS V&V for Space Station Freedom (SSF). The third task is to recommend modifications to the SSF to support the development of KBS using effectiveness software engineering and validation techniques. To accomplish the first task, three complementary techniques will be evaluated: (1) Sensitivity Analysis (Worchester Polytechnic Institute); (2) Formal Verification of Safety Properties (SRI International); and (3) Consistency and Completeness Checking (Lockheed AI Center). During FY89 and FY90, each contractor will independently demonstrate the user of his technique on the fault detection, isolation, and reconfiguration (FDIR) KBS or the manned maneuvering unit (MMU), a rule-based system implemented in LISP. During FY91, the application of each of the techniques to other knowledge representations and KBS architectures will be addressed. After evaluation of the results of the first task and examination of Space Station Freedom V&V requirements for conventional software, a comprehensive KBS V&V methodology will be developed and documented. Development of highly reliable KBS's cannot be accomplished without effective software engineering methods. Using the results of current in-house research to develop and assess software engineering methods for KBS's as well as assessment of techniques being developed elsewhere, an effective software engineering methodology for space station KBS's will be developed, and modification of the SSF to support these tools and methods will be addressed.

Development of an adaptive failure detection and identification system for detecting aircraft control element failures

NASA Technical Reports Server (NTRS)

Bundick, W. Thomas

1990-01-01

A methodology for designing a failure detection and identification (FDI) system to detect and isolate control element failures in aircraft control systems is reviewed. An FDI system design for a modified B-737 aircraft resulting from this methodology is also reviewed, and the results of evaluating this system via simulation are presented. The FDI system performed well in a no-turbulence environment, but it experienced an unacceptable number of false alarms in atmospheric turbulence. An adaptive FDI system, which adjusts thresholds and other system parameters based on the estimated turbulence level, was developed and evaluated. The adaptive system performed well over all turbulence levels simulated, reliably detecting all but the smallest magnitude partially-missing-surface failures.

An Evaluation of a Computer-Based Training on the Visual Analysis of Single-Subject Data

ERIC Educational Resources Information Center

Snyder, Katie

2013-01-01

Visual analysis is the primary method of analyzing data in single-subject methodology, which is the predominant research method used in the fields of applied behavior analysis and special education. Previous research on the reliability of visual analysis suggests that judges often disagree about what constitutes an intervention effect. Considering…

Accelerated Aging of the M119 Simulator

NASA Technical Reports Server (NTRS)

Bixon, Eric R.

2000-01-01

This paper addresses the storage requirement, shelf life, and the reliability of M119 Whistling Simulator. Experimental conditions have been determined and the data analysis has been completed for the accelerated testing of the system. A general methodology to evaluate the shelf life of the system as a function of the storage time, temperature, and relative humidity is discussed.

Assessment of biodeterioration for the screening of new wood preservatives : calculation of stiffness loss in rapid decay testing

Treesearch

Simon R. Przewloka; Douglas M. Crawford; Douglas R. Rammer; Donald L. Buckner; Bessie M. Woodward; Gan Li; Darrel D. Nicholas

2008-01-01

Demand for the development of environmentally benign wood preservatives has increased significantly. To reduce the evaluation time of prospective candidates, reliable accelerated decay methodologies are necessary for laboratory screening of potential preservatives. Ongoing research at Mississippi State University has focused upon utilizing custom built equipment to...

R. & D. in Psychometrics: Technical Reports on Latent Structure Models.

ERIC Educational Resources Information Center

Wilcox, Rand R.

This document contains three papers from the Methodology Project of the Center for the Study of Evaluation. Methods for characterizing test accuracy are reported in the first two papers. "Bounds on the K Out of N Reliability of a Test, and an Exact Test for Hierarchically Related Items" describes and illustrates how an extension of a…

MALDI-TOF Mass Spectrometry Is a Fast and Reliable Platform for Identification and Ecological Studies of Species from Family Rhizobiaceae

PubMed Central

Ferreira, Laura; Sánchez-Juanes, Fernando; García-Fraile, Paula; Rivas, Raúl; Mateos, Pedro F.; Martínez-Molina, Eustoquio; González-Buitrago, José Manuel; Velázquez, Encarna

2011-01-01

Family Rhizobiaceae includes fast growing bacteria currently arranged into three genera, Rhizobium, Ensifer and Shinella, that contain pathogenic, symbiotic and saprophytic species. The identification of these species is not possible on the basis of physiological or biochemical traits and should be based on sequencing of several genes. Therefore alternative methods are necessary for rapid and reliable identification of members from family Rhizobiaceae. In this work we evaluated the suitability of Matrix-Assisted Laser Desorption Ionization-Time-of-Flight Mass Spectrometry (MALDI-TOF MS) for this purpose. Firstly, we evaluated the capability of this methodology to differentiate among species of family Rhizobiaceae including those closely related and then we extended the database of MALDI Biotyper 2.0 including the type strains of 56 species from genera Rhizobium, Ensifer and Shinella. Secondly, we evaluated the identification potential of this methodology by using several strains isolated from different sources previously identified on the basis of their rrs, recA and atpD gene sequences. The 100% of these strains were correctly identified showing that MALDI-TOF MS is an excellent tool for identification of fast growing rhizobia applicable to large populations of isolates in ecological and taxonomic studies. PMID:21655291

Selection and Reporting of Statistical Methods to Assess Reliability of a Diagnostic Test: Conformity to Recommended Methods in a Peer-Reviewed Journal

PubMed Central

Park, Ji Eun; Han, Kyunghwa; Sung, Yu Sub; Chung, Mi Sun; Koo, Hyun Jung; Yoon, Hee Mang; Choi, Young Jun; Lee, Seung Soo; Kim, Kyung Won; Shin, Youngbin; An, Suah; Cho, Hyo-Min

2017-01-01

Objective To evaluate the frequency and adequacy of statistical analyses in a general radiology journal when reporting a reliability analysis for a diagnostic test. Materials and Methods Sixty-three studies of diagnostic test accuracy (DTA) and 36 studies reporting reliability analyses published in the Korean Journal of Radiology between 2012 and 2016 were analyzed. Studies were judged using the methodological guidelines of the Radiological Society of North America-Quantitative Imaging Biomarkers Alliance (RSNA-QIBA), and COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) initiative. DTA studies were evaluated by nine editorial board members of the journal. Reliability studies were evaluated by study reviewers experienced with reliability analysis. Results Thirty-one (49.2%) of the 63 DTA studies did not include a reliability analysis when deemed necessary. Among the 36 reliability studies, proper statistical methods were used in all (5/5) studies dealing with dichotomous/nominal data, 46.7% (7/15) of studies dealing with ordinal data, and 95.2% (20/21) of studies dealing with continuous data. Statistical methods were described in sufficient detail regarding weighted kappa in 28.6% (2/7) of studies and regarding the model and assumptions of intraclass correlation coefficient in 35.3% (6/17) and 29.4% (5/17) of studies, respectively. Reliability parameters were used as if they were agreement parameters in 23.1% (3/13) of studies. Reproducibility and repeatability were used incorrectly in 20% (3/15) of studies. Conclusion Greater attention to the importance of reporting reliability, thorough description of the related statistical methods, efforts not to neglect agreement parameters, and better use of relevant terminology is necessary. PMID:29089821

Reliability and Validity of Survey Instruments to Measure Work-Related Fatigue in the Emergency Medical Services Setting: A Systematic Review.

PubMed

Patterson, P Daniel; Weaver, Matthew D; Fabio, Anthony; Teasley, Ellen M; Renn, Megan L; Curtis, Brett R; Matthews, Margaret E; Kroemer, Andrew J; Xun, Xiaoshuang; Bizhanova, Zhadyra; Weiss, Patricia M; Sequeira, Denisse J; Coppler, Patrick J; Lang, Eddy S; Higgins, J Stephen

2018-02-15

This study sought to systematically search the literature to identify reliable and valid survey instruments for fatigue measurement in the Emergency Medical Services (EMS) occupational setting. A systematic review study design was used and searched six databases, including one website. The research question guiding the search was developed a priori and registered with the PROSPERO database of systematic reviews: "Are there reliable and valid instruments for measuring fatigue among EMS personnel?" (2016:CRD42016040097). The primary outcome of interest was criterion-related validity. Important outcomes of interest included reliability (e.g., internal consistency), and indicators of sensitivity and specificity. Members of the research team independently screened records from the databases. Full-text articles were evaluated by adapting the Bolster and Rourke system for categorizing findings of systematic reviews, and the rated data abstracted from the body of literature as favorable, unfavorable, mixed/inconclusive, or no impact. The Grading of Recommendations, Assessment, Development and Evaluation (GRADE) methodology was used to evaluate the quality of evidence. The search strategy yielded 1,257 unique records. Thirty-four unique experimental and non-experimental studies were determined relevant following full-text review. Nineteen studies reported on the reliability and/or validity of ten different fatigue survey instruments. Eighteen different studies evaluated the reliability and/or validity of four different sleepiness survey instruments. None of the retained studies reported sensitivity or specificity. Evidence quality was rated as very low across all outcomes. In this systematic review, limited evidence of the reliability and validity of 14 different survey instruments to assess the fatigue and/or sleepiness status of EMS personnel and related shift worker groups was identified.

COTS Ceramic Chip Capacitors: An Evaluation of the Parts and Assurance Methodologies

NASA Technical Reports Server (NTRS)

Brusse, Jay A.; Sampson, Michael J.

2004-01-01

Commercial-Off-The-Shelf (COTS) multilayer ceramic chip capacitors (MLCCs) are continually evolving to reduce physical size and increase volumetric efficiency. Designers of high reliability aerospace and military systems are attracted to these attributes of COTS MLCCs and would like to take advantage of them while maintaining the high standards for long-term reliable operation they are accustomed io when selecting military qualified established reliability (MIL-ER) MLCCs. However, MIL-ER MLCCs are not available in the full range of small chip sizes with high capacitance as found in today's COTS MLCCs. The objectives for this evaluation were to assess the long-term performance of small case size COTS MLCCs and to identify effective, lower-cost product assurance methodologies. Fifteen (15) lots of COTS X7R dielectric MLCCs from four (4) different manufacturers and two (2) MIL-ER BX dielectric MLCCs from two (2) of the same manufacturers were evaluated. Both 0805 and 0402 chip sizes were included. Several voltage ratings were tested ranging from a high of 50 volts to a low of 6.3 volts. The evaluation consisted of a comprehensive screening and qualification test program based upon MIL-PRF-55681 (i.e., voltage conditioning, thermal shock, moisture resistance, 2000-hour life test, etc.). In addition, several lot characterization tests were performed including Destructive Physical Analysis (DPA), Highly Accelerated Life Test (HALT) and Dielectric Voltage Breakdown Strength. The data analysis included a comparison of the 2000-hour life test results (used as a metric for long-term performance) relative to the screening and characterization test results. Results of this analysis indicate that the long-term life performance of COTS MLCCs is variable -- some lots perform well, some lots perform poorly. DPA and HALT were found to be promising lot characterization tests to identify substandard COTS MLCC lots prior to conducting more expensive screening and qualification tests. The results indicate that lot- specific screening and qualification are still recommended for high reliability applications. One significant and concerning observation is that MIL- type voltage conditioning (100 hours at twice rated voltage, 125 C) was not an effective screen in removing infant mortality parts for the particular lots of COTS MLCCs evaluated.

APPLICATION OF TRAVEL TIME RELIABILITY FOR PERFORMANCE ORIENTED OPERATIONAL PLANNING OF EXPRESSWAYS

NASA Astrophysics Data System (ADS)

Mehran, Babak; Nakamura, Hideki

Evaluation of impacts of congestion improvement scheme s on travel time reliability is very significant for road authorities since travel time reliability repr esents operational performance of expressway segments. In this paper, a methodology is presented to estimate travel tim e reliability prior to implementation of congestion relief schemes based on travel time variation modeling as a function of demand, capacity, weather conditions and road accident s. For subject expressway segmen ts, traffic conditions are modeled over a whole year considering demand and capacity as random variables. Patterns of demand and capacity are generated for each five minute interval by appl ying Monte-Carlo simulation technique, and accidents are randomly generated based on a model that links acci dent rate to traffic conditions. A whole year analysis is performed by comparing de mand and available capacity for each scenario and queue length is estimated through shockwave analysis for each time in terval. Travel times are estimated from refined speed-flow relationships developed for intercity expressways and buffer time index is estimated consequently as a measure of travel time reliability. For validation, estimated reliability indices are compared with measured values from empirical data, and it is shown that the proposed method is suitable for operational evaluation and planning purposes.

A comparison of kinesthetic-tactual and visual displays via a critical tracking task. [for aircraft control

NASA Technical Reports Server (NTRS)

Jagacinski, R. J.; Miller, D. P.; Gilson, R. D.

1979-01-01

The feasibility of using the critical tracking task to evaluate kinesthetic-tactual displays was examined. The test subjects were asked to control a first-order unstable system with a continuously decreasing time constant by using either visual or tactual unidimensional displays. The results indicate that the critical tracking task is both a feasible and a reliable methodology for assessing tactual tracking. Further, that the critical tracking methodology is as sensitive and valid a measure of tactual tracking as visual tracking is demonstrated by the approximately equal effects of quickening for the tactual and visual displays.

A Method for Evaluating the Safety Impacts of Air Traffic Automation

NASA Technical Reports Server (NTRS)

Kostiuk, Peter; Shapiro, Gerald; Hanson, Dave; Kolitz, Stephan; Leong, Frank; Rosch, Gene; Bonesteel, Charles

1998-01-01

This report describes a methodology for analyzing the safety and operational impacts of emerging air traffic technologies. The approach integrates traditional reliability models of the system infrastructure with models that analyze the environment within which the system operates, and models of how the system responds to different scenarios. Products of the analysis include safety measures such as predicted incident rates, predicted accident statistics, and false alarm rates; and operational availability data. The report demonstrates the methodology with an analysis of the operation of the Center-TRACON Automation System at Dallas-Fort Worth International Airport.

Results of a Demonstration Assessment of Passive System Reliability Utilizing the Reliability Method for Passive Systems (RMPS)

DOE Office of Scientific and Technical Information (OSTI.GOV)

Bucknor, Matthew; Grabaskas, David; Brunett, Acacia

2015-04-26

Advanced small modular reactor designs include many advantageous design features such as passively driven safety systems that are arguably more reliable and cost effective relative to conventional active systems. Despite their attractiveness, a reliability assessment of passive systems can be difficult using conventional reliability methods due to the nature of passive systems. Simple deviations in boundary conditions can induce functional failures in a passive system, and intermediate or unexpected operating modes can also occur. As part of an ongoing project, Argonne National Laboratory is investigating various methodologies to address passive system reliability. The Reliability Method for Passive Systems (RMPS), amore » systematic approach for examining reliability, is one technique chosen for this analysis. This methodology is combined with the Risk-Informed Safety Margin Characterization (RISMC) approach to assess the reliability of a passive system and the impact of its associated uncertainties. For this demonstration problem, an integrated plant model of an advanced small modular pool-type sodium fast reactor with a passive reactor cavity cooling system is subjected to a station blackout using RELAP5-3D. This paper discusses important aspects of the reliability assessment, including deployment of the methodology, the uncertainty identification and quantification process, and identification of key risk metrics.« less

Assessment of the Validity of the Research Diagnostic Criteria for Temporomandibular Disorders: Overview and Methodology

PubMed Central

Schiffman, Eric L.; Truelove, Edmond L.; Ohrbach, Richard; Anderson, Gary C.; John, Mike T.; List, Thomas; Look, John O.

2011-01-01

AIMS The purpose of the Research Diagnostic Criteria for Temporomandibular Disorders (RDC/TMD) Validation Project was to assess the diagnostic validity of this examination protocol. An overview is presented, including Axis I and II methodology and descriptive statistics for the study participant sample. This paper details the development of reliable methods to establish the reference standards for assessing criterion validity of the Axis I RDC/TMD diagnoses. Validity testing for the Axis II biobehavioral instruments was based on previously validated reference standards. METHODS The Axis I reference standards were based on the consensus of 2 criterion examiners independently performing a comprehensive history, clinical examination, and evaluation of imaging. Intersite reliability was assessed annually for criterion examiners and radiologists. Criterion exam reliability was also assessed within study sites. RESULTS Study participant demographics were comparable to those of participants in previous studies using the RDC/TMD. Diagnostic agreement of the criterion examiners with each other and with the consensus-based reference standards was excellent with all kappas ≥ 0.81, except for osteoarthrosis (moderate agreement, k = 0.53). Intrasite criterion exam agreement with reference standards was excellent (k ≥ 0.95). Intersite reliability of the radiologists for detecting computed tomography-disclosed osteoarthrosis and magnetic resonance imaging-disclosed disc displacement was good to excellent (k = 0.71 and 0.84, respectively). CONCLUSION The Validation Project study population was appropriate for assessing the reliability and validity of the RDC/TMD Axis I and II. The reference standards used to assess the validity of Axis I TMD were based on reliable and clinically credible methods. PMID:20213028

Overcoming the Challenges of Unstructured Data in Multi-site, Electronic Medical Record-based Abstraction

PubMed Central

Polnaszek, Brock; Gilmore-Bykovskyi, Andrea; Hovanes, Melissa; Roiland, Rachel; Ferguson, Patrick; Brown, Roger; Kind, Amy JH

2014-01-01

Background Unstructured data encountered during retrospective electronic medical record (EMR) abstraction has routinely been identified as challenging to reliably abstract, as this data is often recorded as free text, without limitations to format or structure. There is increased interest in reliably abstracting this type of data given its prominent role in care coordination and communication, yet limited methodological guidance exists. Objective As standard abstraction approaches resulted in sub-standard data reliability for unstructured data elements collected as part of a multi-site, retrospective EMR study of hospital discharge communication quality, our goal was to develop, apply and examine the utility of a phase-based approach to reliably abstract unstructured data. This approach is examined using the specific example of discharge communication for warfarin management. Research Design We adopted a “fit-for-use” framework to guide the development and evaluation of abstraction methods using a four step, phase-based approach including (1) team building, (2) identification of challenges, (3) adaptation of abstraction methods, and (4) systematic data quality monitoring. Measures Unstructured data elements were the focus of this study, including elements communicating steps in warfarin management (e.g., warfarin initiation) and medical follow-up (e.g., timeframe for follow-up). Results After implementation of the phase-based approach, inter-rater reliability for all unstructured data elements demonstrated kappas of ≥ 0.89 -- an average increase of + 0.25 for each unstructured data element. Conclusions As compared to standard abstraction methodologies, this phase-based approach was more time intensive, but did markedly increase abstraction reliability for unstructured data elements within multi-site EMR documentation. PMID:27624585

Evaluating test-retest reliability in patient-reported outcome measures for older people: A systematic review.

PubMed

Park, Myung Sook; Kang, Kyung Ja; Jang, Sun Joo; Lee, Joo Yun; Chang, Sun Ju

2018-03-01

This study aimed to evaluate the components of test-retest reliability including time interval, sample size, and statistical methods used in patient-reported outcome measures in older people and to provide suggestions on the methodology for calculating test-retest reliability for patient-reported outcomes in older people. This was a systematic literature review. MEDLINE, Embase, CINAHL, and PsycINFO were searched from January 1, 2000 to August 10, 2017 by an information specialist. This systematic review was guided by both the Preferred Reporting Items for Systematic Reviews and Meta-Analyses checklist and the guideline for systematic review published by the National Evidence-based Healthcare Collaborating Agency in Korea. The methodological quality was assessed by the Consensus-based Standards for the selection of health Measurement Instruments checklist box B. Ninety-five out of 12,641 studies were selected for the analysis. The median time interval for test-retest reliability was 14days, and the ratio of sample size for test-retest reliability to the number of items in each measure ranged from 1:1 to 1:4. The most frequently used statistical methods for continuous scores was intraclass correlation coefficients (ICCs). Among the 63 studies that used ICCs, 21 studies presented models for ICC calculations and 30 studies reported 95% confidence intervals of the ICCs. Additional analyses using 17 studies that reported a strong ICC (>0.09) showed that the mean time interval was 12.88days and the mean ratio of the number of items to sample size was 1:5.37. When researchers plan to assess the test-retest reliability of patient-reported outcome measures for older people, they need to consider an adequate time interval of approximately 13days and the sample size of about 5 times the number of items. Particularly, statistical methods should not only be selected based on the types of scores of the patient-reported outcome measures, but should also be described clearly in the studies that report the results of test-retest reliability. Copyright © 2017 Elsevier Ltd. All rights reserved.

Evaluating the Reliability and Impact of a Quality Assurance System for E-Learning Courseware

ERIC Educational Resources Information Center

Sung, Yao-Ting; Chang, Kuo-En; Yu, Wen-Cheng

2011-01-01

Assuring e-learning quality is of interest worldwide. This paper introduces the methods of e-learning courseware quality assurance (a quality certification system) adopted by the eLQSC (e-Learning Quality Service Centre) in Taiwan. A sequential/explanatory design with a mixed methodology was used to gather research data and conduct data analyses.…

Evaluation of a Propolis Water Extract Using a Reliable RP-HPLC Methodology and In Vitro and In Vivo Efficacy and Safety Characterisation

PubMed Central

Rocha, Bruno Alves; Bueno, Paula Carolina Pires; Vaz, Mirela Mara de Oliveira Lima Leite; Nascimento, Andresa Piacezzi; Ferreira, Nathália Ursoli; Moreno, Gabriela de Padua; Rodrigues, Marina Rezende; Costa-Machado, Ana Rita de Mello; Barizon, Edna Aparecida; Campos, Jacqueline Costa Lima; de Oliveira, Pollyanna Francielli; Acésio, Nathália de Oliveira; Martins, Sabrina de Paula Lima; Tavares, Denise Crispim; Berretta, Andresa Aparecida

2013-01-01

Since the beginning of propolis research, several groups have studied its antibacterial, antifungal, and antiviral properties. However, most of these studies have only employed propolis ethanolic extract (PEE) leading to little knowledge about the biological activities of propolis water extract (PWE). Based on this, in a previous study, we demonstrated the anti-inflammatory and immunomodulatory activities of PWE. In order to better understand the equilibrium between effectiveness and toxicity, which is essential for a new medicine, the characteristics of PWE were analyzed. We developed and validated an RP-HPLC method to chemically characterize PWE and PEE and evaluated the in vitro antioxidant/antimicrobial activity for both extracts and the safety of PWE via determining genotoxic potential using in vitro and in vivo mammalian micronucleus assays. We have concluded that the proposed analytical methodology was reliable, and both extracts showed similar chemical composition. The extracts presented antioxidant and antimicrobial effects, while PWE demonstrated higher antioxidant activity and more efficacious for the most of the microorganisms tested than PEE. Finally, PWE was shown to be safe using micronucleus assays. PMID:23710228

Integrated technology rotor/flight research rotor hub concept definition

NASA Technical Reports Server (NTRS)

Dixon, P. G. C.

1983-01-01

Two variations of the helicopter bearingless main rotor hub concept are proposed as bases for further development in the preliminary design phase of the Integrated Technology Rotor/Flight Research Rotor (ITR/FRR) program. This selection was the result of an evaluation of three bearingless hub concepts and two articulated hub concepts with elastomeric bearings. The characteristics of each concept were evaluated by means of simplified methodology. These characteristics included the assessment of stability, vulnerability, weight, drag, cost, stiffness, fatigue life, maintainability, and reliability.

A novel evaluation method for building construction project based on integrated information entropy with reliability theory.

PubMed

Bai, Xiao-ping; Zhang, Xi-wei

2013-01-01

Selecting construction schemes of the building engineering project is a complex multiobjective optimization decision process, in which many indexes need to be selected to find the optimum scheme. Aiming at this problem, this paper selects cost, progress, quality, and safety as the four first-order evaluation indexes, uses the quantitative method for the cost index, uses integrated qualitative and quantitative methodologies for progress, quality, and safety indexes, and integrates engineering economics, reliability theories, and information entropy theory to present a new evaluation method for building construction project. Combined with a practical case, this paper also presents detailed computing processes and steps, including selecting all order indexes, establishing the index matrix, computing score values of all order indexes, computing the synthesis score, sorting all selected schemes, and making analysis and decision. Presented method can offer valuable references for risk computing of building construction projects.

PROOF OF CONCEPT FOR A HUMAN RELIABILITY ANALYSIS METHOD FOR HEURISTIC USABILITY EVALUATION OF SOFTWARE

DOE Office of Scientific and Technical Information (OSTI.GOV)

Ronald L. Boring; David I. Gertman; Jeffrey C. Joe

2005-09-01

An ongoing issue within human-computer interaction (HCI) is the need for simplified or “discount” methods. The current economic slowdown has necessitated innovative methods that are results driven and cost effective. The myriad methods of design and usability are currently being cost-justified, and new techniques are actively being explored that meet current budgets and needs. Recent efforts in human reliability analysis (HRA) are highlighted by the ten-year development of the Standardized Plant Analysis Risk HRA (SPAR-H) method. The SPAR-H method has been used primarily for determining humancentered risk at nuclear power plants. The SPAR-H method, however, shares task analysis underpinnings withmore » HCI. Despite this methodological overlap, there is currently no HRA approach deployed in heuristic usability evaluation. This paper presents an extension of the existing SPAR-H method to be used as part of heuristic usability evaluation in HCI.« less

Designing trials for pressure ulcer risk assessment research: methodological challenges.

PubMed

Balzer, K; Köpke, S; Lühmann, D; Haastert, B; Kottner, J; Meyer, G

2013-08-01

For decades various pressure ulcer risk assessment scales (PURAS) have been developed and implemented into nursing practice despite uncertainty whether use of these tools helps to prevent pressure ulcers. According to current methodological standards, randomised controlled trials (RCTs) are required to conclusively determine the clinical efficacy and safety of this risk assessment strategy. In these trials, PURAS-aided risk assessment has to be compared to nurses' clinical judgment alone in terms of its impact on pressure ulcer incidence and adverse outcomes. However, RCTs evaluating diagnostic procedures are prone to specific risks of bias and threats to the statistical power which may challenge their validity and feasibility. This discussion paper critically reflects on the rigour and feasibility of experimental research needed to substantiate the clinical efficacy of PURAS-aided risk assessment. Based on reflections of the methodological literature, a critical appraisal of available trials on this subject and an analysis of a protocol developed for a methodologically robust cluster-RCT, this paper arrives at the following conclusions: First, available trials do not provide reliable estimates of the impact of PURAS-aided risk assessment on pressure ulcer incidence compared to nurses' clinical judgement alone due to serious risks of bias and insufficient sample size. Second, it seems infeasible to assess this impact by means of rigorous experimental studies since sample size would become extremely high if likely threats to validity and power are properly taken into account. Third, means of evidence linkages seem to currently be the most promising approaches for evaluating the clinical efficacy and safety of PURAS-aided risk assessment. With this kind of secondary research, the downstream effect of use of PURAS on pressure ulcer incidence could be modelled by combining best available evidence for single parts of this pathway. However, to yield reliable modelling results, more robust experimental research evaluating specific parts of the pressure ulcer risk assessment-prevention pathway is needed. Copyright © 2013 Elsevier Ltd. All rights reserved.

Metrics for Evaluating the Accuracy of Solar Power Forecasting: Preprint

DOE Office of Scientific and Technical Information (OSTI.GOV)

Zhang, J.; Hodge, B. M.; Florita, A.

2013-10-01

Forecasting solar energy generation is a challenging task due to the variety of solar power systems and weather regimes encountered. Forecast inaccuracies can result in substantial economic losses and power system reliability issues. This paper presents a suite of generally applicable and value-based metrics for solar forecasting for a comprehensive set of scenarios (i.e., different time horizons, geographic locations, applications, etc.). In addition, a comprehensive framework is developed to analyze the sensitivity of the proposed metrics to three types of solar forecasting improvements using a design of experiments methodology, in conjunction with response surface and sensitivity analysis methods. The resultsmore » show that the developed metrics can efficiently evaluate the quality of solar forecasts, and assess the economic and reliability impact of improved solar forecasting.« less

Quality and rigor of the concept mapping methodology: a pooled study analysis.

PubMed

Rosas, Scott R; Kane, Mary

2012-05-01

The use of concept mapping in research and evaluation has expanded dramatically over the past 20 years. Researchers in academic, organizational, and community-based settings have applied concept mapping successfully without the benefit of systematic analyses across studies to identify the features of a methodologically sound study. Quantitative characteristics and estimates of quality and rigor that may guide for future studies are lacking. To address this gap, we conducted a pooled analysis of 69 concept mapping studies to describe characteristics across study phases, generate specific indicators of validity and reliability, and examine the relationship between select study characteristics and quality indicators. Individual study characteristics and estimates were pooled and quantitatively summarized, describing the distribution, variation and parameters for each. In addition, variation in the concept mapping data collection in relation to characteristics and estimates was examined. Overall, results suggest concept mapping yields strong internal representational validity and very strong sorting and rating reliability estimates. Validity and reliability were consistently high despite variation in participation and task completion percentages across data collection modes. The implications of these findings as a practical reference to assess the quality and rigor for future concept mapping studies are discussed. Copyright © 2011 Elsevier Ltd. All rights reserved.

Multi-criteria decision assessments using Subjective Logic: Methodology and the case of urban water strategies

NASA Astrophysics Data System (ADS)

Moglia, Magnus; Sharma, Ashok K.; Maheepala, Shiroma

2012-07-01

SummaryPlanning of regional and urban water resources, and in particular with Integrated Urban Water Management approaches, often considers inter-relationships between human uses of water, the health of the natural environment as well as the cost of various management strategies. Decision makers hence typically need to consider a combination of social, environmental and economic goals. The types of strategies employed can include water efficiency measures, water sensitive urban design, stormwater management, or catchment management. Therefore, decision makers need to choose between different scenarios and to evaluate them against a number of criteria. This type of problem has a discipline devoted to it, i.e. Multi-Criteria Decision Analysis, which has often been applied in water management contexts. This paper describes the application of Subjective Logic in a basic Bayesian Network to a Multi-Criteria Decision Analysis problem. By doing this, it outlines a novel methodology that explicitly incorporates uncertainty and information reliability. The application of the methodology to a known case study context allows for exploration. By making uncertainty and reliability of assessments explicit, it allows for assessing risks of various options, and this may help in alleviating cognitive biases and move towards a well formulated risk management policy.

Slow Crack Growth and Fatigue Life Prediction of Ceramic Components Subjected to Variable Load History

NASA Technical Reports Server (NTRS)

Jadaan, Osama

2001-01-01

Present capabilities of the NASA CARES/Life (Ceramic Analysis and Reliability Evaluation of Structures/Life) code include probabilistic life prediction of ceramic components subjected to fast fracture, slow crack growth (stress corrosion), and cyclic fatigue failure modes. Currently, this code has the capability to compute the time-dependent reliability of ceramic structures subjected to simple time-dependent loading. For example, in slow crack growth (SCG) type failure conditions CARES/Life can handle the cases of sustained and linearly increasing time-dependent loads, while for cyclic fatigue applications various types of repetitive constant amplitude loads can be accounted for. In real applications applied loads are rarely that simple, but rather vary with time in more complex ways such as, for example, engine start up, shut down, and dynamic and vibrational loads. In addition, when a given component is subjected to transient environmental and or thermal conditions, the material properties also vary with time. The objective of this paper is to demonstrate a methodology capable of predicting the time-dependent reliability of components subjected to transient thermomechanical loads that takes into account the change in material response with time. In this paper, the dominant delayed failure mechanism is assumed to be SCG. This capability has been added to the NASA CARES/Life (Ceramic Analysis and Reliability Evaluation of Structures/Life) code, which has also been modified to have the ability of interfacing with commercially available FEA codes executed for transient load histories. An example involving a ceramic exhaust valve subjected to combustion cycle loads is presented to demonstrate the viability of this methodology and the CARES/Life program.

[Assessment of individual clinical outcomes: regarding an electroconvulsive therapy case].

PubMed

Iraurgi, Ioseba; Gorbeña, Susana; Martínez-Cubillos, Miren-Itxaso; Escribano, Margarita; Gómez-de-Maintenant, Pablo

2015-01-01

Evaluation of therapeutic results and of the efficacy and effectiveness of treatments is an area of interest both for clinicians and researchers. In general, randomized controlled trial designs have been used as the methodology of choice in which intergroup comparisons are made having a minimum of participants in each arm of treatment. However, these procedures are seldom used in daily clinical practice. Despite this fact, the evaluation of treatment results for a specific patient is important for the clinician in order to address if therapeutic goals have been accomplished both in terms of statistical significance and clinical meaningfulness. The methodology based on the reliable change index (Jacobson y Truax)1 provides an estimate of these two criteria. The goal of this article is to propose a procedure to apply the methodology with a single case study of a woman diagnosed with major depression and treated with electroconvulsive therapy. Copyright © 2014 SEP y SEPB. Published by Elsevier España. All rights reserved.

The quality of systematic reviews about interventions for refractive error can be improved: a review of systematic reviews.

PubMed

Mayo-Wilson, Evan; Ng, Sueko Matsumura; Chuck, Roy S; Li, Tianjing

2017-09-05

Systematic reviews should inform American Academy of Ophthalmology (AAO) Preferred Practice Pattern® (PPP) guidelines. The quality of systematic reviews related to the forthcoming Preferred Practice Pattern® guideline (PPP) Refractive Errors & Refractive Surgery is unknown. We sought to identify reliable systematic reviews to assist the AAO Refractive Errors & Refractive Surgery PPP. Systematic reviews were eligible if they evaluated the effectiveness or safety of interventions included in the 2012 PPP Refractive Errors & Refractive Surgery. To identify potentially eligible systematic reviews, we searched the Cochrane Eyes and Vision United States Satellite database of systematic reviews. Two authors identified eligible reviews and abstracted information about the characteristics and quality of the reviews independently using the Systematic Review Data Repository. We classified systematic reviews as "reliable" when they (1) defined criteria for the selection of studies, (2) conducted comprehensive literature searches for eligible studies, (3) assessed the methodological quality (risk of bias) of the included studies, (4) used appropriate methods for meta-analyses (which we assessed only when meta-analyses were reported), (5) presented conclusions that were supported by the evidence provided in the review. We identified 124 systematic reviews related to refractive error; 39 met our eligibility criteria, of which we classified 11 to be reliable. Systematic reviews classified as unreliable did not define the criteria for selecting studies (5; 13%), did not assess methodological rigor (10; 26%), did not conduct comprehensive searches (17; 44%), or used inappropriate quantitative methods (3; 8%). The 11 reliable reviews were published between 2002 and 2016. They included 0 to 23 studies (median = 9) and analyzed 0 to 4696 participants (median = 666). Seven reliable reviews (64%) assessed surgical interventions. Most systematic reviews of interventions for refractive error are low methodological quality. Following widely accepted guidance, such as Cochrane or Institute of Medicine standards for conducting systematic reviews, would contribute to improved patient care and inform future research.

Transcultural Adaptation of GRID Hamilton Rating Scale For Depression (GRID-HAMD) to Brazilian Portuguese and Evaluation of the Impact of Training Upon Inter-Rater Reliability.

PubMed

Henrique-Araújo, Ricardo; Osório, Flávia L; Gonçalves Ribeiro, Mônica; Soares Monteiro, Ivandro; Williams, Janet B W; Kalali, Amir; Alexandre Crippa, José; Oliveira, Irismar Reis De

2014-07-01

GRID-HAMD is a semi-structured interview guide developed to overcome flaws in HAM-D, and has been incorporated into an increasing number of studies. Carry out the transcultural adaptation of GRID-HAMD into the Brazilian Portuguese language, evaluate the inter-rater reliability of this instrument and the training impact upon this measure, and verify the raters' opinions of said instrument. The transcultural adaptation was conducted by appropriate methodology. The measurement of inter-rater reliability was done by way of videos that were evaluated by 85 professionals before and after training for the use of this instrument. The intraclass correlation coefficient (ICC) remained between 0.76 and 0.90 for GRID-HAMD-21 and between 0.72 and 0.91 for GRID-HAMD-17. The training did not have an impact on the ICC, except for a few groups of participants with a lower level of experience. Most of the participants showed high acceptance of GRID-HAMD, when compared to other versions of HAM-D. The scale presented adequate inter-rater reliability even before training began. Training did not have an impact on this measure, except for a few groups with less experience. GRID-HAMD received favorable opinions from most of the participants.

Developing and validating a nutrition knowledge questionnaire: key methods and considerations.

PubMed

Trakman, Gina Louise; Forsyth, Adrienne; Hoye, Russell; Belski, Regina

2017-10-01

To outline key statistical considerations and detailed methodologies for the development and evaluation of a valid and reliable nutrition knowledge questionnaire. Literature on questionnaire development in a range of fields was reviewed and a set of evidence-based guidelines specific to the creation of a nutrition knowledge questionnaire have been developed. The recommendations describe key qualitative methods and statistical considerations, and include relevant examples from previous papers and existing nutrition knowledge questionnaires. Where details have been omitted for the sake of brevity, the reader has been directed to suitable references. We recommend an eight-step methodology for nutrition knowledge questionnaire development as follows: (i) definition of the construct and development of a test plan; (ii) generation of the item pool; (iii) choice of the scoring system and response format; (iv) assessment of content validity; (v) assessment of face validity; (vi) purification of the scale using item analysis, including item characteristics, difficulty and discrimination; (vii) evaluation of the scale including its factor structure and internal reliability, or Rasch analysis, including assessment of dimensionality and internal reliability; and (viii) gathering of data to re-examine the questionnaire's properties, assess temporal stability and confirm construct validity. Several of these methods have previously been overlooked. The measurement of nutrition knowledge is an important consideration for individuals working in the nutrition field. Improved methods in the development of nutrition knowledge questionnaires, such as the use of factor analysis or Rasch analysis, will enable more confidence in reported measures of nutrition knowledge.

Independent predictors of reliability between full time employee-dependent acquisition of functional outcomes compared to non-full time employee-dependent methodologies: a prospective single institutional study.

PubMed

Adogwa, Owoicho; Elsamadicy, Aladine A; Cheng, Joseph; Bagley, Carlos

2016-03-01

The prospective acquisition of reliable patient-reported outcomes (PROs) measures demonstrating the effectiveness of spine surgery, or lack thereof, remains a challenge. The aims of this study are to compare the reliability of functional outcomes metrics obtained using full time employee (FTE) vs. non-FTE-dependent methodologies and to determine the independent predictors of response reliability using non FTE-dependent methodologies. One hundred and nineteen adult patients (male: 65, female: 54) undergoing one- and two-level lumbar fusions at Duke University Medical Center were enrolled in this prospective study. Enrollment criteria included available demographic, clinical and baseline functional outcomes data. All patients were administered two similar sets of baseline questionnaires-(I) phone interviews (FTE-dependent) and (II) hardcopy in clinic (patient self-survey, non-FTE-dependent). All patients had at least a two-week washout period between phone interviews and in-clinic self-surveys to minimize effect of recall. Questionnaires included Oswestry disability index (ODI) and Visual Analog Back and Leg Pain Scale (VAS-BP/LP). Reliability was assessed by the degree to which patient responses to baseline questionnaires differed between both time points. About 26.89% had a history an anxiety disorder and 28.57% reported a history of depression. At least 97.47% of patients had a High School Diploma or GED, with 49.57% attaining a 4-year college degree or post-graduate degree. 29.94% reported full-time employment and 14.28% were on disability. There was a very high correlation between baseline PRO's data captured between FTE-dependent compared to non-FTE-dependent methodologies (r=0.89). In a multivariate logistic regression model, the absence of anxiety and depression, higher levels of education (college or greater) and full-time employment, were independently associated with high response reliability using non-FTE-dependent methodologies. Our study suggests that capturing health-related quality of life data using non-FTE-dependent methodologies is highly reliable and maybe a more cost-effective alternative. Well-educated patients who are employed full-time appear to be the most reliable.

Integrating reliability and maintainability into a concurrent engineering environment

NASA Astrophysics Data System (ADS)

Phillips, Clifton B.; Peterson, Robert R.

1993-02-01

This paper describes the results of a reliability and maintainability study conducted at the University of California, San Diego and supported by private industry. Private industry thought the study was important and provided the university access to innovative tools under cooperative agreement. The current capability of reliability and maintainability tools and how they fit into the design process is investigated. The evolution of design methodologies leading up to today's capability is reviewed for ways to enhance the design process while keeping cost under control. A method for measuring the consequences of reliability and maintainability policy for design configurations in an electronic environment is provided. The interaction of selected modern computer tool sets is described for reliability, maintainability, operations, and other elements of the engineering design process. These tools provide a robust system evaluation capability that brings life cycle performance improvement information to engineers and their managers before systems are deployed, and allow them to monitor and track performance while it is in operation.

Novel Strength Test Battery to Permit Evidence-Based Paralympic Classification

PubMed Central

Beckman, Emma M.; Newcombe, Peter; Vanlandewijck, Yves; Connick, Mark J.; Tweedy, Sean M.

2014-01-01

Abstract Ordinal-scale strength assessment methods currently used in Paralympic athletics classification prevent the development of evidence-based classification systems. This study evaluated a battery of 7, ratio-scale, isometric tests with the aim of facilitating the development of evidence-based methods of classification. This study aimed to report sex-specific normal performance ranges, evaluate test–retest reliability, and evaluate the relationship between the measures and body mass. Body mass and strength measures were obtained from 118 participants—63 males and 55 females—ages 23.2 years ± 3.7 (mean ± SD). Seventeen participants completed the battery twice to evaluate test–retest reliability. The body mass–strength relationship was evaluated using Pearson correlations and allometric exponents. Conventional patterns of force production were observed. Reliability was acceptable (mean intraclass correlation = 0.85). Eight measures had moderate significant correlations with body size (r = 0.30–61). Allometric exponents were higher in males than in females (mean 0.99 vs 0.30). Results indicate that this comprehensive and parsimonious battery is an important methodological advance because it has psychometric properties critical for the development of evidence-based classification. Measures were interrelated with body size, indicating further research is required to determine whether raw measures require normalization in order to be validly applied in classification. PMID:25068950

Methodological quality and reporting of systematic reviews in hand and wrist pathology.

PubMed

Wasiak, J; Shen, A Y; Ware, R; O'Donohoe, T J; Faggion, C M

2017-10-01

The objective of this study was to assess methodological and reporting quality of systematic reviews in hand and wrist pathology. MEDLINE, EMBASE and Cochrane Library were searched from inception to November 2016 for relevant studies. Reporting quality was evaluated using Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and methodological quality using a measurement tool to assess systematic reviews, the Assessment of Multiple Systematic Reviews (AMSTAR). Descriptive statistics and linear regression were used to identify features associated with improved methodological quality. A total of 91 studies were included in the analysis. Most reviews inadequately reported PRISMA items regarding study protocol, search strategy and bias and AMSTAR items regarding protocol, publication bias and funding. Systematic reviews published in a plastics journal, or which included more authors, were associated with higher AMSTAR scores. A large proportion of systematic reviews within hand and wrist pathology literature score poorly with validated methodological assessment tools, which may affect the reliability of their conclusions. I.

Breast Shape Analysis With Curvature Estimates and Principal Component Analysis for Cosmetic and Reconstructive Breast Surgery.

PubMed

Catanuto, Giuseppe; Taher, Wafa; Rocco, Nicola; Catalano, Francesca; Allegra, Dario; Milotta, Filippo Luigi Maria; Stanco, Filippo; Gallo, Giovanni; Nava, Maurizio Bruno

2018-03-20

Breast shape is defined utilizing mainly qualitative assessment (full, flat, ptotic) or estimates, such as volume or distances between reference points, that cannot describe it reliably. We will quantitatively describe breast shape with two parameters derived from a statistical methodology denominated principal component analysis (PCA). We created a heterogeneous dataset of breast shapes acquired with a commercial infrared 3-dimensional scanner on which PCA was performed. We plotted on a Cartesian plane the two highest values of PCA for each breast (principal components 1 and 2). Testing of the methodology on a preoperative and postoperative surgical case and test-retest was performed by two operators. The first two principal components derived from PCA are able to characterize the shape of the breast included in the dataset. The test-retest demonstrated that different operators are able to obtain very similar values of PCA. The system is also able to identify major changes in the preoperative and postoperative stages of a two-stage reconstruction. Even minor changes were correctly detected by the system. This methodology can reliably describe the shape of a breast. An expert operator and a newly trained operator can reach similar results in a test/re-testing validation. Once developed and after further validation, this methodology could be employed as a good tool for outcome evaluation, auditing, and benchmarking.

Characterizing the reliability of a bioMEMS-based cantilever sensor

NASA Astrophysics Data System (ADS)

Bhalerao, Kaustubh D.

2004-12-01

The cantilever-based BioMEMS sensor represents one instance from many competing ideas of biosensor technology based on Micro Electro Mechanical Systems. The advancement of BioMEMS from laboratory-scale experiments to applications in the field will require standardization of their components and manufacturing procedures as well as frameworks to evaluate their performance. Reliability, the likelihood with which a system performs its intended task, is a compact mathematical description of its performance. The mathematical and statistical foundation of systems-reliability has been applied to the cantilever-based BioMEMS sensor. The sensor is designed to detect one aspect of human ovarian cancer, namely the over-expression of the folate receptor surface protein (FR-alpha). Even as the application chosen is clinically motivated, the objective of this study was to demonstrate the underlying systems-based methodology used to design, develop and evaluate the sensor. The framework development can be readily extended to other BioMEMS-based devices for disease detection and will have an impact in the rapidly growing $30 bn industry. The Unified Modeling Language (UML) is a systems-based framework for design and development of object-oriented information systems which has potential application for use in systems designed to interact with biological environments. The UML has been used to abstract and describe the application of the biosensor, to identify key components of the biosensor, and the technology needed to link them together in a coherent manner. The use of the framework is also demonstrated in computation of system reliability from first principles as a function of the structure and materials of the biosensor. The outcomes of applying the systems-based framework to the study are the following: (1) Characterizing the cantilever-based MEMS device for disease (cell) detection. (2) Development of a novel chemical interface between the analyte and the sensor that provides a degree of selectivity towards the disease. (3) Demonstrating the performance and measuring the reliability of the biosensor prototype, and (4) Identification of opportunities in technological development in order to further refine the proposed biosensor. Application of the methodology to design develop and evaluate the reliability of BioMEMS devices will be beneficial in the streamlining the growth of the BioMEMS industry, while providing a decision-support tool in comparing and adopting suitable technologies from available competing options.

Space Station Freedom power - A reliability, availability, and maintainability assessment of the proposed Space Station Freedom electric power system

NASA Technical Reports Server (NTRS)

Turnquist, S. R.; Twombly, M.; Hoffman, D.

1989-01-01

A preliminary reliability, availability, and maintainability (RAM) analysis of the proposed Space Station Freedom electric power system (EPS) was performed using the unit reliability, availability, and maintainability (UNIRAM) analysis methodology. Orbital replacement units (ORUs) having the most significant impact on EPS availability measures were identified. Also, the sensitivity of the EPS to variations in ORU RAM data was evaluated for each ORU. Estimates were made of average EPS power output levels and availability of power to the core area of the space station. The results of assessments of the availability of EPS power and power to load distribution points in the space stations are given. Some highlights of continuing studies being performed to understand EPS availability considerations are presented.

Delirium diagnosis methodology used in research: a survey-based study.

PubMed

Neufeld, Karin J; Nelliot, Archana; Inouye, Sharon K; Ely, E Wesley; Bienvenu, O Joseph; Lee, Hochang Benjamin; Needham, Dale M

2014-12-01

To describe methodology used to diagnose delirium in research studies evaluating delirium detection tools. The authors used a survey to address reference rater methodology for delirium diagnosis, including rater characteristics, sources of patient information, and diagnostic process, completed via web or telephone interview according to respondent preference. Participants were authors of 39 studies included in three recent systematic reviews of delirium detection instruments in hospitalized patients. Authors from 85% (N = 33) of the 39 eligible studies responded to the survey. The median number of raters per study was 2.5 (interquartile range: 2-3); 79% were physicians. The raters' median duration of clinical experience with delirium diagnosis was 7 years (interquartile range: 4-10), with 5% having no prior clinical experience. Inter-rater reliability was evaluated in 70% of studies. Cognitive tests and delirium detection tools were used in the delirium reference rating process in 61% (N = 21) and 45% (N = 15) of studies, respectively, with 33% (N = 11) using both and 27% (N = 9) using neither. When patients were too drowsy or declined to participate in delirium evaluation, 70% of studies (N = 23) used all available information for delirium diagnosis, whereas 15% excluded such patients. Significant variability exists in reference standard methods for delirium diagnosis in published research. Increasing standardization by documenting inter-rater reliability, using standardized cognitive and delirium detection tools, incorporating diagnostic expert consensus panels, and using all available information in patients declining or unable to participate with formal testing may help advance delirium research by increasing consistency of case detection and improving generalizability of research results. Copyright © 2014 American Association for Geriatric Psychiatry. Published by Elsevier Inc. All rights reserved.

Deliberate Imagery Practice: The Reliability of Using a Retrospective Recall Methodology

ERIC Educational Resources Information Center

Cumming, Jennifer; Hall, Craig; Starkes, Janet L.

2005-01-01

This study examined the reliability of a retrospective recall methodology for providing evidence of deliberate imagery practice. A secondary purpose was to determine which imagery activities constituted the sport-specific definition of deliberate practice (Starkes, Deakin, Allard, Hodges, & Hayes, 1996). Ninety-three Canadian athletes from one…

The role of pragmatism in explaining heterogeneity in meta-analyses of randomised trials: a protocol for a cross-sectional methodological review.

PubMed

Aves, Theresa; Allan, Katherine S; Lawson, Daeria; Nieuwlaat, Robby; Beyene, Joseph; Mbuagbaw, Lawrence

2017-09-03

There has been increasing interest in pragmatic trials methodology. As a result, tools such as the Pragmatic-Explanatory Continuum Indicator Summary-2 (PRECIS-2) are being used prospectively to help researchers design randomised controlled trials (RCTs) within the pragmatic-explanatory continuum. There may be value in applying the PRECIS-2 tool retrospectively in a systematic review setting as it could provide important information about how to pool data based on the degree of pragmatism. To investigate the role of pragmatism as a source of heterogeneity in systematic reviews by (1) identifying systematic reviews with meta-analyses of RCTs that have moderate to high heterogeneity, (2) applying PRECIS-2 to RCTs of systematic reviews, (3) evaluating the inter-rater reliability of PRECIS-2, (4) determining how much of this heterogeneity may be explained by pragmatism. A cross-sectional methodological review will be conducted on systematic reviews of RCTs published in the Cochrane Library from 1 January 2014 to 1 January 2017. Included systematic reviews will have a minimum of 10 RCTs in the meta-analysis of the primary outcome and moderate to substantial heterogeneity (I 2 ≥50%). Of the eligible systematic reviews, a random selection of 10 will be included for quantitative evaluation. In each systematic review, RCTs will be scored using the PRECIS-2 tool, in duplicate. Agreement between raters will be measured using the intraclass correlation coefficient. Subgroup analyses and meta-regression will be used to evaluate how much variability in the primary outcome may be due to pragmatism. This review will be among the first to evaluate the PRECIS-2 tool in a systematic review setting. Results from this research will provide inter-rater reliability information about PRECIS-2 and may be used to provide methodological guidance when dealing with pragmatism in systematic reviews and subgroup considerations. On completion, this review will be submitted to a peer-reviewed journal for publication. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2017. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

Statistics-related and reliability-physics-related failure processes in electronics devices and products

NASA Astrophysics Data System (ADS)

Suhir, E.

2014-05-01

The well known and widely used experimental reliability "passport" of a mass manufactured electronic or a photonic product — the bathtub curve — reflects the combined contribution of the statistics-related and reliability-physics (physics-of-failure)-related processes. When time progresses, the first process results in a decreasing failure rate, while the second process associated with the material aging and degradation leads to an increased failure rate. An attempt has been made in this analysis to assess the level of the reliability physics-related aging process from the available bathtub curve (diagram). It is assumed that the products of interest underwent the burn-in testing and therefore the obtained bathtub curve does not contain the infant mortality portion. It has been also assumed that the two random processes in question are statistically independent, and that the failure rate of the physical process can be obtained by deducting the theoretically assessed statistical failure rate from the bathtub curve ordinates. In the carried out numerical example, the Raleigh distribution for the statistical failure rate was used, for the sake of a relatively simple illustration. The developed methodology can be used in reliability physics evaluations, when there is a need to better understand the roles of the statistics-related and reliability-physics-related irreversible random processes in reliability evaluations. The future work should include investigations on how powerful and flexible methods and approaches of the statistical mechanics can be effectively employed, in addition to reliability physics techniques, to model the operational reliability of electronic and photonic products.

Systematic review of communication partner training in aphasia: methodological quality.

PubMed

Cherney, Leora R; Simmons-Mackie, Nina; Raymer, Anastasia; Armstrong, Elizabeth; Holland, Audrey

2013-10-01

Twenty-three studies identified from a previous systematic review examining the effects of communication partner training on persons with aphasia and their communication partners were evaluated for methodological quality. Two reviewers rated the studies on defined methodological quality criteria relevant to each study design. There were 11 group studies, seven single-subject participant design studies, and five qualitative studies. Quality scores were derived for each study. The mean inter-rater reliability of scores for each study design ranged from 85-93%, with Cohen's Kappa indicating substantial agreement between raters. Methodological quality of research on communication partner training in aphasia was highly varied. Overall, group studies employed the least rigorous methodology as compared to single subject and qualitative research. Only two of 11 group studies complied with more than half of the quality criteria. No group studies reported therapist blinding and only one group study reported participant blinding. Across all types of studies, the criterion of treatment fidelity was most commonly omitted. Failure to explicitly report certain methodological quality criteria may account for low ratings. Using methodological rating scales specific to the type of study design may help improve the methodological quality of aphasia treatment studies, including those on communication partner training.

What Have the Difference Scores Not Been Telling Us? A Critique of the Use of Self-Ideal Discrepancy in the Assessment of Body Image and Evaluation of an Alternative Data-Analytic Framework

ERIC Educational Resources Information Center

Cafri, Guy; van den Berg, Patricia; Brannick, Michael T.

2010-01-01

Difference scores are often used as a means of assessing body image satisfaction using silhouette scales. Unfortunately, difference scores suffer from numerous potential methodological problems, including reduced reliability, ambiguity, confounded effects, untested constraints, and dimensional reduction. In this article, the methodological…

Advanced flight control system study

NASA Technical Reports Server (NTRS)

Hartmann, G. L.; Wall, J. E., Jr.; Rang, E. R.; Lee, H. P.; Schulte, R. W.; Ng, W. K.

1982-01-01

A fly by wire flight control system architecture designed for high reliability includes spare sensor and computer elements to permit safe dispatch with failed elements, thereby reducing unscheduled maintenance. A methodology capable of demonstrating that the architecture does achieve the predicted performance characteristics consists of a hierarchy of activities ranging from analytical calculations of system reliability and formal methods of software verification to iron bird testing followed by flight evaluation. Interfacing this architecture to the Lockheed S-3A aircraft for flight test is discussed. This testbed vehicle can be expanded to support flight experiments in advanced aerodynamics, electromechanical actuators, secondary power systems, flight management, new displays, and air traffic control concepts.

Practical no-gold-standard evaluation framework for quantitative imaging methods: application to lesion segmentation in positron emission tomography

PubMed Central

Jha, Abhinav K.; Mena, Esther; Caffo, Brian; Ashrafinia, Saeed; Rahmim, Arman; Frey, Eric; Subramaniam, Rathan M.

2017-01-01

Abstract. Recently, a class of no-gold-standard (NGS) techniques have been proposed to evaluate quantitative imaging methods using patient data. These techniques provide figures of merit (FoMs) quantifying the precision of the estimated quantitative value without requiring repeated measurements and without requiring a gold standard. However, applying these techniques to patient data presents several practical difficulties including assessing the underlying assumptions, accounting for patient-sampling-related uncertainty, and assessing the reliability of the estimated FoMs. To address these issues, we propose statistical tests that provide confidence in the underlying assumptions and in the reliability of the estimated FoMs. Furthermore, the NGS technique is integrated within a bootstrap-based methodology to account for patient-sampling-related uncertainty. The developed NGS framework was applied to evaluate four methods for segmenting lesions from F-Fluoro-2-deoxyglucose positron emission tomography images of patients with head-and-neck cancer on the task of precisely measuring the metabolic tumor volume. The NGS technique consistently predicted the same segmentation method as the most precise method. The proposed framework provided confidence in these results, even when gold-standard data were not available. The bootstrap-based methodology indicated improved performance of the NGS technique with larger numbers of patient studies, as was expected, and yielded consistent results as long as data from more than 80 lesions were available for the analysis. PMID:28331883

Medicine, methodology, and values: trade-offs in clinical science and practice.

PubMed

Ho, Vincent K Y

2011-01-01

The current guidelines of evidence-based medicine (EBM) presuppose that clinical research and clinical practice should advance from rigorous scientific tests as they generate reliable, value-free knowledge. Under this presupposition, hypotheses postulated by doctors and patients in the process of their decision making are preferably tested in randomized clinical trials (RCTs), and in systematic reviews and meta-analyses summarizing outcomes from multiple RCTs. Since testing under this scheme is predominantly focused on the criteria of generality and precision achieved through methodological rigor, at the cost of the criterion of realism, translating test results to clinical practice is often problematic. Choices concerning which methodological criteria should have priority are inevitable, however, as clinical trials, and scientific research in general, cannot meet all relevant criteria at the same time. Since these choices may be informed by considerations external to science, we must acknowledge that science cannot be value-free in a strict sense, and this invites a more prominent role for value-laden considerations in evaluating clinical research. The urgency for this becomes even more apparent when we consider the important yet implicit role of scientific theories in EBM, which may also be subjected to methodological evaluation and for which selectiveness in methodological focus is likewise inevitable.

UNIX-based operating systems robustness evaluation

NASA Technical Reports Server (NTRS)

Chang, Yu-Ming

1996-01-01

Robust operating systems are required for reliable computing. Techniques for robustness evaluation of operating systems not only enhance the understanding of the reliability of computer systems, but also provide valuable feed- back to system designers. This thesis presents results from robustness evaluation experiments on five UNIX-based operating systems, which include Digital Equipment's OSF/l, Hewlett Packard's HP-UX, Sun Microsystems' Solaris and SunOS, and Silicon Graphics' IRIX. Three sets of experiments were performed. The methodology for evaluation tested (1) the exception handling mechanism, (2) system resource management, and (3) system capacity under high workload stress. An exception generator was used to evaluate the exception handling mechanism of the operating systems. Results included exit status of the exception generator and the system state. Resource management techniques used by individual operating systems were tested using programs designed to usurp system resources such as physical memory and process slots. Finally, the workload stress testing evaluated the effect of the workload on system performance by running a synthetic workload and recording the response time of local and remote user requests. Moderate to severe performance degradations were observed on the systems under stress.

Integrated Design Methodology for Highly Reliable Liquid Rocket Engine

NASA Astrophysics Data System (ADS)

Kuratani, Naoshi; Aoki, Hiroshi; Yasui, Masaaki; Kure, Hirotaka; Masuya, Goro

The Integrated Design Methodology is strongly required at the conceptual design phase to achieve the highly reliable space transportation systems, especially the propulsion systems, not only in Japan but also all over the world in these days. Because in the past some catastrophic failures caused some losses of mission and vehicle (LOM/LOV) at the operational phase, moreover did affect severely the schedule delays and cost overrun at the later development phase. Design methodology for highly reliable liquid rocket engine is being preliminarily established and investigated in this study. The sensitivity analysis is systematically performed to demonstrate the effectiveness of this methodology, and to clarify and especially to focus on the correlation between the combustion chamber, turbopump and main valve as main components. This study describes the essential issues to understand the stated correlations, the need to apply this methodology to the remaining critical failure modes in the whole engine system, and the perspective on the engine development in the future.

Design verification of SIFT

NASA Technical Reports Server (NTRS)

Moser, Louise; Melliar-Smith, Michael; Schwartz, Richard

1987-01-01

A SIFT reliable aircraft control computer system, designed to meet the ultrahigh reliability required for safety critical flight control applications by use of processor replications and voting, was constructed for SRI, and delivered to NASA Langley for evaluation in the AIRLAB. To increase confidence in the reliability projections for SIFT, produced by a Markov reliability model, SRI constructed a formal specification, defining the meaning of reliability in the context of flight control. A further series of specifications defined, in increasing detail, the design of SIFT down to pre- and post-conditions on Pascal code procedures. Mechanically checked mathematical proofs were constructed to demonstrate that the more detailed design specifications for SIFT do indeed imply the formal reliability requirement. An additional specification defined some of the assumptions made about SIFT by the Markov model, and further proofs were constructed to show that these assumptions, as expressed by that specification, did indeed follow from the more detailed design specifications for SIFT. This report provides an outline of the methodology used for this hierarchical specification and proof, and describes the various specifications and proofs performed.

NASA PC software evaluation project

NASA Technical Reports Server (NTRS)

Dominick, Wayne D. (Editor); Kuan, Julie C.

1986-01-01

The USL NASA PC software evaluation project is intended to provide a structured framework for facilitating the development of quality NASA PC software products. The project will assist NASA PC development staff to understand the characteristics and functions of NASA PC software products. Based on the results of the project teams' evaluations and recommendations, users can judge the reliability, usability, acceptability, maintainability and customizability of all the PC software products. The objective here is to provide initial, high-level specifications and guidelines for NASA PC software evaluation. The primary tasks to be addressed in this project are as follows: to gain a strong understanding of what software evaluation entails and how to organize a structured software evaluation process; to define a structured methodology for conducting the software evaluation process; to develop a set of PC software evaluation criteria and evaluation rating scales; and to conduct PC software evaluations in accordance with the identified methodology. Communication Packages, Network System Software, Graphics Support Software, Environment Management Software, General Utilities. This report represents one of the 72 attachment reports to the University of Southwestern Louisiana's Final Report on NASA Grant NGT-19-010-900. Accordingly, appropriate care should be taken in using this report out of context of the full Final Report.

Diagnostic methods for assessing maxillary skeletal and dental transverse deficiencies: A systematic review

PubMed Central

Sawchuk, Dena; Currie, Kris; Vich, Manuel Lagravere; Palomo, Juan Martin

2016-01-01

Objective To evaluate the accuracy and reliability of the diagnostic tools available for assessing maxillary transverse deficiencies. Methods An electronic search of three databases was performed from their date of establishment to April 2015, with manual searching of reference lists of relevant articles. Articles were considered for inclusion if they reported the accuracy or reliability of a diagnostic method or evaluation technique for maxillary transverse dimensions in mixed or permanent dentitions. Risk of bias was assessed in the included articles, using the Quality Assessment of Diagnostic Accuracy Studies tool-2. Results Nine articles were selected. The studies were heterogeneous, with moderate to low methodological quality, and all had a high risk of bias. Four suggested that the use of arch width prediction indices with dental cast measurements is unreliable for use in diagnosis. Frontal cephalograms derived from cone-beam computed tomography (CBCT) images were reportedly more reliable for assessing intermaxillary transverse discrepancies than posteroanterior cephalograms. Two studies proposed new three-dimensional transverse analyses with CBCT images that were reportedly reliable, but have not been validated for clinical sensitivity or specificity. No studies reported sensitivity, specificity, positive or negative predictive values or likelihood ratios, or ROC curves of the methods for the diagnosis of transverse deficiencies. Conclusions Current evidence does not enable solid conclusions to be drawn, owing to a lack of reliable high quality diagnostic studies evaluating maxillary transverse deficiencies. CBCT images are reportedly more reliable for diagnosis, but further validation is required to confirm CBCT's accuracy and diagnostic superiority. PMID:27668196

Test-treatment RCTs are susceptible to bias: a review of the methodological quality of randomized trials that evaluate diagnostic tests.

PubMed

Ferrante di Ruffano, Lavinia; Dinnes, Jacqueline; Sitch, Alice J; Hyde, Chris; Deeks, Jonathan J

2017-02-24

There is a growing recognition for the need to expand our evidence base for the clinical effectiveness of diagnostic tests. Many international bodies are calling for diagnostic randomized controlled trials to provide the most rigorous evidence of impact to patient health. Although these so-called test-treatment RCTs are very challenging to undertake due to their methodological complexity, they have not been subjected to a systematic appraisal of their methodological quality. The extent to which these trials may be producing biased results therefore remains unknown. We set out to address this issue by conducting a methodological review of published test-treatment trials to determine how often they implement adequate methods to limit bias and safeguard the validity of results. We ascertained all test-treatment RCTs published 2004-2007, indexed in CENTRAL, including RCTs which randomized patients to diagnostic tests and measured patient outcomes after treatment. Tests used for screening, monitoring or prognosis were excluded. We assessed adequacy of sequence generation, allocation concealment and intention-to-treat, appropriateness of primary analyses, blinding and reporting of power calculations, and extracted study characteristics including the primary outcome. One hundred three trials compared 105 control with 119 experimental interventions, and reported 150 primary outcomes. Randomization and allocation concealment were adequate in 57 and 37% of trials. Blinding was uncommon (patients 5%, clinicians 4%, outcome assessors 21%), as was an adequate intention-to-treat analysis (29%). Overall 101 of 103 trials (98%) were at risk of bias, as judged using standard Cochrane criteria. Test-treatment trials are particularly susceptible to attrition and inadequate primary analyses, lack of blinding and under-powering. These weaknesses pose much greater methodological and practical challenges to conducting reliable RCT evaluations of test-treatment strategies than standard treatment interventions. We suggest a cautious approach that first examines whether a test-treatment intervention can accommodate the methodological safeguards necessary to minimize bias, and highlight that test-treatment RCTs require different methods to ensure reliability than standard treatment trials. Please see the companion paper to this article: http://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-016-0286-0 .

Methodology, Methods, and Metrics for Testing and Evaluating Augmented Cognition Systems

DOE Office of Scientific and Technical Information (OSTI.GOV)

Greitzer, Frank L.

The augmented cognition research community seeks cognitive neuroscience-based solutions to improve warfighter performance by applying and managing mitigation strategies to reduce workload and improve the throughput and quality of decisions. The focus of augmented cognition mitigation research is to define, demonstrate, and exploit neuroscience and behavioral measures that support inferences about the warfighter’s cognitive state that prescribe the nature and timing of mitigation. A research challenge is to develop valid evaluation methodologies, metrics and measures to assess the impact of augmented cognition mitigations. Two considerations are external validity, which is the extent to which the results apply to operational contexts;more » and internal validity, which reflects the reliability of performance measures and the conclusions based on analysis of results. The scientific rigor of the research methodology employed in conducting empirical investigations largely affects the validity of the findings. External validity requirements also compel us to demonstrate operational significance of mitigations. Thus it is important to demonstrate effectiveness of mitigations under specific conditions. This chapter reviews some cognitive science and methodological considerations in designing augmented cognition research studies and associated human performance metrics and analysis methods to assess the impact of augmented cognition mitigations.« less

Fuzzy logic based sensor performance evaluation of vehicle mounted metal detector systems

NASA Astrophysics Data System (ADS)

Abeynayake, Canicious; Tran, Minh D.

2015-05-01

Vehicle Mounted Metal Detector (VMMD) systems are widely used for detection of threat objects in humanitarian demining and military route clearance scenarios. Due to the diverse nature of such operational conditions, operational use of VMMD without a proper understanding of its capability boundaries may lead to heavy causalities. Multi-criteria fitness evaluations are crucial for determining capability boundaries of any sensor-based demining equipment. Evaluation of sensor based military equipment is a multi-disciplinary topic combining the efforts of researchers, operators, managers and commanders having different professional backgrounds and knowledge profiles. Information acquired through field tests usually involves uncertainty, vagueness and imprecision due to variations in test and evaluation conditions during a single test or series of tests. This report presents a fuzzy logic based methodology for experimental data analysis and performance evaluation of VMMD. This data evaluation methodology has been developed to evaluate sensor performance by consolidating expert knowledge with experimental data. A case study is presented by implementing the proposed data analysis framework in a VMMD evaluation scenario. The results of this analysis confirm accuracy, practicability and reliability of the fuzzy logic based sensor performance evaluation framework.

Quantified Risk Ranking Model for Condition-Based Risk and Reliability Centered Maintenance

NASA Astrophysics Data System (ADS)

Chattopadhyaya, Pradip Kumar; Basu, Sushil Kumar; Majumdar, Manik Chandra

2017-06-01

In the recent past, risk and reliability centered maintenance (RRCM) framework is introduced with a shift in the methodological focus from reliability and probabilities (expected values) to reliability, uncertainty and risk. In this paper authors explain a novel methodology for risk quantification and ranking the critical items for prioritizing the maintenance actions on the basis of condition-based risk and reliability centered maintenance (CBRRCM). The critical items are identified through criticality analysis of RPN values of items of a system and the maintenance significant precipitating factors (MSPF) of items are evaluated. The criticality of risk is assessed using three risk coefficients. The likelihood risk coefficient treats the probability as a fuzzy number. The abstract risk coefficient deduces risk influenced by uncertainty, sensitivity besides other factors. The third risk coefficient is called hazardous risk coefficient, which is due to anticipated hazards which may occur in the future and the risk is deduced from criteria of consequences on safety, environment, maintenance and economic risks with corresponding cost for consequences. The characteristic values of all the three risk coefficients are obtained with a particular test. With few more tests on the system, the values may change significantly within controlling range of each coefficient, hence `random number simulation' is resorted to obtain one distinctive value for each coefficient. The risk coefficients are statistically added to obtain final risk coefficient of each critical item and then the final rankings of critical items are estimated. The prioritization in ranking of critical items using the developed mathematical model for risk assessment shall be useful in optimization of financial losses and timing of maintenance actions.

Decision-theoretic methodology for reliability and risk allocation in nuclear power plants

DOE Office of Scientific and Technical Information (OSTI.GOV)

Cho, N.Z.; Papazoglou, I.A.; Bari, R.A.

1985-01-01

This paper describes a methodology for allocating reliability and risk to various reactor systems, subsystems, components, operations, and structures in a consistent manner, based on a set of global safety criteria which are not rigid. The problem is formulated as a multiattribute decision analysis paradigm; the multiobjective optimization, which is performed on a PRA model and reliability cost functions, serves as the guiding principle for reliability and risk allocation. The concept of noninferiority is used in the multiobjective optimization problem. Finding the noninferior solution set is the main theme of the current approach. The assessment of the decision maker's preferencesmore » could then be performed more easily on the noninferior solution set. Some results of the methodology applications to a nontrivial risk model are provided and several outstanding issues such as generic allocation and preference assessment are discussed.« less

One approach for evaluating the Distributed Computing Design System (DCDS)

NASA Technical Reports Server (NTRS)

Ellis, J. T.

1985-01-01

The Distributed Computer Design System (DCDS) provides an integrated environment to support the life cycle of developing real-time distributed computing systems. The primary focus of DCDS is to significantly increase system reliability and software development productivity, and to minimize schedule and cost risk. DCDS consists of integrated methodologies, languages, and tools to support the life cycle of developing distributed software and systems. Smooth and well-defined transistions from phase to phase, language to language, and tool to tool provide a unique and unified environment. An approach to evaluating DCDS highlights its benefits.

Reliability evaluation methodology for NASA applications

NASA Technical Reports Server (NTRS)

Taneja, Vidya S.

1992-01-01

Liquid rocket engine technology has been characterized by the development of complex systems containing large number of subsystems, components, and parts. The trend to even larger and more complex system is continuing. The liquid rocket engineers have been focusing mainly on performance driven designs to increase payload delivery of a launch vehicle for a given mission. In otherwords, although the failure of a single inexpensive part or component may cause the failure of the system, reliability in general has not been considered as one of the system parameters like cost or performance. Up till now, quantification of reliability has not been a consideration during system design and development in the liquid rocket industry. Engineers and managers have long been aware of the fact that the reliability of the system increases during development, but no serious attempts have been made to quantify reliability. As a result, a method to quantify reliability during design and development is needed. This includes application of probabilistic models which utilize both engineering analysis and test data. Classical methods require the use of operating data for reliability demonstration. In contrast, the method described in this paper is based on similarity, analysis, and testing combined with Bayesian statistical analysis.

De-individualized psychophysiological strain assessment during a flight simulation test—Validation of a space methodology

NASA Astrophysics Data System (ADS)

Johannes, Bernd; Salnitski, Vyacheslav; Soll, Henning; Rauch, Melina; Hoermann, Hans-Juergen

For the evaluation of an operator's skill reliability indicators of work quality as well as of psychophysiological states during the work have to be considered. The herein presented methodology and measurement equipment were developed and tested in numerous terrestrial and space experiments using a simulation of a spacecraft docking on a space station. However, in this study the method was applied to a comparable terrestrial task—the flight simulator test (FST) used in the DLR selection procedure for ab initio pilot applicants for passenger airlines. This provided a large amount of data for a statistical verification of the space methodology. For the evaluation of the strain level of applicants during the FST psychophysiological measurements were used to construct a "psychophysiological arousal vector" (PAV) which is sensitive to various individual reaction patterns of the autonomic nervous system to mental load. Its changes and increases will be interpreted as "strain". In the first evaluation study, 614 subjects were analyzed. The subjects first underwent a calibration procedure for the assessment of their autonomic outlet type (AOT) and on the following day they performed the FST, which included three tasks and was evaluated by instructors applying well-established and standardized rating scales. This new method will possibly promote a wide range of other future applications in aviation and space psychology.

A psychometric evaluation of the Rorschach comprehensive system's perceptual thinking index.

PubMed

Dao, Tam K; Prevatt, Frances

2006-04-01

In this study, we investigated evidence for reliability and validity of the Perceptual Thinking Index (PTI; Exner, 2000a, 2000b) among an adult inpatient population. We conducted reliability and validity analyses on 107 patients who met the Diagnostic and Statistical Manual of Mental Disorders (4th ed., text revision; American Psychiatric Association, 2000) criteria for a schizophrenia-spectrum disorder (SSD) or mood disorder with no psychotic features (MD). Results provided support for interrater reliability as well as internal consistency of the PTI. Furthermore, the PTI was an effective index in differentiating SSD patients from patients diagnosed with an MD. Finally, the PTI demonstrated adequate diagnostic statistics that can be useful in the classification of patients diagnosed with SSD and MD. We discuss methodological issues, implications for assessment practice, and directions for future research.

The methodological quality of diagnostic test accuracy studies for musculoskeletal conditions can be improved.

PubMed

Henschke, Nicholas; Keuerleber, Julia; Ferreira, Manuela; Maher, Christopher G; Verhagen, Arianne P

2014-04-01

To provide an overview of reporting and methodological quality in diagnostic test accuracy (DTA) studies in the musculoskeletal field and evaluate the use of the QUality Assessment of Diagnostic Accuracy Studies (QUADAS) checklist. A literature review identified all systematic reviews that evaluated the accuracy of clinical tests to diagnose musculoskeletal conditions and used the QUADAS checklist. Two authors screened all identified reviews and extracted data on the target condition, index tests, reference standard, included studies, and QUADAS items. A descriptive analysis of the QUADAS checklist was performed, along with Rasch analysis to examine the construct validity and internal reliability. A total of 19 systematic reviews were included, which provided data on individual items of the QUADAS checklist for 392 DTA studies. In the musculoskeletal field, uninterpretable or intermediate test results are commonly not reported, with 175 (45%) studies scoring "no" to this item. The proportion of studies fulfilling certain items varied from 22% (item 11) to 91% (item 3). The interrater reliability of the QUADAS checklist was good and Rasch analysis showed excellent construct validity and internal consistency. This overview identified areas where the reporting and performance of diagnostic studies within the musculoskeletal field can be improved. Copyright © 2014 Elsevier Inc. All rights reserved.

Rasch analysis and impact factor methods both yield valid and comparable measures of health status in interstitial lung disease.

PubMed

Patel, Amit S; Siegert, Richard J; Bajwah, Sabrina; Brignall, Kate; Gosker, Harry R; Moxham, John; Maher, Toby M; Renzoni, Elisabetta A; Wells, Athol U; Higginson, Irene J; Birring, Surinder S

2015-09-01

Rasch analysis has largely replaced impact factor methodology for developing health status measures. The aim of this study was to develop a health status questionnaire for patients with interstitial lung disease (ILD) using impact factor methodology and to compare its validity with that of another version developed using Rasch analysis. A preliminary 71-item questionnaire was developed and evaluated in 173 patients with ILD. Items were reduced by the impact factor method (King's Brief ILD questionnaire, KBILD-I) and Rasch analysis (KBILD-R). Both questionnaires were validated by assessing their relationship with forced vital capacity (FVC) and St Georges Respiratory Questionnaire (SGRQ) and by evaluating internal reliability, repeatability, and longitudinal responsiveness. The KBILD-R and KBILD-I comprised 15 items each. The content of eight items differed between the KBILD-R and KBILD-I. Internal and test-retest reliability was good for total scores of both questionnaires. There was a good relationship with SGRQ and moderate relationship with FVC for both questionnaires. Effect sizes were comparable. Both questionnaires discriminated patients with differing disease severity. Despite considerable differences in the content of retained items, both KBILD-R and KBILD-I questionnaires demonstrated acceptable measurement properties and performed comparably in a clinical setting. Copyright © 2015 Elsevier Inc. All rights reserved.

Prediction of Software Reliability using Bio Inspired Soft Computing Techniques.

PubMed

Diwaker, Chander; Tomar, Pradeep; Poonia, Ramesh C; Singh, Vijander

2018-04-10

A lot of models have been made for predicting software reliability. The reliability models are restricted to using particular types of methodologies and restricted number of parameters. There are a number of techniques and methodologies that may be used for reliability prediction. There is need to focus on parameters consideration while estimating reliability. The reliability of a system may increase or decreases depending on the selection of different parameters used. Thus there is need to identify factors that heavily affecting the reliability of the system. In present days, reusability is mostly used in the various area of research. Reusability is the basis of Component-Based System (CBS). The cost, time and human skill can be saved using Component-Based Software Engineering (CBSE) concepts. CBSE metrics may be used to assess those techniques which are more suitable for estimating system reliability. Soft computing is used for small as well as large-scale problems where it is difficult to find accurate results due to uncertainty or randomness. Several possibilities are available to apply soft computing techniques in medicine related problems. Clinical science of medicine using fuzzy-logic, neural network methodology significantly while basic science of medicine using neural-networks-genetic algorithm most frequently and preferably. There is unavoidable interest shown by medical scientists to use the various soft computing methodologies in genetics, physiology, radiology, cardiology and neurology discipline. CBSE boost users to reuse the past and existing software for making new products to provide quality with a saving of time, memory space, and money. This paper focused on assessment of commonly used soft computing technique like Genetic Algorithm (GA), Neural-Network (NN), Fuzzy Logic, Support Vector Machine (SVM), Ant Colony Optimization (ACO), Particle Swarm Optimization (PSO), and Artificial Bee Colony (ABC). This paper presents working of soft computing techniques and assessment of soft computing techniques to predict reliability. The parameter considered while estimating and prediction of reliability are also discussed. This study can be used in estimation and prediction of the reliability of various instruments used in the medical system, software engineering, computer engineering and mechanical engineering also. These concepts can be applied to both software and hardware, to predict the reliability using CBSE.

Measurement properties of tools measuring mental health knowledge: a systematic review.

PubMed

Wei, Yifeng; McGrath, Patrick J; Hayden, Jill; Kutcher, Stan

2016-08-23

Mental health literacy has received great attention recently to improve mental health knowledge, decrease stigma and enhance help-seeking behaviors. We conducted a systematic review to critically appraise the qualities of studies evaluating the measurement properties of mental health knowledge tools and the quality of included measurement properties. We searched PubMed, PsycINFO, EMBASE, CINAHL, the Cochrane Library, and ERIC for studies addressing psychometrics of mental health knowledge tools and published in English. We applied the COSMIN checklist to assess the methodological quality of each study as "excellent", "good", "fair", or "indeterminate". We ranked the level of evidence of the overall quality of each measurement property across studies as "strong", "moderate", "limited", "conflicting", or "unknown". We identified 16 mental health knowledge tools in 17 studies, addressing reliability, validity, responsiveness or measurement errors. The methodological quality of included studies ranged from "poor" to "excellent" including 6 studies addressing the content validity, internal consistency or structural validity demonstrating "excellent" quality. We found strong evidence of the content validity or internal consistency of 6 tools; moderate evidence of the internal consistency, the content validity or the reliability of 8 tools; and limited evidence of the reliability, the structural validity, the criterion validity, or the construct validity of 12 tools. Both the methodological qualities of included studies and the overall evidence of measurement properties are mixed. Based on the current evidence, we recommend that researchers consider using tools with measurement properties of strong or moderate evidence that also reached the threshold for positive ratings according to COSMIN checklist.

Retrieving the Polar Mixed-Phase Cloud Liquid Water Path by Combining CALIOP and IIR Measurements

NASA Astrophysics Data System (ADS)

Luo, Tao; Wang, Zhien; Li, Xuebin; Deng, Shumei; Huang, Yong; Wang, Yingjian

2018-02-01

Mixed-phase cloud (MC) is the dominant cloud type over the polar region, and there are challenging conditions for remote sensing and in situ measurements. In this study, a new methodology of retrieving the stratiform MC liquid water path (LWP) by combining Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) and infrared imaging radiometer (IIR) measurements was developed and evaluated. This new methodology takes the advantage of reliable cloud-phase discrimination by combining lidar and radar measurements. An improved multiple-scattering effect correction method for lidar signals was implemented to provide reliable cloud extinction near cloud top. Then with the adiabatic cloud assumption, the MC LWP can be retrieved by a lookup-table-based method. Simulations with error-free inputs showed that the mean bias and the root mean squared error of the LWP derived from the new method are -0.23 ± 2.63 g/m2, with the mean absolute relative error of 4%. Simulations with erroneous inputs suggested that the new methodology could provide reliable retrieval of LWP to support the statistical or climatology analysis. Two-month A-train satellite retrievals over Arctic region showed that the new method can produce very similar cloud top temperature (CTT) dependence of LWP to the ground-based microwave radiometer measurements, with a bias of -0.78 g/m2 and a correlation coefficient of 0.95 between the two mean CTT-LWP relationships. The new approach can also produce reasonable pattern and value of LWP in spatial distribution over the Arctic region.

Multidisciplinary System Reliability Analysis

NASA Technical Reports Server (NTRS)

Mahadevan, Sankaran; Han, Song; Chamis, Christos C. (Technical Monitor)

2001-01-01

The objective of this study is to develop a new methodology for estimating the reliability of engineering systems that encompass multiple disciplines. The methodology is formulated in the context of the NESSUS probabilistic structural analysis code, developed under the leadership of NASA Glenn Research Center. The NESSUS code has been successfully applied to the reliability estimation of a variety of structural engineering systems. This study examines whether the features of NESSUS could be used to investigate the reliability of systems in other disciplines such as heat transfer, fluid mechanics, electrical circuits etc., without considerable programming effort specific to each discipline. In this study, the mechanical equivalence between system behavior models in different disciplines are investigated to achieve this objective. A new methodology is presented for the analysis of heat transfer, fluid flow, and electrical circuit problems using the structural analysis routines within NESSUS, by utilizing the equivalence between the computational quantities in different disciplines. This technique is integrated with the fast probability integration and system reliability techniques within the NESSUS code, to successfully compute the system reliability of multidisciplinary systems. Traditional as well as progressive failure analysis methods for system reliability estimation are demonstrated, through a numerical example of a heat exchanger system involving failure modes in structural, heat transfer and fluid flow disciplines.

Multi-Disciplinary System Reliability Analysis

NASA Technical Reports Server (NTRS)

Mahadevan, Sankaran; Han, Song

1997-01-01

The objective of this study is to develop a new methodology for estimating the reliability of engineering systems that encompass multiple disciplines. The methodology is formulated in the context of the NESSUS probabilistic structural analysis code developed under the leadership of NASA Lewis Research Center. The NESSUS code has been successfully applied to the reliability estimation of a variety of structural engineering systems. This study examines whether the features of NESSUS could be used to investigate the reliability of systems in other disciplines such as heat transfer, fluid mechanics, electrical circuits etc., without considerable programming effort specific to each discipline. In this study, the mechanical equivalence between system behavior models in different disciplines are investigated to achieve this objective. A new methodology is presented for the analysis of heat transfer, fluid flow, and electrical circuit problems using the structural analysis routines within NESSUS, by utilizing the equivalence between the computational quantities in different disciplines. This technique is integrated with the fast probability integration and system reliability techniques within the NESSUS code, to successfully compute the system reliability of multi-disciplinary systems. Traditional as well as progressive failure analysis methods for system reliability estimation are demonstrated, through a numerical example of a heat exchanger system involving failure modes in structural, heat transfer and fluid flow disciplines.

Reliability and maintainability assessment factors for reliable fault-tolerant systems

NASA Technical Reports Server (NTRS)

Bavuso, S. J.

1984-01-01

A long term goal of the NASA Langley Research Center is the development of a reliability assessment methodology of sufficient power to enable the credible comparison of the stochastic attributes of one ultrareliable system design against others. This methodology, developed over a 10 year period, is a combined analytic and simulative technique. An analytic component is the Computer Aided Reliability Estimation capability, third generation, or simply CARE III. A simulative component is the Gate Logic Software Simulator capability, or GLOSS. The numerous factors that potentially have a degrading effect on system reliability and the ways in which these factors that are peculiar to highly reliable fault tolerant systems are accounted for in credible reliability assessments. Also presented are the modeling difficulties that result from their inclusion and the ways in which CARE III and GLOSS mitigate the intractability of the heretofore unworkable mathematics.

Solving a methodological challenge in work stress evaluation with the Stress Assessment and Research Toolkit (StART): a study protocol.

PubMed

Guglielmi, Dina; Simbula, Silvia; Vignoli, Michela; Bruni, Ilaria; Depolo, Marco; Bonfiglioli, Roberta; Tabanelli, Maria Carla; Violante, Francesco Saverio

2013-06-22

Stress evaluation is a field of strong interest and challenging due to several methodological aspects in the evaluation process. The aim of this study is to propose a study protocol to test a new method (i.e., the Stress Assessment and Research Toolkit) to assess psychosocial risk factors at work. This method addresses several methodological issues (e.g., subjective vs. objective, qualitative vs quantitative data) by assessing work-related stressors using different kinds of data: i) organisational archival data (organisational indicators sheet); ii) qualitative data (focus group); iii) worker perception (questionnaire); and iv) observational data (observational checklist) using mixed methods research. In addition, it allows positive and negative aspects of work to be considered conjointly, using an approach that considers at the same time job demands and job resources. The integration of these sources of data can reduce the theoretical and methodological bias related to stress research in the work setting, allows researchers and professionals to obtain a reliable description of workers' stress, providing a more articulate vision of psychosocial risks, and allows a large amount of data to be collected. Finally, the implementation of the method ensures in the long term a primary prevention for psychosocial risk management in that it aims to reduce or modify the intensity, frequency or duration of organisational demands.

Solving a methodological challenge in work stress evaluation with the Stress Assessment and Research Toolkit (StART): a study protocol

PubMed Central

2013-01-01

Background Stress evaluation is a field of strong interest and challenging due to several methodological aspects in the evaluation process. The aim of this study is to propose a study protocol to test a new method (i.e., the Stress Assessment and Research Toolkit) to assess psychosocial risk factors at work. Design This method addresses several methodological issues (e.g., subjective vs. objective, qualitative vs quantitative data) by assessing work-related stressors using different kinds of data: i) organisational archival data (organisational indicators sheet); ii) qualitative data (focus group); iii) worker perception (questionnaire); and iv) observational data (observational checklist) using mixed methods research. In addition, it allows positive and negative aspects of work to be considered conjointly, using an approach that considers at the same time job demands and job resources. Discussion The integration of these sources of data can reduce the theoretical and methodological bias related to stress research in the work setting, allows researchers and professionals to obtain a reliable description of workers’ stress, providing a more articulate vision of psychosocial risks, and allows a large amount of data to be collected. Finally, the implementation of the method ensures in the long term a primary prevention for psychosocial risk management in that it aims to reduce or modify the intensity, frequency or duration of organisational demands. PMID:23799950

Enhancing treatment fidelity in psychotherapy research: novel approach to measure the components of cognitive behavioural therapy for relapse prevention in first-episode psychosis.

PubMed

Alvarez-Jimenez, Mario; Wade, Darryl; Cotton, Sue; Gee, Donna; Pearce, Tracey; Crisp, Kingsley; McGorry, Patrick D; Gleeson, John F

2008-12-01

Establishing treatment fidelity is one of the most important aspects of psychotherapy research. Treatment fidelity refers to the methodological strategies used to examine and enhance the reliability and validity of psychotherapy. This study sought to develop and evaluate a measure specifically designed to assess fidelity to the different therapeutic components (i.e. therapy phases) of the individual intervention of a psychotherapy clinical trial (the EPISODE II trial). A representative sample of sessions stratified by therapy phase was assessed using a specifically developed fidelity measure (Relapse Prevention Therapy-Fidelity Scale, RPT-FS). Each RPT-FS subscale was designed to include a different component/phase of therapy and its major therapeutic ingredients. The measure was found to be reliable and had good internal consistency. The RPT-FS discriminated, almost perfectly, between therapy phases. The analysis of the therapeutic strategies implemented during the intervention indicated that treatment fidelity was good throughout therapy phases. While therapists primarily engaged in interventions from the appropriate therapeutic phase, flexibility in therapy was evident. This study described the development of a brief, reliable and internally consistent measure to determine both treatment fidelity and the therapy components implemented throughout the intervention. This methodology can be potentially useful to determine those components related to therapeutic change.

Instruments evaluating the quality of the clinical learning environment in nursing education: A systematic review of psychometric properties.

PubMed

Mansutti, Irene; Saiani, Luisa; Grassetti, Luca; Palese, Alvisa

2017-03-01

The clinical learning environment is fundamental to nursing education paths, capable of affecting learning processes and outcomes. Several instruments have been developed in nursing education, aimed at evaluating the quality of the clinical learning environments; however, no systematic review of the psychometric properties and methodological quality of these studies has been performed to date. The aims of the study were: 1) to identify validated instruments evaluating the clinical learning environments in nursing education; 2) to evaluate critically the methodological quality of the psychometric property estimation used; and 3) to compare psychometric properties across the instruments available. A systematic review of the literature (using the Preferred Reporting Items for Systematic Reviews and Meta-Analysis guidelines) and an evaluation of the methodological quality of psychometric properties (using the COnsensus-based Standards for the selection of health Measurement INstruments guidelines). The Medline and CINAHL databases were searched. Eligible studies were those that satisfied the following criteria: a) validation studies of instruments evaluating the quality of clinical learning environments; b) in nursing education; c) published in English or Italian; d) before April 2016. The included studies were evaluated for the methodological quality of the psychometric properties measured and then compared in terms of both the psychometric properties and the methodological quality of the processes used. The search strategy yielded a total of 26 studies and eight clinical learning environment evaluation instruments. A variety of psychometric properties have been estimated for each instrument, with differing qualities in the methodology used. Concept and construct validity were poorly assessed in terms of their significance and rarely judged by the target population (nursing students). Some properties were rarely considered (e.g., reliability, measurement error, criterion validity), whereas others were frequently estimated, but using different coefficients and statistical analyses (e.g., internal consistency, structural validity), thus rendering comparison across instruments difficult. Moreover, the methodological quality adopted in the property assessments was poor or fair in most studies, compromising the goodness of the psychometric values estimated. Clinical learning placements represent the key strategies in educating the future nursing workforce: instruments evaluating the quality of the settings, as well as their capacity to promote significant learning, are strongly recommended. Studies estimating psychometric properties, using an increased quality of research methodologies are needed in order to support nursing educators in the process of clinical placements accreditation and quality improvement. Copyright © 2017 Elsevier Ltd. All rights reserved.

Integrating field methodology and web-based data collection to assess the reliability of the Alcohol Use Disorders Identification Test (AUDIT).

PubMed

Celio, Mark A; Vetter-O'Hagen, Courtney S; Lisman, Stephen A; Johansen, Gerard E; Spear, Linda P

2011-12-01

Field methodologies offer a unique opportunity to collect ecologically valid data on alcohol use and its associated problems within natural drinking environments. However, limitations in follow-up data collection methods have left unanswered questions regarding the psychometric properties of field-based measures. The aim of the current study is to evaluate the reliability of self-report data collected in a naturally occurring environment - as indexed by the Alcohol Use Disorders Identification Test (AUDIT) - compared to self-report data obtained through an innovative web-based follow-up procedure. Individuals recruited outside of bars (N=170; mean age=21; range 18-32) provided a BAC sample and completed a self-administered survey packet that included the AUDIT. BAC feedback was provided anonymously through a dedicated web page. Upon sign in, follow-up participants (n=89; 52%) were again asked to complete the AUDIT before receiving their BAC feedback. Reliability analyses demonstrated that AUDIT scores - both continuous and dichotomized at the standard cut-point - were stable across field- and web-based administrations. These results suggest that self-report data obtained from acutely intoxicated individuals in naturally occurring environments are reliable when compared to web-based data obtained after a brief follow-up interval. Furthermore, the results demonstrate the feasibility, utility, and potential of integrating field methods and web-based data collection procedures. Copyright © 2011 Elsevier Ireland Ltd. All rights reserved.

Symptom research on chronic cough: a historical perspective.

PubMed

Irwin, R S; Madison, J M

2001-05-01

This review provides a perspective on how research on the management of cough has evolved, looks at key methodologic lessons that have been learned from this research and how they may relate to the management of other symptoms, identifies important methodologic challenges that remain to be solved, and lists important questions that still need to be answered. Three important methodologic lessons have been learned. First, cough must be evaluated systematically and according to a neuroanatomic framework. Second, the response to specific therapy must be noted to determine the cause or causes of cough and to characterize the strengths and limitations of diagnostic testing. Third, multiple conditions can simultaneously cause cough. Among the three methodologic challenges that still need to be solved are 1) definitively determining the diagnostic accuracy and reliability of 24-hour esophageal pH monitoring and how best to interpret pH test results, 2) definitively determining the role of nonacid reflux in cough due to gastroesophageal reflux disease, and 3) developing reliable and reproducible subjective and objective methods with which to assess the efficacy of cough therapy. Numerous important clinical questions are still unanswered: What role do empirical therapeutic trials play in diagnosing the cause of chronic cough? What is the most cost-effective approach to the diagnosis and treatment of chronic cough: empirical therapeutic trials or laboratory testing-directed therapeutic trials? How often is environmental air pollution, unrelated to allergies or smoking, responsible for chronic cough?

Construction and validation of educational materials for the prevention of metabolic syndrome in adolescents 1

PubMed Central

de Moura, Ionara Holanda; da Silva, Antônia Fabiana Rodrigues; Rocha, Aparecida do Espírito Santo de Holanda; Lima, Luisa Helena de Oliveira; Moreira, Thereza Maria Magalhães; da Silva, Ana Roberta Vilarouca

2017-01-01

ABSTRACT Objective: To develop and validate an educational technology focused on prevention of metabolic syndrome among adolescents. Methods: This was methodological research. Using an integrative review, the available publications on the subject were analyzed. Then, this knowledge was used to describe the theoretical content and, with the help of a graphic designer, the art and layout of the pages were developed. In the third phase, the booklet was evaluated and validated by 21 specialists and 39 adolescents. Data collection included three different questionnaires, according to the focus of evaluation of each group of participants, analyzed for reliability (Cronbach’s Alpha) and agreement by Infraclass Correlation Coefficient. Results: The mean score attributed by technical content experts was 91.7%, and the content validity index, measured by experts responses, was 0.98, showing high reliability and agreement. In addition, the level of agreement of the positive responses given by adolescents was 88.4%. Conclusion: the educational booklet has proved to be a valid and reliable tool to be used for promoting adolescent health. PMID:29020125

Measurement methods to assess diastasis of the rectus abdominis muscle (DRAM): A systematic review of their measurement properties and meta-analytic reliability generalisation.

PubMed

van de Water, A T M; Benjamin, D R

2016-02-01

Systematic literature review. Diastasis of the rectus abdominis muscle (DRAM) has been linked with low back pain, abdominal and pelvic dysfunction. Measurement is used to either screen or to monitor DRAM width. Determining which methods are suitable for screening and monitoring DRAM is of clinical value. To identify the best methods to screen for DRAM presence and monitor DRAM width. AMED, Embase, Medline, PubMed and CINAHL databases were searched for measurement property studies of DRAM measurement methods. Population characteristics, measurement methods/procedures and measurement information were extracted from included studies. Quality of all studies was evaluated using 'quality rating criteria'. When possible, reliability generalisation was conducted to provide combined reliability estimations. Thirteen studies evaluated measurement properties of the 'finger width'-method, tape measure, calipers, ultrasound, CT and MRI. Ultrasound was most evaluated. Methodological quality of these studies varied widely. Pearson's correlations of r = 0.66-0.79 were found between calipers and ultrasound measurements. Calipers and ultrasound had Intraclass Correlation Coefficients (ICC) of 0.78-0.97 for test-retest, inter- and intra-rater reliability. The 'finger width'-method had weighted Kappa's of 0.73-0.77 for test-retest reliability, but moderate agreement (63%; weighted Kappa = 0.53) between raters. Comparing calipers and ultrasound, low measurement error was found (above the umbilicus), and the methods had good agreement (83%; weighted Kappa = 0.66) for discriminative purposes. The available information support ultrasound and calipers as adequate methods to assess DRAM. For other methods limited measurement information of low to moderate quality is available and further evaluation of their measurement properties is required. Copyright © 2015 Elsevier Ltd. All rights reserved.

Evaluating sensor linearity of chosen infrared sensors

NASA Astrophysics Data System (ADS)

Walczykowski, P.; Orych, A.; Jenerowicz, A.; Karcz, P.

2014-11-01

The paper describes a series of experiments conducted as part of the IRAMSWater Project, the aim of which is to establish methodologies for detecting and identifying pollutants in water bodies using aerial imagery data. The main idea is based on the hypothesis, that it is possible to identify certain types of physical, biological and chemical pollutants based on their spectral reflectance characteristics. The knowledge of these spectral curves is then used to determine very narrow spectral bands in which greatest reflectance variations occur between these pollutants. A frame camera is then equipped with a band pass filter, which allows only the selected bandwidth to be registered. In order to obtain reliable reflectance data straight from the images, the team at the Military University of Technology had developed a methodology for determining the necessary acquisition parameters for the sensor (integration time and f-stop depending on the distance from the scene and it's illumination). This methodology however is based on the assumption, that the imaging sensors have a linear response. This paper shows the results of experiments used to evaluate this linearity.

Mining data from hemodynamic simulations for generating prediction and explanation models.

PubMed

Bosnić, Zoran; Vračar, Petar; Radović, Milos D; Devedžić, Goran; Filipović, Nenad D; Kononenko, Igor

2012-03-01

One of the most common causes of human death is stroke, which can be caused by carotid bifurcation stenosis. In our work, we aim at proposing a prototype of a medical expert system that could significantly aid medical experts to detect hemodynamic abnormalities (increased artery wall shear stress). Based on the acquired simulated data, we apply several methodologies for1) predicting magnitudes and locations of maximum wall shear stress in the artery, 2) estimating reliability of computed predictions, and 3) providing user-friendly explanation of the model's decision. The obtained results indicate that the evaluated methodologies can provide a useful tool for the given problem domain. © 2012 IEEE

[Some critical remarks on standardised assessment instruments in nursing].

PubMed

Bartholomeyczik, Sabine

2007-08-01

The use of standardised instruments in nursing has rapidly grown and can be seen as a symptom of the necessary comprehensive nursing diagnostics. However, these instruments comprise the risk of misuse, if they are not critically evaluated. Published statements about tests of reliability and validity of an instrument are insufficient. First, the critical evaluation has to ask for the instrument's theoretical and content base: Is the instrument relevant for nursing, suitable for practice and leading to nursing actions? Two examples of well known instruments and different kinds of their utilization in nursing are discussed. Next, the instruments have to be questioned as "bodies with numbers". Studies on reliability and validity have to be as carefully evaluated as other empirical research. The sample, the suitability of agreement indicators (interraterreliability), kind and reason of tests have to be questioned. The same has to be done with tests of validity which comprise an even greater challenge. Methodological studies about these questions are missing; guidelines for test user qualifications need to be developed.

Validation of hindi translation of DSM-5 level 1 cross-cutting symptom measure.

PubMed

Goel, Ankit; Kataria, Dinesh

2018-04-01

The DSM-5 Level 1 Cross-Cutting Symptom Measure is a self- or informant-rated measure that assesses mental health domains which are important across psychiatric diagnoses. The absence of this self- or informant-administered instrument in Hindi, which is a major language in India, is an important limitation in using this scale. To translate the English version of the DSM-5 Level 1 Cross-Cutting Symptom Measure to Hindi and evaluate its psychometric properties. The study was conducted at a tertiary care hospital in Delhi. The DSM-5 Level 1 Cross-Cutting Symptom Measure was translated into Hindi using the World Health Organization's translation methodology. Mean and standard deviation were evaluated for continuous variables while for categorical variables frequency and percentages were calculated. The translated version was evaluated for cross-language equivalence, test-retest reliability, internal consistency, and split half reliability. Hindi version was found to have good cross-language equivalence and test-retest reliability at the level of items and domains. Twenty two of the 23 items and all the 23 items had a significant correlation (ρ < 0.001) in cross language concordance and test-retest reliability data, respectively. The Cronbach's alpha was 0.95, and the Spearman-Brown Sphericity value was 0.79 for the Hindi version. The present study shows that cross-language concordance, internal consistency, split-half reliability, and test-retest reliability of the Hindi version of the measure are excellent. Thus, the Hindi version of DSM-5 Level 1 Cross-Cutting Symptom Measure as translated in this study is a valid instrument. Copyright © 2018 Elsevier B.V. All rights reserved.

The National Aviation Operational Monitoring Service (NAOMS): A Documentation of the Development of a Survey Methodology

NASA Technical Reports Server (NTRS)

Connors, Mary M.; Mauro, Robert; Statler, Irving C.

2012-01-01

The National Aviation Operational Monitoring Service (NAOMS) was a research project under NASA s Aviation Safety Program during the years from 2000 to 2005. The purpose of this project was to develop a methodology for gaining reliable information on changes over time in the rates-of-occurrence of safety-related events as a means of assessing the safety of the national airspace. The approach was a scientifically designed survey of the operators of the aviation system concerning their safety-related experiences. This report presents the results of the methodology developed and a demonstration of the NAOMS concept through a survey of nearly 20,000 randomly selected air-carrier pilots. Results give evidence that the NAOMS methodology can provide a statistically sound basis for evaluating trends of incidents that could compromise safety. The approach and results are summarized in the report and supporting documentation and complete analyses of results are presented in 14 appendices.

76 FR 65504 - Proposed Agency Information Collection

Federal Register 2010, 2011, 2012, 2013, 2014

2011-10-21

..., including the validity of the methodology and assumptions used; (c) ways to enhance the quality, utility... Reliability Standard, FAC- 008-3--Facility Ratings, developed by the North American Electric Reliability... Reliability Standard FAC- 008-3 is pending before the Commission. The proposed Reliability Standard modifies...

A diameter-sensitive flow entropy method for reliability consideration in water distribution system design

NASA Astrophysics Data System (ADS)

Liu, Haixing; Savić, Dragan; Kapelan, Zoran; Zhao, Ming; Yuan, Yixing; Zhao, Hongbin

2014-07-01

Flow entropy is a measure of uniformity of pipe flows in water distribution systems. By maximizing flow entropy one can identify reliable layouts or connectivity in networks. In order to overcome the disadvantage of the common definition of flow entropy that does not consider the impact of pipe diameter on reliability, an extended definition of flow entropy, termed as diameter-sensitive flow entropy, is proposed. This new methodology is then assessed by using other reliability methods, including Monte Carlo Simulation, a pipe failure probability model, and a surrogate measure (resilience index) integrated with water demand and pipe failure uncertainty. The reliability assessment is based on a sample of WDS designs derived from an optimization process for each of the two benchmark networks. Correlation analysis is used to evaluate quantitatively the relationship between entropy and reliability. To ensure reliability, a comparative analysis between the flow entropy and the new method is conducted. The results demonstrate that the diameter-sensitive flow entropy shows consistently much stronger correlation with the three reliability measures than simple flow entropy. Therefore, the new flow entropy method can be taken as a better surrogate measure for reliability and could be potentially integrated into the optimal design problem of WDSs. Sensitivity analysis results show that the velocity parameters used in the new flow entropy has no significant impact on the relationship between diameter-sensitive flow entropy and reliability.

Direct access to dithiobenzoate RAFT agent fragmentation rate coefficients by ESR spin-trapping.

PubMed

Ranieri, Kayte; Delaittre, Guillaume; Barner-Kowollik, Christopher; Junkers, Thomas

2014-12-01

The β-scission rate coefficient of tert-butyl radicals fragmenting off the intermediate resulting from their addition to tert-butyl dithiobenzoate-a reversible addition-fragmentation chain transfer (RAFT) agent-is estimated via the recently introduced electron spin resonance (ESR)-trapping methodology as a function of temperature. The newly introduced ESR-trapping methodology is critically evaluated and found to be reliable. At 20 °C, a fragmentation rate coefficient of close to 0.042 s(-1) is observed, whereas the activation parameters for the fragmentation reaction-determined for the first time-read EA = 82 ± 13.3 kJ mol(-1) and A = (1.4 ± 0.25) × 10(13) s(-1) . The ESR spin-trapping methodology thus efficiently probes the stability of the RAFT adduct radical under conditions relevant for the pre-equilibrium of the RAFT process. It particularly indicates that stable RAFT adduct radicals are indeed formed in early stages of the RAFT poly-merization, at least when dithiobenzoates are employed as controlling agents as stipulated by the so-called slow fragmentation theory. By design of the methodology, the obtained fragmentation rate coefficients represent an upper limit. The ESR spin-trapping methodology is thus seen as a suitable tool for evaluating the fragmentation rate coefficients of a wide range of RAFT adduct radicals. © 2014 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim.

Evaluating multidisciplinary health care teams: taking the crisis out of CRM.

PubMed

Sutton, Gigi

2009-08-01

High-reliability organisations are those, such as within the aviation industry, which operate in complex, hazardous environments and yet despite this are able to balance safety and effectiveness. Crew resource management (CRM) training is used to improve the non-technical skills of aviation crews and other high-reliability teams. To date, CRM within the health sector has been restricted to use with "crisis teams" and "crisis events". The purpose of this discussion paper is to examine the application of CRM to acute, ward-based multidisciplinary health care teams and more broadly to argue for the repositioning of health-based CRM to address effective everyday function, of which "crisis events" form just one part. It is argued that CRM methodology could be applied to evaluate ward-based health care teams and design non-technical skills training to increase their efficacy, promote better patient outcomes, and facilitate a range of positive personal and organisational level outcomes.

Durability evaluation of ceramic components using CARES/LIFE

NASA Technical Reports Server (NTRS)

Nemeth, Noel N.; Powers, Lynn M.; Janosik, Lesley A.; Gyekenyesi, John P.

1994-01-01

The computer program CARES/LIFE calculates the time-dependent reliability of monolithic ceramic components subjected to thermomechanical and/or proof test loading. This program is an extension of the CARES (Ceramics Analysis and Reliability Evaluation of Structures) computer program. CARES/LIFE accounts for the phenomenon of subcritical crack growth (SCG) by utilizing the power law, Paris law, or Walker equation. The two-parameter Weibull cumulative distribution function is used to characterize the variation in component strength. The effects of multiaxial stresses are modeled using either the principle of independent action (PIA), the Weibull normal stress averaging method (NSA), or the Batdorf theory. Inert strength and fatigue parameters are estimated from rupture strength data of naturally flawed specimens loaded in static, dynamic, or cyclic fatigue. Application of this design methodology is demonstrated using experimental data from alumina bar and disk flexure specimens which exhibit SCG when exposed to water.

Durability evaluation of ceramic components using CARES/LIFE

DOE Office of Scientific and Technical Information (OSTI.GOV)

Nemeth, N.N.; Janosik, L.A.; Gyekenyesi, J.P.

1996-01-01

The computer program CARES/LIFE calculates the time-dependent reliability of monolithic ceramic components subjected to thermomechanical and/or proof test loading. This program is an extension of the CARES (Ceramics Analysis and Reliability Evaluation of Structures) computer program. CARES/LIFE accounts for the phenomenon of subcritical crack growth (SCG) by utilizing the power law, Paris law, or Walker equation. The two-parameter Weibull cumulative distribution function is used to characterize the variation in component strength. The effects of multiaxial stresses are modeled using either the principle of independent action (PIA), the Weibull normal stress averaging method (NSA), or the Batdorf theory. Inert strength andmore » fatigue parameters are estimated from rupture strength data of naturally flawed specimens loaded in static, dynamic, or cyclic fatigue. Application of this design methodology is demonstrated using experimental data from alumina bar and disk flexure specimens, which exhibit SCG when exposed to water.« less

[Optimization of one-step pelletization technology of Biqiu granules by Plackett-Burman design and Box-Behnken response surface methodology].

PubMed

Zhang, Yan-jun; Liu, Li-li; Hu, Jun-hua; Wu, Yun; Chao, En-xiang; Xiao, Wei

2015-11-01

First with the qualified rate of granules as the evaluation index, significant influencing factors were firstly screened by Plackett-Burman design. Then, with the qualified rate and moisture content as the evaluation indexes, significant factors that affect one-step pelletization technology were further optimized by Box-Behnken design; experimental data were imitated by multiple regression and second-order polynomial equation; and response surface method was used for predictive analysis of optimal technology. The best conditions were as follows: inlet air temperature of 85 degrees C, sample introduction speed of 33 r x min(-1), density of concrete 1. 10. One-step pelletization technology of Biqiu granules by Plackett-Burman design and Box-Behnken response surface methodology was stable and feasible with good predictability, which provided reliable basis for the industrialized production of Biqiu granules.

[Development of MEDUC-PG14 survey to assess postgraduate teaching in medical specialties].

PubMed

Pizarro, Margarita; Solís, Nancy; Rojas, Viviana; Díaz, Luis Antonio; Padilla, Oslando; Letelier, Luz María; Aizman, Andrés; Sarfatis, Alberto; Olivos, Trinidad; Soza, Alejandro; Delfino, Alejandro; Latorre, Gonzalo; Ivanovic-Zuvic, Danisa; Hoyl, Trinidad; Bitran, Marcela; Arab, Juan Pablo; Riquelme, Arnoldo

2015-08-01

Feedback is one of the most important tools to improve teaching in medical education. To develop an instrument to assess the performance of clinical postgraduate teachers in medical specialties. A qualitative methodology consisting in interviews and focus-groups followed by a quantitative methodology to generate consensus, was employed. After generating the instrument, psychometric tests were performed to assess the construct validity (factor analysis) and reliability (Cronbach’s alpha). Experts in medical education, teachers and residents of a medical school participated in interviews and focus groups. With this information, 26 categories (79 items) were proposed and reduced to 14 items (Likert scale 1-5) by an expert’s Delphi panel, generating the MEDUC-PG14 survey, which was answered by 123 residents from different programs of medical specialties. Construct validity was carried out. Factor analysis showed three domains: Teaching and evaluation, respectful behavior towards patients and health care team, and providing feedback. The global score was 4.46 ± 0.94 (89% of the maximum). One teachers’ strength, as evaluated by their residents was “respectful behavior” with 4.85 ± 0.42 (97% of the maximum). “Providing feedback” obtained 4.09 ± 1.0 points (81.8% of the maximum). MEDUC-PG14 survey had a Cronbach’s alpha coefficient of 0.947. MEDUC-PG14 survey is a useful and reliable guide for teacher evaluation in medical specialty programs. Also provides feedback to improve educational skills of postgraduate clinical teachers.

Parts and Components Reliability Assessment: A Cost Effective Approach

NASA Technical Reports Server (NTRS)

Lee, Lydia

2009-01-01

System reliability assessment is a methodology which incorporates reliability analyses performed at parts and components level such as Reliability Prediction, Failure Modes and Effects Analysis (FMEA) and Fault Tree Analysis (FTA) to assess risks, perform design tradeoffs, and therefore, to ensure effective productivity and/or mission success. The system reliability is used to optimize the product design to accommodate today?s mandated budget, manpower, and schedule constraints. Stand ard based reliability assessment is an effective approach consisting of reliability predictions together with other reliability analyses for electronic, electrical, and electro-mechanical (EEE) complex parts and components of large systems based on failure rate estimates published by the United States (U.S.) military or commercial standards and handbooks. Many of these standards are globally accepted and recognized. The reliability assessment is especially useful during the initial stages when the system design is still in the development and hard failure data is not yet available or manufacturers are not contractually obliged by their customers to publish the reliability estimates/predictions for their parts and components. This paper presents a methodology to assess system reliability using parts and components reliability estimates to ensure effective productivity and/or mission success in an efficient manner, low cost, and tight schedule.

Reliability of Performance-Based Clinical Measurements to Assess Shoulder Girdle Kinematics and Positioning: Systematic Review.

PubMed

D'hondt, Norman E; Kiers, Henri; Pool, Jan J M; Hacquebord, Sijmen T; Terwee, Caroline B; Veeger, Dirkjan H E J

2017-01-01

Deviant shoulder girdle movement is suggested as an eminent factor in the etiology of shoulder pain. Reliable measurements of shoulder girdle kinematics are a prerequisite for optimizing clinical management strategies. The purpose of this study was to evaluate the reliability, measurement error, and internal consistency of measurements with performance-based clinical tests for shoulder girdle kinematics and positioning in patients with shoulder pain. The MEDLINE, Embase, CINAHL, and SPORTDiscus databases were systematically searched from inception to August 2015. Articles published in Dutch, English, or German were included if they involved the evaluation of at least one of the measurement properties of interest. Two reviewers independently evaluated the methodological quality per studied measurement property with the 4-point-rating scale of the COSMIN (COnsensus-based Standards for the selection of health Measurement INstruments) checklist, extracted data, and assessed the adequacy of the measurement properties. Forty studies comprising more than 30 clinical tests were included. Actual reported measurements of the tests were categorized into: (1) positional measurement methods, (2) measurement methods to determine dynamic characteristics, and (3) tests to diagnose impairments of shoulder girdle function. Best evidence synthesis of the tests was performed per measurement for each measurement property. All studies had significant limitations, including incongruence between test description and actual reported measurements and a lack of reporting on minimal important change. In general, the methodological quality of the selected studies was fair to poor. High-quality evidence indicates that measurements obtained with the Modified Scapular Assistance Test are not reliable for clinical use. Sound recommendations for the use of other tests could not be made due to inadequate evidence. Across studies, diversity in description, performance, and interpretation of similar tests was present, and different criteria were used to establish similar diagnoses, mostly without taking into account a clinically meaningful context. Consequently, these tests lack face validity, which hampers their clinical use. Further research on validity and how to integrate a clinically meaningful context of movement into clinical tests is warranted. © 2017 American Physical Therapy Association

Reliability approach to rotating-component design. [fatigue life and stress concentration

NASA Technical Reports Server (NTRS)

Kececioglu, D. B.; Lalli, V. R.

1975-01-01

A probabilistic methodology for designing rotating mechanical components using reliability to relate stress to strength is explained. The experimental test machines and data obtained for steel to verify this methodology are described. A sample mechanical rotating component design problem is solved by comparing a deterministic design method with the new design-by reliability approach. The new method shows that a smaller size and weight can be obtained for specified rotating shaft life and reliability, and uses the statistical distortion-energy theory with statistical fatigue diagrams for optimum shaft design. Statistical methods are presented for (1) determining strength distributions for steel experimentally, (2) determining a failure theory for stress variations in a rotating shaft subjected to reversed bending and steady torque, and (3) relating strength to stress by reliability.

Application of a Resilience Framework to Military Installations: A Methodology for Energy Resilience Business Case Decisions

DTIC Science & Technology

2016-10-04

analysis (due to site-level evaluations), but could be added in the future, include: wind turbines (the installations we visited were not interested due...procurement, operation, maintenance , testing, and fueling of the generators, detailed inventory and cost data is difficult to obtain. The DPW is often...understaffed, leading to uneven testing and maintenance of the equipment despite their best efforts. The reliability of these generators is typically

Methodology for urban rail and construction technology research and development planning

NASA Technical Reports Server (NTRS)

Rubenstein, L. D.; Land, J. E.; Deshpande, G.; Dayman, B.; Warren, E. H.

1980-01-01

A series of transit system visits, organized by the American Public Transit Association (APTA), was conducted in which the system operators identified the most pressing development needs. These varied by property and were reformulated into a series of potential projects. To assist in the evaluation, a data base useful for estimating the present capital and operating costs of various transit system elements was generated from published data. An evaluation model was developed which considered the rate of deployment of the research and development project, potential benefits, development time and cost. An outline of an evaluation methodology that considered benefits other than capital and operating cost savings was also presented. During the course of the study, five candidate projects were selected for detailed investigation; (1) air comfort systems; (2) solid state auxiliary power conditioners; (3) door systems; (4) escalators; and (5) fare collection systems. Application of the evaluation model to these five examples showed the usefulness of modeling deployment rates and indicated a need to increase the scope of the model to quantitatively consider reliability impacts.

Measurement properties of instruments evaluating self-care and related concepts in people with chronic obstructive pulmonary disease: A systematic review.

PubMed

Clari, Marco; Matarese, Maria; Alvaro, Rosaria; Piredda, Michela; De Marinis, Maria Grazia

2016-01-01

The use of valid and reliable instruments for assessing self-care is crucial for the evaluation of chronic obstructive pulmonary disease (COPD) management programs. The aim of this review is to evaluate the measurement properties and theoretical foundations of instruments for assessing self-care and related concepts in people with COPD. A systematic review was conducted of articles describing the development and validation of self-care instruments. The methodological quality of the measurement properties was assessed using the COSMIN checklist. Ten studies were included evaluating five instruments: three for assessing self-care and self-management and two for assessing self-efficacy. The COPD Self-Efficacy Scale was the most studied instrument, but due to poor study methodological quality, evidence about its measurement properties is inconclusive. Evidence from the COPD Self-Management Scale is more promising, but only one study tested its properties. Due to inconclusive evidence of their measurement properties, no instrument can be recommended for clinical use. Copyright © 2016 Elsevier Inc. All rights reserved.

Turkish version of the modified Constant-Murley score and standardized test protocol: reliability and validity.

PubMed

Çelik, Derya

2016-01-01

The Constant-Murley score (CMS) is widely used to evaluate disabilities associated with shoulder injuries, but it has been criticized for relying on imprecise terminology and a lack of standardized methodology. A modified guideline, therefore, was published in 2008 with several recommendations. This new version has not yet been translated or culturally adapted for Turkish-speaking populations. The purpose of this study was to translate and cross-culturally adapt the modified CMS and its test protocol, as well as define and measure its reliability and validity. The modified CMS was translated into Turkish, consistent with published methodological guidelines. The measurement properties of the Turkish version of the modified CMS were tested in 30 patients (12 males, 18 females; mean age: 59.5±13.5 years) with a variety of shoulder pathologies. Intraclass correlation coefficients (ICC) were used to estimate test-retest reliability. Construct validity was analyzed with the Turkish version of the American Shoulder and Elbow Surgeons (ASES) Standardized Shoulder Assessment Form and Short-Form Health Survey (SF-12). No difficulties were found in the translation process. The Turkish version of the modified CMS showed excellent test-retest reliability (ICC=0.86). The correlation coefficients between the Turkish version of the modified CMS and the ASES, SF-12-physical component score, and SF-12 mental component scores were found to be 0.48, 0.35, and 0.05, respectively. No floor or ceiling effects were found. The translation and cultural adaptation of the modified CMS and its standardized test protocol into Turkish were successful. The Turkish version of the modified CMS has sufficient reliability and validity to measure a variety of shoulder disorders for Turkish-speaking individuals.

Evaluation of Behaviours of Laminated Glass

NASA Astrophysics Data System (ADS)

Sable, L.; Japins, G.; Kalnins, K.

2015-11-01

Visual appearance of building facades and other load bearing structures, which now are part of modern architecture, is the reason why it is important to investigate in more detail the reliability of laminated glass for civil structures. Laminated glass in particular has become one of the trendy materials, for example Apple© stores have both load carrying capacity and transparent appearance. Glass has high mechanical strength and relatively medium density, however, the risk of sudden brittle failure like concrete or other ceramics determine relatively high conservatism in design practice of glass structures. This should be changed as consumer requirements evolve calling for a safe and reliable design methodology and corresponding building standards. A design methodology for glass and glass laminates should be urgently developed and included as a chapter in Eurocode. This paper presents initial experimental investigation of behaviour of simple glass sheets and laminated glass samples in 4-point bending test. The aim of the current research is to investigate laminated glass characteristic values and to verify the obtained experimental results with finite element method for glass and EVA material in line with future European Structural Design of Glass Components code.

A novel integrated assessment methodology of urban water reuse.

PubMed

Listowski, A; Ngo, H H; Guo, W S; Vigneswaran, S

2011-01-01

Wastewater is no longer considered a waste product and water reuse needs to play a stronger part in securing urban water supply. Although treatment technologies for water reclamation have significantly improved the question that deserves further analysis is, how selection of a particular wastewater treatment technology relates to performance and sustainability? The proposed assessment model integrates; (i) technology, characterised by selected quantity and quality performance parameters; (ii) productivity, efficiency and reliability criteria; (iii) quantitative performance indicators; (iv) development of evaluation model. The challenges related to hierarchy and selections of performance indicators have been resolved through the case study analysis. The goal of this study is to validate a new assessment methodology in relation to performance of the microfiltration (MF) technology, a key element of the treatment process. Specific performance data and measurements were obtained at specific Control and Data Acquisition Points (CP) to satisfy the input-output inventory in relation to water resources, products, material flows, energy requirements, chemicals use, etc. Performance assessment process contains analysis and necessary linking across important parametric functions leading to reliable outcomes and results.

[A systematic social observation tool: methods and results of inter-rater reliability].

PubMed

Freitas, Eulilian Dias de; Camargos, Vitor Passos; Xavier, César Coelho; Caiaffa, Waleska Teixeira; Proietti, Fernando Augusto

2013-10-01

Systematic social observation has been used as a health research methodology for collecting information from the neighborhood physical and social environment. The objectives of this article were to describe the operationalization of direct observation of the physical and social environment in urban areas and to evaluate the instrument's reliability. The systematic social observation instrument was designed to collect information in several domains. A total of 1,306 street segments belonging to 149 different neighborhoods in Belo Horizonte, Minas Gerais, Brazil, were observed. For the reliability study, 149 segments (1 per neighborhood) were re-audited, and Fleiss kappa was used to access inter-rater agreement. Mean agreement was 0.57 (SD = 0.24); 53% had substantial or almost perfect agreement, and 20.4%, moderate agreement. The instrument appears to be appropriate for observing neighborhood characteristics that are not time-dependent, especially urban services, property characterization, pedestrian environment, and security.

Probabilistic Structural Analysis Methods (PSAM) for select space propulsion system components, part 2

NASA Technical Reports Server (NTRS)

1991-01-01

The technical effort and computer code enhancements performed during the sixth year of the Probabilistic Structural Analysis Methods program are summarized. Various capabilities are described to probabilistically combine structural response and structural resistance to compute component reliability. A library of structural resistance models is implemented in the Numerical Evaluations of Stochastic Structures Under Stress (NESSUS) code that included fatigue, fracture, creep, multi-factor interaction, and other important effects. In addition, a user interface was developed for user-defined resistance models. An accurate and efficient reliability method was developed and was successfully implemented in the NESSUS code to compute component reliability based on user-selected response and resistance models. A risk module was developed to compute component risk with respect to cost, performance, or user-defined criteria. The new component risk assessment capabilities were validated and demonstrated using several examples. Various supporting methodologies were also developed in support of component risk assessment.

The maximum specific hydrogen-producing activity of anaerobic mixed cultures: definition and determination

PubMed Central

Mu, Yang; Yang, Hou-Yun; Wang, Ya-Zhou; He, Chuan-Shu; Zhao, Quan-Bao; Wang, Yi; Yu, Han-Qing

2014-01-01

Fermentative hydrogen production from wastes has many advantages compared to various chemical methods. Methodology for characterizing the hydrogen-producing activity of anaerobic mixed cultures is essential for monitoring reactor operation in fermentative hydrogen production, however there is lack of such kind of standardized methodologies. In the present study, a new index, i.e., the maximum specific hydrogen-producing activity (SHAm) of anaerobic mixed cultures, was proposed, and consequently a reliable and simple method, named SHAm test, was developed to determine it. Furthermore, the influences of various parameters on the SHAm value determination of anaerobic mixed cultures were evaluated. Additionally, this SHAm assay was tested for different types of substrates and bacterial inocula. Our results demonstrate that this novel SHAm assay was a rapid, accurate and simple methodology for determining the hydrogen-producing activity of anaerobic mixed cultures. Thus, application of this approach is beneficial to establishing a stable anaerobic hydrogen-producing system. PMID:24912488

Verification methodology for fault-tolerant, fail-safe computers applied to maglev control computer systems. Final report, July 1991-May 1993

DOE Office of Scientific and Technical Information (OSTI.GOV)

Lala, J.H.; Nagle, G.A.; Harper, R.E.

1993-05-01

The Maglev control computer system should be designed to verifiably possess high reliability and safety as well as high availability to make Maglev a dependable and attractive transportation alternative to the public. A Maglev control computer system has been designed using a design-for-validation methodology developed earlier under NASA and SDIO sponsorship for real-time aerospace applications. The present study starts by defining the maglev mission scenario and ends with the definition of a maglev control computer architecture. Key intermediate steps included definitions of functional and dependability requirements, synthesis of two candidate architectures, development of qualitative and quantitative evaluation criteria, and analyticalmore » modeling of the dependability characteristics of the two architectures. Finally, the applicability of the design-for-validation methodology was also illustrated by applying it to the German Transrapid TR07 maglev control system.« less

Management of the aging of critical safety-related concrete structures in light-water reactor plants

DOE Office of Scientific and Technical Information (OSTI.GOV)

Naus, D.J.; Oland, C.B.; Arndt, E.G.

1990-01-01

The Structural Aging Program has the overall objective of providing the USNRC with an improved basis for evaluating nuclear power plant safety-related structures for continued service. The program consists of a management task and three technical tasks: materials property data base, structural component assessment/repair technology, and quantitative methodology for continued-service determinations. Objectives, accomplishments, and planned activities under each of these tasks are presented. Major program accomplishments include development of a materials property data base for structural materials as well as an aging assessment methodology for concrete structures in nuclear power plants. Furthermore, a review and assessment of inservice inspection techniquesmore » for concrete materials and structures has been complete, and work on development of a methodology which can be used for performing current as well as reliability-based future condition assessment of concrete structures is well under way. 43 refs., 3 tabs.« less

The maximum specific hydrogen-producing activity of anaerobic mixed cultures: definition and determination

NASA Astrophysics Data System (ADS)

Mu, Yang; Yang, Hou-Yun; Wang, Ya-Zhou; He, Chuan-Shu; Zhao, Quan-Bao; Wang, Yi; Yu, Han-Qing

2014-06-01

Fermentative hydrogen production from wastes has many advantages compared to various chemical methods. Methodology for characterizing the hydrogen-producing activity of anaerobic mixed cultures is essential for monitoring reactor operation in fermentative hydrogen production, however there is lack of such kind of standardized methodologies. In the present study, a new index, i.e., the maximum specific hydrogen-producing activity (SHAm) of anaerobic mixed cultures, was proposed, and consequently a reliable and simple method, named SHAm test, was developed to determine it. Furthermore, the influences of various parameters on the SHAm value determination of anaerobic mixed cultures were evaluated. Additionally, this SHAm assay was tested for different types of substrates and bacterial inocula. Our results demonstrate that this novel SHAm assay was a rapid, accurate and simple methodology for determining the hydrogen-producing activity of anaerobic mixed cultures. Thus, application of this approach is beneficial to establishing a stable anaerobic hydrogen-producing system.

Psychometric Properties of the Turkish Version of Caregiver Burden Index (CBI) for Parents of Children with Allergies.

PubMed

Ekim, Ayfer; Hecan, Melis; Oren, Serkan

Childhood chronic diseases have a great impact, including physiological, social and financial burdens, on parents. The concept of "caregiver burden" is gaining importance to understand the effects of allergic diseases and plan family-centered strategies. The purpose of this study was to examine the psychometric properties of the Caregiver Burden Index (CBI) in Turkish mothers of children with allergies. The participants of this methodological study were 213 mothers of children with allergies between 6 and 12years. Construct validity was evaluated through factor analysis and reliability was evaluated through internal consistency and item-total correlation. In reliability analysis, the overall Cronbach's alpha value (0.85) demonstrated a high level of reliability. The corrected item-total correlation varied between 0.63 and 0.84. In exploratory factor analysis, it was detected that 3 factors structure explained 73.6% of the total variance. This study indicated that the CBI is a valid and reliable tool to assess the caregiver burden of mothers of Turkish children with allergies. The results of this study contribute to the development and implementation of evidence based models of care that address the caregiver burden needs of parents whose children have allergies. Copyright © 2017 Elsevier Inc. All rights reserved.

Feasibility of groundwater recharge dam projects in arid environments

NASA Astrophysics Data System (ADS)

Jaafar, H. H.

2014-05-01

A new method for determining feasibility and prioritizing investments for agricultural and domestic recharge dams in arid regions is developed and presented. The method is based on identifying the factors affecting the decision making process and evaluating these factors, followed by determining the indices in a GIS-aided environment. Evaluated parameters include results from field surveys and site visits, land cover and soils data, precipitation data, runoff data and modeling, number of beneficiaries, domestic irrigation demand, reservoir objectives, demography, reservoirs yield and reliability, dam structures, construction costs, and operation and maintenance costs. Results of a case study on more than eighty proposed dams indicate that assessment of reliability, annualized cost/demand satisfied and yield is crucial prior to investment decision making in arid areas. Irrigation demand is the major influencing parameter on yield and reliability of recharge dams, even when only 3 months of the demand were included. Reliability of the proposed reservoirs as related to their standardized size and net inflow was found to increase with increasing yield. High priority dams were less than 4% of the total, and less priority dams amounted to 23%, with the remaining found to be not feasible. The results of this methodology and its application has proved effective in guiding stakeholders for defining most favorable sites for preliminary and detailed design studies and commissioning.

The Development and Validation of a Rapid Assessment Tool of Primary Care in China

PubMed Central

Mei, Jie; Liang, Yuan; Shi, LeiYu; Zhao, JingGe; Wang, YuTan; Kuang, Li

2016-01-01

Introduction. With Chinese health care reform increasingly emphasizing the importance of primary care, the need for a tool to evaluate primary care performance and service delivery is clear. This study presents a methodology for a rapid assessment of primary care organizations and service delivery in China. Methods. The study translated and adapted the Primary Care Assessment Tool-Adult Edition (PCAT-AE) into a Chinese version to measure core dimensions of primary care, namely, first contact, continuity, comprehensiveness, and coordination. A cross-sectional survey was conducted to assess the validity and reliability of the Chinese Rapid Primary Care Assessment Tool (CR-PCAT). Eight community health centers in Guangdong province have been selected to participate in the survey. Results. A total of 1465 effective samples were included for data analysis. Eight items were eliminated following principal component analysis and reliability testing. The principal component analysis extracted five multiple-item scales (first contact utilization, first contact accessibility, ongoing care, comprehensiveness, and coordination). The tests of scaling assumptions were basically met. Conclusion. The standard psychometric evaluation indicates that the scales have achieved relatively good reliability and validity. The CR-PCAT provides a rapid and reliable measure of four core dimensions of primary care, which could be applied in various scenarios. PMID:26885509

Test-Retest Reliability of Pediatric Heart Rate Variability: A Meta-Analysis.

PubMed

Weiner, Oren M; McGrath, Jennifer J

2017-01-01

Heart rate variability (HRV), an established index of autonomic cardiovascular modulation, is associated with health outcomes (e.g., obesity, diabetes) and mortality risk. Time- and frequency-domain HRV measures are commonly reported in longitudinal adult and pediatric studies of health. While test-retest reliability has been established among adults, less is known about the psychometric properties of HRV among infants, children, and adolescents. The objective was to conduct a meta-analysis of the test-retest reliability of time- and frequency-domain HRV measures from infancy to adolescence. Electronic searches (PubMed, PsycINFO; January 1970-December 2014) identified studies with nonclinical samples aged ≤ 18 years; ≥ 2 baseline HRV recordings separated by ≥ 1 day; and sufficient data for effect size computation. Forty-nine studies ( N = 5,170) met inclusion criteria. Methodological variables coded included factors relevant to study protocol, sample characteristics, electrocardiogram (ECG) signal acquisition and preprocessing, and HRV analytical decisions. Fisher's Z was derived as the common effect size. Analyses were age-stratified (infant/toddler < 5 years, n = 3,329; child/adolescent 5-18 years, n = 1,841) due to marked methodological differences across the pediatric literature. Meta-analytic results revealed HRV demonstrated moderate reliability; child/adolescent studies ( Z = 0.62, r = 0.55) had significantly higher reliability than infant/toddler studies ( Z = 0.42, r = 0.40). Relative to other reported measures, HF exhibited the highest reliability among infant/toddler studies ( Z = 0.42, r = 0.40), while rMSSD exhibited the highest reliability among child/adolescent studies ( Z = 1.00, r = 0.76). Moderator analyses indicated greater reliability with shorter test-retest interval length, reported exclusion criteria based on medical illness/condition, lower proportion of males, prerecording acclimatization period, and longer recording duration; differences were noted across age groups. HRV is reliable among pediatric samples. Reliability is sensitive to pertinent methodological decisions that require careful consideration by the researcher. Limited methodological reporting precluded several a priori moderator analyses. Suggestions for future research, including standards specified by Task Force Guidelines, are discussed.

Test-Retest Reliability of Pediatric Heart Rate Variability

PubMed Central

Weiner, Oren M.; McGrath, Jennifer J.

2017-01-01

Heart rate variability (HRV), an established index of autonomic cardiovascular modulation, is associated with health outcomes (e.g., obesity, diabetes) and mortality risk. Time- and frequency-domain HRV measures are commonly reported in longitudinal adult and pediatric studies of health. While test-retest reliability has been established among adults, less is known about the psychometric properties of HRV among infants, children, and adolescents. The objective was to conduct a meta-analysis of the test-retest reliability of time- and frequency-domain HRV measures from infancy to adolescence. Electronic searches (PubMed, PsycINFO; January 1970–December 2014) identified studies with nonclinical samples aged ≤ 18 years; ≥ 2 baseline HRV recordings separated by ≥ 1 day; and sufficient data for effect size computation. Forty-nine studies (N = 5,170) met inclusion criteria. Methodological variables coded included factors relevant to study protocol, sample characteristics, electrocardiogram (ECG) signal acquisition and preprocessing, and HRV analytical decisions. Fisher’s Z was derived as the common effect size. Analyses were age-stratified (infant/toddler < 5 years, n = 3,329; child/adolescent 5–18 years, n = 1,841) due to marked methodological differences across the pediatric literature. Meta-analytic results revealed HRV demonstrated moderate reliability; child/adolescent studies (Z = 0.62, r = 0.55) had significantly higher reliability than infant/toddler studies (Z = 0.42, r = 0.40). Relative to other reported measures, HF exhibited the highest reliability among infant/toddler studies (Z = 0.42, r = 0.40), while rMSSD exhibited the highest reliability among child/adolescent studies (Z = 1.00, r = 0.76). Moderator analyses indicated greater reliability with shorter test-retest interval length, reported exclusion criteria based on medical illness/condition, lower proportion of males, prerecording acclimatization period, and longer recording duration; differences were noted across age groups. HRV is reliable among pediatric samples. Reliability is sensitive to pertinent methodological decisions that require careful consideration by the researcher. Limited methodological reporting precluded several a priori moderator analyses. Suggestions for future research, including standards specified by Task Force Guidelines, are discussed. PMID:29307951

Inter-rater agreement in evaluation of disability: systematic review of reproducibility studies

PubMed Central

Barth, Jürgen; de Boer, Wout E L; Busse, Jason W; Hoving, Jan L; Kedzia, Sarah; Couban, Rachel; Fischer, Katrin; von Allmen, David Y; Spanjer, Jerry

2017-01-01

Objectives To explore agreement among healthcare professionals assessing eligibility for work disability benefits. Design Systematic review and narrative synthesis of reproducibility studies. Data sources Medline, Embase, and PsycINFO searched up to 16 March 2016, without language restrictions, and review of bibliographies of included studies. Eligibility criteria Observational studies investigating reproducibility among healthcare professionals performing disability evaluations using a global rating of working capacity and reporting inter-rater reliability by a statistical measure or descriptively. Studies could be conducted in insurance settings, where decisions on ability to work include normative judgments based on legal considerations, or in research settings, where decisions on ability to work disregard normative considerations.Teams of paired reviewers identified eligible studies, appraised their methodological quality and generalisability, and abstracted results with pretested forms. As heterogeneity of research designs and findings impeded a quantitative analysis, a descriptive synthesis stratified by setting (insurance or research) was performed. Results From 4562 references, 101 full text articles were reviewed. Of these, 16 studies conducted in an insurance setting and seven in a research setting, performed in 12 countries, met the inclusion criteria. Studies in the insurance setting were conducted with medical experts assessing claimants who were actual disability claimants or played by actors, hypothetical cases, or short written scenarios. Conditions were mental (n=6, 38%), musculoskeletal (n=4, 25%), or mixed (n=6, 38%). Applicability of findings from studies conducted in an insurance setting to real life evaluations ranged from generalisable (n=7, 44%) and probably generalisable (n=3, 19%) to probably not generalisable (n=6, 37%). Median inter-rater reliability among experts was 0.45 (range intraclass correlation coefficient 0.86 to κ−0.10). Inter-rater reliability was poor in six studies (37%) and excellent in only two (13%). This contrasts with studies conducted in the research setting, where the median inter-rater reliability was 0.76 (range 0.91-0.53), and 71% (5/7) studies achieved excellent inter-rater reliability. Reliability between assessing professionals was higher when the evaluation was guided by a standardised instrument (23 studies, P=0.006). No such association was detected for subjective or chronic health conditions or the studies’ generalisability to real world evaluation of disability (P=0.46, 0.45, and 0.65, respectively). Conclusions Despite their common use and far reaching consequences for workers claiming disabling injury or illness, research on the reliability of medical evaluations of disability for work is limited and indicates high variation in judgments among assessing professionals. Standardising the evaluation process could improve reliability. Development and testing of instruments and structured approaches to improve reliability in evaluation of disability are urgently needed. PMID:28122727

Use of the script concordance approach to evaluate clinical reasoning in food-ruminant practitioners.

PubMed

Dufour, Simon; Latour, Sylvie; Chicoine, Yvan; Fecteau, Gilles; Forget, Sylvain; Moreau, Jean; Trépanier, André

2012-01-01

A script concordance test (SCT) was developed measuring clinical reasoning of food-ruminant practitioners for whom potential clinical competence difficulties were identified by their provincial professional organization. The SCT was designed to be used as part of a broader evaluation procedure. A scoring key was developed based on answers from a reference panel of 12 experts and using the modified aggregate method commonly used for SCTs. A convenient sample of 29 food-ruminant practitioners was constituted to assess the reliability and precision of the SCT and to determine a fair threshold value for success. Cronbach's α coefficients were computed to evaluate internal reliability. To evaluate SCT precision, a test-retest methodology was used and measures of agreement beyond chance were computed at question and test levels. After optimization, the 36-question SCT yielded acceptable internal reliability (Cronbach's α=0.70). Precision of the SCT at question level was excellent with 33 questions (92%) yielding moderate to almost perfect agreement between administrations. At test level, fair agreement (concordance correlation coefficient=0.32) was observed between administrations. A slight SCT score improvement (M=+2.8 points) on the second administration was in part responsible for some of the disagreement and was potentially a result of an adaptation to the SCT format. Scores distribution was used to determine a fair threshold value for success, while considering the underlying objectives of the examination. The data suggest that the developed SCT can be used as a reliable and precise measurement of clinical reasoning of food-ruminant practitioners.

Methodology Series Module 9: Designing Questionnaires and Clinical Record Forms - Part II.

PubMed

Setia, Maninder Singh

2017-01-01

This article is a continuation of the previous module on designing questionnaires and clinical record form in which we have discussed some basic points about designing the questionnaire and clinical record forms. In this section, we will discuss the reliability and validity of questionnaires. The different types of validity are face validity, content validity, criterion validity, and construct validity. The different types of reliability are test-retest reliability, inter-rater reliability, and intra-rater reliability. Some of these parameters are assessed by subject area experts. However, statistical tests should be used for evaluation of other parameters. Once the questionnaire has been designed, the researcher should pilot test the questionnaire. The items in the questionnaire should be changed based on the feedback from the pilot study participants and the researcher's experience. After the basic structure of the questionnaire has been finalized, the researcher should assess the validity and reliability of the questionnaire or the scale. If an existing standard questionnaire is translated in the local language, the researcher should assess the reliability and validity of the translated questionnaire, and these values should be presented in the manuscript. The decision to use a self- or interviewer-administered, paper- or computer-based questionnaire depends on the nature of the questions, literacy levels of the target population, and resources.

Methodology Series Module 9: Designing Questionnaires and Clinical Record Forms – Part II

PubMed Central

Setia, Maninder Singh

2017-01-01

This article is a continuation of the previous module on designing questionnaires and clinical record form in which we have discussed some basic points about designing the questionnaire and clinical record forms. In this section, we will discuss the reliability and validity of questionnaires. The different types of validity are face validity, content validity, criterion validity, and construct validity. The different types of reliability are test-retest reliability, inter-rater reliability, and intra-rater reliability. Some of these parameters are assessed by subject area experts. However, statistical tests should be used for evaluation of other parameters. Once the questionnaire has been designed, the researcher should pilot test the questionnaire. The items in the questionnaire should be changed based on the feedback from the pilot study participants and the researcher's experience. After the basic structure of the questionnaire has been finalized, the researcher should assess the validity and reliability of the questionnaire or the scale. If an existing standard questionnaire is translated in the local language, the researcher should assess the reliability and validity of the translated questionnaire, and these values should be presented in the manuscript. The decision to use a self- or interviewer-administered, paper- or computer-based questionnaire depends on the nature of the questions, literacy levels of the target population, and resources. PMID:28584367

Reliability of recurrence quantification analysis measures of the center of pressure during standing in individuals with musculoskeletal disorders.

PubMed

Mazaheri, Masood; Negahban, Hossein; Salavati, Mahyar; Sanjari, Mohammad Ali; Parnianpour, Mohamad

2010-09-01

Although the application of nonlinear tools including recurrence quantification analysis (RQA) has increasingly grown in the recent years especially in balance-disordered populations, there have been few studies which determine their measurement properties. Therefore, a methodological study was performed to estimate the intersession and intrasession reliability of some dynamic features provided by RQA for nonlinear analysis of center of pressure (COP) signals recorded during quiet standing in a sample of patients with musculoskeletal disorders (MSDs) including low back pain (LBP), anterior cruciate ligament (ACL) injury and functional ankle instability (FAI). The subjects completed postural measurements with three levels of difficulty (rigid surface-eyes open, rigid surface-eyes closed, and foam surface-eyes closed). Four RQA measures (% recurrence, % determinism, entropy, and trend) were extracted from the recurrence plot. Relative reliability of these measures was assessed using intraclass correlation coefficient and absolute reliability using standard error of measurement and coefficient of variation. % Determinism and entropy were the most reliable features of RQA for the both intersession and intrasession reliability measures. High level of reliability of % determinism and entropy in this preliminary investigation may show their clinical promise for discriminative and evaluative purposes of balance performance. 2010 IPEM. Published by Elsevier Ltd. All rights reserved.

Methodological issues in oral health research: intervention studies.

PubMed

O'Mullane, Denis; James, Patrice; Whelton, Helen; Parnell, Carmel

2012-02-01

To provide a broad overview of methodological issues in the design and evaluation of intervention studies in dental public health, with particular emphasis on explanatory trials, pragmatic trials and complex interventions. We present a narrative summary of selected publications from the literature outlining both historical and recent challenges in the design and evaluation of intervention studies and describe some recent tools that may help researchers to address these challenges. It is now recognised that few intervention studies in dental public health are purely explanatory or pragmatic. We describe the PRECIS tool which can be used by trialists to assess and display the position of their trial on a continuum between the extremes of explanatory and pragmatic trials. The tool aims to help trialists make design decisions that are in line with their stated aims. The increasingly complex nature of dental public health interventions presents particular design and evaluation challenges. The revised Medical Research Council (MRC) guidance for the development and evaluation of complex interventions which emphasises the importance of planning and process evaluation is a welcome development. We briefly describe the MRC guidance and outline some examples of complex interventions in the field of oral health. The role of observational studies in monitoring public health interventions when the conduct of RCTs is not appropriate or feasible is acknowledged. We describe the STROBE statement and outline the implications of the STROBE guidelines for dental public health. The methodological challenges in the design, conduct and reporting of intervention studies in oral health are considerable. The need to provide reliable evidence to support innovative new strategies in oral health policy is a major impetus in these fields. No doubt the 'Methodological Issues in Oral Health Research' group will have further opportunities to highlight this work. © 2012 John Wiley & Sons A/S.

[Estimators of internal consistency in health research: the use of the alpha coefficient].

PubMed

da Silva, Franciele Cascaes; Gonçalves, Elizandra; Arancibia, Beatriz Angélica Valdivia; Bento, Gisele Graziele; Castro, Thiago Luis da Silva; Hernandez, Salma Stephany Soleman; da Silva, Rudney

2015-01-01

Academic production has increased in the area of health, increasingly demanding high quality in publications of great impact. One of the ways to consider quality is through methods that increase the consistency of data analysis, such as reliability which, depending on the type of data, can be evaluated by different coefficients, especially the alpha coefficient. Based on this, the present review systematically gathers scientific articles produced in the last five years, which in a methodological manner gave the α coefficient psychometric use as an estimator of internal consistency and reliability in the processes of construction, adaptation and validation of instruments. The identification of the studies was conducted systematically in the databases BioMed Central Journals, Web of Science, Wiley Online Library, Medline, SciELO, Scopus, Journals@Ovid, BMJ and Springer, using inclusion and exclusion criteria. Data analyses were performed by means of triangulation, content analysis and descriptive analysis. It was found that most studies were conducted in Iran (f=3), Spain (f=2) and Brazil (f=2). These studies aimed to test the psychometric properties of instruments, with eight studies using the α coefficient to assess reliability and nine for assessing internal consistency. All studies were classified as methodological research when their objectives were analyzed. In addition, four studies were also classified as correlational and one as descriptive-correlational. It can be concluded that though the α coefficient is widely used as one of the main parameters for assessing internal consistency of questionnaires in health sciences, its use as an estimator of trust of the methodology used and internal consistency has some critiques that should be considered.

Transient Reliability of Ceramic Structures For Heat Engine Applications

NASA Technical Reports Server (NTRS)

Nemeth, Noel N.; Jadaan, Osama M.

2002-01-01

The objectives of this report was to develop a methodology to predict the time-dependent reliability (probability of failure) of brittle material components subjected to transient thermomechanical loading, taking into account the change in material response with time. This methodology for computing the transient reliability in ceramic components subjected to fluctuation thermomechanical loading was developed, assuming SCG (Slow Crack Growth) as the delayed mode of failure. It takes into account the effect of varying Weibull modulus and materials with time. It was also coded into a beta version of NASA's CARES/Life code, and an example demonstrating its viability was presented.

Measurement properties of patient reported outcome measures for spondyloarthritis: A systematic review.

PubMed

Png, Kelly; Kwan, Yu Heng; Leung, Ying Ying; Phang, Jie Kie; Lau, Jia Qi; Lim, Ka Keat; Chew, Eng Hui; Low, Lian Leng; Tan, Chuen Seng; Thumboo, Julian; Fong, Warren; Østbye, Truls

2018-03-21

This systematic review aimed to identify studies investigating measurement properties of patient reported outcome measures (PROMs) for spondyloarthritis (SpA), and to evaluate their methodological quality and level of evidence relating to the measurement properties of PROMs. This systematic review was guided by the preferred reporting items for systematic review and meta-analysis (PRISMA). Articles published before 30 June 2017 were retrieved from PubMed ® , Embase ® , and PsychINFO ® (Ovid). Methodological quality and level of evidence were evaluated according to recommendations from the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN). We identified 60 unique PROMs from 125 studies in 39 countries. Twenty-one PROMs were validated for two or more SpA subtypes. The literature examined hypothesis testing (82.4%) most frequently followed by reliability (60.0%). A percentage of 77.7% and 42.7% of studies that assessed PROMs for hypothesis testing and reliability, respectively had "fair" or better methodological quality. Among the PROMs identified, 41.7% were studied in ankylosing spondylitis (AS) only and 23.3% were studied in psoriatic arthritis (PsA) only. The more extensively assessed PROMs included the ankylosing spondylitis quality of life (ASQoL) and bath ankylosing spondylitis functional index (BASFI) for ankylosing spondylitis, and the psoriatic arthritis quality of life questionnaire (VITACORA-19) for psoriatic arthritis. This study identified 60 unique PROMs through a systematic review and synthesized evidence of the measurement properties of the PROMs. There is a lack of validation of PROMs for use across SpA subtypes. Future studies may consider validating PROMs for use across different SpA subtypes. Copyright © 2018 Elsevier Inc. All rights reserved.

Turkish Version of Kolcaba's Immobilization Comfort Questionnaire: A Validity and Reliability Study.

PubMed

Tosun, Betül; Aslan, Özlem; Tunay, Servet; Akyüz, Aygül; Özkan, Hüseyin; Bek, Doğan; Açıksöz, Semra

2015-12-01

The purpose of this study was to determine the validity and reliability of the Turkish version of the Immobilization Comfort Questionnaire (ICQ). The sample used in this methodological study consisted of 121 patients undergoing lower extremity arthroscopy in a training and research hospital. The validity study of the questionnaire assessed language validity, structural validity and criterion validity. Structural validity was evaluated via exploratory factor analysis. Criterion validity was evaluated by assessing the correlation between the visual analog scale (VAS) scores (i.e., the comfort and pain VAS scores) and the ICQ scores using Spearman's correlation test. The Kaiser-Meyer-Olkin coefficient and Bartlett's test of sphericity were used to determine the suitability of the data for factor analysis. Internal consistency was evaluated to determine reliability. The data were analyzed with SPSS version 15.00 for Windows. Descriptive statistics were presented as frequencies, percentages, means and standard deviations. A p value ≤ .05 was considered statistically significant. A moderate positive correlation was found between the ICQ scores and the VAS comfort scores; a moderate negative correlation was found between the ICQ and the VAS pain measures in the criterion validity analysis. Cronbach α values of .75 and .82 were found for the first and second measurements, respectively. The findings of this study reveal that the ICQ is a valid and reliable tool for assessing the comfort of patients in Turkey who are immobilized because of lower extremity orthopedic problems. Copyright © 2015. Published by Elsevier B.V.

Cross-cultural adaptation of the Nordic musculoskeletal questionnaire.

PubMed

de Barros, E N C; Alexandre, N M C

2003-06-01

Reports in the literature have identified a need for internationally standardized and reliable measurements to analyse musculoskeletal symptoms. Screening of musculoskeletal disorders may serve as a diagnostic tool to evaluate the work environment. The Nordic general questionnaire is a standardized instrument used to analyse musculoskeletal symptoms in an ergonomic or occupational health context. To translate and adapt a version of the Nordic general questionnaire into Brazilian Portuguese and evaluate its reliability. The cross-cultural adaptation was performed according to internationally recommended methodology, using the following guidelines: translation; back-translation; committee review; and pretesting. First, the questionnaire was independently translated into Portuguese by two teachers and one doctor, and a consensus version was generated. Second, two other translators performed a back-translation independently from one another. This version was then submitted to a committee, consisting of six specialists in the area of knowledge of the instrument, to evaluate its equivalence to the original instrument. The final version was pretested on 20 subjects randomly selected in an outpatient clinic. Reliability was assessed by a test-retest procedure at 1-day intervals using the Kappa coefficient in a group of 40 subjects. The Kappa agreement values were calculated for each one of the four questions of the questionnaire. The agreement among the same observers was substantial, varying from 0.88 to 1, according to the Kappa values. these demonstrated strong agreement of the instrument, suggesting that the Brazilian version of the "Standardized Nordic Questionnaire" offers substantial reliability.

Reproducibility, Reliability, and Validity of Fuchsin-Based Beads for the Evaluation of Masticatory Performance.

PubMed

Sánchez-Ayala, Alfonso; Farias-Neto, Arcelino; Vilanova, Larissa Soares Reis; Costa, Marina Abrantes; Paiva, Ana Clara Soares; Carreiro, Adriana da Fonte Porto; Mestriner-Junior, Wilson

2016-08-01

Rehabilitation of masticatory function is inherent to prosthodontics; however, despite the various techniques for evaluating oral comminution, the methodological suitability of these has not been completely studied. The aim of this study was to determine the reproducibility, reliability, and validity of a test food based on fuchsin beads for masticatory function assessment. Masticatory performance was evaluated in 20 dentate subjects (mean age, 23.3 years) using two kinds of test foods and methods: fuchsin beads and ultraviolet-visible spectrophotometry, and silicone cubes and multiple sieving as gold standard. Three examiners conducted five masticatory performance trials with each test food. Reproducibility of the results from both test foods was separately assessed using the intraclass correlation coefficient (ICC). Reliability and validity of fuchsin bead data were measured by comparing the average mean of absolute differences and the measurement means, respectively, regarding silicone cube data using the paired Student's t-test (α = 0.05). Intraexaminer and interexaminer ICC for the fuchsin bead values were 0.65 and 0.76 (p < 0.001), respectively; those for the silicone cubes values were 0.93 and 0.91 (p < 0.001), respectively. Reliability revealed intraexaminer (p < 0.001) and interexaminer (p < 0.05) differences between the average means of absolute differences of each test foods. Validity also showed differences between the measurement means of each test food (p < 0.001). Intra- and interexaminer reproducibility of the test food based on fuchsin beads for evaluation of masticatory performance were good and excellent, respectively; however, the reliability and validity were low, because fuchsin beads do not measure the grinding capacity of masticatory function as silicone cubes do; instead, this test food describes the crushing potential of teeth. Thus, the two kinds of test foods evaluate different properties of masticatory capacity, confirming fushsin beads as a useful tool for this purpose. © 2015 by the American College of Prosthodontists.

A multimedia approach for teaching human embryology: Development and evaluation of a methodology.

PubMed

Moraes, Suzana Guimarães; Pereira, Luis Antonio Violin

2010-12-20

Human embryology requires students to understand the simultaneous changes in embryos, but students find it difficult to grasp the concepts presented and to visualise the related processes in three dimensions. The aims of this study have been to develop and evaluate new educational materials and a teaching methodology based on multimedia approaches to improve the comprehension of human development. The materials developed at the State University of Campinas include clinical histories, movies, animations, and ultrasound, as well as autopsy images from embryos and foetuses. The series of embryology lectures were divided into two parts. The first part of the series addressed the development of the body's structures, while in the second part, clinical history and the corresponding materials were shown to the students, who were encouraged to discuss the malformations. The teaching materials were made available on software used by the students in classes. At the end of the discipline, the material and methodology were evaluated with an attitudinal instrument, interviews, and knowledge examination. The response rate to the attitudinal instrument was 95.35%, and the response rate to the interview was 46%. The students approved of the materials and the teaching methodology (reliability of the attitudinal instrument was 0.9057). The exams showed that most students scored above 6.0. A multimedia approach proved useful for solving an important problem associated with teaching methods in many medical institutions: the lack of integration between basic sciences and clinical disciplines. 2010 Elsevier GmbH. All rights reserved.

An assessment of routine primary care health information system data quality in Sofala Province, Mozambique

PubMed Central

2011-01-01

Background Primary health care is recognized as a main driver of equitable health service delivery. For it to function optimally, routine health information systems (HIS) are necessary to ensure adequate provision of health care and the development of appropriate health policies. Concerns about the quality of routine administrative data have undermined their use in resource-limited settings. This evaluation was designed to describe the availability, reliability, and validity of a sample of primary health care HIS data from nine health facilities across three districts in Sofala Province, Mozambique. HIS data were also compared with results from large community-based surveys. Methodology We used a methodology similar to the Global Fund to Fight AIDS, Tuberculosis and Malaria data verification bottom-up audit to assess primary health care HIS data availability and reliability. The quality of HIS data was validated by comparing three key indicators (antenatal care, institutional birth, and third diptheria, pertussis, and tetanus [DPT] immunization) with population-level surveys over time. Results and discussion The data concordance from facility clinical registries to monthly facility reports on five key indicators--the number of first antenatal care visits, institutional births, third DPT immunization, HIV testing, and outpatient consults--was good (80%). When two sites were excluded from the analysis, the concordance was markedly better (92%). Of monthly facility reports for immunization and maternity services, 98% were available in paper form at district health departments and 98% of immunization and maternity services monthly facility reports matched the Ministry of Health electronic database. Population-level health survey and HIS data were strongly correlated (R = 0.73), for institutional birth, first antenatal care visit, and third DPT immunization. Conclusions Our results suggest that in this setting, HIS data are both reliable and consistent, supporting their use in primary health care program monitoring and evaluation. Simple, rapid tools can be used to evaluate routine data and facilitate the rapid identification of problem areas. PMID:21569533

Mechanical system reliability for long life space systems

NASA Technical Reports Server (NTRS)

Kowal, Michael T.

1994-01-01

The creation of a compendium of mechanical limit states was undertaken in order to provide a reference base for the application of first-order reliability methods to mechanical systems in the context of the development of a system level design methodology. The compendium was conceived as a reference source specific to the problem of developing the noted design methodology, and not an exhaustive or exclusive compilation of mechanical limit states. The compendium is not intended to be a handbook of mechanical limit states for general use. The compendium provides a diverse set of limit-state relationships for use in demonstrating the application of probabilistic reliability methods to mechanical systems. The compendium is to be used in the reliability analysis of moderately complex mechanical systems.

Error Estimation and Uncertainty Propagation in Computational Fluid Mechanics

NASA Technical Reports Server (NTRS)

Zhu, J. Z.; He, Guowei; Bushnell, Dennis M. (Technical Monitor)

2002-01-01

Numerical simulation has now become an integral part of engineering design process. Critical design decisions are routinely made based on the simulation results and conclusions. Verification and validation of the reliability of the numerical simulation is therefore vitally important in the engineering design processes. We propose to develop theories and methodologies that can automatically provide quantitative information about the reliability of the numerical simulation by estimating numerical approximation error, computational model induced errors and the uncertainties contained in the mathematical models so that the reliability of the numerical simulation can be verified and validated. We also propose to develop and implement methodologies and techniques that can control the error and uncertainty during the numerical simulation so that the reliability of the numerical simulation can be improved.

The Scaling of Performance and Losses in Miniature Internal Combustion Engines

DTIC Science & Technology

2010-01-01

reliable measurements of engine performance and losses in these small engines. Methodologies are also developed for measuring volumetric, heat transfer...making reliable measurements of engine performance and losses in these small engines. Methodologies are also developed for measuring volumetric, heat ...the most important challenge as it accounts for 60-70% of total energy losses. Combustion losses are followed in order of importance by heat transfer

Process perspective on image quality evaluation

NASA Astrophysics Data System (ADS)

Leisti, Tuomas; Halonen, Raisa; Kokkonen, Anna; Weckman, Hanna; Mettänen, Marja; Lensu, Lasse; Ritala, Risto; Oittinen, Pirkko; Nyman, Göte

2008-01-01

The psychological complexity of multivariate image quality evaluation makes it difficult to develop general image quality metrics. Quality evaluation includes several mental processes and ignoring these processes and the use of a few test images can lead to biased results. By using a qualitative/quantitative (Interpretation Based Quality, IBQ) methodology, we examined the process of pair-wise comparison in a setting, where the quality of the images printed by laser printer on different paper grades was evaluated. Test image consisted of a picture of a table covered with several objects. Three other images were also used, photographs of a woman, cityscape and countryside. In addition to the pair-wise comparisons, observers (N=10) were interviewed about the subjective quality attributes they used in making their quality decisions. An examination of the individual pair-wise comparisons revealed serious inconsistencies in observers' evaluations on the test image content, but not on other contexts. The qualitative analysis showed that this inconsistency was due to the observers' focus of attention. The lack of easily recognizable context in the test image may have contributed to this inconsistency. To obtain reliable knowledge of the effect of image context or attention on subjective image quality, a qualitative methodology is needed.

Some consideration for evaluation of structural integrity of aging aircraft

NASA Astrophysics Data System (ADS)

Terada, Hiroyuki; Asada, Hiroo

The objective of this paper is to examine the achievement and the limitation of state-of-the-art of the methodology of damage tolerant design and the subjects to be solved for further improvement. The topics discussed are: the basic concept of full-scale fatigue testing, fracture mechanics applications, repair of detected damages, inspection technology, and determination of inspection intervals, reliability assessment for practical application, and the importance of various kinds of data acquisition.

Structural design considerations for micromachined solid-oxide fuel cells

NASA Astrophysics Data System (ADS)

Srikar, V. T.; Turner, Kevin T.; Andrew Ie, Tze Yung; Spearing, S. Mark

Micromachined solid-oxide fuel cells (μSOFCs) are among a class of devices being investigated for portable power generation. Optimization of the performance and reliability of such devices requires robust, scale-dependent, design methodologies. In this first analysis, we consider the structural design of planar, electrolyte-supported, μSOFCs from the viewpoints of electrochemical performance, mechanical stability and reliability, and thermal behavior. The effect of electrolyte thickness on fuel cell performance is evaluated using a simple analytical model. Design diagrams that account explicitly for thermal and intrinsic residual stresses are presented to identify geometries that are resistant to fracture and buckling. Analysis of energy loss due to in-plane heat conduction highlights the importance of efficient thermal isolation in microscale fuel cell design.

Reliability of physical examination tests for the diagnosis of knee disorders: Evidence from a systematic review.

PubMed

Décary, Simon; Ouellet, Philippe; Vendittoli, Pascal-André; Desmeules, François

2016-12-01

Clinicians often rely on physical examination tests to guide them in the diagnostic process of knee disorders. However, reliability of these tests is often overlooked and may influence the consistency of results and overall diagnostic validity. Therefore, the objective of this study was to systematically review evidence on the reliability of physical examination tests for the diagnosis of knee disorders. A structured literature search was conducted in databases up to January 2016. Included studies needed to report reliability measures of at least one physical test for any knee disorder. Methodological quality was evaluated using the QAREL checklist. A qualitative synthesis of the evidence was performed. Thirty-three studies were included with a mean QAREL score of 5.5 ± 0.5. Based on low to moderate quality evidence, the Thessaly test for meniscal injuries reached moderate inter-rater reliability (k = 0.54). Based on moderate to excellent quality evidence, the Lachman for anterior cruciate ligament injuries reached moderate to excellent inter-rater reliability (k = 0.42 to 0.81). Based on low to moderate quality evidence, the Tibiofemoral Crepitus, Joint Line and Patellofemoral Pain/Tenderness, Bony Enlargement and Joint Pain on Movement tests for knee osteoarthritis reached fair to excellent inter-rater reliability (k = 0.29 to 0.93). Based on low to moderate quality evidence, the Lateral Glide, Lateral Tilt, Lateral Pull and Quality of Movement tests for patellofemoral pain reached moderate to good inter-rater reliability (k = 0.49 to 0.73). Many physical tests appear to reach good inter-rater reliability, but this is based on low-quality and conflicting evidence. High-quality research is required to evaluate the reliability of knee physical examination tests. Copyright © 2016 Elsevier Ltd. All rights reserved.

DXA in the assessment of subchondral bone mineral density in knee osteoarthritis--A semi-standardized protocol after systematic review.

PubMed

Sepriano, Alexandre; Roman-Blas, Jorge A; Little, Robert D; Pimentel-Santos, Fernando; Arribas, Jose María; Largo, Raquel; Branco, Jaime C; Herrero-Beaumont, Gabriel

2015-12-01

Subchondral bone mineral density (sBMD) contributes to the initiation and progression of knee osteoarthritis (OA). Reliable methods to assess sBMD status may predict the response of specific OA phenotypes to targeted therapies. While dual-energy X-ray absorptiometry (DXA) of the knee can determine sBMD, no consensus exists regarding its methodology. Construct a semi-standardized protocol for knee DXA to measure sBMD in patients with OA of the knee by evaluating the varying methodologies present in existing literature. We performed a systematic review of original papers published in PubMed and Web of Science from their inception to July 2014 using the following search terms: subchondral bone, osteoarthritis, and bone mineral density. DXA of the knee can be performed with similar reproducibility values to those proposed by the International Society for Clinical Densitometry for the hip and spine. We identified acquisition view, hip rotation, knee positioning and stabilization, ROI location and definition, and the type of analysis software as important sources of variation. A proposed knee DXA protocol was constructed taking into consideration the results of the review. DXA of the knee can be reliably performed in patients with knee OA. Nevertheless, we found substantial methodological variation across previous studies. Methodological standardization may provide a foundation from which to establish DXA of the knee as a valid tool for identification of SB changes and as an outcome measure in clinical trials of disease modifying osteoarthritic drugs. Copyright © 2015 Elsevier Inc. All rights reserved.

Measurement properties of patient-reported outcome measures (PROMS) in Patellofemoral Pain Syndrome: a systematic review.

PubMed

Green, Andrew; Liles, Clive; Rushton, Alison; Kyte, Derek G

2014-12-01

This systematic review investigated the measurement properties of disease-specific patient-reported outcome measures used in Patellofemoral Pain Syndrome. Two independent reviewers conducted a systematic search of key databases (MEDLINE, EMBASE, AMED, CINHAL+ and the Cochrane Library from inception to August 2013) to identify relevant studies. A third reviewer mediated in the event of disagreement. Methodological quality was evaluated using the validated COSMIN (Consensus-based Standards for the Selection of Health Measurement Instruments) tool. Data synthesis across studies determined the level of evidence for each patient-reported outcome measure. The search strategy returned 2177 citations. Following the eligibility review phase, seven studies, evaluating twelve different patient-reported outcome measures, met inclusion criteria. A 'moderate' level of evidence supported the structural validity of several measures: the Flandry Questionnaire, Anterior Knee Pain Scale, Functional Index Questionnaire, Eng and Pierrynowski Questionnaire and Visual Analogue Scales for 'usual' and 'worst' pain. In addition, there was a 'Limited' level of evidence supporting the test-retest reliability and validity (cross-cultural, hypothesis testing) of the Persian version of the Anterior Knee Pain Scale. Other measurement properties were evaluated with poor methodological quality, and many properties were not evaluated in any of the included papers. Current disease-specific outcome measures for Patellofemoral Pain Syndrome require further investigation. Future studies should evaluate all important measurement properties, utilising an appropriate framework such as COSMIN to guide study design, to facilitate optimal methodological quality. Copyright © 2014 Elsevier Ltd. All rights reserved.

A Comprehensive, Multi-modal Evaluation of the Assessment System of an Undergraduate Research Methodology Course: Translating Theory into Practice.

PubMed

Mohammad Abdulghani, Hamza; G Ponnamperuma, Gominda; Ahmad, Farah; Amin, Zubair

2014-03-01

To evaluate assessment system of the 'Research Methodology Course' using utility criteria (i.e. validity, reliability, acceptability, educational impact, and cost-effectiveness). This study demonstrates comprehensive evaluation of assessment system and suggests a framework for similar courses. Qualitative and quantitative methods used for evaluation of the course assessment components (50 MCQ, 3 Short Answer Questions (SAQ) and research project) using the utility criteria. RESULTS of multiple evaluation methods for all the assessment components were collected and interpreted together to arrive at holistic judgments, rather than judgments based on individual methods or individual assessment. Face validity, evaluated using a self-administered questionnaire (response rate-88.7%) disclosed that the students perceived that there was an imbalance in the contents covered by the assessment. This was confirmed by the assessment blueprint. Construct validity was affected by the low correlation between MCQ and SAQ scores (r=0.326). There was a higher correlation between the project and MCQ (r=0.466)/SAQ (r=0.463) scores. Construct validity was also affected by the presence of recall type of MCQs (70%; 35/50), item construction flaws and non-functioning distractors. High discriminating indices (>0.35) were found in MCQs with moderate difficulty indices (0.3-0.7). Reliability of the MCQs was 0.75 which could be improved up to 0.8 by increasing the number of MCQs to at least 70. A positive educational impact was found in the form of the research project assessment driving students to present/publish their work in conferences/peer reviewed journals. Cost per student to complete the course was US$164.50. The multi-modal evaluation of an assessment system is feasible and provides thorough and diagnostic information. Utility of the assessment system could be further improved by modifying the psychometrically inappropriate assessment items.

A Comprehensive, Multi-modal Evaluation of the Assessment System of an Undergraduate Research Methodology Course: Translating Theory into Practice

PubMed Central

Mohammad Abdulghani, Hamza; G. Ponnamperuma, Gominda; Ahmad, Farah; Amin, Zubair

2014-01-01

Objective: To evaluate assessment system of the 'Research Methodology Course' using utility criteria (i.e. validity, reliability, acceptability, educational impact, and cost-effectiveness). This study demonstrates comprehensive evaluation of assessment system and suggests a framework for similar courses. Methods: Qualitative and quantitative methods used for evaluation of the course assessment components (50 MCQ, 3 Short Answer Questions (SAQ) and research project) using the utility criteria. Results of multiple evaluation methods for all the assessment components were collected and interpreted together to arrive at holistic judgments, rather than judgments based on individual methods or individual assessment. Results: Face validity, evaluated using a self-administered questionnaire (response rate-88.7%) disclosed that the students perceived that there was an imbalance in the contents covered by the assessment. This was confirmed by the assessment blueprint. Construct validity was affected by the low correlation between MCQ and SAQ scores (r=0.326). There was a higher correlation between the project and MCQ (r=0.466)/SAQ (r=0.463) scores. Construct validity was also affected by the presence of recall type of MCQs (70%; 35/50), item construction flaws and non-functioning distractors. High discriminating indices (>0.35) were found in MCQs with moderate difficulty indices (0.3-0.7). Reliability of the MCQs was 0.75 which could be improved up to 0.8 by increasing the number of MCQs to at least 70. A positive educational impact was found in the form of the research project assessment driving students to present/publish their work in conferences/peer reviewed journals. Cost per student to complete the course was US$164.50. Conclusions: The multi-modal evaluation of an assessment system is feasible and provides thorough and diagnostic information. Utility of the assessment system could be further improved by modifying the psychometrically inappropriate assessment items. PMID:24772117

Recent advances in computational structural reliability analysis methods

NASA Astrophysics Data System (ADS)

Thacker, Ben H.; Wu, Y.-T.; Millwater, Harry R.; Torng, Tony Y.; Riha, David S.

1993-10-01

The goal of structural reliability analysis is to determine the probability that the structure will adequately perform its intended function when operating under the given environmental conditions. Thus, the notion of reliability admits the possibility of failure. Given the fact that many different modes of failure are usually possible, achievement of this goal is a formidable task, especially for large, complex structural systems. The traditional (deterministic) design methodology attempts to assure reliability by the application of safety factors and conservative assumptions. However, the safety factor approach lacks a quantitative basis in that the level of reliability is never known and usually results in overly conservative designs because of compounding conservatisms. Furthermore, problem parameters that control the reliability are not identified, nor their importance evaluated. A summary of recent advances in computational structural reliability assessment is presented. A significant level of activity in the research and development community was seen recently, much of which was directed towards the prediction of failure probabilities for single mode failures. The focus is to present some early results and demonstrations of advanced reliability methods applied to structural system problems. This includes structures that can fail as a result of multiple component failures (e.g., a redundant truss), or structural components that may fail due to multiple interacting failure modes (e.g., excessive deflection, resonate vibration, or creep rupture). From these results, some observations and recommendations are made with regard to future research needs.

Recent advances in computational structural reliability analysis methods

NASA Technical Reports Server (NTRS)

Thacker, Ben H.; Wu, Y.-T.; Millwater, Harry R.; Torng, Tony Y.; Riha, David S.

1993-01-01

The goal of structural reliability analysis is to determine the probability that the structure will adequately perform its intended function when operating under the given environmental conditions. Thus, the notion of reliability admits the possibility of failure. Given the fact that many different modes of failure are usually possible, achievement of this goal is a formidable task, especially for large, complex structural systems. The traditional (deterministic) design methodology attempts to assure reliability by the application of safety factors and conservative assumptions. However, the safety factor approach lacks a quantitative basis in that the level of reliability is never known and usually results in overly conservative designs because of compounding conservatisms. Furthermore, problem parameters that control the reliability are not identified, nor their importance evaluated. A summary of recent advances in computational structural reliability assessment is presented. A significant level of activity in the research and development community was seen recently, much of which was directed towards the prediction of failure probabilities for single mode failures. The focus is to present some early results and demonstrations of advanced reliability methods applied to structural system problems. This includes structures that can fail as a result of multiple component failures (e.g., a redundant truss), or structural components that may fail due to multiple interacting failure modes (e.g., excessive deflection, resonate vibration, or creep rupture). From these results, some observations and recommendations are made with regard to future research needs.

Does Maltreatment Beget Maltreatment? A Systematic Review of the Intergenerational Literature

PubMed Central

Thornberry, Terence P.; Knight, Kelly E.; Lovegrove, Peter J.

2014-01-01

In this paper, we critically review the literature testing the cycle of maltreatment hypothesis which posits continuity in maltreatment across adjacent generations. That is, we examine whether a history of maltreatment victimization is a significant risk factor for the later perpetration of maltreatment. We begin by establishing 11 methodological criteria that studies testing this hypothesis should meet. They include such basic standards as using representative samples, valid and reliable measures, prospective designs, and different reporters for each generation. We identify 47 studies that investigated this issue and then evaluate them with regard to the 11 methodological criteria. Overall, most of these studies report findings consistent with the cycle of maltreatment hypothesis. Unfortunately, at the same time, few of them satisfy the basic methodological criteria that we established; indeed, even the stronger studies in this area only meet about half of them. Moreover, the methodologically stronger studies present mixed support for the hypothesis. As a result, the positive association often reported in the literature appears to be based largely on the methodologically weaker designs. Based on our systematic methodological review, we conclude that this small and methodologically weak body of literature does not provide a definitive test of the cycle of maltreatment hypothesis. We conclude that it is imperative to develop more robust and methodologically adequate assessments of this hypothesis to more accurately inform the development of prevention and treatment programs. PMID:22673145

Reliability, validity, and responsiveness of the Persian version of Shoulder Activity Scale in a group of patients with shoulder disorders.

PubMed

Negahban, Hossein; Mohtasebi, Elham; Goharpey, Shahin

2015-01-01

The aim of this methodological study was to cross-culturally translate the Shoulder Activity Scale (SAS) into the Persian and determine its clinimetric properties including reliability, validity, and responsiveness in patients with shoulder disorders. Persian version of the SAS was obtained after standard forward-backward translation. Three questionnaires were completed by the respondents: SAS, shoulder pain and disability index (SPADI), and Short-Form 36 Health Survey (SF-36). The patients completed the SAS, 1 week after the first visit to evaluate the test-retest reliability. Construct validity was evaluated by examining the associations between the scores on the SAS and the scores obtained from the SPADI, SF-36, and age of the patients. To assess responsiveness, data were collected in the first visit and then again after 4 weeks physiotherapy intervention. Test-retest reliability and internal consistency were assessed using Intra-class Correlation Coefficient (ICC) and Cronbach's alpha, respectively. To evaluate construct validity, Spearman's rank correlation was used. The ability of the SAS to detect changes was evaluated by the receiver-operating characteristics method. No problem or language difficulties were reported during translation process. Test-retest reliability of the SAS was excellent with an ICC of 0.98. Also, the marginal Cronbach's alpha level of 0.64 was obtained. The correlation between the SAS and the SPADI was low, proving divergent validity, whereas the correlations between the SAS and the SF-36/age were moderate proving convergent validity. A marginally acceptable responsiveness was achieved for the Persian SAS. The study provides some evidences to support the test-retest reliability, internal consistency, construct validity, and responsiveness of the Persian version of the SAS in patients with shoulder disorders. Therefore, it seems that this instrument is a useful measure of shoulder activity level in research setting and clinical practice. The shoulder activity scale (SAS) is a reliable, valid, and responsive measure of shoulder activity level in Persian-speaking patients with different shoulder disorders. The results on clinimetric properties of the Persian SAS are comparable with its original, English version. Persian version of the SAS can be used in "clinical" and "research" settings of patients with shoulder disorders.

Searching for qualitative research for inclusion in systematic reviews: a structured methodological review.

PubMed

Booth, Andrew

2016-05-04

Qualitative systematic reviews or qualitative evidence syntheses (QES) are increasingly recognised as a way to enhance the value of systematic reviews (SRs) of clinical trials. They can explain the mechanisms by which interventions, evaluated within trials, might achieve their effect. They can investigate differences in effects between different population groups. They can identify which outcomes are most important to patients, carers, health professionals and other stakeholders. QES can explore the impact of acceptance, feasibility, meaningfulness and implementation-related factors within a real world setting and thus contribute to the design and further refinement of future interventions. To produce valid, reliable and meaningful QES requires systematic identification of relevant qualitative evidence. Although the methodologies of QES, including methods for information retrieval, are well-documented, little empirical evidence exists to inform their conduct and reporting. This structured methodological overview examines papers on searching for qualitative research identified from the Cochrane Qualitative and Implementation Methods Group Methodology Register and from citation searches of 15 key papers. A single reviewer reviewed 1299 references. Papers reporting methodological guidance, use of innovative methodologies or empirical studies of retrieval methods were categorised under eight topical headings: overviews and methodological guidance, sampling, sources, structured questions, search procedures, search strategies and filters, supplementary strategies and standards. This structured overview presents a contemporaneous view of information retrieval for qualitative research and identifies a future research agenda. This review concludes that poor empirical evidence underpins current information practice in information retrieval of qualitative research. A trend towards improved transparency of search methods and further evaluation of key search procedures offers the prospect of rapid development of search methods.

Systematic review on measurement properties of questionnaires assessing the neighbourhood environment in the context of youth physical activity behaviour

PubMed Central

2013-01-01

Background High-quality measurement instruments for assessing the neighbourhood environment are a prerequisite for identifying associations between the neighbourhood environment and a person’s physical activity. The aim of this systematic review was to identify reliable and valid questionnaires assessing neighbourhood environmental attributes in the context of physical activity behaviours in children and adolescents. In addition, current gaps and best practice models in instrumentation and their evaluation are discussed. Methods We conducted a systematic literature search using six databases (Web of Science, Medline, TRID, SportDISCUS, PsycARTICLES and PsycINFO). Two independent reviewers screened the identified English-language peer-reviewed journal articles. Only studies examining the measurement properties of self- or proxy-report questionnaires on any aspects of the neighbourhood environment in children and adolescents aged 3 to 18 years were included. The methodological quality of the included studies was assessed using the COSMIN checklists. Results We identified 13 questionnaires on attributes of the neighbourhood environment. Most of these studies were conducted in the United States (n = 7). Eight studies evaluated self-report measures, two studies evaluated parent-report measures and three studies included both administration types. While eight studies had poor methodological quality, we identified three questionnaires with substantial test-retest reliability and two questionnaires with acceptable convergent validity based on sufficient evidential basis. Conclusions Based on the results of this review, we recommend that cross-culturally adapted questionnaires should be used and that existing questionnaires should be evaluated especially in diverse samples and in countries other than the United States. Further, high-quality studies on measurement properties should be promoted and measurement models (formative vs. reflexive) should be specified to ensure that appropriate methods for psychometric testing are applied in future studies. PMID:23663328

Systematic review on measurement properties of questionnaires assessing the neighbourhood environment in the context of youth physical activity behaviour.

PubMed

Reimers, Anne K; Mess, Filip; Bucksch, Jens; Jekauc, Darko; Woll, Alexander

2013-05-11

High-quality measurement instruments for assessing the neighbourhood environment are a prerequisite for identifying associations between the neighbourhood environment and a person's physical activity. The aim of this systematic review was to identify reliable and valid questionnaires assessing neighbourhood environmental attributes in the context of physical activity behaviours in children and adolescents. In addition, current gaps and best practice models in instrumentation and their evaluation are discussed. We conducted a systematic literature search using six databases (Web of Science, Medline, TRID, SportDISCUS, PsycARTICLES and PsycINFO). Two independent reviewers screened the identified English-language peer-reviewed journal articles. Only studies examining the measurement properties of self- or proxy-report questionnaires on any aspects of the neighbourhood environment in children and adolescents aged 3 to 18 years were included. The methodological quality of the included studies was assessed using the COSMIN checklists. We identified 13 questionnaires on attributes of the neighbourhood environment. Most of these studies were conducted in the United States (n = 7). Eight studies evaluated self-report measures, two studies evaluated parent-report measures and three studies included both administration types. While eight studies had poor methodological quality, we identified three questionnaires with substantial test-retest reliability and two questionnaires with acceptable convergent validity based on sufficient evidential basis. Based on the results of this review, we recommend that cross-culturally adapted questionnaires should be used and that existing questionnaires should be evaluated especially in diverse samples and in countries other than the United States. Further, high-quality studies on measurement properties should be promoted and measurement models (formative vs. reflexive) should be specified to ensure that appropriate methods for psychometric testing are applied in future studies.

Using meta-quality to assess the utility of volunteered geographic information for science.

PubMed

Langley, Shaun A; Messina, Joseph P; Moore, Nathan

2017-11-06

Volunteered geographic information (VGI) has strong potential to be increasingly valuable to scientists in collaboration with non-scientists. The abundance of mobile phones and other wireless forms of communication open up significant opportunities for the public to get involved in scientific research. As these devices and activities become more abundant, questions of uncertainty and error in volunteer data are emerging as critical components for using volunteer-sourced spatial data. Here we present a methodology for using VGI and assessing its sensitivity to three types of error. More specifically, this study evaluates the reliability of data from volunteers based on their historical patterns. The specific context is a case study in surveillance of tsetse flies, a health concern for being the primary vector of African Trypanosomiasis. Reliability, as measured by a reputation score, determines the threshold for accepting the volunteered data for inclusion in a tsetse presence/absence model. Higher reputation scores are successful in identifying areas of higher modeled tsetse prevalence. A dynamic threshold is needed but the quality of VGI will improve as more data are collected and the errors in identifying reliable participants will decrease. This system allows for two-way communication between researchers and the public, and a way to evaluate the reliability of VGI. Boosting the public's ability to participate in such work can improve disease surveillance and promote citizen science. In the absence of active surveillance, VGI can provide valuable spatial information given that the data are reliable.

Score Reliability: A Retrospective Look Back at 12 Years of Reliability Generalization Studies

ERIC Educational Resources Information Center

Vacha-Haase, Tammi; Thompson, Bruce

2011-01-01

The present study was conducted to characterize (a) the features of the thousands of primary reports synthesized in 47 reliability generalization (RG) measurement meta-analysis studies and (b) typical methodological practice within the RG literature to date. With respect to the treatment of score reliability in the literature, in an astounding…

It's not that Difficult: An Interrater Reliability Study of the DSM–5 Section III Alternative Model for Personality Disorders

DOE PAGES

Garcia, Darren J.; Skadberg, Rebecca M.; Schmidt, Megan; ...

2018-03-05

The Diagnostic and Statistical Manual of Mental Disorders (5th ed. [DSM–5]; American Psychiatric Association, 2013) Section III Alternative Model for Personality Disorders (AMPD) represents a novel approach to the diagnosis of personality disorder (PD). In this model, PD diagnosis requires evaluation of level of impairment in personality functioning (Criterion A) and characterization by pathological traits (Criterion B). Questions about clinical utility, complexity, and difficulty in learning and using the AMPD have been expressed in recent scholarly literature. We examined the learnability, interrater reliability, and clinical utility of the AMPD using a vignette methodology and graduate student raters. Results showed thatmore » student clinicians can learn Criterion A of the AMPD to a high level of interrater reliability and agreement with expert ratings. Interrater reliability of the 25 trait facets of the AMPD varied but showed overall acceptable levels of agreement. Examination of severity indexes of PD impairment showed the level of personality functioning (LPF) added information beyond that of global assessment of functioning (GAF). Clinical utility ratings were generally strong. Lastly, the satisfactory interrater reliability of components of the AMPD indicates the model, including the LPF, is very learnable.« less

It's not that Difficult: An Interrater Reliability Study of the DSM–5 Section III Alternative Model for Personality Disorders

DOE Office of Scientific and Technical Information (OSTI.GOV)

Garcia, Darren J.; Skadberg, Rebecca M.; Schmidt, Megan

The Diagnostic and Statistical Manual of Mental Disorders (5th ed. [DSM–5]; American Psychiatric Association, 2013) Section III Alternative Model for Personality Disorders (AMPD) represents a novel approach to the diagnosis of personality disorder (PD). In this model, PD diagnosis requires evaluation of level of impairment in personality functioning (Criterion A) and characterization by pathological traits (Criterion B). Questions about clinical utility, complexity, and difficulty in learning and using the AMPD have been expressed in recent scholarly literature. We examined the learnability, interrater reliability, and clinical utility of the AMPD using a vignette methodology and graduate student raters. Results showed thatmore » student clinicians can learn Criterion A of the AMPD to a high level of interrater reliability and agreement with expert ratings. Interrater reliability of the 25 trait facets of the AMPD varied but showed overall acceptable levels of agreement. Examination of severity indexes of PD impairment showed the level of personality functioning (LPF) added information beyond that of global assessment of functioning (GAF). Clinical utility ratings were generally strong. Lastly, the satisfactory interrater reliability of components of the AMPD indicates the model, including the LPF, is very learnable.« less

Minimum Control Requirements for Advanced Life Support Systems

NASA Technical Reports Server (NTRS)

Boulange, Richard; Jones, Harry; Jones, Harry

2002-01-01

Advanced control technologies are not necessary for the safe, reliable and continuous operation of Advanced Life Support (ALS) systems. ALS systems can and are adequately controlled by simple, reliable, low-level methodologies and algorithms. The automation provided by advanced control technologies is claimed to decrease system mass and necessary crew time by reducing buffer size and minimizing crew involvement. In truth, these approaches increase control system complexity without clearly demonstrating an increase in reliability across the ALS system. Unless these systems are as reliable as the hardware they control, there is no savings to be had. A baseline ALS system is presented with the minimal control system required for its continuous safe reliable operation. This baseline control system uses simple algorithms and scheduling methodologies and relies on human intervention only in the event of failure of the redundant backup equipment. This ALS system architecture is designed for reliable operation, with minimal components and minimal control system complexity. The fundamental design precept followed is "If it isn't there, it can't fail".

Polymer on Top: Current Limits and Future Perspectives of Quantitatively Evaluating Surface Grafting.

PubMed

Michalek, Lukas; Barner, Leonie; Barner-Kowollik, Christopher

2018-03-07

Well-defined polymer strands covalently tethered onto solid substrates determine the properties of the resulting functional interface. Herein, the current approaches to determine quantitative grafting densities are assessed. Based on a brief introduction into the key theories describing polymer brush regimes, a user's guide is provided to estimating maximum chain coverage and-importantly-examine the most frequently employed approaches for determining grafting densities, i.e., dry thickness measurements, gravimetric assessment, and swelling experiments. An estimation of the reliability of these determination methods is provided via carefully evaluating their assumptions and assessing the stability of the underpinning equations. A practical access guide for comparatively and quantitatively evaluating the reliability of a given approach is thus provided, enabling the field to critically judge experimentally determined grafting densities and to avoid the reporting of grafting densities that fall outside the physically realistic parameter space. The assessment is concluded with a perspective on the development of advanced approaches for determination of grafting density, in particular, on single-chain methodologies. © 2018 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim.

Internet addiction assessment tools: dimensional structure and methodological status.

PubMed

Lortie, Catherine L; Guitton, Matthieu J

2013-07-01

Excessive internet use is becoming a concern, and some have proposed that it may involve addiction. We evaluated the dimensions assessed by, and psychometric properties of, a range of questionnaires purporting to assess internet addiction. Fourteen questionnaires were identified purporting to assess internet addiction among adolescents and adults published between January 1993 and October 2011. Their reported dimensional structure, construct, discriminant and convergent validity and reliability were assessed, as well as the methods used to derive these. Methods used to evaluate internet addiction questionnaires varied considerably. Three dimensions of addiction predominated: compulsive use (79%), negative outcomes (86%) and salience (71%). Less common were escapism (21%), withdrawal symptoms (36%) and other dimensions. Measures of validity and reliability were found to be within normally acceptable limits. There is a broad convergence of questionnaires purporting to assess internet addiction suggesting that compulsive use, negative outcome and salience should be covered and the questionnaires show adequate psychometric properties. However, the methods used to evaluate the questionnaires vary widely and possible factors contributing to excessive use such as social motivation do not appear to be covered. © 2013 Society for the Study of Addiction.

Development and validation of an algorithm for laser application in wound treatment 1

PubMed Central

da Cunha, Diequison Rite; Salomé, Geraldo Magela; Massahud, Marcelo Renato; Mendes, Bruno; Ferreira, Lydia Masako

2017-01-01

ABSTRACT Objective: To develop and validate an algorithm for laser wound therapy. Method: Methodological study and literature review. For the development of the algorithm, a review was performed in the Health Sciences databases of the past ten years. The algorithm evaluation was performed by 24 participants, nurses, physiotherapists, and physicians. For data analysis, the Cronbach’s alpha coefficient and the chi-square test for independence was used. The level of significance of the statistical test was established at 5% (p<0.05). Results: The professionals’ responses regarding the facility to read the algorithm indicated: 41.70%, great; 41.70%, good; 16.70%, regular. With regard the algorithm being sufficient for supporting decisions related to wound evaluation and wound cleaning, 87.5% said yes to both questions. Regarding the participants’ opinion that the algorithm contained enough information to support their decision regarding the choice of laser parameters, 91.7% said yes. The questionnaire presented reliability using the Cronbach’s alpha coefficient test (α = 0.962). Conclusion: The developed and validated algorithm showed reliability for evaluation, wound cleaning, and use of laser therapy in wounds. PMID:29211197

Quantitative evaluation of pairs and RS steganalysis

NASA Astrophysics Data System (ADS)

Ker, Andrew D.

2004-06-01

We give initial results from a new project which performs statistically accurate evaluation of the reliability of image steganalysis algorithms. The focus here is on the Pairs and RS methods, for detection of simple LSB steganography in grayscale bitmaps, due to Fridrich et al. Using libraries totalling around 30,000 images we have measured the performance of these methods and suggest changes which lead to significant improvements. Particular results from the project presented here include notes on the distribution of the RS statistic, the relative merits of different "masks" used in the RS algorithm, the effect on reliability when previously compressed cover images are used, and the effect of repeating steganalysis on the transposed image. We also discuss improvements to the Pairs algorithm, restricting it to spatially close pairs of pixels, which leads to a substantial performance improvement, even to the extent of surpassing the RS statistic which was previously thought superior for grayscale images. We also describe some of the questions for a general methodology of evaluation of steganalysis, and potential pitfalls caused by the differences between uncompressed, compressed, and resampled cover images.

Staphylococcus aureus, Staphylococcus epidermidis and Staphylococcus haemolyticus: methicillin-resistant isolates are detected directly in blood cultures by multiplex PCR.

PubMed

Pereira, Eliezer M; Schuenck, Ricardo P; Malvar, Karoline L; Iorio, Natalia L P; Matos, Pricilla D M; Olendzki, André N; Oelemann, Walter M R; dos Santos, Kátia R N

2010-03-31

In this study, we standardized and evaluated a multiplex-PCR methodology using specific primers to identify Staphylococcus aureus, Staphylococcus epidermidis and Staphylococcus haemolyticus and their methicillin-resistance directly from blood cultures. Staphylococci clinical isolates (149) and control strains (16) previously identified by conventional methods were used to establish the multiplex PCR protocol. Subsequently, this methodology was evaluated using a fast and cheap DNA extraction protocol from 25 staphylococci positive blood cultures. A wash step of the pellet with 0.1% bovine serum albumin (BSA) solution was performed to reduce PCR inhibitors. Amplicons of 154bp (mecA gene), 271bp (S. haemolyticus mvaA gene) and 108 and 124bp (S. aureus and S. epidermidis species-specific fragments, respectively) were observed. Reliable results were obtained for 100% of the evaluated strains, suggesting that this new multiplex-PCR combined with an appropriate DNA-extraction method could be useful in the laboratory for fast and accurate identification of three staphylococci species and simultaneously their methicillin resistance directly in blood cultures.

Authors' response: the primacy of conscious decision making.

PubMed

Shanks, David R; Newell, Ben R

2014-02-01

The target article sought to question the common belief that our decisions are often biased by unconscious influences. While many commentators offer additional support for this perspective, others question our theoretical assumptions, empirical evaluations, and methodological criteria. We rebut in particular the starting assumption that all decision making is unconscious, and that the onus should be on researchers to prove conscious influences. Further evidence is evaluated in relation to the core topics we reviewed (multiple-cue judgment, deliberation without attention, and decisions under uncertainty), as well as priming effects. We reiterate a key conclusion from the target article, namely, that it now seems to be generally accepted that awareness should be operationally defined as reportable knowledge, and that such knowledge can only be evaluated by careful and thorough probing. We call for future research to pay heed to the different ways in which awareness can intervene in decision making (as identified in our lens model analysis) and to employ suitable methodology in the assessment of awareness, including the requirements that awareness assessment must be reliable, relevant, immediate, and sensitive.

Methodological aspects of an adaptive multidirectional pattern search to optimize speech perception using three hearing-aid algorithms

NASA Astrophysics Data System (ADS)

Franck, Bas A. M.; Dreschler, Wouter A.; Lyzenga, Johannes

2004-12-01

In this study we investigated the reliability and convergence characteristics of an adaptive multidirectional pattern search procedure, relative to a nonadaptive multidirectional pattern search procedure. The procedure was designed to optimize three speech-processing strategies. These comprise noise reduction, spectral enhancement, and spectral lift. The search is based on a paired-comparison paradigm, in which subjects evaluated the listening comfort of speech-in-noise fragments. The procedural and nonprocedural factors that influence the reliability and convergence of the procedure are studied using various test conditions. The test conditions combine different tests, initial settings, background noise types, and step size configurations. Seven normal hearing subjects participated in this study. The results indicate that the reliability of the optimization strategy may benefit from the use of an adaptive step size. Decreasing the step size increases accuracy, while increasing the step size can be beneficial to create clear perceptual differences in the comparisons. The reliability also depends on starting point, stop criterion, step size constraints, background noise, algorithms used, as well as the presence of drifting cues and suboptimal settings. There appears to be a trade-off between reliability and convergence, i.e., when the step size is enlarged the reliability improves, but the convergence deteriorates. .

Strategies for Improving U.S. Air Force Productivity: Developing Methodologies for Assessing the Potential Relationship between Communication Behaviors and Productivity.

DTIC Science & Technology

1980-09-01

1176 NL EmillllllllEhmhmmhhhhhmhu ,mhhmmhhhhmhhhu Ehmmhhmhhmhhl EllhlhElhlhEEE Emmhhhhhhmmhu 1 II 12.5 111111.25 1 I1 6 MICROCOPY RESOLUTION TEST ...Technician Performance t/-O7 20J ABSTRCT fContinue an reverse aid* it necesary and Identify by block number) ILJ Liedhe poetwas a preliminary testing of a...of work characteristics. These were evaluated according to re- sponse patterns, factor structure, and/or reliability indicants.. Preliminary testing

Multi-Model Ensemble Wake Vortex Prediction

NASA Technical Reports Server (NTRS)

Koerner, Stephan; Holzaepfel, Frank; Ahmad, Nash'at N.

2015-01-01

Several multi-model ensemble methods are investigated for predicting wake vortex transport and decay. This study is a joint effort between National Aeronautics and Space Administration and Deutsches Zentrum fuer Luft- und Raumfahrt to develop a multi-model ensemble capability using their wake models. An overview of different multi-model ensemble methods and their feasibility for wake applications is presented. The methods include Reliability Ensemble Averaging, Bayesian Model Averaging, and Monte Carlo Simulations. The methodologies are evaluated using data from wake vortex field experiments.

Psychometric properties of the Satisfaction with Life Scale (SWLS): secondary analysis of the Mexican Health and Aging Study.

PubMed

López-Ortega, Mariana; Torres-Castro, Sara; Rosas-Carrasco, Oscar

2016-12-09

The Satisfaction with Life Scale (SWLS) has been widely used and has proven to be a valid and reliable instrument for assessing satisfaction with life in diverse population groups, however, research on satisfaction with life and validation of different measuring instruments in Mexican adults is still lacking. The objective was to evaluate the psychometric properties of the Satisfaction with Life Scale (SWLS) in a representative sample of Mexican adults. This is a methodological study to evaluate a satisfaction with life scale in a sample of 13,220 Mexican adults 50 years of age or older from the 2012 Mexican Health and Aging Study. The scale's reliability (internal consistency) was analysed using Cronbach's alpha and inter-item correlations. An exploratory factor analysis was also performed. Known-groups validity was evaluated comparing good-health and bad-health participants. Comorbidity, perceived financial situation, self-reported general health, depression symptoms, and social support were included to evaluate the validity between these measures and the total score of the scale using Spearman's correlations. The analysis of the scale's reliability showed good internal consistency (α = 0.74). The exploratory factor analysis confirmed the existence of a unique factor structure that explained 54% of the variance. SWLS was related to depression, perceived health, financial situation, and social support, and these relations were all statistically significant (P < .01). There was significant difference in life satisfaction between the good- and bad-health groups. Results show good internal consistency and construct validity of the SWLS. These results are comparable with results from previous studies. Meeting the study's objective to validate the scale, the results show that the Spanish version of the SWLS is a reliable and valid measure of satisfaction with life in the Mexican context.

Reliability of Pseudotyped Influenza Viral Particles in Neutralizing Antibody Detection

PubMed Central

Yang, Jinghui; Li, Weidong; Long, Yunfeng; Song, Shaohui; Liu, Jing; Zhang, Xinwen; Wang, Xiaoguang; Jiang, Shude; Liao, Guoyang

2014-01-01

Background Current influenza control strategies require an active surveillance system. Pseudotyped viral particles (pp) together with the evaluation of pre-existing immunity in a population might satisfy this requirement. However, the reliability of using pp in neutralizing antibody (nAb) detection are undefined. Methodology/Principal Findings Pseudotyped particles of A(H1N1)pmd09 (A/California/7/2009) and HPAI H5N1 (A/Anhui/1/2005), as well as their reassortants, were generated. The reliability of using these pp in nAb detection were compared concurrently with the corresponding viruses by a hemagglutination inhibition test, as well as ELISA-, cytopathic effect-, and fluorescence-based microneutralization assays. In the qualitative detection on nAbs, the pp and their corresponding viruses were in complete agreement, with an R2 value equal to or near 1 in two different populations. In the quantitative detection on nAbs, although the geometric mean titers (95% confidence interval) differed between the pp and viruses, no significant difference was observed. Furthermore, humoral immunity against the reassortants was evaluated; our results indicated strong consistency between the nAbs against reassortant pp and those against naïve pp harboring the same hemagglutinin. Conclusion/Significance The pp displayed high reliability in influenza virus nAb detection. The use of reassortant pp is a safe and convenient strategy for characterizing emerging influenza viruses and surveying the disease burden. PMID:25436460

Methodology for Developing a New EFNEP Food and Physical Activity Behaviors Questionnaire.

PubMed

Murray, Erin K; Auld, Garry; Baker, Susan S; Barale, Karen; Franck, Karen; Khan, Tarana; Palmer-Keenan, Debra; Walsh, Jennifer

2017-10-01

Research methods are described for developing a food and physical activity behaviors questionnaire for the Expanded Food and Nutrition Education Program (EFNEP), a US Department of Agriculture nutrition education program serving low-income families. Mixed-methods observational study. The questionnaire will include 5 domains: (1) diet quality, (2) physical activity, (3) food safety, (4) food security, and (5) food resource management. A 5-stage process will be used to assess the questionnaire's test-retest reliability and content, face, and construct validity. Research teams across the US will coordinate questionnaire development and testing nationally. Convenience samples of low-income EFNEP, or EFNEP-eligible, adult participants across the US. A 5-stage process: (1) prioritize domain concepts to evaluate (2) question generation and content analysis panel, (3) question pretesting using cognitive interviews, (4) test-retest reliability assessment, and (5) construct validity testing. A nationally tested valid and reliable food and physical activity behaviors questionnaire for low-income adults to evaluate EFNEP's effectiveness. Cognitive interviews will be summarized to identify themes and dominant trends. Paired t tests (P ≤ .05) and Spearman and intra-class correlation coefficients (r > .5) will be conducted to assess reliability. Construct validity will be assessed using Wilcoxon t test (P ≤ .05), Spearman correlations, and Bland-Altman plots. Copyright © 2017 Society for Nutrition Education and Behavior. Published by Elsevier Inc. All rights reserved.

Standardization of carbon-phenolic composite test methodology

NASA Technical Reports Server (NTRS)

Hall, W. B.

1986-01-01

The objective of this study was to evaluate the residual volatiles, filler content, and resin flow test procedures for carbon-phenolic prepreg materials. The residual volatile test procedure was rewritten with tighter procedure control which was then evaluated by round robin testing by four laboratories on the same rolls of prepreg. Results indicated that the residual volatiles test was too operator and equipment dependent to be reliable, and it was recommended that the test be discontinued. The resin flow test procedures were rewritten with tighter procedure control, and it is now considered to be an acceptable test. It was recommended that the filler content determination be made prior to prepregging.

The validity and reliability of the four square step test in different adult populations: a systematic review.

PubMed

Moore, Martha; Barker, Karen

2017-09-11

The four square step test (FSST) was first validated in healthy older adults to provide a measure of dynamic standing balance and mobility. The FSST has since been used in a variety of patient populations. The purpose of this systematic review is to determine the validity and reliability of the FSST in these different adult patient populations. The literature search was conducted to highlight all the studies that measured validity and reliability of the FSST. Six electronic databases were searched including AMED, CINAHL, MEDLINE, PEDro, Web of Science and Google Scholar. Grey literature was also searched for any documents relevant to the review. Two independent reviewers carried out study selection and quality assessment. The methodological quality was assessed using the QUADAS-2 tool, which is a validated tool for the quality assessment of diagnostic accuracy studies, and the COSMIN four-point checklist, which contains standards for evaluating reliability studies on the measurement properties of health instruments. Fifteen studies were reviewed studying community-dwelling older adults, Parkinson's disease, Huntington's disease, multiple sclerosis, vestibular disorders, post stroke, post unilateral transtibial amputation, knee pain and hip osteoarthritis. Three of the studies were of moderate methodological quality scoring low in risk of bias and applicability for all domains in the QUADAS-2 tool. Three studies scored "fair" on the COSMIN four-point checklist for the reliability components. The concurrent validity of the FSST was measured in nine of the studies with moderate to strong correlations being found. Excellent Intraclass Correlation Coefficients were found between physiotherapists carrying out the tests (ICC = .99) with good to excellent test-retest reliability shown in nine of the studies (ICC = .73-.98). The FSST may be an effective and valid tool for measuring dynamic balance and a participants' falls risk. It has been shown to have strong correlations with other measures of balance and mobility with good reliability shown in a number of populations. However, the quality of the papers reviewed was variable with key factors, such as sample size and test set up, needing to be addressed before the tool can be confidently used in these specified populations.

Inter-rater agreement in evaluation of disability: systematic review of reproducibility studies.

PubMed

Barth, Jürgen; de Boer, Wout E L; Busse, Jason W; Hoving, Jan L; Kedzia, Sarah; Couban, Rachel; Fischer, Katrin; von Allmen, David Y; Spanjer, Jerry; Kunz, Regina

2017-01-25

 To explore agreement among healthcare professionals assessing eligibility for work disability benefits.  Systematic review and narrative synthesis of reproducibility studies.  Medline, Embase, and PsycINFO searched up to 16 March 2016, without language restrictions, and review of bibliographies of included studies.  Observational studies investigating reproducibility among healthcare professionals performing disability evaluations using a global rating of working capacity and reporting inter-rater reliability by a statistical measure or descriptively. Studies could be conducted in insurance settings, where decisions on ability to work include normative judgments based on legal considerations, or in research settings, where decisions on ability to work disregard normative considerations. : Teams of paired reviewers identified eligible studies, appraised their methodological quality and generalisability, and abstracted results with pretested forms. As heterogeneity of research designs and findings impeded a quantitative analysis, a descriptive synthesis stratified by setting (insurance or research) was performed.  From 4562 references, 101 full text articles were reviewed. Of these, 16 studies conducted in an insurance setting and seven in a research setting, performed in 12 countries, met the inclusion criteria. Studies in the insurance setting were conducted with medical experts assessing claimants who were actual disability claimants or played by actors, hypothetical cases, or short written scenarios. Conditions were mental (n=6, 38%), musculoskeletal (n=4, 25%), or mixed (n=6, 38%). Applicability of findings from studies conducted in an insurance setting to real life evaluations ranged from generalisable (n=7, 44%) and probably generalisable (n=3, 19%) to probably not generalisable (n=6, 37%). Median inter-rater reliability among experts was 0.45 (range intraclass correlation coefficient 0.86 to κ-0.10). Inter-rater reliability was poor in six studies (37%) and excellent in only two (13%). This contrasts with studies conducted in the research setting, where the median inter-rater reliability was 0.76 (range 0.91-0.53), and 71% (5/7) studies achieved excellent inter-rater reliability. Reliability between assessing professionals was higher when the evaluation was guided by a standardised instrument (23 studies, P=0.006). No such association was detected for subjective or chronic health conditions or the studies' generalisability to real world evaluation of disability (P=0.46, 0.45, and 0.65, respectively).  Despite their common use and far reaching consequences for workers claiming disabling injury or illness, research on the reliability of medical evaluations of disability for work is limited and indicates high variation in judgments among assessing professionals. Standardising the evaluation process could improve reliability. Development and testing of instruments and structured approaches to improve reliability in evaluation of disability are urgently needed. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

Bulk Fuel Pricing: DOD Needs to Take Additional Actions to Establish a More Reliable Methodology

DTIC Science & Technology

2015-11-19

Page 1 GAO-16-78R Bulk Fuel Pricing 441 G St. N.W. Washington, DC 20548 November 19, 2015 The Honorable Ashton Carter The Secretary of...Defense Bulk Fuel Pricing : DOD Needs to Take Additional Actions to Establish a More Reliable Methodology Dear Secretary Carter: Each fiscal...year, the Office of the Under Secretary of Defense (Comptroller), in coordination with the Defense Logistics Agency, sets a standard price per barrel

[Teacher's perfomance assessment in Family Medicine specialization].

PubMed

Martínez-González, Adrián; Gómez-Clavelina, Francisco J; Hernández-Torres, Isaías; Flores-Hernández, Fernando; Sánchez-Mendiola, Melchor

2016-01-01

In Mexico there is no systematic evaluation of teachers in medical specialties. It is difficult to identify appropriate teaching practices. The lack of evaluation has limited the recognition and improvement of teaching. The objective of this study was to analyze feedback from students about teaching activities of teachers-tutors responsible for the specialization course in family medicine, and evaluate the evidence of reliability and validity of the instrument applied online. It was an observational and cross-sectional study. Seventy eight teachers of Family Medicine of medical residency were evaluated by 734 resident´s opinion. The anonymous questionnaire to assess teaching performance by resident's opinion and it is composed of 5 dimensions using a Likert scale. Descriptive and inferential statistics (t test, one-way ANOVA and factor analysis) were used. Residents stated that teaching performance is acceptable, with an average of 4.25 ± 0.93. The best valued dimension was "Methodology" with an average of 4.34 ± .92 in contrast to the "assessment" dimension with 4.16 ± 1.04. Teachers of specialization in family medicine have acceptable performance by resident's opinion. The online assessment tool meets the criteria of validity and reliability.

Methods for reliability evaluation of trust and reputation systems

NASA Astrophysics Data System (ADS)

Janiszewski, Marek B.

2016-09-01

Trust and reputation systems are a systematic approach to build security on the basis of observations of node's behaviour. Exchange of node's opinions about other nodes is very useful to indicate nodes which act selfishly or maliciously. The idea behind trust and reputation systems gets significance because of the fact that conventional security measures (based on cryptography) are often not sufficient. Trust and reputation systems can be used in various types of networks such as WSN, MANET, P2P and also in e-commerce applications. Trust and reputation systems give not only benefits but also could be a thread itself. Many attacks aim at trust and reputation systems exist, but such attacks still have not gain enough attention of research teams. Moreover, joint effects of many of known attacks have been determined as a very interesting field of research. Lack of an acknowledged methodology of evaluation of trust and reputation systems is a serious problem. This paper aims at presenting various approaches of evaluation such systems. This work also contains a description of generalization of many trust and reputation systems which can be used to evaluate reliability of such systems in the context of preventing various attacks.

Measurement properties of questionnaires assessing participation in children and adolescents with a disability: a systematic review.

PubMed

Rainey, Linda; van Nispen, Ruth; van der Zee, Carlijn; van Rens, Ger

2014-12-01

To critically appraise the measurement properties of questionnaires measuring participation in children and adolescents (0-18 years) with a disability. Bibliographic databases were searched for studies evaluating the measurement properties of self-report or parent-report questionnaires measuring participation in children and adolescents (0-18 years) with a disability. The methodological quality of the included studies and the results of the measurement properties were evaluated using a checklist developed on consensus-based standards. The search strategy identified 3,977 unique publications, of which 22 were selected; these articles evaluated the development and measurement properties of eight different questionnaires. The Child and Adolescent Scale of Participation was evaluated most extensively, generally showing moderate positive results on content validity, internal consistency, reliability and construct validity. The remaining questionnaires also demonstrated positive results. However, at least 50 % of the measurement properties per questionnaire were not (or only poorly) assessed. Studies of high methodological quality, using modern statistical methods, are needed to accurately assess the measurement properties of currently available questionnaires. Moreover, consensus is required on the definition of the construct 'participation' to determine content validity and to enable meaningful interpretation of outcomes.

Allocating SMART Reliability and Maintainability Goals to NASA Ground Systems

NASA Technical Reports Server (NTRS)

Gillespie, Amanda; Monaghan, Mark

2013-01-01

This paper will describe the methodology used to allocate Reliability and Maintainability (R&M) goals to Ground Systems Development and Operations (GSDO) subsystems currently being designed or upgraded.

Life Cycle Assessment for desalination: a review on methodology feasibility and reliability.

PubMed

Zhou, Jin; Chang, Victor W-C; Fane, Anthony G

2014-09-15

As concerns of natural resource depletion and environmental degradation caused by desalination increase, research studies of the environmental sustainability of desalination are growing in importance. Life Cycle Assessment (LCA) is an ISO standardized method and is widely applied to evaluate the environmental performance of desalination. This study reviews more than 30 desalination LCA studies since 2000s and identifies two major issues in need of improvement. The first is feasibility, covering three elements that support the implementation of the LCA to desalination, including accounting methods, supporting databases, and life cycle impact assessment approaches. The second is reliability, addressing three essential aspects that drive uncertainty in results, including the incompleteness of the system boundary, the unrepresentativeness of the database, and the omission of uncertainty analysis. This work can serve as a preliminary LCA reference for desalination specialists, but will also strengthen LCA as an effective method to evaluate the environment footprint of desalination alternatives. Copyright © 2014 Elsevier Ltd. All rights reserved.

Reliability and Validity of the Turkish Version of the Moral Competence Scale for Public Health Nurses: A Methodological Study.

PubMed

Yildiz, Esra; Güdücü Tüfekci, Fatma

Moral competencies must be improved in nursing area practice. To evaluate the moral competence seems necessary for nurses. The aims of this study are to adapt and evaluate the psychometric properties of the moral competence questionnaire for public health nurses in Turkey. The moral competence scale was translated into Turkish by a skilled translator, after which it was back-translated into English by another translator. We then administered the Turkish version of the moral competence scale to 138 public health nurses working in family and public health centers in Erzurum, a city in eastern Turkey. We analyzed the data using factor analysis and Cronbach's α. Three factors were extracted, which together explained a total of 67.50% of the variance. The Cronbach's α values were .83, .91, .87, and .88 for factors 1, 2, and 3 and for the whole scale, respectively. The Turkish version of the moral competence scale for public health nurses is a valid and reliable assessment tool.

Methodologies for Crawler Based Web Surveys.

ERIC Educational Resources Information Center

Thelwall, Mike

2002-01-01

Describes Web survey methodologies used to study the content of the Web, and discusses search engines and the concept of crawling the Web. Highlights include Web page selection methodologies; obstacles to reliable automatic indexing of Web sites; publicly indexable pages; crawling parameters; and tests for file duplication. (Contains 62…

A hierarchical approach to reliability modeling of fault-tolerant systems. M.S. Thesis

NASA Technical Reports Server (NTRS)

Gossman, W. E.

1986-01-01

A methodology for performing fault tolerant system reliability analysis is presented. The method decomposes a system into its subsystems, evaluates vent rates derived from the subsystem's conditional state probability vector and incorporates those results into a hierarchical Markov model of the system. This is done in a manner that addresses failure sequence dependence associated with the system's redundancy management strategy. The method is derived for application to a specific system definition. Results are presented that compare the hierarchical model's unreliability prediction to that of a more complicated tandard Markov model of the system. The results for the example given indicate that the hierarchical method predicts system unreliability to a desirable level of accuracy while achieving significant computational savings relative to component level Markov model of the system.

The design and evaluation of psychometric properties for a questionnaire on elderly abuse by family caregivers among older adults on hemodialysis.

PubMed

Mahmoudian, Amaneh; Torabi Chafjiri, Razieh; Alipour, Atefeh; Shamsalinia, Abbas; Ghaffari, Fatemeh

2018-01-01

Older adults with chronic disease are more vulnerable to abuse. Early and accurate detection of the elderly abuse phenomenon can help identify health-promoting solutions for the elderly, their family, and society. The purpose of this study was to design and evaluate the psychometric properties of a questionnaire on elderly abuse by family caregivers among older adults on hemodialysis. Qualitative and quantitative research methodologies were used to develop the questionnaire. The item pool was compiled from literature reviews and the Delphi method. The literature reviews comprised 22 studies. The psychometric properties of the questionnaire were verified using face, content, and construct validity, and the reliability was tested using Cronbach's alpha reliability. A 57-item questionnaire was developed after the psychometric evaluation. The Kaiser-Meyer-Olkin index and Bartlett's test of sphericity showed reliable results. Seven components from the exploratory content analysis including psychological misbehavior, authority deprivation, physical misbehavior, financial misbehavior, being abandoned, caring neglect, and emotional misbehavior explained 74.769% of the total variance. Cronbach's alpha was 0.98 and the interclass correlation coefficient was r =0.91 responding to the items twice ( p <0.001), which shows a high level of tool stability. This study developed a questionnaire to assess elderly abuse by family caregivers among older adults on hemodialysis. It is recommended as a mini scale that can be used both in statistical and practical studies, and that is valid and reliable. Nurses or other health care providers can use it in health centers, dialysis centers, or at the house of the patient.

Evaluation of the international standardized 24-h dietary recall methodology (GloboDiet) for potential application in research and surveillance within African settings.

PubMed

Aglago, Elom Kouassivi; Landais, Edwige; Nicolas, Geneviève; Margetts, Barrie; Leclercq, Catherine; Allemand, Pauline; Aderibigbe, Olaide; Agueh, Victoire Damienne; Amuna, Paul; Annor, George Amponsah; El Ati, Jalila; Coates, Jennifer; Colaiezzi, Brooke; Compaore, Ella; Delisle, Hélène; Faber, Mieke; Fungo, Robert; Gouado, Inocent; El Hamdouchi, Asmaa; Hounkpatin, Waliou Amoussa; Konan, Amoin Georgette; Labzizi, Saloua; Ledo, James; Mahachi, Carol; Maruapula, Segametsi Ditshebo; Mathe, Nonsikelelo; Mbabazi, Muniirah; Mirembe, Mandy Wilja; Mizéhoun-Adissoda, Carmelle; Nzi, Clement Diby; Pisa, Pedro Terrence; El Rhazi, Karima; Zotor, Francis; Slimani, Nadia

2017-06-19

Collection of reliable and comparable individual food consumption data is of primary importance to better understand, control and monitor malnutrition and its related comorbidities in low- and middle-income countries (LMICs), including in Africa. The lack of standardised dietary tools and their related research support infrastructure remains a major obstacle to implement concerted and region-specific research and action plans worldwide. Citing the magnitude and importance of this challenge, the International Agency for Research on Cancer (IARC/WHO) launched the "Global Nutrition Surveillance initiative" to pilot test the use of a standardized 24-h dietary recall research tool (GloboDiet), validated in Europe, in other regions. In this regard, the development of the GloboDiet-Africa can be optimised by better understanding of the local specific methodological needs, barriers and opportunities. The study aimed to evaluate the standardized 24-h dietary recall research tool (GloboDiet) as a possible common methodology for research and surveillance across Africa. A consultative panel of African and international experts in dietary assessment participated in six e-workshop sessions. They completed an in-depth e-questionnaire to evaluate the GloboDiet dietary methodology before and after participating in the e-workshop. The 29 experts expressed their satisfaction on the potential of the software to address local specific needs when evaluating the main structure of the software, the stepwise approach for data collection and standardisation concept. Nevertheless, additional information to better describe local foods and recipes, as well as particular culinary patterns (e.g. mortar pounding), were proposed. Furthermore, food quantification in shared-plates and -bowls eating situations and interviewing of populations with low literacy skills, especially in rural settings, were acknowledged as requiring further specific considerations and appropriate solutions. An overall positive evaluation of the GloboDiet methodology by both African and international experts, supports the flexibility and potential applicability of this tool in diverse African settings and sets a positive platform for improved dietary monitoring and surveillance. Following this evaluation, prerequisite for future implementation and/or adaptation of GloboDiet in Africa, rigorous and robust capacity building as well as knowledge transfer will be required to roadmap a stepwise approach to implement this methodology across pilot African countries/regions.

Cryogenic Quenching Process for Electronic Part Screening

NASA Technical Reports Server (NTRS)

Sheldon, Douglas J.; Cressler, John

2011-01-01

The use of electronic parts at cryogenic temperatures (less than 100 C) for extreme environments is not well controlled or developed from a product quality and reliability point of view. This is in contrast to the very rigorous and well-documented procedures to qualify electronic parts for mission use in the 55 to 125 C temperature range. A similarly rigorous methodology for screening and evaluating electronic parts needs to be developed so that mission planners can expect the same level of high reliability performance for parts operated at cryogenic temperatures. A formal methodology for screening and qualifying electronic parts at cryogenic temperatures has been proposed. The methodology focuses on the base physics of failure of the devices at cryogenic temperatures. All electronic part reliability is based on the bathtub curve, high amounts of initial failures (infant mortals), a long period of normal use (random failures), and then an increasing number of failures (end of life). Unique to this is the development of custom screening procedures to eliminate early failures at cold temperatures. The ability to screen out defects will specifically impact reliability at cold temperatures. Cryogenic reliability is limited by electron trap creation in the oxide and defect sites at conductor interfaces. Non-uniform conduction processes due to process marginalities will be magnified at cryogenic temperatures. Carrier mobilities change by orders of magnitude at cryogenic temperatures, significantly enhancing the effects of electric field. Marginal contacts, impurities in oxides, and defects in conductor/conductor interfaces can all be magnified at low temperatures. The novelty is the use of an ultra-low temperature, short-duration quenching process for defect screening. The quenching process is designed to identify those defects that will precisely (and negatively) affect long-term, cryogenic part operation. This quenching process occurs at a temperature that is at least 25 C colder than the coldest expected operating temperature. This quenching process is the opposite of the standard burn-in procedure. Normal burn-in raises the temperature (and voltage) to activate quickly any possible manufacturing defects remaining in the device that were not already rejected at a functional test step. The proposed inverse burn-in or quenching process is custom-tailored to the electronic device being used. The doping profiles, materials, minimum dimensions, interfaces, and thermal expansion coefficients are all taken into account in determining the ramp rate, dwell time, and temperature.

Reliability and validity of Persian version of perceived stress scale (PSS-10) in adults with asthma.

PubMed

Maroufizadeh, Saman; Zareiyan, Armin; Sigari, Naseh

2014-05-01

Asthma is a major public health problem in the world, and recent findings suggest that stress influences asthma and asthma morbidity. The 10-item Perceived Stress Scale (PSS-10) is one of the most frequently used instruments to measure psychological stress. This study was conducted to evaluate the psychometric properties of the Persian versions of the PSS-10 in adults with asthma. In this descriptive cross-sectional study as a methodological research, 106 asthmatic patients referring to several clinics in Sanandaj (western Iran) were selected through convenience sampling. The PSS-10 and the 21-item Depression anxiety and stress scale (DASS-21) were administrated to all patients. Cronbach's alpha was used to evaluate reliability of PSS-10, and confirmatory factor analysis (CFA) and convergent validity were used to evaluate its validity. The results of confirmatory factor analysis indicated that a two-factor structure of PSS-10 provided a good fit to data. The Cronbach's alpha coefficients for negative factor, positive factor and total score (PSS-10) were 0.86, 0.83, and 0.90, respectively. The PSS-10 was positively correlated with the DASS-21 and its subscales, indicating an acceptable convergent validity. Female asthmatic patients scored higher on PSS-10 in comparison with male asthmatic patients. The Persian version of PSS-10 is a valid and reliable instrument to measure perceived stress in adults with asthma.

Evaluation of the reliability, usability, and applicability of AMSTAR, AMSTAR 2, and ROBIS: protocol for a descriptive analytic study.

PubMed

Gates, Allison; Gates, Michelle; Duarte, Gonçalo; Cary, Maria; Becker, Monika; Prediger, Barbara; Vandermeer, Ben; Fernandes, Ricardo M; Pieper, Dawid; Hartling, Lisa

2018-06-13

Systematic reviews (SRs) of randomised controlled trials (RCTs) can provide the best evidence to inform decision-making, but their methodological and reporting quality varies. Tools exist to guide the critical appraisal of quality and risk of bias in SRs, but evaluations of their measurement properties are limited. We will investigate the interrater reliability (IRR), usability, and applicability of A MeaSurement Tool to Assess systematic Reviews (AMSTAR), AMSTAR 2, and Risk Of Bias In Systematic reviews (ROBIS) for SRs in the fields of biomedicine and public health. An international team of researchers at three collaborating centres will undertake the study. We will use a random sample of 30 SRs of RCTs investigating therapeutic interventions indexed in MEDLINE in February 2014. Two reviewers at each centre will appraise the quality and risk of bias in each SR using AMSTAR, AMSTAR 2, and ROBIS. We will record the time to complete each assessment and for the two reviewers to reach consensus for each SR. We will extract the descriptive characteristics of each SR, the included studies, participants, interventions, and comparators. We will also extract the direction and strength of the results and conclusions for the primary outcome. We will summarise the descriptive characteristics of the SRs using means and standard deviations, or frequencies and proportions. To test for interrater reliability between reviewers and between the consensus agreements of reviewer pairs, we will use Gwet's AC 1 statistic. For comparability to previous evaluations, we will also calculate weighted Cohen's kappa and Fleiss' kappa statistics. To estimate usability, we will calculate the mean time to complete the appraisal and to reach consensus for each tool. To inform applications of the tools, we will test for statistical associations between quality scores and risk of bias judgments, and the results and conclusions of the SRs. Appraising the methodological and reporting quality of SRs is necessary to determine the trustworthiness of their conclusions. Which tool may be most reliably applied and how the appraisals should be used is uncertain; the usability of newly developed tools is unknown. This investigation of common (AMSTAR) and newly developed (AMSTAR 2, ROBIS) tools will provide empiric data to inform their application, interpretation, and refinement.

A recursive Bayesian approach for fatigue damage prognosis: An experimental validation at the reliability component level

NASA Astrophysics Data System (ADS)

Gobbato, Maurizio; Kosmatka, John B.; Conte, Joel P.

2014-04-01

Fatigue-induced damage is one of the most uncertain and highly unpredictable failure mechanisms for a large variety of mechanical and structural systems subjected to cyclic and random loads during their service life. A health monitoring system capable of (i) monitoring the critical components of these systems through non-destructive evaluation (NDE) techniques, (ii) assessing their structural integrity, (iii) recursively predicting their remaining fatigue life (RFL), and (iv) providing a cost-efficient reliability-based inspection and maintenance plan (RBIM) is therefore ultimately needed. In contribution to these objectives, the first part of the paper provides an overview and extension of a comprehensive reliability-based fatigue damage prognosis methodology — previously developed by the authors — for recursively predicting and updating the RFL of critical structural components and/or sub-components in aerospace structures. In the second part of the paper, a set of experimental fatigue test data, available in the literature, is used to provide a numerical verification and an experimental validation of the proposed framework at the reliability component level (i.e., single damage mechanism evolving at a single damage location). The results obtained from this study demonstrate (i) the importance and the benefits of a nearly continuous NDE monitoring system, (ii) the efficiency of the recursive Bayesian updating scheme, and (iii) the robustness of the proposed framework in recursively updating and improving the RFL estimations. This study also demonstrates that the proposed methodology can lead to either an extent of the RFL (with a consequent economical gain without compromising the minimum safety requirements) or an increase of safety by detecting a premature fault and therefore avoiding a very costly catastrophic failure.

Development of reliable pavement models.

DOT National Transportation Integrated Search

2011-05-01

The current report proposes a framework for estimating the reliability of a given pavement structure as analyzed by : the Mechanistic-Empirical Pavement Design Guide (MEPDG). The methodology proposes using a previously fit : response surface, in plac...

The Factor Structure of the Spiritual Well-Being Scale in Veterans Experienced Chemical Weapon Exposure.

PubMed

Sharif Nia, Hamid; Pahlevan Sharif, Saeed; Boyle, Christopher; Yaghoobzadeh, Ameneh; Tahmasbi, Bahram; Rassool, G Hussein; Taebei, Mozhgan; Soleimani, Mohammad Ali

2018-04-01

This study aimed to determine the factor structure of the spiritual well-being among a sample of the Iranian veterans. In this methodological research, 211 male veterans of Iran-Iraq warfare completed the Paloutzian and Ellison spiritual well-being scale. Maximum likelihood (ML) with oblique rotation was used to assess domain structure of the spiritual well-being. The construct validity of the scale was assessed using confirmatory factor analysis (CFA), convergent validity, and discriminant validity. Reliability was evaluated with Cronbach's alpha, Theta (θ), and McDonald Omega (Ω) coefficients, intra-class correlation coefficient (ICC), and construct reliability (CR). Results of ML and CFA suggested three factors which were labeled "relationship with God," "belief in fate and destiny," and "life optimism." The ICC, coefficients of the internal consistency, and CR were >.7 for the factors of the scale. Convergent validity and discriminant validity did not fulfill the requirements. The Persian version of spiritual well-being scale demonstrated suitable validity and reliability among the veterans of Iran-Iraq warfare.

International classification of reliability for implanted cochlear implant receiver stimulators.

PubMed

Battmer, Rolf-Dieter; Backous, Douglas D; Balkany, Thomas J; Briggs, Robert J S; Gantz, Bruce J; van Hasselt, Andrew; Kim, Chong Sun; Kubo, Takeshi; Lenarz, Thomas; Pillsbury, Harold C; O'Donoghue, Gerard M

2010-10-01

To design an international standard to be used when reporting reliability of the implanted components of cochlear implant systems to appropriate governmental authorities, cochlear implant (CI) centers, and for journal editors in evaluating manuscripts involving cochlear implant reliability. The International Consensus Group for Cochlear Implant Reliability Reporting was assembled to unify ongoing efforts in the United States, Europe, Asia, and Australia to create a consistent and comprehensive classification system for the implanted components of CI systems across manufacturers. All members of the consensus group are from tertiary referral cochlear implant centers. None. A clinically relevant classification scheme adapted from principles of ISO standard 5841-2:2000 originally designed for reporting reliability of cardiac pacemakers, pulse generators, or leads. Standard definitions for device failure, survival time, clinical benefit, reduced clinical benefit, and specification were generated. Time intervals for reporting back to implant centers for devices tested to be "out of specification," categorization of explanted devices, the method of cumulative survival reporting, and content of reliability reports to be issued by manufacturers was agreed upon by all members. The methodology for calculating Cumulative survival was adapted from ISO standard 5841-2:2000. The International Consensus Group on Cochlear Implant Device Reliability Reporting recommends compliance to this new standard in reporting reliability of implanted CI components by all manufacturers of CIs and the adoption of this standard as a minimal reporting guideline for editors of journals publishing cochlear implant research results.

The Development of a Checklist to Enhance Methodological Quality in Intervention Programs.

PubMed

Chacón-Moscoso, Salvador; Sanduvete-Chaves, Susana; Sánchez-Martín, Milagrosa

2016-01-01

The methodological quality of primary studies is an important issue when performing meta-analyses or systematic reviews. Nevertheless, there are no clear criteria for how methodological quality should be analyzed. Controversies emerge when considering the various theoretical and empirical definitions, especially in relation to three interrelated problems: the lack of representativeness, utility, and feasibility. In this article, we (a) systematize and summarize the available literature about methodological quality in primary studies; (b) propose a specific, parsimonious, 12-items checklist to empirically define the methodological quality of primary studies based on a content validity study; and (c) present an inter-coder reliability study for the resulting 12-items. This paper provides a precise and rigorous description of the development of this checklist, highlighting the clearly specified criteria for the inclusion of items and a substantial inter-coder agreement in the different items. Rather than simply proposing another checklist, however, it then argues that the list constitutes an assessment tool with respect to the representativeness, utility, and feasibility of the most frequent methodological quality items in the literature, one that provides practitioners and researchers with clear criteria for choosing items that may be adequate to their needs. We propose individual methodological features as indicators of quality, arguing that these need to be taken into account when designing, implementing, or evaluating an intervention program. This enhances methodological quality of intervention programs and fosters the cumulative knowledge based on meta-analyses of these interventions. Future development of the checklist is discussed.

The Development of a Checklist to Enhance Methodological Quality in Intervention Programs

PubMed Central

Chacón-Moscoso, Salvador; Sanduvete-Chaves, Susana; Sánchez-Martín, Milagrosa

2016-01-01

The methodological quality of primary studies is an important issue when performing meta-analyses or systematic reviews. Nevertheless, there are no clear criteria for how methodological quality should be analyzed. Controversies emerge when considering the various theoretical and empirical definitions, especially in relation to three interrelated problems: the lack of representativeness, utility, and feasibility. In this article, we (a) systematize and summarize the available literature about methodological quality in primary studies; (b) propose a specific, parsimonious, 12-items checklist to empirically define the methodological quality of primary studies based on a content validity study; and (c) present an inter-coder reliability study for the resulting 12-items. This paper provides a precise and rigorous description of the development of this checklist, highlighting the clearly specified criteria for the inclusion of items and a substantial inter-coder agreement in the different items. Rather than simply proposing another checklist, however, it then argues that the list constitutes an assessment tool with respect to the representativeness, utility, and feasibility of the most frequent methodological quality items in the literature, one that provides practitioners and researchers with clear criteria for choosing items that may be adequate to their needs. We propose individual methodological features as indicators of quality, arguing that these need to be taken into account when designing, implementing, or evaluating an intervention program. This enhances methodological quality of intervention programs and fosters the cumulative knowledge based on meta-analyses of these interventions. Future development of the checklist is discussed. PMID:27917143

[Methodological quality of articles on therapeutic procedures published in Cirugía Española. Evaluation of the period 2005-2008].

PubMed

Manterola, Carlos; Grande, Luís

2010-04-01

To determine methodological quality of therapy articles published in Cirugía Española and to study its association with the publication year, the centre of origin and subjects. A literature study which included all therapy articles published between 2005 and 2008. All kinds of clinical designs were considered, excluding editorials, review articles, letters to editor and experimental studies. Variables analysed included: year of publication, centre of origin, design, and methodological quality of articles. A valid and reliable scale was applied to determine methodological quality. A total of 243 articles [206 series of cases (84.8%), 27 cohort studies (11.1%), 9 clinical trials (3.7%) and 1 case control study (0.4%)] were found. Studies came preferentially from Catalonia and Valencia (22.3% and 12.3% respectively). Thematic areas most frequently found were hepato-bilio-pancreatic and colorectal surgery (20.0% and 16.6%, respectively). Average and median of the methodological quality score calculated for the entire series were 9.5+/-4.3 points and 8 points, respectively. Association between methodological quality and geographical area (p=0.0101), subject area (p=0.0267), and university origin (p=0.0369) was found. A significant increase of methodological quality by publication year was observed (p=0.0004). Methodological quality of therapy articles published in Cirugía Española between 2005 and 2008 is low; but an increase tendency with statistical significance was observed.

Fatigue criterion to system design, life and reliability

NASA Technical Reports Server (NTRS)

Zaretsky, E. V.

1985-01-01

A generalized methodology to structural life prediction, design, and reliability based upon a fatigue criterion is advanced. The life prediction methodology is based in part on work of W. Weibull and G. Lundberg and A. Palmgren. The approach incorporates the computed life of elemental stress volumes of a complex machine element to predict system life. The results of coupon fatigue testing can be incorporated into the analysis allowing for life prediction and component or structural renewal rates with reasonable statistical certainty.

Quality appraisal of generic self-reported instruments measuring health-related productivity changes: a systematic review

PubMed Central

2014-01-01

Background Health impairments can result in disability and changed work productivity imposing considerable costs for the employee, employer and society as a whole. A large number of instruments exist to measure health-related productivity changes; however their methodological quality remains unclear. This systematic review critically appraised the measurement properties in generic self-reported instruments that measure health-related productivity changes to recommend appropriate instruments for use in occupational and economic health practice. Methods PubMed, PsycINFO, Econlit and Embase were systematically searched for studies whereof: (i) instruments measured health-related productivity changes; (ii) the aim was to evaluate instrument measurement properties; (iii) instruments were generic; (iv) ratings were self-reported; (v) full-texts were available. Next, methodological quality appraisal was based on COSMIN elements: (i) internal consistency; (ii) reliability; (iii) measurement error; (iv) content validity; (v) structural validity; (vi) hypotheses testing; (vii) cross-cultural validity; (viii) criterion validity; and (ix) responsiveness. Recommendations are based on evidence syntheses. Results This review included 25 articles assessing the reliability, validity and responsiveness of 15 different generic self-reported instruments measuring health-related productivity changes. Most studies evaluated criterion validity, none evaluated cross-cultural validity and information on measurement error is lacking. The Work Limitation Questionnaire (WLQ) was most frequently evaluated with moderate respectively strong positive evidence for content and structural validity and negative evidence for reliability, hypothesis testing and responsiveness. Less frequently evaluated, the Stanford Presenteeism Scale (SPS) showed strong positive evidence for internal consistency and structural validity, and moderate positive evidence for hypotheses testing and criterion validity. The Productivity and Disease Questionnaire (PRODISQ) yielded strong positive evidence for content validity, evidence for other properties is lacking. The other instruments resulted in mostly fair-to-poor quality ratings with limited evidence. Conclusions Decisions based on the content of the instrument, usage purpose, target country and population, and available evidence are recommended. Until high-quality studies are in place to accurately assess the measurement properties of the currently available instruments, the WLQ and, in a Dutch context, the PRODISQ are cautiously preferred based on its strong positive evidence for content validity. Based on its strong positive evidence for internal consistency and structural validity, the SPS is cautiously recommended. PMID:24495301

Validity and Reliability of Published Comprehensive Theory of Mind Tests for Normal Preschool Children: A Systematic Review.

PubMed

Ziatabar Ahmadi, Seyyede Zohreh; Jalaie, Shohreh; Ashayeri, Hassan

2015-09-01

Theory of mind (ToM) or mindreading is an aspect of social cognition that evaluates mental states and beliefs of oneself and others. Validity and reliability are very important criteria when evaluating standard tests; and without them, these tests are not usable. The aim of this study was to systematically review the validity and reliability of published English comprehensive ToM tests developed for normal preschool children. We searched MEDLINE (PubMed interface), Web of Science, Science direct, PsycINFO, and also evidence base Medicine (The Cochrane Library) databases from 1990 to June 2015. Search strategy was Latin transcription of 'Theory of Mind' AND test AND children. Also, we manually studied the reference lists of all final searched articles and carried out a search of their references. Inclusion criteria were as follows: Valid and reliable diagnostic ToM tests published from 1990 to June 2015 for normal preschool children; and exclusion criteria were as follows: the studies that only used ToM tests and single tasks (false belief tasks) for ToM assessment and/or had no description about structure, validity or reliability of their tests. METHODological quality of the selected articles was assessed using the Critical Appraisal Skills Programme (CASP). In primary searching, we found 1237 articles in total databases. After removing duplicates and applying all inclusion and exclusion criteria, we selected 11 tests for this systematic review. There were a few valid, reliable and comprehensive ToM tests for normal preschool children. However, we had limitations concerning the included articles. The defined ToM tests were different in populations, tasks, mode of presentations, scoring, mode of responses, times and other variables. Also, they had various validities and reliabilities. Therefore, it is recommended that the researchers and clinicians select the ToM tests according to their psychometric characteristics, validity and reliability.

Validity and Reliability of Published Comprehensive Theory of Mind Tests for Normal Preschool Children: A Systematic Review

PubMed Central

Ziatabar Ahmadi, Seyyede Zohreh; Jalaie, Shohreh; Ashayeri, Hassan

2015-01-01

Objective: Theory of mind (ToM) or mindreading is an aspect of social cognition that evaluates mental states and beliefs of oneself and others. Validity and reliability are very important criteria when evaluating standard tests; and without them, these tests are not usable. The aim of this study was to systematically review the validity and reliability of published English comprehensive ToM tests developed for normal preschool children. Method: We searched MEDLINE (PubMed interface), Web of Science, Science direct, PsycINFO, and also evidence base Medicine (The Cochrane Library) databases from 1990 to June 2015. Search strategy was Latin transcription of ‘Theory of Mind’ AND test AND children. Also, we manually studied the reference lists of all final searched articles and carried out a search of their references. Inclusion criteria were as follows: Valid and reliable diagnostic ToM tests published from 1990 to June 2015 for normal preschool children; and exclusion criteria were as follows: the studies that only used ToM tests and single tasks (false belief tasks) for ToM assessment and/or had no description about structure, validity or reliability of their tests. Methodological quality of the selected articles was assessed using the Critical Appraisal Skills Programme (CASP). Result: In primary searching, we found 1237 articles in total databases. After removing duplicates and applying all inclusion and exclusion criteria, we selected 11 tests for this systematic review. Conclusion: There were a few valid, reliable and comprehensive ToM tests for normal preschool children. However, we had limitations concerning the included articles. The defined ToM tests were different in populations, tasks, mode of presentations, scoring, mode of responses, times and other variables. Also, they had various validities and reliabilities. Therefore, it is recommended that the researchers and clinicians select the ToM tests according to their psychometric characteristics, validity and reliability. PMID:27006666

Validity and Reliability of the Turkish Chronic Pain Acceptance Questionnaire

PubMed

Akmaz, Hazel Ekin; Uyar, Meltem; Kuzeyli Yıldırım, Yasemin; Akın Korhan, Esra

2018-05-29

Pain acceptance is the process of giving up the struggle with pain and learning to live a worthwhile life despite it. In assessing patients with chronic pain in Turkey, making a diagnosis and tracking the effectiveness of treatment is done with scales that have been translated into Turkish. However, there is as yet no valid and reliable scale in Turkish to assess the acceptance of pain. To validate a Turkish version of the Chronic Pain Acceptance Questionnaire developed by McCracken and colleagues. Methodological and cross sectional study. A simple randomized sampling method was used in selecting the study sample. The sample was composed of 201 patients, more than 10 times the number of items examined for validity and reliability in the study, which totaled 20. A patient identification form, the Chronic Pain Acceptance Questionnaire, and the Brief Pain Inventory were used to collect data. Data were collected by face-to-face interviews. In the validity testing, the content validity index was used to evaluate linguistic equivalence, content validity, construct validity, and expert views. In reliability testing of the scale, Cronbach’s α coefficient was calculated, and item analysis and split-test reliability methods were used. Principal component analysis and varimax rotation were used in factor analysis and to examine factor structure for construct concept validity. The item analysis established that the scale, all items, and item-total correlations were satisfactory. The mean total score of the scale was 21.78. The internal consistency coefficient was 0.94, and the correlation between the two halves of the scale was 0.89. The Chronic Pain Acceptance Questionnaire, which is intended to be used in Turkey upon confirmation of its validity and reliability, is an evaluation instrument with sufficient validity and reliability, and it can be reliably used to examine patients’ acceptance of chronic pain.

Validity and Reliability of the Turkish Chronic Pain Acceptance Questionnaire

PubMed Central

Akmaz, Hazel Ekin; Uyar, Meltem; Kuzeyli Yıldırım, Yasemin; Akın Korhan, Esra

2018-01-01

Background: Pain acceptance is the process of giving up the struggle with pain and learning to live a worthwhile life despite it. In assessing patients with chronic pain in Turkey, making a diagnosis and tracking the effectiveness of treatment is done with scales that have been translated into Turkish. However, there is as yet no valid and reliable scale in Turkish to assess the acceptance of pain. Aims: To validate a Turkish version of the Chronic Pain Acceptance Questionnaire developed by McCracken and colleagues. Study Design: Methodological and cross sectional study. Methods: A simple randomized sampling method was used in selecting the study sample. The sample was composed of 201 patients, more than 10 times the number of items examined for validity and reliability in the study, which totaled 20. A patient identification form, the Chronic Pain Acceptance Questionnaire, and the Brief Pain Inventory were used to collect data. Data were collected by face-to-face interviews. In the validity testing, the content validity index was used to evaluate linguistic equivalence, content validity, construct validity, and expert views. In reliability testing of the scale, Cronbach’s α coefficient was calculated, and item analysis and split-test reliability methods were used. Principal component analysis and varimax rotation were used in factor analysis and to examine factor structure for construct concept validity. Results: The item analysis established that the scale, all items, and item-total correlations were satisfactory. The mean total score of the scale was 21.78. The internal consistency coefficient was 0.94, and the correlation between the two halves of the scale was 0.89. Conclusion: The Chronic Pain Acceptance Questionnaire, which is intended to be used in Turkey upon confirmation of its validity and reliability, is an evaluation instrument with sufficient validity and reliability, and it can be reliably used to examine patients’ acceptance of chronic pain. PMID:29843496

Quality of care assessment in geriatric evaluation and management units: construction of a chart review tool for a tracer condition.

PubMed

Kergoat, Marie-Jeanne; Leclerc, Bernard-Simon; Leduc, Nicole; Latour, Judith; Berg, Katherine; Bolduc, Aline

2009-07-29

The number of elderly people requiring hospital care is growing, so, quality and assessment of care for elders are emerging and complex areas of research. Very few validated and reliable instruments exist for the assessment of quality of acute care in this field. This study's objective was to create such a tool for Geriatric Evaluation and Management Units (GEMUs). The methodology involved a reliability and feasibility study of a retrospective chart review on 934 older inpatients admitted in 49 GEMUs during the year 2002-2003 for fall-related trauma as a tracer condition. Pertinent indicators for a chart abstraction tool, the Geriatric Care Tool (GCT), were developed and validated according to five dimensions: access to care, comprehensiveness, continuity of care, patient-centred care and appropriateness. Consensus methods were used to develop the content. Participants were experts representing eight main health care professions involved in GEMUs from 19 different sites. Items associated with high quality of care at each step of the multidisciplinary management of patients admitted due to falls were identified. The GCT was tested for intra- and inter-rater reliability using 30 medical charts reviewed by each of three independent and blinded trained nurses. Kappa and agreement measures between pairs of chart reviewers were computed on an item-by-item basis. Three quarters of 169 items identifying the process of care, from the case history to discharge planning, demonstrated good agreement (kappa greater than 0.40 and agreement over 70%). Indicators for the appropriateness of care showed less reliability. Content validity and reliability results, as well as the feasibility of the process, suggest that the chart abstraction tool can gather standardized and pertinent clinical information for further evaluating quality of care in GEMU using admission due to falls as a tracer condition. However, the GCT should be evaluated in other models of acute geriatric units and new strategies should be developed to improve reliability of peer assessments in characterizing the quality of care for elderly patients with complex conditions.

Methodology for Software Reliability Prediction. Volume 1.

DTIC Science & Technology

1987-11-01

SPACECRAFT 0 MANNED SPACECRAFT B ATCH SYSTEM AIRBORNE AVIONICS 0 UNMANNED EVENT C014TROL a REAL TIME CLOSED 0 UNMANNED SPACECRAFT LOOP OPERATINS SPACECRAFT...software reliability. A Software Reliability Measurement Framework was established which spans the life cycle of a software system and includes the...specification, prediction, estimation, and assessment of software reliability. Data from 59 systems , representing over 5 million lines of code, were

Overall Key Performance Indicator to Optimizing Operation of High-Pressure Homogenizers for a Reliable Quantification of Intracellular Components in Pichia pastoris.

PubMed

Garcia-Ortega, Xavier; Reyes, Cecilia; Montesinos, José Luis; Valero, Francisco

2015-01-01

The most commonly used cell disruption procedures may present lack of reproducibility, which introduces significant errors in the quantification of intracellular components. In this work, an approach consisting in the definition of an overall key performance indicator (KPI) was implemented for a lab scale high-pressure homogenizer (HPH) in order to determine the disruption settings that allow the reliable quantification of a wide sort of intracellular components. This innovative KPI was based on the combination of three independent reporting indicators: decrease of absorbance, release of total protein, and release of alkaline phosphatase activity. The yeast Pichia pastoris growing on methanol was selected as model microorganism due to it presents an important widening of the cell wall needing more severe methods and operating conditions than Escherichia coli and Saccharomyces cerevisiae. From the outcome of the reporting indicators, the cell disruption efficiency achieved using HPH was about fourfold higher than other lab standard cell disruption methodologies, such bead milling cell permeabilization. This approach was also applied to a pilot plant scale HPH validating the methodology in a scale-up of the disruption process. This innovative non-complex approach developed to evaluate the efficacy of a disruption procedure or equipment can be easily applied to optimize the most common disruption processes, in order to reach not only reliable quantification but also recovery of intracellular components from cell factories of interest.

Overall Key Performance Indicator to Optimizing Operation of High-Pressure Homogenizers for a Reliable Quantification of Intracellular Components in Pichia pastoris

PubMed Central

Garcia-Ortega, Xavier; Reyes, Cecilia; Montesinos, José Luis; Valero, Francisco

2015-01-01

The most commonly used cell disruption procedures may present lack of reproducibility, which introduces significant errors in the quantification of intracellular components. In this work, an approach consisting in the definition of an overall key performance indicator (KPI) was implemented for a lab scale high-pressure homogenizer (HPH) in order to determine the disruption settings that allow the reliable quantification of a wide sort of intracellular components. This innovative KPI was based on the combination of three independent reporting indicators: decrease of absorbance, release of total protein, and release of alkaline phosphatase activity. The yeast Pichia pastoris growing on methanol was selected as model microorganism due to it presents an important widening of the cell wall needing more severe methods and operating conditions than Escherichia coli and Saccharomyces cerevisiae. From the outcome of the reporting indicators, the cell disruption efficiency achieved using HPH was about fourfold higher than other lab standard cell disruption methodologies, such bead milling cell permeabilization. This approach was also applied to a pilot plant scale HPH validating the methodology in a scale-up of the disruption process. This innovative non-complex approach developed to evaluate the efficacy of a disruption procedure or equipment can be easily applied to optimize the most common disruption processes, in order to reach not only reliable quantification but also recovery of intracellular components from cell factories of interest. PMID:26284241

Study on the performance of different craniofacial superimposition approaches (II): Best practices proposal.

PubMed

Damas, S; Wilkinson, C; Kahana, T; Veselovskaya, E; Abramov, A; Jankauskas, R; Jayaprakash, P T; Ruiz, E; Navarro, F; Huete, M I; Cunha, E; Cavalli, F; Clement, J; Lestón, P; Molinero, F; Briers, T; Viegas, F; Imaizumi, K; Humpire, D; Ibáñez, O

2015-12-01

Craniofacial superimposition, although existing for one century, is still a controversial technique within the scientific community. Objective and unbiased validation studies over a significant number of cases are required to establish a more solid picture on the reliability. However, there is lack of protocols and standards in the application of the technique leading to contradictory information concerning reliability. Instead of following a uniform methodology, every expert tends to apply his own approach to the problem, based on the available technology and deep knowledge on human craniofacial anatomy, soft tissues, and their relationships. The aim of this study was to assess the reliability of different craniofacial superimposition methodologies and the corresponding technical approaches to this type of identification. With all the data generated, some of the most representative experts in craniofacial identification joined in a discussion intended to identify and agree on the most important issues that have to be considered to properly employ the craniofacial superimposition technique. As a consequence, the consortium has produced the current manuscript, which can be considered the first standard in the field; including good and bad practices, sources of error and uncertainties, technological requirements and desirable features, and finally a common scale for the craniofacial matching evaluation. Such a document is intended to be part of a more complete framework for craniofacial superimposition, to be developed during the FP7-founded project MEPROCS, which will favour and standardize its proper application. Copyright © 2015 Elsevier Ireland Ltd. All rights reserved.

Reliability Assurance of Detection of EML4-ALK Rearrangement in Non-Small Cell Lung Cancer: The Results of Proficiency Testing in China.

PubMed

Li, Yulong; Zhang, Rui; Peng, Rongxue; Ding, Jiansheng; Han, Yanxi; Wang, Guojing; Zhang, Kuo; Lin, Guigao; Li, Jinming

2016-06-01

Currently, several approaches are being used to detect echinoderm microtubule associated protein like 4 gene (EML4)-anaplastic lymphoma receptor tyrosine kinase gene (ALK) rearrangement, but the performance of laboratories in China is unknown. To evaluate the proficiency of different laboratories in detecting EML4-ALK rearrangement, we organized a proficiency test (PT). We prepared formalin-fixed, paraffin-embedded samples derived from the xenograft tumor tissue of three non-small cell lung cancer cell lines with different EML4-ALK rearrangements and used PTs to evaluate the detection performance of laboratories in China. We received results from 94 laboratories that used different methods. Of the participants, 75.53% correctly identified all samples in the PT panel. Among the errors made by participants, false-negative errors were likely to occur. According to the methodology applied, 82.86%, 76.67%, 77.78%, and 66.67% of laboratories using reverse transcriptase polymerase chain reaction, fluorescence in situ hybridization, next-generation sequencing, and immunohistochemical analysis, respectively, could analyze all the samples correctly. Moreover, we have found that the laboratories' genotyping capacity is high, especially for variant 3. Our PT survey revealed that the performance and methodological problems of laboratories must be addressed to further increase the reproducibility and accuracy of detection of EML4-ALK rearrangement to ensure reliable results for selection of appropriate patients. Copyright © 2016 International Association for the Study of Lung Cancer. Published by Elsevier Inc. All rights reserved.

Reporting and methodological quality of survival analysis in articles published in Chinese oncology journals

PubMed Central

Zhu, Xiaoyan; Zhou, Xiaobin; Zhang, Yuan; Sun, Xiao; Liu, Haihua; Zhang, Yingying

2017-01-01

Abstract Survival analysis methods have gained widespread use in the filed of oncology. For achievement of reliable results, the methodological process and report quality is crucial. This review provides the first examination of methodological characteristics and reporting quality of survival analysis in articles published in leading Chinese oncology journals. To examine methodological and reporting quality of survival analysis, to identify some common deficiencies, to desirable precautions in the analysis, and relate advice for authors, readers, and editors. A total of 242 survival analysis articles were included to be evaluated from 1492 articles published in 4 leading Chinese oncology journals in 2013. Articles were evaluated according to 16 established items for proper use and reporting of survival analysis. The application rates of Kaplan–Meier, life table, log-rank test, Breslow test, and Cox proportional hazards model (Cox model) were 91.74%, 3.72%, 78.51%, 0.41%, and 46.28%, respectively, no article used the parametric method for survival analysis. Multivariate Cox model was conducted in 112 articles (46.28%). Follow-up rates were mentioned in 155 articles (64.05%), of which 4 articles were under 80% and the lowest was 75.25%, 55 articles were100%. The report rates of all types of survival endpoint were lower than 10%. Eleven of 100 articles which reported a loss to follow-up had stated how to treat it in the analysis. One hundred thirty articles (53.72%) did not perform multivariate analysis. One hundred thirty-nine articles (57.44%) did not define the survival time. Violations and omissions of methodological guidelines included no mention of pertinent checks for proportional hazard assumption; no report of testing for interactions and collinearity between independent variables; no report of calculation method of sample size. Thirty-six articles (32.74%) reported the methods of independent variable selection. The above defects could make potentially inaccurate, misleading of the reported results, or difficult to interpret. There are gaps in the conduct and reporting of survival analysis in studies published in Chinese oncology journals, severe deficiencies were noted. More endorsement by journals of the report guideline for survival analysis may improve articles quality, and the dissemination of reliable evidence to oncology clinicians. We recommend authors, readers, reviewers, and editors to consider survival analysis more carefully and cooperate more closely with statisticians and epidemiologists. PMID:29390340

An Assessment Methodology to Evaluate In-Flight Engine Health Management Effectiveness

NASA Astrophysics Data System (ADS)

Maggio, Gaspare; Belyeu, Rebecca; Pelaccio, Dennis G.

2002-01-01

flight effectiveness of candidate engine health management system concepts. A next generation engine health management system will be required to be both reliable and robust in terms of anomaly detection capability. The system must be able to operate successfully in the hostile, high-stress engine system environment. This implies that its system components, such as the instrumentation, process and control, and vehicle interface and support subsystems, must be highly reliable. Additionally, the system must be able to address a vast range of possible engine operation anomalies through a host of different types of measurements supported by a fast algorithm/architecture processing capability that can identify "true" (real) engine operation anomalies. False anomaly condition reports for such a system must be essentially eliminated. The accuracy of identifying only real anomaly conditions has been an issue with the Space Shuttle Main Engine (SSME) in the past. Much improvement in many of the technologies to address these areas is required. The objectives of this study were to identify and demonstrate a consistent assessment methodology that can evaluate the capability of next generation engine health management system concepts to respond in a correct, timely manner to alleviate an operational engine anomaly condition during flight. Science Applications International Corporation (SAIC), with support from NASA Marshall Space Flight Center, identified a probabilistic modeling approach to assess engine health management system concept effectiveness using a deterministic anomaly-time event assessment modeling approach that can be applied in the engine preliminary design stage of development to assess engine health management system concept effectiveness. Much discussion in this paper focuses on the formulation and application approach in performing this assessment. This includes detailed discussion of key modeling assumptions, the overall assessment methodology approach identified, and the identification of key supporting engine health management system concept design/operation and fault mode information required to utilize this methodology. At the paper's conclusion, discussion focuses on a demonstration benchmark study that applied this methodology to the current SSME health management system. A summary of study results and lessons learned are provided. Recommendations for future work in this area are also identified at the conclusion of the paper. * Please direct all correspondence/communication pertaining to this paper to Dennis G. Pelaccio, Science

Complexity, Representation and Practice: Case Study as Method and Methodology

ERIC Educational Resources Information Center

Miles, Rebecca

2015-01-01

While case study is considered a common approach to examining specific and particular examples in research disciplines such as law, medicine and psychology, in the social sciences case study is often treated as a lesser, flawed or undemanding methodology which is less valid, reliable or theoretically rigorous than other methodologies. Building on…

Methodology for nonwork travel analysis in suburban communities.

DOT National Transportation Integrated Search

1994-01-01

The increase in the number of nonwork trips during the past decade has contributed substantially to congestion and to environmental problems. Data collection methodologies, descriptive information, and reliable models of nonwork travel behavior are n...

Hierarchical specification of the SIFT fault tolerant flight control system

NASA Technical Reports Server (NTRS)

Melliar-Smith, P. M.; Schwartz, R. L.

1981-01-01

The specification and mechanical verification of the Software Implemented Fault Tolerance (SIFT) flight control system is described. The methodology employed in the verification effort is discussed, and a description of the hierarchical models of the SIFT system is given. To meet the objective of NASA for the reliability of safety critical flight control systems, the SIFT computer must achieve a reliability well beyond the levels at which reliability can be actually measured. The methodology employed to demonstrate rigorously that the SIFT computer meets as reliability requirements is described. The hierarchy of design specifications from very abstract descriptions of system function down to the actual implementation is explained. The most abstract design specifications can be used to verify that the system functions correctly and with the desired reliability since almost all details of the realization were abstracted out. A succession of lower level models refine these specifications to the level of the actual implementation, and can be used to demonstrate that the implementation has the properties claimed of the abstract design specifications.

Reliability and Maintainability Analysis of a High Air Pressure Compressor Facility

NASA Technical Reports Server (NTRS)

Safie, Fayssal M.; Ring, Robert W.; Cole, Stuart K.

2013-01-01

This paper discusses a Reliability, Availability, and Maintainability (RAM) independent assessment conducted to support the refurbishment of the Compressor Station at the NASA Langley Research Center (LaRC). The paper discusses the methodologies used by the assessment team to derive the repair by replacement (RR) strategies to improve the reliability and availability of the Compressor Station (Ref.1). This includes a RAPTOR simulation model that was used to generate the statistical data analysis needed to derive a 15-year investment plan to support the refurbishment of the facility. To summarize, study results clearly indicate that the air compressors are well past their design life. The major failures of Compressors indicate that significant latent failure causes are present. Given the occurrence of these high-cost failures following compressor overhauls, future major failures should be anticipated if compressors are not replaced. Given the results from the RR analysis, the study team recommended a compressor replacement strategy. Based on the data analysis, the RR strategy will lead to sustainable operations through significant improvements in reliability, availability, and the probability of meeting the air demand with acceptable investment cost that should translate, in the long run, into major cost savings. For example, the probability of meeting air demand improved from 79.7 percent for the Base Case to 97.3 percent. Expressed in terms of a reduction in the probability of failing to meet demand (1 in 5 days to 1 in 37 days), the improvement is about 700 percent. Similarly, compressor replacement improved the operational availability of the facility from 97.5 percent to 99.8 percent. Expressed in terms of a reduction in system unavailability (1 in 40 to 1 in 500), the improvement is better than 1000 percent (an order of magnitude improvement). It is worthy to note that the methodologies, tools, and techniques used in the LaRC study can be used to evaluate similar high value equipment components and facilities. Also, lessons learned in data collection and maintenance practices derived from the observations, findings, and recommendations of the study are extremely important in the evaluation and sustainment of new compressor facilities.

Neck motion kinematics: an inter-tester reliability study using an interactive neck VR assessment in asymptomatic individuals.

PubMed

Sarig Bahat, Hilla; Sprecher, Elliot; Sela, Itamar; Treleaven, Julia

2016-07-01

The use of virtual reality (VR) for assessment and intervention of neck pain has previously been used and shown reliable for cervical range of motion measures. Neck VR enables analysis of task-oriented neck movement by stimulating responsive movements to external stimuli. Therefore, the purpose of this study was to establish inter-tester reliability of neck kinematic measures so that it can be used as a reliable assessment and treatment tool between clinicians. This reliability study included 46 asymptomatic participants, who were assessed using the neck VR system which displayed an interactive VR scenario via a head-mounted device, controlled by neck movements. The objective of the interactive assessment was to hit 16 targets, randomly appearing in four directions, as fast as possible. Each participant was tested twice by two different testers. Good reliability was found of neck motion kinematic measures in flexion, extension, and rotation (0.64-0.93 inter-class correlation). High reliability was shown for peak velocity globally (0.93), in left rotation (0.9), right rotation and extension (0.88), and flexion (0.86). Mean velocity had a good global reliability (0.84), except for left rotation directed movement with moderate reliability (0.68). Minimal detectable change for peak velocity ranged from 41 to 53 °/s, while mean velocity ranged from 20 to 25 °/s. The results suggest high reliability for peak and mean velocity as measured by the interactive Neck VR assessment of neck motion kinematics. VR appears to provide a reliable and more ecologically valid method of cervical motion evaluation than previous conventional methodologies.

Psychometric Properties of the Farsi Version of “Spiritual Needs Questionnaire” for Cancer Patients in Iran: A Methodological Study

PubMed

Hatamipour, Khadijeh; Rassouli, Maryam; Yaghmaie, Farideh; Zendedel, Kazem

2018-04-25

Background and objectives: Spiritual needs are very important requirements to cancer patients. A valid and reliable instrument is needed for evaluation. This study was conducted to psychometrically evaluate a Spiritual Needs Questionnaire (SpNQ) for cancer patients in Iran. Methods: In this study, the methodology and psychometric properties of the Farsi version of the SpNQ (Büssing et al., (2010)) were evaluated, based on the model proposed by Wilde et al., (2005). The study population included cancer patients referred to the largest referral center in Iran. Some 400 subjects were selected. Then, the content, face and construct validity, as well as the internal consistency and reliability of the Farsi version were assessed. Findings: In the confirmatory factor analysis, the original four-factor version with 19 phrases was not confirmed. Subsequently, an exploratory factor analysis (EFA) was carried out in which phrases were included in three dimensions (peace and active giving, religion, and existence) that explained 48.1% of the variance. Later, a confirmatory factor analysis (CFA) was conducted, which showed a good fit of the model (CFI=0.94, GFI=0.94, RMSEA=0.071, and AGFI=0.96). Cronbach’s alpha was α=0.91 for the whole SpNQ. Cronbach’s alpha values ranged from 0.76 to 0.86 for the three factors. The intra-class correlation coefficient was ICC=0.82 between two tests performed with a two-week interval. Conclusion: The modified Farsi version of the SpNQ shows good psychometric properties for patients and can be used to investigate the spiritual needs of Iranian cancer patients. Creative Commons Attribution License

Interviewer as instrument: accounting for human factors in evaluation research.

PubMed

Brown, Joel H

2006-04-01

This methodological study examines an original data collection model designed to incorporate human factors and enhance data richness in qualitative and evaluation research. Evidence supporting this model is drawn from in-depth youth and adult interviews in one of the largest policy/program evaluations undertaken in the United States, the Drug, Alcohol, and Tobacco Education evaluation (77 districts, 118 schools). When applying the explicit observation technique (EOT)--the strategic and nonjudgmental disclosure of nonverbal human factor cues by the interviewer to the respondent during interview--data revealed the observation disclosure pattern. Here, respondents linked perceptions with policy or program implementation or effectiveness evidence. Although more research is needed, it is concluded that the EOT yields richer data when compared with traditional semistructured interviews and, thus, holds promise to enhance qualitative and evaluation research methods. Validity and reliability as well as qualitative and evaluation research considerations are discussed.

The Society for Implementation Research Collaboration Instrument Review Project: a methodology to promote rigorous evaluation.

PubMed

Lewis, Cara C; Stanick, Cameo F; Martinez, Ruben G; Weiner, Bryan J; Kim, Mimi; Barwick, Melanie; Comtois, Katherine A

2015-01-08

Identification of psychometrically strong instruments for the field of implementation science is a high priority underscored in a recent National Institutes of Health working meeting (October 2013). Existing instrument reviews are limited in scope, methods, and findings. The Society for Implementation Research Collaboration Instrument Review Project's objectives address these limitations by identifying and applying a unique methodology to conduct a systematic and comprehensive review of quantitative instruments assessing constructs delineated in two of the field's most widely used frameworks, adopt a systematic search process (using standard search strings), and engage an international team of experts to assess the full range of psychometric criteria (reliability, construct and criterion validity). Although this work focuses on implementation of psychosocial interventions in mental health and health-care settings, the methodology and results will likely be useful across a broad spectrum of settings. This effort has culminated in a centralized online open-access repository of instruments depicting graphical head-to-head comparisons of their psychometric properties. This article describes the methodology and preliminary outcomes. The seven stages of the review, synthesis, and evaluation methodology include (1) setting the scope for the review, (2) identifying frameworks to organize and complete the review, (3) generating a search protocol for the literature review of constructs, (4) literature review of specific instruments, (5) development of an evidence-based assessment rating criteria, (6) data extraction and rating instrument quality by a task force of implementation experts to inform knowledge synthesis, and (7) the creation of a website repository. To date, this multi-faceted and collaborative search and synthesis methodology has identified over 420 instruments related to 34 constructs (total 48 including subconstructs) that are relevant to implementation science. Despite numerous constructs having greater than 20 available instruments, which implies saturation, preliminary results suggest that few instruments stem from gold standard development procedures. We anticipate identifying few high-quality, psychometrically sound instruments once our evidence-based assessment rating criteria have been applied. The results of this methodology may enhance the rigor of implementation science evaluations by systematically facilitating access to psychometrically validated instruments and identifying where further instrument development is needed.

Integrating Formal Methods and Testing 2002

NASA Technical Reports Server (NTRS)

Cukic, Bojan

2002-01-01

Traditionally, qualitative program verification methodologies and program testing are studied in separate research communities. None of them alone is powerful and practical enough to provide sufficient confidence in ultra-high reliability assessment when used exclusively. Significant advances can be made by accounting not only tho formal verification and program testing. but also the impact of many other standard V&V techniques, in a unified software reliability assessment framework. The first year of this research resulted in the statistical framework that, given the assumptions on the success of the qualitative V&V and QA procedures, significantly reduces the amount of testing needed to confidently assess reliability at so-called high and ultra-high levels (10-4 or higher). The coming years shall address the methodologies to realistically estimate the impacts of various V&V techniques to system reliability and include the impact of operational risk to reliability assessment. Combine formal correctness verification, process and product metrics, and other standard qualitative software assurance methods with statistical testing with the aim of gaining higher confidence in software reliability assessment for high-assurance applications. B) Quantify the impact of these methods on software reliability. C) Demonstrate that accounting for the effectiveness of these methods reduces the number of tests needed to attain certain confidence level. D) Quantify and justify the reliability estimate for systems developed using various methods.

Evaluation of the Validity and Reliability of the Waterlow Pressure Ulcer Risk Assessment Scale

PubMed Central

Charalambous, Charalambos; Koulori, Agoritsa; Vasilopoulos, Aristidis; Roupa, Zoe

2018-01-01

Introduction Prevention is the ideal strategy to tackle the problem of pressure ulcers. Pressure ulcer risk assessment scales are one of the most pivotal measures applied to tackle the problem, much criticisms has been developed regarding the validity and reliability of these scales. Objective To investigate the validity and reliability of the Waterlow pressure ulcer risk assessment scale. Method The methodology used is a narrative literature review, the bibliography was reviewed through Cinahl, Pubmed, EBSCO, Medline and Google scholar, 26 scientific articles where identified. The articles where chosen due to their direct correlation with the objective under study and their scientific relevance. Results The construct and face validity of the Waterlow appears adequate, but with regards to content validity changes in the category age and gender can be beneficial. The concurrent validity cannot be assessed. The predictive validity of the Waterlow is characterized by high specificity and low sensitivity. The inter-rater reliability has been demonstrated to be inadequate, this may be due to lack of clear definitions within the categories and differentiating level of knowledge between the users. Conclusion Due to the limitations presented regarding the validity and reliability of the Waterlow pressure ulcer risk assessment scale, the scale should be used in conjunction with clinical assessment to provide optimum results. PMID:29736104

Evaluation of the Validity and Reliability of the Waterlow Pressure Ulcer Risk Assessment Scale.

PubMed

Charalambous, Charalambos; Koulori, Agoritsa; Vasilopoulos, Aristidis; Roupa, Zoe

2018-04-01

Prevention is the ideal strategy to tackle the problem of pressure ulcers. Pressure ulcer risk assessment scales are one of the most pivotal measures applied to tackle the problem, much criticisms has been developed regarding the validity and reliability of these scales. To investigate the validity and reliability of the Waterlow pressure ulcer risk assessment scale. The methodology used is a narrative literature review, the bibliography was reviewed through Cinahl, Pubmed, EBSCO, Medline and Google scholar, 26 scientific articles where identified. The articles where chosen due to their direct correlation with the objective under study and their scientific relevance. The construct and face validity of the Waterlow appears adequate, but with regards to content validity changes in the category age and gender can be beneficial. The concurrent validity cannot be assessed. The predictive validity of the Waterlow is characterized by high specificity and low sensitivity. The inter-rater reliability has been demonstrated to be inadequate, this may be due to lack of clear definitions within the categories and differentiating level of knowledge between the users. Due to the limitations presented regarding the validity and reliability of the Waterlow pressure ulcer risk assessment scale, the scale should be used in conjunction with clinical assessment to provide optimum results.

Proposed Reliability/Cost Model

NASA Technical Reports Server (NTRS)

Delionback, L. M.

1982-01-01

New technique estimates cost of improvement in reliability for complex system. Model format/approach is dependent upon use of subsystem cost-estimating relationships (CER's) in devising cost-effective policy. Proposed methodology should have application in broad range of engineering management decisions.

A Simple and Reliable Method of Design for Standalone Photovoltaic Systems

NASA Astrophysics Data System (ADS)

Srinivasarao, Mantri; Sudha, K. Rama; Bhanu, C. V. K.

2017-06-01

Standalone photovoltaic (SAPV) systems are seen as a promoting method of electrifying areas of developing world that lack power grid infrastructure. Proliferations of these systems require a design procedure that is simple, reliable and exhibit good performance over its life time. The proposed methodology uses simple empirical formulae and easily available parameters to design SAPV systems, that is, array size with energy storage. After arriving at the different array size (area), performance curves are obtained for optimal design of SAPV system with high amount of reliability in terms of autonomy at a specified value of loss of load probability (LOLP). Based on the array to load ratio (ALR) and levelized energy cost (LEC) through life cycle cost (LCC) analysis, it is shown that the proposed methodology gives better performance, requires simple data and is more reliable when compared with conventional design using monthly average daily load and insolation.

An object-oriented approach to risk and reliability analysis : methodology and aviation safety applications.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Dandini, Vincent John; Duran, Felicia Angelica; Wyss, Gregory Dane

2003-09-01

This article describes how features of event tree analysis and Monte Carlo-based discrete event simulation can be combined with concepts from object-oriented analysis to develop a new risk assessment methodology, with some of the best features of each. The resultant object-based event scenario tree (OBEST) methodology enables an analyst to rapidly construct realistic models for scenarios for which an a priori discovery of event ordering is either cumbersome or impossible. Each scenario produced by OBEST is automatically associated with a likelihood estimate because probabilistic branching is integral to the object model definition. The OBEST methodology is then applied to anmore » aviation safety problem that considers mechanisms by which an aircraft might become involved in a runway incursion incident. The resulting OBEST model demonstrates how a close link between human reliability analysis and probabilistic risk assessment methods can provide important insights into aviation safety phenomenology.« less

Methodological challenges when doing research that includes ethnic minorities: a scoping review.

PubMed

Morville, Anne-Le; Erlandsson, Lena-Karin

2016-11-01

There are challenging methodological issues in obtaining valid and reliable results on which to base occupational therapy interventions for ethnic minorities. The aim of this scoping review is to describe the methodological problems within occupational therapy research, when ethnic minorities are included. A thorough literature search yielded 21 articles obtained from the scientific databases PubMed, Cinahl, Web of Science and PsychInfo. Analysis followed Arksey and O'Malley's framework for scoping reviews, applying content analysis. The results showed methodological issues concerning the entire research process from defining and recruiting samples, the conceptual understanding, lack of appropriate instruments, data collection using interpreters to analyzing data. In order to avoid excluding the ethnic minorities from adequate occupational therapy research and interventions, development of methods for the entire research process is needed. It is a costly and time-consuming process, but the results will be valid and reliable, and therefore more applicable in clinical practice.

Smart Sensor Demonstration Payload

NASA Technical Reports Server (NTRS)

Schmalzel, John; Bracey, Andrew; Rawls, Stephen; Morris, Jon; Turowski, Mark; Franzl, Richard; Figueroa, Fernando

2010-01-01

Sensors are a critical element to any monitoring, control, and evaluation processes such as those needed to support ground based testing for rocket engine test. Sensor applications involve tens to thousands of sensors; their reliable performance is critical to achieving overall system goals. Many figures of merit are used to describe and evaluate sensor characteristics; for example, sensitivity and linearity. In addition, sensor selection must satisfy many trade-offs among system engineering (SE) requirements to best integrate sensors into complex systems [1]. These SE trades include the familiar constraints of power, signal conditioning, cabling, reliability, and mass, and now include considerations such as spectrum allocation and interference for wireless sensors. Our group at NASA s John C. Stennis Space Center (SSC) works in the broad area of integrated systems health management (ISHM). Core ISHM technologies include smart and intelligent sensors, anomaly detection, root cause analysis, prognosis, and interfaces to operators and other system elements [2]. Sensor technologies are the base fabric that feed data and health information to higher layers. Cost-effective operation of the complement of test stands benefits from technologies and methodologies that contribute to reductions in labor costs, improvements in efficiency, reductions in turn-around times, improved reliability, and other measures. ISHM is an active area of development at SSC because it offers the potential to achieve many of those operational goals [3-5].

Coupling long and short term decisions in the design of urban water supply infrastructure for added reliability and flexibility

NASA Astrophysics Data System (ADS)

Marques, G.; Fraga, C. C. S.; Medellin-Azuara, J.

2016-12-01

The expansion and operation of urban water supply systems under growing demands, hydrologic uncertainty and water scarcity requires a strategic combination of supply sources for reliability, reduced costs and improved operational flexibility. The design and operation of such portfolio of water supply sources involves integration of long and short term planning to determine what and when to expand, and how much to use of each supply source accounting for interest rates, economies of scale and hydrologic variability. This research presents an integrated methodology coupling dynamic programming optimization with quadratic programming to optimize the expansion (long term) and operations (short term) of multiple water supply alternatives. Lagrange Multipliers produced by the short-term model provide a signal about the marginal opportunity cost of expansion to the long-term model, in an iterative procedure. A simulation model hosts the water supply infrastructure and hydrologic conditions. Results allow (a) identification of trade offs between cost and reliability of different expansion paths and water use decisions; (b) evaluation of water transfers between urban supply systems; and (c) evaluation of potential gains by reducing water system losses as a portfolio component. The latter is critical in several developing countries where water supply system losses are high and often neglected in favor of more system expansion.

Data Sufficiency Assessment and Pumping Test Design for Groundwater Prediction Using Decision Theory and Genetic Algorithms

NASA Astrophysics Data System (ADS)

McPhee, J.; William, Y. W.

2005-12-01

This work presents a methodology for pumping test design based on the reliability requirements of a groundwater model. Reliability requirements take into consideration the application of the model results in groundwater management, expressed in this case as a multiobjective management model. The pumping test design is formulated as a mixed-integer nonlinear programming (MINLP) problem and solved using a combination of genetic algorithm (GA) and gradient-based optimization. Bayesian decision theory provides a formal framework for assessing the influence of parameter uncertainty over the reliability of the proposed pumping test. The proposed methodology is useful for selecting a robust design that will outperform all other candidate designs under most potential 'true' states of the system

Shuttle payload minimum cost vibroacoustic tests

NASA Technical Reports Server (NTRS)

Stahle, C. V.; Gongloff, H. R.; Young, J. P.; Keegan, W. B.

1977-01-01

This paper is directed toward the development of the methodology needed to evaluate cost effective vibroacoustic test plans for Shuttle Spacelab payloads. Statistical decision theory is used to quantitatively evaluate seven alternate test plans by deriving optimum test levels and the expected cost for each multiple mission payload considered. The results indicate that minimum costs can vary by as much as $6 million for the various test plans. The lowest cost approach eliminates component testing and maintains flight vibration reliability by performing subassembly tests at a relatively high acoustic level. Test plans using system testing or combinations of component and assembly level testing are attractive alternatives. Component testing alone is shown not to be cost effective.

[A quickly methodology for drug intelligence using profiling of illicit heroin samples].

PubMed

Zhang, Jianxin; Chen, Cunyi

2012-07-01

The aim of the paper was to evaluate a link between two heroin seizures using a descriptive method. The system involved the derivation and gas chromatographic separation of samples followed by a fully automatic data analysis and transfer to a database. Comparisons used the square cosine function between two chromatograms assimilated to vectors. The method showed good discriminatory capabilities. The probability of false positives was extremely slight. In conclusion, this method proved to be efficient and reliable, which appeared suitable for estimating the links between illicit heroin samples.

Structural design methodologies for ceramic-based material systems

NASA Technical Reports Server (NTRS)

Duffy, Stephen F.; Chulya, Abhisak; Gyekenyesi, John P.

1991-01-01

One of the primary pacing items for realizing the full potential of ceramic-based structural components is the development of new design methods and protocols. The focus here is on low temperature, fast-fracture analysis of monolithic, whisker-toughened, laminated, and woven ceramic composites. A number of design models and criteria are highlighted. Public domain computer algorithms, which aid engineers in predicting the fast-fracture reliability of structural components, are mentioned. Emphasis is not placed on evaluating the models, but instead is focused on the issues relevant to the current state of the art.

Determination of tocopherols and sitosterols in seeds and nuts by QuEChERS-liquid chromatography.

PubMed

Delgado-Zamarreño, M Milagros; Fernández-Prieto, Cristina; Bustamante-Rangel, Myriam; Pérez-Martín, Lara

2016-02-01

In the present work a simple, reliable and affordable sample treatment method for the simultaneous analysis of tocopherols and free phytosterols in nuts was developed. Analyte extraction was carried out using the QuEChERS methodology and analyte separation and detection were accomplished using HPLC-DAD. The use of this methodology for the extraction of natural occurring substances provides advantages such as speed, simplicity and ease of use. The parameters evaluated for the validation of the method developed included the linearity of the calibration plots, the detection and quantification limits, repeatability, reproducibility and recovery. The proposed method was successfully applied to the analysis of tocopherols and free phytosterols in samples of almonds, cashew nuts, hazelnuts, peanuts, tiger nuts, sun flower seeds and pistachios. Copyright © 2015 Elsevier Ltd. All rights reserved.

Monitoring the fracture behavior of metal matrix composites by combined NDE methodologies

NASA Astrophysics Data System (ADS)

Kordatos, E. Z.; Exarchos, D. A.; Mpalaskas, A. C.; Matikas, T. E.

2015-03-01

Current work deals with the non-destructive evaluation (NDE) of the fatigue behavior of metal matrix composites (MMCs) materials using Infrared Thermography (IRT) and Acoustic Emission (AE). AE monitoring was employed to record a wide spectrum of cracking events enabling the characterization of the severity of fracture in relation to the applied load. IR thermography as a non-destructive, real-time and non-contact technique, allows the detection of heat waves generated by the thermo-mechanical coupling during mechanical loading of the sample. In this study an IR methodology, based on the monitoring of the intrinsically dissipated energy, was applied for the determination of the fatigue limit of A359/SiCp composites. The thermographic monitoring is in agreement with the AE results enabling the reliable monitoring of the MMCs' fatigue behavior.

Uncovering productive morphosyntax in French-learning toddlers: a multidimensional methodology perspective.

PubMed

Barrière, Isabelle; Goyet, Louise; Kresh, Sarah; Legendre, Géraldine; Nazzi, Thierry

2016-09-01

The present study applies a multidimensional methodological approach to the study of the acquisition of morphosyntax. It focuses on evaluating the degree of productivity of an infrequent subject-verb agreement pattern in the early acquisition of French and considers the explanatory role played by factors such as input frequency, semantic transparency of the agreement markers, and perceptual factors in accounting for comprehension of agreement in number (singular vs. plural) in an experimental setting. Results on a pointing task involving pseudo-verbs demonstrate significant comprehension of both singular and plural agreement in children aged 2;6. The experimental results are shown not to reflect input frequency, input marker reliability on its own, or lexically driven knowledge. We conclude that toddlers have knowledge of subject-verb agreement at age 2;6 which is abstract and productive despite its paucity in the input.

Evaluation of tools used to measure calcium and/or dairy consumption in children and adolescents.

PubMed

Magarey, Anthea; Yaxley, Alison; Markow, Kylie; Baulderstone, Lauren; Miller, Michelle

2014-08-01

To identify and critique tools that assess Ca and/or dairy intake in children to ascertain the most accurate and reliable tools available. A systematic review of the literature was conducted using defined inclusion and exclusion criteria. Articles were included on the basis that they reported on a tool measuring Ca and/or dairy intake in children in Western countries and reported on originally developed tools or tested the validity or reliability of existing tools. Defined criteria for reporting reliability and validity properties were applied. Studies in Western countries. Children. Eighteen papers reporting on two tools that assessed dairy intake, ten that assessed Ca intake and five that assessed both dairy and Ca were identified. An examination of tool testing revealed high reliance on lower-order tests such as correlation and failure to differentiate between statistical and clinically meaningful significance. Only half of the tools were tested for reliability and results indicated that only one Ca tool and one dairy tool were reliable. Validation studies showed acceptable levels of agreement (<100 mg difference) and/or sensitivity (62-83 %) and specificity (55-77 %) in three Ca tools. With reference to the testing methodology and results, no tools were considered both valid and reliable for the assessment of dairy intake and only one tool proved valid and reliable for the assessment of Ca intake. These results clearly indicate the need for development and rigorous testing of tools to assess Ca and/or dairy intake in children and adolescents.

Analyzing Reliability and Performance Trade-Offs of HLS-Based Designs in SRAM-Based FPGAs Under Soft Errors

NASA Astrophysics Data System (ADS)

Tambara, Lucas Antunes; Tonfat, Jorge; Santos, André; Kastensmidt, Fernanda Lima; Medina, Nilberto H.; Added, Nemitala; Aguiar, Vitor A. P.; Aguirre, Fernando; Silveira, Marcilei A. G.

2017-02-01

The increasing system complexity of FPGA-based hardware designs and shortening of time-to-market have motivated the adoption of new designing methodologies focused on addressing the current need for high-performance circuits. High-Level Synthesis (HLS) tools can generate Register Transfer Level (RTL) designs from high-level software programming languages. These tools have evolved significantly in recent years, providing optimized RTL designs, which can serve the needs of safety-critical applications that require both high performance and high reliability levels. However, a reliability evaluation of HLS-based designs under soft errors has not yet been presented. In this work, the trade-offs of different HLS-based designs in terms of reliability, resource utilization, and performance are investigated by analyzing their behavior under soft errors and comparing them to a standard processor-based implementation in an SRAM-based FPGA. Results obtained from fault injection campaigns and radiation experiments show that it is possible to increase the performance of a processor-based system up to 5,000 times by changing its architecture with a small impact in the cross section (increasing up to 8 times), and still increasing the Mean Workload Between Failures (MWBF) of the system.

Validation of the Practice Environment Scale to the Brazilian culture.

PubMed

Gasparino, Renata C; Guirardello, Edinêis de B

2017-07-01

To validate the Brazilian version of the Practice Environment Scale. The Practice Environment Scale is a tool that evaluates the presence of characteristics that are favourable for professional nursing practice because a better work environment contributes to positive results for patients, professionals and institutions. Methodological study including 209 nurses. Validity was assessed via a confirmatory factor analysis using structural equation modelling, in which the correlations between the instrument and the following variables were tested: burnout, job satisfaction, safety climate, perception of quality of care and intention to leave the job. Subgroups were compared and the reliability was assessed using Cronbach's alpha and the composite reliability. Factor analysis resulted in exclusion of seven items. Significant correlations were obtained between the subscales and all variables in the study. The reliability was considered acceptable. The Brazilian version of the Practice Environment Scale is a valid and reliable tool used to assess the characteristics that promote professional nursing practice. Use of this tool in Brazilian culture should allow managers to implement changes that contribute to the achievement of better results, in addition to identifying and comparing the environments of health institutions. © 2017 John Wiley & Sons Ltd.

Comparison of methods for profiling O-glycosylation: Human Proteome Organisation Human Disease Glycomics/Proteome Initiative multi-institutional study of IgA1.

PubMed

Wada, Yoshinao; Dell, Anne; Haslam, Stuart M; Tissot, Bérangère; Canis, Kévin; Azadi, Parastoo; Bäckström, Malin; Costello, Catherine E; Hansson, Gunnar C; Hiki, Yoshiyuki; Ishihara, Mayumi; Ito, Hiromi; Kakehi, Kazuaki; Karlsson, Niclas; Hayes, Catherine E; Kato, Koichi; Kawasaki, Nana; Khoo, Kay-Hooi; Kobayashi, Kunihiko; Kolarich, Daniel; Kondo, Akihiro; Lebrilla, Carlito; Nakano, Miyako; Narimatsu, Hisashi; Novak, Jan; Novotny, Milos V; Ohno, Erina; Packer, Nicolle H; Palaima, Elizabeth; Renfrow, Matthew B; Tajiri, Michiko; Thomsson, Kristina A; Yagi, Hirokazu; Yu, Shin-Yi; Taniguchi, Naoyuki

2010-04-01

The Human Proteome Organisation Human Disease Glycomics/Proteome Initiative recently coordinated a multi-institutional study that evaluated methodologies that are widely used for defining the N-glycan content in glycoproteins. The study convincingly endorsed mass spectrometry as the technique of choice for glycomic profiling in the discovery phase of diagnostic research. The present study reports the extension of the Human Disease Glycomics/Proteome Initiative's activities to an assessment of the methodologies currently used for O-glycan analysis. Three samples of IgA1 isolated from the serum of patients with multiple myeloma were distributed to 15 laboratories worldwide for O-glycomics analysis. A variety of mass spectrometric and chromatographic procedures representative of current methodologies were used. Similar to the previous N-glycan study, the results convincingly confirmed the pre-eminent performance of MS for O-glycan profiling. Two general strategies were found to give the most reliable data, namely direct MS analysis of mixtures of permethylated reduced glycans in the positive ion mode and analysis of native reduced glycans in the negative ion mode using LC-MS approaches. In addition, mass spectrometric methodologies to analyze O-glycopeptides were also successful.

Development of Process Control Methodology for Tracking the Quality and Safety of Pain, Agitation, and Sedation Management in Critical Care Units.

PubMed

Walsh, Timothy S; Kydonaki, Kalliopi; Lee, Robert J; Everingham, Kirsty; Antonelli, Jean; Harkness, Ronald T; Cole, Stephen; Quasim, Tara; Ruddy, James; McDougall, Marcia; Davidson, Alan; Rutherford, John; Richards, Jonathan; Weir, Christopher J

2016-03-01

To develop sedation, pain, and agitation quality measures using process control methodology and evaluate their properties in clinical practice. A Sedation Quality Assessment Tool was developed and validated to capture data for 12-hour periods of nursing care. Domains included pain/discomfort and sedation-agitation behaviors; sedative, analgesic, and neuromuscular blocking drug administration; ventilation status; and conditions potentially justifying deep sedation. Predefined sedation-related adverse events were recorded daily. Using an iterative process, algorithms were developed to describe the proportion of care periods with poor limb relaxation, poor ventilator synchronization, unnecessary deep sedation, agitation, and an overall optimum sedation metric. Proportion charts described processes over time (2 monthly intervals) for each ICU. The numbers of patients treated between sedation-related adverse events were described with G charts. Automated algorithms generated charts for 12 months of sequential data. Mean values for each process were calculated, and variation within and between ICUs explored qualitatively. Eight Scottish ICUs over a 12-month period. Mechanically ventilated patients. None. The Sedation Quality Assessment Tool agitation-sedation domains correlated with the Richmond Sedation Agitation Scale score (Spearman ρ = 0.75) and were reliable in clinician-clinician (weighted kappa; κ = 0.66) and clinician-researcher (κ = 0.82) comparisons. The limb movement domain had fair correlation with Behavioral Pain Scale (ρ = 0.24) and was reliable in clinician-clinician (κ = 0.58) and clinician-researcher (κ = 0.45) comparisons. Ventilator synchronization correlated with Behavioral Pain Scale (ρ = 0.54), and reliability in clinician-clinician (κ = 0.29) and clinician-researcher (κ = 0.42) comparisons was fair-moderate. Eight hundred twenty-five patients were enrolled (range, 59-235 across ICUs), providing 12,385 care periods for evaluation (range 655-3,481 across ICUs). The mean proportion of care periods with each quality metric varied between ICUs: excessive sedation 12-38%; agitation 4-17%; poor relaxation 13-21%; poor ventilator synchronization 8-17%; and overall optimum sedation 45-70%. Mean adverse event intervals ranged from 1.5 to 10.3 patients treated. The quality measures appeared relatively stable during the observation period. Process control methodology can be used to simultaneously monitor multiple aspects of pain-sedation-agitation management within ICUs. Variation within and between ICUs could be used as triggers to explore practice variation, improve quality, and monitor this over time.

The Structured Clinical Interview for DSM-IV Childhood Diagnoses (Kid-SCID): first psychometric evaluation in a Dutch sample of clinically referred youths.

PubMed

Roelofs, Jeffrey; Muris, Peter; Braet, Caroline; Arntz, Arnoud; Beelen, Imke

2015-06-01

The Structured Clinical Interview for DSM-IV Childhood Disorders (Kid-SCID) is a semi-structured interview for the classification of psychiatric disorders in children and adolescents. This study presents a first evaluation of the psychometric properties of the Kid-SCID in a Dutch sample of children and adolescents who had been referred to an outpatient treatment centre for mental health problems. Results indicated that the inter-rater reliability of the Kid-SCID classifications and the internal consistency of various (dimensional) criteria of the diagnoses were moderate to good. Further, for most Kid-SCID diagnoses, reasonable agreement between children and parents was found. Finally, the correspondence between the Kid-SCID and the final clinical diagnosis as established after the full intake procedure, which included the information as provided by the Kid-SCID, ranged from poor to good. Results are discussed in the light of methodological issues pertaining to the assessment of psychiatric disorders in youths. The Kid-SCID can generally be seen as a reliable and useful tool that can assist clinicians in carrying out clinical evaluations of children and adolescents.

Development of an interprofessional lean facilitator assessment scale.

PubMed

Bravo-Sanchez, Cindy; Dorazio, Vincent; Denmark, Robert; Heuer, Albert J; Parrott, J Scott

2018-05-01

High reliability is important for optimising quality and safety in healthcare organisations. Reliability efforts include interprofessional collaborative practice (IPCP) and Lean quality/process improvement strategies, which require skilful facilitation. Currently, no validated Lean facilitator assessment tool for interprofessional collaboration exists. This article describes the development and pilot evaluation of such a tool; the Interprofessional Lean Facilitator Assessment Scale (ILFAS), which measures both technical and 'soft' skills, which have not been measured in other instruments. The ILFAS was developed using methodologies and principles from Lean/Shingo, IPCP, metacognition research and Bloom's Taxonomy of Learning Domains. A panel of experts confirmed the initial face validity of the instrument. Researchers independently assessed five facilitators, during six Lean sessions. Analysis included quantitative evaluation of rater agreement. Overall inter-rater agreement of the assessment of facilitator performance was high (92%), and discrepancies in the agreement statistics were analysed. Face and content validity were further established, and usability was evaluated, through primary stakeholder post-pilot feedback, uncovering minor concerns, leading to tool revision. The ILFAS appears comprehensive in the assessment of facilitator knowledge, skills, abilities, and may be useful in the discrimination between facilitators of different skill levels. Further study is needed to explore instrument performance and validity.

Evaluation of oxygen reduction activity by the thin-film rotating disk electrode methodology: The effects of potentiodynamic parameters

DOE PAGES

Chen, Guangyu; Li, Meng; Kuttiyiel, Kurian A.; ...

2016-04-11

Here, an accurate and efficient assessment of activity is critical for the research and development of electrocatalysts for oxygen reduction reaction (ORR). Currently, the methodology combining the thin-film rotating disk electrode (TF-RDE) and potentiodynamic polarization is the most commonly used to pre-evaluate ORR activity, acquire kinetic data (i.e., kinetic current, Tafel slope, etc.), and gain understanding of the ORR mechanism. However, it is often neglected that appropriate potentiodynamic parameters have to be chosen to obtain reliable results. We first evaluate the potentiodynamic and potentiostatic polarization measurements with TF-RDE to examine the ORR activity of Pt nanoelectrocatalyst. Furthermore, our results demonstratemore » that besides depending on the nature of electrocatalyst, the apparent ORR kinetics also strongly depends on the associated potentiodynamic parameters, such as scan rate and scan region, which have a great effect on the coverage of adsorbed OH ad/O ad on Pt surface, thereby affecting the ORR activities of both nanosized and bulk Pt. However, the apparent Tafel slopes remained nearly the same, indicating that the ORR mechanism in all the measurements was not affected by different potentiodynamic parameters.« less

The design and evaluation of psychometric properties for a questionnaire on elderly abuse by family caregivers among older adults on hemodialysis

PubMed Central

Mahmoudian, Amaneh; Torabi Chafjiri, Razieh; Alipour, Atefeh; Shamsalinia, Abbas; Ghaffari, Fatemeh

2018-01-01

Introduction Older adults with chronic disease are more vulnerable to abuse. Early and accurate detection of the elderly abuse phenomenon can help identify health-promoting solutions for the elderly, their family, and society. The purpose of this study was to design and evaluate the psychometric properties of a questionnaire on elderly abuse by family caregivers among older adults on hemodialysis. Methods Qualitative and quantitative research methodologies were used to develop the questionnaire. The item pool was compiled from literature reviews and the Delphi method. The literature reviews comprised 22 studies. The psychometric properties of the questionnaire were verified using face, content, and construct validity, and the reliability was tested using Cronbach’s alpha reliability. Results A 57-item questionnaire was developed after the psychometric evaluation. The Kaiser–Meyer–Olkin index and Bartlett’s test of sphericity showed reliable results. Seven components from the exploratory content analysis including psychological misbehavior, authority deprivation, physical misbehavior, financial misbehavior, being abandoned, caring neglect, and emotional misbehavior explained 74.769% of the total variance. Cronbach’s alpha was 0.98 and the interclass correlation coefficient was r=0.91 responding to the items twice (p<0.001), which shows a high level of tool stability. Conclusion This study developed a questionnaire to assess elderly abuse by family caregivers among older adults on hemodialysis. It is recommended as a mini scale that can be used both in statistical and practical studies, and that is valid and reliable. Nurses or other health care providers can use it in health centers, dialysis centers, or at the house of the patient. PMID:29670340

ECSIN's methodological approach for hazard evaluation of engineered nanomaterials

NASA Astrophysics Data System (ADS)

Bregoli, Lisa; Benetti, Federico; Venturini, Marco; Sabbioni, Enrico

2013-04-01

The increasing production volumes and commercialization of engineered nanomaterials (ENM), together with data on their higher biological reactivity when compared to bulk counterpart and ability to cross biological barriers, have caused concerns about their potential impacts on the health and safety of both humans and the environment. A multidisciplinary component of the scientific community has been called to evaluate the real risks associated with the use of products containing ENM, and is today in the process of developing specific definitions and testing strategies for nanomaterials. At ECSIN we are developing an integrated multidisciplinary methodological approach for the evaluation of the biological effects of ENM on the environment and human health. While our testing strategy agrees with the most widely advanced line of work at the European level, the choice of methods and optimization of protocols is made with an extended treatment of details. Our attention to the methodological and technical details is based on the acknowledgment that the innovative characteristics of matter at the nano-size range may influence the existing testing methods in a partially unpredictable manner, an aspect which is frequently recognized at the discussion level but oftentimes disregarded at the laboratory bench level. This work outlines the most important steps of our testing approach. In particular, each step will be briefly discussed in terms of potential technical and methodological pitfalls that we have encountered, and which are often ignored in nanotoxicology research. The final aim is to draw attention to the need of preliminary studies in developing reliable tests, a crucial aspect to confirm the suitability of the chosen analytical and toxicological methods to be used for the specific tested nanoparticle, and to express the idea that in nanotoxicology,"devil is in the detail".

Criteria for evaluating programme theory diagrams in quality improvement initiatives: a structured method for appraisal.

PubMed

Issen, Laurel; Woodcock, Thomas; McNicholas, Christopher; Lennox, Laura; Reed, Julie E

2018-04-09

Despite criticisms that many quality improvement (QI) initiatives fail due to incomplete programme theory, there is no defined way to evaluate how programme theory has been articulated. The objective of this research was to develop, and assess the usability and reliability of scoring criteria to evaluate programme theory diagrams. Criteria development was informed by published literature and QI experts. Inter-rater reliability was tested between two evaluators. About 63 programme theory diagrams (42 driver diagrams and 21 action-effect diagrams) were reviewed to establish whether the criteria could support comparative analysis of different approaches to constructing diagrams. Components of the scoring criteria include: assessment of overall aim, logical overview, clarity of components, cause-effect relationships, evidence and measurement. Independent reviewers had 78% inter-rater reliability. Scoring enabled direct comparison of different approaches to developing programme theory; action-effect diagrams were found to have had a statistically significant but moderate improvement in programme theory quality over driver diagrams; no significant differences were observed based on the setting in which driver diagrams were developed. The scoring criteria summarise the necessary components of programme theory that are thought to contribute to successful QI projects. The viability of the scoring criteria for practical application was demonstrated. Future uses include assessment of individual programme theory diagrams and comparison of different approaches (e.g. methodological, teaching or other QI support) to produce programme theory. The criteria can be used as a tool to guide the production of better programme theory diagrams, and also highlights where additional support for QI teams could be needed.

Evaluating the evidence for non-monotonic dose-response relationships: A systematic literature review and (re-)analysis of in vivo toxicity data in the area of food safety.

PubMed

Varret, C; Beronius, A; Bodin, L; Bokkers, B G H; Boon, P E; Burger, M; De Wit-Bos, L; Fischer, A; Hanberg, A; Litens-Karlsson, S; Slob, W; Wolterink, G; Zilliacus, J; Beausoleil, C; Rousselle, C

2018-01-15

This study aims to evaluate the evidence for the existence of non-monotonic dose-responses (NMDRs) of substances in the area of food safety. This review was performed following the systematic review methodology with the aim to identify in vivo studies published between January 2002 and February 2015 containing evidence for potential NMDRs. Inclusion and reliability criteria were defined and used to select relevant and reliable studies. A set of six checkpoints was developed to establish the likelihood that the data retrieved contained evidence for NMDR. In this review, 49 in vivo studies were identified as relevant and reliable, of which 42 were used for dose-response analysis. These studies contained 179 in vivo dose-response datasets with at least five dose groups (and a control group) as fewer doses cannot provide evidence for NMDR. These datasets were extracted and analyzed using the PROAST software package. The resulting dose-response relationships were evaluated for possible evidence of NMDRs by applying the six checkpoints. In total, 10 out of the 179 in vivo datasets fulfilled all six checkpoints. While these datasets could be considered as providing evidence for NMDR, replicated studies would still be needed to check if the results can be reproduced to rule out that the non-monotonicity was caused by incidental anomalies in that specific study. This approach, combining a systematic review with a set of checkpoints, is new and appears useful for future evaluations of the dose response datasets regarding evidence of non-monotonicity. Published by Elsevier Inc.

A systematic review on the quality of measurement techniques for the assessment of burn wound depth or healing potential.

PubMed

Jaspers, Mariëlle E H; van Haasterecht, Ludo; van Zuijlen, Paul P M; Mokkink, Lidwine B

2018-06-22

Reliable and valid assessment of burn wound depth or healing potential is essential to treatment decision-making, to provide a prognosis, and to compare studies evaluating different treatment modalities. The aim of this review was to critically appraise, compare and summarize the quality of relevant measurement properties of techniques that aim to assess burn wound depth or healing potential. A systematic literature search was performed using PubMed, EMBASE and Cochrane Library. Two reviewers independently evaluated the methodological quality of included articles using an adapted version of the Consensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist. A synthesis of evidence was performed to rate the measurement properties for each technique and to draw an overall conclusion on quality of the techniques. Thirty-six articles were included, evaluating various techniques, classified as (1) laser Doppler techniques; (2) thermography or thermal imaging; (3) other measurement techniques. Strong evidence was found for adequate construct validity of laser Doppler imaging (LDI). Moderate evidence was found for adequate construct validity of thermography, videomicroscopy, and spatial frequency domain imaging (SFDI). Only two studies reported on the measurement property reliability. Furthermore, considerable variation was observed among comparator instruments. Considering the evidence available, it appears that LDI is currently the most favorable technique; thereby assessing burn wound healing potential. Additional research is needed into thermography, videomicroscopy, and SFDI to evaluate their full potential. Future studies should focus on reliability and measurement error, and provide a precise description of which construct is aimed to measure. Copyright © 2018 Elsevier Ltd and ISBI. All rights reserved.

Levels of Evidence in Cosmetic Surgery: Analysis and Recommendations Using a New CLEAR Classification

PubMed Central

2013-01-01

Background: The Level of Evidence rating was introduced in 2011 to grade the quality of publications. This system evaluates study design but does not assess several other quality indicators. This study introduces a new “Cosmetic Level of Evidence And Recommendation” (CLEAR) classification that includes additional methodological criteria and compares this new classification with the existing system. Methods: All rated publications in the Cosmetic Section of Plastic and Reconstructive Surgery, July 2011 through June 2013, were evaluated. The published Level of Evidence rating (1–5) and criteria relevant to study design and methodology for each study were tabulated. A new CLEAR rating was assigned to each article, including a recommendation grade (A–D). The published Level of Evidence rating (1–5) was compared with the recommendation grade determined using the CLEAR classification. Results: Among the 87 cosmetic articles, 48 studies (55%) were designated as level 4. Three articles were assigned a level 1, but they contained deficiencies sufficient to undermine the conclusions. The correlation between the published Level of Evidence classification (1–5) and CLEAR Grade (A–D) was weak (ρ = 0.11, not significant). Only 41 studies (48%) evaluated consecutive patients or consecutive patients meeting inclusion criteria. Conclusions: The CLEAR classification considers methodological factors in evaluating study reliability. A prospective study among consecutive patients meeting eligibility criteria, with a reported inclusion rate, the use of contemporaneous controls when indicated, and consideration of confounders is a realistic goal. Such measures are likely to improve study quality. PMID:25289261

Tailoring a Human Reliability Analysis to Your Industry Needs

NASA Technical Reports Server (NTRS)

DeMott, D. L.

2016-01-01

Companies at risk of accidents caused by human error that result in catastrophic consequences include: airline industry mishaps, medical malpractice, medication mistakes, aerospace failures, major oil spills, transportation mishaps, power production failures and manufacturing facility incidents. Human Reliability Assessment (HRA) is used to analyze the inherent risk of human behavior or actions introducing errors into the operation of a system or process. These assessments can be used to identify where errors are most likely to arise and the potential risks involved if they do occur. Using the basic concepts of HRA, an evolving group of methodologies are used to meet various industry needs. Determining which methodology or combination of techniques will provide a quality human reliability assessment is a key element to developing effective strategies for understanding and dealing with risks caused by human errors. There are a number of concerns and difficulties in "tailoring" a Human Reliability Assessment (HRA) for different industries. Although a variety of HRA methodologies are available to analyze human error events, determining the most appropriate tools to provide the most useful results can depend on industry specific cultures and requirements. Methodology selection may be based on a variety of factors that include: 1) how people act and react in different industries, 2) expectations based on industry standards, 3) factors that influence how the human errors could occur such as tasks, tools, environment, workplace, support, training and procedure, 4) type and availability of data, 5) how the industry views risk & reliability, and 6) types of emergencies, contingencies and routine tasks. Other considerations for methodology selection should be based on what information is needed from the assessment. If the principal concern is determination of the primary risk factors contributing to the potential human error, a more detailed analysis method may be employed versus a requirement to provide a numerical value as part of a probabilistic risk assessment. Industries involved with humans operating large equipment or transport systems (ex. railroads or airlines) would have more need to address the man machine interface than medical workers administering medications. Human error occurs in every industry; in most cases the consequences are relatively benign and occasionally beneficial. In cases where the results can have disastrous consequences, the use of Human Reliability techniques to identify and classify the risk of human errors allows a company more opportunities to mitigate or eliminate these types of risks and prevent costly tragedies.

Accuracy assessment: The statistical approach to performance evaluation in LACIE. [Great Plains corridor, United States

NASA Technical Reports Server (NTRS)

Houston, A. G.; Feiveson, A. H.; Chhikara, R. S.; Hsu, E. M. (Principal Investigator)

1979-01-01

A statistical methodology was developed to check the accuracy of the products of the experimental operations throughout crop growth and to determine whether the procedures are adequate to accomplish the desired accuracy and reliability goals. It has allowed the identification and isolation of key problems in wheat area yield estimation, some of which have been corrected and some of which remain to be resolved. The major unresolved problem in accuracy assessment is that of precisely estimating the bias of the LACIE production estimator. Topics covered include: (1) evaluation techniques; (2) variance and bias estimation for the wheat production estimate; (3) the 90/90 evaluation; (4) comparison of the LACIE estimate with reference standards; and (5) first and second order error source investigations.

Evaluation of a Method for Rapid Detection of Listeria monocytogenes in Dry-Cured Ham Based on Impedanciometry Combined with Chromogenic Agar.

PubMed

Labrador, Mirian; Rota, María C; Pérez, Consuelo; Herrera, Antonio; Bayarri, Susana

2018-05-01

The food industry is in need of rapid, reliable methodologies for the detection of Listeria monocytogenes in ready-to-eat products, as an alternative to the International Organization of Standardization (ISO) 11290-1 reference method. The aim of this study was to evaluate impedanciometry combined with chromogenic agar culture for the detection of L. monocytogenes in dry-cured ham. The experimental setup consisted in assaying four strains of L. monocytogenes and two strains of Listeria innocua in pure culture. The method was evaluated according to the ISO 16140:2003 standard through a comparative study with the ISO reference method with 119 samples of dry-cured ham. Significant determination coefficients ( R 2 of up to 0.99) for all strains assayed in pure culture were obtained. The comparative study results had 100% accuracy, 100% specificity, and 100% sensitivity. Impedanciometry followed by chromogenic agar culture was capable of detecting 1 CFU/25 g of food. L. monocytogenes was not detected in the 65 commercial samples tested. The method evaluated herein represents a promising alternative for the food industry in its efforts to control L. monocytogenes. Overall analysis time is shorter and the method permits a straightforward analysis of a large number of samples with reliable results.

Applying different equations to evaluate the level of mismatch between students and school furniture.

PubMed

Castellucci, H I; Arezes, P M; Molenbroek, J F M

2014-07-01

The mismatch between students and school furniture is likely to result in a number of negative effects, such as uncomfortable body posture, pain, and ultimately, it may also affect the learning process. This study's main aim is to review the literature describing the criteria equations for defining the mismatch between students and school furniture, to apply these equations to a specific sample and, based on the results, to propose a methodology to evaluate school furniture suitability. The literature review comprises one publications database, which was used to identify the studies carried out in the field of the abovementioned mismatch. The sample used for testing the different equations was composed of 2261 volunteer subjects from 14 schools. Fifteen studies were found to meet the criteria of this review and 21 equations to test 6 furniture dimensions were identified. Regarding seat height, there are considerable differences between the two most frequently used equations. Although seat to desk clearance was evaluated by knee height, this condition seems to be based on the false assumption that students are sitting on a chair with a proper seat height. Finally, the proposed methodology for suitability evaluation of school furniture should allow for a more reliable analysis of school furniture. Copyright © 2014 Elsevier Ltd and The Ergonomics Society. All rights reserved.

Mixed method evaluation of a community-based physical activity program using the RE-AIM framework: practical application in a real-world setting.

PubMed

Koorts, Harriet; Gillison, Fiona

2015-11-06

Communities are a pivotal setting in which to promote increases in child and adolescent physical activity behaviours. Interventions implemented in these settings require effective evaluation to facilitate translation of findings to wider settings. The aims of this paper are to i) present findings from a RE-AIM evaluation of a community-based physical activity program, and ii) review the methodological challenges faced when applying RE-AIM in practice. A single mixed-methods case study was conducted based on a concurrent triangulation design. Five sources of data were collected via interviews, questionnaires, archival records, documentation and field notes. Evidence was triangulated within RE-AIM to assess individual and organisational-level program outcomes. Inconsistent availability of data and a lack of robust reporting challenged assessment of all five dimensions. Reach, Implementation and setting-level Adoption were less successful, Effectiveness and Maintenance at an individual and organisational level were moderately successful. Only community-level Adoption was highly successful, reflecting the key program goal to provide community-wide participation in sport and physical activity. This research highlighted important methodological constraints associated with the use of RE-AIM in practice settings. Future evaluators wishing to use RE-AIM may benefit from a mixed-method triangulation approach to offset challenges with data availability and reliability.

Analysis of travel-time reliability for freight corridors connecting the Pacific Northwest.

DOT National Transportation Integrated Search

2012-11-01

A new methodology and algorithms were developed to combine diverse data sources and to estimate the impacts of recurrent and non-recurrent : congestion on freight movements reliability and delays, costs, and emissions. The results suggest that tra...

Validity and reliability of Turkish Caregiver Burden Scale among family caregivers of haemodialysis patients.

PubMed

Cil Akinci, Ayse; Pinar, Rukiye

2014-02-01

To investigate the validity and reliability of the Caregiver Burden Scale in family members who provide primary care for haemodialysis patients. In Turkey, there is a need for a multi-dimensional instrument to evaluate the caregiver burden in people who provide care for patients with chronic diseases. A methodological study. The study sample consisted of 161 family members who provide primary care for haemodialysis patients. The forward-backward translation method was used to develop the Turkish Caregiver Burden Scale. The reliability was based on internal consistency investigated by Cronbach's alpha and item-total correlation. The factorial construct validity of the scale was tested with confirmatory factor analysis. By means of convergent and divergent validity, correlation between Caregiver Burden Scale and 36-Item Short Form Health Survey (SF-36) and correlation between Caregiver Burden Scale and the Maslach Burnout Scale were investigated. Cronbach's alpha and item-total correlations results suggested that there was good internal reliability. We found five underlying factors similar to original Scale's five-factor solution. The confirmatory factor analysis five-factor model represented an acceptable fit. Factor loadings were significant, with standardised loadings ranging from 0·43-0·81. By means of divergent validity, all sub-dimension scores and the total score of the Caregiver Burden Scale were negatively correlated with the SF-36, whereas there was a positive correlation with the emotional exhaustion and depersonalisation subscales of the Maslach Burnout Scale as expected. These results suggest that the Caregiver Burden Scale is a reliable and valid instrument which can be used with confidence in Turkish caregivers for haemodialysis patients to screen caregiver burden. The burden experienced by people who provide care for patients with chronic diseases can be evaluated with the Caregiver Burden Scale. Additionally, the Caregiver Burden Scale can be used in the evaluation of the effectiveness of attempts to decrease caregiver burden. © 2012 Blackwell Publishing Ltd.

Validity and Reliability of Field-Based Measures for Assessing Movement Skill Competency in Lifelong Physical Activities: A Systematic Review.

PubMed

Hulteen, Ryan M; Lander, Natalie J; Morgan, Philip J; Barnett, Lisa M; Robertson, Samuel J; Lubans, David R

2015-10-01

It has been suggested that young people should develop competence in a variety of 'lifelong physical activities' to ensure that they can be active across the lifespan. The primary aim of this systematic review is to report the methodological properties, validity, reliability, and test duration of field-based measures that assess movement skill competency in lifelong physical activities. A secondary aim was to clearly define those characteristics unique to lifelong physical activities. A search of four electronic databases (Scopus, SPORTDiscus, ProQuest, and PubMed) was conducted between June 2014 and April 2015 with no date restrictions. Studies addressing the validity and/or reliability of lifelong physical activity tests were reviewed. Included articles were required to assess lifelong physical activities using process-oriented measures, as well as report either one type of validity or reliability. Assessment criteria for methodological quality were adapted from a checklist used in a previous review of sport skill outcome assessments. Movement skill assessments for eight different lifelong physical activities (badminton, cycling, dance, golf, racquetball, resistance training, swimming, and tennis) in 17 studies were identified for inclusion. Methodological quality, validity, reliability, and test duration (time to assess a single participant), for each article were assessed. Moderate to excellent reliability results were found in 16 of 17 studies, with 71% reporting inter-rater reliability and 41% reporting intra-rater reliability. Only four studies in this review reported test-retest reliability. Ten studies reported validity results; content validity was cited in 41% of these studies. Construct validity was reported in 24% of studies, while criterion validity was only reported in 12% of studies. Numerous assessments for lifelong physical activities may exist, yet only assessments for eight lifelong physical activities were included in this review. Generalizability of results may be more applicable if more heterogeneous samples are used in future research. Moderate to excellent levels of inter- and intra-rater reliability were reported in the majority of studies. However, future work should look to establish test-retest reliability. Validity was less commonly reported than reliability, and further types of validity other than content validity need to be established in future research. Specifically, predictive validity of 'lifelong physical activity' movement skill competency is needed to support the assertion that such activities provide the foundation for a lifetime of activity.

Modelling Single Tree Structure with Terrestrial Laser Scanner

NASA Astrophysics Data System (ADS)

Yurtseven, H.; Akgül, M.; Gülci, S.

2017-11-01

Recent technological developments, which has reliable accuracy and quality for all engineering works, such as remote sensing tools have wide range use in forestry applications. Last decade, sustainable use and management opportunities of forest resources are favorite topics. Thus, precision of obtained data plays an important role in evaluation of current status of forests' value. The use of aerial and terrestrial laser technology has more reliable and effective models to advance the appropriate natural resource management. This study investigates the use of terrestrial laser scanner (TLS) technology in forestry, and also the methodological data processing stages for tree volume extraction is explained. Z+F Imager 5010C TLS system was used for measure single tree information such as tree height, diameter of breast height, branch volume and canopy closure. In this context more detailed and accurate data can be obtained than conventional inventory sampling in forestry by using TLS systems. However the accuracy of obtained data is up to the experiences of TLS operator in the field. Number of scan stations and its positions are other important factors to reduce noise effect and accurate 3D modelling. The results indicated that the use of point cloud data to extract tree information for forestry applications are promising methodology for precision forestry.

Evaluation of counseling outcomes at a university counseling center: the impact of clinically significant change on problem resolution and academic functioning.

PubMed

Choi, Keum-Hyeong; Buskey, Wendy; Johnson, Bonita

2010-07-01

The main purpose of this study was to investigate how receiving personal counseling at a university counseling center helps students deal with their personal problems and facilitates academic functioning. To that end, this study used both clinical and academic outcome measures that are relevant to the practice of counseling provided at a counseling center and its unique function in an institution of higher education. In addition, this study used the clinical significance methodology (N. S. Jacobson & P. Truax, 1991) that takes into account clients' differences in making clinically reliable and significant change. Pre-intake and post-termination surveys, including the Outcome Questionnaire (M. J. Lambert, K. Lunnen, V. Umphress, N. Hansen, & G. Burlingame, 1994), were completed by 78 clients, and the responses were analyzed using clinical significance methodology. The results revealed that those who made clinically reliable and significant change (i.e., the recovered group) reported the highest level of improvement in academic commitment to their educational goals and problem resolution, compared with those who did not make clinically significant change. The implications of the findings on practice for counseling at university counseling centers and for administrators in higher education institutions are discussed. (c) 2010 APA, all rights reserved.

Controlled cell-seeding methodologies: a first step toward clinically relevant bone tissue engineering strategies.

PubMed

Impens, Saartje; Chen, Yantian; Mullens, Steven; Luyten, Frank; Schrooten, Jan

2010-12-01

The repair of large and complex bone defects could be helped by a cell-based bone tissue engineering strategy. A reliable and consistent cell-seeding methodology is a mandatory step in bringing bone tissue engineering into the clinic. However, optimization of the cell-seeding step is only relevant when it can be reliably evaluated. The cell seeding efficiency (CSE) plays a fundamental role herein. Results showed that cell lysis and the definition used to determine the CSE played a key role in quantifying the CSE. The definition of CSE should therefore be consistent and unambiguous. The study of the influence of five drop-seeding-related parameters within the studied test conditions showed that (i) the cell density and (ii) the seeding vessel did not significantly affect the CSE, whereas (iii) the volume of seeding medium-to-free scaffold volume ratio (MFR), (iv) the seeding time, and (v) the scaffold morphology did. Prolonging the incubation time increased the CSE up to a plateau value at 4 h. Increasing the MFR or permeability by changing the morphology of the scaffolds significantly reduced the CSE. These results confirm that cell seeding optimization is needed and that an evidence-based selection of the seeding conditions is favored.

“Retention Projection” Enables Reliable Use of Shared Gas Chromatographic Retention Data Across Labs, Instruments, and Methods

PubMed Central

Barnes, Brian B.; Wilson, Michael B.; Carr, Peter W.; Vitha, Mark F.; Broeckling, Corey D.; Heuberger, Adam L.; Prenni, Jessica; Janis, Gregory C.; Corcoran, Henry; Snow, Nicholas H.; Chopra, Shilpi; Dhandapani, Ramkumar; Tawfall, Amanda; Sumner, Lloyd W.; Boswell, Paul G.

2014-01-01

Gas chromatography-mass spectrometry (GC-MS) is a primary tool used to identify compounds in complex samples. Both mass spectra and GC retention times are matched to those of standards, but it is often impractical to have standards on hand for every compound of interest, so we must rely on shared databases of MS data and GC retention information. Unfortunately, retention databases (e.g. linear retention index libraries) are experimentally restrictive, notoriously unreliable, and strongly instrument dependent, relegating GC retention information to a minor, often negligible role in compound identification despite its potential power. A new methodology called “retention projection” has great potential to overcome the limitations of shared chromatographic databases. In this work, we tested the reliability of the methodology in five independent laboratories. We found that even when each lab ran nominally the same method, the methodology was 3-fold more accurate than retention indexing because it properly accounted for unintentional differences between the GC-MS systems. When the labs used different methods of their own choosing, retention projections were 4- to 165-fold more accurate. More importantly, the distribution of error in the retention projections was predictable across different methods and labs, thus enabling automatic calculation of retention time tolerance windows. Tolerance windows at 99% confidence were generally narrower than those widely used even when physical standards are on hand to measure their retention. With its high accuracy and reliability, the new retention projection methodology makes GC retention a reliable, precise tool for compound identification, even when standards are not available to the user. PMID:24205931

Evaluation of lower leg function in patients with Achilles tendinopathy.

PubMed

Silbernagel, Karin Grävare; Gustavsson, Alexander; Thomeé, Roland; Karlsson, Jon

2006-11-01

Achilles tendinopathy is considered to be one of the most common overuse injuries in elite and recreational athletes. However, the effect that the Achilles tendinopathy has on patients' physical performance is still unclear. The purpose of this study was to evaluate if Achilles tendinopathy caused functional deficits on the injured side compared with the non-injured side in patients. A test battery comprised of tests for different aspects of muscle-tendon function of the gastrocnemius, soleus and Achilles tendon complex was developed to evaluate lower leg function. The test battery's test-retest reliability and sensitivity (the percent probability that the tests would demonstrate abnormal lower limb symmetry index in patients) were also evaluated. The test battery consisted of three jump tests, a counter movements jump (CMJ), a drop counter movement jump (drop CMJ) and hopping, and two strength tests, concentric toe-raises, eccentric-concentric toe-raises and toe-raises for endurance. The reliability was evaluated through a test-retest design on 15 healthy subjects. The test battery's sensitivity and possible functional deficits in patients with Achilles tendinopathy were evaluated on 42 patients (19 women and 23 men). An excellent reliability was found between test days 1-2 and 2-3 for all tests (ICC = 0.76-0.94) except for concentric toe-raise, test 2-3, which had fair reliability (ICC = 0.73). The methodological error ranged from 8 to 17%. There were significant differences (P = 0.001-0.049) between the non-injured (or least symptomatic) side and injured (most symptomatic) side for hopping, drop CMJ, concentric and eccentric-concentric toe-raises, and significant differences (P = 0.000-0.012) in the level of pain during CMJ, hopping, and drop CMJ. The sensitivity of the test battery at a 90% capacity was 88. Achilles tendinopathy causes not only pain and symptoms in patients but also apparent impairments in various aspects of lower leg muscle-tendon function as measured with the test battery. This test battery is reliable and able to detect differences in lower leg function between the injured or "most symptomatic" and non-injured or "least symptomatic" side in patients with Achilles tendinopathy. The test battery has higher demand on patients' function compared with each individual test.

Validity and reliability of the Persian version of mobile phone addiction scale.

PubMed

Mazaheri, Maryam Amidi; Karbasi, Mojtaba

2014-02-01

With regard to large number of mobile users especially among college students in Iran, addiction to mobile phone is attracting increasing concern. There is an urgent need for reliable and valid instrument to measure this phenomenon. This study examines validity and reliability of the Persian version of mobile phone addiction scale (MPAIS) in college students. this methodological study was down in Isfahan University of Medical Sciences. One thousand one hundred and eighty students were selected by convenience sampling. The English version of the MPAI questionnaire was translated into Persian with the approach of Jones et al. (Challenges in language, culture, and modality: Translating English measures into American Sign Language. Nurs Res 2006; 55: 75-81). Its reliability was tested by Cronbach's alpha and its dimensionality validity was evaluated using Pearson correlation coefficients with other measures of mobile phone use and IAT. Construct validity was evaluated using Exploratory subscale analysis. Cronbach's alpha of 0.86 was obtained for total PMPAS, for subscale1 (eight items) was 0.84, for subscale 2 (five items) was 0.81 and for subscale 3 (two items) was 0.77. There were significantly positive correlations between the score of PMPAS and IAT (r = 0.453, P < 0.001) and other measures of mobile phone use. Principal component subscale analysis yielded a three-subscale structure including: inability to control craving; feeling anxious and lost; mood improvement accounted for 60.57% of total variance. The results of discriminate validity showed that all the item's correlations with related subscale were greater than 0.5 and correlations with unrelated subscale were less than 0.5. Considering lack of a valid and reliable questionnaire for measuring addiction to the mobile phone, PMPAS could be a suitable instrument for measuring mobile phone addiction in future research.

Cross-cultural adaptation, reliability, and validity of the work role functioning questionnaire to Brazilian Portuguese.

PubMed

Gallasch, Cristiane Helena; Alexandre, Neusa Maria Costa; Amick, Benjamin

2007-12-01

The study objectives were to translate and adapt the Work Role Functioning Questionnaire (WRFQ) into the Brazilian Portuguese language and evaluate its reliability in patients experiencing musculoskeletal disorders. The cross-cultural adaptation was performed according to the internationally recommended methodology, using the following guidelines: translation, back-translation, revision by a committee, and pretest. At first, the questionnaire was independently translated by two bilingual translators, who had Portuguese as their mother language. Subsequently, two other translators whose mother language was English did the back-translation. A committee composed of five specialists revised and compared the translations obtained, developing the final version for pretest application. The pretest was carried out with 30 patients experiencing musculoskeletal disorders. Psychometric properties were evaluated by administering the questionnaire to 105 subjects with musculoskeletal disorders and receiving physical therapy treatment. The reliability was estimated through stability and homogeneity assessment. The construct validity was tested comparing subjects experiencing musculoskeletal disorders to healthy workers. The results indicated good content validity and internal consistency (Cronbach alpha = 0.95). Cronbach alpha for each scale was >0.85, except for the social demand scale. The Intraclass Correlation Coefficient for the test-retest reliability was satisfactory for mental demands (ICC = 0.68) and excellent for the others (0.82-0.91). In relation to the construct validity, the mean score obtained for each scale was lower for physical, work scheduling, and output demands in the subjects with musculoskeletal disorders. There was a significant difference (p < 0.001) between the groups in comparison to work scheduling, physical, and output demands. The data showed that the cross-cultural adaptation process was successful and the adapted instrument demonstrated psychometric properties making it reliable to use in Brazilian culture.

Methodological quality evaluation of systematic reviews or meta-analyses on ERCC1 in non-small cell lung cancer: a systematic review.

PubMed

Tao, Huan; Zhang, Yueyuan; Li, Qian; Chen, Jin

2017-11-01

To assess the methodological quality of systematic reviews (SRs) or meta-analysis concerning the predictive value of ERCC1 in platinum chemotherapy of non-small cell lung cancer. We searched the PubMed, EMbase, Cochrane library, international prospective register of systematic reviews, Chinese BioMedical Literature Database, China National Knowledge Infrastructure, Wan Fang and VIP database for SRs or meta-analysis. The methodological quality of included literatures was evaluated by risk of bias in systematic review (ROBIS) scale. Nineteen eligible SRs/meta-analysis were included. The most frequently searched databases were EMbase (74%), PubMed, Medline and CNKI. Fifteen SRs did additional retrieval manually, but none of them retrieved the registration platform. 47% described the two-reviewers model in the screening for eligible original articles, and seven SRs described the two reviewers to extract data. In methodological quality assessment, inter-rater reliability Kappa was 0.87 between two reviewers. Research question were well related to all SRs in phase 1 and the eligibility criteria was suitable for each SR, and rated as 'low' risk bias. But the 'high' risk bias existed in all the SRs regarding methods used to identify and/or select studies, and data collection and study appraisal. More than two-third of SRs or meta-analysis were finished with high risk of bias in the synthesis, findings and the final phase. The study demonstrated poor methodological quality of SRs/meta-analysis assessing the predictive value of ERCC1 in chemotherapy among the NSCLC patients, especially the high performance bias. Registration or publishing the protocol is recommended in future research.

Development and Validation of a Collocated Exposure Monitoring Methodology using Portable Air Monitors

NASA Astrophysics Data System (ADS)

Li, Z.; Che, W.; Frey, H. C.; Lau, A. K. H.

2016-12-01

Portable air monitors are currently being developed and used to enable a move towards exposure monitoring as opposed to fixed site monitoring. Reliable methods are needed regarding capturing spatial and temporal variability in exposure concentration to obtain credible data from which to develop efficient exposure mitigation measures. However, there are few studies that quantify the validity and repeatability of the collected data. The objective of this study is to present and evaluate a collocated exposure monitoring (CEM) methodology including the calibration of portable air monitors against stationary reference equipment, side-by-side comparison of portable air monitors, personal or microenvironmental exposure monitoring and the processing and interpretation of the collected data. The CEM methodology was evaluated based on application to portable monitors TSI DustTrak II Aerosol Monitor 8530 for fine particulate matter (PM2.5) and TSI Q-Trak model 7575 with probe model 982 for CO, CO2, temperature and relative humidity. Taking a school sampling campaign in Hong Kong in January and June, 2015 as an example, the calibrated side-by-side measured 1 Hz PM2.5 concentrations showed good consistency between two sets of portable air monitors. Confidence in side-by-side comparison, PM2.5 concentrations of which most of the time were within 2 percent, enabled robust inference regarding differences when the monitors measured in classroom and pedestrian during school hour. The proposed CEM methodology can be widely applied in sampling campaigns with the objective of simultaneously characterizing pollutant concentrations in two or more locations or microenvironments. The further application of the CEM methodology to transportation exposure will be presented and discussed.

Methodology for safety optimization of highway cross-sections for horizontal curves with restricted sight distance.

PubMed

Ibrahim, Shewkar E; Sayed, Tarek; Ismail, Karim

2012-11-01

Several earlier studies have noted the shortcomings with existing geometric design guides which provide deterministic standards. In these standards the safety margin of the design output is generally unknown and there is little knowledge of the safety implications of deviating from the standards. To mitigate these shortcomings, probabilistic geometric design has been advocated where reliability analysis can be used to account for the uncertainty in the design parameters and to provide a mechanism for risk measurement to evaluate the safety impact of deviations from design standards. This paper applies reliability analysis for optimizing the safety of highway cross-sections. The paper presents an original methodology to select a suitable combination of cross-section elements with restricted sight distance to result in reduced collisions and consistent risk levels. The purpose of this optimization method is to provide designers with a proactive approach to the design of cross-section elements in order to (i) minimize the risk associated with restricted sight distance, (ii) balance the risk across the two carriageways of the highway, and (iii) reduce the expected collision frequency. A case study involving nine cross-sections that are parts of two major highway developments in British Columbia, Canada, was presented. The results showed that an additional reduction in collisions can be realized by incorporating the reliability component, P(nc) (denoting the probability of non-compliance), in the optimization process. The proposed approach results in reduced and consistent risk levels for both travel directions in addition to further collision reductions. Copyright © 2012 Elsevier Ltd. All rights reserved.

Reliability Prediction Analysis: Airborne System Results and Best Practices

NASA Astrophysics Data System (ADS)

Silva, Nuno; Lopes, Rui

2013-09-01

This article presents the results of several reliability prediction analysis for aerospace components, made by both methodologies, the 217F and the 217Plus. Supporting and complementary activities are described, as well as the differences concerning the results and the applications of both methodologies that are summarized in a set of lessons learned that are very useful for RAMS and Safety Prediction practitioners.The effort that is required for these activities is also an important point that is discussed, as is the end result and their interpretation/impact on the system design.The article concludes while positioning these activities and methodologies in an overall process for space and aeronautics equipment/components certification, and highlighting their advantages. Some good practices have also been summarized and some reuse rules have been laid down.

Practical tool to assess reliability of web-based medicines information.

PubMed

Lebanova, Hristina; Getov, Ilko; Grigorov, Evgeni

2014-02-01

Information disseminated by medicines information systems is not always easy to apply. Nowadays internet provides access to enormous volume and range of health information that was previously inaccessible both for medical specialists and consumers. The aim of this study is to assess internet as a source of drug and health related information and to create test methodology to evaluate the top 10 visited health-related web-sites in Bulgaria. Using existing scientific methodologies for evaluation of web sources, a new algorithm of three-step approach consisting of score-card validation of the drug-related information in the 10 most visited Bulgarian web-sites was created. In many cases the drug information in the internet sites contained errors and discrepancies. Some of the published materials were not validated; they were out-of-date and could cause confusion for consumers. The quality of the online health information is a cause for considerable information noise and threat to patients' safety and rational drug use. There is a need of monitoring the drugs information available online in order to prevent patient misinformation and confusion that could lead to medication errors and abuse.

[Evaluation of quality of service in Early Intervention: A systematic review].

PubMed

Jemes Campaña, Inmaculada Concepción; Romero-Galisteo, Rita Pilar; Labajos Manzanares, María Teresa; Moreno Morales, Noelia

2018-06-07

Early Intervention (EI), as a paediatric service, has the duty of quantifying the results and the quality of its services provided. The accessibility of valid and reliable tools allows professionals to evaluate the quality of these services. The aim of this study is to review the scientific literature on tools used to measure the methodological and service quality in EI. A search was made in different databases: Medline (from PubMed), Web of Science, PsycINFO, Cochrane, Scopus, ERIC and Scielo. The methodological quality of the studies was tested using the COSMIN scale. A total of 13 manuscripts met the criteria to be included in this review. Ten of them received a "good" or "reasonable" score based on the COSMIN scale. Despite its importance, there is no consensus among authors on the measurement of service quality in EI. It is often the family of the children attended in EI that are considered the target to study, although the opinion of professionals carries more weight and completes the information. Copyright © 2018. Publicado por Elsevier España, S.L.U.

Prioritization Methodology for Chemical Replacement

NASA Technical Reports Server (NTRS)

Cruit, W.; Schutzenhofer, S.; Goldberg, B.; Everhart, K.

1993-01-01

This project serves to define an appropriate methodology for effective prioritization of efforts required to develop replacement technologies mandated by imposed and forecast legislation. The methodology used is a semiquantitative approach derived from quality function deployment techniques (QFD Matrix). This methodology aims to weigh the full environmental, cost, safety, reliability, and programmatic implications of replacement technology development to allow appropriate identification of viable candidates and programmatic alternatives. The results are being implemented as a guideline for consideration for current NASA propulsion systems.

Probabilistic fatigue methodology for six nines reliability

NASA Technical Reports Server (NTRS)

Everett, R. A., Jr.; Bartlett, F. D., Jr.; Elber, Wolf

1990-01-01

Fleet readiness and flight safety strongly depend on the degree of reliability that can be designed into rotorcraft flight critical components. The current U.S. Army fatigue life specification for new rotorcraft is the so-called six nines reliability, or a probability of failure of one in a million. The progress of a round robin which was established by the American Helicopter Society (AHS) Subcommittee for Fatigue and Damage Tolerance is reviewed to investigate reliability-based fatigue methodology. The participants in this cooperative effort are in the U.S. Army Aviation Systems Command (AVSCOM) and the rotorcraft industry. One phase of the joint activity examined fatigue reliability under uniquely defined conditions for which only one answer was correct. The other phases were set up to learn how the different industry methods in defining fatigue strength affected the mean fatigue life and reliability calculations. Hence, constant amplitude and spectrum fatigue test data were provided so that each participant could perform their standard fatigue life analysis. As a result of this round robin, the probabilistic logic which includes both fatigue strength and spectrum loading variability in developing a consistant reliability analysis was established. In this first study, the reliability analysis was limited to the linear cumulative damage approach. However, it is expected that superior fatigue life prediction methods will ultimately be developed through this open AHS forum. To that end, these preliminary results were useful in identifying some topics for additional study.

Standardizing evaluation of pQCT image quality in the presence of subject movement: qualitative versus quantitative assessment.

PubMed

Blew, Robert M; Lee, Vinson R; Farr, Joshua N; Schiferl, Daniel J; Going, Scott B

2014-02-01

Peripheral quantitative computed tomography (pQCT) is an essential tool for assessing bone parameters of the limbs, but subject movement and its impact on image quality remains a challenge to manage. The current approach to determine image viability is by visual inspection, but pQCT lacks a quantitative evaluation. Therefore, the aims of this study were to (1) examine the reliability of a qualitative visual inspection scale and (2) establish a quantitative motion assessment methodology. Scans were performed on 506 healthy girls (9-13 years) at diaphyseal regions of the femur and tibia. Scans were rated for movement independently by three technicians using a linear, nominal scale. Quantitatively, a ratio of movement to limb size (%Move) provided a measure of movement artifact. A repeat-scan subsample (n = 46) was examined to determine %Move's impact on bone parameters. Agreement between measurers was strong (intraclass correlation coefficient = 0.732 for tibia, 0.812 for femur), but greater variability was observed in scans rated 3 or 4, the delineation between repeat and no repeat. The quantitative approach found ≥95% of subjects had %Move <25 %. Comparison of initial and repeat scans by groups above and below 25% initial movement showed significant differences in the >25 % grouping. A pQCT visual inspection scale can be a reliable metric of image quality, but technicians may periodically mischaracterize subject motion. The presented quantitative methodology yields more consistent movement assessment and could unify procedure across laboratories. Data suggest a delineation of 25% movement for determining whether a diaphyseal scan is viable or requires repeat.

Standardizing Evaluation of pQCT Image Quality in the Presence of Subject Movement: Qualitative vs. Quantitative Assessment

PubMed Central

Blew, Robert M.; Lee, Vinson R.; Farr, Joshua N.; Schiferl, Daniel J.; Going, Scott B.

2013-01-01

Purpose Peripheral quantitative computed tomography (pQCT) is an essential tool for assessing bone parameters of the limbs, but subject movement and its impact on image quality remains a challenge to manage. The current approach to determine image viability is by visual inspection, but pQCT lacks a quantitative evaluation. Therefore, the aims of this study were to (1) examine the reliability of a qualitative visual inspection scale, and (2) establish a quantitative motion assessment methodology. Methods Scans were performed on 506 healthy girls (9–13yr) at diaphyseal regions of the femur and tibia. Scans were rated for movement independently by three technicians using a linear, nominal scale. Quantitatively, a ratio of movement to limb size (%Move) provided a measure of movement artifact. A repeat-scan subsample (n=46) was examined to determine %Move’s impact on bone parameters. Results Agreement between measurers was strong (ICC = .732 for tibia, .812 for femur), but greater variability was observed in scans rated 3 or 4, the delineation between repeat or no repeat. The quantitative approach found ≥95% of subjects had %Move <25%. Comparison of initial and repeat scans by groups above and below 25% initial movement, showed significant differences in the >25% grouping. Conclusions A pQCT visual inspection scale can be a reliable metric of image quality but technicians may periodically mischaracterize subject motion. The presented quantitative methodology yields more consistent movement assessment and could unify procedure across laboratories. Data suggest a delineation of 25% movement for determining whether a diaphyseal scan is viable or requires repeat. PMID:24077875

Assessment of Nutrient Status in Athletes and the Need for Supplementation.

PubMed

Larson-Meyer, D Enette; Woolf, Kathleen; Burke, Louise

2018-03-01

Nutrition assessment is a necessary first step in advising athletes on dietary strategies that include dietary supplementation, and in evaluating the effectiveness of supplementation regimens. Although dietary assessment is the cornerstone component of the nutrition assessment process, it should be performed within the context of a complete assessment that includes collection/evaluation of anthropometric, biochemical, clinical, and environmental data. Collection of dietary intake data can be challenging, with the potential for significant error of validity and reliability, which include inherent errors of the collection methodology, coding of data by dietitians, estimation of nutrient composition using nutrient food tables and/or dietary software programs, and expression of data relative to reference standards including eating guidance systems, macronutrient guidelines for athletes, and recommended dietary allowances. Limitations in methodologies used to complete anthropometric assessment and biochemical analysis also exist, as reference norms for the athlete are not well established and practical and reliable biomarkers are not available for all nutrients. A clinical assessment collected from history information and the nutrition-focused physical exam may help identify overt nutrient deficiencies but may be unremarkable in the well-trained athlete. Assessment of potential food-drug interactions and environmental components further helps make appropriate dietary and supplement recommendations. Overall, the assessment process can help the athlete understand that supplement intake cannot make up for poor food choices and an inadequate diet, while a healthy diet helps ensure maximal benefit from supplementation. Establishment of reference norms specifically for well-trained athletes for the nutrition assessment process is a future research priority.

A brief measure of attitudes toward mixed methods research in psychology.

PubMed

Roberts, Lynne D; Povee, Kate

2014-01-01

The adoption of mixed methods research in psychology has trailed behind other social science disciplines. Teaching psychology students, academics, and practitioners about mixed methodologies may increase the use of mixed methods within the discipline. However, tailoring and evaluating education and training in mixed methodologies requires an understanding of, and way of measuring, attitudes toward mixed methods research in psychology. To date, no such measure exists. In this article we present the development and initial validation of a new measure: Attitudes toward Mixed Methods Research in Psychology. A pool of 42 items developed from previous qualitative research on attitudes toward mixed methods research along with validation measures was administered via an online survey to a convenience sample of 274 psychology students, academics and psychologists. Principal axis factoring with varimax rotation on a subset of the sample produced a four-factor, 12-item solution. Confirmatory factor analysis on a separate subset of the sample indicated that a higher order four factor model provided the best fit to the data. The four factors; 'Limited Exposure,' '(in)Compatibility,' 'Validity,' and 'Tokenistic Qualitative Component'; each have acceptable internal reliability. Known groups validity analyses based on preferred research orientation and self-rated mixed methods research skills, and convergent and divergent validity analyses based on measures of attitudes toward psychology as a science and scientist and practitioner orientation, provide initial validation of the measure. This brief, internally reliable measure can be used in assessing attitudes toward mixed methods research in psychology, measuring change in attitudes as part of the evaluation of mixed methods education, and in larger research programs.

Predicting Cost/Reliability/Maintainability of Advanced General Aviation Avionics Equipment

NASA Technical Reports Server (NTRS)

Davis, M. R.; Kamins, M.; Mooz, W. E.

1978-01-01

A methodology is provided for assisting NASA in estimating the cost, reliability, and maintenance (CRM) requirements for general avionics equipment operating in the 1980's. Practical problems of predicting these factors are examined. The usefulness and short comings of different approaches for modeling coast and reliability estimates are discussed together with special problems caused by the lack of historical data on the cost of maintaining general aviation avionics. Suggestions are offered on how NASA might proceed in assessing cost reliability CRM implications in the absence of reliable generalized predictive models.

PERFORMANCE, RELIABILITY, AND IMPROVEMENT OF A TISSUE-SPECIFIC METABOLIC SIMULATOR

EPA Science Inventory

A methodology is described that has been used to build and enhance a simulator for rat liver metabolism providing reliable predictions within a large chemical domain. The tissue metabolism simulator (TIMES) utilizes a heuristic algorithm to generate plausible metabolic maps using...

Evaluation of the reported association of obsessive-compulsive symptoms or disorder with Tourette's disorder.

PubMed

Shapiro, A K; Shapiro, E

1992-01-01

This review evaluates the evidence reporting an association of obsessive-compulsive symptoms (OCS) and obsessive-compulsive disorder (OCD) with Tourette's syndrome or disorder (TS). Published reports in the literature describing a relationship between OCS-OCD and TS provided the data for the review. The methodological adequacy of the studies are discussed and rated on five criteria: adequacy of the experimental sample, presence and adequacy of the control sample, whether tics are defined as OCS-OCD, whether blind procedures are used to diagnose OCS-OCD in subjects and controls, and evidence for the reliability and validity of OCS-OCD measures. Although there are considerable clinical indications suggesting an association of OCS-OCD with TS and chronic motor tic disorder (CMT), and a possible overlap between OSC-OCD and TS, our evaluation of the evidence does not provide adequate support for an association between these disorders. To meaningfully evaluate the possible relationship between OCS-OCD and TS requires development of specific criteria for classification of OCS-OCD-TS symptoms, use of adequate experimental and control samples, blind evaluation, reliable and valid measures of OCS-OCD-TS, and appropriate statistical analysis. If such studies are performed, it is possible that the strong relationship reported between OCS-OCD and TS is more likely to be artifact than fact, and recent bandwagon effect rather than the latest breakthrough.

Measurement properties of disease-specific questionnaires in patients with neck pain: a systematic review.

PubMed

Schellingerhout, Jasper M; Verhagen, Arianne P; Heymans, Martijn W; Koes, Bart W; de Vet, Henrica C; Terwee, Caroline B

2012-05-01

To critically appraise and compare the measurement properties of the original versions of neck-specific questionnaires. Bibliographic databases were searched for articles concerning the development or evaluation of the measurement properties of an original version of a self-reported questionnaire, evaluating pain and/or disability, which was specifically developed or adapted for patients with neck pain. The methodological quality of the selected studies and the results of the measurement properties were critically appraised and rated using a checklist, specifically designed for evaluating studies on measurement properties. The search strategy resulted in a total of 3,641 unique hits, of which 25 articles, evaluating 8 different questionnaires, were included in our study. The Neck Disability Index is the most frequently evaluated questionnaire and shows positive results for internal consistency, content validity, structural validity, hypothesis testing, and responsiveness, but a negative result for reliability. The other questionnaires show positive results, but the evidence for each measurement property is mostly limited, and at least 50% of the information on measurement properties per questionnaire is lacking. Our findings imply that studies of high methodological quality are needed to properly assess the measurement properties of the currently available questionnaires. Until high quality studies are available, we recommend using these questionnaires with caution. There is no need for the development of new neck-specific questionnaires until the current questionnaires have been adequately assessed.

Prevalidation in pharmaceutical analysis. Part I. Fundamentals and critical discussion.

PubMed

Grdinić, Vladimir; Vuković, Jadranka

2004-05-28

A complete prevalidation, as a basic prevalidation strategy for quality control and standardization of analytical procedure was inaugurated. Fast and simple, the prevalidation methodology based on mathematical/statistical evaluation of a reduced number of experiments (N < or = 24) was elaborated and guidelines as well as algorithms were given in detail. This strategy has been produced for the pharmaceutical applications and dedicated to the preliminary evaluation of analytical methods where linear calibration model, which is very often occurred in practice, could be the most appropriate to fit experimental data. The requirements presented in this paper should therefore help the analyst to design and perform the minimum number of prevalidation experiments needed to obtain all the required information to evaluate and demonstrate the reliability of its analytical procedure. In complete prevalidation process, characterization of analytical groups, checking of two limiting groups, testing of data homogeneity, establishment of analytical functions, recognition of outliers, evaluation of limiting values and extraction of prevalidation parameters were included. Moreover, system of diagnosis for particular prevalidation step was suggested. As an illustrative example for demonstration of feasibility of prevalidation methodology, among great number of analytical procedures, Vis-spectrophotometric procedure for determination of tannins with Folin-Ciocalteu's phenol reagent was selected. Favourable metrological characteristics of this analytical procedure, as prevalidation figures of merit, recognized the metrological procedure as a valuable concept in preliminary evaluation of quality of analytical procedures.

Tests examining skill outcomes in sport: a systematic review of measurement properties and feasibility.

PubMed

Robertson, Samuel J; Burnett, Angus F; Cochrane, Jodie

2014-04-01

A high level of participant skill is influential in determining the outcome of many sports. Thus, tests assessing skill outcomes in sport are commonly used by coaches and researchers to estimate an athlete's ability level, to evaluate the effectiveness of interventions or for the purpose of talent identification. The objective of this systematic review was to examine the methodological quality, measurement properties and feasibility characteristics of sporting skill outcome tests reported in the peer-reviewed literature. A search of both SPORTDiscus and MEDLINE databases was undertaken. Studies that examined tests of sporting skill outcomes were reviewed. Only studies that investigated measurement properties of the test (reliability or validity) were included. A total of 22 studies met the inclusion/exclusion criteria. A customised checklist of assessment criteria, based on previous research, was utilised for the purpose of this review. A range of sports were the subject of the 22 studies included in this review, with considerations relating to methodological quality being generally well addressed by authors. A range of methods and statistical procedures were used by researchers to determine the measurement properties of their skill outcome tests. The majority (95%) of the reviewed studies investigated test-retest reliability, and where relevant, inter and intra-rater reliability was also determined. Content validity was examined in 68% of the studies, with most tests investigating multiple skill domains relevant to the sport. Only 18% of studies assessed all three reviewed forms of validity (content, construct and criterion), with just 14% investigating the predictive validity of the test. Test responsiveness was reported in only 9% of studies, whilst feasibility received varying levels of attention. In organised sport, further tests may exist which have not been investigated in this review. This could be due to such tests firstly not being published in the peer-review literature and secondly, not having their measurement properties (i.e., reliability or validity) examined formally. Of the 22 studies included in this review, items relating to test methodological quality were, on the whole, well addressed. Test-retest reliability was determined in all but one of the reviewed studies, whilst most studies investigated at least two aspects of validity (i.e., content, construct or criterion-related validity). Few studies examined predictive validity or responsiveness. While feasibility was addressed in over half of the studies, practicality and test limitations were rarely addressed. Consideration of study quality, measurement properties and feasibility components assessed in this review can assist future researchers when developing or modifying tests of sporting skill outcomes.

A comprehensive evaluation of tyrosol and hydroxytyrosol derivatives in extra virgin olive oil by microwave-assisted hydrolysis and HPLC-MS/MS.

PubMed

Bartella, Lucia; Mazzotti, Fabio; Napoli, Anna; Sindona, Giovanni; Di Donna, Leonardo

2018-03-01

A rapid and reliable method to assay the total amount of tyrosol and hydroxytyrosol derivatives in extra virgin olive oil has been developed. The methodology intends to establish the nutritional quality of this edible oil addressing recent international health claim legislations (the European Commission Regulation No. 432/2012) and changing the classification of extra virgin olive oil to the status of nutraceutical. The method is based on the use of high-performance liquid chromatography coupled with tandem mass spectrometry and labeled internal standards preceded by a fast hydrolysis reaction step performed through the aid of microwaves under acid conditions. The overall process is particularly time saving, much shorter than any methodology previously reported. The developed approach represents a mix of rapidity and accuracy whose values have been found near 100% on different fortified vegetable oils, while the RSD% values, calculated from repeatability and reproducibility experiments, are in all cases under 7%. Graphical abstract Schematic of the methodology applied to the determination of tyrosol and hydroxytyrosol ester conjugates.

The Ocean Colour Climate Change Initiative: I. A Methodology for Assessing Atmospheric Correction Processors Based on In-Situ Measurements

NASA Technical Reports Server (NTRS)

Muller, Dagmar; Krasemann, Hajo; Brewin, Robert J. W.; Deschamps, Pierre-Yves; Doerffer, Roland; Fomferra, Norman; Franz, Bryan A.; Grant, Mike G.; Groom, Steve B.; Melin, Frederic;

Cognitive Task Analysis of Business Jet Pilots' Weather Flying Behaviors: Preliminary Results

NASA Technical Reports Server (NTRS)

Latorella, Kara; Pliske, Rebecca; Hutton, Robert; Chrenka, Jason

2001-01-01

This report presents preliminary findings from a cognitive task analysis (CTA) of business aviation piloting. Results describe challenging weather-related aviation decisions and the information and cues used to support these decisions. Further, these results demonstrate the role of expertise in business aviation decision-making in weather flying, and how weather information is acquired and assessed for reliability. The challenging weather scenarios and novice errors identified in the results provide the basis for experimental scenarios and dependent measures to be used in future flight simulation evaluations of candidate aviation weather information systems. Finally, we analyzed these preliminary results to recommend design and training interventions to improve business aviation decision-making with weather information. The primary objective of this report is to present these preliminary findings and to document the extended CTA methodology used to elicit and represent expert business aviator decision-making with weather information. These preliminary findings will be augmented with results from additional subjects using this methodology. A summary of the complete results, absent the detailed treatment of methodology provided in this report, will be documented in a separate publication.

Evaluation in medical education: A topical review of target parameters, data collection tools and confounding factors.

PubMed

Schiekirka, Sarah; Feufel, Markus A; Herrmann-Lingen, Christoph; Raupach, Tobias

2015-01-01

Evaluation is an integral part of education in German medical schools. According to the quality standards set by the German Society for Evaluation, evaluation tools must provide an accurate and fair appraisal of teaching quality. Thus, data collection tools must be highly reliable and valid. This review summarises the current literature on evaluation of medical education with regard to the possible dimensions of teaching quality, the psychometric properties of survey instruments and potential confounding factors. We searched Pubmed, PsycINFO and PSYNDEX for literature on evaluation in medical education and included studies published up until June 30, 2011 as well as articles identified in the "grey literature". RESULTS are presented as a narrative review. We identified four dimensions of teaching quality: structure, process, teacher characteristics, and outcome. Student ratings are predominantly used to address the first three dimensions, and a number of reliable tools are available for this purpose. However, potential confounders of student ratings pose a threat to the validity of these instruments. Outcome is usually operationalised in terms of student performance on examinations, but methodological problems may limit the usability of these data for evaluation purposes. In addition, not all examinations at German medical schools meet current quality standards. The choice of tools for evaluating medical education should be guided by the dimension that is targeted by the evaluation. Likewise, evaluation results can only be interpreted within the context of the construct addressed by the data collection tool that was used as well as its specific confounding factors.

Storage reliability analysis summary report. Volume 2: Electro mechanical devices

NASA Astrophysics Data System (ADS)

Smith, H. B., Jr.; Krulac, I. L.

1982-09-01

This document summarizes storage reliability data collected by the US Army Missile Command on electro-mechanical devices over a period of several years. Sources of data are detailed, major failure modes and mechanisms are listed and discussed. Non-operational failure rate prediction methodology is given, and conclusions and recommendations for enhancing the storage reliability of devices are drawn from the analysis of collected data.

Design of an integrated airframe/propulsion control system architecture

NASA Technical Reports Server (NTRS)

Cohen, Gerald C.; Lee, C. William; Strickland, Michael J.

1990-01-01

The design of an integrated airframe/propulsion control system architecture is described. The design is based on a prevalidation methodology that used both reliability and performance tools. An account is given of the motivation for the final design and problems associated with both reliability and performance modeling. The appendices contain a listing of the code for both the reliability and performance model used in the design.

Effects of Assuming Independent Component Failure Times, If They Are Actually Dependent, in a Series System.

DTIC Science & Technology

1985-11-26

etc.).., Major decisions involving reliability ptudies, based on competing risk methodology , have been made in the past and will continue to be made...censoring mechanism. In such instances, the methodology for estimating relevant reliabili- ty probabilities has received considerable attention (cf. David...proposal for a discussion of the general methodology . .,4..% . - ’ -. - ’ . ’ , . * I - " . . - - - - . . ,_ . . . . . . . . .4

A Radial Basis Function Approach to Financial Time Series Analysis

DTIC Science & Technology

1993-12-01

including efficient methods for parameter estimation and pruning, a pointwise prediction error estimator, and a methodology for controlling the "data...collection of practical techniques to address these issues for a modeling methodology . Radial Basis Function networks. These techniques in- clude efficient... methodology often then amounts to a careful consideration of the interplay between model complexity and reliability. These will be recurrent themes

It is Time the United States Air Force Changes the way it Feeds its Airmen

DTIC Science & Technology

2008-03-01

narrative , phenomenology , ethnography , case study , and grounded theory . In purpose, these strategies are...methodology) the research will be analyzed. Methodology A qualitative research methodology and specifically a case study strategy for the...well as theory building in chapter five . Finally, in regards to reliability, Yin’s (2003) case study protocol guidance was used as a means to

Evaluation of errors in quantitative determination of asbestos in rock

NASA Astrophysics Data System (ADS)

Baietto, Oliviero; Marini, Paola; Vitaliti, Martina

2016-04-01

The quantitative determination of the content of asbestos in rock matrices is a complex operation which is susceptible to important errors. The principal methodologies for the analysis are Scanning Electron Microscopy (SEM) and Phase Contrast Optical Microscopy (PCOM). Despite the PCOM resolution is inferior to that of SEM, PCOM analysis has several advantages, including more representativity of the analyzed sample, more effective recognition of chrysotile and a lower cost. The DIATI LAA internal methodology for the analysis in PCOM is based on a mild grinding of a rock sample, its subdivision in 5-6 grain size classes smaller than 2 mm and a subsequent microscopic analysis of a portion of each class. The PCOM is based on the optical properties of asbestos and of the liquids with note refractive index in which the particles in analysis are immersed. The error evaluation in the analysis of rock samples, contrary to the analysis of airborne filters, cannot be based on a statistical distribution. In fact for airborne filters a binomial distribution (Poisson), which theoretically defines the variation in the count of fibers resulting from the observation of analysis fields, chosen randomly on the filter, can be applied. The analysis in rock matrices instead cannot lean on any statistical distribution because the most important object of the analysis is the size of the of asbestiform fibers and bundles of fibers observed and the resulting relationship between the weights of the fibrous component compared to the one granular. The error evaluation generally provided by public and private institutions varies between 50 and 150 percent, but there are not, however, specific studies that discuss the origin of the error or that link it to the asbestos content. Our work aims to provide a reliable estimation of the error in relation to the applied methodologies and to the total content of asbestos, especially for the values close to the legal limits. The error assessments must be made through the repetition of the same analysis on the same sample to try to estimate the error on the representativeness of the sample and the error related to the sensitivity of the operator, in order to provide a sufficiently reliable uncertainty of the method. We used about 30 natural rock samples with different asbestos content, performing 3 analysis on each sample to obtain a trend sufficiently representative of the percentage. Furthermore we made on one chosen sample 10 repetition of the analysis to try to define more specifically the error of the methodology.

34 CFR 462.11 - What must an application contain?

Code of Federal Regulations, 2010 CFR

2010-07-01

... the methodology and procedures used to measure the reliability of the test. (h) Construct validity... previous test, and results from validity, reliability, and equating or standard-setting studies undertaken... NRS educational functioning levels (content validity). Documentation of the extent to which the items...

75 FR 5779 - Proposed Emergency Agency Information Collection

Federal Register 2010, 2011, 2012, 2013, 2014

2010-02-04

... proposed collection of information, including the validity of the methodology and assumptions used; (c... Collection Request Title: Electricity Delivery and Energy Reliability Recovery Act Smart Grid Grant Program..., Chief Operating Officer, Electricity Delivery and Energy Reliability. [FR Doc. 2010-2422 Filed 2-3-10; 8...

Meta-Analysis of Coefficient Alpha

ERIC Educational Resources Information Center

Rodriguez, Michael C.; Maeda, Yukiko

2006-01-01

The meta-analysis of coefficient alpha across many studies is becoming more common in psychology by a methodology labeled reliability generalization. Existing reliability generalization studies have not used the sampling distribution of coefficient alpha for precision weighting and other common meta-analytic procedures. A framework is provided for…

Probabilistic structural mechanics research for parallel processing computers

NASA Technical Reports Server (NTRS)

Sues, Robert H.; Chen, Heh-Chyun; Twisdale, Lawrence A.; Martin, William R.

1991-01-01

Aerospace structures and spacecraft are a complex assemblage of structural components that are subjected to a variety of complex, cyclic, and transient loading conditions. Significant modeling uncertainties are present in these structures, in addition to the inherent randomness of material properties and loads. To properly account for these uncertainties in evaluating and assessing the reliability of these components and structures, probabilistic structural mechanics (PSM) procedures must be used. Much research has focused on basic theory development and the development of approximate analytic solution methods in random vibrations and structural reliability. Practical application of PSM methods was hampered by their computationally intense nature. Solution of PSM problems requires repeated analyses of structures that are often large, and exhibit nonlinear and/or dynamic response behavior. These methods are all inherently parallel and ideally suited to implementation on parallel processing computers. New hardware architectures and innovative control software and solution methodologies are needed to make solution of large scale PSM problems practical.

Commercialization of NESSUS: Status

NASA Technical Reports Server (NTRS)

Thacker, Ben H.; Millwater, Harry R.

1991-01-01

A plan was initiated in 1988 to commercialize the Numerical Evaluation of Stochastic Structures Under Stress (NESSUS) probabilistic structural analysis software. The goal of the on-going commercialization effort is to begin the transfer of Probabilistic Structural Analysis Method (PSAM) developed technology into industry and to develop additional funding resources in the general area of structural reliability. The commercialization effort is summarized. The SwRI NESSUS Software System is a general purpose probabilistic finite element computer program using state of the art methods for predicting stochastic structural response due to random loads, material properties, part geometry, and boundary conditions. NESSUS can be used to assess structural reliability, to compute probability of failure, to rank the input random variables by importance, and to provide a more cost effective design than traditional methods. The goal is to develop a general probabilistic structural analysis methodology to assist in the certification of critical components in the next generation Space Shuttle Main Engine.

Item response theory analysis of the Lichtenberg Financial Decision Screening Scale.

PubMed

Teresi, Jeanne A; Ocepek-Welikson, Katja; Lichtenberg, Peter A

2017-01-01

The focus of these analyses was to examine the psychometric properties of the Lichtenberg Financial Decision Screening Scale (LFDSS). The purpose of the screen was to evaluate the decisional abilities and vulnerability to exploitation of older adults. Adults aged 60 and over were interviewed by social, legal, financial, or health services professionals who underwent in-person training on the administration and scoring of the scale. Professionals provided a rating of the decision-making abilities of the older adult. The analytic sample included 213 individuals with an average age of 76.9 (SD = 10.1). The majority (57%) were female. Data were analyzed using item response theory (IRT) methodology. The results supported the unidimensionality of the item set. Several IRT models were tested. Ten ordinal and binary items evidenced a slightly higher reliability estimate (0.85) than other versions and better coverage in terms of the range of reliable measurement across the continuum of financial incapacity.

Thought Disorder in Preschool Children with Attention Deficit/Hyperactivity Disorder (ADHD).

PubMed

Hutchison, Amanda K; Kelsay, Kimberly; Talmi, Ayelet; Noonan, Kate; Ross, Randal G

2016-08-01

Preschool identification of and intervention for psychiatric symptoms has the potential for lifelong benefits. However, preschool identification of thought disorder, a symptom associated with long term risk for social and cognitive dysfunction, has received little attention with previous work limited to examining preschoolers with severe emotional and behavioral dysregulation. Using story-stem methodology, 12 children with ADHD and 12 children without ADHD, ages 4.0-6.0 years were evaluated for thought disorder. Thought disorder was reliably assessed (Cronbach's alpha = .958). Children with ADHD were significantly more likely than children without ADHD to exhibit thought disorder (75 vs 25 %; Fischer's Exact Test = .0391). Thought disorder can be reliably assessed in preschool children and is present in preschool children with psychiatric illness including preschool children with ADHD. Thought disorder may be identifiable in preschool years across a broad range of psychiatric illnesses and thus may be an appropriate target of intervention.

Induction of lucid dreams: a systematic review of evidence.

PubMed

Stumbrys, Tadas; Erlacher, Daniel; Schädlich, Melanie; Schredl, Michael

2012-09-01

In lucid dreams the dreamer is aware of dreaming and often able to influence the ongoing dream content. Lucid dreaming is a learnable skill and a variety of techniques is suggested for lucid dreaming induction. This systematic review evaluated the evidence for the effectiveness of induction techniques. A comprehensive literature search was carried out in biomedical databases and specific resources. Thirty-five studies were included in the analysis (11 sleep laboratory and 24 field studies), of which 26 employed cognitive techniques, 11 external stimulation and one drug application. The methodological quality of the included studies was relatively low. None of the induction techniques were verified to induce lucid dreams reliably and consistently, although some of them look promising. On the basis of the reviewed studies, a taxonomy of lucid dream induction methods is presented. Several methodological issues are discussed and further directions for future studies are proposed. Copyright © 2012 Elsevier Inc. All rights reserved.

The cost of power outages in the business and public sectors in Israel: Revealed preference vs. subjective evaluation

DOE Office of Scientific and Technical Information (OSTI.GOV)

Beenstock, M.; Goldin, E.; Haitovsky, Y.

1997-05-01

The economic cost of power outages is a central parameter in the cost-benefit analysis of electric power reliability and the design of electric power systems. The authors present a new methodology for estimating the cost of power outages in the business and public sections and illustrate with data for Israel. The methodology is based on the principle of revealed preference, the cost of an outage may be inferred from the actions taken by consumers to mitigate losses induced by unsupplied electricity. If outages impose costs on businesses, managers are likely to invest in back-up power to mitigate the losses thatmore » are incurred when electricity is not supplied. Investment in back-up generators may then be used to impute the mitigated and unmitigated damage from outages. 12 refs., 3 figs., 7 tabs.« less

Ancient DNA studies: new perspectives on old samples

PubMed Central

2012-01-01

In spite of past controversies, the field of ancient DNA is now a reliable research area due to recent methodological improvements. A series of recent large-scale studies have revealed the true potential of ancient DNA samples to study the processes of evolution and to test models and assumptions commonly used to reconstruct patterns of evolution and to analyze population genetics and palaeoecological changes. Recent advances in DNA technologies, such as next-generation sequencing make it possible to recover DNA information from archaeological and paleontological remains allowing us to go back in time and study the genetic relationships between extinct organisms and their contemporary relatives. With the next-generation sequencing methodologies, DNA sequences can be retrieved even from samples (for example human remains) for which the technical pitfalls of classical methodologies required stringent criteria to guaranty the reliability of the results. In this paper, we review the methodologies applied to ancient DNA analysis and the perspectives that next-generation sequencing applications provide in this field. PMID:22697611

Reliability of different methodologies of infrared image analysis of myofascial trigger points in the upper trapezius muscle

PubMed Central

Dibai-Filho, Almir V.; Guirro, Elaine C. O.; Ferreira, Vânia T. K.; Brandino, Hugo E.; Vaz, Maíta M. O. L. L.; Guirro, Rinaldo R. J.

2015-01-01

BACKGROUND: Infrared thermography is recognized as a viable method for evaluation of subjects with myofascial pain. OBJECTIVE: The aim of the present study was to assess the intra- and inter-rater reliability of infrared image analysis of myofascial trigger points in the upper trapezius muscle. METHOD: A reliability study was conducted with 24 volunteers of both genders (23 females) between 18 and 30 years of age (22.12±2.54), all having cervical pain and presence of active myofascial trigger point in the upper trapezius muscle. Two trained examiners performed analysis of point, line, and area of the infrared images at two different periods with a 1-week interval. The intra-class correlation coefficient (ICC2,1) was used to assess the intra- and inter-rater reliability. RESULTS: With regard to the intra-rater reliability, ICC values were between 0.591 and 0.993, with temperatures between 0.13 and 1.57 °C for values of standard error of measurement (SEM) and between 0.36 and 4.35 °C for the minimal detectable change (MDC). For the inter-rater reliability, ICC ranged from 0.615 to 0.918, with temperatures between 0.43 and 1.22 °C for the SEM and between 1.19 and 3.38 °C for the MDC. CONCLUSION: The methods of infrared image analyses of myofascial trigger points in the upper trapezius muscle employed in the present study are suitable for clinical and research practices. PMID:25993626

Advanced approach to the analysis of a series of in-situ nuclear forward scattering experiments

NASA Astrophysics Data System (ADS)

Vrba, Vlastimil; Procházka, Vít; Smrčka, David; Miglierini, Marcel

2017-03-01

This study introduces a sequential fitting procedure as a specific approach to nuclear forward scattering (NFS) data evaluation. Principles and usage of this advanced evaluation method are described in details and its utilization is demonstrated on NFS in-situ investigations of fast processes. Such experiments frequently consist of hundreds of time spectra which need to be evaluated. The introduced procedure allows the analysis of these experiments and significantly decreases the time needed for the data evaluation. The key contributions of the study are the sequential use of the output fitting parameters of a previous data set as the input parameters for the next data set and the model suitability crosscheck option of applying the procedure in ascending and descending directions of the data sets. Described fitting methodology is beneficial for checking of model validity and reliability of obtained results.

Aerospace reliability applied to biomedicine.

NASA Technical Reports Server (NTRS)

Lalli, V. R.; Vargo, D. J.

1972-01-01

An analysis is presented that indicates that the reliability and quality assurance methodology selected by NASA to minimize failures in aerospace equipment can be applied directly to biomedical devices to improve hospital equipment reliability. The Space Electric Rocket Test project is used as an example of NASA application of reliability and quality assurance (R&QA) methods. By analogy a comparison is made to show how these same methods can be used in the development of transducers, instrumentation, and complex systems for use in medicine.

Prioritization methodology for chemical replacement

NASA Technical Reports Server (NTRS)

Goldberg, Ben; Cruit, Wendy; Schutzenhofer, Scott

1995-01-01

This methodology serves to define a system for effective prioritization of efforts required to develop replacement technologies mandated by imposed and forecast legislation. The methodology used is a semi quantitative approach derived from quality function deployment techniques (QFD Matrix). QFD is a conceptual map that provides a method of transforming customer wants and needs into quantitative engineering terms. This methodology aims to weight the full environmental, cost, safety, reliability, and programmatic implications of replacement technology development to allow appropriate identification of viable candidates and programmatic alternatives.

'Emerging technologies for the changing global market' - Prioritization methodology for chemical replacement

NASA Technical Reports Server (NTRS)

Cruit, Wendy; Schutzenhofer, Scott; Goldberg, Ben; Everhart, Kurt

1993-01-01

This project served to define an appropriate methodology for effective prioritization of technology efforts required to develop replacement technologies mandated by imposed and forecast legislation. The methodology used is a semiquantitative approach derived from quality function deployment techniques (QFD Matrix). This methodology aims to weight the full environmental, cost, safety, reliability, and programmatic implications of replacement technology development to allow appropriate identification of viable candidates and programmatic alternatives. The results will be implemented as a guideline for consideration for current NASA propulsion systems.

Identification and evaluation of software measures

NASA Technical Reports Server (NTRS)

Card, D. N.

1981-01-01

A large scale, systematic procedure for identifying and evaluating measures that meaningfully characterize one or more elements of software development is described. The background of this research, the nature of the data involved, and the steps of the analytic procedure are discussed. An example of the application of this procedure to data from real software development projects is presented. As the term is used here, a measure is a count or numerical rating of the occurrence of some property. Examples of measures include lines of code, number of computer runs, person hours expended, and degree of use of top down design methodology. Measures appeal to the researcher and the manager as a potential means of defining, explaining, and predicting software development qualities, especially productivity and reliability.

Fuzzy risk analysis of a modern γ-ray industrial irradiator.

PubMed

Castiglia, F; Giardina, M

2011-06-01

Fuzzy fault tree analyses were used to investigate accident scenarios that involve radiological exposure to operators working in industrial γ-ray irradiation facilities. The HEART method, a first generation human reliability analysis method, was used to evaluate the probability of adverse human error in these analyses. This technique was modified on the basis of fuzzy set theory to more directly take into account the uncertainties in the error-promoting factors on which the methodology is based. Moreover, with regard to some identified accident scenarios, fuzzy radiological exposure risk, expressed in terms of potential annual death, was evaluated. The calculated fuzzy risks for the examined plant were determined to be well below the reference risk suggested by International Commission on Radiological Protection.

A Bayesian-Based Novel Methodology to Generate Reliable Site Response Mapping Sensitive to Data Uncertainties

NASA Astrophysics Data System (ADS)

Chakraborty, A.; Goto, H.

2017-12-01

The 2011 off the Pacific coast of Tohoku earthquake caused severe damage in many areas further inside the mainland because of site-amplification. Furukawa district in Miyagi Prefecture, Japan recorded significant spatial differences in ground motion even at sub-kilometer scales. The site responses in the damage zone far exceeded the levels in the hazard maps. A reason why the mismatch occurred is that mapping follow only the mean value at the measurement locations with no regard to the data uncertainties and thus are not always reliable. Our research objective is to develop a methodology to incorporate data uncertainties in mapping and propose a reliable map. The methodology is based on a hierarchical Bayesian modeling of normally-distributed site responses in space where the mean (μ), site-specific variance (σ2) and between-sites variance(s2) parameters are treated as unknowns with a prior distribution. The observation data is artificially created site responses with varying means and variances for 150 seismic events across 50 locations in one-dimensional space. Spatially auto-correlated random effects were added to the mean (μ) using a conditionally autoregressive (CAR) prior. The inferences on the unknown parameters are done using Markov Chain Monte Carlo methods from the posterior distribution. The goal is to find reliable estimates of μ sensitive to uncertainties. During initial trials, we observed that the tau (=1/s2) parameter of CAR prior controls the μ estimation. Using a constraint, s = 1/(k×σ), five spatial models with varying k-values were created. We define reliability to be measured by the model likelihood and propose the maximum likelihood model to be highly reliable. The model with maximum likelihood was selected using a 5-fold cross-validation technique. The results show that the maximum likelihood model (μ*) follows the site-specific mean at low uncertainties and converges to the model-mean at higher uncertainties (Fig.1). This result is highly significant as it successfully incorporates the effect of data uncertainties in mapping. This novel approach can be applied to any research field using mapping techniques. The methodology is now being applied to real records from a very dense seismic network in Furukawa district, Miyagi Prefecture, Japan to generate a reliable map of the site responses.

Reliability and validity of the Turkish version of the situational self-efficacy scale for fruit and vegetable consumption in adolescents.

PubMed

Kadioglu, Hasibe; Erol, Saime; Ergun, Ayse

2015-01-01

The purpose of this research was to examine the psychometric properties of the Turkish version of the situational self-efficacy scale for vegetable and fruit consumption in adolescents. This was a methodological study. The study was conducted in four public secondary schools in Istanbul, Turkey. Subjects were 1586 adolescents. Content and construct validity were assessed to test the validity of the scale. The reliability was assessed in terms of internal consistency and test-retest reliability. For confirmatory factor analysis, χ(2) statistics plus other fit indices were used, including the goodness-of-fit index, the adjusted goodness-of-fit index, the nonnormed fit index, the comparative fit index, the standardized root mean residual, and the root mean square error of approximation. Pearson's correlation was used for test-retest reliability and item total correlation. The internal consistency was assessed by using Cronbach α. Confirmatory factor analysis strongly supported the three-component structure representing positive social situations (α = .81), negative effect situations (α = .93), and difficult situations (α = .78). Psychometric analyses of the Turkish version of the situational self-efficacy scale indicate high reliability and good content and construct validity. Researchers and health professionals will find it useful to employ the Turkish situational self-efficacy scale in evaluating situational self-efficacy for fruit and vegetable consumption in Turkish adolescents.

Determining minimum staffing levels during snowstorms using an integrated simulation, regression, and reliability model.

PubMed

Kunkel, Amber; McLay, Laura A

2013-03-01

Emergency medical services (EMS) provide life-saving care and hospital transport to patients with severe trauma or medical conditions. Severe weather events, such as snow events, may lead to adverse patient outcomes by increasing call volumes and service times. Adequate staffing levels during such weather events are critical for ensuring that patients receive timely care. To determine staffing levels that depend on weather, we propose a model that uses a discrete event simulation of a reliability model to identify minimum staffing levels that provide timely patient care, with regression used to provide the input parameters. The system is said to be reliable if there is a high degree of confidence that ambulances can immediately respond to a given proportion of patients (e.g., 99 %). Four weather scenarios capture varying levels of snow falling and snow on the ground. An innovative feature of our approach is that we evaluate the mitigating effects of different extrinsic response policies and intrinsic system adaptation. The models use data from Hanover County, Virginia to quantify how snow reduces EMS system reliability and necessitates increasing staffing levels. The model and its analysis can assist in EMS preparedness by providing a methodology to adjust staffing levels during weather events. A key observation is that when it is snowing, intrinsic system adaptation has similar effects on system reliability as one additional ambulance.

How to improve the standardization and the diagnostic performance of the fecal egg count reduction test?

PubMed

Levecke, Bruno; Kaplan, Ray M; Thamsborg, Stig M; Torgerson, Paul R; Vercruysse, Jozef; Dobson, Robert J

2018-04-15

Although various studies have provided novel insights into how to best design, analyze and interpret a fecal egg count reduction test (FECRT), it is still not straightforward to provide guidance that allows improving both the standardization and the analytical performance of the FECRT across a variety of both animal and nematode species. For example, it has been suggested to recommend a minimum number of eggs to be counted under the microscope (not eggs per gram of feces), but we lack the evidence to recommend any number of eggs that would allow a reliable assessment of drug efficacy. Other aspects that need further research are the methodology of calculating uncertainty intervals (UIs; confidence intervals in case of frequentist methods and credible intervals in case of Bayesian methods) and the criteria of classifying drug efficacy into 'normal', 'suspected' and 'reduced'. The aim of this study is to provide complementary insights into the current knowledge, and to ultimately provide guidance in the development of new standardized guidelines for the FECRT. First, data were generated using a simulation in which the 'true' drug efficacy (TDE) was evaluated by the FECRT under varying scenarios of sample size, analytic sensitivity of the diagnostic technique, and level of both intensity and aggregation of egg excretion. Second, the obtained data were analyzed with the aim (i) to verify which classification criteria allow for reliable detection of reduced drug efficacy, (ii) to identify the UI methodology that yields the most reliable assessment of drug efficacy (coverage of TDE) and detection of reduced drug efficacy, and (iii) to determine the required sample size and number of eggs counted under the microscope that optimizes the detection of reduced efficacy. Our results confirm that the currently recommended criteria for classifying drug efficacy are the most appropriate. Additionally, the UI methodologies we tested varied in coverage and ability to detect reduced drug efficacy, thus a combination of UI methodologies is recommended to assess the uncertainty across all scenarios of drug efficacy estimates. Finally, based on our model estimates we were able to determine the required number of eggs to count for each sample size, enabling investigators to optimize the probability of correctly classifying a theoretical TDE while minimizing both financial and technical resources. Copyright © 2018 Elsevier B.V. All rights reserved.

Reporting and methodological quality of survival analysis in articles published in Chinese oncology journals.

PubMed

Zhu, Xiaoyan; Zhou, Xiaobin; Zhang, Yuan; Sun, Xiao; Liu, Haihua; Zhang, Yingying

2017-12-01

Survival analysis methods have gained widespread use in the filed of oncology. For achievement of reliable results, the methodological process and report quality is crucial. This review provides the first examination of methodological characteristics and reporting quality of survival analysis in articles published in leading Chinese oncology journals.To examine methodological and reporting quality of survival analysis, to identify some common deficiencies, to desirable precautions in the analysis, and relate advice for authors, readers, and editors.A total of 242 survival analysis articles were included to be evaluated from 1492 articles published in 4 leading Chinese oncology journals in 2013. Articles were evaluated according to 16 established items for proper use and reporting of survival analysis.The application rates of Kaplan-Meier, life table, log-rank test, Breslow test, and Cox proportional hazards model (Cox model) were 91.74%, 3.72%, 78.51%, 0.41%, and 46.28%, respectively, no article used the parametric method for survival analysis. Multivariate Cox model was conducted in 112 articles (46.28%). Follow-up rates were mentioned in 155 articles (64.05%), of which 4 articles were under 80% and the lowest was 75.25%, 55 articles were100%. The report rates of all types of survival endpoint were lower than 10%. Eleven of 100 articles which reported a loss to follow-up had stated how to treat it in the analysis. One hundred thirty articles (53.72%) did not perform multivariate analysis. One hundred thirty-nine articles (57.44%) did not define the survival time. Violations and omissions of methodological guidelines included no mention of pertinent checks for proportional hazard assumption; no report of testing for interactions and collinearity between independent variables; no report of calculation method of sample size. Thirty-six articles (32.74%) reported the methods of independent variable selection. The above defects could make potentially inaccurate, misleading of the reported results, or difficult to interpret.There are gaps in the conduct and reporting of survival analysis in studies published in Chinese oncology journals, severe deficiencies were noted. More endorsement by journals of the report guideline for survival analysis may improve articles quality, and the dissemination of reliable evidence to oncology clinicians. We recommend authors, readers, reviewers, and editors to consider survival analysis more carefully and cooperate more closely with statisticians and epidemiologists. Copyright © 2017 The Authors. Published by Wolters Kluwer Health, Inc. All rights reserved.

Pulse compression favourable aperiodic infrared imaging approach for non-destructive testing and evaluation of bio-materials

NASA Astrophysics Data System (ADS)

Mulaveesala, Ravibabu; Dua, Geetika; Arora, Vanita; Siddiqui, Juned A.; Muniyappa, Amarnath

2017-05-01

In recent years, aperiodic, transient pulse compression favourable infrared imaging methodologies demonstrated as reliable, quantitative, remote characterization and evaluation techniques for testing and evaluation of various biomaterials. This present work demonstrates a pulse compression favourable aperiodic thermal wave imaging technique, frequency modulated thermal wave imaging technique for bone diagnostics, especially by considering the bone with tissue, skin and muscle over layers. In order to find the capabilities of the proposed frequency modulated thermal wave imaging technique to detect the density variations in a multi layered skin-fat-muscle-bone structure, finite element modeling and simulation studies have been carried out. Further, frequency and time domain post processing approaches have been adopted on the temporal temperature data in order to improve the detection capabilities of frequency modulated thermal wave imaging.

Environmental Profile of a Community’s Health (EPOCH): An Ecometric Assessment of Measures of the Community Environment Based on Individual Perception

PubMed Central

Corsi, Daniel J.; Subramanian, S. V.; McKee, Martin; Li, Wei; Swaminathan, Sumathi; Lopez-Jaramillo, Patricio; Avezum, Alvaro; Lear, Scott A.; Dagenais, Gilles; Rangarajan, Sumathy; Teo, Koon; Yusuf, Salim; Chow, Clara K.

2012-01-01

Background Public health research has turned towards examining upstream, community-level determinants of cardiovascular disease risk factors. Objective measures of the environment, such as those derived from direct observation, and perception-based measures by residents have both been associated with health behaviours. However, current methods are generally limited to objective measures, often derived from administrative data, and few instruments have been evaluated for use in rural areas or in low-income countries. We evaluate the reliability of a quantitative tool designed to capture perceptions of community tobacco, nutrition, and social environments obtained from interviews with residents in communities in 5 countries. Methodology/ Principal Findings Thirteen measures of the community environment were developed from responses to questionnaire items from 2,360 individuals residing in 84 urban and rural communities in 5 countries (China, India, Brazil, Colombia, and Canada) in the Environmental Profile of a Community’s Health (EPOCH) study. Reliability and other properties of the community-level measures were assessed using multilevel models. High reliability (>0.80) was demonstrated for all community-level measures at the mean number of survey respondents per community (n = 28 respondents). Questionnaire items included in each scale were found to represent a common latent factor at the community level in multilevel factor analysis models. Conclusions/ Significance Reliable measures which represent aspects of communities potentially related to cardiovascular disease (CVD)/risk factors can be obtained using feasible sample sizes. The EPOCH instrument is suitable for use in different settings to explore upstream determinants of CVD/risk factors. PMID:22973446

A systematic review of psychometric testing of instruments that measure intention to work with older people.

PubMed

Che, Chong Chin; Hairi, Noran Naqiah; Chong, Mei Chan

2017-09-01

To review systematically the psychometric properties of instruments used to measure intention to work with older people. Nursing students are part of the future healthcare workforce; thus, being aware of their intention to work with older people would give valuable insights to nursing education and practice. Despite a plethora of research on measuring intention to work with older people, a valid and reliable instrument has not been identified. A systematic literature review of evidence and psychometric properties. Eight database searches were conducted between 2006 - 2016. English articles were selected based on inclusion and exclusion criteria. The COSMIN checklist was used to assess instruments reporting a psychometric evaluation of validity and reliability. Of 41 studies identified for full text review, 36 met the inclusion criteria. Seven different types of instruments were identified for psychometric evaluation. Measures of reliability were reported in eight papers and validity in five papers. Evidence for each measurement property was limited, with each instrument demonstrating a lack of information on measurement properties. Based on the COSMIN checklist, the overall quality of the psychometric properties was rated as poor to good. No single instrument was found to be optimal for use. Studies of high methodological quality are needed to properly assess the measurement properties of the instruments that are currently available. Until such studies are available, we recommend using existing instruments with caution. © 2017 John Wiley & Sons Ltd.

A Cross-Correlational Analysis between Electroencephalographic and End-Tidal Carbon Dioxide Signals: Methodological Issues in the Presence of Missing Data and Real Data Results

PubMed Central

Morelli, Maria Sole; Giannoni, Alberto; Passino, Claudio; Landini, Luigi; Emdin, Michele; Vanello, Nicola

2016-01-01

Electroencephalographic (EEG) irreducible artifacts are common and the removal of corrupted segments from the analysis may be required. The present study aims at exploring the effects of different EEG Missing Data Segment (MDS) distributions on cross-correlation analysis, involving EEG and physiological signals. The reliability of cross-correlation analysis both at single subject and at group level as a function of missing data statistics was evaluated using dedicated simulations. Moreover, a Bayesian-based approach for combining the single subject results at group level by considering each subject’s reliability was introduced. Starting from the above considerations, the cross-correlation function between EEG Global Field Power (GFP) in delta band and end-tidal CO2 (PETCO2) during rest and voluntary breath-hold was evaluated in six healthy subjects. The analysis of simulated data results at single subject level revealed a worsening of precision and accuracy in the cross-correlation analysis in the presence of MDS. At the group level, a large improvement in the results’ reliability with respect to single subject analysis was observed. The proposed Bayesian approach showed a slight improvement with respect to simple average results. Real data results were discussed in light of the simulated data tests and of the current physiological findings. PMID:27809243

The importance of establishing reliability and validity of assessment instruments for mental health problems: An example from Somali children and adolescents living in three refugee camps in Ethiopia

PubMed Central

Hall, Brian J.; Puffer, Eve; Murray, Laura K.; Ismael, Abdulkadir; Bass, Judith K.; Sim, Amanda; Bolton, Paul A.

2014-01-01

Assessing mental health problems cross-culturally for children exposed to war and violence presents a number of unique challenges. One of the most important issues is the lack of validated symptom measures to assess these problems. The present study sought to evaluate the psychometric properties of two measures to assess mental health problems: the Achenbach Youth Self-Report and the Child Posttraumatic Stress Disorder Symptom Scale. We conducted a validity study in three refugee camps in Eastern Ethiopia in the outskirts of Jijiga, the capital of the Somali region. A total of 147 child and caregiver pairs were assessed, and scores obtained were submitted to rigorous psychometric evaluation. Excellent internal consistency reliability was obtained for symptom measures for children and their caregivers. Validation of study instruments based on local case definitions was obtained for the caregivers but not consistently for the children. Sensitivity and specificity of study measures were generally low, indicating that these scales would not perform adequately as screening instruments. Combined test-retest and inter-rater reliability was low for all scales. This study illustrates the need for validation and testing of existing measures cross-culturally. Methodological implications for future cross-cultural research studies in low- and middle-income countries are discussed. PMID:24955147

Correcting Fallacies in Validity, Reliability, and Classification

ERIC Educational Resources Information Center

Sijtsma, Klaas

2009-01-01

This article reviews three topics from test theory that continue to raise discussion and controversy and capture test theorists' and constructors' interest. The first topic concerns the discussion of the methodology of investigating and establishing construct validity; the second topic concerns reliability and its misuse, alternative definitions…

Measurement properties of rheumatoid arthritis-specific quality-of-life questionnaires: systematic review of the literature.

PubMed

Lee, Jiyeon; Kim, Soo Hyun; Moon, Seung Hei; Lee, Eun-Hyun

2014-12-01

This study conducted a systematic review of the methodological quality of the psychometric evaluation process and the quality of measurement properties of rheumatoid arthritis (RA)-specific health-related quality-of-life (HRQOL) questionnaires with the purpose of obtaining the best evidence to help in the selection of the most appropriate instrument for measuring HRQOL in RA patients. A systematic literature search was performed to identify RA-specific HRQOL questionnaires in databases. The methodological quality of the studies was assessed using the Consensus-based Standards for the Selection of Health Measurement Instruments checklist. The quality of the measurement properties was assessed using quality criteria. The evidence regarding the measurement properties was pooled using best-evidence synthesis, with considerations of the number and methodological quality of the studies, and the consistency of their findings in terms of the quality of the measurement properties. The search identified 37 studies describing 9 instruments. Best-evidence synthesis suggested that the Rheumatoid Arthritis Quality of Life (RAQoL) questionnaire had the strongest positive evidence, especially with respect to reliability, measurement error, and content validity, and moderate positive evidence with respect to hypothesis testing and responsiveness. The current evidence suggests that the best-validated instrument among the RA-specific HRQOL measures is the RAQoL questionnaire in terms of both methodological quality in the process of psychometric evaluation and the quality of the measurement properties. However, there is limited evidence regarding internal consistency and structural validity of the RAQoL. Further efforts are warranted to establish the psychometric quality of this questionnaire.

Development and psychometric testing of the Cancer Knowledge Scale for Elders.

PubMed

Su, Ching-Ching; Chen, Yuh-Min; Kuo, Bo-Jein

2009-03-01

To develop the Cancer Knowledge Scale for Elders and test its validity and reliability. The number of elders suffering from cancer is increasing. To facilitate cancer prevention behaviours among elders, they shall be educated about cancer-related knowledge. Prior to designing a programme that would respond to the special needs of elders, understanding the cancer-related knowledge within this population was necessary. However, extensive review of the literature revealed a lack of appropriate instruments for measuring cancer-related knowledge. A valid and reliable cancer knowledge scale for elders is necessary. A non-experimental methodological design was used to test the psychometric properties of the Cancer Knowledge Scale for Elders. Item analysis was first performed to screen out items that had low corrected item-total correlation coefficients. Construct validity was examined with a principle component method of exploratory factor analysis. Cancer-related health behaviour was used as the criterion variable to evaluate criterion-related validity. Internal consistency reliability was assessed by the KR-20. Stability was determined by two-week test-retest reliability. The factor analysis yielded a four-factor solution accounting for 49.5% of the variance. For criterion-related validity, cancer knowledge was positively correlated with cancer-related health behaviour (r = 0.78, p < 0.001). The KR-20 coefficients of each factor were 0.85, 0.76, 0.79 and 0.67 and 0.87 for the total scale. Test-retest reliability over a two-week period was 0.83 (p < 0.001). This study provides evidence for content validity, construct validity, criterion-related validity, internal consistency and stability of the Cancer Knowledge Scale for Elders. The results show that this scale is an easy-to-use instrument for elders and has adequate validity and reliability. The scale can be used as an assessment instrument when implementing cancer education programmes for elders. It can also be used to evaluate the effects of education programmes.

Reliability and validity of the Chinese version of the Pediatric Quality Of Life InventoryTM (PedsQLTM) 3.0 neuromuscular module in children with Duchenne muscular dystrophy.

PubMed

Hu, Jun; Jiang, Li; Hong, Siqi; Cheng, Li; Kong, Min; Ye, Yuanzhen

2013-03-15

The Pediatric Quality of Life Inventory(TM) (PedsQL(TM)) is a widely used instrument to measure pediatric health-related quality of life (HRQOL) in children aged 2 to 18 years. The current study aimed to evaluate the reliability and validity of the Chinese version of the PedsQL(TM) 3.0 Neuromuscular Module in children with Duchenne muscular dystrophy (DMD). The PedsQL(TM) 3.0 Neuromuscular Module was translated into Chinese following PedsQL(TM) Measurement Model Translation Methodology. The Chinese version scale was administered to 56 children with DMD and their parents, and the psychometric properties were evaluated. The missing value percentages for each item of the Chinese version scale ranged from 0.00% to 0.54%. Internal consistency reliability approached or exceeded the minimum reliability standard of α = 0.7 (child α = 0.81, parent α = 0.86). Test-retest reliability was satisfactory, with intraclass correlation coefficients (ICCs) of 0.66 for children and 0.88 for parents (P < 0.01). Correlation coefficients between iteims and their hypothesized subscales were higher than those with other subscales (P < 0.05). The subscale of "About My/My Child's Neuromuscular Disease" significantly related to mobility and stair climbing status (Child t = 2.21, Parent t = 2.83, P < 0.05). The inter-correlations among the Chinese version of the PedsQL(TM) 3.0 Neuromuscular Module and the PedsQL(TM) 4.0 Generic Core Scales had medium to large effect sizes (P < 0.05). The child self-report scores were in moderate agreement with the parent proxy-report scores (ICC = 0.51, P < 0.05). The Chinese version of the PedsQLTM 3.0 Neuromuscular Module has acceptable psychometric properties. It is a reliable measure of disease-specific HRQOL in Chinese children with DMD.

Validity and reliability of the French translation of the VISA-A questionnaire for Achilles tendinopathy.

PubMed

Kaux, Jean-François; Delvaux, François; Oppong-Kyei, Julian; Dardenne, Nadia; Beaudart, Charlotte; Buckinx, Fanny; Croisier, Jean-Louis; Forthomme, Bénédicte; Crielaard, Jean-Michel; Bruyère, Oliver

2016-12-01

The Victorian Institute of Sport Assessment - Achilles tendinopathy questionnaire (VISA-A) evaluates the clinical severity of Achilles tendinopathy. The aim of this study was to translate the VISA-A into French and to study the reliability and validity of this French version, the VISA-AF. The VISA-A was translated into French to produce the VISA-AF using a validated methodology in six steps. Thereafter, several psychometric properties of this French version such as test-retest reliability, internal consistency, construct validity and floor and ceiling effects were evaluated. Therefore, we recruited 116 subjects, distributed into 3 groups: pathological patients (n = 31), at-risk athletes (n = 63) and healthy people (n = 22). The final version of the VISA-AF was approved by an expert committee. On a scale ranging from 0 to 100, the average scores of the VISA-AF obtained were 59 (± 18) for the pathological group, 99 (± 1) for the healthy group and 94 (± 7) for the at-risk group. The VISA-AF shows excellent reliability, low correlations with the discriminant subscales of the SF-36 and moderate correlations with the convergent subscales of the SF-36. The French version of the VISA-A is equivalent to its original version and is a reliable and valid questionnaire for French-speaking patients with Achilles tendinopathy. Implication for Rehabilitation The VISA-AF questionnaire is a reliable translation of the original VISA-A, from English into French, which is one of the most widespread languages in the world. The VISA-AF questionnaire is now a valid instrument that can be used by clinicians and researchers to assess the severity of pain and disability of French-speaking subjects with Achilles tendinopathy. The VISA-AF is a questionnaire to assess the severity of Achilles tendinopathy symptoms but is not a diagnostic tool.

High throughput and miniaturised systems for biodegradability assessments.

PubMed

Cregut, Mickael; Jouanneau, Sulivan; Brillet, François; Durand, Marie-José; Sweetlove, Cyril; Chenèble, Jean-Charles; L'Haridon, Jacques; Thouand, Gérald

2014-01-01

The society demands safer products with a better ecological profile. Regulatory criteria have been developed to prevent risks for human health and the environment, for example, within the framework of the European regulation REACH (Regulation (EC) No 1907, 2006). This has driven industry to consider the development of high throughput screening methodologies for assessing chemical biodegradability. These new screening methodologies must be scalable for miniaturisation, reproducible and as reliable as existing procedures for enhanced biodegradability assessment. Here, we evaluate two alternative systems that can be scaled for high throughput screening and conveniently miniaturised to limit costs in comparison with traditional testing. These systems are based on two dyes as follows: an invasive fluorescent dyes that serves as a cellular activity marker (a resazurin-like dye reagent) and a noninvasive fluorescent oxygen optosensor dye (an optical sensor). The advantages and limitations of these platforms for biodegradability assessment are presented. Our results confirm the feasibility of these systems for evaluating and screening chemicals for ready biodegradability. The optosensor is a miniaturised version of a component already used in traditional ready biodegradability testing, whereas the resazurin dye offers an interesting new screening mechanism for chemical concentrations greater than 10 mg/l that are not amenable to traditional closed bottle tests. The use of these approaches allows generalisation of high throughput screening methodologies to meet the need of developing new compounds with a favourable ecological profile and also assessment for regulatory purpose.

Methodology for building confidence measures

NASA Astrophysics Data System (ADS)

Bramson, Aaron L.

2004-04-01

This paper presents a generalized methodology for propagating known or estimated levels of individual source document truth reliability to determine the confidence level of a combined output. Initial document certainty levels are augmented by (i) combining the reliability measures of multiply sources, (ii) incorporating the truth reinforcement of related elements, and (iii) incorporating the importance of the individual elements for determining the probability of truth for the whole. The result is a measure of confidence in system output based on the establishing of links among the truth values of inputs. This methodology was developed for application to a multi-component situation awareness tool under development at the Air Force Research Laboratory in Rome, New York. Determining how improvements in data quality and the variety of documents collected affect the probability of a correct situational detection helps optimize the performance of the tool overall.

Urban land use: Remote sensing of ground-basin permeability

NASA Technical Reports Server (NTRS)

Tinney, L. R.; Jensen, J. R.; Estes, J. E.

1975-01-01

A remote sensing analysis of the amount and type of permeable and impermeable surfaces overlying an urban recharge basin is discussed. An effective methodology for accurately generating this data as input to a safe yield study is detailed and compared to more conventional alternative approaches. The amount of area inventoried, approximately 10 sq. miles, should provide a reliable base against which automatic pattern recognition algorithms, currently under investigation for this task, can be evaluated. If successful, such approaches can significantly reduce the time and effort involved in obtaining permeability data, an important aspect of urban hydrology dynamics.

Advanced extravehicular protective systems for shuttle, space station, lunar base and Mars missions.

NASA Technical Reports Server (NTRS)

Heimlich, P. F.; Sutton, J. G.; Tepper, E. H.

1972-01-01

Advances in extravehicular life support system technology will directly influence future space mission reliability and maintainability considerations. To identify required new technology areas, an appraisal of advanced portable life support system and subsystem concepts was conducted. Emphasis was placed on thermal control and combined CO2 control/O2 supply subsystems for both primary and emergency systems. A description of study methodology, concept evaluation techniques, specification requirements, and selected subsystems and systems are presented. New technology recommendations encompassing thermal control, CO2 control and O2 supply subsystems are also contained herein.

EUVL back-insertion layout optimization

NASA Astrophysics Data System (ADS)

Civay, D.; Laffosse, E.; Chesneau, A.

2018-03-01

Extreme ultraviolet lithography (EUVL) is targeted for front-up insertion at advanced technology nodes but will be evaluated for back insertion at more mature nodes. EUVL can put two or more mask levels back on one mask, depending upon what level(s) in the process insertion occurs. In this paper, layout optimization methods are discussed that can be implemented when EUVL back insertion is implemented. The layout optimizations can be focused on improving yield, reliability or density, depending upon the design needs. The proposed methodology modifies the original two or more colored layers and generates an optimized single color EUVL layout design.

Ceramic Technology Project semiannual progress report, October 1992--March 1993

DOE Office of Scientific and Technical Information (OSTI.GOV)

Johnson, D.R.

1993-09-01

This project was developed to meet the ceramic technology requirements of the OTS`s automotive technology programs. Although progress has been made in developing reliable structural ceramics, further work is needed to reduce cost. The work described in this report is organized according to the following work breakdown structure project elements: Materials and processing (monolithics [Si nitride, carbide], ceramic composites, thermal and wear coatings, joining, cost effective ceramic machining), materials design methodology (contact interfaces, new concepts), data base and life prediction (structural qualification, time-dependent behavior, environmental effects, fracture mechanics, nondestructive evaluation development), and technology transfer.

Systems design study of the Pioneer Venus spacecraft. Volume 2. Preliminary program development plan

NASA Technical Reports Server (NTRS)

1973-01-01

The preliminary development plan for the Pioneer Venus program is presented. This preliminary plan treats only developmental aspects that would have a significant effect on program cost. These significant development areas were: master program schedule planning; test planning - both unit and system testing for probes/orbiter/ probe bus; ground support equipment; performance assurance; and science integration Various test planning options and test method techniques were evaluated in terms of achieving a low-cost program without degrading mission performance or system reliability. The approaches studied and the methodology of the selected approach are defined.

Evaluation of the Telecommunications Protocol Processing Subsystem Using Reconfigurable Interoperable Gate Array

NASA Technical Reports Server (NTRS)

Pang, Jackson; Liddicoat, Albert; Ralston, Jesse; Pingree, Paula

2006-01-01

The current implementation of the Telecommunications Protocol Processing Subsystem Using Reconfigurable Interoperable Gate Arrays (TRIGA) is equipped with CFDP protocol and CCSDS Telemetry and Telecommand framing schemes to replace the CPU intensive software counterpart implementation for reliable deep space communication. We present the hardware/software co-design methodology used to accomplish high data rate throughput. The hardware CFDP protocol stack implementation is then compared against the two recent flight implementations. The results from our experiments show that TRIGA offers more than 3 orders of magnitude throughput improvement with less than one-tenth of the power consumption.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Grabaskas, David; Brunett, Acacia J.; Passerini, Stefano

GE Hitachi Nuclear Energy (GEH) and Argonne National Laboratory (Argonne) participated in a two year collaboration to modernize and update the probabilistic risk assessment (PRA) for the PRISM sodium fast reactor. At a high level, the primary outcome of the project was the development of a next-generation PRA that is intended to enable risk-informed prioritization of safety- and reliability-focused research and development. A central Argonne task during this project was a reliability assessment of passive safety systems, which included the Reactor Vessel Auxiliary Cooling System (RVACS) and the inherent reactivity feedbacks of the metal fuel core. Both systems were examinedmore » utilizing a methodology derived from the Reliability Method for Passive Safety Functions (RMPS), with an emphasis on developing success criteria based on mechanistic system modeling while also maintaining consistency with the Fuel Damage Categories (FDCs) of the mechanistic source term assessment. This paper provides an overview of the reliability analyses of both systems, including highlights of the FMEAs, the construction of best-estimate models, uncertain parameter screening and propagation, and the quantification of system failure probability. In particular, special focus is given to the methodologies to perform the analysis of uncertainty propagation and the determination of the likelihood of violating FDC limits. Additionally, important lessons learned are also reviewed, such as optimal sampling methodologies for the discovery of low likelihood failure events and strategies for the combined treatment of aleatory and epistemic uncertainties.« less

Reliability analysis of repairable systems using Petri nets and vague Lambda-Tau methodology.

PubMed

Garg, Harish

2013-01-01

The main objective of the paper is to developed a methodology, named as vague Lambda-Tau, for reliability analysis of repairable systems. Petri net tool is applied to represent the asynchronous and concurrent processing of the system instead of fault tree analysis. To enhance the relevance of the reliability study, vague set theory is used for representing the failure rate and repair times instead of classical(crisp) or fuzzy set theory because vague sets are characterized by a truth membership function and false membership functions (non-membership functions) so that sum of both values is less than 1. The proposed methodology involves qualitative modeling using PN and quantitative analysis using Lambda-Tau method of solution with the basic events represented by intuitionistic fuzzy numbers of triangular membership functions. Sensitivity analysis has also been performed and the effects on system MTBF are addressed. The methodology improves the shortcomings of the existing probabilistic approaches and gives a better understanding of the system behavior through its graphical representation. The washing unit of a paper mill situated in a northern part of India, producing approximately 200 ton of paper per day, has been considered to demonstrate the proposed approach. The results may be helpful for the plant personnel for analyzing the systems' behavior and to improve their performance by adopting suitable maintenance strategies. Copyright © 2012 ISA. Published by Elsevier Ltd. All rights reserved.

The quality of instruments to assess the process of shared decision making: A systematic review.

PubMed

Gärtner, Fania R; Bomhof-Roordink, Hanna; Smith, Ian P; Scholl, Isabelle; Stiggelbout, Anne M; Pieterse, Arwen H

2018-01-01

To inventory instruments assessing the process of shared decision making and appraise their measurement quality, taking into account the methodological quality of their validation studies. In a systematic review we searched seven databases (PubMed, Embase, Emcare, Cochrane, PsycINFO, Web of Science, Academic Search Premier) for studies investigating instruments measuring the process of shared decision making. Per identified instrument, we assessed the level of evidence separately for 10 measurement properties following a three-step procedure: 1) appraisal of the methodological quality using the COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN) checklist, 2) appraisal of the psychometric quality of the measurement property using three possible quality scores, 3) best-evidence synthesis based on the number of studies, their methodological and psychometrical quality, and the direction and consistency of the results. The study protocol was registered at PROSPERO: CRD42015023397. We included 51 articles describing the development and/or evaluation of 40 shared decision-making process instruments: 16 patient questionnaires, 4 provider questionnaires, 18 coding schemes and 2 instruments measuring multiple perspectives. There is an overall lack of evidence for their measurement quality, either because validation is missing or methods are poor. The best-evidence synthesis indicated positive results for a major part of instruments for content validity (50%) and structural validity (53%) if these were evaluated, but negative results for a major part of instruments when inter-rater reliability (47%) and hypotheses testing (59%) were evaluated. Due to the lack of evidence on measurement quality, the choice for the most appropriate instrument can best be based on the instrument's content and characteristics such as the perspective that they assess. We recommend refinement and validation of existing instruments, and the use of COSMIN-guidelines to help guarantee high-quality evaluations.

Evaluation of the measurement properties of self-reported health-related work-functioning instruments among workers with common mental disorders.

PubMed

Abma, Femke I; van der Klink, Jac J L; Terwee, Caroline B; Amick, Benjamin C; Bültmann, Ute

2012-01-01

During the past decade, common mental disorders (CMD) have emerged as a major public and occupational health problem in many countries. Several instruments have been developed to measure the influence of health on functioning at work. To select appropriate instruments for use in occupational health practice and research, the measurement properties (eg, reliability, validity, responsiveness) must be evaluated. The objective of this study is to appraise critically and compare the measurement properties of self-reported health-related work-functioning instruments among workers with CMD. A systematic review was performed searching three electronic databases. Papers were included that: (i) mainly focused on the development and/or evaluation of the measurement properties of a self-reported health-related work-functioning instrument; (ii) were conducted in a CMD population; and (iii) were fulltext original papers. Quality appraisal was performed using the consensus-based standards for the selection of health status measurement instruments (COSMIN) checklist. Five papers evaluating measurement properties of five self-reported health-related work-functioning instruments in CMD populations were included. There is little evidence available for the measurement properties of the identified instruments in this population, mainly due to low methodological quality of the included studies. The available evidence on measurement properties is based on studies of poor-to-fair methodological quality. Information on a number of measurement properties, such as measurement error, content validity, and cross-cultural validity is still lacking. Therefore, no evidence-based decisions and recommendations can be made for the use of health-related work functioning instruments. Studies of high methodological quality are needed to properly assess the existing instruments' measurement properties.

Validity and reliability of Internet-based physiotherapy assessment for musculoskeletal disorders: a systematic review.

PubMed

Mani, Suresh; Sharma, Shobha; Omar, Baharudin; Paungmali, Aatit; Joseph, Leonard

2017-04-01

Purpose The purpose of this review is to systematically explore and summarise the validity and reliability of telerehabilitation (TR)-based physiotherapy assessment for musculoskeletal disorders. Method A comprehensive systematic literature review was conducted using a number of electronic databases: PubMed, EMBASE, PsycINFO, Cochrane Library and CINAHL, published between January 2000 and May 2015. The studies examined the validity, inter- and intra-rater reliabilities of TR-based physiotherapy assessment for musculoskeletal conditions were included. Two independent reviewers used the Quality Appraisal Tool for studies of diagnostic Reliability (QAREL) and the Quality Assessment of Diagnostic Accuracy Studies (QUADAS) tool to assess the methodological quality of reliability and validity studies respectively. Results A total of 898 hits were achieved, of which 11 articles based on inclusion criteria were reviewed. Nine studies explored the concurrent validity, inter- and intra-rater reliabilities, while two studies examined only the concurrent validity. Reviewed studies were moderate to good in methodological quality. The physiotherapy assessments such as pain, swelling, range of motion, muscle strength, balance, gait and functional assessment demonstrated good concurrent validity. However, the reported concurrent validity of lumbar spine posture, special orthopaedic tests, neurodynamic tests and scar assessments ranged from low to moderate. Conclusion TR-based physiotherapy assessment was technically feasible with overall good concurrent validity and excellent reliability, except for lumbar spine posture, orthopaedic special tests, neurodynamic testa and scar assessment.

A Methodological Approach to Quantifying Plyometric Intensity.

PubMed

Jarvis, Mark M; Graham-Smith, Phil; Comfort, Paul

2016-09-01

Jarvis, MM, Graham-Smith, P, and Comfort, P. A Methodological approach to quantifying plyometric intensity. J Strength Cond Res 30(9): 2522-2532, 2016-In contrast to other methods of training, the quantification of plyometric exercise intensity is poorly defined. The purpose of this study was to evaluate the suitability of a range of neuromuscular and mechanical variables to describe the intensity of plyometric exercises. Seven male recreationally active subjects performed a series of 7 plyometric exercises. Neuromuscular activity was measured using surface electromyography (SEMG) at vastus lateralis (VL) and biceps femoris (BF). Surface electromyography data were divided into concentric (CON) and eccentric (ECC) phases of movement. Mechanical output was measured by ground reaction forces and processed to provide peak impact ground reaction force (PF), peak eccentric power (PEP), and impulse (IMP). Statistical analysis was conducted to assess the reliability intraclass correlation coefficient and sensitivity smallest detectable difference of all variables. Mean values of SEMG demonstrate high reliability (r ≥ 0.82), excluding ECC VL during a 40-cm drop jump (r = 0.74). PF, PEP, and IMP demonstrated high reliability (r ≥ 0.85). Statistical power for force variables was excellent (power = 1.0), and good for SEMG (power ≥0.86) excluding CON BF (power = 0.57). There was no significant difference (p > 0.05) in CON SEMG between exercises. Eccentric phase SEMG only distinguished between exercises involving a landing and those that did not (percentage of maximal voluntary isometric contraction [%MVIC] = no landing -65 ± 5, landing -140 ± 8). Peak eccentric power, PF, and IMP all distinguished between exercises. In conclusion, CON neuromuscular activity does not appear to vary when intent is maximal, whereas ECC activity is dependent on the presence of a landing. Force characteristics provide a reliable and sensitive measure enabling precise description of intensity in plyometric exercises. The present findings provide coaches and scientists with an insightful and precise method of measuring intensity in plyometrics, which will allow for greater control of programming variables.

Assessing the effects of employee assistance programs: a review of employee assistance program evaluations.

PubMed

Colantonio, A

1989-01-01

Employee assistance programs have grown at a dramatic rate, yet the effectiveness of these programs has been called into question. The purpose of this paper was to assess the effectiveness of employee assistance programs (EAPs) by reviewing recently published EAP evaluations. All studies evaluating EAPs published since 1975 from peer-reviewed journals in the English language were included in this analysis. Each of the articles was assessed in the following areas: (a) program description (subjects, setting, type of intervention, format), (b) evaluation design (research design, variables measured, operational methods), and (c) program outcomes. Results indicate numerous methodological and conceptual weaknesses and issues. These weaknesses included lack of controlled research designs and short time lags between pre- and post-test measures. Other problems identified are missing information regarding subjects, type of intervention, how variables are measured (operational methods), and reliability and validity of evaluation instruments. Due to the aforementioned weaknesses, positive outcomes could not be supported. Recommendations are made for future EAP evaluations.

Assessing the effects of employee assistance programs: a review of employee assistance program evaluations.

PubMed Central

Colantonio, A.

1989-01-01

Employee assistance programs have grown at a dramatic rate, yet the effectiveness of these programs has been called into question. The purpose of this paper was to assess the effectiveness of employee assistance programs (EAPs) by reviewing recently published EAP evaluations. All studies evaluating EAPs published since 1975 from peer-reviewed journals in the English language were included in this analysis. Each of the articles was assessed in the following areas: (a) program description (subjects, setting, type of intervention, format), (b) evaluation design (research design, variables measured, operational methods), and (c) program outcomes. Results indicate numerous methodological and conceptual weaknesses and issues. These weaknesses included lack of controlled research designs and short time lags between pre- and post-test measures. Other problems identified are missing information regarding subjects, type of intervention, how variables are measured (operational methods), and reliability and validity of evaluation instruments. Due to the aforementioned weaknesses, positive outcomes could not be supported. Recommendations are made for future EAP evaluations. PMID:2728498

Rating the raters in a mixed model: An approach to deciphering the rater reliability

NASA Astrophysics Data System (ADS)

Shang, Junfeng; Wang, Yougui

2013-05-01

Rating the raters has attracted extensive attention in recent years. Ratings are quite complex in that the subjective assessment and a number of criteria are involved in a rating system. Whenever the human judgment is a part of ratings, the inconsistency of ratings is the source of variance in scores, and it is therefore quite natural for people to verify the trustworthiness of ratings. Accordingly, estimation of the rater reliability will be of great interest and an appealing issue. To facilitate the evaluation of the rater reliability in a rating system, we propose a mixed model where the scores of the ratees offered by a rater are described with the fixed effects determined by the ability of the ratees and the random effects produced by the disagreement of the raters. In such a mixed model, for the rater random effects, we derive its posterior distribution for the prediction of random effects. To quantitatively make a decision in revealing the unreliable raters, the predictive influence function (PIF) serves as a criterion which compares the posterior distributions of random effects between the full data and rater-deleted data sets. The benchmark for this criterion is also discussed. This proposed methodology of deciphering the rater reliability is investigated in the multiple simulated and two real data sets.

Test-retest reliability of prefrontal transcranial Direct Current Stimulation (tDCS) effects on functional MRI connectivity in healthy subjects.

PubMed

Wörsching, Jana; Padberg, Frank; Helbich, Konstantin; Hasan, Alkomiet; Koch, Lena; Goerigk, Stephan; Stoecklein, Sophia; Ertl-Wagner, Birgit; Keeser, Daniel

2017-07-15

Transcranial Direct Current Stimulation (tDCS) of the prefrontal cortex (PFC) can be used for probing functional brain connectivity and meets general interest as novel therapeutic intervention in psychiatric and neurological disorders. Along with a more extensive use, it is important to understand the interplay between neural systems and stimulation protocols requiring basic methodological work. Here, we examined the test-retest (TRT) characteristics of tDCS-induced modulations in resting-state functional-connectivity MRI (RS fcMRI). Twenty healthy subjects received 20minutes of either active or sham tDCS of the dorsolateral PFC (2mA, anode over F3 and cathode over F4, international 10-20 system), preceded and ensued by a RS fcMRI (10minutes each). All subject underwent three tDCS sessions with one-week intervals in between. Effects of tDCS on RS fcMRI were determined at an individual as well as at a group level using both ROI-based and independent-component analyses (ICA). To evaluate the TRT reliability of individual active-tDCS and sham effects on RS fcMRI, voxel-wise intra-class correlation coefficients (ICC) of post-tDCS maps between testing sessions were calculated. For both approaches, results revealed low reliability of RS fcMRI after active tDCS (ICC (2,1) = -0.09 - 0.16). Reliability of RS fcMRI (baselines only) was low to moderate for ROI-derived (ICC (2,1) = 0.13 - 0.50) and low for ICA-derived connectivity (ICC (2,1) = 0.19 - 0.34). Thus, for ROI-based analyses, the distribution of voxel-wise ICC was shifted to lower TRT reliability after active, but not after sham tDCS, for which the distribution was similar to baseline. The intra-individual variation observed here resembles variability of tDCS effects in motor regions and may be one reason why in this study robust tDCS effects at a group level were missing. The data can be used for appropriately designing large scale studies investigating methodological issues such as sources of variability and localisation of tDCS effects. Copyright © 2017 Elsevier Inc. All rights reserved.

A data-driven multi-model methodology with deep feature selection for short-term wind forecasting

DOE Office of Scientific and Technical Information (OSTI.GOV)

Feng, Cong; Cui, Mingjian; Hodge, Bri-Mathias

With the growing wind penetration into the power system worldwide, improving wind power forecasting accuracy is becoming increasingly important to ensure continued economic and reliable power system operations. In this paper, a data-driven multi-model wind forecasting methodology is developed with a two-layer ensemble machine learning technique. The first layer is composed of multiple machine learning models that generate individual forecasts. A deep feature selection framework is developed to determine the most suitable inputs to the first layer machine learning models. Then, a blending algorithm is applied in the second layer to create an ensemble of the forecasts produced by firstmore » layer models and generate both deterministic and probabilistic forecasts. This two-layer model seeks to utilize the statistically different characteristics of each machine learning algorithm. A number of machine learning algorithms are selected and compared in both layers. This developed multi-model wind forecasting methodology is compared to several benchmarks. The effectiveness of the proposed methodology is evaluated to provide 1-hour-ahead wind speed forecasting at seven locations of the Surface Radiation network. Numerical results show that comparing to the single-algorithm models, the developed multi-model framework with deep feature selection procedure has improved the forecasting accuracy by up to 30%.« less

Instruments to assess patients with rotator cuff pathology: a systematic review of measurement properties.

PubMed

Longo, Umile Giuseppe; Saris, Daniël; Poolman, Rudolf W; Berton, Alessandra; Denaro, Vincenzo

2012-10-01

The aims of this study were to obtain an overview of the methodological quality of studies on the measurement properties of rotator cuff questionnaires and to describe how well various aspects of the design and statistical analyses of studies on measurement properties are performed. A systematic review of published studies on the measurement properties of rotator cuff questionnaires was performed. Two investigators independently rated the quality of the studies using the Consensus-based Standards for the selection of health Measurement Instruments checklist. This checklist was developed in an international Delphi consensus study. Sixteen studies were included, in which two measurement instruments were evaluated, namely the Western Ontario Rotator Cuff Index and the Rotator Cuff Quality-of-Life Measure. The methodological quality of the included studies was adequate on some properties (construct validity, reliability, responsiveness, internal consistency, and translation) but need to be improved on other aspects. The most important methodological aspects that need to be developed are as follows: measurement error, content validity, structural validity, cross-cultural validity, criterion validity, and interpretability. Considering the importance of adequate measurement properties, it is concluded that, in the field of rotator cuff pathology, there is room for improvement in the methodological quality of studies measurement properties. II.

Evaluating the uncertainty of predicting future climate time series at the hourly time scale

NASA Astrophysics Data System (ADS)

Caporali, E.; Fatichi, S.; Ivanov, V. Y.

2011-12-01

A stochastic downscaling methodology is developed to generate hourly, point-scale time series for several meteorological variables, such as precipitation, cloud cover, shortwave radiation, air temperature, relative humidity, wind speed, and atmospheric pressure. The methodology uses multi-model General Circulation Model (GCM) realizations and an hourly weather generator, AWE-GEN. Probabilistic descriptions of factors of change (a measure of climate change with respect to historic conditions) are computed for several climate statistics and different aggregation times using a Bayesian approach that weights the individual GCM contributions. The Monte Carlo method is applied to sample the factors of change from their respective distributions thereby permitting the generation of time series in an ensemble fashion, which reflects the uncertainty of climate projections of future as well as the uncertainty of the downscaling procedure. Applications of the methodology and probabilistic expressions of certainty in reproducing future climates for the periods, 2000 - 2009, 2046 - 2065 and 2081 - 2100, using the 1962 - 1992 period as the baseline, are discussed for the location of Firenze (Italy). The climate predictions for the period of 2000 - 2009 are tested against observations permitting to assess the reliability and uncertainties of the methodology in reproducing statistics of meteorological variables at different time scales.

An optimized methodology to analyze biopolymer capsules by environmental scanning electron microscopy.

PubMed

Conforto, Egle; Joguet, Nicolas; Buisson, Pierre; Vendeville, Jean-Eudes; Chaigneau, Carine; Maugard, Thierry

2015-02-01

The aim of this paper is to describe an optimized methodology to study the surface characteristics and internal structure of biopolymer capsules using scanning electron microscopy (SEM) in environmental mode. The main advantage of this methodology is that no preparation is required and, significantly, no metallic coverage is deposited on the surface of the specimen, thus preserving the original capsule shape and its surface morphology. This avoids introducing preparation artefacts which could modify the capsule surface and mask information concerning important feature like porosities or roughness. Using this method gelatin and mainly fatty coatings, difficult to be analyzed by standard SEM technique, unambiguously show fine details of their surface morphology without damage. Furthermore, chemical contrast is preserved in backscattered electron images of unprepared samples, allowing visualizing the internal organization of the capsule, the quality of the envelope, etc... This study provides pointers on how to obtain optimal conditions for the analysis of biological or sensitive material, as this is not always studied using appropriate techniques. A reliable evaluation of the parameters used in capsule elaboration for research and industrial applications, as well as that of capsule functionality is provided by this methodology, which is essential for the technological progress in this domain. Copyright © 2014 Elsevier B.V. All rights reserved.

Construct Definition Methodology and Generalizability Theory Applied to Career Education Measurement.

ERIC Educational Resources Information Center

Stenner, A. Jackson; Rohlf, Richard J.

The merits of generalizability theory in the formulation of construct definitions and in the determination of reliability estimates are discussed. The broadened conceptualization of reliability brought about by Cronbach's generalizability theory is reviewed. Career Maturity Inventory data from a sample of 60 ninth grade students is used to…

34 CFR 668.144 - Application for test approval.

Code of Federal Regulations, 2010 CFR

2010-07-01

... the comparability of scores on the current test to scores on the previous test, and data from validity... explanation of the methodology and procedures for measuring the reliability of the test; (ii) Evidence that different forms of the test, including, if applicable, short forms, are comparable in reliability; (iii...

Allometric scaling theory applied to FIA biomass estimation

Treesearch

David C. Chojnacky

2002-01-01

Tree biomass estimates in the Forest Inventory and Analysis (FIA) database are derived from numerous methodologies whose abundance and complexity raise questions about consistent results throughout the U.S. A new model based on allometric scaling theory ("WBE") offers simplified methodology and a theoretically sound basis for improving the reliability and...

Kohlberg's Moral Judgment Scale: Some Methodological Considerations

ERIC Educational Resources Information Center

Rubin, Kenneth H.; Trotter, Kristin T.

1977-01-01

Examined 3 methodological issues in the use of Kohlberg's Moral Judgment Scale: (1) test-retest reliability, (2) consistency of moral judgment stages from one dilemma to the next, and (3) influence of subject's verbal facility on the projective test scores. Forty children in grades 3 and 5 participated. (JMB)

Inter-rater reliability for movement pattern analysis (MPA): measuring patterning of behaviors versus discrete behavior counts as indicators of decision-making style

PubMed Central

Connors, Brenda L.; Rende, Richard; Colton, Timothy J.

2014-01-01

The unique yield of collecting observational data on human movement has received increasing attention in a number of domains, including the study of decision-making style. As such, interest has grown in the nuances of core methodological issues, including the best ways of assessing inter-rater reliability. In this paper we focus on one key topic – the distinction between establishing reliability for the patterning of behaviors as opposed to the computation of raw counts – and suggest that reliability for each be compared empirically rather than determined a priori. We illustrate by assessing inter-rater reliability for key outcome measures derived from movement pattern analysis (MPA), an observational methodology that records body movements as indicators of decision-making style with demonstrated predictive validity. While reliability ranged from moderate to good for raw counts of behaviors reflecting each of two Overall Factors generated within MPA (Assertion and Perspective), inter-rater reliability for patterning (proportional indicators of each factor) was significantly higher and excellent (ICC = 0.89). Furthermore, patterning, as compared to raw counts, provided better prediction of observable decision-making process assessed in the laboratory. These analyses support the utility of using an empirical approach to inform the consideration of measuring patterning versus discrete behavioral counts of behaviors when determining inter-rater reliability of observable behavior. They also speak to the substantial reliability that may be achieved via application of theoretically grounded observational systems such as MPA that reveal thinking and action motivations via visible movement patterns. PMID:24999336

Inter-rater reliability for movement pattern analysis (MPA): measuring patterning of behaviors versus discrete behavior counts as indicators of decision-making style.

PubMed

Connors, Brenda L; Rende, Richard; Colton, Timothy J

2014-01-01

The unique yield of collecting observational data on human movement has received increasing attention in a number of domains, including the study of decision-making style. As such, interest has grown in the nuances of core methodological issues, including the best ways of assessing inter-rater reliability. In this paper we focus on one key topic - the distinction between establishing reliability for the patterning of behaviors as opposed to the computation of raw counts - and suggest that reliability for each be compared empirically rather than determined a priori. We illustrate by assessing inter-rater reliability for key outcome measures derived from movement pattern analysis (MPA), an observational methodology that records body movements as indicators of decision-making style with demonstrated predictive validity. While reliability ranged from moderate to good for raw counts of behaviors reflecting each of two Overall Factors generated within MPA (Assertion and Perspective), inter-rater reliability for patterning (proportional indicators of each factor) was significantly higher and excellent (ICC = 0.89). Furthermore, patterning, as compared to raw counts, provided better prediction of observable decision-making process assessed in the laboratory. These analyses support the utility of using an empirical approach to inform the consideration of measuring patterning versus discrete behavioral counts of behaviors when determining inter-rater reliability of observable behavior. They also speak to the substantial reliability that may be achieved via application of theoretically grounded observational systems such as MPA that reveal thinking and action motivations via visible movement patterns.

Validation of highly reliable, real-time knowledge-based systems

NASA Technical Reports Server (NTRS)

Johnson, Sally C.

1988-01-01

Knowledge-based systems have the potential to greatly increase the capabilities of future aircraft and spacecraft and to significantly reduce support manpower needed for the space station and other space missions. However, a credible validation methodology must be developed before knowledge-based systems can be used for life- or mission-critical applications. Experience with conventional software has shown that the use of good software engineering techniques and static analysis tools can greatly reduce the time needed for testing and simulation of a system. Since exhaustive testing is infeasible, reliability must be built into the software during the design and implementation phases. Unfortunately, many of the software engineering techniques and tools used for conventional software are of little use in the development of knowledge-based systems. Therefore, research at Langley is focused on developing a set of guidelines, methods, and prototype validation tools for building highly reliable, knowledge-based systems. The use of a comprehensive methodology for building highly reliable, knowledge-based systems should significantly decrease the time needed for testing and simulation. A proven record of delivering reliable systems at the beginning of the highly visible testing and simulation phases is crucial to the acceptance of knowledge-based systems in critical applications.

Psychometric properties of the Persian version of Social Adaptation Self-evaluation Scale in community-dwelling older adults.

PubMed

Farokhnezhad Afshar, Pouya; Foroughan, Mahshid; Vedadhir, AbouAli; Ghazi Tabatabaie, Mahmood

2017-01-01

The Social Adaptation Self-evaluation Scale (SASS) is used to measure social function and social motivation in depressed patients. There is little attention to social function in the treatment of depression. The aim of this study was to assess the validity and reliability of the Persian version of SASS (P-SASS) for older adults. This is a cross-sectional and methodological study. The participants were 550 community-dwelling older adults living in Tehran who were selected randomly from the primary health care centers. To assess the psychometric properties of SASS, we first did translation and cross-cultural adjustment on SASS and then used P-SASS and the Geriatric Depression Scale (GDS) for gathering data. A number of analyses, including Pearson's correlation, exploratory factor analysis, and Cronbach's α , and receiver operating characteristic curve were used to manage the data with the IBM SPSS Statistics V.22. The mean age of the participants was 66.09±6.67 years, and 58.9% of them were male. The Cronbach's α was 0.97. The test-retest reliability correlation coefficient was 0.78. Principal component analysis showed that P-SASS consists of two components. P-SASS score showed a significant negative correlation with GDS ( r =-0.91, P <0.01), which suggests good convergent validity. The P-SASS cutoff point was 28 (sensitivity: 0.97 and specificity: 0.94). P-SASS has good reliability and validity for older adults. So, it can be considered as an appropriate tool to evaluate the social function and social motivation of older persons with and without depression.

Development and validation of surgical training tool: cystectomy assessment and surgical evaluation (CASE) for robot-assisted radical cystectomy for men.

PubMed

Hussein, Ahmed A; Sexton, Kevin J; May, Paul R; Meng, Maxwell V; Hosseini, Abolfazl; Eun, Daniel D; Daneshmand, Siamak; Bochner, Bernard H; Peabody, James O; Abaza, Ronney; Skinner, Eila C; Hautmann, Richard E; Guru, Khurshid A

2018-04-13

We aimed to develop a structured scoring tool: cystectomy assessment and surgical evaluation (CASE) that objectively measures and quantifies performance during robot-assisted radical cystectomy (RARC) for men. A multinational 10-surgeon expert panel collaborated towards development and validation of CASE. The critical steps of RARC in men were deconstructed into nine key domains, each assessed by five anchors. Content validation was done utilizing the Delphi methodology. Each anchor was assessed in terms of context, score concordance, and clarity. The content validity index (CVI) was calculated for each aspect. A CVI ≥ 0.75 represented consensus, and this statement was removed from the next round. This process was repeated until consensus was achieved for all statements. CASE was used to assess de-identified videos of RARC to determine reliability and construct validity. Linearly weighted percent agreement was used to assess inter-rater reliability (IRR). A logit model for odds ratio (OR) was used to assess construct validation. The expert panel reached consensus on CASE after four rounds. The final eight domains of the CASE included: pelvic lymph node dissection, development of the peri-ureteral space, lateral pelvic space, anterior rectal space, control of the vascular pedicle, anterior vesical space, control of the dorsal venous complex, and apical dissection. IRR > 0.6 was achieved for all eight domains. Experts outperformed trainees across all domains. We developed and validated a reliable structured, procedure-specific tool for objective evaluation of surgical performance during RARC. CASE may help differentiate novice from expert performances.

Performance of regional oxygen saturation monitoring by near-infrared spectroscopy (NIRS) in pediatric inter-hospital transports with special reference to air ambulance transports: a methodological study.

PubMed

Hamrin, Tova Hannegård; Radell, Peter J; Fläring, Urban; Berner, Jonas; Eksborg, Staffan

2017-12-28

The aim of the present study was to evaluate the performance of regional oxygen saturation (rSO 2 ) monitoring with near infrared spectroscopy (NIRS) during pediatric inter-hospital transports and to optimize processing of the electronically stored data. Cerebral (rSO 2 -C) and abdominal (rSO 2 -A) NIRS sensors were used during transport in air ambulance and connecting ground ambulance. Data were electronically stored by the monitor during transport, extracted and analyzed off-line after the transport. After removal of all zero and floor effect values, the Savitzky-Golay algorithm of data smoothing was applied on the NIRS-signal. The second order of smoothing polynomial was used and the optimal number of neighboring points for the smoothing procedure was evaluated. NIRS-data from 38 pediatric patients was examined. Reliability, defined as measurements without values of 0 or 15%, was acceptable during transport (> 90% of all measurements). There were, however, individual patients with < 90% reliable measurements during transport, while no patient was found to have < 90% reliable measurements in hospital. Satisfactory noise reduction of the signal, without distortion of the underlying information, was achieved when 20-50 neighbors ("window-size") were used. The use of NIRS for measuring rSO 2 in clinical studies during pediatric transport in ground and air-ambulance is feasible but hampered by unreliable values and signal interference. By applying the Savitzky-Golay algorithm, the signal-to-noise ratio was improved and enabled better post-hoc signal evaluation.

Automated Collection of Real-Time Alerts of Citizens as a Useful Tool to Continuously Monitor Malodorous Emissions.

PubMed

Brattoli, Magda; Mazzone, Antonio; Giua, Roberto; Assennato, Giorgio; de Gennaro, Gianluigi

2016-02-26

The evaluation of odor emissions and dispersion is a very arduous topic to face; the real-time monitoring of odor emissions, the identification of chemical components and, with proper certainty, the source of annoyance represent a challenge for stakeholders such as local authorities. The complaints of people, often not systematic and variously distributed, in general do not allow us to quantify the perceived annoyance. Experimental research has been performed to detect and evaluate olfactory annoyance, based on field testing of an innovative monitoring methodology grounded in automatic recording of citizen alerts. It has been applied in Taranto, in the south of Italy where a relevant industrial area is located, by using Odortel(®) for automated collection of citizen alerts. To evaluate its reliability, the collection system has been integrated with automated samplers, able to sample odorous air in real time, according to the citizen alerts of annoyance and, moreover, with meteorological data (especially the wind direction) and trends in odor marker compounds, recorded by air quality monitoring stations. The results have allowed us, for the first time, to manage annoyance complaints, test their reliability, and obtain information about the distribution and entity of the odor phenomena, such that we were able to identify, with supporting evidence, the source as an oil refinery plant.

CARES/Life Ceramics Durability Evaluation Software Enhanced for Cyclic Fatigue

NASA Technical Reports Server (NTRS)

Nemeth, Noel N.; Powers, Lynn M.; Janosik, Lesley A.

1999-01-01

The CARES/Life computer program predicts the probability of a monolithic ceramic component's failure as a function of time in service. The program has many features and options for materials evaluation and component design. It couples commercial finite element programs--which resolve a component's temperature and stress distribution--to reliability evaluation and fracture mechanics routines for modeling strength-limiting defects. The capability, flexibility, and uniqueness of CARES/Life have attracted many users representing a broad range of interests and has resulted in numerous awards for technological achievements and technology transfer. Recent work with CARES/Life was directed at enhancing the program s capabilities with regards to cyclic fatigue. Only in the last few years have ceramics been recognized to be susceptible to enhanced degradation from cyclic loading. To account for cyclic loads, researchers at the NASA Lewis Research Center developed a crack growth model that combines the Power Law (time-dependent) and the Walker Law (cycle-dependent) crack growth models. This combined model has the characteristics of Power Law behavior (decreased damage) at high R ratios (minimum load/maximum load) and of Walker law behavior (increased damage) at low R ratios. In addition, a parameter estimation methodology for constant-amplitude, steady-state cyclic fatigue experiments was developed using nonlinear least squares and a modified Levenberg-Marquardt algorithm. This methodology is used to give best estimates of parameter values from cyclic fatigue specimen rupture data (usually tensile or flexure bar specimens) for a relatively small number of specimens. Methodology to account for runout data (unfailed specimens over the duration of the experiment) was also included.

Adverse effects of psychosocial work factors on blood pressure: systematic review of studies on demand-control-support and effort-reward imbalance models.

PubMed

Gilbert-Ouimet, Mahée; Trudel, Xavier; Brisson, Chantal; Milot, Alain; Vézina, Michel

2014-03-01

A growing body of research has investigated the adverse effects of psychosocial work factors on blood pressure (BP) elevation. There is now a clear need for an up-to-date, critical synthesis of reliable findings on this topic. This systematic review aimed to evaluate the adverse effects of psychosocial work factors of both the demand-control-support (DCS) and effort-reward imbalance (ERI) models on BP among men and women, according to the methodological quality of the studies. To be eligible, studies had to: (i) evaluate at least one psychosocial work factor, (ii) evaluate BP or hypertension, (iii) comprise ≥100 workers, (iv) be written in English or French, and (v) be published in a peer-reviewed journal. A total of 74 studies were included. Of these, 64 examined the DCS model, and 12 looked at the ERI model, with 2 studies considering both models. Approximately half the studies observed a significant adverse effect of psychosocial work factors on BP. A more consistent effect was observed, however, among men than women. For job strain, a more consistent effect was also observed in studies of higher methodological quality, ie, studies using a prospective design and ambulatory BP measures. A more consistent adverse effect of psychosocial work factors was observed among men than women and in studies of higher methodological quality. These findings contribute to the current effort of primary prevention of cardiovascular disease by documenting the psychosocial etiology of elevated BP, a major cardiovascular risk factor.

ARAMIS project: a more explicit demonstration of risk control through the use of bow-tie diagrams and the evaluation of safety barrier performance.

PubMed

de Dianous, Valérie; Fiévez, Cécile

2006-03-31

Over the last two decades a growing interest for risk analysis has been noted in the industries. The ARAMIS project has defined a methodology for risk assessment. This methodology has been built to help the industrialist to demonstrate that they have a sufficient risk control on their site. Risk analysis consists first in the identification of all the major accidents, assuming that safety functions in place are inefficient. This step of identification of the major accidents uses bow-tie diagrams. Secondly, the safety barriers really implemented on the site are taken into account. The barriers are identified on the bow-ties. An evaluation of their performance (response time, efficiency, and level of confidence) is performed to validate that they are relevant for the expected safety function. At last, the evaluation of their probability of failure enables to assess the frequency of occurrence of the accident. The demonstration of the risk control based on a couple gravity/frequency of occurrence is also possible for all the accident scenarios. During the risk analysis, a practical tool called risk graph is used to assess if the number and the reliability of the safety functions for a given cause are sufficient to reach a good risk control.