OC19 - Measuring feasibility, reliability and validity of the Greek version of PedsQL cardiac module.

PubMed

Drakouli, Maria; Petsios, Konstantinos; Matziou, Vasiliki

2016-05-09

Theme: Cardiology Introduction: Measuring quality of life (QoL) in children and adolescents with congenital heart disease (CHD) is of great clinical importance. The aim of the study was: (a) to adapt the PedsQL Cardiac Module for children aged two to 18 years with CHD in a sample of the Greek population; (b) to determine its reliability and validity. Forward and backward translation methodology was used. Parents and children completed the instrument during: (a) hospitalization and (b) visits in the paediatric cardiology outpatient department. Cross-informant variance between children and parents was thoroughly assessed. Missing item responses did not exceed 5%. All internal consistency reliability coefficients for the inventory exceeded the minimum standards for group comparisons, over 0.75. Hypothesized correlations between cardiac module and core scales were statistically significant, (p<0.05). Agreement between children and parents was relatively high. Pilot study results will be additionally presented. The findings support the feasibility, reliability and validity of the Greek translation of the PedsQL Cardiac Module in children with congenital heart defect (CHD).

Measuring health-related quality of life in children with cancer living in Mainland China: feasibility, reliability and validity of the Chinese Mandarin version of PedsQL 4.0 Generic Core Scales and 3.0 Cancer Module.

PubMed

Ji, Yi; Chen, Siyuan; Li, Kai; Xiao, Nong; Yang, Xue; Zheng, Shan; Xiao, Xianmin

2011-11-23

The Pediatric Quality of Life Inventory (PedsQL) is widely used instrument to measure pediatric health-related quality of life (HRQOL) for children aged 2 to 18 years. The purpose of the current study was to investigate the feasibility, reliability and validity of the Chinese mandarin version of the PedsQL 4.0 Generic Core Scales and 3.0 Cancer Module in a group of Chinese children with cancer. The PedsQL 4.0 Genetic Core Scales and the PedsQL 3.0 Cancer Module were administered to children with cancer (aged 5-18 years) and parents of such children (aged 2-18 years). For comparison, a survey on a demographically group-matched sample of the general population with children (aged 5-18) and parents of children (aged 2-18 years) was conducted with the PedsQL 4.0 Genetic Core Scales. The minimal mean percentage of missing item responses (except the School Functioning scale) supported the feasibility of the PedsQL 4.0 Generic Core Scales and 3.0 Cancer Module for Chinese children with cancer. Most of the scales showed satisfactory reliability with Cronbach's α of exceeding 0.70, and all scales demonstrated sufficient test-retest reliability. Assessing the clinical validity of the questionnaires, statistically significant difference was found between healthy children and children with cancer, and between children on-treatment versus off-treatment ≥12 months. Positive significant correlations were observed between the scores of the PedsQL 4.0 Generic Core Scale and the PedsQL 3.0 Cancer Module. Exploratory factor analysis demonstrated sufficient factorial validity. Moderate to good agreement was found between child self- and parent proxy-reports. The findings support the feasibility, reliability and validity of the Chinese Mandarin version of PedsQL 4.0 Generic Core Scales and 3.0 Cancer Module in children with cancer living in mainland China.

Measuring health-related quality of life in children with cancer living in mainland China: feasibility, reliability and validity of the Chinese mandarin version of PedsQL 4.0 Generic Core Scales and 3.0 Cancer Module

PubMed Central

2011-01-01

Background The Pediatric Quality of Life Inventory (PedsQL) is widely used instrument to measure pediatric health-related quality of life (HRQOL) for children aged 2 to 18 years. The purpose of the current study was to investigate the feasibility, reliability and validity of the Chinese mandarin version of the PedsQL 4.0 Generic Core Scales and 3.0 Cancer Module in a group of Chinese children with cancer. Methods The PedsQL 4.0 Genetic Core Scales and the PedsQL 3.0 Cancer Module were administered to children with cancer (aged 5-18 years) and parents of such children (aged 2-18 years). For comparison, a survey on a demographically group-matched sample of the general population with children (aged 5-18) and parents of children (aged 2-18 years) was conducted with the PedsQL 4.0 Genetic Core Scales. Result The minimal mean percentage of missing item responses (except the School Functioning scale) supported the feasibility of the PedsQL 4.0 Generic Core Scales and 3.0 Cancer Module for Chinese children with cancer. Most of the scales showed satisfactory reliability with Cronbach's α of exceeding 0.70, and all scales demonstrated sufficient test-retest reliability. Assessing the clinical validity of the questionnaires, statistically significant difference was found between healthy children and children with cancer, and between children on-treatment versus off-treatment ≥12 months. Positive significant correlations were observed between the scores of the PedsQL 4.0 Generic Core Scale and the PedsQL 3.0 Cancer Module. Exploratory factor analysis demonstrated sufficient factorial validity. Moderate to good agreement was found between child self- and parent proxy-reports. Conclusion The findings support the feasibility, reliability and validity of the Chinese Mandarin version of PedsQL 4.0 Generic Core Scales and 3.0 Cancer Module in children with cancer living in mainland China. PMID:22111968

Design, testing and validation of an innovative web-based instrument to evaluate school meal quality.

PubMed

Patterson, Emma; Quetel, Anna-Karin; Lilja, Karin; Simma, Marit; Olsson, Linnea; Elinder, Liselotte Schäfer

2013-06-01

To develop a feasible, valid, reliable web-based instrument to objectively evaluate school meal quality in Swedish primary schools. The construct 'school meal quality' was operationalized by an expert panel into six domains, one of which was nutritional quality. An instrument was drafted and pilot-tested. Face validity was evaluated by the panel. Feasibility was established via a large national study. Food-based criteria to predict the nutritional adequacy of school meals in terms of fat quality, iron, vitamin D and fibre content were developed. Predictive validity was evaluated by comparing the nutritional adequacy of school menus based on these criteria with the results from a nutritional analysis. Inter-rater reliability was also assessed. The instrument was developed between 2010 and 2012. It is designed for use in all primary schools by school catering and/or management representatives. A pilot-test of eighty schools in Stockholm (autumn 2010) and a further test of feasibility in 191 schools nationally (spring 2011). The four nutrient-specific food-based criteria predicted nutritional adequacy with sensitivity ranging from 0.85 to 1.0, specificity from 0.45 to 1.0 and accuracy from 0.67 to 1.0. The sample in the national study was statistically representative and the majority of users rated the questionnaire positively, suggesting the instrument is feasible. The inter-rater reliability was fair to almost perfect for continuous variables and agreement was ≥ 67 % for categorical variables. An innovative web-based system to comprehensively monitor school meal quality across several domains, with validated questions in the nutritional domain, is available in Sweden for the first time.

Measuring physical activity in preschoolers: Reliability and validity of The System for Observing Fitness Instruction Time for Preschoolers (SOFIT-P)

PubMed Central

Sharma, Shreela; Chuang, Ru-Jye; Skala, Katherine; Atteberry, Heather

2012-01-01

The purpose of this study is describe the initial feasibility, reliability, and validity of an instrument to measure physical activity in preschoolers using direct observation. The System for Observing Fitness Instruction Time for Preschoolers was developed and tested among 3- to 6-year-old children over fall 2008 for feasibility and reliability (Phase I, n=67) and in fall 2009 for concurrent validity (Phase II, n=27). Phase I showed that preschoolers spent >75% of their active time at preschool in light physical activity. The mean inter-observer agreements scores were ≥.75 for physical activity level and type. Correlation coefficients, measuring construct validity between the lesson context and physical activity types with and with the activity levels, were moderately strong. Phase II showed moderately strong correlations ranging from .50 to .54 between the System for Observing Fitness Instruction Time for Preschoolers and Actigraph accelerometers for physical activity levels. The System for Observing Fitness Instruction Time for Preschoolers shows promising initial results as a new method for measuring physical activity among preschoolers. PMID:22485071

Fatigue in children: reliability and validity of the Dutch PedsQL™ Multidimensional Fatigue Scale.

PubMed

Gordijn, M Suzanne; Suzanne Gordijn, M; Cremers, Eline M P; Kaspers, Gertjan J L; Gemke, Reinoud J B J

2011-09-01

The aim of the study is to report on the feasibility, reliability, validity, and the norm-references of the Dutch version of the PedsQL™ Multidimensional Fatigue Scale. The study participants are four hundred and ninety-seven parents of children aged 2-18 years and 366 children aged 5-18 years from various day care facilities, elementary schools, and a high school who completed the Dutch version of the PedsQL™ Multidimensional Fatigue Scale. The number of missing items was minimal. All scales showed satisfactory internal consistency reliability, with Cronbach's coefficient alpha exceeding 0.70. Test-retest reliability was good to excellent (ICCs 0.68-0.84) and inter-observer reliability varied from moderate to excellent (ICCs 0.56-0.93) for total scores. Parent/child concordance for total scores was poor to good (ICCs 0.25-0.68). The PedsQL™ Multidimensional Fatigue Scale was able to distinguish between healthy children and children with an impaired health condition. The Dutch version of the PedsQL™ Multidimensional Fatigue Scale demonstrates an adequate feasibility, reliability, and validity in another sociocultural context. With the obtained norm-references, it can be utilized as a tool in the evaluation of fatigue in healthy and chronically ill children aged 2-18 years.

Construct validity and reliability of a real-time multidimensional smartphone app to assess pain in children and adolescents with cancer.

PubMed

Stinson, Jennifer N; Jibb, Lindsay A; Nguyen, Cynthia; Nathan, Paul C; Maloney, Anne Marie; Dupuis, L Lee; Gerstle, J Ted; Hopyan, Sevan; Alman, Benjamin A; Strahlendorf, Caron; Portwine, Carol; Johnston, Donna L

2015-12-01

We evaluated the construct validity (including responsiveness), reliability, and feasibility of the Pain Squad multidimensional smartphone-based pain assessment application (app) in children and adolescents with cancer, using 2 descriptive studies with repeated measures. Participants (8-18 years) undergoing cancer treatment were drawn from 4 pediatric cancer centers. In study 1, 92 participants self-reported their level of pain twice daily for 2 weeks using the Pain Squad app to assess app construct validity and reliability. In study 2, 14 participants recorded their level of pain twice a day for 1 week before and 2 weeks after cancer-related surgery to determine app responsiveness. Participants in both studies completed multiple measures to determine the construct validity and feasibility of the Pain Squad app. Correlations between average weekly pain ratings on the Pain Squad app and recalled least, average, and worst weekly pain were moderate to high (0.43-0.68). Correlations with health-related quality of life and pain coping (measured with PedsQL Inventory 4.0, PedsQL Cancer Module, and Pain Coping Questionnaire) were -0.46 to 0.29. The app showed excellent internal consistency (α = 0.96). Pain ratings changed because of surgery with large effect sizes between baseline and the first week postsurgery (>0.85) and small effect sizes between baseline and the second week postsurgery (0.13-0.32). These findings provide evidence of the construct validity, reliability, and feasibility of the Pain Squad app in children and adolescents with cancer. Use of real-time data capture approaches should be considered in future studies of childhood cancer pain. A video accompanying this abstract is available online as Supplemental Digital Content at http://links.lww.com/PAIN/A169.

Objective structured clinical examination for pharmacy students in Qatar: cultural and contextual barriers to assessment.

PubMed

Wilby, K J; Black, E K; Austin, Z; Mukhalalati, B; Aboulsoud, S; Khalifa, S I

2016-07-10

This study aimed to evaluate the feasibility and psychometric defensibility of implementing a comprehensive objective structured clinical examination (OSCE) on the complete pharmacy programme for pharmacy students in a Middle Eastern context, and to identify facilitators and barriers to implementation within new settings. Eight cases were developed, validated, and had standards set according to a blueprint, and were assessed with graduating pharmacy students. Assessor reliability was evaluated using inter-class coefficients (ICCs). Concurrent validity was evaluated by comparing OSCE results to professional skills course grades. Field notes were maintained to generate recommendations for implementation in other contexts. The examination pass mark was 424 points out of 700 (60.6%). All 23 participants passed. Mean performance was 74.6%. Low to moderate inter-rater reliability was obtained for analytical and global components (average ICC 0.77 and 0.48, respectively). In conclusion, OSCE was feasible in Qatar but context-related validity and reliability concerns must be addressed prior to future iterations in Qatar and elsewhere.

A non-invasive assessment of skin carotenoid status through reflection spectroscopy is a feasible, reliable and potentially valid measure of fruit and vegetable consumption in a diverse community sample.

PubMed

Jilcott Pitts, Stephanie Bell; Jahns, Lisa; Wu, Qiang; Moran, Nancy E; Bell, Ronny A; Truesdale, Kimberly P; Laska, Melissa N

2018-06-01

To assess the feasibility, reliability and validity of reflection spectroscopy (RS) to assess skin carotenoids in a racially diverse sample. Study 1 was a cross-sectional study of corner store customers (n 479) who completed the National Cancer Institute Fruit and Vegetable Screener as well as RS measures. Feasibility was assessed by examining the time it took to complete three RS measures, reliability was assessed by examining the variation between three RS measures, and validity was examined by correlation with self-reported fruit and vegetable consumption. In Study 2, validity was assessed in a smaller sample (n 30) by examining associations between RS measures and dietary carotenoids, fruits and vegetables as calculated from a validated FFQ and plasma carotenoids. Eastern North Carolina, USA. It took on average 94·0 s to complete three RS readings per person. The average variation between three readings for each participant was 6·8 %. In Study 2, in models adjusted for age, race and sex, there were statistically significant associations between RS measures and (i) FFQ-estimated carotenoid intake (P<0·0001); (ii) FFQ-estimated fruit and vegetable consumption (P<0·010); and (iii) plasma carotenoids (P<0·0001). RS is a potentially improved method to approximate fruit and vegetable consumption among diverse participants. RS is portable and easy to use in field-based public health nutrition settings. More research is needed to investigate validity and sensitivity in diverse populations.

Accuracy and Feasibility of Video Analysis for Assessing Hamstring Flexibility and Validity of the Sit-and-Reach Test

ERIC Educational Resources Information Center

Mier, Constance M.

2011-01-01

The accuracy of video analysis of the passive straight-leg raise test (PSLR) and the validity of the sit-and-reach test (SR) were tested in 60 men and women. Computer software measured static hip-joint flexion accurately. High within-session reliability of the PSLR was demonstrated (R greater than 0.97). Test-retest (separate days) reliability for…

Validation of the German revised version of the program in palliative care education and practice questionnaire (PCEP-GR).

PubMed

Fetz, Katharina; Wenzel-Meyburg, Ursula; Schulz-Quach, Christian

2017-12-28

The evaluation of the effectiveness of undergraduate palliative care education (UPCE) programs is an essential foundation to providing high-quality UPCE programs. Therefore, the implementation of valid evaluation tools is indispensable. Until today, there has been no general consensus regarding concrete outcome parameters and their accurate measurement. The Program in Palliative Care Education and Practice Questionnaire (German Revised Version; PCEP-GR) is a promising assessment tool for UPCE. The aim of the current study was to evaluate the psychometric properties of PCEP-GR and to demonstrate its feasibility for the evaluation of UPCE programs. The practical feasibility of the PCEP-GR and its acceptance in medical students were investigated in a pilot study with 24 undergraduate medical students at Heinrich Heine University Dusseldorf, Germany. Subsequently, the PCEP-GR was surveyed in a representative sample (N = 680) of medical students in order to investigate its psychometric properties. Factorial validity was investigated by means of principal component analysis (PCA). Reliability was examined by means of split-half-reliability analysis and analysis of internal consistency. After taking into consideration the PCA and distribution analysis results, an evaluation instruction for the PCEP-GR was developed. The PCEP-GR proved to be feasible and well-accepted in medical students. PCA revealed a four-factorial solution indicating four PCEP-GR subscales: preparation to provide palliative care, attitudes towards palliative care, self-estimation of competence in communication with dying patients and their relatives and self-estimation of knowledge and skills in palliative care. The PCEP-GR showed good split-half-reliability and acceptable to good internal consistency of subscales. Attitudes towards palliative care slightly missed the criterion of acceptable internal consistency. The evaluation instruction suggests a global PCEP-GR index and four subscales. The PCEP-GR has proven to be a feasible, economic, valid and reliable tool for the assessment of UPCE that comprises self-efficacy expectation and relevant attitudes towards palliative care.

Measuring metacognitive ability based on science literacy in dynamic electricity topic

NASA Astrophysics Data System (ADS)

Warni; Sunyono; Rosidin

2018-01-01

This study aims to produce an instrument of metacognition ability assessment based on science literacy on theoretically and empirically feasible dynamic electrical material. The feasibility of the assessment instrument includes theoretical validity on material, construction, and language aspects, as well as empirical validity, reliability, difficulty, distinguishing, and distractor indices. The development of assessment instruments refers to the Dick and Carey development model which includes the preliminary study stage, initial product development, validation and revision, and piloting. The instrument was tested to 32 students of class IX in SMP Negeri 20 Bandar Lampung, using the design of One Group Pretest-Postest Design. The result shows that the metacognition ability assessment instrument based on science literacy is feasible theoretically with theoretical validity percentage of 95.44% and empirical validity of 43.75% for the high category, 43.75% for the medium category, and 12.50 % for low category questions; Reliability of assessment instruments of 0.83 high categories; Difficulty level of difficult item is about 31.25% and medium category is equal to 68.75%. Item that has very good distinguishing power is 12.50%, 62.50% for good stage, and medium category is 25.00%; As well as the duplexing function on a matter of multiple choice is 80.00% including good category and 20.00% for medium category.

The Trojan Lifetime Champions Health Survey: development, validity, and reliability.

PubMed

Sorenson, Shawn C; Romano, Russell; Scholefield, Robin M; Schroeder, E Todd; Azen, Stanley P; Salem, George J

2015-04-01

Self-report questionnaires are an important method of evaluating lifespan health, exercise, and health-related quality of life (HRQL) outcomes among elite, competitive athletes. Few instruments, however, have undergone formal characterization of their psychometric properties within this population. To evaluate the validity and reliability of a novel health and exercise questionnaire, the Trojan Lifetime Champions (TLC) Health Survey. Descriptive laboratory study. A large National Collegiate Athletic Association Division I university. A total of 63 university alumni (age range, 24 to 84 years), including former varsity collegiate athletes and a control group of nonathletes. Participants completed the TLC Health Survey twice at a mean interval of 23 days with randomization to the paper or electronic version of the instrument. Content validity, feasibility of administration, test-retest reliability, parallel-form reliability between paper and electronic forms, and estimates of systematic and typical error versus differences of clinical interest were assessed across a broad range of health, exercise, and HRQL measures. Correlation coefficients, including intraclass correlation coefficients (ICCs) for continuous variables and κ agreement statistics for ordinal variables, for test-retest reliability averaged 0.86, 0.90, 0.80, and 0.74 for HRQL, lifetime health, recent health, and exercise variables, respectively. Correlation coefficients, again ICCs and κ, for parallel-form reliability (ie, equivalence) between paper and electronic versions averaged 0.90, 0.85, 0.85, and 0.81 for HRQL, lifetime health, recent health, and exercise variables, respectively. Typical measurement error was less than the a priori thresholds of clinical interest, and we found minimal evidence of systematic test-retest error. We found strong evidence of content validity, convergent construct validity with the Short-Form 12 Version 2 HRQL instrument, and feasibility of administration in an elite, competitive athletic population. These data suggest that the TLC Health Survey is a valid and reliable instrument for assessing lifetime and recent health, exercise, and HRQL, among elite competitive athletes. Generalizability of the instrument may be enhanced by additional, larger-scale studies in diverse populations.

SCREENING FOR PTSD AMONG INCARCERATED MEN

PubMed Central

WOLFF, NANCY; GREGORY CHUGO, M; SHI, JING; HUENING, JESSICA; FRUEH, B. CHRISTOPHER

2014-01-01

Trauma exposure is overrepresented in incarcerated male populations and is linked to psychiatric morbidity, particularly posttraumatic stress disorder (PTSD). This study tests the feasibility, reliability, and validity of using computer-administered interviewing (CAI) versus orally administered interviewing (OAI) to screen for PTSD among incarcerated men. A 2 × 2 factorial design was used to randomly assign 592 incarcerated men to screening modality. Findings indicate that computer screening was feasible. Compared with OAI, CAI produced equally reliable screening information on PTSD symptoms, with test–retest intraclass correlations for the PTSD Checklist (PCL) total score ranging from .774 to .817, and the Clinician-Administered PTSD scale and PCL scores were significantly correlated for OAI and CAI. These findings indicate that data on PTSD symptoms can be reliably and validly obtained from CAI technology, increasing the efficiency by which incarcerated populations can be screened for PTSD, and those at risk can be identified for treatment. PMID:25673900

The international Hip Outcome Tool-33 (iHOT-33): multicenter validation and translation to Spanish.

PubMed

Ruiz-Ibán, Miguel Angel; Seijas, Roberto; Sallent, Andrea; Ares, Oscar; Marín-Peña, Oliver; Muriel, Alfonso; Cuéllar, Ricardo

2015-05-20

The international Hip Outcome Tool-33 (iHOT-33) is a 33-item self administered outcome measure based on a Visual Analogue Scale response format designed for young and active population with hip pathology. The aim of the present study is to translate and validate the iHOT-33 into Spanish. 97 patients undergoing hip arthroscopy were included in this prospective and multicenter study performed between January 2012 and May 2014. Crosscultural adaptation was used to translate iHOT-33 into Spanish. Patients completed the questionnaire before and after surgery. Feasibility, reliability, internal consistency, construct validity (correlation with Western Ontario and McMaster Universities Osteoarthritis Index), ceiling and floor effects and sensitivity to change were assessed for the present study. Mean age was 48 years old. Feasibility: 41.2 % patients had no blank questions, and 71.3 % of patients had fulfilled all but one or two questions. Reliability: ICC for the global questionnaire was 0.97, showing that the questionnaire is highly reproducible. Internal consistency: Cronbach's alpha was 0.98 for the global questionnaire. Construct validity: there was a high correlation with WOMAC (correlation coefficient >0.5). The Ceiling effect (taking into account the minimum detectable change) was 12.1 % and the floor effect was 21.6 %, for the global questionnaire. Large sensitivity to change was shown. the Spanish version of iHOT-33 has shown to be feasible, reliable and sensible to changes for patients undergoing hip arthroscopy. This validated translation of iHOT-33 allows for comparisons between studies involving either Spanish- or English-speaking patients. Prognostic study, Level I.

[Spanish validation of the MacArthur Competence Assessment Tool for Treatment interview to assess patients competence to consent treatment].

PubMed

Alvarez Marrodán, Ignacio; Baón Pérez, Beatriz; Navío Acosta, Mercedes; López-Antón, Raul; Lobo Escolar, Elena; Ventura Faci, Tirso

2014-09-09

To validate the MacArthur Competence Assessment Tool for Treatment (MacCAT-T) Spanish version, which assesses the mental capacity of patients to consent treatment, by examining 4 areas (Understanding, Appreciation, Reasoning and Expressing a choice). 160 subjects (80 Internal Medicine inpatients, 40 Psychiatric inpatients and 40 healthy controls). MacCAT-T, Mini-Mental Status Examination (MMSE). Feasibility study, reliability and validity calculations (against to gold standard of clinical expert). Mean duration of the MacCAT-T interview was 18min. Inter-rater reliability: Intraclass correlation coefficient for Understanding=0.98, Appreciation=0.97, Reasoning=0.98, Expressing a choice=0.91. Internal consistency (Cronbach's alpha): Understanding=0.87, for Appreciation=0.76, for Reasoning=0.86. Patients considered to be incapable (gold standard) scored lower in all the MacCAT-T areas. Poor performance on the MacCAT-T was related to cognitive impairment assessed by MMSE. Spanish version of the MacCAT-T is feasible, reliable, and valid for assessing the capacity of patients to consent treatment. Copyright © 2013 Elsevier España, S.L. All rights reserved.

Puzzling with online games (BAM-COG): reliability, validity, and feasibility of an online self-monitor for cognitive performance in aging adults.

PubMed

Aalbers, Teun; Baars, Maria A E; Olde Rikkert, Marcel G M; Kessels, Roy P C

2013-12-03

Online interventions are aiming increasingly at cognitive outcome measures but so far no easy and fast self-monitors for cognition have been validated or proven reliable and feasible. This study examines a new instrument called the Brain Aging Monitor-Cognitive Assessment Battery (BAM-COG) for its alternate forms reliability, face and content validity, and convergent and divergent validity. Also, reference values are provided. The BAM-COG consists of four easily accessible, short, yet challenging puzzle games that have been developed to measure working memory ("Conveyer Belt"), visuospatial short-term memory ("Sunshine"), episodic recognition memory ("Viewpoint"), and planning ("Papyrinth"). A total of 641 participants were recruited for this study. Of these, 397 adults, 40 years and older (mean 54.9, SD 9.6), were eligible for analysis. Study participants played all games three times with 14 days in between sets. Face and content validity were based on expert opinion. Alternate forms reliability (AFR) was measured by comparing scores on different versions of the BAM-COG and expressed with an intraclass correlation (ICC: two-way mixed; consistency at 95%). Convergent validity (CV) was provided by comparing BAM-COG scores to gold-standard paper-and-pencil and computer-assisted cognitive assessment. Divergent validity (DV) was measured by comparing BAM-COG scores to the National Adult Reading Test IQ (NART-IQ) estimate. Both CV and DV are expressed as Spearman rho correlation coefficients. Three out of four games showed adequate results on AFR, CV, and DV measures. The games Conveyer Belt, Sunshine, and Papyrinth have AFR ICCs of .420, .426, and .645 respectively. Also, these games had good to very good CV correlations: rho=.577 (P=.001), rho=.669 (P<.001), and rho=.400 (P=.04), respectively. Last, as expected, DV correlations were low: rho=-.029 (P=.44), rho=-.029 (P=.45), and rho=-.134 (P=.28) respectively. The game Viewpoint provided less desirable results with an AFR ICC of .167, CV rho=.202 (P=.15), and DV rho=-.162 (P=.21). This study provides evidence for the use of the BAM-COG test battery as a feasible, reliable, and valid tool to monitor cognitive performance in healthy adults in an online setting. Three out of four games have good psychometric characteristics to measure working memory, visuospatial short-term memory, and planning capacity.

Non-Technical Skills for Surgeons (NOTSS): Critical appraisal of its measurement properties.

PubMed

Jung, James J; Borkhoff, Cornelia M; Jüni, Peter; Grantcharov, Teodor P

2018-02-17

To critically appraise the development and measurement properties, including sensibility, reliability, and validity of the Non-Technical Skills of Surgeons (NOTSS) system. Articles that described development process of the NOTSS system were identified. Relevant primary studies that presented evidence of reliability and validity were identified through a comprehensive literature review. NOTSS was developed through robust item generation and reduction strategies. It was shown to have good content validity, acceptability, and feasibility. Inter-rater reliability increased with greater expertise and number of assessors. Studies demonstrated evidence of cross-sectional construct validity, in that the tool was able to differentiate known groups of varied non-technical skill levels. Evidence of longitudinal construct validity also existed to demonstrate that NOTSS detected changes in non-technical skills before and after targeted training. In populations and settings presented in our critical appraisal, NOTSS provided reliable and valid measurements of intraoperative non-technical skills of surgeons. Copyright © 2018 Elsevier Inc. All rights reserved.

Validity and reliability of Internet-based physiotherapy assessment for musculoskeletal disorders: a systematic review.

PubMed

Mani, Suresh; Sharma, Shobha; Omar, Baharudin; Paungmali, Aatit; Joseph, Leonard

2017-04-01

Purpose The purpose of this review is to systematically explore and summarise the validity and reliability of telerehabilitation (TR)-based physiotherapy assessment for musculoskeletal disorders. Method A comprehensive systematic literature review was conducted using a number of electronic databases: PubMed, EMBASE, PsycINFO, Cochrane Library and CINAHL, published between January 2000 and May 2015. The studies examined the validity, inter- and intra-rater reliabilities of TR-based physiotherapy assessment for musculoskeletal conditions were included. Two independent reviewers used the Quality Appraisal Tool for studies of diagnostic Reliability (QAREL) and the Quality Assessment of Diagnostic Accuracy Studies (QUADAS) tool to assess the methodological quality of reliability and validity studies respectively. Results A total of 898 hits were achieved, of which 11 articles based on inclusion criteria were reviewed. Nine studies explored the concurrent validity, inter- and intra-rater reliabilities, while two studies examined only the concurrent validity. Reviewed studies were moderate to good in methodological quality. The physiotherapy assessments such as pain, swelling, range of motion, muscle strength, balance, gait and functional assessment demonstrated good concurrent validity. However, the reported concurrent validity of lumbar spine posture, special orthopaedic tests, neurodynamic tests and scar assessments ranged from low to moderate. Conclusion TR-based physiotherapy assessment was technically feasible with overall good concurrent validity and excellent reliability, except for lumbar spine posture, orthopaedic special tests, neurodynamic testa and scar assessment.

Measurement properties, feasibility and clinical utility of the Doloplus-2 pain scale in older adults with cognitive impairment: a systematic review.

PubMed

Rostad, Hanne Marie; Utne, Inger; Grov, Ellen Karine; Puts, Martine; Halvorsrud, Liv

2017-11-02

The Doloplus-2 is a pain assessment scale for assessing pain in older adults with cognitive impairment. It is used in clinical practice and research. However, evidence for its measurement properties, feasibility and clinical utility remain incomplete. This systematic review synthesizes previous research on the measurement properties, feasibility and clinical utility of the scale. We conducted a systematic search in three databases (CINAHL, Medline and PsycINFO) for studies published in English, French, German, Dutch/Flemish or a Scandinavian language between 1990 and April 2017. We also reviewed the Doloplus-2 homepage and reference lists of included studies to supplement our search. Two reviewers independently reviewed titles and abstracts and performed the quality assessment and data abstraction. A total of 24 studies were included in this systematic review. The quality of the studies varied, but many lacked sufficient detail about the samples and response rates. The Doloplus-2 has been studied using diverse samples in a variety of settings; most study participants were in long-term care and in people with dementia. Sixteen studies addressed various aspects of the scale's feasibility and clinical utility, but their results are limited and inconsistent across settings and samples. Support for the scale's reliability, validity and responsiveness varied widely across the studies. Generally, the reliability coefficients reached acceptable benchmarks, but the evidence for different aspects of the scale's validity and responsiveness was incomplete. Additional high-quality studies are warranted to determine in which populations of older adults with cognitive impairment the Doloplus-2 is reliable, valid and feasible. The ability of the Doloplus-2 to meaningfully quantify pain, measure treatment response and improve patient outcomes also needs further investigation. PROSPERO reg. no.: CRD42016049697 registered 20. Oct. 2016.

PedsQL™ Multidimensional Fatigue Scale in sickle cell disease: feasibility, reliability, and validity.

PubMed

Panepinto, Julie A; Torres, Sylvia; Bendo, Cristiane B; McCavit, Timothy L; Dinu, Bogdan; Sherman-Bien, Sandra; Bemrich-Stolz, Christy; Varni, James W

2014-01-01

Sickle cell disease (SCD) is an inherited blood disorder characterized by a chronic hemolytic anemia that can contribute to fatigue and global cognitive impairment in patients. The study objective was to report on the feasibility, reliability, and validity of the PedsQL™ Multidimensional Fatigue Scale in SCD for pediatric patient self-report ages 5-18 years and parent proxy-report for ages 2-18 years. This was a cross-sectional multi-site study whereby 240 pediatric patients with SCD and 303 parents completed the 18-item PedsQL™ Multidimensional Fatigue Scale. Participants also completed the PedsQL™ 4.0 Generic Core Scales. The PedsQL™ Multidimensional Fatigue Scale evidenced excellent feasibility, excellent reliability for the Total Scale Scores (patient self-report α = 0.90; parent proxy-report α = 0.95), and acceptable reliability for the three individual scales (patient self-report α = 0.77-0.84; parent proxy-report α = 0.90-0.97). Intercorrelations of the PedsQL™ Multidimensional Fatigue Scale with the PedsQL™ Generic Core Scales were predominantly in the large (≥0.50) range, supporting construct validity. PedsQL™ Multidimensional Fatigue Scale Scores were significantly worse with large effects sizes (≥0.80) for patients with SCD than for a comparison sample of healthy children, supporting known-groups discriminant validity. Confirmatory factor analysis demonstrated an acceptable to excellent model fit in SCD. The PedsQL™ Multidimensional Fatigue Scale demonstrated acceptable to excellent measurement properties in SCD. The results demonstrate the relative severity of fatigue symptoms in pediatric patients with SCD, indicating the potential clinical utility of multidimensional assessment of fatigue in patients with SCD in clinical research and practice. © 2013 Wiley Periodicals, Inc.

PedsQL™ Multidimensional Fatigue Scale in Sickle Cell Disease: Feasibility, Reliability and Validity

PubMed Central

Panepinto, Julie A.; Torres, Sylvia; Bendo, Cristiane B.; McCavit, Timothy L.; Dinu, Bogdan; Sherman-Bien, Sandra; Bemrich-Stolz, Christy; Varni, James W.

2013-01-01

Background Sickle cell disease (SCD) is an inherited blood disorder characterized by a chronic hemolytic anemia that can contribute to fatigue and global cognitive impairment in patients. The study objective was to report on the feasibility, reliability, and validity of the PedsQL™ Multidimensional Fatigue Scale in SCD for pediatric patient self-report ages 5–18 years and parent proxy-report for ages 2–18 years. Procedure This was a cross-sectional multi-site study whereby 240 pediatric patients with SCD and 303 parents completed the 18-item PedsQL™ Multidimensional Fatigue Scale. Participants also completed the PedsQL™ 4.0 Generic Core Scales. Results The PedsQL™ Multidimensional Fatigue Scale evidenced excellent feasibility, excellent reliability for the Total Scale Scores (patient self-report α = 0.90; parent proxy-report α = 0.95), and acceptable reliability for the three individual scales (patient self-report α = 0.77–0.84; parent proxy-report α = 0.90–0.97). Intercorrelations of the PedsQL™ Multidimensional Fatigue Scale with the PedsQL™ Generic Core Scales were predominantly in the large (≥ 0.50) range, supporting construct validity. PedsQL™ Multidimensional Fatigue Scale Scores were significantly worse with large effects sizes (≥0.80) for patients with SCD than for a comparison sample of healthy children, supporting known-groups discriminant validity. Confirmatory factor analysis demonstrated an acceptable to excellent model fit in SCD. Conclusions The PedsQL™ Multidimensional Fatigue Scale demonstrated acceptable to excellent measurement properties in SCD. The results demonstrate the relative severity of fatigue symptoms in pediatric patients with SCD, indicating the potential clinical utility of multidimensional assessment of fatigue in patients with SCD in clinical research and practice. PMID:24038960

The development of an integrated assessment instrument for measuring analytical thinking and science process skills

NASA Astrophysics Data System (ADS)

Irwanto, Rohaeti, Eli; LFX, Endang Widjajanti; Suyanta

2017-05-01

This research aims to develop instrument and determine the characteristics of an integrated assessment instrument. This research uses 4-D model, which includes define, design, develop, and disseminate. The primary product is validated by expert judgment, tested it's readability by students, and assessed it's feasibility by chemistry teachers. This research involved 246 students of grade XI of four senior high schools in Yogyakarta, Indonesia. Data collection techniques include interview, questionnaire, and test. Data collection instruments include interview guideline, item validation sheet, users' response questionnaire, instrument readability questionnaire, and essay test. The results show that the integrated assessment instrument has Aiken validity value of 0.95. Item reliability was 0.99 and person reliability was 0.69. Teachers' response to the integrated assessment instrument is very good. Therefore, the integrated assessment instrument is feasible to be applied to measure the students' analytical thinking and science process skills.

Validation of Malayalam Version of National Comprehensive Cancer Network Distress Thermometer and its Feasibility in Oncology Patients.

PubMed

Biji, M S; Dessai, Sampada; Sindhu, N; Aravind, Sithara; Satheesan, B

2018-01-01

This study was designed to translate and validate the National Comprehensive Cancer Network (NCCN) distress thermometer (DT) in regional language " Malayalam" and to see the feasibility of using it in our patients. (1) To translate and validate the NCCN DT. (2) To study the feasibility of using validated Malayalam translated DT in Malabar Cancer center. This is a single-arm prospective observational study. The study was conducted at author's institution between December 8, 2015, and January 20, 2016 in the Department of Cancer Palliative Medicine. This was a prospective observational study carried out in two phases. In Phase 1, the linguistic validation of the NCCN DT was done. In Phase 2, the feasibility, face validity, and utility of the translated of NCCN DT in accordance with QQ-10 too was done. SPSS version 16 (SPSS Inc. Released 2007. SPSS for Windows, Version 16.0. Chicago, SPSS Inc.) was used for analysis. Ten patients were enrolled in Phase 2. The median age was 51.5 years and 40% of patients were male. All patients had completed at least basic education up to the primary level. The primary site of cancer was heterogeneous. The NCCN DT completion rate was 100%. The face validity, utility, reliability, and feasibility were 100%, 100%, 100%, and 90%, respectively. It can be concluded that the Malayalam validated DT has high face validity, utility, and it is feasible for its use.

Feasibility, Validity, and Reliability of the Italian Pediatric Quality of Life Inventory Multidimensional Fatigue Scale for Adults in Inpatients with Severe Obesity

PubMed Central

Manzoni, Gian Mauro; Rossi, Alessandro; Marazzi, Nicoletta; Agosti, Fiorenza; De Col, Alessandra; Pietrabissa, Giada; Castelnuovo, Gianluca; Molinari, Enrico; Sartorio, Allessandro

2018-01-01

Objective This study was aimed to examine the feasibility, validity, and reliability of the Italian Pediatric Quality of Life Inventory Multidimensional Fatigue Scale (PedsQL™ MFS) for adult inpatients with severe obesity. Methods 200 inpatients (81% females) with severe obesity (BMI ≥ 35 kg/m2) completed the PedsQL MFS (General Fatigue, Sleep/Rest Fatigue and Cognitive Fatigue domains), the Fatigue Severity Scale, and the Center for Epidemiologic Studies Depression Scale immediately after admission to a 3-week residential body weight reduction program. A randomized subsample of 48 patients re-completed the PedsQL MFS after 3 days. Results Confirmatory factor analysis showed that a modified hierarchical model with two items moved from the Sleep/Rest Fatigue domain to the General Fatigue domain and a second-order latent factor best fitted the data. Internal consistency and test-retest reliabilities were acceptable to high in all scales, and small to high statistically significant correlations were found with all convergent measures, with the exception of BMI. Significant floor effects were found in two scales (Cognitive Fatigue and Sleep/Rest Fatigue). Conclusion The Italian modified PedsQL MFS for adults showed to be a valid and reliable tool for the assessment of fatigue in inpatients with severe obesity. Future studies should assess its discriminant validity as well as its responsiveness to weight reduction. PMID:29402854

The PedsQL™ Oral Health Scale: feasibility, reliability and validity of the Brazilian Portuguese version

PubMed Central

2012-01-01

Background Oral and orofacial problems may cause a profound impact on children’s oral health-related quality of life (OHRQoL) because of symptoms associated with these conditions that may influence the physical, psychological and social aspects of their daily life. The OHRQoL questionnaires found in the literature are very specific and are not able to measure the impact of oral health on general health domains. Consequently, the objective of this study was to evaluate the psychometric properties of the Portuguese version for Brazilian translation of the Pediatric Quality of Life Inventory™ (PedsQL™) Oral Health Scale in combination with the PedsQL™ 4.0 Generic Core Scales. Methods The PedsQL™ Oral Health Scale was forward-backward translated and cross-culturally adapted for the Brazilian Portuguese language. In order to assess the feasibility, reliability and validity of the Brazilian version of the instrument, a study was carried out in Belo Horizonte with 208 children and adolescents between 2 and 18 years-of-age and their parents. Clinical evaluation of dental caries, socioeconomic information and the Brazilian versions of the PedsQL™ Oral Health Scale, PedsQL™ 4.0 Generic Core Scales, Child Perceptions Questionnaire (CPQ11-14 and CPQ8-10) and Parental-Caregiver Perception Questionnaire (P-CPQ) were administered. Statistical analysis included feasibility (missing values), confirmatory factor analysis (CFA), internal consistency reliability, and test-retest intraclass correlation coefficients (ICC) of the PedsQL™ Oral Health Scale. Results There were no missing data for both child self-report and parent proxy-report on the Brazilian version of the PedsQL™ Oral Health Scale. The CFA showed that the five items of child self-report and parent proxy-report loaded on a single construct. The Cronbach's alpha coefficients for child/adolescent and parent oral health instruments were 0.65 and 0.59, respectively. The test-retest reliability (ICC) for child self-report and parent proxy-report were 0.90 [95% confidence interval (CI) = 0.86-0.93] and 0.86 (95%CI = 0.81-0.90), respectively. The PedsQL™ Oral Health Scale demonstrated acceptable construct validity, convergent validity and discriminant validity. Conclusions These results supported the feasibility, reliability and validity of the Brazilian version of the PedsQL™ Oral Health Scale for child self-report for ages 5–18 years-old and parent proxy-report for ages 2–18 years-old children. PMID:22531004

Tobacco Use Prevention for the Young (TUPY-S): Development, Validity and Reliability of an Interactive Multimedia Strategy from the Adolescents’ Perspective in Malaysia

PubMed Central

Zin, Faridah Mohd; Hillaluddin, Azlin Hilma; Mustaffa, Jamaludin

2017-01-01

Objective: This study aims to develop, validate and determine the reliability of an interactive multimedia strategy to prevent tobacco use among the young (TUPY-S) from an adolescents’ perspective. Methods: A descriptive study design was utilized. A modular instruction guideline by Russel (1974) was followed in the entire process, comprising a feasibility study, a review of existing modules, specification of the objectives, identification of the construct criterion items, learner analysis and entry behavior specification, establishment of the sequence instruction and media selection, a tryout with students and a field test. Result: Feasibility was agreed among the researchers and the school authorities. Culturally suitable rigorously developed tobacco use preventive strategies delivered using information technology (IT) are lacking in the literature. The objective of TUPY-S is to prevent tobacco use among adolescents living in Malaysia. Identified construct criterion items include knowledge, attitude, intention to use, self-efficacy, and refusal skill. The target population was early adolescents belonging to generation-Z. Content was developed from the adolescents’ perspective and delivered using IT in Malay language. Content validity, assessed by six experts in the field and module development, was good at 86%. The students’ tryout showed satisfactory face validity subjectively and objectively (85.5%) and high alpha Cronbach reliability (0.91). Conclusion: TUPY-S was confirmed to suit early adolescents of the current generation living in Malaysia. It demonstrated good content validity among the experts, satisfactory face validity and reliability among the target population. TUPY-S is ready to be evaluated for its effectiveness among early adolescents. PMID:28612599

Questionnaire to assess patient satisfaction with pharmaceutical care in Spanish language.

PubMed

Traverso, María Luz; Salamano, Mercedes; Botta, Carina; Colautti, Marisel; Palchik, Valeria; Pérez, Beatriz

2007-08-01

To develop and validate a questionnaire, in Spanish, for assessing patient satisfaction with pharmaceutical care received in community pharmacies. Selection and translation of questionnaire's items; definition of response scale and demographic questions. Evaluation of face and content validity, feasibility, factor structure, reliability and construct validity. Forty-one community pharmacies of the province of Santa Fe. Argentina. Questionnaire administered to patients receiving pharmaceutical care or traditional pharmacy services. Pilot test to assess feasibility. Factor analysis used principal components and varimax rotation. Reliability established using internal consistency with Cronbach's alpha. Construct validity determined with extreme group method. A self-administered questionnaire with 27 items, 5-point Likert response scale and demographic questions was designed considering multidimensional structure of patient satisfaction. Questionnaire evaluates cumulative experience of patients with comprehensive pharmaceutical care practice in community pharmacies. Two hundred and seventy-four complete questionnaires were obtained. Factor analysis resulted in three factors: Managing therapy, Interpersonal relationship and General satisfaction, with a cumulative variance of 62.51%. Cronbach's alpha for the whole questionnaire was 0.96, and 0.95, 0.88 and 0.76 for the three factors, respectively. Mann-Whitney test for construct validity did not showed significant differences between pharmacies that provide pharmaceutical care and those that do not, however, 23 items showed significant differences between the two groups of pharmacies. The questionnaire developed can be a reliable and valid instrument to assess patient satisfaction with pharmaceutical care in community pharmacies in Spanish. Further research is needed to deepen the validation process.

The PedsQL 4.0 as a pediatric population health measure: feasibility, reliability, and validity.

PubMed

Varni, James W; Burwinkle, Tasha M; Seid, Michael; Skarr, Douglas

2003-01-01

The application of health-related quality of life (HRQOL) as a pediatric population health measure may facilitate risk assessment and resource allocation, the tracking of community health, the identification of health disparities, and the determination of health outcomes from interventions and policy decisions. To determine the feasibility, reliability, and validity of the 23-item PedsQL 4.0 (Pediatric Quality of Life Inventory) Generic Core Scales as a measure of pediatric population health for children and adolescents. Mail survey in February and March 2001 to 20 031 families with children ages 2-16 years throughout the State of California encompassing all new enrollees in the State's Children's Health Insurance Program (SCHIP) for those months and targeted language groups. The PedsQL 4.0 Generic Core Scales (Physical, Emotional, Social, School Functioning) were completed by 10 241 families through a statewide mail survey to evaluate the HRQOL of new enrollees in SCHIP. The PedsQL 4.0 evidenced minimal missing responses, achieved excellent reliability for the Total Scale Score (alpha =.89 child;.92 parent report), and distinguished between healthy children and children with chronic health conditions. The PedsQL 4.0 was also related to indicators of health care access, days missed from school, days sick in bed or too ill to play, and days needing care. The results demonstrate the feasibility, reliability, and validity of the PedsQL 4.0 as a pediatric population health outcome. Measuring pediatric HRQOL may be a way to evaluate the health outcomes of SCHIP.

The PedsQL 4.0 as a school population health measure: feasibility, reliability, and validity.

PubMed

Varni, James W; Burwinkle, Tasha M; Seid, Michael

2006-03-01

The application of health-related quality of life (HRQOL) as a school population health measure may facilitate risk assessment and resource allocation, the tracking of student health at the school and district level, the identification of health disparities among schoolchildren, and the determination of health outcomes from interventions and policy decisions at the school, district, and county level. To determine the feasibility, reliability, and validity of the 23-item PedsQL 4.0 (Pediatric Quality of Life Inventory) Generic Core Scales as a school population health measure for children and adolescents. Survey conducted in 304 classes at 18 elementary schools, 4 middle schools, and 3 high schools within a large metropolitan school district. The PedsQL 4.0 Generic Core Scales (Physical, Emotional, Social, School Functioning) were completed by 2437 children ages 8-18 and 4227 parents of children ages 5-18. The PedsQL 4.0 evidenced minimal missing responses, achieved excellent reliability for the Total Scale Score (alpha = 0.89 child, 0.92, parent report), and distinguished between healthy children and children with chronic health conditions. The PedsQL 4.0 was related to indicators of socioeconomic status (SES) at the school and district level. The PedsQL School Functioning Scale was significantly correlated with standardized achievement scores based on the Stanford 9. The results demonstrate the feasibility, reliability and validity of the PedsQL 4.0 Generic Core Scales as a school population health measure. The implications of measuring HRQOL in schoolchildren at the school, district, and county level for identifying and ameliorating health disparities are discussed.

Feasibility, reliability, and validity of a smartphone based application for the assessment of cognitive function in the elderly.

PubMed

Brouillette, Robert M; Foil, Heather; Fontenot, Stephanie; Correro, Anthony; Allen, Ray; Martin, Corby K; Bruce-Keller, Annadora J; Keller, Jeffrey N

2013-01-01

While considerable knowledge has been gained through the use of established cognitive and motor assessment tools, there is a considerable interest and need for the development of a battery of reliable and validated assessment tools that provide real-time and remote analysis of cognitive and motor function in the elderly. Smartphones appear to be an obvious choice for the development of these "next-generation" assessment tools for geriatric research, although to date no studies have reported on the use of smartphone-based applications for the study of cognition in the elderly. The primary focus of the current study was to assess the feasibility, reliability, and validity of a smartphone-based application for the assessment of cognitive function in the elderly. A total of 57 non-demented elderly individuals were administered a newly developed smartphone application-based Color-Shape Test (CST) in order to determine its utility in measuring cognitive processing speed in the elderly. Validity of this novel cognitive task was assessed by correlating performance on the CST with scores on widely accepted assessments of cognitive function. Scores on the CST were significantly correlated with global cognition (Mini-Mental State Exam: r = 0.515, p<0.0001) and multiple measures of processing speed and attention (Digit Span: r = 0.427, p<0.0001; Trail Making Test: r = -0.651, p<0.00001; Digit Symbol Test: r = 0.508, p<0.0001). The CST was not correlated with naming and verbal fluency tasks (Boston Naming Test, Vegetable/Animal Naming) or memory tasks (Logical Memory Test). Test re-test reliability was observed to be significant (r = 0.726; p = 0.02). Together, these data are the first to demonstrate the feasibility, reliability, and validity of using a smartphone-based application for the purpose of assessing cognitive function in the elderly. The importance of these findings for the establishment of smartphone-based assessment batteries of cognitive and motor function in the elderly is discussed.

Validation of the Spanish version of the Hip Outcome Score: a multicenter study.

PubMed

Seijas, Roberto; Sallent, Andrea; Ruiz-Ibán, Miguel Angel; Ares, Oscar; Marín-Peña, Oliver; Cuéllar, Ricardo; Muriel, Alfonso

2014-05-13

The Hip Outcome Score (HOS) is a self-reported questionnaire evaluating the outcomes of treatment interventions for hip pathologies, divided in 19 items of activities of daily life (ADL) and 9 sports' items. The aim of the present study is to translate and validate HOS into Spanish. A prospective and multicenter study with 100 patients undergoing hip arthroscopy was performed between June 2012 and January 2013. Crosscultural adaptation was used to translate HOS into Spanish. Patients completed the questionnaire before and after surgery. Feasibility, reliability, internal consistency, construct validity (correlation with Western Ontario and McMaster Universities Osteoarthritis Index), ceiling and floor effects and sensitivity to change were assessed for the present study. Mean age was 45.05 years old. 36 women and 64 men were included. Feasibility: 13% had at least one missing item within the ADL subscale and 17% within the sport subscale. Reliability: the translated version of HOS was highly reproducible with intraclass correlation coefficient of 0.95 for ADL and 0.94 for the sports subscale. Internal consistency was confirmed with Cronbach's alpha >0.90 in both subscales. Construct validity showed statistically significant correlation with WOMAC. Ceiling effect was observed in 6% and 12% for ADL and sports subscale, respectively. Floor effect was found in 3% and 37% ADL and sports subscale, respectively. Large sensitivity to change was shown in both subscales. The translated version of HOS into Spanish has shown to be feasible, reliable and sensible to changes for patients undergoing hip arthroscopy. This validated translation of HOS allows for comparisons between studies involving either Spanish- or English-speaking patients. Prognostic study, Level I.

The Imperial Paediatric Emergency Training Toolkit (IPETT) for use in paediatric emergency training: development and evaluation of feasibility and validity.

PubMed

Lambden, Simon; DeMunter, Claudine; Dowson, Anne; Cooper, Mehrengise; Gautama, Sanjay; Sevdalis, Nick

2013-06-01

To develop and test the feasibility, reliability, and validity of a practical toolkit for the assessment and feedback of skills required to manage paediatric emergencies in critical care settings. The Imperial Paediatric Emergency Training Toolkit (IPETT) was developed based on current evidence-base and expert input. IPETT assesses both technical and non-technical skills. The technical component covers skills in the areas of clinical assessment, airway and breathing, cardiovascular, and drugs. The non-technical component is based on the validated NOTECHS tool and covers communication and interaction, cooperation and team skills, leadership and managerial skills, and decision-making. The reliability (internal consistency), content validity (inter-correlations between different skills) and concurrent validity (correlations between global technical and non-technical scores) of IPETT were prospectively evaluated in 45 simulated paediatric crises carried out in a PICU with anaesthetic and paediatric trainees (N=52). Non-parametric analyses were carried out. Significance was set at P<0.05. Cronbach alpha reliability coefficients were overall acceptable for the technical (alpha range=0.638-0.810) and good for the non-technical (alpha range=0.701-0.899) component of IPETT. The median inter-skill correlation was rho=0.564 and rho=0.549 for the technical and non-technical components, respectively. These indicate good content validity, as the skills were inter-related but not redundant. We also demonstrate a correlation between the global technical and non-technical scores (rho=0.471) - all Ps<0.05 during the assessments. IPETT offers a psychometrically viable and feasible to use tool in the context of paediatric emergencies training. This study shows that assessment of technical and non-technical skills in combination may offer a more clinically relevant model for training in paediatric emergencies. Further validation should aim to demonstrate skill retention over time and skill transfer from simulation-based training to real emergencies. Copyright © 2013. Published by Elsevier Ireland Ltd.

Good Evaluation Measures: More than Their Psychometric Properties

ERIC Educational Resources Information Center

Weitzman, Beth C.; Silver, Diana

2013-01-01

In this commentary, we examine Braverman's insights into the trade-offs between feasibility and rigor in evaluation measures and reject his assessment of the trade-off as a zero-sum game. We, argue that feasibility and policy salience are, like reliability and validity, intrinsic to the definition of a good measure. To reduce the tension between…

National audit of continence care: laying the foundation.

PubMed

Mian, Sarah; Wagg, Adrian; Irwin, Penny; Lowe, Derek; Potter, Jonathan; Pearson, Michael

2005-12-01

National audit provides a basis for establishing performance against national standards, benchmarking against other service providers and improving standards of care. For effective audit, clinical indicators are required that are valid, feasible to apply and reliable. This study describes the methods used to develop clinical indicators of continence care in preparation for a national audit. To describe the methods used to develop and test clinical indicators of continence care with regard to validity, feasibility and reliability. A multidisciplinary working group developed clinical indicators that measured the structure, process and outcome of care as well as case-mix variables. Literature searching, consensus workshops and a Delphi process were used to develop the indicators. The indicators were tested in 15 secondary care sites, 15 primary care sites and 15 long-term care settings. The process of development produced indicators that received a high degree of consensus within the Delphi process. Testing of the indicators demonstrated an internal reliability of 0.7 and an external reliability of 0.6. Data collection required significant investment in terms of staff time and training. The method used produced indicators that achieved a high degree of acceptance from health care professionals. The reliability of data collection was high for this audit and was similar to the level seen in other successful national audits. Data collection for the indicators was feasible to collect, however, issues of time and staffing were identified as limitations to such data collection. The study has described a systematic method for developing clinical indicators for national audit. The indicators proved robust and reliable in primary and secondary care as well as long-term care settings.

Validity, reliability and feasibility of commercially available activity trackers in physical therapy for people with a chronic disease: a study protocol of a mixed methods research.

PubMed

Beekman, Emmylou; Braun, Susy M; Ummels, Darcy; van Vijven, Kim; Moser, Albine; Beurskens, Anna J

2017-01-01

For older people and people with a chronic disease, physical activity provides health benefits. Patients and healthcare professionals can use commercially available activity trackers to objectively monitor (alterations in) activity levels and patterns and to support physical activity. However, insight in the validity, reliability, and feasibility of these trackers in people with a chronic disease is needed. In this article, a study protocol is described in which the validity, reliability (part A), and feasibility from a patient and therapist's point of view (part B) of commercially available activity trackers in daily life and health care is investigated. In part A, a quantitative cross-sectional study, an activity protocol that simulates everyday life activities will be used to determine the validity and reliability of nine commercially available activity trackers. Video recordings will act as the gold standard. In part B, a qualitative participatory action research study will be performed to gain insight in the use of activity trackers in peoples' daily life and therapy settings. Objective feasibility of the activity trackers will be measured with questionnaires, and subjective feasibility (experiences) will be explored in a community of practice. Physical therapists ( n  = 8) will regularly meet during 6 months to learn from each other regarding the actual use of activity trackers in therapy. Therapists and patients ( n  = 48) will decide together which tracker will be used in therapy and for which purpose (e.g., monitoring, goal setting). Data from the therapist' and patients' experiences will be collected by interviews (individual and focus groups) and analyzed by a directed content analysis. At the time of submission, selection of activity trackers, development of the activity protocol, and the ethical approval process are finished. Data collection and data processing are ongoing. The relevance of the study as well as the advantages and disadvantages of several aspects of the chosen design are discussed. The results acquired from both study parts can be used to create decision aids that may assist therapists and people with a chronic disease in choosing a suitable activity tracker, and to facilitate use of these activity trackers in health care settings. Ethical approval has been obtained from two medical-ethical committees (nr. 15-N-109, 15-N-48 and MEC-15-07).

An adaptive semantic matching paradigm for reliable and valid language mapping in individuals with aphasia.

PubMed

Wilson, Stephen M; Yen, Melodie; Eriksson, Dana K

2018-04-17

Research on neuroplasticity in recovery from aphasia depends on the ability to identify language areas of the brain in individuals with aphasia. However, tasks commonly used to engage language processing in people with aphasia, such as narrative comprehension and picture naming, are limited in terms of reliability (test-retest reproducibility) and validity (identification of language regions, and not other regions). On the other hand, paradigms such as semantic decision that are effective in identifying language regions in people without aphasia can be prohibitively challenging for people with aphasia. This paper describes a new semantic matching paradigm that uses an adaptive staircase procedure to present individuals with stimuli that are challenging yet within their competence, so that language processing can be fully engaged in people with and without language impairments. The feasibility, reliability and validity of the adaptive semantic matching paradigm were investigated in sixteen individuals with chronic post-stroke aphasia and fourteen neurologically normal participants, in comparison to narrative comprehension and picture naming paradigms. All participants succeeded in learning and performing the semantic paradigm. Test-retest reproducibility of the semantic paradigm in people with aphasia was good (Dice coefficient = 0.66), and was superior to the other two paradigms. The semantic paradigm revealed known features of typical language organization (lateralization; frontal and temporal regions) more consistently in neurologically normal individuals than the other two paradigms, constituting evidence for validity. In sum, the adaptive semantic matching paradigm is a feasible, reliable and valid method for mapping language regions in people with aphasia. © 2018 Wiley Periodicals, Inc.

The reliability and validity of ultrasound to quantify muscles in older adults: a systematic review

PubMed Central

Scafoglieri, Aldo; Jager‐Wittenaar, Harriët; Hobbelen, Johannes S.M.; van der Schans, Cees P.

2017-01-01

Abstract This review evaluates the reliability and validity of ultrasound to quantify muscles in older adults. The databases PubMed, Cochrane, and Cumulative Index to Nursing and Allied Health Literature were systematically searched for studies. In 17 studies, the reliability (n = 13) and validity (n = 8) of ultrasound to quantify muscles in community‐dwelling older adults (≥60 years) or a clinical population were evaluated. Four out of 13 reliability studies investigated both intra‐rater and inter‐rater reliability. Intraclass correlation coefficient (ICC) scores for reliability ranged from −0.26 to 1.00. The highest ICC scores were found for the vastus lateralis, rectus femoris, upper arm anterior, and the trunk (ICC = 0.72 to 1.000). All included validity studies found ICC scores ranging from 0.92 to 0.999. Two studies describing the validity of ultrasound to predict lean body mass showed good validity as compared with dual‐energy X‐ray absorptiometry (r 2 = 0.92 to 0.96). This systematic review shows that ultrasound is a reliable and valid tool for the assessment of muscle size in older adults. More high‐quality research is required to confirm these findings in both clinical and healthy populations. Furthermore, ultrasound assessment of small muscles needs further evaluation. Ultrasound to predict lean body mass is feasible; however, future research is required to validate prediction equations in older adults with varying function and health. PMID:28703496

Puzzling With Online Games (BAM-COG): Reliability, Validity, and Feasibility of an Online Self-Monitor for Cognitive Performance in Aging Adults

PubMed Central

Baars, Maria A E; Olde Rikkert, Marcel G M; Kessels, Roy P C

2013-01-01

Background Online interventions are aiming increasingly at cognitive outcome measures but so far no easy and fast self-monitors for cognition have been validated or proven reliable and feasible. Objective This study examines a new instrument called the Brain Aging Monitor–Cognitive Assessment Battery (BAM-COG) for its alternate forms reliability, face and content validity, and convergent and divergent validity. Also, reference values are provided. Methods The BAM-COG consists of four easily accessible, short, yet challenging puzzle games that have been developed to measure working memory (“Conveyer Belt”), visuospatial short-term memory (“Sunshine”), episodic recognition memory (“Viewpoint”), and planning (“Papyrinth”). A total of 641 participants were recruited for this study. Of these, 397 adults, 40 years and older (mean 54.9, SD 9.6), were eligible for analysis. Study participants played all games three times with 14 days in between sets. Face and content validity were based on expert opinion. Alternate forms reliability (AFR) was measured by comparing scores on different versions of the BAM-COG and expressed with an intraclass correlation (ICC: two-way mixed; consistency at 95%). Convergent validity (CV) was provided by comparing BAM-COG scores to gold-standard paper-and-pencil and computer-assisted cognitive assessment. Divergent validity (DV) was measured by comparing BAM-COG scores to the National Adult Reading Test IQ (NART-IQ) estimate. Both CV and DV are expressed as Spearman rho correlation coefficients. Results Three out of four games showed adequate results on AFR, CV, and DV measures. The games Conveyer Belt, Sunshine, and Papyrinth have AFR ICCs of .420, .426, and .645 respectively. Also, these games had good to very good CV correlations: rho=.577 (P=.001), rho=.669 (P<.001), and rho=.400 (P=.04), respectively. Last, as expected, DV correlations were low: rho=−.029 (P=.44), rho=−.029 (P=.45), and rho=−.134 (P=.28) respectively. The game Viewpoint provided less desirable results with an AFR ICC of .167, CV rho=.202 (P=.15), and DV rho=−.162 (P=.21). Conclusions This study provides evidence for the use of the BAM-COG test battery as a feasible, reliable, and valid tool to monitor cognitive performance in healthy adults in an online setting. Three out of four games have good psychometric characteristics to measure working memory, visuospatial short-term memory, and planning capacity. PMID:24300212

The Trojan Lifetime Champions Health Survey: Development, Validity, and Reliability

PubMed Central

Sorenson, Shawn C.; Romano, Russell; Scholefield, Robin M.; Schroeder, E. Todd; Azen, Stanley P.; Salem, George J.

2015-01-01

Context Self-report questionnaires are an important method of evaluating lifespan health, exercise, and health-related quality of life (HRQL) outcomes among elite, competitive athletes. Few instruments, however, have undergone formal characterization of their psychometric properties within this population. Objective To evaluate the validity and reliability of a novel health and exercise questionnaire, the Trojan Lifetime Champions (TLC) Health Survey. Design Descriptive laboratory study. Setting A large National Collegiate Athletic Association Division I university. Patients or Other Participants A total of 63 university alumni (age range, 24 to 84 years), including former varsity collegiate athletes and a control group of nonathletes. Intervention(s) Participants completed the TLC Health Survey twice at a mean interval of 23 days with randomization to the paper or electronic version of the instrument. Main Outcome Measure(s) Content validity, feasibility of administration, test-retest reliability, parallel-form reliability between paper and electronic forms, and estimates of systematic and typical error versus differences of clinical interest were assessed across a broad range of health, exercise, and HRQL measures. Results Correlation coefficients, including intraclass correlation coefficients (ICCs) for continuous variables and κ agreement statistics for ordinal variables, for test-retest reliability averaged 0.86, 0.90, 0.80, and 0.74 for HRQL, lifetime health, recent health, and exercise variables, respectively. Correlation coefficients, again ICCs and κ, for parallel-form reliability (ie, equivalence) between paper and electronic versions averaged 0.90, 0.85, 0.85, and 0.81 for HRQL, lifetime health, recent health, and exercise variables, respectively. Typical measurement error was less than the a priori thresholds of clinical interest, and we found minimal evidence of systematic test-retest error. We found strong evidence of content validity, convergent construct validity with the Short-Form 12 Version 2 HRQL instrument, and feasibility of administration in an elite, competitive athletic population. Conclusions These data suggest that the TLC Health Survey is a valid and reliable instrument for assessing lifetime and recent health, exercise, and HRQL, among elite competitive athletes. Generalizability of the instrument may be enhanced by additional, larger-scale studies in diverse populations. PMID:25611315

Reliability and criterion validity of an observation protocol for working technique assessments in cash register work.

PubMed

Palm, Peter; Josephson, Malin; Mathiassen, Svend Erik; Kjellberg, Katarina

2016-06-01

We evaluated the intra- and inter-observer reliability and criterion validity of an observation protocol, developed in an iterative process involving practicing ergonomists, for assessment of working technique during cash register work for the purpose of preventing upper extremity symptoms. Two ergonomists independently assessed 17 15-min videos of cash register work on two occasions each, as a basis for examining reliability. Criterion validity was assessed by comparing these assessments with meticulous video-based analyses by researchers. Intra-observer reliability was acceptable (i.e. proportional agreement >0.7 and kappa >0.4) for 10/10 questions. Inter-observer reliability was acceptable for only 3/10 questions. An acceptable inter-observer reliability combined with an acceptable criterion validity was obtained only for one working technique aspect, 'Quality of movements'. Thus, major elements of the cashiers' working technique could not be assessed with an acceptable accuracy from short periods of observations by one observer, such as often desired by practitioners. Practitioner Summary: We examined an observation protocol for assessing working technique in cash register work. It was feasible in use, but inter-observer reliability and criterion validity were generally not acceptable when working technique aspects were assessed from short periods of work. We recommend the protocol to be used for educational purposes only.

Feasibility, Validity, and Reliability of the Italian Pediatric Quality of Life Inventory Multidimensional Fatigue Scale for Adults in Inpatients with Severe Obesity.

PubMed

Manzoni, Gian Mauro; Rossi, Alessandro; Marazzi, Nicoletta; Agosti, Fiorenza; De Col, Alessandra; Pietrabissa, Giada; Castelnuovo, Gianluca; Molinari, Enrico; Sartorio, Allessandro

2018-01-01

This study was aimed to examine the feasibility, validity, and reliability of the Italian Pediatric Quality of Life Inventory Multidimensional Fatigue Scale (PedsQL™ MFS) for adult inpatients with severe obesity. 200 inpatients (81% females) with severe obesity (BMI ≥ 35 kg/m2) completed the PedsQL MFS (General Fatigue, Sleep/Rest Fatigue and Cognitive Fatigue domains), the Fatigue Severity Scale, and the Center for Epidemiologic Studies Depression Scale immediately after admission to a 3-week residential body weight reduction program. A randomized subsample of 48 patients re-completed the PedsQL MFS after 3 days. Confirmatory factor analysis showed that a modified hierarchical model with two items moved from the Sleep/Rest Fatigue domain to the General Fatigue domain and a second-order latent factor best fitted the data. Internal consistency and test-retest reliabilities were acceptable to high in all scales, and small to high statistically significant correlations were found with all convergent measures, with the exception of BMI. Significant floor effects were found in two scales (Cognitive Fatigue and Sleep/Rest Fatigue). The Italian modified PedsQL MFS for adults showed to be a valid and reliable tool for the assessment of fatigue in inpatients with severe obesity. Future studies should assess its discriminant validity as well as its responsiveness to weight reduction. © 2018 The Author(s) Published by S. Karger GmbH, Freiburg.

AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews.

PubMed

Shea, Beverley J; Hamel, Candyce; Wells, George A; Bouter, Lex M; Kristjansson, Elizabeth; Grimshaw, Jeremy; Henry, David A; Boers, Maarten

2009-10-01

Our purpose was to measure the agreement, reliability, construct validity, and feasibility of a measurement tool to assess systematic reviews (AMSTAR). We randomly selected 30 systematic reviews from a database. Each was assessed by two reviewers using: (1) the enhanced quality assessment questionnaire (Overview of Quality Assessment Questionnaire [OQAQ]); (2) Sacks' instrument; and (3) our newly developed measurement tool (AMSTAR). We report on reliability (interobserver kappas of the 11 AMSTAR items), intraclass correlation coefficients (ICCs) of the sum scores, construct validity (ICCs of the sum scores of AMSTAR compared with those of other instruments), and completion times. The interrater agreement of the individual items of AMSTAR was substantial with a mean kappa of 0.70 (95% confidence interval [CI]: 0.57, 0.83) (range: 0.38-1.0). Kappas recorded for the other instruments were 0.63 (95% CI: 0.38, 0.78) for enhanced OQAQ and 0.40 (95% CI: 0.29, 0.50) for the Sacks' instrument. The ICC of the total score for AMSTAR was 0.84 (95% CI: 0.65, 0.92) compared with 0.91 (95% CI: 0.82, 0.96) for OQAQ and 0.86 (95% CI: 0.71, 0.94) for the Sacks' instrument. AMSTAR proved easy to apply, each review taking about 15 minutes to complete. AMSTAR has good agreement, reliability, construct validity, and feasibility. These findings need confirmation by a broader range of assessors and a more diverse range of reviews.

A Validation Study of Early Adolescents' Pubertal Self-Assessments

ERIC Educational Resources Information Center

Schmitz, Katharine E.; Hovell, Melbourne F.; Nichols, Jeanne F.; Irvin, Veronica L.; Keating, Kristen; Simon, Gayle M.; Gehrman, Christine; Jones, Kenneth Lee

2004-01-01

This study aimed to determine whether self-assessed puberty is sufficiently reliable and valid to substitute for physician examination when feasibility of physician examination is low (e.g., behavioral research). Adolescents (convenience sample N = 178 endocrinology patients and N = 125 from educational trial; mean age 12.7 and 11.3 years,…

The measurement of patient attitudes regarding prenatal and preconception genetic carrier screening and translational behavioral medicine: an integrative review.

PubMed

Shiroff, Jennifer J; Gregoski, Mathew J

2017-06-01

Measurement of recessive carrier screening attitudes related to conception and pregnancy is necessary to determine current acceptance, and whether behavioral intervention strategies are needed in clinical practice. To evaluate quantitative survey instruments to measure patient attitudes regarding genetic carrier testing prior to conception and pregnancy databases examining patient attitudes regarding genetic screening prior to conception and pregnancy from 2003-2013 were searched yielding 344 articles; eight studies with eight instruments met criteria for inclusion. Data abstraction on theoretical framework, subjects, instrument description, scoring, method of measurement, reliability, validity, feasibility, level of evidence, and outcomes was completed. Reliability information was provided in five studies with an internal consistency of Cronbach's α >0.70. Information pertaining to validity was presented in three studies and included construct validity via factor analysis. Despite limited psychometric information, these questionnaires are self-administered and can be briefly completed, making them a feasible method of evaluation.

A psychometric study of the multidimensional fatigue inventory to assess fatigue in patients with schizophrenia spectrum disorders.

PubMed

Hedlund, Lena; Gyllensten, Amanda Lundvik; Hansson, Lars

2015-04-01

Fatigue is frequently reported by patients with mental illness. The multidimensional fatigue inventory (MFI-20) is a self-assessment instrument with 20 items including five dimensions of fatigue. The purpose of this study was to examine the test-retest reliability, internal consistency, convergent construct validity and feasibility of using MFI-20 in patients with schizophrenia spectrum disorders. Patients completed two self-assessment instruments, MFI-20 (n = 93) and Visual Analogue Scale (n = 79), twice within 1 week ± 2 days. Fifty-three patients also rated the feasibility of responding to the MFI-20 with a Likert scale. The test-retest reliability and validity were analysed by using Spearman's correlations and internal consistency by calculating Cronbach's α. The test-retest showed a correlation between .66 and .91 for all subscales of MFI. The internal consistency was .92. The analysis of convergent construct validity showed a correlation of .68 (time 1) and .77 (time 2). No item was systematically identified as being difficult to answer.

Health-related quality of life in young adults in education, employment, or training: development of the Japanese version of Pediatric Quality of Life Inventory (PedsQL) Generic Core Scales Young Adult Version.

PubMed

Kaneko, Mei; Sato, Iori; Soejima, Takafumi; Kamibeppu, Kiyoko

2014-09-01

The purpose of the study is to develop a Japanese version of the Pediatric Quality of Life Inventory (PedsQL) Generic Core Scales Young Adult Version (PedsQL-YA-J) and determine the feasibility, reliability, and validity of the scales. Translation equivalence and content validity were verified using back-translation and cognitive debriefing tests. A total of 428 young adults recruited from one university, two vocational schools, or five companies completed questionnaires. We determined questionnaire feasibility, internal consistency, and test-retest reliability; checked concurrent validity against the Center for Epidemiologic Studies Depression Scale (CES-D); determined convergent and discriminant validity with the Medical Outcome Study 36-item Short Form Health Survey (SF-36); described known-groups validity with regard to subjective symptoms, illness or injury requiring regular medical visits, and depression; and verified factorial validity. All scales were internally consistent (Cronbach's coefficient alpha = 0.77-0.86); test-retest reliability was acceptable (intraclass correlation coefficient = 0.57-0.69); and all scales were concurrently valid with depression (Pearson's correlation coefficient = 0.43-0.57). The scales convergent and discriminant validity with the SF-36 and CES-D were acceptable. Evaluation of known-groups validity confirmed that the Physical Functioning scale was sensitive for subjective symptoms, the Emotional Functioning scale for depression, and the Work/School Functioning scale for illness or injury requiring regular medical visits. Exploratory factor analysis found a six-factor structure consistent with the assumed structure (cumulative proportion = 57.0%). The PedsQL-YA-J is suitable for assessing health-related quality of life in young adults in education, employment, or training, and for clinical trials and epidemiological research.

Preliminary validation of 2 magnetic resonance image scoring systems for osteoarthritis of the hip according to the OMERACT filter.

PubMed

Maksymowych, Walter P; Cibere, Jolanda; Loeuille, Damien; Weber, Ulrich; Zubler, Veronika; Roemer, Frank W; Jaremko, Jacob L; Sayre, Eric C; Lambert, Robert G W

2014-02-01

Development of a validated magnetic resonance image (MRI) scoring system is essential in hip OA because radiographs are insensitive to change. We assessed the feasibility and reliability of 2 previously developed scoring methods: (1) the Hip Inflammation MRI Scoring System (HIMRISS) and (2) the Hip Osteoarthritis MRI Scoring System (HOAMS). Six readers (3 radiologists, 3 rheumatologists) participated in 2 reading exercises. In Reading Exercise 1, MRI of the hip of 20 subjects were read at a single time point followed by further standardization of methodology. In Reading Exercise 2, MRI of the hip of 18 subjects from a randomized controlled trial, assessed at 2 timepoints, and 27 subjects from a cross-sectional study were read for HIMRISS and HOAMS bone marrow lesions (BML) and synovitis. Reliability was assessed using intraclass correlation coefficient (ICC) and kappa statistics. Both methods were considered feasible. For Reading 1, HIMRISS ICC were 0.52, 0.61, 0.70, and 0.58 for femoral BML, acetabular BML, effusion, and total scores, respectively; and for HOAMS, summed BML and synovitis ICC were 0.52 and 0.46, respectively. For Reading 2, HIMRISS and HOAMS ICC for BML and synovitis-effusion improved substantially. Interobserver reliability for change scores was 0.81 and 0.71 for HIMRISS femoral and HOAMS summed BML, respectively. Responsiveness and discrimination was moderate to high for synovitis-effusion. Significant associations were noted between BML or synovitis scores and Western Ontario and McMaster Universities Osteoarthritis Index pain scores for baseline values (p ≤ 0.001). The BML and synovitis-effusion components of both HIMRISS and HOAMS scoring systems are feasible and reliable, and should be validated further.

Health Auctions: a Valuation Experiment (HAVE) study protocol.

PubMed

Kularatna, Sanjeewa; Petrie, Dennis; Scuffham, Paul A; Byrnes, Joshua

2016-04-07

Quality-adjusted life years are derived using health state utility weights which adjust for the relative value of living in each health state compared with living in perfect health. Various techniques are used to estimate health state utility weights including time-trade-off and standard gamble. These methods have exhibited limitations in terms of complexity, validity and reliability. A new composite approach using experimental auctions to value health states is introduced in this protocol. A pilot study will test the feasibility and validity of using experimental auctions to value health states in monetary terms. A convenient sample (n=150) from a population of university staff and students will be invited to participate in 30 auction sets with a group of 5 people in each set. The 9 health states auctioned in each auction set will come from the commonly used EQ-5D-3L instrument. At most participants purchase 2 health states, and the participant who acquires the 2 'best' health states on average will keep the amount of money they do not spend in acquiring those health states. The value (highest bid and average bid) of each of the 24 health states will be compared across auctions to test for reliability across auction groups and across auctioneers. A test retest will be conducted for 10% of the sample to assess reliability of responses for health states auctions. Feasibility of conducting experimental auctions to value health states will also be examined. The validity of estimated health states values will be compared with published utility estimates from other methods. This pilot study will explore the feasibility, reliability and validity in using experimental auction for valuing health states. Ethical clearance was obtained from Griffith University ethics committee. The results will be disseminated in peer-reviewed journals and major international conferences. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/

Clinical composite measures of disease activity and damage used to evaluate patients with systemic lupus erythematosus: A systematic literature review.

PubMed

Castrejón, Isabel; Rúa-Figueroa, Iñigo; Rosario, María Piedad; Carmona, Loreto

2014-01-01

To determine the most appropriate indices to evaluate the disease activity and damage in patients with sytemic lupus erythematosus (SLE). A systematic literature search was performed to identify validation studies of indices used to evaluate disease activity and damage. We collected information for each instrument on every aspect of validation including feasibility, reliability, validity and sensitivity to change using ad hoc forms. A total of 38 articles were included addressing the validation of 6 composite indices to evaluate disease activity (BILAG, ECLAM, SLAM, SLEDAI, LAI and SLAQ); and 3 indices to evaluate damage (SLICC/ACE-DI, LDIQ and BILD). Only the SLAQ, LIDIQ and the BILD were self-administered. Feasibility and internal consistency was only studied in 3 indices (BILAG, SLAQ and SDI) with a Cronbach's α ranging from 0.35 to 0.87. The intra-observer reliability was examined by the intraclass correlation coefficient for BILAG with a result of 0.48 (95%CI: 0,23-0,81) and using analysis of variance for SLAM-R (0,78), SLEDAI (0,33) and the LAI (0,81). The inter-observer feasibility was evaluated using the correlation coefficient for ECLAM (0,90-0,93), the SLAM (0,86) and MEX-SLEDAI (0,97-0,89). The construct validity was examined by means of convergence with other instruments, specifically with global assessment by the physician, with similar results between indices (0,48-0,75). Lastly, responsiveness was tested in all indices except LAI, SDI and LDIQ, with a standardized response mean ranging from 0.12 to 0.75. Although multiple instruments have been validated for use in SLE it was not possible to find direct evidence of which is the most appropriate. BILAG and SLEDAI, with moderate feasibility and low responsiveness, are the 2 indices with a most complete validation and more extensively used. Copyright © 2013 Elsevier España, S.L.U. All rights reserved.

Feasibility of surveillance of changes in human fertility and semen quality.

PubMed

Stewart, T M; Brown, E H; Venn, A; Mbizvo, M T; Farley, T M; Garrett, C; Baker, H W

2001-01-01

There is concern that male fertility is declining, but this is difficult to study because few men volunteer for studies of semen quality, and recruitment bias may over-represent the subfertile. The Human Reproduction Programme of the World Health Organization developed a protocol for multicentre studies of fertility involving a questionnaire for pregnant women to obtain time to pregnancy (TTP): the number of menstrual cycles taken to conceive. Male characteristics and semen quality will be determined in a subset of the partners. Our aim was to validate the TTP questionnaire, and to examine potential recruitment bias and feasibility of conducting large-scale surveillance of fertility. The questionnaire was administered to 120 pregnant women (16-32 weeks). Validation included internal reliability by consistency of responses, test-re-test reliability by repeat administration (20 women) and accuracy by comparison of gestational age from first antenatal ultrasound and menstrual dates. Internal reliability was high. Agreement between categorical responses on re-testing was very good (k > 0.8). In both the re-test and gestational age analysis, differences in TTP of 1 cycle were found (standard deviation <0.25 cycles). In this small pilot study there was no evidence of recruitment bias. Response rates indicate the feasibility of surveillance of fertility in large maternity centres.

Assessing patient-centered care: one approach to health disparities education.

PubMed

Wilkerson, LuAnn; Fung, Cha-Chi; May, Win; Elliott, Donna

2010-05-01

Patient-centered care has been described as one approach to cultural competency education that could reduce racial and ethnic health disparities by preparing providers to deliver care that is respectful and responsive to the preferences of each patient. In order to evaluate the effectiveness of a curriculum in teaching patient-centered care (PCC) behaviors to medical students, we drew on the work of Kleinman, Eisenberg, and Good to develop a scale that could be embedded across cases in an objective structured clinical examination (OSCE). To compare the reliability, validity, and feasibility of an embedded patient-centered care scale with the use of a single culturally challenging case in measuring students' use of PCC behaviors as part of a comprehensive OSCE. A total of 322 students from two California medical schools participated in the OSCE as beginning seniors. Cronbach's alpha was used to assess the internal consistency of each approach. Construct validity was addressed by establishing convergent and divergent validity using the cultural challenge case total score and OSCE component scores. Feasibility assessment considered cost and training needs for the standardized patients (SPs). Medical students demonstrated a moderate level of patient-centered skill (mean = 63%, SD = 11%). The PCC Scale demonstrated an acceptable level of internal consistency (alpha = 0.68) over the single case scale (alpha = 0.60). Both convergent and divergent validities were established through low to moderate correlation coefficients. The insertion of PCC items across multiple cases in a comprehensive OSCE can provide a reliable estimate of students' use of PCC behaviors without incurring extra costs associated with implementing a special cross-cultural OSCE. This approach is particularly feasible when an OSCE is already part of the standard assessment of clinical skills. Reliability may be increased with an additional investment in SP training.

Tactical and operational response to major incidents: feasibility and reliability of skills assessment using novel virtual environments.

PubMed

Cohen, Daniel; Sevdalis, Nick; Patel, Vishal; Taylor, Michael; Lee, Henry; Vokes, Mick; Heys, Mick; Taylor, David; Batrick, Nicola; Darzi, Ara

2013-07-01

To determine feasibility and reliability of skills assessment in a multi-agency, triple-site major incident response exercise carried out in a virtual world environment. Skills assessment was carried out across three scenarios. The pre-hospital scenario required paramedics to triage and treat casualties at the site of an explosion. Technical skills assessment forms were developed using training syllabus competencies and national guidelines identified by pre-hospital response experts. Non-technical skills were assessed using a seven-point scale previously developed for use by pre-hospital paramedics. The two in-hospital scenarios, focusing on a trauma team leader and a silver/clinical major incident co-ordinator, utilised the validated Trauma-NOTECHS scale to assess five domains of performance. Technical competencies were assessed using an ATLS-style competency scale for the trauma scenario. For the silver scenario, the assessment document was developed using competencies described from a similar role description in a real-life hospital major incident plan. The technical and non-technical performance of all participants was assessed live by two experts in each of the three scenarios and inter-assessor reliability was computed. Participants also self-assessed their performance using identical proformas immediately after the scenarios were completed. Self and expert assessments were correlated (assessment cross-validation). Twenty-three participants underwent all scenarios and assessments. Performance assessments were feasible for both experts as well as the participants. Non-technical performance was generally scored higher than technical performance. Very good inter-rater reliability was obtained between expert raters across all scenarios and both technical and non-technical aspects of performance (reliability range 0.59-0.90, Ps<0.01). Significant positive correlations were found between self and expert assessment in technical skills across all three scenarios (correlation range 0.52-0.84, Ps<0.05), although no such correlations were observed in non-technical skills. This study establishes feasibility and reliability of virtual environment technical and non-technical skills assessment in major incident scenarios for the first time. The development for further scenarios and validated assessment scales will enable major incident planners to utilise virtual technologies for improved major incident preparation and training. Copyright © 2012 Elsevier Ireland Ltd. All rights reserved.

Improving the residency admissions process by integrating a professionalism assessment: a validity and feasibility study.

PubMed

Bajwa, Nadia M; Yudkowsky, Rachel; Belli, Dominique; Vu, Nu Viet; Park, Yoon Soo

2017-03-01

The purpose of this study was to provide validity and feasibility evidence in measuring professionalism using the Professionalism Mini-Evaluation Exercise (P-MEX) scores as part of a residency admissions process. In 2012 and 2013, three standardized-patient-based P-MEX encounters were administered to applicants invited for an interview at the University of Geneva Pediatrics Residency Program. Validity evidence was gathered for P-MEX content (item analysis); response process (qualitative feedback); internal structure (inter-rater reliability with intraclass correlation and Generalizability); relations to other variables (correlations); and consequences (logistic regression to predict admission). To improve reliability, Kane's formula was used to create an applicant composite score using P-MEX, structured letter of recommendation (SLR), and structured interview (SI) scores. Applicant rank lists using composite scores versus faculty global ratings were compared using the Wilcoxon signed-rank test. Seventy applicants were assessed. Moderate associations were found between pairwise correlations of P-MEX scores and SLR (r = 0.25, P = .036), SI (r = 0.34, P = .004), and global ratings (r = 0.48, P < .001). Generalizability of the P-MEX using three cases was moderate (G-coefficient = 0.45). P-MEX scores had the greatest correlation with acceptance (r = 0.56, P < .001), were the strongest predictor of acceptance (OR 4.37, P < .001), and increased pseudo R-squared by 0.20 points. Including P-MEX scores increased composite score reliability from 0.51 to 0.74. Rank lists of applicants using composite score versus global rating differed significantly (z = 5.41, P < .001). Validity evidence supports the use of P-MEX scores to improve the reliability of the residency admissions process by improving applicant composite score reliability.

Assessing the Generalizable Skills of Post-Secondary Vocational Students. A Validation Study.

ERIC Educational Resources Information Center

Greenan, James P.; Smith, Brandon B.

A study examined the feasibility, reliability, and validity of two instruments designed to assess the degree to which postsecondary vocational students possessed those generalizable skills that are believed to be functionally relevant to success in a vocational program. The instruments, a student self-rating and a teacher rating form, contained 81…

Assessment of the psychometric properties of the Spanish language version of questionnaire ICIQ-Male Lower Urinary Tract Symptoms (ICIQ-MLUTS).

PubMed

Castro-Díaz, D M; Esteban-Fuertes, M; Salinas-Casado, J; Bustamante-Alarma, S; Gago-Ramos, J L; Galacho-Bech, A; García-Matres, M J; Rodríguez-Toves, L A; Zubiaur-Líbano, C; Collado-Serra, A; Batista-Miranda, J E; Ortiz-Gámiz, A

2014-03-01

To evaluate the psychometric properties of the Spanish version of the ICIQ-Male Lower Urinary Tract Symptoms Questionnaire (ICIQ-MLUTS): Feasibility (% of completion and ceiling/ground effects), reliability (Test-retest), convergent validity (vs Bladder Control Self-Assessment Questionnaire [BSAQ] and vs International Prostate Symptom Score [I-PSS]) and criterion validity (according to presence or absence of symptoms). This was an observational, non-interventionist and multicenter study. 223 male patients with lower urinary tract symptoms (LUTS), predominantly storage symptoms and aged 18-65, took part in the study. Patients completed the ICIQ-MLUTS (test), I-PSS and BSAQ questionnaires and referred their urinary symptoms in a single visit, with the exception of a subgroup composed by 49 patients that completed the questionnaire again 15 days after initial visit to evaluate test-retest reliability. The questionnaire includes 13 items divided in 2 sub-scales: Voiding symptoms (V) from 0-20 and Incontinence symptoms (I) from 0-24. Percentage of patients that completed all items: 98.84%. Ground effect is 0 and ceiling effect was under 6% in both sub-scales. Test-retest reliability: Intraclass correlation coefficient (ICC) ranged from 0.68 to 0.88, except on Delay. Kappa shows a good agreement, between 0.60 and 0.81, except for Nocturia. Convergent validity: Correlation (Spearman) between the questionnaire sub-scales scores and the rest of measures is statistically significant (P < .01 and P < .05). Criterion validity: Statistically significant differences (P < .05) between scores on ICIQ-MLUTS, from patients that refer experiencing symptoms and those who do not. The Spanish version of the ICIQ-MLUTS questionnaire shows adequate feasibility, reliability and validity. Copyright © 2013 AEU. Published by Elsevier Espana. All rights reserved.

An initial investigation into the validity of a computer-based auditory processing assessment (Feather Squadron).

PubMed

Barker, Matthew D; Purdy, Suzanne C

2016-01-01

This research investigates a novel method for identifying and measuring school-aged children with poor auditory processing through a tablet computer. Feasibility and test-retest reliability are investigated by examining the percentage of Group 1 participants able to complete the tasks and developmental effects on performance. Concurrent validity was investigated against traditional tests of auditory processing using Group 2. There were 847 students aged 5 to 13 years in group 1, and 46 aged 5 to 14 years in group 2. Some tasks could not be completed by the youngest participants. Significant correlations were found between results of most auditory processing areas assessed by the Feather Squadron test and traditional auditory processing tests. Test-retest comparisons indicated good reliability for most of the Feather Squadron assessments and some of the traditional tests. The results indicate the Feather Squadron assessment is a time-efficient, feasible, concurrently valid, and reliable approach for measuring auditory processing in school-aged children. Clinically, this may be a useful option for audiologists when performing auditory processing assessments as it is a relatively fast, engaging, and easy way to assess auditory processing abilities. Research is needed to investigate further the construct validity of this new assessment by examining the association between performance on Feather Squadron and objective evoked potential, lesion studies, and/or functional imaging measures of auditory function.

[Validation and adhesion to GESIDA quality indicators in patients with HIV infection].

PubMed

Riera, Melchor; Esteban, Herminia; Suarez, Ignacio; Palacios, Rosario; Lozano, Fernando; Blanco, Jose R; Valencia, Eulalia; Ocampo, Antonio; Amador, Concha; Frontera, Guillem; vonWichmann-de Miguel, Miguel Angel

2016-01-01

The objective of the study is to validate the relevant GESIDA quality indicators for HIV infection, assessing the reliability, feasibility and adherence to them. The reliability was evaluated using the reproducibility of 6 indicators in peer review, with the second observer being an outsider. The feasibility and measurement of the level of adherence to the 22 indicators was conducted with annual fragmented retrospective collection of information from specific databases or the clinical charts of the nine participating hospitals. Reliability was very high, with interobserver agreement levels higher than 95% in 5 of the 6 indicators. The median time to achieve the indicators ranged between 5 and 600minutes, but could be achieved progressively from specific databases, enabling obtaining them automatically. As regards adherence to the indicators related with the initial evaluation of the patients, instructions and suitability of the guidelines for ART, adherence to ART, follow-up in clinics, and achieve an undetectable HIV by PCR at week 48 of the ART. Indicators of quality related to the prevention of opportunistic infections and control of comorbidities, the standards set were not achieved, and significant heterogeneity was observed between hospitals. The GESIDA quality indicators of HIV infection enabled the relevant indicators to be feasibly and reliably measured, and should be collected in all the units that care for patients with HIV infection. Copyright © 2015 Elsevier España, S.L.U. and Sociedad Española de Enfermedades Infecciosas y Microbiología Clínica. All rights reserved.

The Strengths and Difficulties Questionnaire: psychometric properties of the parent and teacher version in children aged 4-7.

PubMed

Stone, Lisanne L; Janssens, Jan M A M; Vermulst, Ad A; Van Der Maten, Marloes; Engels, Rutger C M E; Otten, Roy

2015-01-01

The Strengths and Difficulties Questionnaire is one of the most employed screening instruments. Although there is a large research body investigating its psychometric properties, reliability and validity are not yet fully tested using modern techniques. Therefore, we investigate reliability, construct validity, measurement invariance, and predictive validity of the parent and teacher version in children aged 4-7. Besides, we intend to replicate previous studies by investigating test-retest reliability and criterion validity. In a Dutch community sample 2,238 teachers and 1,513 parents filled out questionnaires regarding problem behaviors and parenting, while 1,831 children reported on sociometric measures at T1. These children were followed-up during three consecutive years. Reliability was examined using Cronbach's alpha and McDonald's omega, construct validity was examined by Confirmatory Factor Analysis, and predictive validity was examined by calculating developmental profiles and linking these to measures of inadequate parenting, parenting stress and social preference. Further, mean scores and percentiles were examined in order to establish norms. Omega was consistently higher than alpha regarding reliability. The original five-factor structure was replicated, and measurement invariance was established on a configural level. Further, higher SDQ scores were associated with future indices of higher inadequate parenting, higher parenting stress and lower social preference. Finally, previous results on test-retest reliability and criterion validity were replicated. This study is the first to show SDQ scores are predictively valid, attesting to the feasibility of the SDQ as a screening instrument. Future research into predictive validity of the SDQ is warranted.

Development of a Peer Teaching-Assessment Program and a Peer Observation and Evaluation Tool

PubMed Central

Trujillo, Jennifer M.; Barr, Judith; Gonyeau, Michael; Van Amburgh, Jenny A.; Matthews, S. James; Qualters, Donna

2008-01-01

Objectives To develop a formalized, comprehensive, peer-driven teaching assessment program and a valid and reliable assessment tool. Methods A volunteer taskforce was formed and a peer-assessment program was developed using a multistep, sequential approach and the Peer Observation and Evaluation Tool (POET). A pilot study was conducted to evaluate the efficiency and practicality of the process and to establish interrater reliability of the tool. Intra-class correlation coefficients (ICC) were calculated. Results ICCs for 8 separate lectures evaluated by 2-3 observers ranged from 0.66 to 0.97, indicating good interrater reliability of the tool. Conclusion Our peer assessment program for large classroom teaching, which includes a valid and reliable evaluation tool, is comprehensive, feasible, and can be adopted by other schools of pharmacy. PMID:19325963

Feasibility, reliability and validity of the Thai version of the Pediatric Quality of Life Inventory 3.0 cerebral palsy module.

PubMed

Tantilipikorn, Pinailug; Watter, Pauline; Prasertsukdee, Saipin

2013-03-01

Quality of Life (QOL) and Health-related Quality of Life (HRQOL) are now considered as necessary outcome measures for children with cerebral palsy (CP). Various reliable and valid condition-specific HRQOL tools are available for these children. One of these is Pediatric Quality of Life Inventory (PedsQL) 3.0 CP module which has been widely used and was translated to many languages. As no Thai version is available, the authors have completed this translation. This study then aimed to investigate psychometric properties of the newly translated Thai PedsQL 3.0 CP module and to establish parent confidence in their ratings in the translated tool. Translation of the PedsQL 3.0 CP module was performed based on linguistic translation guidelines. Then, the psychometric properties of the Thai version were established. PedsQL 3.0 CP module was completed by children with CP and their parents or caregivers twice with 2-4 weeks. Respondents were 97 parents or caregivers and 54 children. Minimal missing data were found. Acceptable internal consistency was supported except for Movement and Balance Scale (self-report). Intraclass correlation coefficients for parent proxy and self-report were good to excellent (0.684-0.950). The feasibility, reliability and validity of the translated tool were supported.

Development, initial reliability and validity testing of an observational tool for assessing technical skills of operating room nurses.

PubMed

Sevdalis, Nick; Undre, Shabnam; Henry, Janet; Sydney, Elaine; Koutantji, Mary; Darzi, Ara; Vincent, Charles A

2009-09-01

The recent emergence of the Systems Approach to the safety and quality of surgical care has triggered individual and team skills training modules for surgeons and anaesthetists and relevant observational assessment tools have been developed. To develop an observational tool that captures operating room (OR) nurses' technical skill and can be used for assessment and training. The Imperial College Assessment of Technical Skills for Nurses (ICATS-N) assesses (i) gowning and gloving, (ii) setting up instrumentation, (iii) draping, and (iv) maintaining sterility. Three to five observable behaviours have been identified for each skill and are rated on 1-6 scales. Feasibility and aspects of reliability and validity were assessed in 20 simulation-based crisis management training modules for trainee nurses and doctors, carried out in a Simulated Operating Room. The tool was feasible to use in the context of simulation-based training. Satisfactory reliability (Cronbach alpha) was obtained across trainers' and trainees' scores (analysed jointly and separately). Moreover, trainer nurse's ratings of the four skills correlated positively, thus indicating adequate content validity. Trainer's and trainees' ratings did not correlate. Assessment of OR nurses' technical skill is becoming a training priority. The present evidence suggests that the ICATS-N could be considered for use as an assessment/training tool for junior OR nurses.

Quantifying Engagement: Measuring Player Involvement in Human-Avatar Interactions

PubMed Central

Norris, Anne E.; Weger, Harry; Bullinger, Cory; Bowers, Alyssa

2014-01-01

This research investigated the merits of using an established system for rating behavioral cues of involvement in human dyadic interactions (i.e., face-to-face conversation) to measure involvement in human-avatar interactions. Gameplay audio-video and self-report data from a Feasibility Trial and Free Choice study of an effective peer resistance skill building simulation game (DRAMA-RAMA™) were used to evaluate reliability and validity of the rating system when applied to human-avatar interactions. The Free Choice study used a revised game prototype that was altered to be more engaging. Both studies involved girls enrolled in a public middle school in Central Florida that served a predominately Hispanic (greater than 80%), low-income student population. Audio-video data were coded by two raters, trained in the rating system. Self-report data were generated using measures of perceived realism, predictability and flow administered immediately after game play. Hypotheses for reliability and validity were supported: Reliability values mirrored those found in the human dyadic interaction literature. Validity was supported by factor analysis, significantly higher levels of involvement in Free Choice as compared to Feasibility Trial players, and correlations between involvement dimension sub scores and self-report measures. Results have implications for the science of both skill-training intervention research and game design. PMID:24748718

The comprehensive care project: measuring physician performance in ambulatory practice.

PubMed

Holmboe, Eric S; Weng, Weifeng; Arnold, Gerald K; Kaplan, Sherrie H; Normand, Sharon-Lise; Greenfield, Sheldon; Hood, Sarah; Lipner, Rebecca S

2010-12-01

To investigate the feasibility, reliability, and validity of comprehensively assessing physician-level performance in ambulatory practice. Ambulatory-based general internists in 13 states participated in the assessment. We assessed physician-level performance, adjusted for patient factors, on 46 individual measures, an overall composite measure, and composite measures for chronic, acute, and preventive care. Between- versus within-physician variation was quantified by intraclass correlation coefficients (ICC). External validity was assessed by correlating performance on a certification exam. Medical records for 236 physicians were audited for seven chronic and four acute care conditions, and six age- and gender-appropriate preventive services. Performance on the individual and composite measures varied substantially within (range 5-86 percent compliance on 46 measures) and between physicians (ICC range 0.12-0.88). Reliabilities for the composite measures were robust: 0.88 for chronic care and 0.87 for preventive services. Higher certification exam scores were associated with better performance on the overall (r = 0.19; p<.01), chronic care (r = 0.14, p = .04), and preventive services composites (r = 0.17, p = .01). Our results suggest that reliable and valid comprehensive assessment of the quality of chronic and preventive care can be achieved by creating composite measures and by sampling feasible numbers of patients for each condition. © Health Research and Educational Trust.

Patient-based health status assessments in an outpatient psychiatry setting.

PubMed

Adler, D A; Bungay, K M; Cynn, D J; Kosinski, M

2000-03-01

The reliability, validity, and feasibility of the routine use of a generic health status instrument, the Short-Form-36 Health Survey (SF-36), were examined in a psychiatric outpatient clinic of a general hospital. The sample comprised 411 patients referred to an outpatient psychiatry department between April 1994 and March 1995. They filled out the SF-36 along with their admission forms. Scores and reports were generated, and the results were returned to the charts and used at weekly clinical conference discussions. Feasibility was evaluated using subjective and objective data on administration of the instrument, its psychometric properties, and costs. Results from the outpatient psychiatry patients were compared with those from patients scheduled for elective surgery and a healthy normative sample. Routine administration of the SF-36 was successfully achieved with minimal resistance from staff and patients. The SF-36 provided reliable and valid data. As predicted, patients with emotional disorders scored lower, indicating more impairment, on scales measuring mental health than did the elective surgery patients and the normative sample. However, the psychiatric patients' scores on the physical health scale were lower than clinicians expected. Compared with the elective surgery patients, the psychiatric patients were less impaired on only the physical functioning and bodily pain scales; no difference was found between the two groups in role functioning due to physical problems. Routine use of the SF-36 in a general hospital psychiatric outpatient clinic was feasible, and the results were reliable, valid, and helpful to clinicians. Psychiatric patients' significantly lower scores in physical health and social and role functioning provided additional information about their difficulties.

The assessment of emergency physicians by a regulatory authority.

PubMed

Lockyer, Jocelyn M; Violato, Claudio; Fidler, Herta

2006-12-01

To determine whether it is possible to develop a feasible, valid, and reliable multisource feedback program (360 degree evaluation) for emergency physicians. Surveys with 16, 20, 30, and 31 items were developed to assess emergency physicians by 25 patients, eight coworkers, eight medical colleagues, and self, respectively, using five-point scales along with an "unable to assess" category. Items addressed key competencies related to communication skills, professionalism, collegiality, and self-management. Data from 187 physicians who identified themselves as emergency physicians were available. The mean number of respondents per physician was 21.6 (SD +/- 3.87) (93%) for patients, 7.6 (SD +/- 0.89) (96%) for coworkers, and 7.7 (SD +/- 0.61) (95%) for medical colleagues, suggesting it was a feasible tool. Only the patient survey had four items with "unable to assess" percentages > or = 15%. The factor analysis indicated there were two factors on the patient questionnaire (communication/professionalism and patient education), two on the coworker survey (communication/collegiality and professionalism), and four on the medical colleague questionnaire (clinical performance, professionalism, self-management, and record management) that accounted for 80.0%, 62.5%, and 71.9% of the variance on the surveys, respectively. The factors were consistent with the intent of the instruments, providing empirical evidence of validity for the instruments. Reliability was established for the instruments (Cronbach's alpha > 0.94) and for each physician (generalizability coefficients were 0.68 for patients, 0.85 for coworkers, and 0.84 for medical colleagues). The psychometric examination of the data suggests that the instruments developed to assess emergency physicians were feasible and provide evidence for validity and reliability.

Patient-reported Pediatric Quality of Life Inventory™ 4.0 Generic Core Scales in pediatric patients with attention-deficit/hyperactivity disorder and comorbid psychiatric disorders: feasibility, reliability, and validity.

PubMed

Limbers, Christine A; Ripperger-Suhler, Jane; Heffer, Robert W; Varni, James W

2011-06-01

The primary objective of the study was to evaluate the feasibility, reliability, and validity of the Pediatric Quality of Life Inventory™ (PedsQL) 4.0 Generic Core Scales as a patient self-reported health-related quality of life measurement instrument in pediatric patients with attention-deficit/hyperactivity disorder (ADHD) and physician-diagnosed comorbid psychiatric disorders being seen in a pediatric psychiatric clinic. The secondary objective was to evaluate parent proxy-reported PedsQL in this population. One hundred seventy-nine children with ADHD and comorbid psychiatric disorders ages 5 to 18 years and 181 parents completed the PedsQL 4.0 Generic Core Scales and parents also completed the Vanderbilt ADHD Diagnostic Rating Scales. Known-groups discriminant validity comparisons were made between the sample of pediatric patients with ADHD and comorbid psychiatric disorders and healthy, cancer, and type 1 diabetes samples. The PedsQL evidenced minimal missing responses for patient self-report and parent proxy-report (0.2% and 0.5%, respectively), demonstrated no significant floor or ceiling effects, and achieved excellent reliability for the Total Scale Score (α = 0.85 patient self-report, 0.92 parent proxy-report). Pediatric patients with ADHD and comorbid psychiatric disorders and their parents reported statistically significantly worse PedsQL scores than healthy children, with large effect sizes across all domains, supporting known-groups discriminant validity. Pediatric patients with ADHD and comorbid psychiatric disorders and their parents reported worse PedsQL scores compared to pediatric patients with cancer and diabetes with the exception of physical health, in which pediatric cancer patients manifested lower physical health, indicating the relative severe impact of ADHD and comorbid psychiatric disorders. More severe ADHD symptoms were generally associated with more impaired PedsQL scores, supporting construct validity. These data demonstrate the feasibility, reliability, and validity of patient self-reported PedsQL 4.0 Generic Core Scales in this high risk population of pediatric patients and highlight the profound negative impact of ADHD and comorbid psychiatric disorders on generic health-related quality of life, comparable to or worse than serious pediatric chronic physical diseases. Copyright © 2011 International Society for Pharmacoeconomics and Outcomes Research (ISPOR). Published by Elsevier Inc. All rights reserved.

Diagnosing paratonia in the demented elderly: reliability and validity of the Paratonia Assessment Instrument (PAI).

PubMed

Hobbelen, Johannes S M; Koopmans, Raymond T C M; Verhey, Frans R J; Habraken, Kitty M; de Bie, Rob A

2008-08-01

Paratonia is one of the associated movement disorders characteristic of dementia. The aim of this study was to develop an assessment tool (the Paratonia Assessment Instrument, PAI), based on the new consensus definition of paratonia. An additional aim was to investigate the reliability and validity of the PAI. A three-phase cross-sectional survey was conducted. In the first two phases, the PAI was developed and validated. In the third phase, the inter-observer reliability and feasibility of the instrument was tested. The original PAI consisted of five criteria that all needed to be met in order to make the diagnosis. On the basis of a qualitative analysis, one criterion was reformulated and another was removed. Following this, inter-observer reliability between the two assessors resulted in an improvement of Cohen's kappa from 0.532 in the initial phase to 0.677 in the second phase. This improvement was substantiated in the third phase by two independent assessors with Cohen's kappa ranging from 0.625 to 1. The PAI is a reliable and valid assessment tool for diagnosing paratonia in elderly people with dementia that can be applied easily in daily practice.

Critical-Inquiry-Based-Learning: Model of Learning to Promote Critical Thinking Ability of Pre-service Teachers

NASA Astrophysics Data System (ADS)

Prayogi, S.; Yuanita, L.; Wasis

2018-01-01

This study aimed to develop Critical-Inquiry-Based-Learning (CIBL) learning model to promote critical thinking (CT) ability of preservice teachers. The CIBL learning model was developed by meeting the criteria of validity, practicality, and effectiveness. Validation of the model involves 4 expert validators through the mechanism of the focus group discussion (FGD). CIBL learning model declared valid to promote CT ability, with the validity level (Va) of 4.20 and reliability (r) of 90,1% (very reliable). The practicality of the model was evaluated when it was implemented that involving 17 of preservice teachers. The CIBL learning model had been declared practice, its measuring from learning feasibility (LF) with very good criteria (LF-score = 4.75). The effectiveness of the model was evaluated from the improvement CT ability after the implementation of the model. CT ability were evaluated using the scoring technique adapted from Ennis-Weir Critical Thinking Essay Test. The average score of CT ability on pretest is - 1.53 (uncritical criteria), whereas on posttest is 8.76 (critical criteria), with N-gain score of 0.76 (high criteria). Based on the results of this study, it can be concluded that developed CIBL learning model is feasible to promote CT ability of preservice teachers.

A Reliable, Feasible Method to Observe Neighborhoods at High Spatial Resolution

PubMed Central

Kepper, Maura M.; Sothern, Melinda S.; Theall, Katherine P.; Griffiths, Lauren A.; Scribner, Richard; Tseng, Tung-Sung; Schaettle, Paul; Cwik, Jessica M.; Felker-Kantor, Erica; Broyles, Stephanie T.

2016-01-01

Introduction Systematic social observation (SSO) methods traditionally measure neighborhoods at street level and have been performed reliably using virtual applications to increase feasibility. Research indicates that collection at even higher spatial resolution may better elucidate the health impact of neighborhood factors, but whether virtual applications can reliably capture social determinants of health at the smallest geographic resolution (parcel level) remains uncertain. This paper presents a novel, parcel-level SSO methodology and assesses whether this new method can be collected reliably using Google Street View and is feasible. Methods Multiple raters (N=5) observed 42 neighborhoods. In 2016, inter-rater reliability (observed agreement and kappa coefficient) was compared for four SSO methods: (1) street-level in person; (2) street-level virtual; (3) parcel-level in person; and (4) parcel-level virtual. Intra-rater reliability (observed agreement and kappa coefficient) was calculated to determine whether parcel-level methods produce results comparable to traditional street-level observation. Results Substantial levels of inter-rater agreement were documented across all four methods; all methods had >70% of items with at least substantial agreement. Only physical decay showed higher levels of agreement (83% of items with >75% agreement) for direct versus virtual rating source. Intra-rater agreement comparing street- versus parcel-level methods resulted in observed agreement >75% for all but one item (90%). Conclusions Results support the use of Google Street View as a reliable, feasible tool for performing SSO at the smallest geographic resolution. Validation of a new parcel-level method collected virtually may improve the assessment of social determinants contributing to disparities in health behaviors and outcomes. PMID:27989289

[The Basel Screening Instrument for Psychosis (BSIP): development, structure, reliability and validity].

PubMed

Riecher-Rössler, A; Aston, J; Ventura, J; Merlo, M; Borgwardt, S; Gschwandtner, U; Stieglitz, R-D

2008-04-01

Early detection of psychosis is of growing clinical importance. So far there is, however, no screening instrument for detecting individuals with beginning psychosis in the atypical early stages of the disease with sufficient validity. We have therefore developed the Basel Screening Instrument for Psychosis (BSIP) and tested its feasibility, interrater-reliability and validity. Aim of this paper is to describe the development and structure of the instrument, as well as to report the results of the studies on reliability and validity. The instrument was developed based on a comprehensive search of literature on the most important risk factors and early signs of schizophrenic psychoses. The interraterreliability study was conducted on 24 psychiatric cases. Validity was tested based on 206 individuals referred to our early detection clinic from 3/1/2000 until 2/28/2003. We identified seven categories of relevance for early detection of psychosis and used them to construct a semistructured interview. Interrater-reliability for high risk individuals was high (Kappa .87). Predictive validity was comparable to other, more comprehensive instruments: 16 (32 %) of 50 individuals classified as being at risk for psychosis by the BSIP have in fact developed frank psychosis within an follow-up period of two to five years. The BSIP is the first screening instrument for the early detection of psychosis which has been validated based on transition to psychosis. The BSIP is easy to use by experienced psychiatrists and has a very good interrater-reliability and predictive validity.

[Satisfaction with obstetrical care: development and validation of a scale on quality of care].

PubMed

Ramanah, R; Dumont, A; Schepens, F; Traore, M; Gaye, A; Schaal, J-P; Riethmuller, D; Rude, N

2014-01-01

To develop and validate a subjective and multidimensional scale to measure satisfaction in obstetrical care (SSO) during labour, delivery and two hours postpartum, which is relevant to the French-speaking context. Forty partially directed patient interviews during the 48 hours after delivery and four care-giver interviews were conducted to build up the questionnaire. After a prior feasibility study on 40 patients, the psychometric validity of the questionnaire was evaluated by calculating the Cronbach coefficient of reliability for 432 patients. Hundred and eighty items were initially obtained after content analysis of the patient interviews. Expert meetings finally selected 49 items classified within 5 dimensions. The feasibility study showed that the questionnaire was easily accepted and understood with a mean time of 15 minutes to answer it. Cronbach coefficients were respectively at 0.941, 0.949, 0.808, 0.814 et 0.869 for the 5 dimensions. SSO questionnaire is a reliable and relevant scale to measure immediate postpartum quality of care in French. Copyright © 2014 Elsevier Masson SAS. All rights reserved.

Feasibility and inter-rater reliability of the ICU Mobility Scale.

PubMed

Hodgson, Carol; Needham, Dale; Haines, Kimberley; Bailey, Michael; Ward, Alison; Harrold, Megan; Young, Paul; Zanni, Jennifer; Buhr, Heidi; Higgins, Alisa; Presneill, Jeff; Berney, Sue

2014-01-01

The objectives of this study were to develop a scale for measuring the highest level of mobility in adult ICU patients and to assess its feasibility and inter-rater reliability. Growing evidence supports the feasibility, safety and efficacy of early mobilization in the intensive care unit (ICU). However, there are no adequately validated tools to quickly, easily, and reliably describe the mobility milestones of adult patients in ICU. Identifying or developing such a tool is a priority for evaluating mobility and rehabilitation activities for research and clinical care purposes. This study was performed at two ICUs in Australia. Thirty ICU nursing, and physiotherapy staff assessed the feasibility of the 'ICU Mobility Scale' (IMS) using a 10-item questionnaire. The inter-rater reliability of the IMS was assessed by 2 junior physical therapists, 2 senior physical therapists, and 16 nursing staff in 100 consecutive medical, surgical or trauma ICU patients. An 11 point IMS scale was developed based on multidisciplinary input. Participating clinicians reported that the scale was clear, with 95% of respondents reporting that it took <1 min to complete. The junior and senior physical therapists showed the highest inter-rater reliability with a weighted Kappa (95% confidence interval) of 0.83 (0.76-0.90), while the senior physical therapists and nurses and the junior physical therapists and nurses had a weighted Kappa of 0.72 (0.61-0.83) and 0.69 (0.56-0.81) respectively. The IMS is a feasible tool with strong inter-rater reliability for measuring the maximum level of mobility of adult patients in the ICU. Copyright © 2014 Elsevier Inc. All rights reserved.

[Authorization, translation, back translation and language modification of the simplified Chinese adult comorbidity-27 index].

PubMed

Gao, L; Mao, C; Yu, G Y; Peng, X

2016-10-09

Objective: To translate the adult comorbidity evaluation-27(ACE-27) index authored by professor JF Piccirillo into Chinese and for the purpose of assessing the possible impact of comorbidity on survival of oral cancer patients and improving cancer staging. Methods: The translation included the following steps, obtaining permission from professor Piccirillo, translation, back translation, language modification, adjusted by the advice from the professors of oral and maxillofacial surgery. The test population included 154 patients who were admitted to Peking University of Stomatology during March 2011. Questionnaire survey was conducted on these patients. Retest of reliability, internal consistency reliability, content validity, and structure validity were performed. Results: The simplified Chinese ACE-27 index was established. The Cronbach's α was 0.821 in the internal consistency reliability test. The Kaiser-Meyer-Olkin (KMO) value of 8 items was 0.859 in the structure validity test. Conclusions: The simplified Chinese ACE-27 index has good feasibility and reliability. It is useful to assess the comorbidity of oral cancer patients.

from the Adolescents’ Perspective in Malaysia

PubMed

Mohd Zin, Faridah; Hillaluddin, Azlin Hilma; Mustaffa, Jamaludin

2017-05-01

Objective: This study aims to develop, validate and determine the reliability of an interactive multimedia strategy to prevent tobacco use among the young (TUPY-S) from an adolescents’ perspective. Methods: A descriptive study design was utilized. A modular instruction guideline by Russel (1974) was followed in the entire process, comprising a feasibility study, a review of existing modules, specification of the objectives, identification of the construct criterion items, learner analysis and entry behavior specification, establishment of the sequence instruction and media selection, a tryout with students and a field test. Result: Feasibility was agreed among the researchers and the school authorities. Culturally suitable rigorously developed tobacco use preventive strategies delivered using information technology (IT) are lacking in the literature. The objective of TUPY-S is to prevent tobacco use among adolescents living in Malaysia. Identified construct criterion items include knowledge, attitude, intention to use, self-efficacy, and refusal skill. The target population was early adolescents belonging to generation-Z. Content was developed from the adolescents’ perspective and delivered using IT in Malay language. Content validity, assessed by six experts in the field and module development, was good at 86%. The students’ tryout showed satisfactory face validity subjectively and objectively (85.5%) and high alpha Cronbach reliability (0.91). Conclusion: TUPY-S was confirmed to suit early adolescents of the current generation living in Malaysia. It demonstrated good content validity among the experts, satisfactory face validity and reliability among the target population. TUPY-S is ready to be evaluated for its effectiveness among early adolescents. Creative Commons Attribution License

Evaluation of using the Chinese version of the Spirituality Index of Well-Being (SIWB) scale in Taiwanese elders.

PubMed

Lee, Yi-Hui; Salman, Ali

2016-11-01

Spirituality and spiritual well-being have emerged as important indicators for one's quality of life and health outcomes. Nursing as a profession is concerned with a holistic approach to improve health and overall well-being. To evaluate the outcomes of holistic nursing interventions, using valid and reliable instruments to assess spiritual well-being becomes necessary. There is a lack of instruments for measuring spiritual well-being in Chinese populations. Little has been known about the feasibility of using the Spirituality Index of Well-Being (SIWB) in Taiwanese elders. The purpose of this cross-sectional study was to evaluate the uses of the translated Chinese version of the Spirituality Index of Well-Being (SIWB-C) with Taiwanese elders. A total of 150 individual who were 65 years old or older and living in southern Taiwan were recruited from a public community center. A four-step procedure was used to translate the English version of the SIWB to the traditional Chinese language. Internal consistency, factor analysis, and correlation coefficient were conducted to evaluate the reliability and validity of the SIWB-C. The SIWB-C demonstrated a high internal consistency with Cronbach's alpha .95. The construct validity of SIWB-C was supported by factor analysis and by significant correlations with its subscales and the CES-D scale. The psychometric analysis indicates that the SIWB-C is a valid and reliable instrument for measuring spiritual well-being. This instrument provides a feasible and valid approach for assessing Taiwanese elders' spiritual well-being in the future. Copyright © 2016 Elsevier Inc. All rights reserved.

Validity and reliability of a short questionnaire for assessing the impact of cooking skills interventions.

PubMed

Barton, K L; Wrieden, W L; Anderson, A S

2011-12-01

Food skills programmes are widely used as a means to improve confidence in food preparation, the use of basic food skills and food selections amongst low income communities. However, the impact of such interventions are rarely evaluated as a result of a lack of validated assessment tools appropriate for use within this target group. A two-page questionnaire utilising a closed-question format was designed based on key domains known to be influenced by cooking skills programmes. Content validity was assessed by a panel of public health experts and face validity by individuals, typical of those who may attend cooking skills classes. Internal and repeat reliability were assessed with groups of adults attending community-based classes. The feasibility of using the tool in community settings was also assessed. The draft questionnaire was amended as appropriate subsequent to content and face validity testing. Cronbach's alpha for confidence and knowledge sections was 0.86 and 0.84, respectively, indicating good internal consistency. Spearman correlation coefficients for repeat reliability testing between time 1 and time 2 for each item were in the range 0.46-0.91 (all significant at P < 0.001), indicating that the questionnaire elicited stable responses for repeated use. Feasibility testing highlighted the need for detailed instructions for course tutors on how to distribute and check questionnaires for completion. This tool provides a standardised method of evaluating cooking skills interventions that could be utilised in the development and evaluation of multicentre cooking skills interventions. © 2011 The Authors. Journal of Human Nutrition and Dietetics © 2011 The British Dietetic Association Ltd.

Toddler physical activity study: laboratory and community studies to evaluate accelerometer validity and correlates.

PubMed

Hager, Erin R; Gormley, Candice E; Latta, Laura W; Treuth, Margarita S; Caulfield, Laura E; Black, Maureen M

2016-09-06

Toddlerhood is an important age for physical activity (PA) promotion to prevent obesity and support a physically active lifestyle throughout childhood. Accurate assessment of PA is needed to determine trends/correlates of PA, time spent in sedentary, light, or moderate-vigorous PA (MVPA), and the effectiveness of PA promotion programs. Due to the limited availability of objective measures that have been validated and evaluated for feasibility in community studies, it is unclear which subgroups of toddlers are at the highest risk for inactivity. Using Actical ankle accelerometry, the objectives of this study are to develop valid thresholds, examine feasibility, and examine demographic/ anthropometric PA correlates of MVPA among toddlers from low-income families. Two studies were conducted with toddlers (12-36 months). Laboratory Study (n = 24)- Two Actical accelerometers were placed on the ankle. PA was observed using the Child Activity Rating Scale (CARS, prescribed activities). Analyses included device equivalence reliability (correlation: activity counts of two Acticals), criterion-related validity (correlation: activity counts and CARS ratings), and sensitivity/specificity for thresholds. Community Study (n = 277, low-income mother-toddler dyads recruited)- An Actical was worn on the ankle for > 7 days (goal >5, 24-h days). Height/weight was measured. Mothers reported demographics. Analyses included frequencies (feasibility) and stepwise multiple linear regression (sMLR). Laboratory Study- Acticals demonstrated reliability (r = 0.980) and validity (r = 0.75). Thresholds demonstrated sensitivity (86 %) and specificity (88 %). Community Study- 86 % wore accelerometer, 69 % had valid data (mean = 5.2 days). Primary reasons for missing/invalid data: refusal (14 %) and wear-time ≤2 days (11 %). The MVPA threshold (>2200 cpm) yielded 54 min/day. In sMLR, MVPA was associated with age (older > younger, β = 32.8, p < 0.001), gender (boys > girls, β = -11.21, p = 0.032), maternal MVPA (β = 0.44, p = 0.002) and recruitment location (suburban > urban, β = 19.6, p < 0.001), or race (non-Black > Black, β = 18.5, p = 0.001). No association with toddler weight status. Ankle accelerometry is a valid, reliable, and feasible method of assessing PA in community studies of toddlers from low-income families. Sub-populations of toddlers may be at increased risk for inactivity, including toddlers that are younger, female, Black, those with less active mothers, and those living in an urban location.

Reliability and validity of the Brief Pain Inventory in individuals with chronic obstructive pulmonary disease.

PubMed

Chen, Y-W; HajGhanbari, B; Road, J D; Coxson, H O; Camp, P G; Reid, W D

2018-06-08

Pain is prevalent in chronic obstructive pulmonary disease (COPD) and the Brief Pain Inventory (BPI) appears to be a feasible questionnaire to assess this symptom. However, the reliability and validity of the BPI have not been determined in individuals with COPD. This study aimed to determine the internal consistency, test-retest reliability and validity (construct, convergent, divergent and discriminant) of the BPI in individuals with COPD. In order to examine the test-retest reliability, individuals with COPD were recruited from pulmonary rehabilitation programmes to complete the BPI twice 1 week apart. In order to investigate validity, de-identified data was retrieved from two previous studies, including forced expiratory volume in 1-s, age, sex and data from four questionnaires: the BPI, short-form McGill Pain Questionnaire (SF-MPQ), 36-Item Short Form Survey (SF-36) and Community Health Activities Model Program for Seniors (CHAMPS) questionnaire. In total, 123 participants were included in the analyses (eligible data were retrieved from 86 participants and additional 37 participants were recruited). The BPI demonstrated excellent internal consistency and test-retest reliability. It also showed convergent validity with the SF-MPQ and divergent validity with the SF-36. The factor analysis yielded two factors of the BPI, which demonstrated that the two domains of the BPI measure the intended constructs. The BPI can also discriminate pain levels among COPD patients with varied levels of quality of life (SF-36) and physical activity (CHAMPS). The BPI is a reliable and valid pain questionnaire that can be used to evaluate pain in COPD. This study formally established the reliability and validity of the BPI in individuals with COPD, which have not been determined in this patient group. The results of this study provide strong evidence that assessment results from this pain questionnaire are reliable and valid. © 2018 European Pain Federation - EFIC®.

[Adaptation and study of the measurement properties of a sleep questionnaire for infants and pre-school children].

PubMed

Cassanello, Pia; Díez-Izquierdo, Ana; Gorina, Nuria; Matilla-Santander, Nuria; Martínez-Sanchez, Jose M; Balaguer, Albert

2018-01-06

Although sleep disturbances in infants and toddlers are common, there is no suitable validated tool in Spanish to evaluate sleep disorders. The Brief Infant Sleep Questionnaire (BISQ) is a well-established multidimensional questionnaire widely used internationally. To adapt the BISQ questionnaire to Spanish and analyse its reliability and validity. To explore its feasibility, both in the clinical context, and in epidemiological research. Participants were parents of children between 3 and 30 months of age. The adaptation to Spanish (BISQ-E) was achieved by using both back translation and consensus, following the international guidelines.Reliability was determined by means of test-retest and measurement of agreement (Kappa value) between 2forms of administration of the questionnaire (self-administration and clinical interview). Construct validity was established by analysing its correlation with a sleep diary. A total of 87 families/children participated. The test-retest reliability undertaken in 60 subjects was excellent; r = 0.848 (P<0.001). The kappa value was 0.939 (95% CI: 0.858-1.00, P<0.001).The agreement between BISQ-E and the sleep diary was analysed in 27 families/children, with statistically significant values being obtained for the following variables: bedtime (r = 0.731), hours of night sleep (r = 0.726), hours of daytime sleep (r = 0.867), and number of nocturnal awakenings (r = 0.888) (P < 0.001). The Spanish adaptation of the BISQ shows overall adequate validity and reliability for the evaluation of sleep in infants and pre-school children. Its use as a clinical tool, or for clinical-epidemiological research studies, is feasible. Copyright © 2017. Publicado por Elsevier España, S.L.U.

Development and validation of a trustworthy multisource feedback instrument to support nurse appraisals.

PubMed

Crossley, James G M

2015-01-01

Nurse appraisal is well established in the Western world because of its obvious educational advantages. Appraisal works best with many sources of information on performance. Multisource feedback (MSF) is widely used in business and in other clinical disciplines to provide such information. It has also been incorporated into nursing appraisals, but, so far, none of the instruments in use for nurses has been validated. We set out to develop an instrument aligned with the UK Knowledge and Skills Framework (KSF) and to evaluate its reliability and feasibility across a wide hospital-based nursing population. The KSF framework provided a content template. Focus groups developed an instrument based on consensus. The instrument was administered to all the nursing staff in 2 large NHS hospitals forming a single trust in London, England. We used generalizability analysis to estimate reliability, response rates and unstructured interviews to evaluate feasibility, and factor structure and correlation studies to evaluate validity. On a voluntary basis the response rate was moderate (60%). A failure to engage with information technology and employment-related concerns were commonly cited as reasons for not responding. In this population, 11 responses provided a profile with sufficient reliability to inform appraisal (G = 0.7). Performance on the instrument was closely and significantly correlated with performance on a KSF questionnaire. This is the first contemporary psychometric evaluation of an MSF instrument for nurses. MSF appears to be as valid and reliable as an assessment method to inform appraisal in nurses as it is in other health professional groups. © 2015 The Alliance for Continuing Education in the Health Professions, the Society for Academic Continuing Medical Education, and the Council on Continuing Medical Education, Association for Hospital Medical Education.

Transcultural adaptation and validation of the “Hip and Knee” questionnaire into Spanish

PubMed Central

2014-01-01

Background The purpose of the present study is to translate and validate the “Hip and Knee Outcomes Questionnaire”, developed in English, into Spanish. The ‘Hip and Knee Outcomes Questionnaire is a questionnaire planned to evaluate the impact in quality of life of any problem related to the human musculoskeletal system. 10 scientific associations developed it. Methods The questionnaire underwent a validated translation/retro-translation process. Patients undergoing primary knee arthroplasty, before and six months postoperative, tested the final version in Spanish. Psychometric properties of feasibility, reliability, validity and sensitivity to change were assessed. Convergent validity with SF-36 and WOMAC questionnaires was evaluated. Results 316 patients were included. Feasibility: a high number of missing items in questions 3, 4 and 5 were observed. The number of patients with a missing item was 171 (51.35%) in the preoperative visit and 139 (44.0%) at the postoperative. Internal validity: revision of coefficients in the item-rest correlation recommended removing question 6 during the preoperative visit (coefficient <0.20). Convergent validity: coefficients of correlation with WOMAC and SF-36 scales confirm the questionnaire’s validity. Sensitivity to change: statistically significant differences were found between the mean scores of the first visit compared to the postoperative. Conclusion The proposed translation to Spanish of the ‘Hip and Knee Questionnaire’ is found to be reliable, valid and sensible to changes produced at the clinical practice of patients undergoing primary knee arthroplasty. However, some changes at the completion instructions are recommended. Level of evidence: Level I. Prognostic study. PMID:24885248

The Comprehensive Care Project: Measuring Physician Performance in Ambulatory Practice

PubMed Central

Holmboe, Eric S; Weng, Weifeng; Arnold, Gerald K; Kaplan, Sherrie H; Normand, Sharon-Lise; Greenfield, Sheldon; Hood, Sarah; Lipner, Rebecca S

2010-01-01

Objective To investigate the feasibility, reliability, and validity of comprehensively assessing physician-level performance in ambulatory practice. Data Sources/Study Setting Ambulatory-based general internists in 13 states participated in the assessment. Study Design We assessed physician-level performance, adjusted for patient factors, on 46 individual measures, an overall composite measure, and composite measures for chronic, acute, and preventive care. Between- versus within-physician variation was quantified by intraclass correlation coefficients (ICC). External validity was assessed by correlating performance on a certification exam. Data Collection/Extraction Methods Medical records for 236 physicians were audited for seven chronic and four acute care conditions, and six age- and gender-appropriate preventive services. Principal Findings Performance on the individual and composite measures varied substantially within (range 5–86 percent compliance on 46 measures) and between physicians (ICC range 0.12–0.88). Reliabilities for the composite measures were robust: 0.88 for chronic care and 0.87 for preventive services. Higher certification exam scores were associated with better performance on the overall (r = 0.19; p <.01), chronic care (r = 0.14, p = .04), and preventive services composites (r = 0.17, p = .01). Conclusions Our results suggest that reliable and valid comprehensive assessment of the quality of chronic and preventive care can be achieved by creating composite measures and by sampling feasible numbers of patients for each condition. PMID:20819110

Development and validation of the TOCO-TURBT tool: a summative assessment tool that measures surgical competency in transurethral resection of bladder tumour.

PubMed

de Vries, Anna H; Muijtjens, Arno M M; van Genugten, Hilde G J; Hendrikx, Ad J M; Koldewijn, Evert L; Schout, Barbara M A; van der Vleuten, Cees P M; Wagner, Cordula; Tjiam, Irene M; van Merriënboer, Jeroen J G

2018-06-05

The current shift towards competency-based residency training has increased the need for objective assessment of skills. In this study, we developed and validated an assessment tool that measures technical and non-technical competency in transurethral resection of bladder tumour (TURBT). The 'Test Objective Competency' (TOCO)-TURBT tool was designed by means of cognitive task analysis (CTA), which included expert consensus. The tool consists of 51 items, divided into 3 phases: preparatory (n = 15), procedural (n = 21), and completion (n = 15). For validation of the TOCO-TURBT tool, 2 TURBT procedures were performed and videotaped by 25 urologists and 51 residents in a simulated setting. The participants' degree of competence was assessed by a panel of eight independent expert urologists using the TOCO-TURBT tool. Each procedure was assessed by two raters. Feasibility, acceptability and content validity were evaluated by means of a quantitative cross-sectional survey. Regression analyses were performed to assess the strength of the relation between experience and test scores (construct validity). Reliability was analysed by generalizability theory. The majority of assessors and urologists indicated the TOCO-TURBT tool to be a valid assessment of competency and would support the implementation of the TOCO-TURBT assessment as a certification method for residents. Construct validity was clearly established for all outcome measures of the procedural phase (all r > 0.5, p < 0.01). Generalizability-theory analysis showed high reliability (coefficient Phi ≥ 0.8) when using the format of two assessors and two cases. This study provides first evidence that the TOCO-TURBT tool is a feasible, valid and reliable assessment tool for measuring competency in TURBT. The tool has the potential to be used for future certification of competencies for residents and urologists. The methodology of CTA might be valuable in the development of assessment tools in other areas of clinical practice.

The PU-PROM: A patient-reported outcome measure for peptic ulcer disease.

PubMed

Liu, Na; Lv, Jing; Liu, Jinchun; Zhang, Yanbo

2017-12-01

Patient-reported outcome measure (PROM) conceived to enable description of treatment-related effects, from the patient perspective, bring the potential to improve in clinical research, and to provide patients with accurate information. Therefore, the aim of this study was to develop a patient-centred peptic ulcer patient-reported outcome measure (PU-PROM) and evaluate its reliability, validity, differential item functioning (DIF) and feasibility. To develop a conceptual framework and item pool for the PU-PROM, we performed a literature review and consulted other measures created in China and other countries. Beyond that, we interviewed 10 patients with peptic ulcers, and consulted six key experts to ensure that all germane parameters were included. In the first item selection phase, classical test theory and item response theory were used to select and adjust items to shape the preliminary measure completed by 130 patients and 50 controls. In the next phase, the measure was evaluated used the same methods with 492 patients and 124 controls. Finally, we used the same population in the second item reselection to assess the reliability, validity, DIF and feasibility of the final measure. The final peptic ulcer PRO measure comprised four domains (physiology, psychology, society and treatment), with 11 subdomains, and 54 items. The Cronbach's α coefficient of each subdomain for the measure was >0.800. Confirmatory factory analysis indicated that the construct validity fulfilled expectations. Model fit indices, such as RMR, RMSEA, NFI, NNFI, CFI and IFI, showed acceptable fit. The measure showed a good response rate. The peptic ulcer PRO measure had good reliability, validity, DIF and feasibility, and can be used as a clinical research evaluation instrument with patients with peptic ulcers to assess their condition focus on treatment. This measure may also be applied in other health areas, especially in clinical trials of new drugs, and may be helpful in clinical decision making. © 2017 The Authors Health Expectations Published by John Wiley & Sons Ltd.

Validity and reliability of food security measures.

PubMed

Cafiero, Carlo; Melgar-Quiñonez, Hugo R; Ballard, Terri J; Kepple, Anne W

2014-12-01

This paper reviews some of the existing food security indicators, discussing the validity of the underlying concept and the expected reliability of measures under reasonably feasible conditions. The main objective of the paper is to raise awareness on existing trade-offs between different qualities of possible food security measurement tools that must be taken into account when such tools are proposed for practical application, especially for use within an international monitoring framework. The hope is to provide a timely, useful contribution to the process leading to the definition of a food security goal and the associated monitoring framework within the post-2015 Development Agenda. © 2014 New York Academy of Sciences.

Patient Evaluation of Emotional Comfort Experienced (PEECE): developing and testing a measurement instrument

PubMed Central

Lester, L; Bulsara, C; Petterson, A; Bennett, K; Allen, E; Joske, D

2017-01-01

Objectives The Patient Evaluation of Emotional Comfort Experienced (PEECE) is a 12-item questionnaire which measures the mental well-being state of emotional comfort in patients. The instrument was developed using previous qualitative work and published literature. Design Instrument development. Setting Acute Care Public Hospital, Western Australia. Participants Sample of 374 patients. Interventions A multidisciplinary expert panel assessed the face and content validity of the instrument and following a pilot study, the psychometric properties of the instrument were explored. Main outcome measures Exploratory and confirmatory factor analysis assessed the underlying dimensions of the PEECE instrument; Cronbach's α was used to determine the reliability; κ was used for test–retest reliability of the ordinal items. Results 2 factors were identified in the instrument and named ‘positive emotions’ and ‘perceived meaning’. A greater proportion of male patients were found to report positive emotions compared with female patients. The instrument was found to be feasible, reliable and valid for use with inpatients and outpatients. Conclusions PEECE was found to be a feasible instrument for use with inpatient and outpatients, being easily understood and completed. Further psychometric testing is recommended. PMID:28122833

PedsQL gastrointestinal symptoms module feasibility reliability and validity

USDA-ARS?s Scientific Manuscript database

The objective of this study was to report on the measurement properties of the Pediatric Quality of Life Inventory (PedsQL) Gastrointestinal Symptoms Module for patients with functional gastrointestinal (GI) disorders (FGIDs) and organic GI diseases, hereafter referred to as "GI disorders," for pati...

Score Reliability and Validity of the Student Risk Screening Scale: A Psychometrically Sound, Feasible Tool for Use in Urban Middle Schools

ERIC Educational Resources Information Center

Lane, Kathleen Lynne; Bruhn, Allison L.; Eisner, Shanna L.; Kalberg, Jemma Robertson

2010-01-01

In this article, the authors examine the psychometric properties of the "Student Risk Screening Scale" (SRSS) for use in urban middle schools. Results of Studies 1 and 2 suggest strong internal consistency and test-retest stability. Study 1 supports the predictive validity of the SRSS, with students at low risk being able to be differentiated from…

Validation of the Penn Acoustic Neuroma Quality-of-Life Scale (PANQOL) for Spanish-Speaking Patients.

PubMed

Medina, Maria Del Mar; Carrillo, Alvaro; Polo, Ruben; Fernandez, Borja; Alonso, Daniel; Vaca, Miguel; Cordero, Adela; Perez, Cecilia; Muriel, Alfonso; Cobeta, Ignacio

2017-04-01

Objective To perform translation, cross-cultural adaptation, and validation of the Penn Acoustic Neuroma Quality-of-Life Scale (PANQOL) to the Spanish language. Study Design Prospective study. Setting Tertiary neurotologic referral center. Subjects and Methods PANQOL was translated and translated back, and a pretest trial was performed. The study included 27 individuals diagnosed with vestibular schwannoma. Inclusion criteria were adults with untreated vestibular schwannoma, diagnosed in the past 12 months. Feasibility, internal consistency, test-retest reliability, construct validity, and ceiling and floor effects were assessed for the present study. Results The mean overall score of the PANQOL was 69.21 (0-100 scale, lowest to highest quality of life). Cronbach's α was 0.87. Intraclass correlation coefficient was performed for each item, with an overall score of 0.92. The κ coefficient scores were between moderate and almost perfect in more than 92% of patients. Anxiety and energy domains of the PANQOL were correlated with both physical and mental components of the SF-12. Hearing, balance, and pain domains were correlated with the SF-12 physical component. Facial and general domains were not significantly correlated with any component of the SF-12. Furthermore, the overall score of the PANQOL was correlated with the physical component of the SF-12. Conclusion Feasibility, internal consistency, reliability, and construct validity outcomes in the current study support the validity of the Spanish version of the PANQOL.

Validity and Reliability of the Italian Version of the Functioning Assessment Short Test (FAST) in Bipolar Disorder

PubMed Central

Moro, Maria Francesca; Colom, Francesc; Floris, Francesca; Pintus, Elisa; Pintus, Mirra; Contini, Francesca; Carta, Mauro Giovanni

2012-01-01

Background: Functioning Assessment Short Test (FAST) is a brief instrument designed to assess the main functioning problems experienced by psychiatric patients, specifically bipolar patients. It includes 24 items assessing impairment or disability in six domains of functioning: autonomy, occupational functioning, cognitive functioning, financial issues, interpersonal relationships and leisure time. The aim of this study is to measure the validity and reliability of the Italian version of this instrument. Methods: Twenty-four patients with DSM-IV TR bipolar disorder and 20 healthy controls were recruited and evaluated in three private clinics in Cagliari (Sardinia, Italy). The psychometric properties of FAST (feasibility, internal consistency, concurrent validity, discriminant validity (patients vs controls and eutimic patients vs manic and depressed), and test-retest reliability were analyzed. Results: The internal consistency obtained was very high with a Cronbach's alpha of 0.955. A highly significant negative correlation with GAF was obtained (r = -0.9; p < 0.001) pointing to a reasonable degree of concurrent validity. FAST show a good test-retest reliability between two independent evaluation differing of one week (mean K =0.73). The total FAST scores were lower in controls as compared with Bipolar Patients and in Euthimic patients compared with Depressed or Manic. Conclusion: The Italian version of the FAST showed similar psychometrics properties as far as regard internal consistency and discriminant validity of the original version and show a good test retest reliability measure by means of K statistics. PMID:22905035

The Utrecht questionnaire (U-CEP) measuring knowledge on clinical epidemiology proved to be valid.

PubMed

Kortekaas, Marlous F; Bartelink, Marie-Louise E L; de Groot, Esther; Korving, Helen; de Wit, Niek J; Grobbee, Diederick E; Hoes, Arno W

2017-02-01

Knowledge on clinical epidemiology is crucial to practice evidence-based medicine. We describe the development and validation of the Utrecht questionnaire on knowledge on Clinical epidemiology for Evidence-based Practice (U-CEP); an assessment tool to be used in the training of clinicians. The U-CEP was developed in two formats: two sets of 25 questions and a combined set of 50. The validation was performed among postgraduate general practice (GP) trainees, hospital trainees, GP supervisors, and experts. Internal consistency, internal reliability (item-total correlation), item discrimination index, item difficulty, content validity, construct validity, responsiveness, test-retest reliability, and feasibility were assessed. The questionnaire was externally validated. Internal consistency was good with a Cronbach alpha of 0.8. The median item-total correlation and mean item discrimination index were satisfactory. Both sets were perceived as relevant to clinical practice. Construct validity was good. Both sets were responsive but failed on test-retest reliability. One set took 24 minutes and the other 33 minutes to complete, on average. External GP trainees had comparable results. The U-CEP is a valid questionnaire to assess knowledge on clinical epidemiology, which is a prerequisite for practicing evidence-based medicine in daily clinical practice. Copyright © 2016 Elsevier Inc. All rights reserved.

The Mayo Dysphagia Questionnaire-30: documentation of reliability and validity of a tool for interventional trials in adults with esophageal disease.

PubMed

McElhiney, Judith; Lohse, Matthew R; Arora, Amindra S; Peloquin, Joanna M; Geno, Debra M; Kuntz, Melissa M; Enders, Felicity B; Fredericksen, Mary; Abdalla, Adil A; Khan, Yulia; Talley, Nicholas J; Diehl, Nancy N; Beebe, Timothy J; Harris, Ann M; Farrugia, Gianrico; Graner, Darlene E; Murray, Joseph A; Locke, G Richard; Grothe, Rayna M; Crowell, Michael D; Francis, Dawn L; Grudell, April M B; Dabade, Tushar; Ramirez, Angelica; Alkhatib, MhdMaan; Alexander, Jeffrey A; Kimber, Jessica; Prasad, Ganapathy; Zinsmeister, Alan R; Romero, Yvonne

2010-09-01

The aim of this study was to develop the Mayo Dysphagia Questionnaire-30 Day (MDQ-30), a tool to measure esophageal dysphagia, by adapting items from validated instruments for use in clinical trials, and assess its feasibility, reproducibility, and concurrent validity. Outpatients referred to endoscopy for dysphagia or seen in a specialty clinic were recruited. Feasibility testing was done to identify problematic items. Reproducibility was measured by test-retest format. Concurrent validity reflects agreement between information gathered in a structured interview versus the patients' written responses. The MDQ-30, a 28-item instrument, took 10 min (range = 5-30 min) to complete. Four hundred thirty-one outpatients [210 (49%) men; mean age = 61 years] participated. Overall, most concurrent validity kappa values for dysphagia were very good to excellent with a median of 0.78 (min 0.28, max 0.95). The majority of reproducibility kappa values for dysphagia were moderate to excellent with a median kappa value of 0.66 (min 0.07, max 1.0). Overall, concurrent validity and reproducibility kappa values for gastroesophageal reflux disease (GERD) symptoms were 0.81 (95% CI = 0.72, 0.91) and 0.66 (95% CI = 0.55, 0.77), respectively. Individual item percent agreement was generally very good to excellent. Internal consistency was excellent. We conclude that the MDQ-30 is an easy-to-complete tool to evaluate reliably dysphagia symptoms over the last 30 days.

Measuring Food Use in School-Aged Children.

ERIC Educational Resources Information Center

Randall, Elizabeth

1991-01-01

Ideal measurement of food use in school-aged children possesses validity, reliability, interpretability, and feasibility. The article discusses dietary records and recalls and measures of patterns in food use. Issues to consider when constructing children's food frequency are food list, time intervals, response set, context of questioning, and…

Measuring the anaesthesia clinical learning environment at the department level is feasible and reliable.

PubMed

Castanelli, D J; Smith, N A

2017-05-01

The learning environment describes the context and culture in which trainees learn. In order to establish the feasibility and reliability of measuring the anaesthetic learning environment in individual departments we implemented a previously developed instrument in hospitals across New South Wales. We distributed the instrument to trainees from 25 anaesthesia departments and supplied summarized results to individual departments. Exploratory and confirmatory factor analyses were performed to assess internal structure validity and generalizability theory was used to calculate reliability. The number of trainees required for acceptable precision in results was determined using the standard error of measurement. We received 172 responses (59% response rate). Suitable internal structure validity was confirmed. Measured reliability was acceptable (G-coefficient 0.69) with nine trainees per department. Eight trainees were required for a 95% confidence interval of plus or minus 0.25 in the mean total score. Eight trainees as assessors also allow a 95% confidence interval of approximately plus or minus 0.3 in the subscale mean scores. Results for individual departments varied, with scores below the expected level recorded on individual subscales, particularly the 'teaching' subscale. Our results confirm that, using this instrument, individual departments can obtain acceptable precision in results with achievable trainee numbers. Additionally, with the exception of departments with few trainees, implementation proved feasible across a training region. Repeated use would allow departments or accrediting bodies to monitor their individual learning environment and the impact of changes such as the introduction of new curricular elements, or local initiatives to improve trainee experience. © The Author 2017. Published by Oxford University Press on behalf of the British Journal of Anaesthesia. All rights reserved. For Permissions, please email: journals.permissions@oup.com

Parent proxy-report of their children's health-related quality of life: an analysis of 13,878 parents' reliability and validity across age subgroups using the PedsQL 4.0 Generic Core Scales.

PubMed

Varni, James W; Limbers, Christine A; Burwinkle, Tasha M

2007-01-03

Health-related quality of life (HRQOL) measurement has emerged as an important health outcome in clinical trials, clinical practice improvement strategies, and healthcare services research and evaluation. While pediatric patient self-report should be considered the standard for measuring perceived HRQOL, there are circumstances when children are too young, too cognitively impaired, too ill or fatigued to complete a HRQOL instrument, and reliable and valid parent proxy-report instruments are needed in such cases. Further, it is typically parents' perceptions of their children's HRQOL that influences healthcare utilization. Data from the PedsQL DatabaseSM were utilized to test the reliability and validity of parent proxy-report at the individual age subgroup level for ages 2-16 years as recommended by recent FDA guidelines. The sample analyzed represents parent proxy-report age data on 13,878 children ages 2 to 16 years from the PedsQL 4.0 Generic Core Scales DatabaseSM. Parents were recruited from general pediatric clinics, subspecialty clinics, and hospitals in which their children were being seen for well-child checks, mild acute illness, or chronic illness care (n = 3,718, 26.8%), and from a State Children's Health Insurance Program (SCHIP) in California (n = 10,160, 73.2%). The percentage of missing item responses for the parent proxy-report sample as a whole was 2.1%, supporting feasibility. The majority of the parent proxy-report scales across the age subgroups exceeded the minimum internal consistency reliability standard of 0.70 required for group comparisons, while the Total Scale Scores across the age subgroups approached or exceeded the reliability criterion of 0.90 recommended for analyzing individual patient scale scores. Construct validity was demonstrated utilizing the known groups approach. For each PedsQL scale and summary score, across age subgroups, healthy children demonstrated a statistically significant difference in HRQOL (better HRQOL) than children with a known chronic health condition, with most effect sizes in the medium to large effect size range. The results demonstrate the feasibility, reliability, and validity of parent proxy-report at the individual age subgroup for ages 2-16 years. These analyses are consistent with recent FDA guidelines which require instrument development and validation testing for children and adolescents within fairly narrow age groupings and which determine the lower age limit at which reliable and valid responses across age categories are achievable. Even as pediatric patient self-report is advocated, there remains a fundamental role for parent proxy-report in pediatric clinical trials and health services research.

Invited review: Animal-based indicators for on-farm welfare assessment for dairy goats.

PubMed

Battini, M; Vieira, A; Barbieri, S; Ajuda, I; Stilwell, G; Mattiello, S

2014-11-01

This paper reviews animal-based welfare indicators to develop a valid, reliable, and feasible on-farm welfare assessment protocol for dairy goats. The indicators were considered in the light of the 4 accepted principles (good feeding, good housing, good health, appropriate behavior) subdivided into 12 criteria developed by the European Welfare Quality program. We will only examine the practical indicators to be used on-farm, excluding those requiring the use of specific instruments or laboratory analysis and those that are recorded at the slaughterhouse. Body condition score, hair coat condition, and queuing at the feed barrier or at the drinker seem the most promising indicators for the assessment of the "good feeding" principle. As to "good housing," some indicators were considered promising for assessing "comfort around resting" (e.g., resting in contact with a wall) or "thermal comfort" (e.g., panting score for the detection of heat stress and shivering score for the detection of cold stress). Several indicators related to "good health," such as lameness, claw overgrowth, presence of external abscesses, and hair coat condition, were identified. As to the "appropriate behavior" principle, different criteria have been identified: agonistic behavior is largely used as the "expression of social behavior" criterion, but it is often not feasible for on-farm assessment. Latency to first contact and the avoidance distance test can be used as criteria for assessing the quality of the human-animal relationship. Qualitative behavior assessment seems to be a promising indicator for addressing the "positive emotional state" criterion. Promising indicators were identified for most of the considered criteria; however, no valid indicator has been identified for "expression of other behaviors." Interobserver reliability has rarely been assessed and warrants further attention; in contrast, short-term intraobserver reliability is frequently assessed and some studies consider mid- and long-term reliability. The feasibility of most of the reviewed indicators in commercial farms still needs to be carefully evaluated, as several studies were performed under experimental conditions. Our review highlights some aspects of goat welfare that have been widely studied, but some indicators need to be investigated further and drafted before being included in a valid, reliable, and feasible welfare assessment protocol. The indicators selected and examined may be an invaluable starting point for the development of an on-farm welfare assessment protocol for dairy goats. Copyright © 2014 American Dairy Science Association. Published by Elsevier Inc. All rights reserved.

Measuring Working Memory Deficits in Aphasia

ERIC Educational Resources Information Center

Mayer, Jamie F.; Murray, Laura L.

2012-01-01

Purpose: Many adults with aphasia demonstrate concomitant deficits in working memory (WM), but such deficits are difficult to quantify because of a lack of validated measures as well as the complex interdependence between language and WM. We examined the feasibility, reliability, and internal consistency of an "n"-back task for…

Physical Activity Measurement Methods for Young Children: A Comparative Study

ERIC Educational Resources Information Center

Hands, Beth; Parker, Helen; Larkin, Dawne

2006-01-01

Many behavior patterns that impact on physical activity experiences are established in early childhood, therefore it is important that valid, reliable, and feasible measures are constructed to identify children who are not developing appropriate and healthy activity habits. In this study, measures of physical activity derived by accelerometry and…

Methodological Approaches to Online Scoring of Essays.

ERIC Educational Resources Information Center

Chung, Gregory K. W. K.; O'Neil, Harold F., Jr.

This report examines the feasibility of scoring essays using computer-based techniques. Essays have been incorporated into many of the standardized testing programs. Issues of validity and reliability must be addressed to deploy automated approaches to scoring fully. Two approaches that have been used to classify documents, surface- and word-based…

Reliability of 3D laser-based anthropometry and comparison with classical anthropometry.

PubMed

Kuehnapfel, Andreas; Ahnert, Peter; Loeffler, Markus; Broda, Anja; Scholz, Markus

2016-05-26

Anthropometric quantities are widely used in epidemiologic research as possible confounders, risk factors, or outcomes. 3D laser-based body scans (BS) allow evaluation of dozens of quantities in short time with minimal physical contact between observers and probands. The aim of this study was to compare BS with classical manual anthropometric (CA) assessments with respect to feasibility, reliability, and validity. We performed a study on 108 individuals with multiple measurements of BS and CA to estimate intra- and inter-rater reliabilities for both. We suggested BS equivalents of CA measurements and determined validity of BS considering CA the gold standard. Throughout the study, the overall concordance correlation coefficient (OCCC) was chosen as indicator of agreement. BS was slightly more time consuming but better accepted than CA. For CA, OCCCs for intra- and inter-rater reliability were greater than 0.8 for all nine quantities studied. For BS, 9 of 154 quantities showed reliabilities below 0.7. BS proxies for CA measurements showed good agreement (minimum OCCC > 0.77) after offset correction. Thigh length showed higher reliability in BS while upper arm length showed higher reliability in CA. Except for these issues, reliabilities of CA measurements and their BS equivalents were comparable.

An Approach to measuring Integrated Care within a Maternity Care System: Experiences from the Maternity Care Network Study and the Dutch Birth Centre Study.

PubMed

Boesveld, Inge C; Valentijn, Pim P; Hitzert, Marit; Hermus, Marieke A A; Franx, Arie; de Vries, Raymond G; Wiegers, Therese A; Bruijnzeels, Marc A

2017-06-20

Integrated care is considered to be a means to reduce costs, improve the quality of care and generate better patient outcomes. At present, little is known about integrated care in maternity care systems. We developed questionnaires to examine integrated care in two different settings, using the taxonomy of the Rainbow Model of Integrated Care. The aim of this study was to explore the validity of these questionnaires. We used data collected between 2013 and 2015 from two studies: the Maternity Care Network Study (634 respondents) and the Dutch Birth Centre Study (56 respondents). We assessed the feasibility, discriminative validity, and reliability of the questionnaires. Both questionnaires showed good feasibility (overall missing rate < 20%) and reliability (Cronbach's Alpha coefficient > 0.70). Between-subgroups post-hoc comparisons showed statistically significant differences on integration profiles between regional networks (on all items, dimensions of integration and total integration score) and birth centres (on 50% of the items and dimensions of integration). Both questionnaires are feasible and can discriminate between sites with different integration profiles in The Netherlands. They offer an opportunity to better understand integrated care as one step in understanding the complexity of the concept.

An Approach to measuring Integrated Care within a Maternity Care System: Experiences from the Maternity Care Network Study and the Dutch Birth Centre Study

PubMed Central

Valentijn, Pim P.; Hitzert, Marit; Hermus, Marieke A.A.; Franx, Arie; de Vries, Raymond G.; Wiegers, Therese A.; Bruijnzeels, Marc A.

2017-01-01

Introduction: Integrated care is considered to be a means to reduce costs, improve the quality of care and generate better patient outcomes. At present, little is known about integrated care in maternity care systems. We developed questionnaires to examine integrated care in two different settings, using the taxonomy of the Rainbow Model of Integrated Care. The aim of this study was to explore the validity of these questionnaires. Methods: We used data collected between 2013 and 2015 from two studies: the Maternity Care Network Study (634 respondents) and the Dutch Birth Centre Study (56 respondents). We assessed the feasibility, discriminative validity, and reliability of the questionnaires. Results: Both questionnaires showed good feasibility (overall missing rate < 20%) and reliability (Cronbach’s Alpha coefficient > 0.70). Between-subgroups post-hoc comparisons showed statistically significant differences on integration profiles between regional networks (on all items, dimensions of integration and total integration score) and birth centres (on 50% of the items and dimensions of integration). Discussion: Both questionnaires are feasible and can discriminate between sites with different integration profiles in The Netherlands. They offer an opportunity to better understand integrated care as one step in understanding the complexity of the concept. PMID:28970747

The PedsQL multidimensional fatigue scale in pediatric obesity: feasibility, reliability and validity.

PubMed

Varni, James W; Limbers, Christine A; Bryant, William P; Wilson, Don P

2010-01-01

The PedsQL (Pediatric Quality of Life Inventory) is a modular instrument designed to measure health-related quality of life (HRQOL) and disease-specific symptoms in children and adolescents. The PedsQL Multidimensional Fatigue Scale was designed as a child self-report and parent proxy-report generic symptom-specific instrument to measure fatigue in pediatric patients. The objective of the present study was to determine the feasibility, reliability, and validity of the PedsQL Multidimensional Fatigue Scale in pediatric obesity. The 18-item PedsQL Multidimensional Fatigue Scale (General Fatigue, Sleep/Rest Fatigue, and Cognitive Fatigue domains) and the PedsQL 4.0 Generic Core Scales were completed by 41 pediatric patients with a physician-diagnosis of obesity and 43 parents from a hospital-based Pediatric Endocrinology Clinic. The PedsQL Multidimensional Fatigue Scale evidenced minimal missing responses (1.6%, child report; 0.5%, parent report), achieved excellent reliability for the Total Fatigue Scale Score (alpha = 0.90 child report, 0.90 parent report), distinguished between pediatric patients with obesity and healthy children, and was significantly correlated with the PedsQL 4.0 Generic Core Scales supporting construct validity. Pediatric patients with obesity experienced fatigue comparable with pediatric patients receiving cancer treatment, demonstrating the relative severity of their fatigue symptoms. The results demonstrate the measurement properties of the PedsQL Multidimensional Fatigue Scale in pediatric obesity. The findings suggest that the PedsQL Multidimensional Fatigue Scale may be utilized in the standardized evaluation of fatigue in pediatric patients with obesity.

Improving the governance of patient safety in emergency care: a systematic review of interventions

PubMed Central

Hesselink, Gijs; Berben, Sivera; Beune, Thimpe

2016-01-01

Objectives To systematically review interventions that aim to improve the governance of patient safety within emergency care on effectiveness, reliability, validity and feasibility. Design A systematic review of the literature. Methods PubMed, EMBASE, Cumulative Index to Nursing and Allied Health Literature, the Cochrane Database of Systematic Reviews and PsychInfo were searched for studies published between January 1990 and July 2014. We included studies evaluating interventions relevant for higher management to oversee and manage patient safety, in prehospital emergency medical service (EMS) organisations and hospital-based emergency departments (EDs). Two reviewers independently selected candidate studies, extracted data and assessed study quality. Studies were categorised according to study quality, setting, sample, intervention characteristics and findings. Results Of the 18 included studies, 13 (72%) were non-experimental. Nine studies (50%) reported data on the reliability and/or validity of the intervention. Eight studies (44%) reported on the feasibility of the intervention. Only 4 studies (22%) reported statistically significant effects. The use of a simulation-based training programme and well-designed incident reporting systems led to a statistically significant improvement of safety knowledge and attitudes by ED staff and an increase of incident reports within EDs, respectively. Conclusions Characteristics of the interventions included in this review (eg, anonymous incident reporting and validation of incident reports by an independent party) could provide useful input for the design of an effective tool to govern patient safety in EMS organisations and EDs. However, executives cannot rely on a robust set of evidence-based and feasible tools to govern patient safety within their emergency care organisation and in the chain of emergency care. Established strategies from other high-risk sectors need to be evaluated in emergency care settings, using an experimental design with valid outcome measures to strengthen the evidence base. PMID:26826151

Conducting pilot and feasibility studies.

PubMed

Cope, Diane G

2015-03-01

Planning a well-designed research study can be tedious and laborious work. However, this process is critical and ultimately can produce valid, reliable study findings. Designing a large-scale randomized, controlled trial (RCT)-the gold standard in quantitative research-can be even more challenging. Even the most well-planned study potentially can result in issues with research procedures and design, such as recruitment, retention, or methodology. One strategy that may facilitate sound study design is the completion of a pilot or feasibility study prior to the initiation of a larger-scale trial. This article will discuss pilot and feasibility studies, their advantages and disadvantages, and implications for oncology nursing research. 
.

Study Protocol on Ecological Momentary Assessment of Health-Related Quality of Life Using a Smartphone Application.

PubMed

Mareva, Silvana; Thomson, David; Marenco, Pietro; Estal Muñoz, Víctor; Ott, Caroline V; Schmidt, Barbara; Wingen, Tobias; Kassianos, Angelos P

2016-01-01

Health-Related Quality of Life (HRQoL) is a construct of increasing importance in modern healthcare, and has typically been assessed using retrospective instruments. While such measures have been shown to have predictive utility for clinical outcomes, several cognitive biases associated with human recall and current mood state may undermine their validity and reliability. Retrospective tools can be further criticized for their lack of ecology, as individuals are usually assessed in less natural settings such as hospitals and health centers, and may be obliged to spend time and money traveling to receive assessment. Ecological momentary assessment (EMA) is an alternative, as mobile assessment using mobile health (mHealth) technology has the potential to minimize biases and overcome many of these limitations. Employing an EMA methodology, we will use a smartphone application to collect data on real-time HRQoL, with an adapted version of the widely used WHOQOL-BREF questionnaire. We aim to recruit a total of 450 healthy participants. Participants will be prompted by the application to report their real-time HRQoL over 2 weeks together with information on mood and current activities. At the end of 2 weeks, they will complete a retrospective assessment of their HRQoL and they will provide information about their sleep quality and perceived stress. The psychometric properties of real-time HRQoL will be assessed, including analysis of the factorial structure, reliability and validity of the measure, and compared with retrospective HRQoL responses for the same 2-week testing period. Further, we aim to identify factors associated with real-time HRQoL (e.g., mood, activities), the feasibility of the application, and within- and between-person variability in real-time HRQoL. We expect real-time HRQoL to have adequate validity and reliability, and positive responses on the feasibility of using a smartphone application for routine HRQoL assessment. The direct comparison of real-time and retrospective measures in this study will provide important novel insight into the efficacy of mHealth applications for HRQoL assessment. If shown to be valid, reliable and feasible for the collection of HRQoL data, mHealth applications may have future potential for facilitating clinical assessment, patient-physician communication, and monitoring individual HRQoL over course of treatment.

Documentation of pharmaceutical care: Validation of an intervention oriented classification system.

PubMed

Maes, Karen A; Studer, Helene; Berger, Jérôme; Hersberger, Kurt E; Lampert, Markus L

2017-12-01

During the dispensing process, pharmacists may come across technical and clinical issues requiring a pharmaceutical intervention (PI). An intervention-oriented classification system is a helpful tool to document these PIs in a structured manner. Therefore, we developed the PharmDISC classification system (Pharmacists' Documentation of Interventions in Seamless Care). The aim of this study was to evaluate the PharmDISC system in the daily practice environment (in terms of interrater reliability, appropriateness, interpretability, acceptability, feasibility, and validity); to assess its user satisfaction, the descriptive manual, and the online training; and to explore first implementation aspects. Twenty-one pharmacists from different community pharmacies each classified 30 prescriptions requiring a PI with the PharmDISC system on 5 selected days within 5 weeks. Interrater reliability was determined using model PIs and Fleiss's kappa coefficients (κ) were calculated. User satisfaction was assessed by questionnaire with a 4-point Likert scale. The main outcome measures were interrater reliability (κ); appropriateness, interpretability, validity (ratio of completely classified PIs/all PIs); feasibility, and acceptability (user satisfaction and suggestions). The PharmDISC system reached an average substantial agreement (κ = 0.66). Of documented 519 PIs, 430 (82.9%) were completely classified. Most users found the system comprehensive (median user agreement 3 [2/3.25 quartiles]) and practical (3[2.75/3]). The PharmDISC system raised the awareness regarding drug-related problems for most users (n = 16). To facilitate its implementation, an electronic version that automatically connects to the prescription together with a task manager for PIs needing follow-up was suggested. Barriers could be time expenditure and lack of understanding the benefits. Substantial interrater reliability and acceptable user satisfaction indicate that the PharmDISC system is a valid system to document PIs in daily community pharmacy practice. © 2017 John Wiley & Sons, Ltd.

French adaptation of the new Knee Society Scoring System for total knee arthroplasty.

PubMed

Debette, C; Parratte, S; Maucort-Boulch, D; Blanc, G; Pauly, V; Lustig, S; Servien, E; Neyret, P; Argenson, J N

2014-09-01

In November 2011, the Knee Society published its new KSS score to evaluate objective clinical data and also patient expectations, satisfaction and knee function during various physical activities before and after total knee arthroplasty (TKA). We undertook the French cross-cultural adaptation of this scoring system according to current recommendations. The French version of the new KSS score is a consistent, feasible, reliable and discriminating score. Eighty patients with knee osteoarthritis were recruited from two centers: one group of 40 patients had a TKA indication, while the other group of 40 patients had an indication for conservative treatment. After the new KSS score was translated and back-translated, it was compared to three other validated instruments (KOOS, AMIQUAL and SF-12) to determine construct validity, discriminating power, feasibility in terms of response rate and existence of floor or ceiling effect, internal consistency with Chronbach's alpha and reliability based on reproducibility and sensitivity to change (responsiveness). Due to missing data, two cases were eliminated. We found that the score could discriminate between groups; it had a nearly 100% response rate, a ceiling effect in the "expectations" domain, satisfactory Chronbach's alpha, excellent reproducibility and good responsiveness. These results confirm that the French version of the new KSS score is reliable, feasible, discriminating, consistent and responsive. The novelty of this scoring system resides in the "expectations" and "satisfaction" domains, its availability as a self-assessment questionnaire and the evaluation of function during various activities. Level III. Copyright © 2014 Elsevier Masson SAS. All rights reserved.

Validity of the Spanish 8-item short-form generic health-related quality-of-life questionnaire in surgical patients: a population-based study.

PubMed

Vallès, Jordi; Guilera, Magda; Briones, Zahara; Gomar, Carmen; Canet, Jaume; Alonso, Jordi

2010-05-01

Health-related quality of life is usually reported for specific rather than heterogeneous populations such as those treated in routine anesthesia practice. The 8-item short-form generic health-related quality-of-life questionnaire (SF-8) is a candidate instrument for this setting. The authors evaluated the feasibility, reliability, validity, and responsiveness to change of the Spanish version of SF-8 in a population-based surgical cohort. Recruiting patients from a large population-based study of risk factors for pulmonary complications, before surgery, the authors administered the 1-week recall SF-8 to 2,991 patients undergoing nonobstetric elective or emergency surgery in 59 hospitals, each of which collected data on seven randomly assigned days in 2006. The SF-8 was administered again 3 months later. Reliability was evaluated using the Cronbach alpha coefficient and validity by comparing physical and mental component summary SF-8 scores with clinical variables. Responsiveness after surgery was evaluated using the standardized response mean. Cronbach alpha for the overall test was 0.92. Physical and mental component summary scores and all individual scores were lower (worse quality of life) in women (P < 0. 01) and decreased with age (P < 0.01). Preoperative scores were lower for those in worse clinical condition (higher body mass index, American Society of Anesthesiologists physical status class, or surgical risk scores), with preoperative respiratory symptoms, and in emergency situations (P < 0.01). The standardized response mean ranged from 0.1 to 0.5. The SF-8 is a feasible, reliable, valid, and responsive instrument for assessing health-related quality of life in a broad-spectrum surgical population.

Montreal Cognitive Assessment 5-minute protocol is a brief, valid, reliable, and feasible cognitive screen for telephone administration.

PubMed

Wong, Adrian; Nyenhuis, David; Black, Sandra E; Law, Lorraine S N; Lo, Eugene S K; Kwan, Pauline W L; Au, Lisa; Chan, Anne Y Y; Wong, Lawrence K S; Nasreddine, Ziad; Mok, Vincent

2015-04-01

The National Institute of Neurological Disorders and Stroke-Canadian Stroke Network Vascular Cognitive Impairment Harmonization working group proposed a brief cognitive protocol for screening of vascular cognitive impairment. We investigated the validity, reliability, and feasibility of the Montreal Cognitive Assessment 5-minute protocol (MoCA 5-minute protocol) administered over the telephone. Four items examining attention, verbal learning and memory, executive functions/language, and orientation were extracted from the MoCA to form the MoCA 5-minute protocol. One hundred four patients with stroke or transient ischemic attack, including 53 with normal cognition (Clinical Dementia Rating, 0) and 51 with cognitive impairment (Clinical Dementia Rating, 0.5 or 1), were administered the MoCA in clinic and a month later, the MoCA 5-minute protocol over the telephone. Administration of the MoCA 5-minute protocol took 5 minutes over the telephone. Total score of the MoCA 5-minute protocol correlated negatively with age (r=-0.36; P<0.001) and positively with years of education (r=0.41; P<0.001) but not with sex (ρ=0.03; P=0.773). Total scores of the MoCA and MoCA 5-minute protocol were highly correlated (r=0.87; P<0.001). The MoCA 5-minute protocol performed equally well as the MoCA in differentiating patients with cognitive impairment from those without (areas under receiver operating characteristics curve for MoCA 5-minute protocol, 0.78; MoCA=0.74; P>0.05 for difference; Cohen d for group difference, 0.80-1.13). It differentiated cognitively impaired patients with executive domain impairment from those without (areas under receiver operating characteristics curve, 0.89; P<0.001; Cohen d=1.7 for group difference). Thirty-day test-retest reliability was excellent (intraclass correlation coefficient, 0.89). The MoCA 5-minute protocol is a free, valid, and reliable cognitive screen for stroke and transient ischemic attack. It is brief and highly feasible for telephone administration. © 2015 American Heart Association, Inc.

In-Training Assessment Using Direct Observation of Single-Patient Encounters: A Literature Review

ERIC Educational Resources Information Center

Pelgrim, E. A. M.; Kramer, A. W. M.; Mokkink, H. G. A.; van den Elsen, L.; Grol, R. P. T. M.; van der Vleuten, C. P. M.

2011-01-01

We reviewed the literature on instruments for work-based assessment in single clinical encounters, such as the mini-clinical evaluation exercise (mini-CEX), and examined differences between these instruments in characteristics and feasibility, reliability, validity and educational effect. A PubMed search of the literature published before 8…

Pre-implementation studies of a workforce planning tool for nurse staffing and human resource management in university hospitals.

PubMed

van Oostveen, Catharina J; Ubbink, Dirk T; Mens, Marian A; Pompe, Edwin A; Vermeulen, Hester

2016-03-01

To investigate the reliability, validity and feasibility of the RAFAELA workforce planning system (including the Oulu patient classification system - OPCq), before deciding on implementation in Dutch hospitals. The complexity of care, budgetary restraints and demand for high-quality patient care have ignited the need for transparent hospital workforce planning. Nurses from 12 wards of two university hospitals were trained to test the reliability of the OPCq by investigating the absolute agreement of nursing care intensity (NCI) measurements among nurses. Validity was tested by assessing whether optimal NCI/nurse ratio, as calculated by a regression analysis in RAFAELA, was realistic. System feasibility was investigated through a questionnaire among all nurses involved. Almost 67 000 NCI measurements were performed between December 2013 and June 2014. Agreement using the OPCq varied between 38% and 91%. For only 1 in 12 wards was the optimal NCI area calculated judged as valid. Although the majority of respondents was positive about the applicability and user-friendliness, RAFAELA was not accepted as useful workforce planning system. Nurses' performance using the RAFAELA system did not warrant its implementation. Hospital managers should first focus on enlarging the readiness of nurses regarding the implementation of a workforce planning system. © 2015 John Wiley & Sons Ltd.

Validity, Reliability, and Feasibility of Durometer Measurements of Scleroderma Skin Disease in a Multicenter Treatment Trial

PubMed Central

MERKEL, PETER A.; SILLIMAN, NANCY P.; DENTON, CHRISTOPHER P.; FURST, DANIEL E.; KHANNA, DINESH; EMERY, PAUL; HSU, VIVIEN M.; STREISAND, JAMES B.; POLISSON, RICHARD P.; ÅKESSON, ANITA; COPPOCK, JOHN; van den HOOGEN, FRANK; HERRICK, ARIANE; MAYES, MAUREEN D.; VEALE, DOUGLAS; SEIBOLD, JAMES R.; BLACK, CAROL M.; KORN, JOSEPH H.

2013-01-01

Objective To determine the validity, reliability, and feasibility of durometer measurements of skin hardness as an outcome measure in clinical trials of scleroderma. Methods Skin hardness was measured during a multicenter treatment trial for scleroderma using handheld digital durometers with a continuous scale. Skin thickness was measured by modified Rodnan skin score (MRSS). Other outcome data collected included the Scleroderma Health Assessment Questionnaire. In a reliability exercise in advance of the trial, 9 investigators examined the same 5 scleroderma patients by MRSS and durometry. Results Forty-three patients with early diffuse cutaneous systemic sclerosis were studied at 11 international centers (mean age 49 years [range 24–76], median disease duration 6.4 months [range 0.3–23], and median baseline MRSS 22 [range 11–38]). The reliability of durometer measurements was excellent, with high interobserver intraclass correlation coefficients (ICCs) (0.82–0.92), and each result was greater than the corresponding skin site ICCs for MRSS (0.54–0.85). Baseline durometer scores correlated well with MRSS (r = 0.69, P < 0.0001), patient self-assessments of skin disease (r = 0.69, P < 0.0001), and Health Assessment Questionnaire (HAQ) disability scores (r = 0.34, P = 0.03). Change in durometer scores correlated with change in MRSS (r = 0.70, P < 0.0001), change in patient self-assessments of skin disease (r = 0.52, P = 0.003), and change in HAQ disability scores (r = 0.42, P = 0.017). The effect size was greater for durometry than for MRSS or patient self-assessment. Conclusion Durometer measurements of skin hardness in patients with scleroderma are reliable, simple, accurate, demonstrate good sensitivity to change compared with traditional skin scoring, and reflect patients' self-assessments of their disease. Durometer measurements are valid, objective, and scalable, and should be considered for use as a complementary outcome measure to skin scoring in clinical trials of scleroderma. PMID:18438905

A tool to assess sex-gender when selecting health research projects.

PubMed

Tomás, Concepción; Yago, Teresa; Eguiluz, Mercedes; Samitier, M A Luisa; Oliveros, Teresa; Palacios, Gemma

2015-04-01

To validate the questionnaire "Gender Perspective in Health Research" (GPIHR) to assess the inclusion of gender perspective in research projects. Validation study in two stages. Feasibility was analysed in the first, and reliability, internal consistence and validity in the second. Aragón Institute of Health Science, Aragón, Spain. GPIHR was applied to 118 research projects funded in national and international competitive tenders from 2003 to 2012. Analysis of inter- and intra-observer reliability with Kappa index and internal consistency with Cronbach's alpha. Content validity analysed through literature review and construct validity with an exploratory factor analysis. Validated GPIHR has 10 questions: 3 in the introduction, 1 for objectives, 3 for methodology and 3 for research purpose. Average time of application was 13min Inter-observer reliability (Kappa) varied between 0.35 and 0.94 and intra-observer between 0.40 and 0.94. Theoretical construct is supported in the literature. Factor analysis identifies three levels of GP inclusion: "difference by sex", "gender sensitive" and "feminist research" with an internal consistency of 0.64, 0.87 and 0.81, respectively, which explain 74.78% of variance. GPIHR questionnaire is a valid tool to assess GP and useful for those researchers who would like to include GP in their projects. Copyright © 2014 Elsevier España, S.L.U. All rights reserved.

Furthering the reliable and valid measurement of mental health screening, diagnoses, treatment and outcomes through health information technology.

PubMed

Haberer, Jessica E; Trabin, Tom; Klinkman, Michael

2013-01-01

Measurement of mental health is challenging; however, many solutions may be found through the use of health information technology. This article reviews current approaches to measuring mental health, focusing on screening, diagnosis, treatment, and outcomes. It then identifies several key areas in which health information technology may advance the field and provide reliable and valid measurements that are readily available to and manageable for providers, as well as acceptable, feasible, and sustainable for selected populations. Although new technologies must overcome many challenges, including privacy, efficiency, cost, and scalability, it is an exciting and fast-growing field with many potential applications and clinical benefit. Copyright © 2013 Elsevier Inc. All rights reserved.

A systematic approach for studying the signs and symptoms of fever in adult patients: the fever assessment tool (FAST).

PubMed

Ames, Nancy J; Powers, John H; Ranucci, Alexandra; Gartrell, Kyungsook; Yang, Li; VanRaden, Mark; Leidy, Nancy Kline; Wallen, Gwenyth R

2017-04-27

Although body temperature is one of four key vital signs routinely monitored and treated in clinical practice, relatively little is known about the symptoms associated with febrile states. The purpose of this study was to assess the validity, reliability and feasibility of the Fever Assessment Tool (FAST) in an acute care research setting. Qualitative: To assess content validity and finalize the FAST instrument, 12 adults from an inpatient medical-surgical unit at the National Institutes of Health (NIH) Clinical Center participated in cognitive interviews within approximately 12 h of a febrile state (tympanic temperature ≥ 38° Celsius). Quantitative: To test reliability, validity and feasibility, 56 new adult inpatients completed the 21-item FAST. The cognitive interviews clarified and validated the content of the final 21-item FAST. Fifty-six patients completed the FAST from two to 133 times during routine vital sign assessment, yielding 1,699 temperature time points. Thirty-four percent of the patients (N = 19) experienced fever at one or more time points, with a total of 125 febrile time points. Kuder-Richardson 20 (KR-20) reliability of the FAST was 0.70. Four nonspecific symptom categories, Tired or Run-Down (12), Sleepy (13), Weak or Lacking Energy (11), and Thirsty (9) were among the most frequently reported symptoms in all participants. Using Generalized Estimating Equations (GEE), the odds of reporting eight symptoms, Warm (4), Sweating (5), Thirsty (9), General Body Aches (10), Weak or Lacking Energy (11), Tired or Run Down (12) and Difficulty Breathing (17), were increased when patients had a fever (Fever Now), compared to the two other subgroups-patients who had a fever, but not at that particular time point, (Fever Not Now) and patients who never had a fever (Fever Never). Many, but not all, of the comparisons were significant in both groups. Results suggest the FAST is reliable, valid and easy to administer. In addition to symptoms usually associated with fever (e.g. feeling warm), symptoms such as Difficulty Breathing (17) were identified with fever. Further study in a larger, more diverse patient population is warranted. Clinical Trials Number: NCT01287143 (January 2011).

Comprehensive Deployment Method for Technical Characteristics Base on Multi-failure Modes Correlation Analysis

NASA Astrophysics Data System (ADS)

Zheng, W.; Gao, J. M.; Wang, R. X.; Chen, K.; Jiang, Y.

2017-12-01

This paper put forward a new method of technical characteristics deployment based on Reliability Function Deployment (RFD) by analysing the advantages and shortages of related research works on mechanical reliability design. The matrix decomposition structure of RFD was used to describe the correlative relation between failure mechanisms, soft failures and hard failures. By considering the correlation of multiple failure modes, the reliability loss of one failure mode to the whole part was defined, and a calculation and analysis model for reliability loss was presented. According to the reliability loss, the reliability index value of the whole part was allocated to each failure mode. On the basis of the deployment of reliability index value, the inverse reliability method was employed to acquire the values of technology characteristics. The feasibility and validity of proposed method were illustrated by a development case of machining centre’s transmission system.

Development of the evaluation instrument use CIPP on the implementation of project assessment topic optik

NASA Astrophysics Data System (ADS)

Asfaroh, Jati Aurum; Rosana, Dadan; Supahar

2017-08-01

This research aims to develop an evaluation instrument models CIPP valid and reliable as well as determine the feasibility and practicality of an evaluation instrument models CIPP. An evaluation instrument models CIPP to evaluate the implementation of the project assessment topic optik to measure problem-solving skills of junior high school class VIII in the Yogyakarta region. This research is a model of development that uses 4-D. Subject of product trials are students in class VIII SMP N 1 Galur and SMP N 1 Sleman. Data collection techniques in this research using non-test techniques include interviews, questionnaires and observations. Validity in this research was analyzed using V'Aikens. Reliability analyzed using ICC. This research uses 7 raters are derived from two lecturers expert (expert judgment), two practitioners (science teacher) and three colleagues. The results of this research is the evaluation's instrument model of CIPP is used to evaluate the implementation of the implementation of the project assessment instruments. The validity result of evaluation instrument have V'Aikens values between 0.86 to 1, which means a valid and 0.836 reliability values into categories so well that it has been worth used as an evaluation instrument.

Psychometric properties of a Chinese asthma quality of life questionnaire.

PubMed

Wang, Ningqun; Huang, Xiaobo; Chen, Wenqiang; Zhang, Xiaomei; Zhang, Yongsheng; Chen, Yujing

2017-12-01

To assess the acceptability, reliability, validity, and responsiveness of the Chinese Asthma Quality of Life Questionnaire (C-AQLQ) in a sample of Chinese asthma patients. The C-AQLQ and Short Form 36 Health Survey (SF-36) scales were administered to patients at baseline and 3 months later. Asthma severity condition and lung function were evaluated. Necessary data were gathered to assess the psychometric properties such as the feasibility, internal consistency, test-retest reliability, structural validity, discriminant validity, convergent validity, and responsiveness of the C-AQLQ. One hundred and thirty-seven patients completed the investigation. The Cronbach's alpha coefficient for the total scale was 0.96. Factor analysis yielded five factors that generally corresponded to the five proposed subscales. Patients with mild asthma reported higher scores than patients with moderate/severe asthma on all subscales other than environmental stimuli. Lung function measurement and the asthma severity score correlated significantly with domains of the C-AQOL but with fewer domains of the SF-36. The questionnaire detected within-subject changes in patients' asthma status during follow-up. Results indicated preliminary support that the C-AQLQ is a reliable, valid, discriminating, and responsive measure of quality of life in Chinese asthma patients. It is more sensitive than the generic SF-36 in detecting differences in asthma severity.

Posturography using the Wii Balance Board™: A feasibility study with healthy adults and adults post-stroke.

PubMed

Llorens, Roberto; Latorre, Jorge; Noé, Enrique; Keshner, Emily A

2016-01-01

Posturography systems that incorporate force platforms are considered to assess balance and postural control with greater sensitivity and objectivity than conventional clinical tests. The Wii Balance Board (WBB) system has been shown to have similar performance characteristics as other force platforms, but with lower cost and size. To determine the validity and reliability of a freely available WBB-based posturography system that combined the WBB with several traditional balance assessments, and to assess the performance of a cohort of stroke individuals with respect to healthy individuals. Healthy subjects and individuals with stroke were recruited. Both groups were assessed using the WBB-based posturography system. Individuals with stroke were also assessed using a laboratory grade posturography system and a battery of clinical tests to determine the concurrent validity of the system. A group of subjects were assessed twice with the WBB-based system to determine its reliability. A total of 144 healthy individuals and 53 individuals with stroke participated in the study. Concurrent validity with another posturography system was moderate to high. Correlations with clinical scales were consistent with previous research. The reliability of the system was excellent in almost all measures. In addition, the system successfully characterized individuals with stroke with respect to the healthy population. The WBB-based posturography system exhibited excellent psychometric properties and sensitivity for identifying balance performance of individuals with stroke in comparison with healthy subjects, which supports feasibility of the system as a clinical tool. Copyright © 2015 Elsevier B.V. All rights reserved.

The PedsQL Multidimensional Fatigue Scale in type 1 diabetes: feasibility, reliability, and validity.

PubMed

Varni, James W; Limbers, Christine A; Bryant, William P; Wilson, Don P

2009-08-01

The Pediatric Quality of Life Inventory (PedsQL, Mapi Research Trust, Lyon, France; www.pedsql.org) is a modular instrument designed to measure health-related quality of life and disease-specific symptoms in children and adolescents. The PedsQL Multidimensional Fatigue Scale was designed as a child self-report and parent proxy-report generic symptom-specific instrument to measure fatigue in pediatric patients. The objective of the present study was to determine the feasibility, reliability, and validity of the PedsQL Multidimensional Fatigue Scale in type 1 diabetes. The 18-item PedsQL Multidimensional Fatigue Scale (General Fatigue, Sleep/Rest Fatigue, and Cognitive Fatigue domains) and the PedsQL 4.0 Generic Core Scales were administered to 83 pediatric patients with type 1 diabetes and 84 parents. The PedsQL Multidimensional Fatigue Scale evidenced minimal missing responses (0.3% child report and 0.3% parent report), achieved excellent reliability for the Total Fatigue Scale score (alpha= 0.92 child report, 0.94 parent report), distinguished between pediatric patients with diabetes and healthy children, and was significantly correlated with the PedsQL 4.0 Generic Core Scales supporting construct validity. Pediatric patients with diabetes experienced fatigue that was comparable to pediatric patients with cancer on treatment, demonstrating the relative severity of their fatigue symptoms. The results demonstrate the measurement properties of the PedsQL Multidimensional Fatigue Scale in type 1 diabetes. The findings suggest that the PedsQL Multidimensional Fatigue Scale may be utilized in the standardized evaluation of fatigue in pediatric patients with type 1 diabetes.

Validation of Persian Version of PedsQL™ 4.0™ Generic Core Scales in Toddlers and Children

PubMed Central

Gheissari, Alaleh; Farajzadegan, Ziba; Heidary, Maryam; Salehi, Fatemeh; Masaeli, Ali; Mazrooei, Amin; Varni, James W; Fallah, Zahra; Zandieh, Fariborz

2012-01-01

Introduction: To evaluate the reliability, validity and feasibility of the Persian version of the Pediatric Quality of Life inventory (PedsQL™ 4.0™ 4.0) Generic Core Scales in Iranian healthy students ages 7-15 and chronically ill children ages 2-18. Methods: We followed the translation methodology proposed by developer to validate Persian version of PedsQL™ 4.0™ 4.0 Generic Core Scales for children. Six hundred and sixty children and adolescents and their parents were enrolled. Sample of 160 healthy students were chosen by random cluster method between 4 regions of Isfahan education offices and 60 chronically ill children were recruited from St. Alzahra hospital private clinics. The questionnaires were fulfilled by the participants. Results: The Persian version of PedsQL™ 4.0™ 4.0 Generic Core Scales discriminated between healthy and chronically ill children (healthy students mean score was 12.3 better than chronically ill children, P<0.001). Cronbachs’ alpha internal consistency values exceeded 0.7 for children self reports and proxy reports of children 5-7 years old and 13-18 years old. Reliability of proxy reports for 2-4 years old was much lower than 0.7. Although, proxy reports for chronically ill children 8-12 years old was more than 0.7, these reports for healthy children with same age group was slightly lower than 0.7. Constructive, criterion face and content validity were acceptable. In addition, the Persian version of PedsQL™ 4.0™ 4.0 Generic Core Scales was feasible and easy to complete. Conclusion: Results showed that Persian version of PedsQL™ 4.0™ 4.0 Generic Core Scales is valid and acceptable for pediatric health researches. It is necessary to alternate scoring for 2-4 years old questionnaire and to find a way to increase reliability for healthy children aged 8-12 years especially, according to Iranian culture. PMID:22701775

Networking the Home and University: How Families Can Be Integrated into Proximate/Distant Computer Systems.

ERIC Educational Resources Information Center

Watson, J. Allen; And Others

1989-01-01

Describes study that was conducted to determine the feasibility of networking home microcomputers with a university mainframe system in order to investigate a new family process research paradigm, as well as the design and function of the microcomputer/mainframe system. Test instrumentation is described and systems' reliability and validity are…

Exploring Methods for Developing Behaviorally Anchored Rating Scales for Evaluating Structured Interview Performance. Research Report. ETS RR-17-28

ERIC Educational Resources Information Center

Kell, Harrison J.; Martin-Raugh, Michelle P.; Carney, Lauren M.; Inglese, Patricia A.; Chen, Lei; Feng, Gary

2017-01-01

Behaviorally anchored rating scales (BARS) are an essential component of structured interviews. Use of BARS to evaluate interviewees' performance is associated with greater predictive validity and reliability and less bias. BARS are time-consuming and expensive to construct, however. This report explores the feasibility of gathering participants'…

Testing the Feasibility of Developmental Asset Measures on College Students to Guide Health Promotion Efforts

ERIC Educational Resources Information Center

Zullig, Keith J.; Ward, Rose Marie; King, Keith A.; Patton, Jon M.; Murray, Karen A.

2009-01-01

The purpose of this investigation was to assess the reliability and validity of eight developmental asset measures among a stratified, random sample (N = 540) of college students to guide health promotion efforts. The sample was randomly split to produce exploratory and confirmatory samples for factor analysis using principal axis factoring and…

Automated Pilot Performance Assessment in the T-37: A Feasibility Study. Final Report (May 1968-April 1971).

ERIC Educational Resources Information Center

Knoop, Patricia A.; Welde, William L.

Air Force investigators conducted a three year program to develop a capability for automated quantification and assessment of in-flight pilot performance. Such a capability enhances pilot training by making ratings more objective, valid, reliable and sensitive, and by freeing instructors from rating responsibilities, allowing them to concentrate…

Top 10 Research Questions Related to Musculoskeletal Physical Fitness Testing in Children and Adolescents

ERIC Educational Resources Information Center

Plowman, Sharon Ann

2014-01-01

The purpose of this article is to bring attention to the 10 most pressing questions relevant to musculoskeletal physical fitness testing in children and adolescents. The goal is to stimulate research to answer these questions. The most pressing needs include establishing definitive links between valid, reliable, and feasible field test measures of…

Performance Assessments in Science: Hands-On Tasks and Scoring Guides.

ERIC Educational Resources Information Center

Stecher, Brian M.; Klein, Stephen P.

In 1992, RAND received a grant from the National Science Foundation to study the technical quality of performance assessments in science and to evaluate their feasibility for use in large-scale testing programs. The specific goals of the project were to assess the reliability and validity of hands-on science testing and to investigate the cost and…

Identifying depression post-stroke in patients with aphasia: a systematic review of the reliability, validity and feasibility of available instruments.

PubMed

van Dijk, Mariska J; de Man-van Ginkel, Janneke M; Hafsteinsdóttir, Thóra B; Schuurmans, Marieke J

2016-08-01

To identify and critically appraise the evidence for instruments assessing depression in stroke patients with aphasia. The PubMed, CINAHL, Web of Science, Psych Info and Cochrane databases were searched from inception until May 2015. Of the 383 titles found in the search, 15 articles met the inclusion criteria and six instruments were identified: The Aphasic Depression Rating Scale, the Clinical Global Impression-Scale, the Stroke Aphasic Depression Questionnaire (four versions), the Signs of Depression Scale, the Visual Analogue Mood Scale (three versions) and the Visual Analogue Self Esteem Scale. Supporting evidence for reliability and validity was limited owing to methodological flaws in the studies influencing the ratings of methodological quality. Feasibility data were available for all instruments. Rating time of the instruments ranged from less than one minute to five minutes, two instruments required extensive training. A number of instruments to assess depressive symptoms in patients with aphasia are available. None of the instruments however, were found to be sufficiently investigated and most of the studies identified were of low methodological quality. Given the present evidence, the Stroke Aphasic Depression Questionnaire-10, the Stroke Aphasic Depression Questionnaire-H10 and the Signs of Depression Scale are the most feasible and can be recommended for clinical practice. © The Author(s) 2015.

THE 6-MINUTE WALK TEST AND OTHER CLINICAL ENDPOINTS IN DUCHENNE MUSCULAR DYSTROPHY: RELIABILITY, CONCURRENT VALIDITY, AND MINIMAL CLINICALLY IMPORTANT DIFFERENCES FROM A MULTICENTER STUDY

PubMed Central

McDonald, Craig M; Henricson, Erik K; Abresch, R Ted; Florence, Julaine; Eagle, Michelle; Gappmaier, Eduard; Glanzman, Allan M; Spiegel, Robert; Barth, Jay; Elfring, Gary; Reha, Allen; Peltz, Stuart W

2013-01-01

Introduction: An international clinical trial enrolled 174 ambulatory males ≥5 years old with nonsense mutation Duchenne muscular dystrophy (nmDMD). Pretreatment data provide insight into reliability, concurrent validity, and minimal clinically important differences (MCIDs) of the 6-minute walk test (6MWT) and other endpoints. Methods: Screening and baseline evaluations included the 6-minute walk distance (6MWD), timed function tests (TFTs), quantitative strength by myometry, the PedsQL, heart rate–determined energy expenditure index, and other exploratory endpoints. Results: The 6MWT proved feasible and reliable in a multicenter context. Concurrent validity with other endpoints was excellent. The MCID for 6MWD was 28.5 and 31.7 meters based on 2 statistical distribution methods. Conclusions: The ratio of MCID to baseline mean is lower for 6MWD than for other endpoints. The 6MWD is an optimal primary endpoint for Duchenne muscular dystrophy (DMD) clinical trials that are focused therapeutically on preservation of ambulation and slowing of disease progression. Muscle Nerve 48: 357–368, 2013 PMID:23674289

Feasibility of developing LSI microcircuit reliability prediction models

NASA Technical Reports Server (NTRS)

Ryerson, C. M.

1972-01-01

In the proposed modeling approach, when any of the essential key factors are not known initially, they can be approximated in various ways with a known impact on the accuracy of the final predictions. For example, on any program where reliability predictions are started at interim states of project completion, a-priori approximate estimates of the key factors are established for making preliminary predictions. Later these are refined for greater accuracy as subsequent program information of a more definitive nature becomes available. Specific steps to develop, validate and verify these new models are described.

Quality of care assessment in geriatric evaluation and management units: construction of a chart review tool for a tracer condition.

PubMed

Kergoat, Marie-Jeanne; Leclerc, Bernard-Simon; Leduc, Nicole; Latour, Judith; Berg, Katherine; Bolduc, Aline

2009-07-29

The number of elderly people requiring hospital care is growing, so, quality and assessment of care for elders are emerging and complex areas of research. Very few validated and reliable instruments exist for the assessment of quality of acute care in this field. This study's objective was to create such a tool for Geriatric Evaluation and Management Units (GEMUs). The methodology involved a reliability and feasibility study of a retrospective chart review on 934 older inpatients admitted in 49 GEMUs during the year 2002-2003 for fall-related trauma as a tracer condition. Pertinent indicators for a chart abstraction tool, the Geriatric Care Tool (GCT), were developed and validated according to five dimensions: access to care, comprehensiveness, continuity of care, patient-centred care and appropriateness. Consensus methods were used to develop the content. Participants were experts representing eight main health care professions involved in GEMUs from 19 different sites. Items associated with high quality of care at each step of the multidisciplinary management of patients admitted due to falls were identified. The GCT was tested for intra- and inter-rater reliability using 30 medical charts reviewed by each of three independent and blinded trained nurses. Kappa and agreement measures between pairs of chart reviewers were computed on an item-by-item basis. Three quarters of 169 items identifying the process of care, from the case history to discharge planning, demonstrated good agreement (kappa greater than 0.40 and agreement over 70%). Indicators for the appropriateness of care showed less reliability. Content validity and reliability results, as well as the feasibility of the process, suggest that the chart abstraction tool can gather standardized and pertinent clinical information for further evaluating quality of care in GEMU using admission due to falls as a tracer condition. However, the GCT should be evaluated in other models of acute geriatric units and new strategies should be developed to improve reliability of peer assessments in characterizing the quality of care for elderly patients with complex conditions.

The feasibility of sharing simulation-based evaluation scenarios in anesthesiology.

PubMed

Berkenstadt, Haim; Kantor, Gareth S; Yusim, Yakov; Gafni, Naomi; Perel, Azriel; Ezri, Tiberiu; Ziv, Amitai

2005-10-01

We prospectively assessed the feasibility of international sharing of simulation-based evaluation tools despite differences in language, education, and anesthesia practice, in an Israeli study, using validated scenarios from a multi-institutional United States (US) study. Thirty-one Israeli junior anesthesia residents performed four simulation scenarios. Training sessions were videotaped and performance was assessed using two validated scoring systems (Long and Short Forms) by two independent raters. Subjects scored from 37 to 95 (70 +/- 12) of 108 possible points with the "Long Form" and "Short Form" scores ranging from 18 to 35 (28.2 +/- 4.5) of 40 possible points. Scores >70% of the maximal score were achieved by 61% of participants in comparison to only 5% in the original US study. The scenarios were rated as very realistic by 80% of the participants (grade 4 on a 1-4 scale). Reliability of the original assessment tools was demonstrated by internal consistencies of 0.66 for the Long and 0.75 for the Short Form (Cronbach alpha statistic). Values in the original study were 0.72-0.76 for the Long and 0.71-0.75 for the Short Form. The reliability did not change when a revised Israeli version of the scoring was used. Interrater reliability measured by Pearson correlation was 0.91 for the Long and 0.96 for the Short Form (P < 0.01). The high scores for plausibility given to the scenarios and the similar reliability of the original assessment tool support the feasibility of using simulation-based evaluation tools, developed in the US, in Israel. The higher scores achieved by Israeli residents may be related to the fact that most Israeli residents are immigrants with previous training in anesthesia. Simulation-based assessment tools developed in a multi-institutional study in the United States can be used in Israel despite the differences in language, education, and medical system.

Systematic review found AMSTAR, but not R(evised)-AMSTAR, to have good measurement properties.

PubMed

Pieper, Dawid; Buechter, Roland Brian; Li, Lun; Prediger, Barbara; Eikermann, Michaela

2015-05-01

To summarize all available evidence on measurement properties in terms of reliability, validity, and feasibility of the Assessment of Multiple Systematic Reviews (AMSTAR) tool, including R(evised)-AMSTAR. MEDLINE, EMBASE, Psycinfo, and CINAHL were searched for studies containing information on measurement properties of the tools in October 2013. We extracted data on study characteristics and measurement properties. These data were analyzed following measurement criteria. We included 13 studies, four of them were labeled as validation studies. Nine articles dealt with AMSTAR, two articles dealt with R-AMSTAR, and one article dealt with both instruments. In terms of interrater reliability, most items showed a substantial agreement (>0.6). The median intraclass correlation coefficient (ICC) for the overall score of AMSTAR was 0.83 (range 0.60-0.98), indicating a high agreement. In terms of validity, ICCs were very high with all but one ICC lower than 0.8 when the AMSTAR score was compared with scores from other tools. Scoring AMSTAR takes between 10 and 20 minutes. AMSTAR seems to be reliable and valid. Further investigations for systematic reviews of other study designs than randomized controlled trials are needed. R-AMSTAR should be further investigated as evidence for its use is limited and its measurement properties have not been studied sufficiently. In general, test-retest reliability should be investigated in future studies. Copyright © 2015 Elsevier Inc. All rights reserved.

Measuring Outcomes for Dysphagia: Validity and Reliability of the European Portuguese Eating Assessment Tool (P-EAT-10).

PubMed

Nogueira, Dália Santos; Ferreira, Pedro Lopes; Reis, Elizabeth Azevedo; Lopes, Inês Sousa

2015-10-01

The purpose of this study was to evaluate the validity and the reliability of the European Portuguese version of the EAT-10 (P-EAT-10). This research was conducted in three phases: (i) cultural and linguistic adaptation; (ii) feasibility and reliability test; and (iii) validity tests. The final sample was formed by a cohort of 520 subjects. The P-EAT-10 index was compared for socio-demographic and clinic variables. It was also compared for both dysphagic and non-dysphagic groups as well as for the results of the 3Oz wst. Lastly, the P-EAT-10 scores were correlated with the EuroQol Group Portuguese EQ-5D index. The Cronbach's α obtained for the P-EAT-10 scale was 0.952 and it remained excellent even if any item was deleted. The item-total and the intraclass correlation coefficients were very good. The P-EAT-10 mean of the non-dysphagic cohort was 0.56 and that of the dysphagic cohort was 14.26, the mean comparison between the 3Oz wst groups and the P-EAT-10 scores were significant. A significant higher perception of QoL was also found among the non-dysphagic subjects. P-EAT-10 is a valid and reliable measure that may be used to document dysphagia which makes it useful both for screening in clinical practice and in research.

Psychometric Testing of the Chinese-Version Glover-Nilsson Smoking Behavioral Questionnaire (GN-SBQ-C) for the Identification of Nicotine Dependence in Adult Smokers in Taiwan.

PubMed

Chen, Shu-Ching; Chen, Hsiu-Fang; Peng, Hsi-Ling; Lee, Li-Yun; Chiang, Ting-Yu; Chiu, Hui-Chuan

2017-04-01

The purposes of this study were to evaluate the psychometric properties, reliability, and validity of the Chinese-version Glover-Nilsson Smoking Behavioral Questionnaire (GN-SBQ-C) and assess the behavioral nicotine dependence among community-dwelling adult smokers in Taiwan. The methods used were survey design, administration, and validation. A total of 202 adult smokers completed a survey to assess behavioral dependence, nicotine dependence, depression, social support, and demographic and smoking characteristics. Data analysis included descriptive statistics, internal consistency reliability, t test, exploratory factor analysis, independent t test, and Pearson product moment correlation. The results showed that (1) the GN-SBQ-C has good internal consistency reliability and stability (2-week test-retest reliability); (2) the extracted one factor explained 41.80 % of the variance, indicating construct validity; (3) the scale has acceptable concurrent validity, with significant positive correlation between the GN-SBQ-C and nicotine dependence, depression, and time smoking and negative correlation between the GN-SBQ-C and age and exercise habit; and (4) the instrument has discriminant validity, supported by significant differences between those with high and low-to-moderate nicotine dependence, smokers greater than 43 years old and those 43 years old and younger, and those who smoked 10 years or less and those smoking more than 10 years. The 11-item GN-SBQ-C has satisfactory psychometric properties when applied in a sample of Taiwanese adult smokers. The scale is feasible and valid to use to assess smoking behavioral dependence.

Validity and feasibility of the american college of surgeons colectomy composite outcome quality measure.

PubMed

Merkow, Ryan P; Hall, Bruce L; Cohen, Mark E; Wang, Xue; Adams, John L; Chow, Warren B; Lawson, Elise H; Bilimoria, Karl Y; Richards, Karen; Ko, Clifford Y

2013-03-01

To develop a reliable, robust, parsimonious, risk-adjusted 30-day composite colectomy outcome measure. A fundamental aspect in the pursuit of high-quality care is the development of valid and reliable performance measures in surgery. Colon resection is associated with appreciable morbidity and mortality and therefore is an ideal quality improvement target. From 2010 American College of Surgeons National Surgical Quality Improvement Program data, patients were identified who underwent colon resection for any indication. A composite outcome of death or any serious morbidity within 30 days of the index operation was established. A 6-predictor, parsimonious model was developed and compared with a more complex model with more variables. National caseload requirements were calculated on the basis of increasing reliability thresholds. From 255 hospitals, 22,346 patients were accrued who underwent a colon resection in 2010, most commonly for neoplasm (46.7%). A mortality or serious morbidity event occurred in 4461 patients (20.0%). At the hospital level, the median composite event rate was 20.7% (interquartile range: 15.8%-26.3%). The parsimonious model performed similarly to the full model (Akaike information criterion: 19,411 vs 18,988), and hospital-level performance comparisons were highly correlated (R = 0.97). At a reliability threshold of 0.4, 56 annual colon resections would be required and achievable at an estimated 42% of US and 69% of American College of Surgeons National Surgical Quality Improvement Program hospitals. This 42% of US hospitals performed approximately 84% of all colon resections in the country in 2008. It is feasible to design a measure with a composite outcome of death or serious morbidity after colon surgery that has a low burden for data collection, has substantial clinical importance, and has acceptable reliability.

A systematic review of reliable and valid tools for the measurement of patient participation in healthcare.

PubMed

Phillips, Nicole Margaret; Street, Maryann; Haesler, Emily

2016-02-01

Patient participation in healthcare is recognised internationally as essential for consumer-centric, high-quality healthcare delivery. Its measurement as part of continuous quality improvement requires development of agreed standards and measurable indicators. This systematic review sought to identify strategies to measure patient participation in healthcare and to report their reliability and validity. In the context of this review, patient participation was constructed as shared decision-making, acknowledging the patient as having critical knowledge regarding their own health and care needs and promoting self-care/autonomy. Following a comprehensive search, studies reporting reliability or validity of an instrument used in a healthcare setting to measure patient participation, published in English between January 2004 and March 2014 were eligible for inclusion. From an initial search, which identified 1582 studies, 156 studies were retrieved and screened against inclusion criteria. Thirty-three studies reporting 24 patient participation measurement tools met inclusion criteria, and were critically appraised. The majority of studies were descriptive psychometric studies using prospective, cross-sectional designs. Almost all the tools completed by patients, family caregivers, observers or more than one stakeholder focused on aspects of patient-professional communication. Few tools designed for completion by patients or family caregivers provided valid and reliable measures of patient participation. There was low correlation between many of the tools and other measures of patient satisfaction. Few reliable and valid tools for measurement of patient participation in healthcare have been recently developed. Of those reported in this review, the dyadic Observing Patient Involvement in Decision Making (dyadic-OPTION) tool presents the most promise for measuring core components of patient participation. There remains a need for further study into valid, reliable and feasible strategies for measuring patient participation as part of continuous quality improvement. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/

The use of the EQ-5D-Y health related quality of life outcome measure in children in the Western Cape, South Africa: psychometric properties, feasibility and usefulness - a longitudinal, analytical study.

PubMed

Scott, Des; Ferguson, Gillian D; Jelsma, Jennifer

2017-01-19

The EQ-5D-Y, an outcome measure of Health Related Quality of Life (HRQoL) in children, was developed by an international task team in 2010. The multinational feasibility, reliability and validity study which followed was undertaken with mainly healthy children. The aim of this study was to investigate the psychometric properties of the EQ-5D-Y when used to assess the HRQoL of children with different health states. A sample of 224 children between eight and twelve years were grouped according to their health state. The groups included 52 acutely ill children, 67 children with either a chronic health condition or disability and 105 mostly healthy, mainstream school children as a comparator. They were assessed at baseline, at three months and at six months. An analysis of the psychometric properties was performed to assess the reliability, validity and responsiveness of the EQ-5D-Y in the different groups of children. Cohen's kappa, the intraclass correlation coefficient, Pearson Chi-square, Kruskal-Wallis ANOVA and effect size of Wilcoxon Signed-rank test were used to determine the reliability, validity and responsiveness of the instrument. The EQ-5D-Y dimensions were found to be reliable on test-retest (kappa varying from 0.365 to 0.653), except for the Usual Activities dimension (kappa 0.199). The Visual Analogue Scale (VAS) was also reliable (ICC = 0.77). Post-hoc analysis indicated that dimensions were able to discriminate between acutely ill and healthy children (all differences p < 0.001). The acutely ill children had the lowest ranked VAS (median 50, range 0-100), indicating worst HRQoL and was the only group significantly different from the other three groups (p < 0.001 in all cases). Convergent validity between all similar EQ-5D-Y and PedsQL, WeeFIM and Faces Pain Scale dimensions was only evident in the acutely ill children. As expected the largest treatment effect was also observed in these children (Wilcoxon Signed-rank test for VAS was 0.43). Six of the nine therapists who took part in the study, found the measure quick and easy to apply, used the information in the management of the child and would continue to use it in future. The EQ-5D-Y could be used with confidence as an outcome measure for acutely-ill children, but demonstrated poorer psychometric properties in children with no health condition or chronic conditions. It appears to be feasible and useful to include the EQ-5D-Y in routine assessments of children.

Validity and reliability of the semi-quantitative self-report Home Food Availability Inventory Checklist (HFAI-C) in White and South Asian populations.

PubMed

Bryant, Maria; LeCroy, Madison; Sahota, Pinki; Cai, Jianwen; Stevens, June

2016-05-04

Despite interest in the importance of the home food environment and its potential influence on children's diets and social norms, there remain few self-report checklist methods that have been validated against the gold standard of researcher-conducted inventories. This study aimed to assess the criterion validity and reliability of the 'Home Food Availability Inventory Checklist' (HFAI-C), a 39-item checklist including categories of fruit, vegetables, snacks and drinks. The HFAI-C was completed by 97 participants of White and Pakistani origin in the UK. Validity was determined by comparing participant-reported HFAI-C responses to data from researcher observations of home food availability using PABAK and weighted kappa statistics. The validity of measuring the amount of items (in addition to presence/absence) available was also determined. Test-retest reliability compared repeated administrations of the HFAI-C using intra-class correlation coefficients. Validity and reliability was fair to moderate overall. For validity, the average category-level PABAK ranged from 0.31 (95% CI: 0.25, 0.37) for vegetables to 0.44 (95% CI: 0.40, 0.49) for fruits. Assessment of the presence/absence of items demonstrated higher validity compared to quantity measurements. Reliability was increased when the HFAI-C was repeated close to the time of the first administration. For example, ICCs for reliability of the measurement of fruits were 0.52 (95%CI: 0.47, 0.56) if re-administered within 5 months, 0.58 (95% CI: 0.51, 0.64) within 30 days and 0.97 (95%CI: 0.94, 1.00) if re-administered on the same day. Overall, the HFAI-C demonstrated fair to moderate validity and reliability in a population of White and South Asian participants. This evaluation is consistent with previous work on other checklists in less diverse, more affluent populations. Our research supports the use of the HFAI-C as a useful, albeit imperfect, representation of researcher-conducted inventories. The feasibility of collecting information using the HFAI-C in large, multi-ethnic samples can facilitate examination of home food availability in relation to exposures such as ethnicity and outcomes including behavioural, social and health outcomes. Future work using the HFAI-C could provide important insights into a modifiable influence with potential to impact health.

[Reliability and validity of the PAQ-A questionnaire to assess physical activity in Spanish adolescents].

PubMed

Martínez-Gómez, David; Martínez-de-Haro, Vicente; Pozo, Tamara; Welk, Gregory J; Villagra, Ariel; Calle, Marisa E; Marcos, Ascensión; Veiga, Oscar L

2009-01-01

Questionnaires are feasible instruments to assess physical activity (PA) in large samples. The aim of the current study was to evaluate the reliability and validity of the PAQ-A questionnaire in Spanish adolescents using the measurement of PA by accelerometer as criterion. In a sample of 82 adolescents, aged 12 to 17 years, 1-week PAQ-A test-retest was administered. Reliability was analyzed by the Intraclass Correlation Coefficient (ICC) and the internal consistency by the Cronbach's alpha Coefficient. Two hundred thirty-two adolescents, aged 13-17 years, completed the PAQ-A and wore the ActiGraph GT1M accelerometer during 7-days. The PAQ-A was compared against total PA and moderate to vigorous PA (MVPA) obtained by the accelerometer. Test-retest reliability showed ICC = 0.71 for the final score of PAQ-A. Internal consistency was alpha = 0.65 in the first self-report, alpha = 0.67 in the retest in 82 adolescents sample, and alpha = 0.74 in the 232 adolescents sample. The PAQ-A was moderately correlated with total PA (rho = 0.39) and MVPA (rho= 0.34) assessed by the accelerometer. The PAQ-A obtained significantly moderate correlations in boys but not in girls against the accelerometer. The PAQ-A questionnaire shows an adequate reliability and a reasonable validity for assessing PA in Spanish adolescents.

A comparison of kinesthetic-tactual and visual displays via a critical tracking task. [for aircraft control

NASA Technical Reports Server (NTRS)

Jagacinski, R. J.; Miller, D. P.; Gilson, R. D.

1979-01-01

The feasibility of using the critical tracking task to evaluate kinesthetic-tactual displays was examined. The test subjects were asked to control a first-order unstable system with a continuously decreasing time constant by using either visual or tactual unidimensional displays. The results indicate that the critical tracking task is both a feasible and a reliable methodology for assessing tactual tracking. Further, that the critical tracking methodology is as sensitive and valid a measure of tactual tracking as visual tracking is demonstrated by the approximately equal effects of quickening for the tactual and visual displays.

The Relationship of Eye Anomalies and Reading Ability, and an Analysis of Vision-Screening Programs. (Volumes I and II).

ERIC Educational Resources Information Center

Francis, Leslie John

The purpose of this study was to provide an in-depth analysis of vision-screening programs in relation to their efficacy, appropriateness, and feasibility for public school use. Twenty-two vision-screening programs were analyzed for reliability, validity, efficiency of identification and referral cost, and required testing time. Findings are that…

A Tuning-AHELO Conceptual Framework of Expected Desired/Learning Outcomes in Engineering. OECD Education Working Papers, Number 60

ERIC Educational Resources Information Center

OECD Publishing (NJ1), 2011

2011-01-01

The OECD Secretariat, at the invitation of the AHELO Group of National Experts, contracted the Tuning Association to undertake initial development work on learning outcomes to be used for valid and reliable assessments of students from diverse institutions and countries. The two disciplines selected for the AEHLO Feasibility Study are engineering…

Considering Personality Type in Adult Learning: Using the Myers-Briggs Type Indicator in Instructor Preparation at PricewaterhouseCoopers

ERIC Educational Resources Information Center

Daisley, Richard J.

2011-01-01

This article explores the feasibility of using the Myers-Briggs Type Indicator (MBTI) as a framework for instructor development in a professional services training environment. It explores the consistency of MBTI with common adult learning theory, addresses questions on MBTI's reliability and validity, and explores the applicability of MBTI to the…

[A set of quality and safety indicators for hospitals of the "Agencia Valenciana de Salud"].

PubMed

Nebot-Marzal, C M; Mira-Solves, J J; Guilabert-Mora, M; Pérez-Jover, V; Pablo-Comeche, D; Quirós-Morató, T; Cuesta Peredo, D

2014-01-01

To prepare a set of quality and safety indicators for Hospitals of the «Agencia Valenciana de Salud». The qualitative technique Metaplan® was applied in order to gather proposals on sustainability and nursing. The catalogue of the «Spanish Society of Quality in Healthcare» was adopted as a starting point for clinical indicators. Using the Delphi technique, 207 professionals were invited to participate in the selecting the most reliable and feasible indicators. Lastly, the resulting proposal was validated with the managers of 12 hospitals, taking into account the variability, objectivity, feasibility, reliability and sensitivity, of the indicators. Participation rates varied between 66.67% and 80.71%. Of the 159 initial indicators, 68 were prioritized and selected (21 economic or management indicators, 22 nursing indicators, and 25 clinical or hospital indicators). Three of them were common to all three categories and two did not match the specified criteria during the validation phase, thus obtaining a final catalogue of 63 indicators. A set of quality and safety indicators for Hospitals was prepared. They are currently being monitored using the hospital information systems. Copyright © 2013 SECA. Published by Elsevier Espana. All rights reserved.

A decision-tree approach to the assessment of posttraumatic stress disorder: Engineering empirically rigorous and ecologically valid assessment measures.

PubMed

Stewart, Regan W; Tuerk, Peter W; Metzger, Isha W; Davidson, Tatiana M; Young, John

2016-02-01

Structured diagnostic interviews are widely considered to be the optimal method of assessing symptoms of posttraumatic stress; however, few clinicians report using structured assessments to guide clinical practice. One commonly cited impediment to these assessment approaches is the amount of time required for test administration and interpretation. Empirically keyed methods to reduce the administration time of structured assessments may be a viable solution to increase the use of standardized and reliable diagnostic tools. Thus, the present research conducted an initial feasibility study using a sample of treatment-seeking military veterans (N = 1,517) to develop a truncated assessment protocol based on the Clinician-Administered Posttraumatic Stress Disorder (PTSD) Scale (CAPS). Decision-tree analysis was utilized to identify a subset of predictor variables among the CAPS items that were most predictive of a diagnosis of PTSD. The algorithm-driven, atheoretical sequence of questions reduced the number of items administered by more than 75% and classified the validation sample at 92% accuracy. These results demonstrated the feasibility of developing a protocol to assess PTSD in a way that imposes little assessment burden while still providing a reliable categorization. (c) 2016 APA, all rights reserved).

The reliability, validity, and feasibility of physical activity measurement in adults with traumatic brain injury: an observational study.

PubMed

Hassett, Leanne; Moseley, Anne; Harmer, Alison; van der Ploeg, Hidde P

2015-01-01

To determine the reliability and validity of the Physical Activity Scale for Individuals with a Physical Disability (PASIPD) in adults with severe traumatic brain injury (TBI) and estimate the proportion of the sample participants who fail to meet the World Health Organization guidelines for physical activity. A single-center observational study recruited a convenience sample of 30 community-based ambulant adults with severe TBI. Participants completed the PASIPD on 2 occasions, 1 week apart, and wore an accelerometer (ActiGraph GT3X; ActiGraph LLC, Pensacola, Florida) for the 7 days between these 2 assessments. The PASIPD test-retest reliability was substantial (intraclass correlation coefficient = 0.85; 95% confidence interval, 0.70-0.92), and the correlation with the accelerometer ranged from too low to be meaningful (R = 0.09) to moderate (R = 0.57). From device-based measurement of physical activity, 56% of participants failed to meet the World Health Organization physical activity guidelines. The PASIPD is a reliable measure of the type of physical activity people with severe TBI participate in, but it is not a valid measure of the amount of moderate to vigorous physical activity in which they engage. Accelerometers should be used to quantify moderate to vigorous physical activity in people with TBI.

Development and Validation of the Numeracy Understanding in Medicine Instrument Short Form

PubMed Central

Schapira, Marilyn M.; Walker, Cindy M.; Miller, Tamara; Fletcher, Kathlyn A; Ganschow, Pamela G.; Jacobs, Elizabeth A; Imbert, Diana; O'Connell, Maria; Neuner, Joan M.

2014-01-01

Background Health numeracy can be defined as the ability to understand and use numeric information and quantitative concepts in the context of health. We previously reported the development of the Numeracy Understanding in Medicine Instrument (NUMi); a 20-item test developed using item response theory. We now report the development and validation of a short form of the NUMi. Methods Item statistics were used to identify a subset of 8-items representing a range of difficulty and content areas. Internal reliability was evaluated with Cronbach's alpha. Divergent and convergent validity was assessed by comparing scores of the S-NUMI with existing measures of education, print and numeric health literacy, mathematic achievement, cognitive reasoning, and the original NUMi. Results The 8-item scale had adequate reliability (Cronbach's alpha: 0.72) and was strongly correlated to the 20-item NUMi (0.92). The S-NUMi scores were strongly correlated with the Lipkus numeracy test (0.62), Wide Range of Achievement Test-Mathematics (WRAT-M) (0.72), and Wonderlic cognitive reasoning test (0.76). Moderate correlation was found with education level (0.58) and print literacy as measured by the TOFHLA (0.49). Conclusion The short Numeracy Understanding in Medicine Instrument is a reliable and valid measure of health numeracy feasible for use in clinical and research settings. PMID:25315596

The Psychometric Properties of the Voice Handicap Index in People With Parkinson's Disease.

PubMed

Guimaraes, I; Cardoso, R; Pinto, S; Ferreira, J J

2017-03-01

Psychosocial impact of dysphonia in people with Parkinson disease (PD) has been described with the Voice Handicap Index (VHI); however, its psychometric properties when applied in this population are not described. The objective of this study was to examine the psychometric properties of the VHI in people with PD. A cross-sectional study of 151 subjects without cognitive impairment (90 people with PD and 61 controls) was carried out. The VHI was applied along with clinician-based (Mini Mental State Examination, Hoehn and Yahr staging, and Movement Disorder Society-Unified Parkinson's Disease Rating Scale) and patient-based (self-rated voice severity) outcome measures. The psychometric properties of the VHI analyzed were the feasibility, reliability, and construct validity. The average age of the PD population studied was 67 years; 51% had a primary level of education and 81% were retired. On average, they had disease onset duration of 11 years, a mild disease stage, mild to moderate global motor disability and impairment, and a normal to mild self-rated voice severity. The psychometric attributes of the VHI demonstrated that the questionnaire is feasible (missing data less than 1%), reliable (Cronbach α > 0.9), and valid (71.5% of the total variance is explained by five factors, correlates with voice severity, PD disability, and impairment, and differentiates subjects with PD from subjects without PD). The VHI is a reliable and valid tool that can be recommended for the population under study although further work is required to investigate its utility in advanced stages of disease. Copyright © 2017 The Voice Foundation. Published by Elsevier Inc. All rights reserved.

Validation of the Spanish Acne Severity Scale (Escala de Gravedad del Acné Española--EGAE).

PubMed

Puig, Lluis; Guerra-Tapia, Aurora; Conejo-Mir, Julián; Toribio, Jaime; Berasategui, Carmen; Zsolt, Ilonka

2013-04-01

Several acne grading systems have been described, but consensus is lacking on which shows superiority. A standardized system would facilitate therapeutic decisions and the analysis of clinical trial data. To assess the feasibility, reliability, validity and sensitivity to change of the Spanish Acne Severity Scale (EGAE). A Spanish, multicentre, prospective, observational study was performed in patients with facial, back or chest acne assessed using EGAE, Leeds Revised Acne Grading system (LRAG) and lesion count. Clinicians answered 4 questions regarding EGAE use and time employed. Patients were evaluated at baseline and after 5±1 weeks. Four additional blinded observers, all dermatologists, evaluated patients' pictures using EGAE and LRAG. In total, 349 acne locations were assessed in 328 patients. Of the dermatologists, 95.6% (CI: 92.9-97.5%) reported that EGAE was easy to use, and 75% used it in <3 minutes. Interobserver reliability of the EGAE scale was shown by a Kendall's W of 0.773 (p<0.001). EGAE and LRAG scales showed a high correlation (Spearman's correlation>0.85; p<0.001). EGAE mean score in treatment-compliant patients was significantly lower at follow-up than at baseline (2.14 vs. 1.57, p<0.001, Cohen's d=0.35).The pre-post-treatment difference in EGAE mean score in non-compliant patients was not significant (1.44 vs. 1.32, p<0.102) and Cohen's d was lower (0.19) than in compliant patients. The use of EGAE to evaluate acne grade in daily clinical dermatological practice in Spanish centres has shown feasibility, high interobserver reliability, concurrent validity and sensitivity to detect treatment effects.

Development and validation of the Dutch version of the London Handicap Scale.

PubMed

Groothuis-Oudshoorn, Catharina G M; Chorus, Astrid M J; Verrips, G H W; Detmar, Symone B

2015-01-01

The London Handicap Scale (LHS) was found to be a valid and reliable scale for measuring participation restrictions in adults. This paper describes the development and assesses the construct-related validity of a Dutch version of the London Handicap Scale (DLHS). The DLHS was tested in 798 adults (mean age: 50.7 years, SD=14.5, range 16 to 85) and validated with the 'Impact on Participation and Autonomy' (IPA) questionnaire, the Dutch version of the EQ-5D and questions concerning comorbidity and use of medical devices. The study population consisted of patients with rheumatoid arthritis, chronic obstructive pulmonary disease (COPD), epilepsy, laryngectomy and multiple sclerosis. Feasibility was satisfactory. Large correlations (ρ > 0.6) for the DLHS sum score were found with the IPA subscales 'autonomy outdoors', 'perceiving problems', 'family role', autonomy indoors', 'work and education' and with the EQ-5D. The DLHS sum score differs significantly between subgroups based on the number of chronic diseases, number of medical devices and self-reported burden of disease or handicap (p< 0.001). Based on this evaluation the questionnaire seems feasible and valid for assessing differences in level of participation between subgroups of chronically ill or disabled persons in the Netherlands.

Emotional disorders: cluster 4 of the proposed meta-structure for DSM-V and ICD-11.

PubMed

Goldberg, D P; Krueger, R F; Andrews, G; Hobbs, M J

2009-12-01

The extant major psychiatric classifications DSM-IV, and ICD-10, are atheoretical and largely descriptive. Although this achieves good reliability, the validity of a medical diagnosis would be greatly enhanced by an understanding of risk factors and clinical manifestations. In an effort to group mental disorders on the basis of aetiology, five clusters have been proposed. This paper considers the validity of the fourth cluster, emotional disorders, within that proposal. We reviewed the literature in relation to 11 validating criteria proposed by a Study Group of the DSM-V Task Force, as applied to the cluster of emotional disorders. An emotional cluster of disorders identified using the 11 validators is feasible. Negative affectivity is the defining feature of the emotional cluster. Although there are differences between disorders in the remaining validating criteria, there are similarities that support the feasibility of an emotional cluster. Strong intra-cluster co-morbidity may reflect the action of common risk factors and also shared higher-order symptom dimensions in these emotional disorders. Emotional disorders meet many of the salient criteria proposed by the Study Group of the DSM-V Task Force to suggest a classification cluster.

Selecting process quality indicators for the integrated care of vulnerable older adults affected by cognitive impairment or dementia.

PubMed

Kröger, Edeltraut; Tourigny, André; Morin, Diane; Côté, Lise; Kergoat, Marie-Jeanne; Lebel, Paule; Robichaud, Line; Imbeault, Shirley; Proulx, Solange; Benounissa, Zohra

2007-11-29

This study aimed at evaluating face and content validity, feasibility and reliability of process quality indicators developed previously in the United States or other countries. The indicators can be used to evaluate care and services for vulnerable older adults affected by cognitive impairment or dementia within an integrated service system in Quebec, Canada. A total of 33 clinical experts from three major urban centres in Quebec formed a panel representing two medical specialties (family medicine, geriatrics) and seven health or social services specialties (nursing, occupational therapy, psychology, neuropsychology, pharmacy, nutrition, social work), from primary or secondary levels of care, including long-term care. A modified version of the RAND(R)/University of California at Los Angeles (UCLA) appropriateness method, a two-round Delphi panel, was used to assess face and content validity of process quality indicators. The appropriateness of indicators was evaluated according to a) agreement of the panel with three criteria, defined as a median rating of 7-9 on a nine-point rating scale, and b) agreement among panellists, judged by the statistical measure of the interpercentile range adjusted for symmetry. Feasibility of quality assessment and reliability of appropriate indicators were then evaluated within a pilot study on 29 patients affected by cognitive impairment or dementia. For measurable indicators the inter-observer reliability was calculated with the Kappa statistic. Initially, 82 indicators for care of vulnerable older adults with cognitive impairment or dementia were submitted to the panellists. Of those, 72 (88%) were accepted after two rounds. Among 29 patients for whom medical files of the preceding two years were evaluated, 63 (88%) of these indicators were considered applicable at least once, for at least one patient. Only 22 indicators were considered applicable at least once for ten or more out of 29 patients. Four indicators could be measured with the help of a validated questionnaire on patient satisfaction. Inter-observer reliability was moderate (Kappa = 0.57). A multidisciplinary panel of experts judged a large majority of the initial indicators valid for use in integrated care systems for vulnerable older adults in Quebec, Canada. Most of these indicators can be measured using patient files or patient or caregiver interviews and reliability of assessment from patient-files is moderate.

Selecting process quality indicators for the integrated care of vulnerable older adults affected by cognitive impairment or dementia

PubMed Central

Kröger, Edeltraut; Tourigny, André; Morin, Diane; Côté, Lise; Kergoat, Marie-Jeanne; Lebel, Paule; Robichaud, Line; Imbeault, Shirley; Proulx, Solange; Benounissa, Zohra

2007-01-01

Background This study aimed at evaluating face and content validity, feasibility and reliability of process quality indicators developed previously in the United States or other countries. The indicators can be used to evaluate care and services for vulnerable older adults affected by cognitive impairment or dementia within an integrated service system in Quebec, Canada. Methods A total of 33 clinical experts from three major urban centres in Quebec formed a panel representing two medical specialties (family medicine, geriatrics) and seven health or social services specialties (nursing, occupational therapy, psychology, neuropsychology, pharmacy, nutrition, social work), from primary or secondary levels of care, including long-term care. A modified version of the RAND®/University of California at Los Angeles (UCLA) appropriateness method, a two-round Delphi panel, was used to assess face and content validity of process quality indicators. The appropriateness of indicators was evaluated according to a) agreement of the panel with three criteria, defined as a median rating of 7–9 on a nine-point rating scale, and b) agreement among panellists, judged by the statistical measure of the interpercentile range adjusted for symmetry. Feasibility of quality assessment and reliability of appropriate indicators were then evaluated within a pilot study on 29 patients affected by cognitive impairment or dementia. For measurable indicators the inter-observer reliability was calculated with the Kappa statistic. Results Initially, 82 indicators for care of vulnerable older adults with cognitive impairment or dementia were submitted to the panellists. Of those, 72 (88%) were accepted after two rounds. Among 29 patients for whom medical files of the preceding two years were evaluated, 63 (88%) of these indicators were considered applicable at least once, for at least one patient. Only 22 indicators were considered applicable at least once for ten or more out of 29 patients. Four indicators could be measured with the help of a validated questionnaire on patient satisfaction. Inter-observer reliability was moderate (Kappa = 0.57). Conclusion A multidisciplinary panel of experts judged a large majority of the initial indicators valid for use in integrated care systems for vulnerable older adults in Quebec, Canada. Most of these indicators can be measured using patient files or patient or caregiver interviews and reliability of assessment from patient-files is moderate. PMID:18047668

Reliability testing of a portfolio assessment tool for postgraduate family medicine training in South Africa

PubMed Central

Mash, Bob; Derese, Anselme

2013-01-01

Abstract Background Competency-based education and the validity and reliability of workplace-based assessment of postgraduate trainees have received increasing attention worldwide. Family medicine was recognised as a speciality in South Africa six years ago and a satisfactory portfolio of learning is a prerequisite to sit the national exit exam. A massive scaling up of the number of family physicians is needed in order to meet the health needs of the country. Aim The aim of this study was to develop a reliable, robust and feasible portfolio assessment tool (PAT) for South Africa. Methods Six raters each rated nine portfolios from the Stellenbosch University programme, using the PAT, to test for inter-rater reliability. This rating was repeated three months later to determine test–retest reliability. Following initial analysis and feedback the PAT was modified and the inter-rater reliability again assessed on nine new portfolios. An acceptable intra-class correlation was considered to be > 0.80. Results The total score was found to be reliable, with a coefficient of 0.92. For test–retest reliability, the difference in mean total score was 1.7%, which was not statistically significant. Amongst the subsections, only assessment of the educational meetings and the logbook showed reliability coefficients > 0.80. Conclusion This was the first attempt to develop a reliable, robust and feasible national portfolio assessment tool to assess postgraduate family medicine training in the South African context. The tool was reliable for the total score, but the low reliability of several sections in the PAT helped us to develop 12 recommendations regarding the use of the portfolio, the design of the PAT and the training of raters.

Derivation, Validation and Application of a Pragmatic Risk Prediction Index for Benchmarking of Surgical Outcomes.

PubMed

Spence, Richard T; Chang, David C; Kaafarani, Haytham M A; Panieri, Eugenio; Anderson, Geoffrey A; Hutter, Matthew M

2018-02-01

Despite the existence of multiple validated risk assessment and quality benchmarking tools in surgery, their utility outside of high-income countries is limited. We sought to derive, validate and apply a scoring system that is both (1) feasible, and (2) reliably predicts mortality in a middle-income country (MIC) context. A 5-step methodology was used: (1) development of a de novo surgical outcomes database modeled around the American College of Surgeons' National Surgical Quality Improvement Program (ACS-NSQIP) in South Africa (SA dataset), (2) use of the resultant data to identify all predictors of in-hospital death with more than 90% capture indicating feasibility of collection, (3) use these predictors to derive and validate an integer-based score that reliably predicts in-hospital death in the 2012 ACS-NSQIP, (4) apply the score in the original SA dataset and demonstrate its performance, (5) identify threshold cutoffs of the score to prompt action and drive quality improvement. Following step one-three above, the 13 point Codman's score was derived and validated on 211,737 and 109,079 patients, respectively, and includes: age 65 (1), partially or completely dependent functional status (1), preoperative transfusions ≥4 units (1), emergency operation (2), sepsis or septic shock (2) American Society of Anesthesia score ≥3 (3) and operative procedure (1-3). Application of the score to 373 patients in the SA dataset showed good discrimination and calibration to predict an in-hospital death. A Codman Score of 8 is an optimal cutoff point for defining expected and unexpected deaths. We have designed a novel risk prediction score specific for a MIC context. The Codman Score can prove useful for both (1) preoperative decision-making and (2) benchmarking the quality of surgical care in MIC's.

Feasibility of remote administration of the Fundamentals of Laparoscopic Surgery (FLS) skills test.

PubMed

Okrainec, Allan; Vassiliou, Melina; Kapoor, Andrew; Pitzul, Kristen; Henao, Oscar; Kaneva, Pepa; Jackson, Timothy; Ritter, E Matt

2013-11-01

Fundamentals of Laparoscopic Surgery (FLS) certification testing currently is offered at accredited test centers or at select surgical conferences. Maintaining these test centers requires considerable investment in human and financial resources. Additionally, it can be challenging for individuals outside North America to become FLS certified. The objective of this pilot study was to assess the feasibility of remotely administering and scoring the FLS examination using live videoconferencing compared with standard onsite testing. This parallel mixed-methods study used both FLS scoring data and participant feedback to determine the barriers to feasibility of remote proctoring for the FLS examination. Participants were tested at two accredited FLS testing centers. An official FLS proctor administered and scored the FLS exam remotely while another onsite proctor provided a live score of participants' performance. Participant feedback was collected during testing. Interrater reliabilities of onsite and remote FLS scoring data were compared using intraclass correlation coefficients (ICCs). Participant feedback was analyzed using modified grounded theory to identify themes for barriers to feasibility. The scores of the remote and onsite proctors showed excellent interrater reliability in the total FLS (ICC 0.995, CI [0.985-0.998]). Several barriers led to critical errors in remote scoring, but most were accompanied by a solution incorporated into the study protocol. The most common barrier was the chain of custody for exam accessories. The results of this pilot study suggest that remote administration of the FLS has the potential to decrease costs without altering test-taker scores or exam validity. Further research is required to validate protocols for remote and onsite proctors and to direct execution of these protocols in a controlled environment identical to current FLS test administration.

Feasibility, reliability, and validity of the Pediatric Quality of Life Inventory ™ generic core scales, cancer module, and multidimensional fatigue scale in long-term adult survivors of pediatric cancer.

PubMed

Robert, Rhonda S; Paxton, Raheem J; Palla, Shana L; Yang, Grace; Askins, Martha A; Joy, Shaini E; Ater, Joann L

2012-10-01

Most health-related quality of life assessments are designed for either children or adults and have not been evaluated for adolescent and young adult survivors of pediatric cancer. The objective of this study was to examine the feasibility, reliability, and validity of the Pediatric Quality of Life Inventory (PedsQL ™ Generic Core Scales, Cancer Module, and Multidimensional Fatigue Scale in adult survivors of pediatric cancer. Adult survivors (n = 64; Mean age 35 year old; >2 years after treatment) completed the PedsQL™ Generic Core Scales, Cancer Module, and Multidimensional Fatigue Scale. Feasibility was examined with floor and ceiling effects; and internal consistency was determined by Cronbach's coefficient alpha calculations. Inter-factor correlations were also assessed. Significant ceiling effects were observed for the scales of social function, nausea, procedural anxiety, treatment anxiety, and communication. Internal consistency for all subscales was within the recommended ranges (α ≥ 0.70). Moderate to strong correlations between most Cancer Module and Generic Core Scales (r = 0.25 to r = 0.76) and between the Multidimensional Fatigue Scale and Generic Core Scales (r = 0.37 to r = 0.73). The PedsQL™ Generic Core Scales, Cancer Module, and Multidimensional Fatigue Scale appear to be feasible for an older population of pediatric cancer survivors; however, some of the Cancer Module Scales (nausea, procedural/treatment anxiety, and communication) were deemed not relevant for long-term survivors. More information is needed to determine whether the issues addressed by these modules are meaningful to long-term adult survivors of pediatric cancers. Copyright © 2012 Wiley Periodicals, Inc.

Using a virtual reality game to assess goal-directed hand movements in children: A pilot feasibility study.

PubMed

Gabyzon, M Elboim; Engel-Yeger, B; Tresser, S; Springer, S

2016-01-01

Virtual reality gaming environments may be used as a supplement to the motor performance assessment tool box by providing clinicians with quantitative information regarding motor performance in terms of movement accuracy and speed, as well as sensory motor integration under different levels of dual tasking. To examine the feasibility of using the virtual reality game `Timocco' as an assessment tool for evaluating goal-directed hand movements among typically developing children. In this pilot study, 47 typically-developing children were divided into two age groups, 4-6 years old and 6-8 years old. Performance was measured using two different virtual environment games (Bubble Bath and Falling Fruit), each with two levels of difficulty. Discriminative validity (age effect) was examined by comparing the performance of the two groups, and by comparing the performance between levels of the games for each group (level effect). Test-retest reliability was examined by reassessing the older children 3-7 days after the first session. The older children performed significantly better in terms of response time, action time, game duration, and efficiency in both games compared to the younger children. Both age groups demonstrated poorer performance at the higher game level in the Bubble Bath game compared to the lower level. A similar level effect was found in the Falling Fruit game for both age groups in response time and efficiency, but not in action time. The performance of the older children was not significantly different between the two sessions at both game levels. The discriminative validity and test-retest reliability indicate the feasibility of using the Timocco virtual reality game as a tool for assessing goal-directed hand movements in children. Further studies should examine its feasibility for use in children with disabilities.

Evaluation of Animal-Based Indicators to Be Used in a Welfare Assessment Protocol for Sheep.

PubMed

Richmond, Susan E; Wemelsfelder, Francoise; de Heredia, Ina Beltran; Ruiz, Roberto; Canali, Elisabetta; Dwyer, Cathy M

2017-01-01

Sheep are managed under a variety of different environments (continually outdoors, partially outdoors with seasonal or diurnal variation, continuously indoors) and for different purposes, which makes assessing welfare challenging. This diversity means that resource-based indicators are not particularly useful and, thus, a welfare assessment scheme for sheep, focusing on animal-based indicators, was developed. We focus specifically on ewes, as the most numerous group of sheep present on farm, although many of the indicators may also have relevance to adult male sheep. Using the Welfare Quality ® framework of four Principles and 12 Criteria, we considered the validity, reliability, and feasibility of 46 putative animal-based indicators derived from the literature for these criteria. Where animal-based indicators were potentially unreliably or were not considered feasible, we also considered the resource-based indicators of access to water, stocking density, and floor slipperiness. With the exception of the criteria "Absence of prolonged thirst," we suggest at least one animal-based indicator for each welfare criterion. As a minimum, face validity was available for all indicators; however, for many, we found evidence of convergent validity and discriminant validity (e.g., lameness as measured by gait score, body condition score). The reliability of most of the physical and health measures has been tested in the field and found to be appropriate for use in welfare assessment. However, for the majority of the proposed behavioral indicators (lying synchrony, social withdrawal, postures associated with pain, vocalizations, stereotypy, vigilance, response to surprise, and human approach test), this still needs to be tested. In conclusion, the comprehensive assessment of sheep welfare through largely animal-based measures is supported by the literature through the use of indicators focusing on specific aspects of sheep biology. Further work is required for some indicators to ensure that measures are reliable when used in commercial settings.

Evaluation of Animal-Based Indicators to Be Used in a Welfare Assessment Protocol for Sheep

PubMed Central

Richmond, Susan E.; Wemelsfelder, Francoise; de Heredia, Ina Beltran; Ruiz, Roberto; Canali, Elisabetta; Dwyer, Cathy M.

2017-01-01

Sheep are managed under a variety of different environments (continually outdoors, partially outdoors with seasonal or diurnal variation, continuously indoors) and for different purposes, which makes assessing welfare challenging. This diversity means that resource-based indicators are not particularly useful and, thus, a welfare assessment scheme for sheep, focusing on animal-based indicators, was developed. We focus specifically on ewes, as the most numerous group of sheep present on farm, although many of the indicators may also have relevance to adult male sheep. Using the Welfare Quality® framework of four Principles and 12 Criteria, we considered the validity, reliability, and feasibility of 46 putative animal-based indicators derived from the literature for these criteria. Where animal-based indicators were potentially unreliably or were not considered feasible, we also considered the resource-based indicators of access to water, stocking density, and floor slipperiness. With the exception of the criteria “Absence of prolonged thirst,” we suggest at least one animal-based indicator for each welfare criterion. As a minimum, face validity was available for all indicators; however, for many, we found evidence of convergent validity and discriminant validity (e.g., lameness as measured by gait score, body condition score). The reliability of most of the physical and health measures has been tested in the field and found to be appropriate for use in welfare assessment. However, for the majority of the proposed behavioral indicators (lying synchrony, social withdrawal, postures associated with pain, vocalizations, stereotypy, vigilance, response to surprise, and human approach test), this still needs to be tested. In conclusion, the comprehensive assessment of sheep welfare through largely animal-based measures is supported by the literature through the use of indicators focusing on specific aspects of sheep biology. Further work is required for some indicators to ensure that measures are reliable when used in commercial settings. PMID:29322048

Validity and reliability of a simple, low cost measure to quantify children’s dietary intake in afterschool settings

PubMed Central

Davison, Kirsten K.; Austin, S. Bryn; Giles, Catherine; Cradock, Angie L.; Lee, Rebekka M.; Gortmaker, Steven L.

2017-01-01

Interest in evaluating and improving children’s diets in afterschool settings has grown, necessitating the development of feasible yet valid measures for capturing children’s intake in such settings. This study’s purpose was to test the criterion validity and cost of three unobtrusive visual estimation methods compared to a plate-weighing method: direct on-site observation using a 4-category rating scale and off-site rating of digital photographs taken on-site using 4- and 10-category scales. Participants were 111 children in grades 1–6 attending four afterschool programs in Boston, MA in December 2011. Researchers observed and photographed 174 total snack meals consumed across two days at each program. Visual estimates of consumption were compared to weighed estimates (the criterion measure) using intra-class correlations. All three methods were highly correlated with the criterion measure, ranging from 0.92–0.94 for total calories consumed, 0.86–0.94 for consumption of pre-packaged beverages, 0.90–0.93 for consumption of fruits/vegetables, and 0.92–0.96 for consumption of grains. For water, which was not pre-portioned, coefficients ranged from 0.47–0.52. The photographic methods also demonstrated excellent inter-rater reliability: 0.84–0.92 for the 4-point and 0.92–0.95 for the 10-point scale. The costs of the methods for estimating intake ranged from $0.62 per observation for the on-site direct visual method to $0.95 per observation for the criterion measure. This study demonstrates that feasible, inexpensive methods can validly and reliably measure children’s dietary intake in afterschool settings. Improving precision in measures of children’s dietary intake can reduce the likelihood of spurious or null findings in future studies. PMID:25596895

Indicators of the relative availability of healthy versus unhealthy foods in supermarkets: a validation study.

PubMed

Vandevijvere, Stefanie; Mackenzie, Tara; Mhurchu, Cliona Ni

2017-04-26

In-store availability of healthy and unhealthy foods may influence consumer purchases. Methods used to measure food availability, however, vary widely. A simple, valid, and reliable indicator to collect comparable data on in-store food availability is needed. Cumulative linear shelf length of and variety within 22 healthy and 28 unhealthy food groups, determined based on a comparison of three nutrient profiling systems, were measured in 15 New Zealand supermarkets. Inter-rater reliability was tested in one supermarket by a second researcher. The construct validity of five simple indicators of relative availability of healthy versus unhealthy foods was assessed against this 'gold standard'. Cumulative linear shelf length was a more sensitive and feasible measure of food availability than variety. Four out of five shelf length ratio indicators were significantly associated with the gold standard (ρ = 0.70-0.75). Based on a non-significant difference from the 'gold standard' (d = 0.053 ± 0.040) and feasibility, the ratio of cumulative linear shelf length of fresh and frozen fruits and vegetables versus soft and energy drinks, crisps and snacks, sweet biscuits and confectionery performed best for use in New Zealand supermarkets. Four out of the five shelf length ratio indicators of the relative availability of healthy versus unhealthy foods in-store tested could be used for future research and monitoring, but additional validation studies in other settings and countries are recommended. Consistent use of those shelf length ratio indicators could enhance comparability of supermarket food availability between studies, and help inform policies to create healthy consumer food retail environments.

Reliability and validity of the PHQ-9 for screening late-life depression in Chinese primary care.

PubMed

Chen, Shulin; Chiu, Helen; Xu, Baihua; Ma, Yan; Jin, Tao; Wu, Manhua; Conwell, Yeates

2010-11-01

The aim of this study was to examine the reliability and validation of the 9-item Patient Health Questionnaire (PHQ-9) for late-life depression in Chinese primary care. In the primary care clinics (PCCs) of Hangzhou city, we recruited 364 older patients (aged ≥ 60) for the PHQ-9 screening. Then 77 of them were further interviewed with Structured Clinical Interview for DSM Disorders (SCID) for the diagnosis of major depression in late life. Statistic strategies for the feasibility, reliability, validity, and receiver operating characteristic curve were performed. The mean administration time was 7.5 min, and the Cronbach's α was 0.91. The optimal cut-off score of PHQ-9 ≥ 9 revealed a sensitivity of 0.86, specificity of 0.77, and positive likelihood ratio of 5.73. The area under the curve (AUC) in this study was 0.92 (SD = 0.02, 95% CI 0.88-0.96). The PHQ-2 also revealed good sensitivity (0.84) and specificity (0.90) at the cut-off point ≥ 3. The PHQ-9 performs well and has acceptable psychometric properties for screening of patients with late-life depression in Chinese primary care settings.

Measuring Acculturation Among Central American Women with the Use of a Brief Language Scale

PubMed Central

Wallen, Gwenyth R.; Feldman, Robert H.; Anliker, Jean

2015-01-01

The purpose of this study was to test the reliability and validity of a brief language usage scale as a measure of acculturation in 197 Central American immigrant women. This study presents an analysis of cross-sectional survey data collected during face-to-face interviews conducted in Spanish as part of the program evaluation of the Infant Feeding for Hispanic Supplemental Nutrition Program for Women, Infants, and Children (WIC) Populations a Peer Education Model. The Short Acculturation Scale, a four-item language usage scale exploring the participants’ language preferences, was used as a measure of acculturation. The participant’s age, length of time in the United States, and perceived social support for breastfeeding were used as validation measures. Results demonstrated good internal reliability for the acculturation summary scale. Consistent with previous studies, significant correlations (p < 0.01) were found between acculturation and mother’s age, perceived social support for breastfeeding, and mother’s length of time in the United States. The reliability and validity data from this group of Central American immigrants support the continued use of this brief measure of acculturation in diverse Latino subpopulations when multidimensional measures are neither practical nor feasible. PMID:16228765

The FLIR ONE thermal imager for the assessment of burn wounds: Reliability and validity study.

PubMed

Jaspers, M E H; Carrière, M E; Meij-de Vries, A; Klaessens, J H G M; van Zuijlen, P P M

2017-11-01

Objective measurement tools may be of great value to provide early and reliable burn wound assessment. Thermal imaging is an easy, accessible and objective technique, which measures skin temperature as an indicator of tissue perfusion. These thermal images might be helpful in the assessment of burn wounds. However, before implementation of a novel measurement tool into clinical practice is considered, it is appropriate to test its clinimetric properties (i.e. reliability and validity). The objective of this study was to assess the reliability and validity of the recently introduced FLIR ONE thermal imager. Two observers obtained thermal images of burn wounds in adult patients at day 1-3, 4-7 and 8-10 after burn. Subsequently, temperature differences between the burn wound and healthy skin (ΔT) were calculated on an iPad mini containing the FLIR Tools app. To assess reliability, ΔT values of both observers were compared by calculating the intraclass correlation coefficient (ICC) and measurement error parameters. To assess validity, the ΔT values of the first observer were compared to the registered healing time of the burn wounds, which was specified into three categories: (I) ≤14 days, (II) 15-21 days and (III) >21 days. The ability of the FLIR ONE to discriminate between healing ≤21 days and >21 days was evaluated by means of a receiver operating characteristic curve and an optimal ΔT cut-off value. Reliability: ICCs were 0.99 for each time point, indicating excellent reliability up to 10 days after burn. The standard error of measurement varied between 0.17-0.22°C. the area under the curve was calculated at 0.69 (95% CI 0.54-0.84). A cut-off value of -1.15°C shows a moderate discrimination between burn wound healing ≤21 days and >21 days (46% sensitivity; 82% specificity). Our results show that the FLIR ONE thermal imager is highly reliable, but the moderate validity calls for additional research. However, the FLIR ONE is pre-eminently feasible, allowing easy and fast measurements in clinical burn practice. Copyright © 2017 Elsevier Ltd and ISBI. All rights reserved.

Psychometric performance of the brazilian version of the Mini-cuestionario de calidad de vida en la hipertensión arterial (MINICHAL).

PubMed

Soutello, Ana Lúcia Soares; Rodrigues, Roberta Cunha Matheus; Jannuzzi, Fernanda Freire; Spana, Thaís Moreira; Gallani, Maria Cecília Bueno Jayme; Nadruz Junior, Wilson

2011-01-01

This study aimed to evaluate the feasibility, acceptability, ceiling and floor effects, reliability, and convergent construct validity of the Brazilian version of the Mini Cuestionario de Calidad de Vida en la Hipertensión Arterial (MINICHAL). The study included 200 hypertensive outpatients in a university hospital and a primary healthcare unit. The MINICHAL was applied in 3.0 (± 1.0) minutes with 100% of the items answered. A "ceiling effect" was observed in both dimensions and in the total score, as well as evidence of measurement stability (ICC=0.74). The convergent validity was confirmed by significant positive correlations between similar dimensions of the MINICHAL and the SF-36, and significant negative correlations with the Minnesota Living with Heart Failure Questionnaire - MLHFQ, however, correlations between dissimilar constructs were also observed. It was concluded that the Brazilian version of the MINICHAL presents evidence of reliability and validity when applied to hypertensive outpatients.

Can skills assessment on a virtual reality trainer predict a surgical trainee's talent in laparoscopic surgery?

PubMed

Rosenthal, R; Gantert, W A; Scheidegger, D; Oertli, D

2006-08-01

A number of studies have investigated several aspects of feasibility and validity of performance assessments with virtual reality surgical simulators. However, the validity of performance assessments is limited by the reliability of such measurements, and some issues of reliability still need to be addressed. This study aimed to evaluate the hypothesis that test subjects show logarithmic performance curves on repetitive trials for a component task of laparoscopic cholecystectomy on a virtual reality simulator, and that interindividual differences in performance after considerable training are significant. According to kinesiologic theory, logarithmic performance curves are expected and an individual's learning capacity for a specific task can be extrapolated, allowing quantification of a person's innate ability to develop task-specific skills. In this study, 20 medical students at the University of Basel Medical School performed five trials of a standardized task on the LS 500 virtual reality simulator for laparoscopic surgery. Task completion time, number of errors, economy of instrument movements, and maximum speed of instrument movements were measured. The hypothesis was confirmed by the fact that the performance curves for some of the simulator measurements were very close to logarithmic curves, and there were significant interindividual differences in performance at the end of the repetitive trials. Assessment of perceptual motor skills and the innate ability of an individual with no prior experience in laparoscopic surgery to develop such skills using the LS 500 VR surgical simulator is feasible and reliable.

Using Mechanical Turk for research on cancer survivors.

PubMed

Arch, Joanna J; Carr, Alaina L

2017-10-01

The successful recruitment and study of cancer survivors within psycho-oncology research can be challenging, time-consuming, and expensive, particularly for key subgroups such as young adult cancer survivors. Online crowdsourcing platforms offer a potential solution that has not yet been investigated with regard to cancer populations. The current study assessed the presence of cancer survivors on Amazon's Mechanical Turk (MTurk) and the feasibility of using MTurk as an efficient, cost-effective, and reliable psycho-oncology recruitment and research platform. During a <4-month period, cancer survivors living in the United States were recruited on MTurk to complete two assessments, spaced 1 week apart, relating to psychosocial and cancer-related functioning. The reliability and validity of responses were investigated. Within a <4-month period, 464 self-identified cancer survivors on MTurk consented to and completed an online assessment. The vast majority (79.09%) provided reliable and valid study data according to multiple indices. The sample was highly diverse in terms of U.S. geography, socioeconomic status, and cancer type, and reflected a particularly strong presence of distressed and young adult cancer survivors (median age = 36 years). A majority of participants (58.19%) responded to a second survey sent one week later. Online crowdsourcing represents a feasible, efficient, and cost-effective recruitment and research platform for cancer survivors, particularly for young adult cancer survivors and those with significant distress. We discuss remaining challenges and future recommendations. Copyright © 2016 John Wiley & Sons, Ltd. Copyright © 2016 John Wiley & Sons, Ltd.

Advancing implementation science through measure development and evaluation: a study protocol.

PubMed

Lewis, Cara C; Weiner, Bryan J; Stanick, Cameo; Fischer, Sarah M

2015-07-22

Significant gaps related to measurement issues are among the most critical barriers to advancing implementation science. Three issues motivated the study aims: (a) the lack of stakeholder involvement in defining pragmatic measure qualities; (b) the dearth of measures, particularly for implementation outcomes; and (c) unknown psychometric and pragmatic strength of existing measures. Aim 1: Establish a stakeholder-driven operationalization of pragmatic measures and develop reliable, valid rating criteria for assessing the construct. Aim 2: Develop reliable, valid, and pragmatic measures of three critical implementation outcomes, acceptability, appropriateness, and feasibility. Aim 3: Identify Consolidated Framework for Implementation Research and Implementation Outcome Framework-linked measures that demonstrate both psychometric and pragmatic strength. For Aim 1, we will conduct (a) interviews with stakeholder panelists (N = 7) and complete a literature review to populate pragmatic measure construct criteria, (b) Q-sort activities (N = 20) to clarify the internal structure of the definition, (c) Delphi activities (N = 20) to achieve consensus on the dimension priorities, (d) test-retest and inter-rater reliability assessments of the emergent rating system, and (e) known-groups validity testing of the top three prioritized pragmatic criteria. For Aim 2, our systematic development process involves domain delineation, item generation, substantive validity assessment, structural validity assessment, reliability assessment, and predictive validity assessment. We will also assess discriminant validity, known-groups validity, structural invariance, sensitivity to change, and other pragmatic features. For Aim 3, we will refine our established evidence-based assessment (EBA) criteria, extract the relevant data from the literature, rate each measure using the EBA criteria, and summarize the data. The study outputs of each aim are expected to have a positive impact as they will establish and guide a comprehensive measurement-focused research agenda for implementation science and provide empirically supported measures, tools, and methods for accomplishing this work.

Standardised clients as assessors in a veterinary communication OSCE: a reliability and validity study.

PubMed

Artemiou, E; Adams, C L; Hecker, K G; Vallevand, A; Violato, C; Coe, J B

2014-11-22

In human medicine, standardised patients (SP) have been shown to reliably and accurately assess learners' communication performance in high-stakes certification Objective Structured Clinical Examinations (OSCE), offering a feasible way to reduce the need for recruitment, time commitment and coordination of faculty assessors. In this study, we evaluated the use of standardised clients (SC) as a viable option for assessing veterinary students' communication performance. We designed a four-station, two-track communication skills OSCE. SC assessors used an adapted nine-item Liverpool Undergraduate Communication Assessment Scale (LUCAS). Faculty used a 21-item checklist derived from the Calgary-Cambridge Guide (CCG) and a five-point global rating scale. Participants were second year veterinary students (n=96). For the four stations, intrastation reliability (α) ranged from 0.63 to 0.82 for the LUCAS, and 0.73 to 0.87 for the CCG. The interstation reliability coefficients were 0.85 for the LUCAS and 0.89 for the CGG. The calculated Generalisability (G) coefficients were 0.62 for the LUCAS and 0.60 for the CGG. Supporting construct validity, SC and faculty assessors showed a significant correlation between the LUCAS and CCG total percent scores (r=0.45, P<0.001), and likewise between the LUCAS and global rating scores (r=0.49, P<0.001).Study results support that SC assessors offer a reliable and valid approach for assessing veterinary communication OSCE. British Veterinary Association.

SRB/SLEEC (Solid Rocket Booster/Shingle Lap Extendible Exit Cone) feasibility study, volume 1

NASA Technical Reports Server (NTRS)

Baker, William H., Jr.

1986-01-01

A preliminary design and analysis was completed for a SLEEC (Shingle Lap Extendible Exit Cone) which could be incorporated on the Space Transportation System (STS) Solid Rocket Booster (SRB). Studies were completed which predicted weights and performance increases and development plans were prepared for the full-scale bench and static test of SLEEC. In conjunction with the design studies, a series of supporting analyses were performed to assure the validity and feasibility of performance, fabrication, cost, and reliability for the selected design. The feasibility and required amounts of bench, static firing, and flight tests considered necessary for the successful incorporation of SLEEC on the Shuttle SRBs were determined. Preliminary plans were completed which define both a follow on study effort and a development program.

Development and feasibility of a wearable infant wrist band for the objective measurement of physical activity using accelerometery.

PubMed

Prioreschi, Alessandra; Nappey, Thomas; Westgate, Kate; Olivier, Patrick; Brage, Soren; Micklesfield, Lisa Kim

2018-01-01

It is important to be able to reliably and feasibly measure infant and toddler physical activity in order to determine adherence to current physical activity guidelines and effects on early life development, growth and health. This study aimed to describe the development of an infant wearable wrist-worn band for the measurement of physical activity; to determine the feasibility of the device data for observational measurement of physical activity and to determine the caregiver reported acceptability of the infant wearable wrist band. After various iterations of prototypes and piloting thereof, a final wearable band was designed to fit an Axivity AX3 monitor. Mother and infant/toddler (aged 3-24 months) pairs ( n  = 152) were recruited, and mothers were asked for their child to wear the band with enclosed monitor at all times for 1 week (minimum 3 days). Feasibility was assessed by determining technical reliability of the data, as well as wear time and compliance according to requirements for observational measurement. Acceptability was assessed via questionnaire. Technical reliability of the Axivity AX3 monitors in this age group was good. After excluding days that did not have at least 15 h of wear time, only 2% of participants had less than three valid days of data remaining, and 4% of participants had no data (due to device loss or data loss). Therefore, 94% of participants were compliant, having three or more days of wear with at least 15 h of wear per day, thus providing enough valid data for observational measurement. The majority (60%) of mothers reported being "very happy" with the safety of the device, while only 8% were "a little worried". A large majority (86%) of mothers stated that the band attracted attention from others, although this was mostly attributed to curiosity about the function of the band. Most (80%) of participants rated the comfort of the band as "comfortable", and 10% rated it as "very comfortable". The infant wearable band proved to be feasible and acceptable according to the criteria tested, and compliance wearing the band was good. We have therefore provided a replicable, comfortable and acceptable wearable band for the measurement of infant and toddler physical activity.

Health-Related Quality of Life and Cognitive Functioning from the Perspective of Parents of School-Aged Children with Asperger's Syndrome Utilizing the PedsQL[TM

ERIC Educational Resources Information Center

Limbers, Christine A.; Heffer, Robert W.; Varni, James W.

2009-01-01

HRQOL as a multidimensional construct has not been previously investigated in children with Asperger's Syndrome. The objective of the present study was to examine the initial feasibility, reliability, and validity of the PedsQL[TM] 4.0 Generic Core Scales and PedsQL[TM] Cognitive Functioning Scale parent proxy-report versions in school-aged…

Development and validation of a brief trauma screening measure for children: The Child Trauma Screen.

PubMed

Lang, Jason M; Connell, Christian M

2017-05-01

Childhood exposure to trauma, including violence and abuse, is a major public health concern that has resulted in increased efforts to promote trauma-informed child-serving systems. Trauma screening is an important component of such trauma-informed systems, yet widespread use of trauma screening is rare in part due to the lack of brief, validated trauma screening measures for children. We describe development and validation of the Child Trauma Screen (CTS), a 10-item screening measure of trauma exposure and posttraumatic stress disorder (PTSD) symptoms for children consistent with the DSM-5 definition of PTSD. Study 1 describes measure development incorporating analysis to derive items based on existing measures from 1,065 children and caregivers together with stakeholder input to finalize item selection. Study 2 describes validation of the CTS with a clinical sample of 74 children and their caregivers. Results support the CTS as an empirically derived, reliable measure to screen children for trauma exposure and PTSD symptoms with strong convergent, divergent, and criterion validity. The CTS is a promising measure for rapidly and reliably screening children for trauma exposure and PTSD symptoms. Future research is needed to confirm validation and to examine feasibility and utility of its use across various child-serving systems. (PsycINFO Database Record (c) 2017 APA, all rights reserved).

Reaching clinically relevant outcome measures for new pharmacotherapy and immunotherapy of atopic eczema.

PubMed

Chalmers, Joanne; Deckert, Stefanie; Schmitt, Jochen

2015-06-01

This article describes the core outcome set (COS) for atopic eczema trials. COS describe a minimum set of outcomes to be assessed in a defined situation. COS are required to overcome the current situation of different trials using different endpoints with unclear/insufficient measurement properties resulting in incomparable trials. The global multi-stakeholder Harmonising Outcomes Measures for Eczema initiative developed the Harmonising Outcomes Measures for Eczema roadmap as a generic framework for COS development. Following the establishment of a panel representing all stakeholders, a core set of outcome domains need to be selected based on systematic reviews and consensus methods. Outcome measurement instruments to assess these core domains need to be valid, reliable, and feasible. There is broad global consensus that clinical signs, quality of life, symptoms, and long-term control of flares form the COS for atopic eczema trials. The Eczema Area and Severity Index is recommended to assess clinical signs in atopic eczema trials. Systematic reviews to identify adequate outcome measurement instruments for the other core outcome domains are underway. Clinical signs should be assessed in all atopic eczema trials by at least the Eczema Area and Severity Index. Quality of life, symptoms, and flares should also be assessed in all atopic eczema trials by a valid, reliable, and feasible instrument.

Reliability, Validity, and Minimal Detectable Change of Four-Step Stair Climb Power Test in Community-Dwelling Older Adults.

PubMed

Ni, Meng; Brown, Lorna G; Lawler, Danielle; Bean, Jonathan F

2017-07-01

Stair climb power is an important clinical measure of lower-extremity power. The stair climb power test (SCPT) was validated by requiring individuals to climb a full flight of stairs. A 4-step SCPT (4SCPT) would be more clinically feasible and easier to perform, yet its reliability and validity are unknown. To evaluate reliability, validity, and minimal detectable change of 4SCPT among community-dwelling older adults. This study is a cross-sectional analysis of baseline data from a clinical trial. Fifty older adults ≥65 years of age, at risk for mobility decline, consented to participate in this ancillary study. Test-retest reliability was derived from 2 measurements within each participant measured by a single assessor. Pearson correlation analyses among leg power measures (4SCPT, SCPT, single leg press power at 40% and 70% of the 1-repetition maximum [SLP40, SLP70]) were performed. Separate multivariate linear regressions were conducted evaluating the associations between each leg power measure and 2 mobility outcomes, the Short Physical Performance Battery (SPPB) and habitual gait speed (HGS). Minimal detectable change was based on a 90% confidence interval (MDC 90 ). The 4SCPT had excellent test-retest reliability (ICC(2,1) = 0.951), and strong correlation with SCPT, SLP40, and SLP70 ( r = 0.85-0.96). The 4SCPT explained a greater amount of variance in the SPPB (R 2 = 0.31) than other leg power measurements (R 2 = 0.23-0.25). The 4SCPT (R 2 = 0.41) and SCPT (R 2 = 0.42) described equivalent amounts of variance in HGS, and greater than that with SLP40 (R 2 = 0.28) and SLP70 (R 2 = 0.30). The MDC 90 for 4SCPT was 44.0 watts. This was a cross-sectional analysis within a small, nonrepresentative sample. Interrater reliability was not evaluated. The 4SCPT shows scientific promise as a valid and reliable leg power measurement among community-dwelling older adults. © 2017 American Physical Therapy Association

The Feasibility of First Step to Success with Preschoolers.

PubMed

Frey, Andy J; Small, Jason; Feil, Edward; Seeley, John; Walker, Hill; Golly, Annemieke

2013-07-01

The primary purpose of this study was to examine feasibility of the preschool version of the First Step to Success (FSS) intervention. Toward this end, the following four research questions were addressed: (1) To what extent was the intervention implemented with integrity? (2) To what extent do teachers and parents perceive the intervention to be socially valid? (3) To what extent were teachers and parents satisfied with the intervention? and (4) To what extent was the intervention effective in reducing problem behavior and improving social skills? Twelve students participated in the study. Treatment integrity, social validity, and satisfaction results were analyzed at the aggregate level, and a reliable change index was calculated at the case level for primary outcome measures to assess the potential efficacy of the intervention. Fidelity data suggest the preschool version of the intervention can be implemented with acceptable integrity by coaches and teachers in preschool settings. Social validity outcomes suggest parents' perceptions of the program's goals, procedures, and outcomes were extremely favorable, and social validity from the teacher perspective was acceptable. The results provide initial evidence that participating in the preschool version of the FSS intervention improves children's social skills and decreases problem behavior.

The Feasibility of First Step to Success with Preschoolers

PubMed Central

Frey, Andy J.; Small, Jason; Feil, Edward; Seeley, John; Walker, Hill; Golly, Annemieke

2017-01-01

The primary purpose of this study was to examine feasibility of the preschool version of the First Step to Success (FSS) intervention. Toward this end, the following four research questions were addressed: (1) To what extent was the intervention implemented with integrity? (2) To what extent do teachers and parents perceive the intervention to be socially valid? (3) To what extent were teachers and parents satisfied with the intervention? and (4) To what extent was the intervention effective in reducing problem behavior and improving social skills? Twelve students participated in the study. Treatment integrity, social validity, and satisfaction results were analyzed at the aggregate level, and a reliable change index was calculated at the case level for primary outcome measures to assess the potential efficacy of the intervention. Fidelity data suggest the preschool version of the intervention can be implemented with acceptable integrity by coaches and teachers in preschool settings. Social validity outcomes suggest parents’ perceptions of the program’s goals, procedures, and outcomes were extremely favorable, and social validity from the teacher perspective was acceptable. The results provide initial evidence that participating in the preschool version of the FSS intervention improves children’s social skills and decreases problem behavior. PMID:29225519

Testing the psychometric properties of the Environmental Attitudes Inventory on undergraduate students in the Arab context: A test-retest approach.

PubMed

AlMenhali, Entesar Ali; Khalid, Khalizani; Iyanna, Shilpa

2018-01-01

The Environmental Attitudes Inventory (EAI) was developed to evaluate the multidimensional nature of environmental attitudes; however, it is based on a dataset from outside the Arab context. This study reinvestigated the construct validity of the EAI with a new dataset and confirmed the feasibility of applying it in the Arab context. One hundred and forty-eight subjects in Study 1 and 130 in Study 2 provided valid responses. An exploratory factor analysis (EFA) was used to extract a new factor structure in Study 1, and confirmatory factor analysis (CFA) was performed in Study 2. Both studies generated a seven-factor model, and the model fit was discussed for both the studies. Study 2 exhibited satisfactory model fit indices compared to Study 1. Factor loading values of a few items in Study 1 affected the reliability values and average variance extracted values, which demonstrated low discriminant validity. Based on the results of the EFA and CFA, this study showed sufficient model fit and suggested the feasibility of applying the EAI in the Arab context with a good construct validity and internal consistency.

Testing the psychometric properties of the Environmental Attitudes Inventory on undergraduate students in the Arab context: A test-retest approach

PubMed Central

2018-01-01

The Environmental Attitudes Inventory (EAI) was developed to evaluate the multidimensional nature of environmental attitudes; however, it is based on a dataset from outside the Arab context. This study reinvestigated the construct validity of the EAI with a new dataset and confirmed the feasibility of applying it in the Arab context. One hundred and forty-eight subjects in Study 1 and 130 in Study 2 provided valid responses. An exploratory factor analysis (EFA) was used to extract a new factor structure in Study 1, and confirmatory factor analysis (CFA) was performed in Study 2. Both studies generated a seven-factor model, and the model fit was discussed for both the studies. Study 2 exhibited satisfactory model fit indices compared to Study 1. Factor loading values of a few items in Study 1 affected the reliability values and average variance extracted values, which demonstrated low discriminant validity. Based on the results of the EFA and CFA, this study showed sufficient model fit and suggested the feasibility of applying the EAI in the Arab context with a good construct validity and internal consistency. PMID:29758021

Screening emergency department patients for opioid drug use: A qualitative systematic review.

PubMed

Sahota, Preet Kaur; Shastry, Siri; Mukamel, Dana B; Murphy, Linda; Yang, Narisu; Lotfipour, Shahram; Chakravarthy, Bharath

2018-05-24

The opioid drug epidemic is a major public health concern and an economic burden in the United States. The purpose of this systematic review is to assess the reliability and validity of screening instruments used in emergency medicine settings to detect opioid use in patients and to assess psychometric data for each screening instrument. PubMed/MEDLINE, PsycINFO, Cochrane Database of Systematic Reviews, Cochrane Central Register of Controlled Trials, Web of Science, Cumulative Index to Nursing and Allied Health Literature and ClinicalTrials.gov were searched for articles published up to May 2018. The extracted articles were independently screened for eligibility by two reviewers. We extracted 1555 articles for initial screening and 95 articles were assessed for full-text eligibility. Six articles were extracted from the full-text assessment. Six instruments were identified from the final article list: Screener and Opioid Assessment for Patients with Pain - Revised; Drug Abuse Screening Test; Opioid Risk Tool; Current Opioid Misuse Measure; an Emergency Medicine Providers Clinician Assessment Questionnaire; and an Emergency Provider Impression Data Collection Form. Screening instrument characteristics, and reliability and validity data were extracted from the six studies. A meta-analysis was not conducted due to heterogeneity between the studies. There is a lack of validity and reliability evidence in all six articles; and sensitivity, specificity and predictive values varied between the different instruments. These instruments cannot be validated for use in emergency medicine settings. There is no clear evidence to state which screening instruments are appropriate for use in detecting opioid use disorders in emergency medicine patients. There is a need for brief, reliable, valid and feasible opioid use screening instruments in the emergency medicine setting. Copyright © 2018 Elsevier Ltd. All rights reserved.

A Compact Forearm Crutch Based on Force Sensors for Aided Gait: Reliability and Validity.

PubMed

Chamorro-Moriana, Gema; Sevillano, José Luis; Ridao-Fernández, Carmen

2016-06-21

Frequently, patients who suffer injuries in some lower member require forearm crutches in order to partially unload weight-bearing. These lesions cause pain in lower limb unloading and their progression should be controlled objectively to avoid significant errors in accuracy and, consequently, complications and after effects in lesions. The design of a new and feasible tool that allows us to control and improve the accuracy of loads exerted on crutches during aided gait is necessary, so as to unburden the lower limbs. In this paper, we describe such a system based on a force sensor, which we have named the GCH System 2.0. Furthermore, we determine the validity and reliability of measurements obtained using this tool via a comparison with the validated AMTI (Advanced Mechanical Technology, Inc., Watertown, MA, USA) OR6-7-2000 Platform. An intra-class correlation coefficient demonstrated excellent agreement between the AMTI Platform and the GCH System. A regression line to determine the predictive ability of the GCH system towards the AMTI Platform was found, which obtained a precision of 99.3%. A detailed statistical analysis is presented for all the measurements and also segregated for several requested loads on the crutches (10%, 25% and 50% of body weight). Our results show that our system, designed for assessing loads exerted by patients on forearm crutches during assisted gait, provides valid and reliable measurements of loads.

A Compact Forearm Crutch Based on Force Sensors for Aided Gait: Reliability and Validity

PubMed Central

Chamorro-Moriana, Gema; Sevillano, José Luis; Ridao-Fernández, Carmen

2016-01-01

Frequently, patients who suffer injuries in some lower member require forearm crutches in order to partially unload weight-bearing. These lesions cause pain in lower limb unloading and their progression should be controlled objectively to avoid significant errors in accuracy and, consequently, complications and after effects in lesions. The design of a new and feasible tool that allows us to control and improve the accuracy of loads exerted on crutches during aided gait is necessary, so as to unburden the lower limbs. In this paper, we describe such a system based on a force sensor, which we have named the GCH System 2.0. Furthermore, we determine the validity and reliability of measurements obtained using this tool via a comparison with the validated AMTI (Advanced Mechanical Technology, Inc., Watertown, MA, USA) OR6-7-2000 Platform. An intra-class correlation coefficient demonstrated excellent agreement between the AMTI Platform and the GCH System. A regression line to determine the predictive ability of the GCH system towards the AMTI Platform was found, which obtained a precision of 99.3%. A detailed statistical analysis is presented for all the measurements and also segregated for several requested loads on the crutches (10%, 25% and 50% of body weight). Our results show that our system, designed for assessing loads exerted by patients on forearm crutches during assisted gait, provides valid and reliable measurements of loads. PMID:27338396

Observation of Spontaneous Expressive Language (OSEL): A New Measure for Spontaneous and Expressive Language of Children with Autism Spectrum Disorders and Other Communication Disorders

PubMed Central

Kim, So Hyun; Junker, Dörte; Lord, Catherine

2014-01-01

A new language measure, the Observation of Spontaneous Expressive Language (OSEL), is intended to document spontaneous use of syntax, pragmatics, and semantics in 2-12-year-old children with ASD and other communication disorders with expressive language levels comparable to typical 2-5 year olds. Because the purpose of the OSEL is to provide developmental norms for use of language, the first step involves assessment of the scale’s feasibility, validity, and reliability using a sample of 180 2-5 year-old typically developing children. Pilot data from the OSEL shows strong internal consistency, high reliabilities and validity. Once replicated with a large population-based sample and in special populations, the scale should be helpful in designing appropriate interventions for children with ASD and other communication disorders. PMID:25022249

Observation of Spontaneous Expressive Language (OSEL): a new measure for spontaneous and expressive language of children with autism spectrum disorders and other communication disorders.

PubMed

Kim, So Hyun; Junker, Dörte; Lord, Catherine

2014-12-01

A new language measure, the Observation of Spontaneous Expressive Language (OSEL), is intended to document spontaneous use of syntax, pragmatics, and semantics in 2-12-year-old children with Autism Spectrum Disorder (ASD) and other communication disorders with expressive language levels comparable to typical 2-5 year olds. Because the purpose of the OSEL is to provide developmental norms for use of language, the first step involves assessment of the scale's feasibility, validity, and reliability using a sample of 180 2-5 year-old typically developing children. Pilot data from the OSEL shows strong internal consistency, high reliabilities and validity. Once replicated with a large population-based sample and in special populations, the scale should be helpful in designing appropriate interventions for children with ASD and other communication disorders.

A Serious Game for Clinical Assessment of Cognitive Status: Validation Study.

PubMed

Tong, Tiffany; Chignell, Mark; Tierney, Mary C; Lee, Jacques

2016-05-27

We propose the use of serious games to screen for abnormal cognitive status in situations where it may be too costly or impractical to use standard cognitive assessments (eg, emergency departments). If validated, serious games in health care could enable broader availability of efficient and engaging cognitive screening. The objective of this work is to demonstrate the feasibility of a game-based cognitive assessment delivered on tablet technology to a clinical sample and to conduct preliminary validation against standard mental status tools commonly used in elderly populations. We carried out a feasibility study in a hospital emergency department to evaluate the use of a serious game by elderly adults (N=146; age: mean 80.59, SD 6.00, range 70-94 years). We correlated game performance against a number of standard assessments, including the Mini-Mental State Examination (MMSE), Montreal Cognitive Assessment (MoCA), and the Confusion Assessment Method (CAM). After a series of modifications, the game could be used by a wide range of elderly patients in the emergency department demonstrating its feasibility for use with these users. Of 146 patients, 141 (96.6%) consented to participate and played our serious game. Refusals to play the game were typically due to concerns of family members rather than unwillingness of the patient to play the game. Performance on the serious game correlated significantly with the MoCA (r=-.339, P <.001) and MMSE (r=-.558, P <.001), and correlated (point-biserial correlation) with the CAM (r=.565, P <.001) and with other cognitive assessments. This research demonstrates the feasibility of using serious games in a clinical setting. Further research is required to demonstrate the validity and reliability of game-based assessments for clinical decision making.

A Serious Game for Clinical Assessment of Cognitive Status: Validation Study

PubMed Central

Chignell, Mark; Tierney, Mary C.; Lee, Jacques

2016-01-01

Background We propose the use of serious games to screen for abnormal cognitive status in situations where it may be too costly or impractical to use standard cognitive assessments (eg, emergency departments). If validated, serious games in health care could enable broader availability of efficient and engaging cognitive screening. Objective The objective of this work is to demonstrate the feasibility of a game-based cognitive assessment delivered on tablet technology to a clinical sample and to conduct preliminary validation against standard mental status tools commonly used in elderly populations. Methods We carried out a feasibility study in a hospital emergency department to evaluate the use of a serious game by elderly adults (N=146; age: mean 80.59, SD 6.00, range 70-94 years). We correlated game performance against a number of standard assessments, including the Mini-Mental State Examination (MMSE), Montreal Cognitive Assessment (MoCA), and the Confusion Assessment Method (CAM). Results After a series of modifications, the game could be used by a wide range of elderly patients in the emergency department demonstrating its feasibility for use with these users. Of 146 patients, 141 (96.6%) consented to participate and played our serious game. Refusals to play the game were typically due to concerns of family members rather than unwillingness of the patient to play the game. Performance on the serious game correlated significantly with the MoCA (r=–.339, P <.001) and MMSE (r=–.558, P <.001), and correlated (point-biserial correlation) with the CAM (r=.565, P <.001) and with other cognitive assessments. Conclusions This research demonstrates the feasibility of using serious games in a clinical setting. Further research is required to demonstrate the validity and reliability of game-based assessments for clinical decision making. PMID:27234145

Health literacy screening instruments for eHealth applications: a systematic review.

PubMed

Collins, Sarah A; Currie, Leanne M; Bakken, Suzanne; Vawdrey, David K; Stone, Patricia W

2012-06-01

To systematically review current health literacy (HL) instruments for use in consumer-facing and mobile health information technology screening and evaluation tools. The databases, PubMed, OVID, Google Scholar, Cochrane Library and Science Citation Index, were searched for health literacy assessment instruments using the terms "health", "literacy", "computer-based," and "psychometrics". All instruments identified by this method were critically appraised according to their reported psychometric properties and clinical feasibility. Eleven different health literacy instruments were found. Screening questions, such as asking a patient about his/her need for assistance in navigating health information, were evaluated in seven different studies and are promising for use as a valid, reliable, and feasible computer-based approach to identify patients that struggle with low health literacy. However, there was a lack of consistency in the types of screening questions proposed. There is also a lack of information regarding the psychometric properties of computer-based health literacy instruments. Only English language health literacy assessment instruments were reviewed and analyzed. Current health literacy screening tools demonstrate varying benefits depending on the context of their use. In many cases, it seems that a single screening question may be a reliable, valid, and feasible means for establishing health literacy. A combination of screening questions that assess health literacy and technological literacy may enable tailoring eHealth applications to user needs. Further research should determine the best screening question(s) and the best synthesis of various instruments' content and methodologies for computer-based health literacy screening and assessment. Copyright © 2012 Elsevier Inc. All rights reserved.

Health Literacy Screening Instruments for eHealth Applications: A Systematic Review

PubMed Central

Collins, Sarah A.; Currie, Leanne M.; Bakken, Suzanne; Vawdrey, David K.; Stone, Patricia W.

2012-01-01

Objective To systematically review current health literacy (HL) instruments for use in consumer-facing and mobile health information technology screening and evaluation tools. Design The databases, PubMed, OVID, Google Scholar, Cochrane Library and Science Citation Index, were searched for health literacy assessment instruments using the terms “health”, “literacy”, “computer-based,” and “psychometrics”. All instruments identified by this method were critically appraised according to their reported psychometric properties and clinical feasibility. Results Eleven different health literacy instruments were found. Screening questions, such as asking a patient about his/her need for assistance in navigating health information, were evaluated in 7 different studies and are promising for use as a valid, reliable, and feasible computer-based approach to identify patients that struggle with low health literacy. However, there was a lack of consistency in the types of screening questions proposed. There is also a lack of information regarding the psychometric properties of computer-based health literacy instruments. Limitations Only English language health literacy assessment instruments were reviewed and analyzed. Conclusions Current health literacy screening tools demonstrate varying benefits depending on the context of their use. In many cases, it seems that a single screening question may be a reliable, valid, and feasible means for establishing health literacy. A combination of screening questions that assess health literacy and technological literacy may enable tailoring eHealth applications to user needs. Further research should determine the best screening question(s) and the best synthesis of various instruments’ content and methodologies for computer-based health literacy screening and assessment. PMID:22521719

Reliability and validity of the international physical activity questionnaire for assessing walking.

PubMed

van der Ploeg, Hidde P; Tudor-Locke, Catrine; Marshall, Alison L; Craig, Cora; Hagströmer, Maria; Sjöström, Michael; Bauman, Adrian

2010-03-01

Physical inactivity and its accompanying adverse sequelae (e.g., obesity and diabetes) are global health concerns. The single most commonly reported physical activity in public health surveys is walking (Centers for Disease Control and Prevention, 2000; Rafferty, Reeves, McGee, & Pivarnik, 2002). As evidence accumulates that walking is important for preventing weight gain (Levine et al., 2008) and reducing the risk of diabetes (Jeon, Lokken, Hu, & van Dam, 2007), there is increased need to capture this behavior in a valid and reliable manner. Although the disadvantages of a self-report methodology are well known (Sallis, & Saelens, 2000), it still represents the most feasible approach for conducting population-level surveillance across developed and developing countries. The International Physical Activity Questionnaire (IPAQ) was created and evaluated as a standardized instrument for this purpose. Although two versions of the IPAQwere designed and evaluated (short: nine items; and long: 31 items), the short form was recommended for population monitoring (Craig et al., 2003). However, it has not been recommended for intervention or research studies that require precise physical activity quantification to examine changes in physical activity at the individual level. IPAQ was also not intended to replace instruments that are more responsive to individual changes in activity level, such as objective measures. In addition to walking behaviors, IPAQ also assesses time spent in moderate- and vigorous-intensity activity as well as sitting behaviors, although the latter is not the focus of this analysis. Aggregated IPAQ data have been previously validated compared to accelerometers, and overall reliability was confirmed across 12 countries (Craig et al., 2003). Previous research showed criterion validity Spearman correlations with a median of 0.30 and test-retest reliability Spearman correlations clustered around 0.8 (Craig et al., 2003). The purpose of this study, however, was to reanalyze these data with respect to validity (again compared to an accelerometer) and test-retest reliability specifically for population monitoring of walking.

Comprehensive proficiency-based inanimate training for robotic surgery: reliability, feasibility, and educational benefit.

PubMed

Arain, Nabeel A; Dulan, Genevieve; Hogg, Deborah C; Rege, Robert V; Powers, Cathryn E; Tesfay, Seifu T; Hynan, Linda S; Scott, Daniel J

2012-10-01

We previously developed a comprehensive proficiency-based robotic training curriculum demonstrating construct, content, and face validity. This study aimed to assess reliability, feasibility, and educational benefit associated with curricular implementation. Over an 11-month period, 55 residents, fellows, and faculty (robotic novices) from general surgery, urology, and gynecology were enrolled in a 2-month curriculum: online didactics, half-day hands-on tutorial, and self-practice using nine inanimate exercises. Each trainee completed a questionnaire and performed a single proctored repetition of each task before (pretest) and after (post-test) training. Tasks were scored for time and errors using modified FLS metrics. For inter-rater reliability (IRR), three trainees were scored by two raters and analyzed using intraclass correlation coefficients (ICC). Data from eight experts were analyzed using ICC and Cronbach's α to determine test-retest reliability and internal consistency, respectively. Educational benefit was assessed by comparing baseline (pretest) and final (post-test) trainee performance; comparisons used Wilcoxon signed-rank test. Of the 55 trainees that pretested, 53 (96 %) completed all curricular components in 9-17 h and reached proficiency after completing an average of 72 ± 28 repetitions over 5 ± 1 h. Trainees indicated minimal prior robotic experience and "poor comfort" with robotic skills at baseline (1.8 ± 0.9) compared to final testing (3.1 ± 0.8, p < 0.001). IRR data for the composite score revealed an ICC of 0.96 (p < 0.001). Test-retest reliability was 0.91 (p < 0.001) and internal consistency was 0.81. Performance improved significantly after training for all nine tasks and according to composite scores (548 ± 176 vs. 914 ± 81, p < 0.001), demonstrating educational benefit. This curriculum is associated with high reliability measures, demonstrated feasibility for a large cohort of trainees, and yielded significant educational benefit. Further studies and adoption of this curriculum are encouraged.

Measuring Quadriceps strength in adults with severe or moderate intellectual and visual disabilities: Feasibility and reliability.

PubMed

Dijkhuizen, Annemarie; Douma, Rob K; Krijnen, Wim P; van der Schans, Cees P; Waninge, Aly

2018-05-30

A feasible and reliable instrument to measure strength in persons with severe intellectual and visual disabilities (SIVD) is lacking. The aim of our study was to determine feasibility, learning period and reliability of three strength tests. Twenty-nine participants with SIVD performed the Minimum Sit-to-Stand Height test (MSST), the Leg Extension test (LE) and the 30 seconds Chair-Stand test (30sCS), once per week for 5 weeks. Feasibility was determined by the percentage of successful measurements; learning effect by using paired t test between two consecutive measurements; test-retest reliability by intraclass correlation coefficient and Limits of Agreement and, correlations by Pearson correlations. A sufficient feasibility and learning period of the tests was shown. The methods had sufficient test-retest reliability and moderate-to-sufficient correlations. The MSST, the LE, and the 30sCS are feasible tests for measuring muscle strength in persons with SIVD, having sufficient test re-test reliability. © 2018 John Wiley & Sons Ltd.

Evaluation of high fidelity patient simulator in assessment of performance of anaesthetists.

PubMed

Weller, J M; Bloch, M; Young, S; Maze, M; Oyesola, S; Wyner, J; Dob, D; Haire, K; Durbridge, J; Walker, T; Newble, D

2003-01-01

There is increasing emphasis on performance-based assessment of clinical competence. The High Fidelity Patient Simulator (HPS) may be useful for assessment of clinical practice in anaesthesia, but needs formal evaluation of validity, reliability, feasibility and effect on learning. We set out to assess the reliability of a global rating scale for scoring simulator performance in crisis management. Using a global rating scale, three judges independently rated videotapes of anaesthetists in simulated crises in the operating theatre. Five anaesthetists then independently rated subsets of these videotapes. There was good agreement between raters for medical management, behavioural attributes and overall performance. Agreement was high for both the initial judges and the five additional raters. Using a global scale to assess simulator performance, we found good inter-rater reliability for scoring performance in a crisis. We estimate that two judges should provide a reliable assessment. High fidelity simulation should be studied further for assessing clinical performance.

Development and evaluation of oral Cancer quality-of-life questionnaire (QOL-OC).

PubMed

Nie, Min; Liu, Chang; Pan, Yi-Chen; Jiang, Chen-Xi; Li, Bao-Ru; Yu, Xi-Jie; Wu, Xin-Yu; Zheng, Shu-Ning

2018-05-03

In this study scales and items for the Oral Cancer Quality-of-life Questionnaire (QOL-OC) were designed and the instrument was evaluated. The QOL-OC was developed and modified using the international definition of quality of life (QOL) promulgated by the European Organization for Research and Treatment of Cancer (EORTC) and analysis of the precedent measuring instruments. The contents of each item were determined in the context of the specific characteristics of oral cancer. Two hundred thirteen oral cancer patients were asked to complete both the EORTC core quality of life questionnaire (EORTC QLC-C30) and the QOL-OC. Data collected was used to conduct factor analysis, test-retest reliability, internal consistency, and construct validity. Questionnaire compliance was relatively high. Fourteen of the 213 subjects accepted the same tests after 24 to 48 h demonstrating a high test-retest reliability for all five scales. Overall internal consistency surpasses 0.8. The outcome of the factor analysis coincides substantially with our theoretical conception. Each item shows a higher correlation coefficient within its own scale than the others which indicates high construct validity. QOL-OC demonstrates fairly good statistical reliability, validity, and feasibility. However, further tests and modification are needed to ensure its applicability to the quality-of-life assessment of Chinese oral cancer patients.

Tests for the Assessment of Sport-Specific Performance in Olympic Combat Sports: A Systematic Review With Practical Recommendations.

PubMed

Chaabene, Helmi; Negra, Yassine; Bouguezzi, Raja; Capranica, Laura; Franchini, Emerson; Prieske, Olaf; Hbacha, Hamdi; Granacher, Urs

2018-01-01

The regular monitoring of physical fitness and sport-specific performance is important in elite sports to increase the likelihood of success in competition. This study aimed to systematically review and to critically appraise the methodological quality, validation data, and feasibility of the sport-specific performance assessment in Olympic combat sports like amateur boxing, fencing, judo, karate, taekwondo, and wrestling. A systematic search was conducted in the electronic databases PubMed, Google-Scholar, and Science-Direct up to October 2017. Studies in combat sports were included that reported validation data (e.g., reliability, validity, sensitivity) of sport-specific tests. Overall, 39 studies were eligible for inclusion in this review. The majority of studies (74%) contained sample sizes <30 subjects. Nearly, 1/3 of the reviewed studies lacked a sufficient description (e.g., anthropometrics, age, expertise level) of the included participants. Seventy-two percent of studies did not sufficiently report inclusion/exclusion criteria of their participants. In 62% of the included studies, the description and/or inclusion of a familiarization session (s) was either incomplete or not existent. Sixty-percent of studies did not report any details about the stability of testing conditions. Approximately half of the studies examined reliability measures of the included sport-specific tests (intraclass correlation coefficient [ICC] = 0.43-1.00). Content validity was addressed in all included studies, criterion validity (only the concurrent aspect of it) in approximately half of the studies with correlation coefficients ranging from r = -0.41 to 0.90. Construct validity was reported in 31% of the included studies and predictive validity in only one. Test sensitivity was addressed in 13% of the included studies. The majority of studies (64%) ignored and/or provided incomplete information on test feasibility and methodological limitations of the sport-specific test. In 28% of the included studies, insufficient information or a complete lack of information was provided in the respective field of the test application. Several methodological gaps exist in studies that used sport-specific performance tests in Olympic combat sports. Additional research should adopt more rigorous validation procedures in the application and description of sport-specific performance tests in Olympic combat sports.

Development and Validation of an HIV Risk Exposure and Indicator Conditions Questionnaire to Support Targeted HIV Screening.

PubMed

Elías, María Jesús Pérez; Gómez-Ayerbe, Cristina; Elías, Pilar Pérez; Muriel, Alfonso; de Santiago, Alberto Diaz; Martinez-Colubi, María; Moreno, Ana; Santos, Cristina; Polo, Lidia; Barea, Rafa; Robledillo, Gema; Uranga, Almudena; Espín, Agustina Cano; Quereda, Carmen; Dronda, Fernando; Casado, Jose Luis; Moreno, Santiago

2016-02-01

The aim of our study was to develop a Spanish-structured HIV risk of exposure and indicator conditions (RE&IC) questionnaire. People attending to an emergency room or to a primary clinical care center were offered to participate in a prospective, 1 arm, open label study, in which all enrolled patients filled out our developed questionnaire and were HIV tested. Questionnaire accuracy, feasibility, and reliability were evaluated.Valid paired 5329 HIV RE&IC questionnaire and rapid HIV tests were performed, 69.3% in the primary clinical care center, 49.6% women, median age 37 years old, 74.9% Spaniards, 20.1% Latin-Americans. Confirmed hidden HIV infection was detected in 4.1%, while HIV RE&IC questionnaire was positive in 51.2%. HIV RE&IC questionnaire sensitivity was 100% to predict HIV infection, with a 100% negative predictive value. When considered separately, RE or IC items sensitivity decreases to 86.4% or 91%, and similarly their negative predictive value to 99.9% for both of them. The majority of people studied, 90.8% self-completed HIV RE&IC questionnaire. Median time to complete was 3 minutes. Overall HIV RE&IC questionnaire test-retest Kappa agreement was 0.82 (almost perfect), likewise for IC items 0.89, while for RE items was lower 0.78 (substantial).A feasible and reliable Spanish HIV RE&IC self questionnaire accurately discriminated all non-HIV-infected people without missing any HIV diagnoses, in a low prevalence HIV infection area. The best accuracy and reliability were obtained when combining HIV RE&IC items.

The PedsQL Multidimensional Fatigue Scale in young adults: feasibility, reliability and validity in a University student population.

PubMed

Varni, James W; Limbers, Christine A

2008-02-01

The PedsQL (Pediatric Quality of Life Inventory) is a modular instrument designed to measure health-related quality of life (HRQOL) and disease-specific symptoms in children and adolescents ages 2-18. The PedsQL Multidimensional Fatigue Scale was designed as a generic symptom-specific instrument to measure fatigue in pediatric patients ages 2-18. Since a sizeable number of pediatric patients prefer to remain with their pediatric providers after age 18, the objective of the present study was to determine the feasibility, reliability, and validity of the PedsQL Multidimensional Fatigue Scale in young adults. The 18-item PedsQL Multidimensional Fatigue Scale (General Fatigue, Sleep/Rest Fatigue, and Cognitive Fatigue domains), the PedsQL 4.0 Generic Core Scales Young Adult Version, and the SF-8 Health Survey were completed by 423 university students ages 18-25. The PedsQL Multidimensional Fatigue Scale evidenced minimal missing responses, achieved excellent reliability for the Total Scale Score (alpha = 0.90), distinguished between healthy young adults and young adults with chronic health conditions, was significantly correlated with the relevant PedsQL 4.0 Generic Core Scales and the SF-8 standardized scores, and demonstrated a factor-derived structure largely consistent with the a priori conceptual model. The results demonstrate the measurement properties of the PedsQL Multidimensional Fatigue Scale in a convenience sample of young adult university students. The findings suggest that the PedsQL Multidimensional Fatigue Scale may be utilized in the evaluation of fatigue for a broad age range.

PedsQL Neurofibromatosis Type 1 Module for children, adolescents and young adults: feasibility, reliability, and validity.

PubMed

Nutakki, Kavitha; Varni, James W; Swigonski, Nancy L

2018-04-01

The objective of the present study was to report on the measurement properties of the Pediatric Quality of Life Inventory (PedsQL) Neurofibromatosis Type 1 Module for pediatric patients ages 5-25 from the perspectives of patients and parents. The 104-item PedsQL NF1 Module and 23-item PedsQL Generic Core Scales were completed in a multi-site national study by 323 patients and 335 parents (343 families). Patients were diagnosed with NF1 using the National Institutes of Health diagnostic criteria. In addition to a Total Scale Score, 18 unidimensional scales were derived measuring skin itch bother, skin sensations, pain, pain impact, pain management, cognitive functioning, speech, fine motor, balance, vision, perceived physical appearance, communication, worry, treatment anxiety, medicines, stomach discomfort, constipation, and diarrhea. The PedsQL NF1 Module Scales evidenced excellent feasibility, excellent reliability for the Total Scale Scores (patient self-report α = 0.98; parent proxy-report α = 0.98), and good to excellent reliability for the 18 individual scales (patient self-report α = 0.71-0.96; parent proxy-report α = 0.73-0.98). Intercorrelations with the Generic Core Scales supported construct validity. Factor analysis supported the unidimensionality of the 18 individual scales. The PedsQL NF1 Module Scales demonstrated acceptable to excellent measurement properties, and may be utilized as standardized metrics to assess NF1-specific symptoms and problems in clinical research and practice in children, adolescents, and young adults.

Development and Validation of an HIV Risk Exposure and Indicator Conditions Questionnaire to Support Targeted HIV Screening

PubMed Central

Elías, María Jesús Pérez; Gómez-Ayerbe, Cristina; Elías, Pilar Pérez; Muriel, Alfonso; de Santiago, Alberto Diaz; Martinez-Colubi, María; Moreno, Ana; Santos, Cristina; Polo, Lidia; Barea, Rafa; Robledillo, Gema; Uranga, Almudena; Espín, Agustina Cano; Quereda, Carmen; Dronda, Fernando; Casado, Jose Luis; Moreno, Santiago

2016-01-01

Abstract The aim of our study was to develop a Spanish-structured HIV risk of exposure and indicator conditions (RE&IC) questionnaire. People attending to an emergency room or to a primary clinical care center were offered to participate in a prospective, 1 arm, open label study, in which all enrolled patients filled out our developed questionnaire and were HIV tested. Questionnaire accuracy, feasibility, and reliability were evaluated. Valid paired 5329 HIV RE&IC questionnaire and rapid HIV tests were performed, 69.3% in the primary clinical care center, 49.6% women, median age 37 years old, 74.9% Spaniards, 20.1% Latin-Americans. Confirmed hidden HIV infection was detected in 4.1%, while HIV RE&IC questionnaire was positive in 51.2%. HIV RE&IC questionnaire sensitivity was 100% to predict HIV infection, with a 100% negative predictive value. When considered separately, RE or IC items sensitivity decreases to 86.4% or 91%, and similarly their negative predictive value to 99.9% for both of them. The majority of people studied, 90.8% self-completed HIV RE&IC questionnaire. Median time to complete was 3 minutes. Overall HIV RE&IC questionnaire test-retest Kappa agreement was 0.82 (almost perfect), likewise for IC items 0.89, while for RE items was lower 0.78 (substantial). A feasible and reliable Spanish HIV RE&IC self questionnaire accurately discriminated all non–HIV-infected people without missing any HIV diagnoses, in a low prevalence HIV infection area. The best accuracy and reliability were obtained when combining HIV RE&IC items. PMID:26844471

Hypochondriasis Y-BOCS: a study of the psychometric properties of a clinician-administered semi-structured interview to assess hypochondriacal thoughts and behaviours.

PubMed

Greeven, Anja; Spinhoven, Philip; van Balkom, Anton J L M

2009-01-01

This study investigated the psychometric properties of the first clinician-administered semi-structured interview for assessing the severity of hypochondriacal symptoms. The Hypochondriasis Yale-Brown Obsessive-Compulsive Scale (H-YBOCS) consisted of three a priori dimensions: hypochondriacal obsessions, compulsions and avoidance. The 16-item interview was conducted with 112 participants with Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, hypochondriasis. We analysed factor analytic structure, reliability, construct validity and sensitivity to change. Factor analysis supported a three-factor model similar to the a priori dimensions. Internal consistency ranged from satisfactory to good. Inter-rater reliability was excellent. The construct validity was low to moderate. The H-YBOCS was sensitive for measuring changes in symptom severity. The H-YBOCS is a (factorially) valid and coherent interview with a high level of agreement across different raters. The relatively low discriminant validity could be due to co-morbid anxiety and depressive disorders. Overall, the H-YBOCS seems to be a promising contribution to the assessment of hypochondriasis. *The hypochondriasis Y-BOCS is a feasible clinician rated interview to assess the severity of hypochondriacal complaints.

The Clinical Interview Schedule-Revised (CIS-R)-Malay Version, Clinical Validation.

PubMed

Subramaniam, Kavitha; Krishnaswamy, Saroja; Jemain, Abdul Aziz; Hamid, Abdul; Patel, Vikram

2006-01-01

Use of instruments or questionnaires in different cultural settings without proper validation can result in inaccurate results. Issues like reliability, validity, feasibility and acceptability should be considered in the use of an instrument. The study aims to determine the usefulness of the CIS-R Malay version in detecting common mental health problems specifically to establish the validity. The CIS-R instrument (PROQSY* format) was translated through the back translation process into Malay. Inter rater reliability was established for raters who were medical students. Cases and controls for the study were psychiatric in patients, out patient and relatives or friends accompanying the patients to the clinic or visiting the inpatients. The Malay version of CIS-R was administered to all cases and controls. All cases and controls involved in the study were rated by psychiatrists for psychiatric morbidity using the SCID as a guideline. Specificity and sensitivity of the CIS-R to the assessment by the psychiatrist were determined. The Malay version of CIS-R showed 100% sensitivity and 96.15% specificity at a cut off score of 9. The CIS-R can be a useful instrument for clinical and research use in the Malaysian population for diagnosing common mental disorders like depression and anxiety.

Assessment of nuclear reactor concepts for low power space applications

NASA Technical Reports Server (NTRS)

Klein, Andrew C.; Gedeon, Stephen R.; Morey, Dennis C.

1988-01-01

The results of a preliminary small reactor concepts feasibility and safety evaluation designed to provide a first order validation of the nuclear feasibility and safety of six small reactor concepts are given. These small reactor concepts have potential space applications for missions in the 1 to 20 kWe power output range. It was concluded that low power concepts are available from the U.S. nuclear industry that have the potential for meeting both the operational and launch safety space mission requirements. However, each design has its uncertainties, and further work is required. The reactor concepts must be mated to a power conversion technology that can offer safe and reliable operation.

Measuring decision quality: psychometric evaluation of a new instrument for breast cancer chemotherapy.

PubMed

Lee, Clara N; Wetschler, Matthew H; Chang, Yuchiao; Belkora, Jeffrey K; Moy, Beverly; Partridge, Ann; Sepucha, Karen R

2014-08-20

Women diagnosed with early stage (I or II) breast cancer face a highly challenging decision - whether or not to undergo adjuvant chemotherapy. We developed a decision quality instrument for chemotherapy for early stage breast cancer and sought to evaluate its performance. Cross-sectional, mailed survey of recent breast cancer survivors, providers, and healthy controls and a retest survey of survivors. The decision quality instrument includes questions on knowledge and personal goals. It results in a knowledge score and concordance score, which reflects the percentage of patients who received treatments that match their goals. Hypotheses related to acceptability, feasibility, validity, and reliability of the survey instrument were examined. Responses were received from 352 patients, 89 providers and 35 healthy controls. The decision quality instrument was feasible to implement with few missing data. The knowledge scores had good retest reliability (intraclass correlation coefficient (ICC) =0.75). Knowledge scores discriminated between providers and patients (mean difference 31.1%, 95% CI 26.9, 35.3) and between patients and healthy controls (mean difference 11.2, 95% CI 5.4, 17.1). Most providers reported that the knowledge items covered essential content. Two of the five goal items had a ceiling effect, and one goal had low content validity. The goal items had moderate retest reliability (ICC's 0.57 to 0.78). In the multivariable model of treatment, none of the patient goals was associated with receipt of chemotherapy. Age and hormone receptor status were the only variables independently associated with chemotherapy. Most patients (77.6%) had treatment concordant with that predicted by the model. Patients who had concordant treatment had similar levels of confidence and regret as those who did not. The Decision Quality Instrument is a reliable and valid measure of patient knowledge about chemotherapy, but its ability to measure concordance with patient goals is limited. In this sample, patient goals were not associated with treatment, and most patients reported they were not asked their preference, suggesting that goals were not adequately considered in decision making.

The ECOS-16 questionnaire for the evaluation of health related quality of life in post-menopausal women with osteoporosis

PubMed Central

Badia, Xavier; Díez-Pérez, Adolfo; Lahoz, Raquel; Lizán, Luis; Nogués, Xavier; Iborra, Jordi

2004-01-01

Background The aim of this study is to validate the questionnaire ECOS-16 (Assessment of health related quality of life in osteoporosis) for the evaluation of health related quality of life (HRQoL) in post-menopausal women with osteoporosis. Methods An observational, prospective and multi-centre study was carried out among post-menopausal women with osteoporosis in primary care centres and hospital outpatient clinics. All patients attended 2 visits: at baseline and at 6 months. In addition, the subgroup of outpatients attended another visit a month after the baseline to assess the test-retest reliability. The psychometric properties of the questionnaire were evaluated in terms of feasibility, validity (content validity and construct validity) and internal consistency in baseline, and in terms of test-retest reliability and responsiveness to change in visit at month and visit at 6 months, respectively. In all visits, ECOS-16, EUROQoL-5D (EQ-5D) and four 7-point items about health status (general health status, back pain, limitation in daily activities and emotional status) were administered, whereas only outpatients were given MINI-OQLQ (Mini Osteoporosis Quality of Life Questionnaire), besides all clinical variables; and sociodemographic variables at baseline. Results 316 women were consecutively included, 212 from primary care centres and 104 from hospital outpatient clinics. Feasibility: 94.3% of patients answered all items of the questionnaire. The mean administration time was 12.3 minutes. Validity: factor analysis suggested that the questionnaire was unidimensional. In the multivariate analysis, patients with vertebral fractures, co-morbidity and a lower education level showed to have worse HRQoL. Moderate to high correlations were found between the ECOS-16 score and the other health status questionnaires (0.47–0.82). Reliability: internal consistency (Cronbach's α) was 0.92 and test-retest reliability (ICC) was 0.80. Responsiveness to change: ECOS-16 scores increased according to change perceived by the patient, as well as the effect size (ranges between 1.35 to 0.43), the greater the perception of change in patients' general health status, the greater the changes in patients' scores. The Minimal Clinically Important Difference (MCID) suggested a change of 0.5 points in the ECOS-16 score, representing the least improvement in general health status due to their osteoporosis: "slightly better". Conclusion ECOS-16 has been proven preliminarily to have good psychometric properties, so that it can be potentially a useful tool to evaluate HRQoL of post-menopausal women with osteoporosis in research and routine clinical practice. PMID:15291959

Development and psychometric evaluation of the Undergraduate Clinical Education Environment Measure (UCEEM).

PubMed

Strand, Pia; Sjöborg, Karolina; Stalmeijer, Renée; Wichmann-Hansen, Gitte; Jakobsson, Ulf; Edgren, Gudrun

2013-12-01

There is a paucity of instruments designed to evaluate the multiple dimensions of the workplace as an educational environment for undergraduate medical students. The aim was to develop and psychometrically evaluate an instrument to measure how undergraduate medical students perceive the clinical workplace environment, based on workplace learning theories and empirical findings. Development of the instrument relied on established standards including theoretical and empirical grounding, systematic item development and expert review at various stages to ensure content validity. Qualitative and quantitative methods were employed using a series of steps from conceptualization through psychometric analysis of scores in a Swedish medical student population. The final result was a 25-item instrument with two overarching dimensions, experiential learning and social participation, and four subscales that coincided well with theory and empirical findings: Opportunities to learn in and through work & quality of supervision; Preparedness for student entry; Workplace interaction patterns & student inclusion; and Equal treatment. Evidence from various sources supported content validity, construct validity and reliability of the instrument. The Undergraduate Clinical Education Environment Measure represents a valid, reliable and feasible multidimensional instrument for evaluation of the clinical workplace as a learning environment for undergraduate medical students. Further validation in different populations using various psychometric methods is needed.

Adapting Scott and Bruce's General Decision-Making Style Inventory to Patient Decision Making in Provider Choice.

PubMed

Fischer, Sophia; Soyez, Katja; Gurtner, Sebastian

2015-05-01

Research testing the concept of decision-making styles in specific contexts such as health care-related choices is missing. Therefore, we examine the contextuality of Scott and Bruce's (1995) General Decision-Making Style Inventory with respect to patient choice situations. Scott and Bruce's scale was adapted for use as a patient decision-making style inventory. In total, 388 German patients who underwent elective joint surgery responded to a questionnaire about their provider choice. Confirmatory factor analyses within 2 independent samples assessed factorial structure, reliability, and validity of the scale. The final 4-dimensional, 13-item patient decision-making style inventory showed satisfactory psychometric properties. Data analyses supported reliability and construct validity. Besides the intuitive, dependent, and avoidant style, a new subdimension, called "comparative" decision-making style, emerged that originated from the rational dimension of the general model. This research provides evidence for the contextuality of decision-making style to specific choice situations. Using a limited set of indicators, this report proposes the patient decision-making style inventory as valid and feasible tool to assess patients' decision propensities. © The Author(s) 2015.

Development and pilot test of a new set of good practice indicators for chronic non-malignant pain management.

PubMed

Saturno, Pedro Jesus; Angel-García, Daniel; Martínez-Nicolás, Ismael; López Soriano, Francisco; Escolar Reina, Maria Pilar; Guerrero Díaz, María Beatriz; Ros Martínez, María Encarnación; Medina Mirapeix, Francesc; Saturno Marcos, Mayo

2018-06-08

This study was designed to address the current relative void of valid measures by developing evidence-based quality indicators for pain management of chronic non-malignant pain. We performed a 10-year literature search to identify guidelines and review articles on chronic pain management to identify evidence-based recommendations for the different conditions associated to chronic pain. A complementary search of indicators and indicator-related articles was also performed. Then, we built new indicators or adapted existing ones to cover all the evidence-based recommendations we found. The resulting set was pilot-tested for feasibility, reliability (kappa) and usefulness to identify quality problems, using the Lot Quality Acceptance method, α≤0.05 y β≤0.01, for 75% (40% threshold) and 95% (70% threshold) compliance standards, and estimates with binomial exact 95% confidence intervals. The study reviews clinical records from a primary-care centre, a medium-size hospital (250 beds) and a large hospital (500 beds). Forty-six indicators were developed (six general and forty condition-specific). Thirty-three were feasible in primary care and/or hospitals. Feasible indicators were also reliable (most kappa>0.7). Regarding compliance, four quality indicators obtained compliance levels over 60%, addressing pharmacological treatment, multimodal approach and appropriate use of neuro-image tests; while sixteen obtained compliance scores under 15% (six with 0% compliance). The created set has tested to be feasible, reliable, and useful, with the capacity to serve as the baseline for developing the necessary strategies to improve the management of chronic non-malignant pain, by monitoring and evaluating quality of care. This article is protected by copyright. All rights reserved. This article is protected by copyright. All rights reserved.

Measuring functional, interactive and critical health literacy of Chinese secondary school students: reliable, valid and feasible?

PubMed

Guo, Shuaijun; Davis, Elise; Yu, Xiaoming; Naccarella, Lucio; Armstrong, Rebecca; Abel, Thomas; Browne, Geoffrey; Shi, Yanqin

2018-04-01

Health literacy is an increasingly important topic in the global context. In mainland China, health literacy measures mainly focus on health knowledge and practices or on the functional domain for adolescents. However, little is known about interactive and critical domains. This study aimed to adopt a skills-based and three-domain (functional, interactive and critical) instrument to measure health literacy in Chinese adolescents and to examine the status and determinants of each domain. Using a systematic review, the eight-item Health Literacy Assessment Tool (HLAT-8) was selected and translated from English to Chinese (c-HLAT-8). Following the translation process, a cross-sectional study was conducted in four secondary schools in Beijing, China. A total of 650 students in Years 7-9 were recruited to complete a self-administered questionnaire that assessed socio-demographics, self-efficacy, social support, school environment, community environment and health literacy. Results showed that the c-HLAT-8 had satisfactory reliability (Cronbach's α = 0.79; intra-class correlation coefficient = 0.72) and strong validity (translation validity index (TVI) ≥0.95; χ 2 / df = 3.388, p < 0.001; comparative fit index = 0.975, Tucker and Lewis's index of fit = 0.945, normed fit index = 0.965, root mean error of approximation = 0.061; scores on the c-HLAT-8 were moderately correlated with the Health Literacy Study-Taiwan, but weakly with the Newest Vital Sign). Chinese students had an average score of 26.37 (±5.89) for the c-HLAT-8. When the determinants of each domain of health literacy were examined, social support was the strongest predictor of interactive and critical health literacy. On the contrary, self-efficacy and school environment played more dominant roles in predicting functional health literacy. The c-HLAT-8 was demonstrated to be a reliable, valid and feasible instrument for measuring functional, interactive and critical health literacy among Chinese students. The current findings indicate that increasing self-efficacy, social support and creating supportive environments are important for promoting health literacy in secondary school settings in China.

Development of an objective assessment tool for total laparoscopic hysterectomy: A Delphi method among experts and evaluation on a virtual reality simulator.

PubMed

Knight, Sophie; Aggarwal, Rajesh; Agostini, Aubert; Loundou, Anderson; Berdah, Stéphane; Crochet, Patrice

2018-01-01

Total Laparoscopic hysterectomy (LH) requires an advanced level of operative skills and training. The aim of this study was to develop an objective scale specific for the assessment of technical skills for LH (H-OSATS) and to demonstrate feasibility of use and validity in a virtual reality setting. The scale was developed using a hierarchical task analysis and a panel of international experts. A Delphi method obtained consensus among experts on relevant steps that should be included into the H-OSATS scale for assessment of operative performances. Feasibility of use and validity of the scale were evaluated by reviewing video recordings of LH performed on a virtual reality laparoscopic simulator. Three groups of operators of different levels of experience were assessed in a Marseille teaching hospital (10 novices, 8 intermediates and 8 experienced surgeons). Correlations with scores obtained using a recognised generic global rating tool (OSATS) were calculated. A total of 76 discrete steps were identified by the hierarchical task analysis. 14 experts completed the two rounds of the Delphi questionnaire. 64 steps reached consensus and were integrated in the scale. During the validation process, median time to rate each video recording was 25 minutes. There was a significant difference between the novice, intermediate and experienced group for total H-OSATS scores (133, 155.9 and 178.25 respectively; p = 0.002). H-OSATS scale demonstrated high inter-rater reliability (intraclass correlation coefficient [ICC] = 0.930; p<0.001) and test retest reliability (ICC = 0.877; p<0.001). High correlations were found between total H-OSATS scores and OSATS scores (rho = 0.928; p<0.001). The H-OSATS scale displayed evidence of validity for assessment of technical performances for LH performed on a virtual reality simulator. The implementation of this scale is expected to facilitate deliberate practice. Next steps should focus on evaluating the validity of the scale in the operating room.

Measurement properties and feasibility of clinical tests to assess sit-to-stand/stand-to-sit tasks in subjects with neurological disease: a systematic review

PubMed Central

Silva, Paula F. S.; Quintino, Ludmylla F.; Franco, Juliane; Faria, Christina D. C. M.

2014-01-01

Background Subjects with neurological disease (ND) usually show impaired performance during sit-to-stand and stand-to-sit tasks, with a consequent reduction in their mobility levels. Objective To determine the measurement properties and feasibility previously investigated for clinical tests that evaluate sit-to-stand and stand-to-sit in subjects with ND. Method A systematic literature review following the PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) protocol was performed. Systematic literature searches of databases (MEDLINE/SCIELO/LILACS/PEDro) were performed to identify relevant studies. In all studies, the following inclusion criteria were assessed: investigation of any measurement property or the feasibility of clinical tests that evaluate sit-to-stand and stand-to-sit tasks in subjects with ND published in any language through December 2012. The COSMIN checklist was used to evaluate the methodological quality of the included studies. Results Eleven studies were included. The measurement properties/feasibility were most commonly investigated for the five-repetition sit-to-stand test, which showed good test-retest reliability (Intraclass Correlation Coefficient:ICC=0.94-0.99) for subjects with stroke, cerebral palsy and dementia. The ICC values were higher for this test than for the number of repetitions in the 30-s test. The five-repetition sit-to-stand test also showed good inter/intra-rater reliabilities (ICC=0.97-0.99) for stroke and inter-rater reliability (ICC=0.99) for subjects with Parkinson disease and incomplete spinal cord injury. For this test, the criterion-related validity for subjects with stroke, cerebral palsy and incomplete spinal cord injury was, in general, moderate (correlation=0.40-0.77), and the feasibility and safety were good for subjects with Alzheimer's disease. Conclusions The five-repetition sit-to-stand test was used more often in subjects with ND, and most of the measurement properties were investigated and showed adequate results. PMID:24839043

Coronary flow velocity reserve by echocardiography: feasibility, reproducibility and agreement with PET in overweight and obese patients with stable and revascularized coronary artery disease.

PubMed

Olsen, Rasmus Huan; Pedersen, Lene Rørholm; Snoer, Martin; Christensen, Thomas Emil; Ghotbi, Adam Ali; Hasbak, Philip; Kjaer, Andreas; Haugaard, Steen B; Prescott, Eva

2016-06-07

Coronary flow velocity reserve (CFVR) measured by transthoracic Doppler echocardiography of the LAD is used to assess microvascular function but validation studies in clinical settings are lacking. We aimed to assess feasibility, reproducibility and agreement with myocardial flow reserve (MFR) measured by PET in overweight and obese patients. Participants with revascularized coronary artery disease were examined by CFVR. Subgroups were examined by repeated CFVR (reproducibility) or Rubidium-82-PET (agreement). To account for time variation, results were computed for scans performed within a week (1-week) and for all scans regardless of time gap (total) and to account for scar tissue for patients with and without previous myocardial infarction (MI). Eighty-six patients with median BMI 30.9 (IQR 29.4-32.9) kg × m(-2) and CFVR 2.29 (1.90-2.63) were included. CFVR was feasible in 83 (97 %) using a contrast agent in 14 %. For reproducibility overall (n = 21) limits of agreement (LOA) were (-0.75;0.71), within-subjects coefficient of variation (CV) 11 %, and reliability 0.84. For reproducibility within 1-week (n = 13) LOA were (-0.33;0.25), within-subjects CV 5 %, and reliability 0.97. Agreement with MFR of the LAD territory (n = 35) was without significant bias and overall LOA were (-1.40;1.46). Agreement was best for examinations performed within 1-week of participants without MI of the LAD-territory (n = 12); LOA = (-0.68;0.88). CFVR was highly feasible with a good reproducibility on par with other contemporary measures applied in cardiology. Agreement with MFR was acceptable, though discrepancy related to prior MI has to be considered. CFVR of LAD is a valid tool in overweight and obese patients.

Developing a Method to Test the Validity of 24 Hour Time Use Diaries Using Wearable Cameras: A Feasibility Pilot

PubMed Central

Kelly, Paul; Thomas, Emma; Doherty, Aiden; Harms, Teresa; Burke, Órlaith; Gershuny, Jonathan; Foster, Charlie

2015-01-01

Self-report time use diaries collect a continuous sequenced record of daily activities but the validity of the data they produce is uncertain. This study tests the feasibility of using wearable cameras to generate, through image prompted interview, reconstructed 'near-objective' data to assess their validity. 16 volunteers completed the Harmonised European Time Use Survey (HETUS) diary and used an Autographer wearable camera (recording images at approximately 15 second intervals) for the waking hours of the same 24-hour period. Participants then completed an interview in which visual images were used as prompts to reconstruct a record of activities for comparison with the diary record. 14 participants complied with the full collection protocol. We compared time use and number of discrete activities from the diary and camera records (using 10 classifications of activity). In terms of aggregate totals of daily time use we found no significant difference between the diary and camera data. In terms of number of discrete activities, participants reported a mean of 19.2 activities per day in the diaries, while image prompted interviews revealed 41.1 activities per day. The visualisations of the individual activity sequences reveal some potentially important differences between the two record types, which will be explored at the next project stage. This study demonstrates the feasibility of using wearable cameras to reconstruct time use through image prompted interview in order to test the concurrent validity of 24-hour activity time-use budgets. In future we need a suitably powered study to assess the validity and reliability of 24-hour time use diaries. PMID:26633807

Validation of an adapted arabic version of fibromyalgia syndrome impact questionnaire.

PubMed

El-Naby, Mai Abd; Hefny, Mohamed Ahmed; Fahim, Ayman Ekram; Awadalla, Magdy Ahmed

2013-10-01

Fibromyalgia (FM) is the most common chronic pain syndrome encountered in medical practice, affecting females more than males, and the estimated prevalence of FM in Egypt is 1.3 %. The aim was to translate and adapt the Fibromyalgia Impact Questionnaire (FIQ) into Arabic and assess reliability and validity. The Arabic version of Fibromyalgia Impact Questionnaire (FIQ-A) was adapted following the forward/backward translation approach. Fifty-one female patients with FM were studied to assess psychometric properties of the FIQ-A. Reliability was analyzed by the correlation coefficient between test and retest. Internal consistency was checked by the Cronbach's alpha coefficient. Construct validity was assessed comparing FIQ-A with Health Assessment Questionnaire (HAQ), Health Assessment Questionnaire of Fibromyalgia (FHAQ), The Medical Outcome Survey Short-Form-36 (SF-36), and the Total Visual Analog Scale (TVAS) for FM symptom, and feasibility was assessed by the time taken in completing the FIQ-A and the proportion of patients completed the questionnaire. Patients studied were 33.2 ± 9.8 years old. Translation was concordant. Adaptation affected 4 sub-items of physical function. Test-retest correlation coefficient was 0.89 for total FIQ-A and Cronbach's alpha was 0.76. Excellent to good statistically significant correlations (p < 0.05) were found between the FIQ-A items and HAQ, FHAQ, and SF-36. The FIQ-A is a reliable, valid for measuring health status and physical function in Arabic-speaking FM patients.

Developing Capture Mechanisms and High-Fidelity Dynamic Models for the MXER Tether System

NASA Technical Reports Server (NTRS)

Canfield, Steven L.

2007-01-01

A team consisting of collaborators from Tennessee Technological University (TTU), Marshall Space Flight Center, BD Systems, and the University of Delaware (herein called the TTU team) conducted specific research and development activities in MXER tether systems during the base period of May 15, 2004 through September 30, 2006 under contract number NNM04AB13C. The team addressed two primary topics related to the MXER tether system: 1) Development of validated high-fidelity dynamic models of an elastic rotating tether and 2) development of feasible mechanisms to enable reliable rendezvous and capture. This contractor report will describe in detail the activities that were performed during the base period of this cycle-2 MXER tether activity and will summarize the results of this funded activity. The primary deliverables of this project were the quad trap, a robust capture mechanism proposed, developed, tested, and demonstrated with a high degree of feasibility and the detailed development of a validated high-fidelity elastic tether dynamic model provided through multiple formulations.

The Assessment of Military Multitasking Performance: Validation of a Dual-Task and Multitask Protocol

DTIC Science & Technology

2014-08-01

Revisions based on IRR findings and rater comments Charge of Quarters (CQ) Duty Requires the subject to organize and implement a plan in order to...to evaluate test burden) and malingering are planned . Where feasible, test-retest reliability for several of the tasks is being assessed during...in either a shopping mall or hospital lobby setting (Alderman, Burgess,Knight,&Henman, 2003;Cuberos- Urbano et al., 2013; Dawson et al., 2009

Methods to elicit probability distributions from experts: a systematic review of reported practice in health technology assessment.

PubMed

Grigore, Bogdan; Peters, Jaime; Hyde, Christopher; Stein, Ken

2013-11-01

Elicitation is a technique that can be used to obtain probability distribution from experts about unknown quantities. We conducted a methodology review of reports where probability distributions had been elicited from experts to be used in model-based health technology assessments. Databases including MEDLINE, EMBASE and the CRD database were searched from inception to April 2013. Reference lists were checked and citation mapping was also used. Studies describing their approach to the elicitation of probability distributions were included. Data was abstracted on pre-defined aspects of the elicitation technique. Reports were critically appraised on their consideration of the validity, reliability and feasibility of the elicitation exercise. Fourteen articles were included. Across these studies, the most marked features were heterogeneity in elicitation approach and failure to report key aspects of the elicitation method. The most frequently used approaches to elicitation were the histogram technique and the bisection method. Only three papers explicitly considered the validity, reliability and feasibility of the elicitation exercises. Judged by the studies identified in the review, reports of expert elicitation are insufficient in detail and this impacts on the perceived usability of expert-elicited probability distributions. In this context, the wider credibility of elicitation will only be improved by better reporting and greater standardisation of approach. Until then, the advantage of eliciting probability distributions from experts may be lost.

Measuring health-related quality of life in patients with mild to moderate eczema and psoriasis: clinical validity, reliability and sensitivity to change of the DLQI. The Cavide Research Group.

PubMed

Badia, X; Mascaró, J M; Lozano, R

1999-10-01

The aim of this study was to assess the feasibility, validity, reliability and sensitivity to change of a Spanish version of the Dermatology Life Quality Index (DLQI) in patients with mild to moderate eczema and psoriasis who were treated with topical corticosteroids. The final study sample comprised 237 patients (48% eczema). Discriminant validity was tested by comparing patients' scores with those of a random sample of the general population (n = 100), and convergent validity by analysing correlations between DLQI scores, measures of clinical severity, and domain scores on the Nottingham Health Profile (NHP). Internal consistency and test-retest reliability were tested in clinically stable patients (n = 94), and responsiveness in a clinically unstable group (n = 143) initiating treatment with topical corticosteroids. Patient scores were significantly higher than general population scores (4.3 vs. 0. 27, P < 0.001). Correlations with NHP domains ranged from 0.12 to 0. 32, and there was significant correlation with clinical measures (r = 0.26, P < 0.001). Reliability was good (Cronbach's alpha = 0.83; intraclass correlation coefficient = 0.88), and the instrument proved responsive to change (effect size for the total group of de novo patients = 0.70), though the great majority of changes occurred in items 1 and 2. The NHP Emotional Reactions and Mobility domains were more responsive than some DLQI domains. In clinical trials of treatments for mild to moderate eczema and psoriasis, it is likely that only items 1 and 2 of the DLQI will be needed, and it is probably advisable to include generic instruments alongside the DLQI.

Convergent Validity of Preschool Children's Television Viewing Measures among Low-Income Latino Families: A Cross-Sectional Study

PubMed Central

McLeod, Jessica; Chen, Tzu-An; Nicklas, Theresa A.; Baranowski, Tom

2013-01-01

Abstract Background Television viewing is an important modifiable risk factor for childhood obesity. However, valid methods for measuring children's TV viewing are sparse and few studies have included Latinos, a population disproportionately affected by obesity. The goal of this study was to test the reliability and convergent validity of four TV viewing measures among low-income Latino preschool children in the United States. Methods Latino children (n=96) ages 3–5 years old were recruited from four Head Start centers in Houston, Texas (January, 2009, to June, 2010). TV viewing was measured concurrently over 7 days by four methods: (1) TV diaries (parent reported), (2) sedentary time (accelerometry), (3) TV Allowance (an electronic TV power meter), and (4) Ecological Momentary Assessment (EMA) on personal digital assistants (parent reported). This 7-day procedure was repeated 3–4 weeks later. Test–retest reliability was determined by intraclass correlations (ICC). Spearman correlations (due to nonnormal distributions) were used to determine convergent validity compared to the TV diary. Results The TV diary had the highest test–retest reliability (ICC=0.82, p<0.001), followed by the TV Allowance (ICC=0.69, p<0.001), EMA (ICC=0.46, p<0.001), and accelerometry (ICC=0.36–0.38, p<0.01). The TV Allowance (r=0.45–0.55, p<0.001) and EMA (r=0.47–0.51, p<0.001) methods were significantly correlated with TV diaries. Accelerometer-determined sedentary minutes were not correlated with TV diaries. The TV Allowance and EMA methods were significantly correlated with each other (r=0.48–0.53, p<0.001). Conclusions The TV diary is feasible and is the most reliable method for measuring US Latino preschool children's TV viewing. PMID:23270534

Potential reliability and validity of a modified version of the Unified Parkinson’s Disease Rating Scale that could be administered remotely

PubMed Central

Abdolahi, Amir; Scoglio, Nicholas; Killoran, Annie; Dorsey, Ray; Biglan, Kevin M.

2013-01-01

Background By permitting remote assessments of patients and research participants, telemedicine has the potential to reshape clinical care and clinical trials for Parkinson disease. While the majority of the motor Unified Parkinson’s Disease Rating Scale (UPDRS) items can be conducted visually, rigidity and retropulsion pull testing require hands-on assessment by the rater and are less feasible to perform remotely in patients' homes. Methods In a secondary data analysis of the Comparison of the Agonist pramipexole vs. Levodopa on Motor complications in Parkinson’s Disease (CALM-PD) study, a randomized clinical trial, we assessed the cross-sectional (baseline and 2 years) and longitudinal (change from baseline to 2 years) reliability of a modified motor UPDRS (removing rigidity and retropulsion items) compared to the standard motor UPDRS (all items) using intraclass correlation coefficients (ICC), stratified by treatment group. Internal consistency of the modified UPDRS (mUPDRS) was measured using Cronbach’s alpha, and concurrent validity was assessed using Pearson’s correlation coefficient (r) between the standard motor UPDRS and mUPDRS. Results The mUPDRS versus standard motor UPDRS is cross-sectionally (ICC ≥ 0.92) and longitudinally (ICC ≥ 0.92) reliable for both treatment groups. High internal consistencies were also observed (α ≥ 0.96). The mUPDRS had high concurrent validity with the standard UPDRS at both time points and longitudinally (r ≥ 0.93, p < 0.0001). Conclusions A modified version of the motor UPDRS without rigidity and retropulsion pull testing is reliable and valid and may lay the foundation for its use in remote assessments of patients and research participants. PMID:23102808

Potential reliability and validity of a modified version of the Unified Parkinson's Disease Rating Scale that could be administered remotely.

PubMed

Abdolahi, Amir; Scoglio, Nicholas; Killoran, Annie; Dorsey, E Ray; Biglan, Kevin M

2013-02-01

By permitting remote assessments of patients and research participants, telemedicine has the potential to reshape clinical care and clinical trials for Parkinson disease. While the majority of the motor Unified Parkinson's Disease Rating Scale (UPDRS) items can be conducted visually, rigidity and retropulsion pull testing require hands-on assessment by the rater and are less feasible to perform remotely in patients' homes. In a secondary data analysis of the Comparison of the Agonist pramipexole vs. Levodopa on Motor complications in Parkinson's Disease (CALM-PD) study, a randomized clinical trial, we assessed the cross-sectional (baseline and 2 years) and longitudinal (change from baseline to 2 years) reliability of a modified motor UPDRS (removing rigidity and retropulsion items) compared to the standard motor UPDRS (all items) using intraclass correlation coefficients (ICC), stratified by treatment group. Internal consistency of the modified UPDRS (mUPDRS) was measured using Cronbach's alpha, and concurrent validity was assessed using Pearson's correlation coefficient (r) between the standard motor UPDRS and mUPDRS. The mUPDRS versus standard motor UPDRS is cross-sectionally (ICC ≥ 0.92) and longitudinally (ICC ≥ 0.92) reliable for both treatment groups. High internal consistencies were also observed (α ≥ 0.96). The mUPDRS had high concurrent validity with the standard UPDRS at both time points and longitudinally (r ≥ 0.93, p < 0.0001). A modified version of the motor UPDRS without rigidity and retropulsion pull testing is reliable and valid and may lay the foundation for its use in remote assessments of patients and research participants. Copyright © 2012 Elsevier Ltd. All rights reserved.

Reliability and Concurrent Validity of the Narrow Path Walking Test in Persons With Multiple Sclerosis.

PubMed

Rosenblum, Uri; Melzer, Itshak

2017-01-01

About 90% of people with multiple sclerosis (PwMS) have gait instability and 50% fall. Reliable and clinically feasible methods of gait instability assessment are needed. The study investigated the reliability and validity of the Narrow Path Walking Test (NPWT) under single-task (ST) and dual-task (DT) conditions for PwMS. Thirty PwMS performed the NPWT on 2 different occasions, a week apart. Number of Steps, Trial Time, Trial Velocity, Step Length, Number of Step Errors, Number of Cognitive Task Errors, and Number of Balance Losses were measured. Intraclass correlation coefficients (ICC2,1) were calculated from the average values of NPWT parameters. Absolute reliability was quantified from standard error of measurement (SEM) and smallest real difference (SRD). Concurrent validity of NPWT with Functional Reach Test, Four Square Step Test (FSST), 12-item Multiple Sclerosis Walking Scale (MSWS-12), and 2 Minute Walking Test (2MWT) was determined using partial correlations. Intraclass correlation coefficients (ICCs) for most NPWT parameters during ST and DT ranged from 0.46-0.94 and 0.55-0.95, respectively. The highest relative reliability was found for Number of Step Errors (ICC = 0.94 and 0.93, for ST and DT, respectively) and Trial Velocity (ICC = 0.83 and 0.86, for ST and DT, respectively). Absolute reliability was high for Number of Step Errors in ST (SEM % = 19.53%) and DT (SEM % = 18.14%) and low for Trial Velocity in ST (SEM % = 6.88%) and DT (SEM % = 7.29%). Significant correlations for Number of Step Errors and Trial Velocity were found with FSST, MSWS-12, and 2MWT. In persons with PwMS performing the NPWT, Number of Step Errors and Trial Velocity were highly reliable parameters. Based on correlations with other measures of gait instability, Number of Step Errors was the most valid parameter of dynamic balance under the conditions of our test.Video Abstract available for more insights from the authors (see Supplemental Digital Content 1, available at: http://links.lww.com/JNPT/A159).

The Patient Assessment Questionnaire: initial validation of a measure of treatment effectiveness for patients with schizophrenia and schizoaffective disorder.

PubMed

Mojtabai, Ramin; Corey-Lisle, Patricia K; Ip, Edward Hak-Sing; Kopeykina, Irina; Haeri, Sophia; Cohen, Lisa Janet; Shumaker, Sally

2012-12-30

Investigation of patients' subjective perspective regarding the effectiveness - as opposed to efficacy - of antipsychotic medication has been hampered by a relative shortage of self-report measures of global clinical outcome. This paper presents data supporting the feasibility, inter-item consistency, and construct validity of the Patient Assessment Questionnaire (PAQ)-a self-report measure of psychiatric symptoms, medication side effects and general wellbeing, ultimately intended to assess effectiveness of interventions for schizophrenia-spectrum patients. The original 53-item instrument was developed by a multidisciplinary team which utilized brainstorming sessions for item generation and content analysis, patient focus groups, and expert panel reviews. This instrument and additional validation measures were administered, via Audio Computer-Assisted Self-Interviewing (ACASI), to 300 stable, medicated outpatients diagnosed with schizophrenia or schizoaffective disorder. Item elimination was based on psychometric properties and Item-Response Theory information functions and characteristic curves. Exploratory factor analysis of the resulting 40-item scale yielded a five factor solution. The five subscales (General Distress, Side Effects, Psychotic Symptoms, Cognitive Symptoms, Sleep) showed robust convergent (β's=0.34-0.75, average β=0.49) and discriminant validity. The PAQ demonstrates feasibility, reliability, and construct validity as a self-report measure of multiple domains pertinent to effectiveness. Future research needs to establish the PAQ's sensitivity to change. Copyright © 2012 Elsevier Ireland Ltd. All rights reserved.

Feasibility and Reliability of Physical Fitness Tests in Older Adults with Intellectual Disability: A Pilot Study

ERIC Educational Resources Information Center

Hilgenkamp, Thessa I. M.; van Wijck, Ruud; Evenhuis, Heleen M.

2012-01-01

Background: Physical fitness is relevant for wellbeing and health, but knowledge on the feasibility and reliability of instruments to measure physical fitness for older adults with intellectual disability is lacking. Methods: Feasibility and test-retest reliability of a physical fitness test battery (Box and Block Test, Response Time Test, walking…

Safety, reliability, and validity of a physiologic definition of bronchopulmonary dysplasia.

PubMed

Walsh, Michele C; Wilson-Costello, Deanna; Zadell, Arlene; Newman, Nancy; Fanaroff, Avroy

2003-09-01

Bronchopulmonary dysplasia (BPD) is the focus of many intervention trials, yet the outcome measure when based solely on oxygen administration may be confounded by differing criteria for oxygen administration between physicians. Thus, we wished to define BPD by a standardized oxygen saturation monitoring at 36 weeks corrected age, and compare this physiologic definition with the standard clinical definition of BPD based solely on oxygen administration. A total of 199 consecutive very low birthweight infants (VLBW, 501 to 1500 g birthweight) were assessed prospectively at 36+/-1 weeks corrected age. Neonates on positive pressure support or receiving >30% supplemental oxygen were assigned the outcome BPD. Those receiving < or =30% oxygen underwent a stepwise 2% reduction in supplemental oxygen to room air while under continuous observation and oxygen saturation monitoring. Outcomes of the test were "no BPD" (saturations > or =88% for 60 minutes) or "BPD" (saturation < 88%). At the conclusion of the test, all infants were returned to their baseline oxygen. Safety (apnea, bradycardia, increased oxygen use), inter-rater reliability, test-retest reliability, and validity of the physiologic definition vs the clinical definition were assessed. A total of 199 VLBW were assessed, of whom 45 (36%) were diagnosed with BPD by the clinical definition of oxygen use at 36 weeks corrected age. The physiologic definition identified 15 infants treated with oxygen who successfully passed the saturation monitoring test in room air. The physiologic definition diagnosed BPD in 30 (24%) of the cohort. All infants were safely studied. The test was highly reliable (inter-rater reliability, kappa=1.0; test-retest reliability, kappa=0.83) and highly correlated with discharge home in oxygen, length of hospital stay, and hospital readmissions in the first year of life. The physiologic definition of BPD is safe, feasible, reliable, and valid and improves the precision of the diagnosis of BPD. This may be of benefit in future multicenter clinical trials.

Prior Study of Cross-Cultural Validation of McGill Quality-of-Life Questionnaire in Mainland Mandarin Chinese Patients With Cancer.

PubMed

Hu, Liya; Li, Jingwen; Wang, Xu; Payne, Sheila; Chen, Yuan; Mei, Qi

2015-11-01

The validation of McGill quality-of-life questionnaire (MQOLQ) in mainland China, which had already been used in multicultural palliative care background including Hong Kong and Taiwan, remained unknown. Eligible patients completed the translated Chinese version of McGill questionnaires (MQOL-C), which had been examined before the study. Construct validity was preliminarily assessed through exploratory factor analysis extracting 4 factors that construct a new hypothesis model and then the original model was proved to be better confirmed by confirmatory factor analysis. Internal consistency of all the subscales was within 0.582 to 0.917. Furthermore, test-retest reliability ranged from 0.509 to 0.859, which was determined by Spearman rank correlation coefficient. Face validation and feasibility also confirm the good validity of MQOL-C. The MQOL-C has satisfied validation in mainland Chinese patients with cancer, although cultural difference should be considered while using it. © The Author(s) 2014.

Reliability and validity of Edinburgh visual gait score as an evaluation tool for children with cerebral palsy.

PubMed

Del Pilar Duque Orozco, Maria; Abousamra, Oussama; Church, Chris; Lennon, Nancy; Henley, John; Rogers, Kenneth J; Sees, Julieanne P; Connor, Justin; Miller, Freeman

2016-09-01

Assessment of gait abnormalities in cerebral palsy (CP) is challenging, and access to instrumented gait analysis is not always feasible. Therefore, many observational gait analysis scales have been devised. This study aimed to evaluate the interobserver reliability, intraobserver reliability, and validity of Edinburgh visual gait score (EVGS). Video of 30 children with spastic CP were reviewed by 7 raters (10 children each in GMFCS levels I, II, and III, age 6-12 years). Three observers had high level of experience in gait analysis (10+ years), two had medium level (2-5 years) and two had no previous experience (orthopedic fellows). Interobserver reliability was evaluated using percentage of complete agreement and kappa values. Criterion validity was evaluated by comparing EVGS scores with 3DGA data taken from the same video visit. Interobserver agreement was 60-90% and Kappa values were 0.18-0.85 for the 17 items in EVGS. Reliability was higher for distal segments (foot/ankle/knee 63-90%; trunk/pelvis/hip 60-76%), with greater experience (high 66-91%, medium 62-90%, no-experience 41-87%), with more EVGS practice (1st 10 videos 52-88%, last 10 videos 64-97%) and when used with higher functioning children (GMFCS I 65-96%, II 58-90%, III 35-65%). Intraobserver agreement was 64-92%. Agreement between EVGS and 3DGA was 52-73%. We believe that having EVGS as part of the standardized gait evaluation is helpful in optimizing the visual scoring. EVGS can be a supportive tool that adds quantitative data instead of only qualitative assessment to a video only gait evaluation. Copyright © 2016 Elsevier B.V. All rights reserved.

The development and psychometric testing of East Asian Acculturation Scale among Asian immigrant women in Taiwan.

PubMed

Kuo, Shu-Fen; Chang, Wen-Yin; Chang, Lu-I; Chou, Yu-Hua; Chen, Ching-Min

2013-01-01

This is a report of development and psychometric testing of the East Asian Acculturation Measure-Chinese version (EAAM-C) scale. An instrument validation design with a cross-sectional survey was conducted. The process was carried in two phases. In Phase 1, Barry's East Asian Acculturation Measure was translated and back translated to evaluate its content, face validity, and feasibility validity. In Phase 2, the 16-item EAAM-C was pilot-tested among 485 female immigrants for test-retest reliability, internal consistency, theoretically-supported construct validity and concurrent validity. The pilot work and the survey results indicated the tools possessed adequate content and face validity. The Cronbach's Alphas for the EAAM-C was 0.72, and 0.76-0.79 for its subscales, and the correlation of test-retest reliability (at 3 weeks) was 0.75. After dropping one item, four theoretically-supported factors which explained 61.82% of the variance were abstracted using exploratory factor analysis: assimilation, integration, separation, and marginalization. Based on the underlying four-factor theoretical structures of the EAAM, the confirmatory factor analysis of the EAAM-C was further examined. The analysis revealed that the four-factor model was an acceptable fit for the data which demonstrated adequate finding in its construct validity. These factors were inter-correlated, and showed statistically significant correlation with the Chinese Health Questionnaire, indicating adequate concurrent validity. The scale shows acceptable validity and consistency, and suggests that immigrant acculturation is a complex construct. This quick evaluation instrument can be applied to assess clients' acculturation and in further developing certain interventions to improve their health.

Validity, reliability and support for implementation of independence-scaled procedural assessment in laparoscopic surgery.

PubMed

Kramp, Kelvin H; van Det, Marc J; Veeger, Nic J G M; Pierie, Jean-Pierre E N

2016-06-01

There is no widely used method to evaluate procedure-specific laparoscopic skills. The first aim of this study was to develop a procedure-based assessment method. The second aim was to compare its validity, reliability and feasibility with currently available global rating scales (GRSs). An independence-scaled procedural assessment was created by linking the procedural key steps of the laparoscopic cholecystectomy to an independence scale. Subtitled and blinded videos of a novice, an intermediate and an almost competent trainee, were evaluated with GRSs (OSATS and GOALS) and the independence-scaled procedural assessment by seven surgeons, three senior trainees and six scrub nurses. Participants received a short introduction to the GRSs and independence-scaled procedural assessment before assessment. The validity was estimated with the Friedman and Wilcoxon test and the reliability with the intra-class correlation coefficient (ICC). A questionnaire was used to evaluate user opinion. Independence-scaled procedural assessment and GRS scores improved significantly with surgical experience (OSATS p = 0.001, GOALS p < 0.001, independence-scaled procedural assessment p < 0.001). The ICCs of the OSATS, GOALS and independence-scaled procedural assessment were 0.78, 0.74 and 0.84, respectively, among surgeons. The ICCs increased when the ratings of scrub nurses were added to those of the surgeons. The independence-scaled procedural assessment was not considered more of an administrative burden than the GRSs (p = 0.692). A procedural assessment created by combining procedural key steps to an independence scale is a valid, reliable and acceptable assessment instrument in surgery. In contrast to the GRSs, the reliability of the independence-scaled procedural assessment exceeded the threshold of 0.8, indicating that it can also be used for summative assessment. It furthermore seems that scrub nurses can assess the operative competence of surgical trainees.

Development and validation of The Personal Diabetes Questionnaire (PDQ): a measure of diabetes self-care behaviors, perceptions and barriers.

PubMed

Stetson, Barbara; Schlundt, David; Rothschild, Chelsea; Floyd, Jennifer E; Rogers, Whitney; Mokshagundam, Sri Prakash

2011-03-01

To develop and evaluate the validity and reliability of The Personal Diabetes Questionnaire (PDQ), a brief, yet comprehensive measure of diabetes self-care behaviors, perceptions and barriers. To examine individual items to provide descriptive and normative information and provide data on scale reliability and associations between PDQ scales and concurrently assessed HBA(1c) and BMI. Items were written to address nutritional management, medication utilization, blood glucose monitoring, and physical activity. The initial instrument was reviewed by multidisciplinary diabetes care providers and items subsequently revised until the measure provided complete coverage of the diabetes care domains using as few items as possible. The scoring scheme was generated rationally. Subjects were 790 adults (205 with type 1 and 585 with type 2 diabetes) who completed the PDQ while waiting for clinic appointments. Item completion rates were high, with few items skipped by participants. Subscales demonstrated good internal consistency (Cronbach α=.650-.834) and demonstrated significant associations with BMI (p ≤.001) and HbA(1c) (p ≤.001). The PDQ is a useful measure of diabetes self-care behaviors and related perceptions and barriers that is reliable and valid and feasible to administer in a clinic setting. This measure may be used to obtain data for assessing diabetes self-management and barriers and to guide patient care. Copyright © 2010 Elsevier Ireland Ltd. All rights reserved.

Validation of personal digital photography to assess dietary quality among people with intellectual disabilities.

PubMed

Elinder, L S; Brunosson, A; Bergström, H; Hagströmer, M; Patterson, E

2012-02-01

Dietary assessment is a challenge in general, and specifically in individuals with intellectual disabilities (ID). This study aimed to evaluate personal digital photography as a method of assessing different aspects of dietary quality in this target group. Eighteen adults with ID were recruited from community residences and activity centres in Stockholm County. Participants were instructed to photograph all foods and beverages consumed during 1 day, while observed. Photographs were coded by two raters. Observations and photographs of meal frequency, intake occasions of four specific food and beverage items, meal quality and dietary diversity were compared. Evaluation of inter-rater reliability and validity of the method was performed by intra-class correlation analysis. With reminders from staff, 85% of all observed eating or drinking occasions were photographed. The inter-rater reliability was excellent for all assessed variables (ICC ≥ 0.88), except for meal quality where ICC was 0.66. The correlations between items assessed in photos and observations were strong to almost perfect with ICC values ranging from 0.71 to 0.92 and all were statistically significant. Personal digital photography appears to be a feasible, reliable and valid method for assessing dietary quality in people with mild to moderate ID, who have daily staff support. © 2011 The Authors. Journal of Intellectual Disability Research © 2011 Blackwell Publishing Ltd.

Reliable and valid NEWS for Chinese seniors: measuring perceived neighborhood attributes related to walking.

PubMed

Cerin, Ester; Sit, Cindy Hp; Cheung, Man-Chin; Ho, Sai-Yin; Lee, Lok-Chun Janet; Chan, Wai-Man

2010-11-25

The effects of the built environment on walking in seniors have not been studied in an Asian context. To examine these effects, valid and reliable measures are needed. The aim of this study was to develop and validate a questionnaire of perceived neighborhood characteristics related to walking appropriate for Chinese seniors (Neighborhood Environment Walkability Scale for Chinese Seniors, NEWS-CS). It was based on the Neighborhood Environment Walkability Scale - Abbreviated (NEWS-A), a validated measure of perceived built environment developed in the USA for adults. A secondary study aim was to establish the generalizability of the NEWS-A to an Asian high-density urban context and a different age group. A multidisciplinary panel of experts adapted the original NEWS-A to reflect the built environment of Hong Kong and needs of seniors. The translated instrument was pre-tested on a sample of 50 Chinese-speaking senior residents (65+ years). The final version of the NEWS-CS was interviewer-administered to 484 seniors residing in four selected Hong Kong districts varying in walkability and socio-economic status. Ninety-two participants completed the questionnaire on two separate occasions, 2-3 weeks apart. Test-rest reliability indices were estimated for each item and subscale of the NEWS-CS. Confirmatory factor analysis was used to develop the measurement model of the NEWS-CS and cross-validate that of the NEWS-A. The final version of the NEWS-CS consisted of 14 subscales and four single items (76 items). Test-retest reliability was moderate to good (ICC > 50 or % agreement > 60) except for four items measuring distance to destinations. The originally-proposed measurement models of the NEWS-A and NEWS-CS required 2-3 theoretically-justifiable modifications to fit the data well. The NEWS-CS possesses sufficient levels of reliability and factorial validity to be used for measuring perceived neighborhood environment in Chinese seniors. Further work is needed to assess its construct validity and generalizability to other Asian locations. In general, the measurement model of the original NEWS-A was generalizable to this study context, supporting the feasibility of cross-country and age-group comparisons of the effect of the neighborhood environment on walking using the NEWS-A as a tool to measure the perceived built environment.

The Sickness Impact Profile as a measure of the health status of noncognitively impaired nursing home residents.

PubMed

Rothman, M L; Hedrick, S; Inui, T

1989-03-01

The Sickness Impact Profile (SIP) is a multidimensional, behaviorally based measure of the health status that has been successfully used in a wide range of applications. The characteristics of this measure have not been assessed with nursing home residents. The purpose of this study was to assess the feasibility, reliability (internal consistency), validity, and comprehensiveness of the SIP as a measure of the health status of a selected group of nursing home residents. One hundred sixty-eight veterans residing in community and VA nursing homes responded to a questionnaire consisting of the SIP, Index of Activities of Daily Living, Barthel Index, Life Satisfaction Index Z, and the Philadelphia Geriatric Center Morale Scale. In general, the respondents correctly interpreted instructions; reliability and validity were supported; and the SIP was found to provide a comprehensive assessment of physical function. Adding a measure of psychologic well-being to a study protocol involving this population may, however, provide additional useful information regarding this construct.

Cloud-based bioinformatics workflow platform for large-scale next-generation sequencing analyses

PubMed Central

Liu, Bo; Madduri, Ravi K; Sotomayor, Borja; Chard, Kyle; Lacinski, Lukasz; Dave, Utpal J; Li, Jianqiang; Liu, Chunchen; Foster, Ian T

2014-01-01

Due to the upcoming data deluge of genome data, the need for storing and processing large-scale genome data, easy access to biomedical analyses tools, efficient data sharing and retrieval has presented significant challenges. The variability in data volume results in variable computing and storage requirements, therefore biomedical researchers are pursuing more reliable, dynamic and convenient methods for conducting sequencing analyses. This paper proposes a Cloud-based bioinformatics workflow platform for large-scale next-generation sequencing analyses, which enables reliable and highly scalable execution of sequencing analyses workflows in a fully automated manner. Our platform extends the existing Galaxy workflow system by adding data management capabilities for transferring large quantities of data efficiently and reliably (via Globus Transfer), domain-specific analyses tools preconfigured for immediate use by researchers (via user-specific tools integration), automatic deployment on Cloud for on-demand resource allocation and pay-as-you-go pricing (via Globus Provision), a Cloud provisioning tool for auto-scaling (via HTCondor scheduler), and the support for validating the correctness of workflows (via semantic verification tools). Two bioinformatics workflow use cases as well as performance evaluation are presented to validate the feasibility of the proposed approach. PMID:24462600

Cloud-based bioinformatics workflow platform for large-scale next-generation sequencing analyses.

PubMed

Liu, Bo; Madduri, Ravi K; Sotomayor, Borja; Chard, Kyle; Lacinski, Lukasz; Dave, Utpal J; Li, Jianqiang; Liu, Chunchen; Foster, Ian T

2014-06-01

Due to the upcoming data deluge of genome data, the need for storing and processing large-scale genome data, easy access to biomedical analyses tools, efficient data sharing and retrieval has presented significant challenges. The variability in data volume results in variable computing and storage requirements, therefore biomedical researchers are pursuing more reliable, dynamic and convenient methods for conducting sequencing analyses. This paper proposes a Cloud-based bioinformatics workflow platform for large-scale next-generation sequencing analyses, which enables reliable and highly scalable execution of sequencing analyses workflows in a fully automated manner. Our platform extends the existing Galaxy workflow system by adding data management capabilities for transferring large quantities of data efficiently and reliably (via Globus Transfer), domain-specific analyses tools preconfigured for immediate use by researchers (via user-specific tools integration), automatic deployment on Cloud for on-demand resource allocation and pay-as-you-go pricing (via Globus Provision), a Cloud provisioning tool for auto-scaling (via HTCondor scheduler), and the support for validating the correctness of workflows (via semantic verification tools). Two bioinformatics workflow use cases as well as performance evaluation are presented to validate the feasibility of the proposed approach. Copyright © 2014 Elsevier Inc. All rights reserved.

Measurement of EMG activity with textile electrodes embedded into clothing.

PubMed

Finni, T; Hu, M; Kettunen, P; Vilavuo, T; Cheng, S

2007-11-01

Novel textile electrodes that can be embedded into sports clothing to measure averaged rectified electromyography (EMG) have been developed for easy use in field tests and in clinical settings. The purpose of this study was to evaluate the validity, reliability and feasibility of this new product to measure averaged rectified EMG. The validity was tested by comparing the signals from bipolar textile electrodes (42 cm(2)) and traditional bipolar surface electrodes (1.32 cm(2)) during bilateral isometric knee extension exercise with two electrode locations (A: both electrodes located in the same place, B: traditional electrodes placed on the individual muscles according to SENIAM, n=10 persons for each). Within-session repeatability (the coefficient of variation CV%, n=10) was calculated from five repetitions of 60% maximum voluntary contraction (MVC). The day-to-day repeatability (n=8) was assessed by measuring three different isometric force levels on five consecutive days. The feasibility of the textile electrodes in field conditions was assessed during a maximal treadmill test (n=28). Bland-Altman plots showed a good agreement within 2SD between the textile and traditional electrodes, demonstrating that the textile electrodes provide similar information on the EMG signal amplitude to the traditional electrodes. The within-session CV ranged from 13% to 21% in both the textile and traditional electrodes. The day-to-day CV was smaller, ranging from 4% to 11% for the textile electrodes. A similar relationship (r(2)=0.5) was found between muscle strength and the EMG of traditional and textile electrodes. The feasibility study showed that the textile electrode technique can potentially make EMG measurements very easy in field conditions. This study indicates that textile electrodes embedded into shorts is a valid and feasible method for assessing the average rectified value of EMG.

An instrument measuring prospective mathematics teacher self-regulated learning: validity and reliability

NASA Astrophysics Data System (ADS)

Nugroho, A. A.; Juniati, D.; Siswono, T. Y. E.

2018-03-01

Self Regulated Learning (SRL) is an individual's ability to achieve academic goals by controlling behavior, motivate yourself and use cognitive in learning, so it is important for a teacher especially teachers of mathematics related to the ability of management, design, implementation of learning and evaluation of learning outcomes. The purpose of the research is to develop an instrument to describe the SRL of a prospective mathematics teacher. Data were collected by (1) the study of the theory of SRL produced the indicator SRL used to design the questionnaire SRL; (2) analysis of the questionnaire SRL obtained from several References; and (3) development stage of the SRL questionnaire through validity test of content and empirical validation. The study involved 2 content experts in mathematics, 1 linguist, and 92 prospective mathematics teachers. The results of the research on content validity test based on Indonesian expert and 2 content experts indicate that the content can assess the indicator of the SRL and feasible to be used, in the test of legibility of two prospective mathematics teacher concluded that the instrument has a language that can be understood by the prospective teacher of mathematics and on empirical validation involving 92 prospective mathematics teacher generate data that of 65 statements there are 3 invalid statements. Reliability calculation shows high category that values 0,93. The conclusion is the SRL instrument developed for the prospective mathematics teacher.

Measuring pediatric hematology-oncology fellows' skills in humanism and professionalism: A novel assessment instrument.

PubMed

Kesselheim, Jennifer C; Agrawal, Anurag K; Bhatia, Nita; Cronin, Angel; Jubran, Rima; Kent, Paul; Kersun, Leslie; Rao, Amulya Nageswara; Rose, Melissa; Savelli, Stephanie; Sharma, Mukta; Shereck, Evan; Twist, Clare J; Wang, Michael

2017-05-01

Educators in pediatric hematology-oncology lack rigorously developed instruments to assess fellows' skills in humanism and professionalism. We developed a novel 15-item self-assessment instrument to address this gap in fellowship training. Fellows (N = 122) were asked to assess their skills in five domains: balancing competing demands of fellowship, caring for the dying patient, confronting depression and burnout, responding to challenging relationships with patients, and practicing humanistic medicine. An expert focus group predefined threshold scores on the instrument that could be used as a cutoff to identify fellows who need support. Reliability and feasibility were assessed and concurrent validity was measured using three established instruments: Maslach Burnout Inventory (MBI), Flourishing Scale (FS), and Jefferson Scale of Physician Empathy (JSPE). For 90 participating fellows (74%), the self-assessment proved feasible to administer and had high internal consistency reliability (Cronbach's α = 0.81). It was moderately correlated with the FS and MBI (Pearson's r = 0.41 and 0.4, respectively) and weakly correlated with the JSPE (Pearson's r = 0.15). Twenty-eight fellows (31%) were identified as needing support. The self-assessment had a sensitivity of 50% (95% confidence interval [CI]: 31-69) and a specificity of 77% (95% CI: 65-87) for identifying fellows who scored poorly on at least one of the three established scales. We developed a novel assessment instrument for use in pediatric fellowship training. The new scale proved feasible and demonstrated internal consistency reliability. Its moderate correlation with other established instruments shows that the novel assessment instrument provides unique, nonredundant information as compared to existing scales. © 2016 Wiley Periodicals, Inc.

Cross-cultural challenges for assessing medical professionalism among clerkship physicians in a Middle Eastern country (Bahrain): feasibility and psychometric properties of multisource feedback.

PubMed

Al Ansari, Ahmed; Al Khalifa, Khalid; Al Azzawi, Mohamed; Al Amer, Rashed; Al Sharqi, Dana; Al-Mansoor, Anwar; Munshi, Fadi M

2015-01-01

We aimed to design, implement, and evaluate the feasibility and reliability of a multisource feedback (MSF) system to assess interns in their clerkship year in the Middle Eastern culture, the Kingdom of Bahrain. The study was undertaken in the Bahrain Defense Force Hospital, a military teaching hospital in the Kingdom of Bahrain. A total of 21 interns (who represent the total population of the interns for the given year) were assessed in this study. All of the interns were rotating through our hospital during their year-long clerkship rotation. The study sample consisted of nine males and 12 females. Each participating intern was evaluated by three groups of raters, eight medical intern colleagues, eight senior medical colleagues, and eight coworkers from different departments. A total of 21 interns (nine males and 12 females) were assessed in this study. The total mean response rates were 62.3%. A factor analysis was conducted that found that the data on the questionnaire grouped into three factors that counted for 76.4% of the total variance. These three factors were labeled as professionalism, collaboration, and communication. Reliability analysis indicated that the full instrument scale had high internal consistency (Cronbach's α 0.98). The generalizability coefficients for the surveys were estimated to be 0.78. Based on our results and analysis, we conclude that the MSF tool we used on the interns rotating in their clerkship year within our Middle Eastern culture provides an effective method of evaluation because it offers a reliable, valid, and feasible process.

Feasibility of interactive biking exercise system for telemanagement in elderly.

PubMed

Finkelstein, Joseph; Jeong, In Cheol

2013-01-01

Inexpensive cycling equipment is widely available for home exercise however its use is hampered by lack of tools supporting real-time monitoring of cycling exercise in elderly and coordination with a clinical care team. To address these barriers, we developed a low-cost mobile system aimed at facilitating safe and effective home-based cycling exercise. The system used a miniature wireless 3-axis accelerometer that transmitted the cycling acceleration data to a tablet PC that was integrated with a multi-component disease management system. An exercise dashboard was presented to a patient allowing real-time graphical visualization of exercise progress. The system was programmed to alert patients when exercise intensity exceeded the levels recommended by the patient care providers and to exchange information with a central server. The feasibility of the system was assessed by testing the accuracy of cycling speed monitoring and reliability of alerts generated by the system. Our results demonstrated high validity of the system both for upper and lower extremity exercise monitoring as well as reliable data transmission between home unit and central server.

Adapting the Bayley Scales of infant and toddler development in Ethiopia: evaluation of reliability and validity

PubMed Central

Medhin, G.; Worku, B.; Tomlinson, M.; Alem, A.; Dewey, M.; Prince, M.

2016-01-01

Abstract Background There is a need for valid and reliable observational measures of early child development in low‐income and middle‐income country settings. Methods The aims of the study were to adapt the Bayley Scales of Infant Development (Bayley III) for a rural Ethiopian setting and evaluate reliability and validity. The study was carried out between January 2008 and January 2009 in the Butajira demographic surveillance site, south central Ethiopia. The Bayley III was adapted to be socioculturally appropriate for a rural Ethiopian context. Nurses and high school graduates were trained in administration of the measure for 10 days. Inter‐rater reliability was evaluated (n = 60). Content, construct and convergent validity was then examined on a population‐based cohort of children at the ages of 30 (n = 440) and 42 months (n = 456). Mokken scale analysis was used to determine the scalability of items in unidimensional, hierarchical sub‐scales. The mean score was compared by age of child and by stunting status (less than −2 z scores below the standard height‐for‐age). Results The intra‐class correlations between raters were above 0.90 for all sub‐scales of the child development measure. Some scale items were not contextually relevant and showed poor scalability. However, the majority of items scaled onto the existing sub‐scales of the international measure to form adequate‐to‐strong hierarchical scales with good internal consistency (Cronbach's α above 0.70 except for gross motor and expressive language sub‐scales). Item‐scale coefficients were good. The mean score of all sub‐scales was significantly higher in the older group of children (33.02 higher total score; P < 0.001) and in the children who were stunted (total Bayley score 2.58 (95% confidence interval 0.07 to 5.10) points lower at 30 months and 3.87 (1.94 to 5.81) points lower at 42 months. Conclusions An adapted version of an international, observational measure of child development was found to be reliable, valid and feasible in a rural Ethiopian setting. PMID:27381579

Development and validation of the Vellore Occupational Therapy Evaluation Scale to assess functioning in people with mental illness.

PubMed

Samuel, Reema; Russell, Paul Ss; Paraseth, Tapan Kumar; Ernest, Sharmila; Jacob, K S

2016-08-26

Available occupational therapy assessment scales focus on specific areas of functioning. There is a need for comprehensive evaluation of diverse aspects of functioning in people with mental illness. To develop a comprehensive assessment scale to evaluate diverse aspects of functioning among people with mental illness and to assess its validity and reliability. Available instruments, which evaluate diverse aspects of functioning in people with mental illness, were retrieved. Relevant items, which evaluate specific functions, were selected by a committee of mental health experts and combined to form a comprehensive instrument. Face and content validity and feasibility were assessed and the new instrument was piloted among 60 patients with mental illness. The final version of the instrument was employed in 151 consecutive clients, between 18 and 60 years of age, who were also assessed using Global Assessment of Functioning (GAF), Occupational Therapy Task Observation Scale (OTTOS), Social Functioning Questionnaire (SFQ), Rosenberg Self Esteem Scale (RSES) and Pai and Kapur Family Burden Interview Schedule (FBIS) by two therapists. The inter-rater reliability and test-retest reliability of the new instrument (Vellore Occupational Therapy Evaluation Scale (VOTES)) were also evaluated. The new scale had good internal consistency (Cronbach's alpha = .817), inter-rater reliability .928 (.877-.958) and test-retest reliability .928 (.868-.961). The correlation between the general behaviour domain (Pearson's Correlation Coefficient [PCC] = -.763, p = .000), task behaviour (PCC = -.829, p = .000), social skills (PCC = -.351, p = .000), intrapersonal skills (PCC = -.208, p = .010), instrumental activities of daily living (IADL) (PCC = -.329, p = .038) and leisure activities (PCC = -.433, p = .005) scores of VOTES with the corresponding domains in the scales used for comparison was statistically significant. The correlation between the total score of VOTES and the total scores of OTTOS, SFQ and RSES was also statistically significant suggesting convergent validity. The correlation between the total score of VOTES with the total score of FBI is not statistically significant, implying good divergent validity. VOTES seems to be a promising tool to assess overall functioning of people with mental illness. © The Author(s) 2016.

Adapting the Bayley Scales of infant and toddler development in Ethiopia: evaluation of reliability and validity.

PubMed

Hanlon, C; Medhin, G; Worku, B; Tomlinson, M; Alem, A; Dewey, M; Prince, M

2016-09-01

There is a need for valid and reliable observational measures of early child development in low-income and middle-income country settings. The aims of the study were to adapt the Bayley Scales of Infant Development (Bayley III) for a rural Ethiopian setting and evaluate reliability and validity. The study was carried out between January 2008 and January 2009 in the Butajira demographic surveillance site, south central Ethiopia. The Bayley III was adapted to be socioculturally appropriate for a rural Ethiopian context. Nurses and high school graduates were trained in administration of the measure for 10 days. Inter-rater reliability was evaluated (n = 60). Content, construct and convergent validity was then examined on a population-based cohort of children at the ages of 30 (n = 440) and 42 months (n = 456). Mokken scale analysis was used to determine the scalability of items in unidimensional, hierarchical sub-scales. The mean score was compared by age of child and by stunting status (less than -2 z scores below the standard height-for-age). The intra-class correlations between raters were above 0.90 for all sub-scales of the child development measure. Some scale items were not contextually relevant and showed poor scalability. However, the majority of items scaled onto the existing sub-scales of the international measure to form adequate-to-strong hierarchical scales with good internal consistency (Cronbach's α above 0.70 except for gross motor and expressive language sub-scales). Item-scale coefficients were good. The mean score of all sub-scales was significantly higher in the older group of children (33.02 higher total score; P < 0.001) and in the children who were stunted (total Bayley score 2.58 (95% confidence interval 0.07 to 5.10) points lower at 30 months and 3.87 (1.94 to 5.81) points lower at 42 months. An adapted version of an international, observational measure of child development was found to be reliable, valid and feasible in a rural Ethiopian setting. © 2016 The Authors Child: Care, Health and Development Published by John Wiley & Sons Ltd.

Validation of the Korean version of the pediatric quality of life inventory 4.0 (PedsQL) generic core scales in school children and adolescents using the Rasch model.

PubMed

Kook, Seung Hee; Varni, James W

2008-06-02

The Pediatric Quality of Life Inventory (PedsQL) is a child self-report and parent proxy-report instrument designed to assess health-related quality of life (HRQOL) in healthy and ill children and adolescents. It has been translated into over 70 international languages and proposed as a valid and reliable pediatric HRQOL measure. This study aimed to assess the psychometric properties of the Korean translation of the PedsQL 4.0 Generic Core Scales. Following the guidelines for linguistic validation, the original US English scales were translated into Korean and cognitive interviews were administered. The field testing responses of 1425 school children and adolescents and 1431 parents to the Korean version of PedsQL 4.0 Generic Core Scales were analyzed utilizing confirmatory factor analysis and the Rasch model. Consistent with studies using the US English instrument and other translation studies, score distributions were skewed toward higher HRQOL in a predominantly healthy population. Confirmatory factor analysis supported a four-factor and a second order-factor model. The analysis using the Rasch model showed that person reliabilities are low, item reliabilities are high, and the majority of items fit the model's expectation. The Rasch rating scale diagnostics showed that PedsQL 4.0 Generic Core Scales in general have the optimal number of response categories, but category 4 (almost always a problem) is somewhat problematic for the healthy school sample. The agreements between child self-report and parent proxy-report were moderate. The results demonstrate the feasibility, validity, item reliability, item fit, and agreement between child self-report and parent proxy-report of the Korean version of PedsQL 4.0 Generic Core Scales for school population health research in Korea. However, the utilization of the Korean version of the PedsQL 4.0 Generic Core Scales for healthy school populations needs to consider low person reliability, ceiling effects and cultural differences, and further validation studies on Korean clinical samples are required.

Development and psychometric validation of a brief comprehensive health status assessment scale in older patients with hematological malignancies: The GAH Scale.

PubMed

Bonanad, S; De la Rubia, J; Gironella, M; Pérez Persona, E; González, B; Fernández Lago, C; Arnan, M; Zudaire, M; Hernández Rivas, J A; Soler, A; Marrero, C; Olivier, C; Altés, A; Valcárcel, D; Hernández, M T; Oiartzabal, I; Fernández Ordoño, R; Arnao, M; Esquerra, A; Sarrá, J; González-Barca, E; González, J; Calvo, X; Nomdedeu, M; García Guiñón, A; Ramírez Payer, A; Casado, A; López, S; Durán, M; Marcos, M; Cruz-Jentoft, A J

2015-09-01

The purpose of this study was to develop a new brief, comprehensive geriatric assessment scale for older patients diagnosed with different hematological malignancies, the Geriatric Assessment in Hematology (GAH scale), and to determine its psychometric properties. The 30-item GAH scale was designed through a multi-step process to cover 8 relevant dimensions. This is an observational study conducted in 363 patients aged≥65years, newly diagnosed with different hematological malignancies (myelodysplasic syndrome/acute myeloblastic leukemia, multiple myeloma, or chronic lymphocytic leukemia), and treatment-naïve. The scale psychometric validation process included the analyses of feasibility, floor and ceiling effect, validity and reliability criteria. Mean time taken to complete the GAH scale was 11.9±4.7min that improved through a learning-curve effect. Almost 90% of patients completed all items, and no floor or ceiling effects were identified. Criterion validity was supported by reasonable correlations between the GAH scale dimensions and three contrast variables (global health visual analogue scale, ECOG and Karnofsky), except for comorbidities. Factor analysis (supported by the scree plot) revealed nine factors that explained almost 60% of the total variance. Moderate internal consistency reliability was found (Cronbach's α: 0.610), and test-retest was excellent (ICC coefficients, 0.695-0.928). Our study suggests that the GAH scale is a valid, internally reliable and a consistent tool to assess health status in older patients with different hematological malignancies. Future large studies should confirm whether the GAH scale may be a tool to improve clinical decision-making in older patients with hematological malignancies. Copyright © 2015 Elsevier Inc. All rights reserved.

Reliable sagittal plane kinematic gait assessments are feasible using low-cost webcam technology.

PubMed

Saner, Robert J; Washabaugh, Edward P; Krishnan, Chandramouli

2017-07-01

Three-dimensional (3-D) motion capture systems are commonly used for gait analysis because they provide reliable and accurate measurements. However, the downside of this approach is that it is expensive and requires technical expertise; thus making it less feasible in the clinic. To address this limitation, we recently developed and validated (using a high-precision walking robot) a low-cost, two-dimensional (2-D) real-time motion tracking approach using a simple webcam and LabVIEW Vision Assistant. The purpose of this study was to establish the repeatability and minimal detectable change values of hip and knee sagittal plane gait kinematics recorded using this system. Twenty-one healthy subjects underwent two kinematic assessments while walking on a treadmill at a range of gait velocities. Intraclass correlation coefficients (ICC) and minimal detectable change (MDC) values were calculated for commonly used hip and knee kinematic parameters to demonstrate the reliability of the system. Additionally, Bland-Altman plots were generated to examine the agreement between the measurements recorded on two different days. The system demonstrated good to excellent reliability (ICC>0.75) for all the gait parameters tested on this study. The MDC values were typically low (<5°) for most of the parameters. The Bland-Altman plots indicated that there was no systematic error or bias in kinematic measurements and showed good agreement between measurements obtained on two different days. These results indicate that kinematic gait assessments using webcam technology can be reliably used for clinical and research purposes. Copyright © 2017 Elsevier B.V. All rights reserved.

Measuring Implementation Fidelity in a Community-Based Parenting Intervention

PubMed Central

Breitenstein, Susan M.; Fogg, Louis; Garvey, Christine; Hill, Carri; Resnick, Barbara; Gross, Deborah

2012-01-01

Background Establishing the feasibility and validity of implementation fidelity monitoring strategies is an important methodological step in implementing evidence-based interventions on a large scale. Objectives The objective of the study was to examine the reliability and validity of the Fidelity Checklist, a measure designed to assess group leader adherence and competence delivering a parent training intervention (the Chicago Parent Program) in child care centers serving low-income families. Method The sample included 9 parent groups (12 group sessions each), 12 group leaders, and 103 parents. Independent raters reviewed 106 audiotaped parent group sessions and coded group leaders’ fidelity on the Adherence and Competence Scales of the Fidelity Checklist. Group leaders completed self-report adherence checklists and a measure of parent engagement in the intervention. Parents completed measures of consumer satisfaction and child behavior. Results High interrater agreement (Adherence Scale = 94%, Competence Scale = 85%) and adequate intraclass correlation coefficients (Adherence Scale = .69, Competence Scale = .91) were achieved for the Fidelity Checklist. Group leader adherence changed over time, but competence remained stable. Agreement between group leader self-report and independent ratings on the Adherence Scale was 85%; disagreements were more frequently due to positive bias in group leader self-report. Positive correlations were found between group leader adherence and parent attendance and engagement in the intervention and between group leader competence and parent satisfaction. Although child behavior problems improved, improvements were not related to fidelity. Discussion The results suggest that the Fidelity Checklist is a feasible, reliable, and valid measure of group leader implementation fidelity in a group-based parenting intervention. Future research will be focused on testing the Fidelity Checklist with diverse and larger samples and generalizing to other group-based interventions using a similar intervention model. PMID:20404777

Development and Validation of Two Instruments Measuring Intrinsic, Extraneous, and Germane Cognitive Load.

PubMed

Klepsch, Melina; Schmitz, Florian; Seufert, Tina

2017-01-01

Cognitive Load Theory is one of the most powerful research frameworks in educational research. Beside theoretical discussions about the conceptual parts of cognitive load, the main challenge within this framework is that there is still no measurement instrument for the different aspects of cognitive load, namely intrinsic, extraneous, and germane cognitive load. Hence, the goal of this paper is to develop a differentiated measurement of cognitive load. In Study 1 ( N = 97), we developed and analyzed two strategies to measure cognitive load in a differentiated way: (1) Informed rating: We trained learners in differentiating the concepts of cognitive load, so that they could rate them in an informed way. They were asked then to rate 24 different learning situations or learning materials related to either high or low intrinsic, extraneous, or germane load. (2) Naïve rating: For this type of rating of cognitive load we developed a questionnaire with two to three items for each type of load. With this questionnaire, the same learning situations had to be rated. In the second study ( N = between 65 and 95 for each task), we improved the instrument for the naïve rating. For each study, we analyzed whether the instruments are reliable and valid, for Study 1, we also checked for comparability of the two measurement strategies. In Study 2, we conducted a simultaneous scenario based factor analysis. The informed rating seems to be a promising strategy to assess the different aspects of cognitive load, but it seems not economic and feasible for larger studies and a standardized training would be necessary. The improved version of the naïve rating turned out to be a useful, feasible, and reliable instrument. Ongoing studies analyze the conceptual validity of this measurement with up to now promising results.

Tests for the Assessment of Sport-Specific Performance in Olympic Combat Sports: A Systematic Review With Practical Recommendations

PubMed Central

Chaabene, Helmi; Negra, Yassine; Bouguezzi, Raja; Capranica, Laura; Franchini, Emerson; Prieske, Olaf; Hbacha, Hamdi; Granacher, Urs

2018-01-01

The regular monitoring of physical fitness and sport-specific performance is important in elite sports to increase the likelihood of success in competition. This study aimed to systematically review and to critically appraise the methodological quality, validation data, and feasibility of the sport-specific performance assessment in Olympic combat sports like amateur boxing, fencing, judo, karate, taekwondo, and wrestling. A systematic search was conducted in the electronic databases PubMed, Google-Scholar, and Science-Direct up to October 2017. Studies in combat sports were included that reported validation data (e.g., reliability, validity, sensitivity) of sport-specific tests. Overall, 39 studies were eligible for inclusion in this review. The majority of studies (74%) contained sample sizes <30 subjects. Nearly, 1/3 of the reviewed studies lacked a sufficient description (e.g., anthropometrics, age, expertise level) of the included participants. Seventy-two percent of studies did not sufficiently report inclusion/exclusion criteria of their participants. In 62% of the included studies, the description and/or inclusion of a familiarization session (s) was either incomplete or not existent. Sixty-percent of studies did not report any details about the stability of testing conditions. Approximately half of the studies examined reliability measures of the included sport-specific tests (intraclass correlation coefficient [ICC] = 0.43–1.00). Content validity was addressed in all included studies, criterion validity (only the concurrent aspect of it) in approximately half of the studies with correlation coefficients ranging from r = −0.41 to 0.90. Construct validity was reported in 31% of the included studies and predictive validity in only one. Test sensitivity was addressed in 13% of the included studies. The majority of studies (64%) ignored and/or provided incomplete information on test feasibility and methodological limitations of the sport-specific test. In 28% of the included studies, insufficient information or a complete lack of information was provided in the respective field of the test application. Several methodological gaps exist in studies that used sport-specific performance tests in Olympic combat sports. Additional research should adopt more rigorous validation procedures in the application and description of sport-specific performance tests in Olympic combat sports. PMID:29692739

Validation of the Hospital Ethical Climate Survey for older people care.

PubMed

Suhonen, Riitta; Stolt, Minna; Katajisto, Jouko; Charalambous, Andreas; Olson, Linda L

2015-08-01

The exploration of the ethical climate in the care settings for older people is highlighted in the literature, and it has been associated with various aspects of clinical practice and nurses' jobs. However, ethical climate is seldom studied in the older people care context. Valid, reliable, feasible measures are needed for the measurement of ethical climate. This study aimed to test the reliability, validity, and sensitivity of the Hospital Ethical Climate Survey in healthcare settings for older people. A non-experimental cross-sectional study design was employed, and a survey using questionnaires, including the Hospital Ethical Climate Survey was used for data collection. Data were analyzed using descriptive statistics, inferential statistics, and multivariable methods. Survey data were collected from a sample of nurses working in the care settings for older people in Finland (N = 1513, n = 874, response rate = 58%) in 2011. This study was conducted according to good scientific inquiry guidelines, and ethical approval was obtained from the university ethics committee. The mean score for the Hospital Ethical Climate Survey total was 3.85 (standard deviation = 0.56). Cronbach's alpha was 0.92. Principal component analysis provided evidence for factorial validity. LISREL provided evidence for construct validity based on goodness-of-fit statistics. Pearson's correlations of 0.68-0.90 were found between the sub-scales and the Hospital Ethical Climate Survey. The Hospital Ethical Climate Survey was found able to reveal discrimination across care settings and proved to be a valid and reliable tool for measuring ethical climate in care settings for older people and sensitive enough to reveal variations across various clinical settings. The Finnish version of the Hospital Ethical Climate Survey, used mainly in the hospital settings previously, proved to be a valid instrument to be used in the care settings for older people. Further studies are due to analyze the factor structure and some items of the Hospital Ethical Climate Survey. © The Author(s) 2014.

Study protocol of psychometric properties of the Spanish translation of a competence test in evidence based practice: the Fresno test.

PubMed

Argimon-Pallàs, Josep M; Flores-Mateo, Gemma; Jiménez-Villa, Josep; Pujol-Ribera, Enriqueta; Foz, Gonçal; Bundó-Vidiella, Magda; Juncosa, Sebastià; Fuentes-Bellido, Cruz M; Pérez-Rodríguez, Belén; Margalef-Pallarès, Francesc; Villafafila-Ferrero, Rosa; Forès-Garcia, Dolors; Roman-Martínez, Josep; Vilert-Garroga, Esther

2009-02-24

There are few high-quality instruments for evaluating the effectiveness of Evidence-Based Practice (EBP) curricula with objective outcomes measures. The Fresno test is an instrument that evaluates most of EBP steps with a high reliability and validity in the English original version. The present study has the aims to translate the Fresno questionnaire into Spanish and its subsequent validation to ensure the equivalence of the Spanish version against the English original. The questionnaire will be translated with the back translation technique and tested in Primary Care Teaching Units in Catalonia (PCTU). Participants will be: (a) tutors of Family Medicine residents (expert group); (b) Family Medicine residents in their second year of the Family Medicine training program (novice group), and (c) Family Medicine physicians (intermediate group). The questionnaire will be administered before and after an educational intervention. The educational intervention will be an interactive four half-day sessions designed to develop the knowledge and skills required to EBP. Responsiveness statistics used in the analysis will be the effect size, the standardised response mean and Guyatt's method. For internal consistency reliability, two measures will be used: corrected item-total correlations and Cronbach's alpha. Inter-rater reliability will be tested using Kappa coefficient for qualitative items and intra-class correlation coefficient for quantitative items and the overall score. Construct validity, item difficulty, item discrimination and feasibility will be determined. The validation of the Fresno questionnaire into different languages will enable the expansion of the questionnaire, as well as allowing comparison between countries and the evaluation of different teaching models.

Application and Evaluation of an Expert Judgment Elicitation Procedure for Correlations.

PubMed

Zondervan-Zwijnenburg, Mariëlle; van de Schoot-Hubeek, Wenneke; Lek, Kimberley; Hoijtink, Herbert; van de Schoot, Rens

2017-01-01

The purpose of the current study was to apply and evaluate a procedure to elicit expert judgments about correlations, and to update this information with empirical data. The result is a face-to-face group elicitation procedure with as its central element a trial roulette question that elicits experts' judgments expressed as distributions. During the elicitation procedure, a concordance probability question was used to provide feedback to the experts on their judgments. We evaluated the elicitation procedure in terms of validity and reliability by means of an application with a small sample of experts. Validity means that the elicited distributions accurately represent the experts' judgments. Reliability concerns the consistency of the elicited judgments over time. Four behavioral scientists provided their judgments with respect to the correlation between cognitive potential and academic performance for two separate populations enrolled at a specific school in the Netherlands that provides special education to youth with severe behavioral problems: youth with autism spectrum disorder (ASD), and youth with diagnoses other than ASD. Measures of face-validity, feasibility, convergent validity, coherence, and intra-rater reliability showed promising results. Furthermore, the current study illustrates the use of the elicitation procedure and elicited distributions in a social science application. The elicited distributions were used as a prior for the correlation, and updated with data for both populations collected at the school of interest. The current study shows that the newly developed elicitation procedure combining the trial roulette method with the elicitation of correlations is a promising tool, and that the results of the procedure are useful as prior information in a Bayesian analysis.

The paced auditory serial addition test for working memory assessment: Psychometric properties

PubMed Central

Nikravesh, Maryam; Jafari, Zahra; Mehrpour, Masoud; Kazemi, Roozbeh; Amiri Shavaki, Younes; Hossienifar, Shamim; Azizi, Mohamad Parsa

2017-01-01

Background: The paced auditory serial addition test (PASAT) was primarily developed to assess the effects of traumatic brain injury on cognitive functioning. Working memory (WM) is one of the most important aspects of cognitive function, and WM impairment is one of the clinically remarkable signs of aphasia. To develop the Persian version of PASAT, an initial version was used in individuals with aphasia (IWA). Methods: In this study, 25 individuals with aphasia (29-60 years) and 85 controls (18-60 years) were included. PASAT was presented in the form of recorded 61 single-digit numbers (1 to 9). The participants repeatedly added the 2 recent digits. The psychometric properties of PASAT including convergent validity (using the digit memory span tasks), divergent validity (using results in the control group and IWA group), and face validity were investigated. Test-retest reliability was considered as well. Results: The relationship between the PASAT and digit memory span tests was moderate to strong in the control group (forward digit memory span test: r= 0.52, p< 0.0001; backward digit memory span test: r = 0.48, p< 0.0001). A strong relationship was found in IWA (forward digit memory span test: r= 0.72, p< 0.0001; backward digit memory span test: r= 0.53, p= 0.006). Also, strong testretest reliability (intraclass correlation= 0.95, p< 0.0001) was observed. Conclusion: According to our results, the PASAT is a valid and reliable test to assess working memory, particularly in IWA. It could be used as a feasible tool for clinical and research applications. PMID:29445690

Arm cranking versus wheelchair propulsion for testing aerobic fitness in children with spina bifida who are wheelchair dependent.

PubMed

Bloemen, Manon A T; de Groot, Janke F; Backx, Frank J G; Westerveld, Rosalyne A; Takken, Tim

2015-05-01

To determine the best test performance and feasibility using a Graded Arm Cranking Test vs a Graded Wheelchair Propulsion Test in young people with spina bifida who use a wheelchair, and to determine the reliability of the best test. Validity and reliability study. Young people with spina bifida who use a wheelchair. Physiological responses were measured during a Graded Arm Cranking Test and a Graded Wheelchair Propulsion Test using a heart rate monitor and calibrated mobile gas analysis system (Cortex Metamax). For validity, peak oxygen uptake (VO2peak) and peak heart rate (HRpeak) were compared using paired t-tests. For reliability, the intra-class correlation coefficients, standard error of measurement, and standard detectable change were calculated. VO2peak and HRpeak were higher during wheelchair propulsion compared with arm cranking (23.1 vs 19.5 ml/kg/min, p = 0.11; 165 vs 150 beats/min, p < 0.05). Reliability of wheelchair propulsion showed high intra-class correlation coefficients (ICCs) for both VO2peak (ICC = 0.93) and HRpeak (ICC = 0.90). This pilot study shows higher HRpeak and a tendency to higher VO2peak in young people with spina bifida who are using a wheelchair when tested during wheelchair propulsion compared with arm cranking. Wheelchair propulsion showed good reliability. We recommend performing a wheelchair propulsion test for aerobic fitness testing in this population.

Development and initial cohort validation of the Arthritis Research UK Musculoskeletal Health Questionnaire (MSK-HQ) for use across musculoskeletal care pathways

PubMed Central

Hill, Jonathan C; Kang, Sujin; Benedetto, Elena; Myers, Helen; Blackburn, Steven; Smith, Stephanie; Hay, Elaine; Rees, Jonathan; Beard, David; Glyn-Jones, Sion; Barker, Karen; Ellis, Benjamin; Fitzpatrick, Ray; Price, Andrew

2016-01-01

Objectives Current musculoskeletal outcome tools are fragmented across different healthcare settings and conditions. Our objectives were to develop and validate a single musculoskeletal outcome measure for use throughout the pathway and patients with different musculoskeletal conditions: the Arthritis Research UK Musculoskeletal Health Questionnaire (MSK-HQ). Setting A consensus workshop with stakeholders from across the musculoskeletal community, workshops and individual interviews with a broad mix of musculoskeletal patients identified and prioritised outcomes for MSK-HQ inclusion. Initial psychometric validation was conducted in four cohorts from community physiotherapy, and secondary care orthopaedic hip, knee and shoulder clinics. Participants Stakeholders (n=29) included primary care, physiotherapy, orthopaedic and rheumatology patients (n=8); general practitioners, physiotherapists, orthopaedists, rheumatologists and pain specialists (n=7), patient and professional national body representatives (n=10), and researchers (n=4). The four validation cohorts included 570 participants (n=210 physiotherapy, n=150 hip, n=150 knee, n=60 shoulder patients). Outcome measures Outcomes included the MSK-HQ's acceptability, feasibility, comprehension, readability and responder burden. The validation cohort outcomes were the MSK-HQ's completion rate, test–retest reliability and convergent validity with reference standards (EQ-5D-5L, Oxford Hip, Knee, Shoulder Scores, and the Keele MSK-PROM). Results Musculoskeletal domains prioritised were pain severity, physical function, work interference, social interference, sleep, fatigue, emotional health, physical activity, independence, understanding, confidence to self-manage and overall impact. Patients reported MSK-HQ items to be ‘highly relevant’ and ‘easy to understand’. Completion rates were high (94.2%), with scores normally distributed, and no floor/ceiling effects. Test–retest reliability was excellent, and convergent validity was strong (correlations 0.81–0.88). Conclusions A new musculoskeletal outcome measure has been developed through a coproduction process with patients to capture prioritised outcomes for use throughout the pathway and with different musculoskeletal conditions. Four validation cohorts found that the MSK-HQ had high completion rates, excellent test–retest reliability and strong convergent validity with reference standards. Further validation studies are ongoing, including a cohort with rheumatoid/inflammatory arthritis. PMID:27496243

Considerations in the use of reflective writing for student assessment: issues of reliability and validity.

PubMed

Moniz, Tracy; Arntfield, Shannon; Miller, Kristina; Lingard, Lorelei; Watling, Chris; Regehr, Glenn

2015-09-01

Reflective writing is a popular tool to support the growth of reflective capacity in undergraduate medical learners. Its popularity stems from research suggesting that reflective capacity may lead to improvements in skills such as empathy, communication, collaboration and professionalism. This has led to assumptions that reflective writing can also serve as a tool for student assessment. However, evidence to support the reliability and validity of reflective writing as a meaningful assessment strategy is lacking. Using a published instrument for measuring 'reflective capacity' (the Reflection Evaluation for Learners' Enhanced Competencies Tool [REFLECT]), four trained raters independently scored four samples of writing from each of 107 undergraduate medical students to determine the reliability of reflective writing scores. REFLECT scores were then correlated with scores on a Year 4 objective structured clinical examination (OSCE) and Year 2 multiple-choice question (MCQ) examinations to examine, respectively, convergent and divergent validity. Across four writing samples, four-rater Cronbach's α-values ranged from 0.72 to 0.82, demonstrating reasonable inter-rater reliability with four raters using the REFLECT rubric. However, inter-sample reliability was fairly low (four-sample Cronbach's α = 0.54, single-sample intraclass correlation coefficient: 0.23), which suggests that performance on one reflective writing sample was not strongly indicative of performance on the next. Approximately 14 writing samples are required to achieve reasonable inter-sample reliability. The study found weak, non-significant correlations between reflective writing scores and both OSCE global scores (r = 0.13) and MCQ examination scores (r = 0.10), demonstrating a lack of relationship between reflective writing and these measures of performance. Our findings suggest that to draw meaningful conclusions about reflective capacity as a stable construct in individuals requires 14 writing samples per student, each assessed by four or five raters. This calls into question the feasibility and utility of using reflective writing rigorously as an assessment tool in undergraduate medical education. © 2015 John Wiley & Sons Ltd.

Cross-Cultural Adaptation and Psychometric Properties Testing of the Arabic Anterior Knee Pain Scale

PubMed Central

Alshehri, Abdullah; Lohman, Everett; Daher, Noha S.; Bahijri, Khalid; Alghamdi, Abdulmohsen; Altorairi, Nezar; Arnos, Arin; Matar, Abdullah

2017-01-01

Background PFPS is one of the most frequently occurring overuse injuries affecting the lower limbs. A variety of functional and self-reported outcome measures have been used to assess clinical outcomes of patients with PFPS, however, only the Anterior Knee Pain Scale (AKPS) has been designed for PFPS patients. Material/Methods We followed international recommendations to perform a cross-cultural adaptation of the AKPS. The Arabic AKPS and the Arabic RAND 36-item Health Survey were administered to 40 patients who were diagnosed with PFPS. Participants were assessed at baseline and after 2 to 3 days assessed with the Arabic AKPS only. The measurements tested were reliability, validity, and feasibility. Results The Arabic AKPS showed high reliability for both temporal stability, internal consistency (Cronbach’s alpha was 0.81 for the first assessment and 0.75 for the second), excellent test-retest reliability (Intraclass Correlation Coefficients ICC=0.96; 95% confidence interval (CI): 0.93, 0.98) and good agreement (standard error of measurement SEM=1.8%). The Arabic AKPS was significantly correlated with physical components of the RAND 36-Item Health Survey (Spearman’s rho=0.69: p<0.001). No ceiling or floor effects were observed. Conclusions The Arabic AKPS is a valid and reliable tool and is comparable to the original English version and other translated versions. PMID:28364114

Cross-Cultural Adaptation and Psychometric Properties Testing of the Arabic Anterior Knee Pain Scale.

PubMed

Alshehri, Abdullah; Lohman, Everett; Daher, Noha S; Bahijri, Khalid; Alghamdi, Abdulmohsen; Altorairi, Nezar; Arnons, Arin; Matar, Abdullah

2017-04-01

BACKGROUND PFPS is one of the most frequently occurring overuse injuries affecting the lower limbs. A variety of functional and self-reported outcome measures have been used to assess clinical outcomes of patients with PFPS, however, only the Anterior Knee Pain Scale (AKPS) has been designed for PFPS patients. MATERIAL AND METHODS We followed international recommendations to perform a cross-cultural adaptation of the AKPS. The Arabic AKPS and the Arabic RAND 36-item Health Survey were administered to 40 patients who were diagnosed with PFPS. Participants were assessed at baseline and after 2 to 3 days assessed with the Arabic AKPS only. The measurements tested were reliability, validity, and feasibility. RESULTS The Arabic AKPS showed high reliability for both temporal stability, internal consistency (Cronbach's alpha was 0.81 for the first assessment and 0.75 for the second), excellent test-retest reliability (Intraclass Correlation Coefficients ICC=0.96; 95% confidence interval (CI): 0.93, 0.98) and good agreement (standard error of measurement SEM=1.8%). The Arabic AKPS was significantly correlated with physical components of the RAND 36-Item Health Survey (Spearman's rho=0.69: p<0.001). No ceiling or floor effects were observed. CONCLUSIONS The Arabic AKPS is a valid and reliable tool and is comparable to the original English version and other translated versions.

Reliability and validity of the Tigrigna version of the Pelvic Floor Distress Inventory-Short Form 20 (PFDI-20) and Pelvic Floor Impact Questionnaire-7 (PFIQ-7).

PubMed

Goba, Gelila K; Legesse, Awol Yeman; Zelelow, Yibrah Berhe; Gebreselassie, Mussie Alemayehu; Rogers, Rebecca G; Kenton, Kimberly S; Mueller, Margaret G

2018-03-13

This study adapted the Pelvic Floor Distress Inventory-Short Form 20 (PFDI-20) and the Pelvic Floor Impact Questionnaire-7 (PFIQ-7) into the Tigrigna language of northern Ethiopia and validated the their reliability and validity through patient interviews. Expert translation, cognitive interviewing, and patient interviews using translated questionnaires were conducted. A subset of women was reinterviewed 1 week later. Intraclass correlation coefficients (ICC), Bland-Altman analysis, and Cronbach's alpha values were assessed. Total and subscale scores were compared between women with and without pelvic floor disorders (PFDs) using the Mann-Whitney U test. Spearman's correlation coefficients were used to compare severity of pelvic organ prolapse (POP) stage according to the POP Quantification (POP-Q) system and PFDI-20 and PFIQ-7 and subscale scores. Ten women participated in cognitive interviewing and 118 age 49 ± 10 years, mean ± standard deviation (SD) with and without PFDs were interviewed using the translated questionnaires, both of which presented adequate face validity and test-retest reliability [intraclass correlation coefficient (ICC) 0.765-0.969, p < 0.001]. Construct validity was significant between clinical symptoms and full forms (p <0.001) and their subscales (p <0.001), except for the Pelvic Organ Prolapse Impact Questionnaire (POPIQ). Differences between first and second scores on total PFDI-20 and PFIQ-7 and subscales largely fell within 0 ± 1.96 SD. Cronbach's alpha values were 0.891-0.930 for PFDI-20 and 0.909-0.956 for PFIQ-7 (p < 0.001). Analysis of known groups showed differences PFDI-20 and PFIQ-7 scores between women with and without PFDs (p <0.001 for full forms and subscales, except for anal incontinence (AI) and the Urinary Impact Questionnaire (UIQ)/POPIQ. The translated Tigrigna versions of the PFDI-20 and PFIQ-7 questionnaires are reliable, valid, and feasible tools to evaluate symptoms and quality of life (QoL) of Tigrigna-speaking Ethiopian women with PFDs.

How young can children reliably and validly self-report their health-related quality of life?: an analysis of 8,591 children across age subgroups with the PedsQL 4.0 Generic Core Scales.

PubMed

Varni, James W; Limbers, Christine A; Burwinkle, Tasha M

2007-01-03

The last decade has evidenced a dramatic increase in the development and utilization of pediatric health-related quality of life (HRQOL) measures in an effort to improve pediatric patient health and well-being and determine the value of healthcare services. The emerging paradigm shift toward patient-reported outcomes (PROs) in clinical trials has provided the opportunity to further emphasize the value and essential need for pediatric patient self-reported outcomes measurement. Data from the PedsQL DatabaseSM were utilized to test the hypothesis that children as young as 5 years of age can reliably and validly report their HRQOL. The sample analyzed represented child self-report age data on 8,591 children ages 5 to 16 years from the PedsQL 4.0 Generic Core Scales DatabaseSM. Participants were recruited from general pediatric clinics, subspecialty clinics, and hospitals in which children were being seen for well-child checks, mild acute illness, or chronic illness care (n = 2,603, 30.3%), and from a State Children's Health Insurance Program (SCHIP) in California (n = 5,988, 69.7%). Items on the PedsQL 4.0 Generic Core Scales had minimal missing responses for children as young as 5 years old, supporting feasibility. The majority of the child self-report scales across the age subgroups, including for children as young as 5 years, exceeded the minimum internal consistency reliability standard of 0.70 required for group comparisons, while the Total Scale Scores across the age subgroups approached or exceeded the reliability criterion of 0.90 recommended for analyzing individual patient scale scores. Construct validity was demonstrated utilizing the known groups approach. For each PedsQL scale and summary score, across age subgroups, including children as young as 5 years, healthy children demonstrated a statistically significant difference in HRQOL (better HRQOL) than children with a known chronic health condition, with most effect sizes in the medium to large effect size range. The results demonstrate that children as young as the 5 year old age subgroup can reliably and validly self-report their HRQOL when given the opportunity to do so with an age-appropriate instrument. These analyses are consistent with recent FDA guidelines which require instrument development and validation testing for children and adolescents within fairly narrow age groupings and which determine the lower age limit at which children can provide reliable and valid responses across age categories.

Assessing Nursing Care Needs of Children With Complex Medical Conditions: The Nursing-Kids Intensity of Care Survey (N-KICS).

PubMed

Navarra, Ann-Margaret; Schlau, Rona; Murray, Meghan; Mosiello, Linda; Schneider, Laura; Jackson, Olivia; Cohen, Bevin; Saiman, Lisa; Larson, Elaine L

2016-01-01

Recent medical advances have resulted in increased survival of children with complex medical conditions (CMC), but there are no validated methods to measure their care needs. To design and test the Nursing-Kids Intensity of Care Survey (N-KICS) tool and describe intensity of nursing care for children with CMC. The psychometric evaluation confirmed an acceptable standard for reliability and validity and feasibility. Intensity scores were highest for nursing care related to infection control, medication administration, nutrition, diaper changes, hygiene, neurological and respiratory support, and standing program. Development of a psychometrically sound measure of nursing intensity will help evaluate and plan nursing care for children with CMC. Copyright © 2016 Elsevier Inc. All rights reserved.

Assessing Nursing Care Needs of Children with Complex Medical Conditions: The Nursing Kids Intensity of Care Survey (N-KICS)

PubMed Central

Navarra, Ann-Margaret; Schlau, Rona; Murray, Meghan; Mosiello, Linda; Schneider, Laura; Jackson, Olivia; Cohen, Bevin; Saiman, Lisa; Larson, Elaine L.

2015-01-01

Background Recent medical advances have resulted in increased survival of children with complex medical conditions (CMC), but there are no validated methods to measure their care needs. Objectives/Methods To design and test the Nursing-Kids Intensity of Care Survey (N-KICS) tool and describe intensity of nursing care for children with CMC. Results The psychometric evaluation confirmed an acceptable standard for reliability and validity and feasibility. Intensity scores were highest for nursing care related to infection control, medication administration, nutrition, diaper changes, hygiene, neurological and respiratory support, and standing program. Conclusions Development of a psychometrically sound measure of nursing intensity will help evaluate and plan nursing care for children with CMC. PMID:26777429

Ready, Set, Change! Development and usability testing of an online readiness for change decision support tool for healthcare organizations.

PubMed

Timmings, Caitlyn; Khan, Sobia; Moore, Julia E; Marquez, Christine; Pyka, Kasha; Straus, Sharon E

2016-02-24

To address challenges related to selecting a valid, reliable, and appropriate readiness assessment measure in practice, we developed an online decision support tool to aid frontline implementers in healthcare settings in this process. The focus of this paper is to describe a multi-step, end-user driven approach to developing this tool for use during the planning stages of implementation. A multi-phase, end-user driven approach was used to develop and test the usability of a readiness decision support tool. First, readiness assessment measures that are valid, reliable, and appropriate for healthcare settings were identified from a systematic review. Second, a mapping exercise was performed to categorize individual items of included measures according to key readiness constructs from an existing framework. Third, a modified Delphi process was used to collect stakeholder ratings of the included measures on domains of feasibility, relevance, and likelihood to recommend. Fourth, two versions of a decision support tool prototype were developed and evaluated for usability. Nine valid and reliable readiness assessment measures were included in the decision support tool. The mapping exercise revealed that of the nine measures, most measures (78 %) focused on assessing readiness for change at the organizational versus the individual level, and that four measures (44 %) represented all constructs of organizational readiness. During the modified Delphi process, stakeholders rated most measures as feasible and relevant for use in practice, and reported that they would be likely to recommend use of most measures. Using data from the mapping exercise and stakeholder panel, an algorithm was developed to link users to a measure based on characteristics of their organizational setting and their readiness for change assessment priorities. Usability testing yielded recommendations that were used to refine the Ready, Set, Change! decision support tool . Ready, Set, Change! decision support tool is an implementation support that is designed to facilitate the routine incorporation of a readiness assessment as an early step in implementation. Use of this tool in practice may offer time and resource-saving implications for implementation.

Drug-Associated Risk Tool: development and validation of a self-assessment questionnaire to screen for hospitalised patients at risk for drug-related problems.

PubMed

Kaufmann, Carole P; Stämpfli, Dominik; Mory, Nadine; Hersberger, Kurt E; Lampert, Markus L

2018-03-09

Identifying patients with a high risk for drug-related problems (DRPs) might optimise the allocation of targeted pharmaceutical care during the hospital stay and on discharge. To develop a self-assessment screening tool to identify patients at risk for DRPs and validate the tool regarding feasibility, acceptability and the reliability of the patients' answers. Prospective validation study. Two mid-sized hospitals (300-400 beds). 195 patients, exclusion criteria: under 18 years old, patients with a health status not allowing a meaningful communication (eg, delirium, acute psychosis, advanced dementia, aphasia, clouded consciousness state), palliative or terminally ill patients. Twenty-seven risk factors for the development of DRPs, identified in a previous study, provided the basis of the self-assessment questionnaire, the Drug-Associated Risk Tool (DART). Consenting patients filled in DART, and we compared their answers with objective patient data from medical records and laboratory data. One hundred and sixty-four patients filled in DART V.1.0 in an average time of 7 min. After a first validation, we identified statements with a low sensitivity and revised the wording of the questions related to heart insufficiency, renal impairment or liver impairment. The revised DART (V.2.0) was validated in 31 patients presenting heart insufficiency, renal impairment or liver impairment as comorbidity and reached an average specificity of 88% (range 27-100) and an average sensitivity of 67% (range 21-100). DART showed a satisfying feasibility and reliability. The specificity of the statements was mostly high. The sensitivity varied and was higher in statements concerning diseases that require regular disease control and attention to self-care and drug management. Asking patients about their conditions, medications and related problems can facilitate getting a first, broad picture of the risk for DRPs and possible pharmaceutical needs. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2018. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

Development of an objective assessment tool for total laparoscopic hysterectomy: A Delphi method among experts and evaluation on a virtual reality simulator

PubMed Central

Knight, Sophie; Aggarwal, Rajesh; Agostini, Aubert; Loundou, Anderson; Berdah, Stéphane

2018-01-01

Introduction Total Laparoscopic hysterectomy (LH) requires an advanced level of operative skills and training. The aim of this study was to develop an objective scale specific for the assessment of technical skills for LH (H-OSATS) and to demonstrate feasibility of use and validity in a virtual reality setting. Material and methods The scale was developed using a hierarchical task analysis and a panel of international experts. A Delphi method obtained consensus among experts on relevant steps that should be included into the H-OSATS scale for assessment of operative performances. Feasibility of use and validity of the scale were evaluated by reviewing video recordings of LH performed on a virtual reality laparoscopic simulator. Three groups of operators of different levels of experience were assessed in a Marseille teaching hospital (10 novices, 8 intermediates and 8 experienced surgeons). Correlations with scores obtained using a recognised generic global rating tool (OSATS) were calculated. Results A total of 76 discrete steps were identified by the hierarchical task analysis. 14 experts completed the two rounds of the Delphi questionnaire. 64 steps reached consensus and were integrated in the scale. During the validation process, median time to rate each video recording was 25 minutes. There was a significant difference between the novice, intermediate and experienced group for total H-OSATS scores (133, 155.9 and 178.25 respectively; p = 0.002). H-OSATS scale demonstrated high inter-rater reliability (intraclass correlation coefficient [ICC] = 0.930; p<0.001) and test retest reliability (ICC = 0.877; p<0.001). High correlations were found between total H-OSATS scores and OSATS scores (rho = 0.928; p<0.001). Conclusion The H-OSATS scale displayed evidence of validity for assessment of technical performances for LH performed on a virtual reality simulator. The implementation of this scale is expected to facilitate deliberate practice. Next steps should focus on evaluating the validity of the scale in the operating room. PMID:29293635

Using pedometers to estimate ambulatory physical activity in Vietnam.

PubMed

Thuy, Au Bich; Blizzard, Leigh; Schmidt, Michael; Magnussen, Costan; Hansen, Emily; Dwyer, Terence

2011-01-01

Pedometer measurement of physical activity (PA) has been shown to be reliable and valid in industrialized populations, but its applicability in economically developing Vietnam remains untested. This study assessed the feasibility, stability and validity of pedometer estimates of PA in Vietnam. 250 adults from a population-based survey were randomly selected to wear Yamax pedometers and record activities for 7 consecutive days. Stability and concurrent validity were assessed using intraclass correlation coefficients (ICC) and Spearman correlation coefficients. Overall, 97.6% of participants provided at least 1 day of usable recordings, and 76.2% wore pedometers for all 7 days. Only 5.2% of the sample participants were involved in work activities not measurable by pedometer. The number of steps increased with hours of wear. There was no significant difference between weekday and weekend in number of steps, and at least 3 days of recordings were required (ICC of the 3 days of recordings: men 0.96, women 0.97). Steps per hour were moderately correlated (men r = .42, women r = .26) with record estimates of total PA. It is feasible to use pedometers to estimate PA in Vietnam. The measure should involve at least 3 days of recording irrespective of day of the week. ©2011 Human Kinetics, Inc.

Feasibility of a Semi-computerized Line Bisection Test for Unilateral Visual Neglect Assessment.

PubMed

Jee, H; Kim, J; Kim, C; Kim, T; Park, J

2015-01-01

Commonly used paper-and-pencil based test modalities for assessing the degree of unilateral visual neglect (ULN) in patients with hemispheric cerebral lesions consume human resources with a significant inter and intra-rater variability. To explore the feasibility of a semi-computerized electronic-pen based ULN assessment system (e-system) to improve assessment quality without altering the conventional user interface. Thirty cognitively healthy participants (HG) and 11 participants diagnosed with right-hemispheric lesion and unilateral visual neglect (NG) were recruited to evaluate the e-system. Line bisection tests (LBT) were repeatedly conducted twice for the inter-rater and intra-rater (reliability) comparisons. The LBT results were assessed by the e-system and the golden standard methods (manual rater assessment). The percent deviation (%), assessment duration (sec), and number of neglected line (each) were evaluated. The inter-rater comparisons of the assessed deviation (%) variable showed excellent interrater reliabilities (CCCs) ranging from .84 (.59 to .95 (p < .001)) to .99 (.90 to .99 (p < .001)) for HG and NG. The Bland Altman mean difference (B-A) plots with bias (95% LOA (limits of agreement)) showed similar agreements between the e-system and the raters ranging from -.04 % (-2.10 to 1.97) to 1.30 % (-2.23 to 4.84) for HG and NG. The effect sizes (ES), which show similarities between the assessment methods, yielded smaller ranges from .01 to .30 for HG and NG. The reliability (test-retest) comparisons showed similar assessment results between the e-system, rater 1, and rater 2. The manual rater assessment time ranging from 5.85 to 6.00 minutes and inter- and intraassessment variations were virtually eliminated with the e-system. The semi-computerized system with the conventional paper-and pencil user-interface showed valid and reliable assessment results. It may be a feasible replacement for the manual rater assessment modality even in a clinical setting.

Cross-cultural adaptation and validation of the Japanese version of the new Knee Society Scoring System for osteoarthritic knee with total knee arthroplasty.

PubMed

Hamamoto, Yosuke; Ito, Hiromu; Furu, Moritoshi; Ishikawa, Masahiro; Azukizawa, Masayuki; Kuriyama, Shinichi; Nakamura, Shinichiro; Matsuda, Shuichi

2015-09-01

The purposes of this study were to translate the new Knee Society Score (KSS) into Japanese and to evaluate the construct and content validity, test-retest reliability, and internal consistency of the Japanese version of the new KSS. The Japanese version of the KSS was developed according to cross-cultural guidelines by using the "translation-back translation" method to ensure content validity. KSS data were then obtained from patients who had undergone total knee arthroplasty (TKA). The psychometric properties evaluated were as follows: for feasibility, response rate, and floor and ceiling effects; for construct validity, internal consistency using Cronbach's alpha, and correlations with quality of life. Construct validity was evaluated by using Spearman's correlation coefficient to quantify the correlation between the KSS and the Japanese version of the Oxford 12-item Knee Score or Short Form 36 Health Survey (SF-36) questionnaires. The Japanese version of the KSS was sent to 93 consecutive osteoarthritic patients who underwent primary TKA in our institution. Fifty-five patients completed the questionnaires and were included in this study. Neither a floor nor ceiling effect was observed. The reliability proved excellent in the majority of domains, with intraclass correlation coefficients of 0.65-0.88. Internal consistency, assessed by Cronbach's alpha, was good to excellent for all domains (0.78-0.94). All of the four domains of the KSS correlated significantly with the Oxford 12-item Knee Score. The activity and satisfaction domains of the KSS correlated significantly with all and the majority of subscales of the SF-36, respectively, whereas symptoms and expectation domains showed significant correlations only with bodily pain and vitality subscales and with the physical function, bodily pain, and vitality subscales, respectively. The Japanese version of the new KSS is a valid, reliable, and responsive instrument to capture subjective aspects of the functional symptoms and abilities of patients who undergo TKA.

Current status of validation for robotic surgery simulators - a systematic review.

PubMed

Abboudi, Hamid; Khan, Mohammed S; Aboumarzouk, Omar; Guru, Khurshid A; Challacombe, Ben; Dasgupta, Prokar; Ahmed, Kamran

2013-02-01

To analyse studies validating the effectiveness of robotic surgery simulators. The MEDLINE(®), EMBASE(®) and PsycINFO(®) databases were systematically searched until September 2011. References from retrieved articles were reviewed to broaden the search. The simulator name, training tasks, participant level, training duration and evaluation scoring were extracted from each study. We also extracted data on feasibility, validity, cost-effectiveness, reliability and educational impact. We identified 19 studies investigating simulation options in robotic surgery. There are five different robotic surgery simulation platforms available on the market. In all, 11 studies sought opinion and compared performance between two different groups; 'expert' and 'novice'. Experts ranged in experience from 21-2200 robotic cases. The novice groups consisted of participants with no prior experience on a robotic platform and were often medical students or junior doctors. The Mimic dV-Trainer(®), ProMIS(®), SimSurgery Educational Platform(®) (SEP) and Intuitive systems have shown face, content and construct validity. The Robotic Surgical SimulatorTM system has only been face and content validated. All of the simulators except SEP have shown educational impact. Feasibility and cost-effectiveness of simulation systems was not evaluated in any trial. Virtual reality simulators were shown to be effective training tools for junior trainees. Simulation training holds the greatest potential to be used as an adjunct to traditional training methods to equip the next generation of robotic surgeons with the skills required to operate safely. However, current simulation models have only been validated in small studies. There is no evidence to suggest one type of simulator provides more effective training than any other. More research is needed to validate simulated environments further and investigate the effectiveness of animal and cadaveric training in robotic surgery. © 2012 BJU International.

Concurrent Validity of Walking Speed Values Calculated via the GAITRite Electronic Walkway and 3 Meter Walk Test in the Chronic Stroke Population

PubMed Central

Peters, Denise M.; Middleton, Addie; Donley, Jonathan W.; Blanck, Erika L.; Fritz, Stacy L.

2014-01-01

The purpose of this study was to provide novel information regarding the concurrent validity (primary aim) and reliability (secondary aim) of walking speed (WS) calculated via the GAITRite1 electronic walkway system and 3 meter walk test (3MWT) in the chronic stroke population. The 3MWT is a feasible option for clinicians working in environments where space is limited. Psychometric properties of the test have not been established. Participants with chronic stroke were stratified into three groups: household ambulators (HA) (self-selected WS <0.4 m/s, 12 participants, 31 observations), limited community ambulators (LCA) (self-selected WS 0.4–0.8 m/s, 24 participants, 60 observations), and community ambulators (CA) (self-selected WS >0.8 m/s, 26 participants, 71 observations). Three consecutive trials of GAITRite1 and 3MWT were performed at participant’s self-selected WS. Average WS measurements differed significantly (p <0.05) between GAITRite1 and 3MWT for all three groups. HA group: GAITRite1 0.25 (0.11) m/s, 3MWT 0.27 (0.11) m/s; LCA group: GAITRite1 0.56 (0.11) m/s, 3MWT 0.52 (0.10) m/s; CA group: GAITRite1 1.03 (0.16) m/s, 3MWT 0.89 (0.15) m/s. Both WS measures had excellent within-session reliability (ICC’s ranging from 0.85 to 0.97, SEM95 from 0.04 to 0.12 m/s, and MDC95 from 0.05 to 0.16 m/s). Reliability was highest for HA on both measures. Although both the 3MWT and the GAITRite1 are reliable measures of WS for individuals with chronic stroke, the two measures do not demonstrate concurrent validity. PMID:24164441

Cross-cultural challenges for assessing medical professionalism among clerkship physicians in a Middle Eastern country (Bahrain): feasibility and psychometric properties of multisource feedback

PubMed Central

Al Ansari, Ahmed; Al Khalifa, Khalid; Al Azzawi, Mohamed; Al Amer, Rashed; Al Sharqi, Dana; Al-Mansoor, Anwar; Munshi, Fadi M

2015-01-01

Background We aimed to design, implement, and evaluate the feasibility and reliability of a multisource feedback (MSF) system to assess interns in their clerkship year in the Middle Eastern culture, the Kingdom of Bahrain. Method The study was undertaken in the Bahrain Defense Force Hospital, a military teaching hospital in the Kingdom of Bahrain. A total of 21 interns (who represent the total population of the interns for the given year) were assessed in this study. All of the interns were rotating through our hospital during their year-long clerkship rotation. The study sample consisted of nine males and 12 females. Each participating intern was evaluated by three groups of raters, eight medical intern colleagues, eight senior medical colleagues, and eight coworkers from different departments. Results A total of 21 interns (nine males and 12 females) were assessed in this study. The total mean response rates were 62.3%. A factor analysis was conducted that found that the data on the questionnaire grouped into three factors that counted for 76.4% of the total variance. These three factors were labeled as professionalism, collaboration, and communication. Reliability analysis indicated that the full instrument scale had high internal consistency (Cronbach’s α 0.98). The generalizability coefficients for the surveys were estimated to be 0.78. Conclusion Based on our results and analysis, we conclude that the MSF tool we used on the interns rotating in their clerkship year within our Middle Eastern culture provides an effective method of evaluation because it offers a reliable, valid, and feasible process. PMID:26316836

Serial personal digital assistant data capture of health-related quality of life: A randomized controlled trial in a prostate cancer clinic

PubMed Central

Matthew, Andrew G; Currie, Kristen L; Irvine, Jane; Ritvo, Paul; Santa Mina, Daniel; Jamnicky, Leah; Nam, Robert; Trachtenberg, John

2007-01-01

Background In clinical and research practice linked to prostate cancer treatment, frequent monitoring of patient health-related quality of life (HRQOL) is essential. Practical and analytic limitations of paper questionnaire data capture may be overcome with the use of self-administered personal digital assistant (PDA) data collection. The objective of this study was to assess the reliability, validity, and feasibility of using PDA in place of paper versions of the International Prostate Symptom Score (IPSS), the Patient Oriented Prostate Cancer Utility Survey (PORPUS), and the International Index of Erectile Function-5 (IIEF-5) in a prostate cancer clinic setting. Methods 152 participants were randomly assigned to one of three conditions: 1) paper followed by PDA survey; 2) PDA followed by paper survey; or 3) PDA followed by PDA survey. Evaluation included an assessment of data quality (internal consistency, test-retest reliability, response correlation, completeness of data), and feasibility (participation rates, time to completion, preference and difficultly/ease of using PDA). Results Internal consistency was similar for both PDA and paper applications. Test-retest reliability was confirmed for PDA repeated administration. Data from paper and PDA questionnaires were strongly correlated. Lower missed item rates were found in PDA administration. 82.8% of participants preferred using the PDA or had no preference. Mean difficulty/ease ratings indicated that participants found the PDA easy to use. Age did not significantly correlate with preference or difficulty. Conclusion The results confirm the adaptability of the IPSS, IIEF-5, and the PORPUS to PDA administration. Similarly, the findings of this study support the feasibility of using PDA technology for HRQOL serial data capture in the prostate cancer patient population. PMID:17617906

Validation of heart and lung teleauscultation on an Internet-based system.

PubMed

Fragasso, Gabriele; De Benedictis, Marialuisa; Palloshi, Altin; Moltrasio, Marco; Cappelletti, Alberto; Carlino, Mauro; Marchisi, Angelo; Pala, Mariagrazia; Alfieri, Ottavio; Margonato, Alberto

2003-11-01

The feasibility and accuracy of an Internet-based system for teleauscultation was evaluated in 103 cardiac patients, who were auscultated by the same cardiologist with a conventional stethoscope and with an Internet-based method, using an electronic stethoscope and transmitting heart and lung sounds between computer work stations. In 92% of patients, the results of electronic and acoustic auscultation coincided, indicating that teleauscultation may be considered a reliable method for assessing cardiac patients and could, therefore, be adopted in the context of comprehensive telecare programs.

Validation of Commercial Fiber Optic Components for Aerospace Environments

NASA Technical Reports Server (NTRS)

Ott, Melanie N.

2005-01-01

Full qualification for commercial photonic parts as defined by the Military specification system in the past, is not feasible. Due to changes in the photonic components industry and the Military specification system that NASA had relied upon so heavily in the past, an approach to technology validation of commercial off the shelf parts had to be devised. This approach involves knowledge of system requirements, environmental requirements and failure modes of the particular components under consideration. Synthesizing the criteria together with the major known failure modes to formulate a test plan is an effective way of establishing knowledge based "qualification". Although this does not provide the type of reliability assurance that the Military specification system did in the past, it is an approach that allows for increased risk mitigation. The information presented will introduce the audience to the technology validation approach that is currently applied at NASA for the usage of commercial-off-the-shelf (COTS) fiber optic components for space flight environments. The focus will be on how to establish technology validation criteria for commercial fiber products such that continued reliable performance is assured under the harsh environmental conditions of typical missions. The goal of this presentation is to provide the audience with an approach to formulating a COTS qualification test plan for these devices. Examples from past NASA missions will be discussed.

Author Correction: Reliability and feasibility of gait initiation centre-of-pressure excursions using a Wii® Balance Board in older adults at risk of falling.

PubMed

Lee, James; Webb, Graham; Shortland, Adam P; Edwards, Rebecca; Wilce, Charlotte; Jones, Gareth D

2018-05-12

In the original publication, the article title was incorrectly published as 'Reliability and feasibility of gait initiation centre-of-pressure excursions using a Wii® Balance Board in older adults at risk of failing'. The correct title should read as 'Reliability and feasibility of gait initiation centre-of-pressure excursions using a Wii® Balance Board in older adults at risk of falling'.

The Chinese version of monitoring and evaluation system strengthening tool for human immunodeficiency virus (HIV) capacity building: Development and evaluation.

PubMed

Zhao, Ran; Chen, Ren; Zhang, Bing; Ma, Ying; Qin, Xia; Hu, Zhi

2015-08-01

Monitoring and evaluation (M&E) for human immunodeficiency virus (HIV) capacity building has become a significant step for HIV prevention and control. The M&E system strengthening tool published by the United Nations Joint Programme on HIV/AIDS (UNAIDS) was intended to be the most authoritative assessment tool internationally. Facing the fact that the M&E system in China did not function at an optimum level, we considered taking the international standards for reference. By linguistic validating and different stages' discussions and revisions, we came up with the Chinese version of the capacity diagnosis tool with at least 12 components and tested its validity and reliability. The tool turned out to have a sufficiently linguistic validation and proved to be a scientific and feasible instrument which was suitable for China's national conditions.

The Localized Scleroderma Skin Severity Index and Physician Global Assessment of Disease Activity: A Work in Progress Toward Development of Localized Scleroderma Outcome Measures

PubMed Central

ARKACHAISRI, THASCHAWEE; VILAIYUK, SOAMARAT; LI, SUZANNE; O’NEIL, KATHLEEN M.; POPE, ELENA; HIGGINS, GLORIA C.; PUNARO, MARILYNN; RABINOVICH, EGLA C.; ROSENKRANZ, MARGALIT; KIETZ, DANIEL A.; ROSEN, PAUL; SPALDING, STEVEN J.; HENNON, TERESA R.; TOROK, KATHRYN S.; CASSIDY, ELAINE; MEDSGER, THOMAS A.

2013-01-01

Objective To develop and evaluate a Localized Scleroderma (LS) Skin Severity Index (LoSSI) and global assessments’ clinimetric property and effect on quality of life (QOL). Methods A 3-phase study was conducted. The first phase involved 15 patients with LS and 14 examiners who assessed LoSSI [surface area (SA), erythema (ER), skin thickness (ST), and new lesion/extension (N/E)] twice for inter/intrarater reliability. Patient global assessment of disease severity (PtGA-S) and Children’s Dermatology Life Quality Index (CDLQI) were collected for intrarater reliability evaluation. The second phase was aimed to develop clinical determinants for physician global assessment of disease activity (PhysGA-A) and to assess its content validity. The third phase involved 2 examiners assessing LoSSI and PhysGA-A on 27 patients. Effect of training on improving reliability/validity and sensitivity to change of the LoSSI and PhysGA-A was determined. Results Interrater reliability was excellent for ER [intraclass correlation coefficient (ICC) 0.71], ST (ICC 0.70), LoSSI (ICC 0.80), and PhysGA-A (ICC 0.90) but poor for SA (ICC 0.35); thus, LoSSI was modified to mLoSSI. Examiners’ experience did not affect the scores, but training/practice improved reliability. Intrarater reliability was excellent for ER, ST, and LoSSI (Spearman’s rho = 0.71–0.89) and moderate for SA. PtGA-S and CDLQI showed good intrarater agreement (ICC 0.63 and 0.80). mLoSSI correlated moderately with PhysGA-A and PtGA-S. Both mLoSSI and PhysGA-A were sensitive to change following therapy. Conclusion mLoSSI and PhysGA-A are reliable and valid tools for assessing LS disease severity and show high sensitivity to detect change over time. These tools are feasible for use in routine clinical practice. They should be considered for inclusion in a core set of LS outcome measures for clinical trials. PMID:19833758

Reliability and validity of the Chinese mandarin version of PedsQL™ 3.0 transplant module.

PubMed

Chang, Ying; Luo, Yanhui; Zhou, Yuchen; Wang, Ruixin; Song, Na; Zhu, Guanghua; Wang, Bin; Qin, Maoquan; Yang, Jun; Sun, Yuan; Li, Chunfu; Zhou, Xuan

2016-10-05

Long-term health-related quality of life (HRQoL) of pediatric patients after hematopoietic stem cell transplantation (HSCT) is increasingly studied worldwide. However, few studies have been performed in China, where no uniform scale is available; the PedsQL™ Cancer Module 3.0 Chinese Mandarin version has been used to evaluate HRQoL of patients after HSCT in China. This study aimed to assess the reliability and validity of the Chinese Mandarin version of PedsQL™ 3.0 Transplant Module. Patients between 2 and 18 years old, who underwent HSCT from January 2006 to June 2014, were recruited in Beijing Children's Hospital affiliated to Capital Medical University, the First Affiliated Hospital of Southern Medical University and Beijing Daopei Hospital. 207 parent reports and 182 child self-reports of the PedsQL™ 3.0 Transplant Module Chinese Mandarin version were assigned, of which 362 were returned. No missing item response was observed in the returned reports. Cronbach's alpha coefficient exceeded 0.7 in total scale and every dimension. The intraclass correlation coefficient exceeded 0.8 in all dimensions of child self-reports and parent reports. Spearman's rank correlation coefficients of items and their respective dimensions were 0.6-0.94 in parent reports, and 0.62-0.93 in child self-reports, while a weak association was found between the items and other dimensions. Exploratory factor analysis indicated a good extraction effect, and construct validity of the scale was >60 %. The Chinese Mandarin version of PedsQL™ 3.0 Transplant Module has good feasibility, reliability and validity. Its use may help improve the HRQoL of children after HSCT in China.

Using personal qualities assessment to measure the moral orientation and personal qualities of medical students in a non-Western culture.

PubMed

Tsou, Kuo-Inn; Lin, Chaou-Shune; Cho, Shu-Ling; Powis, David; Bore, Miles; Munro, Don; Sze, Daniel Man-Yuen; Wu, Hsi-Chin; Hsieh, Ming-Shium; Lin, Chyi-Her

2013-06-01

How to select candidates with appropriate personal qualities for medical school is an important issue. This study examined the psychometric properties and group differences of the Personal Qualities Assessment (PQA) to test the feasibility of using it as a tool to assess the medical school applicants in a non-Western culture. Seven hundred forty-six medical students in Taiwan completed two psychometric measures: Mojac to assess moral orientation and NACE to assess four aspects of interpersonal relationships. Thirty-one students completed the tests twice to establish test-retest reliability. A subsample of 127 students also completed a measure of the "Big Five" personality traits to examine the construct validity of these scales. Both Mojac and NACE had acceptable internal consistency and test-retest reliability. Conceptually, coherent and significant relationships were observed between test components and between the NACE and Big Five. NACE but not Mojac varied significantly between different sociodemographic groups. Both tests demonstrated acceptable psychometric properties. However, the predictive validity of PQA requires future studies.

Midwifery education and technology enhanced learning: Evaluating online story telling in preregistration midwifery education.

PubMed

Scamell, Mandie; Hanley, Thomas

2018-03-01

A major issue regarding the implementation of blended learning for preregistration health programmes is the analysis of students' perceptions and attitudes towards their learning. It is the extent of the embedding of Technology Enhanced Learning (TEL) into the higher education curriculum that makes this analysis so vital. This paper reports on the quantitative results of a UK based study that was set up to respond to the apparent disconnect between technology enhanced education provision and reliable student evaluation of this mode of learning. Employing a mixed methods research design, the research described here was carried to develop a reliable and valid evaluation tool to measure acceptability of and satisfaction with a blended learning approach, specifically designed for a preregistration midwifery module offered at level 4. Feasibility testing of 46 completed blended learning evaluation questionnaires - Student Midwife Evaluation of Online Learning Effectiveness (SMEOLE) - using descriptive statistics, reliability and internal consistency tests. Standard deviations and mean scores all followed predicted pattern. Results from the reliability and internal consistency testing confirm the feasibility of SMEOLE as an effective tool for measuring student satisfaction with a blended learning approach to preregistration learning. The analysis presented in this paper suggests that we have been successful in our aim to produce an evaluation tool capable of assessing the quality of technology enhanced, University level learning in Midwifery. This work can provide future benchmarking against which midwifery, and other health, blended learning curriculum planning could be structured and evaluated. Copyright © 2017 Elsevier Ltd. All rights reserved.

Reliability and feasibility of the six minute walk test in subjects with myotonic dystrophy.

PubMed

Kierkegaard, Marie; Tollbäck, Anna

2007-12-01

The objective was to describe test-retest reliability and feasibility of the six minute walk test in adult subjects with myotonic dystrophy type 1. Twelve subjects (28-68 years, mean 44) performed three six minute walk tests on two occasions, one week apart. Relative reliability was high (ICC(2.1)=0.99) and absolute reliability values were low (standard error of measurement 12 m, repeatability 33 m). Feasibility was investigated in a sample of 64 subjects (19-70 years, mean 43). Fifty-two subjects were able to perform two tests on the same day. Subjects with severe proximal weakness had difficulties performing repeated tests. A practice trial followed by a second test on the same day can be recommended for most subjects, and the best test should be used for evaluations. In conclusion, even though the study sample was small, the present study indicates that the six minute walk test is reliable and feasible in subjects with myotonic dystrophy type 1.

Development and Validation of an Instrument for Measuring the Quality of Teamwork in Teaching Teams in Postgraduate Medical Training (TeamQ)

PubMed Central

Slootweg, Irene A.; Lombarts, Kiki M. J. M. H.; Boerebach, Benjamin C. M.; Heineman, Maas Jan; Scherpbier, Albert J. J. A.; van der Vleuten, Cees P. M.

2014-01-01

Background Teamwork between clinical teachers is a challenge in postgraduate medical training. Although there are several instruments available for measuring teamwork in health care, none of them are appropriate for teaching teams. The aim of this study is to develop an instrument (TeamQ) for measuring teamwork, to investigate its psychometric properties and to explore how clinical teachers assess their teamwork. Method To select the items to be included in the TeamQ questionnaire, we conducted a content validation in 2011, using a Delphi procedure in which 40 experts were invited. Next, for pilot testing the preliminary tool, 1446 clinical teachers from 116 teaching teams were requested to complete the TeamQ questionnaire. For data analyses we used statistical strategies: principal component analysis, internal consistency reliability coefficient, and the number of evaluations needed to obtain reliable estimates. Lastly, the median TeamQ scores were calculated for teams to explore the levels of teamwork. Results In total, 31 experts participated in the Delphi study. In total, 114 teams participated in the TeamQ pilot. The median team response was 7 evaluations per team. The principal component analysis revealed 11 factors; 8 were included. The reliability coefficients of the TeamQ scales ranged from 0.75 to 0.93. The generalizability analysis revealed that 5 to 7 evaluations were needed to obtain internal reliability coefficients of 0.70. In terms of teamwork, the clinical teachers scored residents' empowerment as the highest TeamQ scale and feedback culture as the area that would most benefit from improvement. Conclusions This study provides initial evidence of the validity of an instrument for measuring teamwork in teaching teams. The high response rates and the low number of evaluations needed for reliably measuring teamwork indicate that TeamQ is feasible for use by teaching teams. Future research could explore the effectiveness of feedback on teamwork in follow up measurements. PMID:25393006

Development and validation of an instrument for measuring the quality of teamwork in teaching teams in postgraduate medical training (TeamQ).

PubMed

Slootweg, Irene A; Lombarts, Kiki M J M H; Boerebach, Benjamin C M; Heineman, Maas Jan; Scherpbier, Albert J J A; van der Vleuten, Cees P M

2014-01-01

Teamwork between clinical teachers is a challenge in postgraduate medical training. Although there are several instruments available for measuring teamwork in health care, none of them are appropriate for teaching teams. The aim of this study is to develop an instrument (TeamQ) for measuring teamwork, to investigate its psychometric properties and to explore how clinical teachers assess their teamwork. To select the items to be included in the TeamQ questionnaire, we conducted a content validation in 2011, using a Delphi procedure in which 40 experts were invited. Next, for pilot testing the preliminary tool, 1446 clinical teachers from 116 teaching teams were requested to complete the TeamQ questionnaire. For data analyses we used statistical strategies: principal component analysis, internal consistency reliability coefficient, and the number of evaluations needed to obtain reliable estimates. Lastly, the median TeamQ scores were calculated for teams to explore the levels of teamwork. In total, 31 experts participated in the Delphi study. In total, 114 teams participated in the TeamQ pilot. The median team response was 7 evaluations per team. The principal component analysis revealed 11 factors; 8 were included. The reliability coefficients of the TeamQ scales ranged from 0.75 to 0.93. The generalizability analysis revealed that 5 to 7 evaluations were needed to obtain internal reliability coefficients of 0.70. In terms of teamwork, the clinical teachers scored residents' empowerment as the highest TeamQ scale and feedback culture as the area that would most benefit from improvement. This study provides initial evidence of the validity of an instrument for measuring teamwork in teaching teams. The high response rates and the low number of evaluations needed for reliably measuring teamwork indicate that TeamQ is feasible for use by teaching teams. Future research could explore the effectiveness of feedback on teamwork in follow up measurements.

Psychometric Properties of the Diabetes Management Self-Efficacy Scale in Korean Patients with Type 2 Diabetes.

PubMed

Lee, Eun-Hyun; van der Bijl, Jaap; Shortridge-Baggett, Lillie M; Han, Seung Jin; Moon, Seung Hei

2015-01-01

Objectives. The aims of this study were to perform a cultural translation of the DMSES and evaluate the psychometric properties of the translated scale in a Korean population with type 2 diabetics. Methods. This study was conducted in patients with diabetes recruited from university hospitals. The first stage of this study involved translating the DMSES into Korean using a forward- and backward-translation technique. The content validity was assessed by an expert group. In the second stage, the psychometric properties of the Korean version of the DMSES (K-DMSES) were evaluated. Results. The content validity of the K-DMSES was satisfactory. Sixteen-items clustered into four-subscales were extracted by exploratory factor analysis, and supported by confirmatory factor analysis. The construct validity of the K-DMSES with the Summary of Diabetes Self-Care Activities scale was satisfactory (r = 0.50, P<0.001). The Cronbach's alpha and intraclass correlation coefficient were 0.92 and 0.85 (P<0.001; 95% CI = 0.75-0.91), respectively, which indicate excellent internal consistency reliability and test-retest reliability. Conclusions. The K-DMSES is a brief instrument that has demonstrated good psychometric properties. It is therefore feasible to use in practice, and is ready for use in clinical research involving Korean patients with type 2 diabetes.

Measurement of acute pain in infants: a review of behavioral and physiological variables.

PubMed

Hatfield, Linda A; Ely, Elizabeth A

2015-01-01

The use of non-validated pain measurement tools to assess infant pain represents a serious iatrogenic threat to the developing neonatal nervous system. One partial explanation for this practice may be the contradictory empirical data from studies that use newborn pain management tools constructed for infants of different developmental stages or exposed to different environmental stressors. The purpose of this review is to evaluate the evidence regarding the physiologic and behavioral variables that accurately assess and measure acute pain response in infants. A literature search was conducted using PUBMED and CINAHL and the search terms infant, neonate/neonatal, newborn, pain, assessment, and measurement to identify peer-reviewed studies that examined the validity and reliability of behavioral and physiological variables used for investigation of infant pain. Ten articles were identified for critical review. Strong evidence supports the use of the behavioral variables of facial expressions and body movements and the physiologic variables of heart rate and oxygen saturation to assess acute pain in infants. It is incumbent upon researchers and clinical nurses to ensure the validity, reliability, and feasibility of pain measures, so that the outcomes of their investigations and interventions will be developmentally appropriate and effective pain management therapies. © The Author(s) 2014.

Linguistic Validation of the M. D. Anderson Symptom Inventory in Persian-Speaking Iranian Cancer Patients.

PubMed

Saadatpour, Leila; Hemati, Simin; Habibi, Farzaneh; Behzadi, Erfan; Hashemi-Jazi, Marsa Sadat; Kheirabadi, Gholamreza; Mirbagher, Leila; Gholamrezaei, Ali

2015-09-01

Various symptoms frequently affect cancer patients' quality of life. Appropriate assessment of these symptoms provides valuable data for cancer management. This study aimed to validate the Persian version of the M. D. Anderson Symptom Inventory (MDASI-P). This cross-sectional study was conducted at four cancer treatment centers in two cities in Iran. Breast cancer and colorectal cancer patients aged 18 years and older were consecutively included in the study. The standard forward-backward translation method was applied. Patients completed the MDASI-P along with the previously validated Persian version of the European Organization for Research and Treatment of Cancer Quality of Life Questionnaire-Core 30 (EORTC QLQ-C30). Construct validity (factor analysis), criterion validity (against the EORTC QLQ-C30), and reliability (Cronbach's alpha) were analyzed. A total of 146 breast cancer and 94 colorectal cancer patients were studied. Factor analysis for the symptom severity items resulted in a three-factor solution, further reduced to a two-factor solution: general symptoms and gastrointestinal symptoms. Correlation of the MDASI-P symptom severity items with corresponding EORTC QLQ-C30 symptom items (r = 0.48-0.75) and MDASI-P interference items with corresponding EORTC QLQ-C30 functioning domains (r = -0.46 to -0.23) supported the criterion validity. Cronbach's alpha was 0.90, 0.88, and 0.77 for the total questionnaire, symptom severity items, and the interference subscale, respectively. The MDASI-P is a feasible, valid, and reliable instrument for evaluation of symptoms in Persian-speaking cancer patients and can be used to improve symptom management in these patients. Copyright © 2015 American Academy of Hospice and Palliative Medicine. Published by Elsevier Inc. All rights reserved.

Development, validation and psychometric properties of a diagnostic/prognostic tool for breakthrough pain in mixed chronic-pain patients.

PubMed

Samolsky Dekel, Boaz Gedaliahu; Remondini, Francesca; Gori, Alberto; Vasarri, Alessio; Di Nino, GianFranco; Melotti, Rita Maria

2016-02-01

Breakthrough pain (BTP) shows variable prevalence in different clinical contexts of cancer and non-cancer patients. BTP diagnostic tools with demonstrated reliability, validation and prognostic capability are lacking. We report the development, psychometric and validation properties of a diagnostic/prognostic tool, the IQ-BTP, for BTP recognition, its likelihood and clinical features among chronic-pain (CP) patients. n=120 consecutive mixed cancer/non-cancer CP in/outpatients. Development, psychometric analyses and formal validation included: Face/Content validity (by 'experts' opinion and assessing the relationship between the IQ-BTP classes and criteria derived from BTP operational-case-definition); Construct validity, by Principle Component Analysis (PCA); and the strength of Spearman correlation between IQ-BTP classes and the Brief Pain Inventory (BPI) items; Reliability, by Cronbach's alpha statistics. Associations with clinical/demographic moderators were assessed applying χ(2) analysis. Potential-BTP was found in 36.7% of patients (38.4% of non-cancer and 32.4% of cancer patients). Among these the likelihood for BTP diagnosis was 'high' in 25%, 'intermediate' in 41% and, 'low' 34% of patients. Analyses showed significant differences between IQ-BTP classes and between the latter BPI pain-item scores. Correlation between IQ-BTP classes and BPI items was moderate. PCA and scree test identified 3 components accounting for 62.3% of the variance. Cronbach's alpha was 0.71. The IQ-BTP showed satisfactory psychometric and validation properties. With adequate feasibility it enabled the allocating of cancer/non-cancer CP patients in three prognostic classes. Results are sufficient to warrant a subsequent impact study of the IQ-BTP as prognostic model and screening tool for BTP in both CP populations. Copyright © 2016 Elsevier B.V. All rights reserved.

Toward extending the educational interpreter performance assessment to cued speech.

PubMed

Krause, Jean C; Kegl, Judy A; Schick, Brenda

2008-01-01

The Educational Interpreter Performance Assessment (EIPA) is as an important research tool for examining the quality of interpreters who use American Sign Language or a sign system in classroom settings, but it is not currently applicable to educational interpreters who use Cued Speech (CS). In order to determine the feasibility of extending the EIPA to include CS, a pilot EIPA test was developed and administered to 24 educational CS interpreters. Fifteen of the interpreters' performances were evaluated two to three times in order to assess reliability. Results show that the instrument has good construct validity and test-retest reliability. Although more interrater reliability data are needed, intrarater reliability was quite high (0.9), suggesting that the pilot test can be rated as reliably as signing versions of the EIPA. Notably, only 48% of interpreters who formally participated in pilot testing performed at a level that could be considered minimally acceptable. In light of similar performance levels previously reported for interpreters who sign (e.g., Schick, Williams, & Kupermintz, 2006), these results suggest that interpreting services for deaf and hard-of hearing students, regardless of the communication option used, are often inadequate and could seriously hinder access to the classroom environment.

Toward Extending the Educational Interpreter Performance Assessment to Cued Speech

PubMed Central

Krause, Jean C.; Kegl, Judy A.; Schick, Brenda

2008-01-01

The Educational Interpreter Performance Assessment (EIPA) is as an important research tool for examining the quality of interpreters who use American Sign Language or a sign system in classroom settings, but it is not currently applicable to educational interpreters who use Cued Speech (CS). In order to determine the feasibility of extending the EIPA to include CS, a pilot EIPA test was developed and administered to 24 educational CS interpreters. Fifteen of the interpreters’ performances were evaluated two to three times in order to assess reliability. Results show that the instrument has good construct validity and test–retest reliability. Although more interrater reliability data are needed, intrarater reliability was quite high (0.9), suggesting that the pilot test can be rated as reliably as signing versions of the EIPA. Notably, only 48% of interpreters who formally participated in pilot testing performed at a level that could be considered minimally acceptable. In light of similar performance levels previously reported for interpreters who sign (e.g., Schick, Williams, & Kupermintz, 2006), these results suggest that interpreting services for deaf and hard-of hearing students, regardless of the communication option used, are often inadequate and could seriously hinder access to the classroom environment. PMID:18042791

Evaluation of a welfare assessment tool to examine practices for preventing, recognizing, and managing pain at companion-animal veterinary clinics

PubMed Central

Dawson, Lauren C.; Dewey, Cate E.; Stone, Elizabeth A.; Mosley, Cornelia I.; Guerin, Michele T.; Niel, Lee

2017-01-01

Successful prevention, recognition, and treatment of pain are integral to ensuring veterinary patient welfare. A canine and feline welfare assessment tool, incorporating verbal interviews with veterinarians using open-ended questions, was developed to assess pain management practices that safeguard and improve patient welfare. The tool was evaluated in 30 companion- and mixed-animal veterinary clinics in Ontario in order to assess its reliability, feasibility, and validity, while also benchmarking current practices. Responses were analyzed according to a scoring scheme developed based on published literature and expert opinion. Based on weighted kappa statistics, interview scoring had substantial inter-observer (Kw = 0.83, 0.73) and near-perfect intra-observer (Kw = 0.92) agreement, which suggests that the tool reliably collects information about pain management practices. Interviews were completed at all recruited clinics, which indicates high feasibility for the methods. Validity could not be assessed, as participants were reluctant to share information about analgesic administration from their clinical records. Descriptive results indicated areas for which many veterinarians are acting in accordance with best practices for pain management, such as pre-emptive and post-surgical analgesia for ovariohysterectomy patients, and post-surgical care instructions. Areas that offer opportunity for enhancement were also highlighted, e.g., training veterinary staff to recognize signs of pain and duration of analgesia in ovariohysterectomy patients after discharge. Overall, based on this limited sample, most veterinarians appear to be effectively managing their patients’ pain, although areas with opportunity for enhancement were also identified. Further research is needed to assess trends in a broader sample of participants. PMID:29081584

Development of the knee osteoarthritis patient education questionnaire: a new measure for evaluating preoperative patient education programmes for patients undergoing total knee replacement.

PubMed

Huber, Erika O; Bastiaenen, Caroline H; Bischoff-Ferrari, Heike A; Meichtry, André; de Bie, Rob A

2015-01-01

The aim of this study was to develop a generic instrument for the use of patients, named the Knee Osteoarthrtis Patient Education Questionnaire (KOPEQ), to assess the validity of a preoperative educational intervention and to make a preliminary test of its psychometric properties. A patient-reported outcome instrument was designed, using the conceptual framework of Wilson and Cleary as a methodological guide. Likert items with a five-point scale were chosen for the scoring option. The feasibility and interpretability of administering the KOPEQ was tested through conducting interviews with targeted patients. Items of the KOPEQ were linked to the International Classification of Functioning, Disability and Health (ICF). Psychometric testing contained internal consistency for reliability, and factor analysis for validity properties. A final list of 16 items was derived and linked to the ICF. Targeted patients confirmed in interviews, that all 16 questions were highly understandable and that the length of the questionnaire was feasible and acceptable. There was a good internal consistency for the 16-item KOPEQ with a Cronbach's alpha of 0.83 (95% confidence interval 0.71-0.94). Sixty-one percent of the variance was explained by a four-factor model and the factors were named "didactics", "addressability", "empowerment" and "theory". Results of a factor analysis provided a loading of the separate items between 0.469 and 0.958. The KOPEQ can help to provide health professionals with reliable feedback on how patients assessed the applied patient education intervention. Interviews with patients and a factor analysis revealed new and important insight.

The Job Accommodation Scale (JAS): psychometric evaluation of a new measure of employer support for temporary job modifications.

PubMed

Shaw, William S; Kristman, Vicki L; Williams-Whitt, Kelly; Soklaridis, Sophie; Huang, Yueng-Hsiang; Côté, Pierre; Loisel, Patrick

2014-12-01

An employer offer of temporary job modification is a key strategy for facilitating return-to-work for musculoskeletal conditions, but there are no validated scales to assess the level of support for temporary job modifications across a range of job types and organizations. To pilot test a new 21-item self-report measure [the Job Accommodation Scale (JAS)] to assess its applicability, internal consistency, factor structure, and relation to physical job demands. Supervisors (N = 804, 72.8 % male, mean age = 46) were recruited from 19 employment settings in the USA and Canada and completed a 30-min online survey regarding job modification practices. As part of the survey, supervisors nominated and described a job position they supervised and completed the JAS for a hypothetical worker (in that position) with an episode of low back pain. Job characteristics were derived from the occupational informational network job classification database. The full response range (1-4) was utilized on all 21 items, with no ceiling or floor effects. Avoiding awkward postures was the most feasible accommodation and moving the employee to a different site or location was the least feasible. An exploratory factor analysis suggested five underlying factors (Modify physical workload; Modify work environment; Modify work schedule; Find alternate work; and Arrange for assistance), and there was an acceptable goodness-of-fit for the five parceled sub-factor scores as a single latent construct in a measurement model (structural equation model). Job accommodations were less feasible for more physical jobs and for heavier industries. The pilot administration of the JAS with respect to a hypothetical worker with low back pain showed initial support for its applicability, reliability, and validity when administered to supervisors. Future studies should assess its validity for use in actual disability cases, for a range of health conditions, and to assess different stakeholder opinions about the feasibility of job accommodation strategies.

The Job Accommodation Scale (JAS): Psychometric evaluation of a new measure of employer support for temporary job modifications

PubMed Central

Shaw, William S.; Kristman, Vicki L.; Williams-Whitt, Kelly; Soklaridis, Sophie; Huang, Yueng-Hsiang; Côté, Pierre; Loisel, Patrick

2015-01-01

INTRODUCTION An employer offer of temporary job modification is a key strategy for facilitating return-to-work (RTW) for musculoskeletal conditions, but there are no validated scales to assess the level of support for temporary job modifications across a range of job types and organizations. OBJECTIVE To pilot test a new 21-item self-report measure (the Job Accommodation Scale [JAS]) to assess its applicability, internal consistency, factor structure, and relation to physical job demands. METHODS Supervisors (N = 804, 72.8% male, mean age = 46) were recruited from 19 employment settings in the USA and Canada and completed a 30-min online survey regarding job modification practices. As part of the survey, supervisors nominated and described a job position they supervised and completed the JAS for a hypothetical worker (in that position) with an episode of low back pain. Job characteristics were derived from the occupational informational network job classification database. RESULTS The full response range (1–4) was utilized on all 21 items, with no ceiling or floor effects. Avoiding awkward postures was the most feasible accommodation and moving the employee to a different site or location was the least feasible. An exploratory factor analysis suggested five underlying factors (Modify physical workload; Modify work environment; Modify work schedule; Find alternate work; and Arrange for assistance), and there was an acceptable goodness-of-fit for the five parceled sub-factor scores as a single latent construct in a measurement model (structural equation model). Job accommodations were less feasible for more physical jobs and for heavier industries. CONCLUSIONS The pilot administration of the JAS with respect to a hypothetical worker with LBP showed initial support for its applicability, reliability, and validity when administered to supervisors. Future studies should assess its validity for use in actual disability cases, for a range of health conditions, and to assess different stakeholder opinions about the feasibility of job accommodation strategies. PMID:24643785

Development and initial cohort validation of the Arthritis Research UK Musculoskeletal Health Questionnaire (MSK-HQ) for use across musculoskeletal care pathways.

PubMed

Hill, Jonathan C; Kang, Sujin; Benedetto, Elena; Myers, Helen; Blackburn, Steven; Smith, Stephanie; Dunn, Kate M; Hay, Elaine; Rees, Jonathan; Beard, David; Glyn-Jones, Sion; Barker, Karen; Ellis, Benjamin; Fitzpatrick, Ray; Price, Andrew

2016-08-05

Current musculoskeletal outcome tools are fragmented across different healthcare settings and conditions. Our objectives were to develop and validate a single musculoskeletal outcome measure for use throughout the pathway and patients with different musculoskeletal conditions: the Arthritis Research UK Musculoskeletal Health Questionnaire (MSK-HQ). A consensus workshop with stakeholders from across the musculoskeletal community, workshops and individual interviews with a broad mix of musculoskeletal patients identified and prioritised outcomes for MSK-HQ inclusion. Initial psychometric validation was conducted in four cohorts from community physiotherapy, and secondary care orthopaedic hip, knee and shoulder clinics. Stakeholders (n=29) included primary care, physiotherapy, orthopaedic and rheumatology patients (n=8); general practitioners, physiotherapists, orthopaedists, rheumatologists and pain specialists (n=7), patient and professional national body representatives (n=10), and researchers (n=4). The four validation cohorts included 570 participants (n=210 physiotherapy, n=150 hip, n=150 knee, n=60 shoulder patients). Outcomes included the MSK-HQ's acceptability, feasibility, comprehension, readability and responder burden. The validation cohort outcomes were the MSK-HQ's completion rate, test-retest reliability and convergent validity with reference standards (EQ-5D-5L, Oxford Hip, Knee, Shoulder Scores, and the Keele MSK-PROM). Musculoskeletal domains prioritised were pain severity, physical function, work interference, social interference, sleep, fatigue, emotional health, physical activity, independence, understanding, confidence to self-manage and overall impact. Patients reported MSK-HQ items to be 'highly relevant' and 'easy to understand'. Completion rates were high (94.2%), with scores normally distributed, and no floor/ceiling effects. Test-retest reliability was excellent, and convergent validity was strong (correlations 0.81-0.88). A new musculoskeletal outcome measure has been developed through a coproduction process with patients to capture prioritised outcomes for use throughout the pathway and with different musculoskeletal conditions. Four validation cohorts found that the MSK-HQ had high completion rates, excellent test-retest reliability and strong convergent validity with reference standards. Further validation studies are ongoing, including a cohort with rheumatoid/inflammatory arthritis. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/

The Counseling Center Assessment of Psychological Symptoms (CCAPS-62): Acceptance, feasibility, and initial psychometric properties in a UK student population.

PubMed

Broglia, Emma; Millings, Abigail; Barkham, Michael

2017-09-01

The burden and severity of student mental health continue to increase in parallel with increasing financial pressures on students and services alike. There is a need for a student-specific measure of distress that acknowledges their unique context. This study examined the feasibility, acceptance, and initial psychometric properties of a US measure, the Counseling Center Assessment of Psychological Symptoms (CCAPS), in a UK student sample. A sample of 294 UK help-seeking students from two universities completed the CCAPS-62 and Clinical Outcomes in Routine Evaluation (CORE-10) as a comparator. The factor solution and reliability of the CCAPS-62 were examined. Correlations and clinical boundaries were determined between the CCAPS-62 subscales and CORE-10, and comparisons were made with US published norms. The CCAPS-62 demonstrated a strong factor solution that matched the intended subscales. All subscales had good reliability and correlated significantly with the CORE-10. The agreement on caseness between the two measures was 92.8% with 86.3% reaching clinical threshold on both the CCAPS-62 and CORE-10. Severity was most noticeable for academic distress, depression, anxiety, and social anxiety. Compared to US data, UK students showed higher clinical severity for all psychological symptoms. The CCAPS-62 is a reliable and psychometrically valid assessment measure to use with UK students without revision. The overall distress indicated is similar to that of the CORE-10, but the individual subscales are more informative of specific student concerns including academic distress, social anxiety, and substance abuse. Potential benefits of administering a student-focused assessment measure in student counselling services are discussed. University students attending counselling in the UK demonstrate clinical severity for academic distress, depression, anxiety, and social anxiety. Compared to university students in the US, UK students present with higher clinical severity on all contextual measures of student psychological distress. It is advantageous for university counselling services to administer a student-specific clinical measure over measures intended for the general clinical population. CCAPS-62 is an acceptable, feasible, and psychometrically valid measure of student psychological distress that can be used in the UK without revision. It is important for university counselling services to continue to provide support from therapists that are trained and experienced in the university context over services intended for the general clinical population. Copyright © 2017 John Wiley & Sons, Ltd.

The feasibility of measuring and monitoring social determinants of health and the relevance for policy and programme – a qualitative assessment of four countries

PubMed Central

Blas, Erik; Ataguba, John E.; Huda, Tanvir M.; Bao, Giang Kim; Rasella, Davide; Gerecke, Megan R.

2016-01-01

Background Since the publication of the reports by the Commission on Social Determinants of Health (CSDH), many research papers have documented inequities, explaining causal pathways in order to inform policy and programmatic decision-making. At the international level, the sustainable development goals (SDGs) reflect an attempt to bring together these themes and the complexities involved in defining a comprehensive development framework. However, to date, much less has been done to address the monitoring challenges, that is, how data generation, analysis and use are to become routine tasks. Objective To test proposed indicators of social determinants of health (SDH), gender, equity, and human rights with respect to their relevance in tracking progress in universal health coverage and population health (level and distribution). Design In an attempt to explore these monitoring challenges, indicators covering a wide range of social determinants were tested in four country case studies (Bangladesh, Brazil, South Africa, and Vietnam) for their technical feasibility, reliability, and validity, and their communicability and usefulness to policy-makers. Twelve thematic domains with 20 core indicators covering different aspects of equity, human rights, gender, and SDH were tested through a review of data sources, descriptive analyses, key informant interviews, and focus group discussions. To test the communicability and usefulness of the domains, domain narratives that explained the causal pathways were presented to policy-makers, managers, the media, and civil society leaders. Results For most countries, monitoring is possible, as some data were available for most of the core indicators. However, a qualitative assessment showed that technical feasibility, reliability, and validity varied across indicators and countries. Producing understandable and useful information proved challenging, and particularly so in translating indicator definitions and data into meaningful lay and managerial narratives, and effectively communicating links to health and ways in which the information could improve decision-making. Conclusions This exercise revealed that for monitoring to produce reliable data collection, analysis, and discourse, it will need to be adapted to each national context and institutionalised into national systems. This will require that capacities and resources for this and subsequent communication of results are increased across countries for both national and international monitoring, including the successful implementation of the SDGs. PMID:26853897

Senior Friendly Hospitals: Development and Application of Criteria: A Descriptive Study.

PubMed

Rashmi, M R; Kasthuri, Arvind; Rodrigues, Rashmi J

2016-01-01

The world's population is rapidly aging. Between 2000 and 2050, the proportion of the world's population over the age of 60 will double from about 11% to 22%; more so in the developing countries. The prevalence of frailty and morbidity among the elderly is high. There is a need to assess the "preparedness" of the health care system including hospitals to respond to the needs of the elderly. 1. To develop criteria for a senior friendly hospital and 2. To assess the feasibility of application of these criteria. A descriptive study was done at Bangalore, India, involving 100 subjects sampled by purposive sampling. Study population consisted of senior citizens, their caretakers, physicians, hospital support staff, nurses, geriatricians, hospital administrators, and architects. They were interviewed using a validated translated interview schedule. The study consisted of two phases; Phase 1: Developing a checklist to assess senior friendliness of a hospital by using modified Delphi technique. Phase 2: Application of the checklist thus developed to selected hospitals in order to assess the feasibility of administration. The data was then analyzed using Statistical Package for the Social Sciences (SPSS) for frequencies, proportions, central tendency and dispersion, interclass reliability, intraclass reliability, and Cronbach's alpha. A checklist containing 44 items to assess the senior friendliness of a hospital was developed. The checklist was found feasible and easy to administer. The checklist thus developed to assess senior friendliness of a hospital has wider application as it has a potential to be considered for framing senior friendly hospital guidelines/policies.

[Reliability and validity of Parkinson's disease sleep scale-Chinese version in the south west of China].

PubMed

Zhang, J H; Peng, R; Du, Y; Mou, Y; Li, N N; Cheng, L

2016-11-08

Objective: To evaluate the reliability and validity of Parkinson's disease sleep scale-Chinese version (CPDSS) through a study of a large PD population in southwest China, and to explore the prevalence and characteristics of sleep disorders in Parkinson's disease (PD) patients from southwest China. Methods: A total of 544 PD patients and 220 control subjects were enrolled in our study. Demographic data, CPDSS, ESS, PDQ39, HAMD and H-Y stage were assessed in all subjects. Statistical description, Cronbach's alpha coefficient, intra-class correlation coefficient ( ICC ), Spearman rank correlation coefficient and Mann-Whitney U test were used for statistical analyses. Result: The Cronbach's alpha coefficient for CPDSS was 0.79, ICC of the total scale was 0.94 and ICC of each item ranged from 0.73 to 0.97. The factor analysis yielded a five-factor solution, which explained 63.4% of the total variance. Total and each item scores of CPDSS in PD patients were lower than those in healthy controls. 69.3% of PD patients had sleep disorder, while prevalence in the control group was only 29.6%. Negative correlation was found between CPDSS and ESS. Daytime sleepiness was the most common factor (35.9%) leading to sleep disorders. The sleep disorders of PD patients in Southwest China were significantly related with the course of disease, the severity of disease, the quality of life, depression, cognitive level and motor symptoms. Conclusion: CPDSS has good feasibility, reliability and validity in PD population from southwest China. CPDSS is considered as an effective tool for the assessment of sleep disorder in PD patients.

A mobile monitoring system of blood pressure for underserved in China by information and communication technology service.

PubMed

Jiang, Jiehui; Yan, Zhuangzhi; Kandachar, Prabhu; Freudenthal, Adinda

2010-05-01

High blood pressure (BP, hypertension) is a leading chronic condition in China and has become the main risk factor for many high-risk diseases, such as heart attacks. However, the platform for chronic disease measurement and management is still lacking, especially for underserved Chinese. To achieve the early diagnosis of hypertension, one BP monitoring system has been designed. The proposed design consists of three main parts: user domain, server domain, and channel domain. All three units and their materialization, validation tests on reliability, and usability are described in this paper, and the conclusion is that the current design concept is feasible and the system can be developed toward sufficient reliability and affordability with further optimization. This idea might also be extended into one platform for other physiological signals, such as blood sugar and ECG.

Validity and Feasibility of a Digital Diet Estimation Method for Use with Preschool Children: A Pilot Study

ERIC Educational Resources Information Center

Nicklas, Theresa A.; O'Neil, Carol E.; Stuff, Janice; Goodell, Lora Suzanne; Liu, Yan; Martin, Corby K.

2012-01-01

Objective: The goal of the study was to assess the validity and feasibility of a digital diet estimation method for use with preschool children in "Head Start." Methods: Preschool children and their caregivers participated in validation (n = 22) and feasibility (n = 24) pilot studies. Validity was determined in the metabolic research unit using…

Detecting Initiation or Risk for Initiation of Substance Use before High School during Pediatric Well-Child Check-Ups

PubMed Central

Ridenour, Ty A.; Willis, David; Bogen, Debra L.; Novak, Scott; Scherer, Jennifer; Reynolds, Maureen D.; Zhai, Zu Wei; Tarter, Ralph E.

2015-01-01

Background Youth substance use (SU) is prevalent and costly, affecting mental and physical health. American Academy of Pediatrics and Affordable Care Act call for SU screening and prevention. The Youth Risk Index© (YRI) was tested as a screening tool for having initiated and propensity to initiate SU before high school (which forecasts SU disorder). YRI was hypothesized to have good to excellent psychometrics, feasibility and stakeholder acceptability for use during well-child check-ups. Design A high-risk longitudinal design with two cross-sectional replication samples, ages 9–13 was used. Analyses included receiver operating characteristics and regression analyses. Participants A one-year longitudinal sample (N=640) was used for YRI derivation. Replication samples were a cross-sectional sample (N=345) and well-child check-up patients (N=105) for testing feasibility, validity and acceptability as a screening tool. Results YRI has excellent test-retest reliability and good sensitivity and specificity for concurrent and one-year-later SU (odds ratio=7.44 CI=4.3–13.0) and conduct problems (odds ratios=7.33 CI=3.9–13.7). Results were replicated in both cross-sectional samples. Well-child patients, parents and pediatric staff rated YRI screening as important, acceptable, and a needed service. Conclusions Identifying at-risk youth prior to age 13 could reap years of opportunity to intervene before onset of SU disorder. Most results pertained to YRI’s association with concurrent or recent past risky behaviors; further replication ought to specify its predictive validity, especially adolescent-onset risky behaviors. YRI well identifies youth at risk for SU and conduct problems prior to high school, is feasible and valid for screening during well-child check-ups, and is acceptable to stakeholders. PMID:25765481

Nutritional status and nutrition risk screening in hospitalized children in New Zealand.

PubMed

Moeeni, Vesal; Walls, Tony; Day, Andrew S

2013-09-01

Children requiring hospitalization are at risk of malnutrition. This study aimed to define the nutritional status of paediatric inpatients in comparison with healthy children and to compare and contrast the feasibility and validity of three nutritional risk screening (NRS) tools in the hospitalized children. A total of 162 children admitted to Christchurch Hospital were assessed along with a similar group of healthy children. Their nutritional state was assessed and classified using standard criteria. The NRS tools were applied, and patients were classified into low-, medium- and high-risk groups. The feasibility and validity of the tools were assessed. Under-nutrition was more frequent in the inpatient group (9.9% vs. 3.7%; p = 0.04), whereas both groups had similar rates of overweight/obesity. NRS tools were able to identify between 81% and 100% of the malnourished patients in the medium- to high-risk groups. Undernourished patients had longer hospital stay than well-nourished patients. Hospitalized children have higher rates of under-nutrition than healthy children in NZ. The three NRS tools were able to identify children at nutritional risk with differing utility. In this setting, STRONGkids was the most reliable tool. ©2013 Foundation Acta Paediatrica. Published by John Wiley & Sons Ltd.

Development and Validation of Two Instruments Measuring Intrinsic, Extraneous, and Germane Cognitive Load

PubMed Central

Klepsch, Melina; Schmitz, Florian; Seufert, Tina

2017-01-01

Cognitive Load Theory is one of the most powerful research frameworks in educational research. Beside theoretical discussions about the conceptual parts of cognitive load, the main challenge within this framework is that there is still no measurement instrument for the different aspects of cognitive load, namely intrinsic, extraneous, and germane cognitive load. Hence, the goal of this paper is to develop a differentiated measurement of cognitive load. In Study 1 (N = 97), we developed and analyzed two strategies to measure cognitive load in a differentiated way: (1) Informed rating: We trained learners in differentiating the concepts of cognitive load, so that they could rate them in an informed way. They were asked then to rate 24 different learning situations or learning materials related to either high or low intrinsic, extraneous, or germane load. (2) Naïve rating: For this type of rating of cognitive load we developed a questionnaire with two to three items for each type of load. With this questionnaire, the same learning situations had to be rated. In the second study (N = between 65 and 95 for each task), we improved the instrument for the naïve rating. For each study, we analyzed whether the instruments are reliable and valid, for Study 1, we also checked for comparability of the two measurement strategies. In Study 2, we conducted a simultaneous scenario based factor analysis. The informed rating seems to be a promising strategy to assess the different aspects of cognitive load, but it seems not economic and feasible for larger studies and a standardized training would be necessary. The improved version of the naïve rating turned out to be a useful, feasible, and reliable instrument. Ongoing studies analyze the conceptual validity of this measurement with up to now promising results. PMID:29201011

Translation, cross-cultural adaptation and validation of the Italian version of the Nottingham Clavicle Score (NCS).

PubMed

Vascellari, Alberto; Schiavetti, Stefano; Rebuzzi, Enrico; Coletti, Nicolò

2015-11-01

The Nottingham Clavicle Score (NCS) is a specific Patient Reported Outcome Measure of injuries to the clavicle, acromio-clavicular joint (ACJ) and sterno-clavicular joint. The purpose of this study was to translate the NCS into Italian and establish its cultural adaptiveness and validity. The original version of the NCS was translated into Italian in accordance with the cross-cultural adaptation guidelines described by Guillemin. Sixty-six patients [average age 45.7 years (SD 11.3)] who had received surgical treatment for injuries of the ACJ and the clavicle were included in the study. The study population completed the NCS twice within 5 days, the Oxford Shoulder Score (OSS), the Disability of the Arm, Shoulder and Hand (DASH) questionnaire and the short-form 36 (SF-36). Statistical tests assessed the construct validity, discriminant validity, internal consistency, reliability and feasibility of the NCS. The translation and adaptation of the NCS for an Italian context required no major cultural adaptation. Internal consistency was high (Cronbach's α, 0.86). Test-retest reproducibility was excellent (ρ = 0.981, p < 0.00001). Administration time was 45 s (range 1 min 32 s-8 min), and all items were answered. The Italian NCS showed strong correlation with the DASH (-0.87), the OSS (-0.84) and those subscales of the SF-36 (physical functioning, role physical and bodily pain) which aim to measure similar constructs. The Italian NCS scale is a reliable, valid, consistent shoulder assessment form that can be used to assess the functional limitations of patients with injuries of clavicle or ACJ. III.

PILATES (Physical Activity and Diet Survey): An Italian Self-Administered Questionnaire Evaluating Diet Habits of Gym-Goers. Validation Process.

PubMed

Gianfredi, Vincenza; Nucci, Daniele; Ceccarelli, Francesco; Villarini, Milena; Moretti, Massimo

2018-04-19

PILATES study aims are to assess the main characteristics related to food habits, diet behaviors, and nutrition knowledge and how and where gym-goers get information on dietary supplementation. We present evidence for the reliability, feasibility, and construct validity of the PILATES questionnaire. Cohen's kappa statistic (k) for dichotomous variables was used to assess the agreement between the two administrations (interrater agreement). The nutrient composition and energy of food were derived from the Food Composition Database for Epidemiological Studies in Italy (Banca Dati di Composizione degli Alimenti per Studi Epidemiologici in Italia - BDA). Because of energy intake, waist circumference and weight are continuous variables; we calculate the agreement between the two interviews using Spearman's rho coefficient (nonparametric measure of rank correlation). An Italian 21-item self-administered questionnaire was designed and pretested on 28 students in food science and human nutrition at the University of Perugia who were enrolled on a voluntary basis. After verifying the feasibility of the questionnaire, pretest evaluation showed a generally high concordance with an 87.32% of agreement and k value = 0.71 ± 0.23. In addition, weight, daily energy intake, and waist circumference confirmed a statistically significant agreement (p <.001, Spearman rho coefficient). PILATES questionnaire is a valid tool to estimate dietary intakes in a fairly simple, cost-effective, and timesaving manner.

Performance indicators for public mental healthcare: a systematic international inventory

PubMed Central

2012-01-01

Background The development and use of performance indicators (PI) in the field of public mental health care (PMHC) has increased rapidly in the last decade. To gain insight in the current state of PI for PMHC in nations and regions around the world, we conducted a structured review of publications in scientific peer-reviewed journals supplemented by a systematic inventory of PI published in policy documents by (non-) governmental organizations. Methods Publications on PI for PMHC were identified through database- and internet searches. Final selection was based on review of the full content of the publications. Publications were ordered by nation or region and chronologically. Individual PI were classified by development method, assessment level, care domain, performance dimension, diagnostic focus, and data source. Finally, the evidence on feasibility, data reliability, and content-, criterion-, and construct validity of the PI was evaluated. Results A total of 106 publications were included in the sample. The majority of the publications (n = 65) were peer-reviewed journal articles and 66 publications specifically dealt with performance of PMHC in the United States. The objectives of performance measurement vary widely from internal quality improvement to increasing transparency and accountability. The characteristics of 1480 unique PI were assessed. The majority of PI is based on stakeholder opinion, assesses care processes, is not specific to any diagnostic group, and utilizes administrative data sources. The targeted quality dimensions varied widely across and within nations depending on local professional or political definitions and interests. For all PI some evidence for the content validity and feasibility has been established. Data reliability, criterion- and construct validity have rarely been assessed. Only 18 publications on criterion validity were included. These show significant associations in the expected direction on the majority of PI, but mixed results on a noteworthy number of others. Conclusions PI have been developed for a broad range of care levels, domains, and quality dimensions of PMHC. To ensure their usefulness for the measurement of PMHC performance and advancement of transparency, accountability and quality improvement in PMHC, future research should focus on assessment of the psychometric properties of PI. PMID:22433251

Intrinsic and extrinsic motivation and intention to breast-feed.

PubMed

Wells, Kristen J; Thompson, Nancy J; Kloeblen-Tarver, Amy S

2002-01-01

To examine the feasibility of using the cognitive evaluation theory to examine pregnant women's intention to breast-feed. A questionnaire designed to measure intrinsic and extrinsic motivation was administered to 228 pregnant women. Results provide evidence for reliability and validity of the revised instrument in this population. A factor analysis suggests the instrument measures 2 types of intrinsic motivation, one type of extrinsic motivation, and motivation related to the baby. The instrument distinguished differences in motivation between women who intend to breast-feed and those who intend to formula feed. This study helps elucidate motivational factors involved in infant-feeding decisions.

Multi-viewpoint clustering analysis

NASA Technical Reports Server (NTRS)

Mehrotra, Mala; Wild, Chris

1993-01-01

In this paper, we address the feasibility of partitioning rule-based systems into a number of meaningful units to enhance the comprehensibility, maintainability and reliability of expert systems software. Preliminary results have shown that no single structuring principle or abstraction hierarchy is sufficient to understand complex knowledge bases. We therefore propose the Multi View Point - Clustering Analysis (MVP-CA) methodology to provide multiple views of the same expert system. We present the results of using this approach to partition a deployed knowledge-based system that navigates the Space Shuttle's entry. We also discuss the impact of this approach on verification and validation of knowledge-based systems.

Dutch translation and validation of the readiness for interprofessional learning scale (RIPLS) in a primary healthcare context.

PubMed

Pype, Peter; Deveugele, Myriam

2016-12-01

Interprofessional education and collaborative practice are gradually gaining importance in the context of growing healthcare complexity. The readiness for interprofessional learning scale (RIPLS) is a well-known scale that can identify attitudinal barriers and variance across professions, which may affect educational interventions. This study aims to translate the English RIPLS into Dutch and to test its reliability and validity. The scale was translated and back-translated by two pairs of people independently and tested for feasibility and comprehensibility. The translated scale was used with 219 general practitioners, 238 community nurses, and 53 palliative home-care nurses. Exploratory factor analysis was used to assess construct validity. Confirmatory factor analysis was done to generate a fit model. Cronbach's alpha was computed to evaluate internal consistency. Regression analysis was used to evaluate the effect of the RIPLS score on the level of learning through collaboration and to gauge the influence of the participants' gender, age, previous palliative care education, type of practice and years in practice. Confirmatory and exploratory factor analysis confirms the factor structure of the original version. The Dutch version shows good reliability (overall Cronbach's alpha: 0.88; intraclass correlation coefficient after test-retest: 0.718 (95%CI: 0.499-0.852). The RIPLS score correlates with the amount of workplace learning during collaboration (discriminant validity: P < 0.001). The Dutch translation of the RIPLS is now ready for comparative studies.

Prediction of Multidimensional Fatigue After Childhood Brain Injury.

PubMed

Crichton, Alison J; Babl, Franz; Oakley, Ed; Greenham, Mardee; Hearps, Stephen; Delzoppo, Carmel; Hutchison, Jamie; Beauchamp, Miriam; Anderson, Vicki A

To determine (1) the presence of fatigue symptoms and predictors of fatigue after childhood brain injury and examine (2) the feasibility, reliability, and validity of a multidimensional fatigue measure (PedsQL Multidimensional Fatigue Scale [MFS]) obtained from parent and child perspectives. Emergency and intensive care units of a hospital in Melbourne, Australia. Thirty-five families (34 parent-proxies and 32 children) aged 8 to 18 years (mean child age = 13.29 years) with traumatic brain injury (TBI) of all severities (27 mild, 5 moderate, and 3 severe) admitted to the Royal Children's Hospital. Longitudinal prospective study. Fatigue data collected at 6-week follow-up (mean = 6.9 weeks). Postinjury child- and parent-rated fatigue (PedsQL MFS), mood, sleep, and pain based on questionnaire report: TBI severity (mild vs moderate/severe TBI). A score greater than 2 standard deviations below healthy control data indicated the presence of abnormal fatigue, rates of which were higher compared with normative data for both parent and child reports (47% and 29%). Fatigue was predicted by postinjury depression and sleep disturbance for parent, but not child ratings. Fatigue, as rated by children, was not significantly predicted by TBI severity or other symptoms. The PedsQL MFS demonstrated acceptable measurement properties in child TBI participants, evidenced by good feasibility and reliability (Cronbach α values >0.90). Interrater reliability between parent and child reports was poor to moderate. Results underscore the need to assess fatigue and associated sleep-wake disturbance and depression after child TBI from both parent and child perspectives.

Specialty-specific multi-source feedback: assuring validity, informing training.

PubMed

Davies, Helena; Archer, Julian; Bateman, Adrian; Dewar, Sandra; Crossley, Jim; Grant, Janet; Southgate, Lesley

2008-10-01

The white paper 'Trust, Assurance and Safety: the Regulation of Health Professionals in the 21st Century' proposes a single, generic multi-source feedback (MSF) instrument in the UK. Multi-source feedback was proposed as part of the assessment programme for Year 1 specialty training in histopathology. An existing instrument was modified following blueprinting against the histopathology curriculum to establish content validity. Trainees were also assessed using an objective structured practical examination (OSPE). Factor analysis and correlation between trainees' OSPE performance and the MSF were used to explore validity. All 92 trainees participated and the assessor response rate was 93%. Reliability was acceptable with eight assessors (95% confidence interval 0.38). Factor analysis revealed two factors: 'generic' and 'histopathology'. Pearson correlation of MSF scores with OSPE performances was 0.48 (P = 0.001) and the histopathology factor correlated more highly (histopathology r = 0.54, generic r = 0.42; t = - 2.76, d.f. = 89, P < 0.01). Trainees scored least highly in relation to ability to use histopathology to solve clinical problems (mean = 4.39) and provision of good reports (mean = 4.39). Three of six doctors whose means were < 4.0 received free text comments about report writing. There were 83 forms with aggregate scores of < 4. Of these, 19.2% included comments about report writing. Specialty-specific MSF is feasible and achieves satisfactory reliability. The higher correlation of the 'histopathology' factor with the OSPE supports validity. This paper highlights the importance of validating an MSF instrument within the specialty-specific context as, in addition to assuring content validity, the PATH-SPRAT (Histopathology-Sheffield Peer Review Assessment Tool) also demonstrates the potential to inform training as part of a quality improvement model.

Experimental feasibility study of estimation of the normalized central blood pressure waveform from radial photoplethysmogram.

PubMed

Zahedi, Edmond; Sohani, Vahid; Ali, M A Mohd; Chellappan, Kalaivani; Beng, Gan Kok

2015-01-01

The feasibility of a novel system to reliably estimate the normalized central blood pressure (CBPN) from the radial photoplethysmogram (PPG) is investigated. Right-wrist radial blood pressure and left-wrist PPG were simultaneously recorded in five different days. An industry-standard applanation tonometer was employed for recording radial blood pressure. The CBP waveform was amplitude-normalized to determine CBPN. A total of fifteen second-order autoregressive models with exogenous input were investigated using system identification techniques. Among these 15 models, the model producing the lowest coefficient of variation (CV) of the fitness during the five days was selected as the reference model. Results show that the proposed model is able to faithfully reproduce CBPN (mean fitness = 85.2% ± 2.5%) from the radial PPG for all 15 segments during the five recording days. The low CV value of 3.35% suggests a stable model valid for different recording days.

Feasibility study of self-powered magnetorheological damper systems

NASA Astrophysics Data System (ADS)

Chen, Chao; Liao, Wei-Hsin

2012-04-01

This paper is aimed to provide a feasibility study of self-powered magnetorheological (MR) damper systems, which could convert vibration and shock energy into electrical energy to power itself under control. The self-powered feature could bring merits such as higher reliability, energy saving, and less maintenance for the MR damper systems. A self-powered MR damper system is proposed and modeled. The criterion whether the MR damper system is self-powered or not is proposed. A prototype of MR damper with power generation is designed, fabricated, and tested. The modeling of this damper is experimentally validated. Then the damper is applied to a 2 DOF suspension system under on-off skyhook controller, to obtain the self-powered working range and vibration control performance. Effects of key factors on the self-powered MR damper systems are studied. Design considerations are given in order to increase the self-powered working range.

Use of a single bipolar electrode in the posterior arytenoid muscles for bilateral monitoring of the recurrent laryngeal nerves in thyroid surgery.

PubMed

Haerle, Stephan; Sidler, D; Linder, Th; Mueller, W

2008-12-01

The aims were to assess the technical feasibility of using a single electrode in the posterior arytenoid muscles (PAM) for intraoperative monitoring of the recurrent laryngeal nerve (RLN) in thyroid surgery, to validate the new method against the insertion of electrodes placed in the vocal cord muscle, and to report the results of the clinical application of the new concept. A total of 52 patients were enrolled. The handling and safety of RLN monitoring was tested by simultaneous registration of the EMG response from vocal fold electrodes and PAM electrodes. Acoustically and electromyographically we found nearly the same values for the arytenoid muscles as for the vocal folds, although the signals taken from the vocal folds were slightly stronger. PAM recording using a single bipolar electrode is technically feasible and as reliable compared to the standard vocal cord monitoring.

Measuring the food and built environments in urban centres: reliability and validity of the EURO-PREVOB Community Questionnaire.

PubMed

Pomerleau, J; Knai, C; Foster, C; Rutter, H; Darmon, N; Derflerova Brazdova, Z; Hadziomeragic, A F; Pekcan, G; Pudule, I; Robertson, A; Brunner, E; Suhrcke, M; Gabrijelcic Blenkus, M; Lhotska, L; Maiani, G; Mistura, L; Lobstein, T; Martin, B W; Elinder, L S; Logstrup, S; Racioppi, F; McKee, M

2013-03-01

The authors designed an instrument to measure objectively aspects of the built and food environments in urban areas, the EURO-PREVOB Community Questionnaire, within the EU-funded project 'Tackling the social and economic determinants of nutrition and physical activity for the prevention of obesity across Europe' (EURO-PREVOB). This paper describes its development, reliability, validity, feasibility and relevance to public health and obesity research. The Community Questionnaire is designed to measure key aspects of the food and built environments in urban areas of varying levels of affluence or deprivation, within different countries. The questionnaire assesses (1) the food environment and (2) the built environment. Pilot tests of the EURO-PREVOB Community Questionnaire were conducted in five to 10 purposively sampled urban areas of different socio-economic status in each of Ankara, Brno, Marseille, Riga, and Sarajevo. Inter-rater reliability was compared between two pairs of fieldworkers in each city centre using three methods: inter-observer agreement (IOA), kappa statistics, and intraclass correlation coefficients (ICCs). Data were collected successfully in all five cities. Overall reliability of the EURO-PREVOB Community Questionnaire was excellent (inter-observer agreement (IOA) > 0.87; intraclass correlation coefficients (ICC)s > 0.91 and kappa statistics > 0.7. However, assessment of certain aspects of the quality of the built environment yielded slightly lower IOA coefficients than the quantitative aspects. The EURO-PREVOB Community Questionnaire was found to be a reliable and practical observational tool for measuring differences in community-level data on environmental factors that can impact on dietary intake and physical activity. The next step is to evaluate its predictive power by collecting behavioural and anthropometric data relevant to obesity and its determinants. Copyright © 2013 The Royal Society for Public Health. Published by Elsevier Ltd. All rights reserved.

[Construction of a psychological aging scale for healthy people].

PubMed

Lin, Fei; Long, Yao; Zeng, Ni; Wu, Lei; Huang, Helang

2017-04-28

To construct a psychological aging scale, and to provide a tool and indexes for scientific evaluation on aging.
 Methods: The age-related psychological items were collected through literature screening and expert interview. The importance, feasibilityand the degree of authority for the psychological index system were graded by two rounds of Delphi method. Using analytic hierarchy process, the weight of dimensions and items were determined. The analysis for internal consistency reliability, correlation and exploratory factor was performed to evaluate the reliability and validity of the scales.
 Results: By two rounds of Delphi method, 17 experts offered the results as follows: the coefficient of expert authorities was 0.88±0.06, the coordination coefficients for the importance and feasibility in second round were 0.456 (P<0.01) and 0.666 (P<0.01), respectively. The consistency was good. The psychological aging scale for healthy people included 4 dimensions as follows: cognitive function, emotion, personality and motivation. The weight coefficients for the 4 dimensions were 0.338, 0.250, 0.166 and 0.258, respectively. The Cronbach's α coefficient for the scale was 0.822, the reliability was 0.817, the content validity index (CVI) was 0.847, and the cumulative contribution rate for the 5 factors was51.42%.
 Conclusion: The psychological aging scale is satisfied, which can provide reference for the evaluation for aging. The indicators were representative and well-recognized.

Preliminary validation of the Knee Inflammation MRI Scoring System (KIMRISS) for grading bone marrow lesions in osteoarthritis of the knee: data from the Osteoarthritis Initiative

PubMed Central

Jeffery, Dean; Buller, M; Wichuk, Stephanie; McDougall, Dave; Lambert, Robert GW; Maksymowych, Walter P

2017-01-01

Objective Bone marrow lesions (BML) are an MRI feature of osteoarthritis (OA) offering a potential target for therapy. We developed the Knee Inflammation MRI Scoring System (KIMRISS) to semiquantitatively score BML with high sensitivity to small changes, and compared feasibility, reliability and responsiveness versus the established MRI Osteoarthritis Knee Score (MOAKS). Methods KIMRISS incorporates a web-based graphic overlay to facilitate detailed regional BML scoring. Observers scored BML by MOAKS and KIMRISS on sagittal fluid-sensitive sequences. Exercise 1 focused on interobserver reliability in Osteoarthritis Initiative observational data, with 4 readers (two experienced/two new to KIMRISS) scoring BML in 80 patients (baseline/1 year). Exercise 2 focused on responsiveness in an open-label trial of adalimumab, with 2 experienced readers scoring BML in 16 patients (baseline/12 weeks). Results Scoring time was similar for KIMRISS and MOAKS. Interobserver reliability of KIMRISS was equivalent to MOAKS for BML status (ICC=0.84 vs 0.79), but consistently better than MOAKS for change in BML: Exercise 1 (ICC 0.82 vs 0.53), Exercise 2 (ICC 0.90 vs 0.32), and in new readers (0.87–0.92 vs 0.32–0.51). KIMRISS BML was more responsive than MOAKS BML: post-treatment BML improvement in Exercise 2 reached statistical significance for KIMRISS (SRM −0.69, p=0.015), but not MOAKS (SRM −0.12, p=0.625). KIMRISS BML also more strongly correlated to WOMAC scores than MOAKS BML (r=0.80 vs 0.58, p<0.05). Conclusions KIMRISS BML scoring was highly feasible, and was more reliable for assessment of change and more responsive to change than MOAKS BML for expert and new readers. PMID:28123780

Portfolio assessment during medical internships: How to obtain a reliable and feasible assessment procedure?

PubMed

Michels, Nele R M; Driessen, Erik W; Muijtjens, Arno M M; Van Gaal, Luc F; Bossaert, Leo L; De Winter, Benedicte Y

2009-12-01

A portfolio is used to mentor and assess students' clinical performance at the workplace. However, students and raters often perceive the portfolio as a time-consuming instrument. In this study, we investigated whether assessment during medical internship by a portfolio can combine reliability and feasibility. The domain-oriented reliability of 61 double-rated portfolios was measured, using a generalisability analysis with portfolio tasks and raters as sources of variation in measuring the performance of a student. We obtained reliability (Phi coefficient) of 0.87 with this internship portfolio containing 15 double-rated tasks. The generalisability analysis showed that an acceptable level of reliability (Phi = 0.80) was maintained when the amount of portfolio tasks was decreased to 13 or 9 using one and two raters, respectively. Our study shows that a portfolio can be a reliable method for the assessment of workplace learning. The possibility of reducing the amount of tasks or raters while maintaining a sufficient level of reliability suggests an increase in feasibility of portfolio use for both students and raters.

Assessing the validity and reliability of self-report data on contraception use in the MObile Technology for Improved Family Planning (MOTIF) randomised controlled trial.

PubMed

Smith, Chris; Edwards, Phil; Free, Caroline

2018-03-15

A variety of different approaches to measuring contraceptive use have been used or proposed, either to assess current use or adherence over time, using subjective or objective measures. This paper reports an overview of approaches to measuring adherence to the oral contraceptive, intra-uterine device, sub-dermal implant, and injectable and describes how we assessed contraception use in the MObile Technology for Improved Family Planning (MOTIF) trial in Cambodia. We summarise and discuss advantages and disadvantages of different subjective and objective approaches to measuring adherence to the oral contraceptive, intra-uterine device, sub-dermal implant, and injectable such as self-reports, clinic records, electronic monitoring devices, clinical examination and biomarkers. For the MOTIF trial, we did not consider it feasible to measure objective contraception use as many participants lived a long distance from the clinic and we were concerned whether it was appropriate to ask women to return to clinic for a physical examination simply to verify self-report information already provided. We aimed to assess the validity of the four-month data with 50 participants, calculating the sensitivity and specificity of self-reported data compared with objective measurement. For the 46 valid measurements obtained, the sensitivity and specificity was 100% for self-reported contraception use compared to objective measurement but this study had some limitations. To assess reliability of self-report data we compared calendar data collected on effective contraception use at months 1-4 post-abortion, collected separately at four and 12 months. Agreement ranged from 80 to 84% with a kappa statistic ranging from 0·59 to 0·67 indicating fair to good agreement. There is no perfect method of assessing contraception use and researchers designing future studies should give consideration of what to measure, for example current use or detailed patterns of use over time, and remain mindful of what will be feasible and acceptable to the study population. Although self-reported data on contraception use are considered less reliable, and prone to social desirability bias, it is often the standard approach for contraception research and provides data comparable to previous studies. ClinicalTrials.gov Identifier: NCT01823861 . Registered: March 30, 2013.

The reliability of workplace-based assessment in postgraduate medical education and training: a national evaluation in general practice in the United Kingdom.

PubMed

Murphy, Douglas J; Bruce, David A; Mercer, Stewart W; Eva, Kevin W

2009-05-01

To investigate the reliability and feasibility of six potential workplace-based assessment methods in general practice training: criterion audit, multi-source feedback from clinical and non-clinical colleagues, patient feedback (the CARE Measure), referral letters, significant event analysis, and video analysis of consultations. Performance of GP registrars (trainees) was evaluated with each tool to assess the reliabilities of the tools and feasibility, given raters and number of assessments needed. Participant experience of process determined by questionnaire. 171 GP registrars and their trainers, drawn from nine deaneries (representing all four countries in the UK), participated. The ability of each tool to differentiate between doctors (reliability) was assessed using generalisability theory. Decision studies were then conducted to determine the number of observations required to achieve an acceptably high reliability for "high-stakes assessment" using each instrument. Finally, descriptive statistics were used to summarise participants' ratings of their experience using these tools. Multi-source feedback from colleagues and patient feedback on consultations emerged as the two methods most likely to offer a reliable and feasible opinion of workplace performance. Reliability co-efficients of 0.8 were attainable with 41 CARE Measure patient questionnaires and six clinical and/or five non-clinical colleagues per doctor when assessed on two occasions. For the other four methods tested, 10 or more assessors were required per doctor in order to achieve a reliable assessment, making the feasibility of their use in high-stakes assessment extremely low. Participant feedback did not raise any major concerns regarding the acceptability, feasibility, or educational impact of the tools. The combination of patient and colleague views of doctors' performance, coupled with reliable competence measures, may offer a suitable evidence-base on which to monitor progress and completion of doctors' training in general practice.

Structured assessment of microsurgery skills in the clinical setting.

PubMed

Chan, WoanYi; Niranjan, Niri; Ramakrishnan, Venkat

2010-08-01

Microsurgery is an essential component in plastic surgery training. Competence has become an important issue in current surgical practice and training. The complexity of microsurgery requires detailed assessment and feedback on skills components. This article proposes a method of Structured Assessment of Microsurgery Skills (SAMS) in a clinical setting. Three types of assessment (i.e., modified Global Rating Score, errors list and summative rating) were incorporated to develop the SAMS method. Clinical anastomoses were recorded on videos using a digital microscope system and were rated by three consultants independently and in a blinded fashion. Fifteen clinical cases of microvascular anastomoses performed by trainees and a consultant microsurgeon were assessed using SAMS. The consultant had consistently the highest scores. Construct validity was also demonstrated by improvement of SAMS scores of microsurgery trainees. The overall inter-rater reliability was strong (alpha=0.78). The SAMS method provides both formative and summative assessment of microsurgery skills. It is demonstrated to be a valid, reliable and feasible assessment tool of operating room performance to provide systematic and comprehensive feedback as part of the learning cycle. Copyright 2009 British Association of Plastic, Reconstructive and Aesthetic Surgeons. Published by Elsevier Ltd. All rights reserved.

Development and Validation of the Pediatric Diabetes Routines Questionnaire for Adolescents.

PubMed

Pierce, Jessica S; Jordan, Sara S; Arnau, Randolph C

2018-04-06

This study describes the development and psychometric evaluation of an adolescent self-report version of the Pediatric Diabetes Routines Questionnaire (PDRQ:A), a measure of diabetes-specific routines for youth with type 1 diabetes, and further validation of the parent-version (PDRQ:P) in an adolescent sample. Participants included 120 parent-adolescent dyads (ages 12-17) and an additional 24 parents who completed measures of diabetes-specific adolescent routines, general adolescent routines, diabetes self-care, and family support of youth diabetes care. The PDRQ:P/A demonstrated good internal consistency, test-retest reliability, and parent-child agreement, and adequate validity coefficients. Confirmatory factor analysis supported a one-factor model. Promising results were obtained. The PDRQ:P/A is a clinically feasible parent- and self-report measure that can provide valuable information regarding how frequently adolescents engage in their diabetes management tasks in a consistent manner. Addition of an adolescent report format will enhance the utility of the measure for clinical and research use.

Understanding and Measuring Health Care Insecurity

PubMed Central

Tomsik, Philip E.; Smith, Samantha; Mason, Mary Jane; Zyzanski, Stephen J.; Stange, Kurt C.; Werner, James J.; Flocke, Susan A.

2015-01-01

Purpose To define the concept of “health care insecurity,” validate a new self-report measure, and examine the impact of beginning care at a free clinic on uninsured patients’ health care insecurity. Methods Consecutive new patients presenting at a free clinic completed 15 items assessing domains of health care insecurity (HCI) at their first visit and again four to eight weeks later. Psychometrics and change of the HCI measure were examined. Results The HCI measure was found to have high internal consistency (α=0.94). Evidence of concurrent validity was indicated by negative correlation with VR-12 health-related quality of life physical and mental health components and positive correlation with the Perceived Stress Scale. Predictive validity was shown among the 83% of participants completing follow-up: HCI decreased after beginning care at a free clinic (p<.001). Conclusion Reliably assessing patient experience of health care insecurity is feasible and has potential to inform efforts to improve quality and access to care among underserved populations. PMID:25418245

Development and reliability of the rating of compensatory movements in upper limb prosthesis wearers during work-related tasks.

PubMed

van der Laan, Tallie M J; Postema, Sietke G; Reneman, Michiel F; Bongers, Raoul M; van der Sluis, Corry K

2018-02-10

Reliability study. Quantifying compensatory movements during work-related tasks may help to prevent musculoskeletal complaints in individuals with upper limb absence. (1) To develop a qualitative scoring system for rating compensatory shoulder and trunk movements in upper limb prosthesis wearers during the performance of functional capacity evaluation tests adjusted for use by 1-handed individuals (functional capacity evaluation-one handed [FCE-OH]); (2) to examine the interrater and intrarater reliability of the scoring system; and (3) to assess its feasibility. Movement patterns of 12 videotaped upper limb prosthesis wearers and 20 controls were analyzed. Compensatory movements were defined for each FCE-OH test, and a scoring system was developed, pilot tested, and adjusted. During reliability testing, 18 raters (12 FCE experts and 6 physiotherapists/gait analysts) scored videotapes of upper limb prosthesis wearers performing 4 FCE-OH tests 2 times (2 weeks apart). Agreement was expressed in % and kappa value. Feasibility (focus area's "acceptability", "demand," and "implementation") was determined by using a questionnaire. After 2 rounds of pilot testing and adjusting, reliability of a third version was tested. The interrater reliability for the first and second rating sessions were к = 0.54 (confidence interval [CI]: 0.52-0.57) and к = 0.64 (CI: 0.61-0.66), respectively. The intrarater reliability was к = 0.77 (CI: 0.72-0.82). The feasibility was good but could be improved by a training program. It seems possible to identify compensatory movements in upper limb prosthesis wearers during the performance of FCE-OH tests reliably by observation using the developed observational scoring system. Interrater reliability was satisfactory in most instances; intrarater reliability was good. Feasibility was established. Copyright © 2018 Hanley & Belfus. Published by Elsevier Inc. All rights reserved.

Measuring Vincristine-Induced Peripheral Neuropathy in Children with Acute Lymphoblastic Leukemia

PubMed Central

Lavoie Smith, Ellen M.; Li, Lang; Hutchinson, Raymond J.; Ho, Richard; Burnette, W. Bryan; Wells, Elizabeth; Bridges, Celia; Renbarger, Jamie

2014-01-01

Background Vincristine-induced peripheral neuropathy (VIPN) is difficult to quantify in children. Objective The study objective was to examine the reliability, validity, and clinical feasibility of several VIPN measures for use in children with acute lymphoblastic leukemia. Interventions/Methods Children (N = 65) aged 1–18 years receiving vincristine at four academic centers participated in the study. Baseline and pre-vincristine VIPN assessments were obtained using the Total Neuropathy Score-Pediatric Vincristine (TNS-PV), the National Cancer Institute Common Terminology Criteria for Adverse Events, the Balis grading scale, and the FACES pain scale. TNS-PV scores (n = 806) were obtained over 15 weeks. Blood was obtained at several time-points to quantify pharmacokinetic parameters. Results Cronbach’s alpha for a reduced TNS-PV scale was 0.84. TNS-PV scores correlated with cumulative vincristine dosage (r = 0.53, p = 0.01), pharmacokinetic parameters (r = 0.41, p = 0.05), and grading scale scores (r = 0.46 – 0.52; p = 0.01). FACES scores correlated with the TNS-PV neuropathic pain item (r = 0.48; p = 0.01), and were attainable in all ages. A 2-item V-Rex score (vibration and reflex items) was the most responsive to change (es 0.65, p < 0.001). TNS-PV scores were attainable in 95% of children ≥ 6 years. Conclusions The TNS-PV is reliable and valid for measuring VIPN. It is sensitive to change over time (15 weeks) and feasible for use in children ≥ 6 years of age. Implications for Practice The TNS-PV may be a useful tool for assessing vincristine toxicity in children with acute lymphoblastic leukemia. PMID:23842524

Recommendation for measuring clinical outcome in distal radius fractures: a core set of domains for standardized reporting in clinical practice and research.

PubMed

Goldhahn, Jörg; Beaton, Dorcas; Ladd, Amy; Macdermid, Joy; Hoang-Kim, Amy

2014-02-01

Lack of standardization of outcome measurement has hampered an evidence-based approach to clinical practice and research. We adopted a process of reviewing evidence on current use of measures and appropriate theoretical frameworks for health and disability to inform a consensus process that was focused on deriving the minimal set of core domains in distal radius fracture. We agreed on the following seven core recommendations: (1) pain and function were regarded as the primary domains, (2) very brief measures were needed for routine administration in clinical practice, (3) these brief measures could be augmented by additional measures that provide more detail or address additional domains for clinical research, (4) measurement of pain should include measures of both intensity and frequency as core attributes, (5) a numeric pain scale, e.g. visual analogue scale or visual numeric scale or the pain subscale of the patient-reported wrist evaluation (PRWE) questionnaires were identified as reliable, valid and feasible measures to measure these concepts, (6) for function, either the Quick Disability of the arm, shoulder and hand questionnaire or PRWE-function subscale was identified as reliable, valid and feasible measures, and (7) a measure of participation and treatment complications should be considered core outcomes for both clinical practice and research. We used a sound methodological approach to form a comprehensive foundation of content for outcomes in the area of distal radius fractures. We recommend the use of symptom and function as separate domains in the ICF core set in clinical research or practice for patients with wrist fracture. Further research is needed to provide more definitive measurement properties of measures across all domains.

Are Validity and Reliability "Relevant" in Qualitative Evaluation Research?

ERIC Educational Resources Information Center

Goodwin, Laura D.; Goodwin, William L.

1984-01-01

The views of prominant qualitative methodologists on the appropriateness of validity and reliability estimation for the measurement strategies employed in qualitative evaluations are summarized. A case is made for the relevance of validity and reliability estimation. Definitions of validity and reliability for qualitative measurement are presented…

Investigating a self-scoring interview simulation for learning and assessment in the medical consultation.

PubMed

Bruen, Catherine; Kreiter, Clarence; Wade, Vincent; Pawlikowska, Teresa

2017-01-01

Experience with simulated patients supports undergraduate learning of medical consultation skills. Adaptive simulations are being introduced into this environment. The authors investigate whether it can underpin valid and reliable assessment by conducting a generalizability analysis using IT data analytics from the interaction of medical students (in psychiatry) with adaptive simulations to explore the feasibility of adaptive simulations for supporting automated learning and assessment. The generalizability (G) study was focused on two clinically relevant variables: clinical decision points and communication skills. While the G study on the communication skills score yielded low levels of true score variance, the results produced by the decision points, indicating clinical decision-making and confirming user knowledge of the process of the Calgary-Cambridge model of consultation, produced reliability levels similar to what might be expected with rater-based scoring. The findings indicate that adaptive simulations have potential as a teaching and assessment tool for medical consultations.

Rapid, Reliable Shape Setting of Superelastic Nitinol for Prototyping Robots

PubMed Central

Gilbert, Hunter B.; Webster, Robert J.

2016-01-01

Shape setting Nitinol tubes and wires in a typical laboratory setting for use in superelastic robots is challenging. Obtaining samples that remain superelastic and exhibit desired precurvatures currently requires many iterations, which is time consuming and consumes a substantial amount of Nitinol. To provide a more accurate and reliable method of shape setting, in this paper we propose an electrical technique that uses Joule heating to attain the necessary shape setting temperatures. The resulting high power heating prevents unintended aging of the material and yields consistent and accurate results for the rapid creation of prototypes. We present a complete algorithm and system together with an experimental analysis of temperature regulation. We experimentally validate the approach on Nitinol tubes that are shape set into planar curves. We also demonstrate the feasibility of creating general space curves by shape setting a helical tube. The system demonstrates a mean absolute temperature error of 10°C. PMID:27648473

Rapid, Reliable Shape Setting of Superelastic Nitinol for Prototyping Robots.

PubMed

Gilbert, Hunter B; Webster, Robert J

Shape setting Nitinol tubes and wires in a typical laboratory setting for use in superelastic robots is challenging. Obtaining samples that remain superelastic and exhibit desired precurvatures currently requires many iterations, which is time consuming and consumes a substantial amount of Nitinol. To provide a more accurate and reliable method of shape setting, in this paper we propose an electrical technique that uses Joule heating to attain the necessary shape setting temperatures. The resulting high power heating prevents unintended aging of the material and yields consistent and accurate results for the rapid creation of prototypes. We present a complete algorithm and system together with an experimental analysis of temperature regulation. We experimentally validate the approach on Nitinol tubes that are shape set into planar curves. We also demonstrate the feasibility of creating general space curves by shape setting a helical tube. The system demonstrates a mean absolute temperature error of 10°C.

Reliability and Validity of the Turkish Version of the PedsQL 3.0 Cancer Module for 2- to 7-Year-Old and the PedsQL 4.0 Generic Core Scales for 5- to 7-Year-Old: The Hacettepe University Experience.

PubMed

Yıldız Kabak, Vesile; Yakut, Yavuz; Çetin, Mualla; Düger, Tülin

2016-09-05

The aim of this study was to investigate the reliability and validity of the Turkish version of the Pediatric Quality of Life Inventory (PedsQL) 3.0 Cancer Module for 2- to 7-year-old and the PedsQL 4.0 Generic Core Scales for 5- to 7-year-old in childhood cancer. The PedsQL 3.0 Cancer Module and PedsQL 4.0 Generic Core Scales were administered to children with cancer and their parents at Hacettepe University. Internal consistency was determined by using Cronbach's alpha and test-retest reliability was determined by using the intraclass correlation coefficient (ICC). Construct validity was assessed by comparing the results of the PedsQL 3.0 Cancer Module with those of the PedsQL 4.0 Generic Core Scales. Cronbach's alpha of the PedsQL 3.0 Cancer Module varied from 0.803 to 0.873 and that of the PedsQL 4.0 Generic Core Scales from 0.665 to 0.841. Test-retest ICC values of the PedsQL 3.0 Cancer Module varied from 0.877 to 0.949 and those of the PedsQL 4.0 Generic Core Scales from 0.681 to 0.824. The correlation of the PedsQL 3.0 Cancer Module with subscale scores of the PedsQL 4.0 Generic Core Scales showed that there were excellent to fair correlations between the two scales. The relationship between parent proxy-report and child self-report of the PedsQL 3.0 Cancer Module had very good correlation (r=0.694, p<0.001), as did the PedsQL 4.0 Generic Core Scales (r=0.540, p=0.002). This study demonstrated the reliability, validity, and feasibility of the Turkish version of the PedsQL 3.0 Cancer Module in 2- to 4-year-old and 5- to 7-year-old and the PedsQL 4.0 Generic Core Scales in 5- to 7-year-old in childhood cancer.

Reliability and validity in a nutshell.

PubMed

Bannigan, Katrina; Watson, Roger

2009-12-01

To explore and explain the different concepts of reliability and validity as they are related to measurement instruments in social science and health care. There are different concepts contained in the terms reliability and validity and these are often explained poorly and there is often confusion between them. To develop some clarity about reliability and validity a conceptual framework was built based on the existing literature. The concepts of reliability, validity and utility are explored and explained. Reliability contains the concepts of internal consistency and stability and equivalence. Validity contains the concepts of content, face, criterion, concurrent, predictive, construct, convergent (and divergent), factorial and discriminant. In addition, for clinical practice and research, it is essential to establish the utility of a measurement instrument. To use measurement instruments appropriately in clinical practice, the extent to which they are reliable, valid and usable must be established.

Concurrent validity of walking speed values calculated via the GAITRite electronic walkway and 3 meter walk test in the chronic stroke population.

PubMed

Peters, Denise M; Middleton, Addie; Donley, Jonathan W; Blanck, Erika L; Fritz, Stacy L

2014-04-01

The purpose of this study was to provide novel information regarding the concurrent validity (primary aim) and reliability (secondary aim) of walking speed (WS) calculated via the GAITRite electronic walkway system and 3 meter walk test (3MWT) in the chronic stroke population. The 3MWT is a feasible option for clinicians working in environments where space is limited. Psychometric properties of the test have not been established. Participants with chronic stroke were stratified into three groups: (1) household ambulators (HA) (self-selected WS < 0.4 m/s, 12 participants, 31 observations); (2) limited community ambulators (LCA) (self-selected WS 0.4-0.8 m/s, 24 participants, 60 observations); and (3) community ambulators (CA) (self-selected WS > 0.8 m/s, 26 participants, 71 observations). Three consecutive trials of GAITRite and 3MWT were performed at participant's self-selected WS. Average WS measurements differed significantly (p < 0.05) between GAITRite and 3MWT for all three groups. HA group: GAITRite 0.25 (0.11) m/s, 3MWT 0.27 (0.11) m/s; LCA group: GAITRite 0.56 (0.11) m/s, 3MWT 0.52 (0.10) m/s; CA group: GAITRite 1.03 (0.16) m/s, 3MWT 0.89 (0.15) m/s. Both WS measures had excellent within-session reliability (ICC's ranging from 0.85 to 0.97, SEM95 from 0.04 to 0.12 m/s and MDC95 from 0.05 to 0.16 m/s). Reliability was highest for HA on both measures. Although both the 3MWT and the GAITRite are reliable measures of WS for individuals with chronic stroke, the two measures do not demonstrate concurrent validity.

Applications of computerized adaptive testing (CAT) to the assessment of headache impact.

PubMed

Ware, John E; Kosinski, Mark; Bjorner, Jakob B; Bayliss, Martha S; Batenhorst, Alice; Dahlöf, Carl G H; Tepper, Stewart; Dowson, Andrew

2003-12-01

To evaluate the feasibility of computerized adaptive testing (CAT) and the reliability and validity of CAT-based estimates of headache impact scores in comparison with 'static' surveys. Responses to the 54-item Headache Impact Test (HIT) were re-analyzed for recent headache sufferers (n = 1016) who completed telephone interviews during the National Survey of Headache Impact (NSHI). Item response theory (IRT) calibrations and the computerized dynamic health assessment (DYNHA) software were used to simulate CAT assessments by selecting the most informative items for each person and estimating impact scores according to pre-set precision standards (CAT-HIT). Results were compared with IRT estimates based on all items (total-HIT), computerized 6-item dynamic estimates (CAT-HIT-6), and a developmental version of a 'static' 6-item form (HIT-6-D). Analyses focused on: respondent burden (survey length and administration time), score distributions ('ceiling' and 'floor' effects), reliability and standard errors, and clinical validity (diagnosis, level of severity). A random sample (n = 245) was re-assessed to test responsiveness. A second study (n = 1103) compared actual CAT surveys and an improved 'static' HIT-6 among current headache sufferers sampled on the Internet. Respondents completed measures from the first study and the generic SF-8 Health Survey; some (n = 540) were re-tested on the Internet after 2 weeks. In the first study, simulated CAT-HIT and total-HIT scores were highly correlated (r = 0.92) without 'ceiling' or 'floor' effects and with a substantial reduction (90.8%) in respondent burden. Six of the 54 items accounted for the great majority of item administrations (3603/5028, 77.6%). CAT-HIT reliability estimates were very high (0.975-0.992) in the range where 95% of respondents scored, and relative validity (RV) coefficients were high for diagnosis (RV = 0.87) and severity (RV = 0.89); patient-level classifications were accurate 91.3% for a diagnosis of migraine. For all three criteria of change, CAT-HIT scores were more responsive than all other measures. In the second study, estimates of respondent burden, item usage, reliability and clinical validity were replicated. The test-retest reliability of CAT-HIT was 0.79 and alternate forms coefficients ranged from 0.85 to 0.91. All correlations with the generic SF-8 were negative. CAT-based administrations of headache impact items achieved very large reductions in respondent burden without compromising validity for purposes of patient screening or monitoring changes in headache impact over time. IRT models and CAT-based dynamic health assessments warrant testing among patients with other conditions.

Quality assessment of a new surgical simulator for neuroendoscopic training.

PubMed

Filho, Francisco Vaz Guimarães; Coelho, Giselle; Cavalheiro, Sergio; Lyra, Marcos; Zymberg, Samuel T

2011-04-01

Ideal surgical training models should be entirely reliable, atoxic, easy to handle, and, if possible, low cost. All available models have their advantages and disadvantages. The choice of one or another will depend on the type of surgery to be performed. The authors created an anatomical model called the S.I.M.O.N.T. (Sinus Model Oto-Rhino Neuro Trainer) Neurosurgical Endotrainer, which can provide reliable neuroendoscopic training. The aim in the present study was to assess both the quality of the model and the development of surgical skills by trainees. The S.I.M.O.N.T. is built of a synthetic thermoretractable, thermosensible rubber called Neoderma, which, combined with different polymers, produces more than 30 different formulas. Quality assessment of the model was based on qualitative and quantitative data obtained from training sessions with 9 experienced and 13 inexperienced neurosurgeons. The techniques used for evaluation were face validation, retest and interrater reliability, and construct validation. The experts considered the S.I.M.O.N.T. capable of reproducing surgical situations as if they were real and presenting great similarity with the human brain. Surgical results of serial training showed that the model could be considered precise. Finally, development and improvement in surgical skills by the trainees were observed and considered relevant to further training. It was also observed that the probability of any single error was dramatically decreased after each training session, with a mean reduction of 41.65% (range 38.7%-45.6%). Neuroendoscopic training has some specific requirements. A unique set of instruments is required, as is a model that can resemble real-life situations. The S.I.M.O.N.T. is a new alternative model specially designed for this purpose. Validation techniques followed by precision assessments attested to the model's feasibility.

Developing a tool for assessing competency in root cause analysis.

PubMed

Gupta, Priyanka; Varkey, Prathibha

2009-01-01

Root cause analysis (RCA) is a tool for identifying the key cause(s) contributing to a sentinel event or near miss. Although training in RCA is gaining popularity in medical education, there is no published literature on valid or reliable methods for assessing competency in the same. A tool for assessing competency in RCA was pilot tested as part of an eight-station Objective Structured Clinical Examination that was conducted at the completion of a three-week quality improvement (QI) curriculum for the Mayo Clinic Preventive Medicine and Endocrinology fellowship programs. As part of the curriculum, fellows completed a QI project to enhance physician communication of the diagnosis and treatment plan at the end of a patient visit. They had a didactic session on RCA, followed by process mapping of the information flow at the project clinic, after which fellows conducted an actual RCA using the Ishikawa fishbone diagram. For the RCA competency assessment, fellows performed an RCA regarding a scenario describing an adverse medication event and provided possible solutions to prevent such errors in the future. All faculty strongly agreed or agreed that they were able to accurately assess competency in RCA using the tool. Interrater reliability for the global competency rating and checklist scoring were 0.96 and 0.85, respectively. Internal consistency (Cronbach's alpha) was 0.76. Six of eight of the fellows found the difficulty level of the test to be optimal. Assessment methods must accompany education programs to ensure that graduates are competent in QI methodologies and are able to apply them effectively in the workplace. The RCA assessment tool was found to be a valid, reliable, feasible, and acceptable method for assessing competency in RCA. Further research is needed to examine its predictive validity and generalizability.

[Validation of the Spanish version of the QOLIE-10 quality of life in epilepsy questionnaire].

PubMed

Viteri, C; Codina, M; Cobaleda, S; Lahuerta, J; Barriga, J; Barrera, S; Morales, M D

2008-04-01

Epilepsy patients not only have to deal with the disease itself but also the side effects of some treatments, fear about sudden occurrence of seizures, stigmatization, and restrictions in activities of daily living. For this reason, it is accepted that seizure control is only one feature of the clinical management of epilepsy, since cognitive, physical and psychological deterioration also affects quality of life. It is essential to have measurement tools that rapidly and accurately evaluate the complex aspects included in the concept of quality of life in persons with epilepsy. This study has aimed to validate the specific Health-Related Quality of Life (HRQoL) questionnaire for epileptic patients, QOLIE-10 (Quality of Life in Epilepsy Inventory-10), a reduced version having easier clinical application than the QOLIE-31 in a Spanish population. Naturalistic, prospective, national and multicenter study, in which 21 Spanish neurologists took part. Patients with partial or generalized epilepsy from 18 to 50 years, who were under monotherapy with lamotrigine or valproic acid and clinically stables, were included. Two visits were carried out, one basal visit and a 6, month follow-up visit. Sociodemographic, clinical (ti-me since diagnosis, previous treatment, current treatment and perceived adverse events) and social variables (QOLIE-31, QOLIE-10) were collected. A total of 107 patients were included. Mean age was 30.4 (deviation standard [DE]: 9.1) years and 61.7% were women. Mean time since epilepsy diagnosis was 8 (SD: 8.1) years. A total of 84.1% of the patients answered all the items correctly (feasibility). The Kaiser- Myer-Olkin measure was 0.822, with an associated pvalue<0.001 (content validity). QOLIE-10 scores did not show any relationship with the type of seizures (p>0.05), either globally or for its different domains (cross-sectional validity). The mean score went from 73.9 (SD: 15.7) to 75.5 (SD: 14.9) between the two visits, equivalent to an increase of HRQoL of 1.6 (SD: 7.6) points (p=0.37) (longitudinal validity). A Cronbach's alpha coefficient of 0.811 was obtained (internal consistency). Interclass correlation coefficients were higher than 0.7 (reliability). In the overall score the effect size obtained was of 0.10 (responsiveness to change). The Spanish version of the QOLIE-10 has been shown to be feasible, valid, reliable and responsive to changes. The QOLIE-10 is a useful tool to measure HRQoL in usual clinical practice.

[Validation of a Spanish version of the Childhood Asthma Control Test (Sc-ACT) for use in Spain].

PubMed

Pérez-Yarza, E G; Castro-Rodriguez, J A; Villa Asensi, J R; Garde Garde, J; Hidalgo Bermejo, F J

2015-08-01

The Childhood Asthma Control Test (c-ACT) is a validated tool for determining pediatric asthma control. However, it is not validated in the Spanish language in Spain. We evaluated the psychometric properties of the Spanish version of the Childhood Asthma Control Test (Sc-ACT) for assessing asthma control in children ages 4 to11. This national, multicentre, prospective study was conducted in Spain with asthmatic children and their caregivers. Patients were assessed at 3 visits (Baseline, 2 Weeks, and 4 Months). Clinical variables included: symptoms, exacerbations, FEV1, asthma classification, PAQLQ and PACQLQ questionnaire scores, and asthma control as perceived by physicians, patients and caregivers. The Sc-ACT feasibility, validity, reliability, and sensitivity to change were assessed. A total of 394 children were included; mean (SD) time to complete the Sc-ACT was 5.3 (4.4) minutes. Sc-ACT score was correlated with asthma control as perceived by physician (-0.52), patient (-0.53), and caregiver (-0.51) and with the PAQLQ (0.56) and PACQLQ (0.55) scores. Sc-ACT was found to be significantly related to intensity and frequency of asthma symptoms. Cronbach alpha coefficient α was 0.81 and intraclass correlation coefficient was ≥0.85 for all of the items. The global effect size of Sc-ACT was 0.55. The cutoff point scores of 21 or higher indicated a good asthma control and their MCID was 4 points. The Spanish version of the c-ACT was found to be a reliable and valid questionnaire for evaluating asthma control in Spanish-speaking children ages 4 to 11 in Spain. Copyright © 2014 Asociación Española de Pediatría. Published by Elsevier España, S.L.U. All rights reserved.

The Behavior Pain Assessment Tool for critically ill adults: a validation study in 28 countries.

PubMed

Gélinas, Céline; Puntillo, Kathleen A; Levin, Pavel; Azoulay, Elie

2017-05-01

Many critically ill adults are unable to communicate their pain through self-report. The study purpose was to validate the use of the 8-item Behavior Pain Assessment Tool (BPAT) in patients hospitalized in 192 intensive care units from 28 countries. A total of 4812 procedures in 3851 patients were included in data analysis. Patients were assessed with the BPAT before and during procedures by 2 different raters (mostly nurses and physicians). Those who were able to self-report were asked to rate their pain intensity and pain distress on 0 to 10 numeric rating scales. Interrater reliability of behavioral observations was supported by moderate (0.43-0.60) to excellent (>0.60) kappa coefficients. Mixed effects multilevel logistic regression models showed that most behaviors were more likely to be present during the procedure than before and in less sedated patients, demonstrating discriminant validation of the tool use. Regarding criterion validation, moderate positive correlations were found during procedures between the mean BPAT scores and the mean pain intensity (r = 0.54) and pain distress (r = 0.49) scores (P < 0.001). Regression models showed that all behaviors were significant predictors of pain intensity and pain distress, accounting for 35% and 29% of their total variance, respectively. A BPAT cut-point score >3.5 could classify patients with or without severe levels (≥8) of pain intensity and distress with sensitivity and specificity findings ranging from 61.8% to 75.1%. The BPAT was found to be reliable and valid. Its feasibility for use in practice and the effect of its clinical implementation on patient pain and intensive care unit outcomes need further research.

Ethical Implications of Validity-vs.-Reliability Trade-Offs in Educational Research

ERIC Educational Resources Information Center

Fendler, Lynn

2016-01-01

In educational research that calls itself empirical, the relationship between validity and reliability is that of trade-off: the stronger the bases for validity, the weaker the bases for reliability (and vice versa). Validity and reliability are widely regarded as basic criteria for evaluating research; however, there are ethical implications of…

What to Do With "Moderate" Reliability and Validity Coefficients?

PubMed

Post, Marcel W

2016-07-01

Clinimetric studies may use criteria for test-retest reliability and convergent validity such that correlation coefficients as low as .40 are supportive of reliability and validity. It can be argued that moderate (.40-.60) correlations should not be interpreted in this way and that reliability coefficients <.70 should be considered as indicative of unreliability. Convergent validity coefficients in the .40 to .60 or .40 to .70 range should be considered as indications of validity problems, or as inconclusive at best. Studies on reliability and convergent should be designed in such a way that it is realistic to expect high reliability and validity coefficients. Multitrait multimethod approaches are preferred to study construct (convergent-divergent) validity. Copyright © 2016 American Congress of Rehabilitation Medicine. Published by Elsevier Inc. All rights reserved.

Quantifying Risk for Anxiety Disorders in Preschool Children: A Machine Learning Approach.

PubMed

Carpenter, Kimberly L H; Sprechmann, Pablo; Calderbank, Robert; Sapiro, Guillermo; Egger, Helen L

2016-01-01

Early childhood anxiety disorders are common, impairing, and predictive of anxiety and mood disorders later in childhood. Epidemiological studies over the last decade find that the prevalence of impairing anxiety disorders in preschool children ranges from 0.3% to 6.5%. Yet, less than 15% of young children with an impairing anxiety disorder receive a mental health evaluation or treatment. One possible reason for the low rate of care for anxious preschoolers is the lack of affordable, timely, reliable and valid tools for identifying young children with clinically significant anxiety. Diagnostic interviews assessing psychopathology in young children require intensive training, take hours to administer and code, and are not available for use outside of research settings. The Preschool Age Psychiatric Assessment (PAPA) is a reliable and valid structured diagnostic parent-report interview for assessing psychopathology, including anxiety disorders, in 2 to 5 year old children. In this paper, we apply machine-learning tools to already collected PAPA data from two large community studies to identify sub-sets of PAPA items that could be developed into an efficient, reliable, and valid screening tool to assess a young child's risk for an anxiety disorder. Using machine learning, we were able to decrease by an order of magnitude the number of items needed to identify a child who is at risk for an anxiety disorder with an accuracy of over 96% for both generalized anxiety disorder (GAD) and separation anxiety disorder (SAD). Additionally, rather than considering GAD or SAD as discrete/binary entities, we present a continuous risk score representing the child's risk of meeting criteria for GAD or SAD. Identification of a short question-set that assesses risk for an anxiety disorder could be a first step toward development and validation of a relatively short screening tool feasible for use in pediatric clinics and daycare/preschool settings.

Quantifying Risk for Anxiety Disorders in Preschool Children: A Machine Learning Approach

PubMed Central

Calderbank, Robert; Sapiro, Guillermo; Egger, Helen L.

2016-01-01

Early childhood anxiety disorders are common, impairing, and predictive of anxiety and mood disorders later in childhood. Epidemiological studies over the last decade find that the prevalence of impairing anxiety disorders in preschool children ranges from 0.3% to 6.5%. Yet, less than 15% of young children with an impairing anxiety disorder receive a mental health evaluation or treatment. One possible reason for the low rate of care for anxious preschoolers is the lack of affordable, timely, reliable and valid tools for identifying young children with clinically significant anxiety. Diagnostic interviews assessing psychopathology in young children require intensive training, take hours to administer and code, and are not available for use outside of research settings. The Preschool Age Psychiatric Assessment (PAPA) is a reliable and valid structured diagnostic parent-report interview for assessing psychopathology, including anxiety disorders, in 2 to 5 year old children. In this paper, we apply machine-learning tools to already collected PAPA data from two large community studies to identify sub-sets of PAPA items that could be developed into an efficient, reliable, and valid screening tool to assess a young child’s risk for an anxiety disorder. Using machine learning, we were able to decrease by an order of magnitude the number of items needed to identify a child who is at risk for an anxiety disorder with an accuracy of over 96% for both generalized anxiety disorder (GAD) and separation anxiety disorder (SAD). Additionally, rather than considering GAD or SAD as discrete/binary entities, we present a continuous risk score representing the child’s risk of meeting criteria for GAD or SAD. Identification of a short question-set that assesses risk for an anxiety disorder could be a first step toward development and validation of a relatively short screening tool feasible for use in pediatric clinics and daycare/preschool settings. PMID:27880812

Older, vulnerable patient view: a pilot and feasibility study of the patient measure of safety (PMOS) with patients in Australia.

PubMed

Taylor, Natalie; Hogden, Emily; Clay-Williams, Robyn; Li, Zhicheng; Lawton, Rebecca; Braithwaite, Jeffrey

2016-06-08

The UK-developed patient measure of safety (PMOS) is a validated tool which captures patient perceptions of safety in hospitals. We aimed (1) to investigate the extent to which the PMOS is appropriate for use with stroke, acute myocardial infarction (AMI) and hip fracture patients in Australian hospitals and (2) to pilot the PMOS for use in a large-scale, national study 'Deepening our Understanding of Quality in Australia' (DUQuA). Stroke, AMI and hip fracture patients (n=34) receiving care in 3 wards in 1 large hospital. 2 phases were conducted. First, a 'think aloud' study was used to determine the validity of PMOS with this population in an international setting, and to make amendments based on patient feedback. The second phase tested the revised measure to establish the internal consistency reliability of the revised subscales, and piloted the recruitment and administration processes to ensure feasibility of the PMOS for use in DUQuA. Of the 43 questions in the PMOS, 13 (30%) were amended based on issues patients highlighted for improvement in phase 1. In phase 2, a total of 34 patients were approached and 29 included, with a mean age of 71.3 years (SD=16.39). Internal consistency reliability was established using interitem correlation and Cronbach's α for all but 1 subscale. The most and least favourably rated aspects of safety differed between the 3 wards. A study log was categorised into 10 key feasibility factors, including liaising with wards to understand operational procedures and identify patterns of patient discharge. Capturing patient perceptions of care is crucial in improving patient safety. The revised PMOS is appropriate for use with vulnerable older adult groups. The findings from this study have informed key decisions made for the deployment of this measure as part of the DUQuA study. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/

A model to predict accommodations needed by disabled persons.

PubMed

Babski-Reeves, Kari; Williams, Sabrina; Waters, Tzer Nan; Crumpton-Young, Lesia L; McCauley-Bell, Pamela

2005-09-01

In this paper, several approaches to assist employers in the accommodation process for disabled employees are discussed and a mathematical model is proposed to assist employers in predicting the accommodation level needed by an individual with a mobility-related disability. This study investigates the validity and reliability of this model in assessing the accommodation level needed by individuals utilizing data collected from twelve individuals with mobility-related disabilities. Based on the results of the statistical analyses, this proposed model produces a feasible preliminary measure for assessing the accommodation level needed for persons with mobility-related disabilities. Suggestions for practical application of this model in an industrial setting are addressed.

Patient-centered technological assessment and monitoring of depression for low-income patients.

PubMed

Wu, Shinyi; Vidyanti, Irene; Liu, Pai; Hawkins, Caitlin; Ramirez, Magaly; Guterman, Jeffrey; Gross-Schulman, Sandra; Sklaroff, Laura Myerchin; Ell, Kathleen

2014-01-01

Depression is a significant challenge for ambulatory care because it worsens health status and outcomes, increases health care utilizations and costs, and elevates suicide risk. An automatic telephonic assessment (ATA) system that links with tasks and alerts to providers may improve quality of depression care and increase provider productivity. We used ATA system in a trial to assess and monitor depressive symptoms of 444 safety-net primary care patients with diabetes. We assessed system properties, evaluated preliminary clinical outcomes, and estimated cost savings. The ATA system is feasible, reliable, valid, safe, and likely cost-effective for depression screening and monitoring for low-income primary care population.

Developing Cognitive Models for Social Simulation from Survey Data

NASA Astrophysics Data System (ADS)

Alt, Jonathan K.; Lieberman, Stephen

The representation of human behavior and cognition continues to challenge the modeling and simulation community. The use of survey and polling instruments to inform belief states, issue stances and action choice models provides a compelling means of developing models and simulations with empirical data. Using these types of data to population social simulations can greatly enhance the feasibility of validation efforts, the reusability of social and behavioral modeling frameworks, and the testable reliability of simulations. We provide a case study demonstrating these effects, document the use of survey data to develop cognitive models, and suggest future paths forward for social and behavioral modeling.

Health and Safety Checklist for Early Care and Education Programs to Assess Key National Health and Safety Standards.

PubMed

Alkon, Abbey; Rose, Roberta; Wolff, Mimi; Kotch, Jonathan B; Aronson, Susan S

2016-01-01

The project aims were to (1) develop an observational Health and Safety Checklist to assess health and safety practices and conditions in early care and education (ECE) programs using Stepping Stones To Caring For Our Children, 3rd Edition national standards, (2) pilot test the Checklist, completed by nurse child care health consultants, to assess feasibility, ease of completion, objectivity, validity, and reliability, and (3) revise the Checklist based on the qualitative and quantitative results of the pilot study. The observable national health and safety standards were identified and then rated by health, safety, and child care experts using a Delphi technique to validate the standards as essential to prevent harm and promote health. Then, child care health consultants recruited ECE centers and pilot tested the 124-item Checklist. The pilot study was conducted in Arizona, California and North Carolina. The psychometric properties of the Checklist were assessed. The 37 participating ECE centers had 2627 children from ethnically-diverse backgrounds and primarily low-income families. The child care health consultants found the Checklist easy to complete, objective, and useful for planning health and safety interventions. The Checklist had content and face validity, inter-rater reliability, internal consistency, and concurrent validity. Based on the child care health consultant feedback and psychometric properties of the Checklist, the Checklist was revised and re-written at an 8th grade literacy level. The Health and Safety Checklist provides a standardized instrument of observable, selected national standards to assess the quality of health and safety in ECE centers.

Is the Veterans Specific Activity Questionnaire Valid to Assess Older Adults Aerobic Fitness?

PubMed

de Carvalho Bastone, Alessandra; de Souza Moreira, Bruno; Teixeira, Claudine Patrícia; Dias, João Marcos Domingues; Dias, Rosângela Corrêa

2016-01-01

Aerobic fitness in older adults is related to health status, incident disability, nursing home admission, and all-cause mortality. The most accurate quantification of aerobic fitness, expressed as peak oxygen consumption in mL·kg·min, is the cardiorespiratory exercise test; however, it is not feasible in all settings and might offer risk to patients. The Veterans Specific Activity Questionnaire (VSAQ) is a 13-item self-administered symptom questionnaire that estimates aerobic fitness expressed in metabolic equivalents (METs) and has been validated to cardiovascular patients. The purpose of this study was to assess the validity and reliability of the VSAQ in older adults without specific health conditions. A methodological study with a cross-sectional design was conducted with 28 older adults (66-86 years). The VSAQ was administered on 3 occasions by 2 evaluators. Aerobic capacity in METs as measured by the VSAQ was compared with the METs found in an incremental shuttle walk test (ISWT) performed with a portable metabolic measurement system and with accelerometer data. The validity of the VSAQ was found to be moderate-to-good when compared with the METs and distance measured by the ISWT and with the moderate activity per day and steps per day obtained by accelerometry. The Bland-Altman graph analysis showed no values outside the limits of agreement, suggesting good precision between the METs estimated by questionnaire and the METs measured by the ISWT. Also, the intrarater and interrater reliabilities of the instrument were good. The results showed that the VSAQ is a valuable tool to assess the aerobic fitness of older adults.

Development of a Computer-Based Format for the Alcohol, Smoking, and Substance Involvement Screening Test (ASSIST) With University Students.

PubMed

Christoff, Adriana Oliveira; Barreto, Heloisa Gomm Arruda; Boerngen-Lacerda, Roseli

2016-07-28

The Alcohol, Smoking, and Substance Involvement Screening Test (ASSIST) is a reliable and valid tool for the early detection of harmful and hazardous drug use in primary care settings when administered by interview in the general population. In university students, substance use is high, so a reliable and feasible screening instrument is needed. To compare the computer-based ASSIST (ASSISTc) with the interview format (ASSISTi). A convenience sample with counterbalanced design was used alternating between the ASSISTi and ASSISTc with 15-day interval. Although this is not a traditional test-retest reliability study, the same statistical analysis was used: intraclass correlations (ICC), kappa (κ), and Cronbach's alpha (α) to compare the two formats. A satisfaction questionnaire was applied immediately after the second session. Both formats were completed by the students (n = 809) over 15 days. The scores of involvement with all substances and with tobacco, alcohol, cannabis, and cocaine obtained with the two formats demonstrated excellent ICC (> .77). The level of agreement was considered substantial for tobacco (κ = .69) and cannabis (κ = .70) and moderate for alcohol (κ = .58). The consistency of the ASSISTc was considered satisfactory (α: .85 for tobacco, .73 for alcohol, and .87 for cannabis). The analysis of satisfaction and feasibility showed that the ASSISTi was easier to understand, but the two formats were considered similar when considering acceptability, ease of responding, and degree of intimidation. The two formats are acceptable, the scores are comparable, and they can be used interchangeably.

Validation of the Novaco Anger Scale-Provocation Inventory (Danish) With Nonclinical, Clinical, and Offender Samples.

PubMed

Moeller, Stine Bjerrum; Novaco, Raymond W; Heinola-Nielsen, Vivian; Hougaard, Helle

2016-10-01

Anger has high prevalence in clinical and forensic settings, and it is associated with aggressive behavior and ward atmosphere on psychiatric units. Dysregulated anger is a clinical problem in Danish mental health care systems, but no anger assessment instruments have been validated in Danish. Because the Novaco Anger Scale and Provocation Inventory (NAS-PI) has been extensively validated with different clinical populations and lends itself to clinical case formulation, it was selected for translation and evaluation in the present multistudy project. Psychometric properties of the NAS-PI were investigated with samples of 477 nonclinical, 250 clinical, 167 male prisoner, and 64 male forensic participants. Anger prevalence and its relationship with other anger measures, anxiety/depression, and aggression were examined. NAS-PI was found to have high reliability, concurrent validity, and discriminant validity, and its scores discriminated the samples. High scores in the offender group demonstrated the feasibility of obtaining self-report assessments of anger with this population. Retrospective and prospective validity of the NAS were tested with the forensic patient sample regarding physically aggressive behavior in hospital. Regression analyses showed that higher scores on NAS increase the risk of having acted aggressively in the past and of acting aggressively in the future. © The Author(s) 2015.

A Novel Health Evaluation Strategy for Multifunctional Self-Validating Sensors

PubMed Central

Shen, Zhengguang; Wang, Qi

2013-01-01

The performance evaluation of sensors is very important in actual application. In this paper, a theory based on multi-variable information fusion is studied to evaluate the health level of multifunctional sensors. A novel conception of health reliability degree (HRD) is defined to indicate a quantitative health level, which is different from traditional so-called qualitative fault diagnosis. To evaluate the health condition from both local and global perspectives, the HRD of a single sensitive component at multiple time points and the overall multifunctional sensor at a single time point are defined, respectively. The HRD methodology is emphasized by using multi-variable data fusion technology coupled with a grey comprehensive evaluation method. In this method, to acquire the distinct importance of each sensitive unit and the sensitivity of different time points, the information entropy and analytic hierarchy process method are used, respectively. In order to verify the feasibility of the proposed strategy, a health evaluating experimental system for multifunctional self-validating sensors was designed. The five different health level situations have been discussed. Successful results show that the proposed method is feasible, the HRD could be used to quantitatively indicate the health level and it does have a fast response to the performance changes of multifunctional sensors. PMID:23291576

Modernising speech audiometry: using a smartphone application to test word recognition.

PubMed

van Zyl, Marianne; Swanepoel, De Wet; Myburgh, Hermanus C

2018-04-20

This study aimed to develop and assess a method to measure word recognition abilities using a smartphone application (App) connected to an audiometer. Word lists were recorded in South African English and Afrikaans. Analyses were conducted to determine the effect of hardware used for presentation (computer, compact-disc player, or smartphone) on the frequency content of recordings. An Android App was developed to enable presentation of recorded materials via a smartphone connected to the auxiliary input of the audiometer. Experiments were performed to test feasibility and validity of the developed App and recordings. Participants were 100 young adults (18-30 years) with pure tone thresholds ≤15 dB across the frequency spectrum (250-8000 Hz). Hardware used for presentation had no significant effect on the frequency content of recordings. Listening experiments indicated good inter-list reliability for recordings in both languages, with no significant differences between scores on different lists at each of the tested intensities. Performance-intensity functions had slopes of 4.05%/dB for English and 4.75%/dB for Afrikaans lists at the 50% point. The developed smartphone App constitutes a feasible and valid method for measuring word recognition scores, and can support standardisation and accessibility of recorded speech audiometry.

On computing stress in polymer systems involving multi-body potentials from molecular dynamics simulation

NASA Astrophysics Data System (ADS)

Fu, Yao; Song, Jeong-Hoon

2014-08-01

Hardy stress definition has been restricted to pair potentials and embedded-atom method potentials due to the basic assumptions in the derivation of a symmetric microscopic stress tensor. Force decomposition required in the Hardy stress expression becomes obscure for multi-body potentials. In this work, we demonstrate the invariance of the Hardy stress expression for a polymer system modeled with multi-body interatomic potentials including up to four atoms interaction, by applying central force decomposition of the atomic force. The balance of momentum has been demonstrated to be valid theoretically and tested under various numerical simulation conditions. The validity of momentum conservation justifies the extension of Hardy stress expression to multi-body potential systems. Computed Hardy stress has been observed to converge to the virial stress of the system with increasing spatial averaging volume. This work provides a feasible and reliable linkage between the atomistic and continuum scales for multi-body potential systems.

Monte Carlo simulation of Ray-Scan 64 PET system and performance evaluation using GATE toolkit

NASA Astrophysics Data System (ADS)

Li, Suying; Zhang, Qiushi; Vuletic, Ivan; Xie, Zhaoheng; Yang, Kun; Ren, Qiushi

2017-02-01

In this study, we aimed to develop a GATE model for the simulation of Ray-Scan 64 PET scanner and model its performance characteristics. A detailed implementation of system geometry and physical process were included in the simulation model. Then we modeled the performance characteristics of Ray-Scan 64 PET system for the first time, based on National Electrical Manufacturers Association (NEMA) NU-2 2007 protocols and validated the model against experimental measurement, including spatial resolution, sensitivity, counting rates and noise equivalent count rate (NECR). Moreover, an accurate dead time module was investigated to simulate the counting rate performance. Overall results showed reasonable agreement between simulation and experimental data. The validation results showed the reliability and feasibility of the GATE model to evaluate major performance of Ray-Scan 64 PET system. It provided a useful tool for a wide range of research applications.

Reliability and validity of the revised Gibson Test of Cognitive Skills, a computer-based test battery for assessing cognition across the lifespan.

PubMed

Moore, Amy Lawson; Miller, Terissa M

2018-01-01

The purpose of the current study is to evaluate the validity and reliability of the revised Gibson Test of Cognitive Skills, a computer-based battery of tests measuring short-term memory, long-term memory, processing speed, logic and reasoning, visual processing, as well as auditory processing and word attack skills. This study included 2,737 participants aged 5-85 years. A series of studies was conducted to examine the validity and reliability using the test performance of the entire norming group and several subgroups. The evaluation of the technical properties of the test battery included content validation by subject matter experts, item analysis and coefficient alpha, test-retest reliability, split-half reliability, and analysis of concurrent validity with the Woodcock Johnson III Tests of Cognitive Abilities and Tests of Achievement. Results indicated strong sources of evidence of validity and reliability for the test, including internal consistency reliability coefficients ranging from 0.87 to 0.98, test-retest reliability coefficients ranging from 0.69 to 0.91, split-half reliability coefficients ranging from 0.87 to 0.91, and concurrent validity coefficients ranging from 0.53 to 0.93. The Gibson Test of Cognitive Skills-2 is a reliable and valid tool for assessing cognition in the general population across the lifespan.

Evaluation of tools used to measure calcium and/or dairy consumption in adults.

PubMed

Magarey, Anthea; Baulderstone, Lauren; Yaxley, Alison; Markow, Kylie; Miller, Michelle

2015-05-01

To identify and critique tools for the assessment of Ca and/or dairy intake in adults, in order to ascertain the most accurate and reliable tools available. A systematic review of the literature was conducted using defined inclusion and exclusion criteria. Articles reporting on originally developed tools or testing the reliability or validity of existing tools that measure Ca and/or dairy intake in adults were included. Author-defined criteria for reporting reliability and validity properties were applied. Studies conducted in Western countries. Adults. Thirty papers, utilising thirty-six tools assessing intake of dairy, Ca or both, were identified. Reliability testing was conducted on only two dairy and five Ca tools, with results indicating that only one dairy and two Ca tools were reliable. Validity testing was conducted for all but four Ca-only tools. There was high reliance in validity testing on lower-order tests such as correlation and failure to differentiate between statistical and clinically meaningful differences. Results of the validity testing suggest one dairy and five Ca tools are valid. Thus one tool was considered both reliable and valid for the assessment of dairy intake and only two tools proved reliable and valid for the assessment of Ca intake. While several tools are reliable and valid, their application across adult populations is limited by the populations in which they were tested. These results indicate a need for tools that assess Ca and/or dairy intake in adults to be rigorously tested for reliability and validity.

Validity and reliability of a health care service evaluation instrument for tuberculosis

PubMed Central

Scatena, Lucia Marina; Wysocki, Anneliese Domingues; Beraldo, Aline Ale; Magnabosco, Gabriela Tavares; Brunello, Maria Eugênia Firmino; Netto, Antonio Ruffino; Nogueira, Jordana de Almeida; Silva, Reinaldo Antonio; Brito, Ewerton William Gomes; Alexandre, Patricia Borges Dias; Monroe, Aline Aparecida; Villa, Tereza Cristina Scatena

2015-01-01

OBJECTIVE To evaluate the validity and reliability of an instrument that evaluates the structure of primary health care units for the treatment of tuberculosis. METHODS This cross-sectional study used simple random sampling and evaluated 1,037 health care professionals from five Brazilian municipalities (Natal, state of Rio Grande do Norte; Cabedelo, state of Paraíba; Foz do Iguaçu, state of Parana; Sao José do Rio Preto, state of Sao Paulo, and Uberaba, state of Minas Gerais) in 2011. Structural indicators were identified and validated, considering different methods of organization of the health care system in the municipalities of different population sizes. Each structure represented the organization of health care services and contained the resources available for the execution of health care services: physical resources (equipment, consumables, and facilities); human resources (number and qualification); and resources for maintenance of the existing infrastructure and technology (deemed as the organization of health care services). The statistical analyses used in the validation process included reliability analysis, exploratory factor analysis, and confirmatory factor analysis. RESULTS The validation process indicated the retention of five factors, with 85.9% of the total variance explained, internal consistency between 0.6460 and 0.7802, and quality of fit of the confirmatory factor analysis of 0.995 using the goodness-of-fit index. The retained factors comprised five structural indicators: professionals involved in the care of tuberculosis patients, training, access to recording instruments, availability of supplies, and coordination of health care services with other levels of care. Availability of supplies had the best performance and the lowest coefficient of variation among the services evaluated. The indicators of assessment of human resources and coordination with other levels of care had satisfactory performance, but the latter showed the highest coefficient of variation. The performance of the indicators “training” and “access to recording instruments” was inferior to that of other indicators. CONCLUSIONS The instrument showed feasibility of application and potential to assess the structure of primary health care units for the treatment of tuberculosis. PMID:25741651

Reliable LC3 and p62 autophagy marker detection in formalin fixed paraffin embedded human tissue by immunohistochemistry.

PubMed

Schläfli, A M; Berezowska, S; Adams, O; Langer, R; Tschan, M P

2015-05-05

Autophagy assures cellular homeostasis, and gains increasing importance in cancer, where it impacts on carcinogenesis, propagation of the malignant phenotype and development of resistance. To date, its tissue-based analysis by immunohistochemistry remains poorly standardized. Here we show the feasibility of specifically and reliably assessing the autophagy markers LC3B and p62 (SQSTM1) in formalin fixed and paraffin embedded human tissue by immunohistochemistry. Preceding functional experiments consisted of depleting LC3B and p62 in H1299 lung cancer cells with subsequent induction of autophagy. Western blot and immunofluorescence validated antibody specificity, knockdown efficiency and autophagy induction prior to fixation in formalin and embedding in paraffin. LC3B and p62 antibodies were validated on formalin fixed and paraffin embedded cell pellets of treated and control cells and finally applied on a tissue microarray with 80 human malignant and non-neoplastic lung and stomach formalin fixed and paraffin embedded tissue samples. Dot-like staining of various degrees was observed in cell pellets and 18/40 (LC3B) and 22/40 (p62) tumors, respectively. Seventeen tumors were double positive for LC3B and p62. P62 displayed additional significant cytoplasmic and nuclear staining of unknown significance. Interobserver-agreement for grading of staining intensities and patterns was substantial to excellent (kappa values 0.60 - 0.83). In summary, we present a specific and reliable IHC staining of LC3B and p62 on formalin fixed and paraffin embedded human tissue. Our presented protocol is designed to aid reliable investigation of dysregulated autophagy in solid tumors and may be used on large tissue collectives.

Development, sensibility, and reliability of the Toronto Axial Spondyloarthritis Questionnaire in inflammatory bowel disease.

PubMed

Alnaqbi, Khalid A; Touma, Zahi; Passalent, Laura; Johnson, Sindhu R; Tomlinson, George A; Carty, Adele; Inman, Robert D

2013-10-01

There is an unacceptable delay in the diagnosis of axial spondyloarthritis (axSpA) in its early stages among patients at high risk, in particular those with inflammatory bowel disease (IBD). Our objectives were to develop a sensible and reliable questionnaire to identify undetected axSpA among patients with IBD. Literature was reviewed for item generation in the Toronto axSpA Questionnaire on IBD (TASQ-IBD). Sensibility of the questionnaire was assessed among healthcare professionals and patients. This assessment was related to purpose and framework (clinical function, clinical justification, and clinical applicability), face validity, comprehensiveness [oligo-variability (limiting the questionnaire to important items) and transparency], replicability, content validity, and feasibility. The test-retest reliability study was administered to 77 patients with established IBD and axSpA. Kappa agreement coefficients and absolute agreement were calculated for items. Three domains included IBD, inflammatory back symptoms, and extraaxial features. The entry criterion required a patient to have IBD and back pain or stiffness that ever persisted for ≥ 3 months. Iterative sensibility assessment involved 16 items and a diagram of the back. Kappa coefficients ranged from 0.81-1.00 for each item. Absolute agreement across all items ranged from 91% to 100%. TASQ-IBD is a newly developed, sensible, and reliable case-finding questionnaire to be administered to patients with IBD who have ever had chronic back pain or stiffness persisting for ≥ 3 months. It should facilitate identification and timely referral of patients with IBD to rheumatologists and minimize the delay in diagnosis of axSpA. Consequently, it should assess the prevalence of axSpA in IBD.

Relative and absolute reliability of the clinical version of the Narrow Path Walking Test (NPWT) under single and dual task conditions.

PubMed

Gimmon, Yoav; Jacob, Grinshpon; Lenoble-Hoskovec, Constanze; Büla, Christophe; Melzer, Itshak

2013-01-01

Decline in gait stability has been associated with increased fall risk in older adults. Reliable and clinically feasible methods of gait instability assessment are needed. This study evaluated the relative and absolute reliability and concurrent validity of the testing procedure of the clinical version of the Narrow Path Walking Test (NPWT) under single task (ST) and dual task (DT) conditions. Thirty independent community-dwelling older adults (65-87 years) were tested twice. Participants were instructed to walk within the 6-m narrow path without stepping out. Trial time, number of steps, trial velocity, number of step errors, and number of cognitive task errors were determined. Intraclass correlation coefficients (ICCs) were calculated as indices of agreement, and a graphic approach called "mountain plot" was applied to help interpret the direction and magnitude of disagreements between testing procedures. Smallest detectable change and smallest real difference (SRD) were computed to determine clinically relevant improvement at group and individual levels, respectively. Concurrent validity was assessed using Performance Oriented Mobility Assessment Tool (POMA) and the Short Physical Performance Battery (SPPB). Test-retest agreement (ICC1,2) varied from 0.77 to 0.92 in ST and from 0.78 to 0.92 in DT conditions, with no apparent systematic differences between testing procedures demonstrated by the mountain plot graphs. Smallest detectable change and smallest real change were small for motor task performance and larger for cognitive errors. Significant correlations were observed for trial velocity and trial time with POMA and SPPB. The present results indicate that the NPWT testing procedure is highly reliable and reproducible. Copyright © 2013 Elsevier Ireland Ltd. All rights reserved.

[Evaluation of Suicide Risk Levels in Hospitals: Validity and Reliability Tests].

PubMed

Macagnino, Sandro; Steinert, Tilman; Uhlmann, Carmen

2018-05-01

Examination of in-hospital suicide risk levels concerning their validity and their reliability. The internal suicide risk levels were evaluated in a cross sectional study of in 163 inpatients. A reliability check was performed via determining interrater-reliability of senior physician, therapist and the responsible nurse. Within the scope of the validity check, we conducted analyses of criterion validity and construct validity. For the total sample an "acceptable" to "good" interrater-reliability (Kendalls W = .77) of suicide risk levels were obtained. Schizophrenic disorders showed the lowest values, for personality disorders we found the highest level of interrater-reliability. When examining the criterion validity, Item-9 of the BDI-II is substantial correlated to our suicide risk levels (ρ m  = .54, p < .01). Within the scope of construct validity check, affective disorders showed the highest correlation (ρ = .77), compatible also with "convergent validity". They differed with schizophrenic disorders which showed the least concordance (ρ = .43). In-hospital suicide risk levels may represent an important contribution to the assessment of suicidal behavior of inpatients experiencing psychiatric treatment due to their overall good validity and reliability. © Georg Thieme Verlag KG Stuttgart · New York.

Goal achievement as a patient‐generated outcome measure for stress urinary incontinence

PubMed Central

Milne, Jill L.; Robert, Magali; Tang, Selphee; Drummond, Neil; Ross, Sue

2009-01-01

Abstract Objectives  To explore women’s goals and goal attainment for the conservative and surgical treatment of stress urinary incontinence (SUI), and to examine the feasibility of Goal Attainment Scaling (GAS) as an outcome measure in this population. Background  Despite the range of treatments for SUI, little is known about the outcomes patients consider important. Current instruments measure the impact of SUI on the ability to live a ‘normal’ life without addressing what normal looks like for the patient. Patient‐generated measures that address what a patient aims to achieve may fill this gap. Design  A mixed‐methods exploratory design combined semi‐structured interviews with validated questionnaires and individualized rating of goal achievement. Setting and participants  Participants with SUI (n = 18) were interviewed in their homes prior to initiation of treatment and 3–6 months afterwards. Main variables  Participants reported individualized goals pre‐treatment and rated goal attainment after surgical and conservative therapy. Quality of life impact and change were measured using short forms of the Incontinence Impact Questionnaire and Urinary Distress Inventory. Results  Women expressed a median of four highly individualized treatment‐related goals but goal achievement following conservative treatment was poor. GAS was not feasible as an outcome measure; women readily identified personal goals but could not independently identify graded levels of attainment for each goal. Conclusions  Although further work is needed to examine the most feasible, valid, and reliable method of measuring goal achievement in research, asking patients with UI to identify pre‐treatment goals may provide useful information to guide treatment‐related decision making. PMID:19754692

Educational testing validity and reliability in pharmacy and medical education literature.

PubMed

Hoover, Matthew J; Jung, Rose; Jacobs, David M; Peeters, Michael J

2013-12-16

To evaluate and compare the reliability and validity of educational testing reported in pharmacy education journals to medical education literature. Descriptions of validity evidence sources (content, construct, criterion, and reliability) were extracted from articles that reported educational testing of learners' knowledge, skills, and/or abilities. Using educational testing, the findings of 108 pharmacy education articles were compared to the findings of 198 medical education articles. For pharmacy educational testing, 14 articles (13%) reported more than 1 validity evidence source while 83 articles (77%) reported 1 validity evidence source and 11 articles (10%) did not have evidence. Among validity evidence sources, content validity was reported most frequently. Compared with pharmacy education literature, more medical education articles reported both validity and reliability (59%; p<0.001). While there were more scholarship of teaching and learning (SoTL) articles in pharmacy education compared to medical education, validity, and reliability reporting were limited in the pharmacy education literature.

The Work Disability Functional Assessment Battery (WD-FAB): Feasibility and Psychometric Properties

PubMed Central

Meterko, Mark; Marfeo, Elizabeth E.; McDonough, Christine M.; Jette, Alan M.; Ni, Pengsheng; Bogusz, Kara; Rasch, Elizabeth K; Brandt, Diane E.; Chan, Leighton

2015-01-01

Objectives To assess the feasibility and psychometric properties of eight scales covering two domains of the newly developed Work Disability Functional Assessment Battery (WD-FAB): physical function (PF) and behavioral health (BH) function. Design Cross-sectional. Setting Community. Participants Adults unable to work due to a physical (n=497) or mental (n=476) disability. Interventions None. Main Outcome Measures Each disability group responded to a survey consisting of the relevant WD-FAB scales and existing measures of established validity. The WD-FAB scales were evaluated with regard to data quality (score distribution; percent “I don’t know” responses), efficiency of administration (number of items required to achieve reliability criterion; time required to complete the scale) by computerized adaptive testing (CAT), and measurement accuracy as tested by person fit. Construct validity was assessed by examining both convergent and discriminant correlations between the WD-FAB scales and scores on same-domain and cross-domain established measures. Results Data quality was good and CAT efficiency was high across both WD-FAB domains. Measurement accuracy was very good for the PF scales; BH scales demonstrated more variability. Construct validity correlations, both convergent and divergent, between all WD-FAB scales and established measures were in the expected direction and range of magnitude. Conclusions The data quality, CAT efficacy, person fit and construct validity of the WD-FAB scales were well supported and suggest that the WD-FAB could be used to assess physical and behavioral health function related to work disability. Variation in scale performance suggests the need for future work on item replenishment and refinement, particularly regarding the Self-Efficacy scale. PMID:25528263

Care provider allocation on admissions to acute mental health wards: The development and validation of the Admission Team Score List.

PubMed

van den Berg, Sjobha R N; Stringer, Barbara; van de Sande, Roland; Draisma, Stasja

2018-05-18

Currently, support tools are lacking to prioritize steps in the care coordination process to enable safe practice and effective clinical pathways in the first phase of acute psychiatric admissions. This study describes the development, validity, and reliability of an acute care coordination support tool, the Admission Team Score List (ATSL). The ATSL assists in care provider allocation during admissions. Face validity and feasibility of the ATSL were tested in 77 acute admissions. Endscores of filled out ATSL's were translated to recommended team compositions. These ATSL team (ATSL-T) compositions were compared to the actually present team (AP-T) and the most preferred team (MP-T) composition in hindsight. Consistency between the ATSL-T and the MP-T was substantial; K w  = 0.70, P < 0.001, 95% CI [0.55-0.84]. The consistency between the ATSL-T and AP-T was moderate; K w  = 0.43, P < 0.001, 95% CI [0.23-0.62]. The ATSL has an adequate (inter-rater) reliability; ICC = 0.90, P < 0.001, 95% CI [0.65-0.91]. The ATSL study is an important step to promote safety and efficient care based on care provider allocation, for service users experiencing an acute admission. The ATSL may stimulate structured clinical decision-making during the hectic process around acute psychiatric admissions. © 2018 Australian College of Mental Health Nurses Inc.

Mother reports of maternal support following child sexual abuse: Preliminary psychometric data on the Maternal Self-report Support Questionnaire (MSSQ).

PubMed

Smith, Daniel W; Sawyer, Genelle K; Jones, Lisa M; Cross, Theodore; McCart, Michael R; Ralston, M Elizabeth

2010-10-01

Maternal support is an important factor in predicting outcomes following disclosure of child sexual abuse; however, definition of the construct has been unclear and existing measures of maternal support are utilized inconsistently and have limited psychometric data. The purpose of this study was to develop a reliable and valid mother-report measure for assessing maternal support following the disclosure of child sexual abuse. Data from 2 very similar samples of mother-child pairs seeking forensic evaluation following the discovery of child sexual abuse were combined, resulting in a final sample of 246. Exploratory factor analysis resulted in two reliable 7-item factors labeled "Emotional Support" and "Blame/Doubt," each of which had acceptable internal consistency. Analyses with a child-report measure of general maternal support the construct validity of the MSSQ. Concurrent validity analyses revealed unique relations with maternal ratings of child behavior problems and case characteristic data. The study resulted in the development of a brief, easily scored self-report measure of maternal support with reasonable preliminary psychometric properties that could easily be utilized in other studies of sexually abused children. Adoption of this promising measure in future research will reduce the lack of cross-study measurement comparability that has characterized the maternal support literature to date, increase the feasibility of expanding upon current literature on maternal support, and may produce important information leading to clinical and theoretical innovation. Copyright © 2010. Published by Elsevier Ltd.

Portable fluorescence lifetime spectroscopy system for in-situ interrogation of biological tissues

NASA Astrophysics Data System (ADS)

Saito Nogueira, Marcelo; Cosci, Alessandro; Teixeira Rosa, Ramon Gabriel; Salvio, Ana Gabriela; Pratavieira, Sebastião; Kurachi, Cristina

2017-12-01

Fluorescence spectroscopy and lifetime techniques are potential methods for optical diagnosis and characterization of biological tissues with an in-situ, fast, and noninvasive interrogation. Several diseases may be diagnosed due to differences in the fluorescence spectra of targeted fluorophores, when, these spectra are similar, considering steady-state fluorescence, others may be detected by monitoring their fluorescence lifetime. Despite this complementarity, most of the current fluorescence lifetime systems are not robust and portable, and not being feasible for clinical applications. We describe the assembly of a fluorescence lifetime spectroscopy system in a suitcase, its characterization, and validation with clinical measurements of skin lesions. The assembled system is all encased and robust, maintaining its mechanical, electrical, and optical stability during transportation, and is feasible for clinical measurements. The instrument response function measured was about 300 ps, and the system is properly calibrated. At the clinical study, the system showed to be reliable, and the achieved spectroscopy results support its potential use as an auxiliary tool for skin diagnostics.

Reliability and validity of non-radiographic methods of thoracic kyphosis measurement: a systematic review.

PubMed

Barrett, Eva; McCreesh, Karen; Lewis, Jeremy

2014-02-01

A wide array of instruments are available for non-invasive thoracic kyphosis measurement. Guidelines for selecting outcome measures for use in clinical and research practice recommend that properties such as validity and reliability are considered. This systematic review reports on the reliability and validity of non-invasive methods for measuring thoracic kyphosis. A systematic search of 11 electronic databases located studies assessing reliability and/or validity of non-invasive thoracic kyphosis measurement techniques. Two independent reviewers used a critical appraisal tool to assess the quality of retrieved studies. Data was extracted by the primary reviewer. The results were synthesized qualitatively using a level of evidence approach. 27 studies satisfied the eligibility criteria and were included in the review. The reliability, validity and both reliability and validity were investigated by sixteen, two and nine studies respectively. 17/27 studies were deemed to be of high quality. In total, 15 methods of thoracic kyphosis were evaluated in retrieved studies. All investigated methods showed high (ICC ≥ .7) to very high (ICC ≥ .9) levels of reliability. The validity of the methods ranged from low to very high. The strongest levels of evidence for reliability exists in support of the Debrunner kyphometer, Spinal Mouse and Flexicurve index, and for validity supports the arcometer and Flexicurve index. Further reliability and validity studies are required to strengthen the level of evidence for the remaining methods of measurement. This should be addressed by future research. Copyright © 2013 Elsevier Ltd. All rights reserved.

Improving the quality of discrete-choice experiments in health: how can we assess validity and reliability?

PubMed

Janssen, Ellen M; Marshall, Deborah A; Hauber, A Brett; Bridges, John F P

2017-12-01

The recent endorsement of discrete-choice experiments (DCEs) and other stated-preference methods by regulatory and health technology assessment (HTA) agencies has placed a greater focus on demonstrating the validity and reliability of preference results. Areas covered: We present a practical overview of tests of validity and reliability that have been applied in the health DCE literature and explore other study qualities of DCEs. From the published literature, we identify a variety of methods to assess the validity and reliability of DCEs. We conceptualize these methods to create a conceptual model with four domains: measurement validity, measurement reliability, choice validity, and choice reliability. Each domain consists of three categories that can be assessed using one to four procedures (for a total of 24 tests). We present how these tests have been applied in the literature and direct readers to applications of these tests in the health DCE literature. Based on a stakeholder engagement exercise, we consider the importance of study characteristics beyond traditional concepts of validity and reliability. Expert commentary: We discuss study design considerations to assess the validity and reliability of a DCE, consider limitations to the current application of tests, and discuss future work to consider the quality of DCEs in healthcare.

Measuring perceived exercise capability and investigating its relationship with childhood obesity: a feasibility study.

PubMed

Taylor, M J; Arriscado, D; Vlaev, I; Taylor, D; Gately, P; Darzi, A

2016-01-01

According to the COM-B ('Capability', 'Opportunity', 'Motivation' and 'Behaviour') model of behaviour, three factors are essential for behaviour to occur: capability, opportunity and motivation. Obese children are less likely to feel capable of exercising. The implementation of a new methodological approach to investigate the relationship between perceived exercise capability (PEC) and childhood obesity was conducted, which involved creating a new instrument, and demonstrating how it can be used to measure obesity intervention outcomes. A questionnaire aiming to measure perceived exercise capability, opportunity and motivation was systematically constructed using the COM-B model and administered to 71 obese children (aged 9-17 years (12.24±0.2.01), body mass index (BMI) standard deviation scores (SDS) 2.80±0.660) at a weight-management camp in northern England. Scale validity and reliability was assessed. Relationships between PEC, as measured by the questionnaire, and BMI SDS were investigated for the children at the weight-management camp, and for 45 Spanish schoolchildren (aged 9-13 years, (10.52±1.23), BMI SDS 0.80±0.99). A pilot study, demonstrating how the questionnaire can be used to measure the effectiveness of an intervention aiming to bring about improved PEC for weight-management camp attendees, was conducted. No participants withdrew from these studies. The questionnaire domain (exercise capability, opportunity and motivation) composite scales were found to have adequate internal consistency (a=0.712-0.796) and construct validity (χ(2)/degrees of freedom=1.55, root mean square error of approximation=0.072, comparative fit index=0.92). Linear regression revealed that low PEC was associated with higher baseline BMI SDS for both UK (b=-0.289, P=0.010) and Spanish (b=-0.446, P=0.047) participants. Pilot study findings provide preliminary evidence for PEC improvements through intervention being achievable, and measurable using the questionnaire. Evidence is presented for reliability and validity of the questionnaire, and for feasibility of its use in the context of a childhood obesity intervention. Future research could investigate the link between PEC and childhood obesity further.

77 FR 56650 - Food and Drug Administration/American Glaucoma Society Workshop on the Validity, Reliability, and...

Federal Register 2010, 2011, 2012, 2013, 2014

2012-09-13

...] Food and Drug Administration/American Glaucoma Society Workshop on the Validity, Reliability, and... entitled ``FDA/American Glaucoma Society (AGS) Workshop on the Validity, Reliability, and Usability of... research. The purpose of this public workshop is to provide a forum for discussing the validity...

Cognitive and adaptive measurement endpoints for clinical trials in mucopolysaccharidoses types I, II, and III: A review of the literature.

PubMed

Janzen, Darren; Delaney, Kathleen A; Shapiro, Elsa G

2017-06-01

Sensitive, reliable measurement instruments are critical for the evaluation of disease progression and new treatments that affect the brain in the mucopolysaccharidoses (MPS). MPS I, II, and III have early onset clinical phenotypes that affect the brain during development and result in devastating cognitive decline and ultimately death without treatment. Comparisons of outcomes are hindered by diverse protocols and approaches to assessment including applicability to international trials necessary in rare diseases. We review both cognitive and adaptive measures with the goal of providing evidence to a Delphi panel to come to a consensus about recommendations for clinical trials for various age groups. The results of the consensus panel are reported in an accompanying article. The following data were gathered (from internet resources and from test manuals) for each measure and summarized in the discussion: reliability, validity, date and adequacy of normative data, applicability of the measure's metrics, cross cultural validity including translations and adaptations, feasibility in the MPS population, familiarity to sites, sensitivity to change, and interpretability. If, resulting from this consensus, standard protocols are used for both natural history and treatment studies, patients, their families, and health care providers will benefit from the ability to compare study outcomes. Copyright © 2017 The Authors. Published by Elsevier Inc. All rights reserved.

A Roadmap for the Development and Validation of ERP Biomarkers in Schizophrenia Research

PubMed Central

Luck, Steven J.; Mathalon, Daniel H.; O'Donnell, Brian F.; Hämäläinen, Matti S.; Spencer, Kevin M.; Javitt, Daniel C.; Uhlhaas, Peter J.

2010-01-01

New efforts to develop treatments for cognitive dysfunction in mental illnesses would benefit enormously from biomarkers that provide sensitive and reliable measures of the neural events underlying cognition. Here we evaluate the promise of event-related potentials (ERPs) as biomarkers of cognitive dysfunction in schizophrenia. We conclude that ERPs have several desirable properties: (a) they provide a direct measure of electrical activity during neurotransmission; (b) their high temporal resolutions makes it possible to measure neural synchrony and oscillations; (c) they are relatively inexpensive and convenient to record; (d) animal models are readily available for several ERP components; (e) decades of research has established the sensitivity and reliability of ERP measures in psychiatric illnesses; and (f) feasibility of large N (>500) multi-site studies has been demonstrated for key measures. Consequently, ERPs may be useful for identifying endophenotypes and defining treatment targets, for evaluating new compounds in animals and in humans, and for identifying individuals who are good candidates for early interventions or for specific treatments. However, several challenges must be overcome before ERPs gain widespread use as biomarkers in schizophrenia research, and we make several recommendations for the research that is necessary to develop and validate ERP-based biomarkers that can have a real impact on treatment development. PMID:21111401

The reliability, validity and feasibility of tools used to screen for caregiver burden: a systematic review.

PubMed

Whalen, Kimberly J; Buchholz, Susan W

The overall objective of this review is to quantitatively measure the psychometric properties and the feasibility of caregiver burden screening tools. The more specific objectives were to determine the reliability, validity as well as feasibility of tools that are used to screen for caregiver burden and strain. This review considered international quantitative research papers that addressed the psychometric properties and feasibility of caregiver burden screening tools. The search strategy aimed to find both published and unpublished studies from 1980-2007 published only in the English language. An initial limited search of MEDLINE and CINAHL was undertaken followed by analysis of the text words contained in the title and abstract and the index terms used to describe the article. A second search identified keywords and index terms across major databases. Third, the reference list of identified reports and articles was searched for additional studies. Each paper was assessed by two independent reviewers for methodological quality prior to inclusion in the review using an appropriate critical appraisal instrument from the Joanna Briggs Institutes' System for the Unified Management, Assessment and Review (SUMARI) package. Because burden is a multidimensional construct defined internationally with a multitude of other terms, only those studies whose title, abstract or keywords contained the search terminology developed for this review were identified for retrieval. The construct of caregiver burden is not standardized, and many terms are used to describe burden. A caregiver is also identified as a carer. Instruments exist in multiple languages and have been tested in multiple populations. A total of 112 papers, experimental and non-experimental in nature, were included in the review. The majority of papers were non-experimental studies that tested or used a caregiver burden screening tool. Because of the nature of these papers, a meta-analysis of the results was not possible. Instead a table is used to depict the 74 caregiver burden screening tools that meet the psychometric and feasibility standards of this review. The Zarit Burden Interview (ZBI), in particular the 22-item version, has been examined the most throughout the literature. In addition to its sound psychometric properties, the ZBI has been widely used across languages and cultures. The significant amount of research that has already been done on psychometric testing of caregiver burden tools has provided a solid foundation for additional research. Although some tools have been well tested, many tools have published limited psychometric properties and feasibility data. The clinician needs to be aware of this and may need to team up with a researcher to obtain additional research data on their specific population before using a minimally tested caregiver burden screening tool. Because caregiver burden is multidimensional and many different terms are used to describe burden, both the clinician and researcher need to be precise in their selection of the appropriate tool for their work.

Validity and Reliability of Field-Based Measures for Assessing Movement Skill Competency in Lifelong Physical Activities: A Systematic Review.

PubMed

Hulteen, Ryan M; Lander, Natalie J; Morgan, Philip J; Barnett, Lisa M; Robertson, Samuel J; Lubans, David R

2015-10-01

It has been suggested that young people should develop competence in a variety of 'lifelong physical activities' to ensure that they can be active across the lifespan. The primary aim of this systematic review is to report the methodological properties, validity, reliability, and test duration of field-based measures that assess movement skill competency in lifelong physical activities. A secondary aim was to clearly define those characteristics unique to lifelong physical activities. A search of four electronic databases (Scopus, SPORTDiscus, ProQuest, and PubMed) was conducted between June 2014 and April 2015 with no date restrictions. Studies addressing the validity and/or reliability of lifelong physical activity tests were reviewed. Included articles were required to assess lifelong physical activities using process-oriented measures, as well as report either one type of validity or reliability. Assessment criteria for methodological quality were adapted from a checklist used in a previous review of sport skill outcome assessments. Movement skill assessments for eight different lifelong physical activities (badminton, cycling, dance, golf, racquetball, resistance training, swimming, and tennis) in 17 studies were identified for inclusion. Methodological quality, validity, reliability, and test duration (time to assess a single participant), for each article were assessed. Moderate to excellent reliability results were found in 16 of 17 studies, with 71% reporting inter-rater reliability and 41% reporting intra-rater reliability. Only four studies in this review reported test-retest reliability. Ten studies reported validity results; content validity was cited in 41% of these studies. Construct validity was reported in 24% of studies, while criterion validity was only reported in 12% of studies. Numerous assessments for lifelong physical activities may exist, yet only assessments for eight lifelong physical activities were included in this review. Generalizability of results may be more applicable if more heterogeneous samples are used in future research. Moderate to excellent levels of inter- and intra-rater reliability were reported in the majority of studies. However, future work should look to establish test-retest reliability. Validity was less commonly reported than reliability, and further types of validity other than content validity need to be established in future research. Specifically, predictive validity of 'lifelong physical activity' movement skill competency is needed to support the assertion that such activities provide the foundation for a lifetime of activity.

Feasibility and Reliability of Two Different Walking Tests in People with Severe Intellectual and Sensory Disabilities

ERIC Educational Resources Information Center

Waninge, A.; Evenhuis, I. J.; van Wijck, R.; van der Schans, C. P.

2011-01-01

Background: The purpose of this study is to describe feasibility and test-retest reliability of the six-minute walking distance test (6MWD) and an adapted shuttle run test (aSRT) in persons with severe intellectual and sensory (multiple) disabilities. Materials and Methods: Forty-seven persons with severe multiple disabilities, with Gross Motor…

Feasibility and Reliability of the Modified Berg Balance Scale in Persons with Severe Intellectual and Visual Disabilities

ERIC Educational Resources Information Center

Waninge, A.; van Wijck, R.; Steenbergen, B.; van der Schans, C. P.

2011-01-01

Background: The purpose of this study was to determine the feasibility and reliability of the modified Berg Balance Scale (mBBS) in persons with severe intellectual and visual disabilities (severe multiple disabilities, SMD) assigned Gross Motor Function Classification System (GMFCS) grades I and II. Method: Thirty-nine participants with SMD and…

Validity and Reliability of Turkish Male Breast Self-Examination Instrument.

PubMed

Erkin, Özüm; Göl, İlknur

2018-04-01

This study aims to measure the validity and reliability of Turkish male breast self-examination (MBSE) instrument. The methodological study was performed in 2016 at Ege University, Faculty of Nursing, İzmir, Turkey. The MBSE includes ten steps. For validity studies, face validity, content validity, and construct validity (exploratory factor analysis) were done. For reliability study, Kuder Richardson was calculated. The content validity index was found to be 0.94. Kendall W coefficient was 0.80 (p=0.551). The total variance explained by the two factors was found to be 63.24%. Kuder Richardson 21 was done for reliability study and found to be 0.97 for the instrument. The final instrument included 10 steps and two stages. The Turkish version of MBSE is a valid and reliable instrument for early diagnose. The MBSE can be used in Turkish speaking countries and cultures with two stages and 10 steps.

Consistency of clinical biomechanical measures between three different institutions: implications for multi-center biomechanical and epidemiological research.

PubMed

Myer, Gregory D; Wordeman, Samuel C; Sugimoto, Dai; Bates, Nathaniel A; Roewer, Benjamin D; Medina McKeon, Jennifer M; DiCesare, Christopher A; Di Stasi, Stephanie L; Barber Foss, Kim D; Thomas, Staci M; Hewett, Timothy E

2014-05-01

Multi-center collaborations provide a powerful alternative to overcome the inherent limitations to single-center investigations. Specifically, multi-center projects can support large-scale prospective, longitudinal studies that investigate relatively uncommon outcomes, such as anterior cruciate ligament injury. This project was conceived to assess within- and between-center reliability of an affordable, clinical nomogram utilizing two-dimensional video methods to screen for risk of knee injury. The authors hypothesized that the two-dimensional screening methods would provide good-to-excellent reliability within and between institutions for assessment of frontal and sagittal plane biomechanics. Nineteen female, high school athletes participated. Two-dimensional video kinematics of the lower extremity during a drop vertical jump task were collected on all 19 study participants at each of the three facilities. Within-center and between-center reliability were assessed with intra- and inter-class correlation coefficients. Within-center reliability of the clinical nomogram variables was consistently excellent, but between-center reliability was fair-to-good. Within-center intra-class correlation coefficient for all nomogram variables combined was 0.98, while combined between-center inter-class correlation coefficient was 0.63. Injury risk screening protocols were reliable within and repeatable between centers. These results demonstrate the feasibility of multi-site biomechanical studies and establish a framework for further dissemination of injury risk screening algorithms. Specifically, multi-center studies may allow for further validation and optimization of two-dimensional video screening tools. 2b.

Reliability of a rating procedure to monitor industry self-regulation codes governing alcohol advertising content.

PubMed

Babor, Thomas F; Xuan, Ziming; Proctor, Dwayne

2008-03-01

The purposes of this study were to develop reliable procedures to monitor the content of alcohol advertisements broadcast on television and in other media, and to detect violations of the content guidelines of the alcohol industry's self-regulation codes. A set of rating-scale items was developed to measure the content guidelines of the 1997 version of the U.S. Beer Institute Code. Six focus groups were conducted with 60 college students to evaluate the face validity of the items and the feasibility of the procedure. A test-retest reliability study was then conducted with 74 participants, who rated five alcohol advertisements on two occasions separated by 1 week. Average correlations across all advertisements using three reliability statistics (r, rho, and kappa) were almost all statistically significant and the kappas were good for most items, which indicated high test-retest agreement. We also found high interrater reliabilities (intraclass correlations) among raters for item-level and guideline-level violations, indicating that regardless of the specific item, raters were consistent in their general evaluations of the advertisements. Naïve (untrained) raters can provide consistent (reliable) ratings of the main content guidelines proposed in the U.S. Beer Institute Code. The rating procedure may have future applications for monitoring compliance with industry self-regulation codes and for conducting research on the ways in which alcohol advertisements are perceived by young adults and other vulnerable populations.

Psychometric instrumentation: reliability and validity of instruments used for clinical practice, evidence-based practice projects and research studies.

PubMed

Mayo, Ann M

2015-01-01

It is important for CNSs and other APNs to consider the reliability and validity of instruments chosen for clinical practice, evidence-based practice projects, or research studies. Psychometric testing uses specific research methods to evaluate the amount of error associated with any particular instrument. Reliability estimates explain more about how well the instrument is designed, whereas validity estimates explain more about scores that are produced by the instrument. An instrument may be architecturally sound overall (reliable), but the same instrument may not be valid. For example, if a specific group does not understand certain well-constructed items, then the instrument does not produce valid scores when used with that group. Many instrument developers may conduct reliability testing only once, yet continue validity testing in different populations over many years. All CNSs should be advocating for the use of reliable instruments that produce valid results. Clinical nurse specialists may find themselves in situations where reliability and validity estimates for some instruments that are being utilized are unknown. In such cases, CNSs should engage key stakeholders to sponsor nursing researchers to pursue this most important work.

Cross-cultural adaptation and validation of the Korean version of the Michigan hand questionnaire.

PubMed

Roh, Young Hak; Yang, Bo Kyu; Noh, Jung Ho; Baek, Goo Hyun; Song, Cheol Ho; Gong, Hyun Sik

2011-09-01

The Michigan hand questionnaire (MHQ) is increasingly being used and has been adapted cross-culturally in some Western and Asian countries, but the validation process for an Asian translation of MHQ has not been well described. In this study, we translated and adapted the original MHQ cross-culturally to produce a Korean version, and then assessed the validity and reliability of the Korean version of the MHQ. A total of 176 patients with common hand disorders completed the Korean version of the MHQ and the Disabilities of the Arm, Shoulder, and Hand questionnaire. We included the pain score assessed by a visual analog scale during activity, range of motion, measurement of grip strength, and subjective assessment of the functional state by use of Cooney's scale in the validation process. There were no major linguistic or cultural problems during forward and backward translations of the MHQ, except for a minor change owing to cultural discrepancies in eating, such as the dominant hand using a spoon and chopsticks instead of both hands using a knife and fork. All subscales of the MHQ showed satisfactory internal consistency. The reproducibility test showed no significant difference. The construct validity revealed a moderate to strong correlation between every subscale of the Korean MHQ against DASH disabilities and symptoms. The aesthetic and satisfaction domains, unique domains of the MHQ, had little correlation with the objective measure of the pain visual analog scale, grip strength, motion and subjective functional state. The Korean version of MHQ showed satisfactory internal consistency, test-retest reliability, and validity and demonstrated a significant correlation with the patient-based upper extremity questionnaire and clinical assessment. We found the application and evaluation of the instrument to be feasible and understandable among patients in Korea. Copyright © 2011 American Society for Surgery of the Hand. Published by Elsevier Inc. All rights reserved.

Establishing the feasibility of direct observation in the assessment of tics in children with chronic tic disorders.

PubMed

Himle, Michael B; Chang, Susanna; Woods, Douglas W; Pearlman, Amanda; Buzzella, Brian; Bunaciu, Liviu; Piacentini, John C

2006-01-01

Behavior analysis has been at the forefront in establishing effective treatments for children and adults with chronic tic disorders. As is customary in behavior analysis, the efficacy of these treatments has been established using direct-observation assessment methods. Although behavior-analytic treatments have enjoyed acceptance and integration into mainstream health care practices for tic disorders (e.g., psychiatry and neurology), the use of direct observation as a primary assessment tool has been neglected in favor of less objective methods. Hesitation to use direct observation appears to stem largely from concerns about the generalizability of clinic observations to other settings (e.g., home) and a lack of consensus regarding the most appropriate and feasible techniques for conducting and scoring direct observation. The purpose of the current study was to evaluate and establish a reliable, valid, and feasible direct-observation protocol capable of being transported to research and clinical settings. A total of 43 children with tic disorders, collected from two outpatient specialty clinics, were assessed using direct (videotape samples) and indirect (Yale Global Tic Severity Scale; YGTSS) methods. Videotaped observation samples were collected across 3 consecutive weeks and two different settings (clinic and home), were scored using both exact frequency counts and partial-interval coding, and were compared to data from a common indirect measure of tic severity (the YGTSS). In addition, various lengths of videotaped segments were scored to determine the optimal observation length. Results show that (a) clinic-based observations correspond well to home-based observations, (b) brief direct-observation segments scored with time-sampling methods reliably quantified tics, and (c) indirect methods did not consistently correspond with the direct methods.

Feasibility of an interactive voice response system for monitoring depressive symptoms in a lower-middle income Latin American country.

PubMed

Janevic, Mary R; Aruquipa Yujra, Amparo C; Marinec, Nicolle; Aguilar, Juvenal; Aikens, James E; Tarrazona, Rosa; Piette, John D

2016-01-01

Innovative, scalable solutions are needed to address the vast unmet need for mental health care in low- and middle-income countries (LMICs). We conducted a feasibility study of a 14-week automated telephonic interactive voice response (IVR) depression self-care service among Bolivian primary care patients with at least moderately severe depressive symptoms. We analyzed IVR call completion rates, the reliability and validity of IVR-collected data, and participant satisfaction. Of the 32 participants, the majority were women (78 % or 25/32) and non-indigenous (75 % or 24/32). Participants had moderate depressive symptoms at baseline (PHQ-8 score mean 13.3, SD = 3.5) and reported good or fair general health status (88 % or 28/32). Fifty-four percent of weekly IVR calls (approximately 7 out of 13 active call-weeks) were completed. Neither PHQ-8 scores nor IVR call completion differed significantly by ethnicity, education, self-reported depression diagnosis, self-reported overall health, number of chronic conditions, or health literacy. The reliability for IVR-collected PHQ-8 scores was good (Cronbach's alpha = 0.83). Virtually every participant (97 %) was "mostly" or "very" satisfied with the program. Many described the program as beneficial for their mood and self-care, albeit limited by some technological difficulties and the lack of human interaction. Findings suggest that IVR could feasibly be used to provide monitoring and self-care education to depressed patients in Bolivia. An expanded stepped-care service offering contact with lay health workers for more depressed individuals and expanded mHealth content may foster greater patient engagement and enhance its therapeutic value while remaining cost-effective. Trial registration ISRCTN ISRCTN 18403214. Registered 14 September 2016. Retrospectively registered.

Development of a prognostic scale for severely hemiplegic stroke patients in a rehabilitation hospital.

PubMed

Hirano, Yoshitake; Nitta, Osamu; Hayashi, Takeshi; Takahashi, Hidetoshi; Miyazaki, Yasuhiro; Kigawa, Hiroshi

2017-07-01

For patients with severe hemiplegia in a rehabilitation hospital, early prediction of the functional prognosis and outcomes is challenging. The purpose of this study was to create and verify a prognostic scale in severely hemiplegic stroke patients and allowing for prediction of (1) the ability to walk at the time of hospital discharge, (2) the ability to carry out activities of daily living (ADL), and (3) feasibility of home discharge. The study was conducted on 80 severely hemiplegic stroke patients. A prognostic scale was created as an analysis method using the following items: mini-mental state examination (MMSE) at the time of admission, modified NIH stroke scale (m-NIHSS); trunk control test (TCT); and the ratio of the knee extensor strength on the non-paralyzed side to the body weight (KES/BW-US). We verified the reliability and validity of this scale. We established a prognostic scale using the MMSE, m-NIHSS, TCT, and KES/BW-US. A score of 56.8 or higher on the prognostic scale suggested that the patient would be able to walk and that assistance with ADL would be unnecessary at the time of hospital discharge. In addition, a score of 41.3 points indicated that the patient's return home was feasible. The reliability and the results were in good agreement. These findings showed that the ability or inability to walk was predictable in 85%, the need of assistance with ADL in 82.5%, and the feasibility of home return in 76.3% of cases. At the time of admission, four evaluation items permitted the prediction of three outcomes at time of discharge. Our formula predicts three outcomes with an accuracy of more than 76%. Copyright © 2017 Elsevier B.V. All rights reserved.

Reliability and validity of generalizable skills instruments for students who are deaf, blind, or visually impaired.

PubMed

Loeding, B L; Greenan, J P

1998-12-01

The study examined the validity and reliability of four assessments, with three instruments per domain. Domains included generalizable mathematics, communication, interpersonal relations, and reasoning skills. Participants were deaf, legally blind, or visually impaired students enrolled in vocational classes at residential secondary schools. The researchers estimated the internal consistency reliability, test-retest reliability, and construct validity correlations of three subinstruments: student self-ratings, teacher ratings, and performance assessments. The data suggest that these instruments are highly internally consistent measures of generalizable vocational skills. Four performance assessments have high-to-moderate test-retest reliability estimates, and were generally considered to possess acceptable validity and reliability.

Internal Consistency, Retest Reliability, and their Implications For Personality Scale Validity

PubMed Central

McCrae, Robert R.; Kurtz, John E.; Yamagata, Shinji; Terracciano, Antonio

2010-01-01

We examined data (N = 34,108) on the differential reliability and validity of facet scales from the NEO Inventories. We evaluated the extent to which (a) psychometric properties of facet scales are generalizable across ages, cultures, and methods of measurement; and (b) validity criteria are associated with different forms of reliability. Composite estimates of facet scale stability, heritability, and cross-observer validity were broadly generalizable. Two estimates of retest reliability were independent predictors of the three validity criteria; none of three estimates of internal consistency was. Available evidence suggests the same pattern of results for other personality inventories. Internal consistency of scales can be useful as a check on data quality, but appears to be of limited utility for evaluating the potential validity of developed scales, and it should not be used as a substitute for retest reliability. Further research on the nature and determinants of retest reliability is needed. PMID:20435807

Maximising harm reduction in early specialty training for general practice: validation of a safety checklist

PubMed Central

2012-01-01

Background Making health care safer is a key policy priority worldwide. In specialty training, medical educators may unintentionally impact on patient safety e.g. through failures of supervision; providing limited feedback on performance; and letting poorly developed behaviours continue unchecked. Doctors-in-training are also known to be susceptible to medical error. Ensuring that all essential educational issues are addressed during training is problematic given the scale of the tasks to be undertaken. Human error and the reliability of local systems may increase the risk of safety-critical topics being inadequately covered. However adherence to a checklist reminder may improve the reliability of task delivery and maximise harm reduction. We aimed to prioritise the most safety-critical issues to be addressed in the first 12-weeks of specialty training in the general practice environment and validate a related checklist reminder. Methods We used mixed methods with different groups of GP educators (n = 127) and specialty trainees (n = 9) in two Scottish regions to prioritise, develop and validate checklist content. Generation and refinement of checklist themes and items were undertaken on an iterative basis using a range of methods including small group work in dedicated workshops; a modified-Delphi process; and telephone interviews. The relevance of potential checklist items was rated using a 4-point scale content validity index to inform final inclusion. Results 14 themes (e.g. prescribing safely; dealing with medical emergency; implications of poor record keeping; and effective & safe communication) and 47 related items (e.g. how to safety-net face-to-face or over the telephone; knowledge of practice systems for results handling; recognition of harm in children) were judged to be essential safety-critical educational issues to be covered. The mean content validity index ratio was 0.98. Conclusion A checklist was developed and validated for educational supervisors to assist in the reliable delivery of safety-critical educational issues in the opening 12-week period of training, and aligned with national curriculum competencies. The tool can also be adapted for use as a self-assessment instrument by trainees to guide patient safety-related learning needs. Dissemination and implementation of the checklist and self-rating scale are proceeding on a national, voluntary basis with plans to evaluate its feasibility and educational impact. PMID:22721273

Maximising harm reduction in early specialty training for general practice: validation of a safety checklist.

PubMed

Bowie, Paul; McKay, John; Kelly, Moya

2012-06-21

Making health care safer is a key policy priority worldwide. In specialty training, medical educators may unintentionally impact on patient safety e.g. through failures of supervision; providing limited feedback on performance; and letting poorly developed behaviours continue unchecked. Doctors-in-training are also known to be susceptible to medical error. Ensuring that all essential educational issues are addressed during training is problematic given the scale of the tasks to be undertaken. Human error and the reliability of local systems may increase the risk of safety-critical topics being inadequately covered. However adherence to a checklist reminder may improve the reliability of task delivery and maximise harm reduction. We aimed to prioritise the most safety-critical issues to be addressed in the first 12-weeks of specialty training in the general practice environment and validate a related checklist reminder. We used mixed methods with different groups of GP educators (n=127) and specialty trainees (n=9) in two Scottish regions to prioritise, develop and validate checklist content. Generation and refinement of checklist themes and items were undertaken on an iterative basis using a range of methods including small group work in dedicated workshops; a modified-Delphi process; and telephone interviews. The relevance of potential checklist items was rated using a 4-point scale content validity index to inform final inclusion. 14 themes (e.g. prescribing safely; dealing with medical emergency; implications of poor record keeping; and effective & safe communication) and 47 related items (e.g. how to safety-net face-to-face or over the telephone; knowledge of practice systems for results handling; recognition of harm in children) were judged to be essential safety-critical educational issues to be covered. The mean content validity index ratio was 0.98. A checklist was developed and validated for educational supervisors to assist in the reliable delivery of safety-critical educational issues in the opening 12-week period of training, and aligned with national curriculum competencies. The tool can also be adapted for use as a self-assessment instrument by trainees to guide patient safety-related learning needs. Dissemination and implementation of the checklist and self-rating scale are proceeding on a national, voluntary basis with plans to evaluate its feasibility and educational impact.

Study on the Validity and Reliability of Melbourne Decision Making Scale in Turkey

ERIC Educational Resources Information Center

Çolakkadioglu, Oguzhan; Deniz, M. Engin

2015-01-01

This study is to analyze the validity and reliability of Melbourne Decision Making Questionnaire (MDMQ). The sample consisted of 650 university students. The structural validity of the MDMQ, as well as correlations among its sub-scales, measure-bound validity, internal consistency, item total correlations and test-retest reliability coefficients…

A Model for Estimating the Reliability and Validity of Criterion-Referenced Measures.

ERIC Educational Resources Information Center

Edmonston, Leon P.; Randall, Robert S.

A decision model designed to determine the reliability and validity of criterion referenced measures (CRMs) is presented. General procedures which pertain to the model are discussed as to: Measures of relationship, Reliability, Validity (content, criterion-oriented, and construct validation), and Item Analysis. The decision model is presented in…

The Chinese-version of the CARE Measure reliably differentiates between doctors in primary care: a cross-sectional study in Hong Kong

PubMed Central

2011-01-01

Background The Consultation and Relational Empathy (CARE) Measure is a widely used patient-rated experience measure which has recently been translated into Chinese and has undergone preliminary qualitative and quantitative validation. The objective of this study was to determine the reliability of the Chinese-version of the CARE Measure in reliably differentiating between doctors in a primary care setting in Hong Kong Methods Data were collected from 984 primary care patients attending 20 doctors with differing levels of training in family medicine in 5 public clinics in Hong Kong. The acceptability of the Chinese-CARE measure to patients was assessed. The reliability of the measure in discriminating effectively between doctors was analysed by Generalisability-theory (G-Theory) Results The items in the Chinese-CARE measure were regarded as important by patients and there were few 'not applicable' responses. The measure showed high internal reliability (coefficient 0.95) and effectively differentiated between doctors with only 15-20 patient ratings per doctor (inter-rater reliability > 0.8). Doctors' mean CARE measure scores varied widely, ranging from 24.1 to 45.9 (maximum possible score 50) with a mean of 34.6. CARE Measure scores were positively correlated with level of training in family medicine (Spearman's rho 0.493, p < 0.05). Conclusion These data demonstrate the acceptability, feasibility and reliability of using the Chinese-CARE Measure in primary care in Hong Kong to differentiate between doctors interpersonal competencies. Training in family medicine appears to enhance these key interpersonal skills. PMID:21631927

Progress in the RAMI analysis of a conceptual LHCD system for DEMO

NASA Astrophysics Data System (ADS)

Mirizzi, F.

2014-02-01

Reliability, Availability, Maintainability and Inspectability (RAMI) concepts and techniques, that acquired great importance during the first manned space missions, have been progressively extended to industrial, scientific and consumer equipments to assure them satisfactory performances and lifetimes. In the design of experimental facilities, like tokamaks, mainly aimed at demonstrating validity and feasibility of scientific theories, RAMI analysis has been often left aside. DEMO, the future prototype fusion reactors, will be instead designed for steadily delivering electrical energy to commercial grids, so that the RAMI aspects will assume an absolute relevance since their initial design phases. A preliminary RAMI analysis of the LHCD system for the conceptual EU DEMO reactor is given in the paper.

Patient-Centered Technological Assessment and Monitoring of Depression for Low-Income Patients

PubMed Central

Wu, Shinyi; Vidyanti, Irene; Liu, Pai; Hawkins, Caitlin; Ramirez, Magaly; Guterman, Jeffrey; Gross-Schulman, Sandra; Sklaroff, Laura Myerchin; Ell, Kathleen

2014-01-01

Depression is a significant challenge for ambulatory care because it worsens health status and outcomes, increases health care utilizations and costs, and elevates suicide risk. An automatic telephonic assessment (ATA) system that links with tasks and alerts to providers may improve quality of depression care and increase provider productivity. We used ATA system in a trial to assess and monitor depressive symptoms of 444 safety-net primary care patients with diabetes. We assessed system properties, evaluated preliminary clinical outcomes, and estimated cost savings. The ATA system is feasible, reliable, valid, safe, and likely cost-effective for depression screening and monitoring for low-income primary care population. PMID:24525531

Validation of the 'United Registries for Clinical Assessment and Research' [UR-CARE], a European Online Registry for Clinical Care and Research in Inflammatory Bowel Disease.

PubMed

Burisch, Johan; Gisbert, Javier P; Siegmund, Britta; Bettenworth, Dominik; Thomsen, Sandra Bohn; Cleynen, Isabelle; Cremer, Anneline; Ding, Nik John Sheng; Furfaro, Federica; Galanopoulos, Michail; Grunert, Philip Christian; Hanzel, Jurij; Ivanovski, Tamara Knezevic; Krustins, Eduards; Noor, Nurulamin; O'Morain, Neil; Rodríguez-Lago, Iago; Scharl, Michael; Tua, Julia; Uzzan, Mathieu; Ali Yassin, Nuha; Baert, Filip; Langholz, Ebbe

2018-04-27

The 'United Registries for Clinical Assessment and Research' [UR-CARE] database is an initiative of the European Crohn's and Colitis Organisation [ECCO] to facilitate daily patient care and research studies in inflammatory bowel disease [IBD]. Herein, we sought to validate the database by using fictional case histories of patients with IBD that were to be entered by observers of varying experience in IBD. Nineteen observers entered five patient case histories into the database. After 6 weeks, all observers entered the same case histories again. For each case history, 20 key variables were selected to calculate the accuracy for each observer. We assumed that the database was such that ≥ 90% of the entered data would be correct. The overall proportion of correctly entered data was calculated using a beta-binomial regression model to account for inter-observer variation and compared to the expected level of validity. Re-test reliability was assessed using McNemar's test. For all case histories, the overall proportion of correctly entered items and their confidence intervals included the target of 90% (Case 1: 92% [88-94%]; Case 2: 87% [83-91%]; Case 3: 93% [90-95%]; Case 4: 97% [94-99%]; Case 5: 91% [87-93%]). These numbers did not differ significantly from those found 6 weeks later [NcNemar's test p > 0.05]. The UR-CARE database appears to be feasible, valid and reliable as a tool and easy to use regardless of prior user experience and level of clinical IBD experience. UR-CARE has the potential to enhance future European collaborations regarding clinical research in IBD.

Assessment protocols of maximum oxygen consumption in young people with Down syndrome--a review.

PubMed

Seron, Bruna Barboza; Greguol, Márcia

2014-03-01

Maximum oxygen consumption is considered the gold standard measure of cardiorespiratory fitness. Young people with Down syndrome (DS) present low values of this indicator compared to their peers without disabilities and to young people with an intellectual disability but without DS. The use of reliable and valid assessment methods provides more reliable results for the diagnosis of cardiorespiratory fitness and the response of this variable to exercise. The aim of the present study was to review the literature on the assessment protocols used to measure maximum oxygen consumption in children and adolescents with Down syndrome giving emphasis to the protocols used, the validation process and their feasibility. The search was carried out in eight electronic databases--Scopus, Medline-Pubmed, Web of science, SportDiscus, Cinhal, Academic Search Premier, Scielo, and Lilacs. The inclusion criteria were: (a) articles which assessed VO2peak and/or VO2max (independent of the validation method), (b) samples composed of children and/or adolescents with Down syndrome, (c) participants of up to 20 years old, and (d) studies performed after 1990. Fifteen studies were selected and, of these, 11 measured the VO2peak using tests performed in a laboratory, 2 used field tests and the remaining 2 used both laboratory and field tests. The majority of the selected studies used maximal tests and conducted familiarization sessions. All the studies took into account the clinical conditions that could hamper testing or endanger the individuals. However, a large number of studies used tests which had not been specifically validated for the evaluated population. Finally, the search emphasized the small number of studies which use field tests to evaluate oxygen consumption. Copyright © 2013 Elsevier Ltd. All rights reserved.

PedsQL™ Eosinophilic Esophagitis Module: Feasibility, Reliability and Validity

PubMed Central

Franciosi, James P.; Hommel, Kevin A.; Bendo, Cristiane B.; King, Eileen C.; Collins, Margaret H.; Eby, Michael D.; Marsolo, Keith; Abonia, J. Pablo; von Tiehl, Karl F.; Putnam, Philip E.; Greenler, Alexandria J.; Greenberg, Allison B.; Bryson, Ronald A.; Davis, Carla M.; Olive, Anthony P.; Gupta, Sandeep K.; Erwin, Elizabeth A.; Klinnert, Mary D.; Spergel, Jonathan M.; Denham, Jolanda M.; Furuta, Glenn T.; Rothenberg, Marc E.; Varni, James W.

2014-01-01

Objective Eosinophilic esophagitis (EoE) is a chronic esophageal inflammatory condition with a paucity of information on health-related quality of life (HRQOL). The objective of the study was to report on the measurement properties of the PedsQL™ EoE Module. Methods The PedsQL™ EoE Module was completed in a multisite study by 196 pediatric EoE patients and 262 EoE parents. Results The PedsQL™ EoE Module scales evidenced excellent feasibility (0.6%–3.1% missing), excellent group comparison reliability across total scale scores (patient α = 0.93; parent proxy α = 0.94), good reliability for the seven individual scales (patient α = 0.75–0.87; parent proxy α = 0.81–0.92), excellent test-retest reliability (patient ICC = 0.88; parent ICC= 0.82), demonstrated no floor effects and low ceiling effects, and demonstrated a high percentage of scaling success for most scales. Intercorrelations with the PedsQL™ Generic Core Scales were in the medium (0.30) to large (0.50) range. PedsQL™ EoE Module scores were worse among patients with active histologic disease (> 5 eos/hpf) compared to those in remission (patient self-report: 63.3 vs. 69.9 [p<0.05]; parent proxy-report: 65.1 vs. 72.3 [p<0.01]), and those treated with dietary restrictions compared to those with no restrictions (patient self-report: 61.6 vs. 74.3 [p< 0.01]; parent proxy-report: 65.5 vs. 74.7 [p<0.01]). Conclusions The results demonstrate excellent measurement properties of the PedsQL™ EoE Module. Patients with active histologic disease and those treated with dietary restrictions demonstrated worse PedsQL™ scores. The PedsQL™ EoE Module may be utilized in evaluation of pediatric EoE disease-specific HRQOL in clinical research and practice. PMID:23478422

Reliability and Validity of the Turkish Version of the PedsQL 3.0 Cancer Module for 2- to 7-Year-Old and the PedsQL 4.0 Generic Core Scales for 5- to 7-Year-Old: The Hacettepe University Experience

PubMed Central

Yıldız Kabak, Vesile; Yakut, Yavuz; Çetin, Mualla; Düger, Tülin

2016-01-01

Objective: The aim of this study was to investigate the reliability and validity of the Turkish version of the Pediatric Quality of Life Inventory (PedsQL) 3.0 Cancer Module for 2- to 7-year-old and the PedsQL 4.0 Generic Core Scales for 5- to 7-year-old in childhood cancer. Materials and Methods: The PedsQL 3.0 Cancer Module and PedsQL 4.0 Generic Core Scales were administered to children with cancer and their parents at Hacettepe University. Internal consistency was determined by using Cronbach’s alpha and test-retest reliability was determined by using the intraclass correlation coefficient (ICC). Construct validity was assessed by comparing the results of the PedsQL 3.0 Cancer Module with those of the PedsQL 4.0 Generic Core Scales. Results: Cronbach’s alpha of the PedsQL 3.0 Cancer Module varied from 0.803 to 0.873 and that of the PedsQL 4.0 Generic Core Scales from 0.665 to 0.841. Test-retest ICC values of the PedsQL 3.0 Cancer Module varied from 0.877 to 0.949 and those of the PedsQL 4.0 Generic Core Scales from 0.681 to 0.824. The correlation of the PedsQL 3.0 Cancer Module with subscale scores of the PedsQL 4.0 Generic Core Scales showed that there were excellent to fair correlations between the two scales. The relationship between parent proxy-report and child self-report of the PedsQL 3.0 Cancer Module had very good correlation (r=0.694, p<0.001), as did the PedsQL 4.0 Generic Core Scales (r=0.540, p=0.002). Conclusion: This study demonstrated the reliability, validity, and feasibility of the Turkish version of the PedsQL 3.0 Cancer Module in 2- to 4-year-old and 5- to 7-year-old and the PedsQL 4.0 Generic Core Scales in 5- to 7-year-old in childhood cancer. PMID:27095607

Feasibility and Reliability of Tests Measuring Health-Related Physical Fitness in Children with Moderate to Severe Levels of Intellectual Disability

ERIC Educational Resources Information Center

Wouters, Marieke; van der Zanden, Anna M.; Evenhuis, Heleen M.; Hilgenkamp, Thessa I. M.

2017-01-01

Physical fitness is an important marker for health. In this study we investigated the feasibility and reliability of health-related physical fitness tests in children with moderate to severe levels of intellectual disability. Thirty-nine children (2-18 yrs) performed tests for muscular strength and endurance, the modified 6-minute walk test (6mwt)…

OVERALL PROCEDURES PROTOCOL AND PATIENT ENROLLMENT PROTOCOL: TESTING FEASIBILITY OF 3D ULTRASOUND DATA ACQUISITION AND RELIABILITY OF DATA RETRIEVAL FROM STORED 3D IMAGES

EPA Science Inventory

The purpose of this study is to examine the feasibility of collecting, transmitting,

and analyzing 3-D ultrasound data in the context of a multi-center study of pregnant

women. The study will also examine the reliability of measurements obtained from 3-D

imag...

Symptom severity scale of the DSM5 for schizophrenia, and other psychotic disorders: diagnostic validity and clinical feasibility.

PubMed

Ritsner, Michael S; Mar, Maria; Arbitman, Marina; Grinshpoon, Alexander

2013-06-30

Innovations in DSM5 include dimensional diagnosis of schizophrenia (SZ) and other psychotic (OP) disorders using the symptom severity scale (SS-DSM5). We evaluated the psychometric properties and diagnostic validity of the SS-DSM5 scale using a cross-sectional design and an unselected convenience unselected sample of 314 inpatients and outpatients with SZ/OP and mood disorders who received standard care in routine clinical practice. The SS-DSM5 scale, the Clinical Global Impression-Severity scale (CGI-S), the Positive and Negative Syndrome Scale (PANSS), and the Bech-Rafaelsen Mania Scale (BRMS) were administered. Factor structure, reliability, internal consistency, convergent and diagnostic ability of the DSM5-SS were evaluated. Factor analysis indicated two latent factors underlying the SS-DSM5 (Psychotic and Deficit sub-scales). Cronbach's alpha was >0.70. Convergent validity of the SS-DSM5 was highly significant. Patients with SZ/PO disorders were correctly diagnosed (77.9%) using the SS-DSM5 scale (72% using PANSS). The agreement of the diagnostic decisions between the SS-DSM5 and PANSS was substantial for SZ/PO disorders (Kappa=0.75). Classifying participants with SZ/PO versus mood disorders using SS-DSM5 provided a sensitivity of 95%, and specificity of 34%. Thus, this study suggests that the SS-DSM5 has acceptable psychometric properties and that its use in clinical practice and research is feasible in clinical settings. The dimensional option for the diagnosis of schizophrenia and related disorders using SS-DSM5 is discussed. Copyright © 2013 Elsevier Ireland Ltd. All rights reserved.

Development, reliability, and validity of the My Child's Play (MCP) questionnaire.

PubMed

Schneider, Eleanor; Rosenblum, Sara

2014-01-01

This article describes the development, reliability, and validity of My Child's Play (MCP), a parent questionnaire designed to evaluate the play of children ages 3-9 yr. The first phase of the study determined the questionnaire's content and face validity. Subsequently, the internal reliability consistency and construct and concurrent validity were demonstrated using 334 completed questionnaires. The MCP showed good internal consistency (α = .86). The factor analysis revealed four distinct factors with acceptable levels of internal reliability (Cronbach's αs = .63-.81) and gender- and age-related differences in play characteristics; both findings attest to the tool's construct validity. Significant correlations (r = .33, p < .0001) with the Parent as a Teacher Inventory demonstrate the MCP's concurrent validity. The MCP demonstrated acceptable reliability and validity. It appears to be a promising standardized assessment tool for use in research and practice to promote understanding of a child's play. Copyright © 2014 by the American Occupational Therapy Association, Inc.

Crowd-sourced assessment of technical skills: an opportunity for improvement in the assessment of laparoscopic surgical skills.

PubMed

Deal, Shanley B; Lendvay, Thomas S; Haque, Mohamad I; Brand, Timothy; Comstock, Bryan; Warren, Justin; Alseidi, Adnan

2016-02-01

Objective, unbiased assessment of surgical skills remains a challenge in surgical education. We sought to evaluate the feasibility and reliability of Crowd-Sourced Assessment of Technical Skills. Seven volunteer general surgery interns were given time for training and then testing, on laparoscopic peg transfer, precision cutting, and intracorporeal knot-tying. Six faculty experts (FEs) and 203 Amazon.com Mechanical Turk crowd workers (CWs) evaluated 21 deidentified video clips using the Global Objective Assessment of Laparoscopic Skills validated rating instrument. Within 19 hours and 15 minutes we received 662 eligible ratings from 203 CWs and 126 ratings from 6 FEs over 10 days. FE video ratings were of borderline internal consistency (Krippendorff's alpha = .55). FE ratings were highly correlated with CW ratings (Pearson's correlation coefficient = .78, P < .001). We propose the use of Crowd-Sourced Assessment of Technical Skills as a reliable, basic tool to standardize the evaluation of technical skills in general surgery. Copyright © 2016 Elsevier Inc. All rights reserved.

Applications of ultrasensitive magnetic measurement technologies (invited) (abstract)

NASA Astrophysics Data System (ADS)

Hirschkoff, Eugene C.

1993-05-01

The development of reliable, easy-to-use magnetic measurement systems with significantly enhanced levels of sensitivity has opened up a number of broad new areas of application for magnetic sensing. Magnetometers based on optical pumping offer sensitivities at the picotesla level, while those that utilize superconducting quantum interference devices can operate at the femtotesla level. These systems are finding applications in areas as diverse as geophysical exploration, communications, and medical diagnostics. This review briefly surveys the capabilities and application areas for a number of magnetic sensing technologies. The emphasis then focuses on the application of the most sensitive of these to the field of medical diagnostics and functional imaging. Protocols for specific applications to noninvasive presurgical planning and to the noninvasive assay of cortical dysfunction in diseases ranging from epilepsy to migraine and schizophrenia will be described in detail. Data will be presented reporting independent validation of these techniques in ten patients who subsequently underwent surgery. Routine and reliable utilization of this ultrasensitive magnetic sensing technology in the clinic is now feasible and practical.

Assuring Electronics Reliability: What Could and Should Be Done Differently

NASA Astrophysics Data System (ADS)

Suhir, E.

The following “ ten commandments” for the predicted and quantified reliability of aerospace electronic, and photonic products are addressed and discussed: 1) The best product is the best compromise between the needs for reliability, cost effectiveness and time-to-market; 2) Reliability cannot be low, need not be higher than necessary, but has to be adequate for a particular product; 3) When reliability is imperative, ability to quantify it is a must, especially if optimization is considered; 4) One cannot design a product with quantified, optimized and assured reliability by limiting the effort to the highly accelerated life testing (HALT) that does not quantify reliability; 5) Reliability is conceived at the design stage and should be taken care of, first of all, at this stage, when a “ genetically healthy” product should be created; reliability evaluations and assurances cannot be delayed until the product is fabricated and shipped to the customer, i.e., cannot be left to the prognostics-and-health-monitoring/managing (PHM) stage; it is too late at this stage to change the design or the materials for improved reliability; that is why, when reliability is imperative, users re-qualify parts to assess their lifetime and use redundancy to build a highly reliable system out of insufficiently reliable components; 6) Design, fabrication, qualification and PHM efforts should consider and be specific for particular products and their most likely actual or at least anticipated application(s); 7) Probabilistic design for reliability (PDfR) is an effective means for improving the state-of-the-art in the field: nothing is perfect, and the difference between an unreliable product and a robust one is “ merely” the probability of failure (PoF); 8) Highly cost-effective and highly focused failure oriented accelerated testing (FOAT) geared to a particular pre-determined reliability model and aimed at understanding the physics of failure- anticipated by this model is an important constituent part of the PDfR effort; 9) Predictive modeling (PM) is another important constituent of the PDfR approach; in combination with FOAT, it is a powerful means to carry out sensitivity analyses (SA), to quantify and nearly eliminate failures (“ principle of practical confidence” ); 10) Consistent, comprehensive and physically meaningful PDfR can effectively contribute to the most feasible and the most effective qualification test (QT) methodologies, practices and specifications. The general concepts addressed in the paper are illustrated by numerical examples. It is concluded that although the suggested concept is promising and fruitful, further research, refinement, and validations are needed before this concept becomes widely accepted by the engineering community and implemented into practice. It is important that this novel approach is introduced gradually, whenever feasible and appropriate, in addition to, and in some situations even instead of, the currently employed various types and modifications of the forty year old HALT.

Development and preliminary validation of an Observation List for detecting mental disorders and social Problems in the elderly in primary and home care (OLP).

PubMed

Tak, Erwin C P M; van Hespen, Ariëtte T H; Verhaak, Peter F M; Eekhof, Just; Hopman-Rock, Marijke

2016-07-01

Even though the prevalence of mental disorders and social problems is high among elderly patients, it is difficult to detect these in a primary (home) care setting. Goal was the development and preliminary validation of a short observation list to detect six problem areas: anxiety, depression, cognition, suspicion, loneliness, and somatisation. A draft list of indicators identified from a short review of the literature and the opinions of 22 experts was evaluated by general practitioners (GPs) and home care organisations for feasibility. It was then used by GPs and home care personnel to observe patients, who also completed validated tests for psychological disorders (General Health Questionnaire 12 item version (GHQ-12)), depression (Geriatric Depression Scale 15-item version (GDS-15)), anxiety and suspicion (Symptom Checklist-90 (SCL-90)), loneliness (University of California, Los Angeles (UCLA)), somatisation (Illness Attitude Scale (IAS)), and cognition (Mini-Mental State Examination (MMSE)). GPs and home care personnel observed 180 patients (mean age 78.4 years; 66% female) and evaluated the draft list during a regular visit. Cronbach's α was 0.87 for the draft list and ≥0.80 for the draft problem areas (loneliness and suspicion excepted). Principal component analysis identified six components (cognition, depression + loneliness, somatisation, anxiety + suspicion, depression (other signs), and an ambiguous component). Convergent validity was shown for the indicators list as a whole (using the GHQ-12), and the subscales of depression, anxiety, loneliness, cognition, and somatisation. Using pre-set agreed criteria, the list was reduced to 14 final indicators divided over five problem areas. The Observation List for mental disorders and social Problems (OLP) proved to be preliminarily valid, reliable, and feasible for use in primary and home care settings. Copyright © John Wliey & Sons, Ltd. Copyright © 2015 John Wiley & Sons, Ltd.

Soldier Dimensions in Combat Models

DTIC Science & Technology

1990-05-07

and performance. Questionnaires, SQTs, and ARTEPs were often used. Many scales had estimates of reliability but few had validity data. Most studies...pending its validation . Research plans were provided for applications in simulated combat and with simulation devices, for data previously gathered...regarding reliability and validity . Lack of information following an instrument indicates neither reliability nor validity information was provided by the

Development and validation of the FertiMed questionnaire assessing patients' experiences with hormonal fertility medication.

PubMed

Lankreijer, K; D'Hooghe, T; Sermeus, W; van Asseldonk, F P M; Repping, S; Dancet, E A F

2016-08-01

Can a valid and reliable questionnaire be developed to assess patients' experiences with all of the characteristics of hormonal fertility medication valued by them? The FertiMed questionnaire is a valid and reliable tool that assesses patients' experiences with all medication characteristics valued by them and that can be used for all hormonal fertility medications, irrespective of their route of administration. Hormonal fertility medications cause emotional strain and differ in their dosage regime and route of administration, although they often have comparable effectiveness. Medication experiences of former patients would be informative for medication choices. A recent literature review showed that there is no trustworthy tool to compare patients' experiences of medications with differing routes of administration, regarding all medication characteristics which patients value. The items of the new FertiMed questionnaire were generated by literature review and 23 patient interviews. In 2013, 411 IVF-patients were asked to retrospectively complete the FertiMed questionnaire to assess 1 out of the 8 different medications used for ovarian stimulation, induction of pituitary quiescence, ovulation triggering or luteal support. In total, 276 patients (on average 35 per medication) from 2 university fertility clinics (Belgium, the Netherlands) completed the FertiMed questionnaire (67% response rate). The FertiMed questionnaire questioned whether items were valued by patients and whether these items were experienced while using the assessed medication. Hence, the final outcome 'Experiences with Valued Aspects Scores' (EVAS) combined importance and experience ratings. The content and face validity, reliability, feasibility and discriminative potential of the FertiMed questionnaire were tested and changes were made accordingly. Patient interviews defined 51 items relevant to seven medication characteristics previously proved to be important to patients. Item analysis deleted 10 items. The combined results from the reliability and content validity analysis identified 10 characteristics instead of 7. The final FertiMed questionnaire was valid (Adapted Goodness of Fit Index = 0.95) and all but one characteristic ('ease of use: disturbance') could be assessed reliably (Cronbach's α > 0.60). The EVAS per characteristic differed between the medications inducing pituitary quiescence (P = 0.001). As all eight medications prescribed in the recruiting clinics were questioned, sample sizes per medication were rather small for presenting EVAS per medication and for testing the discriminative potential of the FertiMed questionnaire. The FertiMed questionnaire can be used for all hormonal fertility medications to assess in a valid and reliable way whether patients experience what they value regarding 10 medication characteristics (e.g. side effects and ease of use). Future randomized controlled trials (RCT) comparing medications could include the FertiMed questionnaire as a Patient Reported Experience Measure (PREM). Insights from these RCTs could be used to develop evidence-based decision aids aiming to facilitate shared physician-patient medication choices. Funding was received from the University of Leuven and Amsterdam University Medical Centre. E.A.F.D. holds a postdoctoral fellowship of the Research Foundation of Flanders. T.D. was appointed Vice-President and Head Global Medical Affairs Fertility at Merck (Darmstadt, Germany) on 1 October 2015. The reported project was initiated and finished before this date. The other authors had no conflicts of interest to declare. © The Author 2016. Published by Oxford University Press on behalf of the European Society of Human Reproduction and Embryology. All rights reserved. For Permissions, please email: journals.permissions@oup.com.

Methodology Series Module 9: Designing Questionnaires and Clinical Record Forms - Part II.

PubMed

Setia, Maninder Singh

2017-01-01

This article is a continuation of the previous module on designing questionnaires and clinical record form in which we have discussed some basic points about designing the questionnaire and clinical record forms. In this section, we will discuss the reliability and validity of questionnaires. The different types of validity are face validity, content validity, criterion validity, and construct validity. The different types of reliability are test-retest reliability, inter-rater reliability, and intra-rater reliability. Some of these parameters are assessed by subject area experts. However, statistical tests should be used for evaluation of other parameters. Once the questionnaire has been designed, the researcher should pilot test the questionnaire. The items in the questionnaire should be changed based on the feedback from the pilot study participants and the researcher's experience. After the basic structure of the questionnaire has been finalized, the researcher should assess the validity and reliability of the questionnaire or the scale. If an existing standard questionnaire is translated in the local language, the researcher should assess the reliability and validity of the translated questionnaire, and these values should be presented in the manuscript. The decision to use a self- or interviewer-administered, paper- or computer-based questionnaire depends on the nature of the questions, literacy levels of the target population, and resources.

Methodology Series Module 9: Designing Questionnaires and Clinical Record Forms – Part II

PubMed Central

Setia, Maninder Singh

2017-01-01

This article is a continuation of the previous module on designing questionnaires and clinical record form in which we have discussed some basic points about designing the questionnaire and clinical record forms. In this section, we will discuss the reliability and validity of questionnaires. The different types of validity are face validity, content validity, criterion validity, and construct validity. The different types of reliability are test-retest reliability, inter-rater reliability, and intra-rater reliability. Some of these parameters are assessed by subject area experts. However, statistical tests should be used for evaluation of other parameters. Once the questionnaire has been designed, the researcher should pilot test the questionnaire. The items in the questionnaire should be changed based on the feedback from the pilot study participants and the researcher's experience. After the basic structure of the questionnaire has been finalized, the researcher should assess the validity and reliability of the questionnaire or the scale. If an existing standard questionnaire is translated in the local language, the researcher should assess the reliability and validity of the translated questionnaire, and these values should be presented in the manuscript. The decision to use a self- or interviewer-administered, paper- or computer-based questionnaire depends on the nature of the questions, literacy levels of the target population, and resources. PMID:28584367

Reliability and Validity of the Chinese Version of FACIT-AI, a New Tool for Assessing Quality of Life in Patients with Malignant Ascites.

PubMed

Lou, Yanni; Lu, Linghui; Li, Yuan; Liu, Meng; Bredle, Jason M; Jia, Liqun

2015-10-01

The study objective was to determine the reliability and validity of the Chinese version of the Functional Assessment of Chronic Illness Therapy - Ascites Index (FACIT-AI). A forward-backward translation procedure was adopted to develop the Chinese version of the FACIT-AI, which was tested in 69 patients with malignant ascites. Cronbach's α, split-half reliability, and test-retest reliability were used to assess the reliability of the scale. The content validity index was used to assess the content validity, while factor analysis was used for construct validity and correlation analysis was used for criterion validity. The Cronbach's α was 0.772 for the total scale, and the split-half reliability was 0.693. The test-retest correlation was 0.972. The content validity index for the scale was 0.8-1.0. Four factors were extracted by factor analysis, and these contributed 63.51% of the total variance. Item-total correlations ranged from 0.591 to 0.897, and these were correlated with visual analog scale scores (correlation coefficient, 0.889; P<0.01). The Chinese version of the FACIT-AI has good reliability and validity and can be used as a tool to measure quality of life in Chinese patients with malignant ascites.

Methods to approximate reliabilities in single-step genomic evaluation

USDA-ARS?s Scientific Manuscript database

Reliability of predictions from single-step genomic BLUP (ssGBLUP) can be calculated by inversion, but that is not feasible for large data sets. Two methods of approximating reliability were developed based on decomposition of a function of reliability into contributions from records, pedigrees, and...

Evaluation of Measurement Tools for Tobacco Product Displays: Is there an App for that?

PubMed Central

Combs, Todd B.; Moreland-Russell, Sarah; Roche, Jason

2015-01-01

Tobacco product displays are a pervasive presence in convenience stores, supermarkets, pharmacies, and other retailers nationwide. The influence that tobacco product displays have on purchases and tobacco product initiation, particularly on young people and other vulnerable populations, is well known. An objective measurement tool that is valid, reliable, and feasible to use is needed to assess product displays in the retail setting. This study reports on the relative accuracy of various tools that measure area and/or distance in photos and thus could be applied to product displays. We compare results of repeated trials using five tools. Three tools are smartphone apps that measure objects in photos taken on the device; these are narrowed down from a list of 284 candidate apps. Another tool uses photos taken with any device and calculates relative area via a built-in function in the Microsoft Office Suite. The fifth uses photos taken with the Narrative Clip, a “life-logging” wearable camera. To evaluate validity and reliability, we assess each instrument's measurements and calculate intra-class correlation coefficients. Mean differences between observed measurements (via tape measure) and those from the five tools range from just over one square foot to just over two square feet. Most instruments produce reliable estimates though some are sensitive to the size of the display. Results of this study indicate need for future research to test innovative measurement tools. This paper also solicits further discussion on how best to transform anecdotal knowledge of product displays as targeted and disproportionate marketing tactics into a scientific evidence base for public policy change. PMID:29188220

Single-item measure for assessing quality of life in children with drug-resistant epilepsy.

PubMed

Conway, Lauryn; Widjaja, Elysa; Smith, Mary Lou

2018-03-01

The current study investigated the psychometric properties of a single-item quality of life (QOL) measure, the Global Quality of Life in Childhood Epilepsy question (G-QOLCE), in children with drug-resistant epilepsy. Data came from the Impact of Pediatric Epilepsy Surgery on Health-Related Quality of Life Study (PESQOL), a multicenter prospective cohort study (n = 118) with observations collected at baseline and at 6 months of follow-up on children aged 4-18 years. QOL was measured with the QOLCE-76 and KIDSCREEN-27. The G-QOLCE was an overall QOL question derived from the QOLCE-76. Construct validity and reliability were assessed with Spearman's correlation and intraclass correlation coefficient (ICC). Responsiveness was examined through distribution-based and anchor-based methods. The G-QOLCE showed moderate (r ≥ 0.30) to strong (r ≥ 0.50) correlations with composite scores, and most subscales of the QOLCE-76 and KIDSCREEN-27 at baseline and 6-month follow-up. The G-QOLCE had moderate test-retest reliability (ICC range: 0.49-0.72) and was able to detect clinically important change in patients' QOL (standardized response mean: 0.38; probability of change: 0.65; Guyatt's responsiveness statistics: 0.62 and 0.78). Caregiver anxiety and family functioning contributed most strongly to G-QOLCE scores over time. Results offer promising preliminary evidence regarding the validity, reliability, and responsiveness of the proposed single-item QOL measure. The G-QOLCE is a potentially useful tool that can be feasibly administered in a busy clinical setting to evaluate clinical status and impact of treatment outcomes in pediatric epilepsy.

Objective color measurements: clinimetric performance of three devices on normal skin and scar tissue.

PubMed

van der Wal, Martijn; Bloemen, Monica; Verhaegen, Pauline; Tuinebreijer, Wim; de Vet, Henrica; van Zuijlen, Paul; Middelkoop, Esther

2013-01-01

Color measurements are an essential part of scar evaluation. Thus, vascularization (erythema) and pigmentation (melanin) are common outcome parameters in scar research. The aim of this study was to investigate the clinimetric properties and clinical feasibility of the Mexameter, Colorimeter, and the DSM II ColorMeter for objective measurements on skin and scars. Fifty scars with a mean age of 6 years (2 months to 53 years) were included. Reliability was tested using the single-measure interobserver intraclass correlation coefficient. Validity was determined by measuring the Pearson correlation with the Fitzpatrick skin type classification (for skin) and the Patient and Observer Scar Assessment Scale (for scar tissue). All three instruments provided reliable readings (intraclass correlation coefficient ≥ 0.83; confidence interval: 0.71-0.90) on normal skin and scar tissue. Parameters with the highest correlations with the Fitzpatrick classification were melanin (Mexameter), 0.72; ITA (Colorimeter), -0.74; and melanin (DSM II), 0.70. On scars, the highest correlations with the Patient and Observer Scar Assessment Scale vascularization scores were the following: erythema (Mexameter), 0.59; LAB2 (Colorimeter), 0.69; and erythema (DSM II), 0.66. For hyperpigmentation, the highest correlations were melanin (Mexameter), 0.75; ITA (Colorimeter), -0.80; and melanin (DSM II), 0.83. This study shows that all three instruments can provide reliable color data on skin and scars with a single measurement. The authors also demonstrated that they can assist in objective skin type classification. For scar assessment, the most valid parameters in each instrument were identified.

Reliability and Validity of Ultrasound Cross Sectional Area Measurements for Long-Duration Spaceflight

NASA Technical Reports Server (NTRS)

Scott, Jessica M.; Martin, David S.; Cunningham, David; Matz, Timothy; Caine, Timothy; Hackney, Kyle J.; Arzeno, Natalia; Ploutz-Snyder, Lori

2010-01-01

Limb muscle atrophy and the accompanying decline in function can adversely affect the performance of astronauts during mission-related activities and upon re-ambulation in a gravitational environment. Previous characterization of space flight-induced muscle atrophy has been performed using pre and post flight magnetic resonance imaging (MRI). In addition to being costly and time consuming, MRI is an impractical methodology for assessing in-flight changes in muscle size. Given the mobility of ultrasound (US) equipment, it may be more feasible to evaluate changes in muscle size using this technique. PURPOSE: To examine the reliability and validity of using a customized template to acquire panoramic ultrasound (US) images for determining quadriceps and gastrocnemius anatomical cross sectional area (CSA). METHODS: Vastus lateralis (VL), rectus femoris (RF), medial gastrocnemius (MG), and lateral gastrocnemius (LG) CSA were assessed in 10 healthy individuals (36+/-2 yrs) using US and MRI. Panoramic US images were acquired by 2 sonographers using a customized template placed on the thigh and calf and analyzed by the same 2 sonographers (CX50 Philips). MRI images of the leg were acquired while subjects were supine in a 1.5T scanner (Signa Horizon LX, General Electric) and were analyzed by 3 trained investigators. The average of the 2 US and 3 MRI values were used for validity analysis. RESULTS: High inter-experimenter reliability was found for both the US template and MRI analysis as coefficients of variation across muscles ranged from 2.4 to 4.1% and 2.8 to 3.8%, respectively. Significant correlations were found between US and MRI CSA measures (VL, r = 0.85; RF, r = 0.60; MG, r = 0.86; LG, r = 0.73; p < 0.05). Furthermore, the standard error of measurement between US and MRI ranged from 0.91 to 2.09 sq cm with high limits of agreement analyzed by Bland-Altman plots. However, there were significant differences between absolute values of MRI and US for all muscles. CONCLUSION: The present results indicate that utilizing a customized US template provides reliable measures of leg muscle CSA, and thus could be used to characterize changes in muscle CSA both in flight and on the ground.

[Implementation of quality of care indicators for third-level public hospitals in Mexico].

PubMed

Saturno-Hernández, Pedro Jesús; Martínez-Nicolás, Ismael; Poblano-Verástegui, Ofelia; Vértiz-Ramírez, José de Jesús; Suárez-Ortiz, Erasto Cosme; Magaña-Izquierdo, Manuel; Kawa-Karasik, Simón

2017-01-01

To select, pilot test and implement a set of indicators for tertiary public hospitals. Quali-quantitative study in four stages: identification of indicators used internationally; selection and prioritization by utility, feasibility and reliability; exploration of the quality of sources of information in six hospitals; pilot feasibility and reliability, and follow-up measurement. From 143 indicators, 64 were selected and eight were prioritized. The scan revealed sources of information deficient. In the pilot, three indicators were feasible with reliability limited. Has conducted workshops to improve records and sources of information; nine hospitals reported measurements of a quarter. Eight priority indicators could not be measured immediately due to limitations in the data sources for its construction. It is necessary to improve mechanisms of registration and processing of data in this group of hospital.

Field Validity and Feasibility of Four Techniques for the Detection of Trichuris in Simians: A Model for Monitoring Drug Efficacy in Public Health?

PubMed Central

Levecke, Bruno; De Wilde, Nathalie; Vandenhoute, Els; Vercruysse, Jozef

2009-01-01

Background Soil-transmitted helminths, such as Trichuris trichiura, are of major concern in public health. Current efforts to control these helminth infections involve periodic mass treatment in endemic areas. Since these large-scale interventions are likely to intensify, monitoring the drug efficacy will become indispensible. However, studies comparing detection techniques based on sensitivity, fecal egg counts (FEC), feasibility for mass diagnosis and drug efficacy estimates are scarce. Methodology/Principal Findings In the present study, the ether-based concentration, the Parasep Solvent Free (SF), the McMaster and the FLOTAC techniques were compared based on both validity and feasibility for the detection of Trichuris eggs in 100 fecal samples of nonhuman primates. In addition, the drug efficacy estimates of quantitative techniques was examined using a statistical simulation. Trichuris eggs were found in 47% of the samples. FLOTAC was the most sensitive technique (100%), followed by the Parasep SF (83.0% [95% confidence interval (CI): 82.4–83.6%]) and the ether-based concentration technique (76.6% [95% CI: 75.8–77.3%]). McMaster was the least sensitive (61.7% [95% CI: 60.7–62.6%]) and failed to detect low FEC. The quantitative comparison revealed a positive correlation between the four techniques (Rs = 0.85–0.93; p<0.0001). However, the ether-based concentration technique and the Parasep SF detected significantly fewer eggs than both the McMaster and the FLOTAC (p<0.0083). Overall, the McMaster was the most feasible technique (3.9 min/sample for preparing, reading and cleaning of the apparatus), followed by the ether-based concentration technique (7.7 min/sample) and the FLOTAC (9.8 min/sample). Parasep SF was the least feasible (17.7 min/sample). The simulation revealed that the sensitivity is less important for monitoring drug efficacy and that both FLOTAC and McMaster were reliable estimators. Conclusions/Significance The results of this study demonstrated that McMaster is a promising technique when making use of FEC to monitor drug efficacy in Trichuris. PMID:19172171

Assessing reliability and validity measures in managed care studies.

PubMed

Montoya, Isaac D

2003-01-01

To review the reliability and validity literature and develop an understanding of these concepts as applied to managed care studies. Reliability is a test of how well an instrument measures the same input at varying times and under varying conditions. Validity is a test of how accurately an instrument measures what one believes is being measured. A review of reliability and validity instructional material was conducted. Studies of managed care practices and programs abound. However, many of these studies utilize measurement instruments that were developed for other purposes or for a population other than the one being sampled. In other cases, instruments have been developed without any testing of the instrument's performance. The lack of reliability and validity information may limit the value of these studies. This is particularly true when data are collected for one purpose and used for another. The usefulness of certain studies without reliability and validity measures is questionable, especially in cases where the literature contradicts itself

ICRS Recommendation Document

PubMed Central

Roos, Ewa M.; Engelhart, Luella; Ranstam, Jonas; Anderson, Allen F.; Irrgang, Jay J.; Marx, Robert G.; Tegner, Yelverton; Davis, Aileen M.

2011-01-01

Objective: The purpose of this article is to describe and recommend patient-reported outcome instruments for use in patients with articular cartilage lesions undergoing cartilage repair interventions. Methods: Nonsystematic literature search identifying measures addressing pain and function evaluated for validity and psychometric properties in patients with articular cartilage lesions. Results: The knee-specific instruments, titled the International Knee Documentation Committee Subjective Knee Form and the Knee injury and Osteoarthritis and Outcome Score, both fulfill the basic requirements for reliability, validity, and responsiveness in cartilage repair patients. A major difference between them is that the former results in a single score and the latter results in 5 subscores. A single score is preferred for simplicity’s sake, whereas subscores allow for evaluation of separate constructs at all levels according to the International Classification of Functioning. Conclusions: Because there is no obvious superiority of either instrument at this time, both outcome measures are recommended for use in cartilage repair. Rescaling of the Lysholm Scoring Scale has been suggested, and confirmatory longitudinal studies are needed prior to recommending this scale for use in cartilage repair. Inclusion of a generic measure is feasible in cartilage repair studies and allows analysis of health-related quality of life and health economic outcomes. The Marx or Tegner Activity Rating Scales are feasible and have been evaluated in patients with knee injuries. However, activity measures require age and sex adjustment, and data are lacking in people with cartilage repair. PMID:26069575

Development of quality indicators to evaluate antibiotic treatment of patients with community-acquired pneumonia in Indonesia.

PubMed

Farida, Helmia; Rondags, Angelique; Gasem, M Hussein; Leong, Katharina; Adityana, A; van den Broek, Peterhans J; Keuter, Monique; Natsch, Stephanie

2015-04-01

To develop an instrument for evaluating the quality of antibiotic management of patients with community-acquired pneumonia (CAP) applicable in a middle-income developing country. A previous study and Indonesian guidelines were reviewed to derive potential quality of care indicators (QIs). An expert panel performed a two-round Delphi consensus procedure on the QI's relevance to patient recovery, reduction of antimicrobial resistance and cost containment. Applicability in practice, including reliability, feasibility and opportunity for improvement, was determined in a data set of 128 patients hospitalised with CAP in Semarang, Indonesia. Fifteen QIs were selected by the consensus procedure. Five QIs did not pass feasibility criteria, because of inappropriate documentation, inefficient laboratory services or patient factors. Three QIs provided minor opportunity for improvement. Two QIs contradicted each other; one of these was considered not valid and excluded. A final set of six QIs was defined for use in the Indonesian setting. Using the Delphi method, we defined a list of QIs for assessing the quality of care, in particular antibiotic treatment, for CAP in Indonesia. For further improvement, a modified Delphi method that includes discussion, a sound medical documentation system, improvement of microbiology laboratory services, and multi-center applicability tests are needed to develop a valid and applicable QI list for the Indonesian setting. © 2014 John Wiley & Sons Ltd.

FINAL INTERIM REPORT, CANDIDATE SITES, MACHINES IN USE, DATA STORAGE AND TRANSMISSION METHODS: TESTING FEASIBILITY OF 3D ULTRASOUND DATA ACQUISITION AND RELIABILITY OF DATA RETRIEVAL FROM STORED 3D IMAGES

EPA Science Inventory

The purpose of this Work Assignment, 02-03, is to examine the feasibility of collecting transmitting, and analyzing 3-D ultrasound data in the context of a multi-center study of pregnant women. The study will also examine the reliability of measurements obtained from 3-D images< ...

Measurement of fatigue: Comparison of the reliability and validity of single-item and short measures to a comprehensive measure.

PubMed

Kim, Hee-Ju; Abraham, Ivo

2017-01-01

Evidence is needed on the clinicometric properties of single-item or short measures as alternatives to comprehensive measures. We examined whether two single-item fatigue measures (i.e., Likert scale, numeric rating scale) or a short fatigue measure were comparable to a comprehensive measure in reliability (i.e., internal consistency and test-retest reliability) and validity (i.e., convergent, concurrent, and predictive validity) in Korean young adults. For this quantitative study, we selected the Functional Assessment of Chronic Illness Therapy-Fatigue for the comprehensive measure and the Profile of Mood States-Brief, Fatigue subscale for the short measure; and constructed two single-item measures. A total of 368 students from four nursing colleges in South Korea participated. We used Cronbach's alpha and item-total correlation for internal consistency reliability and intraclass correlation coefficient for test-retest reliability. We assessed Pearson's correlation with a comprehensive measure for convergent validity, with perceived stress level and sleep quality for concurrent validity and the receiver operating characteristic curve for predictive validity. The short measure was comparable to the comprehensive measure in internal consistency reliability (Cronbach's alpha=0.81 vs. 0.88); test-retest reliability (intraclass correlation coefficient=0.66 vs. 0.61); convergent validity (r with comprehensive measure=0.79); concurrent validity (r with perceived stress=0.55, r with sleep quality=0.39) and predictive validity (area under curve=0.88). Single-item measures were not comparable to the comprehensive measure. A short fatigue measure exhibited similar levels of reliability and validity to the comprehensive measure in Korean young adults. Copyright © 2016 Elsevier Ltd. All rights reserved.

Dynamic MRI to quantify musculoskeletal motion: A systematic review of concurrent validity and reliability, and perspectives for evaluation of musculoskeletal disorders.

PubMed

Borotikar, Bhushan; Lempereur, Mathieu; Lelievre, Mathieu; Burdin, Valérie; Ben Salem, Douraied; Brochard, Sylvain

2017-01-01

To report evidence for the concurrent validity and reliability of dynamic MRI techniques to evaluate in vivo joint and muscle mechanics, and to propose recommendations for their use in the assessment of normal and impaired musculoskeletal function. The search was conducted on articles published in Web of science, PubMed, Scopus, Academic search Premier, and Cochrane Library between 1990 and August 2017. Studies that reported the concurrent validity and/or reliability of dynamic MRI techniques for in vivo evaluation of joint or muscle mechanics were included after assessment by two independent reviewers. Selected articles were assessed using an adapted quality assessment tool and a data extraction process. Results for concurrent validity and reliability were categorized as poor, moderate, or excellent. Twenty articles fulfilled the inclusion criteria with a mean quality assessment score of 66% (±10.4%). Concurrent validity and/or reliability of eight dynamic MRI techniques were reported, with the knee being the most evaluated joint (seven studies). Moderate to excellent concurrent validity and reliability were reported for seven out of eight dynamic MRI techniques. Cine phase contrast and real-time MRI appeared to be the most valid and reliable techniques to evaluate joint motion, and spin tag for muscle motion. Dynamic MRI techniques are promising for the in vivo evaluation of musculoskeletal mechanics; however results should be evaluated with caution since validity and reliability have not been determined for all joints and muscles, nor for many pathological conditions.

Validity and Reliability of the 8-Item Work Limitations Questionnaire.

PubMed

Walker, Timothy J; Tullar, Jessica M; Diamond, Pamela M; Kohl, Harold W; Amick, Benjamin C

2017-12-01

Purpose To evaluate factorial validity, scale reliability, test-retest reliability, convergent validity, and discriminant validity of the 8-item Work Limitations Questionnaire (WLQ) among employees from a public university system. Methods A secondary analysis using de-identified data from employees who completed an annual Health Assessment between the years 2009-2015 tested research aims. Confirmatory factor analysis (CFA) (n = 10,165) tested the latent structure of the 8-item WLQ. Scale reliability was determined using a CFA-based approach while test-retest reliability was determined using the intraclass correlation coefficient. Convergent/discriminant validity was tested by evaluating relations between the 8-item WLQ with health/performance variables for convergent validity (health-related work performance, number of chronic conditions, and general health) and demographic variables for discriminant validity (gender and institution type). Results A 1-factor model with three correlated residuals demonstrated excellent model fit (CFI = 0.99, TLI = 0.99, RMSEA = 0.03, and SRMR = 0.01). The scale reliability was acceptable (0.69, 95% CI 0.68-0.70) and the test-retest reliability was very good (ICC = 0.78). Low-to-moderate associations were observed between the 8-item WLQ and the health/performance variables while weak associations were observed between the demographic variables. Conclusions The 8-item WLQ demonstrated sufficient reliability and validity among employees from a public university system. Results suggest the 8-item WLQ is a usable alternative for studies when the more comprehensive 25-item WLQ is not available.

A Turkish version of myocardial infarction dimensional assessment scale (TR-MIDAS): reliability-validity assesment.

PubMed

Uysal, Hilal; Ozcan, Şeyda

2011-06-01

Many new measuring devices have been developed so that broader psychometric measurements in the coronary artery disease, disease-specific health status measurements, and identification of the broader quality of life can be performed in the recent years. The study was intended to determine whether, and to what extent, MIDAS is a valid and reliable measurement to the patients suffering from myocardial infarction for the first time in Turkey. The research was conducted with the patients hospitalized and treated with myocardial infarction in the cardiology departments of 2 hospitals in Istanbul, Turkey, between 2007 and 2008. Psychometric evaluations of TR-MIDAS were used for validity studies; language validity, content validity, construct validity were examined. For reliability studies; the tool's internal consistency reliability, Cronbach's alpha reliability coefficient, and test-retest reliability were completed. The instrument's content validity index was determined to be "0.95". Principal component analysis revealed six factors with an eigenvalue >1.5. Cronbach's alpha was found to be 0.89 for total scale which was an acceptable value. The total's test-retest reliability was 0.51 (p<0.01). Data obtained at the end of the study supports that Turkish Myocardial Infarction Dimensional Assessment Scale is a valid and reliable instrument as a disease-specific scale to assess the patients' quality of life suffering from myocardial infarction in Turkey. Copyright © 2010 European Society of Cardiology. Published by Elsevier B.V. All rights reserved.

Mid-infrared spectroscopy combined with chemometrics to detect Sclerotinia stem rot on oilseed rape (Brassica napus L.) leaves.

PubMed

Zhang, Chu; Feng, Xuping; Wang, Jian; Liu, Fei; He, Yong; Zhou, Weijun

2017-01-01

Detection of plant diseases in a fast and simple way is crucial for timely disease control. Conventionally, plant diseases are accurately identified by DNA, RNA or serology based methods which are time consuming, complex and expensive. Mid-infrared spectroscopy is a promising technique that simplifies the detection procedure for the disease. Mid-infrared spectroscopy was used to identify the spectral differences between healthy and infected oilseed rape leaves. Two different sample sets from two experiments were used to explore and validate the feasibility of using mid-infrared spectroscopy in detecting Sclerotinia stem rot (SSR) on oilseed rape leaves. The average mid-infrared spectra showed differences between healthy and infected leaves, and the differences varied among different sample sets. Optimal wavenumbers for the 2 sample sets selected by the second derivative spectra were similar, indicating the efficacy of selecting optimal wavenumbers. Chemometric methods were further used to quantitatively detect the oilseed rape leaves infected by SSR, including the partial least squares-discriminant analysis, support vector machine and extreme learning machine. The discriminant models using the full spectra and the optimal wavenumbers of the 2 sample sets were effective for classification accuracies over 80%. The discriminant results for the 2 sample sets varied due to variations in the samples. The use of two sample sets proved and validated the feasibility of using mid-infrared spectroscopy and chemometric methods for detecting SSR on oilseed rape leaves. The similarities among the selected optimal wavenumbers in different sample sets made it feasible to simplify the models and build practical models. Mid-infrared spectroscopy is a reliable and promising technique for SSR control. This study helps in developing practical application of using mid-infrared spectroscopy combined with chemometrics to detect plant disease.

Technical Analysis Feasibility Study on Smart Microgrid System in Sekolah Tinggi Teknik PLN

NASA Astrophysics Data System (ADS)

Suyanto, Heri

2018-02-01

Nowadays application of new and renewable energy as main resource of power plant has greatly increased. High penetration of renewable energy into the grid will influence the quality and reliability of the electricity system, due to the intermittent characteristic of new and renewable energy resources. Smart grid or microgrid technology has the ability to deal with this intermittent characteristic especially if these renewable energy resources integrated to grid in large scale, so it can improve the reliability and efficiency of the grid. We plan to implement smart microgrid system at Sekolah Tinggi Teknik PLN as a pilot project. Before the pilot project start, the feasibility study must be conducted. In this feasibility study, the renewable energy resources and load characteristic at the site will be measured. Then the technical aspect of this feasibility study will be analyzed. This paper explains that analysis of ths feasibility study.

Inter-observer agreement, diagnostic sensitivity and specificity of animal-based indicators of young lamb welfare.

PubMed

Phythian, C J; Toft, N; Cripps, P J; Michalopoulou, E; Winter, A C; Jones, P H; Grove-White, D; Duncan, J S

2013-07-01

A scientific literature review and consensus of expert opinion used the welfare definitions provided by the Farm Animal Welfare Council (FAWC) Five Freedoms as the framework for selecting a set of animal-based indicators that were sensitive to the current on-farm welfare issues of young lambs (aged ≤ 6 weeks). Ten animal-based indicators assessed by observation - demeanour, response to stimulation, shivering, standing ability, posture, abdominal fill, body condition, lameness, eye condition and salivation were tested as part of the objective of developing valid, reliable and feasible animal-based measures of lamb welfare The indicators were independently tested on 966 young lambs from 17 sheep flocks across Northwest England and Wales during December 2008 to April 2009 by four trained observers. Inter-observer reliability was assessed using Fleiss's kappa (κ), and the pair-wise agreement with an experienced, observer designated as the 'test standard observer' (TSO) was examined using Cohen's κ. Latent class analysis (LCA) estimated the sensitivity (Se) and specificity (Sp) of each observer without assuming a gold standard and predicted the Se and Sp of randomly selected observers who may apply the indicators in the future. Overall, good levels of inter-observer reliability, and high levels of Sp were identified for demeanour (κ = 0.54, Se ≥ 0.70, Sp ≥ 0.98), stimulation (κ = 0.57, Se = 0.30 to 0.77, Sp ≥ 0.98), shivering (κ = 0.55, Se = 0.37 to 0.85, Sp ≥ 0.99), standing ability (0.54, Se ≥ 0.80, Sp ≥ 0.99), posture (κ = 0.45, Se ≥ 0.56, Sp = 0.99), abdominal fill (κ = 0.44, Se = 0.39 to 0.98, Sp = 0.99), body condition (κ = 0.72, Se ⩾ 0.38 to 0.90, Sp = 0.99), lameness (κ = 0.68, Se > 0.73, Sp = 1.00), and eye condition (κ = 0.72, Se ≥ 0.86, Sp = 0.99). LCA predicted that randomly selected observers had Se > 0.77 (acceptable), and Sp ≥ 0.98 (high) for assessments of demeanour, lameness, abdominal fill posture, body condition and eye condition. The diagnostic performance of some indicators was influenced by the composition of the study population, and it would be useful to test the indicators on lambs with a greater level of outcomes associated with poor welfare. The findings presented in this paper could be applied in the selection of valid, reliable and feasible indicators used for the purposes of on-farm assessments of lamb welfare.

Sleep Estimates Using Microelectromechanical Systems (MEMS)

PubMed Central

te Lindert, Bart H. W.; Van Someren, Eus J. W.

2013-01-01

Study Objectives: Although currently more affordable than polysomnography, actigraphic sleep estimates have disadvantages. Brand-specific differences in data reduction impede pooling of data in large-scale cohorts and may not fully exploit movement information. Sleep estimate reliability might improve by advanced analyses of three-axial, linear accelerometry data sampled at a high rate, which is now feasible using microelectromechanical systems (MEMS). However, it might take some time before these analyses become available. To provide ongoing studies with backward compatibility while already switching from actigraphy to MEMS accelerometry, we designed and validated a method to transform accelerometry data into the traditional actigraphic movement counts, thus allowing for the use of validated algorithms to estimate sleep parameters. Design: Simultaneous actigraphy and MEMS-accelerometry recording. Setting: Home, unrestrained. Participants: Fifteen healthy adults (23-36 y, 10 males, 5 females). Interventions: None. Measurements: Actigraphic movement counts/15-sec and 50-Hz digitized MEMS-accelerometry. Analyses: Passing-Bablok regression optimized transformation of MEMS-accelerometry signals to movement counts. Kappa statistics calculated agreement between individual epochs scored as wake or sleep. Bland-Altman plots evaluated reliability of common sleep variables both between and within actigraphs and MEMS-accelerometers. Results: Agreement between epochs was almost perfect at the low, medium, and high threshold (kappa = 0.87 ± 0.05, 0.85 ± 0.06, and 0.83 ± 0.07). Sleep parameter agreement was better between two MEMS-accelerometers or a MEMS-accelerometer and an actigraph than between two actigraphs. Conclusions: The algorithm allows for continuity of outcome parameters in ongoing actigraphy studies that consider switching to MEMS-accelerometers. Its implementation makes backward compatibility feasible, while collecting raw data that, in time, could provide better sleep estimates and promote cross-study data pooling. Citation: te Lindert BHW; Van Someren EJW. Sleep estimates using microelectromechanical systems (MEMS). SLEEP 2013;36(5):781-789. PMID:23633761

Designing trials for pressure ulcer risk assessment research: methodological challenges.

PubMed

Balzer, K; Köpke, S; Lühmann, D; Haastert, B; Kottner, J; Meyer, G

2013-08-01

For decades various pressure ulcer risk assessment scales (PURAS) have been developed and implemented into nursing practice despite uncertainty whether use of these tools helps to prevent pressure ulcers. According to current methodological standards, randomised controlled trials (RCTs) are required to conclusively determine the clinical efficacy and safety of this risk assessment strategy. In these trials, PURAS-aided risk assessment has to be compared to nurses' clinical judgment alone in terms of its impact on pressure ulcer incidence and adverse outcomes. However, RCTs evaluating diagnostic procedures are prone to specific risks of bias and threats to the statistical power which may challenge their validity and feasibility. This discussion paper critically reflects on the rigour and feasibility of experimental research needed to substantiate the clinical efficacy of PURAS-aided risk assessment. Based on reflections of the methodological literature, a critical appraisal of available trials on this subject and an analysis of a protocol developed for a methodologically robust cluster-RCT, this paper arrives at the following conclusions: First, available trials do not provide reliable estimates of the impact of PURAS-aided risk assessment on pressure ulcer incidence compared to nurses' clinical judgement alone due to serious risks of bias and insufficient sample size. Second, it seems infeasible to assess this impact by means of rigorous experimental studies since sample size would become extremely high if likely threats to validity and power are properly taken into account. Third, means of evidence linkages seem to currently be the most promising approaches for evaluating the clinical efficacy and safety of PURAS-aided risk assessment. With this kind of secondary research, the downstream effect of use of PURAS on pressure ulcer incidence could be modelled by combining best available evidence for single parts of this pathway. However, to yield reliable modelling results, more robust experimental research evaluating specific parts of the pressure ulcer risk assessment-prevention pathway is needed. Copyright © 2013 Elsevier Ltd. All rights reserved.

Longitudinal Models of Reliability and Validity: A Latent Curve Approach.

ERIC Educational Resources Information Center

Tisak, John; Tisak, Marie S.

1996-01-01

Dynamic generalizations of reliability and validity that will incorporate longitudinal or developmental models, using latent curve analysis, are discussed. A latent curve model formulated to depict change is incorporated into the classical definitions of reliability and validity. The approach is illustrated with sociological and psychological…

Scoring Rubric Development: Validity and Reliability.

ERIC Educational Resources Information Center

Moskal, Barbara M.; Leydens, Jon A.

2000-01-01

Provides clear definitions of the terms "validity" and "reliability" in the context of developing scoring rubrics and illustrates these definitions through examples. Also clarifies how validity and reliability may be addressed in the development of scoring rubrics, defined as descriptive scoring schemes developed to guide the analysis of the…

An alternative to the balance error scoring system: using a low-cost balance board to improve the validity/reliability of sports-related concussion balance testing.

PubMed

Chang, Jasper O; Levy, Susan S; Seay, Seth W; Goble, Daniel J

2014-05-01

Recent guidelines advocate sports medicine professionals to use balance tests to assess sensorimotor status in the management of concussions. The present study sought to determine whether a low-cost balance board could provide a valid, reliable, and objective means of performing this balance testing. Criterion validity testing relative to a gold standard and 7 day test-retest reliability. University biomechanics laboratory. Thirty healthy young adults. Balance ability was assessed on 2 days separated by 1 week using (1) a gold standard measure (ie, scientific grade force plate), (2) a low-cost Nintendo Wii Balance Board (WBB), and (3) the Balance Error Scoring System (BESS). Validity of the WBB center of pressure path length and BESS scores were determined relative to the force plate data. Test-retest reliability was established based on intraclass correlation coefficients. Composite scores for the WBB had excellent validity (r = 0.99) and test-retest reliability (R = 0.88). Both the validity (r = 0.10-0.52) and test-retest reliability (r = 0.61-0.78) were lower for the BESS. These findings demonstrate that a low-cost balance board can provide improved balance testing accuracy/reliability compared with the BESS. This approach provides a potentially more valid/reliable, yet affordable, means of assessing sports-related concussion compared with current methods.

Patient reported outcomes in hip arthroplasty registries.

PubMed

Paulsen, Aksel

2014-05-01

PROs are used increasingly in orthopedics and in joint registries, but still many aspects of use in this area have not been examined in depth. To be able to introduce PROs in the DHR in a scientific fashion, my studies were warranted; the feasibility of four often used PROs (OHS, HOOS, EQ-5D and SF-12) was examined in a registry context. Having the PROs in the target language is an absolute necessity, so I translated, cross-culturally adapted and validated a Danish language version of an often used PRO (OHS), since this PRO had no properly developed Danish language version. To minimize data loss and to maximize the data quality I validated our data capture procedure, an up to date AFP system, by comparing scannable, paper-based PROs, with manual single-key- and double-key entered data. To help further registry-PRO studies, I calculated the number of patients needed to discriminate between subgroups of age, sex, diagnosis, and prosthesis type for each of four often used PROs (OHS, HOOS, EQ-5D and SF-12), and to simplify the clinical interpretation of PRO scores and PRO change scores in PRO studies, I estimated MCII and PASS for two often used PROs (EQ-5D and HOOS). The feasibility study included 5,747 THA patients registered in the DHR, and I found only minor differences between the disease-specific and the generic PROs regarding ceiling and floor effects as well as discarded items. The HOOS, the OHS, the SF-12, and the EQ-5D are all appropriate PROs for administration in a hip registry. I found that group sizes from 51 to 1,566 were needed for subgroup analysis, depending on descriptive factors and choice of PRO. The AFP study included 200 THA patients (398 PROs, 4,875 items and 21,887 data fields), and gave excellent results provided use of highly structured questionnaires. OMR performed equally as well as manual double-key entering, and better than single-key entering. The PRO translation and validation study included 2,278 patients (and 212 patients for the test-retest). I found that the translated PRO had good feasibility, an excellent response rate, no floor effect, but a high ceiling effect (as was expected with our postoperative patients) and few patients missed too many items to calculate a sum score. The translated PRO had high test-retest reliability and very high internal consistency, and appears to be a valid and reliable tool for outcome studies on THA patients in a hip registry setting. The MCII and PASS study included 1,335 patients, and I estimated that one year after THA, an improvement of 38-55% from mean baseline PRO score and absolute follow-up scores of 57-91% of the maximum score correspond to a minimal important improvement and acceptable symptom state, respectively.

The cross-cultural adaptation, reliability, and validity of the Copenhagen Neck Functional Disability Scale in patients with chronic neck pain: Turkish version study.

PubMed

Yapali, Gökmen; Günel, Mintaze Kerem; Karahan, Sevilay

2012-05-15

The study design was cross-cultural adaptation and investigation of reliability and validity of the Copenhagen Neck Functional Disability Scale (CNFDS). The aim of this study was to translate the CNFDS into Turkish language and assess its reliability and validity among patients with neck pain in Turkish population. The CNFDS is a reliable and valid evaluation instrument for disability, but there is no published the Turkish version of the CNFDS. One hundred one subjects who had chronic neck pain were included in this study. The CNFDS, Neck Pain and Disability Scale, and visual analogue scale were administered to all subjects. For investigating test-retest reliability, correlation between CNFDS scores, applied at 1-week interval, intraclass correlation coefficient score for test-retest reliability was 0.86 (95% confidence interval = 0.679-0.935). There was no difference between test-retest scores (P < 0.001). For investigating concurrent validity, correlation between total score of the CNFDS and the mean visual analogue scale was r = 0.73 (P < 0.001). Concurrent validity of the CNFDS was very good. For investigating construct validity, correlation between total score of the CNFDS and the Neck Pain and Disability Scale was r = 0.78 (P < 0.001). Construct validity of the CNFDS was also very good. Our results suggest that the Turkish version of the CNFDS is a reliable and valid instrument for Turkish people.

Development of a Conservative Model Validation Approach for Reliable Analysis

DTIC Science & Technology

2015-01-01

CIE 2015 August 2-5, 2015, Boston, Massachusetts, USA [DRAFT] DETC2015-46982 DEVELOPMENT OF A CONSERVATIVE MODEL VALIDATION APPROACH FOR RELIABLE...obtain a conservative simulation model for reliable design even with limited experimental data. Very little research has taken into account the...3, the proposed conservative model validation is briefly compared to the conventional model validation approach. Section 4 describes how to account

Evaluation of Validity and Reliability for Hierarchical Scales Using Latent Variable Modeling

ERIC Educational Resources Information Center

Raykov, Tenko; Marcoulides, George A.

2012-01-01

A latent variable modeling method is outlined, which accomplishes estimation of criterion validity and reliability for a multicomponent measuring instrument with hierarchical structure. The approach provides point and interval estimates for the scale criterion validity and reliability coefficients, and can also be used for testing composite or…

The Reliability and Validity of a Scale to Measure Teachers' Attitudes toward Integration in an Australian Context.

ERIC Educational Resources Information Center

Roberts, Clare; Pratt, Chris

1988-01-01

The study evaluated the psychometric properties of reliability and construct validity of the Attitude Toward Mainstreaming Scale (ATMS) in an Australian context. It was concluded that the scale is both reliable and factorially valid in an Australian context. (Author/DB)

Self-esteem among nursing assistants: reliability and validity of the Rosenberg Self-Esteem Scale.

PubMed

McMullen, Tara; Resnick, Barbara

2013-01-01

To establish the reliability and validity of the Rosenberg Self-Esteem Scale (RSES) when used with nursing assistants (NAs). Testing the RSES used baseline data from a randomized controlled trial testing the Res-Care Intervention. Female NAs were recruited from nursing homes (n = 508). Validity testing for the positive and negative subscales of the RSES was based on confirmatory factor analysis (CFA) using structural equation modeling and Rasch analysis. Estimates of reliability were based on Rasch analysis and the person separation index. Evidence supports the reliability and validity of the RSES in NAs although we recommend minor revisions to the measure for subsequent use. Establishing reliable and valid measures of self-esteem in NAs will facilitate testing of interventions to strengthen workplace self-esteem, job satisfaction, and retention.

Construct Validity and Reliability of the Questionnaire on the Quality of Physician-Patient Interaction in Adults With Hypertension.

PubMed

Hickman, Ronald L; Clochesy, John M; Hetland, Breanna; Alaamri, Marym

2017-04-01

There are limited reliable and valid measures of the patient- provider interaction among adults with hypertension. Therefore, the purpose of this report is to describe the construct validity and reliability of the Questionnaire on the Quality of Physician-Patient Interaction (QQPPI), in community-dwelling adults with hypertension. A convenience sample of 109 participants with hypertension was recruited and administered the QQPPI at baseline and 8 weeks later. The exploratory factor analysis established a 12-item, 2-factor structure for the QQPPI was valid in this sample. The modified QQPPI proved to have sufficient internal consistency and test- retest reliability. The modified QQPPI is a valid and reliable measure of the provider-patient interaction, a construct posited to impact self-management, in adults with hypertension.

So you want to conduct a randomised trial? Learnings from a 'failed' feasibility study of a Crisis Resource Management prompt during simulated paediatric resuscitation.

PubMed

Teis, Rachel; Allen, Jyai; Lee, Nigel; Kildea, Sue

2017-02-01

No study has tested a Crisis Resource Management prompt on resuscitation performance. We conducted a feasibility, unblinded, parallel-group, randomised controlled trial at one Australian paediatric hospital (June-September 2014). Eligible participants were any doctor, nurse, or nurse manager who would normally be involved in a Medical Emergency Team simulation. The unit of block randomisation was one of six scenarios (3 control:3 intervention) with or without a verbal prompt. The primary outcomes tested the feasibility and utility of the intervention and data collection tools. The secondary outcomes measured resuscitation quality and team performance. Data were analysed from six resuscitation scenarios (n=49 participants); three control groups (n=25) and three intervention groups (n=24). The ability to measure all data items on the data collection tools was hindered by problems with the recording devices both in the mannequins and the video camera. For a pilot study, greater training for the prompt role and pre-briefing participants about assessment of their cardio-pulmonary resuscitation quality should be undertaken. Data could be analysed in real time with independent video analysis to validate findings. Two cameras would strengthen reliability of the methods. Copyright © 2016 College of Emergency Nursing Australasia. Published by Elsevier Ltd. All rights reserved.

Psychometrics of the Home Safety Self-Assessment Tool (HSSAT) to prevent falls in community-dwelling older adults.

PubMed

Tomita, Machiko R; Saharan, Sumandeep; Rajendran, Sheela; Nochajski, Susan M; Schweitzer, Jo A

2014-01-01

OBJECTIVE. To identify psychometric properties of the Home Safety Self-Assessment Tool (HSSAT) to prevent falls in community-dwelling older adults. METHOD. We tested content validity, test-retest reliability, interrater reliability, construct validity, convergent and discriminant validity, and responsiveness to change. RESULTS. The content validity index was .98, the intraclass correlation coefficient for test-retest reliability was .97, and the interrater reliability was .89. The difference on identified risk factors between the use and nonuse of the HSSAT was significant (p = .005). Convergent validity with the Centers for Disease Control and Prevention Home Safety Checklist was high (r = .65), and discriminant validity with fear of falling was very low (r = .10). The responsiveness to change was moderate (standardized response mean = 0.57). CONCLUSION. The HSSAT is a reliable and valid instrument to identify fall risks in a home environment, and the HSSAT booklet is effective as educational material leading to improvement in home safety. Copyright © 2014 by the American Occupational Therapy Association, Inc.

Further Development of the Assessment of Military Multitasking Performance: Iterative Reliability Testing

PubMed Central

McCulloch, Karen L.; Radomski, Mary V.; Finkelstein, Marsha; Cecchini, Amy S.; Davidson, Leslie F.; Heaton, Kristin J.; Smith, Laurel B.; Scherer, Matthew R.

2017-01-01

The Assessment of Military Multitasking Performance (AMMP) is a battery of functional dual-tasks and multitasks based on military activities that target known sensorimotor, cognitive, and exertional vulnerabilities after concussion/mild traumatic brain injury (mTBI). The AMMP was developed to help address known limitations in post concussive return to duty assessment and decision making. Once validated, the AMMP is intended for use in combination with other metrics to inform duty-readiness decisions in Active Duty Service Members following concussion. This study used an iterative process of repeated interrater reliability testing and feasibility feedback to drive modifications to the 9 tasks of the original AMMP which resulted in a final version of 6 tasks with metrics that demonstrated clinically acceptable ICCs of > 0.92 (range of 0.92–1.0) for the 3 dual tasks and > 0.87 (range 0.87–1.0) for the metrics of the 3 multitasks. Three metrics involved in recording subject errors across 2 tasks did not achieve ICCs above 0.85 set apriori for multitasks (0.64) and above 0.90 set for dual-tasks (0.77 and 0.86) and were not used for further analysis. This iterative process involved 3 phases of testing with between 13 and 26 subjects, ages 18–42 years, tested in each phase from a combined cohort of healthy controls and Service Members with mTBI. Study findings support continued validation of this assessment tool to provide rehabilitation clinicians further return to duty assessment methods robust to ceiling effects with strong face validity to injured Warriors and their leaders. PMID:28056045

Validation of the Individualized Numeric Rating Scale (INRS): a pain assessment tool for nonverbal children with intellectual disability.

PubMed

Solodiuk, Jean C; Scott-Sutherland, Jennifer; Meyers, Margie; Myette, Beth; Shusterman, Christine; Karian, Victoria E; Harris, Sion Kim; Curley, Martha A Q

2010-08-01

Clinical observations suggest that nonverbal children with severe intellectual disability exhibit pain in a wide variety yet uniquely individual ways. Here, we investigate the feasibility and describe the initial psychometrics properties of the Individualized Numeric Rating Scale (INRS), a personalized pain assessment tool for nonverbal children with intellectual disability based on the parent's knowledge of the child. Parents of 50 nonverbal children with severe intellectual disability scheduled for surgery were able to complete the task of describing then rank ordering their child's usual and pain indicators. The parent, bedside nurse and research assistant (RA) triad then simultaneously yet independently scored the patient's post-operative pain using the INRS for a maximum of two sets of pre/post paired observations. A total of 170 triad assessments were completed before (n=85) and after (n=85) an intervention to manage the child's pain. INRS inter-rater agreement between the parents and research nurse was high (ICC 0.82-0.87) across all ratings. Parent and bedside nurse agreement (ICC 0.65-0.74) and bedside nurse and research nurse agreement (ICC 0.74-0.80) also suggest good reliability. A moderate to strong correlation (0.63-0.73) between INRS ratings and NCCPC-PV total scores provides evidence of convergent validity. These results provide preliminary data that the INRS is a valid and reliable tool for assessing pain in nonverbal children with severe intellectual disability in an acute care setting. Copyright (c) 2010 International Association for the Study of Pain. Published by Elsevier B.V. All rights reserved.

Creation and validation of a visual macroscopic hematuria scale for optimal communication and an objective hematuria index.

PubMed

Wong, Lih-Ming; Chum, Jia-Min; Maddy, Peter; Chan, Steven T F; Travis, Douglas; Lawrentschuk, Nathan

2010-07-01

Macroscopic hematuria is a common symptom and sign that is challenging to quantify and describe. The degree of hematuria communicated is variable due to health worker experience combined with lack of a reliable grading tool. We produced a reliable, standardized visual scale to describe hematuria severity. Our secondary aim was to validate a new laboratory test to quantify hemoglobin in hematuria specimens. Nurses were surveyed to ascertain current hematuria descriptions. Blood and urine were titrated at varying concentrations and digitally photographed in catheter bag tubing. Photos were processed and printed on transparency paper to create a prototype swatch or card showing light, medium, heavy and old hematuria. Using the swatch 60 samples were rated by nurses and laymen. Interobserver variability was reported using the generalized kappa coefficient of agreement. Specimens were analyzed for hemolysis by measuring optical density at oxyhemoglobin absorption peaks. Interobserver agreement between nurses and laymen was good (kappa = 0.51, p <0.001). Subgroup analysis showed substantial agreement for light hematuria (kappa = 0.71). Overall agreement improved when the moderate (kappa = 0.28) and heavy (kappa = 0.53) hematuria categories were combined (kappa = 0.70). Compared to known blood concentrations the assay of optical density at oxyhemoglobin absorption peaks showed a linear trend. A simple visual scale to grade and communicate hematuria with adequate interobserver agreement is feasible. The test for optical density at oxyhemoglobin absorption peaks is a new method, validated in our study, to quantify hemoglobin in a hematuria specimen. Copyright (c) 2010 American Urological Association Education and Research, Inc. Published by Elsevier Inc. All rights reserved.

In-Hospital Quality-of-Care Measures for Pediatric Sepsis Syndrome.

PubMed

Odetola, Folafoluwa O; Freed, Gary; Shevrin, Caroline; Madden, Brian; McCormick, Julie; Dombkowski, Kevin

2017-07-24

Sepsis syndrome, comprising sepsis, severe sepsis, and septic shock, is a leading cause of child mortality and morbidity, for which the delivery of time-sensitive care leads to improved survival. We aimed to describe the development and testing of quality measures for in-hospital care of pediatric sepsis syndrome. Seven measures of quality of care for children hospitalized with sepsis syndrome were developed by using an iterative process including literature review, development of concepts and candidate measures, and selection of measures for feasibility and importance by 2 panels of experts. The measures were tested for reliability and validity among children 0 to 18 years of age hospitalized with sepsis syndrome from January 1, 2012, to June 30, 2013. Of 27 hospitals, 59% had no protocol for the identification and treatment of pediatric sepsis syndrome. Blood culture was performed in only 70% of patients with pediatric sepsis syndrome. Antibiotics were administered within 1 hour of diagnosis in 70% of patients with pediatric severe sepsis or septic shock, and timely fluid resuscitation was performed in 50% of patients with severe sepsis or septic shock. Documentation of heart rate during fluid resuscitation of children with severe sepsis or septic shock was observed in 18% of cases. Two measures could not be rigorously tested for validity and reliability given the rarity of septic shock and were deemed infeasible. This multisite study to develop and validate measures of the quality of hospital care of children with sepsis syndrome highlights the existence of important gaps in delivery of care. Copyright © 2017 by the American Academy of Pediatrics.

Cross-Cultural Adaptation, Reliability and Validity Study of the Persian Version of the Clinical COPD Questionnaire.

PubMed

Hasanpour, Neda; Attarbashi Moghadam, Behrouz; Sami, Ramin; Tavakol, Kamran

2016-08-01

The clinical COPD questionnaire (CCQ) has been developed to measure the health status of COPD patients. The aim of this study was to translate CCQ into the Persian language and assess the validity and reliability of the translated version. We used a forward-backward procedure to translate the questionnaire. In a cross-sectional study 100 COPD patients and 50 healthy subjects over 40 years old were selected to assess the reliability and construct validity of the instrument. The face and content validity were used for the questionnaire validity. Validity was examined in a population of patients with COPD, using the Persian validated version of the St George's Respiratory Questionnaire (PSGRQ). In order to assess the questionnaire's reliability, the Intraclass correlation coefficient (ICC) and Cronbach's alpha were calculated. Test-retest reliability was tested by re-administering the Persian version of the CCQ (PCCQ) after 1 week. Test-retest carry out of data demonstrates that the PCCQ has excellent reliability (ICC for all 3 domains were higher than 0.9). Internal consistency was found by Cronbach's alpha to be 0.96, 0.94, 0.97, and 0.98 for the symptom, mental state, functional state and total scores respectively. In addition, the correlation between the components of PCCQ and PSGRQ showed satisfactory construct validity. Analyzing the data from healthy subjects and patients divulged that the PCCQ has acceptable discriminant validity. In general, the PCCQ had satisfactory reliability and validity for assessing health-related quality of life status of Iranian COPD patients.

Validity, Reliability, and the Questionable Role of Psychometrics in Plastic Surgery

PubMed Central

2014-01-01

Summary: This report examines the meaning of validity and reliability and the role of psychometrics in plastic surgery. Study titles increasingly include the word “valid” to support the authors’ claims. Studies by other investigators may be labeled “not validated.” Validity simply refers to the ability of a device to measure what it intends to measure. Validity is not an intrinsic test property. It is a relative term most credibly assigned by the independent user. Similarly, the word “reliable” is subject to interpretation. In psychometrics, its meaning is synonymous with “reproducible.” The definitions of valid and reliable are analogous to accuracy and precision. Reliability (both the reliability of the data and the consistency of measurements) is a prerequisite for validity. Outcome measures in plastic surgery are intended to be surveys, not tests. The role of psychometric modeling in plastic surgery is unclear, and this discipline introduces difficult jargon that can discourage investigators. Standard statistical tests suffice. The unambiguous term “reproducible” is preferred when discussing data consistency. Study design and methodology are essential considerations when assessing a study’s validity. PMID:25289354

The Validation of a Case-Based, Cumulative Assessment and Progressions Examination

PubMed Central

Coker, Adeola O.; Copeland, Jeffrey T.; Gottlieb, Helmut B.; Horlen, Cheryl; Smith, Helen E.; Urteaga, Elizabeth M.; Ramsinghani, Sushma; Zertuche, Alejandra; Maize, David

2016-01-01

Objective. To assess content and criterion validity, as well as reliability of an internally developed, case-based, cumulative, high-stakes third-year Annual Student Assessment and Progression Examination (P3 ASAP Exam). Methods. Content validity was assessed through the writing-reviewing process. Criterion validity was assessed by comparing student scores on the P3 ASAP Exam with the nationally validated Pharmacy Curriculum Outcomes Assessment (PCOA). Reliability was assessed with psychometric analysis comparing student performance over four years. Results. The P3 ASAP Exam showed content validity through representation of didactic courses and professional outcomes. Similar scores on the P3 ASAP Exam and PCOA with Pearson correlation coefficient established criterion validity. Consistent student performance using Kuder-Richardson coefficient (KR-20) since 2012 reflected reliability of the examination. Conclusion. Pharmacy schools can implement internally developed, high-stakes, cumulative progression examinations that are valid and reliable using a robust writing-reviewing process and psychometric analyses. PMID:26941435

Study to validate the Non-Interference Performance Assessment (NIPA) technique

NASA Technical Reports Server (NTRS)

Seeman, J. S.; Murphy, G. L.

1973-01-01

The NIPA (Non-Interference Performance Assessment) technique involves direct observation of group verbal activities by trained observers who rate the emotional content (affect) of each verbal interaction as either positive, negative, or neutral. During the test, in which four men were confined for 90 consecutive days, feasibility of the NIPA technique was demonstrated and observer reliability was verified. However, the validity of the test was not proved because an independent criterion measure of morale for the confined crew was lacking. There were indications, however, that NIPA measures were tracking changes in crew morale. At approximately the two-thirds point (Days 60 to 70), morale apparently fell dramatically for a period of about ten days, and simultaneously NIPA measure of positive verbalization decreased in number. A need was indicated for a separate study to apply the NIPA technique under experimental conditions and using a clearly defined criterion measure against which the ability of NIPA observations to truly measure morale changes could be determined.

On computing stress in polymer systems involving multi-body potentials from molecular dynamics simulation

DOE Office of Scientific and Technical Information (OSTI.GOV)

Fu, Yao, E-mail: fu5@mailbox.sc.edu, E-mail: jhsong@cec.sc.edu; Song, Jeong-Hoon, E-mail: fu5@mailbox.sc.edu, E-mail: jhsong@cec.sc.edu

2014-08-07

Hardy stress definition has been restricted to pair potentials and embedded-atom method potentials due to the basic assumptions in the derivation of a symmetric microscopic stress tensor. Force decomposition required in the Hardy stress expression becomes obscure for multi-body potentials. In this work, we demonstrate the invariance of the Hardy stress expression for a polymer system modeled with multi-body interatomic potentials including up to four atoms interaction, by applying central force decomposition of the atomic force. The balance of momentum has been demonstrated to be valid theoretically and tested under various numerical simulation conditions. The validity of momentum conservation justifiesmore » the extension of Hardy stress expression to multi-body potential systems. Computed Hardy stress has been observed to converge to the virial stress of the system with increasing spatial averaging volume. This work provides a feasible and reliable linkage between the atomistic and continuum scales for multi-body potential systems.« less

High-power ultrasonic system for the enhancement of mass transfer in supercritical CO2 extraction processes

NASA Astrophysics Data System (ADS)

Riera, Enrique; Blanco, Alfonso; García, José; Benedito, José; Mulet, Antonio; Gallego-Juárez, Juan A.; Blasco, Miguel

2010-01-01

Oil is an important component of almonds and other vegetable substrates that can show an influence on human health. In this work the development and validation of an innovative, robust, stable, reliable and efficient ultrasonic system at pilot scale to assist supercritical CO2 extraction of oils from different substrates is presented. In the extraction procedure ultrasonic energy represents an efficient way of producing deep agitation enhancing mass transfer processes because of some mechanisms (radiation pressure, streaming, agitation, high amplitude vibrations, etc.). A previous work to this research pointed out the feasibility of integrating an ultrasonic field inside a supercritical extractor without losing a significant volume fraction. This pioneer method enabled to accelerate mass transfer and then, improving supercritical extraction times. To commercially develop the new procedure fulfilling industrial requirements, a new configuration device has been designed, implemented, tested and successfully validated for supercritical fluid extraction of oil from different vegetable substrates.

Quality of care indicators for the rehabilitation of children with traumatic brain injury.

PubMed

Rivara, Frederick P; Ennis, Stephanie K; Mangione-Smith, Rita; MacKenzie, Ellen J; Jaffe, Kenneth M

2012-03-01

To develop measurement tools for assessing compliance with identifiable processes of inpatient care for children with traumatic brain injury (TBI) that are reliable, valid, and amenable to implementation. Literature review and expert panel using the RAND/UCLA Appropriateness Method and a Delphi technique. Not applicable. Children with TBI. Not applicable. Quality of care indicators. A total of 119 indicators were developed across the domains of general management; family-centered care; cognitive-communication, speech, language, and swallowing impairments; gross and fine motor skill impairments; neuropsychologic, social, and behavioral impairments; school reentry; and community integration. There was a high degree of agreement on these indicators as valid and feasible quality measures for children with TBI. These indicators are an important step toward building a better base of evidence about the effectiveness and efficiency of the components of acute inpatient rehabilitation for pediatric patients with TBI. Copyright © 2012 American Congress of Rehabilitation Medicine. Published by Elsevier Inc. All rights reserved.

Identifying compensatory movement patterns in the upper extremity using a wearable sensor system.

PubMed

Ranganathan, Rajiv; Wang, Rui; Dong, Bo; Biswas, Subir

2017-11-30

Movement impairments such as those due to stroke often result in the nervous system adopting atypical movements to compensate for movement deficits. Monitoring these compensatory patterns is critical for improving functional outcomes during rehabilitation. The purpose of this study was to test the feasibility and validity of a wearable sensor system for detecting compensatory trunk kinematics during activities of daily living. Participants with no history of neurological impairments performed reaching and manipulation tasks with their upper extremity, and their movements were recorded by a wearable sensor system and validated using a motion capture system. Compensatory movements of the trunk were induced using a brace that limited range of motion at the elbow. Our results showed that the elbow brace elicited compensatory movements of the trunk during reaching tasks but not manipulation tasks, and that a wearable sensor system with two sensors could reliably classify compensatory movements (~90% accuracy). These results show the potential of the wearable system to assess and monitor compensatory movements outside of a lab setting.

Validity and Reliability of the Clinical Competency Evaluation Instrument for Use among Physiotherapy Students: Pilot study.

PubMed

Muhamad, Zailani; Ramli, Ayiesah; Amat, Salleh

2015-05-01

The aim of this study was to determine the content validity, internal consistency, test-retest reliability and inter-rater reliability of the Clinical Competency Evaluation Instrument (CCEVI) in assessing the clinical performance of physiotherapy students. This study was carried out between June and September 2013 at University Kebangsaan Malaysia (UKM), Kuala Lumpur, Malaysia. A panel of 10 experts were identified to establish content validity by evaluating and rating each of the items used in the CCEVI with regards to their relevance in measuring students' clinical competency. A total of 50 UKM undergraduate physiotherapy students were assessed throughout their clinical placement to determine the construct validity of these items. The instrument's reliability was determined through a cross-sectional study involving a clinical performance assessment of 14 final-year undergraduate physiotherapy students. The content validity index of the entire CCEVI was 0.91, while the proportion of agreement on the content validity indices ranged from 0.83-1.00. The CCEVI construct validity was established with factor loading of ≥0.6, while internal consistency (Cronbach's alpha) overall was 0.97. Test-retest reliability of the CCEVI was confirmed with a Pearson's correlation range of 0.91-0.97 and an intraclass coefficient correlation range of 0.95-0.98. Inter-rater reliability of the CCEVI domains ranged from 0.59 to 0.97 on initial and subsequent assessments. This pilot study confirmed the content validity of the CCEVI. It showed high internal consistency, thereby providing evidence that the CCEVI has moderate to excellent inter-rater reliability. However, additional refinement in the wording of the CCEVI items, particularly in the domains of safety and documentation, is recommended to further improve the validity and reliability of the instrument.

Validity and Reliability of the Turkish Chronic Pain Acceptance Questionnaire

PubMed

Akmaz, Hazel Ekin; Uyar, Meltem; Kuzeyli Yıldırım, Yasemin; Akın Korhan, Esra

2018-05-29

Pain acceptance is the process of giving up the struggle with pain and learning to live a worthwhile life despite it. In assessing patients with chronic pain in Turkey, making a diagnosis and tracking the effectiveness of treatment is done with scales that have been translated into Turkish. However, there is as yet no valid and reliable scale in Turkish to assess the acceptance of pain. To validate a Turkish version of the Chronic Pain Acceptance Questionnaire developed by McCracken and colleagues. Methodological and cross sectional study. A simple randomized sampling method was used in selecting the study sample. The sample was composed of 201 patients, more than 10 times the number of items examined for validity and reliability in the study, which totaled 20. A patient identification form, the Chronic Pain Acceptance Questionnaire, and the Brief Pain Inventory were used to collect data. Data were collected by face-to-face interviews. In the validity testing, the content validity index was used to evaluate linguistic equivalence, content validity, construct validity, and expert views. In reliability testing of the scale, Cronbach’s α coefficient was calculated, and item analysis and split-test reliability methods were used. Principal component analysis and varimax rotation were used in factor analysis and to examine factor structure for construct concept validity. The item analysis established that the scale, all items, and item-total correlations were satisfactory. The mean total score of the scale was 21.78. The internal consistency coefficient was 0.94, and the correlation between the two halves of the scale was 0.89. The Chronic Pain Acceptance Questionnaire, which is intended to be used in Turkey upon confirmation of its validity and reliability, is an evaluation instrument with sufficient validity and reliability, and it can be reliably used to examine patients’ acceptance of chronic pain.

Validity and Reliability of the Turkish Chronic Pain Acceptance Questionnaire

PubMed Central

Akmaz, Hazel Ekin; Uyar, Meltem; Kuzeyli Yıldırım, Yasemin; Akın Korhan, Esra

2018-01-01

Background: Pain acceptance is the process of giving up the struggle with pain and learning to live a worthwhile life despite it. In assessing patients with chronic pain in Turkey, making a diagnosis and tracking the effectiveness of treatment is done with scales that have been translated into Turkish. However, there is as yet no valid and reliable scale in Turkish to assess the acceptance of pain. Aims: To validate a Turkish version of the Chronic Pain Acceptance Questionnaire developed by McCracken and colleagues. Study Design: Methodological and cross sectional study. Methods: A simple randomized sampling method was used in selecting the study sample. The sample was composed of 201 patients, more than 10 times the number of items examined for validity and reliability in the study, which totaled 20. A patient identification form, the Chronic Pain Acceptance Questionnaire, and the Brief Pain Inventory were used to collect data. Data were collected by face-to-face interviews. In the validity testing, the content validity index was used to evaluate linguistic equivalence, content validity, construct validity, and expert views. In reliability testing of the scale, Cronbach’s α coefficient was calculated, and item analysis and split-test reliability methods were used. Principal component analysis and varimax rotation were used in factor analysis and to examine factor structure for construct concept validity. Results: The item analysis established that the scale, all items, and item-total correlations were satisfactory. The mean total score of the scale was 21.78. The internal consistency coefficient was 0.94, and the correlation between the two halves of the scale was 0.89. Conclusion: The Chronic Pain Acceptance Questionnaire, which is intended to be used in Turkey upon confirmation of its validity and reliability, is an evaluation instrument with sufficient validity and reliability, and it can be reliably used to examine patients’ acceptance of chronic pain. PMID:29843496

Dynamic MRI to quantify musculoskeletal motion: A systematic review of concurrent validity and reliability, and perspectives for evaluation of musculoskeletal disorders

PubMed Central

Lempereur, Mathieu; Lelievre, Mathieu; Burdin, Valérie; Ben Salem, Douraied; Brochard, Sylvain

2017-01-01

Purpose To report evidence for the concurrent validity and reliability of dynamic MRI techniques to evaluate in vivo joint and muscle mechanics, and to propose recommendations for their use in the assessment of normal and impaired musculoskeletal function. Materials and methods The search was conducted on articles published in Web of science, PubMed, Scopus, Academic search Premier, and Cochrane Library between 1990 and August 2017. Studies that reported the concurrent validity and/or reliability of dynamic MRI techniques for in vivo evaluation of joint or muscle mechanics were included after assessment by two independent reviewers. Selected articles were assessed using an adapted quality assessment tool and a data extraction process. Results for concurrent validity and reliability were categorized as poor, moderate, or excellent. Results Twenty articles fulfilled the inclusion criteria with a mean quality assessment score of 66% (±10.4%). Concurrent validity and/or reliability of eight dynamic MRI techniques were reported, with the knee being the most evaluated joint (seven studies). Moderate to excellent concurrent validity and reliability were reported for seven out of eight dynamic MRI techniques. Cine phase contrast and real-time MRI appeared to be the most valid and reliable techniques to evaluate joint motion, and spin tag for muscle motion. Conclusion Dynamic MRI techniques are promising for the in vivo evaluation of musculoskeletal mechanics; however results should be evaluated with caution since validity and reliability have not been determined for all joints and muscles, nor for many pathological conditions. PMID:29232401

Clinical instruments: reliability and validity critical appraisal.

PubMed

Brink, Yolandi; Louw, Quinette A

2012-12-01

RATIONALE, AIM AND OBJECTIVES: There is a lack of health care practitioners using objective clinical tools with sound psychometric properties. There is also a need for researchers to improve their reporting of the validity and reliability results of these clinical tools. Therefore, to promote the use of valid and reliable tools or tests for clinical evaluation, this paper reports on the development of a critical appraisal tool to assess the psychometric properties of objective clinical tools. A five-step process was followed to develop the new critical appraisal tool: (1) preliminary conceptual decisions; (2) defining key concepts; (3) item generation; (4) assessment of face validity; and (5) formulation of the final tool. The new critical appraisal tool consists of 13 items, of which five items relate to both validity and reliability studies, four items to validity studies only and four items to reliability studies. The 13 items could be scored as 'yes', 'no' or 'not applicable'. This critical appraisal tool will aid both the health care practitioner to critically appraise the relevant literature and researchers to improve the quality of reporting of the validity and reliability of objective clinical tools. © 2011 Blackwell Publishing Ltd.

Assessment of a condition-specific quality-of-life measure for patients with developmentally absent teeth: validity and reliability testing.

PubMed

Akram, A J; Ireland, A J; Postlethwaite, K C; Sandy, J R; Jerreat, A S

2013-11-01

This article describes the process of validity and reliability testing of a condition-specific quality-of-life measure for patients with hypodontia presenting for orthodontic treatment. The development of the instrument is described in a previous article. Royal Devon and Exeter NHS Foundation Trust & Musgrove Park Hospital, Taunton. The child perception questionnaire was used as a standard against which to test criterion validity. The Bland and Altman method was used to check agreement between the two questionnaires. Construct validity was tested using principal component analysis on the four sections of the questionnaire. Test-retest reliability was tested using intraclass correlation coefficient and Bland and Altman method. Cronbach's alpha was used to test internal consistency reliability. Overall the questionnaire showed good reliability, criterion and construct validity. This together with previous evidence of good face and content validity suggests that the instrument may prove useful in clinical practice and further research. This study has demonstrated that the newly developed condition-specific quality-of-life questionnaire is both valid and reliable for use in young patients with hypodontia. © 2013 John Wiley & Sons A/S. Published by Blackwell Publishing Ltd.

Validity and reliability of a scale to measure genital body image.

PubMed

Zielinski, Ruth E; Kane-Low, Lisa; Miller, Janis M; Sampselle, Carolyn

2012-01-01

Women's body image dissatisfaction extends to body parts usually hidden from view--their genitals. Ability to measure genital body image is limited by lack of valid and reliable questionnaires. We subjected a previously developed questionnaire, the Genital Self Image Scale (GSIS) to psychometric testing using a variety of methods. Five experts determined the content validity of the scale. Then using four participant groups, factor analysis was performed to determine construct validity and to identify factors. Further construct validity was established using the contrasting groups approach. Internal consistency and test-retest reliability was determined. Twenty one of 29 items were considered content valid. Two items were added based on expert suggestions. Factor analysis was undertaken resulting in four factors, identified as Genital Confidence, Appeal, Function, and Comfort. The revised scale (GSIS-20) included 20 items explaining 59.4% of the variance. Women indicating an interest in genital cosmetic surgery exhibited significantly lower scores on the GSIS-20 than those who did not. The final 20 item scale exhibited internal reliability across all sample groups as well as test-retest reliability. The GSIS-20 provides a measure of genital body image demonstrating reliability and validity across several populations of women.

[The appraisal of reliability and validity of subjective workload assessment technique and NASA-task load index].

PubMed

Xiao, Yuan-mei; Wang, Zhi-ming; Wang, Mian-zhen; Lan, Ya-jia

2005-06-01

To test the reliability and validity of two mental workload assessment scales, i.e. subjective workload assessment technique (SWAT) and NASA task load index (NASA-TLX). One thousand two hundred and sixty-eight mental workers were sampled from various kinds of occupations, such as scientific research, education, administration and medicine, etc, with randomized cluster sampling. The re-test reliability, split-half reliability, Cronbach's alpha coefficient and correlation coefficients between item score and total score were adopted to test the reliability. The test of validity included structure validity. The re-test reliability coefficients of these two scales and their items were ranged from 0.516 to 0.753 (P < 0.01), indicating the two scales had good re-test reliability; the split-half reliability of SWAT was 0.645, and its Cronbach's alpha coefficient was more than 0.80, all the correlation coefficients between its items score and total score were more than 0.70; as for NASA-TLX, both the split-half reliability and Cronbach's alpha coefficient were more than 0.80, the correlation coefficients between its items score and total score were all more than 0.60 (P < 0.01) except the item of performance. Both scales had good inner consistency. The Pearson correlation coefficient between the two scales was 0.492 (P < 0.01), implying the results of the two scales had good consistency. Factor analysis showed that the two scales had good structure validity. Both SWAT and NASA-TLX have good reliability and validity and may be used as a valid tool to assess mental workload in China after being revised properly.

Assessing the validity and reliability of the Pool Activity Level (PAL) Checklist for use with older people with dementia.

PubMed

Wenborn, Jennifer; Challis, David; Pool, Jackie; Burgess, Jane; Elliott, Nicola; Orrell, Martin

2008-03-01

Activity is key to maintaining physical and mental health and well-being. However, as dementia affects the ability to engage in activity, care-givers can find it difficult to provide appropriate activities. The Pool Activity Level (PAL) Checklist guides the selection of appropriate, personally meaningful activities. The aim of this study was to assess the reliability and validity of the PAL Checklist when used with older people with dementia. A postal questionnaire sent to activity providers assessed content validity. Validity and reliability were measured in a sample of 60 older people with dementia. The questionnaire response rate was 83% (102/122). Most respondents felt no important items were missing. Seven of the nine activities were ranked as 'very important' or 'essential' by at least 77% of the sample, indicating very good content validity. Correlation with measures of cognition, severity of dementia and activity performance demonstrated strong concurrent validity. Inter-item correlation indicated strong construct validity. Cronbach's alpha coefficient measured internal consistency as excellent (0.95). All items achieved acceptable test-retest reliability, and the majority demonstrated acceptable inter-rater reliability. We conclude that the PAL Checklist demonstrates adequate validity and reliability when used with older people with dementia and appears a useful tool for a variety of care settings.

Validity and test-retest reliability in assessing current body size with figure drawings in Chinese adolescents.

PubMed

Lo, Wing-Sze; Ho, Sai-Yin; Wong, Bonny Yee-Man; Mak, Kwok-Kei; Lam, Tai-Hing

2011-06-01

The reliability and validity of Stunkard's Figure Rating Scale (FRS) as a measure of current body size (CBS) was established in Western adolescent girls but not in non-Western population. We examined the validity and test-retest reliability of Stunkard's FRS in assessing CBS among Chinese adolescents. Methods. In a school-based survey in Hong Kong, 5666 adolescents (boys: 45.1%; mean age 14.7 years) provided data on self-reported height and weight, CBS, perceived weight status, and health-related quality of life using the Medical Outcomes Study Short-Form version 2 (SF-12v2). Height and weight were also objectively measured. Spearman's correlation was used to assess construct validity, concurrent validity and test-retest reliability. Convergent and discriminant validity were good: CBS correlated strongly with weight and self-reported/measured BMI, but only weakly with SF-12v2. CBS correlated strongly with perceived weight status, showing concurrent validity. Spearman's correlation (r) for CBS was 0.78 for girls and 0.72 for boys indicating good test-retest reliability. Validity and reliability results did not differ significantly between senior and junior grade adolescents. Our findings support the use of Stunkard's FRS to measure body size among Chinese adolescents.

A systematic review of the reliability and validity of discrete choice experiments in valuing non-market environmental goods.

PubMed

Rakotonarivo, O Sarobidy; Schaafsma, Marije; Hockley, Neal

2016-12-01

While discrete choice experiments (DCEs) are increasingly used in the field of environmental valuation, they remain controversial because of their hypothetical nature and the contested reliability and validity of their results. We systematically reviewed evidence on the validity and reliability of environmental DCEs from the past thirteen years (Jan 2003-February 2016). 107 articles met our inclusion criteria. These studies provide limited and mixed evidence of the reliability and validity of DCE. Valuation results were susceptible to small changes in survey design in 45% of outcomes reporting reliability measures. DCE results were generally consistent with those of other stated preference techniques (convergent validity), but hypothetical bias was common. Evidence supporting theoretical validity (consistency with assumptions of rational choice theory) was limited. In content validity tests, 2-90% of respondents protested against a feature of the survey, and a considerable proportion found DCEs to be incomprehensible or inconsequential (17-40% and 10-62% respectively). DCE remains useful for non-market valuation, but its results should be used with caution. Given the sparse and inconclusive evidence base, we recommend that tests of reliability and validity are more routinely integrated into DCE studies and suggest how this might be achieved. Copyright © 2016 The Authors. Published by Elsevier Ltd.. All rights reserved.

An exploratory study into the effect of time-restricted internet access on face-validity, construct validity and reliability of postgraduate knowledge progress testing

PubMed Central

2013-01-01

Background Yearly formative knowledge testing (also known as progress testing) was shown to have a limited construct-validity and reliability in postgraduate medical education. One way to improve construct-validity and reliability is to improve the authenticity of a test. As easily accessible internet has become inseparably linked to daily clinical practice, we hypothesized that allowing internet access for a limited amount of time during the progress test would improve the perception of authenticity (face-validity) of the test, which would in turn improve the construct-validity and reliability of postgraduate progress testing. Methods Postgraduate trainees taking the yearly knowledge progress test were asked to participate in a study where they could access the internet for 30 minutes at the end of a traditional pen and paper test. Before and after the test they were asked to complete a short questionnaire regarding the face-validity of the test. Results Mean test scores increased significantly for all training years. Trainees indicated that the face-validity of the test improved with internet access and that they would like to continue to have internet access during future testing. Internet access did not improve the construct-validity or reliability of the test. Conclusion Improving the face-validity of postgraduate progress testing, by adding the possibility to search the internet for a limited amount of time, positively influences test performance and face-validity. However, it did not change the reliability or the construct-validity of the test. PMID:24195696

Intelligence Assessment Instruments in Adult Prison Populations: A Systematic Review.

PubMed

van Esch, A Y M; Denzel, A D; Scherder, E J A; Masthoff, E D M

2017-10-01

Detection of intellectual disability (ID) in the penitentiary system is important for the following reasons: (a) to provide assistance to people with ID in understanding their legal rights and court proceedings; (b) to facilitate rehabilitation programs tailored to ID patients, which improves the enhancement of their quality of life and reduces their risk of reoffending; and (c) to provide a reliable estimate of the risk of offence recidivism. It requires a short assessment instrument that provides a reliable estimation of a person's intellectual functioning at the earliest possible stage of this process. The aim of this systematic review is (a) to provide an overview of recent short assessment instruments that provide a full-scale IQ score in adult prison populations and (b) to achieve a quality measurement of the validation studies regarding these instruments to determine which tests are most feasible in this target population. The Preferred Reporting Items for Systematic reviews and Meta-Analyses Statement is used to ensure reliability. The Satz-Mögel, an item-reduction short form of the Wechsler Adult Intelligence Scale, shows the highest correlation with the golden standard and is described to be most reliable. Nevertheless, when it comes to applicability in prison populations, the shorter and less verbal Quick Test can be preferred over others. Without affecting these conclusions, major limitations emerge from the present systematic review, which give rise to several important recommendations for further research.

The Reliability and Validity of Zimbardo Time Perspective Inventory Scores in Academically Talented Adolescents

ERIC Educational Resources Information Center

Worrell, Frank C.; Mello, Zena R.

2007-01-01

In this study, the authors examined the reliability, structural validity, and concurrent validity of Zimbardo Time Perspective Inventory (ZTPI) scores in a group of 815 academically talented adolescents. Reliability estimates of the purported factors' scores were in the low to moderate range. Exploratory factor analysis supported a five-factor…

Reliability and Validity of Information about Student Achievement: Comparing Large-Scale and Classroom Testing Contexts

ERIC Educational Resources Information Center

Cizek, Gregory J.

2009-01-01

Reliability and validity are two characteristics that must be considered whenever information about student achievement is collected. However, those characteristics--and the methods for evaluating them--differ in large-scale testing and classroom testing contexts. This article presents the distinctions between reliability and validity in the two…

Investigating Postgraduate College Admission Interviews: Generalizability Theory Reliability and Incremental Predictive Validity

ERIC Educational Resources Information Center

Arce-Ferrer, Alvaro J.; Castillo, Irene Borges

2007-01-01

The use of face-to-face interviews is controversial for college admissions decisions in light of the lack of availability of validity and reliability evidence for most college admission processes. This study investigated reliability and incremental predictive validity of a face-to-face postgraduate college admission interview with a sample of…

An Integrated Approach to Establish Validity and Reliability of Reading Tests

ERIC Educational Resources Information Center

Razi, Salim

2012-01-01

This study presents the processes of developing and establishing reliability and validity of a reading test by administering an integrative approach as conventional reliability and validity measures superficially reveals the difficulty of a reading test. In this respect, analysing vocabulary frequency of the test is regarded as a more eligible way…

A Validity and Reliability Update on the Informal Reading Inventory with Suggestions for Improvement.

ERIC Educational Resources Information Center

Klesius, Janell P.; Homan, Susan P.

1985-01-01

The article reviews validity and reliability studies on the informal reading inventory, a diagnostic instrument to identify reading grade-level placement and strengths and weaknesses in work recognition and comprehension. Gives suggestions to improve the validity and reliability of existing inventories and to evaluate them in newly published…

Utility of eButton images for identifying food preparation behaviors and meal-related tasks in adolescents.

PubMed

Raber, Margaret; Patterson, Monika; Jia, Wenyan; Sun, Mingui; Baranowski, Tom

2018-02-24

Food preparation skills may encourage healthy eating. Traditional assessment of child food preparation employs self- or parent proxy-reporting methods, which are prone to error. The eButton is a wearable all-day camera that has promise as an objective, passive method for measuring child food preparation practices. This paper explores the feasibility of the eButton to reliably capture home food preparation behaviors and practices in a sample of pre- and early adolescents (ages 9 to 13). This is a secondary analysis of two eButton pilot projects evaluating the dietary intake of pre- and early adolescents in or around Houston, Texas. Food preparation behaviors were coded into seven major categories including: browsing, altering food/adding seasoning, food media, meal related tasks, prep work, cooking and observing. Inter-coder reliability was measured using Cohen's kappa and percent agreement. Analysis was completed on data for 31 participants. The most common activity was browsing in the pantry or fridge. Few participants demonstrated any food preparation work beyond unwrapping of food packages and combining two or more ingredients; actual cutting or measuring of foods were rare. Although previous research suggests children who "help" prepare meals may obtain some dietary benefit, accurate assessment tools of food preparation behavior are lacking. The eButton offers a feasible approach to food preparation behavior measurement among pre- and early adolescents. Follow up research exploring the validity of this method in a larger sample, and comparisons between cooking behavior and dietary intake are needed.

Validity and reliability of the Diagnostic Adaptive Behaviour Scale.

PubMed

Tassé, M J; Schalock, R L; Balboni, G; Spreat, S; Navas, P

2016-01-01

The Diagnostic Adaptive Behaviour Scale (DABS) is a new standardised adaptive behaviour measure that provides information for evaluating limitations in adaptive behaviour for the purpose of determining a diagnosis of intellectual disability. This article presents validity evidence and reliability data for the DABS. Validity evidence was based on comparing DABS scores with scores obtained on the Vineland Adaptive Behaviour Scale, second edition. The stability of the test scores was measured using a test and retest, and inter-rater reliability was assessed by computing the inter-respondent concordance. The DABS convergent validity coefficients ranged from 0.70 to 0.84, while the test-retest reliability coefficients ranged from 0.78 to 0.95, and the inter-rater concordance as measured by intraclass correlation coefficients ranged from 0.61 to 0.87. All obtained validity and reliability indicators were strong and comparable with the validity and reliability coefficients of the most commonly used adaptive behaviour instruments. These results and the advantages of the DABS for clinician and researcher use are discussed. © 2015 MENCAP and International Association of the Scientific Study of Intellectual and Developmental Disabilities and John Wiley & Sons Ltd.

[Reliability and validity of Driving Anger Scale in professional drivers in China].

PubMed

Li, Z; Yang, Y M; Zhang, C; Li, Y; Hu, J; Gao, L W; Zhou, Y X; Zhang, X J

2017-11-10

Objective: To assess the reliability and validity of the Chinese version of Driving Anger Scale (DAS) in professional drivers in China and provide a scientific basis for the application of the scale in drivers in China. Methods: Professional drivers, including taxi drivers, bus drivers, truck drivers and school bus drivers, were selected to complete the questionnaire. Cronbach's α and split-half reliability were calculated to evaluate the reliability of DAS, and content, contract, discriminant and convergent validity were performed to measure the validity of the scale. Results: The overall Cronbach's α of DAS was 0.934 and the split-half reliability was 0.874. The correlation coefficient of each subscale with the total scale was 0.639-0.922. The simplified version of DAS supported a presupposed six-factor structure, explaining 56.371% of the total variance revealed by exploratory factor analysis. The DAS had good convergent and discriminant validity, with the success rate of calibration experiment of 100%. Conclusion: DAS has a good reliability and validity in professional drivers in China, and the use of DAS is worth promoting in divers.

Assessing the surgical skills of trainees in the operating theatre: a prospective observational study of the methodology.

PubMed

Beard, J D; Marriott, J; Purdie, H; Crossley, J

2011-01-01

To compare user satisfaction and acceptability, reliability and validity of three different methods of assessing the surgical skills of trainees by direct observation in the operating theatre across a range of different surgical specialties and index procedures. A 2-year prospective, observational study in the operating theatres of three teaching hospitals in Sheffield. The assessment methods were procedure-based assessment (PBA), Objective Structured Assessment of Technical Skills (OSATS) and Non-technical Skills for Surgeons (NOTSS). The specialties were obstetrics and gynaecology (O&G) and upper gastrointestinal, colorectal, cardiac, vascular and orthopaedic surgery. Two to four typical index procedures were selected from each specialty. Surgical trainees were directly observed performing typical index procedures and assessed using a combination of two of the three methods (OSATS or PBA and NOTSS for O&G, PBA and NOTSS for the other specialties) by the consultant clinical supervisor for the case and the anaesthetist and/or scrub nurse, as well as one or more independent assessors from the research team. Information on user satisfaction and acceptability of each assessment method from both assessor and trainee perspectives was obtained from structured questionnaires. The reliability of each method was measured using generalisability theory. Aspects of validity included the internal structure of each tool and correlation between tools, construct validity, predictive validity, interprocedural differences, the effect of assessor designation and the effect of assessment on performance. Of the 558 patients who were consented, a total of 437 (78%) cases were included in the study: 51 consultant clinical supervisors, 56 anaesthetists, 39 nurses, 2 surgical care practitioners and 4 independent assessors provided 1635 assessments on 85 trainees undertaking the 437 cases. A total of 749 PBAs, 695 NOTSS and 191 OSATSs were performed. Non-O&G clinical supervisors and trainees provided mixed, but predominantly positive, responses about a range of applications of PBA. Most felt that PBA was important in surgical education, and would use it again in the future and did not feel that it added time to the operating list. The overall satisfaction of O&G clinical supervisors and trainees with OSATS was not as high, and a majority of those who used both preferred PBA. A majority of anaesthetists and nurses felt that NOTSS allowed them to rate interpersonal skills (communication, teamwork and leadership) more easily than cognitive skills (situation awareness and decision-making), that it had formative value and that it was a valuable adjunct to the assessment of technical skills. PBA demonstrated high reliability (G > 0.8 for only three assessor judgements on the same index procedure). OSATS had lower reliability (G > 0.8 for five assessor judgements on the same index procedure). Both were less reliable on a mix of procedures because of strong procedure-specific factors. A direct comparison of PBA between O&G and non-O&G cases showed a striking difference in reliability. Within O&G, a good level of reliability (G > 0.8) could not be obtained using a feasible number of assessments. Conversely, the reliability within non-O&G cases was exceptionally high, with only two assessor judgements being required. The reasons for this difference probably include the more summative purpose of assessment in O&G and the much higher proportion of O&G trainees in this study with training concerns (42% vs 4%). The reliability of NOTSS was lower than that for PBA. Reliability for the same procedure (G > 0.8) required six assessor judgements. However, as procedure-specific factors exerted a lesser influence on NOTSS, reliability on a mix of procedures could be achieved using only eight assessor judgements. NOTSS also demonstrated a valid internal structure. The strongest correlations between NOTSS and PBA or OSATS were in the 'decision-making' domain. PBA and NOTSS showed better construct validity than OSATS, the year of training and the number of recent index procedures performed being significant independent predictors of performance. There was little variation in scoring between different procedures or different designations of assessor. The results suggest that PBA is a reliable and acceptable method of assessing surgical skills, with good construct validity. Specialties that use OSATS may wish to consider changing the design or switching to PBA. Whatever workplace-based assessment method is used, the purpose, timing and frequency of assessment require detailed guidance. NOTSS is a promising tool for the assessment of non-technical skills, and surgical specialties may wish to consider its inclusion in their assessment framework. Further research is required into the use of health-care professionals other than consultant surgeons to assess trainees, the relationship between performance and experience, the educational impact of assessment and the additional value of video recording.