Expert agreements and disagreements on induced seismicity by Enhanced Geothermal Systems

NASA Astrophysics Data System (ADS)

Trutnevyte, E.; Azevedo, I. L.

2016-12-01

Enhanced or Engineered Geothermal Systems (EGS) are at an early stage of development and only a handful of projects exist worldwide. In face of limited empirical evidence on EGS induced seismicity, expert elicitation provides a complementary view to quantitative assessments and basic science. We present the results of an international expert elicitation exercise with 14 experts from 6 countries. The elicitation aimed at evaluating induced seismicity hazard and risk for EGS and characterizing associated uncertainty. The state-of-the-art expert elicitation method was used: it combines technical analysis with behavioral science-informed elicitation of expert judgement in order to minimize subjectivity. The experts assessed a harmonized scenario of an EGS plant, its operational characteristics, geological context, and surrounding buildings and infrastructures. The experts provided quantitative estimates of exceedance probabilities of induced M>=3 and M>=5, maximum magnitudes that could be observed, and made judgements on economic loss, injuries, and fatalities in the case of M=3 and M=5. The experts also rated the importance of factors that influence induced seismicity hazard and risk (e.g. reservoir depth, injected volumes, exposed building stock etc.) and the potential uncertainty reductions through future research. We present the findings of this elicitation and highlight the points of expert agreements and disagreements.

Group elicitations yield more consistent, yet more uncertain experts in understanding risks to ecosystem services in New Zealand bays

PubMed Central

Sinner, Jim; Ellis, Joanne; Kandlikar, Milind; Halpern, Benjamin S.; Satterfield, Terre; Chan, Kai

2017-01-01

The elicitation of expert judgment is an important tool for assessment of risks and impacts in environmental management contexts, and especially important as decision-makers face novel challenges where prior empirical research is lacking or insufficient. Evidence-driven elicitation approaches typically involve techniques to derive more accurate probability distributions under fairly specific contexts. Experts are, however, prone to overconfidence in their judgements. Group elicitations with diverse experts can reduce expert overconfidence by allowing cross-examination and reassessment of prior judgements, but groups are also prone to uncritical “groupthink” errors. When the problem context is underspecified the probability that experts commit groupthink errors may increase. This study addresses how structured workshops affect expert variability among and certainty within responses in a New Zealand case study. We find that experts’ risk estimates before and after a workshop differ, and that group elicitations provided greater consistency of estimates, yet also greater uncertainty among experts, when addressing prominent impacts to four different ecosystem services in coastal New Zealand. After group workshops, experts provided more consistent ranking of risks and more consistent best estimates of impact through increased clarity in terminology and dampening of extreme positions, yet probability distributions for impacts widened. The results from this case study suggest that group elicitations have favorable consequences for the quality and uncertainty of risk judgments within and across experts, making group elicitation techniques invaluable tools in contexts of limited data. PMID:28767694

Application and Evaluation of an Expert Judgment Elicitation Procedure for Correlations.

PubMed

Zondervan-Zwijnenburg, Mariëlle; van de Schoot-Hubeek, Wenneke; Lek, Kimberley; Hoijtink, Herbert; van de Schoot, Rens

2017-01-01

The purpose of the current study was to apply and evaluate a procedure to elicit expert judgments about correlations, and to update this information with empirical data. The result is a face-to-face group elicitation procedure with as its central element a trial roulette question that elicits experts' judgments expressed as distributions. During the elicitation procedure, a concordance probability question was used to provide feedback to the experts on their judgments. We evaluated the elicitation procedure in terms of validity and reliability by means of an application with a small sample of experts. Validity means that the elicited distributions accurately represent the experts' judgments. Reliability concerns the consistency of the elicited judgments over time. Four behavioral scientists provided their judgments with respect to the correlation between cognitive potential and academic performance for two separate populations enrolled at a specific school in the Netherlands that provides special education to youth with severe behavioral problems: youth with autism spectrum disorder (ASD), and youth with diagnoses other than ASD. Measures of face-validity, feasibility, convergent validity, coherence, and intra-rater reliability showed promising results. Furthermore, the current study illustrates the use of the elicitation procedure and elicited distributions in a social science application. The elicited distributions were used as a prior for the correlation, and updated with data for both populations collected at the school of interest. The current study shows that the newly developed elicitation procedure combining the trial roulette method with the elicitation of correlations is a promising tool, and that the results of the procedure are useful as prior information in a Bayesian analysis.

Use (and abuse) of expert elicitation in support of decision making for public policy

PubMed Central

Morgan, M. Granger

2014-01-01

The elicitation of scientific and technical judgments from experts, in the form of subjective probability distributions, can be a valuable addition to other forms of evidence in support of public policy decision making. This paper explores when it is sensible to perform such elicitation and how that can best be done. A number of key issues are discussed, including topics on which there are, and are not, experts who have knowledge that provides a basis for making informed predictive judgments; the inadequacy of only using qualitative uncertainty language; the role of cognitive heuristics and of overconfidence; the choice of experts; the development, refinement, and iterative testing of elicitation protocols that are designed to help experts to consider systematically all relevant knowledge when they make their judgments; the treatment of uncertainty about model functional form; diversity of expert opinion; and when it does or does not make sense to combine judgments from different experts. Although it may be tempting to view expert elicitation as a low-cost, low-effort alternative to conducting serious research and analysis, it is neither. Rather, expert elicitation should build on and use the best available research and analysis and be undertaken only when, given those, the state of knowledge will remain insufficient to support timely informed assessment and decision making. PMID:24821779

Assessing nutrient flows in septic tanks by eliciting expert judgement: a promising method in the context of developing countries.

PubMed

Montangero, Agnes; Belevi, Hasan

2007-03-01

Simple models based on the physical and biochemical processes occurring in septic tanks, pit and urine diversion latrines were developed to determine the nutrient flows in these systems. Nitrogen and phosphorus separation in different output materials from these on-site sanitation installations were thus determined. Moreover, nutrient separation in septic tanks was also assessed through literature values and by eliciting expert judgement. Use of formal expert elicitation technique proved to be effective, particularly in the context of developing countries where data is often scarce but expert judgement readily available. In Vietnam, only 5-14% and 11-27% of the nitrogen and phosphorus input, respectively, are removed from septic tanks with the faecal sludge. The remaining fraction leaves the tank via the liquid effluent. Unlike septic tanks, urine diversion latrines allow to immobilize most of the nutrients either in form of stored urine or dehydrated faecal matter. These latrines thus contribute to reducing the nutrient load in the environment and lowering consumption of energy and non-renewable resources for fertiliser production.

A methodology for uncertainty quantification in quantitative technology valuation based on expert elicitation

NASA Astrophysics Data System (ADS)

Akram, Muhammad Farooq Bin

The management of technology portfolios is an important element of aerospace system design. New technologies are often applied to new product designs to ensure their competitiveness at the time they are introduced to market. The future performance of yet-to- be designed components is inherently uncertain, necessitating subject matter expert knowledge, statistical methods and financial forecasting. Estimates of the appropriate parameter settings often come from disciplinary experts, who may disagree with each other because of varying experience and background. Due to inherent uncertain nature of expert elicitation in technology valuation process, appropriate uncertainty quantification and propagation is very critical. The uncertainty in defining the impact of an input on performance parameters of a system makes it difficult to use traditional probability theory. Often the available information is not enough to assign the appropriate probability distributions to uncertain inputs. Another problem faced during technology elicitation pertains to technology interactions in a portfolio. When multiple technologies are applied simultaneously on a system, often their cumulative impact is non-linear. Current methods assume that technologies are either incompatible or linearly independent. It is observed that in case of lack of knowledge about the problem, epistemic uncertainty is the most suitable representation of the process. It reduces the number of assumptions during the elicitation process, when experts are forced to assign probability distributions to their opinions without sufficient knowledge. Epistemic uncertainty can be quantified by many techniques. In present research it is proposed that interval analysis and Dempster-Shafer theory of evidence are better suited for quantification of epistemic uncertainty in technology valuation process. Proposed technique seeks to offset some of the problems faced by using deterministic or traditional probabilistic approaches for uncertainty propagation. Non-linear behavior in technology interactions is captured through expert elicitation based technology synergy matrices (TSM). Proposed TSMs increase the fidelity of current technology forecasting methods by including higher order technology interactions. A test case for quantification of epistemic uncertainty on a large scale problem of combined cycle power generation system was selected. A detailed multidisciplinary modeling and simulation environment was adopted for this problem. Results have shown that evidence theory based technique provides more insight on the uncertainties arising from incomplete information or lack of knowledge as compared to deterministic or probability theory methods. Margin analysis was also carried out for both the techniques. A detailed description of TSMs and their usage in conjunction with technology impact matrices and technology compatibility matrices is discussed. Various combination methods are also proposed for higher order interactions, which can be applied according to the expert opinion or historical data. The introduction of technology synergy matrix enabled capturing the higher order technology interactions, and improvement in predicted system performance.

Integrating clinicians, knowledge and data: expert-based cooperative analysis in healthcare decision support

PubMed Central

2010-01-01

Background Decision support in health systems is a highly difficult task, due to the inherent complexity of the process and structures involved. Method This paper introduces a new hybrid methodology Expert-based Cooperative Analysis (EbCA), which incorporates explicit prior expert knowledge in data analysis methods, and elicits implicit or tacit expert knowledge (IK) to improve decision support in healthcare systems. EbCA has been applied to two different case studies, showing its usability and versatility: 1) Bench-marking of small mental health areas based on technical efficiency estimated by EbCA-Data Envelopment Analysis (EbCA-DEA), and 2) Case-mix of schizophrenia based on functional dependency using Clustering Based on Rules (ClBR). In both cases comparisons towards classical procedures using qualitative explicit prior knowledge were made. Bayesian predictive validity measures were used for comparison with expert panels results. Overall agreement was tested by Intraclass Correlation Coefficient in case "1" and kappa in both cases. Results EbCA is a new methodology composed by 6 steps:. 1) Data collection and data preparation; 2) acquisition of "Prior Expert Knowledge" (PEK) and design of the "Prior Knowledge Base" (PKB); 3) PKB-guided analysis; 4) support-interpretation tools to evaluate results and detect inconsistencies (here Implicit Knowledg -IK- might be elicited); 5) incorporation of elicited IK in PKB and repeat till a satisfactory solution; 6) post-processing results for decision support. EbCA has been useful for incorporating PEK in two different analysis methods (DEA and Clustering), applied respectively to assess technical efficiency of small mental health areas and for case-mix of schizophrenia based on functional dependency. Differences in results obtained with classical approaches were mainly related to the IK which could be elicited by using EbCA and had major implications for the decision making in both cases. Discussion This paper presents EbCA and shows the convenience of completing classical data analysis with PEK as a mean to extract relevant knowledge in complex health domains. One of the major benefits of EbCA is iterative elicitation of IK.. Both explicit and tacit or implicit expert knowledge are critical to guide the scientific analysis of very complex decisional problems as those found in health system research. PMID:20920289

Integrating clinicians, knowledge and data: expert-based cooperative analysis in healthcare decision support.

PubMed

Gibert, Karina; García-Alonso, Carlos; Salvador-Carulla, Luis

2010-09-30

Decision support in health systems is a highly difficult task, due to the inherent complexity of the process and structures involved. This paper introduces a new hybrid methodology Expert-based Cooperative Analysis (EbCA), which incorporates explicit prior expert knowledge in data analysis methods, and elicits implicit or tacit expert knowledge (IK) to improve decision support in healthcare systems. EbCA has been applied to two different case studies, showing its usability and versatility: 1) Bench-marking of small mental health areas based on technical efficiency estimated by EbCA-Data Envelopment Analysis (EbCA-DEA), and 2) Case-mix of schizophrenia based on functional dependency using Clustering Based on Rules (ClBR). In both cases comparisons towards classical procedures using qualitative explicit prior knowledge were made. Bayesian predictive validity measures were used for comparison with expert panels results. Overall agreement was tested by Intraclass Correlation Coefficient in case "1" and kappa in both cases. EbCA is a new methodology composed by 6 steps:. 1) Data collection and data preparation; 2) acquisition of "Prior Expert Knowledge" (PEK) and design of the "Prior Knowledge Base" (PKB); 3) PKB-guided analysis; 4) support-interpretation tools to evaluate results and detect inconsistencies (here Implicit Knowledg -IK- might be elicited); 5) incorporation of elicited IK in PKB and repeat till a satisfactory solution; 6) post-processing results for decision support. EbCA has been useful for incorporating PEK in two different analysis methods (DEA and Clustering), applied respectively to assess technical efficiency of small mental health areas and for case-mix of schizophrenia based on functional dependency. Differences in results obtained with classical approaches were mainly related to the IK which could be elicited by using EbCA and had major implications for the decision making in both cases. This paper presents EbCA and shows the convenience of completing classical data analysis with PEK as a mean to extract relevant knowledge in complex health domains. One of the major benefits of EbCA is iterative elicitation of IK.. Both explicit and tacit or implicit expert knowledge are critical to guide the scientific analysis of very complex decisional problems as those found in health system research.

Quantifying catchment water balances and their uncertainties by expert elicitation

NASA Astrophysics Data System (ADS)

Sebok, Eva; Refsgaard, Jens Christian; Warmink, Jord J.; Stisen, Simon; Høgh Jensen, Karsten

2017-04-01

The increasing demand on water resources necessitates a more responsible and sustainable water management requiring a thorough understanding of hydrological processes both on small scale and on catchment scale. On catchment scale, the characterization of hydrological processes is often carried out by calculating a water balance based on the principle of mass conservation in hydrological fluxes. Assuming a perfect water balance closure and estimating one of these fluxes as a residual of the water balance is a common practice although this estimate will contain uncertainties related to uncertainties in the other components. Water balance closure on the catchment scale is also an issue in Denmark, thus, it was one of the research objectives of the HOBE hydrological observatory, that has been collecting data in the Skjern river catchment since 2008. Water balance components in the 1050 km2 Ahlergaarde catchment and the nested 120 km2 Holtum catchment, located in the glacial outwash plan of the Skjern catchment, were estimated using a multitude of methods. As the collected data enables the complex assessment of uncertainty of both the individual water balance components and catchment-scale water balances, the expert elicitation approach was chosen to integrate the results of the hydrological observatory. This approach relies on the subjective opinion of experts whose available knowledge and experience about the subject allows to integrate complex information from multiple sources. In this study 35 experts were involved in a multi-step elicitation process with the aim of (1) eliciting average annual values of water balance components for two nested catchments and quantifying the contribution of different sources of uncertainties to the total uncertainty in these average annual estimates; (2) calculating water balances for two catchments by reaching consensus among experts interacting in form of group discussions. To address the complex problem of water balance closure, the water balance was separated into five components: precipitation, evapotranspiration, surface runoff, recharge and subsurface outflow. During the study, experts first participated in individual interviews where they gave their opinion on the probability distribution of their water balance component of interest. The average annual values and uncertainty of water balance components and catchment-scale water balances were obtained at a later stage by reaching consensus during group discussions. The obtained water balance errors for the Ahlergaarde catchment and the Holtum catchment were -5 and -62 mm/yr, respectively, with an uncertainty of 66 and 86 mm/yr, respectively. As an advantage of the expert elicitation, drawing on the intuitive experience and capabilities of experts to assess complex, site-specific problems, not only the uncertainty of the water balance error was quantified, but the uncertainty of individual water balance components as well.

Expert elicitation, uncertainty, and the value of information in controlling invasive species

USGS Publications Warehouse

Johnson, Fred A.; Smith, Brian J.; Bonneau, Mathieu; Martin, Julien; Romagosa, Christina; Mazzotti, Frank J.; Waddle, J. Hardin; Reed, Robert; Eckles, Jennifer Kettevrlin; Vitt, Laurie J.

2017-01-01

We illustrate the utility of expert elicitation, explicit recognition of uncertainty, and the value of information for directing management and research efforts for invasive species, using tegu lizards (Salvator merianae) in southern Florida as a case study. We posited a post-birth pulse, matrix model in which four age classes of tegus are recognized: hatchlings, 1 year-old, 2 year-olds, and 3 + year-olds. This matrix model was parameterized using a 3-point process to elicit estimates of tegu demographic rates in southern Florida from 10 herpetology experts. We fit statistical distributions for each parameter and for each expert, then drew and pooled a large number of replicate samples from these to form a distribution for each demographic parameter. Using these distributions, as well as the observed correlations among elicited values, we generated a large sample of matrix population models to infer how the tegu population would respond to control efforts. We used the concepts of Pareto efficiency and stochastic dominance to conclude that targeting older age classes at relatively high rates appears to have the best chance of minimizing tegu abundance and control costs. We conclude that expert opinion combined with an explicit consideration of uncertainty can be valuable in conducting an initial assessment of what control strategy, effort, and monetary resources are needed to reduce and eventually eliminate the invader. Scientists, in turn, can use the value of information to focus research in a way that not only increases the efficacy of control, but minimizes costs as well.

75 FR 76467 - Draft Concept for Government-Wide “ExpertNet” Platform and Process To Elicit Expert Public...

Federal Register 2010, 2011, 2012, 2013, 2014

2010-12-08

... GENERAL SERVICES ADMINISTRATION [Docket 2010-0005; Sequence 14] Draft Concept for Government-Wide... Questions AGENCY: U.S. General Services Administration. ACTION: Notice; request for public comment. SUMMARY: With this notice, the U.S. General Services Administration (GSA) requests input, comment, and ideas...

Elicitation of neurological knowledge with argument-based machine learning.

PubMed

Groznik, Vida; Guid, Matej; Sadikov, Aleksander; Možina, Martin; Georgiev, Dejan; Kragelj, Veronika; Ribarič, Samo; Pirtošek, Zvezdan; Bratko, Ivan

2013-02-01

The paper describes the use of expert's knowledge in practice and the efficiency of a recently developed technique called argument-based machine learning (ABML) in the knowledge elicitation process. We are developing a neurological decision support system to help the neurologists differentiate between three types of tremors: Parkinsonian, essential, and mixed tremor (comorbidity). The system is intended to act as a second opinion for the neurologists, and most importantly to help them reduce the number of patients in the "gray area" that require a very costly further examination (DaTSCAN). We strive to elicit comprehensible and medically meaningful knowledge in such a way that it does not come at the cost of diagnostic accuracy. To alleviate the difficult problem of knowledge elicitation from data and domain experts, we used ABML. ABML guides the expert to explain critical special cases which cannot be handled automatically by machine learning. This very efficiently reduces the expert's workload, and combines expert's knowledge with learning data. 122 patients were enrolled into the study. The classification accuracy of the final model was 91%. Equally important, the initial and the final models were also evaluated for their comprehensibility by the neurologists. All 13 rules of the final model were deemed as appropriate to be able to support its decisions with good explanations. The paper demonstrates ABML's advantage in combining machine learning and expert knowledge. The accuracy of the system is very high with respect to the current state-of-the-art in clinical practice, and the system's knowledge base is assessed to be very consistent from a medical point of view. This opens up the possibility to use the system also as a teaching tool. Copyright © 2012 Elsevier B.V. All rights reserved.

National-level long-term eruption forecasts by expert elicitation

NASA Astrophysics Data System (ADS)

Bebbington, Mark S.; Stirling, Mark W.; Cronin, Shane; Wang, Ting; Jolly, Gill

2018-06-01

Volcanic hazard estimation is becoming increasingly quantitative, creating the potential for land-use decisions and engineering design to use volcanic information in an analogous manner to seismic codes. The initial requirement is to characterize the possible hazard sources, quantifying the likely timing, magnitude and location of the next eruption in each case. This is complicated by the extremely different driving processes at individual volcanoes, and incomplete and uneven records of past activity at various volcanoes. To address these issues, we carried out an expert elicitation approach to estimate future eruption potential for 12 volcanoes of interest in New Zealand. A total of 28 New Zealand experts provided estimates that were combined using Cooke's classical method to arrive at a hazard estimate. In 11 of the 12 cases, the elicited eruption duration increased with VEI, and was correlated with expected repose, differing little between volcanoes. Most of the andesitic volcanoes had very similar elicited distributions for the VEI of a future eruption, except that Taranaki was expected to produce a larger eruption, due to the current long repose. Elicited future vent locations for Tongariro and Okataina reflect strongly the most recent eruptions. In the poorly studied Bay of Islands volcanic field, the estimated vent location distribution was centred on the centroid of the previous vent locations, while in the Auckland field, it was focused on regions within the field without past eruptions. The elicited median dates for the next eruptions ranged from AD2022 (Whakaari/White Island) to AD4390 (Tuhua/Mayor Island).

A Delphi Investigation into Future Trends in E-Learning in Israel

ERIC Educational Resources Information Center

Aharony, Noa; Bronstein, Jenny

2014-01-01

The purpose of this study is to investigate the views and opinions of e-learning experts regarding future trends in the e-learning arena. The Delphi technique was chosen as a method of study. This technique is an efficient and effective group communication process designed to systematically elicit judgments from experts in their selected area of…

Cognitive Task Analysis for Instruction in Single-Injection Ultrasound Guided-Regional Anesthesia

ERIC Educational Resources Information Center

Gucev, Gligor V.

2012-01-01

Cognitive task analysis (CTA) is methodology for eliciting knowledge from subject matter experts. CTA has been used to capture the cognitive processes, decision-making, and judgments that underlie expert behaviors. A review of the literature revealed that CTA has not yet been used to capture the knowledge required to perform ultrasound guided…

Development of a practical approach to expert elicitation for randomised controlled trials with missing health outcomes: Application to the IMPROVE trial.

PubMed

Mason, Alexina J; Gomes, Manuel; Grieve, Richard; Ulug, Pinar; Powell, Janet T; Carpenter, James

2017-08-01

The analyses of randomised controlled trials with missing data typically assume that, after conditioning on the observed data, the probability of missing data does not depend on the patient's outcome, and so the data are 'missing at random' . This assumption is usually implausible, for example, because patients in relatively poor health may be more likely to drop out. Methodological guidelines recommend that trials require sensitivity analysis, which is best informed by elicited expert opinion, to assess whether conclusions are robust to alternative assumptions about the missing data. A major barrier to implementing these methods in practice is the lack of relevant practical tools for eliciting expert opinion. We develop a new practical tool for eliciting expert opinion and demonstrate its use for randomised controlled trials with missing data. We develop and illustrate our approach for eliciting expert opinion with the IMPROVE trial (ISRCTN 48334791), an ongoing multi-centre randomised controlled trial which compares an emergency endovascular strategy versus open repair for patients with ruptured abdominal aortic aneurysm. In the IMPROVE trial at 3 months post-randomisation, 21% of surviving patients did not complete health-related quality of life questionnaires (assessed by EQ-5D-3L). We address this problem by developing a web-based tool that provides a practical approach for eliciting expert opinion about quality of life differences between patients with missing versus complete data. We show how this expert opinion can define informative priors within a fully Bayesian framework to perform sensitivity analyses that allow the missing data to depend upon unobserved patient characteristics. A total of 26 experts, of 46 asked to participate, completed the elicitation exercise. The elicited quality of life scores were lower on average for the patients with missing versus complete data, but there was considerable uncertainty in these elicited values. The missing at random analysis found that patients randomised to the emergency endovascular strategy versus open repair had higher average (95% credible interval) quality of life scores of 0.062 (-0.005 to 0.130). Our sensitivity analysis that used the elicited expert information as pooled priors found that the gain in average quality of life for the emergency endovascular strategy versus open repair was 0.076 (-0.054 to 0.198). We provide and exemplify a practical tool for eliciting the expert opinion required by recommended approaches to the sensitivity analyses of randomised controlled trials. We show how this approach allows the trial analysis to fully recognise the uncertainty that arises from making alternative, plausible assumptions about the reasons for missing data. This tool can be widely used in the design, analysis and interpretation of future trials, and to facilitate this, materials are available for download.

Uncertainty in geological linework: communicating the expert's tacit model to the data user(s) by expert elicitation.

NASA Astrophysics Data System (ADS)

Lawley, Russell; Barron, Mark; Lee, Katy

2014-05-01

Uncertainty in geological linework: communicating the expert's tacit model to the data user(s) by expert elicitation. R. Lawley, M. Barron and K. Lee. NERC - British Geological Survey, Environmental Science Centre, Keyworth, Nottingham, UK, NG12 5GG The boundaries mapped in traditional field geological survey are subject to a wide range of inherent uncertainties. A map at a survey-scale of 1:10,000 is created by a combination of terrain interpretation, direct observations from boreholes and exposures (often sparsely distributed), and indirect interpretation of proxy variables such as soil properties, vegetation and remotely sensed images. A critical factor influencing the quality of the final map is the skill and experience of the surveyor to bring this information together in a coherent conceptual model. The users of geological data comprising or based on mapped boundaries are increasingly aware of these uncertainties, and want to know how to manage them. The growth of 3D modelling, which takes 2D surveys as a starting point, adds urgency to the need for a better understanding of survey uncertainties; particularly where 2D mapping of variable vintage has been compiled into a national coverage. Previous attempts to apply confidence on the basis of metrics such as data density, survey age or survey techniques have proved useful for isolating single, critical, factors but do not generally succeed in evaluating geological mapping 'in the round', because they cannot account for the 'conceptual' skill set of the surveyor. The British Geological Survey (BGS) is using expert elicitation methods to gain a better understanding of uncertainties within the national geological map of Great Britain. The expert elicitation approach starts with the assumption that experienced surveyors have an intuitive sense of the uncertainty of the boundaries that they map, based on a tacit model of geology and its complexity and the nature of the surveying process. The objective of elicitation is to extract this model in a useable, quantitative, form by a robust and transparent procedure. At BGS expert elicitation is being used to evaluate the uncertainty of mapped boundaries in different common mapping scenarios, with a view to building a 'collective' understanding of the challenges each scenario presents. For example, a 'sharp contact (at surface) between highly contrasting sedimentary rocks' represents one level of survey challenge that should be accurately met by all surveyors, even novices. In contrast, a 'transitional boundary defined by localised facies-variation' may require much more experience to resolve (without recourse to significantly more sampling). We will describe the initial phase of this exercise in which uncertainty models were elicited for mapped boundaries in six contrasting scenarios. Each scenario was presented to a panel of experts with varied expertise and career history. In five cases it was possible to arrive at a consensus model, in a sixth case experts with different experience took different views of the nature of the mapping problem. We will discuss our experience of the use of elicitation methodology and the implications of our results for further work at the BGS to quantify uncertainty in map products. In particular we will consider the value of elicitation as a means to capture the expertise of individuals as they retire, and as the composition of the organization's staff changes in response to the management and policy decisions.

Psychology of developing and designing expert systems

DOE Office of Scientific and Technical Information (OSTI.GOV)

Tonn, B.; MacGregor, D.

This paper discusses psychological problems relevant to developing and designing expert systems. With respect to the former, the psychological literature suggests that several cognitive biases may affect the elicitation of a valid knowledge base from the expert. The literature also suggests that common expert system inference engines may be quite inconsistent with reasoning heuristics employed by experts. With respect to expert system user interfaces, care should be taken when eliciting uncertainty estimates from users, presenting system conclusions, and ordering questions.

Risk analysis for autonomous underwater vehicle operations in extreme environments.

PubMed

Brito, Mario Paulo; Griffiths, Gwyn; Challenor, Peter

2010-12-01

Autonomous underwater vehicles (AUVs) are used increasingly to explore hazardous marine environments. Risk assessment for such complex systems is based on subjective judgment and expert knowledge as much as on hard statistics. Here, we describe the use of a risk management process tailored to AUV operations, the implementation of which requires the elicitation of expert judgment. We conducted a formal judgment elicitation process where eight world experts in AUV design and operation were asked to assign a probability of AUV loss given the emergence of each fault or incident from the vehicle's life history of 63 faults and incidents. After discussing methods of aggregation and analysis, we show how the aggregated risk estimates obtained from the expert judgments were used to create a risk model. To estimate AUV survival with mission distance, we adopted a statistical survival function based on the nonparametric Kaplan-Meier estimator. We present theoretical formulations for the estimator, its variance, and confidence limits. We also present a numerical example where the approach is applied to estimate the probability that the Autosub3 AUV would survive a set of missions under Pine Island Glacier, Antarctica in January-March 2009. © 2010 Society for Risk Analysis.

Meet EPA Scientist Jordan West, Ph.D.

EPA Pesticide Factsheets

Jordan West, Ph.D. is an aquatic ecologist at EPA. Her areas of expertise include freshwater & marine ecology, climate change impacts and adaptation, resilience and threshold theory, environmental risk assessment, expert elicitation & stakeholder processes

Development of a practical approach to expert elicitation for randomised controlled trials with missing health outcomes: Application to the IMPROVE trial

PubMed Central

Mason, Alexina J; Gomes, Manuel; Grieve, Richard; Ulug, Pinar; Powell, Janet T; Carpenter, James

2017-01-01

Background/aims: The analyses of randomised controlled trials with missing data typically assume that, after conditioning on the observed data, the probability of missing data does not depend on the patient’s outcome, and so the data are ‘missing at random’ . This assumption is usually implausible, for example, because patients in relatively poor health may be more likely to drop out. Methodological guidelines recommend that trials require sensitivity analysis, which is best informed by elicited expert opinion, to assess whether conclusions are robust to alternative assumptions about the missing data. A major barrier to implementing these methods in practice is the lack of relevant practical tools for eliciting expert opinion. We develop a new practical tool for eliciting expert opinion and demonstrate its use for randomised controlled trials with missing data. Methods: We develop and illustrate our approach for eliciting expert opinion with the IMPROVE trial (ISRCTN 48334791), an ongoing multi-centre randomised controlled trial which compares an emergency endovascular strategy versus open repair for patients with ruptured abdominal aortic aneurysm. In the IMPROVE trial at 3 months post-randomisation, 21% of surviving patients did not complete health-related quality of life questionnaires (assessed by EQ-5D-3L). We address this problem by developing a web-based tool that provides a practical approach for eliciting expert opinion about quality of life differences between patients with missing versus complete data. We show how this expert opinion can define informative priors within a fully Bayesian framework to perform sensitivity analyses that allow the missing data to depend upon unobserved patient characteristics. Results: A total of 26 experts, of 46 asked to participate, completed the elicitation exercise. The elicited quality of life scores were lower on average for the patients with missing versus complete data, but there was considerable uncertainty in these elicited values. The missing at random analysis found that patients randomised to the emergency endovascular strategy versus open repair had higher average (95% credible interval) quality of life scores of 0.062 (−0.005 to 0.130). Our sensitivity analysis that used the elicited expert information as pooled priors found that the gain in average quality of life for the emergency endovascular strategy versus open repair was 0.076 (−0.054 to 0.198). Conclusion: We provide and exemplify a practical tool for eliciting the expert opinion required by recommended approaches to the sensitivity analyses of randomised controlled trials. We show how this approach allows the trial analysis to fully recognise the uncertainty that arises from making alternative, plausible assumptions about the reasons for missing data. This tool can be widely used in the design, analysis and interpretation of future trials, and to facilitate this, materials are available for download. PMID:28675302

What defines an Expert? - Uncertainty in the interpretation of seismic data

NASA Astrophysics Data System (ADS)

Bond, C. E.

2008-12-01

Studies focusing on the elicitation of information from experts are concentrated primarily in economics and world markets, medical practice and expert witness testimonies. Expert elicitation theory has been applied in the natural sciences, most notably in the prediction of fluid flow in hydrological studies. In the geological sciences expert elicitation has been limited to theoretical analysis with studies focusing on the elicitation element, gaining expert opinion rather than necessarily understanding the basis behind the expert view. In these cases experts are defined in a traditional sense, based for example on: standing in the field, no. of years of experience, no. of peer reviewed publications, the experts position in a company hierarchy or academia. Here traditional indicators of expertise have been compared for significance on affective seismic interpretation. Polytomous regression analysis has been used to assess the relative significance of length and type of experience on the outcome of a seismic interpretation exercise. Following the initial analysis the techniques used by participants to interpret the seismic image were added as additional variables to the analysis. Specific technical skills and techniques were found to be more important for the affective geological interpretation of seismic data than the traditional indicators of expertise. The results of a seismic interpretation exercise, the techniques used to interpret the seismic and the participant's prior experience have been combined and analysed to answer the question - who is and what defines an expert?

Knowledge Elicitation: Phase 1 Final Report. Volume 1

DTIC Science & Technology

1989-06-01

34 i.e., superficial features such as type of apparatus, while experts rely on basic principles of physics (e.g., conservation of energy ) and generic...process. This last part of the model would typically consist of descriptions of the impact of the process on one or more of the objects. Figure 3-4...goals. The elicitor is probing for an underlying mental model. 9. Expert: To kill him before he can take any action that would impact on our forces. 10

Evaluation of a Performance-Based Expert Elicitation: WHO Global Attribution of Foodborne Diseases.

PubMed

Aspinall, W P; Cooke, R M; Havelaar, A H; Hoffmann, S; Hald, T

2016-01-01

For many societally important science-based decisions, data are inadequate, unreliable or non-existent, and expert advice is sought. In such cases, procedures for eliciting structured expert judgments (SEJ) are increasingly used. This raises questions regarding validity and reproducibility. This paper presents new findings from a large-scale international SEJ study intended to estimate the global burden of foodborne disease on behalf of WHO. The study involved 72 experts distributed over 134 expert panels, with panels comprising thirteen experts on average. Elicitations were conducted in five languages. Performance-based weighted solutions for target questions of interest were formed for each panel. These weights were based on individual expert's statistical accuracy and informativeness, determined using between ten and fifteen calibration variables from the experts' field with known values. Equal weights combinations were also calculated. The main conclusions on expert performance are: (1) SEJ does provide a science-based method for attribution of the global burden of foodborne diseases; (2) equal weighting of experts per panel increased statistical accuracy to acceptable levels, but at the cost of informativeness; (3) performance-based weighting increased informativeness, while retaining accuracy; (4) due to study constraints individual experts' accuracies were generally lower than in other SEJ studies, and (5) there was a negative correlation between experts' informativeness and statistical accuracy which attenuated as accuracy improved, revealing that the least accurate experts drive the negative correlation. It is shown, however, that performance-based weighting has the ability to yield statistically accurate and informative combinations of experts' judgments, thereby offsetting this contrary influence. The present findings suggest that application of SEJ on a large scale is feasible, and motivate the development of enhanced training and tools for remote elicitation of multiple, internationally-dispersed panels.

Expert Elicitation Methods in Quantifying the Consequences of Acoustic Disturbance from Offshore Renewable Energy Developments.

PubMed

Donovan, Carl; Harwood, John; King, Stephanie; Booth, Cormac; Caneco, Bruno; Walker, Cameron

2016-01-01

There are many developments for offshore renewable energy around the United Kingdom whose installation typically produces large amounts of far-reaching noise, potentially disturbing many marine mammals. The potential to affect the favorable conservation status of many species means extensive environmental impact assessment requirements for the licensing of such installation activities. Quantification of such complex risk problems is difficult and much of the key information is not readily available. Expert elicitation methods can be employed in such pressing cases. We describe the methodology used in an expert elicitation study conducted in the United Kingdom for combining expert opinions based on statistical distributions and copula-like methods.

Evaluation of a Performance-Based Expert Elicitation: WHO Global Attribution of Foodborne Diseases

PubMed Central

Aspinall, W. P.; Cooke, R. M.; Havelaar, A. H.; Hoffmann, S.; Hald, T.

2016-01-01

For many societally important science-based decisions, data are inadequate, unreliable or non-existent, and expert advice is sought. In such cases, procedures for eliciting structured expert judgments (SEJ) are increasingly used. This raises questions regarding validity and reproducibility. This paper presents new findings from a large-scale international SEJ study intended to estimate the global burden of foodborne disease on behalf of WHO. The study involved 72 experts distributed over 134 expert panels, with panels comprising thirteen experts on average. Elicitations were conducted in five languages. Performance-based weighted solutions for target questions of interest were formed for each panel. These weights were based on individual expert’s statistical accuracy and informativeness, determined using between ten and fifteen calibration variables from the experts' field with known values. Equal weights combinations were also calculated. The main conclusions on expert performance are: (1) SEJ does provide a science-based method for attribution of the global burden of foodborne diseases; (2) equal weighting of experts per panel increased statistical accuracy to acceptable levels, but at the cost of informativeness; (3) performance-based weighting increased informativeness, while retaining accuracy; (4) due to study constraints individual experts’ accuracies were generally lower than in other SEJ studies, and (5) there was a negative correlation between experts' informativeness and statistical accuracy which attenuated as accuracy improved, revealing that the least accurate experts drive the negative correlation. It is shown, however, that performance-based weighting has the ability to yield statistically accurate and informative combinations of experts' judgments, thereby offsetting this contrary influence. The present findings suggest that application of SEJ on a large scale is feasible, and motivate the development of enhanced training and tools for remote elicitation of multiple, internationally-dispersed panels. PMID:26930595

Probabilistic accident consequence uncertainty analysis: Dispersion and deposition uncertainty assessment, appendices A and B

DOE Office of Scientific and Technical Information (OSTI.GOV)

Harper, F.T.; Young, M.L.; Miller, L.A.

The development of two new probabilistic accident consequence codes, MACCS and COSYMA, completed in 1990, estimate the risks presented by nuclear installations based on postulated frequencies and magnitudes of potential accidents. In 1991, the US Nuclear Regulatory Commission (NRC) and the Commission of the European Communities (CEC) began a joint uncertainty analysis of the two codes. The objective was to develop credible and traceable uncertainty distributions for the input variables of the codes. Expert elicitation, developed independently, was identified as the best technology available for developing a library of uncertainty distributions for the selected consequence parameters. The study was formulatedmore » jointly and was limited to the current code models and to physical quantities that could be measured in experiments. To validate the distributions generated for the wet deposition input variables, samples were taken from these distributions and propagated through the wet deposition code model along with the Gaussian plume model (GPM) implemented in the MACCS and COSYMA codes. Resulting distributions closely replicated the aggregated elicited wet deposition distributions. Project teams from the NRC and CEC cooperated successfully to develop and implement a unified process for the elaboration of uncertainty distributions on consequence code input parameters. Formal expert judgment elicitation proved valuable for synthesizing the best available information. Distributions on measurable atmospheric dispersion and deposition parameters were successfully elicited from experts involved in the many phenomenological areas of consequence analysis. This volume is the second of a three-volume document describing the project and contains two appendices describing the rationales for the dispersion and deposition data along with short biographies of the 16 experts who participated in the project.« less

Expert nurses' clinical reasoning under uncertainty: representation, structure, and process.

PubMed Central

Fonteyn, M. E.; Grobe, S. J.

1992-01-01

How do expert nurses reason when planning care and making clinical decisions for a patient who is at risk, and whose outcome is uncertain? In this study, a case study involving a critically ill elderly woman whose condition deteriorated over time, was presented in segments to ten expert critical care nurses. Think aloud method was used to elicit knowledge from these experts to provide conceptual information about their knowledge and to reveal their reasoning processes and problem-solving strategies. The verbatim transcripts were then analyzed using a systematic three-step method that makes analysis easier and adds creditability to study findings by providing a means of retracing and explaining analysis results. Findings revealed information about how patient problems were represented during reasoning, the manner in which experts subjects structured their plan of care, and the reasoning processes and heuristics they used to formulate solutions for resolving the patient's problems and preventing deterioration in the patient's condition. PMID:1482907

Assessing the probability of introduction and spread of avian influenza (AI) virus in commercial Australian poultry operations using an expert opinion elicitation.

PubMed

Singh, Mini; Toribio, Jenny-Ann; Scott, Angela Bullanday; Groves, Peter; Barnes, Belinda; Glass, Kathryn; Moloney, Barbara; Black, Amanda; Hernandez-Jover, Marta

2018-01-01

The objective of this study was to elicit experts' opinions and gather estimates on the perceived probability of introduction and spread of avian influenza (AI) virus in the Australian broiler and layer industry. Using a modified Delphi method and a 4-step elicitation process, 11 experts were asked to give initial individual estimates for the various pathways and practices in the presented scenarios using a questionnaire. Following this, a workshop was conducted to present group averages of estimates and discussion was facilitated to obtain final individual estimates. For each question, estimates for all experts were combined using a discrete distribution, with weights allocated representing the level of expertise. Indirect contact with wild birds either via a contaminated water source or fomites was considered the most likely pathway of introduction of low pathogenic avian influenza (LPAI) on poultry farms. Presence of a water body near the poultry farm was considered a potential pathway for introduction only when the operation type was free range and the water body was within 500m distance from the shed. The probability that LPAI will mutate to highly pathogenic avian influenza (HPAI) was considered to be higher in layer farms. Shared personnel, equipment and aerosol dispersion were the most likely pathways of shed to shed spread of the virus. For LPAI and HPAI spread from farm to farm, shared pick-up trucks for broiler and shared egg trays and egg pallets for layer farms were considered the most likely pathways. Findings from this study provide an insight on most influential practices on the introduction and spread of AI virus among commercial poultry farms in Australia, as elicited from opinions of experts. These findings will be used to support parameterization of a modelling study assessing the risk of AI introduction and spread among commercial poultry farms in Australia.

Expert elicitation survey on future wind energy costs

DOE PAGES

Wiser, Ryan; Jenni, Karen; Seel, Joachim; ...

2016-09-12

Wind energy supply has grown rapidly over the last decade. However, the long-term contribution of wind to future energy supply, and the degree to which policy support is necessary to motivate higher levels of deployment, depends - in part - on the future costs of both onshore and offshore wind. In this paper, we summarize the results of an expert elicitation survey of 163 of the world's foremost wind experts, aimed at better understanding future costs and technology advancement possibilities. Results suggest significant opportunities for cost reductions, but also underlying uncertainties. Under the median scenario, experts anticipate 24-30% reductions bymore » 2030 and 35-41% reductions by 2050 across the three wind applications studied. Costs could be even lower: experts predict a 10% chance that reductions will be more than 40% by 2030 and more than 50% by 2050. Insights gained through expert elicitation complement other tools for evaluating cost-reduction potential, and help inform policy and planning, R & D and industry strategy.« less

Expert elicitation survey on future wind energy costs

DOE Office of Scientific and Technical Information (OSTI.GOV)

Wiser, Ryan; Jenni, Karen; Seel, Joachim

Wind energy supply has grown rapidly over the last decade. However, the long-term contribution of wind to future energy supply, and the degree to which policy support is necessary to motivate higher levels of deployment, depends -- in part -- on the future costs of both onshore and offshore wind. Here, we summarize the results of an expert elicitation survey of 163 of the world's foremost wind experts, aimed at better understanding future costs and technology advancement possibilities. Results suggest significant opportunities for cost reductions, but also underlying uncertainties. Under the median scenario, experts anticipate 24-30% reductions by 2030 andmore » 35-41% reductions by 2050 across the three wind applications studied. Costs could be even lower: experts predict a 10% chance that reductions will be more than 40% by 2030 and more than 50% by 2050. Insights gained through expert elicitation complement other tools for evaluating cost-reduction potential, and help inform policy and planning, R&D and industry strategy.« less

Expert elicitation survey on future wind energy costs

DOE Office of Scientific and Technical Information (OSTI.GOV)

Wiser, Ryan; Jenni, Karen; Seel, Joachim

Wind energy supply has grown rapidly over the last decade. However, the long-term contribution of wind to future energy supply, and the degree to which policy support is necessary to motivate higher levels of deployment, depends - in part - on the future costs of both onshore and offshore wind. In this paper, we summarize the results of an expert elicitation survey of 163 of the world's foremost wind experts, aimed at better understanding future costs and technology advancement possibilities. Results suggest significant opportunities for cost reductions, but also underlying uncertainties. Under the median scenario, experts anticipate 24-30% reductions bymore » 2030 and 35-41% reductions by 2050 across the three wind applications studied. Costs could be even lower: experts predict a 10% chance that reductions will be more than 40% by 2030 and more than 50% by 2050. Insights gained through expert elicitation complement other tools for evaluating cost-reduction potential, and help inform policy and planning, R & D and industry strategy.« less

Domain specificity versus expertise: factors influencing distinct processing of faces.

PubMed

Carmel, David; Bentin, Shlomo

2002-02-01

To explore face specificity in visual processing, we compared the role of task-associated strategies and expertise on the N170 event-related potential (ERP) component elicited by human faces with the ERPs elicited by cars, birds, items of furniture, and ape faces. In Experiment 1, participants performed a car monitoring task and an animacy decision task. In Experiment 2, participants monitored human faces while faces of apes were the distracters. Faces elicited an equally conspicuous N170, significantly larger than the ERPs elicited by non-face categories regardless of whether they were ignored or had an equal status with other categories (Experiment 1), or were the targets (in Experiment 2). In contrast, the negative component elicited by cars during the same time range was larger if they were targets than if they were not. Furthermore, unlike the posterior-temporal distribution of the N170, the negative component elicited by cars and its modulation by task were more conspicuous at occipital sites. Faces of apes elicited an N170 that was similar in amplitude to that elicited by the human face targets, albeit peaking 10 ms later. As our participants were not ape experts, this pattern indicates that the N170 is face-specific, but not specie-specific, i.e. it is elicited by particular face features regardless of expertise. Overall, these results demonstrate the domain specificity of the visual mechanism implicated in processing faces, a mechanism which is not influenced by either task or expertise. The processing of other objects is probably accomplished by a more general visual processor, which is sensitive to strategic manipulations and attention.

Expert knowledge elicitation using computer simulation: the organization of frail elderly case management as an illustration.

PubMed

Chiêm, Jean-Christophe; Van Durme, Thérèse; Vandendorpe, Florence; Schmitz, Olivier; Speybroeck, Niko; Cès, Sophie; Macq, Jean

2014-08-01

Various elderly case management projects have been implemented in Belgium. This type of long-term health care intervention involves contextual factors and human interactions. These underlying complex mechanisms can be usefully informed with field experts' knowledge, which are hard to make explicit. However, computer simulation has been suggested as one possible method of overcoming the difficulty of articulating such elicited qualitative views. A simulation model of case management was designed using an agent-based methodology, based on the initial qualitative research material. Variables and rules of interaction were formulated into a simple conceptual framework. This model has been implemented and was used as a support for a structured discussion with experts in case management. The rigorous formulation provided by the agent-based methodology clarified the descriptions of the interventions and the problems encountered regarding: the diverse network topologies of health care actors in the project; the adaptation time required by the intervention; the communication between the health care actors; the institutional context; the organization of the care; and the role of the case manager and his or hers personal ability to interpret the informal demands of the frail older person. The simulation model should be seen primarily as a tool for thinking and learning. A number of insights were gained as part of a valuable cognitive process. Computer simulation supporting field experts' elicitation can lead to better-informed decisions in the organization of complex health care interventions. © 2013 John Wiley & Sons, Ltd.

Toward the integration of expert knowledge and instrumental data to control food processes: application to Camembert-type cheese ripening.

PubMed

Sicard, M; Perrot, N; Leclercq-Perlat, M-N; Baudrit, C; Corrieu, G

2011-01-01

Modeling the cheese ripening process remains a challenge because of its complexity. We still lack the knowledge necessary to understand the interactions that take place at different levels of scale during the process. However, information may be gathered from expert knowledge. Combining this expertise with knowledge extracted from experimental databases may allow a better understanding of the entire ripening process. The aim of this study was to elicit expert knowledge and to check its validity to assess the evolution of organoleptic quality during a dynamic food process: Camembert cheese ripening. Experiments on a pilot scale were carried out at different temperatures and relative humidities to obtain contrasting ripening kinetics. During these experiments, macroscopic evolution was evaluated from an expert's point of view and instrumental measurements were carried out to simultaneously monitor microbiological, physicochemical, and biochemical kinetics. A correlation of 76% was established between the microbiological, physicochemical, and biochemical data and the sensory phases measured according to expert knowledge, highlighting the validity of the experts' measurements. In the future, it is hoped that this expert knowledge may be integrated into food process models to build better decision-aid systems that will make it possible to preserve organoleptic qualities by linking them to other phenomena at the microscopic level. Copyright © 2011 American Dairy Science Association. Published by Elsevier Inc. All rights reserved.

Analyzing Environmental Policies for Chlorinated Solvents with a Model of Markets and Regulations

DTIC Science & Technology

1991-01-01

electronics, aerospace, fabricated metal products, and dry cleaning depend heavily on chlorinated solvents in their production processes . For example...production processes . The second of the model’s components is a group of economic equations that represents all of the solvent substitutions in...Instead, the process for numerically specifying the substitution parameters involves eliciting expert judgments and then normalizing the parameters

Estimating unknown parameters in haemophilia using expert judgement elicitation.

PubMed

Fischer, K; Lewandowski, D; Janssen, M P

2013-09-01

The increasing attention to healthcare costs and treatment efficiency has led to an increasing demand for quantitative data concerning patient and treatment characteristics in haemophilia. However, most of these data are difficult to obtain. The aim of this study was to use expert judgement elicitation (EJE) to estimate currently unavailable key parameters for treatment models in severe haemophilia A. Using a formal expert elicitation procedure, 19 international experts provided information on (i) natural bleeding frequency according to age and onset of bleeding, (ii) treatment of bleeds, (iii) time needed to control bleeding after starting secondary prophylaxis, (iv) dose requirements for secondary prophylaxis according to onset of bleeding, and (v) life-expectancy. For each parameter experts provided their quantitative estimates (median, P10, P90), which were combined using a graphical method. In addition, information was obtained concerning key decision parameters of haemophilia treatment. There was most agreement between experts regarding bleeding frequencies for patients treated on demand with an average onset of joint bleeding (1.7 years): median 12 joint bleeds per year (95% confidence interval 0.9-36) for patients ≤ 18, and 11 (0.8-61) for adult patients. Less agreement was observed concerning estimated effective dose for secondary prophylaxis in adults: median 2000 IU every other day The majority (63%) of experts expected that a single minor joint bleed could cause irreversible damage, and would accept up to three minor joint bleeds or one trauma related joint bleed annually on prophylaxis. Expert judgement elicitation allowed structured capturing of quantitative expert estimates. It generated novel data to be used in computer modelling, clinical care, and trial design. © 2013 John Wiley & Sons Ltd.

Internal Medicine House Officers' Performance as Assessed by Experts and Standardized Patients.

ERIC Educational Resources Information Center

Calhoun, Judith G.; And Others

1987-01-01

Three chronically ill patients were trained to evaluate the performance of 31 second-year internal medicine house officers based upon: a checklist for the medical data elicited during the medical interview; the process of the interview; and the physical examination technique. (Author/MLW)

Quantifying Novice and Expert Differences in Visual Diagnostic Reasoning in Veterinary Pathology Using Eye-Tracking Technology.

PubMed

Warren, Amy L; Donnon, Tyrone L; Wagg, Catherine R; Priest, Heather; Fernandez, Nicole J

2018-01-18

Visual diagnostic reasoning is the cognitive process by which pathologists reach a diagnosis based on visual stimuli (cytologic, histopathologic, or gross imagery). Currently, there is little to no literature examining visual reasoning in veterinary pathology. The objective of the study was to use eye tracking to establish baseline quantitative and qualitative differences between the visual reasoning processes of novice and expert veterinary pathologists viewing cytology specimens. Novice and expert participants were each shown 10 cytology images and asked to formulate a diagnosis while wearing eye-tracking equipment (10 slides) and while concurrently verbalizing their thought processes using the think-aloud protocol (5 slides). Compared to novices, experts demonstrated significantly higher diagnostic accuracy (p<.017), shorter time to diagnosis (p<.017), and a higher percentage of time spent viewing areas of diagnostic interest (p<.017). Experts elicited more key diagnostic features in the think-aloud protocol and had more efficient patterns of eye movement. These findings suggest that experts' fast time to diagnosis, efficient eye-movement patterns, and preference for viewing areas of interest supports system 1 (pattern-recognition) reasoning and script-inductive knowledge structures with system 2 (analytic) reasoning to verify their diagnosis.

A Simplified Method of Eliciting Information from Novices.

ERIC Educational Resources Information Center

Brandt, D. Scott; Uden, Lorna

2002-01-01

Discusses the use of applied cognitive task analysis (ACTA) to interview novices and gain insight into their cognitive skills and processes. Focuses particularly on novice Internet searchers at the University of Staffordshire (United Kingdom) and reviews attempts to modify ACTA, which is intended to gather information from experts as part of…

Applying Cluster Analysis to Physics Education Research Data

ERIC Educational Resources Information Center

Springuel, R. Padraic

2010-01-01

One major thrust of Physics Education Research (PER) is the identification of student ideas about specific physics concepts, both correct ideas and those that differ from the expert consensus. Typically the research process of eliciting the spectrum of student ideas involves the administration of specially designed questions to students. One major…

Probabilistic accident consequence uncertainty analysis -- Late health effects uncertain assessment. Volume 2: Appendices

DOE Office of Scientific and Technical Information (OSTI.GOV)

Little, M.P.; Muirhead, C.R.; Goossens, L.H.J.

1997-12-01

The development of two new probabilistic accident consequence codes, MACCS and COSYMA, was completed in 1990. These codes estimate the consequence from the accidental releases of radiological material from hypothesized accidents at nuclear installations. In 1991, the US Nuclear Regulatory Commission and the Commission of the European Communities began cosponsoring a joint uncertainty analysis of the two codes. The ultimate objective of this joint effort was to systematically develop credible and traceable uncertainty distributions for the respective code input variables. A formal expert judgment elicitation and evaluation process was identified as the best technology available for developing a library ofmore » uncertainty distributions for these consequence parameters. This report focuses on the results of the study to develop distribution for variables related to the MACCS and COSYMA late health effects models. This volume contains appendices that include (1) a summary of the MACCS and COSYMA consequence codes, (2) the elicitation questionnaires and case structures, (3) the rationales and results for the expert panel on late health effects, (4) short biographies of the experts, and (5) the aggregated results of their responses.« less

Assessing the probability of introduction and spread of avian influenza (AI) virus in commercial Australian poultry operations using an expert opinion elicitation

PubMed Central

Toribio, Jenny-Ann; Scott, Angela Bullanday; Groves, Peter; Barnes, Belinda; Glass, Kathryn; Moloney, Barbara; Black, Amanda; Hernandez-Jover, Marta

2018-01-01

The objective of this study was to elicit experts’ opinions and gather estimates on the perceived probability of introduction and spread of avian influenza (AI) virus in the Australian broiler and layer industry. Using a modified Delphi method and a 4-step elicitation process, 11 experts were asked to give initial individual estimates for the various pathways and practices in the presented scenarios using a questionnaire. Following this, a workshop was conducted to present group averages of estimates and discussion was facilitated to obtain final individual estimates. For each question, estimates for all experts were combined using a discrete distribution, with weights allocated representing the level of expertise. Indirect contact with wild birds either via a contaminated water source or fomites was considered the most likely pathway of introduction of low pathogenic avian influenza (LPAI) on poultry farms. Presence of a water body near the poultry farm was considered a potential pathway for introduction only when the operation type was free range and the water body was within 500m distance from the shed. The probability that LPAI will mutate to highly pathogenic avian influenza (HPAI) was considered to be higher in layer farms. Shared personnel, equipment and aerosol dispersion were the most likely pathways of shed to shed spread of the virus. For LPAI and HPAI spread from farm to farm, shared pick-up trucks for broiler and shared egg trays and egg pallets for layer farms were considered the most likely pathways. Findings from this study provide an insight on most influential practices on the introduction and spread of AI virus among commercial poultry farms in Australia, as elicited from opinions of experts. These findings will be used to support parameterization of a modelling study assessing the risk of AI introduction and spread among commercial poultry farms in Australia. PMID:29494696

Electrophysiological correlates of aesthetic processing of webpages: a comparison of experts and laypersons

PubMed Central

Hösker, Thomas M.; Hirschfeld, Gerrit; Thielsch, Meinald T.

2017-01-01

We investigated whether design experts or laypersons evaluate webpages differently. Twenty participants, 10 experts and 10 laypersons, judged the aesthetic value of a webpage in an EEG-experiment. Screenshots of 150 webpages, judged as aesthetic or as unaesthetic by another 136 participants, served as stimulus material. Behaviorally, experts and laypersons evaluated unaesthetic webpages similarly, but they differed in their evaluation of aesthetic ones: experts evaluated aesthetic webpages as unaesthetic more often than laypersons did. The ERP-data show main effects of level of expertise and of aesthetic value only. There was no interaction of expertise and aesthetics. In a time-window of 110–130 ms after stimulus onset, aesthetic webpages elicited a more negative EEG-amplitude than unaesthetic webpages. In the same time window, experts had more negative EEG-amplitudes than laypersons. This patterning of results continued until a time window of 600–800 ms in which group and aesthetic differences diminished. An interaction of perceiver characteristics and object properties that several interactionist theories postulate was absent in the EEG-data. Experts seem to process the stimuli in a more thorough manner than laypersons. The early activation differences between aesthetic and unaesthetic webpages is in contrast with some theories of aesthetic processing and has not been reported before. PMID:28603676

Electrophysiological correlates of aesthetic processing of webpages: a comparison of experts and laypersons.

PubMed

Bölte, Jens; Hösker, Thomas M; Hirschfeld, Gerrit; Thielsch, Meinald T

2017-01-01

We investigated whether design experts or laypersons evaluate webpages differently. Twenty participants, 10 experts and 10 laypersons, judged the aesthetic value of a webpage in an EEG-experiment. Screenshots of 150 webpages, judged as aesthetic or as unaesthetic by another 136 participants, served as stimulus material. Behaviorally, experts and laypersons evaluated unaesthetic webpages similarly, but they differed in their evaluation of aesthetic ones: experts evaluated aesthetic webpages as unaesthetic more often than laypersons did. The ERP-data show main effects of level of expertise and of aesthetic value only. There was no interaction of expertise and aesthetics. In a time-window of 110-130 ms after stimulus onset, aesthetic webpages elicited a more negative EEG-amplitude than unaesthetic webpages. In the same time window, experts had more negative EEG-amplitudes than laypersons. This patterning of results continued until a time window of 600-800 ms in which group and aesthetic differences diminished. An interaction of perceiver characteristics and object properties that several interactionist theories postulate was absent in the EEG-data. Experts seem to process the stimuli in a more thorough manner than laypersons. The early activation differences between aesthetic and unaesthetic webpages is in contrast with some theories of aesthetic processing and has not been reported before.

Final Technical Report on Quantifying Dependability Attributes of Software Based Safety Critical Instrumentation and Control Systems in Nuclear Power Plants

DOE Office of Scientific and Technical Information (OSTI.GOV)

Smidts, Carol; Huang, Funqun; Li, Boyuan

With the current transition from analog to digital instrumentation and control systems in nuclear power plants, the number and variety of software-based systems have significantly increased. The sophisticated nature and increasing complexity of software raises trust in these systems as a significant challenge. The trust placed in a software system is typically termed software dependability. Software dependability analysis faces uncommon challenges since software systems’ characteristics differ from those of hardware systems. The lack of systematic science-based methods for quantifying the dependability attributes in software-based instrumentation as well as control systems in safety critical applications has proved itself to be amore » significant inhibitor to the expanded use of modern digital technology in the nuclear industry. Dependability refers to the ability of a system to deliver a service that can be trusted. Dependability is commonly considered as a general concept that encompasses different attributes, e.g., reliability, safety, security, availability and maintainability. Dependability research has progressed significantly over the last few decades. For example, various assessment models and/or design approaches have been proposed for software reliability, software availability and software maintainability. Advances have also been made to integrate multiple dependability attributes, e.g., integrating security with other dependability attributes, measuring availability and maintainability, modeling reliability and availability, quantifying reliability and security, exploring the dependencies between security and safety and developing integrated analysis models. However, there is still a lack of understanding of the dependencies between various dependability attributes as a whole and of how such dependencies are formed. To address the need for quantification and give a more objective basis to the review process -- therefore reducing regulatory uncertainty -- measures and methods are needed to assess dependability attributes early on, as well as throughout the life-cycle process of software development. In this research, extensive expert opinion elicitation is used to identify the measures and methods for assessing software dependability. Semi-structured questionnaires were designed to elicit expert knowledge. A new notation system, Causal Mechanism Graphing, was developed to extract and represent such knowledge. The Causal Mechanism Graphs were merged, thus, obtaining the consensus knowledge shared by the domain experts. In this report, we focus on how software contributes to dependability. However, software dependability is not discussed separately from the context of systems or socio-technical systems. Specifically, this report focuses on software dependability, reliability, safety, security, availability, and maintainability. Our research was conducted in the sequence of stages found below. Each stage is further examined in its corresponding chapter. Stage 1 (Chapter 2): Elicitation of causal maps describing the dependencies between dependability attributes. These causal maps were constructed using expert opinion elicitation. This chapter describes the expert opinion elicitation process, the questionnaire design, the causal map construction method and the causal maps obtained. Stage 2 (Chapter 3): Elicitation of the causal map describing the occurrence of the event of interest for each dependability attribute. The causal mechanisms for the “event of interest” were extracted for each of the software dependability attributes. The “event of interest” for a dependability attribute is generally considered to be the “attribute failure”, e.g. security failure. The extraction was based on the analysis of expert elicitation results obtained in Stage 1. Stage 3 (Chapter 4): Identification of relevant measurements. Measures for the “events of interest” and their causal mechanisms were obtained from expert opinion elicitation for each of the software dependability attributes. The measures extracted are presented in this chapter. Stage 4 (Chapter 5): Assessment of the coverage of the causal maps via measures. Coverage was assessed to determine whether the measures obtained were sufficient to quantify software dependability, and what measures are further required. Stage 5 (Chapter 6): Identification of “missing” measures and measurement approaches for concepts not covered. New measures, for concepts that had not been covered sufficiently as determined in Stage 4, were identified using supplementary expert opinion elicitation as well as literature reviews. Stage 6 (Chapter 7): Building of a detailed quantification model based on the causal maps and measurements obtained. Ability to derive such a quantification model shows that the causal models and measurements derived from the previous stages (Stage 1 to Stage 5) can form the technical basis for developing dependability quantification models. Scope restrictions have led us to prioritize this demonstration effort. The demonstration was focused on a critical system, i.e. the reactor protection system. For this system, a ranking of the software dependability attributes by nuclear stakeholders was developed. As expected for this application, the stakeholder ranking identified safety as the most critical attribute to be quantified. A safety quantification model limited to the requirements phase of development was built. Two case studies were conducted for verification. A preliminary control gate for software safety for the requirements stage was proposed and applied to the first case study. The control gate allows a cost effective selection of the duration of the requirements phase.« less

A Delphi-Based Framework for systems architecting of in-orbit exploration infrastructure for human exploration beyond Low Earth Orbit

NASA Astrophysics Data System (ADS)

Aliakbargolkar, Alessandro; Crawley, Edward F.

2014-01-01

The current debate in the U.S. Human Spaceflight Program focuses on the development of the next generation of man-rated heavy lift launch vehicles. While launch vehicle systems are of critical importance for future exploration, a comprehensive analysis of the entire exploration infrastructure is required to avoid costly pitfalls at early stages of the design process. This paper addresses this need by presenting a Delphi-Based Systems Architecting Framework for integrated architectural analysis of future in-orbit infrastructure for human space exploration beyond Low Earth Orbit. The paper is structured in two parts. The first part consists of an expert elicitation study to identify objectives for the in-space transportation infrastructure. The study was conducted between November 2011 and January 2012 with 15 senior experts involved in human spaceflight in the United States and Europe. The elicitation study included the formation of three expert panels representing exploration, science, and policy stakeholders engaged in a 3-round Delphi study. The rationale behind the Delphi approach, as imported from social science research, is discussed. Finally, a novel version of the Delphi method is presented and applied to technical decision-making and systems architecting in the context of human space exploration. The second part of the paper describes a tradespace exploration study of in-orbit infrastructure coupled with a requirements definition exercise informed by expert elicitation. The uncertainties associated with technical requirements and stakeholder goals are explicitly considered in the analysis. The outcome of the expert elicitation process portrays an integrated view of perceived stakeholder needs within the human spaceflight community. Needs are subsequently converted into requirements and coupled to the system architectures of interest to analyze the correlation between exploration, science, and policy goals. Pareto analysis is used to identify architectures of interest for further consideration by decision-makers. The paper closes with a summary of insights and develops a strategy for evolutionary development of the exploration infrastructure of the incoming decades. The most important result produced by this analysis is the identification of a critical irreducible ambiguity undermining value delivery for the in-space transportation infrastructure of the next three decades: destination choice. Consensus on destination is far from being reached by the community at large, with particular reference to exploration and policy stakeholders. The realization of this ambiguity is a call for NASA to promote an open forum on this topic, and to develop a strong case for policy makers to incentivize investments in the human spaceflight industry in the next decades.

Effects of competitive pressure on expert performance: underlying psychological, physiological, and kinematic mechanisms.

PubMed

Cooke, Andrew; Kavussanu, Maria; McIntyre, David; Boardley, Ian D; Ring, Christopher

2011-08-01

Although it is well established that performance is influenced by competitive pressure, our understanding of the mechanisms which underlie the pressure-performance relationship is limited. The current experiment examined mediators of the relationship between competitive pressure and motor skill performance of experts. Psychological, physiological, and kinematic responses to three levels of competitive pressure were measured in 50 expert golfers, during a golf putting task. Elevated competitive pressure increased putting accuracy, anxiety, effort, and heart rate, but decreased grip force. Quadratic effects of pressure were noted for self-reported conscious processing and impact velocity. Mediation analyses revealed that effort and heart rate partially mediated improved performance. The findings indicate that competitive pressure elicits effects on expert performance through both psychological and physiological pathways. Copyright © 2011 Society for Psychophysiological Research.

A new way to ask the experts: Rating radioactive waste risks

DOE Office of Scientific and Technical Information (OSTI.GOV)

Kerr, R.A.

1996-11-08

The possible risks of a proposed nuclear waste repository at Yucca Mountain include the dozen or more young volcanos near by. Now some earth scientists have a new approach to evaluating hazards accounting for uncertainty at every step - `expert elicitation.` This pulls together a panel of experts, carefully assesses the uncertainties of each of their views then mathematically combines their risk estimates along with the accompanying uncertainties. The article goes on to describe just such a panel which considered seismic hazards to Yucca Mountain, how they came to their conclusions, the arguments about the conclusions, and the future ofmore » expert elicitation in evaluating the risks of nuclear waste disposal.« less

Integrating expert judgment in veterinary epidemiology: example guidance for disease freedom surveillance.

PubMed

Gustafson, L L; Gustafson, D H; Antognoli, M C; Remmenga, M D

2013-04-01

Expert opinions supplement empirical data in many epidemiologic assessments. For veterinary disease freedom surveillance, where the geographic scope of concern is often broad, populations subject to change, decisions eminent and empirical data, expert opinion can be a critical component of the decision making process. However, opinion is by definition subjective and the manner in which opinion is sought can impact the quality and reliability of estimates. Group interaction can hinder or improve the estimation process, depending on its facilitation. Further, whether and how validation is conducted can limit or increase acceptance of the resulting model. While the utility of expert opinion is widely recognized in many fields, and the impact of its use or misuse implicit, standards for application to veterinary assessments are not readily available. This paper aims to foster discussion on this influential component of epidemiology, with disease freedom application as a focus. Benefits and concerns attributed to expert judgment and guidelines for its structured elicitation are described, borrowing insights from its long history of use in decision science fields and examples from recent veterinary assessments. Published by Elsevier B.V.

Expert assessments of the cost of light water small modular reactors

PubMed Central

Abdulla, Ahmed; Azevedo, Inês Lima; Morgan, M. Granger

2013-01-01

Analysts and decision makers frequently want estimates of the cost of technologies that have yet to be developed or deployed. Small modular reactors (SMRs), which could become part of a portfolio of carbon-free energy sources, are one such technology. Existing estimates of likely SMR costs rely on problematic top-down approaches or bottom-up assessments that are proprietary. When done properly, expert elicitations can complement these approaches. We developed detailed technical descriptions of two SMR designs and then conduced elicitation interviews in which we obtained probabilistic judgments from 16 experts who are involved in, or have access to, engineering-economic assessments of SMR projects. Here, we report estimates of the overnight cost and construction duration for five reactor-deployment scenarios that involve a large reactor and two light water SMRs. Consistent with the uncertainty introduced by past cost overruns and construction delays, median estimates of the cost of new large plants vary by more than a factor of 2.5. Expert judgments about likely SMR costs display an even wider range. Median estimates for a 45 megawatts-electric (MWe) SMR range from $4,000 to $16,300/kWe and from $3,200 to $7,100/kWe for a 225-MWe SMR. Sources of disagreement are highlighted, exposing the thought processes of experts involved with SMR design. There was consensus that SMRs could be built and brought online about 2 y faster than large reactors. Experts identify more affordable unit cost, factory fabrication, and shorter construction schedules as factors that may make light water SMRs economically viable. PMID:23716682

Expert assessments of the cost of light water small modular reactors.

PubMed

Abdulla, Ahmed; Azevedo, Inês Lima; Morgan, M Granger

2013-06-11

Analysts and decision makers frequently want estimates of the cost of technologies that have yet to be developed or deployed. Small modular reactors (SMRs), which could become part of a portfolio of carbon-free energy sources, are one such technology. Existing estimates of likely SMR costs rely on problematic top-down approaches or bottom-up assessments that are proprietary. When done properly, expert elicitations can complement these approaches. We developed detailed technical descriptions of two SMR designs and then conduced elicitation interviews in which we obtained probabilistic judgments from 16 experts who are involved in, or have access to, engineering-economic assessments of SMR projects. Here, we report estimates of the overnight cost and construction duration for five reactor-deployment scenarios that involve a large reactor and two light water SMRs. Consistent with the uncertainty introduced by past cost overruns and construction delays, median estimates of the cost of new large plants vary by more than a factor of 2.5. Expert judgments about likely SMR costs display an even wider range. Median estimates for a 45 megawatts-electric (MWe) SMR range from $4,000 to $16,300/kWe and from $3,200 to $7,100/kWe for a 225-MWe SMR. Sources of disagreement are highlighted, exposing the thought processes of experts involved with SMR design. There was consensus that SMRs could be built and brought online about 2 y faster than large reactors. Experts identify more affordable unit cost, factory fabrication, and shorter construction schedules as factors that may make light water SMRs economically viable.

Not All Wizards Are from Oz: Iterative Design of Intelligent Learning Environments by Communication Capacity Tapering

ERIC Educational Resources Information Center

Mavrikis, Manolis; Gutierrez-Santos, Sergio

2010-01-01

This paper presents a methodology for the design of intelligent learning environments. We recognise that in the educational technology field, theory development and system-design should be integrated and rely on an iterative process that addresses: (a) the difficulty to elicit precise, concise, and operationalized knowledge from "experts" and (b)…

Development of an Expert Judgement Elicitation and Calibration Methodology for Risk Analysis in Conceptual Vehicle Design

NASA Technical Reports Server (NTRS)

Unal, Resit; Keating, Charles; Conway, Bruce; Chytka, Trina

2004-01-01

A comprehensive expert-judgment elicitation methodology to quantify input parameter uncertainty and analysis tool uncertainty in a conceptual launch vehicle design analysis has been developed. The ten-phase methodology seeks to obtain expert judgment opinion for quantifying uncertainties as a probability distribution so that multidisciplinary risk analysis studies can be performed. The calibration and aggregation techniques presented as part of the methodology are aimed at improving individual expert estimates, and provide an approach to aggregate multiple expert judgments into a single probability distribution. The purpose of this report is to document the methodology development and its validation through application to a reference aerospace vehicle. A detailed summary of the application exercise, including calibration and aggregation results is presented. A discussion of possible future steps in this research area is given.

Experts correctly describe demography associated with historical decline of the endangered Indiana bat, but not recent period of stationarity

USGS Publications Warehouse

Thogmartin, Wayne E.; Sanders-Reed, Carol A.; Szymanski, Jennifer; Pruitt, Lori; Runge, Michael C.

2017-01-01

Demographic characteristics of bats are often insufficiently described for modeling populations. In data poor situations, experts are often relied upon for characterizing ecological systems. In concert with the development of a matrix model describing Indiana bat (Myotis sodalis) demography, we elicited estimates for parameterizing this model from 12 experts. We conducted this elicitation in two stages, requesting expert values for 12 demographic rates. These rates were adult and juvenile seasonal (winter, summer, fall) survival rates, pup survival in fall, and propensity and success at breeding. Experts were most in agreement about adult fall survival (3% Coefficient of Variation) and least in agreement about propensity of juveniles to breed (37% CV). The experts showed greater concordance for adult ( mean CV, adult = 6.2%) than for juvenile parameters ( mean CV, juvenile = 16.4%), and slightly more agreement for survival (mean CV, survival = 9.8%) compared to reproductive rates ( mean CV, reproduction = 15.1%). However, survival and reproduction were negatively and positively biased, respectively, relative to a stationary dynamic. Despite the species exhibiting near stationary dynamics for two decades prior to the onset of a potential extinction-causing agent, white-nose syndrome, expert estimates indicated a population decline of -11% per year (95% CI = -2%, -20%); quasi-extinction was predicted within a century ( mean = 61 years to QE, range = 32, 97) by 10 of the 12 experts. Were we to use these expert estimates in our modeling efforts, we would have errantly trained our models to a rapidly declining demography asymptomatic of recent demographic behavior. While experts are sometimes the only source of information, a clear understanding of the temporal and spatial context of the information being elicited is necessary to guard against wayward predictions.

Flammable gas double shell tank expert elicitation presentations (Part A and Part B)

DOE Office of Scientific and Technical Information (OSTI.GOV)

Bratzel, D.R.

1998-04-17

This document is a compilation of presentation packages and white papers for the Flammable Gas Double Shell Tank Expert Elicitation Workshop {number_sign}2. For each presentation given by the different authors, a separate section was developed. The purpose for issuing these workshop presentation packages and white papers as a supporting document is to provide traceability and a Quality Assurance record for future reference to these packages.

Global priorities for research and the relative importance of different research outcomes: an international Delphi survey of malaria research experts.

PubMed

Mulligan, Jo-Ann; Conteh, Lesong

2016-12-06

As global research investment increases, attention inevitably turns to assessing and measuring the outcomes and impact from research programmes. Research can have many different outcomes such as producing advances in scientific knowledge, building research capacity and, ultimately, health and broader societal benefits. The aim of this study was to test the use of a Delphi methodology as a way of gathering views from malaria research experts on research priorities and eliciting relative valuations of the different types of health research impact. An international Delphi survey of 60 malaria research experts was used to understand views on research outcomes and priorities within malaria and across global health more widely. The study demonstrated the application of the Delphi technique to eliciting views on malaria specific research priorities, wider global health research priorities and the values assigned to different types of research impact. In terms of the most important past research successes, the development of new anti-malarial drugs and insecticide-treated bed nets were rated as the most important. When asked about research priorities for future funding, respondents ranked tackling emerging drug and insecticide resistance the highest. With respect to research impact, the panel valued research that focuses on health and health sector benefits and informing policy and product development. Contributions to scientific knowledge, although highly valued, came lower down the ranking, suggesting that efforts to move research discoveries to health products and services are valued more highly than pure advances in scientific knowledge. Although the Delphi technique has been used to elicit views on research questions in global health this was the first time it has been used to assess how a group of research experts value or rank different types of research impact. The results suggest it is feasible to inject the views of a key stakeholder group into the research prioritization process and the Delphi approach is a useful tool for eliciting views on the value or importance of research impact. Future work will explore other methods for assessing and valuing research impact and test the feasibility of developing a composite tool for measuring research outcomes weighted by the values of different stakeholders.

Common problems in the elicitation and analysis of expert opinion affecting probabilistic safety assessments

DOE Office of Scientific and Technical Information (OSTI.GOV)

Meyer, M.A.; Booker, J.M.

1990-01-01

Expert opinion is frequently used in probabilistic safety assessment (PSA), particularly in estimating low probability events. In this paper, we discuss some of the common problems encountered in eliciting and analyzing expert opinion data and offer solutions or recommendations. The problems are: that experts are not naturally Bayesian. People fail to update their existing information to account for new information as it becomes available, as would be predicted by the Bayesian philosophy; that experts cannot be fully calibrated. To calibrate experts, the feedback from the known quantities must be immediate, frequent, and specific to the task; that experts are limitedmore » in the number of things that they can mentally juggle at a time to 7 {plus minus} 2; that data gatherers and analysts can introduce bias by unintentionally causing an altering of the expert's thinking or answers; that the level of detail the data, or granularity, can affect the analyses; and the conditioning effect poses difficulties in gathering and analyzing of the expert data. The data that the expert gives can be conditioned on a variety of factors that can affect the analysis and the interpretation of the results. 31 refs.« less

An expert elicitation process to project the frequency and magnitude of Florida manatee mortality events caused by red tide (Karenia brevis)

USGS Publications Warehouse

Martin, Julien; Runge, Michael C.; Flewelling, Leanne J.; Deutsch, Charles J.; Landsberg, Jan H.

2017-11-20

Red tides (blooms of the harmful alga Karenia brevis) are one of the major sources of mortality for the Florida manatee (Trichechus manatus latirostris), especially in southwest Florida. It has been hypothesized that the frequency and severity of red tides may increase in the future because of global climate change and other factors. To improve our ecological forecast for the effects of red tides on manatee population dynamics and long-term persistence, we conducted a formal expert judgment process to estimate probability distributions for the frequency and relative magnitude of red-tide-related manatee mortality (RTMM) events over a 100-year time horizon in three of the four regions recognized as manatee management units in Florida. This information was used to update a population viability analysis for the Florida manatee (the Core Biological Model). We convened a panel of 12 experts in manatee biology or red-tide ecology; the panel met to frame, conduct, and discuss the elicitation. Each expert provided a best estimate and plausible low and high values (bounding a confidence level of 80 percent) for each parameter in each of three regions (Northwest, Southwest, and Atlantic) of the subspecies’ range (excluding the Upper St. Johns River region) for two time periods (0−40 and 41−100 years from present). We fitted probability distributions for each parameter, time period, and expert by using these three elicited values. We aggregated the parameter estimates elicited from individual experts and fitted a parametric distribution to the aggregated results.Across regions, the experts expected the future frequency of RTMM events to be higher than historical levels, which is consistent with the hypothesis that global climate change (among other factors) may increase the frequency of red-tide blooms. The experts articulated considerable uncertainty, however, about the future frequency of RTMM events. The historical frequency of moderate and intense RTMM (combined) in the Southwest region was 0.35 (80-percent confidence interval [CI]: 0.21−0.52), whereas the forecast probability was 0.48 (80-percent CI: 0.30−0.64) over a 40-year projected time horizon. Moderate and intense RTMM events are expected to continue to be most frequent in the Southwest region, to increase in mean frequency in the Northwest region (historical frequency of moderate and intense RTMM events [combined] in the Northwest region was 0, whereas the forecast probability was 0.12 [80-percent CI: 0.02−0.39] over a 40-year projected time horizon) and in the Atlantic region (historical frequency of moderate and intense RTMM events [combined] in the Atlantic region was 0.05 [80-percent CI: 0.005–0.18], whereas the forecast probability was 0.11 [80-percent CI: 0.03−0.25] over a 40-year projected time horizon), and to remain absent from the Upper St. Johns River region. The impact of red-tide blooms on manatee mortality has been measured for the Southwest region but not for the Northwest and Atlantic regions, where such events have been rare. The expert panel predicted that the median magnitude of RTMM events in the Atlantic and Northwest regions will be much smaller than that in the Southwest; given the large uncertainties, however, they acknowledged the possibility that these events could be larger in their mortality impacts than in the Southwest region. By its nature, forecasting requires expert judgment because it is impossible to have empirical evidence about the future. The large uncertainties in parameter estimates over a 100-year timeframe are to be expected and may also indicate that the training provided to panelists successfully minimized one common pitfall of expert judgment, that of overconfidence. This study has provided useful and needed inputs to the Florida manatee population viability analysis associated with an important and recurrent source of mortality from harmful algal blooms.

Induced seismicity hazard and risk by enhanced geothermal systems: an expert elicitation approach

NASA Astrophysics Data System (ADS)

Trutnevyte, Evelina; Azevedo, Inês L.

2018-03-01

Induced seismicity is a concern for multiple geoenergy applications, including low-carbon enhanced geothermal systems (EGS). We present the results of an international expert elicitation (n = 14) on EGS induced seismicity hazard and risk. Using a hypothetical scenario of an EGS plant and its geological context, we show that expert best-guess estimates of annualized exceedance probabilities of an M ≥ 3 event range from 0.2%-95% during reservoir stimulation and 0.2%-100% during operation. Best-guess annualized exceedance probabilities of M ≥ 5 event span from 0.002%-2% during stimulation and 0.003%-3% during operation. Assuming that tectonic M7 events could occur, some experts do not exclude induced (triggered) events of up to M7 too. If an induced M = 3 event happens at 5 km depth beneath a town with 10 000 inhabitants, most experts estimate a 50% probability that the loss is contained within 500 000 USD without any injuries or fatalities. In the case of an induced M = 5 event, there is 50% chance that the loss is below 50 million USD with the most-likely outcome of 50 injuries and one fatality or none. As we observe a vast diversity in quantitative expert judgements and underlying mental models, we conclude with implications for induced seismicity risk governance. That is, we suggest documenting individual expert judgements in induced seismicity elicitations before proceeding to consensual judgements, to convene larger expert panels in order not to cherry-pick the experts, and to aim for multi-organization multi-model assessments of EGS induced seismicity hazard and risk.

Expert AIV: Study and Prototyping of an Expert System, To Support the Conceptual AIV Phases Of Space Programs

NASA Astrophysics Data System (ADS)

Andrina, G.; Basso, V.; Saitta, L.

2004-08-01

The effort in optimising the AIV process has been mainly focused in the recent years on the standardisation of approaches and on the application of new methodologies. But the earlier the intervention, the greater the benefits in terms of cost and schedule. Early phases of AIV process relied up to now on standards that need to be tailored through company and personal expertise. A study has then been conducted in order to exploit the possibility to develop an expert system helping in making choices in the early, conceptual phase of Assembly, Integration and Verification, namely the Model Philosophy and the test definition. The work focused on a hybrid approach, allowing interaction between historical data and human expertise. The expert system that has been prototyped exploits both information elicited from domain experts and results of a Data Mining activity on the existent data bases of completed projects verification data. The Data Mining algorithms allow the extraction of past experience resident on ESA/ MATD data base, which contains information in the form of statistical summaries, costs, frequencies of on-ground and in flight failures. Finding non-trivial associations could then be utilised by the experts to manage new decisions in a controlled way (Standards driven) at the beginning or during the AIV Process Moreover, the Expert AIV could allow compilation of a set of feasible AIV schedules to support further programmatic-driven choices.

a New Method for Fmeca Based on Fuzzy Theory and Expert System

NASA Astrophysics Data System (ADS)

Byeon, Yoong-Tae; Kim, Dong-Jin; Kim, Jin-O.

2008-10-01

Failure Mode Effects and Criticality Analysis (FMECA) is one of most widely used methods in modern engineering system to investigate potential failure modes and its severity upon the system. FMECA evaluates criticality and severity of each failure mode and visualize the risk level matrix putting those indices to column and row variable respectively. Generally, those indices are determined subjectively by experts and operators. However, this process has no choice but to include uncertainty. In this paper, a method for eliciting expert opinions considering its uncertainty is proposed to evaluate the criticality and severity. In addition, a fuzzy expert system is constructed in order to determine the crisp value of risk level for each failure mode. Finally, an illustrative example system is analyzed in the case study. The results are worth considering in deciding the proper policies for each component of the system.

Protocol for evaluation of the cost-effectiveness of ePrescribing systems and candidate prototype for other related health information technologies

PubMed Central

2014-01-01

Background This protocol concerns the assessment of cost-effectiveness of hospital health information technology (HIT) in four hospitals. Two of these hospitals are acquiring ePrescribing systems incorporating extensive decision support, while the other two will implement systems incorporating more basic clinical algorithms. Implementation of an ePrescribing system will have diffuse effects over myriad clinical processes, so the protocol has to deal with a large amount of information collected at various ‘levels’ across the system. Methods/Design The method we propose is use of Bayesian ideas as a philosophical guide. Assessment of cost-effectiveness requires a number of parameters in order to measure incremental cost utility or benefit – the effectiveness of the intervention in reducing frequency of preventable adverse events; utilities for these adverse events; costs of HIT systems; and cost consequences of adverse events averted. There is no single end-point that adequately and unproblematically captures the effectiveness of the intervention; we therefore plan to observe changes in error rates and adverse events in four error categories (death, permanent disability, moderate disability, minimal effect). For each category we will elicit and pool subjective probability densities from experts for reductions in adverse events, resulting from deployment of the intervention in a hospital with extensive decision support. The experts will have been briefed with quantitative and qualitative data from the study and external data sources prior to elicitation. Following this, there will be a process of deliberative dialogues so that experts can “re-calibrate” their subjective probability estimates. The consolidated densities assembled from the repeat elicitation exercise will then be used to populate a health economic model, along with salient utilities. The credible limits from these densities can define thresholds for sensitivity analyses. Discussion The protocol we present here was designed for evaluation of ePrescribing systems. However, the methodology we propose could be used whenever research cannot provide a direct and unbiased measure of comparative effectiveness. PMID:25038609

Eliciting improved quantitative judgements using the IDEA protocol: A case study in natural resource management.

PubMed

Hemming, Victoria; Walshe, Terry V; Hanea, Anca M; Fidler, Fiona; Burgman, Mark A

2018-01-01

Natural resource management uses expert judgement to estimate facts that inform important decisions. Unfortunately, expert judgement is often derived by informal and largely untested protocols, despite evidence that the quality of judgements can be improved with structured approaches. We attribute the lack of uptake of structured protocols to the dearth of illustrative examples that demonstrate how they can be applied within pressing time and resource constraints, while also improving judgements. In this paper, we demonstrate how the IDEA protocol for structured expert elicitation may be deployed to overcome operational challenges while improving the quality of judgements. The protocol was applied to the estimation of 14 future abiotic and biotic events on the Great Barrier Reef, Australia. Seventy-six participants with varying levels of expertise related to the Great Barrier Reef were recruited and allocated randomly to eight groups. Each participant provided their judgements using the four-step question format of the IDEA protocol ('Investigate', 'Discuss', 'Estimate', 'Aggregate') through remote elicitation. When the events were realised, the participant judgements were scored in terms of accuracy, calibration and informativeness. The results demonstrate that the IDEA protocol provides a practical, cost-effective, and repeatable approach to the elicitation of quantitative estimates and uncertainty via remote elicitation. We emphasise that i) the aggregation of diverse individual judgements into pooled group judgments almost always outperformed individuals, and ii) use of a modified Delphi approach helped to remove linguistic ambiguity, and further improved individual and group judgements. Importantly, the protocol encourages review, critical appraisal and replication, each of which is required if judgements are to be used in place of data in a scientific context. The results add to the growing body of literature that demonstrates the merit of using structured elicitation protocols. We urge decision-makers and analysts to use insights and examples to improve the evidence base of expert judgement in natural resource management.

Bayesian Networks for Modeling Dredging Decisions

DTIC Science & Technology

2011-10-01

change scenarios. Arctic Expert elicitation Netica Bacon et al . 2002 Identify factors that might lead to a change in land use from farming to...tree) algorithms developed by Lauritzen and Spiegelhalter (1988) and Jensen et al . (1990). Statistical inference is simply the process of...causality when constructing a Bayesian network (Kjaerulff and Madsen 2008, Darwiche 2009, Marcot et al . 2006). A knowledge representation approach is the

Cost estimation: An expert-opinion approach. [cost analysis of research projects using the Delphi method (forecasting)

NASA Technical Reports Server (NTRS)

Buffalano, C.; Fogleman, S.; Gielecki, M.

1976-01-01

A methodology is outlined which can be used to estimate the costs of research and development projects. The approach uses the Delphi technique a method developed by the Rand Corporation for systematically eliciting and evaluating group judgments in an objective manner. The use of the Delphi allows for the integration of expert opinion into the cost-estimating process in a consistent and rigorous fashion. This approach can also signal potential cost-problem areas. This result can be a useful tool in planning additional cost analysis or in estimating contingency funds. A Monte Carlo approach is also examined.

Numerical Algorithms for the Analysis of Expert Opinions Elicited in Text Format

DTIC Science & Technology

2013-04-01

generative process just described actually means. No “real” intelligible document will ever be gen- erated from such a process, rather, a bag of words...Linguistics, 1992, pp. 977–981. Bra89. N. Bratchell, Cluster analysis, Chememeotrics and Intelligent Laboratory Sys- tems 6 (1989), 105–125. Bre99. P...with graphical models, Journal of Arti- ficial Intelligence Research 2 96 (1994), 159–225. UNCLASSIFIED 39 DSTO–TR–2797 UNCLASSIFIED Bun02. Wray L

Expert Elicitation Provides a Rapid Alternative to Formal Case-Control Study of an H7N9 Avian Influenza Outbreak in the United States.

PubMed

Gustafson, L; Jones, R; Dufour-Zavala, L; Jensen, E; Malinak, C; McCarter, S; Opengart, K; Quinn, J; Slater, T; Delgado, A; Talbert, M; Garber, L; Remmenga, M; Smeltzer, M

2018-06-01

An expert elicitation was staged to rapidly decipher plausible routes and risks of pathogen transmission in the 2017 H7N9 avian influenza (AI) outbreak in the four-state region of Tennessee, Alabama, Georgia, and Kentucky. The process included the identification of risk factors found in a preponderance of commercial broiler breeder case farms over matched controls and an opinion-based weighting of risks and mitigations perceived influential to this outbreak. Although the two highly pathogenic AI case farms had general location and company ownership in common, obvious connections were lacking for the remainder of H7N9-infected (all low pathogenicity) commercial farms. Expert elicitation of differences between known cases and controls suggested a key role for environmental rather than lateral (business network) pathways in the distribution of low pathogenicity AI across commercial broiler breeder operations. Factors with greatest strength as predictors of disease, whether or not they were causal, included mesopredator or rodent incursions, enclosure defects, and habitat disturbance that might attract wildlife to the farm (e.g., feed spills and vacating of neighboring properties). Business affiliations that may have facilitated farm-to-farm transfer, in contrast, were limited. Biosecurity standards varied across this study group but were no more or less stringent among cases over controls. However, results from a parallel hypothetical scenario staged to address field data gaps suggest that uniformity and consistency in the implementation of biosecurity practices may impact risk of disease introduction. Importantly, this study was conducted within a few weeks and with little disruption to emergency response activities. As such, the approach offers an alternative model for interim field investigation of new or emerging high-consequence diseases with immediate decision support needs.

Eliciting and Representing High-Level Knowledge Requirements to Discover Ecological Knowledge in Flower-Visiting Data

PubMed Central

2016-01-01

Observations of individual organisms (data) can be combined with expert ecological knowledge of species, especially causal knowledge, to model and extract from flower–visiting data useful information about behavioral interactions between insect and plant organisms, such as nectar foraging and pollen transfer. We describe and evaluate a method to elicit and represent such expert causal knowledge of behavioral ecology, and discuss the potential for wider application of this method to the design of knowledge-based systems for knowledge discovery in biodiversity and ecosystem informatics. PMID:27851814

Creating a test blueprint for a progress testing program: A paired-comparisons approach.

PubMed

von Bergmann, HsingChi; Childs, Ruth A

2018-03-01

Creating a new testing program requires the development of a test blueprint that will determine how the items on each test form are distributed across possible content areas and practice domains. To achieve validity, categories of a blueprint are typically based on the judgments of content experts. How experts judgments are elicited and combined is important to the quality of resulting test blueprints. Content experts in dentistry participated in a day-long faculty-wide workshop to discuss, refine, and confirm the categories and their relative weights. After reaching agreement on categories and their definitions, experts judged the relative importance between category pairs, registering their judgments anonymously using iClicker, an audience response system. Judgments were combined in two ways: a simple calculation that could be performed during the workshop and a multidimensional scaling of the judgments performed later. Content experts were able to produce a set of relative weights using this approach. The multidimensional scaling yielded a three-dimensional model with the potential to provide deeper insights into the basis of the experts' judgments. The approach developed and demonstrated in this study can be applied across academic disciplines to elicit and combine content experts judgments for the development of test blueprints.

Probabilistic accident consequence uncertainty analysis -- Uncertainty assessment for internal dosimetry. Volume 2: Appendices

DOE Office of Scientific and Technical Information (OSTI.GOV)

Goossens, L.H.J.; Kraan, B.C.P.; Cooke, R.M.

1998-04-01

The development of two new probabilistic accident consequence codes, MACCS and COSYMA, was completed in 1990. These codes estimate the consequence from the accidental releases of radiological material from hypothesized accidents at nuclear installations. In 1991, the US Nuclear Regulatory Commission and the Commission of the European Communities began cosponsoring a joint uncertainty analysis of the two codes. The ultimate objective of this joint effort was to systematically develop credible and traceable uncertainty distributions for the respective code input variables. A formal expert judgment elicitation and evaluation process was identified as the best technology available for developing a library ofmore » uncertainty distributions for these consequence parameters. This report focuses on the results of the study to develop distribution for variables related to the MACCS and COSYMA internal dosimetry models. This volume contains appendices that include (1) a summary of the MACCS and COSYMA consequence codes, (2) the elicitation questionnaires and case structures, (3) the rationales and results for the panel on internal dosimetry, (4) short biographies of the experts, and (5) the aggregated results of their responses.« less

Expert opinion on landslide susceptibility elicted by probabilistic inversion from scenario rankings

NASA Astrophysics Data System (ADS)

Lee, Katy; Dashwood, Claire; Lark, Murray

2016-04-01

For many natural hazards the opinion of experts, with experience in assessing susceptibility under different circumstances, is a valuable source of information on which to base risk assessments. This is particularly important where incomplete process understanding, and limited data, limit the scope to predict susceptibility by mechanistic or statistical modelling. The expert has a tacit model of a system, based on their understanding of processes and their field experience. This model may vary in quality, depending on the experience of the expert. There is considerable interest in how one may elicit expert understanding by a process which is transparent and robust, to provide a basis for decision support. One approach is to provide experts with a set of scenarios, and then to ask them to rank small overlapping subsets of these with respect to susceptibility. Methods of probabilistic inversion have been used to compute susceptibility scores for each scenario, implicit in the expert ranking. It is also possible to model these scores as functions of measurable properties of the scenarios. This approach has been used to assess susceptibility of animal populations to invasive diseases, to assess risk to vulnerable marine environments and to assess the risk in hypothetical novel technologies for food production. We will present the results of a study in which a group of geologists with varying degrees of expertise in assessing landslide hazards were asked to rank sets of hypothetical simplified scenarios with respect to land slide susceptibility. We examine the consistency of their rankings and the importance of different properties of the scenarios in the tacit susceptibility model that their rankings implied. Our results suggest that this is a promising approach to the problem of how experts can communicate their tacit model of uncertain systems to those who want to make use of their expertise.

A reusable knowledge acquisition shell: KASH

NASA Technical Reports Server (NTRS)

Westphal, Christopher; Williams, Stephen; Keech, Virginia

1991-01-01

KASH (Knowledge Acquisition SHell) is proposed to assist a knowledge engineer by providing a set of utilities for constructing knowledge acquisition sessions based on interviewing techniques. The information elicited from domain experts during the sessions is guided by a question dependency graph (QDG). The QDG defined by the knowledge engineer, consists of a series of control questions about the domain that are used to organize the knowledge of an expert. The content information supplies by the expert, in response to the questions, is represented in the form of a concept map. These maps can be constructed in a top-down or bottom-up manner by the QDG and used by KASH to generate the rules for a large class of expert system domains. Additionally, the concept maps can support the representation of temporal knowledge. The high degree of reusability encountered in the QDG and concept maps can vastly reduce the development times and costs associated with producing intelligent decision aids, training programs, and process control functions.

Two-dimensional fuzzy fault tree analysis for chlorine release from a chlor-alkali industry using expert elicitation.

PubMed

Renjith, V R; Madhu, G; Nayagam, V Lakshmana Gomathi; Bhasi, A B

2010-11-15

The hazards associated with major accident hazard (MAH) industries are fire, explosion and toxic gas releases. Of these, toxic gas release is the worst as it has the potential to cause extensive fatalities. Qualitative and quantitative hazard analyses are essential for the identification and quantification of these hazards related to chemical industries. Fault tree analysis (FTA) is an established technique in hazard identification. This technique has the advantage of being both qualitative and quantitative, if the probabilities and frequencies of the basic events are known. This paper outlines the estimation of the probability of release of chlorine from storage and filling facility of chlor-alkali industry using FTA. An attempt has also been made to arrive at the probability of chlorine release using expert elicitation and proven fuzzy logic technique for Indian conditions. Sensitivity analysis has been done to evaluate the percentage contribution of each basic event that could lead to chlorine release. Two-dimensional fuzzy fault tree analysis (TDFFTA) has been proposed for balancing the hesitation factor involved in expert elicitation. Copyright © 2010 Elsevier B.V. All rights reserved.

The Different Inhibition of Return (IOR) Effects of Emergency Managerial Experts and Novices: An Event-Related Potentials Study

PubMed Central

Cao, Rong; Wu, Lü; Wang, Shuzhen

2017-01-01

Inhibition of return (IOR) is an important effect of attention. However, the IOR of emergency managerial experts is unknown. By employing emergency and natural scene pictures in expert-novice paradigm, the present study explored the neural activity underlying the IOR effects for emergency managerial experts and novices. In behavioral results, there were no differences of IOR effects between novices and emergency managerial experts, while the event-related potentials (ERPs) results were different between novices and experts. In Experiment 1 (novice group), ERPs results showed no any IOR was robust at both stimulus-onset asynchrony (SOA) of 200 ms and 400 ms. In Experiment 2 (expert group), ERPs results showed an enhanced N2 at SOA of 200 ms and attenuated P3 at cued location in the right parietal lobe and adjacent brain regions than uncued location at SOA of 200 ms. The findings of the two experiments showed that, relative to the novices, IOR for the emergency managerial experts was robust, and dominated in the right parietal lobe and adjacent brain regions, suggesting more flexible attentional processing and higher visual search efficiency of the emergency managerial experts. The findings indicate that the P3, possible N2, over the right parietal lobe and adjacent brain regions are the biological indicators for IOR elicited by post-cued emergency pictures for emergency managerial experts. PMID:28588459

Developing and Testing a Model to Predict Outcomes of Organizational Change

PubMed Central

Gustafson, David H; Sainfort, François; Eichler, Mary; Adams, Laura; Bisognano, Maureen; Steudel, Harold

2003-01-01

Objective To test the effectiveness of a Bayesian model employing subjective probability estimates for predicting success and failure of health care improvement projects. Data Sources Experts' subjective assessment data for model development and independent retrospective data on 221 healthcare improvement projects in the United States, Canada, and the Netherlands collected between 1996 and 2000 for validation. Methods A panel of theoretical and practical experts and literature in organizational change were used to identify factors predicting the outcome of improvement efforts. A Bayesian model was developed to estimate probability of successful change using subjective estimates of likelihood ratios and prior odds elicited from the panel of experts. A subsequent retrospective empirical analysis of change efforts in 198 health care organizations was performed to validate the model. Logistic regression and ROC analysis were used to evaluate the model's performance using three alternative definitions of success. Data Collection For the model development, experts' subjective assessments were elicited using an integrative group process. For the validation study, a staff person intimately involved in each improvement project responded to a written survey asking questions about model factors and project outcomes. Results Logistic regression chi-square statistics and areas under the ROC curve demonstrated a high level of model performance in predicting success. Chi-square statistics were significant at the 0.001 level and areas under the ROC curve were greater than 0.84. Conclusions A subjective Bayesian model was effective in predicting the outcome of actual improvement projects. Additional prospective evaluations as well as testing the impact of this model as an intervention are warranted. PMID:12785571

Cancer risk elicitation and communication: lessons from the psychology of risk perception.

PubMed

Klein, William M P; Stefanek, Michael E

2007-01-01

Cancer risk perceptions are a key predictor of risk-reduction practices, health behaviors, and processing of cancer information. Nevertheless, patients and the general public (as well as health care providers) exhibit a number of errors and biases in the way they think about risk, such that their risk perceptions and decisions deviate greatly from those prescribed by normative decision models and by experts in risk assessment. For example, people are more likely to engage in screening behaviors such as mammography when faced with loss-based messages than gain-framed messages, and they often ignore the base rate of a given disease when assessing their own risk of obtaining this disease. In this article, we review many of the psychological processes that underlie risk perception and discuss how these processes lead to such deviations. Among these processes are difficulties with use of numerical information (innumeracy), cognitive processes (eg, use of time-saving heuristics), motivational factors (eg, loss and regret aversion), and emotion. We conclude with suggestions for future research in the area, as well as implications for improving the elicitation and communication of personal cancer risk.

Uncertainty in geohazard understanding and communicating hazards and risk between scientists and the UK Insurance industry.

NASA Astrophysics Data System (ADS)

Lawley, Russell; Lee, Kathryn; Lark, Murray

2015-04-01

At BGS, expert elicitation has been used to evaluate the relative impacts arising from multiple geohazards that can affect UK housing stock. In turn this 'consensus' understanding has been developed into a tool to assist the UK insurance industry underwrite the domestic property market. BGS models six geohazards deemed relevant to UK Housing: Landslides, Shrink-swell (heave), Compressibles, Dissolution (Karst), collapsibles and running sand. The models are widely used and have been developed over 2 decades of research. However, stakeholders such as the insurance industry are not well equipped to managed geohazard models directly and need the information to be categorised in a meaningful way, appropriate to their business models. Creating terminologies to communicate the relative threats for each geohazard has been relatively straightforward, but communicating the impacts of multiple geohazards, or comparing the relative risks of one geohazard against another has proved more difficult. Expert elicitation has been used since 2010 to try and build a consensus model for geohazards and to help BGS communicate its knowledge to stakeholders. Typically, the BGS geohazard models are provided with 5 levels of susceptibility: A (low or absent) ,B, C, D and E (high). Resolving individual models is relatively simple, but the insurance market is extremely dynamic and a need to simplify and convey the possible threats from all geohazards into a single 'rating' of susceptibility has emerged. This poses a problem when trying to convey the geological understanding behind the models. For example, how do you convey the combined (or comparative) susceptibility of a high susceptibility to Dissolution, with a moderate susceptibility to Landslides. This complexity is further hampered when needing to consider that stakeholders resolve spatial distributions via use of frameworks such as 'Postcode' sectors, and that the outputs of most geohazard models are sensitive to scope and scale of such frameworks. The elicitation process (the first to be deployed by BGS) allowed a significant degree of structured knowledge-exchange between experts of differing geohazards backgrounds. Consensus over likely impacts arising from the geohazards was achieved (where previously there had been none). In the process of harmonising the models it became clear that further elicitation (within BGS and externally) could be used to refine the models on a more regular basis and provide a consistency relevant to other industries (such as construction). By establishing a consensus, it has been possible to provide improved understanding to the insurance industry with simpler metrics, whilst maintaining scope for also conveying the underlying complexity and natural variance in the models. We will discuss our experience of the use of elicitation methodology and the implications of our results for further work at the BGS to convey uncertain and complex models to stakeholders and non-geologists.

Hybrid expert system for decision supporting in the medical area: complexity and cognitive computing.

PubMed

Brasil, L M; de Azevedo, F M; Barreto, J M

2001-09-01

This paper proposes a hybrid expert system (HES) to minimise some complexity problems pervasive to the artificial intelligence such as: the knowledge elicitation process, known as the bottleneck of expert systems; the model choice for knowledge representation to code human reasoning; the number of neurons in the hidden layer and the topology used in the connectionist approach; the difficulty to obtain the explanation on how the network arrived to a conclusion. Two algorithms applied to developing of HES are also suggested. One of them is used to train the fuzzy neural network and the other to obtain explanations on how the fuzzy neural network attained a conclusion. To overcome these difficulties the cognitive computing was integrated to the developed system. A case study is presented (e.g. epileptic crisis) with the problem definition and simulations. Results are also discussed.

Elicitation Support Requirements of Multi-Expertise Teams

ERIC Educational Resources Information Center

Bitter-Rijpkema, Marlies; Martens, Rob; Jochems, Wim

2005-01-01

Tools to support knowledge elicitation are used more and more in situations where employees or students collaborate using the computer. Studies indicate that differences exist between experts and novices regarding their methods of work and reasoning. However, the commonly preferred approach tends to deal with team members as a single system with…

Expert elicitation as a means to attribute 28 enteric pathogens to foodborne, waterborne, animal contact, and person-to-person transmission routes in Canada.

PubMed

Butler, Ainslie J; Thomas, M Kate; Pintar, Katarina D M

2015-04-01

Enteric illness contributes to a significant burden of illness in Canada and globally. Understanding its sources is a critical step in identifying and preventing health risks. Expert elicitation is a powerful tool, used previously, to obtain information about enteric illness source attribution where information is difficult or expensive to obtain. Thirty-one experts estimated transmission of 28 pathogens via major transmission routes (foodborne, waterborne, animal contact, person-to-person, and other) at the point of consumption. The elicitation consisted of a (snowball) recruitment phase; administration of a pre-survey to collect background information, an introductory webinar, an elicitation survey, a 1-day discussion, survey readministration, and a feedback exercise, and surveys were administered online. Experts were prompted to quantify changes in contamination at the point of entry into the kitchen versus point of consumption. Estimates were combined via triangular probability distributions, and medians and 90% credible-interval estimates were produced. Transmission was attributed primarily to food for Bacillus cereus, Clostridium perfringens, Cyclospora cayetanensis, Trichinella spp., all three Vibrio spp. categories explored, and Yersinia enterocolitica. Multisource pathogens (e.g., transmitted commonly through both water and food) such as Campylobacter spp., four Escherichia coli categories, Listeria monocytogenes, Salmonella spp., and Staphylococcus aureus were also estimated as mostly foodborne. Water was the primary pathway for Giardia spp. and Cryptosporidium spp., and person-to-person transmission dominated for six enteric viruses and Shigella spp. Consideration of the point of attribution highlighted the importance of food handling and cross-contamination in the transmission pathway. This study provides source attribution estimates of enteric illness for Canada, considering all possible transmission routes. Further research is necessary to improve our understanding of poorly characterized pathogens such as sapovirus and E. coli subgroups in Canada.

Attribution of global foodborne disease to specific foods: Findings from a World Health Organization structured expert elicitation.

PubMed

Hoffmann, Sandra; Devleesschauwer, Brecht; Aspinall, Willy; Cooke, Roger; Corrigan, Tim; Havelaar, Arie; Angulo, Frederick; Gibb, Herman; Kirk, Martyn; Lake, Robin; Speybroeck, Niko; Torgerson, Paul; Hald, Tine

2017-01-01

Recently the World Health Organization, Foodborne Disease Burden Epidemiology Reference Group (FERG) estimated that 31 foodborne diseases (FBDs) resulted in over 600 million illnesses and 420,000 deaths worldwide in 2010. Knowing the relative role importance of different foods as exposure routes for key hazards is critical to preventing illness. This study reports the findings of a structured expert elicitation providing globally comparable food source attribution estimates for 11 major FBDs in each of 14 world subregions. We used Cooke's Classical Model to elicit and aggregate judgments of 73 international experts. Judgments were elicited from each expert individually and aggregated using both equal and performance weights. Performance weighted results are reported as they increased the informativeness of estimates, while retaining accuracy. We report measures of central tendency and uncertainty bounds on food source attribution estimate. For some pathogens we see relatively consistent food source attribution estimates across subregions of the world; for others there is substantial regional variation. For example, for non-typhoidal salmonellosis, pork was of minor importance compared to eggs and poultry meat in the American and African subregions, whereas in the European and Western Pacific subregions the importance of these three food sources were quite similar. Our regional results broadly agree with estimates from earlier European and North American food source attribution research. As in prior food source attribution research, we find relatively wide uncertainty bounds around our median estimates. We present the first worldwide estimates of the proportion of specific foodborne diseases attributable to specific food exposure routes. While we find substantial uncertainty around central tendency estimates, we believe these estimates provide the best currently available basis on which to link FBDs and specific foods in many parts of the world, providing guidance for policy actions to control FBDs.

Eliciting climate experts' knowledge to address model uncertainties in regional climate projections: a case study of Guanacaste, Northwest Costa Rica

NASA Astrophysics Data System (ADS)

Grossmann, I.; Steyn, D. G.

2014-12-01

Global general circulation models typically cannot provide the detailed and accurate regional climate information required by stakeholders for climate adaptation efforts, given their limited capacity to resolve the regional topography and changes in local sea surface temperature, wind and circulation patterns. The study region in Northwest Costa Rica has a tropical wet-dry climate with a double-peak wet season. During the dry season the central Costa Rican mountains prevent tropical Atlantic moisture from reaching the region. Most of the annual precipitation is received following the northward migration of the ITCZ in May that allows the region to benefit from moist southwesterly flow from the tropical Pacific. The wet season begins with a short period of "early rains" and is interrupted by the mid-summer drought associated with the intensification and westward expansion of the North Atlantic subtropical high in late June. Model projections for the 21st century indicate a lengthening and intensification of the mid-summer drought and a weakening of the early rains on which current crop cultivation practices rely. We developed an expert elicitation to systematically address uncertainties in the available model projections of changes in the seasonal precipitation pattern. Our approach extends an elicitation approach developed previously at Carnegie Mellon University. Experts in the climate of the study region or Central American climate were asked to assess the mechanisms driving precipitation during each part of the season, uncertainties regarding these mechanisms, expected changes in each mechanism in a warming climate, and the capacity of current models to reproduce these processes. To avoid overconfidence bias, a step-by-step procedure was followed to estimate changes in the timing and intensity of precipitation during each part of the season. The questions drew upon interviews conducted with the regions stakeholders to assess their climate information needs. This study is part of the FuturAgua project funded by the Belmont Freshwater Security call. The expert opinions on expected changes in the seasonal precipitation pattern are being used to inform regional efforts to build drought resilience and to create and compare alternative water management strategies with the region's stakeholders.

At-risk adolescents as experts in a new requirements elicitation procedure for the development of a smart phone psychoeducational trauma-informed care application.

PubMed

Sockolow, Paulina; Schug, Seran; Zhu, Jichen; Smith, T J; Senathirajah, Yalini; Bloom, Sandra

2017-01-01

Adolescents from urban, socioeconomically disadvantaged communities of color encounter high rates of adverse childhood experiences. To address the resulting multidimensional problems, we developed an innovative approach, Experiential Participatory and Interactive Knowledge Elicitation (EPIKE), using remote experiential needs elicitation methods to generate design and content requirements for a mobile health (mHealth) psychoeducational intervention. At a community-based organization in a northeastern city, the research team developed EPIKE by incorporating elicitation of input on the graphics and conducting remotely recorded experiential meetings and iterative reviews of the design to produce an mHealth smartphone story application (app) prototype for the participants to critique. The 22 participants were 13- to 17-year-olds, predominantly African American and female, from underresourced communities. The four goals of the design process were attained: 1) story development from participant input; 2) needs-elicitation that reflected the patient-centered care approach; 3) interactive story game creation that accommodates the participants' emotional and cognitive developmental needs; 4) development of a game that adolescents can relate to and that which matches their comfort levels of emotional intensity. The EPIKE approach can be used successfully to identify the needs of adolescents across the digital divide to inform the design and development of mHealth apps.

COMPARISON OF METHODS FOR ESTIMATING CONTEMPORARY COSTS: AN APPLICATION TO LIVER TRANSPLANTATION IN THE UNITED KINGDOM.

PubMed

Singh, Jeshika; Longworth, Louise

2017-01-01

Our study addresses the important issue of estimating treatment costs from historical data. It is a problem frequently faced by health technology assessment analysts. We compared four approaches used to estimate current costs when good quality contemporary data are not available using liver transplantation as an example. First, the total cost estimates extracted for patients from a cohort study, conducted in the 1990s, were inflated using a published inflation multiplier. Second, resource use estimates from the cohort study were extracted for hepatitis C patients and updated using current unit costs. Third, expert elicitation was carried out to identify changes in clinical practice over time and quantify current resource use. Fourth, routine data on resource use were obtained from National Health Service Blood and Transplant (NHSBT). The first two methods did not account for changes in clinical practice. Also the first was not specific to hepatitis patients. The use of experts confirmed significant changes in clinical practice. However, the quantification of resource use using experts is challenging as clinical specialists may not have a complete overview of clinical pathway. The NHSBT data are the most accurate reflection of transplantation and posttransplantation phase; however, data were not available for the whole pathway of care. The best estimate of total cost, combining NHSBT data and expert elicitation, is £121,211. Observational data from routine care are potentially the most reliable reflection of current resource use. Efforts should be made to make such data readily available and accessible to researchers. Expert elicitation provided reasonable estimates.

Probabilistic accident consequence uncertainty analysis -- Uncertainty assessment for deposited material and external doses. Volume 2: Appendices

DOE Office of Scientific and Technical Information (OSTI.GOV)

Goossens, L.H.J.; Kraan, B.C.P.; Cooke, R.M.

1997-12-01

The development of two new probabilistic accident consequence codes, MACCS and COSYMA, was completed in 1990. These codes estimate the consequence from the accidental releases of radiological material from hypothesized accidents at nuclear installations. In 1991, the US Nuclear Regulatory Commission and the Commission of the European Communities began cosponsoring a joint uncertainty analysis of the two codes. The ultimate objective of this joint effort was to systematically develop credible and traceable uncertainty distributions for the respective code input variables. A formal expert judgment elicitation and evaluation process was identified as the best technology available for developing a library ofmore » uncertainty distributions for these consequence parameters. This report focuses on the results of the study to develop distribution for variables related to the MACCS and COSYMA deposited material and external dose models. This volume contains appendices that include (1) a summary of the MACCS and COSYMA consequence codes, (2) the elicitation questionnaires and case structures, (3) the rationales and results for the panel on deposited material and external doses, (4) short biographies of the experts, and (5) the aggregated results of their responses.« less

Job Exposure Matrix for Electric Shock Risks with Their Uncertainties

PubMed Central

Vergara, Ximena P.; Fischer, Heidi J.; Yost, Michael; Silva, Michael; Lombardi, David A.; Kheifets, Leeka

2015-01-01

We present an update to an electric shock job exposure matrix (JEM) that assigned ordinal electric shocks exposure for 501 occupational titles based on electric shocks and electrocutions from two available data sources and expert judgment. Using formal expert elicitation and starting with data on electric injury, we arrive at a consensus-based JEM. In our new JEM, we quantify exposures by adding three new dimensions: (1) the elicited median proportion; (2) the elicited 25th percentile; and (3) and the elicited 75th percentile of those experiencing occupational electric shocks in a working lifetime. We construct the relative interquartile range (rIQR) based on uncertainty interval and the median. Finally, we describe overall results, highlight examples demonstrating the impact of cut point selection on exposure assignment, and evaluate potential impacts of such selection on epidemiologic studies of the electric work environment. In conclusion, novel methods allowed for consistent exposure estimates that move from qualitative to quantitative measures in this population-based JEM. Overlapping ranges of median exposure in various categories reflect our limited knowledge about this exposure. PMID:25856552

Job exposure matrix for electric shock risks with their uncertainties.

PubMed

Vergara, Ximena P; Fischer, Heidi J; Yost, Michael; Silva, Michael; Lombardi, David A; Kheifets, Leeka

2015-04-08

We present an update to an electric shock job exposure matrix (JEM) that assigned ordinal electric shocks exposure for 501 occupational titles based on electric shocks and electrocutions from two available data sources and expert judgment. Using formal expert elicitation and starting with data on electric injury, we arrive at a consensus-based JEM. In our new JEM, we quantify exposures by adding three new dimensions: (1) the elicited median proportion; (2) the elicited 25th percentile; and (3) and the elicited 75th percentile of those experiencing occupational electric shocks in a working lifetime. We construct the relative interquartile range (rIQR) based on uncertainty interval and the median. Finally, we describe overall results, highlight examples demonstrating the impact of cut point selection on exposure assignment, and evaluate potential impacts of such selection on epidemiologic studies of the electric work environment. In conclusion, novel methods allowed for consistent exposure estimates that move from qualitative to quantitative measures in this population-based JEM. Overlapping ranges of median exposure in various categories reflect our limited knowledge about this exposure.

The tale of hearts and reason: the influence of mood on decision making.

PubMed

Laborde, Sylvain; Raab, Markus

2013-08-01

In decision-making research, one important aspect of real-life decisions has so far been neglected: the mood of the decision maker when generating options. The authors tested the use of the take-the-first (TTF) heuristic and extended the TTF model to understand how mood influences the option-generation process of individuals in two studies, the first using a between-subjects design (30 nonexperts, 30 near-experts, and 30 experts) and the second conceptually replicating the first using a within-subject design (30 nonexperts). Participants took part in an experimental option-generation task, with 31 three-dimensional videos of choices in team handball. Three moods were elicited: positive, neutral, and negative. The findings (a) replicate previous results concerning TTF and (b) show that the option-generation process was associated with the physiological component of mood, supporting the neurovisceral integration model. The extension of TTF to processing emotional factors is an important step forward in explaining fast choices in real-life situations.

Expert judgments about RD&D and the future of nuclear energy.

PubMed

Anadón, Laura D; Bosetti, Valentina; Bunn, Matthew; Catenacci, Michela; Lee, Audrey

2012-11-06

Probabilistic estimates of the cost and performance of future nuclear energy systems under different scenarios of government research, development, and demonstration (RD&D) spending were obtained from 30 U.S. and 30 European nuclear technology experts. We used a novel elicitation approach which combined individual and group elicitation. With no change from current RD&D funding levels, experts on average expected current (Gen. III/III+) designs to be somewhat more expensive in 2030 than they were in 2010, and they expected the next generation of designs (Gen. IV) to be more expensive still as of 2030. Projected costs of proposed small modular reactors (SMRs) were similar to those of Gen. IV systems. The experts almost unanimously recommended large increases in government support for nuclear RD&D (generally 2-3 times current spending). The majority expected that such RD&D would have only a modest effect on cost, but would improve performance in other areas, such as safety, waste management, and uranium resource utilization. The U.S. and E.U. experts were in relative agreement regarding how government RD&D funds should be allocated, placing particular focus on very high temperature reactors, sodium-cooled fast reactors, fuels and materials, and fuel cycle technologies.

An iterative requirements specification procedure for decision support systems.

PubMed

Brookes, C H

1987-08-01

Requirements specification is a key element in a DSS development project because it not only determines what is to be done, it also drives the evolution process. A procedure for requirements elicitation is described that is based on the decomposition of the DSS design task into a number of functions, subfunctions, and operators. It is postulated that the procedure facilitates the building of a DSS that is complete and integrates MIS, modelling and expert system components. Some examples given are drawn from the health administration field.

Experiences of Structured Elicitation for Model-Based Cost-Effectiveness Analyses.

PubMed

Soares, Marta O; Sharples, Linda; Morton, Alec; Claxton, Karl; Bojke, Laura

2018-06-01

Empirical evidence supporting the cost-effectiveness estimates of particular health care technologies may be limited, or it may even be missing entirely. In these situations, additional information, often in the form of expert judgments, is needed to reach a decision. There are formal methods to quantify experts' beliefs, termed as structured expert elicitation (SEE), but only limited research is available in support of methodological choices. Perhaps as a consequence, the use of SEE in the context of cost-effectiveness modelling is limited. This article reviews applications of SEE in cost-effectiveness modelling with the aim of summarizing the basis for methodological choices made in each application and recording the difficulties and challenges reported by the authors in the design, conduct, and analyses. The methods used in each application were extracted along with the criteria used to support methodological and practical choices and any issues or challenges discussed in the text. Issues and challenges were extracted using an open field, and then categorised and grouped for reporting. The review demonstrates considerable heterogeneity in methods used, and authors acknowledge great methodological uncertainty in justifying their choices. Specificities of the context area emerging as potentially important in determining further methodological research in elicitation are between- expert variation and its interpretation, the fact that substantive experts in the area may not be trained in quantitative subjects, that judgments are often needed on various parameter types, the need for some form of assessment of validity, and the need for more integration with behavioural research to devise relevant debiasing strategies. This review of experiences of SEE highlights a number of specificities/constraints that can shape the development of guidance and target future research efforts in this area. Copyright © 2018 International Society for Pharmacoeconomics and Outcomes Research (ISPOR). Published by Elsevier Inc. All rights reserved.

Integrating Science and Management to Assess Forest Ecosystem Vulnerability to Climate Change

Treesearch

Leslie A. Brandt; Patricia R. Butler; Stephen D. Handler; Maria K. Janowiak; P. Danielle Shannon; Christopher W. Swanston

2017-01-01

We developed the ecosystem vulnerability assessment approach (EVAA) to help inform potential adaptation actions in response to a changing climate. EVAA combines multiple quantitative models and expert elicitation from scientists and land managers. In each of eight assessment areas, a panel of local experts determined potential vulnerability of forest ecosystems to...

76 FR 55673 - Vulnerability Assessments in Support of the Climate Ready Estuaries Program: A Novel Approach...

Federal Register 2010, 2011, 2012, 2013, 2014

2011-09-08

... exercise was designed to elicit judgments from experts in a workshop setting, regarding climate change... influence under both current and future climate change scenarios. The experts also discussed the high impact... in the diagram, and the potential for threshold changes. These reports show how climate-sensitive...

Information/Knowledge Acquisition Methods for Decision Support Systems and Expert Systems.

ERIC Educational Resources Information Center

Yang, Heng-Li

1995-01-01

Compares information requirement-elicitation (IRE) methods for decision support systems (DSS) with knowledge acquisition (KA) methods for expert systems (ES) development. The definition and architectures of ES and DSS are compared and the systems' development cycles and IRE/KA methods are discussed. Differences are noted between ES and DSS…

We’re only in it for the knowledge? A problem solving turn in environment and health expert elicitation

PubMed Central

2012-01-01

Background The FP6 EU HENVINET project aimed at synthesizing the scientific information available on a number of topics of high relevance to policy makers in environment and health. The goal of the current paper is to reflect on the methodology that was used in the project, in view of exploring the usefulness of this and similar methodologies to the policy process. The topics investigated included health impacts of the brominated flame retardants decabrominated diphenylether (decaBDE) and hexabromocyclododecane (HBCD), phthalates highlighting di(2-ethylhexyl)phthalate (DEHP), the pesticide chlorpyrifos (CPF), nanoparticles, the impacts of climate change on asthma and other respiratory disorders, and the influence of environment health stressors on cancer induction. Methods Initially the focus was on identifying knowledge gaps in the state of the art in scientific knowledge. Literature reviews covered all elements that compose the causal chain of the different environmental health issues from emissions to exposures, to effects and to health impacts. Through expert elicitation, knowledge gaps were highlighted by assessing expert confidence using calibrated confidence scales. During this work a complementary focus to that on knowledge gaps was developed through interdisciplinary reflections. By extending the scope of the endeavour from only a scientific perspective, to also include the more problem solving oriented policy perspective, the question of which kind of policy action experts consider justifiable was addressed. This was addressed by means of a questionnaire. In an expert workshop the results of both questionnaires were discussed as a basis for policy briefs. Results The expert elicitation, the application of the calibrated confidence levels and the problem solving approach were all experienced as being quite challenging for the experts involved, as these approaches did not easily relate to mainstream environment and health scientific practices. Even so, most experts were quite positive about it. In particular, the opportunity to widen one’s own horizon and to interactively exchange knowledge and debate with a diversity of experts seemed to be well appreciated in this approach. Different parts of the approach also helped in focussing on specific relevant aspects of scientific knowledge, and as such can be considered of reflective value. Conclusions The approach developed by HENVINET was part of a practice of learning by doing and of interdisciplinary cooperation and negotiation. Ambitions were challenged by unforeseen complexities and difference of opinion and as no Holy Grail approach was at hand to copy or follow, it was quite an interesting but also complicated endeavour. Perfection, if this could be defined, seemed out of reach all the time. Nevertheless, many involved were quite positive about it. It seems that many felt that it fitted some important needs in current science when addressing the needs of policy making on such important issues, without anyone really having a clue on how to actually do this. Challenging questions remain on the quality of such approach and its product. Practice tells us that there probably is no best method and that the best we can do is dependent on contextual negotiation and learning from experiences that we think are relevant. PMID:22759503

The use of concept maps during knowledge elicitation in ontology development processes – the nutrigenomics use case

PubMed Central

Castro, Alexander Garcia; Rocca-Serra, Philippe; Stevens, Robert; Taylor, Chris; Nashar, Karim; Ragan, Mark A; Sansone, Susanna-Assunta

2006-01-01

Background Incorporation of ontologies into annotations has enabled 'semantic integration' of complex data, making explicit the knowledge within a certain field. One of the major bottlenecks in developing bio-ontologies is the lack of a unified methodology. Different methodologies have been proposed for different scenarios, but there is no agreed-upon standard methodology for building ontologies. The involvement of geographically distributed domain experts, the need for domain experts to lead the design process, the application of the ontologies and the life cycles of bio-ontologies are amongst the features not considered by previously proposed methodologies. Results Here, we present a methodology for developing ontologies within the biological domain. We describe our scenario, competency questions, results and milestones for each methodological stage. We introduce the use of concept maps during knowledge acquisition phases as a feasible transition between domain expert and knowledge engineer. Conclusion The contributions of this paper are the thorough description of the steps we suggest when building an ontology, example use of concept maps, consideration of applicability to the development of lower-level ontologies and application to decentralised environments. We have found that within our scenario conceptual maps played an important role in the development process. PMID:16725019

Aviation Safety Risk Modeling: Lessons Learned From Multiple Knowledge Elicitation Sessions

NASA Technical Reports Server (NTRS)

Luxhoj, J. T.; Ancel, E.; Green, L. L.; Shih, A. T.; Jones, S. M.; Reveley, M. S.

2014-01-01

Aviation safety risk modeling has elements of both art and science. In a complex domain, such as the National Airspace System (NAS), it is essential that knowledge elicitation (KE) sessions with domain experts be performed to facilitate the making of plausible inferences about the possible impacts of future technologies and procedures. This study discusses lessons learned throughout the multiple KE sessions held with domain experts to construct probabilistic safety risk models for a Loss of Control Accident Framework (LOCAF), FLightdeck Automation Problems (FLAP), and Runway Incursion (RI) mishap scenarios. The intent of these safety risk models is to support a portfolio analysis of NASA's Aviation Safety Program (AvSP). These models use the flexible, probabilistic approach of Bayesian Belief Networks (BBNs) and influence diagrams to model the complex interactions of aviation system risk factors. Each KE session had a different set of experts with diverse expertise, such as pilot, air traffic controller, certification, and/or human factors knowledge that was elicited to construct a composite, systems-level risk model. There were numerous "lessons learned" from these KE sessions that deal with behavioral aggregation, conditional probability modeling, object-oriented construction, interpretation of the safety risk results, and model verification/validation that are presented in this paper.

Attribution of global foodborne disease to specific foods: Findings from a World Health Organization structured expert elicitation

PubMed Central

Devleesschauwer, Brecht; Aspinall, Willy; Cooke, Roger; Corrigan, Tim; Havelaar, Arie; Angulo, Frederick; Gibb, Herman; Kirk, Martyn; Lake, Robin; Speybroeck, Niko; Torgerson, Paul; Hald, Tine

2017-01-01

Background Recently the World Health Organization, Foodborne Disease Burden Epidemiology Reference Group (FERG) estimated that 31 foodborne diseases (FBDs) resulted in over 600 million illnesses and 420,000 deaths worldwide in 2010. Knowing the relative role importance of different foods as exposure routes for key hazards is critical to preventing illness. This study reports the findings of a structured expert elicitation providing globally comparable food source attribution estimates for 11 major FBDs in each of 14 world subregions. Methods and findings We used Cooke’s Classical Model to elicit and aggregate judgments of 73 international experts. Judgments were elicited from each expert individually and aggregated using both equal and performance weights. Performance weighted results are reported as they increased the informativeness of estimates, while retaining accuracy. We report measures of central tendency and uncertainty bounds on food source attribution estimate. For some pathogens we see relatively consistent food source attribution estimates across subregions of the world; for others there is substantial regional variation. For example, for non-typhoidal salmonellosis, pork was of minor importance compared to eggs and poultry meat in the American and African subregions, whereas in the European and Western Pacific subregions the importance of these three food sources were quite similar. Our regional results broadly agree with estimates from earlier European and North American food source attribution research. As in prior food source attribution research, we find relatively wide uncertainty bounds around our median estimates. Conclusions We present the first worldwide estimates of the proportion of specific foodborne diseases attributable to specific food exposure routes. While we find substantial uncertainty around central tendency estimates, we believe these estimates provide the best currently available basis on which to link FBDs and specific foods in many parts of the world, providing guidance for policy actions to control FBDs. PMID:28910293

Expert elicitation for a national-level volcano hazard model

NASA Astrophysics Data System (ADS)

Bebbington, Mark; Stirling, Mark; Cronin, Shane; Wang, Ting; Jolly, Gill

2016-04-01

The quantification of volcanic hazard at national level is a vital pre-requisite to placing volcanic risk on a platform that permits meaningful comparison with other hazards such as earthquakes. New Zealand has up to a dozen dangerous volcanoes, with the usual mixed degrees of knowledge concerning their temporal and spatial eruptive history. Information on the 'size' of the eruptions, be it in terms of VEI, volume or duration, is sketchy at best. These limitations and the need for a uniform approach lend themselves to a subjective hazard analysis via expert elicitation. Approximately 20 New Zealand volcanologists provided estimates for the size of the next eruption from each volcano and, conditional on this, its location, timing and duration. Opinions were likewise elicited from a control group of statisticians, seismologists and (geo)chemists, all of whom had at least heard the term 'volcano'. The opinions were combined via the Cooke classical method. We will report on the preliminary results from the exercise.

Vulnerability of bridges to scour: insights from an international expert elicitation workshop

NASA Astrophysics Data System (ADS)

Lamb, Rob; Aspinall, Willy; Odbert, Henry; Wagener, Thorsten

2017-08-01

Scour (localised erosion) during flood events is one of the most significant threats to bridges over rivers and estuaries, and has been the cause of numerous bridge failures, with damaging consequences. Mitigation of the risk of bridges being damaged by scour is therefore important to many infrastructure owners, and is supported by industry guidance. Even after mitigation, some residual risk remains, though its extent is difficult to quantify because of the uncertainties inherent in the prediction of scour and the assessment of the scour risk. This paper summarises findings from an international expert workshop on bridge scour risk assessment that explores uncertainties about the vulnerability of bridges to scour. Two specialised structured elicitation methods were applied to explore the factors that experts in the field consider important when assessing scour risk and to derive pooled expert judgements of bridge failure probabilities that are conditional on a range of assumed scenarios describing flood event severity, bridge and watercourse types and risk mitigation protocols. The experts' judgements broadly align with industry good practice, but indicate significant uncertainty about quantitative estimates of bridge failure probabilities, reflecting the difficulty in assessing the residual risk of failure. The data and findings presented here could provide a useful context for the development of generic scour fragility models and their associated uncertainties.

An Expert Elicitation of the Proliferation Resistance of Using Small Modular Reactors (SMR) for the Expansion of Civilian Nuclear Systems.

PubMed

Siegel, Jonas; Gilmore, Elisabeth A; Gallagher, Nancy; Fetter, Steve

2018-02-01

To facilitate the use of nuclear energy globally, small modular reactors (SMRs) may represent a viable alternative or complement to large reactor designs. One potential benefit is that SMRs could allow for more proliferation resistant designs, manufacturing arrangements, and fuel-cycle practices at widespread deployment. However, there is limited work evaluating the proliferation resistance of SMRs, and existing proliferation assessment approaches are not well suited for these novel arrangements. Here, we conduct an expert elicitation of the relative proliferation resistance of scenarios for future nuclear energy deployment driven by Generation III+ light-water reactors, fast reactors, or SMRs. Specifically, we construct the scenarios to investigate relevant technical and institutional features that are postulated to enhance the proliferation resistance of SMRs. The experts do not consistently judge the scenario with SMRs to have greater overall proliferation resistance than scenarios that rely on conventional nuclear energy generation options. Further, the experts disagreed on whether incorporating a long-lifetime sealed core into an SMR design would strengthen or weaken proliferation resistance. However, regardless of the type of reactor, the experts judged that proliferation resistance would be enhanced by improving international safeguards and operating several multinational fuel-cycle facilities rather than supporting many more national facilities. © 2017 Society for Risk Analysis.

Expert surgical consensus for prenatal counseling using the Delphi method.

PubMed

Berman, Loren; Jackson, Jordan; Miller, Kristen; Kowalski, Rebecca; Kolm, Paul; Luks, Francois I

2017-11-28

Pediatric surgeons frequently offer prenatal consultation for congenital pulmonary airway malformation (CPAM) and congenital diaphragmatic hernia (CDH); however, there is no evidence-based consensus to guide prenatal decision making and counseling for these conditions. Eliciting feedback from experts is integral to defining best practice regarding prenatal counseling and intervention. A Delphi consensus process was undertaken using a panel of pediatric surgeons identified as experts in fetal therapy to address current limitations. Areas of discrepancy in the literature on CPAM and CDH were identified and used to generate a list of content and intervention questions. Experts were invited to participate in an online Delphi survey. Items that did not reach first-round consensus were broken down into additional questions, and consensus was achieved in the second round. Fifty-four surgeons (69%) responded to at least one of the two survey rounds. During round one, consensus was reached on 54 of 89 survey questions (61%), and 45 new questions were developed. During round two, consensus was reached on 53 of 60 survey questions (88%). We determined expert consensus to establish guidelines regarding perinatal management of CPAM and CDH. Our results can help educate pediatric surgeons participating in perinatal care of these patients. V. Copyright © 2017 Elsevier Inc. All rights reserved.

A Bayesian approach for calibrating probability judgments

NASA Astrophysics Data System (ADS)

Firmino, Paulo Renato A.; Santana, Nielson A.

2012-10-01

Eliciting experts' opinions has been one of the main alternatives for addressing paucity of data. In the vanguard of this area is the development of calibration models (CMs). CMs are models dedicated to overcome miscalibration, i.e. judgment biases reflecting deficient strategies of reasoning adopted by the expert when inferring about an unknown. One of the main challenges of CMs is to determine how and when to intervene against miscalibration, in order to enhance the tradeoff between costs (time spent with calibration processes) and accuracy of the resulting models. The current paper dedicates special attention to this issue by presenting a dynamic Bayesian framework for monitoring, diagnosing, and handling miscalibration patterns. The framework is based on Beta-, Uniform, or Triangular-Bernoulli models and classes of judgmental calibration theories. Issues regarding the usefulness of the proposed framework are discussed and illustrated via simulation studies.

Teaching Evaluations and Comments of Pre-Service Music Teachers regarding Expert and Novice Choral Conductors

ERIC Educational Resources Information Center

Johnson, Christopher M.; Price, Harry E.; Schroeder, Linda K.

2009-01-01

The purpose of the present study was to determine if pre-service music educators could discriminate between novice and expert choral directors irrespective of the proficiency of the choral ensemble. A secondary purpose was to compare study results elicited in the USA with those from participants in Europe, South America and Asia. Previous research…

The Sport Expert's Attention Superiority on Skill-related Scene Dynamic by the Activation of left Medial Frontal Gyrus: An ERP and LORETA Study.

PubMed

He, Mengyang; Qi, Changzhu; Lu, Yang; Song, Amanda; Hayat, Saba Z; Xu, Xia

2018-05-21

Extensive studies have shown that a sports expert is superior to a sports novice in visually perceptual-cognitive processes of sports scene information, however the attentional and neural basis of it has not been thoroughly explored. The present study examined whether a sport expert has the attentional superiority on scene information relevant to his/her sport skill, and explored what factor drives this superiority. To address this problem, EEGs were recorded as participants passively viewed sport scenes (tennis vs. non-tennis) and negative emotional faces in the context of a visual attention task, where the pictures of sport scenes or of negative emotional faces randomly followed the pictures with overlapping sport scenes and negative emotional faces. ERP results showed that for experts, the evoked potential of attentional competition elicited by the overlap of tennis scene was significantly larger than that evoked by the overlap of non-tennis scene, while this effect was absent for novices. The LORETA showed that the experts' left medial frontal gyrus (MFG) cortex was significantly more active as compared to the right MFG when processing the overlap of tennis scene, but the lateralization effect was not significant in novices. Those results indicate that experts have attentional superiority on skill-related scene information, despite intruding the scene through negative emotional faces that are prone to cause negativity bias toward their visual field as a strong distractor. This superiority is actuated by the activation of left MFG cortex and probably due to self-reference. Copyright © 2018 The Authors. Published by Elsevier Ltd.. All rights reserved.

Expert stakeholder attitudes and support for alternative water sources in a groundwater depleted region.

PubMed

Boyer, Treavor H; Overdevest, Christine; Christiansen, Lisa; Ishii, Stephanie K L

2012-10-15

The main objectives of this research were to quantify the risks/benefits and impacts of alternative water sources (AWSs) as perceived by expert stakeholders and to evaluate the overall support for multiple AWSs by expert stakeholders. The St. Johns River (SJR) basin, FL, USA was chosen as a case study for AWSs because it is a fresh groundwater depleted region and there are ongoing activities related to water supply planning. Expert stakeholders included federal, state, and local governments, public utilities, consulting engineering and industry, and environmental and social non-governmental organizations. AWSs under consideration in the SJR basin include surface water, desalination, water reclamation, and water conservation. A two-phase research approach was followed that focused on expert stakeholders. First, an elicitation study was used to identify salient beliefs about AWSs. Open-ended questions were asked about the risks/benefits of AWSs in terms of the three pillars of sustainability: ecological, economic, and human health impacts. Second, an online survey was constructed using beliefs identified during the elicitation study. The online survey was used to quantify attitudes toward and overall support for AWSs. The salient beliefs of expert stakeholders were dominated by the ecological pillar of sustainability. The support of expert stakeholders for AWSs, from least favorable to most favorable, was surface water withdrawals

Interpretation of the margin of exposure for genotoxic carcinogens - elicitation of expert knowledge about the form of the dose response curve at human relevant exposures.

PubMed

Boobis, Alan; Flari, Villie; Gosling, John Paul; Hart, Andy; Craig, Peter; Rushton, Lesley; Idahosa-Taylor, Ehi

2013-07-01

The general approach to risk assessment of genotoxic carcinogens has been to advise reduction of exposure to "as low as reasonably achievable/practicable" (ALARA/P). However, whilst this remains the preferred risk management option, it does not provide guidance on the urgency or extent of risk management actions necessary. To address this, the "Margin of Exposure" (MOE) approach has been proposed. The MOE is the ratio between the point of departure for carcinogenesis and estimated human exposure. However, interpretation of the MOE requires implicit or explicit consideration of the shape of the dose-response curve at human relevant exposures. In a structured elicitation exercise, we captured expert opinion on available scientific evidence for low dose-response relationships for genotoxic carcinogens. This allowed assessment of: available evidence for the nature of dose-response relationships at human relevant exposures; the generality of judgments about such dose-response relationships; uncertainties affecting judgments on the nature of such dose-response relationships; and whether this last should differ for different classes of genotoxic carcinogens. Elicitation results reflected the variability in experts' views on the form of the dose-response curve for low dose exposure and major sources of uncertainty affecting the assumption of a linear relationship. Copyright © 2013 Elsevier Ltd. All rights reserved.

Looking back to inform the future: The role of cognition in forest disturbance characterization from remote sensing imagery

NASA Astrophysics Data System (ADS)

Bianchetti, Raechel Anne

Remotely sensed images have become a ubiquitous part of our daily lives. From novice users, aiding in search and rescue missions using tools such as TomNod, to trained analysts, synthesizing disparate data to address complex problems like climate change, imagery has become central to geospatial problem solving. Expert image analysts are continually faced with rapidly developing sensor technologies and software systems. In response to these cognitively demanding environments, expert analysts develop specialized knowledge and analytic skills to address increasingly complex problems. This study identifies the knowledge, skills, and analytic goals of expert image analysts tasked with identification of land cover and land use change. Analysts participating in this research are currently working as part of a national level analysis of land use change, and are well versed with the use of TimeSync, forest science, and image analysis. The results of this study benefit current analysts as it improves their awareness of their mental processes used during the image interpretation process. The study also can be generalized to understand the types of knowledge and visual cues that analysts use when reasoning with imagery for purposes beyond land use change studies. Here a Cognitive Task Analysis framework is used to organize evidence from qualitative knowledge elicitation methods for characterizing the cognitive aspects of the TimeSync image analysis process. Using a combination of content analysis, diagramming, semi-structured interviews, and observation, the study highlights the perceptual and cognitive elements of expert remote sensing interpretation. Results show that image analysts perform several standard cognitive processes, but flexibly employ these processes in response to various contextual cues. Expert image analysts' ability to think flexibly during their analysis process was directly related to their amount of image analysis experience. Additionally, results show that the basic Image Interpretation Elements continue to be important despite technological augmentation of the interpretation process. These results are used to derive a set of design guidelines for developing geovisual analytic tools and training to support image analysis.

Estimating distribution and connectivity of recolonizing American marten in the northeastern United States using expert elicitation techniques

USGS Publications Warehouse

Aylward, C.M.; Murdoch, J.D.; Donovan, Therese M.; Kilpatrick, C.W.; Bernier, C.; Katz, J.

2018-01-01

The American marten Martes americana is a species of conservation concern in the northeastern United States due to widespread declines from over‐harvesting and habitat loss. Little information exists on current marten distribution and how landscape characteristics shape patterns of occupancy across the region, which could help develop effective recovery strategies. The rarity of marten and lack of historical distribution records are also problematic for region‐wide conservation planning. Expert opinion can provide a source of information for estimating species–landscape relationships and is especially useful when empirical data are sparse. We created a survey to elicit expert opinion and build a model that describes marten occupancy in the northeastern United States as a function of landscape conditions. We elicited opinions from 18 marten experts that included wildlife managers, trappers and researchers. Each expert estimated occupancy probability at 30 sites in their geographic region of expertise. We, then, fit the response data with a set of 58 models that incorporated the effects of covariates related to forest characteristics, climate, anthropogenic impacts and competition at two spatial scales (1.5 and 5 km radii), and used model selection techniques to determine the best model in the set. Three top models had strong empirical support, which we model averaged based on AIC weights. The final model included effects of five covariates at the 5‐km scale: percent canopy cover (positive), percent spruce‐fir land cover (positive), winter temperature (negative), elevation (positive) and road density (negative). A receiver operating characteristic curve indicated that the model performed well based on recent occurrence records. We mapped distribution across the region and used circuit theory to estimate movement corridors between isolated core populations. The results demonstrate the effectiveness of expert‐opinion data at modeling occupancy for rare species and provide tools for planning marten recovery in the northeastern United States.

Expert Elicitations of 2100 Emission of CO2

NASA Astrophysics Data System (ADS)

Ho, Emily; Bosetti, Valentina; Budescu, David; Keller, Klaus; van Vuuren, Detlef

2017-04-01

Emission scenarios such as Shared Socioeconomic Pathways (SSPs) and Representative Concentration Pathways (RCPs) are used intensively for climate research (e.g. climate change projections) and policy analysis. While the range of these scenarios provides an indication of uncertainty, these scenarios are typically not associated with probability values. Some studies (e.g. Vuuren et al, 2007; Gillingham et al., 2015) took a different approach associating baseline emission pathways (conditionally) with probability distributions. This paper summarizes three studies where climate change experts were asked to conduct pair-wise comparisons of possible ranges of 2100 greenhouse gas emissions and rate the relative likelihood of the ranges. The elicitation was performed under two sets of assumptions: 1) a situation where no climate policies are introduced beyond the ones already in place (baseline scenario), and 2) a situation in which countries have ratified the voluntary policies in line with the long term target embedded in the 2015 Paris Agreement. These indirect relative judgments were used to construct subjective cumulative distribution functions. We show that by using a ratio scaling method that invokes relative likelihoods of scenarios, a subjective probability distribution can be derived for each expert that expresses their beliefs in the projected greenhouse gas emissions range in 2100. This method is shown to elicit stable estimates that require minimal adjustment and is relatively invariant to the partition of the domain of interest. Experts also rated the method as being easy and intuitive to use. We also report results of a study that allowed participants to choose their own ranges of greenhouse gas emissions to remove potential anchoring bias. We discuss the implications of the use of this method for facilitating comparison and communication of beliefs among diverse users of climate science research.

Inverting faces elicits sensitivity to race on the N170 component: a cross-cultural study.

PubMed

Vizioli, Luca; Foreman, Kay; Rousselet, Guillaume A; Caldara, Roberto

2010-01-29

Human beings are natural experts at processing faces, with some notable exceptions. Same-race faces are better recognized than other-race faces: the so-called other-race effect (ORE). Inverting faces impairs recognition more than for any other inverted visual object: the so-called face inversion effect (FIE). Interestingly, the FIE is stronger for same- compared to other-race faces. At the electrophysiological level, inverted faces elicit consistently delayed and often larger N170 compared to upright faces. However, whether the N170 component is sensitive to race is still a matter of ongoing debate. Here we investigated the N170 sensitivity to race in the framework of the FIE. We recorded EEG from Western Caucasian and East Asian observers while presented with Western Caucasian, East Asian and African American faces in upright and inverted orientations. To control for potential confounds in the EEG signal that might be evoked by the intrinsic and salient differences in the low-level properties of faces from different races, we normalized their amplitude-spectra, luminance and contrast. No differences on the N170 were observed for upright faces. Critically, inverted same-race faces lead to greater recognition impairment and elicited larger N170 amplitudes compared to inverted other-race faces. Our results indicate a finer-grained neural tuning for same-race faces at early stages of processing in both groups of observers.

Design Features in Games for Health: Disciplinary and Interdisciplinary Expert Perspectives.

PubMed

Kelley, Christina; Wilcox, Lauren; Ng, Wendy; Schiffer, Jade; Hammer, Jessica

2017-06-01

Games for health (G4H) aim to improve health outcomes and encourage behavior change. While existing theoretical frameworks describe features of both games and health interventions, there has been limited systematic investigation into how disciplinary and interdisciplinary stakeholders understand design features in G4H. We recruited 18 experts from the fields of game design, behavioral health, and games for health, and prompted them with 16 sample games. Applying methods including open card sorting and triading, we elicited themes and features (e.g., real-world interaction, game mechanics) around G4H. We found evidence of conceptual differences suggesting that a G4H perspective is not simply the sum of game and health perspectives. At the same time, we found evidence of convergence in stakeholder views, including areas where game experts provided insights about health and vice versa. We discuss how this work can be applied to provide conceptual tools, improve the G4H design process, and guide approaches to encoding G4H-related data for large-scale empirical analysis.

Design Features in Games for Health: Disciplinary and Interdisciplinary Expert Perspectives

PubMed Central

Kelley, Christina; Wilcox, Lauren; Ng, Wendy; Schiffer, Jade; Hammer, Jessica

2017-01-01

Games for health (G4H) aim to improve health outcomes and encourage behavior change. While existing theoretical frameworks describe features of both games and health interventions, there has been limited systematic investigation into how disciplinary and interdisciplinary stakeholders understand design features in G4H. We recruited 18 experts from the fields of game design, behavioral health, and games for health, and prompted them with 16 sample games. Applying methods including open card sorting and triading, we elicited themes and features (e.g., real-world interaction, game mechanics) around G4H. We found evidence of conceptual differences suggesting that a G4H perspective is not simply the sum of game and health perspectives. At the same time, we found evidence of convergence in stakeholder views, including areas where game experts provided insights about health and vice versa. We discuss how this work can be applied to provide conceptual tools, improve the G4H design process, and guide approaches to encoding G4H–related data for large-scale empirical analysis. PMID:28868523

Probabilistic seismic hazard analyses for ground motions and fault displacement at Yucca Mountain, Nevada

USGS Publications Warehouse

Stepp, J.C.; Wong, I.; Whitney, J.; Quittmeyer, R.; Abrahamson, N.; Toro, G.; Young, S.R.; Coppersmith, K.; Savy, J.; Sullivan, T.

2001-01-01

Probabilistic seismic hazard analyses were conducted to estimate both ground motion and fault displacement hazards at the potential geologic repository for spent nuclear fuel and high-level radioactive waste at Yucca Mountain, Nevada. The study is believed to be the largest and most comprehensive analyses ever conducted for ground-shaking hazard and is a first-of-a-kind assessment of probabilistic fault displacement hazard. The major emphasis of the study was on the quantification of epistemic uncertainty. Six teams of three experts performed seismic source and fault displacement evaluations, and seven individual experts provided ground motion evaluations. State-of-the-practice expert elicitation processes involving structured workshops, consensus identification of parameters and issues to be evaluated, common sharing of data and information, and open exchanges about the basis for preliminary interpretations were implemented. Ground-shaking hazard was computed for a hypothetical rock outcrop at -300 m, the depth of the potential waste emplacement drifts, at the designated design annual exceedance probabilities of 10-3 and 10-4. The fault displacement hazard was calculated at the design annual exceedance probabilities of 10-4 and 10-5.

Expert system training and control based on the fuzzy relation matrix

NASA Technical Reports Server (NTRS)

Ren, Jie; Sheridan, T. B.

1991-01-01

Fuzzy knowledge, that for which the terms of reference are not crisp but overlapped, seems to characterize human expertise. This can be shown from the fact that an experienced human operator can control some complex plants better than a computer can. Proposed here is fuzzy theory to build a fuzzy expert relation matrix (FERM) from given rules or/and examples, either in linguistic terms or in numerical values to mimic human processes of perception and decision making. The knowledge base is codified in terms of many implicit fuzzy rules. Fuzzy knowledge thus codified may also be compared with explicit rules specified by a human expert. It can also provide a basis for modeling the human operator and allow comparison of what a human operator says to what he does in practice. Two experiments were performed. In the first, control of liquid in a tank, demonstrates how the FERM knowledge base is elicited and trained. The other shows how to use a FERM, build up from linguistic rules, and to control an inverted pendulum without a dynamic model.

Factors influencing analysis of complex cognitive tasks: a framework and example from industrial process control.

PubMed

Prietula, M J; Feltovich, P J; Marchak, F

2000-01-01

We propose that considering four categories of task factors can facilitate knowledge elicitation efforts in the analysis of complex cognitive tasks: materials, strategies, knowledge characteristics, and goals. A study was conducted to examine the effects of altering aspects of two of these task categories on problem-solving behavior across skill levels: materials and goals. Two versions of an applied engineering problem were presented to expert, intermediate, and novice participants. Participants were to minimize the cost of running a steam generation facility by adjusting steam generation levels and flows. One version was cast in the form of a dynamic, computer-based simulation that provided immediate feedback on flows, costs, and constraint violations, thus incorporating key variable dynamics of the problem context. The other version was cast as a static computer-based model, with no dynamic components, cost feedback, or constraint checking. Experts performed better than the other groups across material conditions, and, when required, the presentation of the goal assisted the experts more than the other groups. The static group generated richer protocols than the dynamic group, but the dynamic group solved the problem in significantly less time. Little effect of feedback was found for intermediates, and none for novices. We conclude that demonstrating differences in performance in this task requires different materials than explicating underlying knowledge that leads to performance. We also conclude that substantial knowledge is required to exploit the information yielded by the dynamic form of the task or the explicit solution goal. This simple model can help to identify the contextual factors that influence elicitation and specification of knowledge, which is essential in the engineering of joint cognitive systems.

Min and max are the only continuous ampersand-, V-operations for finite logics

NASA Technical Reports Server (NTRS)

Kreinovich, Vladik

1992-01-01

Experts usually express their degrees of belief in their statements by the words of a natural language (like 'maybe', 'perhaps', etc.). If an expert system contains the degrees of beliefs t(A) and t(B) that correspond to the statements A and B, and a user asks this expert system whether 'A&B' is true, then it is necessary to come up with a reasonable estimate for the degree of belief of A&B. The operation that processes t(A) and t(B) into such an estimate t(A&B) is called an &-operation. Many different &-operations have been proposed. Which of them to choose? This can be (in principle) done by interviewing experts and eliciting a &-operation from them, but such a process is very time-consuming and therefore, not always possible. So, usually, to choose a &-operation, the finite set of actually possible degrees of belief is extended to an infinite set (e.g., to an interval (0,1)), define an operation there, and then restrict this operation to the finite set. Only this original finite set is considered. It is shown that a reasonable assumption that an &-operation is continuous (i.e., that gradual change in t(A) and t(B) must lead to a gradual change in t(A&B)), uniquely determines min as an &-operation. Likewise, max is the only continuous V-operation. These results are in good accordance with the experimental analysis of 'and' and 'or' in human beliefs.

World Health Organization Estimates of the Relative Contributions of Food to the Burden of Disease Due to Selected Foodborne Hazards: A Structured Expert Elicitation

PubMed Central

Hald, Tine; Aspinall, Willy; Devleesschauwer, Brecht; Cooke, Roger; Corrigan, Tim; Havelaar, Arie H.; Gibb, Herman J.; Torgerson, Paul R.; Kirk, Martyn D.; Angulo, Fred J.; Lake, Robin J.; Speybroeck, Niko; Hoffmann, Sandra

2016-01-01

Background The Foodborne Disease Burden Epidemiology Reference Group (FERG) was established in 2007 by the World Health Organization (WHO) to estimate the global burden of foodborne diseases (FBDs). This estimation is complicated because most of the hazards causing FBD are not transmitted solely by food; most have several potential exposure routes consisting of transmission from animals, by humans, and via environmental routes including water. This paper describes an expert elicitation study conducted by the FERG Source Attribution Task Force to estimate the relative contribution of food to the global burden of diseases commonly transmitted through the consumption of food. Methods and Findings We applied structured expert judgment using Cooke’s Classical Model to obtain estimates for 14 subregions for the relative contributions of different transmission pathways for eleven diarrheal diseases, seven other infectious diseases and one chemical (lead). Experts were identified through international networks followed by social network sampling. Final selection of experts was based on their experience including international working experience. Enrolled experts were scored on their ability to judge uncertainty accurately and informatively using a series of subject-matter specific ‘seed’ questions whose answers are unknown to the experts at the time they are interviewed. Trained facilitators elicited the 5th, and 50th and 95th percentile responses to seed questions through telephone interviews. Cooke’s Classical Model uses responses to the seed questions to weigh and aggregate expert responses. After this interview, the experts were asked to provide 5th, 50th, and 95th percentile estimates for the ‘target’ questions regarding disease transmission routes. A total of 72 experts were enrolled in the study. Ten panels were global, meaning that the experts should provide estimates for all 14 subregions, whereas the nine panels were subregional, with experts providing estimates for one or more subregions, depending on their experience in the region. The size of the 19 hazard-specific panels ranged from 6 to 15 persons with several experts serving on more than one panel. Pathogens with animal reservoirs (e.g. non-typhoidal Salmonella spp. and Toxoplasma gondii) were in general assessed by the experts to have a higher proportion of illnesses attributable to food than pathogens with mainly a human reservoir, where human-to-human transmission (e.g. Shigella spp. and Norovirus) or waterborne transmission (e.g. Salmonella Typhi and Vibrio cholerae) were judged to dominate. For many pathogens, the foodborne route was assessed relatively more important in developed subregions than in developing subregions. The main exposure routes for lead varied across subregions, with the foodborne route being assessed most important only in two subregions of the European region. Conclusions For the first time, we present worldwide estimates of the proportion of specific diseases attributable to food and other major transmission routes. These findings are essential for global burden of FBD estimates. While gaps exist, we believe the estimates presented here are the best current source of guidance to support decision makers when allocating resources for control and intervention, and for future research initiatives. PMID:26784029

World Health Organization Estimates of the Relative Contributions of Food to the Burden of Disease Due to Selected Foodborne Hazards: A Structured Expert Elicitation.

PubMed

Hald, Tine; Aspinall, Willy; Devleesschauwer, Brecht; Cooke, Roger; Corrigan, Tim; Havelaar, Arie H; Gibb, Herman J; Torgerson, Paul R; Kirk, Martyn D; Angulo, Fred J; Lake, Robin J; Speybroeck, Niko; Hoffmann, Sandra

2016-01-01

The Foodborne Disease Burden Epidemiology Reference Group (FERG) was established in 2007 by the World Health Organization (WHO) to estimate the global burden of foodborne diseases (FBDs). This estimation is complicated because most of the hazards causing FBD are not transmitted solely by food; most have several potential exposure routes consisting of transmission from animals, by humans, and via environmental routes including water. This paper describes an expert elicitation study conducted by the FERG Source Attribution Task Force to estimate the relative contribution of food to the global burden of diseases commonly transmitted through the consumption of food. We applied structured expert judgment using Cooke's Classical Model to obtain estimates for 14 subregions for the relative contributions of different transmission pathways for eleven diarrheal diseases, seven other infectious diseases and one chemical (lead). Experts were identified through international networks followed by social network sampling. Final selection of experts was based on their experience including international working experience. Enrolled experts were scored on their ability to judge uncertainty accurately and informatively using a series of subject-matter specific 'seed' questions whose answers are unknown to the experts at the time they are interviewed. Trained facilitators elicited the 5th, and 50th and 95th percentile responses to seed questions through telephone interviews. Cooke's Classical Model uses responses to the seed questions to weigh and aggregate expert responses. After this interview, the experts were asked to provide 5th, 50th, and 95th percentile estimates for the 'target' questions regarding disease transmission routes. A total of 72 experts were enrolled in the study. Ten panels were global, meaning that the experts should provide estimates for all 14 subregions, whereas the nine panels were subregional, with experts providing estimates for one or more subregions, depending on their experience in the region. The size of the 19 hazard-specific panels ranged from 6 to 15 persons with several experts serving on more than one panel. Pathogens with animal reservoirs (e.g. non-typhoidal Salmonella spp. and Toxoplasma gondii) were in general assessed by the experts to have a higher proportion of illnesses attributable to food than pathogens with mainly a human reservoir, where human-to-human transmission (e.g. Shigella spp. and Norovirus) or waterborne transmission (e.g. Salmonella Typhi and Vibrio cholerae) were judged to dominate. For many pathogens, the foodborne route was assessed relatively more important in developed subregions than in developing subregions. The main exposure routes for lead varied across subregions, with the foodborne route being assessed most important only in two subregions of the European region. For the first time, we present worldwide estimates of the proportion of specific diseases attributable to food and other major transmission routes. These findings are essential for global burden of FBD estimates. While gaps exist, we believe the estimates presented here are the best current source of guidance to support decision makers when allocating resources for control and intervention, and for future research initiatives.

Thermal-Hydraulics Phenomena Important in Modeling and Simulation of Liquid-Fuel Molten Salt Reactors

DOE Office of Scientific and Technical Information (OSTI.GOV)

Bajorek, Stephen; Diamond, David J.

This paper discusses liquid-fuel molten salt reactors, how they will operate under normal, transient, and accident conditions, and the results of an expert elicitation to determine the corresponding thermalhydraulic phenomena important to understanding their behavior. Identifying these phenomena will enable the U.S. Nuclear Regulatory Commission (NRC) to develop or identify modeling functionalities and tools required to carry out confirmatory analyses that examine the validity and accuracy of an applicant’s calculations and help determine the margin of safety in plant design. NRC frequently does an expert elicitation using a Phenomena Identification and Ranking Table (PIRT) to identify and evaluate the statemore » of knowledge of important modeling phenomena. However, few details about the design of these reactors and the sequence of events during accidents are known, so the process used was considered a preliminary PIRT. A panel met to define phenomena that would need to be modeled and considered the impact/importance of each phenomenon with respect to specific figures-of-merit (FoMs) (e.g., salt temperature, velocity, and composition). Each FoM reflected a potential impact on radionuclide release or loss of a barrier to release. The panel considered what the path forward might be with respect to being able to model the phenomenon in a simulation code. Results are explained for both thermal and fast spectrum designs.« less

Neutronics Phenomena Important in Modeling and Simulation of Liquid-Fuel Molten Salt Reactors

DOE Office of Scientific and Technical Information (OSTI.GOV)

Diamond, David J.

This paper discusses liquid-fuel molten salt reactors, how they will operate under normal, transient, and accident conditions, and the results of an expert elicitation to determine the corresponding neutronic phenomena important to understanding their behavior. Identifying these phenomena will enable the U.S. Nuclear Regulatory Commission (NRC) to develop or identify modeling functionalities and tools required to carry out confirmatory analyses that examine the validity and accuracy of applicants’ calculations and help determine the margin of safety in plant design. NRC frequently does an expert elicitation using a Phenomena Identification and Ranking Table (PIRT) to identify and evaluate the state ofmore » knowledge of important modeling phenomena. However, few details about the design of these reactors and the sequence of events during accidents are known, so the process used was considered a preliminary PIRT. A panel met to define phenomena that would need to be modeled and considered the impact/importance of each phenomenon with respect to specific figures-of-merit (FoMs) (e.g., power distribution, fluence, kinetics parameters and reactivity). Each FoM reflected a potential impact on radionuclide release or loss of a barrier to release. The panel considered what the path forward might be with respect to being able to model the phenomenon in a simulation code. Results are explained for both thermal and fast spectrum designs.« less

Case Studies in Modelling, Control in Food Processes.

PubMed

Glassey, J; Barone, A; Montague, G A; Sabou, V

This chapter discusses the importance of modelling and control in increasing food process efficiency and ensuring product quality. Various approaches to both modelling and control in food processing are set in the context of the specific challenges in this industrial sector and latest developments in each area are discussed. Three industrial case studies are used to demonstrate the benefits of advanced measurement, modelling and control in food processes. The first case study illustrates the use of knowledge elicitation from expert operators in the process for the manufacture of potato chips (French fries) and the consequent improvements in process control to increase the consistency of the resulting product. The second case study highlights the economic benefits of tighter control of an important process parameter, moisture content, in potato crisp (chips) manufacture. The final case study describes the use of NIR spectroscopy in ensuring effective mixing of dry multicomponent mixtures and pastes. Practical implementation tips and infrastructure requirements are also discussed.

Institutional Memory Preservation at NASA Glenn Research Center

NASA Technical Reports Server (NTRS)

Coffey, J.; Moreman, Douglas; Dyer, J.; Hemminger, J. A.

1999-01-01

In this era of downsizing and deficit reduction, the preservation of institutional memory is a widespread concern for U.S. companies and governmental agencies. The National Aeronautical and Space Administration faces the pending retirement of many of the agency's long-term, senior engineers. NASA has a marvelous long-term history of success, but the agency faces a recurring problem caused by the loss of these engineers' unique knowledge and perspectives on NASA's role in aeronautics and space exploration. The current work describes a knowledge elicitation effort aimed at demonstrating the feasibility of preserving the more personal, heuristic knowledge accumulated over the years by NASA engineers, as contrasted with the "textbook" knowledge of launch vehicles. Work on this project was performed at NASA Glenn Research Center and elsewhere, and focused on launch vehicle systems integration. The initial effort was directed toward an historic view of the Centaur upper stage which is powered by two RL-10 engines. Various experts were consulted, employing a variety of knowledge elicitation techniques, regarding the Centaur and RL-10. Their knowledge is represented in searchable Web-based multimedia presentations. This paper discusses the various approaches to knowledge elicitation and knowledge representation employed, and assesses successes and challenges in trying to perform large-scale knowledge preservation of institutional memory. It is anticipated that strategies for knowledge elicitation and representation that have been developed in this grant will be utilized to elicit knowledge in a variety of domains including the complex heuristics that underly use of simulation software packages such as that being explored in the Expert System Architecture for Rocket Engine Numerical Simulators.

Cognitive task analysis for instruction in single-injection ultrasound guided-regional anesthesia

NASA Astrophysics Data System (ADS)

Gucev, Gligor V.

Cognitive task analysis (CTA) is methodology for eliciting knowledge from subject matter experts. CTA has been used to capture the cognitive processes, decision-making, and judgments that underlie expert behaviors. A review of the literature revealed that CTA has not yet been used to capture the knowledge required to perform ultrasound guided regional anesthesia (UGRA). The purpose of this study was to utilize CTA to extract knowledge from UGRA experts and to determine whether instruction based on CTA of UGRA will produce results superior to the results of traditional training. This study adds to the knowledge base of CTA in being the first one to effectively capture the expert knowledge of UGRA. The derived protocol was used in a randomized, double blinded experiment involving UGRA instruction to 39 novice learners. The results of this study strongly support the hypothesis that CTA-based instruction in UGRA is more effective than conventional clinical instruction, as measured by conceptual pre- and post-tests, performance of a simulated UGRA procedure, and time necessary for the task performance. This study adds to the number of studies that have proven the superiority of CTA-informed instruction. Finally, it produced several validated instruments that can be used in instructing and evaluating UGRA.

A theoretical framework for holistic hospital management in the Japanese healthcare context.

PubMed

Liu, Hu-Chen

2013-11-01

This paper develops a conceptual framework for performance measurement as a pilot study on holistic hospital management in the Japanese healthcare context. We primarily used two data sources as well as expert statements obtained through interviews: a systematic review of literature and a questionnaire survey to healthcare experts. The systematic survey searched PubMed and PubMed Central, and 24 relevant papers were elicited. The expert questionnaire asked respondents to rate the degree of "usefulness" for each of 66 indicators on a three-point scale. Applying the theoretical framework, a minimum set of performance indicators was selected for holistic hospital management, which well fit the healthcare context in Japan. This indicator set comprised 35 individual indicators and several factors measured through questionnaire surveys. The indicators were confirmed by expert judgments from viewpoints of face, content and construct validities as well as their usefulness. A theoretical framework of performance measurement was established from primary healthcare stakeholders' perspectives. Performance indicators were largely divided into healthcare outcomes and performance shaping factors. Indicators in the former category may be applied for the detection of operational problems, while their latent causes can be effectively addressed by the latter category in terms of process, structure and culture/climate within the organization. Copyright © 2013 Elsevier Ireland Ltd. All rights reserved.

Identification of the Emplacement of Improvised Explosive Devices by Experienced Mission Payload Operators.

PubMed

McNeese, Nathan J; Cooke, Nancy J; Branaghan, Russell; Knobloch, Ashley; Taylor, Amanda

2017-04-01

Improvised Explosive Devices (IEDs) have become one of the deadliest threats to military personnel, resulting in over 50% of American combat casualties in Iraq and Afghanistan. Identification of IED emplacement is conducted by mission payload operators (MPOs). Yet, experienced MPOs are limited in number, making MPO training a critical intervention. In this article, we implement a Cognitive Engineering Based on Expert Skill methodology to better understand how experienced MPOs identify the emplacement of IEDs for the purposes of improving training. First, expert knowledge was elicited through interviews and questionnaires to identify the types of perceptual cues used and how these cues are cognitively processed. Results indicate that there are many different static and dynamic cues that interact with each other over time and space. Using data from the interviews and questionnaires, an empirically grounded framework is presented that explains the cognitive process of IED emplacement detection. Using the overall findings and the framework, IED emplacement training scenarios were developed and built into a simulation. Copyright © 2016 Elsevier Ltd. All rights reserved.

Expert explanations of honeybee losses in areas of extensive agriculture in France: Gaucho® compared with other supposed causal factors

NASA Astrophysics Data System (ADS)

Maxim, L.; van der Sluijs, J. P.

2010-01-01

Debates on causality are at the core of controversies as regards environmental changes. The present paper presents a new method for analyzing controversies on causality in a context of social debate and the results of its empirical testing. The case study used is the controversy as regards the role played by the insecticide Gaucho®, compared with other supposed causal factors, in the substantial honeybee (Apis mellifera L.) losses reported to have occurred in France between 1994 and 2004. The method makes use of expert elicitation of the perceived strength of evidence regarding each of Bradford Hill's causality criteria, as regards the link between each of eight possible causal factors identified in attempts to explain each of five signs observed in honeybee colonies. These judgments are elicited from stakeholders and experts involved in the debate, i.e., representatives of Bayer Cropscience, of the Ministry of Agriculture, of the French Food Safety Authority, of beekeepers and of public scientists. We show that the intense controversy observed in confused and passionate public discourses is much less salient when the various arguments are structured using causation criteria. The contradictions between the different expert views have a triple origin: (1) the lack of shared definition and quantification of the signs observed in colonies; (2) the lack of specialist knowledge on honeybees; and (3) the strategic discursive practices associated with the lack of trust between experts representing stakeholders having diverging stakes in the case.

Traceable accounts of subjective probability judgments in the IPCC and beyond

NASA Astrophysics Data System (ADS)

Baer, P. G.

2012-12-01

One of the major sources of controversy surrounding the reports of the IPCC has been the characterization of uncertainty. Although arguably the IPCC has paid more attention to the process of uncertainty analysis and communication than any comparable assessment body, its efforts to achieve consistency have produced mixed results. In particular, the extensive use of subjective probability assessment has attracted widespread criticism. Statements such as "Average Northern Hemisphere temperatures during the second half of the 20th century were very likely higher than during any other 50-year period in the last 500 years" are ubiquitous (one online database lists nearly 3000 such claims), and indeed are the primary way in which its key "findings" are reported. Much attention is drawn to the precise quantitative definition of such statements (e.g., "very likely" means >90% probability, vs. "extremely likely" which means >95% certainty). But there is no process by which the decision regarding the choice of such uncertainty level for a given finding is formally made or reported, and thus they are easily by disputed by anyone, expert or otherwise, who disagrees with the assessment. In the "Uncertainty Guidance Paper" for the Third Assessment Report, Richard Moss and Steve Schneider defined the concept of a "traceable account," which gave exhaustive detail regarding how one ought to provide documentation of such an uncertainty assessment. But the guidance, while appearing straightforward and reasonable, in fact was an unworkable recipe, which would have taken near-infinite time if used for more than a few key results, and would have required a different structuring of the text than the conventional scientific assessment. And even then it would have left a gap when it came to the actual provenance of any such specific judgments, because there simply is no formal step at which individuals turn their knowledge of the evidence on some finding into a probability judgment. The Uncertainty Guidance Papers for the TAR and subsequent assessments have left open the possibility of using such an expert elicitation within the IPCC drafting process, but to my knowledge it has never been done. Were it in fact attempted, it would reveal the inconvenient truth that there is no uniquely correct method for aggregating probability statements; indeed the standard practice within climate-related expert elicitations has been to report all individual estimates without aggregation. But if a report requires a single "consensus estimate," once you have even a single divergent opinion, the question of how to aggregate becomes unavoidable. In this paper, I review in greater detail the match or lack of it between the vision of a "traceable account" and IPCC practice, and the public discussion of selected examples of probabilistic judgments in AR4. I propose elements of a structure based on a flexible software architecture that could facilitate the development and documentation of what I call "collective subjective probability." Using a simple prototype and a pair of sample "findings" from AR4, I demonstrate an example of how such a structure could be used by a small expert community to implement a practical model of a "traceable account." I conclude with as discussion of the prospects of using such modular elicitations in support of, or as an alternative to, conventional IPCC assessment processes.

Task-Oriented, Naturally Elicited Speech (TONE) Database for the Force Requirements Expert System, Hawaii (FRESH)

DTIC Science & Technology

1988-09-01

Group Subgroup Command and control; Computational linguistics; expert system voice recognition; man- machine interface; U.S. Government 19 Abstract...simulates the characteristics of FRESH on a smaller scale. This study assisted NOSC in developing a voice-recognition, man- machine interface that could...scale. This study assisted NOSC in developing a voice-recogni- tion, man- machine interface that could be used with TONE and upgraded at a later date

Probabilistic fault tree analysis of a radiation treatment system.

PubMed

Ekaette, Edidiong; Lee, Robert C; Cooke, David L; Iftody, Sandra; Craighead, Peter

2007-12-01

Inappropriate administration of radiation for cancer treatment can result in severe consequences such as premature death or appreciably impaired quality of life. There has been little study of vulnerable treatment process components and their contribution to the risk of radiation treatment (RT). In this article, we describe the application of probabilistic fault tree methods to assess the probability of radiation misadministration to patients at a large cancer treatment center. We conducted a systematic analysis of the RT process that identified four process domains: Assessment, Preparation, Treatment, and Follow-up. For the Preparation domain, we analyzed possible incident scenarios via fault trees. For each task, we also identified existing quality control measures. To populate the fault trees we used subjective probabilities from experts and compared results with incident report data. Both the fault tree and the incident report analysis revealed simulation tasks to be most prone to incidents, and the treatment prescription task to be least prone to incidents. The probability of a Preparation domain incident was estimated to be in the range of 0.1-0.7% based on incident reports, which is comparable to the mean value of 0.4% from the fault tree analysis using probabilities from the expert elicitation exercise. In conclusion, an analysis of part of the RT system using a fault tree populated with subjective probabilities from experts was useful in identifying vulnerable components of the system, and provided quantitative data for risk management.

Critical components of reflective supervision: responses from expert supervisors in the field.

PubMed

Tomlin, Angela M; Weatherston, Deborah J; Pavkov, Thomas

2014-01-01

This article offers a brief review of the history of supervision, defines reflective supervision, and reports the results of a Delphi study designed to identify critical components of reflective supervision. Academicians and master clinicians skilled in providing reflective supervision participated in a three-phase survey to elicit beliefs about best practice when engaging in reflective supervision. The process yielded consensus descriptions of optimal characteristics and behaviors of supervisors and supervisees when entering into supervisory relationships that encourage reflective practice. These results, although preliminary, suggest that it is possible to identify elements that are integral to effective reflective supervision. These initial findings may be used for future study of the reflective supervisory process. © 2013 Michigan Association for Infant Mental Health.

The use of belief-based probabilistic methods in volcanology: Scientists' views and implications for risk assessments

NASA Astrophysics Data System (ADS)

Donovan, Amy; Oppenheimer, Clive; Bravo, Michael

2012-12-01

This paper constitutes a philosophical and social scientific study of expert elicitation in the assessment and management of volcanic risk on Montserrat during the 1995-present volcanic activity. It outlines the broader context of subjective probabilistic methods and then uses a mixed-method approach to analyse the use of these methods in volcanic crises. Data from a global survey of volcanologists regarding the use of statistical methods in hazard assessment are presented. Detailed qualitative data from Montserrat are then discussed, particularly concerning the expert elicitation procedure that was pioneered during the eruptions. These data are analysed and conclusions about the use of these methods in volcanology are drawn. The paper finds that while many volcanologists are open to the use of these methods, there are still some concerns, which are similar to the concerns encountered in the literature on probabilistic and determinist approaches to seismic hazard analysis.

Health risk assessment for nanoparticles: A case for using expert judgment

NASA Astrophysics Data System (ADS)

Kandlikar, Milind; Ramachandran, Gurumurthy; Maynard, Andrew; Murdock, Barbara; Toscano, William A.

2007-01-01

Uncertainties in conventional quantitative risk assessment typically relate to values of parameters in risk models. For many environmental contaminants, there is a lack of sufficient information about multiple components of the risk assessment framework. In such cases, the use of default assumptions and extrapolations to fill in the data gaps is a common practice. Nanoparticle risks, however, pose a new form of risk assessment challenge. Besides a lack of data, there is deep scientific uncertainty regarding every aspect of the risk assessment framework: (a) particle characteristics that may affect toxicity; (b) their fate and transport through the environment; (c) the routes of exposure and the metrics by which exposure ought to be measured; (d) the mechanisms of translocation to different parts of the body; and (e) the mechanisms of toxicity and disease. In each of these areas, there are multiple and competing models and hypotheses. These are not merely parametric uncertainties but uncertainties about the choice of the causal mechanisms themselves and the proper model variables to be used, i.e., structural uncertainties. While these uncertainties exist for PM2.5 as well, risk assessment for PM2.5 has avoided dealing with these issues because of a plethora of epidemiological studies. However, such studies don't exist for the case of nanoparticles. Even if such studies are done in the future, they will be very specific to a particular type of engineered nanoparticle and not generalizable to other nanoparticles. Therefore, risk assessment for nanoparticles will have to deal with the various uncertainties that were avoided in the case of PM2.5. Consequently, uncertainties in estimating risks due to nanoparticle exposures may be characterized as `extreme'. This paper proposes a methodology by which risk analysts can cope with such extreme uncertainty. One way to make these problems analytically tractable is to use expert judgment approaches to study the degree of consensus and/or disagreement between experts on different parts of the exposure-response paradigm. This can be done by eliciting judgments from a wide range of experts on different parts of the risk causal chain. We also use examples to illustrate how studying expert consensus/disagreement helps in research prioritization and budget allocation exercises. The expert elicitation can be repeated over the course of several years, over which time, the state of scientific knowledge will also improve and uncertainties may possibly reduce. Results from expert the elicitation exercise can be used by risk managers or managers of funding agencies as a tool for research prioritization.

Using XML and XSLT for flexible elicitation of mental-health risk knowledge.

PubMed

Buckingham, C D; Ahmed, A; Adams, A E

2007-03-01

Current tools for assessing risks associated with mental-health problems require assessors to make high-level judgements based on clinical experience. This paper describes how new technologies can enhance qualitative research methods to identify lower-level cues underlying these judgements, which can be collected by people without a specialist mental-health background. Content analysis of interviews with 46 multidisciplinary mental-health experts exposed the cues and their interrelationships, which were represented by a mind map using software that stores maps as XML. All 46 mind maps were integrated into a single XML knowledge structure and analysed by a Lisp program to generate quantitative information about the numbers of experts associated with each part of it. The knowledge was refined by the experts, using software developed in Flash to record their collective views within the XML itself. These views specified how the XML should be transformed by XSLT, a technology for rendering XML, which resulted in a validated hierarchical knowledge structure associating patient cues with risks. Changing knowledge elicitation requirements were accommodated by flexible transformations of XML data using XSLT, which also facilitated generation of multiple data-gathering tools suiting different assessment circumstances and levels of mental-health knowledge.

DOE Office of Scientific and Technical Information (OSTI.GOV)

F.V. Perry; A. Cogbill; R. Kelley

The U.S. Department of Energy (DOE) considers volcanism to be a potentially disruptive class of events that could affect the safety of the proposed high-level waste repository at Yucca Mountain. Volcanic hazard assessment in monogenetic volcanic fields depends on an adequate understanding of the temporal and spatial pattern of past eruptions. At Yucca Mountain, the hazard is due to an 11 Ma-history of basaltic volcanism with the latest eruptions occurring in three Pleistocene episodes to the west and south of Yucca Mountain. An expert elicitation convened in 1995-1996 by the DOE estimated the mean hazard of volcanic disruption of themore » repository as slightly greater than 10{sup -8} dike intersections per year with an uncertainty of about two orders of magnitude. Several boreholes in the region have encountered buried basalt in alluvial-filled basins; the youngest of these basalts is dated at 3.8 Ma. The possibility of additional buried basalt centers is indicated by a previous regional aeromagnetic survey conducted by the USGS that detected approximately 20 magnetic anomalies that could represent buried basalt volcanoes. Sensitivity studies indicate that the postulated presence of buried post-Miocene volcanoes to the east of Yucca Mountain could increase the hazard by an order of magnitude, and potentially significantly impact the results of the earlier expert elicitation. Our interpretation of the aeromagnetic data indicates that post-Miocene basalts are not present east of Yucca Mountain, but that magnetic anomalies instead represent faulted and buried Miocene basalt that correlates with nearby surface exposures. This interpretation is being tested by drilling. The possibility of uncharacterized buried volcanoes that could significantly change hazard estimates led DOE to support an update of the expert elicitation in 2004-2006. In support of the expert elicitation data needs, the DOE is sponsoring (1) a new higher-resolution, helicopter-borne aeromagnetic survey, completed in mid-2004, and (2) drilling of selected anomalies based on the aeromagnetic survey results to better characterize the number, location and age of buried volcanoes, which began in mid-2005. The new aeromagnetic survey detected the presence of 33 anomalies interpreted as possible buried volcanoes or faulted tuff bedrock. A program to drill ten of the anomalies has begun, with the selection of drill holes prioritized based on their potential impact on the hazard assessment.« less

Public perceptions of the risks of an unfamiliar technology: The case of using nuclear energy sources for space missions

NASA Astrophysics Data System (ADS)

Maharik, Michael

This thesis addresses the public perception of the risk of a technology not widely known to laypeople. Its aims were (1) to characterize public perceptions of the risk of using nuclear energy in space and decisions related to this risk, and (2) to extend the 'mental model' methodology to studying public perception of unfamiliar, risky technologies. A model of the physical processes capable of creating risks from using nuclear energy sources in space was first constructed. Then, knowledge and beliefs related to this topic were elicited from three different groups of people. The generality of the findings was examined in a constructive replication with environmentally-oriented people. The possibility of involving the public in decision-making processes related to engineering macro-design was then investigated. Finally, a communication regarding these risk processes was developed and evaluated in an experiment comparing it with communications produced by NASA. Although they included large portions of the expert model, people's beliefs also had gaps and misconceptions. Respondents often used scientific terms without a clear understanding of what they meant. Respondents' mental models sometimes contained scattered and inconsistent entries. The impact of pre-existing mental models was clearly seen. Different groups of people had different patterns of knowledge and beliefs. Nevertheless, respondents expressed reasonable and coherent opinions on choices among engineering options. The CMU brochure, derived from the study of readers' existing mental models, provided a better risk communication tool than NASA's material, reflecting primarily experts' perspective. The better performance of subjects reading either brochure generally reflected adding knowledge on issues that they had not previously known, rather than correcting wrong beliefs. The communication study confirmed a hypothesis that improving knowledge on risk processes related to the use of a technology causes a more favorable attitude towards that technology. Recommendations related to the design and targeting of risk communication, and to public participation in decision-making on using new and risky technologies, are derived. Additional studies that will elicit laypeople's definitions of risk related to specific technologies, and link their detailed understanding of risk-development processes to the perceived dimensions of risk, are suggested.

Patient Health Goals Elicited During Home Care Admission: A Categorization.

PubMed

Sockolow, Paulina; Radhakrishnan, Kavita; Chou, Edgar Y; Wojciechowicz, Christine

2017-11-01

Home care agencies are initiating "patient health goal elicitation" activities as part of home care admission planning. We categorized elicited goals and identified "clinically informative" goals at a home care agency. We examined patient goals that admitting clinicians documented in the point-of-care electronic health record; conducted content analysis on patient goal data to develop a coding scheme; grouped goal themes into codes; assigned codes to each goal; and identified goals that were in the patient voice. Of the 1,763 patient records, 16% lacked a goal; only 15 goals were in a patient's voice. Nurse and physician experts identified 12 of the 20 codes as clinically important accounting for 82% of goal occurrences. The most frequent goal documented was safety/falls (23%). Training and consistent communication of the intent and operationalization of patient goal elicitation may address the absence of patient voice and the less than universal recording of home care patients' goals.

Advancing the Guánica Bay (Puerto Rico) Watershed Management Plan

EPA Science Inventory

Consideration of stakeholder values in watershed planning and management is a necessity, but sufficiently eliciting, understanding, and organizing those values can be daunting. Many studies have demonstrated the usefulness of formal decision analysis to integrate expert knowledge...

Assessing the Vulnerability of Older Americans to Climate Change

EPA Science Inventory

This project is comprised of a series of activities – listening sessions, literature reviews, and an expert elicitation research agenda-setting workshop – designed to examine and characterize the vulnerability of older adults to climate change and opportunities for adaptation. ...

Balancing research and funding using value of information and portfolio tools for nanomaterial risk classification

NASA Astrophysics Data System (ADS)

Bates, Matthew E.; Keisler, Jeffrey M.; Zussblatt, Niels P.; Plourde, Kenton J.; Wender, Ben A.; Linkov, Igor

2016-02-01

Risk research for nanomaterials is currently prioritized by means of expert workshops and other deliberative processes. However, analytical techniques that quantify and compare alternative research investments are increasingly recommended. Here, we apply value of information and portfolio decision analysis—methods commonly applied in financial and operations management—to prioritize risk research for multiwalled carbon nanotubes and nanoparticulate silver and titanium dioxide. We modify the widely accepted CB Nanotool hazard evaluation framework, which combines nano- and bulk-material properties into a hazard score, to operate probabilistically with uncertain inputs. Literature is reviewed to develop uncertain estimates for each input parameter, and a Monte Carlo simulation is applied to assess how different research strategies can improve hazard classification. The relative cost of each research experiment is elicited from experts, which enables identification of efficient research portfolios—combinations of experiments that lead to the greatest improvement in hazard classification at the lowest cost. Nanoparticle shape, diameter, solubility and surface reactivity were most frequently identified within efficient portfolios in our results.

Balancing research and funding using value of information and portfolio tools for nanomaterial risk classification.

PubMed

Bates, Matthew E; Keisler, Jeffrey M; Zussblatt, Niels P; Plourde, Kenton J; Wender, Ben A; Linkov, Igor

2016-02-01

Risk research for nanomaterials is currently prioritized by means of expert workshops and other deliberative processes. However, analytical techniques that quantify and compare alternative research investments are increasingly recommended. Here, we apply value of information and portfolio decision analysis-methods commonly applied in financial and operations management-to prioritize risk research for multiwalled carbon nanotubes and nanoparticulate silver and titanium dioxide. We modify the widely accepted CB Nanotool hazard evaluation framework, which combines nano- and bulk-material properties into a hazard score, to operate probabilistically with uncertain inputs. Literature is reviewed to develop uncertain estimates for each input parameter, and a Monte Carlo simulation is applied to assess how different research strategies can improve hazard classification. The relative cost of each research experiment is elicited from experts, which enables identification of efficient research portfolios-combinations of experiments that lead to the greatest improvement in hazard classification at the lowest cost. Nanoparticle shape, diameter, solubility and surface reactivity were most frequently identified within efficient portfolios in our results.

Short-term leprosy forecasting from an expert opinion survey.

PubMed

Deiner, Michael S; Worden, Lee; Rittel, Alex; Ackley, Sarah F; Liu, Fengchen; Blum, Laura; Scott, James C; Lietman, Thomas M; Porco, Travis C

2017-01-01

We conducted an expert survey of leprosy (Hansen's Disease) and neglected tropical disease experts in February 2016. Experts were asked to forecast the next year of reported cases for the world, for the top three countries, and for selected states and territories of India. A total of 103 respondents answered at least one forecasting question. We elicited lower and upper confidence bounds. Comparing these results to regression and exponential smoothing, we found no evidence that any forecasting method outperformed the others. We found evidence that experts who believed it was more likely to achieve global interruption of transmission goals and disability reduction goals had higher error scores for India and Indonesia, but lower for Brazil. Even for a disease whose epidemiology changes on a slow time scale, forecasting exercises such as we conducted are simple and practical. We believe they can be used on a routine basis in public health.

Short-term leprosy forecasting from an expert opinion survey

PubMed Central

Deiner, Michael S.; Worden, Lee; Rittel, Alex; Ackley, Sarah F.; Liu, Fengchen; Blum, Laura; Scott, James C.; Lietman, Thomas M.

2017-01-01

We conducted an expert survey of leprosy (Hansen’s Disease) and neglected tropical disease experts in February 2016. Experts were asked to forecast the next year of reported cases for the world, for the top three countries, and for selected states and territories of India. A total of 103 respondents answered at least one forecasting question. We elicited lower and upper confidence bounds. Comparing these results to regression and exponential smoothing, we found no evidence that any forecasting method outperformed the others. We found evidence that experts who believed it was more likely to achieve global interruption of transmission goals and disability reduction goals had higher error scores for India and Indonesia, but lower for Brazil. Even for a disease whose epidemiology changes on a slow time scale, forecasting exercises such as we conducted are simple and practical. We believe they can be used on a routine basis in public health. PMID:28813531

The auditory scene: an fMRI study on melody and accompaniment in professional pianists.

PubMed

Spada, Danilo; Verga, Laura; Iadanza, Antonella; Tettamanti, Marco; Perani, Daniela

2014-11-15

The auditory scene is a mental representation of individual sounds extracted from the summed sound waveform reaching the ears of the listeners. Musical contexts represent particularly complex cases of auditory scenes. In such a scenario, melody may be seen as the main object moving on a background represented by the accompaniment. Both melody and accompaniment vary in time according to harmonic rules, forming a typical texture with melody in the most prominent, salient voice. In the present sparse acquisition functional magnetic resonance imaging study, we investigated the interplay between melody and accompaniment in trained pianists, by observing the activation responses elicited by processing: (1) melody placed in the upper and lower texture voices, leading to, respectively, a higher and lower auditory salience; (2) harmonic violations occurring in either the melody, the accompaniment, or both. The results indicated that the neural activation elicited by the processing of polyphonic compositions in expert musicians depends upon the upper versus lower position of the melodic line in the texture, and showed an overall greater activation for the harmonic processing of melody over accompaniment. Both these two predominant effects were characterized by the involvement of the posterior cingulate cortex and precuneus, among other associative brain regions. We discuss the prominent role of the posterior medial cortex in the processing of melodic and harmonic information in the auditory stream, and propose to frame this processing in relation to the cognitive construction of complex multimodal sensory imagery scenes. Copyright © 2014 Elsevier Inc. All rights reserved.

Prior elicitation and Bayesian analysis of the Steroids for Corneal Ulcers Trial.

PubMed

See, Craig W; Srinivasan, Muthiah; Saravanan, Somu; Oldenburg, Catherine E; Esterberg, Elizabeth J; Ray, Kathryn J; Glaser, Tanya S; Tu, Elmer Y; Zegans, Michael E; McLeod, Stephen D; Acharya, Nisha R; Lietman, Thomas M

2012-12-01

To elicit expert opinion on the use of adjunctive corticosteroid therapy in bacterial corneal ulcers. To perform a Bayesian analysis of the Steroids for Corneal Ulcers Trial (SCUT), using expert opinion as a prior probability. The SCUT was a placebo-controlled trial assessing visual outcomes in patients receiving topical corticosteroids or placebo as adjunctive therapy for bacterial keratitis. Questionnaires were conducted at scientific meetings in India and North America to gauge expert consensus on the perceived benefit of corticosteroids as adjunct treatment. Bayesian analysis, using the questionnaire data as a prior probability and the primary outcome of SCUT as a likelihood, was performed. For comparison, an additional Bayesian analysis was performed using the results of the SCUT pilot study as a prior distribution. Indian respondents believed there to be a 1.21 Snellen line improvement, and North American respondents believed there to be a 1.24 line improvement with corticosteroid therapy. The SCUT primary outcome found a non-significant 0.09 Snellen line benefit with corticosteroid treatment. The results of the Bayesian analysis estimated a slightly greater benefit than did the SCUT primary analysis (0.19 lines verses 0.09 lines). Indian and North American experts had similar expectations on the effectiveness of corticosteroids in bacterial corneal ulcers; that corticosteroids would markedly improve visual outcomes. Bayesian analysis produced results very similar to those produced by the SCUT primary analysis. The similarity in result is likely due to the large sample size of SCUT and helps validate the results of SCUT.

Teacher Knowledge for Active-Learning Instruction: Expert-Novice Comparison Reveals Differences.

PubMed

Auerbach, A J; Higgins, M; Brickman, P; Andrews, T C

2018-01-01

Active-learning strategies can improve science, technology, engineering, and mathematics (STEM) undergraduates' abilities to learn fundamental concepts and skills. However, the results instructors achieve vary substantially. One explanation for this is that instructors commonly implement active learning differently than intended. An important factor affecting how instructors implement active learning is knowledge of teaching and learning. We aimed to discover knowledge that is important to effective active learning in large undergraduate courses. We developed a lesson-analysis instrument to elicit teacher knowledge, drawing on the theoretical construct of teacher noticing. We compared the knowledge used by expert ( n = 14) and novice ( n = 29) active-learning instructors as they analyzed lessons. Experts and novices differed in what they noticed, with experts more commonly considering how instructors hold students accountable, topic-specific student difficulties, whether the instructor elicited and responded to student thinking, and opportunities students had to generate their own ideas and work. Experts were also better able to support their lesson analyses with reasoning. This work provides foundational knowledge for the future design of preparation and support for instructors adopting active learning. Improving teacher knowledge will improve the implementation of active learning, which will be necessary to widely realize the potential benefits of active learning in undergraduate STEM. © 2018 A. J. Auerbach et al. CBE—Life Sciences Education © 2018 The American Society for Cell Biology. This article is distributed by The American Society for Cell Biology under license from the author(s). It is available to the public under an Attribution–Noncommercial–Share Alike 3.0 Unported Creative Commons License (http://creativecommons.org/licenses/by-nc-sa/3.0).

BAYESIAN META-ANALYSIS ON MEDICAL DEVICES: APPLICATION TO IMPLANTABLE CARDIOVERTER DEFIBRILLATORS

PubMed Central

Youn, Ji-Hee; Lord, Joanne; Hemming, Karla; Girling, Alan; Buxton, Martin

2012-01-01

Objectives: The aim of this study is to describe and illustrate a method to obtain early estimates of the effectiveness of a new version of a medical device. Methods: In the absence of empirical data, expert opinion may be elicited on the expected difference between the conventional and modified devices. Bayesian Mixed Treatment Comparison (MTC) meta-analysis can then be used to combine this expert opinion with existing trial data on earlier versions of the device. We illustrate this approach for a new four-pole implantable cardioverter defibrillator (ICD) compared with conventional ICDs, Class III anti-arrhythmic drugs, and conventional drug therapy for the prevention of sudden cardiac death in high risk patients. Existing RCTs were identified from a published systematic review, and we elicited opinion on the difference between four-pole and conventional ICDs from experts recruited at a cardiology conference. Results: Twelve randomized controlled trials were identified. Seven experts provided valid probability distributions for the new ICDs compared with current devices. The MTC model resulted in estimated relative risks of mortality of 0.74 (0.60–0.89) (predictive relative risk [RR] = 0.77 [0.41–1.26]) and 0.83 (0.70–0.97) (predictive RR = 0.84 [0.55–1.22]) with the new ICD therapy compared to Class III anti-arrhythmic drug therapy and conventional drug therapy, respectively. These results showed negligible differences from the preliminary results for the existing ICDs. Conclusions: The proposed method incorporating expert opinion to adjust for a modification made to an existing device may play a useful role in assisting decision makers to make early informed judgments on the effectiveness of frequently modified healthcare technologies. PMID:22559753

Management of uncertainties on parameters elicited by experts - Applications to sea-level rise and to CO2 storage operations risk assessment

NASA Astrophysics Data System (ADS)

Manceau, Jean-Charles; Loschetter, Annick; Rohmer, Jérémy; Le Cozannet, Gonéri; Lary Louis, de; Guénan Thomas, Le; Ken, Hnottavange-Telleen

2017-04-01

In a context of high degree of uncertainty, when very few data are available, experts are commonly requested to provide their opinions on input parameters of risk assessment models. Not only might each expert express a certain degree of uncertainty on his/her own statements, but the set of information collected from the pool of experts introduces an additional level of uncertainty. It is indeed very unlikely that all experts agree on exactly the same data, especially regarding parameters needed for natural risk assessments. In some cases, their opinions may differ only slightly (e.g. the most plausible value for a parameter is similar for different experts, and they only disagree on the level of uncertainties that taint the said value) while on other cases they may express incompatible opinions for a same parameter. Dealing with these different kinds of uncertainties remains a challenge for assessing geological hazards or/and risks. Extra-probabilistic approaches (such as the Dempster-Shafer theory or the possibility theory) have shown to offer promising solutions for representing parameters on which the knowledge is limited. It is the case for instance when the available information prevents an expert from identifying a unique probability law to picture the total uncertainty. Moreover, such approaches are known to be particularly flexible when it comes to aggregating several and potentially conflicting opinions. We therefore propose to discuss the opportunity of applying these new theories for managing the uncertainties on parameters elicited by experts, by a comparison with the application of more classical probability approaches. The discussion is based on two different examples. The first example deals with the estimation of the injected CO2 plume extent in a reservoir in the context of CO2 geological storage. This estimation requires information on the effective porosity of the reservoir, which has been estimated by 14 different experts. The Dempster-Shafer theory has been used to represent and aggregate these pieces of information. The results of different aggregation rules as well as those of a classical probabilistic approach are compared with the purpose of highlighting the elements each of them could provide to the decision-maker (Manceau et al., 2016). The second example focuses on projections of future sea-level rise. Based on IPCC's constraints on the projection quantiles, and on the scientific community consensus level on the physical limits to future sea-level rise, a possibility distribution of the projections by 2100 under the RCP 8.5 scenario has been established. This possibility distribution has been confronted with a set of previously published probabilistic sea-level projections, with a focus on their ability to explore high ranges of sea-level rise (Le Cozannet et al., 2016). These two examples are complementary in the sense that they allow to address various aspects of the problem (e.g. representation of different types of information, conflict among experts, sources dependence). Moreover, we believe that the issues faced during these two experiences can be generalized to many risks/hazards assessment situations. References Manceau, JC., Loschetter, A., Rohmer, J., de Lary, L., Le Guénan, T., Hnottavange-Telleen, K. (2016). Dealing with uncertainty on parameters elicited from a pool of experts for CCS risk assessment. Congrès λμ 20 (St-Malo, France). Le Cozannet G., Manceau JC., Rohmer, J. (2016). Bounding probabilistic sea-level rise projections within the framework of the possibility theory. Accepted in Environmental Research Letters.

Developing a Methodology for Eliciting Subjective Probability Estimates During Expert Evaluations of Safety Interventions: Application for Bayesian Belief Networks

NASA Technical Reports Server (NTRS)

Wiegmann, Douglas A.a

2005-01-01

The NASA Aviation Safety Program (AvSP) has defined several products that will potentially modify airline and/or ATC operations, enhance aircraft systems, and improve the identification of potential hazardous situations within the National Airspace System (NAS). Consequently, there is a need to develop methods for evaluating the potential safety benefit of each of these intervention products so that resources can be effectively invested to produce the judgments to develop Bayesian Belief Networks (BBN's) that model the potential impact that specific interventions may have. Specifically, the present report summarizes methodologies for improving the elicitation of probability estimates during expert evaluations of AvSP products for use in BBN's. The work involved joint efforts between Professor James Luxhoj from Rutgers University and researchers at the University of Illinois. The Rutgers' project to develop BBN's received funding by NASA entitled "Probabilistic Decision Support for Evaluating Technology Insertion and Assessing Aviation Safety System Risk." The proposed project was funded separately but supported the existing Rutgers' program.

Evaluating oversight of human drugs and medical devices: a case study of the FDA and implications for nanobiotechnology.

PubMed

Paradise, Jordan; Tisdale, Alison W; Hall, Ralph F; Kokkoli, Efrosini

2009-01-01

This article evaluates the oversight of drugs and medical devices by the U.S. Food and Drug Administration (FDA) using an integration of public policy, law, and bioethics approaches and employing multiple assessment criteria, including economic, social, safety, and technological. Criteria assessment and expert elicitation are combined with existing literature, case law, and regulations in an integrative historical case studies approach. We then use our findings as a tool to explore possibilities for effective oversight and regulatory mechanisms for nanobiotechnology. Section I describes oversight mechanisms for human drugs and medical devices and presents current nanotechnology products. Section II describes the results of expert elicitation research. Section III highlights key criteria and relates them to the literature and larger debate. We conclude with broad lessons for the oversight of nanobiotechnology informed by Sections I-III in order to provide useful analysis from multiple disciplines and perspectives to guide discussions regarding appropriate FDA oversight.

What do potential jurors know about police interrogation techniques and false confessions?

PubMed

Leo, Richard A; Liu, Brittany

2009-01-01

Psychological police interrogation methods in America inevitably involve some level of pressure and persuasion to achieve their goal of eliciting confessions of guilt from custodial suspects. In this article, we surveyed potential jurors about their perceptions of a range of psychological interrogation techniques, the likelihood that such techniques would elicit a true confession from guilty suspects, and the likelihood that such techniques could elicit a false confession from innocent suspects. Participants recognized that these interrogation techniques may be psychologically coercive and may elicit true confessions, but believed that psychologically coercive interrogation techniques are not likely to elicit false confessions. The findings from this survey study indicate that potential jurors believe that false confessions are both counter- intuitive and unlikely, even in response to psychologically coercive interrogation techniques that have been shown to lead to false confessions from the innocent. This study provides empirical support for the idea that expert witnesses may helpfully inform jurors about the social science research on psychologically coercive interrogation methods and how and why such interrogation techniques can lead to false confessions. (c) 2009 John Wiley & Sons, Ltd.

The making of expert clinicians: reflective practice.

PubMed

Maestre, J M; Szyld, D; Del Moral, I; Ortiz, G; Rudolph, J W

2014-05-01

Debriefing is a rigorous reflection process which helps trainees recognize and resolve clinical and behavioral dilemmas raised by a clinical case. This approach emphasizes eliciting trainees'assumptions about the situation and their reasons for performing as they did (mental models). It analyses their impact on actions, to understand if it is necessary to maintain them or construct new ones that may lead to better performance in the future. It blends evidence and theory from education research, the social and cognitive sciences, and experience drawn from conducting and teaching debriefing to clinicians worldwide, on how to improve professional effectiveness through "reflective practice". Copyright © 2013 Elsevier España, S.L. All rights reserved.

Efficient fuzzy Bayesian inference algorithms for incorporating expert knowledge in parameter estimation

NASA Astrophysics Data System (ADS)

Rajabi, Mohammad Mahdi; Ataie-Ashtiani, Behzad

2016-05-01

Bayesian inference has traditionally been conceived as the proper framework for the formal incorporation of expert knowledge in parameter estimation of groundwater models. However, conventional Bayesian inference is incapable of taking into account the imprecision essentially embedded in expert provided information. In order to solve this problem, a number of extensions to conventional Bayesian inference have been introduced in recent years. One of these extensions is 'fuzzy Bayesian inference' which is the result of integrating fuzzy techniques into Bayesian statistics. Fuzzy Bayesian inference has a number of desirable features which makes it an attractive approach for incorporating expert knowledge in the parameter estimation process of groundwater models: (1) it is well adapted to the nature of expert provided information, (2) it allows to distinguishably model both uncertainty and imprecision, and (3) it presents a framework for fusing expert provided information regarding the various inputs of the Bayesian inference algorithm. However an important obstacle in employing fuzzy Bayesian inference in groundwater numerical modeling applications is the computational burden, as the required number of numerical model simulations often becomes extremely exhaustive and often computationally infeasible. In this paper, a novel approach of accelerating the fuzzy Bayesian inference algorithm is proposed which is based on using approximate posterior distributions derived from surrogate modeling, as a screening tool in the computations. The proposed approach is first applied to a synthetic test case of seawater intrusion (SWI) in a coastal aquifer. It is shown that for this synthetic test case, the proposed approach decreases the number of required numerical simulations by an order of magnitude. Then the proposed approach is applied to a real-world test case involving three-dimensional numerical modeling of SWI in Kish Island, located in the Persian Gulf. An expert elicitation methodology is developed and applied to the real-world test case in order to provide a road map for the use of fuzzy Bayesian inference in groundwater modeling applications.

Setting priorities for safe motherhood programme evaluation: a participatory process in three developing countries.

PubMed

Madi, Banyana Cecilia; Hussein, Julia; Hounton, Sennen; D'Ambruoso, Lucia; Achadi, Endang; Arhinful, Daniel Kojo

2007-09-01

A participatory approach to priority setting in programme evaluation may help improve the allocation and more efficient use of scarce resources especially in low-income countries. Research agendas that are the result of collaboration between researchers, programme managers, policy makers and other stakeholders have the potential to ensure rigorous studies are conducted on matters of local priority, based on local, expert knowledge. This paper describes a process involving key stakeholders to elicit and prioritise evaluation needs for safe motherhood in three developing countries. A series of reiterative consultations with safe motherhood stakeholders from each country was conducted over a period of 36 months. In each country, the consultation process consisted of a series of participatory workshops; firstly, stakeholder's views on evaluation were elicited with parallel descriptive work on the contexts. Secondly, priorities for evaluation were identified from stakeholders; thirdly, the evaluation-priorities were refined; and finally, the evaluation research questions, reflecting the identified priorities, were agreed and finalised. Three evaluation-questions were identified in each country, and one selected, on which a full scale evaluation was undertaken. While there is a great deal written about the importance of transparent and participatory priority setting in evaluation; few examples of how such processes could be implemented exist, particularly for maternal health programmes. Our experience demonstrates that the investment in a participatory priority-setting effort is high but the process undertaken resulted in both globally and contextually-relevant priorities for evaluation. This experience provides useful lessons for public health practitioners committed to bridging the research-policy interface.

Training for Collaboration: Collaborative Practice Skills for Mental Health Professionals

ERIC Educational Resources Information Center

Bischoff, Richard J.; Springer, Paul R.; Reisbig, Allison M. J.; Lyons, Sheena; Likcani, Adriatik

2012-01-01

The purpose of the study was to identify skills that mental health practitioners need for successful collaborative practice in medical settings. Known experts in the field of collaborative health care completed a survey designed to elicit their suggestions about what is needed for successful collaborative care practice. Through qualitative…

Establishing a Taxonometric Structure for the Study of Biotechnology in Secondary School Technology Education.

ERIC Educational Resources Information Center

Wells, John G.

1994-01-01

A Delphi panel of 19 experts identified 8 main knowledge areas of biotechnology: bioprocessing, foundations, genetic engineering, agriculture, biochemistry, medicine, environment, and bioethics. Round 2 elicited 84 subdivisions and round 3 adjusted the ratings. The resulting classification suggests a different context and focus for technology…

Representing and Inferring Visual Perceptual Skills in Dermatological Image Understanding

ERIC Educational Resources Information Center

Li, Rui

2013-01-01

Experts have a remarkable capability of locating, perceptually organizing, identifying, and categorizing objects in images specific to their domains of expertise. Eliciting and representing their visual strategies and some aspects of domain knowledge will benefit a wide range of studies and applications. For example, image understanding may be…

Added value of experts' knowledge to improve a quantitative microbial exposure assessment model--Application to aseptic-UHT food products.

PubMed

Pujol, Laure; Johnson, Nicholas Brian; Magras, Catherine; Albert, Isabelle; Membré, Jeanne-Marie

2015-10-15

In a previous study, a quantitative microbial exposure assessment (QMEA) model applied to an aseptic-UHT food process was developed [Pujol, L., Albert, I., Magras, C., Johnson, N. B., Membré, J. M. Probabilistic exposure assessment model to estimate aseptic UHT product failure rate. 2015 International Journal of Food Microbiology. 192, 124-141]. It quantified Sterility Failure Rate (SFR) associated with Bacillus cereus and Geobacillus stearothermophilus per process module (nine modules in total from raw material reception to end-product storage). Previously, the probabilistic model inputs were set by experts (using knowledge and in-house data). However, only the variability dimension was taken into account. The model was then improved using expert elicitation knowledge in two ways. First, the model was refined by adding the uncertainty dimension to the probabilistic inputs, enabling to set a second order Monte Carlo analysis. The eight following inputs, and their impact on SFR, are presented in detail in this present study: D-value for each bacteria of interest (B. cereus and G. stearothermophilus) associated with the inactivation model for the UHT treatment step, i.e., two inputs; log reduction (decimal reduction) number associated with the inactivation model for the packaging sterilization step for each bacterium and each part of the packaging (product container and sealing component), i.e., four inputs; and bacterial spore air load of the aseptic tank and the filler cabinet rooms, i.e., two inputs. Second, the model was improved by leveraging expert knowledge to develop further the existing model. The proportion of bacteria in the product which settled on surface of pipes (between the UHT treatment and the aseptic tank on one hand, and between the aseptic tank and the filler cabinet on the other hand) leading to a possible biofilm formation for each bacterium, was better characterized. It was modeled as a function of the hygienic design level of the aseptic-UHT line: the experts provided the model structure and most of the model parameters values. Mean of SFR was estimated to 10×10(-8) (95% Confidence Interval=[0×10(-8); 350×10(-8)]) and 570×10(-8) (95% CI=[380×10(-8); 820×10(-8)]) for B. cereus and G. stearothermophilus, respectively. These estimations were more accurate (since the confidence interval was provided) than those given by the model with only variability (for which the estimates were 15×10(-8) and 580×10(-8) for B. cereus and G. stearothermophilus, respectively). The updated model outputs were also compared with those obtained when inputs were described by a generic distribution, without specific information related to the case-study. Results showed that using a generic distribution can lead to unrealistic estimations (e.g., 3,181,000 product units contaminated by G. stearothermophilus among 10(8) product units produced) and emphasized the added value of eliciting information from experts from the relevant specialist field knowledge. Copyright © 2015 Elsevier B.V. All rights reserved.

Prior Elicitation and Bayesian Analysis of the Steroids for Corneal Ulcers Trial

PubMed Central

See, Craig W.; Srinivasan, Muthiah; Saravanan, Somu; Oldenburg, Catherine E.; Esterberg, Elizabeth J.; Ray, Kathryn J.; Glaser, Tanya S.; Tu, Elmer Y.; Zegans, Michael E.; McLeod, Stephen D.; Acharya, Nisha R.; Lietman, Thomas M.

2013-01-01

Purpose To elicit expert opinion on the use of adjunctive corticosteroid therapy in bacterial corneal ulcers. To perform a Bayesian analysis of the Steroids for Corneal Ulcers Trial (SCUT), using expert opinion as a prior probability. Methods The SCUT was a placebo-controlled trial assessing visual outcomes in patients receiving topical corticosteroids or placebo as adjunctive therapy for bacterial keratitis. Questionnaires were conducted at scientific meetings in India and North America to gauge expert consensus on the perceived benefit of corticosteroids as adjunct treatment. Bayesian analysis, using the questionnaire data as a prior probability and the primary outcome of SCUT as a likelihood, was performed. For comparison, an additional Bayesian analysis was performed using the results of the SCUT pilot study as a prior distribution. Results Indian respondents believed there to be a 1.21 Snellen line improvement, and North American respondents believed there to be a 1.24 line improvement with corticosteroid therapy. The SCUT primary outcome found a non-significant 0.09 Snellen line benefit with corticosteroid treatment. The results of the Bayesian analysis estimated a slightly greater benefit than did the SCUT primary analysis (0.19 lines verses 0.09 lines). Conclusion Indian and North American experts had similar expectations on the effectiveness of corticosteroids in bacterial corneal ulcers; that corticosteroids would markedly improve visual outcomes. Bayesian analysis produced results very similar to those produced by the SCUT primary analysis. The similarity in result is likely due to the large sample size of SCUT and helps validate the results of SCUT. PMID:23171211

Using a fuzzy comprehensive evaluation method to determine product usability: A proposed theoretical framework.

PubMed

Zhou, Ronggang; Chan, Alan H S

2017-01-01

In order to compare existing usability data to ideal goals or to that for other products, usability practitioners have tried to develop a framework for deriving an integrated metric. However, most current usability methods with this aim rely heavily on human judgment about the various attributes of a product, but often fail to take into account of the inherent uncertainties in these judgments in the evaluation process. This paper presents a universal method of usability evaluation by combining the analytic hierarchical process (AHP) and the fuzzy evaluation method. By integrating multiple sources of uncertain information during product usability evaluation, the method proposed here aims to derive an index that is structured hierarchically in terms of the three usability components of effectiveness, efficiency, and user satisfaction of a product. With consideration of the theoretical basis of fuzzy evaluation, a two-layer comprehensive evaluation index was first constructed. After the membership functions were determined by an expert panel, the evaluation appraisals were computed by using the fuzzy comprehensive evaluation technique model to characterize fuzzy human judgments. Then with the use of AHP, the weights of usability components were elicited from these experts. Compared to traditional usability evaluation methods, the major strength of the fuzzy method is that it captures the fuzziness and uncertainties in human judgments and provides an integrated framework that combines the vague judgments from multiple stages of a product evaluation process.

Using expert judgments to explore robust alternatives for forest management under climate change.

PubMed

McDaniels, Timothy; Mills, Tamsin; Gregory, Robin; Ohlson, Dan

2012-12-01

We develop and apply a judgment-based approach to selecting robust alternatives, which are defined here as reasonably likely to achieve objectives, over a range of uncertainties. The intent is to develop an approach that is more practical in terms of data and analysis requirements than current approaches, informed by the literature and experience with probability elicitation and judgmental forecasting. The context involves decisions about managing forest lands that have been severely affected by mountain pine beetles in British Columbia, a pest infestation that is climate-exacerbated. A forest management decision was developed as the basis for the context, objectives, and alternatives for land management actions, to frame and condition the judgments. A wide range of climate forecasts, taken to represent the 10-90% levels on cumulative distributions for future climate, were developed to condition judgments. An elicitation instrument was developed, tested, and revised to serve as the basis for eliciting probabilistic three-point distributions regarding the performance of selected alternatives, over a set of relevant objectives, in the short and long term. The elicitations were conducted in a workshop comprising 14 regional forest management specialists. We employed the concept of stochastic dominance to help identify robust alternatives. We used extensive sensitivity analysis to explore the patterns in the judgments, and also considered the preferred alternatives for each individual expert. The results show that two alternatives that are more flexible than the current policies are judged more likely to perform better than the current alternatives on average in terms of stochastic dominance. The results suggest judgmental approaches to robust decision making deserve greater attention and testing. © 2012 Society for Risk Analysis.

Expert scientific judgment and cancer risk assessment: a pilot study of pharmacokinetic data.

PubMed

Hawkins, N C; Graham, J D

1988-12-01

When high-dose tumor data are extrapolated to low doses, it is typically assumed that the dose of a carcinogen delivered to target cells is proportional to the dose administered to test animals, even at exposure levels below the experimental range. Since pharmacokinetic data are becoming available that in some cases question the validity of this assumption, risk assessors must decide whether to maintain the standard assumption. A pilot study of formaldehyde is reported that was undertaken to demonstrate how expert scientific judgment can help guide a controversial risk assessment where pharmacokinetic data are considered inconclusive. Eight experts on pharmacokinetic data were selected by a formal procedure, and each was interviewed personally using a structured interview protocol. The results suggest that expert scientific opinion is polarized in this case, a situation that risk assessors can respond to with a range of risk characterizations considered biologically plausible by the experts. Convergence of expert opinion is likely in this case of several specific research strategies ar executed in a competent fashion. Elicitation of expert scientific judgment is a promising vehicle for evaluating the quality of pharmacokinetic data, expressing uncertainty in risk assessment, and fashioning a research agenda that offers possible forging of scientific consensus.

A novel method of fuzzy fault tree analysis combined with VB program to identify and assess the risk of coal dust explosions

PubMed Central

Li, Jia; Wang, Deming; Huang, Zonghou

2017-01-01

Coal dust explosions (CDE) are one of the main threats to the occupational safety of coal miners. Aiming to identify and assess the risk of CDE, this paper proposes a novel method of fuzzy fault tree analysis combined with the Visual Basic (VB) program. In this methodology, various potential causes of the CDE are identified and a CDE fault tree is constructed. To overcome drawbacks from the lack of exact probability data for the basic events, fuzzy set theory is employed and the probability data of each basic event is treated as intuitionistic trapezoidal fuzzy numbers. In addition, a new approach for calculating the weighting of each expert is also introduced in this paper to reduce the error during the expert elicitation process. Specifically, an in-depth quantitative analysis of the fuzzy fault tree, such as the importance measure of the basic events and the cut sets, and the CDE occurrence probability is given to assess the explosion risk and acquire more details of the CDE. The VB program is applied to simplify the analysis process. A case study and analysis is provided to illustrate the effectiveness of this proposed method, and some suggestions are given to take preventive measures in advance and avoid CDE accidents. PMID:28793348

A novel method of fuzzy fault tree analysis combined with VB program to identify and assess the risk of coal dust explosions.

PubMed

Wang, Hetang; Li, Jia; Wang, Deming; Huang, Zonghou

2017-01-01

Coal dust explosions (CDE) are one of the main threats to the occupational safety of coal miners. Aiming to identify and assess the risk of CDE, this paper proposes a novel method of fuzzy fault tree analysis combined with the Visual Basic (VB) program. In this methodology, various potential causes of the CDE are identified and a CDE fault tree is constructed. To overcome drawbacks from the lack of exact probability data for the basic events, fuzzy set theory is employed and the probability data of each basic event is treated as intuitionistic trapezoidal fuzzy numbers. In addition, a new approach for calculating the weighting of each expert is also introduced in this paper to reduce the error during the expert elicitation process. Specifically, an in-depth quantitative analysis of the fuzzy fault tree, such as the importance measure of the basic events and the cut sets, and the CDE occurrence probability is given to assess the explosion risk and acquire more details of the CDE. The VB program is applied to simplify the analysis process. A case study and analysis is provided to illustrate the effectiveness of this proposed method, and some suggestions are given to take preventive measures in advance and avoid CDE accidents.

A sunny future: expert elicitation of China's solar photovoltaic technologies

NASA Astrophysics Data System (ADS)

Lam, Long T.; Branstetter, Lee; Azevedo, Inês L.

2018-03-01

China has emerged as the global manufacturing center for solar photovoltaic (PV) products. Chinese firms have entered all stages of the supply chain, producing most of the installed solar modules around the world. Meanwhile, production costs are at record lows. The decisions that Chinese solar producers make today will influence the path for the solar industry and its role towards de-carbonization of global energy systems in the years to come. However, to date, there have been no assessments of the future costs and efficiency of solar PV systems produced by the Chinese PV industry. We perform an expert elicitation to assess the technological and non-technological factors that led to the success of China’s silicon PV industry as well as likely future costs and performance. Experts evaluated key metrics such as efficiency, costs, and commercial viability of 17 silicon and non-silicon solar PV technologies by 2030. Silicon-based technologies will continue to be the mainstream product for large-scale electricity generation application in the near future, with module efficiency reaching as high as 23% and production cost as low as 0.24/W. The levelized cost of electricity for solar will be around 34/MWh, allowing solar PV to be competitive with traditional energy resources like coal. The industry’s future developments may be affected by overinvestment, overcapacity, and singular short-term focus.

A Regional Decision Support Scheme for Pest Risk Analysis in Southeast Asia.

PubMed

Soliman, T; MacLeod, A; Mumford, J D; Nghiem, T P L; Tan, H T W; Papworth, S K; Corlett, R T; Carrasco, L R

2016-05-01

A key justification to support plant health regulations is the ability of quarantine services to conduct pest risk analyses (PRA). Despite the supranational nature of biological invasions and the close proximity and connectivity of Southeast Asian countries, PRAs are conducted at the national level. Furthermore, some countries have limited experience in the development of PRAs, which may result in inadequate phytosanitary responses that put their plant resources at risk to pests vectored via international trade. We review existing decision support schemes for PRAs and, following international standards for phytosanitary measures, propose new methods that adapt existing practices to suit the unique characteristics of Southeast Asia. Using a formal written expert elicitation survey, a panel of regional scientific experts was asked to identify and rate unique traits of Southeast Asia with respect to PRA. Subsequently, an expert elicitation workshop with plant protection officials was conducted to verify the potential applicability of the developed methods. Rich biodiversity, shortage of trained personnel, social vulnerability, tropical climate, agriculture-dependent economies, high rates of land-use change, and difficulties in implementing risk management options were identified as challenging Southeast Asian traits. The developed methods emphasize local Southeast Asian conditions and could help support authorities responsible for carrying out PRAs within the region. These methods could also facilitate the creation of other PRA schemes in low- and middle-income tropical countries. © 2016 Society for Risk Analysis.

Framing of scientific knowledge as a new category of health care research.

PubMed

Salvador-Carulla, Luis; Fernandez, Ana; Madden, Rosamond; Lukersmith, Sue; Colagiuri, Ruth; Torkfar, Ghazal; Sturmberg, Joachim

2014-12-01

The new area of health system research requires a revision of the taxonomy of scientific knowledge that may facilitate a better understanding and representation of complex health phenomena in research discovery, corroboration and implementation. A position paper by an expert group following and iterative approach. 'Scientific evidence' should be differentiated from 'elicited knowledge' of experts and users, and this latter typology should be described beyond the traditional qualitative framework. Within this context 'framing of scientific knowledge' (FSK) is defined as a group of studies of prior expert knowledge specifically aimed at generating formal scientific frames. To be distinguished from other unstructured frames, FSK must be explicit, standardized, based on the available evidence, agreed by a group of experts and subdued to the principles of commensurability, transparency for corroboration and transferability that characterize scientific research. A preliminary typology of scientific framing studies is presented. This typology includes, among others, health declarations, position papers, expert-based clinical guides, conceptual maps, classifications, expert-driven health atlases and expert-driven studies of costs and burden of illness. This grouping of expert-based studies constitutes a different kind of scientific knowledge and should be clearly differentiated from 'evidence' gathered from experimental and observational studies in health system research. © 2014 John Wiley & Sons, Ltd.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Szoka de Valladares, M.R.; Mack, S.

The DOE Hydrogen Program needs to develop criteria as part of a systematic evaluation process for proposal identification, evaluation and selection. The H Scan component of this process provides a framework in which a project proposer can fully describe their candidate technology system and its components. The H Scan complements traditional methods of capturing cost and technical information. It consists of a special set of survey forms designed to elicit information so expert reviewers can assess the proposal relative to DOE specified selection criteria. The Analytic Hierarchy Process (AHP) component of the decision process assembles the management defined evaluation andmore » selection criteria into a coherent multi-level decision construct by which projects can be evaluated in pair-wise comparisons. The AHP model will reflect management`s objectives and it will assist in the ranking of individual projects based on the extent to which each contributes to management`s objectives. This paper contains a detailed description of the products and activities associated with the planning and evaluation process: The objectives or criteria; the H Scan; and The Analytic Hierarchy Process (AHP).« less

Knowledge Structures: Where Can We Find Them?

ERIC Educational Resources Information Center

Law, Nancy

Knowledge elicitation through programming was studied to determine students' intuitive ideas about motion. The subjects, 17-year-old sixth-form science students and 14-year-old third-form students, were asked to write expert systems programs about motion; and their interactions with their own knowledge were observed. The 17-year-old students had…

Equitable Leadership on the Ground: Converging on High-Leverage Practices

ERIC Educational Resources Information Center

Galloway, Mollie K.; Ishimaru, Ann M.

2017-01-01

What would leadership standards look like if developed through a lens and language of equity? We engaged with a group of 40 researchers, practitioners, and community leaders recognized as having expertise on equity in education to address this question. Using a Delphi technique, an approach designed to elicit expert feedback and measure…

Forensic Impact of the Child Sexual Abuse Medical Examination.

ERIC Educational Resources Information Center

Myers, John E. B.

1998-01-01

This commentary on an article (EC 619 279) about research issues at the interface of medicine and law concerning medical evaluation for child sexual abuse focuses on empirically testable questions: (1) the medical history--its accuracy, interviewing issues, and elicitation and preservation of verbal evidence of abuse; and, (2) expert testimony.…

Assessing the impact of climate change on vector-borne viruses in the EU through the elicitation of expert opinion.

PubMed

Gale, P; Brouwer, A; Ramnial, V; Kelly, L; Kosmider, R; Fooks, A R; Snary, E L

2010-02-01

Expert opinion was elicited to undertake a qualitative risk assessment to estimate the current and future risks to the European Union (EU) from five vector-borne viruses listed by the World Organization for Animal Health. It was predicted that climate change will increase the risk of incursions of African horse sickness virus (AHSV), Crimean-Congo haemorrhagic fever virus (CCHFV) and Rift Valley fever virus (RVFV) into the EU from other parts of the world, with African swine fever virus (ASFV) and West Nile virus (WNV) being less affected. Currently the predicted risks of incursion were lowest for RVFV and highest for ASFV. Risks of incursion were considered for six routes of entry (namely vectors, livestock, meat products, wildlife, pets and people). Climate change was predicted to increase the risk of incursion from entry of vectors for all five viruses to some degree, the strongest effects being predicted for AHSV, CCHFV and WNV. This work will facilitate identification of appropriate risk management options in relation to adaptations to climate change.

Epistemic uncertainties and natural hazard risk assessment - Part 1: A review of the issues

NASA Astrophysics Data System (ADS)

Beven, K. J.; Aspinall, W. P.; Bates, P. D.; Borgomeo, E.; Goda, K.; Hall, J. W.; Page, T.; Phillips, J. C.; Rougier, J. T.; Simpson, M.; Stephenson, D. B.; Smith, P. J.; Wagener, T.; Watson, M.

2015-12-01

Uncertainties in natural hazard risk assessment are generally dominated by the sources arising from lack of knowledge or understanding of the processes involved. There is a lack of knowledge about frequencies, process representations, parameters, present and future boundary conditions, consequences and impacts, and the meaning of observations in evaluating simulation models. These are the epistemic uncertainties that can be difficult to constrain, especially in terms of event or scenario probabilities, even as elicited probabilities rationalized on the basis of expert judgements. This paper reviews the issues raised by trying to quantify the effects of epistemic uncertainties. Such scientific uncertainties might have significant influence on decisions that are made for risk management, so it is important to communicate the meaning of an uncertainty estimate and to provide an audit trail of the assumptions on which it is based. Some suggestions for good practice in doing so are made.

Development of an overactive bladder assessment tool (BAT): A potential improvement to the standard bladder diary.

PubMed

Kelleher, Con; Chapple, Christopher; Johnson, Nathan; Payne, Christopher; Homma, Yukio; Hakimi, Zalmai; Siddiqui, Emad; Evans, Christopher; Egan, Shayna; Kopp, Zoe

2018-01-23

To develop a comprehensive patient-reported bladder assessment tool (BAT) for assessing overactive bladder (OAB) symptoms, bother, impacts, and satisfaction with treatment. Subjects were consented and eligibility was confirmed by a recruiting physician; subjects were then scheduled for in-person interviews. For concept elicitation and cognitive interviews, 30 and 20 subjects, respectively, were targeted for recruitment from US sites. All interviews were conducted face-to-face, audio-recorded, transcribed verbatim, anonymized, and analyzed using a qualitative data analysis software program. A draft BAT was created based on the results of the concept elicitation interviews and further revised based on cognitive interviews as well as feedback from an advisory board of clinical and patient-reported outcome (PRO) experts. Nocturia, daytime frequency, and urgency were reported by all subjects (n = 30, 100.0%), and incontinence was reported by most subjects (n = 25, 83.3%). The most frequently reported impacts were waking up to urinate (n = 30, 100.0%), embarrassment/shame (n = 24, 80.0%), stress/anxiety (n = 23, 76.7%), and lack of control (n = 23, 76.7%). Following analysis, item generation, cognitive interviews, and advisory board feedback, the resulting BAT contains four hypothesized domains (symptom frequency, symptom bother, impacts, and satisfaction with treatment) and 17 items with a 7-day recall period. The BAT has been developed in multiple stages with input from both OAB patients and clinical experts following the recommended processes included in the FDA PRO Guidance for Industry. Once fully validated, we believe it will offer a superior alternative to use of the bladder diary and other PROs for monitoring OAB patients in clinical trials and clinical practice. © 2018 Wiley Periodicals, Inc.

Using expert opinion to prioritize impacts of climate change on sea turtles' nesting grounds.

PubMed

Fuentes, M M P B; Cinner, J E

2010-12-01

Managers and conservationists often need to prioritize which impacts from climate change to deal with from a long list of threats. However, data which allows comparison of the relative impact from climatic threats for decision-making is often unavailable. This is the case for the management of sea turtles in the face of climate change. The terrestrial life stages of sea turtles can be negatively impacted by various climatic processes, such as sea level rise, altered cyclonic activity, and increased sand temperatures. However, no study has systematically investigated the relative impact of each of these climatic processes, making it challenging for managers to prioritize their decisions and resources. To address this we offer a systematic method for eliciting expert knowledge to estimate the relative impact of climatic processes on sea turtles' terrestrial reproductive phase. For this we used as an example the world's largest population of green sea turtles and asked 22 scientists and managers to answer a paper based survey with a series of pair-wise comparison matrices that compared the anticipated impacts from each climatic process. Both scientists and managers agreed that increased sand temperature will likely cause the most threat to the reproductive output of the nGBR green turtle population followed by sea level rise, then altered cyclonic activity. The methodology used proved useful to determine the relative impact of the selected climatic processes on sea turtles' reproductive output and provided valuable information for decision-making. Thus, the methodological approach can potentially be applied to other species and ecosystems of management concern. Copyright © 2009 Elsevier Ltd. All rights reserved.

Assessing the capability to experience a 'good death': A qualitative study to directly elicit expert views on a new supportive care measure grounded in Sen's capability approach

PubMed Central

2018-01-01

Background Sen’s capability approach is underspecified; one decision left to those operationalising the approach is how to identify sets of relevant and important capabilities. Sen has suggested that lists be developed for specific policy or research objectives through a process of public reasoning and discussion. Robeyns offers further guidance in support of Sen’s position, suggesting that lists should be explicit, discussed and defended; methods be openly scrutinised; lists be considered both in terms of what is ideal and what is practical (‘generality’); and that lists be exhaustive. Here, the principles suggested by Robeyns are operationalised to facilitate external scrutiny of a list of capabilities identified for use in the evaluation of supportive end of life care. Methods This work started with an existing list of seven capabilities (the ICECAP-SCM), identified as being necessary for a person to experience a good death. Semi-structured qualitative interviews were conducted with 20 experts in economics, psychology, ethics and palliative care, to facilitate external scrutiny of the developed list. Interviews were recorded, transcribed and analysed using constant comparison. Results The seven capabilities were found to encompass concepts identified as important by expert stakeholders (to be exhaustive) and the measure was considered feasible for use with patients receiving care at the end of life. Conclusion The rigorous development of lists of capabilities using both initial participatory approaches with affected population groups, and subsequent assessment by experts, strengthens their democratic basis and may encourage their use in policy contexts. PMID:29466414

Integrated wildfire risk assessment: framework development and application on the Lewis and Clark National Forest in Montana, USA.

PubMed

Thompson, Matthew P; Scott, Joe; Helmbrecht, Don; Calkin, Dave E

2013-04-01

The financial, socioeconomic, and ecological impacts of wildfire continue to challenge federal land management agencies in the United States. In recent years, policymakers and managers have increasingly turned to the field of risk analysis to better manage wildfires and to mitigate losses to highly valued resources and assets (HVRAs). Assessing wildfire risk entails the interaction of multiple components, including integrating wildfire simulation outputs with geospatial identification of HVRAs and the characterization of fire effects to HVRAs. We present an integrated and systematic risk assessment framework that entails 3 primary analytical components: 1) stochastic wildfire simulation and burn probability modeling to characterize wildfire hazard, 2) expert-based modeling to characterize fire effects, and 3) multicriteria decision analysis to characterize preference structures across at-risk HVRAs. We demonstrate application of this framework for a wildfire risk assessment performed on the Little Belts Assessment Area within the Lewis and Clark National Forest in Montana, United States. We devote particular attention to our approach to eliciting and encapsulating expert judgment, in which we: 1) adhered to a structured process for using expert judgment in ecological risk assessment, 2) used as our expert base local resource scientists and fire/fuels specialists who have a direct connection to the specific landscape and HVRAs in question, and 3) introduced multivariate response functions to characterize fire effects to HVRAs that consider biophysical variables beyond fire behavior. We anticipate that this work will further the state of wildfire risk science and will lead to additional application of risk assessment to inform land management planning. Copyright © 2012 SETAC.

Human Emotion Experiences Can Be Predicted on Theoretical Grounds: Evidence from Verbal Labeling

PubMed Central

Scherer, Klaus R.; Meuleman, Ben

2013-01-01

In an effort to demonstrate that the verbal labeling of emotional experiences obeys lawful principles, we tested the feasibility of using an expert system called the Geneva Emotion Analyst (GEA), which generates predictions based on an appraisal theory of emotion. Several thousand respondents participated in an Internet survey that applied GEA to self-reported emotion experiences. Users recalled appraisals of emotion-eliciting events and labeled the experienced emotion with one or two words, generating a massive data set on realistic, intense emotions in everyday life. For a final sample of 5969 respondents we show that GEA achieves a high degree of predictive accuracy by matching a user’s appraisal input to one of 13 theoretically predefined emotion prototypes. The first prediction was correct in 51% of the cases and the overall diagnosis was considered as at least partially correct or appropriate in more than 90% of all cases. These results support a component process model that encourages focused, hypothesis-guided research on elicitation and differentiation, memory storage and retrieval, and categorization and labeling of emotion episodes. We discuss the implications of these results for the study of emotion terms in natural language semantics. PMID:23483988

From user needs to system specifications: multi-disciplinary thematic seminars as a collaborative design method for development of health information systems.

PubMed

Scandurra, I; Hägglund, M; Koch, S

2008-08-01

This paper presents a new multi-disciplinary method for user needs analysis and requirements specification in the context of health information systems based on established theories from the fields of participatory design and computer supported cooperative work (CSCW). Whereas conventional methods imply a separate, sequential needs analysis for each profession, the "multi-disciplinary thematic seminar" (MdTS) method uses a collaborative design process. Application of the method in elderly homecare resulted in prototypes that were well adapted to the intended user groups. Vital information in the points of intersection between different care professions was elicited and a holistic view of the entire care process was obtained. Health informatics-usability specialists and clinical domain experts are necessary to apply the method. Although user needs acquisition can be time-consuming, MdTS was perceived to efficiently identify in-context user needs, and transformed these directly into requirements specifications. Consequently the method was perceived to expedite the entire ICT implementation process.

Building the BIKE: Development and Testing of the Biotechnology Instrument for Knowledge Elicitation (BIKE)

NASA Astrophysics Data System (ADS)

Witzig, Stephen B.; Rebello, Carina M.; Siegel, Marcelle A.; Freyermuth, Sharyn K.; Izci, Kemal; McClure, Bruce

2014-10-01

Identifying students' conceptual scientific understanding is difficult if the appropriate tools are not available for educators. Concept inventories have become a popular tool to assess student understanding; however, traditionally, they are multiple choice tests. International science education standard documents advocate that assessments should be reform based, contain diverse question types, and should align with instructional approaches. To date, no instrument of this type targeting student conceptions in biotechnology has been developed. We report here the development, testing, and validation of a 35-item Biotechnology Instrument for Knowledge Elicitation (BIKE) that includes a mix of question types. The BIKE was designed to elicit student thinking and a variety of conceptual understandings, as opposed to testing closed-ended responses. The design phase contained nine steps including a literature search for content, student interviews, a pilot test, as well as expert review. Data from 175 students over two semesters, including 16 student interviews and six expert reviewers (professors from six different institutions), were used to validate the instrument. Cronbach's alpha on the pre/posttest was 0.664 and 0.668, respectively, indicating the BIKE has internal consistency. Cohen's kappa for inter-rater reliability among the 6,525 total items was 0.684 indicating substantial agreement among scorers. Item analysis demonstrated that the items were challenging, there was discrimination among the individual items, and there was alignment with research-based design principles for construct validity. This study provides a reliable and valid conceptual understanding instrument in the understudied area of biotechnology.

Core principles of evolutionary medicine

PubMed Central

Grunspan, Daniel Z; Nesse, Randolph M; Barnes, M Elizabeth; Brownell, Sara E

2018-01-01

Abstract Background and objectives Evolutionary medicine is a rapidly growing field that uses the principles of evolutionary biology to better understand, prevent and treat disease, and that uses studies of disease to advance basic knowledge in evolutionary biology. Over-arching principles of evolutionary medicine have been described in publications, but our study is the first to systematically elicit core principles from a diverse panel of experts in evolutionary medicine. These principles should be useful to advance recent recommendations made by The Association of American Medical Colleges and the Howard Hughes Medical Institute to make evolutionary thinking a core competency for pre-medical education. Methodology The Delphi method was used to elicit and validate a list of core principles for evolutionary medicine. The study included four surveys administered in sequence to 56 expert panelists. The initial open-ended survey created a list of possible core principles; the three subsequent surveys winnowed the list and assessed the accuracy and importance of each principle. Results Fourteen core principles elicited at least 80% of the panelists to agree or strongly agree that they were important core principles for evolutionary medicine. These principles over-lapped with concepts discussed in other articles discussing key concepts in evolutionary medicine. Conclusions and implications This set of core principles will be helpful for researchers and instructors in evolutionary medicine. We recommend that evolutionary medicine instructors use the list of core principles to construct learning goals. Evolutionary medicine is a young field, so this list of core principles will likely change as the field develops further. PMID:29493660

Core principles of evolutionary medicine: A Delphi study.

PubMed

Grunspan, Daniel Z; Nesse, Randolph M; Barnes, M Elizabeth; Brownell, Sara E

2018-01-01

Evolutionary medicine is a rapidly growing field that uses the principles of evolutionary biology to better understand, prevent and treat disease, and that uses studies of disease to advance basic knowledge in evolutionary biology. Over-arching principles of evolutionary medicine have been described in publications, but our study is the first to systematically elicit core principles from a diverse panel of experts in evolutionary medicine. These principles should be useful to advance recent recommendations made by The Association of American Medical Colleges and the Howard Hughes Medical Institute to make evolutionary thinking a core competency for pre-medical education. The Delphi method was used to elicit and validate a list of core principles for evolutionary medicine. The study included four surveys administered in sequence to 56 expert panelists. The initial open-ended survey created a list of possible core principles; the three subsequent surveys winnowed the list and assessed the accuracy and importance of each principle. Fourteen core principles elicited at least 80% of the panelists to agree or strongly agree that they were important core principles for evolutionary medicine. These principles over-lapped with concepts discussed in other articles discussing key concepts in evolutionary medicine. This set of core principles will be helpful for researchers and instructors in evolutionary medicine. We recommend that evolutionary medicine instructors use the list of core principles to construct learning goals. Evolutionary medicine is a young field, so this list of core principles will likely change as the field develops further.

Using Video to Examine Formative Assessment Practices as Measures of Expertise for Mathematics and Science Teachers

ERIC Educational Resources Information Center

Gotwals, Amelia Wenk; Philhower, Joanne; Cisterna, Dante; Bennett, Steven

2015-01-01

Formative assessment practices, including eliciting a broad range of student ideas, noticing the nuances in students' ideas, using these ideas to guide instruction, and promoting student self-regulation of learning are key components of expert teaching. Given the inherent dialogical nature of formative assessment in the classroom, video can…

42 CFR 34.8 - Reexamination; convening of review boards; expert witnesses; reports.

Code of Federal Regulations, 2012 CFR

2012-10-01

... welfare of the alien or others; or (ii) Having a history of a physical or mental disorder and behavior... alien or others and which behavior is likely to recur or lead to other harmful behavior; or (iii) Being... information elicited from the alien's medical history; (3) Consideration of statements regarding the alien's...

42 CFR 34.8 - Reexamination; convening of review boards; expert witnesses; reports.

Code of Federal Regulations, 2013 CFR

2013-10-01

... welfare of the alien or others; or (ii) Having a history of a physical or mental disorder and behavior... alien or others and which behavior is likely to recur or lead to other harmful behavior; or (iii) Being... information elicited from the alien's medical history; (3) Consideration of statements regarding the alien's...

42 CFR 34.8 - Reexamination; convening of review boards; expert witnesses; reports.

Code of Federal Regulations, 2011 CFR

2011-10-01

... welfare of the alien or others; or (ii) Having a history of a physical or mental disorder and behavior... alien or others and which behavior is likely to recur or lead to other harmful behavior; or (iii) Being... information elicited from the alien's medical history; (3) Consideration of statements regarding the alien's...

Diagnostic games: from adequate formalization of clinical experience to structure discovery.

PubMed

Shifrin, Michael A; Kasparova, Eva I

2008-01-01

A method of obtaining well-founded and reproducible results in clinical decision making is presented. It is based on "diagnostic games", a procedure of elicitation and formalization of experts' knowledge and experience. The use of this procedure allows formulating decision rules in the terms of an adequate language, that are both unambiguous and clinically clear.

Using a fuzzy comprehensive evaluation method to determine product usability: A proposed theoretical framework

PubMed Central

Zhou, Ronggang; Chan, Alan H. S.

2016-01-01

BACKGROUND: In order to compare existing usability data to ideal goals or to that for other products, usability practitioners have tried to develop a framework for deriving an integrated metric. However, most current usability methods with this aim rely heavily on human judgment about the various attributes of a product, but often fail to take into account of the inherent uncertainties in these judgments in the evaluation process. OBJECTIVE: This paper presents a universal method of usability evaluation by combining the analytic hierarchical process (AHP) and the fuzzy evaluation method. By integrating multiple sources of uncertain information during product usability evaluation, the method proposed here aims to derive an index that is structured hierarchically in terms of the three usability components of effectiveness, efficiency, and user satisfaction of a product. METHODS: With consideration of the theoretical basis of fuzzy evaluation, a two-layer comprehensive evaluation index was first constructed. After the membership functions were determined by an expert panel, the evaluation appraisals were computed by using the fuzzy comprehensive evaluation technique model to characterize fuzzy human judgments. Then with the use of AHP, the weights of usability components were elicited from these experts. RESULTS AND CONCLUSIONS: Compared to traditional usability evaluation methods, the major strength of the fuzzy method is that it captures the fuzziness and uncertainties in human judgments and provides an integrated framework that combines the vague judgments from multiple stages of a product evaluation process. PMID:28035943

Cultural adaptation of an intervention to reduce sexual risk behaviors among patients attending a STI clinic in St. Petersburg, Russia.

PubMed

Grau, Lauretta E; Krasnoselskikh, Tatiana V; Shaboltas, Alla V; Skochilov, Roman V; Kozlov, Andrei P; Abdala, Nadia

2013-08-01

Cultural adaptation is an important step in the process of implementing health promotion interventions that, having been proven to be effective in one culture, are being applied in another. This study describes the results of a formative investigation to culturally adapt a STI/HIV risk reduction intervention for use in St. Petersburg, Russia. Analyses of data from brief elicitation interviews, focus groups, community experts, and a pilot test of the adapted intervention identified environmental, cognitive-information processing, and affect-motivation factors that needed to be addressed during the adaptation process. The participant/counselor relationship was adapted to reflect a hierarchical (cf. collaborative) relationship in order to accommodate Russian expectations about patient interactions with healthcare experts. Key skills building activities (e.g., identification of personal risk behaviors, role-playing) were approached gradually or indirectly in order to maintain participants' engagement in the intervention, and close-ended questions were added to assist participants in understanding unfamiliar concepts such as "triggers" and self-efficacy. Information about the prevalence of HIV/STI infections and alcohol use included data specific to St. Petersburg to increase the personal relevance of these materials and messages. Intervention components were tailored to participants' risk reduction and informational needs. No gender differences that would have justified adaptation of the intervention approach or content were noted. Examples of specific adaptations and the key issues to attend to when adapting behavioral interventions for use in Russian clinical settings are discussed.

Computer-assisted knowledge acquisition for hypermedia systems

NASA Technical Reports Server (NTRS)

Steuck, Kurt

1990-01-01

The usage of procedural and declarative knowledge to set up the structure or 'web' of a hypermedia environment is described. An automated knowledge acquisition tool was developed that helps a knowledge engineer elicit and represent an expert's knowledge involved in performing procedural tasks. The tool represents both procedural and prerequisite, declarative knowledge that supports each activity performed by the expert. This knowledge is output and subsequently read by a hypertext scripting language to generate the link between blank, but labeled cards. Each step of the expert's activity and each piece of supporting declarative knowledge is set up as an empty node. An instructional developer can then enter detailed instructional material concerning each step and declarative knowledge into these empty nodes. Other research is also described that facilitates the translation of knowledge from one form into a form more readily useable by computerized systems.

Understanding causality and uncertainty in volcanic observations: An example of forecasting eruptive activity on Soufrière Hills Volcano, Montserrat

NASA Astrophysics Data System (ADS)

Sheldrake, T. E.; Aspinall, W. P.; Odbert, H. M.; Wadge, G.; Sparks, R. S. J.

2017-07-01

Following a cessation in eruptive activity it is important to understand how a volcano will behave in the future and when it may next erupt. Such an assessment can be based on the volcano's long-term pattern of behaviour and insights into its current state via monitoring observations. We present a Bayesian network that integrates these two strands of evidence to forecast future eruptive scenarios using expert elicitation. The Bayesian approach provides a framework to quantify the magmatic causes in terms of volcanic effects (i.e., eruption and unrest). In October 2013, an expert elicitation was performed to populate a Bayesian network designed to help forecast future eruptive (in-)activity at Soufrière Hills Volcano. The Bayesian network was devised to assess the state of the shallow magmatic system, as a means to forecast the future eruptive activity in the context of the long-term behaviour at similar dome-building volcanoes. The findings highlight coherence amongst experts when interpreting the current behaviour of the volcano, but reveal considerable ambiguity when relating this to longer patterns of volcanism at dome-building volcanoes, as a class. By asking questions in terms of magmatic causes, the Bayesian approach highlights the importance of using short-term unrest indicators from monitoring data as evidence in long-term forecasts at volcanoes. Furthermore, it highlights potential biases in the judgements of volcanologists and identifies sources of uncertainty in terms of magmatic causes rather than scenario-based outcomes.

Working memory, reasoning, and expertise in medicine-insights into their relationship using functional neuroimaging.

PubMed

Hruska, Pam; Krigolson, Olav; Coderre, Sylvain; McLaughlin, Kevin; Cortese, Filomeno; Doig, Christopher; Beran, Tanya; Wright, Bruce; Hecker, Kent G

2016-12-01

Clinical reasoning is dependent upon working memory (WM). More precisely, during the clinical reasoning process stored information within long-term memory is brought into WM to facilitate the internal deliberation that affords a clinician the ability to reason through a case. In the present study, we examined the relationship between clinical reasoning and WM while participants read clinical cases with functional magnetic resonance imaging (fMRI). More specifically, we examined the impact of clinical case difficulty (easy, hard) and clinician level of expertise (2nd year medical students, senior gastroenterologists) on neural activity within regions of cortex associated with WM (i.e., the prefrontal cortex) during the reasoning process. fMRI was used to scan ten second-year medical students and ten practicing gastroenterologists while they reasoned through sixteen clinical cases [eight straight forward (easy) and eight complex (hard)] during a single 1-h scanning session. Within-group analyses contrasted the easy and hard cases which were then subsequently utilized for a between-group analysis to examine effects of expertise (novice > expert, expert > novice). Reading clinical cases evoked multiple neural activations in occipital, prefrontal, parietal, and temporal cortical regions in both groups. Importantly, increased activation in the prefrontal cortex in novices for both easy and hard clinical cases suggests novices utilize WM more so than experts during clinical reasoning. We found that clinician level of expertise elicited differential activation of regions of the human prefrontal cortex associated with WM during clinical reasoning. This suggests there is an important relationship between clinical reasoning and human WM. As such, we suggest future models of clinical reasoning take into account that the use of WM is not consistent throughout all clinical reasoning tasks, and that memory structure may be utilized differently based on level of expertise.

A novel expert system for objective masticatory efficiency assessment

PubMed Central

2018-01-01

Most of the tools and diagnosis models of Masticatory Efficiency (ME) are not well documented or severely limited to simple image processing approaches. This study presents a novel expert system for ME assessment based on automatic recognition of mixture patterns of masticated two-coloured chewing gums using a combination of computational intelligence and image processing techniques. The hypotheses tested were that the proposed system could accurately relate specimens to the number of chewing cycles, and that it could identify differences between the mixture patterns of edentulous individuals prior and after complete denture treatment. This study enrolled 80 fully-dentate adults (41 females and 39 males, 25 ± 5 years of age) as the reference population; and 40 edentulous adults (21 females and 19 males, 72 ± 8.9 years of age) for the testing group. The system was calibrated using the features extracted from 400 samples covering 0, 10, 15, and 20 chewing cycles. The calibrated system was used to automatically analyse and classify a set of 160 specimens retrieved from individuals in the testing group in two appointments. The ME was then computed as the predicted number of chewing strokes that a healthy reference individual would need to achieve a similar degree of mixture measured against the real number of cycles applied to the specimen. The trained classifier obtained a Mathews Correlation Coefficient score of 0.97. ME measurements showed almost perfect agreement considering pre- and post-treatment appointments separately (κ ≥ 0.95). Wilcoxon signed-rank test showed that a complete denture treatment for edentulous patients elicited a statistically significant increase in the ME measurements (Z = -2.31, p < 0.01). We conclude that the proposed expert system proved able and reliable to accurately identify patterns in mixture and provided useful ME measurements. PMID:29385165

Regional-scale brine migration along vertical pathways due to CO2 injection - Part 1: The participatory modeling approach

NASA Astrophysics Data System (ADS)

Scheer, Dirk; Konrad, Wilfried; Class, Holger; Kissinger, Alexander; Knopf, Stefan; Noack, Vera

2017-06-01

Saltwater intrusion into potential drinking water aquifers due to the injection of CO2 into deep saline aquifers is one of the potential hazards associated with the geological storage of CO2. Thus, in a site selection process, models for predicting the fate of the displaced brine are required, for example, for a risk assessment or the optimization of pressure management concepts. From the very beginning, this research on brine migration aimed at involving expert and stakeholder knowledge and assessment in simulating the impacts of injecting CO2 into deep saline aquifers by means of a participatory modeling process. The involvement exercise made use of two approaches. First, guideline-based interviews were carried out, aiming at eliciting expert and stakeholder knowledge and assessments of geological structures and mechanisms affecting CO2-induced brine migration. Second, a stakeholder workshop including the World Café format yielded evaluations and judgments of the numerical modeling approach, scenario selection, and preliminary simulation results. The participatory modeling approach gained several results covering brine migration in general, the geological model sketch, scenario development, and the review of the preliminary simulation results. These results were included in revised versions of both the geological model and the numerical model, helping to improve the analysis of regional-scale brine migration along vertical pathways due to CO2 injection.

An ontological model of the practice transformation process.

PubMed

Sen, Arun; Sinha, Atish P

2016-06-01

Patient-centered medical home is defined as an approach for providing comprehensive primary care that facilitates partnerships between individual patients and their personal providers. The current state of the practice transformation process is ad hoc and no methodological basis exists for transforming a practice into a patient-centered medical home. Practices and hospitals somehow accomplish the transformation and send the transformation information to a certification agency, such as the National Committee for Quality Assurance, completely ignoring the development and maintenance of the processes that keep the medical home concept alive. Many recent studies point out that such a transformation is hard as it requires an ambitious whole-practice reengineering and redesign. As a result, the practices suffer change fatigue in getting the transformation done. In this paper, we focus on the complexities of the practice transformation process and present a robust ontological model for practice transformation. The objective of the model is to create an understanding of the practice transformation process in terms of key process areas and their activities. We describe how our ontology captures the knowledge of the practice transformation process, elicited from domain experts, and also discuss how, in the future, that knowledge could be diffused across stakeholders in a healthcare organization. Our research is the first effort in practice transformation process modeling. To build an ontological model for practice transformation, we adopt the Methontology approach. Based on the literature, we first identify the key process areas essential for a practice transformation process to achieve certification status. Next, we develop the practice transformation ontology by creating key activities and precedence relationships among the key process areas using process maturity concepts. At each step, we employ a panel of domain experts to verify the intermediate representations of the ontology. Finally, we implement a prototype of the practice transformation ontology using Protégé. Copyright © 2016 Elsevier Inc. All rights reserved.

Expert assessment concludes negative emissions scenarios may not deliver

NASA Astrophysics Data System (ADS)

Vaughan, Naomi E.; Gough, Clair

2016-09-01

Many integrated assessment models (IAMs) rely on the availability and extensive use of biomass energy with carbon capture and storage (BECCS) to deliver emissions scenarios consistent with limiting climate change to below 2 °C average temperature rise. BECCS has the potential to remove carbon dioxide (CO2) from the atmosphere, delivering ‘negative emissions’. The deployment of BECCS at the scale assumed in IAM scenarios is highly uncertain: biomass energy is commonly used but not at such a scale, and CCS technologies have been demonstrated but not commercially established. Here we present the results of an expert elicitation process that explores the explicit and implicit assumptions underpinning the feasibility of BECCS in IAM scenarios. Our results show that the assumptions are considered realistic regarding technical aspects of CCS but unrealistic regarding the extent of bioenergy deployment, and development of adequate societal support and governance structures for BECCS. The results highlight concerns about the assumed magnitude of carbon dioxide removal achieved across a full BECCS supply chain, with the greatest uncertainty in bioenergy production. Unrealistically optimistic assumptions regarding the future availability of BECCS in IAM scenarios could lead to the overshoot of critical warming limits and have significant impacts on near-term mitigation options.

Assessment of watershed vulnerability to climate change for the Uinta-Wasatch-Cache and Ashley National Forests, Utah

Treesearch

Janine Rice; Tim Bardsley; Pete Gomben; Dustin Bambrough; Stacey Weems; Sarah Leahy; Christopher Plunkett; Charles Condrat; Linda A. Joyce

2017-01-01

Watersheds on the Uinta-Wasatch-Cache and Ashley National Forests provide many ecosystem services, and climate change poses a risk to these services. We developed a watershed vulnerability assessment to provide scientific information for land managers facing the challenge of managing these watersheds. Literature-based information and expert elicitation is used to...

An Investigation of a Parenting Videotape Targeted to Parents of Young Children with Disabilities. Final Report, Phase I.

ERIC Educational Resources Information Center

McKinney, David D.

This feasibility study examined the need and the appropriateness of developing a videotape targeted to parents of young children with disabilities. The study involved a review of the literature, interviews with experts and practitioners, a focus group discussion to elicit the opinions and suggestions of parents of children with disabilities,…

Projected Hg dietary exposure of 3 bird species nesting on a contaminated floodplain (South River, Virginia, USA).

PubMed

Wang, Jincheng; Newman, Michael C

2013-04-01

Dietary Hg exposure was modeled for Carolina wren (Thryothorus ludovicianus), Eastern song sparrow (Melospiza melodia), and Eastern screech owl (Otus asio) nesting on the contaminated South River floodplain (Virginia, USA). Parameterization of Monte-Carlo models required formal expert elicitation to define bird body weight and feeding ecology characteristics because specific information was either unavailable in the published literature or too difficult to collect reliably by field survey. Mercury concentrations and weights for candidate food items were obtained directly by field survey. Simulations predicted the probability that an adult bird during breeding season would ingest specific amounts of Hg during daily foraging and the probability that the average Hg ingestion rate for the breeding season of an adult bird would exceed published rates reported to cause harm to other birds (>100 ng total Hg/g body weight per day). Despite the extensive floodplain contamination, the probabilities that these species' average ingestion rates exceeded the threshold value were all <0.01. Sensitivity analysis indicated that overall food ingestion rate was the most important factor determining projected Hg ingestion rates. Expert elicitation was useful in providing sufficiently reliable information for Monte-Carlo simulation. Copyright © 2013 SETAC.

Initial development of a patient-reported outcome measure of experience with cognitive impairment associated with schizophrenia.

PubMed

Welch, Lisa C; Trudeau, Jeremiah J; Silverstein, Steven M; Sand, Michael; Henderson, David C; Rosen, Raymond C

2017-01-01

Cognitive impairment is a serious, often distressing aspect of schizophrenia that affects patients' day-to-day lives. Although several interview-based instruments exist to assess cognitive functioning, a reliable measure developed based on the experiences of patients facing cognitive difficulties is needed to complement the objective performance-based assessments. The present article describes the initial development of a patient-reported outcome (PRO) measure to assess the subjective experience of cognitive impairment among patients with schizophrenia, the Patient-Reported Experience of Cognitive Impairment in Schizophrenia (PRECIS). The phases of development included the construction of a conceptual model based on the existing knowledge and two sets of qualitative interviews with patients: 1) concept elicitation interviews to ensure face and content validity from the perspective of people with schizophrenia and 2) cognitive debriefing of the initial item pool. Input from experts was elicited throughout the process. The initial conceptual model included seven domains. The results from concept elicitation interviews (n=80) supported these domains but yielded substantive changes to concepts within domains and to terminology. Based on these results, an initial pool of 53 items was developed to reflect the most common descriptions and languages used by the study participants. Cognitive debriefing interviews (n=22) resulted in the removal of 18 items and modification of 22 other items. The remaining 35 items represented 23 concepts within six domains plus two items assessing bother. The draft PRO measure is currently undergoing psychometric testing as a precursor to broad-based clinical and research use.

ELICITED EXPERT PERCEPTIONS FOR CLIMATE CHANGE RISKS AND ADAPTATION IN AGRICULTURE AND FOOD PRODUCTION THROUGH MENTAL MODELS APPROACH

NASA Astrophysics Data System (ADS)

Suda, Eiko; Kubota, Hiromi; Baba, Kenshi; Hijioka, Yasuaki; Takahashi, Kiyoshi; Hanasaki, Naota

Impacts of climate change have become obvious in agriculture and food production in Japan these days, and researches to adapt to their risks have been conducted as a key effort to cope with the climate change. Numerous scientific findings on climate change impacts have been presented so far; however, prospective risks to be adapted to and their management in the context of individual on-site situations have not been investigated in detail. The structure of climate change risks and their management vary depending on geographical and social features in the regions where the adaptation options should be applied; therefore, a practical adaptation strategy should consider actual on-site situations. This study intended to clarify climate change risks to be adapted to in the Japanese agricultural sector, and factors to be considered in adaptation options, for encouragement of decision-making on adaptation implementation in the field. Semi-structured individual interviews have been conducted with 9 multidisciplinary experts engaging in climate change impacts research in agricultural production, economics, engineering, policy, and so on. Based on the results of the interviews, and the latest literatures available for risk assessment and adaptation, an expert mental model including their perceptions which cover the process from climate change impacts assessment to adaptation has been developed. The prospective risks, adaptation options, and issues to be examined to progress the development of practical and effective adaptation options and to support individual or social decision-making, have been shown on the developed expert mental model. It is the basic information for developing social communication and stakeholders cooperations in climate change adaptation strategies in agriculture and food production in Japan.

Assessing the structure of non-routine decision processes in Airline Operations Control.

PubMed

Richters, Floor; Schraagen, Jan Maarten; Heerkens, Hans

2016-03-01

Unfamiliar severe disruptions challenge Airline Operations Control professionals most, as their expertise is stretched to its limits. This study has elicited the structure of Airline Operations Control professionals' decision process during unfamiliar disruptions by mapping three macrocognitive activities on the decision ladder: sensemaking, option evaluation and action planning. The relationship between this structure and decision quality was measured. A simulated task was staged, based on which think-aloud protocols were obtained. Results show that the general decision process structure resembles the structure of experts working under routine conditions, in terms of the general structure of the macrocognitive activities, and the rule-based approach used to identify options and actions. Surprisingly, high quality of decision outcomes was found to relate to the use of rule-based strategies. This implies that successful professionals are capable of dealing with unfamiliar problems by reframing them into familiar ones, rather than to engage in knowledge-based processing. Practitioner Summary: We examined the macrocognitive structure of Airline Operations Control professionals' decision process during a simulated unfamiliar disruption in relation to decision quality. Results suggest that successful professionals are capable of dealing with unfamiliar problems by reframing them into familiar ones, rather than to engage in knowledge-based processing.

The expert explorer: a tool for hospital data visualization and adverse drug event rules validation.

PubMed

Băceanu, Adrian; Atasiei, Ionuţ; Chazard, Emmanuel; Leroy, Nicolas

2009-01-01

An important part of adverse drug events (ADEs) detection is the validation of the clinical cases and the assessment of the decision rules to detect ADEs. For that purpose, a software called "Expert Explorer" has been designed by Ideea Advertising. Anonymized datasets have been extracted from hospitals into a common repository. The tool has 3 main features. (1) It can display hospital stays in a visual and comprehensive way (diagnoses, drugs, lab results, etc.) using tables and pretty charts. (2) It allows designing and executing dashboards in order to generate knowledge about ADEs. (3) It finally allows uploading decision rules obtained from data mining. Experts can then review the rules, the hospital stays that match the rules, and finally give their advice thanks to specialized forms. Then the rules can be validated, invalidated, or improved (knowledge elicitation phase).

Delphi Study to Determine Rehabilitation Research Priorities for Older Adults With Cancer.

PubMed

Lyons, Kathleen Doyle; Radomski, Mary Vining; Alfano, Catherine M; Finkelstein, Marsha; Sleight, Alix G; Marshall, Timothy F; McKenna, Raymond; Fu, Jack B

2017-05-01

To solicit expert opinions and develop consensus around the research that is needed to improve cancer rehabilitation for older adults. Delphi methods provided a structured process to elicit and prioritize research questions from national experts. National, Web-based survey. Members (N=32) of the American Congress of Rehabilitation Medicine completed at least 1 of 3 investigator-developed surveys. Not applicable. In the first survey, participants identified up to 5 research questions that needed to be answered to improve cancer rehabilitation for older adults. In 2 subsequent surveys, participants viewed the compilation of questions, rated the importance of each question, and identified the 5 most important questions. This generated priority scores for each question. Consensus scores were created to describe the degree of agreement around the priority of each question. Highest priority research concerns the epidemiology and measurement of function and disability in older adult cancer survivors; the effects of cancer rehabilitation interventions on falls, disability, participation, survival, costs, quality of care, and health care utilization; and testing models of care that facilitate referrals from oncology to rehabilitation providers as part of coordinated, multicomponent care. A multipronged approach is needed to fill these gaps, including targeted funding opportunities developed with an advisory panel of cancer rehabilitation experts, development of a research network to facilitate novel collaborations and grant proposals, and coordinated efforts of clinical groups to advocate for funding, practice change, and policy change. Copyright © 2016 American Congress of Rehabilitation Medicine. Published by Elsevier Inc. All rights reserved.

Using process elicitation and validation to understand and improve chemotherapy ordering and delivery.

PubMed

Mertens, Wilson C; Christov, Stefan C; Avrunin, George S; Clarke, Lori A; Osterweil, Leon J; Cassells, Lucinda J; Marquard, Jenna L

2012-11-01

Chemotherapy ordering and administration, in which errors have potentially severe consequences, was quantitatively and qualitatively evaluated by employing process formalism (or formal process definition), a technique derived from software engineering, to elicit and rigorously describe the process, after which validation techniques were applied to confirm the accuracy of the described process. The chemotherapy ordering and administration process, including exceptional situations and individuals' recognition of and responses to those situations, was elicited through informal, unstructured interviews with members of an interdisciplinary team. The process description (or process definition), written in a notation developed for software quality assessment purposes, guided process validation (which consisted of direct observations and semistructured interviews to confirm the elicited details for the treatment plan portion of the process). The overall process definition yielded 467 steps; 207 steps (44%) were dedicated to handling 59 exceptional situations. Validation yielded 82 unique process events (35 new expected but not yet described steps, 16 new exceptional situations, and 31 new steps in response to exceptional situations). Process participants actively altered the process as ambiguities and conflicts were discovered by the elicitation and validation components of the study. Chemotherapy error rates declined significantly during and after the project, which was conducted from October 2007 through August 2008. Each elicitation method and the subsequent validation discussions contributed uniquely to understanding the chemotherapy treatment plan review process, supporting rapid adoption of changes, improved communication regarding the process, and ensuing error reduction.

[Patients' Priorities in the Treatment of Neuroendocrine Tumours: An Analytical Hierarchy Process].

PubMed

Mühlbacher, A C; Juhnke, C; Kaczynski, A

2016-10-01

Background: Neuroendocrine tumours (NET) are relatively rare, usually slow-growing malignant tumours. So far there are no data on the patient preferences/priorities regarding the therapy for NET. This empirical study aimed at the elicitation of patient priorities in the drug treatment of NET. Method: Qualitative patient interviews (N=9) were conducted. To elicit the patient's perspective regarding various treatment aspects of NET a self-administered questionnaire using the Analytical Hierarchy Process (AHP) was developed. The data collection was carried out using paper questionnaires supported by an item response system in a group discussion. To evaluate the patient-relevant outcomes, the eigenvector method was applied. Results: N=24 patients, experts and relatives participated in the AHP survey. In the AHP all respondents had clear priorities for all considered attributes. The attribute "overall survival" was the most significant feature of a drug therapy for all respondents. As in the qualitative interviews, "efficacy attributes" dominated the side effects in the AHP as well. The evaluation of all participants thus showed the attributes "overall survival" (Wglobal:0.418), "progression-free survival" (Wglobal:0.172) and "response to treatment" (Wglobal:0.161) to be most relevant. "Occurrence of abdominal pain" (Wglobal:0.051) was ranked sixth, with "tiredness/fatigue" and "risk of a hypoglycaemia" (Wglobal:0.034) in a shared seventh place. Conclusion: The results thus provide evidence about how much influence a treatment capacity has on therapeutic decisions. Using the AHP major aspects of drug therapy from the perspective of those affected were captured, and positive and negative therapeutic properties could be related against each other. Based on the assessment of the patient's perspective further investigation must elicit patient preferences for NET drug therapy. In the context of a discrete choice experiment or another choice-based method of preference measurement, the results obtained here can be validated and the therapeutic features weighted according to their preferability. © Georg Thieme Verlag KG Stuttgart · New York.

Applying Subject Matter Expertise (SME) Elicitation Techniques to TRAC Studies

DTIC Science & Technology

2014-09-30

prioritisation, budgeting and resource allocation with multi-criteria decision analysis and decision conferencing ”. English. In: Annals of Operations... electronically . Typically, in responding to survey items, experts are not expected to elaborate beyond providing responses in the format requested in the...between them, however irrelevant to probability Kynn and Ayyub.84 For example, an electronic jamming device might disrupt a cell phone signal at certain

76 FR 29762 - Agency Information Collection Activities; Submission for Office of Management and Budget Review...

Federal Register 2010, 2011, 2012, 2013, 2014

2011-05-23

... questions in the survey are designed to elicit expert opinions and clarify areas of agreement and... Federal Register for comment prior to finalizing them for the pretest and the actual survey. (CVM's... respondent responses (in hours) \\2\\ Pretest 5 1 5 20/60 1.65 Survey 100 1 100 20/60 33 Total 34.65 \\1\\ There...

Learning from the Experts: A Thematic Analysis of Parent's Experiences of Attending a Therapeutic Group for Parents of Children with Learning Disabilities and Challenging Behaviour

ERIC Educational Resources Information Center

Thompson-Janes, Emily; Brice, Samuel; McElroy, Rebecca; Abbott, Jennie; Ball, June

2016-01-01

The Confident Parenting group is a therapeutic group for parents of children with learning disabilities and challenging behaviour, which is informed by the principles of behavioural theory and acceptance and commitment therapy. Parent's experiences of the group were elicited through participation in a large focus group which followed a…

Building Interagency Partnerships Curriculum: Instructor’s Guide

DTIC Science & Technology

2016-03-01

includes instructor-led components, documentary -style footage of subject matter experts, and true stories from the field to elicit reflection and discussion...context. The documentary -style films included in the curriculum are based on an analysis of interviews with military personnel, U.S. government... management tasks while collaborating with interagency partners. 29 Lesson 1: Boundary-Spanning OBJECTIVES The student will be: • Introduced to the

Nobody Is Perfect: ERP Effects Prior to Performance Errors in Musicians Indicate Fast Monitoring Processes

PubMed Central

Maidhof, Clemens; Rieger, Martina; Prinz, Wolfgang; Koelsch, Stefan

2009-01-01

Background One central question in the context of motor control and action monitoring is at what point in time errors can be detected. Previous electrophysiological studies investigating this issue focused on brain potentials elicited after erroneous responses, mainly in simple speeded response tasks. In the present study, we investigated brain potentials before the commission of errors in a natural and complex situation. Methodology/Principal Findings Expert pianists bimanually played scales and patterns while the electroencephalogram (EEG) was recorded. Event-related potentials (ERPs) were computed for correct and incorrect performances. Results revealed differences already 100 ms prior to the onset of a note (i.e., prior to auditory feedback). We further observed that erroneous keystrokes were delayed in time and pressed more slowly. Conclusions Our data reveal neural mechanisms in musicians that are able to detect errors prior to the execution of erroneous movements. The underlying mechanism probably relies on predictive control processes that compare the predicted outcome of an action with the action goal. PMID:19337379

Which uncertainty? Using expert elicitation and expected value of information to design an adaptive program

USGS Publications Warehouse

Runge, Michael C.; Converse, Sarah J.; Lyons, James E.

2011-01-01

Natural resource management is plagued with uncertainty of many kinds, but not all uncertainties are equally important to resolve. The promise of adaptive management is that learning in the short-term will improve management in the long-term; that promise is best kept if the focus of learning is on those uncertainties that most impede achievement of management objectives. In this context, an existing tool of decision analysis, the expected value of perfect information (EVPI), is particularly valuable in identifying the most important uncertainties. Expert elicitation can be used to develop preliminary predictions of management response under a series of hypotheses, as well as prior weights for those hypotheses, and the EVPI can be used to determine how much management could improve if uncertainty was resolved. These methods were applied to management of whooping cranes (Grus americana), an endangered migratory bird that is being reintroduced in several places in North America. The Eastern Migratory Population of whooping cranes had exhibited almost no successful reproduction through 2009. Several dozen hypotheses can be advanced to explain this failure, and many of them lead to very different management responses. An expert panel articulated the hypotheses, provided prior weights for them, developed potential management strategies, and made predictions about the response of the population to each strategy under each hypothesis. Multi-criteria decision analysis identified a preferred strategy in the face of uncertainty, and analysis of the expected value of information identified how informative each strategy could be. These results provide the foundation for design of an adaptive management program.

The use of cognitive task analysis to reveal the instructional limitations of experts in the teaching of procedural skills.

PubMed

Sullivan, Maura E; Yates, Kenneth A; Inaba, Kenji; Lam, Lydia; Clark, Richard E

2014-05-01

Because of the automated nature of knowledge, experts tend to omit information when describing a task. A potential solution is cognitive task analysis (CTA). The authors investigated the percentage of knowledge experts omitted when teaching a cricothyrotomy to determine the percentage of additional knowledge gained during a CTA interview. Three experts were videotaped teaching a cricothyrotomy in 2010 at the University of Southern California. After transcription, they participated in CTA interviews for the same procedure. Three additional surgeons were recruited to perform a CTA for the procedure, and a "gold standard" task list was created. Transcriptions from the teaching sessions were compared with the task list to identify omitted steps (both "what" and "how" to do). Transcripts from the CTA interviews were compared against the task list to determine the percentage of knowledge articulated by each expert during the initial "free recall" (unprompted) phase of the CTA interview versus the amount of knowledge gained by using CTA elicitation techniques (prompted). Experts omitted an average of 71% (10/14) of clinical knowledge steps, 51% (14/27) of action steps, and 73% (3.6/5) of decision steps. For action steps, experts described "how to do it" only 13% (3.6/27) of the time. The average number of steps that were described increased from 44% (20/46) when unprompted to 66% (31/46) when prompted. This study supports previous research that experts unintentionally omit knowledge when describing a procedure. CTA is a useful method to extract automated knowledge and augment expert knowledge recall during teaching.

Developing a competency framework for U.S. state food and feed testing laboratory personnel.

PubMed

Kaml, Craig; Weiss, Christopher C; Dezendorf, Paul; Ishida, Maria; Rice, Daniel H; Klein, Ron; Salfinger, Yvonne

2014-01-01

A competency-based training curriculum framework for U.S. state food and feed testing laboratories personnel is being developed by the International Food Protection Training Institute (IFPTI) and three partners. The framework will help laboratories catalog existing training courses/modules, identify training gaps, inform training curricula, and create career-spanning professional development learning paths, ensuring consistent performance expectations and increasing confidence in shared test results. Ultimately, the framework will aid laboratories in meeting the requirements of ISO/IEC 17025 (2005) international accreditation and the U.S. Food Safety Modernization Act (U.S. Public Law 111-353). In collaboration with the Association of Food and Drug Officials, the Association of Public Health Laboratories, and the Association of American Feed Control Officials, IFPTI is carrying out the project in two phases. In 2013, an expert panel of seven subject matter experts developed competency and curriculum frameworks for five professional levels (entry, mid-level, expert, supervisor/manager, and senior administration) across four competency domains (technical, communication, programmatic, and leadership) including approximately 80 competencies. In 2014 the expert panel will elicit feedback from peers and finalize the framework.

Fiber Optic Tamper Indicating Enclosure (TIE); A Case Study in Authentication

DOE Office of Scientific and Technical Information (OSTI.GOV)

Anheier, Norman C.; Benz, Jacob M.; Tanner, Jennifer E.

2015-07-15

A robust fiber optic-based tamper-indicating enclosure (TIE) has been developed by PNNL through funding by the National Nuclear Security Administration Office of Nuclear Verification over the past few years. The objective of this work is to allow monitors to have confidence in both the authenticity and integrity of the TIE and the monitoring equipment inside, throughout the time it may be located at a host facility. Incorporating authentication features into the design were the focus of fiscal year 2014 development efforts. Throughout the development process, modifications have been made to the physical TIE design based on lessons learned via exercisesmore » and expert elicitation. The end result is a robust and passive TIE which can be utilized to protect monitoring party equipment left in a host facility.« less

V&V framework

DOE Office of Scientific and Technical Information (OSTI.GOV)

Hills, Richard G.; Maniaci, David Charles; Naughton, Jonathan W.

2015-09-01

A Verification and Validation (V&V) framework is presented for the development and execution of coordinated modeling and experimental program s to assess the predictive capability of computational models of complex systems through focused, well structured, and formal processes.The elements of the framework are based on established V&V methodology developed by various organizations including the Department of Energy, National Aeronautics and Space Administration, the American Institute of Aeronautics and Astronautics, and the American Society of Mechanical Engineers. Four main topics are addressed: 1) Program planning based on expert elicitation of the modeling physics requirements, 2) experimental design for model assessment, 3)more » uncertainty quantification for experimental observations and computational model simulations, and 4) assessment of the model predictive capability. The audience for this document includes program planners, modelers, experimentalist, V &V specialist, and customers of the modeling results.« less

Patients' preferences for primary health care - a systematic literature review of discrete choice experiments.

PubMed

Kleij, Kim-Sarah; Tangermann, Ulla; Amelung, Volker E; Krauth, Christian

2017-07-11

Primary care is a key element of health care systems and addresses the main health problems of the population. Due to the demographic change, primary care even gains in importance. The knowledge of the patients' preferences can help policy makers as well as physicians to set priorities in their effort to make health care delivery more responsive to patients' needs. Our objective was to describe which aspects of primary care were included in preference studies and which of them were the most preferred aspects. In order to elicit the preferences for primary care, a systematic literature search was conducted. Two researchers searched three electronic databases (PubMed, Scopus, and PsycINFO) and conducted a narrative synthesis. Inclusion criteria were: focus on primary health care delivery, discrete choice experiment as elicitation method, and studies published between 2006 and 2015 in English language. We identified 18 studies that elicited either the patients' or the population's preferences for primary care based on a discrete choice experiment. Altogether the studies used 16 structure attributes, ten process attributes and four outcome attributes. The most commonly applied structure attribute was "Waiting time till appointment", the most frequently used process attribute was "Shared decision making / professional's attention paid to your views". "Receiving the 'best' treatment" was the most commonly applied outcome attribute. Process attributes were most often the ones of highest importance for patients or the population. The attributes and attribute levels used in the discrete choice experiments were identified by literature research, qualitative research, expert interviews, or the analysis of policy documents. The results of the DCE studies show different preferences for primary health care. The diversity of the results may have several reasons, such as the method of analysis, the selection procedure of the attributes and their levels or the specific research question of the study. As the results of discrete choice experiments depend on many different factors, it is important for a better comprehensibility of the studies to transparently report the steps undertaken in a study as well as the interim results regarding the identification of attributes and levels.

Risk Associated with the Release of Wolbachia-Infected Aedes aegypti Mosquitoes into the Environment in an Effort to Control Dengue.

PubMed

Murray, Justine V; Jansen, Cassie C; De Barro, Paul

2016-01-01

In an effort to eliminate dengue, a successful technology was developed with the stable introduction of the obligate intracellular bacteria Wolbachia pipientis into the mosquito Aedes aegypti to reduce its ability to transmit dengue fever due to life shortening and inhibition of viral replication effects. An analysis of risk was required before considering release of the modified mosquito into the environment. Expert knowledge and a risk assessment framework were used to identify risk associated with the release of the modified mosquito. Individual and group expert elicitation was performed to identify potential hazards. A Bayesian network (BN) was developed to capture the relationship between hazards and the likelihood of events occurring. Risk was calculated from the expert likelihood estimates populating the BN and the consequence estimates elicited from experts. The risk model for "Don't Achieve Release" provided an estimated 46% likelihood that the release would not occur by a nominated time but generated an overall risk rating of very low. The ability to obtain compliance had the greatest influence on the likelihood of release occurring. The risk model for "Cause More Harm" provided a 12.5% likelihood that more harm would result from the release, but the overall risk was considered negligible. The efficacy of mosquito management had the most influence, with the perception that the threat of dengue fever had been eliminated, resulting in less household mosquito control, and was scored as the highest ranked individual hazard (albeit low risk). The risk analysis was designed to incorporate the interacting complexity of hazards that may affect the release of the technology into the environment. The risk analysis was a small, but important, implementation phase in the success of this innovative research introducing a new technology to combat dengue transmission in the environment.

Comparison of two multi-criteria decision techniques for eliciting treatment preferences in people with neurological disorders.

PubMed

Ijzerman, Maarten J; van Til, Janine A; Snoek, Govert J

2008-12-01

To present and compare two multi-criteria decision techniques (analytic hierarchy process [AHP] and conjoint analysis [CA]) for eliciting preferences in patients with cervical spinal cord injury (SCI) who are eligible for surgical augmentation of hand function, either with or without implantation of a neuroprosthesis. The methods were compared in respect to attribute weights, overall preference, and practical experiences. Two previously designed and administered multi-criteria decision surveys in patients with SCI were compared and further analysed. Attributes and their weights in the AHP experiment were determined by an expert panel, followed by determination of the weights in the patient group. Attributes for the CA were selected and validated using an expert panel, piloted in six patients with SCI and subsequently administered to the same group of patients as participated in the AHP experiment. Both experiments showed the importance of non-outcome-related factors such as inpatient stay and number of surgical procedures. In particular, patients were less concerned with clinical outcomes in actual decision making. Overall preference in both the AHP and CA was in favor of tendon reconstruction (0.6 vs 0.4 for neuroprosthetic implantation). Both methods were easy to apply, but AHP was less easily explained and understood. Both the AHP and CA methods produced similar outcomes, which may have been caused by the obvious preferences of patients. CA may be preferred because of the holistic approach of considering all treatment attributes simultaneously and, hence, its power in simulating real market decisions. On the other hand, the AHP method is preferred as a hands-on, easy-to-implement task with immediate feedback to the respondent. This flexibility allows AHP to be used in shared decision making. However, the way the technique is composed results in many inconsistencies. Patients preferred CA but complained about the number of choice tasks.

Mechanisms and risk of cumulative impacts to coastal ecosystem services: An expert elicitation approach.

PubMed

Singh, Gerald G; Sinner, Jim; Ellis, Joanne; Kandlikar, Milind; Halpern, Benjamin S; Satterfield, Terre; Chan, Kai M A

2017-09-01

Coastal environments are some of the most populated on Earth, with greater pressures projected in the future. Managing coastal systems requires the consideration of multiple uses, which both benefit from and threaten multiple ecosystem services. Thus understanding the cumulative impacts of human activities on coastal ecosystem services would seem fundamental to management, yet there is no widely accepted approach for assessing these. This study trials an approach for understanding the cumulative impacts of anthropogenic change, focusing on Tasman and Golden Bays, New Zealand. Using an expert elicitation procedure, we collected information on three aspects of cumulative impacts: the importance and magnitude of impacts by various activities and stressors on ecosystem services, and the causal processes of impact on ecosystem services. We assessed impacts to four ecosystem service benefits - fisheries, shellfish aquaculture, marine recreation and existence value of biodiversity-addressing three main research questions: (1) how severe are cumulative impacts on ecosystem services (correspondingly, what potential is there for restoration)?; (2) are threats evenly distributed across activities and stressors, or do a few threats dominate?; (3) do prominent activities mainly operate through direct stressors, or do they often exacerbate other impacts? We found (1) that despite high uncertainty in the threat posed by individual stressors and impacts, total cumulative impact is consistently severe for all four ecosystem services. (2) A subset of drivers and stressors pose important threats across the ecosystem services explored, including climate change, commercial fishing, sedimentation and pollution. (3) Climate change and commercial fishing contribute to prominent indirect impacts across ecosystem services by exacerbating regional impacts, namely sedimentation and pollution. The prevalence and magnitude of these indirect, networked impacts highlights the need for approaches like this to understand mechanisms of impact, in order to develop strategies to manage them. Copyright © 2017 Elsevier Ltd. All rights reserved.

Concurrent Data Elicitation Procedures, Processes, and the Early Stages of L2 Learning: A Critical Overview

ERIC Educational Resources Information Center

Leow, Ronald P.; Grey, Sarah; Marijuan, Silvia; Moorman, Colleen

2014-01-01

Given the current methodological interest in eliciting direct data on the cognitive processes L2 learners employ as they interact with L2 data during the early stages of the learning process, this article takes a critical and comparative look at three concurrent data elicitation procedures currently employed in the SLA literature: Think aloud (TA)…

Action specificity increases anticipatory performance and the expert advantage in natural interceptive tasks.

PubMed

Mann, David L; Abernethy, Bruce; Farrow, Damian

2010-09-01

The relationship between perception-action coupling and anticipatory skill in an interceptive task was examined using an in-situ temporal occlusion paradigm. Skilled and novice cricket batsmen were required to predict the direction of balls bowled towards them under four counterbalanced response conditions of increasing perception-action coupling: (i) verbal, (ii) lower-body movement only, (iii) full-body movement (no bat), and (iv) full-body movement with bat (i.e., the usual batting response). Skilled but not novice anticipation was found to improve as a function of coupling when responses were based on either no ball-flight, or early ball-flight information, with a response requiring even the lowest degree of body movement found to enhance anticipation when compared to a verbal prediction. Most importantly, a full-body movement using a bat elicited greater anticipation than an equivalent movement with no bat. This result highlights the important role that the requirement and/or opportunity to make bat-ball interception may play in eliciting skill differences for anticipation. Results verify the importance of using experimental conditions and task demands that closely reflect the natural performance environment in order to reveal the full nature of the expert advantage. 2010. Published by Elsevier B.V. All rights reserved.

The conceptual foundation of environmental decision support.

PubMed

Reichert, Peter; Langhans, Simone D; Lienert, Judit; Schuwirth, Nele

2015-05-01

Environmental decision support intends to use the best available scientific knowledge to help decision makers find and evaluate management alternatives. The goal of this process is to achieve the best fulfillment of societal objectives. This requires a careful analysis of (i) how scientific knowledge can be represented and quantified, (ii) how societal preferences can be described and elicited, and (iii) how these concepts can best be used to support communication with authorities, politicians, and the public in environmental management. The goal of this paper is to discuss key requirements for a conceptual framework to address these issues and to suggest how these can best be met. We argue that a combination of probability theory and scenario planning with multi-attribute utility theory fulfills these requirements, and discuss adaptations and extensions of these theories to improve their application for supporting environmental decision making. With respect to (i) we suggest the use of intersubjective probabilities, if required extended to imprecise probabilities, to describe the current state of scientific knowledge. To address (ii), we emphasize the importance of value functions, in addition to utilities, to support decisions under risk. We discuss the need for testing "non-standard" value aggregation techniques, the usefulness of flexibility of value functions regarding attribute data availability, the elicitation of value functions for sub-objectives from experts, and the consideration of uncertainty in value and utility elicitation. With respect to (iii), we outline a well-structured procedure for transparent environmental decision support that is based on a clear separation of scientific prediction and societal valuation. We illustrate aspects of the suggested methodology by its application to river management in general and with a small, didactical case study on spatial river rehabilitation prioritization. Copyright © 2015 The Authors. Published by Elsevier Ltd.. All rights reserved.

Explosion probability of unexploded ordnance: expert beliefs.

PubMed

MacDonald, Jacqueline Anne; Small, Mitchell J; Morgan, M G

2008-08-01

This article reports on a study to quantify expert beliefs about the explosion probability of unexploded ordnance (UXO). Some 1,976 sites at closed military bases in the United States are contaminated with UXO and are slated for cleanup, at an estimated cost of $15-140 billion. Because no available technology can guarantee 100% removal of UXO, information about explosion probability is needed to assess the residual risks of civilian reuse of closed military bases and to make decisions about how much to invest in cleanup. This study elicited probability distributions for the chance of UXO explosion from 25 experts in explosive ordnance disposal, all of whom have had field experience in UXO identification and deactivation. The study considered six different scenarios: three different types of UXO handled in two different ways (one involving children and the other involving construction workers). We also asked the experts to rank by sensitivity to explosion 20 different kinds of UXO found at a case study site at Fort Ord, California. We found that the experts do not agree about the probability of UXO explosion, with significant differences among experts in their mean estimates of explosion probabilities and in the amount of uncertainty that they express in their estimates. In three of the six scenarios, the divergence was so great that the average of all the expert probability distributions was statistically indistinguishable from a uniform (0, 1) distribution-suggesting that the sum of expert opinion provides no information at all about the explosion risk. The experts' opinions on the relative sensitivity to explosion of the 20 UXO items also diverged. The average correlation between rankings of any pair of experts was 0.41, which, statistically, is barely significant (p= 0.049) at the 95% confidence level. Thus, one expert's rankings provide little predictive information about another's rankings. The lack of consensus among experts suggests that empirical studies are needed to better understand the explosion risks of UXO.

Assessing Residents' Readiness for OR Autonomy: A Qualitative Descriptive Study of Expert Surgical Teachers' Best Practices.

PubMed

Chen, Xiaodong Phoenix; Sullivan, Amy M; Alseidi, Adnan; Kwakye, Gifty; Smink, Douglas S

Providing resident autonomy in the operating room (OR) is one of the major challenges for surgical educators today. The purpose of this study was to explore what approaches expert surgical teachers use to assess residents' readiness for autonomy in the OR. We particularly focused on the assessments that experts make prior to conducting the surgical time-out. We conducted semistructured in-depth interviews with expert surgical teachers from March 2016 to September 2016. Purposeful sampling and snowball sampling were applied to identify and recruit expert surgical teachers from general surgery residency programs across the United States to represent a range of clinical subspecialties. All interviews were audio-recorded, deidentified, and transcribed. We applied the Framework Method of content analysis, discussed and reached final consensus on the themes. We interviewed 15 expert teachers from 9 institutions. The majority (13/15) were Program or Associate Program Directors; 47% (7/15) primarily performed complex surgical operations (e.g., endocrine surgery). Five themes regarding how expert surgical teachers determine residents' readiness for OR autonomy before the surgical time-out emerged. These included 3 domains of evidence elicited about the resident (resident characteristics, medical knowledge, and beyond the current OR case), 1 variable relating to attending characteristics, and 1 variable composed of contextual factors. Experts obtained one or more examples of evidence, and adjusted residents' initial autonomy using factors from the attending variable and the context variable. Expert surgical teachers' assessments of residents' readiness for OR autonomy included 5 key components. Better understanding these inputs can contribute to both faculty and resident development, enabling increased resident autonomy and preparation for independent practice. Copyright © 2017 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

Four Common Simplifications of Multi-Criteria Decision Analysis do not hold for River Rehabilitation

PubMed Central

2016-01-01

River rehabilitation aims at alleviating negative effects of human impacts such as loss of biodiversity and reduction of ecosystem services. Such interventions entail difficult trade-offs between different ecological and often socio-economic objectives. Multi-Criteria Decision Analysis (MCDA) is a very suitable approach that helps assessing the current ecological state and prioritizing river rehabilitation measures in a standardized way, based on stakeholder or expert preferences. Applications of MCDA in river rehabilitation projects are often simplified, i.e. using a limited number of objectives and indicators, assuming linear value functions, aggregating individual indicator assessments additively, and/or assuming risk neutrality of experts. Here, we demonstrate an implementation of MCDA expert preference assessments to river rehabilitation and provide ample material for other applications. To test whether the above simplifications reflect common expert opinion, we carried out very detailed interviews with five river ecologists and a hydraulic engineer. We defined essential objectives and measurable quality indicators (attributes), elicited the experts´ preferences for objectives on a standardized scale (value functions) and their risk attitude, and identified suitable aggregation methods. The experts recommended an extensive objectives hierarchy including between 54 and 93 essential objectives and between 37 to 61 essential attributes. For 81% of these, they defined non-linear value functions and in 76% recommended multiplicative aggregation. The experts were risk averse or risk prone (but never risk neutral), depending on the current ecological state of the river, and the experts´ personal importance of objectives. We conclude that the four commonly applied simplifications clearly do not reflect the opinion of river rehabilitation experts. The optimal level of model complexity, however, remains highly case-study specific depending on data and resource availability, the context, and the complexity of the decision problem. PMID:26954353

Monte Carlo simulation of expert judgments on human errors in chemical analysis--a case study of ICP-MS.

PubMed

Kuselman, Ilya; Pennecchi, Francesca; Epstein, Malka; Fajgelj, Ales; Ellison, Stephen L R

2014-12-01

Monte Carlo simulation of expert judgments on human errors in a chemical analysis was used for determination of distributions of the error quantification scores (scores of likelihood and severity, and scores of effectiveness of a laboratory quality system in prevention of the errors). The simulation was based on modeling of an expert behavior: confident, reasonably doubting and irresolute expert judgments were taken into account by means of different probability mass functions (pmfs). As a case study, 36 scenarios of human errors which may occur in elemental analysis of geological samples by ICP-MS were examined. Characteristics of the score distributions for three pmfs of an expert behavior were compared. Variability of the scores, as standard deviation of the simulated score values from the distribution mean, was used for assessment of the score robustness. A range of the score values, calculated directly from elicited data and simulated by a Monte Carlo method for different pmfs, was also discussed from the robustness point of view. It was shown that robustness of the scores, obtained in the case study, can be assessed as satisfactory for the quality risk management and improvement of a laboratory quality system against human errors. Copyright © 2014 Elsevier B.V. All rights reserved.

A fuzzy Bayesian network approach to quantify the human behaviour during an evacuation

NASA Astrophysics Data System (ADS)

Ramli, Nurulhuda; Ghani, Noraida Abdul; Ahmad, Nazihah

2016-06-01

Bayesian Network (BN) has been regarded as a successful representation of inter-relationship of factors affecting human behavior during an emergency. This paper is an extension of earlier work of quantifying the variables involved in the BN model of human behavior during an evacuation using a well-known direct probability elicitation technique. To overcome judgment bias and reduce the expert's burden in providing precise probability values, a new approach for the elicitation technique is required. This study proposes a new fuzzy BN approach for quantifying human behavior during an evacuation. Three major phases of methodology are involved, namely 1) development of qualitative model representing human factors during an evacuation, 2) quantification of BN model using fuzzy probability and 3) inferencing and interpreting the BN result. A case study of three inter-dependencies of human evacuation factors such as danger assessment ability, information about the threat and stressful conditions are used to illustrate the application of the proposed method. This approach will serve as an alternative to the conventional probability elicitation technique in understanding the human behavior during an evacuation.

Assessing the Expected Value of Research Studies in Reducing Uncertainty and Improving Implementation Dynamics.

PubMed

Grimm, Sabine E; Dixon, Simon; Stevens, John W

2017-07-01

With low implementation of cost-effective health technologies being a problem in many health systems, it is worth considering the potential effects of research on implementation at the time of health technology assessment. Meaningful and realistic implementation estimates must be of dynamic nature. To extend existing methods for assessing the value of research studies in terms of both reduction of uncertainty and improvement in implementation by considering diffusion based on expert beliefs with and without further research conditional on the strength of evidence. We use expected value of sample information and expected value of specific implementation measure concepts accounting for the effects of specific research studies on implementation and the reduction of uncertainty. Diffusion theory and elicitation of expert beliefs about the shape of diffusion curves inform implementation dynamics. We illustrate use of the resulting dynamic expected value of research in a preterm birth screening technology and results are compared with those from a static analysis. Allowing for diffusion based on expert beliefs had a significant impact on the expected value of research in the case study, suggesting that mistakes are made where static implementation levels are assumed. Incorporating the effects of research on implementation resulted in an increase in the expected value of research compared to the expected value of sample information alone. Assessing the expected value of research in reducing uncertainty and improving implementation dynamics has the potential to complement currently used analyses in health technology assessments, especially in recommendations for further research. The combination of expected value of research, diffusion theory, and elicitation described in this article is an important addition to the existing methods of health technology assessment.

Embedding Human Expert Cognition Into Autonomous UAS Trajectory Planning.

PubMed

Narayan, Pritesh; Meyer, Patrick; Campbell, Duncan

2013-04-01

This paper presents a new approach for the inclusion of human expert cognition into autonomous trajectory planning for unmanned aerial systems (UASs) operating in low-altitude environments. During typical UAS operations, multiple objectives may exist; therefore, the use of multicriteria decision aid techniques can potentially allow for convergence to trajectory solutions which better reflect overall mission requirements. In that context, additive multiattribute value theory has been applied to optimize trajectories with respect to multiple objectives. A graphical user interface was developed to allow for knowledge capture from a human decision maker (HDM) through simulated decision scenarios. The expert decision data gathered are converted into value functions and corresponding criteria weightings using utility additive theory. The inclusion of preferences elicited from HDM data within an automated decision system allows for the generation of trajectories which more closely represent the candidate HDM decision preferences. This approach has been demonstrated in this paper through simulation using a fixed-wing UAS operating in low-altitude environments.

Expert searcher, teacher, content manager, and patient advocate: an exploratory study of clinical librarian roles.

PubMed

Tan, Maria C; Maggio, Lauren A

2013-01-01

The research explored the roles of practicing clinical librarians embedded in a patient care team. Six clinical librarians from Canada and one from the United States were interviewed to elicit detailed descriptions of their clinical roles and responsibilities and the context in which these were performed. Participants were embedded in a wide range of clinical service areas, working with a diverse complement of health professionals. As clinical librarians, participants wore many hats, including expert searcher, teacher, content manager, and patient advocate. Unique aspects of how these roles played out included a sense of urgency surrounding searching activities, the broad dissemination of responses to clinical questions, and leverage of the roles of expert searcher, teacher, and content manager to advocate for patients. Detailed role descriptions of clinical librarians embedded in patient care teams suggest possible new practices for existing clinical librarians, provide direction for training new librarians working in patient care environments, and raise awareness of the clinical librarian specialty among current and budding health information professionals.

Development of the chronic obstructive pulmonary disease morning symptom diary (COPD-MSD).

PubMed

Globe, Gary; Currie, Brooke; Leidy, Nancy Kline; Jones, Paul; Mannino, David; Martinez, Fernando; Klekotka, Paul; O'Quinn, Sean; Karlsson, Niklas; Wiklund, Ingela

2016-07-16

The morning tends to be the most difficult time of day for many patients with chronic obstructive pulmonary disease (COPD) when symptoms can limit one's ability to perform even simple activities. Morning symptoms have been linked to higher levels of work absenteeism, thereby increasing the already substantial economic burden associated with COPD. A validated patient-reported outcome (PRO) instrument designed to capture morning symptoms will allow for a more comprehensive approach to the evaluation of treatment benefit in COPD clinical trials. A qualitative interview study was conducted among a sample of symptomatic adults with COPD. Concept elicitation interviews (n = 35) were conducted to identify COPD morning symptoms, followed by cognitive interviews (n = 21) to ensure patient comprehension of the items, instructions and response options of the draft COPD Morning Symptom Diary (COPD-MSD). All interview transcript data were coded using ATLAS.ti software for content analysis. Mean age of the concept elicitation and cognitive interview sample was 65.0 years (±7.5) and 62.3 years (±8.3), respectively. The study sample represented the full range of COPD severity (Global Initiative for Chronic Lung Disease [GOLD] classifications I-IV) and included a mix of racial backgrounds, employment status and educational achievement. During the concept elicitation interviews, the three most frequently reported morning symptoms were shortness of breath (n = 35/35; 100 %), phlegm/mucus (n = 31/35; 88.6 %), and cough (n = 30/35; 85.7 %). A group of clinical and instrument development experts convened to review the concept elicitation data and develop the initial 32-item draft COPD-MSD. Cognitive interviews indicated subjects found the draft COPD-MSD to be comprehensive, clear, and easy to understand. The COPD-MSD underwent minor editorial revisions and streamlining based on cognitive interviews and input from the experts to yield the final 19-item daily diary. This study supports the content validity of the new COPD-MSD and positions the diary for quantitative psychometric testing.

Assessing the multidimensionality of coastal erosion risks: public participation and multicriteria analysis in a Mediterranean coastal system.

PubMed

Roca, Elisabet; Gamboa, Gonzalo; Tàbara, J David

2008-04-01

The complex and multidimensional nature of coastal erosion risks makes it necessary to move away from single-perspective assessment and management methods that have conventionally predominated in coastal management. This article explores the suitability of participatory multicriteria analysis (MCA) for improving the integration of diverse expertises and values and enhancing the social-ecological robustness of the processes that lead to the definition of relevant policy options to deal with those risks. We test this approach in the Mediterranean coastal locality of Lido de Sète in France. Results show that the more adaptive alternatives such as "retreating the shoreline" were preferred by our selected stakeholders to those corresponding to "protecting the shoreline" and the business as usual proposals traditionally put forward by experts and policymakers on these matters. Participative MCA contributed to represent coastal multidimensionality, elicit and integrate different views and preferences, facilitated knowledge exchange, and allowed highlighting existing uncertainties.

Uncertainty and Risk Assessment in the Design Process for Wind

DOE Office of Scientific and Technical Information (OSTI.GOV)

Damiani, Rick R.

This report summarizes the concepts and opinions that emerged from an initial study on the subject of uncertainty in wind design that included expert elicitation during a workshop held at the National Wind Technology Center at the National Renewable Energy Laboratory July 12-13, 2016. In this paper, five major categories of uncertainties are identified. The first category is associated with direct impacts on turbine loads, (i.e., the inflow including extreme events, aero-hydro-servo-elastic response, soil-structure inter- action, and load extrapolation). The second category encompasses material behavior and strength. Site suitability and due-diligence aspects pertain to the third category. Calibration of partialmore » safety factors and optimal reliability levels make up the fourth one. And last but not least, is the category associated with uncertainties in computational modeling. The main sections of this paper follow this organization.« less

Generic domain models in software engineering

NASA Technical Reports Server (NTRS)

Maiden, Neil

1992-01-01

This paper outlines three research directions related to domain-specific software development: (1) reuse of generic models for domain-specific software development; (2) empirical evidence to determine these generic models, namely elicitation of mental knowledge schema possessed by expert software developers; and (3) exploitation of generic domain models to assist modelling of specific applications. It focuses on knowledge acquisition for domain-specific software development, with emphasis on tool support for the most important phases of software development.

DETERMINATION OF RELATIVE IMPORTANCE OF NONPROLIFERATION FACTORS

DOE Office of Scientific and Technical Information (OSTI.GOV)

Richard Metcalf

2009-07-01

Methodologies to determine the proliferation resistance (PR) of nuclear facilities often rely on either expert elicitation, a resource-intensive approach without easily reproducible results, or numeric evaluations, which can fail to take into account the institutional knowledge and expert experience of the nonproliferation community. In an attempt to bridge the gap and bring the institutional knowledge into numeric evaluations of PR, a survey was conducted of 33 individuals to find the relative importance of a set of 62 nonproliferation factors, subsectioned into groups under the headings of Diversion, Transportation, Transformation, and Weaponization. One third of the respondents were self-described nonproliferation professionals,more » and the remaining two thirds were from secondary professions related to nonproliferation, such as industrial engineers or policy analysts. The factors were taken from previous work which used multi-attribute utility analysis with uniform weighting of attributes and did not include institutional knowledge. In both expert and non-expert groups, all four headings and the majority of factors had different relative importance at a confidence of 95% (p=0.05). This analysis and survey demonstrates that institutional knowledge can be brought into numeric evaluations of PR, if there is a sufficient investment of resources made prior to the evaluation.« less

Meditation-related activations are modulated by the practices needed to obtain it and by the expertise: an ALE meta-analysis study

PubMed Central

Tomasino, Barbara; Fregona, Sara; Skrap, Miran; Fabbro, Franco

2013-01-01

The brain network governing meditation has been studied using a variety of meditation practices and techniques practices eliciting different cognitive processes (e.g., silence, attention to own body, sense of joy, mantras, etc.). It is very possible that different practices of meditation are subserved by largely, if not entirely, disparate brain networks. This assumption was tested by conducting an activation likelihood estimation (ALE) meta-analysis of meditation neuroimaging studies, which assessed 150 activation foci from 24 experiments. Different ALE meta-analyses were carried out. One involved the subsets of studies involving meditation induced through exercising focused attention (FA). The network included clusters bilaterally in the medial gyrus, the left superior parietal lobe, the left insula and the right supramarginal gyrus (SMG). A second analysis addressed the studies involving meditation states induced by chanting or by repetition of words or phrases, known as “mantra.” This type of practice elicited a cluster of activity in the right SMG, the SMA bilaterally and the left postcentral gyrus. Furthermore, the last analyses addressed the effect of meditation experience (i.e., short- vs. long-term meditators). We found that frontal activation was present for short-term, as compared with long-term experience meditators, confirming that experts are better enabled to sustain attentional focus, rather recruiting the right SMG and concentrating on aspects involving disembodiment. PMID:23316154

Perception of chemesthetic stimuli in groups who differ by food involvement and culinary experience.

PubMed

Byrnes, Nadia; Loss, Christopher R; Hayes, John E

2015-12-01

In the English language, there is generally a limited lexicon when referring to the sensations elicited by chemesthetic stimuli like capsaicin, allyl isothiocyanate, and eugenol, the orally irritating compounds found in chiles, wasabi, and cloves, respectively. Elsewhere, experts and novices have been shown to use language differently, with experts using more precise language. Here, we compare perceptual maps and word usage across three cohorts: experts with formal culinary education, naïve individuals with high Food Involvement Scale (FIS) scores, and naïve individuals with low FIS scores. We hypothesized that increased experience with foods, whether through informal experiential learning or formal culinary education, would have a significant influence on the perceptual maps generated from a sorting task conducted with chemesthetic stimuli, as well as on language use in a descriptive follow-up task to this sorting task. The low- and highFIS non-expert cohorts generated significantly similar maps, though in other respects the highFIS cohort was an intermediate between the lowFIS and expert cohorts. The highFIS and expert cohorts generated more attributes but used language more idiosyncratically than the lowFIS group. Overall, the results from the expert group with formal culinary education differed from the two naïve cohorts both in the perceptual map generated using MDS as well as the mean number of attributes generated. Present data suggest that both formal education and informal experiential learning result in lexical development, but the level and type of learning can have a significant influence on language use and the approach to a sorting task.

Perception of chemesthetic stimuli in groups who differ by food involvement and culinary experience

PubMed Central

Byrnes, Nadia; Loss, Christopher R.; Hayes, John E.

2015-01-01

In the English language, there is generally a limited lexicon when referring to the sensations elicited by chemesthetic stimuli like capsaicin, allyl isothiocyanate, and eugenol, the orally irritating compounds found in chiles, wasabi, and cloves, respectively. Elsewhere, experts and novices have been shown to use language differently, with experts using more precise language. Here, we compare perceptual maps and word usage across three cohorts: experts with formal culinary education, naïve individuals with high Food Involvement Scale (FIS) scores, and naïve individuals with low FIS scores. We hypothesized that increased experience with foods, whether through informal experiential learning or formal culinary education, would have a significant influence on the perceptual maps generated from a sorting task conducted with chemesthetic stimuli, as well as on language use in a descriptive follow-up task to this sorting task. The low- and highFIS non-expert cohorts generated significantly similar maps, though in other respects the highFIS cohort was an intermediate between the lowFIS and expert cohorts. The highFIS and expert cohorts generated more attributes but used language more idiosyncratically than the lowFIS group. Overall, the results from the expert group with formal culinary education differed from the two naïve cohorts both in the perceptual map generated using MDS as well as the mean number of attributes generated. Present data suggest that both formal education and informal experiential learning result in lexical development, but the level and type of learning can have a significant influence on language use and the approach to a sorting task. PMID:26516297

Breakfast barriers and opportunities for children living in a Dutch disadvantaged neighbourhood.

PubMed

van Kleef, Ellen; Vingerhoeds, Monique H; Vrijhof, Milou; van Trijp, Hans C M

2016-12-01

The objective of this study was to explore parents', children's, and experts' beliefs and experiences about breakfast motivation, opportunity, and ability and elicit their thoughts on effective interventions to encourage healthy breakfast consumption. The setting was a disadvantaged neighbourhood in Rotterdam, the Netherlands. Focus groups with mothers and children and semi-structured individual interviews with experts were conducted. Interview guides were developed based on the motivation, opportunity, and ability consumer psychology model. Thirty-two mothers of primary school children participated in five group discussions, eight focus groups were conducted with 44 children, and nine experts participated in interviews. Data from expert interviews and group discussions were coded and thematically analysed. The following themes emerged from the focus groups: (1) generally high motivation to have breakfast, (2) improved performance at school is key motivator, (3) limited time hinders breakfast, and (4) lack of nutritional knowledge about high quality breakfast. Experts mentioned lack of effort, knowledge, and time; financial constraints; and environmental issues (food availability) as barriers to breakfasting healthily. Several ways to encourage healthy breakfasting habits were identified: (1) involvement of both children and parents, (2) role models inspiring change, and (3) interactive educational approaches. Experts perceived more problems and challenges in achieving healthy breakfast habits than did mothers and children. Lack of opportunity (according to the children and experts) and ability (according to the experts) were identified, although the motivation to eat a healthy breakfast was present. Predominant barriers are lack of time and nutritional knowledge. Overall, findings suggest educational and social marketing approaches as interventions to encourage healthy breakfast consumption. Copyright © 2016 Elsevier Ltd. All rights reserved.

MERS-CoV at the Animal–Human Interface: Inputs on Exposure Pathways from an Expert-Opinion Elicitation

PubMed Central

Funk, Anna L.; Goutard, Flavie Luce; Miguel, Eve; Bourgarel, Mathieu; Chevalier, Veronique; Faye, Bernard; Peiris, J. S. Malik; Van Kerkhove, Maria D.; Roger, Francois Louis

2016-01-01

Nearly 4 years after the first report of the emergence of Middle-East respiratory syndrome Coronavirus (MERS-CoV) and nearly 1800 human cases later, the ecology of MERS-CoV, its epidemiology, and more than risk factors of MERS-CoV transmission between camels are poorly understood. Knowledge about the pathways and mechanisms of transmission from animals to humans is limited; as of yet, transmission risks have not been quantified. Moreover the divergent sanitary situations and exposures to animals among populations in the Arabian Peninsula, where human primary cases appear to dominate, vs. other regions in the Middle East and Africa, with no reported human clinical cases and where the virus has been detected only in dromedaries, represents huge scientific and health challenges. Here, we have used expert-opinion elicitation in order to obtain ideas on relative importance of MERS-CoV risk factors and estimates of transmission risks from various types of contact between humans and dromedaries. Fourteen experts with diverse and extensive experience in MERS-CoV relevant fields were enrolled and completed an online questionnaire that examined pathways based on several scenarios, e.g., camels–camels, camels–human, bats/other species to camels/humans, and the role of diverse biological substances (milk, urine, etc.) and potential fomites. Experts believed that dromedary camels play the largest role in MERS-CoV infection of other dromedaries; however, they also indicated a significant influence of the season (i.e. calving or weaning periods) on transmission risk. All experts thought that MERS-CoV-infected dromedaries and asymptomatic humans play the most important role in infection of humans, with bats and other species presenting a possible, but yet undefined, risk. Direct and indirect contact of humans with dromedary camels were identified as the most risky types of contact, when compared to consumption of various camel products, with estimated “most likely” incidence risks of at least 22 and 13% for direct and indirect contact, respectively. The results of our study are consistent with available, yet very limited, published data regarding the potential pathways of transmission of MERS-CoV at the animal–human interface. These results identify key knowledge gaps and highlight the need for more comprehensive, yet focused research to be conducted to better understand transmission between dromedaries and humans. PMID:27761437

Processing Demands Impact 3-Year-Olds' Performance in a Spontaneous-Response Task: New Evidence for the Processing-Load Account of Early False-Belief Understanding.

PubMed

Scott, Rose M; Roby, Erin

2015-01-01

Prior to age four, children succeed in non-elicited-response false-belief tasks but fail elicited-response false-belief tasks. To explain this discrepancy, the processing-load account argues that the capacity to represent beliefs emerges in infancy, as indicated by early success on non-elicited-response tasks, but that children's ability to demonstrate this capacity depends on the processing demands of the task and children's processing skills. When processing demands exceed young children's processing abilities, such as in standard elicited-response tasks, children fail despite their capacity to represent beliefs. Support for this account comes from recent evidence that reducing processing demands improves young children's performance: when demands are sufficiently reduced, 2.5-year-olds succeed in elicited-response tasks. Here we sought complementary evidence for the processing-load account by examining whether increasing processing demands impeded children's performance in a non-elicited-response task. 3-year-olds were tested in a preferential-looking task in which they heard a change-of-location false-belief story accompanied by a picture book; across children, we manipulated the amount of linguistic ambiguity in the story. The final page of the book showed two images: one that was consistent with the main character's false belief and one that was consistent with reality. When the story was relatively unambiguous, children looked reliably longer at the false-belief-consistent image, successfully demonstrating their false-belief understanding. When the story was ambiguous, however, this undermined children's performance: looking times to the belief-consistent image were correlated with verbal ability, and only children with verbal skills in the upper quartile of the sample demonstrated a significant preference for the belief-consistent image. These results support the processing-load account by demonstrating that regardless of whether a task involves an elicited response, children's performance depends on the processing demands of the task and their processing skills. These findings also have implications for alternative, deflationary accounts of early false-belief understanding.

Processing Demands Impact 3-Year-Olds’ Performance in a Spontaneous-Response Task: New Evidence for the Processing-Load Account of Early False-Belief Understanding

PubMed Central

Scott, Rose M.; Roby, Erin

2015-01-01

Prior to age four, children succeed in non-elicited-response false-belief tasks but fail elicited-response false-belief tasks. To explain this discrepancy, the processing-load account argues that the capacity to represent beliefs emerges in infancy, as indicated by early success on non-elicited-response tasks, but that children’s ability to demonstrate this capacity depends on the processing demands of the task and children’s processing skills. When processing demands exceed young children’s processing abilities, such as in standard elicited-response tasks, children fail despite their capacity to represent beliefs. Support for this account comes from recent evidence that reducing processing demands improves young children’s performance: when demands are sufficiently reduced, 2.5-year-olds succeed in elicited-response tasks. Here we sought complementary evidence for the processing-load account by examining whether increasing processing demands impeded children’s performance in a non-elicited-response task. 3-year-olds were tested in a preferential-looking task in which they heard a change-of-location false-belief story accompanied by a picture book; across children, we manipulated the amount of linguistic ambiguity in the story. The final page of the book showed two images: one that was consistent with the main character’s false belief and one that was consistent with reality. When the story was relatively unambiguous, children looked reliably longer at the false-belief-consistent image, successfully demonstrating their false-belief understanding. When the story was ambiguous, however, this undermined children’s performance: looking times to the belief-consistent image were correlated with verbal ability, and only children with verbal skills in the upper quartile of the sample demonstrated a significant preference for the belief-consistent image. These results support the processing-load account by demonstrating that regardless of whether a task involves an elicited response, children’s performance depends on the processing demands of the task and their processing skills. These findings also have implications for alternative, deflationary accounts of early false-belief understanding. PMID:26562840

Towards a Methodology for Identifying Program Constraints During Requirements Analysis

NASA Technical Reports Server (NTRS)

Romo, Lilly; Gates, Ann Q.; Della-Piana, Connie Kubo

1997-01-01

Requirements analysis is the activity that involves determining the needs of the customer, identifying the services that the software system should provide and understanding the constraints on the solution. The result of this activity is a natural language document, typically referred to as the requirements definition document. Some of the problems that exist in defining requirements in large scale software projects includes synthesizing knowledge from various domain experts and communicating this information across multiple levels of personnel. One approach that addresses part of this problem is called context monitoring and involves identifying the properties of and relationships between objects that the system will manipulate. This paper examines several software development methodologies, discusses the support that each provide for eliciting such information from experts and specifying the information, and suggests refinements to these methodologies.

Suburban watershed nitrogen retention: Estimating the effectiveness of stormwater management structures

USGS Publications Warehouse

Koch, Benjamin J.; Febria, Catherine M.; Cooke, Roger M.; Hosen, Jacob D.; Baker, Matthew E.; Colson, Abigail R.; Filoso, Solange; Hayhoe, Katharine; Loperfido, J. V.; Stoner, Anne M.K.; Palmer, Margaret A.

2015-01-01

Expert knowledge indicated wide uncertainty in BMP performance, with N removal efficiencies ranging from <0% (BMP acting as a source of N during a rain event) to >40%. Experts believed that the amount of rain was the primary identifiable source of variability in BMP efficiency, which is relevant given climate projections of more frequent heavy rain events in the mid-Atlantic. To assess the extent to which those projected changes might alter N export from suburban BMPs and watersheds, we combined downscaled estimates of rainfall with distributions of N loads for different-sized rain events derived from our elicitation. The model predicted higher and more variable N loads under a projected future climate regime, suggesting that current BMP regulations for reducing nutrients may be inadequate in the future.

Infants experience-dependent processing of male and female faces: Insights from eye tracking and event-related potentials

PubMed Central

Righi, Giulia; Westerlund, Alissa; Congdon, Eliza L.; Troller-Renfree, Sonya; Nelson, Charles A.

2013-01-01

The goal of the present study was to investigate infants’ processing of female and male faces. We used an event-related potential (ERP) priming task, as well as a visual-paired comparison (VPC) eye tracking task to explore how 7-month-old “female expert” infants differed in their responses to faces of different genders. Female faces elicited larger N290 amplitudes than male faces. Furthermore, infants showed a priming effect for female faces only, whereby the N290 was significantly more negative for novel females compared to primed female faces. The VPC experiment was designed to test whether infants could reliably discriminate between two female and two male faces. Analyses showed that infants were able to differentiate faces of both genders. The results of the present study suggest that 7-month olds with a large amount of female face experience show a processing advantage for forming a neural representation of female faces, compared to male faces. However, the enhanced neural sensitivity to the repetition of female faces is not due to the infants' inability to discriminate male faces. Instead, the combination of results from the two tasks suggests that the differential processing for female faces may be a signature of expert-level processing. PMID:24200421

A novel process for integrating patient stories into patient education interventions: incorporating lessons from theater arts.

PubMed

Fix, Gemmae M; Houston, Thomas K; Barker, Anna M; Wexler, Laura; Cook, Natasha; Volkman, Julie E; Bokhour, Barbara G

2012-09-01

Patient narratives, or stories, are an effective means of educating patients because they increase personal relevance and may reduce counter-arguing. However, such stories must seamlessly combine evidenced-based health information while being true to real patient experiences. The purpose of this paper is to describe the process of developing an educational intervention using African-American patients' success stories controlling hypertension. We identified a process to address stories development challenges. (1) To help identify story tellers, we conducted a literature review and subsequently streamlined the process of storyteller identification through screening and telephone interviews. (2) To better elicit stories, we consulted with experts in storytelling and incorporated principles from theater. (3) To select stories, we used intervention mapping to map the intervention to theory and key clinical concepts, and also engaged members of the target community to ensure scientific criteria and maintain authenticity. Using personal narratives as intervention requires weaving together science, theory and clinically sound content, while still being true to the art of storytelling. Through a careful process of identifying storytellers and story selection and drawing upon theater arts, creating stories for intervention can be streamlined while meeting the goals of authenticity and scientific soundness. Published by Elsevier Ireland Ltd.

Rank aggregation of local expert knowledge for conservation planning of the critically endangered saola.

PubMed

Wilkinson, Nicholas M; Van Duc, Luong

2017-06-01

There has been much recent interest in using local knowledge and expert opinion for conservation planning, particularly for hard-to-detect species. Although it is possible to ask for direct estimation of quantities such as population size, relative abundance is easier to estimate. However, an expert's knowledge is often geographically restricted relative to the area of interest. Combining (or aggregating) experts' assessments of relative abundance is difficult when each expert only knows a part of the area of interest. We used Google's PageRank algorithm to aggregate ranked abundance scores elicited from local experts through a rapid rural-appraisal method. We applied this technique to conservation planning for the saola (Pseudoryx nghetinhensis), a poorly known bovid. Near a priority landscape for the species, composed of 3 contiguous protected areas, we asked groups of local people to indicate relative abundances of saola and other species by placing beans on community maps. For each village, we used this information to rank areas within the knowledge area of that village for saola abundance. We used simulations to compare alternative methods to aggregate the rankings from the different villages. The best-performing method was then used to produce a single map of relative abundance across the entire landscape, an area larger than that known to any one village. This map has informed prioritization of surveys and conservation action in the continued absence of direct information about the saola. © 2016 Society for Conservation Biology.

Verification and Validation of Digitally Upgraded Control Rooms

DOE Office of Scientific and Technical Information (OSTI.GOV)

Boring, Ronald; Lau, Nathan

2015-09-01

As nuclear power plants undertake main control room modernization, a challenge is the lack of a clearly defined human factors process to follow. Verification and validation (V&V) as applied in the nuclear power community has tended to involve efforts such as integrated system validation, which comes at the tail end of the design stage. To fill in guidance gaps and create a step-by-step process for control room modernization, we have developed the Guideline for Operational Nuclear Usability and Knowledge Elicitation (GONUKE). This approach builds on best practices in the software industry, which prescribe an iterative user-centered approach featuring multiple cyclesmore » of design and evaluation. Nuclear regulatory guidance for control room design emphasizes summative evaluation—which occurs after the design is complete. In the GONUKE approach, evaluation is also performed at the formative stage of design—early in the design cycle using mockups and prototypes for evaluation. The evaluation may involve expert review (e.g., software heuristic evaluation at the formative stage and design verification against human factors standards like NUREG-0700 at the summative stage). The evaluation may also involve user testing (e.g., usability testing at the formative stage and integrated system validation at the summative stage). An additional, often overlooked component of evaluation is knowledge elicitation, which captures operator insights into the system. In this report we outline these evaluation types across design phases that support the overall modernization process. The objective is to provide industry-suitable guidance for steps to be taken in support of the design and evaluation of a new human-machine interface (HMI) in the control room. We suggest the value of early-stage V&V and highlight how this early-stage V&V can help improve the design process for control room modernization. We argue that there is a need to overcome two shortcomings of V&V in current practice—the propensity for late-stage V&V and the use of increasingly complex psychological assessment measures for V&V.« less

Development of the Neuron Assessment for Measuring Biology Students’ Use of Experimental Design Concepts and Representations

PubMed Central

Dasgupta, Annwesa P.; Anderson, Trevor R.; Pelaez, Nancy J.

2016-01-01

Researchers, instructors, and funding bodies in biology education are unanimous about the importance of developing students’ competence in experimental design. Despite this, only limited measures are available for assessing such competence development, especially in the areas of molecular and cellular biology. Also, existing assessments do not measure how well students use standard symbolism to visualize biological experiments. We propose an assessment-design process that 1) provides background knowledge and questions for developers of new “experimentation assessments,” 2) elicits practices of representing experiments with conventional symbol systems, 3) determines how well the assessment reveals expert knowledge, and 4) determines how well the instrument exposes student knowledge and difficulties. To illustrate this process, we developed the Neuron Assessment and coded responses from a scientist and four undergraduate students using the Rubric for Experimental Design and the Concept-Reasoning Mode of representation (CRM) model. Some students demonstrated sound knowledge of concepts and representations. Other students demonstrated difficulty with depicting treatment and control group data or variability in experimental outcomes. Our process, which incorporates an authentic research situation that discriminates levels of visualization and experimentation abilities, shows potential for informing assessment design in other disciplines. PMID:27146159

Funding Decisions for Newborn Screening: A Comparative Review of 22 Decision Processes in Europe

PubMed Central

Fischer, Katharina Elisabeth; Rogowski, Wolf Henning

2014-01-01

Decision-makers need to make choices to improve public health. Population-based newborn screening (NBS) is considered as one strategy to prevent adverse health outcomes and address rare disease patients’ needs. The aim of this study was to describe key characteristics of decisions for funding new NBS programmes in Europe. We analysed past decisions using a conceptual framework. It incorporates indicators that capture the steps of decision processes by health care payers. Based on an internet survey, we compared 22 decisions for which answers among two respondents were validated for each observation. The frequencies of indicators were calculated to elicit key characteristics. All decisions resulted in positive, mostly unrestricted funding. Stakeholder participation was diverse focusing on information provision or voting. Often, decisions were not fully transparent. Assessment of NBS technologies concentrated on expert opinion, literature review and rough cost estimates. Most important appraisal criteria were effectiveness (i.e., health gain from testing for the children being screened), disease severity and availability of treatments. Some common and diverging key characteristics were identified. Although no evidence of explicit healthcare rationing was found, processes may be improved in respect of transparency and scientific rigour of assessment. PMID:24852389

Physiotherapists' experiences of the management of anterior cruciate ligament injuries.

PubMed

von Aesch, Arlene V; Perry, Meredith; Sole, Gisela

2016-05-01

While extensive research has been reported for management of anterior cruciate ligament (ACL) injuries, variation in treatment by physiotherapists is evident. To explore physiotherapists' experiences regarding ACL injury rehabilitation and factors that influenced physiotherapists' decision making for ACL rehabilitation, and to elicit what research physiotherapists perceived would support their management of these patients. Qualitative study. Fifteen physiotherapists from six private clinics in New Zealand participated in semi-structured interviews. The interviews were recorded, transcribed verbatim and the general inductive approach was used to develop key themes. Participant's management strengths were evident by their intent and commitment to provide expert rehabilitation, using a biopsychosocial approach and evidence-informed practice. The lengthy management process (including prolonged rehabilitation and referral processes) and interprofessional disconnect concerned participants. Translational research was needed for clear directions for exercise prescription and milestones for return to sports and occupation following ACL injury. Participants provided a biopsychosocial and evidence-based approach to ACL injury management. Potential areas of improvement include simplifying the referral process and enhancing communication between physiotherapists and other health professionals. Future research should focus on clarifying areas of ACL rehabilitation uncertainty, or collating results in an accessible and usable format for clinical practice. Copyright © 2015 Elsevier Ltd. All rights reserved.

Integrated Knowledge Elicitation and Representation Framework.

DTIC Science & Technology

1991-04-01

applications. Given the fast paced, time-stressed battlefield environment, expert and decision sup- port systems will play an important role in future...can weaken them a little bit, it will help." I: "Why would they use dummy guns to divert us?" E: "We use imagery a lot from a fast moving airplane. A...found that subjects are often slow to update their assessment of a situation, even when new information conflicts with their old impressions. Leddo and

Probabilistic 21st and 22nd Century Sea-Level Projections at a Global Network of Tide-Gauge Sites

NASA Technical Reports Server (NTRS)

Kopp, Robert E.; Horton, Radley M.; Little, Christopher M.; Mitrovica, Jerry X.; Oppenheimer, Michael; Rasmussen, D. J.; Strauss, Benjamin H.; Tebaldi, Claudia

2014-01-01

Sea-level rise due to both climate change and non-climatic factors threatens coastal settlements, infrastructure, and ecosystems. Projections of mean global sea-level (GSL) rise provide insufficient information to plan adaptive responses; local decisions require local projections that accommodate different risk tolerances and time frames and that can be linked to storm surge projections. Here we present a global set of local sea-level (LSL) projections to inform decisions on timescales ranging from the coming decades through the 22nd century. We provide complete probability distributions, informed by a combination of expert community assessment, expert elicitation, and process modeling. Between the years 2000 and 2100, we project a very likely (90% probability) GSL rise of 0.5–1.2?m under representative concentration pathway (RCP) 8.5, 0.4–0.9?m under RCP 4.5, and 0.3–0.8?m under RCP 2.6. Site-to-site differences in LSL projections are due to varying non-climatic background uplift or subsidence, oceanographic effects, and spatially variable responses of the geoid and the lithosphere to shrinking land ice. The Antarctic ice sheet (AIS) constitutes a growing share of variance in GSL and LSL projections. In the global average and at many locations, it is the dominant source of variance in late 21st century projections, though at some sites oceanographic processes contribute the largest share throughout the century. LSL rise dramatically reshapes flood risk, greatly increasing the expected number of “1-in-10” and “1-in-100” year events.

Probabilistic 21st and 22nd century sea-level projections at a global network of tide-gauge sites

NASA Astrophysics Data System (ADS)

Kopp, Robert E.; Horton, Radley M.; Little, Christopher M.; Mitrovica, Jerry X.; Oppenheimer, Michael; Rasmussen, D. J.; Strauss, Benjamin H.; Tebaldi, Claudia

2014-08-01

Sea-level rise due to both climate change and non-climatic factors threatens coastal settlements, infrastructure, and ecosystems. Projections of mean global sea-level (GSL) rise provide insufficient information to plan adaptive responses; local decisions require local projections that accommodate different risk tolerances and time frames and that can be linked to storm surge projections. Here we present a global set of local sea-level (LSL) projections to inform decisions on timescales ranging from the coming decades through the 22nd century. We provide complete probability distributions, informed by a combination of expert community assessment, expert elicitation, and process modeling. Between the years 2000 and 2100, we project a very likely (90% probability) GSL rise of 0.5-1.2 m under representative concentration pathway (RCP) 8.5, 0.4-0.9 m under RCP 4.5, and 0.3-0.8 m under RCP 2.6. Site-to-site differences in LSL projections are due to varying non-climatic background uplift or subsidence, oceanographic effects, and spatially variable responses of the geoid and the lithosphere to shrinking land ice. The Antarctic ice sheet (AIS) constitutes a growing share of variance in GSL and LSL projections. In the global average and at many locations, it is the dominant source of variance in late 21st century projections, though at some sites oceanographic processes contribute the largest share throughout the century. LSL rise dramatically reshapes flood risk, greatly increasing the expected number of "1-in-10" and "1-in-100" year events.

Using Expert Elicitation to Estimate the Impacts of Plastic Pollution on Marine Wildlife

NASA Astrophysics Data System (ADS)

Mallos, N. J.; Wilcox, C.; Leonard, G. H.; Rodriquez, A. G.; Hardesty, B. D.

2016-02-01

With the rapid increase in global plastics production and the resulting large volume of litter that enters the marine environment, determining the consequences of this debris on marine fauna and ocean health has now become a critical environmental priority, particularly for threatened and endangered species. However, there are limited data about the impacts on debris on marine species from which to draw conclusions about the population consequences of anthropogenic debris. To address this knowledge gap, information was elicited from experts on the ecological threat of entanglement, ingestion and chemical contamination for three major marine taxa: seabirds, sea turtles and marine mammals. The threat assessment focused on the most common types of litter that are found along the world's coastlines, based on data gathered during three decades of international coastal clean-up efforts. Fishing related gear, balloons and plastic bags were estimated to pose the greatest entanglement risk to marine fauna. In contrast, experts identified a broader suite of items of concern for ingestion, with plastic bags and plastic utensils ranked as the greatest threats. Entanglement and ingestion affected a similar range of taxa, although entanglement was slightly worse as it is more likely to be lethal. Contamination was scored the lowest in terms of its impact, affecting a smaller portion of the taxa and being rated as having solely non-lethal impacts. Research designed to better understand and quantify the impacts of chemical contamination on marine fauna at individual, population and species levels should be a priority for conservation biologists. This work points towards a number of opportunities for both policy-based and consumer-driven changes in plastics use that could have demonstrable affects for a range of taxa that are ecologically important and serve as indicators of marine ecosystem health. Based on threat rankings, entanglement and ingestion should be a similar priority for management action, as experts ranked them nearly identically in terms of expected population impacts. Fishing gear, plastic bags, and plastic utensils were all ranked as having substantial impacts on more than half of all three marine taxa, and should be priorities for management action.

Modelling seasonal habitat suitability for wide-ranging species: Invasive wild pigs in northern Australia.

PubMed

Froese, Jens G; Smith, Carl S; Durr, Peter A; McAlpine, Clive A; van Klinken, Rieks D

2017-01-01

Invasive wildlife often causes serious damage to the economy and agriculture as well as environmental, human and animal health. Habitat models can fill knowledge gaps about species distributions and assist planning to mitigate impacts. Yet, model accuracy and utility may be compromised by small study areas and limited integration of species ecology or temporal variability. Here we modelled seasonal habitat suitability for wild pigs, a widespread and harmful invader, in northern Australia. We developed a resource-based, spatially-explicit and regional-scale approach using Bayesian networks and spatial pattern suitability analysis. We integrated important ecological factors such as variability in environmental conditions, breeding requirements and home range movements. The habitat model was parameterized during a structured, iterative expert elicitation process and applied to a wet season and a dry season scenario. Model performance and uncertainty was evaluated against independent distributional data sets. Validation results showed that an expert-averaged model accurately predicted empirical wild pig presences in northern Australia for both seasonal scenarios. Model uncertainty was largely associated with different expert assumptions about wild pigs' resource-seeking home range movements. Habitat suitability varied considerably between seasons, retracting to resource-abundant rainforest, wetland and agricultural refuge areas during the dry season and expanding widely into surrounding grassland floodplains, savanna woodlands and coastal shrubs during the wet season. Overall, our model suggested that suitable wild pig habitat is less widely available in northern Australia than previously thought. Mapped results may be used to quantify impacts, assess risks, justify management investments and target control activities. Our methods are applicable to other wide-ranging species, especially in data-poor situations.

Modelling seasonal habitat suitability for wide-ranging species: Invasive wild pigs in northern Australia

PubMed Central

2017-01-01

Invasive wildlife often causes serious damage to the economy and agriculture as well as environmental, human and animal health. Habitat models can fill knowledge gaps about species distributions and assist planning to mitigate impacts. Yet, model accuracy and utility may be compromised by small study areas and limited integration of species ecology or temporal variability. Here we modelled seasonal habitat suitability for wild pigs, a widespread and harmful invader, in northern Australia. We developed a resource-based, spatially-explicit and regional-scale approach using Bayesian networks and spatial pattern suitability analysis. We integrated important ecological factors such as variability in environmental conditions, breeding requirements and home range movements. The habitat model was parameterized during a structured, iterative expert elicitation process and applied to a wet season and a dry season scenario. Model performance and uncertainty was evaluated against independent distributional data sets. Validation results showed that an expert-averaged model accurately predicted empirical wild pig presences in northern Australia for both seasonal scenarios. Model uncertainty was largely associated with different expert assumptions about wild pigs’ resource-seeking home range movements. Habitat suitability varied considerably between seasons, retracting to resource-abundant rainforest, wetland and agricultural refuge areas during the dry season and expanding widely into surrounding grassland floodplains, savanna woodlands and coastal shrubs during the wet season. Overall, our model suggested that suitable wild pig habitat is less widely available in northern Australia than previously thought. Mapped results may be used to quantify impacts, assess risks, justify management investments and target control activities. Our methods are applicable to other wide-ranging species, especially in data-poor situations. PMID:28472113

Involving women in personalised decision-making on mode of delivery after caesarean section: the development and pilot testing of a patient decision aid.

PubMed

Schoorel, E N C; Vankan, E; Scheepers, H C J; Augustijn, B C C; Dirksen, C D; de Koning, M; van Kuijk, S M J; Kwee, A; Melman, S; Nijhuis, J G; Aardenburg, R; de Boer, K; Hasaart, T H M; Mol, B W J; Nieuwenhuijze, M; van Pampus, M G; van Roosmalen, J; Roumen, F J M E; de Vries, R; Wouters, M G A J; van der Weijden, T; Hermens, R P M G

2014-01-01

To develop a patient decision aid (PtDA) for mode of delivery after caesarean section that integrates personalised prediction of vaginal birth after caesarean (VBAC) with the elicitation of patient preferences and evidence-based information. A PtDA was developed and pilot tested using the International Patients Decision Aid Standards (IPDAS) criteria. Obstetric health care in the Netherlands. A multidisciplinary steering group, an expert panel, and 25 future users of the PtDA, i.e. women with a previous caesarean section. The development consisted of a construction phase (definition of scope and purpose, and selection of content, framework, and format) and a pilot testing phase by interview. The process was supervised by a multidisciplinary steering group. Usability, clarity, and relevance. The construction phase resulted in a booklet including unbiased balanced information on mode of birth after caesarean section, a preference elicitation exercise, and tailored risk information, including a prediction model for successful VBAC. During pilot testing, visualisation of risks and clarity formed the main basis for revisions. Pilot testing showed the availability of tailored structured information to be the main factor involving women in decision-making. The PtDA meets 39 out of 50 IPDAS criteria (78%): 23 out of 23 criteria for content (100%) and 16 out of 20 criteria for the development process (80%). Criteria for effectiveness (n = 7) were not evaluated. An evidence-based PtDA was developed, with the probability of successful VBAC and the availability of structured information as key items. It is likely that the PtDA enhances the quality of decision-making on mode of birth after caesarean section. © 2013 Royal College of Obstetricians and Gynaecologists.

Making policy decisions about population screening for breast cancer: the role of citizens' deliberation.

PubMed

Paul, Charlotte; Nicholls, Rachel; Priest, Patricia; McGee, Rob

2008-03-01

To test a method of assessing whether a community of interest - when well informed - would be prepared to support or reject a public policy decision about cancer screening. In particular, whether the New Zealand government should offer free mammography screening to all women aged 40-49 years. Eleven women aged from 40 to 49 years, randomly selected from the electoral roll, agreed to participate in trial of a citizens' jury: a deliberative method of gathering the views of the public. Only selected aspects of the jury method were trialled. Participants met over a day and a half to hear evidence from expert witnesses with differing views and to deliberate the verdict. All but one woman changed their minds during the jury process, and voted against government provision of mammography screening in this age group. The main reasons reported were the inaccuracy of the test and the potential for harm, and the lack of firm evidence of saving lives in this age group. A deliberative 'citizens' jury' approach is a feasible way of eliciting a well informed, considered community view about screening or other population health initiatives. Pro-screening views of affected populations may change when individuals are given accurate information and enabled to deliberate about benefits and harms. This method could be used to determine how complex benefits and harms are weighed by affected populations, particularly where experts and advocacy groups disagree.

The Quantification of Consistent Subjective Logic Tree Branch Weights for PSHA

NASA Astrophysics Data System (ADS)

Runge, A. K.; Scherbaum, F.

2012-04-01

The development of quantitative models for the rate of exceedance of seismically generated ground motion parameters is the target of probabilistic seismic hazard analysis (PSHA). In regions of low to moderate seismicity, the selection and evaluation of source- and/or ground-motion models is often a major challenge to hazard analysts and affected by large epistemic uncertainties. In PSHA this type of uncertainties is commonly treated within a logic tree framework in which the branch weights express the degree-of-belief values of an expert in the corresponding set of models. For the calculation of the distribution of hazard curves, these branch weights are subsequently used as subjective probabilities. However the quality of the results depends strongly on the "quality" of the expert knowledge. A major challenge for experts in this context is to provide weight estimates which are logically consistent (in the sense of Kolmogorov's axioms) and to be aware of and to deal with the multitude of heuristics and biases which affect human judgment under uncertainty. For example, people tend to give smaller weights to each branch of a logic tree the more branches it has, starting with equal weights for all branches and then adjusting this uniform distribution based on his/her beliefs about how the branches differ. This effect is known as pruning bias.¹ A similar unwanted effect, which may even wrongly suggest robustness of the corresponding hazard estimates, will appear in cases where all models are first judged according to some numerical quality measure approach and the resulting weights are subsequently normalized to sum up to one.2 To address these problems, we have developed interactive graphical tools for the determination of logic tree branch weights in form of logically consistent subjective probabilities, based on the concepts suggested in Curtis and Wood (2004).3 Instead of determining the set of weights for all the models in a single step, the computer driven elicitation process is performed as a sequence of evaluations of relative weights for small subsets of models which are presented to the analyst. From these, the distribution of logic tree weights for the whole model set is determined as solution of an optimization problem. The model subset presented to the analyst in each step is designed to maximize the expected information. The result of this process is a set of logically consistent weights together with a measure of confidence determined from the amount of conflicting information which is provided by the expert during the relative weighting process.

Experts' Views on Using History and Philosophy of Science in the Practice of Physics Instruction

NASA Astrophysics Data System (ADS)

Galili, Igal; Hazan, Amnon

This study examines the views of a representative sample of experts in physics, physics education and history and philosophy of science (HPS) on the incorporation of HPS based materials in physics instruction. The obtained spectrum of views addresses three areas: the rationale to include HPS, the most appropriate ways of doing so, and anticipated difficulties in such a new educational approach. The elicited views, interpreted and categorized, reflect the attitude of the community of science educators in Israeli colleges and universities with regard to the subject. The constructed profiles indicate low awareness of the recent changes in the understanding of learning and the role of HPS in the light of these changes. Such knowledge can guide the activities of those who devote their efforts to constructing and implementing learning materials utilizing HPS contents in science education.

Authorities for Military Operations Against Terrorist Groups: The State of the Debate and Options for Congress

DTIC Science & Technology

2015-12-07

terrorist threat and the likely trajectory it will take in the coming five years. The second, eliciting the perspectives of legal and policy experts...focused on specific elements of notional ver- sions of a congressional AUMF. We thus sought to survey the current thinking among both terrorism and legal ...United States faces are no longer those that posed the greatest threat in 2001. Operations can continue under the 2001 authority, but not without legal

Expert searcher, teacher, content manager, and patient advocate: an exploratory study of clinical librarian roles

PubMed Central

Tan, Maria C.; Maggio, Lauren A.

2013-01-01

Objective: The research explored the roles of practicing clinical librarians embedded in a patient care team. Methods: Six clinical librarians from Canada and one from the United States were interviewed to elicit detailed descriptions of their clinical roles and responsibilities and the context in which these were performed. Results: Participants were embedded in a wide range of clinical service areas, working with a diverse complement of health professionals. As clinical librarians, participants wore many hats, including expert searcher, teacher, content manager, and patient advocate. Unique aspects of how these roles played out included a sense of urgency surrounding searching activities, the broad dissemination of responses to clinical questions, and leverage of the roles of expert searcher, teacher, and content manager to advocate for patients. Conclusions: Detailed role descriptions of clinical librarians embedded in patient care teams suggest possible new practices for existing clinical librarians, provide direction for training new librarians working in patient care environments, and raise awareness of the clinical librarian specialty among current and budding health information professionals. PMID:23405048

Virtual reality training improves students' knowledge structures of medical concepts.

PubMed

Stevens, Susan M; Goldsmith, Timothy E; Summers, Kenneth L; Sherstyuk, Andrei; Kihmm, Kathleen; Holten, James R; Davis, Christopher; Speitel, Daniel; Maris, Christina; Stewart, Randall; Wilks, David; Saland, Linda; Wax, Diane; Panaiotis; Saiki, Stanley; Alverson, Dale; Caudell, Thomas P

2005-01-01

Virtual environments can provide training that is difficult to achieve under normal circumstances. Medical students can work on high-risk cases in a realistic, time-critical environment, where students practice skills in a cognitively demanding and emotionally compelling situation. Research from cognitive science has shown that as students acquire domain expertise, their semantic organization of core domain concepts become more similar to those of an expert's. In the current study, we hypothesized that students' knowledge structures would become more expert-like as a result of their diagnosing and treating a patient experiencing a hematoma within a virtual environment. Forty-eight medical students diagnosed and treated a hematoma case within a fully immersed virtual environment. Student's semantic organization of 25 case-related concepts was assessed prior to and after training. Students' knowledge structures became more integrated and similar to an expert knowledge structure of the concepts as a result of the learning experience. The methods used here for eliciting, representing, and evaluating knowledge structures offer a sensitive and objective means for evaluating student learning in virtual environments and medical simulations.

Expertise in ill-defined problem-solving domains as effective strategy use.

PubMed

Schunn, Christian D; McGregor, Mark U; Saner, Lelyn D

2005-12-01

Expertise consists of many different cognitive structures. Lemaire and Siegler (1995) have proposed a four-layered account of expertise from a strategies perspective: Experts have better strategies, tend to use strategies that are better overall more often, are better able to select the circumstances to which a strategy best applies, and are better able to execute a given strategy. Originally, this account came from work in simple, well-defined domains. We explored this account in the complex, ill-defined domain of platoon leadership. In Experiment 1A, we elicited free-text responses to leadership scenarios from novices, intermediates, and experts, finding expertise effects for strategy base rates and choice, but not for strategy existence or the number of strategies used. In Experiment 1B, we used a new group of experts to gather ratings of the execution accuracy of the responses in Experiment 1A and found expertise differences in the ability to execute the same strategies. We propose several elaborations to the original four-layered strategies account of expertise on the basis of these results.

Temporal dynamics of interference in Simon and Eriksen tasks considered within the context of a dual-process model.

PubMed

Mansfield, Karen L; van der Molen, Maurits W; Falkenstein, Michael; van Boxtel, Geert J M

2013-08-01

Behavioral and brain potential measures were employed to compare interference in Eriksen and Simon tasks. Assuming a dual-process model of interference elicited in speeded response tasks, we hypothesized that only lateralized stimuli in the Simon task induce fast S-R priming via direct unconditional processes, while Eriksen interference effects are induced later via indirect conditional processes. Delays to responses for incongruent trials were indeed larger in the Eriksen than in the Simon task. Only lateralized stimuli in the Simon task elicited early S-R priming, maximal at parietal areas. Incongruent flankers in the Eriksen task elicited interference later, visible as a lateralized N2. Eriksen interference also elicited an additional component (N350), which accounted for the larger behavioral interference effects in the Eriksen task. The findings suggest that interference and its resolution involve different processes for Simon and Eriksen tasks. Copyright © 2013 Elsevier Inc. All rights reserved.

Amplifying Each Patient's Voice: A Systematic Review of Multi-criteria Decision Analyses Involving Patients.

PubMed

Marsh, Kevin; Caro, J Jaime; Hamed, Alaa; Zaiser, Erica

2017-04-01

Qualitative methods tend to be used to incorporate patient preferences into healthcare decision making. However, for patient preferences to be given adequate consideration by decision makers they need to be quantified. Multi-criteria decision analysis (MCDA) is one way to quantify and capture the patient voice. The objective of this review was to report on existing MCDAs involving patients to support the future use of MCDA to capture the patient voice. MEDLINE and EMBASE were searched in June 2014 for English-language papers with no date restriction. The following search terms were used: 'multi-criteria decision*', 'multiple criteria decision*', 'MCDA', 'benefit risk assessment*', 'risk benefit assessment*', 'multicriteri* decision*', 'MCDM', 'multi-criteri* decision*'. Abstracts were included if they reported the application of MCDA to assess healthcare interventions where patients were the source of weights. Abstracts were excluded if they did not apply MCDA, such as discussions of how MCDA could be used; or did not evaluate healthcare interventions, such as MCDAs to assess the level of health need in a locality. Data were extracted on weighting method, variation in patient and expert preferences, and discussion on different weighting techniques. The review identified ten English-language studies that reported an MCDA to assess healthcare interventions and involved patients as a source of weights. These studies reported 12 applications of MCDA. Different methods of preference elicitation were employed: direct weighting in workshops; discrete choice experiment surveys; and the analytical hierarchy process using both workshops and surveys. There was significant heterogeneity in patient responses and differences between patients, who put greater weight on disease characteristics and treatment convenience, and experts, who put more weight on efficacy. The studies highlighted cognitive challenges associated with some weighting methods, though patients' views on their ability to undertake weighting tasks was positive. This review identified several recent examples of MCDA used to elicit patient preferences, which support the feasibility of using MCDA to capture the patient voice. Challenges identified included, how best to reflect the heterogeneity of patient preferences in decision making and how to manage the cognitive burden associated with some MCDA tasks.

Neural Indices of Semantic Processing in Early Childhood Distinguish Eventual Stuttering Persistence and Recovery.

PubMed

Kreidler, Kathryn; Hampton Wray, Amanda; Usler, Evan; Weber, Christine

2017-11-09

Maturation of neural processes for language may lag in some children who stutter (CWS), and event-related potentials (ERPs) distinguish CWS who have recovered from those who have persisted. The current study explores whether ERPs indexing semantic processing may distinguish children who will eventually persist in stuttering (CWS-ePersisted) from those who will recover from stuttering (CWS-eRecovered). Fifty-six 5-year-old children with normal receptive language listened to naturally spoken sentences in a story context. ERP components elicited for semantic processing (N400, late positive component [LPC]) were compared for CWS-ePersisted, CWS-eRecovered, and children who do not stutter (CWNS). The N400 elicited by semantic violations had a more focal scalp distribution (left lateralized and less anterior) in the CWS-eRecovered compared with CWS-ePersisted. Although the LPC elicited in CWS-eRecovered and CWNS did not differ, the LPC elicited in the CWS-ePersisted was smaller in amplitude compared with that in CWNS. ERPs elicited in 5-year-old CWS-eRecovered compared with CWS-ePersisted suggest that future recovery from stuttering may be associated with earlier maturation of semantic processes in the preschool years. Subtle differences in ERP indices offer a window into neural maturation processes for language and may help distinguish the course of stuttering development.

Automated interviews on clinical case reports to elicit directed acyclic graphs.

PubMed

Luciani, Davide; Stefanini, Federico M

2012-05-01

Setting up clinical reports within hospital information systems makes it possible to record a variety of clinical presentations. Directed acyclic graphs (Dags) offer a useful way of representing causal relations in clinical problem domains and are at the core of many probabilistic models described in the medical literature, like Bayesian networks. However, medical practitioners are not usually trained to elicit Dag features. Part of the difficulty lies in the application of the concept of direct causality before selecting all the causal variables of interest for a specific patient. We designed an automated interview to tutor medical doctors in the development of Dags to represent their understanding of clinical reports. Medical notions were analyzed to find patterns in medical reasoning that can be followed by algorithms supporting the elicitation of causal Dags. Clinical relevance was defined to help formulate only relevant questions by driving an expert's attention towards variables causally related to nodes already inserted in the graph. Key procedural features of the proposed interview are described by four algorithms. The automated interview comprises questions on medical notions, phrased in medical terms. The first elicitation session produces questions concerning the patient's chief complaints and the outcomes related to diseases serving as diagnostic hypotheses, their observable manifestations and risk factors. The second session focuses on questions that refine the initial causal paths by considering syndromes, dysfunctions, pathogenic anomalies, biases and effect modifiers. A case study concerning a gastro-enterological problem and one dealing with an infected patient illustrate the output produced by the algorithms, depending on the answers provided by the doctor. The proposed elicitation framework is characterized by strong consistency with medical background and by a progressive introduction of relevant medical topics. Revision and testing of the subjectively elicited Dag is performed by matching the collected answers with the evidence included in accepted sources of biomedical knowledge. Copyright © 2011 Elsevier B.V. All rights reserved.

Skilful force control in expert pianists.

PubMed

Oku, Takanori; Furuya, Shinichi

2017-05-01

Dexterous object manipulation in skilful behaviours such as surgery, craft making, and musical performance involves fast, precise, and efficient control of force with the fingers. A challenge in playing musical instruments is the requirement of independent control of the magnitude and rate of force production, which typically vary in relation to loudness and tempo. However, it is unknown how expert musicians skilfully control finger force to elicit tones with a wide range of loudness and tempi. Here, we addressed this issue by comparing the variation of spatiotemporal characteristics of force during repetitive and simultaneous piano keystrokes in relation to the loudness and tempo between pianists and musically untrained individuals. While the peak key-descending velocity varied with loudness but not with tempo in both groups, the peak and impulse of the key-depressing force were smaller in pianists than in the non-musicians, specifically when eliciting loud tones, suggesting superior energetic efficiency in the trained individuals. The key-depressing force was more consistent across strikes in pianists than in the non-musicians at all loudness levels but only at slow tempi, confirming expertise-dependency of precise force control. A regression analysis demonstrated that individual differences in the keystroke rates when playing at the fastest tempo across the trained pianists were negatively associated with the force impulse during the key depression but not with the peak force only at the loudest tone. This suggests that rapid reductions of force following the key depression plays a role in considerably fast performance of repetitive piano keystrokes.

Literally experts: expertise and the processing of analogical metaphors in pharmaceutical advertising.

PubMed

Delbaere, Marjorie; Smith, Malcolm C

2014-01-01

This research examined differences between novices and experts in processing analogical metaphors appearing in prescription drug advertisements. In contrast to previous studies on knowledge transfer, no evidence of the superiority of experts in processing metaphors was found. The results from an experiment suggest that expert consumers were more likely to process a metaphor in an ad literally than novices. Our findings point to a condition in which the expertise effect with processing analogies is not the linear relationship assumed in previous studies.

Program risk analysis handbook

NASA Technical Reports Server (NTRS)

Batson, R. G.

1987-01-01

NASA regulations specify that formal risk analysis be performed on a program at each of several milestones. Program risk analysis is discussed as a systems analysis approach, an iterative process (identification, assessment, management), and a collection of techniques. These techniques, which range from extremely simple to complex network-based simulation, are described in this handbook in order to provide both analyst and manager with a guide for selection of the most appropriate technique. All program risk assessment techniques are shown to be based on elicitation and encoding of subjective probability estimates from the various area experts on a program. Techniques to encode the five most common distribution types are given. Then, a total of twelve distinct approaches to risk assessment are given. Steps involved, good and bad points, time involved, and degree of computer support needed are listed. Why risk analysis should be used by all NASA program managers is discussed. Tools available at NASA-MSFC are identified, along with commercially available software. Bibliography (150 entries) and a program risk analysis check-list are provided.

The Viewpoints of Students and Evaluation Experts About Performance Processes of Faculty Member Evaluation at Mazandaran University of Medical Sciences, 2014.

PubMed

Ghahrani, Nassim; Balaghafari, Azita; Aligolbandi, Kobra; Vahedi, Mohammad; Siamian, Hasan

2015-06-01

One of the most common ways used in most of the countries and Iran to determine the status of teacher training is the evaluation by students. The most common method of evaluation is the survey questionnaire, the content of a number of questions about educational activities provided to the students. The researchers plan to evaluate the students' and experts' performances at Mazandaran University of Medical Sciences on the process of evaluating the performance of teachers, they examined in 2014. This study surveys the students and experts in the evaluation of faculty members' performance process. The study subjects were 3904 students and 37 evaluation expert of Mazandaran University of Medical Sciences. Using Cochran sampling formula of 350 students through proportional stratified random sampling were selected. The experts' viewpoint, method was used. Data collection tools consisted of 14 questions with answers Yes, or, I don't know. Descriptive Statistical analysis of the data and chi-square test was performed. From total of 350 students, 346 and the entire 37 evaluations expert participated in this study. Most of the students, 80 (23.12%) and the largest number of experts, 8 (21.62%) were from Sari Allied Medical Sciences Faculty. Most of the demographic information about gender were, 255 female students (74.56%) and 29 female experts (78.37%). In most age groups of students, 188 (55.62 percent) were in the category of 18 to 20 years, and the experts, 19 (51.35%) were in the category of 22 and 31 years. Most students, 232 of them (70.95%) were in semester 2 and 4. Most experts, 20 (54.05 percent) were under 10 years of work experience. The comparison between the views of students and experts in the evaluation process between the schools of Mazandaran University of Medical Sciences, Sari School of Nursing and Midwifery, there was difference between the opinions of experts and students (p-value=0.01. It showed 86.7% student and 33.3% of experts is satisfied with the evaluation process. on comparison of students and experts viewpoints on the implementation of the evaluation process, it is noteworthy that among students of different opinions on how the evaluation process. It worth to mention that there is insignificant difference between their viewpoints and majority of students and evaluation experts with the evaluation the process. In addition, the experts evaluated at different schools, most of them are satisfied the process.

Exploring Self-regulation of More or Less Expert College-Age Video Game Players: A Sequential Explanatory Design.

PubMed

Yilmaz Soylu, Meryem; Bruning, Roger H

2016-01-01

This study examined differences in self-regulation among college-age expert, moderately expert, and non-expert video game players in playing video games for fun. Winne's model of self-regulation (Winne, 2001) guided the study. The main assumption of this study was that expert video game players used more processes of self-regulation than the less-expert players. We surveyed 143 college students about their game playing frequency, habits, and use of self-regulation. Data analysis indicated that while playing recreational video games, expert gamers self-regulated more than moderately expert and non-expert players and moderately expert players used more processes of self-regulation than non-experts. Semi-structured interviews also were conducted with selected participants at each of the expertise levels. Qualitative follow-up analyses revealed five themes: (1) characteristics of expert video gamers, (2) conditions for playing a video game, (3) figuring out a game, (4) how gamers act and, (5) game context. Overall, findings indicated that playing a video game is a highly self-regulated activity and that becoming an expert video game player mobilizes multiple sets of self-regulation related skills and processes. These findings are seen as promising for educators desiring to encourage student self-regulation, because they indicate the possibility of supporting students via recreational video games by recognizing that their play includes processes of self-regulation.

Exploring Self-regulation of More or Less Expert College-Age Video Game Players: A Sequential Explanatory Design

PubMed Central

Yilmaz Soylu, Meryem; Bruning, Roger H.

2016-01-01

This study examined differences in self-regulation among college-age expert, moderately expert, and non-expert video game players in playing video games for fun. Winne's model of self-regulation (Winne, 2001) guided the study. The main assumption of this study was that expert video game players used more processes of self-regulation than the less-expert players. We surveyed 143 college students about their game playing frequency, habits, and use of self-regulation. Data analysis indicated that while playing recreational video games, expert gamers self-regulated more than moderately expert and non-expert players and moderately expert players used more processes of self-regulation than non-experts. Semi-structured interviews also were conducted with selected participants at each of the expertise levels. Qualitative follow-up analyses revealed five themes: (1) characteristics of expert video gamers, (2) conditions for playing a video game, (3) figuring out a game, (4) how gamers act and, (5) game context. Overall, findings indicated that playing a video game is a highly self-regulated activity and that becoming an expert video game player mobilizes multiple sets of self-regulation related skills and processes. These findings are seen as promising for educators desiring to encourage student self-regulation, because they indicate the possibility of supporting students via recreational video games by recognizing that their play includes processes of self-regulation. PMID:27729881

Corrugator activity confirms immediate negative affect in surprise

PubMed Central

Topolinski, Sascha; Strack, Fritz

2015-01-01

The emotion of surprise entails a complex of immediate responses, such as cognitive interruption, attention allocation to, and more systematic processing of the surprising stimulus. All these processes serve the ultimate function to increase processing depth and thus cognitively master the surprising stimulus. The present account introduces phasic negative affect as the underlying mechanism responsible for this switch in operating mode. Surprising stimuli are schema-discrepant and thus entail cognitive disfluency, which elicits immediate negative affect. This affect in turn works like a phasic cognitive tuning switching the current processing mode from more automatic and heuristic to more systematic and reflective processing. Directly testing the initial elicitation of negative affect by surprising events, the present experiment presented high and low surprising neutral trivia statements to N = 28 participants while assessing their spontaneous facial expressions via facial electromyography. High compared to low surprising trivia elicited higher corrugator activity, indicative of negative affect and mental effort, while leaving zygomaticus (positive affect) and frontalis (cultural surprise expression) activity unaffected. Future research shall investigate the mediating role of negative affect in eliciting surprise-related outcomes. PMID:25762956

Corrugator activity confirms immediate negative affect in surprise.

PubMed

Topolinski, Sascha; Strack, Fritz

2015-01-01

The emotion of surprise entails a complex of immediate responses, such as cognitive interruption, attention allocation to, and more systematic processing of the surprising stimulus. All these processes serve the ultimate function to increase processing depth and thus cognitively master the surprising stimulus. The present account introduces phasic negative affect as the underlying mechanism responsible for this switch in operating mode. Surprising stimuli are schema-discrepant and thus entail cognitive disfluency, which elicits immediate negative affect. This affect in turn works like a phasic cognitive tuning switching the current processing mode from more automatic and heuristic to more systematic and reflective processing. Directly testing the initial elicitation of negative affect by surprising events, the present experiment presented high and low surprising neutral trivia statements to N = 28 participants while assessing their spontaneous facial expressions via facial electromyography. High compared to low surprising trivia elicited higher corrugator activity, indicative of negative affect and mental effort, while leaving zygomaticus (positive affect) and frontalis (cultural surprise expression) activity unaffected. Future research shall investigate the mediating role of negative affect in eliciting surprise-related outcomes.

Development of Elderly Quality of Life Index – Eqoli: Item Reduction and Distribution into Dimensions

PubMed Central

Paschoal, Sérgio Márcio Pacheco; Filho, Wilson Jacob; Litvoc, Júlio

2008-01-01

OBJECTIVE To describe item reduction and its distribution into dimensions in the construction process of a quality of life evaluation instrument for the elderly. METHODS The sampling method was chosen by convenience through quotas, with selection of elderly subjects from four programs to achieve heterogeneity in the “health status”, “functional capacity”, “gender”, and “age” variables. The Clinical Impact Method was used, consisting of the spontaneous and elicited selection by the respondents of relevant items to the construct Quality of Life in Old Age from a previously elaborated item pool. The respondents rated each item’s importance using a 5-point Likert scale. The product of the proportion of elderly selecting the item as relevant (frequency) and the mean importance score they attributed to it (importance) represented the overall impact of that item in their quality of life (impact). The items were ordered according to their impact scores and the top 46 scoring items were grouped in dimensions by three experts. A review of the negative items was performed. RESULTS One hundred and ninety three people (122 women and 71 men) were interviewed. Experts distributed the 46 items into eight dimensions. Closely related items were grouped and dimensions not reaching the minimum expected number of items received additional items resulting in eight dimensions and 43 items. DISCUSSION The sample was heterogeneous and similar to what was expected. The dimensions and items demonstrated the multidimensionality of the construct. The Clinical Impact Method was appropriate to construct the instrument, which was named Elderly Quality of Life Index - EQoLI. An accuracy process will be examined in the future. PMID:18438571

Investigating the anticipatory nature of pattern perception in sport.

PubMed

Gorman, Adam D; Abernethy, Bruce; Farrow, Damian

2011-07-01

The aim of the present study was to examine the anticipatory nature of pattern perception in sport by using static and moving basketball patterns across three different display types. Participants of differing skill levels were included in order to determine whether the effects would be moderated by the knowledge and experience of the observer in the same manner reported previously for simple images. The results from a pattern recognition task showed that both expert and recreational participants were more likely to anticipate the next likely state of a pattern when it was presented as a moving video, but only the experts appeared to have the depth of understanding required to elicit the same anticipatory encoding for patterns presented as schematic images. The results extend those reported in previous research and provide further evidence of an anticipatory encoding in pattern perception for images containing complex, interrelated patterns.

The effect of falsely balanced reporting of the autism-vaccine controversy on vaccine safety perceptions and behavioral intentions.

PubMed

Dixon, Graham; Clarke, Christopher

2013-04-01

Controversy surrounding an autism-vaccine link has elicited considerable news media attention. Despite being widely discredited, research suggests that journalists report this controversy by presenting claims both for and against a link in a relatively 'balanced' fashion. To investigate how this reporting style influences judgments of vaccine risk, we randomly assigned 320 undergraduate participants to read a news article presenting either claims both for/against an autism-vaccine link, link claims only, no-link claims only or non-health-related information. Participants who read the balanced article were less certain that vaccines are safe, more likely to believe experts were less certain that vaccines are safe and less likely to have their future children vaccinated. Results suggest that balancing conflicting views of the autism-vaccine controversy may lead readers to erroneously infer the state of expert knowledge regarding vaccine safety and negatively impact vaccine intentions.

Good Food, Bad Food, and White Rice: Understanding Child Feeding Using Visual-Narrative Elicitation.

PubMed

Wentworth, Chelsea

2017-01-01

Visual-narrative elicitation, a process combining photo elicitation and pile sorting in applied medical anthropology, sheds light on food consumption patterns in urban areas of Vanuatu where childhood malnutrition is a persistent problem. Groups of participants took photographs of the foods they feed their children, and the resources and barriers they encounter in accessing foodstuffs. This revealed how imported and local foods are assigned value as "good" or "bad" foods when contributing to dietary diversity and creating appropriate meals for children, particularly in the context of consuming white rice. The process of gathering and working with photographs illuminated the complex negotiations in which caregivers engaged when making food and nutritional choices for their children. At the nexus of visual and medical anthropology, the visual-narrative elicitation process yielded nuanced, comprehensive understandings of how caregivers value the various foods they feed their children.

Eliciting User Requirements Using Appreciative Inquiry

ERIC Educational Resources Information Center

Gonzales, Carol Kernitzki

2010-01-01

Many software development projects fail because they do not meet the needs of users, are over-budget, and abandoned. To address this problem, the user requirements elicitation process was modified based on principles of Appreciative Inquiry. Appreciative Inquiry, commonly used in organizational development, aims to build organizations, processes,…

Making Information Useful: Engagement in the Sustained National Climate Assessment Process

NASA Astrophysics Data System (ADS)

Lough, G. C.; Cloyd, E.

2015-12-01

Creation of actionable information requires that the producers of that information understand the needs of the intended users and decision makers. To that end, the U.S. Global Change Research Program's sustained National Climate Assessment process includes a focus on engaging users through an inclusive, broad-based, and ongoing process. Such a process provides opportunities for scientific experts and decision makers to share knowledge about the climate-related issues, impacts, and potential response actions that are most important in a particular region or sector. Such a process is also highly transparent in order to produce results that are credible, salient, and legitimate for both scientists and decision makers, ultimately making the results extremely useful. To implement these principles, USGCRP implements a broad-based engagement strategy that invites participation from users and stakeholder communities and considers methods for communicating with potential users at every step. The strategy elicits contributions to help shape the framing of the assessment process and products, improve the transparency of the process, and increase the utility of the final information. Specific user inputs are gathered through workshops, public comment opportunities, town hall meetings, presentations, requests for information, submitted documents, and open meetings. Further, a network of contributors self-organizes around topics of interest to extend assessment activities to a wider range of user groups. Here, we describe the outcomes of these innovations in assessment engagement and identify clear successes, notable surprises, future evaluation needs, and areas for new ideas.

Knowledge elicitation for an operator assistant system in process control tasks

NASA Technical Reports Server (NTRS)

Boy, Guy A.

1988-01-01

A knowledge based system (KBS) methodology designed to study human machine interactions and levels of autonomy in allocation of process control tasks is presented. Users are provided with operation manuals to assist them in normal and abnormal situations. Unfortunately, operation manuals usually represent only the functioning logic of the system to be controlled. The user logic is often totally different. A method is focused on which illicits user logic to refine a KBS shell called an Operator Assistant (OA). If the OA is to help the user, it is necessary to know what level of autonomy gives the optimal performance of the overall man-machine system. For example, for diagnoses that must be carried out carefully by both the user and the OA, interactions are frequent, and processing is mostly sequential. Other diagnoses can be automated, in which the case the OA must be able to explain its reasoning in an appropriate level of detail. OA structure was used to design a working KBS called HORSES (Human Orbital Refueling System Expert System). Protocol analysis of pilots interacting with this system reveals that the a-priori analytical knowledge becomes more structured with training and the situation patterns more complex and dynamic. This approach can improve the a-priori understanding of human and automatic reasoning.

Participatory modeling and structured decision making

USGS Publications Warehouse

Robinson, Kelly F.; Fuller, Angela K.

2016-01-01

Structured decision making (SDM) provides a framework for making sound decisions even when faced with uncertainty, and is a transparent, defensible, and replicable method used to understand complex problems. A hallmark of SDM is the explicit incorporation of values and science, which often includes participation from multiple stakeholders, helping to garner trust and ultimately result in a decision that is more likely to be implemented. The core steps in the SDM process are used to structure thinking about natural resources management choices, and include: (1) properly defining the problem and the decision context, (2) determining the objectives that help describe the aspirations of the decision maker, (3) devising management actions or alternatives that can achieve those objectives, (4) evaluating the outcomes or consequences of each alternative on each of the objectives, (5) evaluating trade-offs, and (6) implementing the decision. Participatory modeling for SDM includes engaging stakeholders in some or all of the steps of the SDM process listed above. In addition, participatory modeling often is crucial for creating qualitative and quantitative models of how the system works, providing data for these models, and eliciting expert opinion when data are unavailable. In these ways, SDM provides a framework for decision making in natural resources management that includes participation from stakeholder groups throughout the process, including the modeling phase.

LinkIT: a ludic elicitation game for eliciting risk perceptions.

PubMed

Cao, Yan; McGill, William L

2013-06-01

The mental models approach, a leading strategy to develop risk communications, involves a time- and labor-intensive interview process and a lengthy questionnaire to elicit group-level risk perceptions. We propose that a similarity ratings approach for structural knowledge elicitation can be adopted to assist the risk mental models approach. The LinkIT game, inspired by games with a purpose (GWAP) technology, is a ludic elicitation tool designed to elicit group understanding of the relations between risk factors in a more enjoyable and productive manner when compared to traditional approaches. That is, consistent with the idea of ludic elicitation, LinkIT was designed to make the elicitation process fun and enjoyable in the hopes of increasing participation and data quality in risk studies. Like the mental models approach, the group mental model obtained via the LinkIT game can hence be generated and represented in a form of influence diagrams. In order to examine the external validity of LinkIT, we conducted a study to compare its performance with respect to a more conventional questionnaire-driven approach. Data analysis results conclude that the two group mental models elicited from the two approaches are similar to an extent. Yet, LinkIT was more productive and enjoyable than the questionnaire. However, participants commented that the current game has some usability concerns. This presentation summarizes the design and evaluation of the LinkIT game and suggests areas for future work. © 2012 Society for Risk Analysis.

Test blueprints for psychiatry residency in-training written examinations in Riyadh, Saudi Arabia

PubMed Central

Gaffas, Eisha M; Sequeira, Reginald P; Namla, Riyadh A Al; Al-Harbi, Khalid S

2012-01-01

Background The postgraduate training program in psychiatry in Saudi Arabia, which was established in 1997, is a 4-year residency program. Written exams comprising of multiple choice questions (MCQs) are used as a summative assessment of residents in order to determine their eligibility for promotion from one year to the next. Test blueprints are not used in preparing examinations. Objective To develop test blueprints for the written examinations used in the psychiatry residency program. Methods Based on the guidelines of four professional bodies, documentary analysis was used to develop global and detailed test blueprints for each year of the residency program. An expert panel participated during piloting and final modification of the test blueprints. Their opinion about the content, weightage for each content domain, and proportion of test items to be sampled in each cognitive category as defined by modified Bloom’s taxonomy were elicited. Results Eight global and detailed test blueprints, two for each year of the psychiatry residency program, were developed. The global test blueprints were reviewed by experts and piloted. Six experts participated in the final modification of test blueprints. Based on expert consensus, the content, total weightage for each content domain, and proportion of test items to be included in each cognitive category were determined for each global test blueprint. Experts also suggested progressively decreasing the weightage for recall test items and increasing problem solving test items in examinations, from year 1 to year 4 of the psychiatry residence program. Conclusion A systematic approach using a documentary and content analysis technique was used to develop test blueprints with additional input from an expert panel as appropriate. Test blueprinting is an important step to ensure the test validity in all residency programs. PMID:23762000

Test blueprints for psychiatry residency in-training written examinations in Riyadh, Saudi Arabia.

PubMed

Gaffas, Eisha M; Sequeira, Reginald P; Namla, Riyadh A Al; Al-Harbi, Khalid S

2012-01-01

The postgraduate training program in psychiatry in Saudi Arabia, which was established in 1997, is a 4-year residency program. Written exams comprising of multiple choice questions (MCQs) are used as a summative assessment of residents in order to determine their eligibility for promotion from one year to the next. Test blueprints are not used in preparing examinations. To develop test blueprints for the written examinations used in the psychiatry residency program. Based on the guidelines of four professional bodies, documentary analysis was used to develop global and detailed test blueprints for each year of the residency program. An expert panel participated during piloting and final modification of the test blueprints. Their opinion about the content, weightage for each content domain, and proportion of test items to be sampled in each cognitive category as defined by modified Bloom's taxonomy were elicited. Eight global and detailed test blueprints, two for each year of the psychiatry residency program, were developed. The global test blueprints were reviewed by experts and piloted. Six experts participated in the final modification of test blueprints. Based on expert consensus, the content, total weightage for each content domain, and proportion of test items to be included in each cognitive category were determined for each global test blueprint. Experts also suggested progressively decreasing the weightage for recall test items and increasing problem solving test items in examinations, from year 1 to year 4 of the psychiatry residence program. A systematic approach using a documentary and content analysis technique was used to develop test blueprints with additional input from an expert panel as appropriate. Test blueprinting is an important step to ensure the test validity in all residency programs.

A Participatory Method to Identify Root Determinants of Health: The Heart of the Matter

PubMed Central

Barnidge, Ellen; Baker, Elizabeth A.; Motton, Freda; Rose, Frank; Fitzgerald, Teresa

2010-01-01

Background Co-learning is one of the core principles of community-based participatory research (CBPR). Often, it is difficult to engage community members beyond those involved in the formal partnership in co-learning processes. However, to understand and address locally relevant root factors of health, it is essential to engage the broader community in participatory dialogues around these factors. Objective This article provides a glimpse into how using a photo-elicitation process allowed a community–academic partnership to engage community members in a participatory dialogue about root factors influencing health. The article details the decision to use photo-elicitation and describes the photo-elicitation method. Method Similar to a focus group process, photo-elicitation uses photographs and questions to prompt reflection and dialogue. Used in conjunction with an economic development framework, this method allows participants to discuss underlying, or root, community processes and structures that influence health. Conclusion Photo-elicitation is one way to engage community members in a participatory dialogue that stimulates action around root factors of health. To use this method successfully within a CBPR approach, it is important to build on existing relationships of trust among community and academic partners and create opportunities for community partners to determine the issues for discussion. PMID:20364079

Visualising community: using participant-driven photo-elicitation for research and application

Treesearch

Paul M. Van Auken; Svein J. Frisvoll; Susan I. Stewart

2010-01-01

Despite a contemporary socio-culture revolving around cultural consumption of imagery, metaphors, representations and "gaze", photo-elicitation is a rarely used method for social scientists and planners to acquire knowledge. In this paper, we discuss participant-driven photo-elicitation, a process in which participant photos are paired with in-depth...

Business Process Elicitation, Modeling, and Reengineering: Teaching and Learning with Simulated Environments

ERIC Educational Resources Information Center

Jeyaraj, Anand

2010-01-01

The design of enterprise information systems requires students to master technical skills for elicitation, modeling, and reengineering business processes as well as soft skills for information gathering and communication. These tacit skills and behaviors cannot be effectively taught students but rather experienced and learned by students. This…

Learning a Health Knowledge Graph from Electronic Medical Records.

PubMed

Rotmensch, Maya; Halpern, Yoni; Tlimat, Abdulhakim; Horng, Steven; Sontag, David

2017-07-20

Demand for clinical decision support systems in medicine and self-diagnostic symptom checkers has substantially increased in recent years. Existing platforms rely on knowledge bases manually compiled through a labor-intensive process or automatically derived using simple pairwise statistics. This study explored an automated process to learn high quality knowledge bases linking diseases and symptoms directly from electronic medical records. Medical concepts were extracted from 273,174 de-identified patient records and maximum likelihood estimation of three probabilistic models was used to automatically construct knowledge graphs: logistic regression, naive Bayes classifier and a Bayesian network using noisy OR gates. A graph of disease-symptom relationships was elicited from the learned parameters and the constructed knowledge graphs were evaluated and validated, with permission, against Google's manually-constructed knowledge graph and against expert physician opinions. Our study shows that direct and automated construction of high quality health knowledge graphs from medical records using rudimentary concept extraction is feasible. The noisy OR model produces a high quality knowledge graph reaching precision of 0.85 for a recall of 0.6 in the clinical evaluation. Noisy OR significantly outperforms all tested models across evaluation frameworks (p < 0.01).

Embodied mental rotation: a special link between egocentric transformation and the bodily self

PubMed Central

Kaltner, Sandra; Riecke, Bernhard E.; Jansen, Petra

2014-01-01

This experiment investigated the influence of motor expertise on object-based versus egocentric transformations in a chronometric mental rotation task using images of either the own or another person’s body as stimulus material. According to the embodied cognition viewpoint, we hypothesized motor-experts to outperform non-motor experts specifically in the egocentric condition because of higher kinesthetic representation and motor simulations compared to object-based transformations. In line with this, we expected that images of the own body are solved faster than another person’s body stimuli. Results showed a benefit of motor expertise and representations of another person’s body, but only for the object-based transformation task. That is, this other-advantage diminishes in egocentric transformations. Since motor experts did not show any specific expertise in rotational movements, we concluded that using human bodies as stimulus material elicits embodied spatial transformations, which facilitates performance exclusively for egocentric transformations. Regarding stimulus material, the other-advantage ascribed to increased self-awareness-consciousness distracting attention-demanding resources, disappeared in the egocentric condition. This result may be due to the stronger link between the bodily self and motor representations compared to that emerging in object-based transformations. PMID:24917832

Identifying and measuring stakeholder preferences for disease prioritisation: A case study of the pig industry in Australia.

PubMed

Brookes, V J; Hernández-Jover, M; Neslo, R; Cowled, B; Holyoake, P; Ward, M P

2014-01-01

We describe stakeholder preference modelling using a combination of new and recently developed techniques to elicit criterion weights to incorporate into a multi-criteria decision analysis framework to prioritise exotic diseases for the pig industry in Australia. Australian pig producers were requested to rank disease scenarios comprising nine criteria in an online questionnaire. Parallel coordinate plots were used to visualise stakeholder preferences, which aided identification of two diverse groups of stakeholders - one group prioritised diseases with impacts on livestock, and the other group placed more importance on diseases with zoonotic impacts. Probabilistic inversion was used to derive weights for the criteria to reflect the values of each of these groups, modelling their choice using a weighted sum value function. Validation of weights against stakeholders' rankings for scenarios based on real diseases showed that the elicited criterion weights for the group who prioritised diseases with livestock impacts were a good reflection of their values, indicating that the producers were able to consistently infer impacts from the disease information in the scenarios presented to them. The highest weighted criteria for this group were attack rate and length of clinical disease in pigs, and market loss to the pig industry. The values of the stakeholders who prioritised zoonotic diseases were less well reflected by validation, indicating either that the criteria were inadequate to consistently describe zoonotic impacts, the weighted sum model did not describe stakeholder choice, or that preference modelling for zoonotic diseases should be undertaken separately from livestock diseases. Limitations of this study included sampling bias, as the group participating were not necessarily representative of all pig producers in Australia, and response bias within this group. The method used to elicit criterion weights in this study ensured value trade-offs between a range of potential impacts, and that the weights were implicitly related to the scale of measurement of disease criteria. Validation of the results of the criterion weights against real diseases - a step rarely used in MCDA - added scientific rigour to the process. The study demonstrated that these are useful techniques for elicitation of criterion weights for disease prioritisation by stakeholders who are not disease experts. Preference modelling for zoonotic diseases needs further characterisation in this context. Copyright © 2013 Elsevier B.V. All rights reserved.

The Viewpoints of Students and Evaluation Experts About Performance Processes of Faculty Member Evaluation at Mazandaran University of Medical Sciences, 2014

PubMed Central

Ghahrani, Nassim; Balaghafari, Azita; Aligolbandi, Kobra; Vahedi, Mohammad; Siamian, Hasan

2015-01-01

Background and purpose: One of the most common ways used in most of the countries and Iran to determine the status of teacher training is the evaluation by students. The most common method of evaluation is the survey questionnaire, the content of a number of questions about educational activities provided to the students. The researchers plan to evaluate the students’ and experts’ performances at Mazandaran University of Medical Sciences on the process of evaluating the performance of teachers, they examined in 2014. Materials and methods: This study surveys the students and experts in the evaluation of faculty members’ performance process. The study subjects were 3904 students and 37 evaluation expert of Mazandaran University of Medical Sciences. Using Cochran sampling formula of 350 students through proportional stratified random sampling were selected. The experts’ viewpoint, method was used. Data collection tools consisted of 14 questions with answers Yes, or, I don’t know. Descriptive Statistical analysis of the data and chi-square test was performed. Results: From total of 350 students, 346 and the entire 37 evaluations expert participated in this study. Most of the students, 80 (23.12%) and the largest number of experts, 8 (21.62%) were from Sari Allied Medical Sciences Faculty. Most of the demographic information about gender were, 255 female students (74.56%) and 29 female experts (78.37%). In most age groups of students, 188 (55.62 percent) were in the category of 18 to 20 years, and the experts, 19 (51.35%) were in the category of 22 and 31 years. Most students, 232 of them (70.95%) were in semester 2 and 4. Most experts, 20 (54.05 percent) were under 10 years of work experience. The comparison between the views of students and experts in the evaluation process between the schools of Mazandaran University of Medical Sciences, Sari School of Nursing and Midwifery, there was difference between the opinions of experts and students (p-value=0.01. It showed 86.7% student and 33.3% of experts is satisfied with the evaluation process. Conclusion: on comparison of students and experts viewpoints on the implementation of the evaluation process, it is noteworthy that among students of different opinions on how the evaluation process. It worth to mention that there is insignificant difference between their viewpoints and majority of students and evaluation experts with the evaluation the process. In addition, the experts evaluated at different schools, most of them are satisfied the process. PMID:26236169

Photo-Elicitation and Visual Semiotics: A Unique Methodology for Studying Inclusion for Children with Disabilities

ERIC Educational Resources Information Center

Stockall, Nancy

2013-01-01

The methodology in this paper discusses the use of photographs as an elicitation strategy that can reveal the thinking processes of participants in a qualitatively rich manner. Photo-elicitation techniques combined with a Piercian semiotic perspective offer a unique method for creating a frame of action for later participant analysis. Illustrative…

Pre-Service Teachers' Modelling Processes through Engagement with Model Eliciting Activities with a Technological Tool

ERIC Educational Resources Information Center

Daher, Wajeeh M.; Shahbari, Juhaina Awawdeh

2015-01-01

Engaging mathematics students with modelling activities helps them learn mathematics meaningfully. This engagement, in the case of model eliciting activities, helps the students elicit mathematical models by interpreting real-world situation in mathematical ways. This is especially true when the students utilize technology to build the models.…

Cognitive Validity: Can Multiple-Choice Items Tap Historical Thinking Processes?

ERIC Educational Resources Information Center

Smith, Mark D.

2017-01-01

Cognitive validity examines the relationship between what an assessment aims to measure and what it actually elicits from test takers. The present study examined whether multiple-choice items from the National Assessment of Educational Progress (NAEP) grade 12 U.S. history exam elicited the historical thinking processes they were designed to…

Improving Requirements Generation Thoroughness in User-Centered Workshops: The Role of Prompting and Shared User Stories

ERIC Educational Resources Information Center

Read, Aaron

2013-01-01

The rise of stakeholder centered software development has led to organizations engaging users early in the development process to help define system requirements. To facilitate user involvement in the requirements elicitation process, companies can use Group Support Systems (GSS) to conduct requirements elicitation workshops. The effectiveness of…

Event-related potential N270 as an index of social information conflict in explicit processing.

PubMed

Wang, Pei; Tan, Chen-Hao; Li, Yu; Zhang, Qin; Wang, Yi-Bo; Luo, Jun-Long

2018-01-01

As N270 has been widely shown to be sensitive to nonsocial information conflict, the present study investigated whether social information conflict can elicit increased N270 in either explicit or implicit processing conditions. Gender stereotype-related picture-word pairs and picture-word pairs in specific colors were used as social and nonsocial information, respectively. Participants performed an explicit task based on the S1-S2 paradigm in Study 1, and both social and nonsocial information conditions elicited larger N270 than the no-conflict condition. In Study 2, participants performed a word judgment task that was modified from the S1-S2 paradigm of Study 1. However, neither social information nor nonsocial information elicited larger N270 within the conflict condition. Social trials generally elicited a more negative ERP waveform than nonsocial trials overall. These findings suggest that N270 may reflect the processing of social information conflict only in explicit conditions and also that the cognitive basis of N270 is thus a general but explicit processing of working memory representation conflict. Copyright © 2017. Published by Elsevier B.V.

Bothered by abstractness or engaged by cohesion? Experts' explanations enhance novices' deep-learning.

PubMed

Lachner, Andreas; Nückles, Matthias

2015-03-01

Experts' explanations have been shown to better enhance novices' transfer as compared with advanced students' explanations. Based on research on expertise and text comprehension, we investigated whether the abstractness or the cohesion of experts' and intermediates' explanations accounted for novices' learning. In Study 1, we showed that the superior cohesion of experts' explanations accounted for most of novices' transfer, whereas the degree of abstractness did not impact novices' transfer performance. In Study 2, we investigated novices' processing while learning with experts' and intermediates' explanations. We found that novices studying experts' explanations actively self-regulated their processing of the explanations, as they showed mainly deep-processing activities, whereas novices learning with intermediates' explanations were mainly engaged in shallow-processing activities by paraphrasing the explanations. Thus, we concluded that subject-matter expertise is a crucial prerequisite for instructors. Despite the abstract character of experts' explanations, their subject-matter expertise enables them to generate highly cohesive explanations that serve as a valuable scaffold for students' construction of flexible knowledge by engaging them in deep-level processing. PsycINFO Database Record (c) 2015 APA, all rights reserved.

Breadth in Design Problem Scoping: Using Insights from Experts to Investigate Student Processes. Research Brief

ERIC Educational Resources Information Center

Morozov, Andrew; Kilgore, Deborah; Atman, Cynthia

2007-01-01

In this study, the authors used two methods for analyzing expert data: verbal protocol analysis (VPA) and narrative analysis. VPA has been effectively used to describe the design processes employed by engineering students, expert designers, and expert-novice comparative research. VPA involves asking participants to "think aloud" while…

Using expert opinion surveys to rank threats to endangered species: a case study with sea turtles.

PubMed

Donlan, C Josh; Wingfield, Dana K; Crowder, Larry B; Wilcox, Chris

2010-12-01

Little is known about how specific anthropogenic hazards affect the biology of organisms. Quantifying the effect of regional hazards is particularly challenging for species such as sea turtles because they are migratory, difficult to study, long lived, and face multiple anthropogenic threats. Expert elicitation, a technique used to synthesize opinions of experts while assessing uncertainty around those views, has been in use for several decades in the social science and risk assessment sectors. We conducted an internet-based survey to quantify expert opinion on the relative magnitude of anthropogenic hazards to sea turtle populations at the regional level. Fisheries bycatch and coastal development were most often ranked as the top hazards to sea turtle species in a geographic region. Nest predation and direct take followed as the second and third greatest threats, respectively. Survey results suggest most experts believe sea turtles are threatened by multiple factors, including substantial at-sea threats such as fisheries bycatch. Resources invested by the sea turtle community, however, appear biased toward terrestrial-based impacts. Results from the survey are useful for conservation planning because they provide estimates of relative impacts of hazards on sea turtles and a measure of consensus on the magnitude of those impacts among researchers and practitioners. Our survey results also revealed patterns of expert bias, which we controlled for in our analysis. Respondents with no experience with respect to a sea turtle species tended to rank hazards affecting that sea turtle species higher than respondents with experience. A more-striking pattern was with hazard-based expertise: the more experience a respondent had with a specific hazard, the higher the respondent scored the impact of that hazard on sea turtle populations. Bias-controlled expert opinion surveys focused on threatened species and their hazards can help guide and expedite species recovery plans. © 2010 Society for Conservation Biology.

Early Childhood Stuttering and Electrophysiological Indices of Language Processing

PubMed Central

Weber-Fox, Christine; Wray, Amanda Hampton; Arnold, Hayley

2013-01-01

We examined neural activity mediating semantic and syntactic processing in 27 preschool-age children who stutter (CWS) and 27 preschool-age children who do not stutter (CWNS) matched for age, nonverbal IQ and language abilities. All participants displayed language abilities and nonverbal IQ within the normal range. Event-related brain potentials (ERPs) were elicited while participants watched a cartoon video and heard naturally spoken sentences that were either correct or contained semantic or syntactic (phrase structure) violations. ERPs in CWS, compared to CWNS, were characterized by longer N400 peak latencies elicited by semantic processing. In the CWS, syntactic violations elicited greater negative amplitudes for the early time window (150–350 ms) over medial sites compared to CWNS. Additionally, the amplitude of the P600 elicited by syntactic violations relative to control words was significant over the left hemisphere for the CWNS but showed the reverse pattern in CWS, a robust effect only over the right hemisphere. Both groups of preschoolage children demonstrated marked and differential effects for neural processes elicited by semantic and phrase structure violations; however, a significant proportion of young CWS exhibit differences in the neural functions mediating language processing compared to CWNS despite normal language abilities. These results are the first to show that differences in event-related brain potentials reflecting language processing occur as early as the preschool years in CWS and provide the first evidence that atypical lateralization of hemispheric speech/language functions previously observed in the brains of adults who stutter begin to emerge near the onset of developmental stuttering. PMID:23773672

Modelling health care processes for eliciting user requirements: a way to link a quality paradigm and clinical information system design.

PubMed

Staccini, P; Joubert, M; Quaranta, J F; Fieschi, D; Fieschi, M

2000-01-01

Hospital information systems have to support quality improvement objectives. The design issues of health care information system can be classified into three categories: 1) time-oriented and event-labelled storage of patient data; 2) contextual support of decision-making; 3) capabilities for modular upgrading. The elicitation of the requirements has to meet users' needs in relation to both the quality (efficacy, safety) and the monitoring of all health care activities (traceability). Information analysts need methods to conceptualize clinical information systems that provide actors with individual benefits and guide behavioural changes. A methodology is proposed to elicit and structure users' requirements using a process-oriented analysis, and it is applied to the field of blood transfusion. An object-oriented data model of a process has been defined in order to identify its main components: activity, sub-process, resources, constrains, guidelines, parameters and indicators. Although some aspects of activity, such as "where", "what else", and "why" are poorly represented by the data model alone, this method of requirement elicitation fits the dynamic of data input for the process to be traced. A hierarchical representation of hospital activities has to be found for this approach to be generalised within the organisation, for the processes to be interrelated, and for their characteristics to be shared.

Modelling health care processes for eliciting user requirements: a way to link a quality paradigm and clinical information system design.

PubMed

Staccini, P; Joubert, M; Quaranta, J F; Fieschi, D; Fieschi, M

2001-12-01

Healthcare institutions are looking at ways to increase their efficiency by reducing costs while providing care services with a high level of safety. Thus, hospital information systems have to support quality improvement objectives. The elicitation of the requirements has to meet users' needs in relation to both the quality (efficacy, safety) and the monitoring of all health care activities (traceability). Information analysts need methods to conceptualise clinical information systems that provide actors with individual benefits and guide behavioural changes. A methodology is proposed to elicit and structure users' requirements using a process-oriented analysis, and it is applied to the blood transfusion process. An object-oriented data model of a process has been defined in order to organise the data dictionary. Although some aspects of activity, such as 'where', 'what else', and 'why' are poorly represented by the data model alone, this method of requirement elicitation fits the dynamic of data input for the process to be traced. A hierarchical representation of hospital activities has to be found for the processes to be interrelated, and for their characteristics to be shared, in order to avoid data redundancy and to fit the gathering of data with the provision of care.

Valence and magnitude ambiguity in feedback processing.

PubMed

Gu, Ruolei; Feng, Xue; Broster, Lucas S; Yuan, Lu; Xu, Pengfei; Luo, Yue-Jia

2017-05-01

Outcome feedback which indicates behavioral consequences are crucial for reinforcement learning and environmental adaptation. Nevertheless, outcome information in daily life is often totally or partially ambiguous. Studying how people interpret this kind of information would provide important knowledge about the human evaluative system. This study concentrates on the neural processing of partially ambiguous feedback, that is, either its valence or magnitude is unknown to participants. To address this topic, we sequentially presented valence and magnitude information; electroencephalography (EEG) response to each kind of presentation was recorded and analyzed. The event-related potential components feedback-related negativity (FRN) and P3 were used as indices of neural activity. Consistent with previous literature, the FRN elicited by ambiguous valence was not significantly different from that elicited by negative valence. On the other hand, the FRN elicited by ambiguous magnitude was larger than both the large and small magnitude, indicating the motivation to seek unambiguous magnitude information. The P3 elicited by ambiguous valence and ambiguous magnitude was not significantly different from that elicited by negative valence and small magnitude, respectively, indicating the emotional significance of feedback ambiguity. Finally, the aforementioned effects also manifested in the stage of information integration. These findings indicate both similarities and discrepancies between the processing of valence ambiguity and that of magnitude ambiguity, which may help understand the mechanisms of ambiguous information processing.

Strategic conservation planning for the Eastern North Carolina/Southeastern Virginia Strategic Habitat Conservation Team

USGS Publications Warehouse

Alexander-Vaughn, Louise B.; Collazo, Jaime A.; Drew, C. Ashton

2014-01-01

The Eastern North Carolina/Southeastern Virginia Strategic Habitat Conservation Team (ENCSEVA) is a partnership among local federal agencies and programs with a mission to apply Strategic Habitat Conservation to accomplish priority landscape-level conservation within its geographic region. ENCSEVA seeks to further landscape-scale conservation through collaboration with local partners. To accomplish this mission, ENCSEVA is developing a comprehensive Strategic Habitat Conservation Plan (Plan) to provide guidance for its members, partners, and collaborators by establishing mutual conservation goals, objectives, strategies, and metrics to gauge the success of conservation efforts. Identifying common goals allows the ENCSEVA team to develop strategies that leverage joint resources and are more likely to achieve desired impacts across the landscape. The Plan will also provide an approach for ENCSEVA to meet applied research needs (identify knowledge gaps), foster adaptive management principles, identify conservation priorities, prioritize threats (including potential impacts of climate change), and identify the required capacity to implement strategies to create more resilient landscapes. ENCSEVA seeks to support the overarching goals of the South Atlantic Landscape Conservation Cooperative (SALCC) and to provide scientific and technical support for conservation at landscape scales as well as inform the management of natural resources in response to shifts in climate, habitat fragmentation and loss, and other landscape-level challenges (South Atlantic LCC 2012). The ENCSEVA ecoregion encompasses the northern third of the SALCC geography and offers a unique opportunity to apply landscape conservation at multiple scales through the guidance of local conservation and natural resource management efforts and by reporting metrics that reflect the effectiveness of those efforts (Figure 1). The Environmental Decision Analysis Team, housed within the North Carolina Cooperative Fish and Wildlife Research Unit at North Carolina State University, is assisting the ENCSEVA team in developing a scientifically sound basis for the Plan though the elicitation of expert knowledge and the organization of that knowledge using the Open Standards for the Practice of Conservation. The Open Standards for the Practice of Conservation is a framework that is well suited to incorporating decision-making tools such as Structured Decision Making and provides a multi-step process to conceptually organize conservation projects in a manner that enhances the rigor and transparency of expert and knowledge-based plans. It helps define explicit pathways from 2 planned conservation activities and ultimate impact, as well as indicators to measure success (Stem et al. 2005). Specifically, the framework identifies conservation targets, key ecological attributes, threats, and associated indicators to monitor responses given the implementation of a conservation action (Conservation Measures Partnership 2007). This report serves to provide a scientific foundation for the Plan by summarizing the expert opinion of wildlife biologists, ecologists, hydrologists, researchers, natural resource managers, and conservation practitioners regarding five environments (wetlands, riverine systems, estuaries, uplands, and barrier islands) within the ENCSEVA geography. Specifically, this report describes (1) the approach to elicit expert knowledge meant to support the strategic plan, (2) how this knowledge can inform collaborative conservation planning, and (3) a summary of opportunities available for the ENCSEVA team to address threats and impacts associated with climate change within the ecoregion.

Eliciting and Developing Teachers' Conceptions of Random Processes in a Probability and Statistics Course

ERIC Educational Resources Information Center

Smith, Toni M.; Hjalmarson, Margret A.

2013-01-01

The purpose of this study is to examine prospective mathematics specialists' engagement in an instructional sequence designed to elicit and develop their understandings of random processes. The study was conducted with two different sections of a probability and statistics course for K-8 teachers. Thirty-two teachers participated. Video analyses…

Cost Risk Analysis Based on Perception of the Engineering Process

NASA Technical Reports Server (NTRS)

Dean, Edwin B.; Wood, Darrell A.; Moore, Arlene A.; Bogart, Edward H.

1986-01-01

In most cost estimating applications at the NASA Langley Research Center (LaRC), it is desirable to present predicted cost as a range of possible costs rather than a single predicted cost. A cost risk analysis generates a range of cost for a project and assigns a probability level to each cost value in the range. Constructing a cost risk curve requires a good estimate of the expected cost of a project. It must also include a good estimate of expected variance of the cost. Many cost risk analyses are based upon an expert's knowledge of the cost of similar projects in the past. In a common scenario, a manager or engineer, asked to estimate the cost of a project in his area of expertise, will gather historical cost data from a similar completed project. The cost of the completed project is adjusted using the perceived technical and economic differences between the two projects. This allows errors from at least three sources. The historical cost data may be in error by some unknown amount. The managers' evaluation of the new project and its similarity to the old project may be in error. The factors used to adjust the cost of the old project may not correctly reflect the differences. Some risk analyses are based on untested hypotheses about the form of the statistical distribution that underlies the distribution of possible cost. The usual problem is not just to come up with an estimate of the cost of a project, but to predict the range of values into which the cost may fall and with what level of confidence the prediction is made. Risk analysis techniques that assume the shape of the underlying cost distribution and derive the risk curve from a single estimate plus and minus some amount usually fail to take into account the actual magnitude of the uncertainty in cost due to technical factors in the project itself. This paper addresses a cost risk method that is based on parametric estimates of the technical factors involved in the project being costed. The engineering process parameters are elicited from the engineer/expert on the project and are based on that expert's technical knowledge. These are converted by a parametric cost model into a cost estimate. The method discussed makes no assumptions about the distribution underlying the distribution of possible costs, and is not tied to the analysis of previous projects, except through the expert calibrations performed by the parametric cost analyst.

Eliciting interval beliefs: An experimental study

PubMed Central

Peeters, Ronald; Wolk, Leonard

2017-01-01

In this paper we study the interval scoring rule as a mechanism to elicit subjective beliefs under varying degrees of uncertainty. In our experiment, subjects forecast the termination time of a time series to be generated from a given but unknown stochastic process. Subjects gradually learn more about the underlying process over time and hence the true distribution over termination times. We conduct two treatments, one with a high and one with a low volatility process. We find that elicited intervals are better when subjects are facing a low volatility process. In this treatment, participants learn to position their intervals almost optimally over the course of the experiment. This is in contrast with the high volatility treatment, where subjects, over the course of the experiment, learn to optimize the location of their intervals but fail to provide the optimal length. PMID:28380020

Using task analysis to improve the requirements elicitation in health information system.

PubMed

Teixeira, Leonor; Ferreira, Carlos; Santos, Beatriz Sousa

2007-01-01

This paper describes the application of task analysis within the design process of a Web-based information system for managing clinical information in hemophilia care, in order to improve the requirements elicitation and, consequently, to validate the domain model obtained in a previous phase of the design process (system analysis). The use of task analysis in this case proved to be a practical and efficient way to improve the requirements engineering process by involving users in the design process.

Photolithography diagnostic expert systems: a systematic approach to problem solving in a wafer fabrication facility

NASA Astrophysics Data System (ADS)

Weatherwax Scott, Caroline; Tsareff, Christopher R.

1990-06-01

One of the main goals of process engineering in the semiconductor industry is to improve wafer fabrication productivity and throughput. Engineers must work continuously toward this goal in addition to performing sustaining and development tasks. To accomplish these objectives, managers must make efficient use of engineering resources. One of the tools being used to improve efficiency is the diagnostic expert system. Expert systems are knowledge based computer programs designed to lead the user through the analysis and solution of a problem. Several photolithography diagnostic expert systems have been implemented at the Hughes Technology Center to provide a systematic approach to process problem solving. This systematic approach was achieved by documenting cause and effect analyses for a wide variety of processing problems. This knowledge was organized in the form of IF-THEN rules, a common structure for knowledge representation in expert system technology. These rules form the knowledge base of the expert system which is stored in the computer. The systems also include the problem solving methodology used by the expert when addressing a problem in his area of expertise. Operators now use the expert systems to solve many process problems without engineering assistance. The systems also facilitate the collection of appropriate data to assist engineering in solving unanticipated problems. Currently, several expert systems have been implemented to cover all aspects of the photolithography process. The systems, which have been in use for over a year, include wafer surface preparation (HMDS), photoresist coat and softbake, align and expose on a wafer stepper, and develop inspection. These systems are part of a plan to implement an expert system diagnostic environment throughout the wafer fabrication facility. In this paper, the systems' construction is described, including knowledge acquisition, rule construction, knowledge refinement, testing, and evaluation. The roles played by the process engineering expert and the knowledge engineer are discussed. The features of the systems are shown, particularly the interactive quality of the consultations and the ease of system use.

Expert system prototype developments for NASA-KSC business and engineering applications

NASA Technical Reports Server (NTRS)

Ragusa, James M.; Gonzalez, Avelino J.

1988-01-01

Prototype expert systems developed for a variety of NASA projects in the business/management and engineering domains are discussed. Business-related problems addressed include an assistant for simulating launch vehicle processing, a plan advisor for the acquisition of automated data processing equipment, and an expert system for the identification of customer requirements. Engineering problems treated include an expert system for detecting potential ignition sources in LOX and gaseous-oxygen transportation systems and an expert system for hazardous-gas detection.

Development of a standardised training curriculum for robotic surgery: a consensus statement from an international multidisciplinary group of experts.

PubMed

Ahmed, Kamran; Khan, Reenam; Mottrie, Alexandre; Lovegrove, Catherine; Abaza, Ronny; Ahlawat, Rajesh; Ahlering, Thomas; Ahlgren, Goran; Artibani, Walter; Barret, Eric; Cathelineau, Xavier; Challacombe, Ben; Coloby, Patrick; Khan, Muhammad S; Hubert, Jacques; Michel, Maurice Stephan; Montorsi, Francesco; Murphy, Declan; Palou, Joan; Patel, Vipul; Piechaud, Pierre-Thierry; Van Poppel, Hendrik; Rischmann, Pascal; Sanchez-Salas, Rafael; Siemer, Stefan; Stoeckle, Michael; Stolzenburg, Jens-Uwe; Terrier, Jean-Etienne; Thüroff, Joachim W; Vaessen, Christophe; Van Der Poel, Henk G; Van Cleynenbreugel, Ben; Volpe, Alessandro; Wagner, Christian; Wiklund, Peter; Wilson, Timothy; Wirth, Manfred; Witt, Jörn; Dasgupta, Prokar

2015-07-01

To explore the views of experts about the development and validation of a robotic surgery training curriculum, and how this should be implemented. An international expert panel was invited to a structured session for discussion. The study was of a mixed design, including qualitative and quantitative components based on focus group interviews during the European Association of Urology (EAU) Robotic Urology Section (ERUS) (2012), EAU (2013) and ERUS (2013) meetings. After introduction to the aims, principles and current status of the curriculum development, group responses were elicited. After content analysis of recorded interviews generated themes were discussed at the second meeting, where consensus was achieved on each theme. This discussion also underwent content analysis, and was used to draft a curriculum proposal. At the third meeting, a quantitative questionnaire about this curriculum was disseminated to attendees to assess the level of agreement with the key points. In all, 150 min (19 pages) of the focus group discussion was transcribed (21 316 words). Themes were agreed by two raters (median agreement κ 0.89) and they included: need for a training curriculum (inter-rater agreement κ 0.85); identification of learning needs (κ 0.83); development of the curriculum contents (κ 0.81); an overview of available curricula (κ 0.79); settings for robotic surgery training ((κ 0.89); assessment and training of trainers (κ 0.92); requirements for certification and patient safety (κ 0.83); and need for a universally standardised curriculum (κ 0.78). A training curriculum was proposed based on the above discussions. This group proposes a multi-step curriculum for robotic training. Studies are in process to validate the effectiveness of the curriculum and to assess transfer of skills to the operating room. © 2015 The Authors BJU International © 2015 BJU International Published by John Wiley & Sons Ltd.

Bridging the Expert and Citizen Divide: Integrating Public Deliberation to Inform NASA's Asteroid Initiative

NASA Astrophysics Data System (ADS)

Farooque, M.; Tomblin, D. C.; Sittenfeld, D.

2017-12-01

The demand for public engagement in upstream science and technology is fast becoming mainstream. From the National Academies to the European Commission, from geoengineering to gene editing, from artificial intelligence to synthetic biology—there is a growing recognition of the socio-technical nature of the inherent challenges and a variety of calls for earlier and sustained engagement with diverse stakeholders and the general public. Despite a significant increase in the number and sophistication of approaches, institutional and cultural barriers remain, particularly in linking techno-scientific discourse with socio-political discourse. We will report on a 2014 study to use Participatory Technology Assessment (pTA), a method for eliciting informed, deliberative, diverse, and representative citizen views prior to making decisions about science and technology, to inform upstream decisions concerning NASA's Asteroid Initiative. In partnership with NASA, the Expert and Citizen Assessment of Science and Technology (ECAST) network conducted pTA forums in Boston and Phoenix to assess citizens' preferences and values about potential options for asteroid detection, mitigation, and retrieval and the deployment of the Capability Driven Framework as a planning instrument for a journey to Mars. We describe the three-step trans-disciplinary research process applied for (a) issue framing and deliberation design, (b) content development and participant recruitment, and (c) value assessments and results integration. We present result highlights, describe how they were used, and what kind of impact they had on decisions made by NASA. We discuss the influence this project had on subsequent initiatives by NOAA for climate resilience planning and by DOE for nuclear waste management. We conclude with our thoughts on (i) a new institutional model and (ii) research, application and adaptation opportunities going forward focusing on the role pTA can play to bridge the divide between societal needs and experts decision-making.

A parallel expert system for the control of a robotic air vehicle

NASA Technical Reports Server (NTRS)

Shakley, Donald; Lamont, Gary B.

1988-01-01

Expert systems can be used to govern the intelligent control of vehicles, for example the Robotic Air Vehicle (RAV). Due to the nature of the RAV system the associated expert system needs to perform in a demanding real-time environment. The use of a parallel processing capability to support the associated expert system's computational requirement is critical in this application. Thus, algorithms for parallel real-time expert systems must be designed, analyzed, and synthesized. The design process incorporates a consideration of the rule-set/face-set size along with representation issues. These issues are looked at in reference to information movement and various inference mechanisms. Also examined is the process involved with transporting the RAV expert system functions from the TI Explorer, where they are implemented in the Automated Reasoning Tool (ART), to the iPSC Hypercube, where the system is synthesized using Concurrent Common LISP (CCLISP). The transformation process for the ART to CCLISP conversion is described. The performance characteristics of the parallel implementation of these expert systems on the iPSC Hypercube are compared to the TI Explorer implementation.

Making Information Useful: Engagement in the National Climate Assessment Process

NASA Astrophysics Data System (ADS)

Lough, G. C.; Cloyd, E.

2014-12-01

Creation of actionable information requires that the producers of that information understand the needs of the intended users and decision makers. To that end, development of the Third National Climate Assessment included a focus on engaging users through an inclusive, broad-based, and sustained process. Such a process provides opportunities for scientific experts and decision makers to share knowledge about the climate-related issues, impacts, and potential response actions that are most important in a particular region or sector. Such a process is also highly transparent in order to produce results that are credible, salient, and legitimate for both scientists and decision makers, ultimately making the results extremely useful. To implement these principles for the recent NCA, a broad-based engagement strategy was implemented from the start of the process. The strategy invited participation from users and stakeholder communities at each stage of the process, and considered methods for communicating with potential users at every step. The strategy was designed to elicit contributions to help shape the framing of the assessment, improve the transparency of the process, and increase the utility of the final information. Specific user inputs were gathered through a series of workshops, public comment opportunities, town hall meetings, presentations, requests for information, submitted documents, and open meetings. Further, a network of contributors self-organized around topics of interest to extend the NCA to a wider range of user groups. Here, we describe the outcomes of these innovations in assessment engagement and identify clear successes, notable surprises, future evaluation needs, and areas for new ideas.

Interactive knowledge discovery with the doctor-in-the-loop: a practical example of cerebral aneurysms research.

PubMed

Girardi, Dominic; Küng, Josef; Kleiser, Raimund; Sonnberger, Michael; Csillag, Doris; Trenkler, Johannes; Holzinger, Andreas

2016-09-01

Established process models for knowledge discovery find the domain-expert in a customer-like and supervising role. In the field of biomedical research, it is necessary to move the domain-experts into the center of this process with far-reaching consequences for both their research output and the process itself. In this paper, we revise the established process models for knowledge discovery and propose a new process model for domain-expert-driven interactive knowledge discovery. Furthermore, we present a research infrastructure which is adapted to this new process model and demonstrate how the domain-expert can be deeply integrated even into the highly complex data-mining process and data-exploration tasks. We evaluated this approach in the medical domain for the case of cerebral aneurysms research.

The digital storytelling process: A comparative analysis from various experts

NASA Astrophysics Data System (ADS)

Hussain, Hashiroh; Shiratuddin, Norshuhada

2016-08-01

Digital Storytelling (DST) is a method of delivering information to the audience. It combines narrative and digital media content infused with the multimedia elements. In order for the educators (i.e the designers) to create a compelling digital story, there are sets of processes introduced by experts. Nevertheless, the experts suggest varieties of processes to guide them; of which some are redundant. The main aim of this study is to propose a single guide process for the creation of DST. A comparative analysis is employed where ten DST models from various experts are analysed. The process can also be implemented in other multimedia materials that used the concept of DST.

The Extent of Active Processing of a Long-Duration Stimulus Modulates the Scalp-Recorded Sustained Potential

ERIC Educational Resources Information Center

Campbell, Kenneth; Herzig, Alyssa; Jashmidi, Parastoo

2009-01-01

A long-duration stimulus will elicit a negative sustained potential (SP) that is maximum in amplitude over fronto-central areas of the scalp. This study examines how the duration of active attentional processing of the stimulus might also elicit a nonsensory contingent negative variation (CNV) that overlaps and summates to the SP. Subjects were…

Patient-Centered Research to Support the Development of the Symptoms of Major Depressive Disorder Scale (SMDDS): Initial Qualitative Research.

PubMed

McCarrier, Kelly P; Deal, Linda S; Abraham, Lucy; Blum, Steven I; Bush, Elizabeth Nicole; Martin, Mona L; Thase, Michael E; Coons, Stephen Joel

2016-04-01

Content valid, patient-reported outcome (PRO) measures of major depressive disorder (MDD) symptoms are needed to assess MDD treatment benefit. While a range of questionnaires are currently available to evaluate aspects of depression from the patient's perspective, their comprehensiveness and qualitative development histories are unclear. The objective of this study was to describe the process and results of the preliminary qualitative development of a new symptom-based PRO measure intended to assess treatment benefit in MDD clinical trials. Qualitative interviews were conducted with adult MDD patients in the USA who recently experienced a major depressive episode. Experienced interviewers conducted concept elicitation (CE) and cognitive interviews using semi-structured interview guides. The CE interview guide was used to elicit spontaneous reports of symptom experiences along with probing to further explore and confirm concepts. The cognitive interview guide was developed to evaluate concept relevance, understandability, and structure of the draft items, and to facilitate further instrument refinement. Forty patients participated in the CE interviews. A total of 3022 symptom codes, representing 84 different concepts were derived from the transcripts. Data from the CE interviews were considered alongside existing literature and clinical expert opinion during an item-generation process, leading to development of a preliminary version of the Symptoms of Major Depressive Disorder Scale (SMDDS). Fifteen patients participated in three waves of cognitive interviews, during which the SMDDS was further refined. The SMDDS is a 35-item PRO measure intended for use as an endpoint in MDD clinical trials to support medical product labeling. The SMDDS uses a 7-day recall period and verbal rating scales. It was developed in accordance with the US Food and Drug Administration (FDA)'s PRO Guidance and best practices. Qualitative interviews have provided evidence for content validity. Future quantitative studies will confirm the SMDDS's measurement properties and support FDA qualification.

Emergency preparedness: community-based short-term eruption forecasting at Campi Flegrei

NASA Astrophysics Data System (ADS)

Selva, Jacopo; Marzocchi, Warner; Civetta, Lucia; Del Pezzo, Edoardo; Papale, Paolo

2010-05-01

A key element in emergency preparedness is to define advance tools to assist decision makers and emergency management groups during crises. Such tools must be prepared in advance, accounting for all of expertise and scientific knowledge accumulated through time. During a pre-eruptive phase, the key for sound short-term eruption forecasting is the analysis of the monitoring signals. This involves the capability (i) to recognize anomalous signals and to relate single or combined anomalies to physical processes, assigning them probability values, and (ii) to quickly provide an answer to the observed phenomena even when unexpected. Here we present a > 4 years long process devoted to define the pre-eruptive Event Tree (ET) for Campi Flegrei. A community of about 40 experts in volcanology and volcano monitoring participating to two Italian Projects on Campi Flegrei funded by the Italian Civil Protection, has been constituted and trained during periodic meetings on the statistical methods and the model BET_EF (Marzocchi et al., 2008) that forms the statistical package tool for ET definition. Model calibration has been carried out through public elicitation sessions, preceded and followed by devoted meetings and web forum discussion on the monitoring parameters, their accuracy and relevance, and their potential meanings. The calibrated ET allows anomalies in the monitored parameters to be recognized and interpreted, assigning probability values to each set of data. This process de-personalizes the difficult task of interpreting multi-parametric sets of data during on-going emergencies, and provides a view of the observed variations that accounts for the averaged, weighted opinion of the scientific community. An additional positive outcome of the described ET calibration process is that of providing a picture of the degree of confidence by the expert community on the capability of the many different monitored quantities of recognizing significant variations in the state of the volcano. This picture is particularly useful since it can be used to guide future implementations in the monitoring network, as well as research investments aimed at substantially improving the capability to forecast the short-term volcanic hazard.

A comprehensive information technology system to support physician learning at the point of care.

PubMed

Cook, David A; Sorensen, Kristi J; Nishimura, Rick A; Ommen, Steve R; Lloyd, Farrell J

2015-01-01

MayoExpert is a multifaceted information system integrated with the electronic medical record (EMR) across Mayo Clinic's multisite health system. It was developed as a technology-based solution to manage information, standardize clinical practice, and promote and document learning in clinical contexts. Features include urgent test result notifications; models illustrating expert-approved care processes; concise, expert-approved answers to frequently asked questions (FAQs); a directory of topic-specific experts; and a portfolio for provider licensure and credentialing. The authors evaluate MayoExpert's reach, effectiveness, adoption, implementation, and maintenance. Evaluation data sources included usage statistics, user surveys, and pilot studies.As of October 2013, MayoExpert was available at 94 clinical sites in 12 states and contained 1,368 clinical topics, answers to 7,640 FAQs, and 92 care process models. In 2012, MayoExpert was accessed at least once by 2,578/3,643 (71%) staff physicians, 900/1,374 (66%) midlevel providers, and 1,728/2,291 (75%) residents and fellows. In a 2013 survey of MayoExpert users with 536 respondents, all features were highly rated (≥67% favorable). More providers reported using MayoExpert to answer questions before/after than during patient visits (68% versus 36%). During November 2012 to April 2013, MayoExpert sent 1,660 notifications of new-onset atrial fibrillation and 1,590 notifications of prolonged QT. MayoExpert has become part of routine clinical and educational operations, and its care process models now define Mayo Clinic best practices. MayoExpert's infrastructure and content will continue to expand with improved templates and content organization, new care process models, additional notifications, better EMR integration, and improved support for credentialing activities.

Improving Behavioral Support for Smoking Cessation in Pregnancy: What Are the Barriers to Stopping and Which Behavior Change Techniques Can Influence Them? Application of Theoretical Domains Framework

PubMed Central

Coleman-Haynes, Tom; Lorencatto, Fabiana; Ussher, Michael; Dyas, Jane; Coleman, Tim

2018-01-01

Behavioral support interventions are used to help pregnant smokers stop; however, of those tested, few are proven effective. Systematic research developing effective pregnancy-specific behavior change techniques (BCTs) is ongoing. This paper reports contributory work identifying potentially-effective BCTs relative to known important barriers and facilitators (B&Fs) to smoking cessation in pregnancy; to detect priority areas for BCTs development. A Nominal Group Technique with cessation experts (n = 12) elicited an expert consensus on B&Fs most influencing women’s smoking cessation and those most modifiable through behavioral support. Effective cessation interventions in randomized trials from a recent Cochrane review were coded into component BCTs using existing taxonomies. B&Fs were categorized using Theoretical Domains Framework (TDF) domains. Matrices, mapping BCT taxonomies against TDF domains, were consulted to investigate the extent to which BCTs in existing interventions target key B&Fs. Experts ranked ‘smoking a social norm’ and ‘quitting not a priority’ as most important barriers and ‘desire to protect baby’ an important facilitator to quitting. From 14 trials, 23 potentially-effective BCTs were identified (e.g., ‘information about consequences). Most B&Fs fell into ‘Social Influences’, ‘Knowledge’, ‘Emotions’ and ‘Intentions’ TDF domains; few potentially-effective BCTs mapped onto every TDF domain. B&Fs identified by experts as important to cessation, are not sufficiently targeted by BCT’s currently within interventions for smoking cessation in pregnancy. PMID:29462994

Ethics, Risk and Benefits Associated with Different Applications of Nanotechnology: a Comparison of Expert and Consumer Perceptions of Drivers of Societal Acceptance.

PubMed

Gupta, N; Fischer, A R H; Frewer, L J

Examining those risk and benefit perceptions utilised in the formation of attitudes and opinions about emerging technologies such as nanotechnology can be useful for both industry and policy makers involved in their development, implementation and regulation. A broad range of different socio-psychological and affective factors may influence consumer responses to different applications of nanotechnology, including ethical concerns. A useful approach to identifying relevant consumer concerns and innovation priorities is to develop predictive constructs which can be used to differentiate applications of nanotechnology in a way which is meaningful to consumers. This requires elicitation of attitudinal constructs from consumers, rather than measuring attitudes assumed to be important by the researcher. Psychological factors influencing societal responses to 15 applications of nanotechnology drawn from different application areas (e.g. medicine, agriculture and environment, food, military, sports, and cosmetics) were identified using repertory grid method in conjunction with generalised Procrustes analysis. The results suggested that people differentiate nanotechnology applications based on the extent to which they perceive them to be beneficial, useful, necessary and important. The benefits may be offset by perceived risks focusing on fear and ethical concerns. Compared to an earlier expert study on societal acceptance of nanotechnology, consumers emphasised ethical issues compared to experts but had less concern regarding potential physical contact with the product and time to market introduction. Consumers envisaged fewer issues with several applications compared to experts, in particular food applications.

Eight habitats, 38 threats and 55 experts: Assessing ecological risk in a multi-use marine region

PubMed Central

Jones, Alice R.; Deveney, Marty R.; Ward, Tim M.; Gillanders, Bronwyn M.

2017-01-01

Identifying the relative risk human activities pose to a habitat, and the ecosystem services they provide, can guide management prioritisation and resource allocation. Using a combination of expert elicitation to assess the probable effect of a threat and existing data to assess the level of threat exposure, we conducted a risk assessment for 38 human-mediated threats to eight marine habitats (totalling 304 threat-habitat combinations) in Spencer Gulf, Australia. We developed a score-based survey to collate expert opinion and assess the relative effect of each threat to each habitat, as well as a novel and independent measure of knowledge-based uncertainty. Fifty-five experts representing multiple sectors and institutions participated in the study, with 6 to 15 survey responses per habitat (n = 81 surveys). We identified key threats specific to each habitat; overall, climate change threats received the highest risk rankings, with nutrient discharge identified as a key local-scale stressor. Invasive species and most fishing-related threats, which are commonly identified as major threats to the marine environment, were ranked as low-tier threats to Spencer Gulf, emphasising the importance of regionally-relevant assessments. Further, we identified critical knowledge gaps and quantified uncertainty scores for each risk. Our approach will facilitate prioritisation of resource allocation in a region of increasing social, economic and environmental importance, and can be applied to marine regions where empirical data are lacking. PMID:28489912

Rating impacts in a multi-stressor world: a quantitative assessment of 50 stressors affecting the Great Lakes.

PubMed

Smith, Sigrid D P; Mcintyre, Peter B; Halpern, Benjamin S; Cooke, Roger M; Marino, Adrienne L; Boyer, Gregory L; Buchsbaum, Andy; Burton, G A; Campbell, Linda M; Ciborowski, Jan J H; Doran, Patrick J; Infante, Dana M; Johnson, Lucinda B; Read, Jennifer G; Rose, Joan B; Rutherford, Edward S; Steinman, Alan D; Allan, J David

2015-04-01

Ecosystems often experience multiple environmental stressors simultaneously that can differ widely in their pathways and strengths of impact. Differences in the relative impact of environmental stressors can guide restoration and management prioritization, but few studies have empirically assessed a comprehensive suite of stressors acting on a given ecosystem. To fill this gap in the Laurentian Great Lakes, where considerable restoration investments are currently underway, we used expert elicitation via a detailed online survey to develop ratings of the relative impacts of 50 potential stressors. Highlighting the multiplicity of stressors in this system, experts assessed all 50 stressors as having some impact on ecosystem condition, but ratings differed greatly among stressors. Individual stressors related to invasive and nuisance species (e.g., dreissenid mussels and ballast invasion risk) and climate change were assessed as having the greatest potential impacts. These results mark a shift away from the longstanding emphasis on nonpoint phosphorus and persistent bioaccumulative toxic substances in the Great Lakes. Differences in impact ratings among lakes and ecosystem zones were weak, and experts exhibited surprisingly high levels of agreement on the relative impacts of most stressors. Our results provide a basin-wide, quantitative summary of expert opinion on the present-day influence of all major Great Lakes stressors. The resulting ratings can facilitate prioritizing stressors to achieve management objectives in a given location, as well as providing a baseline for future stressor impact assessments in the Great Lakes and elsewhere.

Guidelines for locoregional therapy in primary breast cancer in developing countries: The results of an expert panel at the 8th Annual Women's Cancer Initiative – Tata Memorial Hospital (WCI-TMH) Conference

PubMed Central

Munshi, Anusheel; Gupta, Sudeep; Anderson, Benjamin; Yarnold, John; Parmar, Vani; Jalali, Rakesh; Sharma, Suresh Chander; Desai, Sangeeta; Thakur, Meenakshi; Baijal, Gunjan; Sarin, Rajiv; Mittra, Indraneel; Ghosh, Jaya; Badwe, Rajendra

2012-01-01

Background: Limited guidelines exist for breast cancer management in developing countries. In this context, the Women's Cancer Initiative - Tata Memorial Hospital (WCI-TMH) organised its 8th Annual Conference to update guidelines in breast cancer. Materials and Methods: Appropriately formulated guideline questions on each topic and subtopic in the surgical, radiation and systemic management of primary breast cancer were developed by the scientific committee and shared with the guest faculty of the Conference. Majority of the questions had multiple choice answers. The opinion of the audience, comprising academic and community oncologists, was electronically cumulated, followed by focussed presentations by eminent national and international experts on each topic. The guidelines were finally developed through an expert panel that voted on each guideline question after all talks had been delivered and audience opinion elicited. Separate panels were constituted for locoregional and systemic therapy in primary breast cancer. Results: Based on the voting results of the expert panel, guidelines for locoregional therapy of breast cancer have been formulated. Voting patterns for each question are reported. Conclusions: The updated guidelines on locoregional management of primary breast cancer in the context of developing countries are presented in this article. These recommendations have been designed to allow centers in the developing world to improve the quality of care for breast cancer patients. PMID:22988354

Improving Behavioral Support for Smoking Cessation in Pregnancy: What Are the Barriers to Stopping and Which Behavior Change Techniques Can Influence These? Application of Theoretical Domains Framework.

PubMed

Campbell, Katarzyna A; Fergie, Libby; Coleman-Haynes, Tom; Cooper, Sue; Lorencatto, Fabiana; Ussher, Michael; Dyas, Jane; Coleman, Tim

2018-02-17

Behavioral support interventions are used to help pregnant smokers stop; however, of those tested, few are proven effective. Systematic research developing effective pregnancy-specific behavior change techniques (BCTs) is ongoing. This paper reports contributory work identifying potentially-effective BCTs relative to known important barriers and facilitators (B&Fs) to smoking cessation in pregnancy; to detect priority areas for BCTs development. A Nominal Group Technique with cessation experts ( n = 12) elicited an expert consensus on B&Fs most influencing women's smoking cessation and those most modifiable through behavioral support. Effective cessation interventions in randomized trials from a recent Cochrane review were coded into component BCTs using existing taxonomies. B&Fs were categorized using Theoretical Domains Framework (TDF) domains. Matrices, mapping BCT taxonomies against TDF domains, were consulted to investigate the extent to which BCTs in existing interventions target key B&Fs. Experts ranked "smoking a social norm" and "quitting not a priority" as most important barriers and "desire to protect baby" an important facilitator to quitting. From 14 trials, 23 potentially-effective BCTs were identified (e.g., information about consequences). Most B&Fs fell into "Social Influences", "Knowledge", "Emotions" and "Intentions" TDF domains; few potentially-effective BCTs mapped onto every TDF domain. B&Fs identified by experts as important to cessation, are not sufficiently targeted by BCT's currently within interventions for smoking cessation in pregnancy.

Evocative gene–environment correlation in the mother–child relationship: A twin study of interpersonal processes

PubMed Central

KLAHR, ASHLEA M.; THOMAS, KATHERINE M.; HOPWOOD, CHRISTOPHER J.; KLUMP, KELLY L.; BURT, S. ALEXANDRA

2014-01-01

The behavior genetic literature suggests that genetically influenced characteristics of the child elicit specific behaviors from the parent. However, little is known about the processes by which genetically influenced child characteristics evoke parental responses. Interpersonal theory provides a useful framework for identifying reciprocal behavioral processes between children and mothers. The theory posits that, at any given moment, interpersonal behavior varies along the orthogonal dimensions of warmth and control and that the interpersonal behavior of one individual tends to elicit corresponding or contrasting behavior from the other (i.e., warmth elicits warmth, whereas control elicits submission). The current study thus examined these dimensions of interpersonal behavior as they relate to the parent–child relationship in 546 twin families. A computer joystick was used to rate videos of mother–child interactions in real time, yielding information on mother and child levels of warmth and control throughout the interaction. Analyses indicated that maternal control, but not maternal warmth, was influenced by evocative gene–environment correlational processes, such that genetic influences on maternal control and child control were largely overlapping. Moreover, these common genetic influences were present both cross-sectionally and over the course of the interaction. Such findings not only confirm the presence of evocative gene–environment correlational processes in the mother–child relationship but also illuminate at least one of the specific interpersonal behaviors that underlie this evocative process. PMID:23398756

Review of the OSHA framework for oversight of occupational environments.

PubMed

Choi, Jae-Young; Ramachandran, Gurumurthy

2009-01-01

The OSHA system for oversight of chemicals in the workplace was evaluated to derive lessons for oversight of nanotechnology. Criteria relating to the development, attributes, evolution, and outcomes of the system were used for evaluation that was based upon quantitative expert elicitation and historical literature analysis. The oversight system had inadequate resources in terms of finances, expertise, and personnel, and insufficient incentive for compliance. The system showed a lack of flexibility in novel situations. There were minimal requirements on companies for data on health and safety of their products. These factors have a strong influence on public confidence and health and safety. The oversight system also scored low on attributes such as public input, transparency, empirical basis, conflict of interest, and informed consent. The experts in our sample tend to believe that the current oversight system for chemicals in the workplace is neither adequate nor effective. It is very likely that the performance of the OSHA oversight system for nanomaterials will be equally inadequate.

The use of mental models in chemical risk protection: developing a generic workplace methodology.

PubMed

Cox, Patrick; Niewöhmer, Jörg; Pidgeon, Nick; Gerrard, Simon; Fischhoff, Baruch; Riley, Donna

2003-04-01

We adopted a comparative approach to evaluate and extend a generic methodology to analyze the different sets of beliefs held about chemical hazards in the workplace. Our study mapped existing knowledge structures about the risks associated with the use of perchloroethylene and rosin-based solder flux in differing workplaces. "Influence diagrams" were used to represent beliefs held by chemical experts; "user models" were developed from data elicited from open-ended interviews with the workplace users of the chemicals. The juxtaposition of expert and user understandings of chemical risks enabled us to identify knowledge gaps and misunderstandings and to reinforce appropriate sets of safety beliefs and behavior relevant to chemical risk communications. By designing safety information to be more relevant to the workplace context of users, we believe that employers and employees may gain improved knowledge about chemical hazards in the workplace, such that better chemical risk management, self-protection, and informed decision making develop over time.

Integration of vegetation community spatial data into a prescribed fire planning process at Shenandoah National Park, Virginia (USA)

USGS Publications Warehouse

Young, John A.; Mahan, Carolyn G.; Forder, Melissa

2017-01-01

Many eastern forest communities depend on fire for regeneration or are enhanced by fire as a restoration practice. However, the use of prescribed fire in the mesic forested environments and the densely populated regions of the eastern United States has been limited. The objective of our research was to develop a science-based approach to prioritizing the use of prescribed fire in appropriate forest types in the eastern United States based on a set of desired management outcomes. Through a process of expert elicitation and data analysis, we assessed and integrated recent vegetation community mapping results along with other available spatial data layers into a spatial prioritization tool for prescribed fire planning at Shenandoah National Park (Virginia, USA). The integration of vegetation spatial data allowed for development of per-pixel priority rankings and exclusion areas enabling precise targeting of fire management activities on the ground, as well as a park-wide ranking of fire planning compartments. We demonstrate the use and evaluation of this approach through implementation and monitoring of a prescribed burn and show that progress is being made toward desired conditions. Integration of spatial data into the fire planning process has served as a collaborative tool for the implementation of prescribed fire projects, which assures projects will be planned in the most appropriate areas to meet objectives that are supported by current science.

A demonstration of expert systems applications in transportation engineering : volume I, transportation engineers and expert systems.

DOT National Transportation Integrated Search

1987-01-01

Expert systems, a branch of artificial-intelligence studies, is introduced with a view to its relevance in transportation engineering. Knowledge engineering, the process of building expert systems or transferring knowledge from human experts to compu...

[Medical expert assessment in criminal processes from the legal viewpoint].

PubMed

Ulsenheimer, K

1996-11-01

In the area of medical professional blunder, the medical expert witness is the one participant in a trial whose statement is practically decisive for the court or the prosecutor. Legally, the responsibility remains naturally in the legal hand as the expert witness is only the assistant of the judge. The most important demands on the expert witness are strict objectiveness including towards the colleague, no independent inquiries or interrogations, comprehensive processing of the expert assessment, readiness to revise a written expert assessment according to better knowledge or new facts, independence from the client, no legal comments, clarity of language and intellectual honesty.

Brain-heart linear and nonlinear dynamics during visual emotional elicitation in healthy subjects.

PubMed

Valenza, G; Greco, A; Gentili, C; Lanata, A; Toschi, N; Barbieri, R; Sebastiani, L; Menicucci, D; Gemignani, A; Scilingo, E P

2016-08-01

This study investigates brain-heart dynamics during visual emotional elicitation in healthy subjects through linear and nonlinear coupling measures of EEG spectrogram and instantaneous heart rate estimates. To this extent, affective pictures including different combinations of arousal and valence levels, gathered from the International Affective Picture System, were administered to twenty-two healthy subjects. Time-varying maps of cortical activation were obtained through EEG spectral analysis, whereas the associated instantaneous heartbeat dynamics was estimated using inhomogeneous point-process linear models. Brain-Heart linear and nonlinear coupling was estimated through the Maximal Information Coefficient (MIC), considering EEG time-varying spectra and point-process estimates defined in the time and frequency domains. As a proof of concept, we here show preliminary results considering EEG oscillations in the θ band (4-8 Hz). This band, indeed, is known in the literature to be involved in emotional processes. MIC highlighted significant arousal-dependent changes, mediated by the prefrontal cortex interplay especially occurring at intermediate arousing levels. Furthermore, lower and higher arousing elicitations were associated to not significant brain-heart coupling changes in response to pleasant/unpleasant elicitations.

Three CLIPS-based expert systems for solving engineering problems

NASA Technical Reports Server (NTRS)

Parkinson, W. J.; Luger, G. F.; Bretz, R. E.

1990-01-01

We have written three expert systems, using the CLIPS PC-based expert system shell. These three expert systems are rule based and are relatively small, with the largest containing slightly less than 200 rules. The first expert system is an expert assistant that was written to help users of the ASPEN computer code choose the proper thermodynamic package to use with their particular vapor-liquid equilibrium problem. The second expert system was designed to help petroleum engineers choose the proper enhanced oil recovery method to be used with a given reservoir. The effectiveness of each technique is highly dependent upon the reservoir conditions. The third expert system is a combination consultant and control system. This system was designed specifically for silicon carbide whisker growth. Silicon carbide whiskers are an extremely strong product used to make ceramic and metal composites. The manufacture of whiskers is a very complicated process. which to date. has defied a good mathematical model. The process was run by experts who had gained their expertise by trial and error. A system of rules was devised by these experts both for procedure setup and for the process control. In this paper we discuss the three problem areas of the design, development and evaluation of the CLIPS-based programs.

Design and implementation of a status at a glance user interface for a power distribution expert system

NASA Technical Reports Server (NTRS)

Liberman, Eugene M.; Manner, David B.; Dolce, James L.; Mellor, Pamela A.

1993-01-01

Expert systems are widely used in health monitoring and fault detection applications. One of the key features of an expert system is that it possesses a large body of knowledge about the application for which it was designed. When the user consults this knowledge base, it is essential that the expert system's reasoning process and its conclusions be as concise as possible. If, in addition, an expert system is part of a process monitoring system, the expert system's conclusions must be combined with current events of the process. Under these circumstances, it is difficult for a user to absorb and respond to all the available information. For example, a user can become distracted and confused if two or more unrelated devices in different parts of the system require attention. A human interface designed to integrate expert system diagnoses with process data and to focus the user's attention to the important matters provides a solution to the 'information overload' problem. This paper will discuss a user interface to the power distribution expert system for Space Station Freedom. The importance of features which simplify assessing system status and which minimize navigating through layers of information will be discussed. Design rationale and implementation choices will also be presented.

Spacelab Data Processing Facility (SLDPF) quality assurance expert systems development

NASA Technical Reports Server (NTRS)

Basile, Lisa R.; Kelly, Angelita C.

1987-01-01

The Spacelab Data Processing Facility (SLDPF) is an integral part of the Space Shuttle data network for missions that involve attached scientific payloads. Expert system prototypes were developed to aid in the performance of the quality assurance function of the Spacelab and/or Attached Shuttle Payloads processed telemetry data. The Spacelab Input Processing System (SIPS) and the Spacelab Output Processing System (SOPS), two expert systems, were developed to determine their feasibility and potential in the quality assurance of processed telemetry data. The capabilities and performance of these systems are discussed.

Visual processing of music notation: a study of event-related potentials.

PubMed

Lee, Horng-Yih; Wang, Yu-Sin

2011-04-01

In reading music, the acquisition of pitch information depends mostly on the spatial position of notes, hence more spatial processing, whereas the acquisition of temporal information depends mostly on the visual features of notes and object recognition. This study used both electrophysiological and behavioral methods to compare the processing of pitch and duration in reading single musical notes. It was observed that in the early stage of note reading, identification of pitch could elicit greater N1 and N2 amplitude than identification of duration at the parietal lobe electrodes. In the later stages of note reading, identifying pitch elicited a greater negative slow wave at parietal electrodes than did identifying note duration. The sustained contribution of parietal processes for pitch suggests that the dorsal pathway is essential for pitch processing. However, the duration task did not elicit greater amplitude of any early ERP components than the pitch task at temporal electrodes. Accordingly, a double dissociation, suggesting involvement of the dorsal visual stream, was not observed in spatial pitch processing and ventral visual stream in processing of note durations.

Scope Complexity Options Risks Excursions (SCORE) Factor Mathematical Description.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Gearhart, Jared Lee; Samberson, Jonell Nicole; Shettigar, Subhasini

The purpose of the Scope, Complexity, Options, Risks, Excursions (SCORE) model is to estimate the relative complexity of design variants of future warhead options, resulting in scores. SCORE factors extend this capability by providing estimates of complexity relative to a base system (i.e., all design options are normalized to one weapon system). First, a clearly defined set of scope elements for a warhead option is established. The complexity of each scope element is estimated by Subject Matter Experts (SMEs), including a level of uncertainty, relative to a specific reference system. When determining factors, complexity estimates for a scope element canmore » be directly tied to the base system or chained together via comparable scope elements in a string of reference systems that ends with the base system. The SCORE analysis process is a growing multi-organizational Nuclear Security Enterprise (NSE) effort, under the management of the NA-12 led Enterprise Modeling and Analysis Consortium (EMAC). Historically, it has provided the data elicitation, integration, and computation needed to support the out-year Life Extension Program (LEP) cost estimates included in the Stockpile Stewardship Management Plan (SSMP).« less

Improving oncology nurses' communication skills for difficult conversations.

PubMed

Baer, Linda; Weinstein, Elizabeth

2013-06-01

When oncology nurses have strong communication skills, they play a pivotal role in influencing patient satisfaction, adherence to plans of care, and overall clinical outcomes. However, research studies indicate that nurses tend to keep communication with patients and families at a superficial, nontherapeutic level. Processes for teaching goals-of-care communication skills and for implementing skills into clinical practice are not clearly defined. Nurses at a large comprehensive cancer center recognized the need for help with this skill set and sought out communication experts to assist in providing the needed education. An educational project was developed to improve therapeutic communication skills in oncology nurses during goals-of-care discussions and giving bad news. The program was tailored to nurses and social workers providing care to patients in a busy, urban, academic, outpatient oncology setting. Program topics included exploring the patient's world, eliciting hopes and concerns, and dealing with conflict about goals. Sharing and discussing specific difficult questions and scenarios were encouraged throughout the program. The program was well attended and well received by oncology nurses and social workers. Participants expressed interest in the continuation of communication programs to further enhance skills.

Envisioning the future of wildlife in a changing climate: Collaborative learning for adaptation planning

USGS Publications Warehouse

LeDee, Olivia E.; Karasov, W.H.; Martin, Karl J.; Meyer, Michael W.; Ribic, Christine; Van Deelen, Timothy R.

2011-01-01

Natural resource managers are tasked with assessing the impacts of climate change on conservation targets and developing adaptation strategies to meet agency goals. The complex, transboundary nature of climate change demands the collaboration of scientists, managers, and stakeholders in this effort. To share, integrate, and apply knowledge from these diverse perspectives, we must engage in social learning. In 2009, we initiated a process to engage university researchers and agency scientists and managers in collaborative learning to assess the impacts of climate change on terrestrial fauna in the state of Wisconsin, USA. We constructed conceptual Bayesian networks to depict the influence of climate change, key biotic and abiotic factors, and existing stressors on the distribution and abundance of 3 species: greater prairie-chicken (Tympanuchus cupido), wood frog (Lithobates sylvaticus), and Karner blue butterfly (Plebejus melissa samuelis). For each species, we completed a 2-stage expert review that elicited dialogue on information gaps, management opportunities, and research priorities. From our experience, collaborative network modeling proved to be a powerful tool to develop a common vision of the potential impacts of climate change on conservation targets.

Expert knowledge as a foundation for the management of secretive species and their habitat

USGS Publications Warehouse

Drew, C. Ashton; Collazo, Jaime

2012-01-01

In this chapter, we share lessons learned during the elicitation and application of expert knowledge in the form of a belief network model for the habitat of a waterbird, the King Rail (Rallus elegans). A belief network is a statistical framework used to graphically represent and evaluate hypothesized cause and effect relationships among variables. Our model was a pilot project to explore the value of such a model as a tool to help the US Fish and Wildlife Service (USFWS) conserve species that lack sufficient empirical data to guide management decisions. Many factors limit the availability of empirical data that can support landscape-scale conservation planning. Globally, most species simply have not yet been subject to empirical study (Wilson 2000). Even for well-studied species, data are often restricted to specific geographic extents, to particular seasons, or to specific segments of a species’ life history. The USFWS mandates that the agency’s conservation actions (1) be coordinated across regional landscapes, (2) be founded on the best available science (with testable assumptions), and (3) support adaptive management through monitoring and assessment of action outcomes. Given limits on the available data, the concept of “best available science” in the context of conservation planning generally includes a mix of empirical data and expert knowledge (Sullivan et al. 2006).

Transformation in the pharmaceutical industry: transformation-induced quality risks--a survey.

PubMed

Shafiei, Nader; Ford, James L; Morecroft, Charles W; Lisboa, Paulo J; Taylor, Mark J; Mouzughi, Yusra

2013-01-01

This paper is the fourth in a series that explores ongoing transformation in the pharmaceutical industry and its impact on pharmaceutical quality from the perspective of risk identification. The aim of this paper is to validate proposed quality risks through elicitation of expert opinion and define the resultant quality risk model. Expert opinion was obtained using a questionnaire-based survey with participants with recognized expertise in pharmaceutical regulation, product lifecycle, or technology. The results of the survey validate the theoretical and operational evidence in support of the four main pharmaceutical transformation triggers previously identified. The quality risk model resulting from the survey indicated a firm relationship between the pharmaceutical quality risks and regulatory compliance outcomes during the marketing approval and post-marketing phases of the product lifecycle and a weaker relationship during the pre-market evaluation phase. In this paper through conduct of an expert opinion survey the proposed quality risks carried forward from an earlier part of the research are validated and resultant quality risk model is defined. The survey results validate the theoretical and operational evidence previously identified. The quality risk model indicates that transformation-related risks have a larger regulatory compliance impact during product approval, manufacturing, distribution, and commercial use than during the development phase.

Creation and preliminary validation of the screening for self-medication safety post-stroke scale (S-5).

PubMed

Kaizer, Franceen; Kim, Angela; Van, My Tram; Korner-Bitensky, Nicol

2010-03-01

Patients with stroke should be screened for safety prior to starting a self-medication regime. An extensive literature review revealed no standardized self-medication tool tailored to the multi-faceted needs of the stroke population. The aim of this study was to create and validate a condition-specific tool to be used in screening for self-medication safety in individuals with stroke. Items were generated using expert consultation and review of the existing tools. The draft tool was pilot-tested on expert stroke clinicians to receive feedback on content, clarity, optimal cueing and domain omissions. The final version was piloted on patients with stroke using a structured interviewer-administered interview. The tool was progressively refined and validated according to feedback from the 11 expert reviewers. The subsequent version was piloted on patients with stroke. The final version includes 16 questions designed to elicit information on 5 domains: cognition, communication, motor, visual-perception and, judgement/executive function/self-efficacy. The Screening for Safe Self-medication post-Stroke Scale (S-5) has been created and validated for use by health professionals to screen self-medication safety readiness of patients after stroke. Its use should also help to guide clinicians' recommendations and interventions aimed at enhancing self-medication post-stroke.

A New Perspective on Modeling Groundwater-Driven Health Risk With Subjective Information

NASA Astrophysics Data System (ADS)

Ozbek, M. M.

2003-12-01

Fuzzy rule-based systems provide an efficient environment for the modeling of expert information in the context of risk management for groundwater contamination problems. In general, their use in the form of conditional pieces of knowledge, has been either as a tool for synthesizing control laws from data (i.e., conjunction-based models), or in a knowledge representation and reasoning perspective in Artificial Intelligence (i.e., implication-based models), where only the latter may lead to coherence problems (e.g., input data that leads to logical inconsistency when added to the knowledge base). We implement a two-fold extension to an implication-based groundwater risk model (Ozbek and Pinder, 2002) including: 1) the implementation of sufficient conditions for a coherent knowledge base, and 2) the interpolation of expert statements to supplement gaps in knowledge. The original model assumes statements of public health professionals for the characterization of the exposed individual and the relation of dose and pattern of exposure to its carcinogenic effects. We demonstrate the utility of the extended model in that it: 1)identifies inconsistent statements and establishes coherence in the knowledge base, and 2) minimizes the burden of knowledge elicitation from the experts for utilizing existing knowledge in an optimal fashion.ÿÿ

Quantifying pollen-vegetation relationships to reconstruct ancient forests using 19th-century forest composition and pollen data

USGS Publications Warehouse

Dawson, Andria; Paciorek, Christopher J.; McLachlan, Jason S.; Goring, Simon; Williams, John W.; Jackson, Stephen T.

2016-01-01

Mitigation of climate change and adaptation to its effects relies partly on how effectively land-atmosphere interactions can be quantified. Quantifying composition of past forest ecosystems can help understand processes governing forest dynamics in a changing world. Fossil pollen data provide information about past forest composition, but rigorous interpretation requires development of pollen-vegetation models (PVMs) that account for interspecific differences in pollen production and dispersal. Widespread and intensified land-use over the 19th and 20th centuries may have altered pollen-vegetation relationships. Here we use STEPPS, a Bayesian hierarchical spatial PVM, to estimate key process parameters and associated uncertainties in the pollen-vegetation relationship. We apply alternate dispersal kernels, and calibrate STEPPS using a newly developed Euro-American settlement-era calibration data set constructed from Public Land Survey data and fossil pollen samples matched to the settlement-era using expert elicitation. Models based on the inverse power-law dispersal kernel outperformed those based on the Gaussian dispersal kernel, indicating that pollen dispersal kernels are fat tailed. Pine and birch have the highest pollen productivities. Pollen productivity and dispersal estimates are generally consistent with previous understanding from modern data sets, although source area estimates are larger. Tests of model predictions demonstrate the ability of STEPPS to predict regional compositional patterns.

Fuzzy fault tree assessment based on improved AHP for fire and explosion accidents for steel oil storage tanks.

PubMed

Shi, Lei; Shuai, Jian; Xu, Kui

2014-08-15

Fire and explosion accidents of steel oil storage tanks (FEASOST) occur occasionally during the petroleum and chemical industry production and storage processes and often have devastating impact on lives, the environment and property. To contribute towards the development of a quantitative approach for assessing the occurrence probability of FEASOST, a fault tree of FEASOST is constructed that identifies various potential causes. Traditional fault tree analysis (FTA) can achieve quantitative evaluation if the failure data of all of the basic events (BEs) are available, which is almost impossible due to the lack of detailed data, as well as other uncertainties. This paper makes an attempt to perform FTA of FEASOST by a hybrid application between an expert elicitation based improved analysis hierarchy process (AHP) and fuzzy set theory, and the occurrence possibility of FEASOST is estimated for an oil depot in China. A comparison between statistical data and calculated data using fuzzy fault tree analysis (FFTA) based on traditional and improved AHP is also made. Sensitivity and importance analysis has been performed to identify the most crucial BEs leading to FEASOST that will provide insights into how managers should focus effective mitigation. Copyright © 2014 Elsevier B.V. All rights reserved.

Modeling with uncertain science: estimating mitigation credits from abating lead poisoning in Golden Eagles.

PubMed

Fitts Cochrane, Jean; Lonsdorf, Eric; Allison, Taber D; Sanders-Reed, Carol A

2015-09-01

Challenges arise when renewable energy development triggers "no net loss" policies for protected species, such as where wind energy facilities affect Golden Eagles in the western United States. When established mitigation approaches are insufficient to fully avoid or offset losses, conservation goals may still be achievable through experimental implementation of unproven mitigation methods provided they are analyzed within a framework that deals transparently and rigorously with uncertainty. We developed an approach to quantify and analyze compensatory mitigation that (1) relies on expert opinion elicited in a thoughtful and structured process to design the analysis (models) and supplement available data, (2) builds computational models as hypotheses about cause-effect relationships, (3) represents scientific uncertainty in stochastic model simulations, (4) provides probabilistic predictions of "relative" mortality with and without mitigation, (5) presents results in clear formats useful to applying risk management preferences (regulatory standards) and selecting strategies and levels of mitigation for immediate action, and (6) defines predictive parameters in units that could be monitored effectively, to support experimental adaptive management and reduction in uncertainty. We illustrate the approach with a case study characterized by high uncertainty about underlying biological processes and high conservation interest: estimating the quantitative effects of voluntary strategies to abate lead poisoning in Golden Eagles in Wyoming due to ingestion of spent game hunting ammunition.

The use and value of polling to determine public opinion on GMOs in Europe: limitations and ways forward.

PubMed

Desaint, Nilsy; Varbanova, Mariana

2013-01-01

In this article, we review and assess existing surveys that attempt to gauge public opinion about GM crops. This review shows that consumer surveys can be something of a blunt instrument. Questionnaires, however well-constructed and professionally delivered, are answered in a vacuum of knowledge and elicit misleading responses. People recurrently admit they lack information on the technology behind GM food. It is a part of the general unfamiliarity with the food production process with which people show equal, if not greater, concern. Lacking control over a process involving such an emotive subject like food makes people uneasy and reluctant to accept "unknowns". In addition, if people give answers to hypothetical questions, they do so as "cautious citizens" rather than consumers and this is not a good guide to actual behavior in real life. Qualitative studies using focus groups can be a much better guide to how people really feel about issues, but they also need expert facilitation and analysis to be of real value. An even better guide to acceptance and purchasing behavior (if that is the objective) is to put people in a situation where they are actually making a choice of whether or not to buy GM products.

Quantifying pollen-vegetation relationships to reconstruct ancient forests using 19th-century forest composition and pollen data

NASA Astrophysics Data System (ADS)

Dawson, Andria; Paciorek, Christopher J.; McLachlan, Jason S.; Goring, Simon; Williams, John W.; Jackson, Stephen T.

2016-04-01

Mitigation of climate change and adaptation to its effects relies partly on how effectively land-atmosphere interactions can be quantified. Quantifying composition of past forest ecosystems can help understand processes governing forest dynamics in a changing world. Fossil pollen data provide information about past forest composition, but rigorous interpretation requires development of pollen-vegetation models (PVMs) that account for interspecific differences in pollen production and dispersal. Widespread and intensified land-use over the 19th and 20th centuries may have altered pollen-vegetation relationships. Here we use STEPPS, a Bayesian hierarchical spatial PVM, to estimate key process parameters and associated uncertainties in the pollen-vegetation relationship. We apply alternate dispersal kernels, and calibrate STEPPS using a newly developed Euro-American settlement-era calibration data set constructed from Public Land Survey data and fossil pollen samples matched to the settlement-era using expert elicitation. Models based on the inverse power-law dispersal kernel outperformed those based on the Gaussian dispersal kernel, indicating that pollen dispersal kernels are fat tailed. Pine and birch have the highest pollen productivities. Pollen productivity and dispersal estimates are generally consistent with previous understanding from modern data sets, although source area estimates are larger. Tests of model predictions demonstrate the ability of STEPPS to predict regional compositional patterns.

Structured decision making as a framework for large-scale wildlife harvest management decisions

USGS Publications Warehouse

Robinson, Kelly F.; Fuller, Angela K.; Hurst, Jeremy E.; Swift, Bryan L.; Kirsch, Arthur; Farquhar, James F.; Decker, Daniel J.; Siemer, William F.

2016-01-01

Fish and wildlife harvest management at large spatial scales often involves making complex decisions with multiple objectives and difficult tradeoffs, population demographics that vary spatially, competing stakeholder values, and uncertainties that might affect management decisions. Structured decision making (SDM) provides a formal decision analytic framework for evaluating difficult decisions by breaking decisions into component parts and separating the values of stakeholders from the scientific evaluation of management actions and uncertainty. The result is a rigorous, transparent, and values-driven process. This decision-aiding process provides the decision maker with a more complete understanding of the problem and the effects of potential management actions on stakeholder values, as well as how key uncertainties can affect the decision. We use a case study to illustrate how SDM can be used as a decision-aiding tool for management decision making at large scales. We evaluated alternative white-tailed deer (Odocoileus virginianus) buck-harvest regulations in New York designed to reduce harvest of yearling bucks, taking into consideration the values of the state wildlife agency responsible for managing deer, as well as deer hunters. We incorporated tradeoffs about social, ecological, and economic management concerns throughout the state. Based on the outcomes of predictive models, expert elicitation, and hunter surveys, the SDM process identified management alternatives that optimized competing objectives. The SDM process provided biologists and managers insight about aspects of the buck-harvest decision that helped them adopt a management strategy most compatible with diverse hunter values and management concerns.

Comparison of elicitation methods for moral and affective beliefs in the theory of planned behaviour.

PubMed

Dean, M; Arvola, A; Vassallo, M; Lähteenmäki, L; Raats, M M; Saba, A; Shepherd, R

2006-09-01

Although the theory of planned behaviour (TPB) has been applied successfully in the area of food choice, it has been criticized for its pure utilitarian approach to the factors determining behaviour. Despite the increase in predictive power of the model with added components such as affective attitude and moral and ethical concerns, in most studies the elicitation process still only addresses people's utilitarian beliefs about the behaviour with little attention paid to other aspects. This study compares the traditional method of elicitation of advantages and disadvantages with two other methods (word association and open-ended) in the elicitations of beliefs, attitudes and moral concerns in relation to the consumption of organic foods. Results show the traditional method to be best for eliciting cognitive beliefs, open-ended emotion task for eliciting emotional beliefs and open-ended beliefs task best for moral concerns. The advantages and disadvantages of each method are discussed.

Evolving Understanding of Antarctic Ice-Sheet Physics and Ambiguity in Probabilistic Sea-Level Projections

NASA Astrophysics Data System (ADS)

Kopp, Robert E.; DeConto, Robert M.; Bader, Daniel A.; Hay, Carling C.; Horton, Radley M.; Kulp, Scott; Oppenheimer, Michael; Pollard, David; Strauss, Benjamin H.

2017-12-01

Mechanisms such as ice-shelf hydrofracturing and ice-cliff collapse may rapidly increase discharge from marine-based ice sheets. Here, we link a probabilistic framework for sea-level projections to a small ensemble of Antarctic ice-sheet (AIS) simulations incorporating these physical processes to explore their influence on global-mean sea-level (GMSL) and relative sea-level (RSL). We compare the new projections to past results using expert assessment and structured expert elicitation about AIS changes. Under high greenhouse gas emissions (Representative Concentration Pathway [RCP] 8.5), median projected 21st century GMSL rise increases from 79 to 146 cm. Without protective measures, revised median RSL projections would by 2100 submerge land currently home to 153 million people, an increase of 44 million. The use of a physical model, rather than simple parameterizations assuming constant acceleration of ice loss, increases forcing sensitivity: overlap between the central 90% of simulations for 2100 for RCP 8.5 (93-243 cm) and RCP 2.6 (26-98 cm) is minimal. By 2300, the gap between median GMSL estimates for RCP 8.5 and RCP 2.6 reaches >10 m, with median RSL projections for RCP 8.5 jeopardizing land now occupied by 950 million people (versus 167 million for RCP 2.6). The minimal correlation between the contribution of AIS to GMSL by 2050 and that in 2100 and beyond implies current sea-level observations cannot exclude future extreme outcomes. The sensitivity of post-2050 projections to deeply uncertain physics highlights the need for robust decision and adaptive management frameworks.

Research Priorities for Studies Linking Intake of Low-Calorie Sweeteners and Potentially Related Health Outcomes

PubMed Central

Bright, Oliver-John M; Wang, Ding Ding; Shams-White, Marissa; Bleich, Sara N; Foreyt, John; Franz, Marion; Johnson, Guy; Manning, Beth Trickett; Mattes, Rick; Pi-Sunyer, Xavier; Schneeman, Barbara; Parrott, James Scott; Steffen, Dan; Sylvetsky, Allison; Ziegler, Paula

2017-01-01

Abstract Background: In a world of finite research funding, efforts to prioritize future research topics are increasingly necessary. Objective: The aim of this study was to identify and prioritize the direction of future research in the broad area of low-calorie sweetener (LCS) intake and potentially related health outcomes by using a novel method that incorporates evidence mapping in the Agency for Healthcare Research and Quality's Future Research Needs (FRN) process. Methods: A diverse expert stakeholder panel was convened and engaged to identify research gaps and prioritize future research needs. An independent research team hosted a number of interactive webinars and elicited feedback through surveys and individual interviews with the stakeholder panel, which included policymakers, lay audience members, health providers, a research funder, individuals with food industry experience, and researchers of several different specialties. Results: The stakeholder panel generated and ranked a list of 18 FRN questions across 5 broad research areas. Overall, stakeholder panel members unanimously agreed that the research questions that will have the largest public health impact are those that address outcomes related to body weight, appetite, and dietary intake. Although the LCSs included in this FRN project have all been Generally Recognized as Safe by the FDA or approved as food additives, the recurrent concerns and confusions with regard to the “safety” of LCSs by consumers underscore the importance of communicating the science to the general public. Conclusion: Our project provides evidence that engaging a diverse expert stakeholder panel is an effective method of translating gaps in nutrition research into prioritized areas of future research.

Evolving Understanding of Antarctic Ice-Sheet Physics and Ambiguity in Probabilistic Sea-Level Projections

NASA Technical Reports Server (NTRS)

Kopp, Robert E.; DeConto, Robert M.; Bader, Daniel A.; Hay, Carling C.; Horton, Radley M.; Kulp, Scott; Oppenheimer, Michael; Pollard, David; Strauss, Benjamin

2017-01-01

Mechanisms such as ice-shelf hydrofracturing and ice-cliff collapse may rapidly increase discharge from marine-based ice sheets. Here, we link a probabilistic framework for sea-level projections to a small ensemble of Antarctic ice-sheet (AIS) simulations incorporating these physical processes to explore their influence on global-mean sea-level (GMSL) and relative sea-level (RSL). We compare the new projections to past results using expert assessment and structured expert elicitation about AIS changes. Under high greenhouse gas emissions (Representative Concentration Pathway [RCP] 8.5), median projected 21st century GMSL rise increases from 79 to 146 cm. Without protective measures, revised median RSL projections would by 2100 submerge land currently home to 153 million people, an increase of 44 million. The use of a physical model, rather than simple parameterizations assuming constant acceleration of ice loss, increases forcing sensitivity: overlap between the central 90% of simulations for 2100 for RCP 8.5 (93-243 cm) and RCP 2.6 (26-98 cm) is minimal. By 2300, the gap between median GMSL estimates for RCP 8.5 and RCP 2.6 reaches >10 m, with median RSL projections for RCP 8.5 jeopardizing land now occupied by 950 million people (versus 167 million for RCP 2.6). The minimal correlation between the contribution of AIS to GMSL by 2050 and that in 2100 and beyond implies current sea-level observations cannot exclude future extreme outcomes. The sensitivity of post-2050 projections to deeply uncertain physics highlights the need for robust decision and adaptive management frameworks.

A model to inform management actions as a response to chytridiomycosis-associated decline

USGS Publications Warehouse

Converse, Sarah J.; Bailey, Larissa L.; Mosher, Brittany A.; Funk, W. Chris; Gerber, Brian D.; Muths, Erin L.

2017-01-01

Decision-analytic models provide forecasts of how systems of interest will respond to management. These models can be parameterized using empirical data, but sometimes require information elicited from experts. When evaluating the effects of disease in species translocation programs, expert judgment is likely to play a role because complete empirical information will rarely be available. We illustrate development of a decision-analytic model built to inform decision-making regarding translocations and other management actions for the boreal toad (Anaxyrus boreas boreas), a species with declines linked to chytridiomycosis caused by Batrachochytrium dendrobatidis (Bd). Using the model, we explored the management implications of major uncertainties in this system, including whether there is a genetic basis for resistance to pathogenic infection by Bd, how translocation can best be implemented, and the effectiveness of efforts to reduce the spread of Bd. Our modeling exercise suggested that while selection for resistance to pathogenic infectionDecision-analytic models provide forecasts of how systems of interest will respond to management. These models can be parameterized using empirical data, but sometimes require information elicited from experts. When evaluating the effects of disease in species translocation programs, expert judgment is likely to play a role because complete empirical information will rarely be available. We illustrate development of a decision-analytic model built to inform decision-making regarding translocations and other management actions for the boreal toad (Anaxyrus boreas boreas), a species with declines linked to chytridiomycosis caused by Batrachochytrium dendrobatidis (Bd). Using the model, we explored the management implications of major uncertainties in this system, including whether there is a genetic basis for resistance to pathogenic infection by Bd, how translocation can best be implemented, and the effectiveness of efforts to reduce the spread of Bd. Our modeling exercise suggested that while selection for resistance to pathogenic infection by Bd could increase numbers of sites occupied by toads, and translocations could increase the rate of toad recovery, efforts to reduce the spread of Bd may have little effect. We emphasize the need to continue developing and parameterizing models necessary to assess management actions for combating chytridiomycosis-associated declines. by Bd could increase numbers of sites occupied by toads, and translocations could increase the rate of toad recovery, efforts to reduce the spread of Bd may have little effect. We emphasize the need to continue developing and parameterizing models necessary to assess management actions for combating chytridiomycosis-associated declines.

Extending Hypothesis Testing of Edge Organizations Using Functional Magnetic Resonance Imaging (fMRI) During ELICIT

DTIC Science & Technology

2010-06-01

task difficulty and response correctness on neural systems supporting fluid reasoning. Cognitive Neurodynamics 1 (1): 71-84. Kaplan, J.T., Iacoboni...dynamic influences on decision-making and trust during social interaction. ELICITing Behavior ELICIT is designed to explore social and cognitive ...a person’s own self-awareness in the game experience, (2) their cognitive processes of reasoning, and (3) the modulation of uncertainty that primes

Current debates on end-of-life sedation: an international expert elicitation study.

PubMed

Papavasiliou, Evangelia Evie; Payne, Sheila; Brearley, Sarah

2014-08-01

End-of-life sedation, though increasingly prevalent and widespread internationally, remains one of the most highly debated medical practices in the context of palliative medicine. This qualitative study aims to elicit and record the perspectives of leading international palliative care experts on current debates. Twenty-one professionals from diverse backgrounds, sharing field-specific knowledge/expertise defined by significant scholarly contribution on end-of-life sedation, were recruited. Open-ended, semi-structured interviews, following a topic-oriented structure reflecting on current debates, were conducted. Results were analysed using thematic content analysis. Three main aspects of sedation were identified and discussed as potentially problematic: (a) continuous deep sedation as an extreme facet of end-of-life sedation, (b) psycho-existential suffering as an ambivalent indication for sedation and (c) withdrawal or withholding of artificial nutrition and hydration as potentially life-shortening. On these grounds, concerns were reported over end-of-life sedation being morally equivalent to euthanasia. Considerable emphasis was placed on intentions as the distinguishing factor between end-of-life acts, and protective safeguards were introduced to distance sedation from euthanasia. This study shows that, despite the safeguards introduced, certain aspects of sedation, including the intentions associated with the practice, are still under question, parallels being drawn between end-of-life sedation and euthanasia. This reaffirms the existence of a grey area surrounding the two practices, already evidenced in countries where euthanasia is legalized. More clarity over the issues that generate this grey area, with their causes being uncovered and eliminated, is imperative to resolve current debates and effectively inform research, policy and practice of end-of-life sedation.

Assessing the impact of growth hormone deficiency and treatment in adults: development of a new disease-specific measure.

PubMed

Brod, Meryl; Højbjerre, Lise; Adalsteinsson, Johan Erpur; Rasmussen, Michael Højby

2014-04-01

Approximately 50 000 adults in the United States are diagnosed with GH deficiency, which has negative impacts on cognitive functioning, psychological well-being, and quality of life. This paper presents development and validation of a patient-reported outcome measure (PRO), the Treatment-Related Impact Measure-Adult Growth Hormone Deficiency (TRIM-AGHD). The TRIM-AGHD was developed to measure the impact of GH deficiency and its treatment. The development and validation of the TRIM-AGHD was conducted according to the Food and Drug Administration guidance on the development of PROs. Concept elicitation, conducted in three countries included interviews with patients, clinical experts, and literature review. Qualitative data were analyzed based on grounded theory principles, and draft items were cognitively debriefed. The measure underwent psychometric validation in a US clinic-based population. An a priori statistical analysis plan included assessment of the measurement model, reliability, and validity. Item functioning was reviewed using item response theory analyses. Forty-eight patients and six clinical experts participated in concept elicitation and 169 patients completed the validation study. TRIM-AGHD was measured. Factor analysis resulted in four domains: energy level, physical health, emotional health, and cognitive ability. The item response theory confirmed adequate item fit and placement within their domain. Internal consistency ranged from 0.82 to 0.95 and test-retest ranged from 0.80 to 0.92. All prespecified hypotheses for convergent validity and all but two for discriminant validity were met. The final 26-item TRIM-AGHD can be considered a reliable and valid PRO of the impact of disease and treatment for adult GH deficiency.

Holistic processing of musical notation: Dissociating failures of selective attention in experts and novices.

PubMed

Wong, Yetta Kwailing; Gauthier, Isabel

2010-12-01

Holistic processing (i.e., the tendency to process objects as wholes) is associated with face perception and also with expertise individuating novel objects. Surprisingly, recent work also reveals holistic effects in novice observers. It is unclear whether the same mechanisms support holistic effects in experts and in novices. In the present study, we measured holistic processing of music sequences using a selective attention task in participants who vary in music-reading expertise. We found that holistic effects were strategic in novices but were relatively automatic in experts. Correlational analyses revealed that individual holistic effects were predicted by both individual music-reading ability and neural responses for musical notation in the right fusiform face area (rFFA), but in opposite directions for experts and novices, suggesting that holistic effects in the two groups may be of different natures. To characterize expert perception, it is important not only to measure the tendency to process objects as wholes, but also to test whether this effect is dependent on task constraints.

Content validation using an expert panel: assessment process for assistive technology adopted by farmers with disabilities.

PubMed

Mathew, S N; Field, W E; French, B F

2011-07-01

This article reports the use of an expert panel to perform content validation of an experimental assessment process for the safety of assistive technology (AT) adopted by farmers with disabilities. The validation process was conducted by a panel of six experts experienced in the subject matter, i.e., design, use, and assessment of AT for farmers with disabilities. The exercise included an evaluation session and two focus group sessions. The evaluation session consisted of using the assessment process under consideration by the panel to evaluate a set of nine ATs fabricated by a farmer on his farm site. The expert panel also participated in the focus group sessions conducted immediately before and after the evaluation session. The resulting data were analyzed using discursive analysis, and the results were incorporated into the final assessment process. The method and the results are presented with recommendations for the use of expert panels in research projects and validation of assessment tools.

The neural organization of perception in chess experts.

PubMed

Krawczyk, Daniel C; Boggan, Amy L; McClelland, M Michelle; Bartlett, James C

2011-07-20

The human visual system responds to expertise, and it has been suggested that regions that process faces also process other objects of expertise including chess boards by experts. We tested whether chess and face processing overlap in brain activity using fMRI. Chess experts and novices exhibited face selective areas, but these regions showed no selectivity to chess configurations relative to other stimuli. We next compared neural responses to chess and to scrambled chess displays to isolate areas relevant to expertise. Areas within the posterior cingulate, orbitofrontal cortex, and right temporal cortex were active in this comparison in experts over novices. We also compared chess and face responses within the posterior cingulate and found this area responsive to chess only in experts. These findings indicate that the configurations in chess are not strongly processed by face-selective regions that are selective for faces in individuals who have expertise in both domains. Further, the area most consistently involved in chess did not show overlap with faces. Overall, these results suggest that expert visual processing may be similar at the level of recognition, but need not show the same neural correlates. Copyright © 2011 Elsevier Ireland Ltd. All rights reserved.

Integrating uncertainty into public energy research and development decisions

NASA Astrophysics Data System (ADS)

Anadón, Laura Díaz; Baker, Erin; Bosetti, Valentina

2017-05-01

Public energy research and development (R&D) is recognized as a key policy tool for transforming the world's energy system in a cost-effective way. However, managing the uncertainty surrounding technological change is a critical challenge for designing robust and cost-effective energy policies. The design of such policies is particularly important if countries are going to both meet the ambitious greenhouse-gas emissions reductions goals set by the Paris Agreement and achieve the required harmonization with the broader set of objectives dictated by the Sustainable Development Goals. The complexity of informing energy technology policy requires, and is producing, a growing collaboration between different academic disciplines and practitioners. Three analytical components have emerged to support the integration of technological uncertainty into energy policy: expert elicitations, integrated assessment models, and decision frameworks. Here we review efforts to incorporate all three approaches to facilitate public energy R&D decision-making under uncertainty. We highlight emerging insights that are robust across elicitations, models, and frameworks, relating to the allocation of public R&D investments, and identify gaps and challenges that remain.

Causality Assessment in Drug-Induced Liver Injury Using a Structured Expert Opinion Process: Comparison to the Roussel-Uclaf Causality Assessment Method

PubMed Central

Rockey, Don C.; Seeff, Leonard B.; Rochon, James; Freston, James; Chalasani, Naga; Bonacini, Maurizio; Fontana, Robert J.; Hayashi, Paul H.

2011-01-01

Drug-induced liver injury (DILI) is largely a diagnosis of exclusion and is therefore challenging. The US Drug-Induced Liver Injury Network (DILIN) prospective study used two methods to assess DILI causality: a structured expert opinion process and the Roussel-Uclaf Causality Assessment Method (RUCAM). Causality assessment focused on detailed clinical and laboratory data from patients with suspected DILI. The adjudication process used standardized numerical and descriptive definitions and scored cases as definite, highly likely, probable, possible, or unlikely. Results of the structured expert opinion procedure were compared with those derived by the RUCAM approach. Among 250 patients with suspected DILI, the expert opinion adjudication process scored 78 patients (31%) as definite, 102 (41%) as highly likely, 37 (15%) as probable, 25 (10%) as possible, and 8 (3%) as unlikely. Among 187 enrollees who had received a single implicated drug, initial complete agreement was reached for 50 (27%) with the expert opinion process and for 34 (19%) with a five-category RUCAM scale (P = 0.08), and the two methods demonstrated a modest correlation with each other (Spearman's r = 0.42, P = 0.0001). Importantly, the RUCAM approach substantially shifted the causality likelihood toward lower probabilities in comparison with the DILIN expert opinion process. Conclusion The structured DILIN expert opinion process produced higher agreement rates and likelihood scores than RUCAM in assessing causality, but there was still considerable interobserver variability in both. Accordingly, a more objective, reliable, and reproducible means of assessing DILI causality is still needed. PMID:20512999

Effects of unexpected chords and of performer's expression on brain responses and electrodermal activity.

PubMed

Koelsch, Stefan; Kilches, Simone; Steinbeis, Nikolaus; Schelinski, Stefanie

2008-07-09

There is lack of neuroscientific studies investigating music processing with naturalistic stimuli, and brain responses to real music are, thus, largely unknown. This study investigates event-related brain potentials (ERPs), skin conductance responses (SCRs) and heart rate (HR) elicited by unexpected chords of piano sonatas as they were originally arranged by composers, and as they were played by professional pianists. From the musical excerpts played by the pianists (with emotional expression), we also created versions without variations in tempo and loudness (without musical expression) to investigate effects of musical expression on ERPs and SCRs. Compared to expected chords, unexpected chords elicited an early right anterior negativity (ERAN, reflecting music-syntactic processing) and an N5 (reflecting processing of meaning information) in the ERPs, as well as clear changes in the SCRs (reflecting that unexpected chords also elicited emotional responses). The ERAN was not influenced by emotional expression, whereas N5 potentials elicited by chords in general (regardless of their chord function) differed between the expressive and the non-expressive condition. These results show that the neural mechanisms of music-syntactic processing operate independently of the emotional qualities of a stimulus, justifying the use of stimuli without emotional expression to investigate the cognitive processing of musical structure. Moreover, the data indicate that musical expression affects the neural mechanisms underlying the processing of musical meaning. Our data are the first to reveal influences of musical performance on ERPs and SCRs, and to show physiological responses to unexpected chords in naturalistic music.

Holistic processing of fingerprints by expert forensic examiners.

PubMed

Vogelsang, Macgregor D; Palmeri, Thomas J; Busey, Thomas A

2017-01-01

Holistic processing is often characterized as a process by which objects are perceived as a whole rather than a compilation of individual features. This mechanism may play an important role in the development of perceptual expertise because it allows for rapid integration across image regions. The present work explores whether holistic processing is present in latent fingerprint examiners, who compare fingerprints collected from crime scenes against a set of standards taken from a suspect. We adapted a composite task widely used in the face recognition and perceptual expertise literatures, in which participants were asked to match only a particular half of a fingerprint with a previous image while ignoring the other half. We tested both experts and novices, using both upright and inverted fingerprints. For upright fingerprints, we found weak evidence for holistic processing, but with no differences between experts and novices with respect to holistic processing. For inverted fingerprints, we found stronger evidence of holistic processing, with weak evidence for differences between experts and novices. These relatively weak holistic processing effects contrast with robust evidence for holistic processing with faces and with objects in other domains of perceptual expertise. The data constrain models of holistic processing by demonstrating that latent fingerprint experts and novices may not substantively differ in terms of the amount of holistic processing and that inverted stimuli actually produced more evidence for holistic processing than upright stimuli. Important differences between the present fingerprint stimuli and those in the literature include the lack of verbal labels for experts and the absence of strong vertical asymmetries, both of which might contribute to stronger holistic processing signatures in other stimulus domains.

Knowledge acquisition for a simple expert controller

NASA Technical Reports Server (NTRS)

Bieker, B.

1987-01-01

A method is presented for process control which has the properties of being incremental, cyclic and top-down. It is described on the basis of the development of an expert controller for a simple, but nonlinear control route. A quality comparison between expert controller and process operator shows the ability of the method for knowledge acquisition.

Wishful Thinking? Inside the Black Box of Exposure Assessment.

PubMed

Money, Annemarie; Robinson, Christine; Agius, Raymond; de Vocht, Frank

2016-05-01

Decision-making processes used by experts when undertaking occupational exposure assessment are relatively unknown, but it is often assumed that there is a common underlying method that experts employ. However, differences in training and experience of assessors make it unlikely that one general method for expert assessment would exist. Therefore, there are concerns about formalizing, validating, and comparing expert estimates within and between studies that are difficult, if not impossible, to characterize. Heuristics on the other hand (the processes involved in decision making) have been extensively studied. Heuristics are deployed by everyone as short-cuts to make the often complex process of decision-making simpler, quicker, and less burdensome. Experts' assessments are often subject to various simplifying heuristics as a way to reach a decision in the absence of sufficient data. Therefore, investigating the underlying heuristics or decision-making processes involved may help to shed light on the 'black box' of exposure assessment. A mixed method study was conducted utilizing both a web-based exposure assessment exercise incorporating quantitative and semiqualitative elements of data collection, and qualitative semi-structured interviews with exposure assessors. Qualitative data were analyzed using thematic analysis. Twenty-five experts completed the web-based exposure assessment exercise and 8 of these 25 were randomly selected to participate in the follow-up interview. Familiar key themes relating to the exposure assessment exercise emerged; 'intensity'; 'probability'; 'agent'; 'process'; and 'duration' of exposure. However, an important aspect of the detailed follow-up interviews revealed a lack of structure and order with which participants described their decision making. Participants mostly described some form of an iterative process, heavily relying on the anchoring and adjustment heuristic, which differed between experts. In spite of having undertaken comparable training (in occupational hygiene or exposure assessment), experts use different methods to assess exposure. Decision making appears to be an iterative process with heavy reliance on the key heuristic of anchoring and adjustment. Using multiple experts to assess exposure while providing some form of anchoring scenario to build from, and additional training in understanding the impact of simple heuristics on the process of decision making, is likely to produce a more methodical approach to assessment; thereby improving consistency and transparency in expert exposure assessment. © The Author 2016. Published by Oxford University Press on behalf of the British Occupational Hygiene Society.

Knowledge-based fault diagnosis system for refuse collection vehicle

DOE Office of Scientific and Technical Information (OSTI.GOV)

Tan, CheeFai; Juffrizal, K.; Khalil, S. N.

The refuse collection vehicle is manufactured by local vehicle body manufacturer. Currently; the company supplied six model of the waste compactor truck to the local authority as well as waste management company. The company is facing difficulty to acquire the knowledge from the expert when the expert is absence. To solve the problem, the knowledge from the expert can be stored in the expert system. The expert system is able to provide necessary support to the company when the expert is not available. The implementation of the process and tool is able to be standardize and more accurate. The knowledgemore » that input to the expert system is based on design guidelines and experience from the expert. This project highlighted another application on knowledge-based system (KBS) approached in trouble shooting of the refuse collection vehicle production process. The main aim of the research is to develop a novel expert fault diagnosis system framework for the refuse collection vehicle.« less

Sherlock Holmes: an expert's view of expertise.

PubMed

André, Didierjean; Fernand, Gobet

2008-02-01

In recent years, there has been an intense research effort to understand the cognitive processes and structures underlying expert behaviour. Work in different fields, including scientific domains, sports, games and mnemonics, has shown that there are vast differences in perceptual abilities between experts and novices, and that these differences may underpin other cognitive differences in learning, memory and problem solving. In this article, we evaluate the progress made in the last years through the eyes of an outstanding, albeit fictional, expert: Sherlock Holmes. We first use the Sherlock Holmes character to illustrate expert processes as described by current research and theories. In particular, the role of perception, as well as the nature and influence of expert knowledge, are all present in the description of Conan Doyle's hero. In the second part of the article, we discuss a number of issues that current research on expertise has barely addressed. These gaps include, for example, several forms of reasoning, the influence of emotions on cognition, and the effect of age on experts' knowledge and cognitive processes. Thus, although nearly 120-year-old, Conan Doyle's books show remarkable illustrations of expert behaviour, including the coverage of themes that have mostly been overlooked by current research.

Perceptual expertise: can sensorimotor experience change holistic processing and left-side bias?

PubMed

Tso, Ricky Van-yip; Au, Terry Kit-fong; Hsiao, Janet Hui-wen

2014-09-01

Holistic processing and left-side bias are both behavioral markers of expert face recognition. By contrast, expert recognition of characters in Chinese orthography involves left-side bias but reduced holistic processing, although faces and Chinese characters share many visual properties. Here, we examined whether this reduction in holistic processing of Chinese characters can be better explained by writing experience than by reading experience. Compared with Chinese nonreaders, Chinese readers who had limited writing experience showed increased holistic processing, whereas Chinese readers who could write characters fluently showed reduced holistic processing. This result suggests that writing and sensorimotor experience can modulate holistic-processing effects and that the reduced holistic processing observed in expert Chinese readers may depend mostly on writing experience. However, both expert writers and writers with limited experience showed similarly stronger left-side bias than novices did in processing mirror-symmetric Chinese characters; left-side bias may therefore be a robust expertise marker for object recognition that is uninfluenced by sensorimotor experience. © The Author(s) 2014.

Using expert knowledge to incorporate uncertainty in cause-of-death assignments for modeling of cause-specific mortality

USGS Publications Warehouse

Walsh, Daniel P.; Norton, Andrew S.; Storm, Daniel J.; Van Deelen, Timothy R.; Heisy, Dennis M.

2018-01-01

Implicit and explicit use of expert knowledge to inform ecological analyses is becoming increasingly common because it often represents the sole source of information in many circumstances. Thus, there is a need to develop statistical methods that explicitly incorporate expert knowledge, and can successfully leverage this information while properly accounting for associated uncertainty during analysis. Studies of cause-specific mortality provide an example of implicit use of expert knowledge when causes-of-death are uncertain and assigned based on the observer's knowledge of the most likely cause. To explicitly incorporate this use of expert knowledge and the associated uncertainty, we developed a statistical model for estimating cause-specific mortality using a data augmentation approach within a Bayesian hierarchical framework. Specifically, for each mortality event, we elicited the observer's belief of cause-of-death by having them specify the probability that the death was due to each potential cause. These probabilities were then used as prior predictive values within our framework. This hierarchical framework permitted a simple and rigorous estimation method that was easily modified to include covariate effects and regularizing terms. Although applied to survival analysis, this method can be extended to any event-time analysis with multiple event types, for which there is uncertainty regarding the true outcome. We conducted simulations to determine how our framework compared to traditional approaches that use expert knowledge implicitly and assume that cause-of-death is specified accurately. Simulation results supported the inclusion of observer uncertainty in cause-of-death assignment in modeling of cause-specific mortality to improve model performance and inference. Finally, we applied the statistical model we developed and a traditional method to cause-specific survival data for white-tailed deer, and compared results. We demonstrate that model selection results changed between the two approaches, and incorporating observer knowledge in cause-of-death increased the variability associated with parameter estimates when compared to the traditional approach. These differences between the two approaches can impact reported results, and therefore, it is critical to explicitly incorporate expert knowledge in statistical methods to ensure rigorous inference.

Uncertainty Quantification Techniques for Population Density Estimates Derived from Sparse Open Source Data

DOE Office of Scientific and Technical Information (OSTI.GOV)

Stewart, Robert N; White, Devin A; Urban, Marie L

2013-01-01

The Population Density Tables (PDT) project at the Oak Ridge National Laboratory (www.ornl.gov) is developing population density estimates for specific human activities under normal patterns of life based largely on information available in open source. Currently, activity based density estimates are based on simple summary data statistics such as range and mean. Researchers are interested in improving activity estimation and uncertainty quantification by adopting a Bayesian framework that considers both data and sociocultural knowledge. Under a Bayesian approach knowledge about population density may be encoded through the process of expert elicitation. Due to the scale of the PDT effort whichmore » considers over 250 countries, spans 40 human activity categories, and includes numerous contributors, an elicitation tool is required that can be operationalized within an enterprise data collection and reporting system. Such a method would ideally require that the contributor have minimal statistical knowledge, require minimal input by a statistician or facilitator, consider human difficulties in expressing qualitative knowledge in a quantitative setting, and provide methods by which the contributor can appraise whether their understanding and associated uncertainty was well captured. This paper introduces an algorithm that transforms answers to simple, non-statistical questions into a bivariate Gaussian distribution as the prior for the Beta distribution. Based on geometric properties of the Beta distribution parameter feasibility space and the bivariate Gaussian distribution, an automated method for encoding is developed that responds to these challenging enterprise requirements. Though created within the context of population density, this approach may be applicable to a wide array of problem domains requiring informative priors for the Beta distribution.« less

Development of a novel observer-reported outcome measure for the assessment of Respiratory Syncytial Virus (RSV) infection symptoms in pediatric clinical trials.

PubMed

Lewis, Sandy; DeMuro, Carla; Block, Stan L; Senders, Shelly; Wisman, Paul; Toback, Seth; Chien, Jason W; Williams, Valerie

2017-01-01

Respiratory syncytial virus (RSV) is a seasonal infection affecting most children by 2 years of age and the leading cause of lower respiratory tract infection requiring hospitalization in infants. Novel antiviral medications are in development to improve the clinical outcomes of RSV; however, no clinical outcome assessments (COAs) for RSV have been developed in alignment with the United States Food and Drug Administration patient-reported outcome guidance to assist in the evaluation of new therapies. To address this need, an observer-reported outcome (ObsRO) measure designed to assess observable RSV symptoms was created. The literature was reviewed to evaluate existing COAs and identify constructs of interest. Individual caregiver interviews elicited concepts that informed item development, and candidate items were subsequently evaluated in two rounds of cognitive testing. Separate cohorts of caregivers of RSV-infected nonhospitalized and hospitalized infants participated. Therapeutic-area experts provided input throughout the instrument development process. Caregivers of 39 children < 24 months old with RSV (31 nonhospitalized, 8 hospitalized) participated in in-depth, individual interviews during concept elicitation and cognitive debriefing, resulting in 21 concepts identified as potentially observable and relevant to young children with RSV. The item pool was reduced to 12 cardinal symptoms and behavior impacts reported to be directly observable by caregivers, with 10 daytime and 9 nighttime symptoms to capture diurnal variation in severity. The RSV Caregiver Diary assesses RSV symptom severity and change from the parent or caregiver perspective in a standardized manner to measure treatment benefit. Following psychometric evaluation and refinement, this tool is expected to be suitable for assisting in the clinical development of RSV therapeutics.

Modeling Water Filtration

ERIC Educational Resources Information Center

Parks, Melissa

2014-01-01

Model-eliciting activities (MEAs) are not new to those in engineering or mathematics, but they were new to Melissa Parks. Model-eliciting activities are simulated real-world problems that integrate engineering, mathematical, and scientific thinking as students find solutions for specific scenarios. During this process, students generate solutions…

Homosexual imagery in print advertisements: attended, remembered, but disliked.

PubMed

Angelini, James R; Bradley, Samuel D

2010-01-01

This study examines whether print advertisements featuring homosexual imagery elicit greater attention and recall while eliciting more negative responses than advertisements featuring heterosexual images. Data indicate that these advertisements were indeed better remembered and required more time to cognitively process, likely because of the advertisement's imagery being inconsistent with existing gender schema. Other responses demonstrated that homosexual imagery negatively impacted opinions about the advertisement itself and the brand featured, and elicited more negative self-reported valence and arousal.

Capturing security requirements for software systems.

PubMed

El-Hadary, Hassan; El-Kassas, Sherif

2014-07-01

Security is often an afterthought during software development. Realizing security early, especially in the requirement phase, is important so that security problems can be tackled early enough before going further in the process and avoid rework. A more effective approach for security requirement engineering is needed to provide a more systematic way for eliciting adequate security requirements. This paper proposes a methodology for security requirement elicitation based on problem frames. The methodology aims at early integration of security with software development. The main goal of the methodology is to assist developers elicit adequate security requirements in a more systematic way during the requirement engineering process. A security catalog, based on the problem frames, is constructed in order to help identifying security requirements with the aid of previous security knowledge. Abuse frames are used to model threats while security problem frames are used to model security requirements. We have made use of evaluation criteria to evaluate the resulting security requirements concentrating on conflicts identification among requirements. We have shown that more complete security requirements can be elicited by such methodology in addition to the assistance offered to developers to elicit security requirements in a more systematic way.

Capturing security requirements for software systems

PubMed Central

El-Hadary, Hassan; El-Kassas, Sherif

2014-01-01

Security is often an afterthought during software development. Realizing security early, especially in the requirement phase, is important so that security problems can be tackled early enough before going further in the process and avoid rework. A more effective approach for security requirement engineering is needed to provide a more systematic way for eliciting adequate security requirements. This paper proposes a methodology for security requirement elicitation based on problem frames. The methodology aims at early integration of security with software development. The main goal of the methodology is to assist developers elicit adequate security requirements in a more systematic way during the requirement engineering process. A security catalog, based on the problem frames, is constructed in order to help identifying security requirements with the aid of previous security knowledge. Abuse frames are used to model threats while security problem frames are used to model security requirements. We have made use of evaluation criteria to evaluate the resulting security requirements concentrating on conflicts identification among requirements. We have shown that more complete security requirements can be elicited by such methodology in addition to the assistance offered to developers to elicit security requirements in a more systematic way. PMID:25685514

Toward the Decision Tree for Inferring Requirements Maturation Types

NASA Astrophysics Data System (ADS)

Nakatani, Takako; Kondo, Narihito; Shirogane, Junko; Kaiya, Haruhiko; Hori, Shozo; Katamine, Keiichi

Requirements are elicited step by step during the requirements engineering (RE) process. However, some types of requirements are elicited completely after the scheduled requirements elicitation process is finished. Such a situation is regarded as problematic situation. In our study, the difficulties of eliciting various kinds of requirements is observed by components. We refer to the components as observation targets (OTs) and introduce the word “Requirements maturation.” It means when and how requirements are elicited completely in the project. The requirements maturation is discussed on physical and logical OTs. OTs Viewed from a logical viewpoint are called logical OTs, e.g. quality requirements. The requirements of physical OTs, e.g., modules, components, subsystems, etc., includes functional and non-functional requirements. They are influenced by their requesters' environmental changes, as well as developers' technical changes. In order to infer the requirements maturation period of each OT, we need to know how much these factors influence the OTs' requirements maturation. According to the observation of actual past projects, we defined the PRINCE (Pre Requirements Intelligence Net Consideration and Evaluation) model. It aims to guide developers in their observation of the requirements maturation of OTs. We quantitatively analyzed the actual cases with their requirements elicitation process and extracted essential factors that influence the requirements maturation. The results of interviews of project managers are analyzed by WEKA, a data mining system, from which the decision tree was derived. This paper introduces the PRINCE model and the category of logical OTs to be observed. The decision tree that helps developers infer the maturation type of an OT is also described. We evaluate the tree through real projects and discuss its ability to infer the requirements maturation types.

Cognitive integration of asynchronous natural or non-natural auditory and visual information in videos of real-world events: an event-related potential study.

PubMed

Liu, B; Wang, Z; Wu, G; Meng, X

2011-04-28

In this paper, we aim to study the cognitive integration of asynchronous natural or non-natural auditory and visual information in videos of real-world events. Videos with asynchronous semantically consistent or inconsistent natural sound or speech were used as stimuli in order to compare the difference and similarity between multisensory integrations of videos with asynchronous natural sound and speech. The event-related potential (ERP) results showed that N1 and P250 components were elicited irrespective of whether natural sounds were consistent or inconsistent with critical actions in videos. Videos with inconsistent natural sound could elicit N400-P600 effects compared to videos with consistent natural sound, which was similar to the results from unisensory visual studies. Videos with semantically consistent or inconsistent speech could both elicit N1 components. Meanwhile, videos with inconsistent speech would elicit N400-LPN effects in comparison with videos with consistent speech, which showed that this semantic processing was probably related to recognition memory. Moreover, the N400 effect elicited by videos with semantically inconsistent speech was larger and later than that elicited by videos with semantically inconsistent natural sound. Overall, multisensory integration of videos with natural sound or speech could be roughly divided into two stages. For the videos with natural sound, the first stage might reflect the connection between the received information and the stored information in memory; and the second one might stand for the evaluation process of inconsistent semantic information. For the videos with speech, the first stage was similar to the first stage of videos with natural sound; while the second one might be related to recognition memory process. Copyright © 2011 IBRO. Published by Elsevier Ltd. All rights reserved.

Setting priorities for non-regulatory animal health in Ireland: results from an expert Policy Delphi study and a farmer priority identification survey.

PubMed

More, Simon J; McKenzie, Ken; O'Flaherty, Joe; Doherty, Michael L; Cromie, Andrew R; Magan, Mike J

2010-07-01

Agriculture is a very important contributor to the Irish economy. In Ireland, national animal health services have been a government, rather than an industry, responsibility. In 2009, Animal Health Ireland (AHI) was established to provide a partnership approach to national leadership of non-regulatory animal health issues (those not subject to national and/or EU regulation). The objectives of this study were to elicit opinion from experts and farmers about non-regulatory animal health issues facing Irish livestock industries, including prioritisation of animal health issues and identification of opportunities to maximise the effective use of AHI resources. The study was conducted with experts using Policy Delphi methodology over three rounds, and with farmers using a priority identification survey. Non-regulatory bovine diseases/conditions were prioritised by both experts and farmers based on impact and international competitiveness. For each high-priority disease/condition, experts were asked to provide an assessment based on cost, impact, international perception, impediment to international market access and current resource usage effectiveness. Further information was also sought from experts about resource allocation preferences, methods to improve education and coordination, and innovative measures to improve prevention and management. There was close agreement between responses from experts and dairy farmers: each gave highest priority to 3 diseases with a biosecurity risk (subsequently termed 'biosecure diseases') (bovine viral diarrhoea [BVD], infectious bovine rhinotracheitis [IBR], paratuberculosis) and 4 diseases/conditions generally without a biosecurity risk ('non-biosecure diseases/conditions') (fertility, udder health/milk quality, lameness, calf health). Beef farmers also prioritised parasitic conditions and weanling pneumonia. The adverse impact of biosecure diseases is currently considered relatively minor by experts, but would increase substantially in time. There are already substantial costs to farms and agribusiness from non-biosecure diseases/conditions. Experts preferred an equal allocation of resources between these biosecure and non-biosecure diseases/conditions, with emphasis on adopting/adapting international models, education and awareness-raising. The results from this study provide robust insights about non-regulatory animal health priorities in Ireland, as perceived by experts and farmers, using methodologies that are both transparent and inclusive. They have already been extremely influential in shaping national policy, as a foundation for interdisciplinary (and multi-agency) cooperation, as a contribution to efforts to encourage stakeholder responsibility-taking, and to ongoing development of postgraduate and undergraduate veterinary education in Ireland. Copyright (c) 2010. Published by Elsevier B.V.

A HUMAN FACTORS ENGINEERING PROCESS TO SUPPORT HUMAN-SYSTEM INTERFACE DESIGN IN CONTROL ROOM MODERNIZATION

DOE Office of Scientific and Technical Information (OSTI.GOV)

Kovesdi, C.; Joe, J.; Boring, R.

The primary objective of the United States (U.S.) Department of Energy (DOE) Light Water Reactor Sustainability (LWRS) program is to sustain operation of the existing commercial nuclear power plants (NPPs) through a multi-pathway approach in conducting research and development (R&D). The Advanced Instrumentation, Information, and Control (II&C) System Technologies pathway conducts targeted R&D to address aging and reliability concerns with legacy instrumentation and control (I&C) and other information systems in existing U.S. NPPs. Control room modernization is an important part following this pathway, and human factors experts at Idaho National Laboratory (INL) have been involved in conducting R&D to supportmore » migration of new digital main control room (MCR) technologies from legacy analog and legacy digital I&C. This paper describes a human factors engineering (HFE) process that supports human-system interface (HSI) design in MCR modernization activities, particularly with migration of old digital to new digital I&C. The process described in this work is an expansion from the LWRS Report INL/EXT-16-38576, and is a requirements-driven approach that aligns with NUREG-0711 requirements. The work described builds upon the existing literature by adding more detail around key tasks and decisions to make when transitioning from HSI Design into Verification and Validation (V&V). The overall objective of this process is to inform HSI design and elicit specific, measurable, and achievable human factors criteria for new digital technologies. Upon following this process, utilities should have greater confidence with transitioning from HSI design into V&V.« less

Discovering the knowledge creation process of an expert group in women-friendly policy: The policy case of Seoul City.

PubMed

Oh, Young Sam; Nam, SungHee; Kim, Yuna

2016-01-01

This research explores how expert knowledge is created in the process of women-friendly policy making, based on actor network theory (ANT). To address this purpose, this study uses the "Women's Happiness in the City of Seoul" policy initiated by the local government of Seoul as one example of policy development. Research findings demonstrate that knowledge creation in expert groups followed the four stages suggested by ANT. In addition, this study found that various types of knowledge emerged from individual experts. This research elucidates the process of knowledge creation and its meanings for women-friendly policy.

How Expert Clinicians Intuitively Recognize a Medical Diagnosis.

PubMed

Brush, John E; Sherbino, Jonathan; Norman, Geoffrey R

2017-06-01

Research has shown that expert clinicians make a medical diagnosis through a process of hypothesis generation and verification. Experts begin the diagnostic process by generating a list of diagnostic hypotheses using intuitive, nonanalytic reasoning. Analytic reasoning then allows the clinician to test and verify or reject each hypothesis, leading to a diagnostic conclusion. In this article, we focus on the initial step of hypothesis generation and review how expert clinicians use experiential knowledge to intuitively recognize a medical diagnosis. Copyright © 2017 Elsevier Inc. All rights reserved.

Event-related potential variations in the encoding and retrieval of different amounts of contextual information.

PubMed

Estrada-Manilla, Cinthya; Cansino, Selene

2012-06-15

Episodic memory events occur within multidimensional contexts; however, the electrophysiological manifestations associated with processing of more than one context have been rarely investigated. The effect of the amount of context on the ERPs was studied using two single and one double source memory tasks and by comparing full and partial context retrieval within a double source task. The single source tasks elicited waveforms with a larger amplitude during successful encoding and retrieval than the double source task. Compared with the waveforms elicited with a full source response, a partial source response elicited waveforms with a smaller amplitude, probably because the retrieval success for one context was combined with the retrieval attempt processes for the missing source. Comparing the tasks revealed that the larger the amount of contextual information processed, the smaller the amplitude of the ERPs, indicating that greater effort or further control processes were required during double source retrieval. Copyright © 2012 Elsevier B.V. All rights reserved.

Controlling Real-Time Processes On The Space Station With Expert Systems

NASA Astrophysics Data System (ADS)

Leinweber, David; Perry, John

1987-02-01

Many aspects of space station operations involve continuous control of real-time processes. These processes include electrical power system monitoring, propulsion system health and maintenance, environmental and life support systems, space suit checkout, on-board manufacturing, and servicing of attached vehicles such as satellites, shuttles, orbital maneuvering vehicles, orbital transfer vehicles and remote teleoperators. Traditionally, monitoring of these critical real-time processes has been done by trained human experts monitoring telemetry data. However, the long duration of space station missions and the high cost of crew time in space creates a powerful economic incentive for the development of highly autonomous knowledge-based expert control procedures for these space stations. In addition to controlling the normal operations of these processes, the expert systems must also be able to quickly respond to anomalous events, determine their cause and initiate corrective actions in a safe and timely manner. This must be accomplished without excessive diversion of system resources from ongoing control activities and any events beyond the scope of the expert control and diagnosis functions must be recognized and brought to the attention of human operators. Real-time sensor based expert systems (as opposed to off-line, consulting or planning systems receiving data via the keyboard) pose particular problems associated with sensor failures, sensor degradation and data consistency, which must be explicitly handled in an efficient manner. A set of these systems must also be able to work together in a cooperative manner. This paper describes the requirements for real-time expert systems in space station control, and presents prototype implementations of space station expert control procedures in PICON (process intelligent control). PICON is a real-time expert system shell which operates in parallel with distributed data acquisition systems. It incorporates a specialized inference engine with a specialized scheduling portion specifically designed to match the allocation of system resources with the operational requirements of real-time control systems. Innovative knowledge engineering techniques used in PICON to facilitate the development of real-time sensor-based expert systems which use the special features of the inference engine are illustrated in the prototype examples.

Electrophysiological evidence for the modulation of retrieval orientation by depth of study processing.

PubMed

Rugg, M D; Allan, K; Birch, C S

2000-07-01

Event-related potentials (ERPs) were employed to investigate whether brain activity elicited by retrieval cues in a memory test varies according to the encoding task undertaken at study. Two recognition memory test blocks were administered, preceded, in one case, by a "shallow" study task (alphabetic judgement) and, in the other case, by a "deep" task (sentence generation). ERPs elicited by the new words in each test block differed, the ERPs elicited in the block following the shallow study task exhibiting the more positive-going waveforms. This finding was taken as evidence that subjects adopt different "retrieval sets" when attempting to retrieve items that had been encoded in terms of alphabetic versus semantic attributes. Differences between the ERPs elicited by correctly classified old and new words (old/new effects) also varied with encoding task. The effects for deeply studied words resembled those found in previous ERP studies of recognition memory, whereas old/new effects for shallowly studied words were confined to a late-onsetting, right frontal positivity. Together, the findings indicate that the depth of study processing influences two kinds of memory-related neural activity, associated with memory search operations, and the processing of retrieved information, respectively.

System perspectives of experts and farmers regarding the role of livelihood assets in risk perception: results from the structured mental model approach.

PubMed

Schoell, Regina; Binder, Claudia R

2009-02-01

Pesticide application is increasing and despite extensive educational programs farmers continue to take high health and environmental risks when applying pesticides. The structured mental model approach (SMMA) is a new method for risk perception analysis. It embeds farmers' risk perception into their livelihood system in the elaboration of a mental model (MM). Results from its first application are presented here. The study region is Vereda la Hoya (Colombia), an area characterized by subsistence farming, high use of pesticides, and a high incidence of health problems. Our hypothesis was that subsistence farmers were constrained by economic, environmental, and sociocultural factors, which consequently should influence their mental models. Thirteen experts and 10 farmers were interviewed and their MMs of the extended pesticide system elicited. The interviews were open-ended with the questions structured in three parts: (i) definition and ranking of types of capital with respect to their importance for the sustainability of farmers' livelihood; (ii) understanding the system and its dynamics; and (iii) importance of the agents in the farmers' agent network. Following this structure, each part of the interview was analyzed qualitatively and statistically. Our analyses showed that the mental models of farmers and experts differed significantly from each other. By applying the SMMA, we were also able to identify reasons for the divergence of experts' and farmers' MMs. Of major importance are the following factors: (i) culture and tradition; (ii) trust in the source of information; and (iii) feedback on knowledge.

HIV and Alcohol Research Priorities of City, State, and Federal Policymakers: Results of a Delphi Study

PubMed Central

Li, Lingfeng; Braithwaite, Ronald Scott

2015-01-01

We identified the research areas related to HIV and alcohol consumption that were of highest priority to city, state, and federal policymakers. From June to July 2014, we conducted a 3-round Delphi study to elicit from experts a list of alcohol- and HIV-related clinical trial research questions that were important to fund and rank order the list to identify questions of highest priority. Translating evidence into practice must be improved because some questions that have been extensively studied with results published in peer-reviewed journals were identified by the panel as areas needing additional research. PMID:26180968

HIV and Alcohol Research Priorities of City, State, and Federal Policymakers: Results of a Delphi Study.

PubMed

Uyei, Jennifer; Li, Lingfeng; Braithwaite, Ronald Scott

2015-09-01

We identified the research areas related to HIV and alcohol consumption that were of highest priority to city, state, and federal policymakers. From June to July 2014, we conducted a 3-round Delphi study to elicit from experts a list of alcohol- and HIV-related clinical trial research questions that were important to fund and rank order the list to identify questions of highest priority. Translating evidence into practice must be improved because some questions that have been extensively studied with results published in peer-reviewed journals were identified by the panel as areas needing additional research.

A comparison of analytic hierarchy process and conjoint analysis methods in assessing treatment alternatives for stroke rehabilitation.

PubMed

Ijzerman, Maarten J; van Til, Janine A; Bridges, John F P

2012-01-01

With growing emphasis on patient involvement in health technology assessment, there is a need for scientific methods that formally elicit patient preferences. Analytic hierarchy process (AHP) and conjoint analysis (CA) are two established scientific methods - albeit with very different objectives. The objective of this study was to compare the performance of AHP and CA in eliciting patient preferences for treatment alternatives for stroke rehabilitation. Five competing treatments for drop-foot impairment in stroke were identified. One survey, including the AHP and CA questions, was sent to 142 patients, resulting in 89 patients for final analysis (response rate 63%). Standard software was used to calculate attribute weights from both AHP and CA. Performance weights for the treatments were obtained from an expert panel using AHP. Subsequently, the mean predicted preference for each of the five treatments was calculated using the AHP and CA weights. Differences were tested using non-parametric tests. Furthermore, all treatments were rank ordered for each individual patient, using the AHP and CA weights. Important attributes in both AHP and CA were the clinical outcome (0.3 in AHP and 0.33 in CA) and risk of complications (about 0.2 in both AHP and CA). Main differences between the methods were found for the attributes 'impact of treatment' (0.06 for AHP and 0.28 for two combined attributes in CA) and 'cosmetics and comfort' (0.28 for two combined attributes in AHP and 0.05 for CA). On a group level, the most preferred treatments were soft tissue surgery (STS) and orthopedic shoes (OS). However, STS was most preferred using AHP weights versus OS using CA weights (p < 0.001). This difference was even more obvious when interpreting the individual treatment ranks. Nearly all patients preferred STS according to the AHP predictions, while >50% of the patients chose OS instead of STS, as most preferred treatment using CA weights. While we found differences between AHP and CA, these differences were most likely caused by the labeling of the attributes and the elicitation of performance judgments. CA scenarios are built using the level descriptions, and hence provide realistic treatment scenarios. In AHP, patients only compared less concrete attributes such as 'impact of treatment.' This led to less realistic choices, and thus overestimation of the preference for the surgical scenarios. Several recommendations are given on how to use AHP and CA in assessing patient preferences.

The effects of rational and experiential information processing of expert testimony in death penalty cases.

PubMed

Krauss, Daniel A; Lieberman, Joel D; Olson, Jodi

2004-01-01

Past research examining the effects of actuarial and clinical expert testimony on defendants' dangerousness in Texas death penalty sentencing has found that jurors are more influenced by less scientific pure clinical expert testimony and less influenced by more scientific actuarial expert testimony (Krauss & Lee, 2003; Krauss & Sales, 2001). By applying cognitive-experiential self-theory (CEST) to juror decision-making, the present study was undertaken in an attempt to offer a theoretical rationale for these findings. Based on past CEST research, 163 mock jurors were either directed into a rational mode or experiential mode of processing. Consistent with CEST and inconsistent with previous research using the same stimulus materials, results demonstrate that jurors in a rational mode of processing more heavily weighted actuarial expert testimony in their dangerousness assessments, while those jurors in the experiential condition were more influenced by clinical expert testimony. The policy implications of these findings are discussed. Copyright 2004 John Wiley & Sons, Ltd.

76 FR 62072 - Center for Devices and Radiological Health; Standard Operating Procedures for Network of Experts...

Federal Register 2010, 2011, 2012, 2013, 2014

2011-10-06

...] Center for Devices and Radiological Health; Standard Operating Procedures for Network of Experts; Request... procedures (SOPs) for a new ``Network of Experts.'' The draft SOPs describe a new process for staff at the... FDA is announcing the availability of two draft SOPs, one entitled, ``Network of Experts--Expert...

Development of the Paris definition of early Crohn's disease for disease-modification trials: results of an international expert opinion process.

PubMed

Peyrin-Biroulet, Laurent; Billioud, Vincent; D'Haens, Geert; Panaccione, Remo; Feagan, Brian; Panés, Julian; Danese, Silvio; Schreiber, Stefan; Ogata, Haruhiko; Hibi, Toshifumi; Higgins, Peter D R; Beaugerie, Laurent; Chowers, Yehuda; Louis, Edouard; Steinwurz, Flávio; Reinisch, Walter; Rutgeerts, Paul; Colombel, Jean-Frédéric; Travis, Simon; Sandborn, William J

2012-12-01

We report the findings and outputs of an international expert opinion process to develop a definition of early Crohn's disease (CD) that could be used in future disease-modification trials. Nineteen experts on inflammatory bowel diseases held an international expert opinion meeting to discuss and agree on a definition for early CD to be used in disease-modification trials. The process included literature searches for the relevant basic-science and clinical evidence. A published preliminary definition of early CD was used as the basis for development of a proposed definition that was discussed at the expert opinion meeting. The participants then derived a final definition, based on best current knowledge, that it is hoped will be of practical use in disease-modification trials in CD.

Expert models and modeling processes associated with a computer-modeling tool

NASA Astrophysics Data System (ADS)

Zhang, Baohui; Liu, Xiufeng; Krajcik, Joseph S.

2006-07-01

Holding the premise that the development of expertise is a continuous process, this study concerns expert models and modeling processes associated with a modeling tool called Model-It. Five advanced Ph.D. students in environmental engineering and public health used Model-It to create and test models of water quality. Using think aloud technique and video recording, we captured their computer screen modeling activities and thinking processes. We also interviewed them the day following their modeling sessions to further probe the rationale of their modeling practices. We analyzed both the audio-video transcripts and the experts' models. We found the experts' modeling processes followed the linear sequence built in the modeling program with few instances of moving back and forth. They specified their goals up front and spent a long time thinking through an entire model before acting. They specified relationships with accurate and convincing evidence. Factors (i.e., variables) in expert models were clustered, and represented by specialized technical terms. Based on the above findings, we made suggestions for improving model-based science teaching and learning using Model-It.

Developing attributes and attribute-levels for a discrete choice experiment on micro health insurance in rural Malawi.

PubMed

Abiiro, Gilbert Abotisem; Leppert, Gerald; Mbera, Grace Bongololo; Robyn, Paul J; De Allegri, Manuela

2014-05-22

Discrete choice experiments (DCEs) are attribute-driven experimental techniques used to elicit stakeholders' preferences to support the design and implementation of policy interventions. The validity of a DCE, therefore, depends on the appropriate specification of the attributes and their levels. There have been recent calls for greater rigor in implementing and reporting on the processes of developing attributes and attribute-levels for discrete choice experiments (DCEs). This paper responds to such calls by carefully reporting a systematic process of developing micro health insurance attributes and attribute-levels for the design of a DCE in rural Malawi. Conceptual attributes and attribute-levels were initially derived from a literature review which informed the design of qualitative data collection tools to identify context specific attributes and attribute-levels. Qualitative data was collected in August-September 2012 from 12 focus group discussions with community residents and 8 in-depth interviews with health workers. All participants were selected according to stratified purposive sampling. The material was tape-recorded, fully transcribed, and coded by three researchers to identify context-specific attributes and attribute-levels. Expert opinion was used to scale down the attributes and levels. A pilot study confirmed the appropriateness of the selected attributes and levels for a DCE. First, a consensus, emerging from an individual level analysis of the qualitative transcripts, identified 10 candidate attributes. Levels were assigned to all attributes based on data from transcripts and knowledge of the Malawian context, derived from literature. Second, through further discussions with experts, four attributes were discarded based on multiple criteria. The 6 remaining attributes were: premium level, unit of enrollment, management structure, health service benefit package, transportation coverage and copayment levels. A final step of revision and piloting confirmed that the retained attributes satisfied the credibility criteria of DCE attributes. This detailed description makes our attribute development process transparent, and provides the reader with a basis to assess the rigor of this stage of constructing the DCE. This paper contributes empirical evidence to the limited methodological literature on attributes and levels development for DCE, thereby providing further empirical guidance on the matter, specifically within rural communities of low- and middle-income countries.

“I see it now”: Using photo elicitation to understand chronic illness self-management

PubMed Central

Fritz, Heather; Lysack, Cathy

2018-01-01

Background How people integrate self-management into daily life remains underexamined, and such processes are difficult to elicit through traditional approaches used to understand human occupation. Purpose This paper will provide a brief overview of one visual research method, photo elicitation, that holds promise for studying self-management of health behaviours and will present findings from an analysis of how the use of photo elicitation interviews contributed additional insights into self-management beyond those generated from the data collected through the other methods used in the study. Method A qualitative, multiple-methods, multiple-case study was conducted with a purposive sample of 10 low-income women ages 40 to 64 with type 2 diabetes. Findings The photo elicitation interviews contributed insights beyond those generated from other study methods about how individuals viewed their self-management behaviours and how occupations changed across time. Implications Photo elicitation is a valuable research method for better understanding clients' chronic illness self-management practices. PMID:29898501

Expert system for web based collaborative CAE

NASA Astrophysics Data System (ADS)

Hou, Liang; Lin, Zusheng

2006-11-01

An expert system for web based collaborative CAE was developed based on knowledge engineering, relational database and commercial FEA (Finite element analysis) software. The architecture of the system was illustrated. In this system, the experts' experiences, theories and typical examples and other related knowledge, which will be used in the stage of pre-process in FEA, were categorized into analysis process and object knowledge. Then, the integrated knowledge model based on object-oriented method and rule based method was described. The integrated reasoning process based on CBR (case based reasoning) and rule based reasoning was presented. Finally, the analysis process of this expert system in web based CAE application was illustrated, and an analysis example of a machine tool's column was illustrated to prove the validity of the system.

Elicitation of anti-Sendai virus cytotoxic T lymphocytes by viral and H-2 antigens incorporated into the same lipid bilayer by membrane fusion and by reconstitution into liposomes.

PubMed

Hale, A H; Lyles, D S; Fan, D P

1980-02-01

We have investigated the minimal molecular requirements for elicitation of anti-Sendai virus cytotoxic T lymphocytes (CTL), and the minimal molecular requirements for the recognition and lysis processes associated with anti-Sendai virus CTL-target cell interactions. This report demonstrates a) that the hemagglutinin-neuraminidase and/or fusion glycoproteins of Sendai virus can elicit anti-Sendai virus CTL and b) that these glycoproteins and H-2 antigens must be within the same membrane lipid bilayer for effective elicitation of anti-Sendai-virus CTL and for effective recognition and lysis of target cells by anti-Sendai virus CTL.

The Principles of Designing an Expert System in Teaching Mathematics

ERIC Educational Resources Information Center

Salekhova, Lailya; Nurgaliev, Albert; Zaripova, Rinata; Khakimullina, Nailya

2013-01-01

This study reveals general didactic concepts of the Expert Systems (ES) development process in the educational area. The proof of concept is based on the example of teaching the 8th grade Algebra subject. The main contribution in this work is the implementation of innovative approaches in analysis and processing of data by expert system as well as…

Regressive Imagery in Creative Problem-Solving: Comparing Verbal Protocols of Expert and Novice Visual Artists and Computer Programmers

ERIC Educational Resources Information Center

Kozbelt, Aaron; Dexter, Scott; Dolese, Melissa; Meredith, Daniel; Ostrofsky, Justin

2015-01-01

We applied computer-based text analyses of regressive imagery to verbal protocols of individuals engaged in creative problem-solving in two domains: visual art (23 experts, 23 novices) and computer programming (14 experts, 14 novices). Percentages of words involving primary process and secondary process thought, plus emotion-related words, were…

Proceedings of the international conference on cybernetics and societ

DOE Office of Scientific and Technical Information (OSTI.GOV)

Not Available

1985-01-01

This book presents the papers given at a conference on artificial intelligence, expert systems and knowledge bases. Topics considered at the conference included automating expert system development, modeling expert systems, causal maps, data covariances, robot vision, image processing, multiprocessors, parallel processing, VLSI structures, man-machine systems, human factors engineering, cognitive decision analysis, natural language, computerized control systems, and cybernetics.

Effects of Unexpected Chords and of Performer's Expression on Brain Responses and Electrodermal Activity

PubMed Central

Koelsch, Stefan; Kilches, Simone; Steinbeis, Nikolaus; Schelinski, Stefanie

2008-01-01

Background There is lack of neuroscientific studies investigating music processing with naturalistic stimuli, and brain responses to real music are, thus, largely unknown. Methodology/Principal Findings This study investigates event-related brain potentials (ERPs), skin conductance responses (SCRs) and heart rate (HR) elicited by unexpected chords of piano sonatas as they were originally arranged by composers, and as they were played by professional pianists. From the musical excerpts played by the pianists (with emotional expression), we also created versions without variations in tempo and loudness (without musical expression) to investigate effects of musical expression on ERPs and SCRs. Compared to expected chords, unexpected chords elicited an early right anterior negativity (ERAN, reflecting music-syntactic processing) and an N5 (reflecting processing of meaning information) in the ERPs, as well as clear changes in the SCRs (reflecting that unexpected chords also elicited emotional responses). The ERAN was not influenced by emotional expression, whereas N5 potentials elicited by chords in general (regardless of their chord function) differed between the expressive and the non-expressive condition. Conclusions/Significance These results show that the neural mechanisms of music-syntactic processing operate independently of the emotional qualities of a stimulus, justifying the use of stimuli without emotional expression to investigate the cognitive processing of musical structure. Moreover, the data indicate that musical expression affects the neural mechanisms underlying the processing of musical meaning. Our data are the first to reveal influences of musical performance on ERPs and SCRs, and to show physiological responses to unexpected chords in naturalistic music. PMID:18612459

Signal Processing Expert Code (SPEC)

DOE Office of Scientific and Technical Information (OSTI.GOV)

Ames, H.S.

1985-12-01

The purpose of this paper is to describe a prototype expert system called SPEC which was developed to demonstrate the utility of providing an intelligent interface for users of SIG, a general purpose signal processing code. The expert system is written in NIL, runs on a VAX 11/750 and consists of a backward chaining inference engine and an English-like parser. The inference engine uses knowledge encoded as rules about the formats of SIG commands and about how to perform frequency analyses using SIG. The system demonstrated that expert system can be used to control existing codes.

Untangling syntactic and sensory processing: an ERP study of music perception.

PubMed

Koelsch, Stefan; Jentschke, Sebastian; Sammler, Daniela; Mietchen, Daniel

2007-05-01

The present study investigated music-syntactic processing with chord sequences that ended on either regular or irregular chord functions. Sequences were composed such that perceived differences in the cognitive processing between syntactically regular and irregular chords could not be due to the sensory processing of acoustic factors like pitch repetition, pitch commonality (the major component of "sensory dissonance"), or roughness. Three experiments with independent groups of subjects were conducted: a behavioral experiment and two experiments using electroencephalography. Irregular chords elicited an early right anterior negativity (ERAN) in the event-related brain potentials (ERPs) under both task-relevant and task-irrelevant conditions. Behaviorally, participants detected around 75% of the irregular chords, indicating that these chords were only moderately salient. Nevertheless, the irregular chords reliably elicited clear ERP effects. Amateur musicians were slightly more sensitive to musical irregularities than nonmusicians, supporting previous studies demonstrating effects of musical training on music-syntactic processing. The findings indicate that the ERAN is an index of music-syntactic processing and that the ERAN can be elicited even when irregular chords are not detectable based on acoustical factors such as pitch repetition, sensory dissonance, or roughness.

EoE (Eosinophilic Esophagitis)

MedlinePlus

... Sheet Q & A with Experts Patient Stories Social Security Disability Application Process For Kids For Teens Managing ... Q & A with Experts Health Information Sheet Social Security Disability Application Process For Family and Friends For ...

Harnessing expert knowledge: Defining a Bayesian network decision model with limited data-Model structure for the vibration qualification problem

DOE Office of Scientific and Technical Information (OSTI.GOV)

Rizzo, Davinia B.; Blackburn, Mark R.

As systems become more complex, systems engineers rely on experts to inform decisions. There are few experts and limited data in many complex new technologies. This challenges systems engineers as they strive to plan activities such as qualification in an environment where technical constraints are coupled with the traditional cost, risk, and schedule constraints. Bayesian network (BN) models provide a framework to aid systems engineers in planning qualification efforts with complex constraints by harnessing expert knowledge and incorporating technical factors. By quantifying causal factors, a BN model can provide data about the risk of implementing a decision supplemented with informationmore » on driving factors. This allows a systems engineer to make informed decisions and examine “what-if” scenarios. This paper discusses a novel process developed to define a BN model structure based primarily on expert knowledge supplemented with extremely limited data (25 data sets or less). The model was developed to aid qualification decisions—specifically to predict the suitability of six degrees of freedom (6DOF) vibration testing for qualification. The process defined the model structure with expert knowledge in an unbiased manner. Finally, validation during the process execution and of the model provided evidence the process may be an effective tool in harnessing expert knowledge for a BN model.« less

Harnessing expert knowledge: Defining a Bayesian network decision model with limited data-Model structure for the vibration qualification problem

DOE PAGES

Rizzo, Davinia B.; Blackburn, Mark R.

2018-03-30

As systems become more complex, systems engineers rely on experts to inform decisions. There are few experts and limited data in many complex new technologies. This challenges systems engineers as they strive to plan activities such as qualification in an environment where technical constraints are coupled with the traditional cost, risk, and schedule constraints. Bayesian network (BN) models provide a framework to aid systems engineers in planning qualification efforts with complex constraints by harnessing expert knowledge and incorporating technical factors. By quantifying causal factors, a BN model can provide data about the risk of implementing a decision supplemented with informationmore » on driving factors. This allows a systems engineer to make informed decisions and examine “what-if” scenarios. This paper discusses a novel process developed to define a BN model structure based primarily on expert knowledge supplemented with extremely limited data (25 data sets or less). The model was developed to aid qualification decisions—specifically to predict the suitability of six degrees of freedom (6DOF) vibration testing for qualification. The process defined the model structure with expert knowledge in an unbiased manner. Finally, validation during the process execution and of the model provided evidence the process may be an effective tool in harnessing expert knowledge for a BN model.« less

Attention-dependent sound offset-related brain potentials.

PubMed

Horváth, János

2016-05-01

When performing sensory tasks, knowing the potentially occurring goal-relevant and irrelevant stimulus events allows the establishment of selective attention sets, which result in enhanced sensory processing of goal-relevant events. In the auditory modality, such enhancements are reflected in the increased amplitude of the N1 ERP elicited by the onsets of task-relevant sounds. It has been recently suggested that ERPs to task-relevant sound offsets are similarly enhanced in a tone-focused state in comparison to a distracted one. The goal of the present study was to explore the influence of attention on ERPs elicited by sound offsets. ERPs elicited by tones in a duration-discrimination task were compared to ERPs elicited by the same tones in not-tone-focused attentional setting. Tone offsets elicited a consistent, attention-dependent biphasic (positive-negative--P1-N1) ERP waveform for tone durations ranging from 150 to 450 ms. The evidence, however, did not support the notion that the offset-related ERPs reflected an offset-specific attention set: The offset-related ERPs elicited in a duration-discrimination condition (in which offsets were task relevant) did not significantly differ from those elicited in a pitch-discrimination condition (in which the offsets were task irrelevant). Although an N2 reflecting the processing of offsets in task-related terms contributed to the observed waveform, this contribution was separable from the offset-related P1 and N1. The results demonstrate that when tones are attended, offset-related ERPs may substantially overlap endogenous ERP activity in the postoffset interval irrespective of tone duration, and attention differences may cause ERP differences in such postoffset intervals. © 2016 Society for Psychophysiological Research.

ISA implementation and uncertainty: a literature review and expert elicitation study.

PubMed

van der Pas, J W G M; Marchau, V A W J; Walker, W E; van Wee, G P; Vlassenroot, S H

2012-09-01

Each day, an average of over 116 people die from traffic accidents in the European Union. One out of three fatalities is estimated to be the result of speeding. The current state of technology makes it possible to make speeding more difficult, or even impossible, by placing intelligent speed limiters (so called ISA devices) in vehicles. Although the ISA technology has been available for some years now, and reducing the number of road traffic fatalities and injuries has been high on the European political agenda, implementation still seems to be far away. Experts indicate that there are still too many uncertainties surrounding ISA implementation, and dealing with these uncertainties is essential for implementing ISA. In this paper, a systematic and representative inventory of the uncertainties is made based upon the literature. Furthermore, experts in the field of ISA were surveyed and asked which uncertainties are barriers for ISA implementation, and how uncertain these uncertainties are. We found that the long-term effects and the effects of large-scale implementation of ISA are still uncertain and are the most important barriers for the implementation of the most effective types of ISA. One way to deal with these uncertainties would be to start implementation on a small scale and gradually expand the penetration, in order to learn how ISA influences the transport system over time. Copyright © 2010 Elsevier Ltd. All rights reserved.

Understanding chemotherapy treatment pathways of advanced colorectal cancer patients to inform an economic evaluation in the United Kingdom

PubMed Central

Shabaruddin, F H; Elliott, R A; Valle, J W; Newman, W G; Payne, K

2010-01-01

Background: Accurate description of current practice within advanced colorectal cancer (CRC) specialties were needed to inform an economic evaluation of the UGT1A1 pharmacogenetic test for irinotecan in the United Kingdom. Methods: The study was based on a literature review and elicitation of expert opinion. The expert panel comprised 44 consultant oncologists in NHS Hospital Trusts across England. Results: Ten first-line, 10 second-line and 12 third-line chemotherapy regimens were reported, reflecting wide variations in treatment pathways. Predominant pathways emerged with: first-line treatment with oxaliplatin-based regimens, second-line treatment with irinotecan-based regimens and third-line treatment with mitomycin-based regimens. Experts estimated the frequency of febrile neutropaenia 8.4% (95% CI: 6.7–10.0), septic neutropaenia 4.7% (95% CI: 3.4–6.0) and severe diarrhoea 13.1% (95% CI: 10.8–15.5). Approaches for the clinical management of neutropaenia within the NHS were described. Conclusions: This study identified wide variations in the clinical management of advanced CRC patients. Descriptions of current treatment pathways are necessary for economic evaluations. Variations in clinical practice must be reflected in the model to ensure the findings from an economic evaluation of UGT1A1 testing are sufficient to inform policy regarding the cost-effective use of NHS resources. PMID:20661248

A Participatory Design Approach to Develop a Web-Based Self-Care Program Supporting Early Rehabilitation among Patients after Total Laryngectomy.

PubMed

Cnossen, Ingrid C; van Uden-Kraan, Cornelia F; Eerenstein, Simone E J; Rinkel, Rico N P M; Aalders, Ijke J; van den Berg, Klaske; de Goede, Cees J T; van Stijgeren, Ans J; Cruijff-Bijl, Yvonne; de Bree, Remco; Leemans, C René; Verdonck-de Leeuw, Irma M

2015-01-01

To develop a Web-based self-care program for patients after total laryngectomy according to a participatory design approach. We conducted a needs assessment with laryngectomees (n = 9) and their partners (n = 3) by means of a focus group interview. In 4 focus group sessions, a requirement plan was formulated by a team of health care professionals (n = 10) and translated into a prototype. An e-health application was built including illustrated information on functional changes after total laryngectomy as well as video demonstrations of skills and exercises. Usability of the prototype was tested by end users (n = 4) and expert users (n = 10). Interviews were held to elicit the intention to use and the desired implementation strategy. Six main self-care topics were identified: (1) nutrition, (2) tracheostomy care, (3) voice prosthesis care, (4) speech rehabilitation, (5) smell rehabilitation, and (6) mobility of head, neck, and shoulder muscles. Expert users expressed concerns regarding tailored exercises, indicated a positive intent to implement the intervention in routine care, and expressed a need for guidance when implementing the intervention. End users and expert users appreciated the content completeness and multimedia-based information built into the application. The participatory design is a valuable approach to develop a self-care program to help meet users' needs. © 2016 S. Karger AG, Basel.

Identifying Barriers and Pathways to Success for Renewable Energy Development on American Indian Lands

DOE Office of Scientific and Technical Information (OSTI.GOV)

Necefer, Len Edward; Jones, Thomas Elisha

American Indian tribes possess lands rich with renewable energy (RE) resources. Tribes have great potential and need to develop these resources, yet face a host of barriers that continue to impede development. Understanding these challenges as well as the pathways that can be taken to overcome them may facilitate more economic development to meet community needs and better position tribes to play a role in securing a low-carbon energy future for the United States. This paper presents the results of an expert elicitation of 24 tribal energy experts from federal, tribal, academic, and private industry backgrounds to identify barriers andmore » opportunities for federally recognized tribes in the lower 48 states. Experts identified a number of unique challenges facing tribes including financing and funding, infrastructure, tribal leadership and staff, state-level influence, and partnerships. Cultural factors were seen only to be of concern with large-scale development. Tribal sovereignty is a significant motivation for RE development and has yet to be fully realized. Cultural considerations are critical to the success of future projects; smaller residential and community-scale projects may be a better fit. Improving partnerships between tribes and the private sector can increase RE deployment and overcome historical distrust. States can have a double-ended influence on projects within tribal lands through taxation.« less

A novel design process for selection of attributes for inclusion in discrete choice experiments: case study exploring variation in clinical decision-making about thrombolysis in the treatment of acute ischaemic stroke.

PubMed

De Brún, Aoife; Flynn, Darren; Ternent, Laura; Price, Christopher I; Rodgers, Helen; Ford, Gary A; Rudd, Matthew; Lancsar, Emily; Simpson, Stephen; Teah, John; Thomson, Richard G

2018-06-22

A discrete choice experiment (DCE) is a method used to elicit participants' preferences and the relative importance of different attributes and levels within a decision-making process. DCEs have become popular in healthcare; however, approaches to identify the attributes/levels influencing a decision of interest and to selection methods for their inclusion in a DCE are under-reported. Our objectives were: to explore the development process used to select/present attributes/levels from the identified range that may be influential; to describe a systematic and rigorous development process for design of a DCE in the context of thrombolytic therapy for acute stroke; and, to discuss the advantages of our five-stage approach to enhance current guidance for developing DCEs. A five-stage DCE development process was undertaken. Methods employed included literature review, qualitative analysis of interview and ethnographic data, expert panel discussions, a quantitative structured prioritisation (ranking) exercise and pilot testing of the DCE using a 'think aloud' approach. The five-stage process reported helped to reduce the list of 22 initial patient-related factors to a final set of nine variable factors and six fixed factors for inclusion in a testable DCE using a vignette model of presentation. In order for the data and conclusions generated by DCEs to be deemed valid, it is crucial that the methods of design and development are documented and reported. This paper has detailed a rigorous and systematic approach to DCE development which may be useful to researchers seeking to establish methods for reducing and prioritising attributes for inclusion in future DCEs.

Visual Word Recognition in Deaf Readers: Lexicality Is Modulated by Communication Mode

PubMed Central

Barca, Laura; Pezzulo, Giovanni; Castrataro, Marianna; Rinaldi, Pasquale; Caselli, Maria Cristina

2013-01-01

Evidence indicates that adequate phonological abilities are necessary to develop proficient reading skills and that later in life phonology also has a role in the covert visual word recognition of expert readers. Impairments of acoustic perception, such as deafness, can lead to atypical phonological representations of written words and letters, which in turn can affect reading proficiency. Here, we report an experiment in which young adults with different levels of acoustic perception (i.e., hearing and deaf individuals) and different modes of communication (i.e., hearing individuals using spoken language, deaf individuals with a preference for sign language, and deaf individuals using the oral modality with less or no competence in sign language) performed a visual lexical decision task, which consisted of categorizing real words and consonant strings. The lexicality effect was restricted to deaf signers who responded faster to real words than consonant strings, showing over-reliance on whole word lexical processing of stimuli. No effect of stimulus type was found in deaf individuals using the oral modality or in hearing individuals. Thus, mode of communication modulates the lexicality effect. This suggests that learning a sign language during development shapes visuo-motor representations of words, which are tuned to the actions used to express them (phono-articulatory movements vs. hand movements) and to associated perceptions. As these visuo-motor representations are elicited during on-line linguistic processing and can overlap with the perceptual-motor processes required to execute the task, they can potentially produce interference or facilitation effects. PMID:23554976

Visual word recognition in deaf readers: lexicality is modulated by communication mode.

PubMed

Barca, Laura; Pezzulo, Giovanni; Castrataro, Marianna; Rinaldi, Pasquale; Caselli, Maria Cristina

2013-01-01

Evidence indicates that adequate phonological abilities are necessary to develop proficient reading skills and that later in life phonology also has a role in the covert visual word recognition of expert readers. Impairments of acoustic perception, such as deafness, can lead to atypical phonological representations of written words and letters, which in turn can affect reading proficiency. Here, we report an experiment in which young adults with different levels of acoustic perception (i.e., hearing and deaf individuals) and different modes of communication (i.e., hearing individuals using spoken language, deaf individuals with a preference for sign language, and deaf individuals using the oral modality with less or no competence in sign language) performed a visual lexical decision task, which consisted of categorizing real words and consonant strings. The lexicality effect was restricted to deaf signers who responded faster to real words than consonant strings, showing over-reliance on whole word lexical processing of stimuli. No effect of stimulus type was found in deaf individuals using the oral modality or in hearing individuals. Thus, mode of communication modulates the lexicality effect. This suggests that learning a sign language during development shapes visuo-motor representations of words, which are tuned to the actions used to express them (phono-articulatory movements vs. hand movements) and to associated perceptions. As these visuo-motor representations are elicited during on-line linguistic processing and can overlap with the perceptual-motor processes required to execute the task, they can potentially produce interference or facilitation effects.

A demonstration of expert systems applications in transportation engineering : volume III, evaluation of the prototype expert system TRANZ.

DOT National Transportation Integrated Search

1990-01-01

The validation and evaluation of an expert system for traffic control in highway work zones (TRANZ) is described. The stages in the evaluation process consisted of the following: revisit the experts, selectively distribute copies of TRANZ with docume...

The Value of Wind Technology Innovation: Implications for the U.S. Power System, Wind Industry, Electricity Consumers, and Environment

DOE Office of Scientific and Technical Information (OSTI.GOV)

Mai, Trieu T; Lantz, Eric J; Mowers, Matthew

Improvements to wind technologies have, in part, led to substantial deployment of U.S. wind power in recent years. The degree to which technology innovation will continue is highly uncertain adding to uncertainties in future wind deployment. We apply electric sector modeling to estimate the potential wind deployment opportunities across a range of technology advancement projections. The suite of projections considered span a wide range of possible cost and technology innovation trajectories, including those from a recent expert elicitation of wind energy experts, a projection based on the broader literature, and one reflecting estimates based on a U.S. DOE research initiative.more » In addition, we explore how these deployment pathways may impact the electricity system, electricity consumers, the environment, and the wind-related workforce. Overall, our analysis finds that wind technology innovation can have consequential implications for future wind power development throughout the United States, impact the broader electricity system, lower electric system and consumer costs, provide potential environmental benefits, and grow the U.S. wind workforce.« less

Using expert knowledge for test linking.

PubMed

Bolsinova, Maria; Hoijtink, Herbert; Vermeulen, Jorine Adinda; Béguin, Anton

2017-12-01

Linking and equating procedures are used to make the results of different test forms comparable. In the cases where no assumption of random equivalent groups can be made some form of linking design is used. In practice the amount of data available to link the two tests is often very limited due to logistic and security reasons, which affects the precision of linking procedures. This study proposes to enhance the quality of linking procedures based on sparse data by using Bayesian methods which combine the information in the linking data with background information captured in informative prior distributions. We propose two methods for the elicitation of prior knowledge about the difference in difficulty of two tests from subject-matter experts and explain how these results can be used in the specification of priors. To illustrate the proposed methods and evaluate the quality of linking with and without informative priors, an empirical example of linking primary school mathematics tests is presented. The results suggest that informative priors can increase the precision of linking without decreasing the accuracy. (PsycINFO Database Record (c) 2017 APA, all rights reserved).

Acquiring Expertise and Cognitive Skills in the Process of Constructing an Expert System: A Preliminary Study.

ERIC Educational Resources Information Center

Lai, Kwok-Wing

This study investigated the educational implications of having a group of 17 first-year New Zealand nursing students develop a medical expert system. Goals of the study were to determine whether, in the process of developing the expert system, the learners could: (1) acquire a more in-depth understanding of the specific subject domain under study…

Real-time diagnostics for a reusable rocket engine

NASA Technical Reports Server (NTRS)

Guo, T. H.; Merrill, W.; Duyar, A.

1992-01-01

A hierarchical, decentralized diagnostic system is proposed for the Real-Time Diagnostic System component of the Intelligent Control System (ICS) for reusable rocket engines. The proposed diagnostic system has three layers of information processing: condition monitoring, fault mode detection, and expert system diagnostics. The condition monitoring layer is the first level of signal processing. Here, important features of the sensor data are extracted. These processed data are then used by the higher level fault mode detection layer to do preliminary diagnosis on potential faults at the component level. Because of the closely coupled nature of the rocket engine propulsion system components, it is expected that a given engine condition may trigger more than one fault mode detector. Expert knowledge is needed to resolve the conflicting reports from the various failure mode detectors. This is the function of the diagnostic expert layer. Here, the heuristic nature of this decision process makes it desirable to use an expert system approach. Implementation of the real-time diagnostic system described above requires a wide spectrum of information processing capability. Generally, in the condition monitoring layer, fast data processing is often needed for feature extraction and signal conditioning. This is usually followed by some detection logic to determine the selected faults on the component level. Three different techniques are used to attack different fault detection problems in the NASA LeRC ICS testbed simulation. The first technique employed is the neural network application for real-time sensor validation which includes failure detection, isolation, and accommodation. The second approach demonstrated is the model-based fault diagnosis system using on-line parameter identification. Besides these model based diagnostic schemes, there are still many failure modes which need to be diagnosed by the heuristic expert knowledge. The heuristic expert knowledge is implemented using a real-time expert system tool called G2 by Gensym Corp. Finally, the distributed diagnostic system requires another level of intelligence to oversee the fault mode reports generated by component fault detectors. The decision making at this level can best be done using a rule-based expert system. This level of expert knowledge is also implemented using G2.

The N400 and the P300 are not all that independent.

PubMed

Arbel, Yael; Spencer, Kevin M; Donchin, Emanuel

2011-06-01

This study assessed whether two ERP components that are elicited by unexpected events interact. The conditions that are known to elicit the N400 and the P300 ERP components were applied separately and in combination to terminal-words in sentences. Each sentence ended with a terminal-word that was highly expected, semantically unexpected, physically deviant, or both semantically unexpected and physically deviant. In addition, we varied the level of semantic relatedness between the unexpected terminal-words and the expected exemplars. Physically deviant words elicited a P300, whereas semantically unexpected words elicited an N400, whose amplitude was sensitive to the level of semantic relatedness. Words that were both semantically unexpected and physically deviant elicited both an N400 with enhanced amplitude, and a P300 with reduced amplitude. These results suggest an interaction between the processes manifested by the two components. Copyright © 2010 Society for Psychophysiological Research.

The influence of expert opinions on the selection of wastewater treatment alternatives: a group decision-making approach.

PubMed

Kalbar, Pradip P; Karmakar, Subhankar; Asolekar, Shyam R

2013-10-15

The application of multiple-attribute decision-making (MADM) to real life decision problems suggests that avoiding the loss of information through scenario-based approaches and including expert opinions in the decision-making process are two major challenges that require more research efforts. Recently, a wastewater treatment technology selection effort has been made with a 'scenario-based' method of MADM. This paper focuses on a novel approach to incorporate expert opinions into the scenario-based decision-making process, as expert opinions play a major role in the selection of treatment technologies. The sets of criteria and the indicators that are used consist of both qualitative and quantitative criteria. The group decision-making (GDM) approach that is implemented for aggregating expert opinions is based on an analytical hierarchy process (AHP), which is the most widely used MADM method. The pairwise comparison matrices (PCMs) for qualitative criteria are formed based on expert opinions, whereas, a novel approach is proposed for generating PCMs for quantitative criteria. It has been determined that the experts largely prefer natural treatment systems because they are more sustainable in any scenario. However, PCMs based on expert opinions suggest that advanced technologies such as the sequencing batch reactor (SBR) can also be appropriate for a given decision scenario. The proposed GDM approach is a rationalized process that will be more appropriate in realistic scenarios where multiple stakeholders with local and regional societal priorities are involved in the selection of treatment technology. Copyright © 2013 Elsevier Ltd. All rights reserved.

Attention that covers letters is necessary for the left-lateralization of an early print-tuned ERP in Japanese hiragana.

PubMed

Okumura, Yasuko; Kasai, Tetsuko; Murohashi, Harumitsu

2015-03-01

Extensive experience with reading develops expertise in acquiring information from print, and this is reflected in specific enhancement of the left-lateralized N170 component in event-related potentials. The N170 is generally considered to reflect visual/orthographic processing; while modulations of its left-lateralization related to phonological processes have also been indicated. However, in our previous study, N170-like response to Hiragana strings lacked left-lateralization when the stimuli were completely task-irrelevant in rapid-presentation sequences [Okumura et al. (2014). Early print-tuned ERP response with minimal involvement of linguistic processing in Japanese Hiragana strings. Neuroreport 25, 410-414]. This suggests that, despite the highly transparent character-to-syllable correspondence, the phonological mapping of Hiragana strings requires some kind of attention toward print. To verify this notion, the present study examined ERPs under the same experimental condition as in the previous study, except that the task required attention to a stimulus attribute (i.e., color). As a result, Hiragana words and nonwords elicited left-lateralized negative deflection in the occipito-temporal region during 130-170ms post-stimulus in comparison to symbol strings, but only when the print had a narrow intercharacter spacing. Moreover, we observed the enhancement of very early occipital ERP in response to words during 70-100ms. The present results suggest that visual attention plays a role in early print processing, which may contribute to our understanding of the mechanisms that underlie expert as well as impaired reading. Copyright © 2015 Elsevier Ltd. All rights reserved.

The Health Equity and Effectiveness of Policy Options to Reduce Dietary Salt Intake in England: Policy Forecast.

PubMed

Gillespie, Duncan O S; Allen, Kirk; Guzman-Castillo, Maria; Bandosz, Piotr; Moreira, Patricia; McGill, Rory; Anwar, Elspeth; Lloyd-Williams, Ffion; Bromley, Helen; Diggle, Peter J; Capewell, Simon; O'Flaherty, Martin

2015-01-01

Public health action to reduce dietary salt intake has driven substantial reductions in coronary heart disease (CHD) over the past decade, but avoidable socio-economic differentials remain. We therefore forecast how further intervention to reduce dietary salt intake might affect the overall level and inequality of CHD mortality. We considered English adults, with socio-economic circumstances (SEC) stratified by quintiles of the Index of Multiple Deprivation. We used IMPACTSEC, a validated CHD policy model, to link policy implementation to salt intake, systolic blood pressure and CHD mortality. We forecast the effects of mandatory and voluntary product reformulation, nutrition labelling and social marketing (e.g., health promotion, education). To inform our forecasts, we elicited experts' predictions on further policy implementation up to 2020. We then modelled the effects on CHD mortality up to 2025 and simultaneously assessed the socio-economic differentials of effect. Mandatory reformulation might prevent or postpone 4,500 (2,900-6,100) CHD deaths in total, with the effect greater by 500 (300-700) deaths or 85% in the most deprived than in the most affluent. Further voluntary reformulation was predicted to be less effective and inequality-reducing, preventing or postponing 1,500 (200-5,000) CHD deaths in total, with the effect greater by 100 (-100-600) deaths or 49% in the most deprived than in the most affluent. Further social marketing and improvements to labelling might each prevent or postpone 400-500 CHD deaths, but minimally affect inequality. Mandatory engagement with industry to limit salt in processed-foods appears a promising and inequality-reducing option. For other policy options, our expert-driven forecast warns that future policy implementation might reach more deprived individuals less well, limiting inequality reduction. We therefore encourage planners to prioritise equity.

Wishful Thinking? Inside the Black Box of Exposure Assessment

PubMed Central

Money, Annemarie; Robinson, Christine; Agius, Raymond; de Vocht, Frank

2016-01-01

Background: Decision-making processes used by experts when undertaking occupational exposure assessment are relatively unknown, but it is often assumed that there is a common underlying method that experts employ. However, differences in training and experience of assessors make it unlikely that one general method for expert assessment would exist. Therefore, there are concerns about formalizing, validating, and comparing expert estimates within and between studies that are difficult, if not impossible, to characterize. Heuristics on the other hand (the processes involved in decision making) have been extensively studied. Heuristics are deployed by everyone as short-cuts to make the often complex process of decision-making simpler, quicker, and less burdensome. Experts’ assessments are often subject to various simplifying heuristics as a way to reach a decision in the absence of sufficient data. Therefore, investigating the underlying heuristics or decision-making processes involved may help to shed light on the ‘black box’ of exposure assessment. Methods: A mixed method study was conducted utilizing both a web-based exposure assessment exercise incorporating quantitative and semiqualitative elements of data collection, and qualitative semi-structured interviews with exposure assessors. Qualitative data were analyzed using thematic analysis. Results: Twenty-five experts completed the web-based exposure assessment exercise and 8 of these 25 were randomly selected to participate in the follow-up interview. Familiar key themes relating to the exposure assessment exercise emerged; ‘intensity’; ‘probability’; ‘agent’; ‘process’; and ‘duration’ of exposure. However, an important aspect of the detailed follow-up interviews revealed a lack of structure and order with which participants described their decision making. Participants mostly described some form of an iterative process, heavily relying on the anchoring and adjustment heuristic, which differed between experts. Conclusion: In spite of having undertaken comparable training (in occupational hygiene or exposure assessment), experts use different methods to assess exposure. Decision making appears to be an iterative process with heavy reliance on the key heuristic of anchoring and adjustment. Using multiple experts to assess exposure while providing some form of anchoring scenario to build from, and additional training in understanding the impact of simple heuristics on the process of decision making, is likely to produce a more methodical approach to assessment; thereby improving consistency and transparency in expert exposure assessment. PMID:26764244

Photo-Elicitation: Reflexivity on Method, Analysis, and Graphic Portraits

ERIC Educational Resources Information Center

Richard, Veronica M.; Lahman, Maria K. E.

2015-01-01

In this methodological discussion, the authors detail and reflect on the processes of using photo-elicitation interviewing as a way to align with positive qualitative methodologies, to gain access to participant beliefs and values, and to highlight participant voices through their choices of words and visuals. A review of the literature and an…

Differential Neural Activity during Search of Specific and General Autobiographical Memories Elicited by Musical Cues

ERIC Educational Resources Information Center

Ford, Jaclyn Hennessey; Addis, Donna Rose; Giovanello, Kelly S.

2011-01-01

Previous neuroimaging studies that have examined autobiographical memory specificity have utilized retrieval cues associated with prior searches of the event, potentially changing the retrieval processes being investigated. In the current study, musical cues were used to naturally elicit memories from multiple levels of specificity (i.e., lifetime…

Small Knowledge-Based Systems in Education and Training: Something New Under the Sun.

ERIC Educational Resources Information Center

Wilson, Brent G.; Welsh, Jack R.

1986-01-01

Discusses artificial intelligence, robotics, natural language processing, and expert or knowledge-based systems research; examines two large expert systems, MYCIN and XCON; and reviews the resources required to build large expert systems and affordable smaller systems (intelligent job aids) for training. Expert system vendors and products are…

StarPower: Elevating Prospective Student Interest through Expert and Celebrity Endorsements--Relevant Message, Relevant Media

ERIC Educational Resources Information Center

Tucciarone, Kristy

2016-01-01

This study investigates how universities can increase the effectiveness of the search process by featuring in their advertisements expert and celebrity endorsers who attended the institution. How can universities gain the attention of prospective students using the star power of experts and celebrities? Experts and celebrities promoting a…

Rhetorical Consequences of the Computer Society: Expert Systems and Human Communication.

ERIC Educational Resources Information Center

Skopec, Eric Wm.

Expert systems are computer programs that solve selected problems by modelling domain-specific behaviors of human experts. These computer programs typically consist of an input/output system that feeds data into the computer and retrieves advice, an inference system using the reasoning and heuristic processes of human experts, and a knowledge…

Expert Systems: Implications for the Diagnosis and Treatment of Learning Disabilities.

ERIC Educational Resources Information Center

Hofmeister, Alan M.; Lubke, Margaret M.

1988-01-01

The article examines characteristics and present or potential applications of expert systems technology for diagnosis and treatment of learning disabilities. Preliminary findings indicate that expert systems can perform as well as humans in specific areas, and that the process of organizing knowledge bases for expert systems helps clarify existing…

A Semiautomated Framework for Integrating Expert Knowledge into Disease Marker Identification

PubMed Central

Wang, Jing; Webb-Robertson, Bobbie-Jo M.; Matzke, Melissa M.; Varnum, Susan M.; Brown, Joseph N.; Riensche, Roderick M.; Adkins, Joshua N.; Jacobs, Jon M.; Hoidal, John R.; Scholand, Mary Beth; Pounds, Joel G.; Blackburn, Michael R.; Rodland, Karin D.; McDermott, Jason E.

2013-01-01

Background. The availability of large complex data sets generated by high throughput technologies has enabled the recent proliferation of disease biomarker studies. However, a recurring problem in deriving biological information from large data sets is how to best incorporate expert knowledge into the biomarker selection process. Objective. To develop a generalizable framework that can incorporate expert knowledge into data-driven processes in a semiautomated way while providing a metric for optimization in a biomarker selection scheme. Methods. The framework was implemented as a pipeline consisting of five components for the identification of signatures from integrated clustering (ISIC). Expert knowledge was integrated into the biomarker identification process using the combination of two distinct approaches; a distance-based clustering approach and an expert knowledge-driven functional selection. Results. The utility of the developed framework ISIC was demonstrated on proteomics data from a study of chronic obstructive pulmonary disease (COPD). Biomarker candidates were identified in a mouse model using ISIC and validated in a study of a human cohort. Conclusions. Expert knowledge can be introduced into a biomarker discovery process in different ways to enhance the robustness of selected marker candidates. Developing strategies for extracting orthogonal and robust features from large data sets increases the chances of success in biomarker identification. PMID:24223463

A Semiautomated Framework for Integrating Expert Knowledge into Disease Marker Identification

DOE Office of Scientific and Technical Information (OSTI.GOV)

Wang, Jing; Webb-Robertson, Bobbie-Jo M.; Matzke, Melissa M.

2013-10-01

Background. The availability of large complex data sets generated by high throughput technologies has enabled the recent proliferation of disease biomarker studies. However, a recurring problem in deriving biological information from large data sets is how to best incorporate expert knowledge into the biomarker selection process. Objective. To develop a generalizable framework that can incorporate expert knowledge into data-driven processes in a semiautomated way while providing a metric for optimization in a biomarker selection scheme. Methods. The framework was implemented as a pipeline consisting of five components for the identification of signatures from integrated clustering (ISIC). Expert knowledge was integratedmore » into the biomarker identification process using the combination of two distinct approaches; a distance-based clustering approach and an expert knowledge-driven functional selection. Results. The utility of the developed framework ISIC was demonstrated on proteomics data from a study of chronic obstructive pulmonary disease (COPD). Biomarker candidates were identified in a mouse model using ISIC and validated in a study of a human cohort. Conclusions. Expert knowledge can be introduced into a biomarker discovery process in different ways to enhance the robustness of selected marker candidates. Developing strategies for extracting orthogonal and robust features from large data sets increases the chances of success in biomarker identification.« less

Using multiple data sets to populate probabilistic volcanic event trees

USGS Publications Warehouse

Newhall, C.G.; Pallister, John S.

2014-01-01

The key parameters one needs to forecast outcomes of volcanic unrest are hidden kilometers beneath the Earth’s surface, and volcanic systems are so complex that there will invariably be stochastic elements in the evolution of any unrest. Fortunately, there is sufficient regularity in behaviour that some, perhaps many, eruptions can be forecast with enough certainty for populations to be evacuated and kept safe. Volcanologists charged with forecasting eruptions must try to understand each volcanic system well enough that unrest can be interpreted in terms of pre-eruptive process, but must simultaneously recognize and convey uncertainties in their assessment. We have found that use of event trees helps to focus discussion, integrate data from multiple sources, reach consensus among scientists about both pre-eruptive process and uncertainties and, in some cases, to explain all of this to officials. Figure 1 shows a generic volcanic event tree from Newhall and Hoblitt (2002) that can be modified as needed for each specific volcano. This paper reviews how we and our colleagues have used such trees during a number of volcanic crises worldwide, for rapid hazard assessments in situations in which more formal expert elicitations could not be conducted. We describe how Multiple Data Sets can be used to estimate probabilities at each node and branch. We also present case histories of probability estimation during crises, how the estimates were used by public officials, and some suggestions for future improvements.

Parallel processing and expert systems

NASA Technical Reports Server (NTRS)

Yan, Jerry C.; Lau, Sonie

1991-01-01

Whether it be monitoring the thermal subsystem of Space Station Freedom, or controlling the navigation of the autonomous rover on Mars, NASA missions in the 90's cannot enjoy an increased level of autonomy without the efficient use of expert systems. Merely increasing the computational speed of uniprocessors may not be able to guarantee that real time demands are met for large expert systems. Speed-up via parallel processing must be pursued alongside the optimization of sequential implementations. Prototypes of parallel expert systems have been built at universities and industrial labs in the U.S. and Japan. The state-of-the-art research in progress related to parallel execution of expert systems was surveyed. The survey is divided into three major sections: (1) multiprocessors for parallel expert systems; (2) parallel languages for symbolic computations; and (3) measurements of parallelism of expert system. Results to date indicate that the parallelism achieved for these systems is small. In order to obtain greater speed-ups, data parallelism and application parallelism must be exploited.

ERP manifestations of processing printed words at different psycholinguistic levels: time course and scalp distribution.

PubMed

Bentin, S; Mouchetant-Rostaing, Y; Giard, M H; Echallier, J F; Pernier, J

1999-05-01

The aim of the present study was to examine the time course and scalp distribution of electrophysiological manifestations of the visual word recognition mechanism. Event-related potentials (ERPs) elicited by visually presented lists of words were recorded while subjects were involved in a series of oddball tasks. The distinction between the designated target and nontarget stimuli was manipulated to induce a different level of processing in each session (visual, phonological/phonetic, phonological/lexical, and semantic). The ERPs of main interest in this study were those elicited by nontarget stimuli. In the visual task the targets were twice as big as the nontargets. Words, pseudowords, strings of consonants, strings of alphanumeric symbols, and strings of forms elicited a sharp negative peak at 170 msec (N170); their distribution was limited to the occipito-temporal sites. For the left hemisphere electrode sites, the N170 was larger for orthographic than for nonorthographic stimuli and vice versa for the right hemisphere. The ERPs elicited by all orthographic stimuli formed a clearly distinct cluster that was different from the ERPs elicited by nonorthographic stimuli. In the phonological/phonetic decision task the targets were words and pseudowords rhyming with the French word vitrail, whereas the nontargets were words, pseudowords, and strings of consonants that did not rhyme with vitrail. The most conspicuous potential was a negative peak at 320 msec, which was similarly elicited by pronounceable stimuli but not by nonpronounceable stimuli. The N320 was bilaterally distributed over the middle temporal lobe and was significantly larger over the left than over the right hemisphere. In the phonological/lexical processing task we compared the ERPs elicited by strings of consonants (among which words were selected), pseudowords (among which words were selected), and by words (among which pseudowords were selected). The most conspicuous potential in these tasks was a negative potential peaking at 350 msec (N350) elicited by phonologically legal but not by phonologically illegal stimuli. The distribution of the N350 was similar to that of the N320, but it was broader and including temporo-parietal areas that were not activated in the "rhyme" task. Finally, in the semantic task the targets were abstract words, and the nontargets were concrete words, pseudowords, and strings of consonants. The negative potential in this task peaked at 450 msec. Unlike the lexical decision, the negative peak in this task significantly distinguished not only between phonologically legal and illegal words but also between meaningful (words) and meaningless (pseudowords) phonologically legal structures. The distribution of the N450 included the areas activated in the lexical decision task but also areas in the fronto-central regions. The present data corroborated the functional neuroanatomy of word recognition systems suggested by other neuroimaging methods and described their timecourse, supporting a cascade-type process that involves different but interconnected neural modules, each responsible for a different level of processing word-related information.

Automaticity of higher cognitive functions: neurophysiological evidence for unconscious syntactic processing of masked words.

PubMed

Jiménez-Ortega, Laura; García-Milla, Marcos; Fondevila, Sabela; Casado, Pilar; Hernández-Gutiérrez, David; Martín-Loeches, Manuel

2014-12-01

Models of language comprehension assume that syntactic processing is automatic, at least at early stages. However, the degree of automaticity of syntactic processing is still controversial. Evidence of automaticity is either indirect or has been observed for pairs of words, which might provide a poor syntactic context in comparison to sentences. The present study investigates the automaticity of syntactic processing using event-related brain potentials (ERPs) during sentence processing. To this end, masked adjectives that could either be syntactically correct or incorrect relative to a sentence being processed appeared just prior to the presentation of supraliminal adjectives. The latter could also be correct or incorrect. According to our data, subliminal gender agreement violations embedded in a sentence trigger an early anterior negativity-like modulation, whereas supraliminal gender agreement violations elicited a later anterior negativity. First-pass syntactic parsing thus appears to be unconsciously and automatically elicited. Interestingly, a P600-like modulation of short duration and early latency could also be observed for masked violations. In addition, masked violations also modulated the P600 component elicited by unmasked targets, probably reflecting that the mechanisms of revising a structural mismatch appear affected by subliminal information. According to our findings, both conscious and unconscious processes apparently contribute to syntactic processing. These results are discussed in line with most recent theories of automaticity and syntactic processing. Copyright © 2014 Elsevier B.V. All rights reserved.

Foundations for Creativity in the Writing Process: Rhetorical Representations of Ill-Defined Problems.

ERIC Educational Resources Information Center

Carey, L. J.; Flower, Linda

This report examines the composing processes of expert writers to determine which cognitive processes in expository writing produce an opportunity for a creative response. The first section considers how the ill-defined nature of many writing problems and the cognitive processes experts use to solve these problems interact to provide an…

Reflecting on the use of photo elicitation with children.

PubMed

Whiting, Lisa S

2015-01-01

To reflect on the use of photo elicitation as a data collection method when conducting research with primary school age children (nine to 11 years). There is recognition that children feel an affinity with the visual medium; as a result, visual methods can be useful when conducting research with children. Photo elicitation is one such method, but there has been little discussion of its use with primary school children within a health context. This paper considers the main issues that researchers should consider. This paper draws on a research study conducted by the author that used an ethnographic approach and photo elicitation to identify the assets underpinning children's wellbeing. A reflective discussion is used to highlight issues relating to the use of photo elicitation to collect data from primary school children. Photo elicitation is not without its challenges: it creates additional ethical considerations, and can be more time-consuming and expensive. However, children value the opportunity to be involved in research and have their opinions sought, and photo elicitation provides a method of collecting data that is appropriate for children's developmental and cognitive maturational stages. Photo elicitation can be a positive experience for children, and one that is not only fun and engaging, but that is also empowering and valuing of their contributions. Research that uses photo elicitation needs to be carefully planned to ensure that the study is supported appropriately. The visual process can offer a unique insight into children's lives that allows health professionals to deepen their understanding of children's experiences.

Time course of brain activation elicited by basic emotions.

PubMed

Hot, Pascal; Sequeira, Henrique

2013-11-13

Whereas facial emotion recognition protocols have shown that each discrete emotion has a specific time course of brain activation, there is no electrophysiological evidence to support these findings for emotional induction by complex pictures. Our objective was to specify the differences between the time courses of brain activation elicited by feelings of happiness and, with unpleasant pictures, by feelings of disgust and sadness. We compared event-related potentials (ERPs) elicited by the watching of high-arousing pictures from the International Affective Picture System, selected to induce specific emotions. In addition to a classical arousal effect on late positive components, we found specific ERP patterns for each emotion in early temporal windows (<200 ms). Disgust was the first emotion to be associated with different brain processing after 140 ms, whereas happiness and sadness differed in ERPs elicited at the frontal and central sites after 160 ms. Our findings highlight the limits of the classical averaging of ERPs elicited by different emotions inside the same valence and suggest that each emotion could elicit a specific temporal pattern of brain activation, similar to those observed with emotional face recognition.

Expert opinion on "best practices" in the delivery of health care services to immigrants in Denmark.

PubMed

Jensen, Natasja Koitzsch; Nielsen, Signe Smith; Krasnik, Allan

2010-08-01

Delivery of health care to immigrants is an emerging field of interest. Immigrants are frequently characterised by health outcomes that are inferior to those of other groups with regard to morbidity and mortality. In addition, health professionals report difficulties associated with the encounter with immigrant patients. A Delphi process with eight Danish experts from the field of immigrant health was performed as part of an EU project. The objective of the Delphi process was to investigate expert opinion on "best practice in the delivery of healthcare to immigrants". Initially, 60 factors were suggested by the experts. Next, these factors were summarised into 32 factors that the experts were invited to rate and, if possible, agree on. The top 11 factors identified in the Delphi process were access to interpreters, quality of interpretation, ensuring medication compliance, having sufficient consultation time, coherence of offers, interdisciplinary collaboration, allocation of resources, the role of the practitioner, acknowledgement of the individual patient, education of health professionals and students and access to telephone interpretation to supplement other services. The Delphi process can be a valuable tool in the investigation of expert opinion and may thereby help to guide future policy directives. In the light of the importance experts placed on access to interpreters and on the quality of the interpretation services offered, it seems as an untenable strategy to introduce as from June 2011 self-payment for interpretation services provided to immigrants who have stayed in the country for more than seven years.

Compassion-based emotion regulation up-regulates experienced positive affect and associated neural networks

PubMed Central

Singer, Tania

2015-01-01

Emotion regulation research has primarily focused on techniques that attenuate or modulate the impact of emotional stimuli. Recent evidence suggests that this mode regulation can be problematic in the context of regulation of emotion elicited by the suffering of others, resulting in reduced emotional connectedness. Here, we investigated the effects of an alternative emotion regulation technique based on the up-regulation of positive affect via Compassion-meditation on experiential and neural affective responses to depictions of individuals in distress, and compared these with the established emotion regulation strategy of Reappraisal. Using fMRI, we scanned 15 expert practitioners of Compassion-meditation either passively viewing, or using Compassion-meditation or Reappraisal to modulate their emotional reactions to film clips depicting people in distress. Both strategies effectively, but differentially regulated experienced affect, with Compassion primarily increasing positive and Reappraisal primarily decreasing negative affect. Imaging results showed that Compassion, relative to both passive-viewing and Reappraisal increased activation in regions involved in affiliation, positive affect and reward processing including ventral striatum and medial orbitfrontal cortex. This network was shown to be active prior to stimulus presentation, suggesting that the regulatory mechanism of Compassion is the stimulus-independent endogenous generation of positive affect. PMID:25698699

Seagrass ecosystem services - What's next?

PubMed

Nordlund, Lina Mtwana; Jackson, Emma L; Nakaoka, Masahiro; Samper-Villarreal, Jimena; Beca-Carretero, Pedro; Creed, Joel C

2017-09-20

Seagrasses, marine flowering plants, provide a wide range of ecosystem services, defined here as natural processes and components that directly or indirectly benefit human needs. Recent research has shown that there are still many gaps in our comprehension of seagrass ecosystem service provision. Furthermore, there seems to be little public knowledge of seagrasses in general and the benefits they provide. This begs the questions: how do we move forward with the information we have? What other information do we need and what actions do we need to take in order to improve the situation and appreciation for seagrass? Based on the outcomes from an international expert knowledge eliciting workshop, three key areas to advance seagrass ecosystem service research were identified: 1) Variability of ecosystem services within seagrass meadows and among different meadows; 2) Seagrass ecosystem services in relation to, and their connection with, other coastal habitats; and 3) Improvement in the communication of seagrass ecosystem services to the public. Here we present ways forward to advance seagrass ecosystem service research in order to raise the profile of seagrass globally, as a means to establish more effective conservation and restoration of these important coastal habitats around the world. Copyright © 2017 The Authors. Published by Elsevier Ltd.. All rights reserved.

A structured approach to Exposure Based Waiving of human health endpoints under REACH developed in the OSIRIS project.

PubMed

Marquart, Hans; Meijster, Tim; Van de Bovenkamp, Marja; Ter Burg, Wouter; Spaan, Suzanne; Van Engelen, Jacqueline

2012-03-01

Exposure Based Waiving (EBW) is one of the options in REACH when there is insufficient hazard data on a specific endpoint. Rules for adaptation of test requirements are specified and a general option for EBW is given via Appendix XI of REACH, allowing waiving of repeated dose toxicity studies, reproductive toxicity studies and carcinogenicity studies under a number of conditions if exposure is very low. A decision tree is described that was developed in the European project OSIRIS (Optimised Strategies for Risk Assessment of Industrial Chemicals through Integration of Non-Test and Test Information) to help decide in what cases EBW can be justified. The decision tree uses specific criteria as well as more general questions. For the latter, guidance on interpretation and resulting conclusions is provided. Criteria and guidance are partly based on an expert elicitation process. Among the specific criteria a number of proposed Thresholds of Toxicological Concern are used. The decision tree, expanded with specific parts on absorption, distribution, metabolism and excretion that are not described in this paper, is implemented in the OSIRIS webtool on integrated testing strategies. Copyright © 2011 Elsevier Inc. All rights reserved.

Parallel processing and expert systems

NASA Technical Reports Server (NTRS)

Lau, Sonie; Yan, Jerry C.

1991-01-01

Whether it be monitoring the thermal subsystem of Space Station Freedom, or controlling the navigation of the autonomous rover on Mars, NASA missions in the 1990s cannot enjoy an increased level of autonomy without the efficient implementation of expert systems. Merely increasing the computational speed of uniprocessors may not be able to guarantee that real-time demands are met for larger systems. Speedup via parallel processing must be pursued alongside the optimization of sequential implementations. Prototypes of parallel expert systems have been built at universities and industrial laboratories in the U.S. and Japan. The state-of-the-art research in progress related to parallel execution of expert systems is surveyed. The survey discusses multiprocessors for expert systems, parallel languages for symbolic computations, and mapping expert systems to multiprocessors. Results to date indicate that the parallelism achieved for these systems is small. The main reasons are (1) the body of knowledge applicable in any given situation and the amount of computation executed by each rule firing are small, (2) dividing the problem solving process into relatively independent partitions is difficult, and (3) implementation decisions that enable expert systems to be incrementally refined hamper compile-time optimization. In order to obtain greater speedups, data parallelism and application parallelism must be exploited.

The characteristics of railway service disruption: implications for disruption management.

PubMed

Golightly, D; Dadashi, N

2017-03-01

Rail disruption management is central to operational continuity and customer satisfaction. Disruption is not a unitary phenomenon - it varies by time, cause, location and complexity of coordination. Effective, user-centred technology for rail disruption must reflect this variety. A repertory grid study was conducted to elicit disruption characteristics. Construct elicitation with a group of experts (n = 7) captured 26 characteristics relevant to rail disruption. A larger group of operational staff (n = 28) rated 10 types of rail incident against the 26 characteristics. The results revealed distinctions such as business impact and public perception, and the importance of management of the disruption over initial detection. There were clear differences between those events that stop the traffic, as opposed to those that only slow the traffic. The results also demonstrate the utility of repertory grid for capturing the characteristics of complex work domains. Practitioner Summary: The aim of the paper is to understand how variety in rail disruption influences socio-technical design. It uses repertory grid to identify and prioritise 26 constructs, and group 10 disruption types, identifying critical factors such as whether an incident stops or merely slows the service, and business reputation.

Sound to Language: Different Cortical Processing for First and Second Languages in Elementary School Children as Revealed by a Large-Scale Study Using fNIRS

PubMed Central

Ojima, Shiro; Matsuba-Kurita, Hiroko; Dan, Ippeita; Tsuzuki, Daisuke; Katura, Takusige; Hagiwara, Hiroko

2011-01-01

A large-scale study of 484 elementary school children (6–10 years) performing word repetition tasks in their native language (L1-Japanese) and a second language (L2-English) was conducted using functional near-infrared spectroscopy. Three factors presumably associated with cortical activation, language (L1/L2), word frequency (high/low), and hemisphere (left/right), were investigated. L1 words elicited significantly greater brain activation than L2 words, regardless of semantic knowledge, particularly in the superior/middle temporal and inferior parietal regions (angular/supramarginal gyri). The greater L1-elicited activation in these regions suggests that they are phonological loci, reflecting processes tuned to the phonology of the native language, while phonologically unfamiliar L2 words were processed like nonword auditory stimuli. The activation was bilateral in the auditory and superior/middle temporal regions. Hemispheric asymmetry was observed in the inferior frontal region (right dominant), and in the inferior parietal region with interactions: low-frequency words elicited more right-hemispheric activation (particularly in the supramarginal gyrus), while high-frequency words elicited more left-hemispheric activation (particularly in the angular gyrus). The present results reveal the strong involvement of a bilateral language network in children’s brains depending more on right-hemispheric processing while acquiring unfamiliar/low-frequency words. A right-to-left shift in laterality should occur in the inferior parietal region, as lexical knowledge increases irrespective of language. PMID:21350046

Sound to language: different cortical processing for first and second languages in elementary school children as revealed by a large-scale study using fNIRS.

PubMed

Sugiura, Lisa; Ojima, Shiro; Matsuba-Kurita, Hiroko; Dan, Ippeita; Tsuzuki, Daisuke; Katura, Takusige; Hagiwara, Hiroko

2011-10-01

A large-scale study of 484 elementary school children (6-10 years) performing word repetition tasks in their native language (L1-Japanese) and a second language (L2-English) was conducted using functional near-infrared spectroscopy. Three factors presumably associated with cortical activation, language (L1/L2), word frequency (high/low), and hemisphere (left/right), were investigated. L1 words elicited significantly greater brain activation than L2 words, regardless of semantic knowledge, particularly in the superior/middle temporal and inferior parietal regions (angular/supramarginal gyri). The greater L1-elicited activation in these regions suggests that they are phonological loci, reflecting processes tuned to the phonology of the native language, while phonologically unfamiliar L2 words were processed like nonword auditory stimuli. The activation was bilateral in the auditory and superior/middle temporal regions. Hemispheric asymmetry was observed in the inferior frontal region (right dominant), and in the inferior parietal region with interactions: low-frequency words elicited more right-hemispheric activation (particularly in the supramarginal gyrus), while high-frequency words elicited more left-hemispheric activation (particularly in the angular gyrus). The present results reveal the strong involvement of a bilateral language network in children's brains depending more on right-hemispheric processing while acquiring unfamiliar/low-frequency words. A right-to-left shift in laterality should occur in the inferior parietal region, as lexical knowledge increases irrespective of language.

Imprecise probability assessment of tipping points in the climate system

PubMed Central

Kriegler, Elmar; Hall, Jim W.; Held, Hermann; Dawson, Richard; Schellnhuber, Hans Joachim

2009-01-01

Major restructuring of the Atlantic meridional overturning circulation, the Greenland and West Antarctic ice sheets, the Amazon rainforest and ENSO, are a source of concern for climate policy. We have elicited subjective probability intervals for the occurrence of such major changes under global warming from 43 scientists. Although the expert estimates highlight large uncertainty, they allocate significant probability to some of the events listed above. We deduce conservative lower bounds for the probability of triggering at least 1 of those events of 0.16 for medium (2–4 °C), and 0.56 for high global mean temperature change (above 4 °C) relative to year 2000 levels. PMID:19289827

Photo-elicitation with adolescents in qualitative research: an example of its use in exploring family interactions in adolescent psychiatry.

PubMed

Sibeoni, J; Costa-Drolon, E; Poulmarc'h, L; Colin, S; Valentin, M; Pradère, J; Revah-Levy, A

2017-01-01

Photo-elicitation is a method used increasingly often in qualitative health research, and its positive effect on the research process is well established today. Photo-elicitation appears to facilitate verbalization and insight and to improve relationships between the researcher and participants, thereby enriching the quality of the data collected. Nonetheless, it is barely used at all in the field of adolescent psychiatry. With the aim of exploring the potential of these methods for research with adolescents receiving psychiatric care, we conducted a qualitative photo-elicitation data collection study with this population, asking them about family interactions around food. The data were collected from 15 adolescents and 17 parents during semi-structured interviews in which a photo taken by the adolescent served as the focus of discussion. Data were explored through inductive thematic analysis. Photo-elicitation played a threefold role in this study: (1) it induced the teens' interest, thought, and pleasure, (2) it played a mediating function during the interviews, and (3) it enabled family interactions to be viewed from the adolescent's perspective. Three themes concerning family interactions were found: (1) parent-child relationship patterns, (2) the functioning of the family group, and (3) the adolescent's individual relation with food, that is, the issue of the adolescent's autonomy. Photo-elicitation proved to be an innovative technique in qualitative research in the area of adolescent psychiatry, one that enriched the data and enabled the emergence of new themes in this field, related in particular to the process by which adolescents develop autonomy.

Effects of perceptual and semantic cues on ERP modulations associated with prospective memory.

PubMed

Cousens, Ross; Cutmore, Timothy; Wang, Ya; Wilson, Jennifer; Chan, Raymond C K; Shum, David H K

2015-10-01

Prospective memory involves the formation and execution of intended actions and is essential for autonomous living. In this study (N=32), the effect of the nature of PM cues (semantic versus perceptual) on established event-related potentials (ERPs) elicited in PM tasks (N300 and prospective positivity) was investigated. PM cues defined by their perceptual features clearly elicited the N300 and prospective positivity whereas PM cues defined by semantic relatedness elicited prospective positivity. This calls into question the view that the N300 is a marker of general processes underlying detection of PM cues, but supports existing research showing that prospective positivity represents general post-retrieval processes that follow detection of PM cues. Continued refinement of ERP paradigms for understanding the neural correlates of PM is needed. Copyright © 2015 Elsevier B.V. All rights reserved.

Modeling of ETL-Processes and Processed Information in Clinical Data Warehousing.

PubMed

Tute, Erik; Steiner, Jochen

2018-01-01

Literature describes a big potential for reuse of clinical patient data. A clinical data warehouse (CDWH) is a means for that. To support management and maintenance of processes extracting, transforming and loading (ETL) data into CDWHs as well as to ease reuse of metadata between regular IT-management, CDWH and secondary data users by providing a modeling approach. Expert survey and literature review to find requirements and existing modeling techniques. An ETL-modeling-technique was developed extending existing modeling techniques. Evaluation by exemplarily modeling existing ETL-process and a second expert survey. Nine experts participated in the first survey. Literature review yielded 15 included publications. Six existing modeling techniques were identified. A modeling technique extending 3LGM² and combining it with openEHR information models was developed and evaluated. Seven experts participated in the evaluation. The developed approach can help in management and maintenance of ETL-processes and could serve as interface between regular IT-management, CDWH and secondary data users.

Using the "Expert Performance Approach" as a Framework for Improving Understanding of Expert Learning

ERIC Educational Resources Information Center

Williams, A. Mark; Fawver, Bradley; Hodges, Nicola J.

2017-01-01

The expert performance approach, initially proposed by Ericsson and Smith (1991), is reviewed as a systematic framework for the study of "expert" learning. The need to develop representative tasks to capture learning is discussed, as is the need to employ process-tracing measures during acquisition to examine what actually changes during…

Reasoning process characteristics in the diagnostic skills of beginner, competent, and expert dentists.

PubMed

Crespo, Kathleen E; Torres, José E; Recio, María E

2004-12-01

The purpose of this study was to evaluate qualitative differences in the diagnostic reasoning process at different developmental stages of expertise. A qualitative design was used to study cognitive processes that characterize the diagnosis of oral disease at the stages of beginner (five junior students who had passed the NBDE I), competent (five GPR first-year residents), and expert dentists (five general dentists with ten or more years of experience). Individually, each participant was asked to determine the diagnosis of an oral condition based on a written clinical case, using the think aloud technique and retrospective reports. A subsequent interview was conducted to obtain the participants' diagnostic process model and pathophysiology of the case. The analysis of the verbal protocols indicated that experts referred to the patient's sociomedical context more frequently, demonstrated better organization of ideas, could determine key clinical findings, and had an ability to plan for the search of pertinent information. Fewer diagnostic hypotheses were formulated by participants who used forward reasoning, independent of the stage of development. Beginners requested additional diagnostic aids (radiographs, laboratory tests) more frequently than the competent/expert dentists. Experts recalled typical experiences with patients, while competent/beginner dentists recalled information from didactic courses. Experts evidenced cognitive diagnostic schemas that integrate pathophysiology of disease, while competent and beginner participants had not achieved this integration. We conclude that expert performance is a combination of a knowledge base, reasoning skills, and an accumulation of experiences with patients that is qualitatively different from that of competent and beginner dentists. It is important for dental education to emphasize the teaching of cognitive processes and to incorporate a wide variety of clinical experiences in addition to the teaching of disciplinary content.

Building a Foreign Military Sales Construction Delivery Strategy Decision Support System

DTIC Science & Technology

1991-09-01

DSS, formulates it into a computer model and produces solutions using information and expert heuristics. Using the Expert Systeic Process to Build a DSS...computer model . There are five stages in the development of an expert system. They are: 1) Identify and characterize the important aspects of the problem...and Steven A. Hidreth. U.S. Security Assistance: The Political Process. Massachusetts: Heath and Company, 1985. 19. Guirguis , Amir A., Program

A knowledge based expert system for propellant system monitoring at the Kennedy Space Center

NASA Technical Reports Server (NTRS)

Jamieson, J. R.; Delaune, C.; Scarl, E.

1985-01-01

The Lox Expert System (LES) is the first attempt to build a realtime expert system capable of simulating the thought processes of NASA system engineers, with regard to fluids systems analysis and troubleshooting. An overview of the hardware and software describes the techniques used, and possible applications to other process control systems. LES is now in the advanced development stage, with a full implementation planned for late 1985.

Cognitive Science.

ERIC Educational Resources Information Center

Cocking, Rodney R.; Mestre, Jose P.

The focus of this paper is on cognitive science as a model for understanding the application of human skills toward effective problem-solving. Sections include: (1) "Introduction" (discussing information processing framework, expert-novice distinctions, schema theory, and learning process); (2) "Application: The Expert-Novice…

Disorder-specific emotional imagery for differential and quantitative assessment of agoraphobia.

PubMed

Engel, Kirsten-Rita; Bandelow, Borwin; Neumann, Charlott; Obst, Katrin; Wedekind, Dirk

2012-01-01

Visual emotional stimulation is supposed to elicit psycho-vegetative reactions, which are similar to as the ones elicited by exposure to actual experience. Visual stimulation paradigms have been widely used in studies on agoraphobia with and without panic disorder. However, the applied imagery has hardly ever been disorder- and subject- specific. 51 patients with an ICD-10 and DSM-IV diagnosis of agoraphobia with or without panic disorder (PDA) and matching healthy controls have been examined. Subjects were confronted with 146 picture showing characteristic agoraphobic situations (high places, narrow places, crowds, public transport facilities, or wide places) or pictures associated with acute physical emergency (panic) situations, which had been pre-selected by anxiety experts. Participants were asked to rate emotional arousal induced by the respective images on the Self- Assessment Manikin scale (SAM). Data on PDA severity (PAS) depressive symptoms (MADRS) and sociodemographic data were recorded. Saliva cortisol levels were measured before and after exposure in a second test applying the individually mostly feared stimuli combined with emotionally neutral pictures for every single patient. 117 of the PDA-specific images were rated significantly more fear-eliciting by patients than by healthy individuals. Sub-categorization into agoraphobia clusters showed differential effects of clusters with regard to gender distribution, severity of PDA and cortisol secretion during exposure. In this study disorder specific and individual characteristics of agoraphobia were assessed for use in future trials applying emotional imagery. It could be used for the differential assessment of PDA and associated neurobiological and psychological phenomena and in neuroimaging paradigms.

Hydraulic Characteristics Of Two Bicycle-Safe Grate Inlet Designs

DOT National Transportation Integrated Search

1988-12-01

Expert Systems are computer programs designed to include a simulation of the reasoning and decision-making processes of human experts. This report provides a set of general guidelines for the development and distribution of highway related expert sys...

Attitudes towards evaluation of psychiatric disability claims: a survey of Swiss stakeholders.

PubMed

Schandelmaier, Stefan; Leibold, Andrea; Fischer, Katrin; Mager, Ralph; Hoffmann-Richter, Ulrike; Bachmann, Monica Susanne; Kedzia, Sarah; Busse, Jason Walter; Guyatt, Gordon Henry; Jeger, Joerg; Marelli, Renato; De Boer, Wout Ernst Lodewijk; Kunz, Regina

2015-01-01

In Switzerland, evaluation of work capacity in individuals with mental disorders has come under criticism. We surveyed stakeholders about their concerns and expectations of the current claim process. We conducted a nationwide online survey among five stakeholder groups. We asked 37 questions addressing the claim process and the evaluation of work capacity, the maximum acceptable disagreement in judgments on work capacity, and its documentation. Response rate among 704 stakeholders (95 plaintiff lawyers, 285 treating psychiatrists, 129 expert psychiatrists evaluating work capacity, 64 social judges, 131 insurers) varied between 71% and 29%. Of the lawyers, 92% were dissatisfied with the current claim process, as were psychiatrists (73%) and experts (64%), whereas the majority of judges (72%) and insurers (81%) were satisfied. Stakeholders agreed in their concerns, such as the lack of a transparent relationship between the experts' findings and their conclusions regarding work capacity, medical evaluations inappropriately addressing legal issues, and the experts' delay in finalising the report. Findings mirror the characteristics that stakeholders consider important for an optimal work capacity evaluation. For a scenario where two experts evaluate the same claimant, stakeholders considered an inter-rater difference of 10%‒20% in work capacity at maximum acceptable. Plaintiff lawyers, treating psychiatrists and experts perceive major problems in work capacity evaluation of psychiatric claims whereas judges and insurers see the process more positively. Efforts to improve the process should include clarifying the basis on which judgments are made, restricting judgments to areas of expertise, and ensuring prompt submission of evaluations.

Use of the self-organising map network (SOMNet) as a decision support system for regional mental health planning.

PubMed

Chung, Younjin; Salvador-Carulla, Luis; Salinas-Pérez, José A; Uriarte-Uriarte, Jose J; Iruin-Sanz, Alvaro; García-Alonso, Carlos R

2018-04-25

Decision-making in mental health systems should be supported by the evidence-informed knowledge transfer of data. Since mental health systems are inherently complex, involving interactions between its structures, processes and outcomes, decision support systems (DSS) need to be developed using advanced computational methods and visual tools to allow full system analysis, whilst incorporating domain experts in the analysis process. In this study, we use a DSS model developed for interactive data mining and domain expert collaboration in the analysis of complex mental health systems to improve system knowledge and evidence-informed policy planning. We combine an interactive visual data mining approach, the self-organising map network (SOMNet), with an operational expert knowledge approach, expert-based collaborative analysis (EbCA), to develop a DSS model. The SOMNet was applied to the analysis of healthcare patterns and indicators of three different regional mental health systems in Spain, comprising 106 small catchment areas and providing healthcare for over 9 million inhabitants. Based on the EbCA, the domain experts in the development team guided and evaluated the analytical processes and results. Another group of 13 domain experts in mental health systems planning and research evaluated the model based on the analytical information of the SOMNet approach for processing information and discovering knowledge in a real-world context. Through the evaluation, the domain experts assessed the feasibility and technology readiness level (TRL) of the DSS model. The SOMNet, combined with the EbCA, effectively processed evidence-based information when analysing system outliers, explaining global and local patterns, and refining key performance indicators with their analytical interpretations. The evaluation results showed that the DSS model was feasible by the domain experts and reached level 7 of the TRL (system prototype demonstration in operational environment). This study supports the benefits of combining health systems engineering (SOMNet) and expert knowledge (EbCA) to analyse the complexity of health systems research. The use of the SOMNet approach contributes to the demonstration of DSS for mental health planning in practice.

Evolving Regulatory Processes Used by Students and Experts in the Acquiring of Improvisational Skills: A Qualitative Study

ERIC Educational Resources Information Center

de Bruin, Leon R.

2018-01-01

The way an improviser practices is a vital and significant aspect to a musician's means and capacities of expression. Expert music performers utilize extensive self-regulatory processes involving planning, strategic development, and systemized approaches to learning and reflective practice. Scholars posit that these processes are constructivist…

The Holistic Processing Account of Visual Expertise in Medical Image Perception: A Review

PubMed Central

Sheridan, Heather; Reingold, Eyal M.

2017-01-01

In the field of medical image perception, the holistic processing perspective contends that experts can rapidly extract global information about the image, which can be used to guide their subsequent search of the image (Swensson, 1980; Nodine and Kundel, 1987; Kundel et al., 2007). In this review, we discuss the empirical evidence supporting three different predictions that can be derived from the holistic processing perspective: Expertise in medical image perception is domain-specific, experts use parafoveal and/or peripheral vision to process large regions of the image in parallel, and experts benefit from a rapid initial glimpse of an image. In addition, we discuss a pivotal recent study (Litchfield and Donovan, 2016) that seems to contradict the assumption that experts benefit from a rapid initial glimpse of the image. To reconcile this finding with the existing literature, we suggest that global processing may serve multiple functions that extend beyond the initial glimpse of the image. Finally, we discuss future research directions, and we highlight the connections between the holistic processing account and similar theoretical perspectives and findings from other domains of visual expertise. PMID:29033865

The Holistic Processing Account of Visual Expertise in Medical Image Perception: A Review.

PubMed

Sheridan, Heather; Reingold, Eyal M

2017-01-01

In the field of medical image perception, the holistic processing perspective contends that experts can rapidly extract global information about the image, which can be used to guide their subsequent search of the image (Swensson, 1980; Nodine and Kundel, 1987; Kundel et al., 2007). In this review, we discuss the empirical evidence supporting three different predictions that can be derived from the holistic processing perspective: Expertise in medical image perception is domain-specific, experts use parafoveal and/or peripheral vision to process large regions of the image in parallel, and experts benefit from a rapid initial glimpse of an image. In addition, we discuss a pivotal recent study (Litchfield and Donovan, 2016) that seems to contradict the assumption that experts benefit from a rapid initial glimpse of the image. To reconcile this finding with the existing literature, we suggest that global processing may serve multiple functions that extend beyond the initial glimpse of the image. Finally, we discuss future research directions, and we highlight the connections between the holistic processing account and similar theoretical perspectives and findings from other domains of visual expertise.

Effect of elicitation on growth, respiration, and nutrient uptake of root and cell suspension cultures of Hyoscyamus muticus.

PubMed

Carvalho, Edgard B; Curtis, Wayne R

2002-01-01

The elicitation of Hyoscyamus muticus root and cell suspension cultures by fungal elicitor from Rhizoctonia solani causes dramatic changes in respiration, nutrient yields, and growth. Cells and mature root tissues have similar specific oxygen uptake rates (SOUR) before and after the onset of the elicitation process. Cell suspension SOUR were 11 and 18 micromol O2/g FW x h for non-elicited control and elicited cultures, respectively. Mature root SOUR were 11 and 24 micromol O2/g FW x h for control and elicited tissue, respectively. Tissue growth is significantly reduced upon the addition of elicitor to these cultures. Inorganic yield remains fairly constant, whereas yield on sugar is reduced from 0.532 to 0.352 g dry biomass per g sugar for roots and 0.614 to 0.440 g dry biomass per g sugar for cells. This reduction in yield results from increased energy requirements for the defense response. Growth reduction is reflected in a reduction in root meristem (tip) SOUR, which decreased from 189 to 70 micromol O2/g FW x h upon elicitation. Therefore, despite the increase in total respiration, the maximum local oxygen fluxes are reduced as a result of the reduction in metabolic activity at the meristem. This distribution of oxygen uptake throughout the mature tissue could reduce mass transfer requirements during elicited production. However, this was not found to be the case for sesquiterpene elicitation, where production of lubimin and solavetivone were found to increase linearly up to oxygen partial pressures of 40% O2 in air. SOUR is shown to similarly increase in both bubble column and tubular reactors despite severe mass transfer limitations, suggesting the possibility of metabolically induced increases in tissue convective transport during elicitation.

Knowledge acquisition from natural language for expert systems based on classification problem-solving methods

NASA Technical Reports Server (NTRS)

Gomez, Fernando

1989-01-01

It is shown how certain kinds of domain independent expert systems based on classification problem-solving methods can be constructed directly from natural language descriptions by a human expert. The expert knowledge is not translated into production rules. Rather, it is mapped into conceptual structures which are integrated into long-term memory (LTM). The resulting system is one in which problem-solving, retrieval and memory organization are integrated processes. In other words, the same algorithm and knowledge representation structures are shared by these processes. As a result of this, the system can answer questions, solve problems or reorganize LTM.

Informing Hospital Change Processes through Visualization and Simulation: A Case Study at a Children's Emergency Clinic.

PubMed

Persson, Johanna; Dalholm, Elisabeth Hornyánszky; Johansson, Gerd

2014-01-01

To demonstrate the use of visualization and simulation tools in order to involve stakeholders and inform the process in hospital change processes, illustrated by an empirical study from a children's emergency clinic. Reorganization and redevelopment of a hospital is a complex activity that involves many stakeholders and demands. Visualization and simulation tools have proven useful for involving practitioners and eliciting relevant knowledge. More knowledge is desired about how these tools can be implemented in practice for hospital planning processes. A participatory planning process including practitioners and researchers was executed over a 3-year period to evaluate a combination of visualization and simulation tools to involve stakeholders in the planning process and to elicit knowledge about needs and requirements. The initial clinic proposal from the architect was discarded as a result of the empirical study. Much general knowledge about the needs of the organization was extracted by means of the adopted tools. Some of the tools proved to be more accessible than others for the practitioners participating in the study. The combination of tools added value to the process by presenting information in alternative ways and eliciting questions from different angles. Visualization and simulation tools inform a planning process (or other types of change processes) by providing the means to see beyond present demands and current work structures. Long-term involvement in combination with accessible tools is central for creating a participatory setting where the practitioners' knowledge guides the process. © 2014 Vendome Group, LLC.

20 CFR 405.10 - Medical and Vocational Expert System.

Code of Federal Regulations, 2011 CFR

2011-04-01

... Section 405.10 Employees' Benefits SOCIAL SECURITY ADMINISTRATION ADMINISTRATIVE REVIEW PROCESS FOR... and Vocational Expert Unit and a national network of qualified medical, psychological, and vocational... and psychological experts from the national network may assist a State agency in determining...

Inhibition of irrelevant information is not necessary to performance of expert chess players.

PubMed

Postal, Virginie

2012-08-01

Some studies on expertise have demonstrated that the difference between novices and experts can be partly due to a lack of knowledge about which information is relevant for a given situation. This lack of knowledge seems to be associated with the selection of correct information and with inhibitory processes. However, while the efficiency of inhibitory processes can lead to better performance in the normal population, it seems that experts in chess do not base their performance on this process but rather on an automatic and parallel encoding of information. Two experiments investigated the processes involved in a check detection task. The congruence of the information was manipulated in a Stroop situation similar to Reingold, Charness, Scheltetus, & Stampe (2001). The results showed that the experts did not benefit from cuing with a congruent cue and that they did not show any interference effect by the incongruent cue, contrary to less skilled chess players who benefited from cuing (Exp. 1). An attentional priming procedure confirmed the automatic encoding of chess relations in the more skilled chess players by showing no advantage from the prime in this group (Exp. 2). Taken together, the results indicate that the processing was serial for the less skilled chess players and that it was automatic and parallel for the more expert chess players. The inhibition of irrelevant information does not seem necessary to process information rapidly and efficiently.

Surgical process improvement tools: defining quality gaps and priority areas in gastrointestinal cancer surgery.

PubMed

Wei, A C; Devitt, K S; Wiebe, M; Bathe, O F; McLeod, R S; Urbach, D R

2014-04-01

Surgery is a cornerstone of cancer treatment, but significant differences in the quality of surgery have been reported. Surgical process improvement tools (spits) modify the processes of care as a means to quality improvement (qi). We were interested in developing spits in the area of gastrointestinal (gi) cancer surgery. We report the recommendations of an expert panel held to define quality gaps and establish priority areas that would benefit from spits. The present study used the knowledge-to-action cycle was as a framework. Canadian experts in qi and in gi cancer surgery were assembled in a nominal group workshop. Participants evaluated the merits of spits, described gaps in current knowledge, and identified and ranked processes of care that would benefit from qi. A qualitative analysis of the workshop deliberations using modified grounded theory methods identified major themes. The expert panel consisted of 22 participants. Experts confirmed that spits were an important strategy for qi. The top-rated spits included clinical pathways, electronic information technology, and patient safety tools. The preferred settings for use of spits included preoperative and intraoperative settings and multidisciplinary contexts. Outcomes of interest were cancer-related outcomes, process, and the technical quality of surgery measures. Surgical process improvement tools were confirmed as an important strategy. Expert panel recommendations will be used to guide future research efforts for spits in gi cancer surgery.

Surgical process improvement tools: defining quality gaps and priority areas in gastrointestinal cancer surgery

PubMed Central

Wei, A.C.; Devitt, K.S.; Wiebe, M.; Bathe, O.F.; McLeod, R.S.; Urbach, D.R.

2014-01-01

Background Surgery is a cornerstone of cancer treatment, but significant differences in the quality of surgery have been reported. Surgical process improvement tools (spits) modify the processes of care as a means to quality improvement (qi). We were interested in developing spits in the area of gastrointestinal (gi) cancer surgery. We report the recommendations of an expert panel held to define quality gaps and establish priority areas that would benefit from spits. Methods The present study used the knowledge-to-action cycle was as a framework. Canadian experts in qi and in gi cancer surgery were assembled in a nominal group workshop. Participants evaluated the merits of spits, described gaps in current knowledge, and identified and ranked processes of care that would benefit from qi. A qualitative analysis of the workshop deliberations using modified grounded theory methods identified major themes. Results The expert panel consisted of 22 participants. Experts confirmed that spits were an important strategy for qi. The top-rated spits included clinical pathways, electronic information technology, and patient safety tools. The preferred settings for use of spits included preoperative and intraoperative settings and multidisciplinary contexts. Outcomes of interest were cancer-related outcomes, process, and the technical quality of surgery measures. Conclusions Surgical process improvement tools were confirmed as an important strategy. Expert panel recommendations will be used to guide future research efforts for spits in gi cancer surgery. PMID:24764704

Development of an instructional expert system for hole drilling processes

NASA Technical Reports Server (NTRS)

Al-Mutawa, Souhaila; Srinivas, Vijay; Moon, Young Bai

1990-01-01

An expert system which captures the expertise of workshop technicians in the drilling domain was developed. The expert system is aimed at novice technicians who know how to operate the machines but have not acquired the decision making skills that are gained with experience. This paper describes the domain background and the stages of development of the expert system.

Spacelab data processing facility (SLDPF) quality assurance (QA)/data accounting (DA) expert systems - Transition from prototypes to operational systems

NASA Technical Reports Server (NTRS)

Basile, Lisa

1988-01-01

The SLDPF is responsible for the capture, quality monitoring processing, accounting, and shipment of Spacelab and/or Attached Shuttle Payloads (ASP) telemetry data to various user facilities. Expert systems will aid in the performance of the quality assurance and data accounting functions of the two SLDPF functional elements: the Spacelab Input Processing System (SIPS) and the Spacelab Output Processing System (SOPS). Prototypes were developed for each as independent efforts. The SIPS Knowledge System Prototype (KSP) used the commercial shell OPS5+ on an IBM PC/AT; the SOPS Expert System Prototype used the expert system shell CLIPS implemented on a Macintosh personal computer. Both prototypes emulate the duties of the respective QA/DA analysts based upon analyst input and predetermined mission criteria parameters, and recommended instructions and decisions governing the reprocessing, release, or holding for further analysis of data. These prototypes demonstrated feasibility and high potential for operational systems. Increase in productivity, decrease of tedium, consistency, concise historical records, and a training tool for new analyses were the principal advantages. An operational configuration, taking advantage of the SLDPF network capabilities, is under development with the expert systems being installed on SUN workstations. This new configuration in conjunction with the potential of the expert systems will enhance the efficiency, in both time and quality, of the SLDPF's release of Spacelab/AST data products.

Spacelab data processing facility (SLDPF) Quality Assurance (QA)/Data Accounting (DA) expert systems: Transition from prototypes to operational systems

NASA Technical Reports Server (NTRS)

Basile, Lisa

1988-01-01

The SLDPF is responsible for the capture, quality monitoring processing, accounting, and shipment of Spacelab and/or Attached Shuttle Payloads (ASP) telemetry data to various user facilities. Expert systems will aid in the performance of the quality assurance and data accounting functions of the two SLDPF functional elements: the Spacelab Input Processing System (SIPS) and the Spacelab Output Processing System (SOPS). Prototypes were developed for each as independent efforts. The SIPS Knowledge System Prototype (KSP) used the commercial shell OPS5+ on an IBM PC/AT; the SOPS Expert System Prototype used the expert system shell CLIPS implemented on a Macintosh personal computer. Both prototypes emulate the duties of the respective QA/DA analysts based upon analyst input and predetermined mission criteria parameters, and recommended instructions and decisions governing the reprocessing, release, or holding for further analysis of data. These prototypes demonstrated feasibility and high potential for operational systems. Increase in productivity, decrease of tedium, consistency, concise historial records, and a training tool for new analyses were the principal advantages. An operational configuration, taking advantage of the SLDPF network capabilities, is under development with the expert systems being installed on SUN workstations. This new configuration in conjunction with the potential of the expert systems will enhance the efficiency, in both time and quality, of the SLDPF's release of Spacelab/AST data products.